cjones/restic-automatic-backup-scheduler
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'launch_check'

Browse Source 
* launch_check:
  Modified: CHANGELOG.md
  Add restic-check launchagent
  Rename logs
  Rename backup launchagent
This commit is contained in:
Erik Westrup
2022-02-15 12:43:05 +01:00
parent 7c0b89a9c3 92205981b6
commit ef731f33b2
5 changed files with 105 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+3
View File
@@ -5,6 +5,9 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased] ## [Unreleased]
### Added
- restic-check launchagent.
### Changed ### Changed
- [README.md](README.md) is restructured with easier TL;DR for each OS and a more general detailed section for the interested. - [README.md](README.md) is restructured with easier TL;DR for each OS and a more general detailed section for the interested.
Library/LaunchAgents/com.github.erikw.restic-automatic-backup-scheduler.plist → Library/LaunchAgents/com.github.erikw.restic-backup.plist
+5 -5
View File
@@ -6,22 +6,22 @@
<plist version="1.0"> <plist version="1.0">
<dict> <dict>
<key>Label</key> <key>Label</key>
<string>com.github.erikw.restic-automatic-backup-scheduler</string> <string>com.github.erikw.restic-backup</string>
<key>ProgramArguments</key> <key>ProgramArguments</key>
<!-- exec in subshell to 1) source *.env.sh 2) expand $HOME to logpath (ref. https://apple.stackexchange.com/a/365880/197493) --> <!-- exec in subshell to 1) source *.env.sh 2) expand $HOME to logpath (ref. https://apple.stackexchange.com/a/365880/197493) -->
<array> <array>
<string>/bin/bash</string> <string>/bin/bash</string>
<string>-c</string> <string>-c</string>
<string>source /usr/local/etc/restic/default.env.sh &amp;&amp; /usr/local/bin/restic_backup.sh >>$HOME/$LOG_OUT 2>>$HOME/$LOG_ERR</string> <string>source $INSTALL_PREFIX/etc/restic/default.env.sh &amp;&amp; $INSTALL_PREFIX/bin/restic_backup.sh >>$HOME/$LOG_OUT 2>>$HOME/$LOG_ERR</string>
</array> </array>
<key>EnvironmentVariables</key> <key>EnvironmentVariables</key>
<dict> <dict>
<key>PATH</key> <key>PATH</key>
<string>/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string> <string>$INSTALL_PREFIX/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
<key>LOG_OUT</key> <key>LOG_OUT</key>
<string>/Library/Logs/restic/restic_stdout.log</string> <string>/Library/Logs/restic/backup_stdout.log</string>
<key>LOG_ERR</key> <key>LOG_ERR</key>
<string>/Library/Logs/restic/restic_stderr.log</string> <string>/Library/Logs/restic/backup_stderr.log</string>
</dict> </dict>
<key>RunAtLoad</key> <key>RunAtLoad</key>
<true/> <true/>
Library/LaunchAgents/com.github.erikw.restic-check.plist
+37
View File
@@ -0,0 +1,37 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.github.erikw.restic-check</string>
<key>ProgramArguments</key>
<array>
<string>/bin/bash</string>
<string>-c</string>
<string>source $INSTALL_PREFIX/etc/restic/default.env.sh &amp;&amp; $INSTALL_PREFIX/bin/restic_check.sh >>$HOME/$LOG_OUT 2>>$HOME/$LOG_ERR</string>
</array>
<key>EnvironmentVariables</key>
<dict>
<key>PATH</key>
<string>$INSTALL_PREFIX/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
<key>LOG_OUT</key>
<string>/Library/Logs/restic/check_stdout.log</string>
<key>LOG_ERR</key>
<string>/Library/Logs/restic/check_stderr.log</string>
</dict>
<key>RunAtLoad</key>
<true/>
<!-- Will schedule check every 1st of the month at 20:00 -->
<key>StartCalendarInterval</key>
<array>
<dict>
<key>Day</key>
<integer>1</integer>
<key>Hour</key>
<integer>20</integer>
<key>Minute</key>
<integer>0</integer>
</dict>
</array>
</dict>
</plist>
Makefile
+32 -13
View File
@@ -1,4 +1,3 @@
# TODO add install for launchagent completely, and unisntall target using bootstrap?
#### Notes #################################################################### #### Notes ####################################################################
# This build process is done in three stages (out-of-source build): # This build process is done in three stages (out-of-source build):
# 1. copy source files to the local build directory. # 1. copy source files to the local build directory.
@@ -20,7 +19,8 @@
install-targets-script install-targets-conf install-targets-systemd \ install-targets-script install-targets-conf install-targets-systemd \
install-targets-cron install-targets-launchagent \ install-targets-cron install-targets-launchagent \
install-targets-schedtask uninstall-targets-schedtask \ install-targets-schedtask uninstall-targets-schedtask \
activate-launchagent deactivate-launchagent activate-launchagent-backup deactivate-launchagent-backup \
activate-launchagent-chec deactivate-launchagent-check
#### Macros ################################################################### #### Macros ###################################################################
NOW := $(shell date +%Y-%m-%d_%H:%M:%S) NOW := $(shell date +%Y-%m-%d_%H:%M:%S)
@@ -52,8 +52,10 @@ MKDIR_PARENTS=sh -c '\
# LaunchAgent names. # LaunchAgent names.
UID := $(shell id -u) UID := $(shell id -u)
LAUNCHAGENT = com.github.erikw.restic-automatic-backup-scheduler LAUNCHAGENT_BACKUP = com.github.erikw.restic-backup
LAUNCHAGENT_TARGET = gui/$(UID)/$(LAUNCHAGENT) LAUNCHAGENT_CHECK = com.github.erikw.restic-check
LAUNCHAGENT_TARGET_BACKUP = gui/$(UID)/$(LAUNCHAGENT_BACKUP)
LAUNCHAGENT_TARGET_CHECK = gui/$(UID)/$(LAUNCHAGENT_CHECK)
# What to substitute $INSTALL_PREFIX in sources to. # What to substitute $INSTALL_PREFIX in sources to.
# This can be useful to set to empty on commandline when building e.g. an AUR # This can be useful to set to empty on commandline when building e.g. an AUR
@@ -126,6 +128,9 @@ DEST_TARGS_SYSTEMD = $(addprefix $(PREFIX)/, $(SRCS_SYSTEMD))
DEST_TARGS_CRON = $(addprefix $(PREFIX)/, $(SRCS_CRON)) DEST_TARGS_CRON = $(addprefix $(PREFIX)/, $(SRCS_CRON))
DEST_TARGS_LAUNCHAGENT = $(addprefix $(LAUNCHAGENTDIR)/, $(SRCS_LAUNCHAGENT)) DEST_TARGS_LAUNCHAGENT = $(addprefix $(LAUNCHAGENTDIR)/, $(SRCS_LAUNCHAGENT))
DEST_LAUNCHAGENT_BACKUP = $(DEST_DIR_LAUNCHAGENT)/$(LAUNCHAGENT_BACKUP).plist
DEST_LAUNCHAGENT_CHECK = $(DEST_DIR_LAUNCHAGENT)/$(LAUNCHAGENT_CHECK).plist
INSTALLED_FILES = $(DEST_TARGS_SCRIPT) $(DEST_TARGS_CONF) \ INSTALLED_FILES = $(DEST_TARGS_SCRIPT) $(DEST_TARGS_CONF) \
$(DEST_TARGS_SYSTEMD) $(DEST_TARGS_CRON) \ $(DEST_TARGS_SYSTEMD) $(DEST_TARGS_CRON) \
$(DEST_TARGS_LAUNCHAGENT) $(DEST_TARGS_LAUNCHAGENT)
@@ -158,10 +163,14 @@ install-systemd: install-targets-script install-targets-conf \
# target: install-cron - Install cron setup. # target: install-cron - Install cron setup.
install-cron: install-targets-script install-targets-conf install-targets-cron install-cron: install-targets-script install-targets-conf install-targets-cron
# target: install-launchagent - Install LaunchAgent setup. # target: install-launchagent - Install backup LaunchAgent setup.
install-launchagent: install-targets-script install-targets-conf \ install-launchagent: install-targets-script install-targets-conf \
install-targets-launchagent install-targets-launchagent
# target: install-launchagent-check - Install check LaunchAgent setup.
# Intended to be run after install-launchagent, thus not requiring scripts/conf
#install-launchagent: install-targets-launchagent
# target: install-schedtask - Install Windows ScheduledTasks # target: install-schedtask - Install Windows ScheduledTasks
install-schedtask: install-targets-script install-targets-conf \ install-schedtask: install-targets-script install-targets-conf \
install-targets-schedtask install-targets-schedtask
@@ -218,12 +227,22 @@ $(DEST_DIR_LAUNCHAGENT)/%: $(BUILD_DIR_LAUNCHAGENT)/%
$(DEST_DIR_MAC_LOG): $(DEST_DIR_MAC_LOG):
mkdir -p $@ mkdir -p $@
# target: activate-launchagent - Activate the LaunchAgent. # target: activate-launchagent-backup - Activate the backup LaunchAgent.
activate-launchagent: activate-launchagent-backup:
launchctl bootstrap gui/$(UID) $(DEST_TARGS_LAUNCHAGENT) launchctl bootstrap gui/$(UID) $(DEST_LAUNCHAGENT_BACKUP)
launchctl enable $(LAUNCHAGENT_TARGET) launchctl enable $(LAUNCHAGENT_TARGET_BACKUP)
launchctl kickstart -p $(LAUNCHAGENT_TARGET) launchctl kickstart -p $(LAUNCHAGENT_TARGET_BACKUP)
# target: deactivate-launchagent - Deactivate and remove the LaunchAgent. # target: activate-launchagent-check - Activate the check LaunchAgent.
deactivate-launchagent: activate-launchagent-check:
launchctl bootout $(LAUNCHAGENT_TARGET) launchctl bootstrap gui/$(UID) $(DEST_LAUNCHAGENT_CHECK)
launchctl enable $(LAUNCHAGENT_TARGET_CHECK)
launchctl kickstart -p $(LAUNCHAGENT_TARGET_CHECK)
# target: deactivate-launchagent-backup - Remove the backup LaunchAgent.
deactivate-launchagent-backup:
launchctl bootout $(LAUNCHAGENT_TARGET_BACKUP)
# target: deactivate-launchagent-check - Remove the check LaunchAgent.
deactivate-launchagent-check:
launchctl bootout $(LAUNCHAGENT_TARGET_CHECK)
README.md
+28 -18
View File
@@ -64,7 +64,7 @@ Depending on your system, the setup will look different. Choose one of:
Many Linux distributions nowadays use [Systemd](https://en.wikipedia.org/wiki/Systemd), which features good support for running services and scheduled jobs. If your distribution is no on Systemd, check out the [cron setup](#setup-cron) instead. Many Linux distributions nowadays use [Systemd](https://en.wikipedia.org/wiki/Systemd), which features good support for running services and scheduled jobs. If your distribution is no on Systemd, check out the [cron setup](#setup-cron) instead.
**TL;DR setup** **TL;DR setup**
1. [Create](#1-create-backblaze-b2-account-bucket-and-keys) B2 bucket + credentials 1. [Create](#1-create-backblaze-b2-account-bucket-and-keys) B2 bucket + credentials
1. Install scripts, configs systemd units/timers: 1. Install scripts, configs systemd units/timers:
* With `make`: * With `make`:
```console ```console
@@ -114,7 +114,7 @@ Many Linux distributions nowadays use [Systemd](https://en.wikipedia.org/wiki/Sy
[Launchd](https://www.launchd.info/) is the modern built-in service scheduler in macOS. It has support for running services as root (Daemon) or as a normal user (Agent). Here we set up an LauchAgent to be run as your normal user for starting regular backups. [Launchd](https://www.launchd.info/) is the modern built-in service scheduler in macOS. It has support for running services as root (Daemon) or as a normal user (Agent). Here we set up an LauchAgent to be run as your normal user for starting regular backups.
**TL;DR setup** **TL;DR setup**
1. [Create](#1-create-backblaze-b2-account-bucket-and-keys) B2 bucket + credentials 1. [Create](#1-create-backblaze-b2-account-bucket-and-keys) B2 bucket + credentials
1. Install scripts, configs and LaunchAgent: 1. Install scripts, configs and LaunchAgent:
* (recommended) with Homebrew from the [erikw/homebrew-tap](https://github.com/erikw/homebrew-tap): * (recommended) with Homebrew from the [erikw/homebrew-tap](https://github.com/erikw/homebrew-tap):
```console ```console
@@ -133,29 +133,39 @@ Many Linux distributions nowadays use [Systemd](https://en.wikipedia.org/wiki/Sy
``` ```
1. Configure [how often](https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html#//apple_ref/doc/uid/10000172i-CH1-SW1) backups should be done. If needed, edit `OnCalendar` in 1. Configure [how often](https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html#//apple_ref/doc/uid/10000172i-CH1-SW1) backups should be done. If needed, edit `OnCalendar` in
* Homebrew install: `~/Library/LaunchAgents/homebrew.mxcl.restic-automatic-backup-scheduler.plist`. * Homebrew install: `~/Library/LaunchAgents/homebrew.mxcl.restic-automatic-backup-scheduler.plist`.
* `make` install: `~/Library/LaunchAgents/com.github.erikw.restic-automatic-backup-scheduler.plist`. * `make` install: `~/Library/LaunchAgents/com.github.erikw.restic-backup.plist`.
1. Enable automated backup for starting with the system & make the first backup: 1. Enable automated backup for starting with the system & make the first backup:
* Homebrew install: * Homebrew install:
```console ```console
$ brew services start restic-automatic-backup-scheduler $ brew services start restic-automatic-backup-scheduler
``` ```
* `make` install: * `make` install:
```console ```console
$ launchctl bootstrap gui/$UID ~/Library/LaunchAgents/com.github.erikw.restic-automatic-backup-scheduler.plist $ launchctl bootstrap gui/$UID ~/Library/LaunchAgents/com.github.erikw.restic-backup.plist
$ launchctl enable gui/$UID/com.github.erikw.restic-automatic-backup-scheduler $ launchctl enable gui/$UID/com.github.erikw.restic-backup
$ launchctl kickstart -p gui/$UID/com.github.erikw.restic-automatic-backup-scheduler $ launchctl kickstart -p gui/$UID/com.github.erikw.restic-backup
``` ```
As a convenience, a shortcut for the above commands are `$ make activate-launchagent`. As a convenience, a shortcut for the above commands are `$ make activate-launchagent-backup`.
1. Watch the first backup progress from the log files: 1. Watch the first backup progress from the log files:
```console ```console
$ tail -f ~/Library/Logs/restic/restic_* $ tail -f ~/Library/Logs/restic/backup*
``` ```
1. Verify the backup 1. Verify the backup
```console ```console
$ restic snapshots $ restic snapshots
``` ```
1. (recommended) Enable the check job that verifies that the backups for the profile are all intact. 1. (recommended) Enable the check job that verifies that the backups for the profile are all intact.
TODO #81 * Homebrew install:
```console
$ brew services start restic-automatic-backup-scheduler-check
```
* `make` install:
```console
$ launchctl bootstrap gui/$UID ~/Library/LaunchAgents/com.github.erikw.restic-check.plist
$ launchctl enable gui/$UID/com.github.erikw.restic-check
$ launchctl kickstart -p gui/$UID/com.github.erikw.restic-check
```
As a convenience, a shortcut for the above commands are `$ make activate-launchagent-check`.
1. Consider more [optional features](#optional-features). 1. Consider more [optional features](#optional-features).
### Homebrew Setup Notes ### Homebrew Setup Notes
@@ -168,15 +178,15 @@ $ brew services stop restic-automatic-backup-scheduler
If `services start` fails, it might be due to previous version installed. In that case remove the existing version and try again: If `services start` fails, it might be due to previous version installed. In that case remove the existing version and try again:
```console ```console
$ launchctl bootout gui/$UID/com.github.erikw.restic-automatic-backup-scheduler $ launchctl bootout gui/$UID/com.github.erikw.restic-backup
$ brew services start restic-automatic-backup-scheduler $ brew services start restic-automatic-backup-scheduler
``` ```
### Make Setup Notes ### Make Setup Notes
Use the `disable` command to temporarily pause the agent, or `bootout` to uninstall it. Use the `disable` command to temporarily pause the agent, or `bootout` to uninstall it.
``` ```
$ launchctl disable gui/$UID/com.github.erikw.restic-automatic-backup-scheduler $ launchctl disable gui/$UID/com.github.erikw.restic-backup
$ launchctl bootout gui/$UID/com.github.erikw.restic-automatic-backup-scheduler $ launchctl bootout gui/$UID/com.github.erikw.restic-backup
``` ```
If you updated the `.plist` file, you need to issue the `bootout` followed by `bootrstrap` and `enable` sub-commands of `launchctl`. This will guarantee that the file is properly reloaded. If you updated the `.plist` file, you need to issue the `bootout` followed by `bootrstrap` and `enable` sub-commands of `launchctl`. This will guarantee that the file is properly reloaded.
@@ -252,7 +262,7 @@ Any system that has a cron-like system can easily setup restic backups as well.
**TL;DR setup** **TL;DR setup**
1. [Create](#1-create-backblaze-b2-account-bucket-and-keys) B2 bucket + credentials 1. [Create](#1-create-backblaze-b2-account-bucket-and-keys) B2 bucket + credentials
1. Install scripts, configs systemd units/timers: 1. Install scripts, configs systemd units/timers:
```console ```console
$ sudo make install-cron $ sudo make install-cron
@@ -297,7 +307,7 @@ and you should now see that all files have been changed like e.g.
+export RESTIC_PASSWORD_FILE="/etc/restic/pw.txt" +export RESTIC_PASSWORD_FILE="/etc/restic/pw.txt"
``` ```
Why? The OS specific TL;DR setups above all use the [Makefile](Makefile) or a package manager to install these files. The placeholder string `$INSTALL_PREFIX` is in the source files for portability reasons, so that the Makefile can support all different operating systems. `make` users can set a different `$PREFIX` when installing like `PREFIX=/usr/local make install-systemd`. Why? The OS specific TL;DR setups above all use the [Makefile](Makefile) or a package manager to install these files. The placeholder string `$INSTALL_PREFIX` is in the source files for portability reasons, so that the Makefile can support all different operating systems. `make` users can set a different `$PREFIX` when installing like `PREFIX=/usr/local make install-systemd`.
In this detailed manual setup we will copy all files manually to `/etc`and `/bin`. Thus we need to remove the placeholder string `$INSTALL_PREFIX` in the source files as a first step. In this detailed manual setup we will copy all files manually to `/etc`and `/bin`. Thus we need to remove the placeholder string `$INSTALL_PREFIX` in the source files as a first step.
@@ -317,7 +327,7 @@ For restic to be able to connect to your bucket, you want to in the B2 settings
#### 2. Configure B2 Credentials Locally #### 2. Configure B2 Credentials Locally
Put these files in `/etc/restic/`: Put these files in `/etc/restic/`:
* `_global.env.sh`: Fill this file out with your global settings including B2 keyID & applicationKey. * `_global.env.sh`: Fill this file out with your global settings including B2 keyID & applicationKey.
* `default.env.sh`: This is the default profile. Fill this out with bucket name, backup paths and retention policy. This file sources `_global.env.sh` and is thus self-contained and can be sourced in the shell when you want to issue some manual restic commands. For example: * `default.env.sh`: This is the default profile. Fill this out with bucket name, backup paths and retention policy. This file sources `_global.env.sh` and is thus self-contained and can be sourced in the shell when you want to issue some manual restic commands. For example:
```console ```console
$ source /etc/restic/default.env.sh $ source /etc/restic/default.env.sh
@@ -421,7 +431,7 @@ To create a different backup and use you can do:
```console ```console
# cp /etc/restic/default.env.sh /etc/restic/other.env.sh # cp /etc/restic/default.env.sh /etc/restic/other.env.sh
# vim /etc/restic/default.other.sh # Set backup path, bucket etc. # vim /etc/restic/default.other.sh # Set backup path, bucket etc.
# source /etc/restic/default.other.sh # source /etc/restic/default.other.sh
# restic_backup.sh # restic_backup.sh
``` ```