235 Commits

Author SHA1 Message Date
Erik Westrup
f17848bed2 Fix notification stats bug with more than 2 latest snapshots 2022-04-29 20:11:49 +02:00
Erik Westrup
d5497b04b4 Add missing $INSTALL_PREFIX in help text 2022-04-29 17:57:33 +02:00
Erik Westrup
855997d7d7 Update CHANGELOG.md 2022-04-14 09:35:27 +02:00
Erik Westrup
77b5a2c653 Update CHANGELOG.md 2022-04-11 21:09:18 +02:00
Erik Westrup
489046c603 Modified: README.md 2022-04-11 18:07:03 +02:00
Erik Westrup
6a43b95222 Fix systemd service OnFailure
Fixes #86
2022-04-11 17:48:00 +02:00
Erik Westrup
329fa40c30 Rename master branch to main 2022-04-05 08:28:16 +02:00
Matteo Mardegan
7c64d861ae correct the environtment file name 2022-03-22 09:27:26 +01:00
Erik Westrup
7ebeb00761 chmod a-x bin/*
The source files won't execute util after the build step. Remove
execution bit to not invite users to try to run these files.

Fixes #89
2022-02-27 22:14:21 +01:00
Erik Westrup
041d3374cc Fix service names in README.md
Fixes #88
2022-02-27 22:11:52 +01:00
Erik Westrup
5ed72bb87f Command for copy cron.d to crontab 2022-02-24 11:20:08 +01:00
Erik Westrup
9797543231 Update README.md 2022-02-24 05:50:10 +01:00
Erik Westrup
7e5496d00a Update README.md 2022-02-24 05:49:18 +01:00
Gerard Bosch
22d6af802f Update README: Metered connection check (#87)
* Update README: Metered connection check

Add a single command to copy and paste to install the metered connection
check.

* Reformat to be consistent with DIY style of optionals

Co-authored-by: Erik Westrup <erik.westrup@gmail.com>
2022-02-16 19:31:46 +01:00
Gerard Bosch
38b3243b4c Redirect stderr to systemd journal (#86)
Errors were not written to the journal as systemd-cat was only reading
stdin. Now, errors printed by restic are shown in the journal.
2022-02-16 13:51:28 +01:00
Erik Westrup
d5a141fc9a Modified: CHANGELOG.md 2022-02-16 12:23:59 +01:00
Erik Westrup
f4d1d6f1b2 chmod u+x bin/ to try execute scripts in build dir
before install step
2022-02-16 12:20:54 +01:00
Gerard Bosch
14f9741b82 Add docopt.sh option: DOCOPT_OPTIONS_FIRST=true
This will allow to use `resticw` as a true wrapper, enabling to pass
restic options through it, for example:

```
resticw --profile profileX snapshots --compact
```

will now be suported by the wraper :)
2022-02-16 12:19:44 +01:00
Gerard Bosch
43b718ea86 Fix README/resticw commands (#84)
* Fix README/resticw commands

As per restic 0.12.1, the special snapshot ID `latest` is not supported by `diff` command.

* Update `resticw diff` description
2022-02-16 10:14:01 +01:00
Erik Westrup
ef059ebd68 Update macOS notifier examples URL 2022-02-16 09:39:38 +01:00
Gerard Bosch
de1b851d3d Update README/Metered Connections (#83) 2022-02-15 19:30:08 +01:00
Erik Westrup
b00ae89812 Add desktop notification screenshot 2022-02-15 19:19:46 +01:00
Erik Westrup
2c14351cd4 Document desktop notifications in README 2022-02-15 18:38:14 +01:00
Erik Westrup
cfbc6c6c51 Merge pull request #82 from gerardbosch/patch-1
Minor fix on README
2022-02-15 18:24:14 +01:00
Gerard Bosch
e8a2647cae Minor fix on README 2022-02-15 18:22:45 +01:00
Erik Westrup
4952cc3ee1 Modified: CHANGELOG.md 2022-02-15 17:49:11 +01:00
Erik Westrup
3ecc3c3510 Consistent default values 2022-02-15 17:47:51 +01:00
Erik Westrup
65172e650b Merge pull request #76 from gerardbosch/feature/desktop-notifications
Add optional desktop notifications
2022-02-15 17:45:07 +01:00
Gerard Bosch
705248ee00 Final cleanup 2022-02-15 16:06:19 +01:00
Gerard Bosch
be2c3163a8 Rework notification data fetching: Use stats+diff 2022-02-15 13:50:10 +01:00
Erik Westrup
84e083c62e Modified: README.md 2022-02-15 12:46:27 +01:00
Erik Westrup
c5c4da3618 Modified: CHANGELOG.md 2022-02-15 12:43:52 +01:00
Erik Westrup
ef731f33b2 Merge branch 'launch_check'
* launch_check:
  Modified: CHANGELOG.md
  Add restic-check launchagent
  Rename logs
  Rename backup launchagent
2022-02-15 12:43:05 +01:00
Erik Westrup
92205981b6 Modified: CHANGELOG.md 2022-02-15 12:42:51 +01:00
Erik Westrup
37f4775eaa Add restic-check launchagent 2022-02-15 12:42:25 +01:00
Erik Westrup
4f7da2c975 Rename logs 2022-02-15 11:36:20 +01:00
Erik Westrup
7bc15e2254 Rename backup launchagent 2022-02-15 11:33:30 +01:00
Erik Westrup
7c0b89a9c3 Set LICENSE properly to be detected by GitHub 2022-02-15 08:17:07 +01:00
Erik Westrup
09692f7a5f Update LICENSE 2022-02-14 22:36:17 +01:00
Erik Westrup
25207799e6 Update LICENSE 2022-02-14 22:35:38 +01:00
Erik Westrup
e3a5a2d250 Update LICENSE 2022-02-14 22:33:35 +01:00
Erik Westrup
8fd61d768b Modified: README.md 2022-02-14 17:06:09 +01:00
Erik Westrup
ce936fceb3 Group notes 2022-02-14 16:53:07 +01:00
Erik Westrup
eac2177042 Improve detailed section 2022-02-14 16:50:44 +01:00
Erik Westrup
ef1c464efd Explain sed 2022-02-14 16:47:37 +01:00
Erik Westrup
418217f746 Modified: README.md 2022-02-14 16:39:51 +01:00
Erik Westrup
86ece14d73 Make B2 creation step easier 2022-02-14 16:39:14 +01:00
Erik Westrup
9f1158f0cb Plus icon 2022-02-14 16:34:22 +01:00
Erik Westrup
11d600f7c0 How to use cron_mail 2022-02-14 16:30:20 +01:00
Erik Westrup
d8cfd6f6a6 Restructure cron section
Part of #79
2022-02-14 16:26:54 +01:00
Erik Westrup
58cfbba0cb Add icon for manual section 2022-02-14 16:13:13 +01:00
Erik Westrup
66c47afd77 Add SVGs for all sections 2022-02-14 16:07:04 +01:00
Erik Westrup
92d9e7cd1f Collected changes 2022-02-14 16:06:01 +01:00
Erik Westrup
1dbc2d4799 Add SVG 2022-02-14 16:05:19 +01:00
Erik Westrup
23e134c5e7 Modified: README.md 2022-02-14 15:59:57 +01:00
Erik Westrup
72088212af Restructure windows
Part of #79
2022-02-14 15:57:01 +01:00
Erik Westrup
148e43c8c3 Collected changes 2022-02-14 15:47:49 +01:00
Erik Westrup
8dd1beb3d2 Restructure systemd & launchagent
part of #79
2022-02-14 15:43:42 +01:00
Erik Westrup
99e51e297a Remove Variations section
As the new EXTRA_ARGS covers this

Part of #79
2022-02-14 12:10:35 +01:00
Erik Westrup
c1ce0d069b Restructure README #79 2022-02-14 12:09:41 +01:00
Erik Westrup
da2ad5f6f3 Modified: README.md 2022-02-13 19:46:19 +01:00
Erik Westrup
9b4018ce62 Modified: README.md 2022-02-13 19:45:45 +01:00
Erik Westrup
3a8ca5a815 Modified: CHANGELOG.md 2022-02-13 19:20:15 +01:00
Erik Westrup
37e38b4cb5 Modified: CHANGELOG.md 2022-02-13 19:20:02 +01:00
Erik Westrup
8f734ccc69 How to disable old launchagent 2022-02-13 19:19:00 +01:00
Erik Westrup
56c4fca8f6 Merge branch 'optionals'
* optionals:
  Made scripts optional
  Cron not using cron_email by default
2022-02-13 19:18:37 +01:00
Erik Westrup
9449d78596 Made scripts optional 2022-02-13 19:15:55 +01:00
Erik Westrup
c50e66b48f Cron not using cron_email by default 2022-02-13 18:47:31 +01:00
Erik Westrup
c1edb2efe1 Remove execution bit on bin/ as these are set when installing 2022-02-13 18:45:52 +01:00
Erik Westrup
8275f708ea Link to schedtask doc 2022-02-13 18:37:04 +01:00
Erik Westrup
eac8b97932 Modified: CHANGELOG.md 2022-02-13 17:53:36 +01:00
Erik Westrup
bb40afa67a Rename launchagent for consistency 2022-02-13 17:50:10 +01:00
Erik Westrup
cacfe58269 Merge branch 'rename_repo'
* rename_repo:
  replace old with new name
  Update into
2022-02-13 16:40:35 +01:00
Erik Westrup
96e9bc23ab replace old with new name 2022-02-13 16:39:25 +01:00
Erik Westrup
911f9f44d2 Update into 2022-02-13 16:33:12 +01:00
Erik Westrup
abac4b11ba Remove duplicated readme line 2022-02-13 15:54:05 +01:00
Erik Westrup
cfaab7eb72 Move schedtasks to own subdir 2022-02-13 11:46:02 +01:00
Erik Westrup
978ca2becb Merge branch 'winfix'
* winfix:
  Modified: Makefile
  Add uninstall
  rename schedtask ps1
  scoop prefix git
2022-02-13 11:35:08 +01:00
Erik Westrup
6cb6b41066 Modified: Makefile 2022-02-13 11:32:58 +01:00
Erik Westrup
16302e5d35 Add uninstall 2022-02-13 11:29:04 +01:00
Erik Westrup
aeec8991b6 rename schedtask ps1 2022-02-13 11:14:04 +01:00
Erik Westrup
0f5a8f0ca5 scoop prefix git 2022-02-13 11:10:38 +01:00
Erik Westrup
4a2ca821d7 Merge pull request #78 from erikw/windows
Add Windows support with TaskScheduler
2022-02-12 20:46:39 +01:00
Erik Westrup
3b08a42495 Modified: CHANGELOG.md 2022-02-12 20:43:25 +01:00
Erik Westrup
9ec3b1cb79 Add screenshot
Fixes #52
2022-02-12 20:40:07 +01:00
Erik Westrup
35c1c45352 Modified: install_restic_scheduledtask.ps1 2022-02-12 20:14:34 +01:00
Erik Westrup
f45b6298a4 Modified: install_restic_scheduledtask.ps1 2022-02-12 20:10:33 +01:00
Erik Westrup
9fa5074699 Modified: README.md install_restic_scheduledtask.ps1 2022-02-12 20:07:58 +01:00
Erik Westrup
71d0e8c5c1 dynamic fs args 2022-02-12 19:59:03 +01:00
Erik Westrup
0d92cca32d windows install instructions 2022-02-12 19:47:22 +01:00
Erik Westrup
30d18b8b0e shebang 2022-02-12 19:15:59 +01:00
Erik Westrup
9ff51d691a WIP make target for scheduledtask 2022-02-12 19:11:47 +01:00
Erik Westrup
edaafdf10a Fix phony target 2022-02-12 19:00:15 +01:00
Erik Westrup
07541cfaf2 Homebrew servcie control 2022-02-12 17:59:41 +01:00
Erik Westrup
399e7cd4ce Fix launchagent macro 2022-02-12 17:40:26 +01:00
Erik Westrup
0f3d39dfe1 Custom launchagent dir 2022-02-12 17:33:29 +01:00
Gerard Bosch
7dde85f25e Reformat 2022-02-12 10:02:03 +01:00
Gerard Bosch
9ebc9ea641 Replace 'here string' syntax with echo + pipe 2022-02-12 08:47:09 +01:00
Erik Westrup
f7c13f7290 Add homebrew formula
Fixes #73
2022-02-11 23:06:17 +01:00
Erik Westrup
c07f0ca027 Modified: CHANGELOG.md 2022-02-11 22:13:53 +01:00
Erik Westrup
a084830441 Modified: CHANGELOG.md 2022-02-11 22:13:41 +01:00
Erik Westrup
7d67142b6e Use correct profile in launchagent 2022-02-11 22:13:19 +01:00
Erik Westrup
b838ebcb1a Modified: CHANGELOG.md 2022-02-11 22:08:10 +01:00
Erik Westrup
8f118b6ddf Allow make to override etc dir 2022-02-11 22:07:26 +01:00
Erik Westrup
fc01b56e1e Modified: CHANGELOG.md 2022-02-11 21:47:34 +01:00
Erik Westrup
289016f8ea No need to write perm on scripts 2022-02-11 21:46:56 +01:00
Erik Westrup
0c43f93901 Modified: CHANGELOG.md 2022-02-11 20:45:49 +01:00
Erik Westrup
2a287910cb Allow override INSTALL_PREFIX on make cli 2022-02-11 20:34:33 +01:00
Erik Westrup
5f3ebd5b9d resticw: fix install prefix 2022-02-11 10:27:35 +01:00
Erik Westrup
8111a5f663 Bash is dependency 2022-02-10 10:58:41 +01:00
Erik Westrup
ae900a3834 Merge pull request #75 from gerardbosch/patch-1 2022-02-09 18:47:21 +01:00
Gerard Bosch
1db2395353 Minor typos 2022-02-09 18:41:20 +01:00
Gerard Bosch
3f000ce137 Add optional desktop notifications
When backing up a desktop system it is handy to have regular and
immediate feedback about backups. This features a notification of backup
stats summary, including the added size to the repository, so you can
have a quick follow-up about what are you uploading to the repository.

This may be very useful if you upload by accident a big file or directory
which should be in your exclusions, so it warns you and gives the chance
to fix the exclusions and remove the undesired snapshot later.
2022-02-09 18:19:48 +01:00
Erik Westrup
59899f5683 Merge pull request #74 from erikw/launchagent
Add macOS launchagent support
2022-02-09 17:25:31 +01:00
Erik Westrup
4cae1ca2f0 Update CHANGELOG.md
Fixes #51
2022-02-09 17:23:09 +01:00
Erik Westrup
caf95ef393 Add make target to activate launchagent 2022-02-09 17:16:18 +01:00
Erik Westrup
99b0bf5ac5 Modified: README.md 2022-02-09 16:29:14 +01:00
Erik Westrup
d20954b9f4 Modified: README.md 2022-02-09 15:58:26 +01:00
Erik Westrup
991476715a install instructions 2022-02-09 15:56:37 +01:00
Erik Westrup
c4429681d8 Modified: Library/LaunchAgents/com.github.erikw.restic-automatic-backup.plist 2022-02-09 15:49:25 +01:00
Erik Westrup
f073f10ddf Modified: Library/LaunchAgents/com.github.erikw.restic-automatic-backup.plist 2022-02-09 15:47:04 +01:00
Erik Westrup
a28160371c Fix log path 2022-02-09 15:45:49 +01:00
Erik Westrup
1977b3c67f Add OS icons 2022-02-09 15:31:42 +01:00
Erik Westrup
9412a8750c trim 2022-02-09 15:08:57 +01:00
Erik Westrup
d38616d697 Path working 2022-02-09 14:58:31 +01:00
Erik Westrup
c8ab0b4421 iterate 2022-02-09 14:04:07 +01:00
Erik Westrup
c1c25d718c schedule 2022-02-09 13:32:25 +01:00
Erik Westrup
f7f69a0142 instructions 2022-02-09 13:32:19 +01:00
Erik Westrup
2f4f48a59b Modified: CHANGELOG.md 2022-02-09 13:09:10 +01:00
Erik Westrup
e9d77d0a31 Surpress mkdir 2022-02-09 13:08:12 +01:00
Erik Westrup
bd3123bef4 Modified: Makefile 2022-02-09 13:08:12 +01:00
Erik Westrup
ddd2233c58 Install LaunchAgent in user Library 2022-02-09 13:08:12 +01:00
Erik Westrup
7973eedae4 Modified: Makefile 2022-02-09 13:08:12 +01:00
Erik Westrup
db26c4e517 Launch instructions 2022-02-09 13:08:12 +01:00
Erik Westrup
193bf926cf PoC souce files 2022-02-09 13:08:12 +01:00
Erik Westrup
8ea1d70a15 Merge pull request #72 from gerardbosch/patch-1
Update comment
2022-02-09 13:07:38 +01:00
Gerard Bosch
8eb7d56fdd Update comment 2022-02-09 12:59:54 +01:00
Erik Westrup
03d9399686 Merge pull request #71 from gerardbosch/fix/resticw-arguments
Fix resticw arguments line
2022-02-08 18:57:42 +01:00
Gerard Bosch
59da5da0da Fix resticw arguments line
It needs to be treated as an array so that complex commands like
`stats latest` work.
2022-02-08 18:51:20 +01:00
Erik Westrup
7a1409ca03 Allow users to execute scripts 2022-02-08 18:20:49 +01:00
Erik Westrup
b5dcce5d2a Adjust comment 2022-02-08 17:58:46 +01:00
Erik Westrup
e0139ff9c8 Address linter issues 2022-02-08 17:33:24 +01:00
Erik Westrup
78f5cb7b62 Modified: CHANGELOG.md 2022-02-08 17:27:22 +01:00
Erik Westrup
ab928d0dfd Rename *.env files to *.env.sh
Fixes #66
2022-02-08 17:26:08 +01:00
Erik Westrup
27d757fc29 Merge pull request #65 from erikw/fix/49
Truly support custom `PREFIX=` install
2022-02-08 17:17:25 +01:00
Erik Westrup
6c1d182e21 Only sleep in service if other restic running 2022-02-08 17:13:26 +01:00
Erik Westrup
e688499838 Surpress mkdir in make install 2022-02-08 16:49:10 +01:00
Gerard Bosch
2dbd618344 Fix README: Enabling/starting systemd unit (#69)
systemctl start restic-backup@default.timer was failing with the following message:

```console
❯ sudo systemctl start restic-backup@default.timer
Failed to start restic-backup@default.timer: Unit nm-unmetered-connection.service failed to load properly, please adjust/correct and reload service manager: Device or resource busy
See system logs and 'systemctl status restic-backup@default.timer' for details.
```

and by doing `systemctl daemon-reload` before that command solved the issue.

But according to `man systemd` in order to avoid this, it seems that `enable` command must be issued before `start`, as it reloads the system manager configuration (in a way equivalent to daemon-reload). So, once properly sorted, it seems that both commands can be fused into a single `enable --now` command :)
2022-02-08 16:43:30 +01:00
Gerard Bosch
b4684d3cfb Fix README: Backup frequency (#68) 2022-02-08 13:01:32 +01:00
Erik Westrup
645df1a0d4 Move /sbin to /bin
As more users would have /bin than /sbin in their PATH
2022-02-07 18:22:36 +01:00
Erik Westrup
54e2d17e23 Modified: CHANGELOG.md Makefile 2022-02-06 15:28:56 +01:00
Erik Westrup
0fc7c29c2f Push image down 2022-02-06 15:26:43 +01:00
Erik Westrup
a57971e27e Add navigation tip 2022-02-06 15:25:45 +01:00
Erik Westrup
eb6e7b4a8d Begin structure README after system setup 2022-02-06 15:16:19 +01:00
Erik Westrup
c01bb5a00e Fix broken help target 2022-02-06 15:10:31 +01:00
Erik Westrup
8db3d27174 Rename top-level install targets
"make install" -> "make install-systemd"
so that I could now add a
""make install-cron"
target.

Soon there will be
"make install-macos"
"make install-windows"

Thus it does not make sense to have a general `make install` anymore.
2022-02-06 14:49:00 +01:00
Erik Westrup
1ee0e2c919 Fix cron with multiple profiles 2022-02-06 14:36:41 +01:00
Erik Westrup
21bb0535ee Note about make needing to be GNU make 2022-02-05 15:56:32 +01:00
Erik Westrup
0255ad03d8 Format comments 2022-02-05 15:52:36 +01:00
Erik Westrup
5942762514 Correct build source depdendency 2022-02-05 15:48:58 +01:00
Erik Westrup
6b315814d0 spell 2022-02-05 15:45:45 +01:00
Erik Westrup
dd092af6a1 Remove legacy .template 2022-02-05 15:43:57 +01:00
Erik Westrup
7f86dbd9f4 Consistent macro names in Makefile 2022-02-05 15:40:00 +01:00
Erik Westrup
494bcc29f0 s/SCRIPTS/SCRIPT/g 2022-02-05 15:40:00 +01:00
Erik Westrup
dc28ad2b52 Update README.md
Co-authored-by: Gerard Bosch <30733556+gerardbosch@users.noreply.github.com>
2022-02-05 15:28:14 +01:00
Erik Westrup
c43367fab3 Simpler sed expression 2022-02-05 15:27:43 +01:00
Erik Westrup
67cd6ee94b Add back dependency on builddir 2022-02-05 15:18:39 +01:00
Erik Westrup
a117c95508 Format Makefile 2022-02-05 15:13:01 +01:00
Erik Westrup
fc04da5a2d Fix comment 2022-02-05 15:04:44 +01:00
Erik Westrup
25666f17a0 Remove old out-commented solution 2022-02-05 15:02:53 +01:00
Erik Westrup
ccf39e822d Remove comment not relevant 2022-02-05 15:02:24 +01:00
Erik Westrup
1a890de7ac Remove target dependency that was not cleaned up before 2022-02-05 14:55:24 +01:00
Erik Westrup
4236c93b57 spell 2022-02-05 14:53:32 +01:00
Erik Westrup
861535a86e Check properly for GNU install 2022-02-05 14:48:38 +01:00
Erik Westrup
9760cd05ec Truly support custom PREFIX= install
* Before, doing `$ PREFIX=/usr/local make install` would install files to`/usr/local/usr/local..` which is wrong
* With this PR, files will be installed to the expected location e.g. `/usr/local/etc/restic`
* `Makefile` almost completely rewritten
   * As e.g. `default.env` would source `_global.env`, `default.env` must be edited to find the right location of `_global.env` depending on what `$PREFIX` was set to.
   * see documented build stages in the `Makefile` itself.
   *  Made sure that the rules are correct so that only modifed files are installed, not all at once unnecessarily like before.
* A sub-goal was that the [PKGBUILD](https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=restic-systemd-automatic-backup#n20) for Arch should not need to do any custom install configuration, to keep everything easier to maintain. `$ make install` should work out of the box for Arch.
* Additionally added the `-b` flag to `install(1)` that makes a backup of existing `etc/restic/*` files before installing a newer version.

Fixes #49
2022-02-03 20:52:00 +01:00
Gerard Bosch
3852e305b6 Add resticw (restic wrapper) utility (#60)
The script provides a convenient way to load environment config, deal
with profiles and act as a pass-through to restic. The overall thing is
to improve the UX when running restic, integrating the features this
project provides.

## Note

The script itself is a very simple thing.
The command line parser is auto-generated using docopt.sh driven from
the script's DOC. It can be refreshed upon DOC changes with:
`docopt.sh path/to/resticw`.

## How to use it

### Examples

```console
sudo resticw stats latest

sudo resticw -p profileA snapshots
```

### Help

```console
❯ resticw --help
A little wrapper over restic just to handle profiles and environment loading.

  It loads the backup profile/environment in a subshell to avoid any credential leak (Note: Run it with sudo so it can load the environment).

  Usage:
    resticw [options] <restic_arguments>

    The restic_arguments is just the regular unwrapped restic arguments, e.g. stats latest

  Options:
    -p --profile=<name>        Specify the profile to load or use default [default: default].

  Examples:
    sudo resticw --profile profileA snapshots
    sudo resticw stats latest  # this will use the profile: default
```



Co-authored-by: Erik Westrup <erik.westrup@gmail.com>
2022-02-03 20:40:45 +01:00
Erik Westrup
73bce43f7d s/TODO/*EDIT*/ to not confuse with development TODOs 2022-02-03 14:41:21 +01:00
Erik Westrup
7499dbefcd Modified: .github/workflows/linter.yml 2022-02-02 08:10:19 +01:00
Erik Westrup
10c46e8d54 Manually trigger linter 2022-02-02 07:57:02 +01:00
Erik Westrup
ce73193128 Trigger linter on all usr/local/sbin edits 2022-02-02 07:53:30 +01:00
Erik Westrup
90358c4f7a README: project scope 2022-02-01 19:51:35 +01:00
Erik Westrup
7f3647fb0c Rel should have been major as of breaking change 2022-02-01 19:41:02 +01:00
Erik Westrup
dd073dfc73 Fix arguments by using arrays to buildup 2022-02-01 19:24:38 +01:00
Erik Westrup
b23d552f0b Allow set but empty envvars 2022-02-01 18:31:27 +01:00
Erik Westrup
4142dbf20e Modified: CHANGELOG.md 2022-02-01 18:18:21 +01:00
Erik Westrup
629fd4c182 Align tag badge next to aur ver badge 2022-02-01 18:05:02 +01:00
Erik Westrup
7808dd6ecc Revert "Test that lint fails"
This reverts commit 082089a203.
2022-02-01 18:04:37 +01:00
Erik Westrup
082089a203 Test that lint fails 2022-02-01 18:03:11 +01:00
Erik Westrup
5e51341c51 Trigger linter on self-modify 2022-02-01 17:53:30 +01:00
Erik Westrup
130372e641 Restrict linting to bash 2022-02-01 17:52:39 +01:00
Erik Westrup
b3c64ca2ee Add lint badge 2022-02-01 17:37:23 +01:00
Erik Westrup
33a16ddb75 Modified: usr/local/sbin/restic_backup.sh 2022-02-01 17:20:30 +01:00
Erik Westrup
f1304f7db9 Modified: README.md 2022-02-01 17:20:00 +01:00
Erik Westrup
fad429fd34 Add shellcheck linter workflow
Fixes #57
2022-02-01 17:19:08 +01:00
Erik Westrup
9b7db6d999 Run shellcheck on all shellscripts 2022-02-01 17:15:47 +01:00
Erik Westrup
79d13a1e64 Prefix envvars with RESTIC_ for consistencty
Fix #63
2022-02-01 16:59:12 +01:00
Erik Westrup
4e8b8adff6 Rename backup_exclude to backup_exclude.txt
Fixes #64
2022-02-01 16:52:20 +01:00
Erik Westrup
687111fddf Assert that all envvars are set in scripts
Remind user to source profile before executing.

Fixes #62
2022-02-01 16:45:19 +01:00
Erik Westrup
f4b90c2499 Add RESTIC_VERBOSITY_LEVEL
Fixes #50
2022-02-01 16:19:05 +01:00
Erik Westrup
341f3e79ec Allow extra args to restic-backup with RESTIC_BACKUP_EXTRA_ARGS
Fixes #56
2022-02-01 16:15:18 +01:00
Erik Westrup
a4cd65db5a Align terminology of credentials with B2
Fixes #59
2022-02-01 16:04:40 +01:00
Erik Westrup
84bf1cfcd3 Modified: LICENSE 2022-02-01 14:59:09 +01:00
Erik Westrup
c51e5ffb03 Add release instructions 2022-02-01 13:31:56 +01:00
Erik Westrup
cf978b00fe Modified: CHANGELOG.md 2022-02-01 13:08:49 +01:00
Erik Westrup
db27be517f Modified: CHANGELOG.md 2022-02-01 13:08:28 +01:00
Erik Westrup
3c9fa21b2e Modified: CHANGELOG.md 2022-02-01 13:07:55 +01:00
Gerard Bosch
d8f25cdf88 Multiple profile configuration (#45)
Adds possibility to have multiple profiles with different backup paths, retention etc.

Co-authored-by: Matt Feifarek <matt.feifarek@gmail.com>
Co-authored-by: Erik Westrup <erik.westrup@gmail.com>
2022-02-01 10:09:36 +01:00
Erik Westrup
16dbe699ab Revert "Empty line"
This reverts commit a011bb6dd2.
2022-01-30 20:09:26 +01:00
Erik Westrup
a011bb6dd2 Empty line 2022-01-30 20:09:07 +01:00
Erik Westrup
15a02d4bd3 Add note about Make as recommended reqirements for installing 2022-01-19 12:02:20 +01:00
Erik Westrup
3a51ffb795 Modified: README.md 2022-01-18 12:01:58 +01:00
Erik Westrup
1b57069d30 README: anecdotal pricing 2022-01-18 12:00:29 +01:00
Erik Westrup
5eeab95c84 Create backup of existing config files on install 2022-01-17 15:53:17 +01:00
Erik Westrup
3e22d8f99d Remove local generated files from templates
As discussed in #46, it's not necessary to keep the local files
generated from the templates as the users should edit the installed
confs, not in the git.
2022-01-17 15:51:39 +01:00
Erik Westrup
9cb85a8571 Add notes to README about development 2022-01-17 15:51:33 +01:00
Erik Westrup
7141426699 Spell 2022-01-17 13:26:08 +01:00
Erik Westrup
9e292218cf Spell fix 2022-01-17 13:16:49 +01:00
Erik Westrup
546845c687 Merge pull request #48 from erikw/fix/46
Fix make install failing when run multiple times
2022-01-17 09:52:08 +01:00
Erik Westrup
828ff79e7a Fix make install failing when run multiple times
Due to the security feature preventing checking in B2 credentials to
git, the install target would get the same file twice due to pattern
subsitution.

This issue is not visible with BSD install as it ignores the issue, but
GNU install wil fail.

Fixes #46
2022-01-16 20:59:50 +01:00
Erik Westrup
8932e60e4a Spell fixes 2022-01-16 20:56:00 +01:00
Erik Westrup
36a90b963e Fix URL 2022-01-01 13:59:45 +01:00
Erik Westrup
5a3c8afb43 Link to #44
Fixes #44
2022-01-01 13:58:33 +01:00
Erik Westrup
aff6211407 Modified: README.md 2021-12-18 11:11:11 +01:00
Erik Westrup
3e9233c0fc Merge pull request #43 from erikw/fix/42
Add comment explaining dynamic backup path
2021-12-04 18:40:35 +01:00
Erik Westrup
1d1145296d Add comment explaining dynamic backup path
Fixes #42
2021-12-04 18:38:34 +01:00
Erik Westrup
ceac35abc8 Remove SLOC badge 2021-12-03 19:51:24 +01:00
Erik Westrup
d01d2995f5 Update CHANGELOG.md 2021-12-03 18:40:24 +01:00
Erik Westrup
97a9475550 Update CHANGELOG.md 2021-12-03 18:35:51 +01:00
Erik Westrup
2e55d1cfef Modified: CHANGELOG.md 2021-12-03 18:35:07 +01:00
Erik Westrup
7a8861adc4 Merge pull request #41 from erikw/fix/40
Fix installation of template files
2021-12-03 18:32:26 +01:00
Erik Westrup
bf9b29575a Update README.md 2021-12-03 18:16:39 +01:00
Erik Westrup
428f4be872 Fix installation of template files
The target that copies the *.template files (#15) had a '/' prefixed which
should not be there. The copy should be locally here. The real install
happens in the install-conf that respects $PREFIX

Fixes #40
2021-12-03 17:09:14 +01:00
Erik Westrup
99f4ba4436 Add AUR maintainer badge 2021-12-02 15:31:33 +01:00
Erik Westrup
ae2b122134 Add AUR instructions 2021-12-02 15:28:40 +01:00
Erik Westrup
8e1853458c Add AUR badge 2021-12-02 15:26:55 +01:00
39 changed files with 1485 additions and 322 deletions

17
.editorconfig Normal file
View File

@@ -0,0 +1,17 @@
root = true
[*]
indent_style = tab
indent_size = 4
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true
#max_line_length = 120
[Makefile]
# Enforce tab (Makefiles require tabs)
indent_style = tab
[*.md]
trim_trailing_whitespace = false

33
.github/workflows/linter.yml vendored Normal file
View File

@@ -0,0 +1,33 @@
name: Lint Code Base
on:
workflow_dispatch:
push:
branches: main
paths:
- '**.sh'
- '.github/workflows/linter.yml'
- 'bin/**'
pull_request:
branches: main
paths:
- '**.sh'
- '.github/workflows/linter.yml'
- 'bin/**'
jobs:
build:
name: Lint Code Base
runs-on: ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout@v2
with:
# Full git history is needed to get a proper list of changed files within `super-linter`
fetch-depth: 0
- name: Lint Code Base
uses: github/super-linter@v4
env:
VALIDATE_ALL_CODEBASE: true
VALIDATE_BASH: true
DEFAULT_BRANCH: main
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

10
.gitignore vendored
View File

@@ -1,2 +1,8 @@
/etc/restic/b2_pw.txt
/etc/restic/b2_env.sh
# make install
/build
# IntelliJ
.idea/
*.iml
# VSCode
.vscode/

View File

@@ -6,6 +6,137 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [7.3.4] - 2022-04-29
### Fixed
- Backup stats notifications: fix issue where `restic snapshots --latest 2` will show more than two snapshots due to different backup paths used.
## [7.3.3] - 2022-04-14
### Fixed
- Trying to fix broken Homebrew bottles due to GitHub API issues.
## [7.3.2] - 2022-04-11
### Fixed
- Trying to fix broken Homebrew bottles
## [7.3.1] - 2022-04-11
### Fixed
- `resticw` is now a true wrapper in that it support `--` args to restic.
- OnFailure no longer masked by the stderr redirect to systemd-cat. [#86](https://github.com/erikw/restic-automatic-backup-scheduler/pull/86)
## [7.3.0] - 2022-02-15
### Added
- optional user-controlled notification. See `RESTIC_NOTIFY_BACKUP_STATS` and in `backup.sh`.
## [7.2.0] - 2022-02-15
### Added
- restic-check LaunchAgent.
### Changed
- [README.md](README.md) is restructured with easier TL;DR for each OS and a more general detailed section for the interested.
## [7.1.0] - 2022-02-13
### Changed
- Minimize base install. The following features are now opt-in: nm-unmetered detection, cron_mail, systemd-email.
## [7.0.0] - 2022-02-13
### Changed
- Renamed project from `restic-systemd-automatic-backup` to `restic-automatic-backup-scheduler` to fit all now supported setups.
## [6.0.0] - 2022-02-12
### Added
- Windows support with native ScheduledTask! New target `$ make install-schedtask` for Windows users.
## [5.3.1] - 2022-02-12
### Fixed
- Launchagentdir make macro
## [5.3.0] - 2022-02-12
### Added
- Allow custom launchagent dir, used by Homebrew.
## [5.2.1] - 2022-02-11
### Added
- Homebrew Formula at [erikw/homebrew-tap](https://github.com/erikw/homebrew-tap). You can now install with `$ brew install erikw/tap/restic-automatic-backup-scheduler`!
### Fixed
- Use default profile in LaunchAgent.
## [5.2.0] - 2022-02-11
### Added
- Make option to override destination dir for configuration files. Needed for Homebrew.
### Changed
- Write permissions on installed scripts removed (0755 -> 0555). Homebrew was complaining.
## [5.1.0] - 2022-02-11
### Added
- macos LaunchAgent support. Install with `make install-launchagent` and activate with `make activate-launchagent`. See [README.md](README.md) for details.
- make option INSTALL_PREFIX to make PKGBUILD and such easier to write.
## [5.0.0] - 2022-02-08
### Added
- `resticw` wrapper for working with different profiles without the need to source the profiles first.
- `$ make install-systemd` will now make a timestamped backup of any existing `/etc/restic/*` files before installing a newer version.
- `$ make install-cron` for installing the cron-job.
### Changed
- **BREAKING CHANGE** moved systemd installation with makefile from `/etc/systemd/system` to `/usr/lib/systemd/system` as this is what packages should do. This is to be able to simplify the arch [PKGBUILD](https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=restic-automatic-backup-scheduler) so that it does not need to do anything else than `make install`.
- If you upgrade form an existing install, you should disable and then re-enable the timer, so that the symlink is pointing to the new location of the timer.
```console
# systemctl disable restic-backup@<profile>.timer
# systemctl enable restic-backup@<profile>.timer
```
- **BREAKING CHANGE** moved script installation with makefile from `/usr/local/sbin` to `/bin` to have a simpler interface to work with `$PREFIX`.
- **BREAKING CHANGE** renamed `etc/restic/*.env` files to `etc/restic/*.env.sh` to clearly communicate that it's a shell script that will be executed (source), and also hint at code editors what file this is to set corect syntax highligting etc. This also enables the shellcheck linter to work more easily on these files as well.
- Renamed top level make install targets. The old `$ make install` is now `$ make install-systemd`
### Fixed
- Installation with custom `PREFIX` now works properly with Make: `$ PREFIX=/usr/local make install` whill now install everything at the expected location. With this, it's easy to use this script as non-root user on e.g. an macOS system.
## [4.0.0] - 2022-02-01
### Fixed
- Use arrays to build up command lines. When fixing `shellcheck(1)` errors, quotes would disable expansion on e.g. $RESTIC_BACKUP_PATHS
- **BREAKING CHANGE** `RESTIC_BACKUP_PATHS` is now a string with `:` separated values
## [3.0.1] - 2022-02-01
### Fixed
- Environment variable assertion should allow empty values e.g. `RESTIC_BACKUP_EXTRA_ARGS`
## [3.0.0] - 2022-02-01
### Added
- Allow extra arguments to restic-backup with `$RESTIC_BACKUP_EXTRA_ARGS`.
- Add `$RESTIC_VERBOSITY_LEVEL` for debugging.
- Assertion on all needed environment variables in the backup and check scripts.
- Added linter (`shellcheck(1)`) that is run on push and PRs.
### Changed
- **BREAKING CHANGE** renamed
- `/etc/restic/backup_exclude` to `/etc/restic/backup_exclude.txt`
- `<backup-dest>/.backup_exclude` to `<backup-dest>/.backup_exclude.txt`.
- **BREAKING CHANGE** renamed envvars for consistency
- `BACKUP_PATHS` -> `RESTIC_BACKUP_PATHS`
- `BACKUP_TAG` -> `RESTIC_BACKUP_TAG`
- `RETENTION_DAYS` -> `RESTIC_RETENTION_DAYS`
- `RETENTION_WEEKS` -> `RESTIC_RETENTION_WEEKS`
- `RETENTION_MONTHS` -> `RESTIC_RETENTION_MONTHS`
- `RETENTION_YEARS` -> `RESTIC_RETENTION_YEARS`
- Align terminology used in README with the one used by B2 for credentials (keyId + applicationKey pair).
## [2.0.0] - 2022-02-01
### Changed
- **BREAKING CHANGE** [#45](https://github.com/erikw/restic-automatic-backup-scheduler/pull/45): multiple backup profiles are now supported. Please backup your configuration before upgrading. The setup of configuration files are now laied out differently. See the [README.md](README.md) TL;DR setup section.
- `restic_backup.sh` has had variables extracted to profiles instead, to allow for configuration of different backups on the same system.
- `b2_env.sh` is split to two files `_global.env` and `default.env` (the default profile). `_global.env` will have B2 accountID and accountKey and `default.env` has backup paths, and retention.
- `b2_pw.sh` renamed to pw.txt
### Fixed
- `restic_backup.sh` now finds `.backup_exclude` files on each backup path as intended.
- Install executeables to `$PREFIX/sbin` instead of `$PREFIX/user/local/sbin`, so that `$ PREFIX=/usr/local make install` does what is expected.
## [1.0.1] - 2021-12-03
### Fixed
- $(make install) now works for the *.template files ([#40](https://github.com/erikw/restic-automatic-backup-scheduler/issues/40))
## [1.0.0] - 2021-12-02
It's time to call this a proper major version!
@@ -19,4 +150,4 @@ It's time to call this a proper major version!
- Conflicts for restic-check service
## [0.1.0] - 2019-07-23
- First tagged version.
- First tagged version to allow Arch's AUR to download a tarball archive to install.

16
LICENSE
View File

@@ -1,15 +1,11 @@
restic-systemd-automatic-backup - My restic backup solution using Backblaze B2 storage, systemd timers (or cron) and email notifications on failure.
Copyright (c) 2018, see commit log for auhtors
All rights reserved.
Copyright 2018 Erik Westrup
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
* Neither the name of the <ORGANIZATION> nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
http://opensource.org/licenses/BSD-3-Clause

View File

@@ -0,0 +1,39 @@
<?xml version="1.0" encoding="UTF-8"?>
<!-- See launchd.plist(5)for documentation on this file. -->
<!-- See https://www.launchd.info/ for a tutorial. -->
<!-- Debug with: $ tail -f /var/log/com.apple.xpc.launchd/launchd.log | grep erikw.restic -->
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.github.erikw.restic-backup</string>
<key>ProgramArguments</key>
<!-- exec in subshell to 1) source *.env.sh 2) expand $HOME to logpath (ref. https://apple.stackexchange.com/a/365880/197493) -->
<array>
<string>/bin/bash</string>
<string>-c</string>
<string>source $INSTALL_PREFIX/etc/restic/default.env.sh &amp;&amp; $INSTALL_PREFIX/bin/restic_backup.sh >>$HOME/$LOG_OUT 2>>$HOME/$LOG_ERR</string>
</array>
<key>EnvironmentVariables</key>
<dict>
<key>PATH</key>
<string>$INSTALL_PREFIX/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
<key>LOG_OUT</key>
<string>/Library/Logs/restic/backup_stdout.log</string>
<key>LOG_ERR</key>
<string>/Library/Logs/restic/backup_stderr.log</string>
</dict>
<key>RunAtLoad</key>
<true/>
<!-- Will schedule backup every day at 19:00 -->
<key>StartCalendarInterval</key>
<array>
<dict>
<key>Hour</key>
<integer>19</integer>
<key>Minute</key>
<integer>0</integer>
</dict>
</array>
</dict>
</plist>

View File

@@ -0,0 +1,37 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.github.erikw.restic-check</string>
<key>ProgramArguments</key>
<array>
<string>/bin/bash</string>
<string>-c</string>
<string>source $INSTALL_PREFIX/etc/restic/default.env.sh &amp;&amp; $INSTALL_PREFIX/bin/restic_check.sh >>$HOME/$LOG_OUT 2>>$HOME/$LOG_ERR</string>
</array>
<key>EnvironmentVariables</key>
<dict>
<key>PATH</key>
<string>$INSTALL_PREFIX/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
<key>LOG_OUT</key>
<string>/Library/Logs/restic/check_stdout.log</string>
<key>LOG_ERR</key>
<string>/Library/Logs/restic/check_stderr.log</string>
</dict>
<key>RunAtLoad</key>
<true/>
<!-- Will schedule check every 1st of the month at 20:00 -->
<key>StartCalendarInterval</key>
<array>
<dict>
<key>Day</key>
<integer>1</integer>
<key>Hour</key>
<integer>20</integer>
<key>Minute</key>
<integer>0</integer>
</dict>
</array>
</dict>
</plist>

281
Makefile
View File

@@ -1,57 +1,248 @@
# Not file targets.
.PHONY: help install install-scripts install-conf install-systemd uninstall
#### Notes ####################################################################
# This build process is done in three stages (out-of-source build):
# 1. copy source files to the local build directory.
# 2. build dir: replace the string "$INSTALL_PREFIX" with the value of $PREFIX
# 3. install files from the build directory to the target directory.
#
# Why this dance?
# * To fully support that a user can install this project to a custom path e.g.
# $(PREFIX=/usr/local make install), we need to modify the files that refer
# to other files on disk. We do this by having a placeholder
# "$INSTALL_PREFIX" that is substituted with the value of $PREFIX when
# installed.
# * We don't want to modify the files that are controlled by git, thus let's
# copy them to a build directory and then modify.
### Macros ###
SRCS_SCRIPTS = $(filter-out %cron_mail, $(wildcard usr/local/sbin/*))
SRCS_CONF = $(filter-out %template, $(wildcard etc/restic/*))
SRCS_SYSTEMD = $(wildcard etc/systemd/system/*)
#### Non-file targets #########################################################
.PHONY: help clean uninstall \
install-systemd install-cron \
install-targets-script install-targets-conf install-targets-systemd \
install-targets-cron install-targets-launchagent \
install-targets-schedtask uninstall-targets-schedtask \
activate-launchagent-backup deactivate-launchagent-backup \
activate-launchagent-chec deactivate-launchagent-check
# Just set PREFIX in envionment, like
# $ PREFIX=/tmp/test make
DEST_SCRIPTS = $(PREFIX)/usr/local/sbin
DEST_CONF = $(PREFIX)/etc/restic
DEST_SYSTEMD = $(PREFIX)/etc/systemd/system
#### Macros ###################################################################
NOW := $(shell date +%Y-%m-%d_%H:%M:%S)
INSTALLED_FILES = $(addprefix $(PREFIX)/, $(SRCS_SCRIPTS) $(SRCS_CONF) $(SRCS_SYSTEMD)) \
$(DEST_CONF)/b2_env.sh $(DEST_CONF)/b2_pw.txt
# GNU and macOS install have incompatible command line arguments.
GNU_INSTALL := $(shell install --version 2>/dev/null | \
grep -q GNU && echo true || echo false)
ifeq ($(GNU_INSTALL),true)
BAK_SUFFIX = --suffix=.$(NOW).bak
else
BAK_SUFFIX = -B .$(NOW).bak
endif
### Targets ###
# target: all - Default target.
all: install
# target: help - Display all targets.
# Source: https://stackoverflow.com/a/14777895/265508
ifeq ($(OS),Windows_NT)
CUR_OS := Windows
else
CUR_OS := $(shell uname)
endif
# Create parent directories of a file, if not existing.
# Reference: https://stackoverflow.com/a/25574592/265508
MKDIR_PARENTS=sh -c '\
dir=$$(dirname $$1); \
test -d $$dir || mkdir -p $$dir \
' MKDIR_PARENTS
# LaunchAgent names.
UID := $(shell id -u)
LAUNCHAGENT_BACKUP = com.github.erikw.restic-backup
LAUNCHAGENT_CHECK = com.github.erikw.restic-check
LAUNCHAGENT_TARGET_BACKUP = gui/$(UID)/$(LAUNCHAGENT_BACKUP)
LAUNCHAGENT_TARGET_CHECK = gui/$(UID)/$(LAUNCHAGENT_CHECK)
# What to substitute $INSTALL_PREFIX in sources to.
# This can be useful to set to empty on commandline when building e.g. an AUR
# package in a separate build directory (PREFIX).
INSTALL_PREFIX := $(PREFIX)
# Where to install persistent configuration files. Used by Homebrew.
SYSCONFDIR := $(PREFIX)
# Where to install LaunchAgent. Used by Homebrew.
LAUNCHAGENTDIR := $(HOME)
# ScheduledTask powershell scripts.
SCHEDTASK_INSTALL = install.ps1
SCHEDTASK_UNINSTALL = uninstall.ps1
# Source directories.
DIR_SCRIPT = bin
DIR_CONF = etc/restic
DIR_SYSTEMD = usr/lib/systemd/system
DIR_CRON = etc/cron.d
DIR_LAUNCHAGENT = Library/LaunchAgents
DIR_SCHEDTASK = ScheduledTask
# Source files.
SRCS_SCRIPT = $(filter-out \
%cron_mail \
%systemd-email \
%nm-unmetered-connection.sh \
, $(wildcard $(DIR_SCRIPT)/*))
SRCS_CONF = $(wildcard $(DIR_CONF)/*)
SRCS_SYSTEMD = $(filter-out \
%status-email-user@.service \
%nm-unmetered-connection.service \
, $(wildcard $(DIR_SYSTEMD)/*))
SRCS_CRON = $(wildcard $(DIR_CRON)/*)
SRCS_LAUNCHAGENT= $(wildcard $(DIR_LAUNCHAGENT)/*)
SRCS_SCHEDTASK = $(wildcard $(DIR_SCHEDTASK)/*)
# Local build directory. Sources will be copied here,
# modified and then installed from this directory.
BUILD_DIR = build
BUILD_DIR_SCRIPT = $(BUILD_DIR)/$(DIR_SCRIPT)
BUILD_DIR_CONF = $(BUILD_DIR)/$(DIR_CONF)
BUILD_DIR_SYSTEMD = $(BUILD_DIR)/$(DIR_SYSTEMD)
BUILD_DIR_CRON = $(BUILD_DIR)/$(DIR_CRON)
BUILD_DIR_LAUNCHAGENT = $(BUILD_DIR)/$(DIR_LAUNCHAGENT)
BUILD_DIR_SCHEDTASK = $(BUILD_DIR)/$(DIR_SCHEDTASK)
# Sources copied to build directory.
BUILD_SRCS_SCRIPT = $(addprefix $(BUILD_DIR)/, $(SRCS_SCRIPT))
BUILD_SRCS_CONF = $(addprefix $(BUILD_DIR)/, $(SRCS_CONF))
BUILD_SRCS_SYSTEMD = $(addprefix $(BUILD_DIR)/, $(SRCS_SYSTEMD))
BUILD_SRCS_CRON = $(addprefix $(BUILD_DIR)/, $(SRCS_CRON))
BUILD_SRCS_LAUNCHAGENT = $(addprefix $(BUILD_DIR)/, $(SRCS_LAUNCHAGENT))
BUILD_SRCS_SCHEDTASK = $(addprefix $(BUILD_DIR)/, $(SRCS_SCHEDTASK))
# Destination directories
DEST_DIR_SCRIPT = $(PREFIX)/$(DIR_SCRIPT)
DEST_DIR_CONF = $(SYSCONFDIR)/$(DIR_CONF)
DEST_DIR_SYSTEMD = $(PREFIX)/$(DIR_SYSTEMD)
DEST_DIR_CRON = $(PREFIX)/$(DIR_CRON)
DEST_DIR_LAUNCHAGENT= $(LAUNCHAGENTDIR)/$(DIR_LAUNCHAGENT)
DEST_DIR_MAC_LOG = $(HOME)/Library/Logs/restic
# Destination file targets.
DEST_TARGS_SCRIPT = $(addprefix $(PREFIX)/, $(SRCS_SCRIPT))
DEST_TARGS_CONF = $(addprefix $(SYSCONFDIR)/, $(SRCS_CONF))
DEST_TARGS_SYSTEMD = $(addprefix $(PREFIX)/, $(SRCS_SYSTEMD))
DEST_TARGS_CRON = $(addprefix $(PREFIX)/, $(SRCS_CRON))
DEST_TARGS_LAUNCHAGENT = $(addprefix $(LAUNCHAGENTDIR)/, $(SRCS_LAUNCHAGENT))
DEST_LAUNCHAGENT_BACKUP = $(DEST_DIR_LAUNCHAGENT)/$(LAUNCHAGENT_BACKUP).plist
DEST_LAUNCHAGENT_CHECK = $(DEST_DIR_LAUNCHAGENT)/$(LAUNCHAGENT_CHECK).plist
INSTALLED_FILES = $(DEST_TARGS_SCRIPT) $(DEST_TARGS_CONF) \
$(DEST_TARGS_SYSTEMD) $(DEST_TARGS_CRON) \
$(DEST_TARGS_LAUNCHAGENT)
#### Targets ##################################################################
# target: help - Default target; displays all targets.
help:
@egrep "#\starget:" [Mm]akefile | sed 's/\s-\s/\t\t\t/' | cut -d " " -f3- | sort -d
@egrep "#\starget:" [Mm]akefile | cut -d " " -f3- | sort -d
# target: install - Install all files
install: install-scripts install-conf install-systemd
# target: clean - Remove build files.
clean:
$(RM) -r $(BUILD_DIR)
# target: install-scripts - Install executables.
install-scripts:
install -d $(DEST_SCRIPTS)
install -m 0744 $(SRCS_SCRIPTS) $(DEST_SCRIPTS)
etc/restic/b2_env.sh:
install -m 0600 etc/restic/b2_env.sh.template /etc/restic/b2_env.sh
etc/restic/b2_pw.txt:
install -m 0600 etc/restic/b2_pw.txt.template /etc/restic/b2_pw.txt
# target: install-conf - Install restic configuration files.
# will create these files locally only if they don't already exist
install-conf: | etc/restic/b2_env.sh etc/restic/b2_pw.txt
install -d $(DEST_CONF)
install -m 0600 $(SRCS_CONF) $(DEST_CONF)
# target: install-systemd - Install systemd timer and service files
install-systemd:
install -d $(DEST_SYSTEMD)
install -m 0644 $(SRCS_SYSTEMD) $(DEST_SYSTEMD)
# target: uninstall - Uninstall files from the install targets
uninstall:
# target: uninstall - Uninstall ALL installed (including config) files.
uninstall: uninstall-schedtask
@for file in $(INSTALLED_FILES); do \
echo $(RM) $$file; \
$(RM) $$file; \
done
# To change the installation root path,
# set the PREFIX variable in your shell's environment, like:
# $ PREFIX=/usr/local make install-systemd
# $ PREFIX=/tmp/test make install-systemd
# target: install-systemd - Install systemd setup.
install-systemd: install-targets-script install-targets-conf \
install-targets-systemd
# target: install-cron - Install cron setup.
install-cron: install-targets-script install-targets-conf install-targets-cron
# target: install-launchagent - Install backup LaunchAgent setup.
install-launchagent: install-targets-script install-targets-conf \
install-targets-launchagent
# target: install-launchagent-check - Install check LaunchAgent setup.
# Intended to be run after install-launchagent, thus not requiring scripts/conf
#install-launchagent: install-targets-launchagent
# target: install-schedtask - Install Windows ScheduledTasks
install-schedtask: install-targets-script install-targets-conf \
install-targets-schedtask
# target: uninstall-schedtask - Uninstall Windows ScheduledTasks
uninstall-schedtask: uninstall-targets-schedtask
# Install targets. Prereq build sources as well,
# so that build dir is re-created if deleted.
install-targets-script: $(DEST_TARGS_SCRIPT) $(BUILD_SRCS_SCRIPT)
install-targets-conf: $(DEST_TARGS_CONF) $(BUILD_SRCS_CONF)
install-targets-systemd: $(DEST_TARGS_SYSTEMD) $(BUILD_SRCS_SYSTEMD)
install-targets-cron: $(DEST_TARGS_CRON) $(BUILD_SRCS_CRON)
install-targets-launchagent: $(DEST_TARGS_LAUNCHAGENT) \
$(BUILD_SRCS_LAUNCHAGENT) $(DEST_DIR_MAC_LOG)
install-targets-schedtask: $(BUILD_DIR_SCHEDTASK)/$(SCHEDTASK_INSTALL)
test $(CUR_OS) != Windows || ./$<
uninstall-targets-schedtask: $(BUILD_DIR_SCHEDTASK)/$(SCHEDTASK_UNINSTALL)
test $(CUR_OS) != Windows || ./$<
# Copies sources to build directory & replace "$INSTALL_PREFIX".
$(BUILD_DIR)/% : %
@${MKDIR_PARENTS} $@
cp $< $@
sed -i.bak -e 's|$$INSTALL_PREFIX|$(INSTALL_PREFIX)|g' $@; rm $@.bak
# Install destination script files.
$(DEST_DIR_SCRIPT)/%: $(BUILD_DIR_SCRIPT)/%
@${MKDIR_PARENTS} $@
install -m 0555 $< $@
# Install destination conf files. Additionally backup existing files.
$(DEST_DIR_CONF)/%: $(BUILD_DIR_CONF)/%
@${MKDIR_PARENTS} $@
install -m 0600 -b $(BAK_SUFFIX) $< $@
# Install destination systemd files.
$(DEST_DIR_SYSTEMD)/%: $(BUILD_DIR_SYSTEMD)/%
@${MKDIR_PARENTS} $@
install -m 0644 $< $@
# Install destination cron files.
$(DEST_DIR_CRON)/%: $(BUILD_DIR_CRON)/%
@${MKDIR_PARENTS} $@
install -m 0644 $< $@
# Install destination launchagent files.
$(DEST_DIR_LAUNCHAGENT)/%: $(BUILD_DIR_LAUNCHAGENT)/%
@${MKDIR_PARENTS} $@
install -m 0444 $< $@
# Install destination mac log dir.
$(DEST_DIR_MAC_LOG):
mkdir -p $@
# target: activate-launchagent-backup - Activate the backup LaunchAgent.
activate-launchagent-backup:
launchctl bootstrap gui/$(UID) $(DEST_LAUNCHAGENT_BACKUP)
launchctl enable $(LAUNCHAGENT_TARGET_BACKUP)
launchctl kickstart -p $(LAUNCHAGENT_TARGET_BACKUP)
# target: activate-launchagent-check - Activate the check LaunchAgent.
activate-launchagent-check:
launchctl bootstrap gui/$(UID) $(DEST_LAUNCHAGENT_CHECK)
launchctl enable $(LAUNCHAGENT_TARGET_CHECK)
launchctl kickstart -p $(LAUNCHAGENT_TARGET_CHECK)
# target: deactivate-launchagent-backup - Remove the backup LaunchAgent.
deactivate-launchagent-backup:
launchctl bootout $(LAUNCHAGENT_TARGET_BACKUP)
# target: deactivate-launchagent-check - Remove the check LaunchAgent.
deactivate-launchagent-check:
launchctl bootout $(LAUNCHAGENT_TARGET_CHECK)

587
README.md
View File

@@ -1,154 +1,565 @@
# Automatic restic backups using systemd services and timers
[![GitHub Stars](https://img.shields.io/github/stars/erikw/restic-systemd-automatic-backup?style=social)](#)
[![GitHub Forks](https://img.shields.io/github/forks/erikw/restic-systemd-automatic-backup?style=social)](#)
# Automatic restic Backups using Native OS Task Schedulers
*formerly named restic-systemd-automatic-backup*
[![GitHub Stars](https://img.shields.io/github/stars/erikw/restic-automatic-backup-scheduler?style=social)](#)
[![GitHub Forks](https://img.shields.io/github/forks/erikw/restic-automatic-backup-scheduler?style=social)](#)
<br>
[![SLOC](https://img.shields.io/tokei/lines/github/erikw/restic-systemd-automatic-backup)](#)
[![Open issues](https://img.shields.io/github/issues/erikw/restic-systemd-automatic-backup)](https://github.com/erikw/restic-systemd-automatic-backup/issues)
[![Closed issues](https://img.shields.io/github/issues-closed/erikw/restic-systemd-automatic-backup?color=success)](https://github.com/erikw/restic-systemd-automatic-backup/issues?q=is%3Aissue+is%3Aclosed)
[![Closed PRs](https://img.shields.io/github/issues-pr-closed/erikw/restic-systemd-automatic-backup?color=success)](https://github.com/erikw/restic-systemd-automatic-backup/pulls?q=is%3Apr+is%3Aclosed)
[![License](https://img.shields.io/badge/license-BSD--3-blue)](LICENSE.txt)
[![OSS Lifecycle](https://img.shields.io/osslifecycle/erikw/restic-systemd-automatic-backup)](https://github.com/Netflix/osstracker)
[![Latest tag](https://img.shields.io/github/v/tag/erikw/restic-systemd-automatic-backup)](https://github.com/erikw/restic-systemd-automatic-backup/tags)
[![Lint Code Base](https://github.com/erikw/restic-automatic-backup-scheduler/actions/workflows/linter.yml/badge.svg)](https://github.com/erikw/restic-automatic-backup-scheduler/actions/workflows/linter.yml)
[![Latest tag](https://img.shields.io/github/v/tag/erikw/restic-automatic-backup-scheduler)](https://github.com/erikw/restic-automatic-backup-scheduler/tags)
[![AUR version](https://img.shields.io/aur/version/restic-automatic-backup-scheduler)](https://aur.archlinux.org/packages/restic-automatic-backup-scheduler/)
[![AUR maintainer](https://img.shields.io/aur/maintainer/restic-automatic-backup-scheduler?label=AUR%20maintainer)](https://aur.archlinux.org/packages/restic-automatic-backup-scheduler/)
[![Homebrew Formula](https://img.shields.io/badge/homebrew-erikw%2Ftap-orange)](https://github.com/erikw/homebrew-tap)
[![Open issues](https://img.shields.io/github/issues/erikw/restic-automatic-backup-scheduler)](https://github.com/erikw/restic-automatic-backup-scheduler/issues)
[![Closed issues](https://img.shields.io/github/issues-closed/erikw/restic-automatic-backup-scheduler?color=success)](https://github.com/erikw/restic-automatic-backup-scheduler/issues?q=is%3Aissue+is%3Aclosed)
[![Closed PRs](https://img.shields.io/github/issues-pr-closed/erikw/restic-automatic-backup-scheduler?color=success)](https://github.com/erikw/restic-automatic-backup-scheduler/pulls?q=is%3Apr+is%3Aclosed)
[![License](https://img.shields.io/badge/license-BSD--3-blue)](LICENSE)
[![OSS Lifecycle](https://img.shields.io/osslifecycle/erikw/restic-automatic-backup-scheduler)](https://github.com/Netflix/osstracker)
<br>
[![Contributors](https://img.shields.io/github/contributors/erikw/restic-systemd-automatic-backup)](https://github.com/erikw/restic-systemd-automatic-backup/graphs/contributors) including these top contributors:
<a href = "https://github.com/erikw/restic-systemd-automatic-backup/graphs/contributors">
<img src = "https://contrib.rocks/image?repo=erikw/restic-systemd-automatic-backup&max=24"/>
[![Contributors](https://img.shields.io/github/contributors/erikw/restic-automatic-backup-scheduler)](https://github.com/erikw/restic-automatic-backup-scheduler/graphs/contributors) including these top contributors:
<a href = "https://github.com/erikw/restic-automatic-backup-scheduler/graphs/contributors">
<img src = "https://contrib.rocks/image?repo=erikw/restic-automatic-backup-scheduler&max=24"/>
</a>
# Intro
[restic](https://restic.net/) is a command-line tool for making backups, the right way. Check the official website for a feature explanation. As a storage backend, I recommend [Backblaze B2](https://www.backblaze.com/b2/cloud-storage.html) as restic works well with it, and it is (at the time of writing) very affordable for the hobbyist hacker! (anecdotal: I pay for my full-systems backups each month typically < 1 USD).
[restic](https://restic.net/) is a command-line tool for making backups, the right way. Check the official website for a feature explanation. As a storage backend, I recommend [Backblaze B2](https://www.backblaze.com/b2/cloud-storage.html) as restic works well with it, and it is (at the time of writing) very affordable for the hobbyist hacker!
Unfortunately restic does not come pre-configured with a way to run automated backups, say every day. However it's possible to set this up yourself using systemd/cron and some wrappers. This example also features email notifications when a backup fails to complete.
Unfortunately restic does not come pre-configured with a way to run automated backups, say every day. However it's possible to set this up yourself using built-in tools in your OS and some wrappers. For Linux with systemd, it's convenient to use systemd timers. For macOS systems, we can use built-in LaunchAgents. For Windows we can use ScheduledTasks. Any OS having something cron-like will also work!
Here follows a step-by step tutorial on how to set it up, with my sample script and configurations that you can modify to suit your needs.
Note, you can use any restic's supported [storage backends](https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html). The setup should be similar but you will have to use other configuration variables to match your backend of choice.
## Project Scope
The scope for this is not to be a full-fledged super solution that solves all the problems and all possible setups. The aim is to be a hackable code base for you to start sewing up the perfect backup solution that fits your requirements!
Nevertheless the project should work out of the box, be minimal but still open the doors for configuration and extensions by users.
## Notes
* Tip: Navigate this document easily from the Section icon in the top left corner.
![README Sections](img/readme_sections.png)
***Note**: in the command listing in this document, `$` means a user shell and `#` means a root shell (or use `sudo`).
Note, you can use any of the supported [storage backends](https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html). The setup should be similar but you will have to use other configuration variables to match your backend of choice.
# Requirements
* `restic >=v0.9.6`
* `bash >=v4.0.0`
* (recommended) GNU `make` if you want an automated install
* Arch: part of the `base-devel` meta package, Debian/Ubuntu: part of the `build-essential` meta package, macOS: use the pre-installed or a more recent with Homebrew
# Set up
Tip: The steps in this section will instruct you to copy files from this repo to system directories. If you don't want to do this manually, you can use the Makefile:
# Setup
Depending on your system, the setup will look different. Choose one of:
* <img height="16" width="16" src="https://unpkg.com/simple-icons@v6/icons/linux.svg" /> [Linux + Systemd](#setup-linux-systemd)
* <img height="16" width="16" src="https://unpkg.com/simple-icons@v6/icons/apple.svg" /> [macOS + LaunchAgent](#setup-macos-launchagent)
* <img height="16" width="16" src="https://unpkg.com/simple-icons@v6/icons/windows.svg" /> [Windows + ScheduledTask](#setup-windows-scheduledtask)
* <img height="16" width="16" src="https://unpkg.com/simple-icons@v6/icons/clockify.svg" /> [Cron](#setup-cron) - for any system having a cron daemon. Tested on FreeBSD and macOS.
## Setup Linux Systemd
<img height="64" width="64" src="https://unpkg.com/simple-icons@v6/icons/linux.svg" />
**Note** The Linux setup here will assume an installation to `/`.
Many Linux distributions nowadays use [Systemd](https://en.wikipedia.org/wiki/Systemd), which features good support for running services and scheduled jobs. If your distribution is no on Systemd, check out the [cron setup](#setup-cron) instead.
**TL;DR setup**
1. [Create](#1-create-backblaze-b2-account-bucket-and-keys) B2 bucket + credentials
1. Install scripts, configs systemd units/timers:
* With `make`:
```console
$ sudo make install-systemd
```
* <img height="16" width="16" src="https://unpkg.com/simple-icons@v6/icons/archlinux.svg" /> Arch Linux users: use the [AUR](https://aur.archlinux.org/packages/restic-automatic-backup-scheduler) package, e.g.
```console
$ yay -S restic-automatic-backup-scheduler
```
1. Fill out [configuration values](#2-configure-b2-credentials-locally) in `/etc/restic`.
1. [Initialize](#3-initialize-remote-repo) the remote repo.
Source the profile to make all needed configuration available to `restic(1)`. All commands after this assumes the profile is sourced in the current shell.
```console
# source /etc/restic/default.env.sh
# restic init
```
1. Configure [how often](https://www.freedesktop.org/software/systemd/man/systemd.time.html#Calendar%20Events) backups should be done.
* If needed, edit `OnCalendar` in `/usr/lib/systemd/system/restic-backup@.timer`.
1. Enable automated backup for starting with the system & make the first backup:
```console
# systemctl enable --now restic-backup@default.timer
```
1. Watch the first backup progress with Systemd journal:
```console
# journalctl -f --lines=50 -u restic-backup@default
```
1. Verify the backup
```console
# restic snapshots
```
1. (recommended) Enable the check job that verifies that the backups for the profile are all intact.
```console
# systemctl enable --now restic-check@default.timer
````
1. (optional) Define multiple profiles: just make a copy of the `default.env.sh` and use the defined profile name in place of `default` to run backups or enable timers. Notice that the value after `@` works as a parameter.
```console
# systemctl enable restic-backup@other_profile.timer
```
1. Consider more [optional features](#optional-features).
## Setup macOS LaunchAgent
<img height="64" width="64" src="https://unpkg.com/simple-icons@v6/icons/apple.svg" />
☝ **Note** The macOS setup here will assume an installation to `/usr/local`, as [custom](https://docs.brew.sh/FAQ#why-does-homebrew-say-sudo-is-bad) with Homebrew installations.
[Launchd](https://www.launchd.info/) is the modern built-in service scheduler in macOS. It has support for running services as root (Daemon) or as a normal user (Agent). Here we set up an LauchAgent to be run as your normal user for starting regular backups.
**TL;DR setup**
1. [Create](#1-create-backblaze-b2-account-bucket-and-keys) B2 bucket + credentials
1. Install scripts, configs and LaunchAgent:
* (recommended) with Homebrew from the [erikw/homebrew-tap](https://github.com/erikw/homebrew-tap):
```console
$ brew install erikw/tap/restic-automatic-backup-scheduler
```
* Using `make`:
```console
$ make PREFIX=/usr/local install-launchagent
```
1. Fill out [configuration values](#2-configure-b2-credentials-locally) in `/usr/local/etc/restic`.
1. [Initialize](#3-initialize-remote-repo) the remote repo.
Source the profile to make all needed configuration available to `restic(1)`. All commands after this assumes the profile is sourced in the current shell.
```console
$ source /usr/local/etc/restic/default.env.sh
$ restic init
```
1. Configure [how often](https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html#//apple_ref/doc/uid/10000172i-CH1-SW1) backups should be done. If needed, edit `OnCalendar` in
* Homebrew install: `~/Library/LaunchAgents/homebrew.mxcl.restic-automatic-backup-scheduler.plist`.
* `make` install: `~/Library/LaunchAgents/com.github.erikw.restic-backup.plist`.
1. Enable automated backup for starting with the system & make the first backup:
* Homebrew install:
```console
$ brew services start restic-automatic-backup-scheduler
```
* `make` install:
```console
$ launchctl bootstrap gui/$UID ~/Library/LaunchAgents/com.github.erikw.restic-backup.plist
$ launchctl enable gui/$UID/com.github.erikw.restic-backup
$ launchctl kickstart -p gui/$UID/com.github.erikw.restic-backup
```
As a convenience, a shortcut for the above commands are `$ make activate-launchagent-backup`.
1. Watch the first backup progress from the log files:
```console
$ tail -f ~/Library/Logs/restic/backup*
```
1. Verify the backup
```console
$ restic snapshots
```
1. (recommended) Enable the check job that verifies that the backups for the profile are all intact.
* Homebrew install:
```console
$ brew services start restic-automatic-backup-scheduler-check
```
* `make` install:
```console
$ launchctl bootstrap gui/$UID ~/Library/LaunchAgents/com.github.erikw.restic-check.plist
$ launchctl enable gui/$UID/com.github.erikw.restic-check
$ launchctl kickstart -p gui/$UID/com.github.erikw.restic-check
```
As a convenience, a shortcut for the above commands are `$ make activate-launchagent-check`.
1. Consider more [optional features](#optional-features).
### Homebrew Setup Notes
Then control the service with homebrew:
```console
$ git clone https://github.com/erikw/restic-systemd-automatic-backup.git
$ cd restic-systemd-automatic-backup
$ sudo make install
````
## 1. Create Backblaze B2 account
First, see this official Backblaze [tutorial](https://help.backblaze.com/hc/en-us/articles/115002880514-How-to-configure-Backblaze-B2-with-Restic-on-Linux) on restic, and follow the instructions ("Create Backblaze account with B2 enabled") there on how to create a new B2 bucket.
Take note of the your account ID, application key and password for the next steps.
## 2. Configure your B2 account locally
Put these files in `/etc/restic/`:
* `b2_env.sh`: Fill this file out with your B2 bucket settings etc. The reason for putting these in a separate file is that it can be used also for you to simply source, when you want to issue some restic commands. For example:
```console
$ source /etc/restic/b2_env.sh
$ restic snapshots # You don't have to supply all parameters like --repo, as they are now in your environment!
````
* `b2_pw.txt`: This file should contain the restic repository password. This is a new password what soon will be used when initializing the new repository. It should be unique to this restic backup repository and is needed for restoring from it. Don't re-use your b2 login password, this should be different.
## 3. Initialize remote repo
Now we must initialize the repository on the remote end:
```console
$ source /etc/restic/b2_env.sh
$ restic init
$ brew services start restic-automatic-backup-scheduler
$ brew services restart restic-automatic-backup-scheduler
$ brew services stop restic-automatic-backup-scheduler
```
## 4. Script for doing the backup
Put this file in `/usr/local/sbin`:
* `restic_backup.sh`: A script that defines how to run the backup. Edit this file to respect your needs in terms of backup which paths to backup, retention (number of backups to save), etc.
Copy this file to `/etc/restic/backup_exclude` or `~/.backup_exclude`:
* `.backup_exclude`: A list of file pattern paths to exclude from you backups, files that just occupy storage space, backup-time, network and money.
## 5. Make first backup & verify
Now see if the backup itself works, by running
If `services start` fails, it might be due to previous version installed. In that case remove the existing version and try again:
```console
$ /usr/local/sbin/restic_backup.sh
$ restic snapshots
$ launchctl bootout gui/$UID/com.github.erikw.restic-backup
$ brew services start restic-automatic-backup-scheduler
```
### Make Setup Notes
Use the `disable` command to temporarily pause the agent, or `bootout` to uninstall it.
```
$ launchctl disable gui/$UID/com.github.erikw.restic-backup
$ launchctl bootout gui/$UID/com.github.erikw.restic-backup
```
If you updated the `.plist` file, you need to issue the `bootout` followed by `bootrstrap` and `enable` sub-commands of `launchctl`. This will guarantee that the file is properly reloaded.
## Setup Windows ScheduledTask
<img height="64" width="64" src="https://unpkg.com/simple-icons@v6/icons/windows.svg" />
Windows comes with a built-in task scheduler called [ScheduledTask](https://docs.microsoft.com/en-us/powershell/module/scheduledtasks/new-scheduledtask?view=windowsserver2022-ps). The frontend app is "Task Scheduler" (`taskschd.msc`) and we can use PowerShell commands to install a new scheduled task.
I describe here one of may ways you can get restic and this backup script working on Windows. Here I chose to work with `scoop` and `git-bash`.
**TL;DR setup**
1. Install [scoop](https://scoop.sh/)
1. Install dependencies from a PowerShell with *administrator privileges*:
```console
powershell> scoop install restic make git
```
1. In a *non-privileged* PowerShell, start git-bash and clone this repo
```console
powershell> git-bash
git-bash$ mkdir ~/src && cd ~/src/
git-bash$ git clone https://github.com/erikw/restic-automatic-backup-scheduler.git && cd $(basename "$_" .git)
```
1. Install scripts, configs and ScheduledTasks
```console
git-bash$ make install-schedtask
```
1. Fill out [configuration values](#2-configure-b2-credentials-locally) in `/etc/restic`.
```console
git-bash$ vim /etc/restic/*
```
Note that you should use cygwin/git-bash paths. E.g. in `default.env.sh` you could have
```bash
export RESTIC_BACKUP_PATHS='/c/Users/<username>/My Documents'
```
1. [Initialize](#3-initialize-remote-repo) the remote repo.
Source the profile to make all needed configuration available to `restic(1)`. All commands after this assumes the profile is sourced in the current shell.
```console
git-bash$ source /etc/restic/default.env.sh
git-bash$ restic init
```
1. Make the first backup
```console
git-bash$ restic_backup.sh
```
1. Verify the backup
```console
git-bash$ restic snapshots
```
1. Inspect the installed ScheduledTasks and make a test run
1. Open the app "Task Scheduler" (`taskschd.msc`)
1. Go to the local "Task Scheduler Library"
1. Right click on one of the newly installed tasks e.g. `restic_backup` and click "run".
- If the tasks are not there, maybe you opended it up before `make install-schedtask`: just close and start it again to refresh.
1. Now a git-bash window should open running `restic_backup.sh`, and the next time the configured schedule hits!
1. Consider more [optional features](#optional-features).
With `taskschd.msc` you can easily start, stop, delete and configure the scheduled tasks to your liking:
<a href="img/tasksched.png" title="Windows Task Scheduler"><img src="img/tasksched.png" width="512" alt="Windows Task Schedulder"></a>
## Setup Cron
<img height="64" width="64" src="https://unpkg.com/simple-icons@v6/icons/clockify.svg" />
☝ **Note** There are many different cron [implementations](https://wiki.archlinux.org/title/Cron) out there and they all work slightly different.
Any system that has a cron-like system can easily setup restic backups as well. However if you system supports any of the previous setups, those are recommended over cron as they provide more features and reliability for your backups.
**TL;DR setup**
1. [Create](#1-create-backblaze-b2-account-bucket-and-keys) B2 bucket + credentials
1. Install scripts, configs systemd units/timers:
```console
$ sudo make install-cron
```
* This assumes that your cron supports dropping files into `/etc/cron.d/`. If that is not the case, simply copy the relevant contents of the installed `/etc/cron.d/restic` in to your `/etc/crontab`.
```console
# grep "^@.*restic_" /etc/cron.d/restic >> /etc/crontab
```
1. Fill out [configuration values](#2-configure-b2-credentials-locally) in `/etc/restic`.
1. [Initialize](#3-initialize-remote-repo) the remote repo.
Source the profile to make all needed configuration available to `restic(1)`. All commands after this assumes the profile is sourced in the current shell.
```console
# source /etc/restic/default.env.sh
# restic init
```
1. Make the first backup
```console
# restic_backup.sh
```
1. Verify the backup
```console
# restic snapshots
```
1. Configure [how often](https://crontab.guru/) backups should be done by directly editing `/etc/cron.d/restic` (or `/etc/crontab`).
1. Consider more [optional features](#optional-features).
## Detailed Manual Setup
<img height="64" width="64" src="img/pen-paper.png" />
This is a more detailed explanation than the TL;DR sections above that will give you more understanding in the setup. This section is more general, but uses Linux + Systemd as the example setup.
#### 0. Clone Repo
```console
$ git clone https://github.com/erikw/restic-automatic-backup-scheduler.git && cd $(basename "$_" .git)
````
## 6. Backup automatically; systemd service + timer
Now we can do the modern version of a cron-job, a systemd service + timer, to run the backup every day!
Make a quick search-and-replace in the source files:
```console
$ find etc bin -type f -exec sed -i.bak -e 's|$INSTALL_PREFIX||g' {} \; -exec rm {}.bak \;
```
and you should now see that all files have been changed like e.g.
```diff
-export RESTIC_PASSWORD_FILE="$INSTALL_PREFIX/etc/restic/pw.txt"
+export RESTIC_PASSWORD_FILE="/etc/restic/pw.txt"
```
Why? The OS specific TL;DR setups above all use the [Makefile](Makefile) or a package manager to install these files. The placeholder string `$INSTALL_PREFIX` is in the source files for portability reasons, so that the Makefile can support all different operating systems. `make` users can set a different `$PREFIX` when installing like `PREFIX=/usr/local make install-systemd`.
In this detailed manual setup we will copy all files manually to `/etc`and `/bin`. Thus we need to remove the placeholder string `$INSTALL_PREFIX` in the source files as a first step.
Put these files in `/etc/systemd/system/`:
* `restic-backup.service`: A service that calls the backup script.
* `restic-backup.timer`: A timer that starts the backup every day.
#### 1. Create Backblaze B2 Account, Bucket and Keys
In short:
1. Create a [Backblaze](https://www.backblaze.com/) account (use 2FA!).
1. Create a new [B2 bucket](https://secure.backblaze.com/b2_buckets.htm).
* Private, without B2 encryption and without the object lock feature
1. Create a pair of [keyId and applicationKey](https://secure.backblaze.com/app_keys.htm?bznetid=17953438771644852981527)
* Limit scope of the new id and key pair to only the above created bucket.
First, see this official Backblaze [tutorial](https://help.backblaze.com/hc/en-us/articles/4403944998811-Quickstart-Guide-for-Restic-and-Backblaze-B2-Cloud-Storage) on restic, and follow the instructions ("Create Backblaze account with B2 enabled") there on how to create a new B2 bucket. In general, you'd want a private bucket, without B2 encryption (restic does the encryption client side for us) and without the object lock feature.
For restic to be able to connect to your bucket, you want to in the B2 settings create a pair of keyID and applicationKey. It's a good idea to create a separate pair of ID and Key with for each bucket that you will use, with limited read&write access to only that bucket.
#### 2. Configure B2 Credentials Locally
Put these files in `/etc/restic/`:
* `_global.env.sh`: Fill this file out with your global settings including B2 keyID & applicationKey.
* `default.env.sh`: This is the default profile. Fill this out with bucket name, backup paths and retention policy. This file sources `_global.env.sh` and is thus self-contained and can be sourced in the shell when you want to issue some manual restic commands. For example:
```console
$ source /etc/restic/default.env.sh
$ restic snapshots # You don't have to supply all parameters like --repo, as they are now in your environment!
````
* `pw.txt`: This file should contain the restic password (single line) used to encrypt the repository. This is a new password what soon will be used when initializing the new repository. It should be unique to this restic backup repository and is needed for restoring from it. Don't re-use your B2 login password, this should be different. For example you can generate a 128 character password (must all be on one line) with:
```console
$ openssl rand -base64 128 | tr -d '\n' > /etc/restic/pw.txt
```
* `backup_exclude.txt`: List of file patterns to ignore. This will trim down your backup size and the speed of the backup a lot when done properly!
#### 3. Initialize remote repo
Now we must initialize the repository on the remote end:
```console
$ sudo -i
# source /etc/restic/default.env.sh
# restic init
```
#### 4. Script for doing the backup
Put this file in `/bin`:
* `restic_backup.sh`: A script that defines how to run the backup. The intention is that you should not need to edit this script yourself, but be able to control everything from the `*.env.sh` profiles.
Restic support exclude files. They list file pattern paths to exclude from you backups, files that just occupy storage space, backup-time, network and money. `restic_backup.sh` allows for a few different exclude files.
* `/etc/restic/backup_exclude.txt` - global exclude list. You can use only this one if your setup is easy. This is set in `_global.env.sh`. If you need a different file for another profile, you can override the envvar `RESTIC_BACKUP_EXCLUDE_FILE` in this profile.
* `.backup_exclude.txt` per backup path. If you have e.g. an USB disk mounted at /mnt/media and this path is included in the `$RESTIC_BACKUP_PATHS`, you can place a file `/mnt/media/.backup_exclude.txt` and it will automatically picked up. The nice thing about this is that the backup paths are self-contained in terms of what they shoud exclude!
#### 5. Make first backup
Now see if the backup itself works, by running as root
```console
# source /etc/restic/default.env.sh
# /bin/restic_backup.sh
````
#### 6. Verify the backup
As the `default.env.sh` is already sourced in your root shell, you can now just list the snapshost
```console
# restic snapshots
```
Alternatively you can mount the restic snapshots to a directory set `/mnt/restic`
```console
# restic mount /mnt/restic
# ls /mnt/restic
```
#### 7. Backup automatically
All OS setups differs in what task scheduler they use. As a demonstration, let's look at how we can do this with systemd under Linux here.
Put these files in `/etc/systemd/system` (note that the Makefile installs as package to `/usr/lib/systemd/system`)
* `restic-backup@.service`: A service that calls the backup script with the specified profile. The profile is specified
by the value after `@` when running it (see below).
* `restic-backup@.timer`: A timer that starts the former backup every day (same thing about profile here).
* If needed, edit this file to configure [how often](https://www.freedesktop.org/software/systemd/man/systemd.time.html#Calendar%20Events) back up should be made. See the `OnCalendar` key in the file.
Now simply enable the timer with:
```console
$ systemctl start restic-backup.timer
$ systemctl enable restic-backup.timer
# systemctl enable --now restic-backup@default.timer
````
You can see when your next backup is scheduled to run with
```console
$ systemctl list-timers | grep restic
# systemctl list-timers | grep restic
```
and see the status of a currently running backup with
and see the status of a currently running backup with:
```console
$ systemctl status restic-backup
# systemctl status restic-backup
```
or start a backup manually
or start a backup manually:
```console
$ systemctl start restic-backup
$ systemctl start restic-backup@default
```
You can follow the backup stdout output live as backup is running with:
```console
$ journalctl -f -u restic-backup.service
$ journalctl -f -u restic-backup@default.service
````
(skip `-f` to see all backups that has run)
#### Recommended: Automated Backup Checks
Once in a while it can be good to do a health check of the remote repository, to make sure it's not getting corrupt. This can be done with `$ restic check`.
## 7. Email notification on failure
There is companion scripts, service and timer (`*check*`) to restic-backup.sh that checks the restic backup for errors; look in the repo in `usr/lib/systemd/system/` and `bin/` and copy what you need over to their corresponding locations.
```console
# systemctl enable --now restic-check@default.timer
````
## Optional Features
<img height="64" width="64" src="img/plus.png" />
### Optional: Multiple profiles
To have different backup jobs having e.g. different buckets, backup path of schedule, just make a copy of the `default.env.sh` and use the defined profile name in place of `default` in the previous steps.
To create a different backup and use you can do:
```console
# cp /etc/restic/default.env.sh /etc/restic/other.env.sh
# vim /etc/restic/default.other.sh # Set backup path, bucket etc.
# source /etc/restic/default.other.sh
# restic_backup.sh
```
### Optional: Desktop Notifications
<img src="img/macos_notification.png" align="right" />
It's a good idea to be on top of your backups to make sure that they don't increase a lot in size and incur high costs. However it's notoriously tricky to make GUI notifications correctly from a non-user process (e.g. root).
Therefore this project provides a lightweight solution for desktop notifications that works like this: Basically `restic_backup.sh` will append a summary line of the last backup to a user-owned file (the user running your OS's desktop environment) in a fire-and-forget fashion. Then the user has a process that reads this and forward each line as a new message to the desktop environment in use.
To set desktop notifications up:
1. Create a special FIFO file as your desktop user:
```console
$ mkfifo /home/user/.cache/notification-queue
```
1. In your profile, e.g. `/etc/restic/default.sh`, set:
```bash
RESTIC_NOTIFY_BACKUP_STATS=true
RESTIC_BACKUP_NOTIFICATION_FILE=/home/user/.cache/notification-queue
```
1. Create a listener on the notification queue file that forwards to desktop notifications
* Linux auto start + cross-platform notifier / notify-send
* [notification-queue-notifier](https://github.com/gerardbosch/dotfiles/blob/ddc1491056822eab45dedd131f1946308ef62135/home/bin/notification-queue-notifier)
* [notification-queue.desktop](https://github.com/gerardbosch/dotfiles-linux/blob/ea0f75bfd7a356945544ecaa42a2fc35c9fab3a1/home/.config/autostart/notification-queue.desktop)
* macOS auto start + [terminal-notifier](https://github.com/julienXX/terminal-notifier)
* [notification-queue-notifier.sh](https://github.com/erikw/dotfiles/blob/8a942defe268292200b614951cdf433ddccf7170/bin/notification-queue-notifier.sh)
* [com.user.notificationqueue.plist](https://github.com/erikw/dotfiles/blob/8a942defe268292200b614951cdf433ddccf7170/.config/LaunchAgents/com.user.notificationqueue.plist)
### Optional: Email Notification on Failure
#### Systemd
We want to be aware when the automatic backup fails, so we can fix it. Since my laptop does not run a mail server, I went for a solution to set up my laptop to be able to send emails with [postfix via my Gmail](https://easyengine.io/tutorials/linux/ubuntu-postfix-gmail-smtp/). Follow the instructions over there.
Put this file in `/usr/local/sbin`:
Put this file in `/bin`:
* `systemd-email`: Sends email using sendmail(1). This script also features time-out for not spamming Gmail servers and getting my account blocked.
Put this files in `/etc/systemd/system/`:
* `status-email-user@.service`: A service that can notify you via email when a systemd service fails. Edit the target email address in this file.
As you maybe noticed already before, `restic-backup.service` is configured to start `status-email-user.service` on failure.
Now edit `restic-backup@.service` and `status-email-user@.service` to call this service failure.
```
OnFailure=status-email-user@%n.service
```
#### Cron
Use `bin/cron_mail`: A wrapper for running cron jobs, that sends output of the job as an email using the mail(1) command. This assumes that the `mail` program is correctly setup on the system to send emails.
To use this, wrap the restic script command with it in your cron file like:
```diff
-@midnight root . /etc/restic/default.sh && restic_backup.sh
+@midnight root . /etc/restic/default.sh && cron_mail restic_backup.sh
```
## 8. Optional: automated backup checks
Once in a while it can be good to do a health check of the remote repository, to make sure it's not getting corrupt. This can be done with `$ restic check`.
### Optional: No Backup on Metered Connections (Linux/systemd only)
For a laptop, it can make sense to not do heavy backups when your on a metered connection like a shared connection from you mobile phone. To solve this we can set up a systemd service that is in success state only when a connection is unmetered. Then we can tell our backup service to depend on this service simply! When the unmetered service detects an unmetered connection it will go to failed state. Then our backup service will not run as it requires this other service to be in success state.
There are some `*-check*`-files in this git repo. Install these in the same way you installed the `*-backup*`-files.
1. Edit `restic-backup@.service` and `restic-check@.service` to require the new service to be in success state:
```
Requires=nm-unmetered-connection.service
```
1. Copy and paste the command below, it will install the following files and refresh systemd daemon:
1. Put this file in `/etc/systemd/system/`:
* `nm-unmetered-connection.service`: A service that is in success state only if the connection is unmetered.
1. Install this file in `/bin`:
* `nm-unmetered-connection.sh`: Detects metered connections and returns an error code if one is detected. This scripts requires the Gnome [NetworkManager](https://wiki.gnome.org/Projects/NetworkManager) to be installed (modify this script if your system has a different network manager).
1. Reload systemd with
```console
# systemctl daemon-reload
```
☝ **Tip**: All steps but the first can be done in one go if you use the Makefile. Set `$PREFIX` as needed or leave empty for install to `/`.
```bash
sudo bash -c 'export PREFIX=
make build/usr/lib/systemd/system/nm-unmetered-connection.service
install -m 0644 build/usr/lib/systemd/system/nm-unmetered-connection.service $PREFIX/etc/systemd/system
install -m 0555 bin/nm-unmetered-connection.sh /bin
systemctl daemon-reload
'
```
### Optional: Restic Wrapper Script
For convenience there's a `restic` wrapper script that makes loading profiles and **running restic**
straightforward (it needs to run with sudo to read environment). Just run:
* `sudo resticw WHATEVER` (e.g. `sudo resticw snapshots`) to use the default profile.
* You can run the wrapper by passing a specific profile: `resticw -p anotherprofile snapshots`.
Useful commands:
| Command | Description |
|---------------------------------------------------|-------------------------------------------------------------------|
| `resticw snapshots` | List backup snapshots |
| `resticw diff <snapshotId-1> <snapshotId-2>` | Show the changes between backup snapshots |
| `resticw stats` / `resticw stats snapshotId ...` | Show the statistics for the whole repo or the specified snapshots |
| `resticw mount /mnt/restic` | Mount your remote repository |
# Cron?
If you want to run an all-classic cron job instead, do like this:
* `etc/cron.d/restic`: Depending on your system's cron, put this in `/etc/cron.d/` or similar, or copy the contents to $(sudo crontab -e). The format of this file is tested under FreeBSD, and might need adaptions depending on your cron.
* `usr/local/sbin/cron_mail`: A wrapper for running cron jobs, that sends output of the job as an email using the mail(1) command.
# Uninstall
There is a make target to remove all files (scripts and configs) that were installed by `sudo make install`. Just run:
There is a make target to remove all files (scripts and **configs)** that were installed by `sudo make install-*`. Just run:
```console
$ sudo make uninstall
```
# Development
* To not mess up your real installation when changing the `Makefile` simply install to a `$PREFIX` like
```console
$ PREFIX=/tmp/restic-test make install-systemd
```
* **Updating the `resticw` parser:** If you ever update the usage `DOC`, you will need to refresh the auto-generated parser:
```console
$ pip install doctopt.sh
$ doctopt.sh usr/local/bin/resticw
```
# Releasing
To make a new release:
1. Create a new tag:
```console
$ vi CHANGELOG.md && git commit -am "Update CHANGELOG.md"
$ git tag vX.Y.Z
$ git push && git push --tags
```
1. Update version in the AUR [PKGBUILD](https://aur.archlinux.org/packages/restic-automatic-backup-scheduler/)
1. Update version in the Homebrew Formulas (see the repo README):
* [restic-automatic-backup-scheduler](https://github.com/erikw/homebrew-tap/blob/main/Formula/restic-automatic-backup-scheduler.rb)
* [restic-automatic-backup-scheduler-check](https://github.com/erikw/homebrew-tap/blob/main/Formula/restic-automatic-backup-scheduler-check.rb)

19
ScheduledTask/install.ps1 Normal file
View File

@@ -0,0 +1,19 @@
#!/usr/bin/env pwsh
# Install restic scheduled tasks.
# Test run the installed actions by
# 1. open the app "Task Scheduler" (taskschd.msc)
# 2. go to the local "Task Scheduler Library"
# 3. right click on the new tasks and click "run".
# Reference: https://blogs.technet.microsoft.com/heyscriptingguy/2015/01/13/use-powershell-to-create-scheduled-tasks/
# Reference: https://www.davidjnice.com/cygwin_scheduled_tasks.html
# Install restic_backup.sh
$action = New-ScheduledTaskAction -Execute "$(scoop prefix git)\git-bash.exe" -Argument '-l -c "source $INSTALL_PREFIX/etc/restic/default.env.sh && $INSTALL_PREFIX/bin/restic_backup.sh"'
$trigger = New-ScheduledTaskTrigger -Daily -At 7pm
Register-ScheduledTask -Action $action -Trigger $trigger -TaskName "restic_backup" -Description "Daily backup to B2 with restic."
# Install restic_check.sh
$action = New-ScheduledTaskAction -Execute "$(scoop prefix git)\git-bash.exe" -Argument '-l -c "source $INSTALL_PREFIX/etc/restic/default.env.sh && /INSTALL_PREFIX/bin/restic_check.sh"'
$trigger = New-ScheduledTaskTrigger -Weekly -WeeksInterval 4 -DaysOfWeek Sunday -At 8pm -RandomDelay 128
Register-ScheduledTask -Action $action -Trigger $trigger -TaskName "restic_check" -Description "Check B2 backups with restic."

View File

@@ -0,0 +1,5 @@
#!/usr/bin/env pwsh
# Uninstall restic scheduled tasks.
Unregister-ScheduledTask -TaskName "restic_backup" -Confirm:$false
Unregister-ScheduledTask -TaskName "restic_check" -Confirm:$false

4
usr/local/sbin/cron_mail → bin/cron_mail Executable file → Normal file
View File

@@ -5,7 +5,7 @@
# Why? Because of FreeBSD the system cron uses sendmail, and I want to use ssmtp.
# Make your crontab files like:
#SHELL=/bin/sh
#PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/sbin
#PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/sbin:$INSTALL_PREFIX/bin
#@daily root cron_mail freebsd-update cron
mail_target=root
@@ -15,7 +15,7 @@ if [ $# -eq 0 ]; then
echo "No program to run given!" >&2
exit 1
fi
cmd="$@"
cmd="$*"
body=$(eval "$cmd" 2>&1)

View File

@@ -1,19 +1,19 @@
#!/bin/bash
#!/usr/bin/env bash
# Requires Gnome NetworkManager
systemctl is-active dbus.service >/dev/null 2>&1 || exit 0
systemctl is-active NetworkManager.service >/dev/null 2>&1 || exit 0
metered_status=$(dbus-send --system --print-reply=literal \
--system --dest=org.freedesktop.NetworkManager \
/org/freedesktop/NetworkManager \
org.freedesktop.DBus.Properties.Get \
string:org.freedesktop.NetworkManager string:Metered \
| grep -o ".$")
--system --dest=org.freedesktop.NetworkManager \
/org/freedesktop/NetworkManager \
org.freedesktop.DBus.Properties.Get \
string:org.freedesktop.NetworkManager string:Metered \
| grep -o ".$")
if [[ $metered_status =~ (1|3) ]]; then
echo Current connection is metered
exit 1
else
else
exit 0
fi

128
bin/restic_backup.sh Normal file
View File

@@ -0,0 +1,128 @@
#!/usr/bin/env bash
# Make a backup with restic to Backblaze B2.
#
# This script is typically run (as root user) either like:
# - from restic service/timer: $PREFIX/etc/systemd/system/restic-backup.{service,timer}
# - from a cronjob: $PREFIX/etc/cron.d/restic
# - manually by a user. For it to work, the environment variables must be set in the shell where this script is executed
# $ source $PREFIX/etc/default.env.sh
# $ restic_backup.sh
# Exit on error, unset var, pipe failure
set -euo pipefail
# Clean up lock if we are killed.
# If killed by systemd, like $(systemctl stop restic), then it kills the whole cgroup and all it's subprocesses.
# However if we kill this script ourselves, we need this trap that kills all subprocesses manually.
exit_hook() {
echo "In exit_hook(), being killed" >&2
jobs -p | xargs kill
restic unlock
}
trap exit_hook INT TERM
# Assert that all needed environment variables are set.
# TODO in future if this grows, move this to a restic_lib.sh
assert_envvars() {
local varnames=("$@")
for varname in "${varnames[@]}"; do
if [ -z ${!varname+x} ]; then
printf "%s must be set for this script to work.\n\nDid you forget to source a $INSTALL_PREFIX/etc/restic/*.env.sh profile in the current shell before executing this script?\n" "$varname" >&2
exit 1
fi
done
}
assert_envvars \
B2_ACCOUNT_ID B2_ACCOUNT_KEY B2_CONNECTIONS \
RESTIC_BACKUP_PATHS RESTIC_BACKUP_TAG \
RESTIC_BACKUP_EXCLUDE_FILE RESTIC_BACKUP_EXTRA_ARGS RESTIC_PASSWORD_FILE RESTIC_REPOSITORY RESTIC_VERBOSITY_LEVEL \
RESTIC_RETENTION_DAYS RESTIC_RETENTION_MONTHS RESTIC_RETENTION_WEEKS RESTIC_RETENTION_YEARS
# Convert to arrays, as arrays should be used to build command lines. See https://github.com/koalaman/shellcheck/wiki/SC2086
IFS=':' read -ra backup_paths <<< "$RESTIC_BACKUP_PATHS"
IFS=' ' read -ra extra_args <<< "$RESTIC_BACKUP_EXTRA_ARGS"
# Set up exclude files: global + path-specific ones
# NOTE that restic will fail the backup if not all listed --exclude-files exist. Thus we should only list them if they are really all available.
## Global backup configuration.
exclusion_args=(--exclude-file "$RESTIC_BACKUP_EXCLUDE_FILE")
## Self-contained backup exclusion files per backup path. E.g. having an USB disk at /mnt/media in RESTIC_BACKUP_PATHS,
# then a file /mnt/media/.backup_exclude.txt will automatically be detected and used:
for backup_path in "${backup_paths[@]}"; do
if [ -f "$backup_path/.backup_exclude.txt" ]; then
exclusion_args=("${exclusion_args[@]}" --exclude-file "$backup_path/.backup_exclude.txt")
fi
done
# --one-file-system is not supportd on Windows (=msys).
FS_ARG=
test "$OSTYPE" = msys || FS_ARG=--one-file-system
# NOTE start all commands in background and wait for them to finish.
# Reason: bash ignores any signals while child process is executing and thus the trap exit hook is not triggered.
# However if put in subprocesses, wait(1) waits until the process finishes OR signal is received.
# Reference: https://unix.stackexchange.com/questions/146756/forward-sigterm-to-child-in-bash
# Remove locks from other stale processes to keep the automated backup running.
restic unlock &
wait $!
# Do the backup!
# See restic-backup(1) or http://restic.readthedocs.io/en/latest/040_backup.html
# --one-file-system makes sure we only backup exactly those mounted file systems specified in $RESTIC_BACKUP_PATHS, and thus not directories like /dev, /sys etc.
# --tag lets us reference these backups later when doing restic-forget.
restic backup \
--verbose="$RESTIC_VERBOSITY_LEVEL" \
$FS_ARG \
--tag "$RESTIC_BACKUP_TAG" \
--option b2.connections="$B2_CONNECTIONS" \
"${exclusion_args[@]}" \
"${extra_args[@]}" \
"${backup_paths[@]}" &
wait $!
# Dereference and delete/prune old backups.
# See restic-forget(1) or http://restic.readthedocs.io/en/latest/060_forget.html
# --group-by only the tag and path, and not by hostname. This is because I create a B2 Bucket per host, and if this hostname accidentially change some time, there would now be multiple backup sets.
restic forget \
--verbose="$RESTIC_VERBOSITY_LEVEL" \
--tag "$RESTIC_BACKUP_TAG" \
--option b2.connections="$B2_CONNECTIONS" \
--prune \
--group-by "paths,tags" \
--keep-daily "$RESTIC_RETENTION_DAYS" \
--keep-weekly "$RESTIC_RETENTION_WEEKS" \
--keep-monthly "$RESTIC_RETENTION_MONTHS" \
--keep-yearly "$RESTIC_RETENTION_YEARS" &
wait $!
# Check repository for errors.
# NOTE this takes much time (and data transfer from remote repo?), do this in a separate systemd.timer which is run less often.
#restic check &
#wait $!
echo "Backup & cleaning is done."
# (optionally) Notify about backup summary stats.
if [ "$RESTIC_NOTIFY_BACKUP_STATS" = true ]; then
if [ -w "$RESTIC_BACKUP_NOTIFICATION_FILE" ]; then
echo 'Notifications are enabled: Silently computing backup summary stats...'
snapshot_size=$(restic stats latest --tag "$RESTIC_BACKUP_TAG" | grep -i 'total size:' | cut -d ':' -f2 | xargs) # xargs acts as trim
latest_snapshot_diff=$(restic snapshots --tag "$RESTIC_BACKUP_TAG" --latest 2 --compact \
| grep -Ei "^[abcdef0-9]{8} " \
| awk '{print $1}' \
| tail -2 \
| tr '\n' ' ' \
| xargs restic diff)
added=$(echo "$latest_snapshot_diff" | grep -i 'added:' | awk '{print $2 " " $3}')
removed=$(echo "$latest_snapshot_diff" | grep -i 'removed:' | awk '{print $2 " " $3}')
echo "Added: ${added}. Removed: ${removed}. Snap size: ${snapshot_size}" >> "$RESTIC_BACKUP_NOTIFICATION_FILE"
else
echo "[WARN] Couldn't write the backup summary stats. File not found or not writable: ${RESTIC_BACKUP_NOTIFICATION_FILE}"
fi
fi

42
bin/restic_check.sh Normal file
View File

@@ -0,0 +1,42 @@
#!/usr/bin/env bash
# Check the backups made with restic to Backblaze B2 for errors.
# See restic_backup.sh on how this script is run (as it's analogous for this script).
# Exit on error, unset var, pipe failure
set -euo pipefail
# Clean up lock if we are killed.
# If killed by systemd, like $(systemctl stop restic), then it kills the whole cgroup and all it's subprocesses.
# However if we kill this script ourselves, we need this trap that kills all subprocesses manually.
exit_hook() {
echo "In exit_hook(), being killed" >&2
jobs -p | xargs kill
restic unlock
}
trap exit_hook INT TERM
# Assert that all needed environment variables are set.
assert_envvars() {
local varnames=("$@")
for varname in "${varnames[@]}"; do
if [ -z ${!varname+x} ]; then
printf "%s must be set for this script to work.\n\nDid you forget to source a $INSTALL_PREFIX/etc/restic/*.env.sh profile in the current shell before executing this script?\n" "$varname" >&2
exit 1
fi
done
}
assert_envvars \
B2_ACCOUNT_ID B2_ACCOUNT_KEY B2_CONNECTIONS \
RESTIC_PASSWORD_FILE RESTIC_REPOSITORY RESTIC_VERBOSITY_LEVEL
# Remove locks from other stale processes to keep the automated backup running.
# NOTE nope, don't unlock like restic_backup.sh. restic_backup.sh should take precedence over this script.
#restic unlock &
#wait $!
# Check repository for errors.
restic check \
--option b2.connections="$B2_CONNECTIONS" \
--verbose="$RESTIC_VERBOSITY_LEVEL" &
wait $!

147
bin/resticw Normal file
View File

@@ -0,0 +1,147 @@
#!/usr/bin/env bash
DOC="A little wrapper over restic just to handle profiles and environment loading.
Usage:
resticw [options] <restic_arguments_line>...
The <restic_arguments_line> is just the regular unwrapped restic command arguments, e.g. stats latest
Options:
-p --profile=<name> Specify the profile to load or use default [default: default].
Examples:
resticw --profile profileA snapshots
resticw stats latest # this will use the profile: default
"
# The following argument parser is generated with docopt.sh from the above docstring.
# See https://github.com/andsens/docopt.sh. If the DOC is updated or new options are added, refresh the parser!
# docopt parser below, refresh this parser with `docopt.sh resticw`
# shellcheck disable=2016,1075,2154
docopt() { parse() { if ${DOCOPT_DOC_CHECK:-true}; then local doc_hash
if doc_hash=$(printf "%s" "$DOC" | (sha256sum 2>/dev/null || shasum -a 256)); then
if [[ ${doc_hash:0:5} != "$digest" ]]; then
stderr "The current usage doc (${doc_hash:0:5}) does not match \
what the parser was generated with (${digest})
Run \`docopt.sh\` to refresh the parser."; _return 70; fi; fi; fi
local root_idx=$1; shift; argv=("$@"); parsed_params=(); parsed_values=()
left=(); testdepth=0; local arg; while [[ ${#argv[@]} -gt 0 ]]; do
if [[ ${argv[0]} = "--" ]]; then for arg in "${argv[@]}"; do
parsed_params+=('a'); parsed_values+=("$arg"); done; break
elif [[ ${argv[0]} = --* ]]; then parse_long
elif [[ ${argv[0]} = -* && ${argv[0]} != "-" ]]; then parse_shorts
elif ${DOCOPT_OPTIONS_FIRST:-false}; then for arg in "${argv[@]}"; do
parsed_params+=('a'); parsed_values+=("$arg"); done; break; else
parsed_params+=('a'); parsed_values+=("${argv[0]}"); argv=("${argv[@]:1}"); fi
done; local idx; if ${DOCOPT_ADD_HELP:-true}; then
for idx in "${parsed_params[@]}"; do [[ $idx = 'a' ]] && continue
if [[ ${shorts[$idx]} = "-h" || ${longs[$idx]} = "--help" ]]; then
stdout "$trimmed_doc"; _return 0; fi; done; fi
if [[ ${DOCOPT_PROGRAM_VERSION:-false} != 'false' ]]; then
for idx in "${parsed_params[@]}"; do [[ $idx = 'a' ]] && continue
if [[ ${longs[$idx]} = "--version" ]]; then stdout "$DOCOPT_PROGRAM_VERSION"
_return 0; fi; done; fi; local i=0; while [[ $i -lt ${#parsed_params[@]} ]]; do
left+=("$i"); ((i++)) || true; done
if ! required "$root_idx" || [ ${#left[@]} -gt 0 ]; then error; fi; return 0; }
parse_shorts() { local token=${argv[0]}; local value; argv=("${argv[@]:1}")
[[ $token = -* && $token != --* ]] || _return 88; local remaining=${token#-}
while [[ -n $remaining ]]; do local short="-${remaining:0:1}"
remaining="${remaining:1}"; local i=0; local similar=(); local match=false
for o in "${shorts[@]}"; do if [[ $o = "$short" ]]; then similar+=("$short")
[[ $match = false ]] && match=$i; fi; ((i++)) || true; done
if [[ ${#similar[@]} -gt 1 ]]; then
error "${short} is specified ambiguously ${#similar[@]} times"
elif [[ ${#similar[@]} -lt 1 ]]; then match=${#shorts[@]}; value=true
shorts+=("$short"); longs+=(''); argcounts+=(0); else value=false
if [[ ${argcounts[$match]} -ne 0 ]]; then if [[ $remaining = '' ]]; then
if [[ ${#argv[@]} -eq 0 || ${argv[0]} = '--' ]]; then
error "${short} requires argument"; fi; value=${argv[0]}; argv=("${argv[@]:1}")
else value=$remaining; remaining=''; fi; fi; if [[ $value = false ]]; then
value=true; fi; fi; parsed_params+=("$match"); parsed_values+=("$value"); done
}; parse_long() { local token=${argv[0]}; local long=${token%%=*}
local value=${token#*=}; local argcount; argv=("${argv[@]:1}")
[[ $token = --* ]] || _return 88; if [[ $token = *=* ]]; then eq='='; else eq=''
value=false; fi; local i=0; local similar=(); local match=false
for o in "${longs[@]}"; do if [[ $o = "$long" ]]; then similar+=("$long")
[[ $match = false ]] && match=$i; fi; ((i++)) || true; done
if [[ $match = false ]]; then i=0; for o in "${longs[@]}"; do
if [[ $o = $long* ]]; then similar+=("$long"); [[ $match = false ]] && match=$i
fi; ((i++)) || true; done; fi; if [[ ${#similar[@]} -gt 1 ]]; then
error "${long} is not a unique prefix: ${similar[*]}?"
elif [[ ${#similar[@]} -lt 1 ]]; then
[[ $eq = '=' ]] && argcount=1 || argcount=0; match=${#shorts[@]}
[[ $argcount -eq 0 ]] && value=true; shorts+=(''); longs+=("$long")
argcounts+=("$argcount"); else if [[ ${argcounts[$match]} -eq 0 ]]; then
if [[ $value != false ]]; then
error "${longs[$match]} must not have an argument"; fi
elif [[ $value = false ]]; then
if [[ ${#argv[@]} -eq 0 || ${argv[0]} = '--' ]]; then
error "${long} requires argument"; fi; value=${argv[0]}; argv=("${argv[@]:1}")
fi; if [[ $value = false ]]; then value=true; fi; fi; parsed_params+=("$match")
parsed_values+=("$value"); }; required() { local initial_left=("${left[@]}")
local node_idx; ((testdepth++)) || true; for node_idx in "$@"; do
if ! "node_$node_idx"; then left=("${initial_left[@]}"); ((testdepth--)) || true
return 1; fi; done; if [[ $((--testdepth)) -eq 0 ]]; then
left=("${initial_left[@]}"); for node_idx in "$@"; do "node_$node_idx"; done; fi
return 0; }; optional() { local node_idx; for node_idx in "$@"; do
"node_$node_idx"; done; return 0; }; oneormore() { local i=0
local prev=${#left[@]}; while "node_$1"; do ((i++)) || true
[[ $prev -eq ${#left[@]} ]] && break; prev=${#left[@]}; done
if [[ $i -ge 1 ]]; then return 0; fi; return 1; }; value() { local i
for i in "${!left[@]}"; do local l=${left[$i]}
if [[ ${parsed_params[$l]} = "$2" ]]; then
left=("${left[@]:0:$i}" "${left[@]:((i+1))}")
[[ $testdepth -gt 0 ]] && return 0; local value
value=$(printf -- "%q" "${parsed_values[$l]}"); if [[ $3 = true ]]; then
eval "var_$1+=($value)"; else eval "var_$1=$value"; fi; return 0; fi; done
return 1; }; stdout() { printf -- "cat <<'EOM'\n%s\nEOM\n" "$1"; }; stderr() {
printf -- "cat <<'EOM' >&2\n%s\nEOM\n" "$1"; }; error() {
[[ -n $1 ]] && stderr "$1"; stderr "$usage"; _return 1; }; _return() {
printf -- "exit %d\n" "$1"; exit "$1"; }; set -e; trimmed_doc=${DOC:0:450}
usage=${DOC:79:53}; digest=dc31d; shorts=(-p); longs=(--profile); argcounts=(1)
node_0(){ value __profile 0; }; node_1(){ value _restic_arguments_line_ a true
}; node_2(){ optional 0; }; node_3(){ optional 2; }; node_4(){ oneormore 1; }
node_5(){ required 3 4; }; node_6(){ required 5; }; cat <<<' docopt_exit() {
[[ -n $1 ]] && printf "%s\n" "$1" >&2; printf "%s\n" "${DOC:79:53}" >&2; exit 1
}'; unset var___profile var__restic_arguments_line_; parse 6 "$@"
local prefix=${DOCOPT_PREFIX:-''}; unset "${prefix}__profile" \
"${prefix}_restic_arguments_line_"
eval "${prefix}"'__profile=${var___profile:-default}'
if declare -p var__restic_arguments_line_ >/dev/null 2>&1; then
eval "${prefix}"'_restic_arguments_line_=("${var__restic_arguments_line_[@]}")'
else eval "${prefix}"'_restic_arguments_line_=()'; fi; local docopt_i=1
[[ $BASH_VERSION =~ ^4.3 ]] && docopt_i=2; for ((;docopt_i>0;docopt_i--)); do
declare -p "${prefix}__profile" "${prefix}_restic_arguments_line_"; done; }
# docopt parser above, complete command for generating this parser is `docopt.sh resticw`
# Parse arguments - See https://github.com/andsens/docopt.sh for the magic :)
DOCOPT_OPTIONS_FIRST=true # treat everything after the first non-option as commands/arguments
eval "$(docopt "$@")"
# --^^^-- END OF GENERATED COMMAND LINE PARSING STUFF --^^^--
#
# --vvv-- ACTUAL SCRIPT BELOW --vvv--
# Exit on error, unbound variable, pipe error
set -euo pipefail
ENV_DIR=$INSTALL_PREFIX/etc/restic
ERR_NO_SUCH_PROFILE=2
ERR_PROFILE_NO_READ_PERM=3
# shellcheck disable=SC2154
profile_file="${ENV_DIR}/${__profile}.env.sh"
[[ ! -f "$profile_file" ]] && echo "Invalid profile: No such environment file ${profile_file}" && exit "$ERR_NO_SUCH_PROFILE"
if [[ ! -r "$profile_file" ]]; then
echo "Error: Could not read the environment file ${profile_file}. Are you running this script as the correct user? Maybe try sudo with the right user."
exit "$ERR_PROFILE_NO_READ_PERM"
fi
echo -e "‣ Using profile: ${__profile} -- (${profile_file})\n"
# shellcheck disable=SC2154,SC1090
source "$profile_file" && restic "${_restic_arguments_line_[@]}"

View File

@@ -1,4 +1,4 @@
#!/usr/bin/env sh
#!/usr/bin/env bash
# Send email notification from systemd.
# Source: https://serverfault.com/questions/876233/how-to-send-an-email-if-a-systemd-service-is-restarted
# Source: https://wiki.archlinux.org/index.php/Systemd/Timers#MAILTO
@@ -15,19 +15,19 @@
# (24 * 60 * 60) / 100 = 864 second
# One option that I used with my old Axis cameras it to use my gmx.com accunt for sending emails instead, as there are (no?) higher limits there.
MIN_WAIT_TIME_S=900
SCRIPT_NAME=$(basename $0)
SCRIPT_NAME=$(basename "$0")
LAST_RUN_FILE="/tmp/${SCRIPT_NAME}_last_run.txt"
last_touch() {
stat -c %Y $1
stat -c %Y "$1"
}
waited_long_enough() {
retval=1
if [ -e $LAST_RUN_FILE ]; then
if [ -e "$LAST_RUN_FILE" ]; then
now=$(date +%s)
last=$(last_touch $LAST_RUN_FILE)
wait_s=$(expr $now - $last)
last=$(last_touch "$LAST_RUN_FILE")
wait_s=$((now - last))
if [ "$wait_s" -gt "$MIN_WAIT_TIME_S" ]; then
retval=0
fi
@@ -35,7 +35,7 @@ waited_long_enough() {
retval=0
fi
[ $retval -eq 0 ] && touch $LAST_RUN_FILE
[ $retval -eq 0 ] && touch "$LAST_RUN_FILE"
return $retval
}

View File

@@ -1,8 +1,13 @@
SHELL=/bin/sh
PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin/:/usr/local/sbin/
PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin/:$INSTALL_PREFIX/bin/
# Order of crontab fields
# minute hour mday month wday command
# Reference: https://www.freebsd.org/doc/handbook/configtuning-cron.html
# Reference: crontab(5).
@midnight root cron_mail restic_backup.sh
@monthly root cron_mail restic_check.sh
@midnight root . $INSTALL_PREFIX/etc/restic/default.env.sh && restic_backup.sh
@monthly root . $INSTALL_PREFIX/etc/restic/default.env.sh && restic_check.sh
# Email notification version. Make sure bin/cron_mail is in the above $PATH
#@midnight root . $INSTALL_PREFIX/etc/restic/default.env.sh && cron_mail restic_backup.sh
#@monthly root . $INSTALL_PREFIX/etc/restic/default.env.sh && cron_mail restic_check.sh

35
etc/restic/_global.env.sh Normal file
View File

@@ -0,0 +1,35 @@
# shellcheck shell=sh
# Global envionment variables
# These variables are sourced FIRST, and any values inside of *.env.sh files for
# specific configurations will override if also defined there.
# Official instructions on how to setup the restic variables for Backblaze B2 can be found at
# https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#backblaze-b2
# The restic repository encryption key
export RESTIC_PASSWORD_FILE="$INSTALL_PREFIX/etc/restic/pw.txt"
# The global restic exclude file
export RESTIC_BACKUP_EXCLUDE_FILE="$INSTALL_PREFIX/etc/restic/backup_exclude.txt"
# Backblaze B2 credentials keyID & applicationKey pair.
# Restic environment variables are documented at https://restic.readthedocs.io/en/latest/040_backup.html#environment-variables
export B2_ACCOUNT_ID="<b2-key-id>" # *EDIT* fill with your keyID
export B2_ACCOUNT_KEY="<b2-application-key>" # *EDIT* fill with your applicationKey
# How many network connections to set up to B2. Default is 5.
export B2_CONNECTIONS=10
# Optional extra space-separated args to restic-backup.
# This is empty here and profiles can override this after sourcing this file.
export RESTIC_BACKUP_EXTRA_ARGS=
# Verbosity level from 0-3. 0 means no --verbose.
# Override this value in a profile if needed.
export RESTIC_VERBOSITY_LEVEL=0
# (optional) Desktop notifications. See restic_backup.sh for details on how to set this up.
export RESTIC_NOTIFY_BACKUP_STATS=false
export RESTIC_BACKUP_NOTIFICATION_FILE=

View File

@@ -1,8 +0,0 @@
# B2 credentials.
# Extracted settings so both systemd timers and user can just source this when want to work on my B2 backup.
# See https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html
export RESTIC_REPOSITORY="b2:<b2-repo-name>"
export RESTIC_PASSWORD_FILE="/etc/restic/b2_pw.txt"
export B2_ACCOUNT_ID="<b2-account-id>"
export B2_ACCOUNT_KEY="<b2-account-key>"

View File

@@ -1 +0,0 @@
<b2-password>

44
etc/restic/default.env.sh Normal file
View File

@@ -0,0 +1,44 @@
# shellcheck shell=sh
# This is the default profile. Fill it with your desired configuration.
# Additionally, you can create and use more profiles by copying this file.
# This file (and other .env.sh files) has two purposes:
# - being sourced by systemd timers to setup the backup before running restic_backup.sh
# - being sourced in a user's shell to work directly with restic commands e.g.
# $ source /etc/restic/default.env.sh
# $ restic snapshots
# Thus you don't have to provide all the arguments like
# $ restic --repo ... --password-file ...
# shellcheck source=etc/restic/_global.env.sh
. "$INSTALL_PREFIX/etc/restic/_global.env.sh"
# Envvars below will override those in _global.env.sh if present.
export RESTIC_REPOSITORY="b2:<b2-repo-name>" # *EDIT* fill with your repo name
# What to backup. Colon-separated paths e.g. to different mountpoints "/home:/mnt/usb_disk".
# To backup only your home directory, set "/home/your-user"
export RESTIC_BACKUP_PATHS="" # *EDIT* fill conveniently with one or multiple paths
# Example below of how to dynamically add a path that is mounted e.g. external USB disk.
# restic does not fail if a specified path is not mounted, but it's nicer to only add if they are available.
#test -d /mnt/media && RESTIC_BACKUP_PATHS+=" /mnt/media"
# A tag to identify backup snapshots.
export RESTIC_BACKUP_TAG=systemd.timer
# Retention policy - How many backups to keep.
# See https://restic.readthedocs.io/en/stable/060_forget.html?highlight=month#removing-snapshots-according-to-a-policy
export RESTIC_RETENTION_DAYS=14
export RESTIC_RETENTION_WEEKS=16
export RESTIC_RETENTION_MONTHS=18
export RESTIC_RETENTION_YEARS=3
# Optional extra space-separated arguments to restic-backup.
# Example: Add two additional exclude files to the global one in RESTIC_PASSWORD_FILE.
#RESTIC_BACKUP_EXTRA_ARGS="--exclude-file /path/to/extra/exclude/file/a --exclude-file /path/to/extra/exclude/file/b"
# Example: exclude all directories that have a .git/ directory inside it.
#RESTIC_BACKUP_EXTRA_ARGS="--exclude-if-present .git"

1
etc/restic/pw.txt Normal file
View File

@@ -0,0 +1 @@
<restic-encryption-password>

View File

@@ -1,11 +0,0 @@
[Unit]
Description=Backup with restic to Backblaze B2
OnFailure=status-email-user@%n.service
Requires=nm-unmetered-connection.service
[Service]
Type=simple
Nice=10
ExecStart=/usr/local/sbin/restic_backup.sh
# $HOME or $XDG_CACHE_HOME must be set for restic to find /root/.cache/restic/
Environment="HOME=/root"

View File

@@ -1,10 +0,0 @@
[Unit]
Description=Check restic backup Backblaze B2 for errors
OnFailure=status-email-user@%n.service
Conflicts=restic-backup.service
Requires=nm-unmetered-connection.service
[Service]
Type=simple
Nice=10
ExecStart=/usr/local/sbin/restic_check.sh

BIN
img/macos_notification.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 33 KiB

BIN
img/pen-paper.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 7.5 KiB

BIN
img/plus.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.9 KiB

BIN
img/readme_sections.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

BIN
img/tasksched.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 136 KiB

View File

@@ -3,5 +3,4 @@ Description=Check if the current NetworkManager connection is metered
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/nm-unmetered-connection.sh
ExecStart=$INSTALL_PREFIX/bin/nm-unmetered-connection.sh

View File

@@ -0,0 +1,16 @@
[Unit]
Description=Backup with restic to Backblaze B2
# Email on failure require special setup. See README.md
#OnFailure=status-email-user@%n.service
# Prevent backup on unmetered connection. Needs special setup. See README.md.
#Requires=nm-unmetered-connection.service
[Service]
Type=simple
Nice=10
# $HOME or $XDG_CACHE_HOME must be set for restic to find /root/.cache/restic/
Environment="HOME=/root"
# pipefail: so that redirecting stderr from the script to systemd-cat does not hide the failed command from OnFailure above.
# Random sleep (in seconds): in the case of multiple backup profiles. Many restic instances started at the same time could case high load or network bandwith usage.
# `systemd-cat` allows showing the restic output to the systemd journal
ExecStart=bash -c 'set -o pipefail; ps cax | grep -q restic && sleep $(shuf -i 0-300 -n 1); source $INSTALL_PREFIX/etc/restic/%I.env.sh && $INSTALL_PREFIX/bin/restic_backup.sh 2>&1 | systemd-cat'

View File

@@ -1,6 +1,5 @@
[Unit]
Description=Backup with restic on schedule
Requires=nm-unmetered-connection.service
[Timer]
OnCalendar=daily

View File

@@ -0,0 +1,14 @@
[Unit]
Description=Check restic backup Backblaze B2 for errors
# Email on failure require special setup. See README.md
#OnFailure=status-email-user@%n.service
Conflicts=restic-backup.service
# Prevent backup on unmetered connection. Needs special setup. See README.md.
#Requires=nm-unmetered-connection.service
[Service]
Type=simple
Nice=10
# pipefail: so that redirecting stderr from the script to systemd-cat does not hide the failed command from OnFailure above.
# `systemd-cat`: allows showing the restic output to the systemd journal
ExecStart=bash -c 'set -o pipefail; source $INSTALL_PREFIX/etc/restic/%I.env.sh && $INSTALL_PREFIX/bin/restic_check.sh 2>&1 | systemd-cat'

View File

@@ -1,6 +1,5 @@
[Unit]
Description=Check restic backup Backblaze B2 for errors on a schedule
Requires=nm-unmetered-connection.service
[Timer]
OnCalendar=monthly

View File

@@ -6,6 +6,6 @@ Description=Send status email for %i to user
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/systemd-email abc@gmail.com %i
ExecStart=$INSTALL_PREFIX/bin/systemd-email abc@gmail.com %i
User=root
Group=systemd-journal

View File

@@ -1,88 +0,0 @@
#!/usr/bin/env bash
# Make backup my system with restic to Backblaze B2.
# This script is typically run by: /etc/systemd/system/restic-backup.{service,timer}
# Exit on failure, pipe failure
set -e -o pipefail
# Clean up lock if we are killed.
# If killed by systemd, like $(systemctl stop restic), then it kills the whole cgroup and all it's subprocesses.
# However if we kill this script ourselves, we need this trap that kills all subprocesses manually.
exit_hook() {
echo "In exit_hook(), being killed" >&2
jobs -p | xargs kill
restic unlock
}
trap exit_hook INT TERM
# How many backups to keep.
RETENTION_DAYS=14
RETENTION_WEEKS=16
RETENTION_MONTHS=18
RETENTION_YEARS=3
# What to backup, and what to not
BACKUP_PATHS="/ /boot /home"
[ -d /mnt/media ] && BACKUP_PATHS+=" /mnt/media"
BACKUP_EXCLUDES="--exclude-file /etc/restic/backup_exclude"
for dir in /home/*
do
if [ -f "$dir/.backup_exclude" ]
then
BACKUP_EXCLUDES+=" --exclude-file $dir/.backup_exclude"
fi
done
BACKUP_TAG=systemd.timer
# Set all environment variables like
# B2_ACCOUNT_ID, B2_ACCOUNT_KEY, RESTIC_REPOSITORY etc.
source /etc/restic/b2_env.sh
# How many network connections to set up to B2. Default is 5.
B2_CONNECTIONS=50
# NOTE start all commands in background and wait for them to finish.
# Reason: bash ignores any signals while child process is executing and thus my trap exit hook is not triggered.
# However if put in subprocesses, wait(1) waits until the process finishes OR signal is received.
# Reference: https://unix.stackexchange.com/questions/146756/forward-sigterm-to-child-in-bash
# Remove locks from other stale processes to keep the automated backup running.
restic unlock &
wait $!
# Do the backup!
# See restic-backup(1) or http://restic.readthedocs.io/en/latest/040_backup.html
# --one-file-system makes sure we only backup exactly those mounted file systems specified in $BACKUP_PATHS, and thus not directories like /dev, /sys etc.
# --tag lets us reference these backups later when doing restic-forget.
restic backup \
--verbose \
--one-file-system \
--tag $BACKUP_TAG \
--option b2.connections=$B2_CONNECTIONS \
$BACKUP_EXCLUDES \
$BACKUP_PATHS &
wait $!
# Dereference and delete/prune old backups.
# See restic-forget(1) or http://restic.readthedocs.io/en/latest/060_forget.html
# --group-by only the tag and path, and not by hostname. This is because I create a B2 Bucket per host, and if this hostname accidentially change some time, there would now be multiple backup sets.
restic forget \
--verbose \
--tag $BACKUP_TAG \
--option b2.connections=$B2_CONNECTIONS \
--prune \
--group-by "paths,tags" \
--keep-daily $RETENTION_DAYS \
--keep-weekly $RETENTION_WEEKS \
--keep-monthly $RETENTION_MONTHS \
--keep-yearly $RETENTION_YEARS &
wait $!
# Check repository for errors.
# NOTE this takes much time (and data transfer from remote repo?), do this in a separate systemd.timer which is run less often.
#restic check &
#wait $!
echo "Backup & cleaning is done."

View File

@@ -1,33 +0,0 @@
#!/usr/bin/env bash
# Check my backup with restic to Backblaze B2 for errors.
# This script is typically run by: /etc/systemd/system/restic-check.{service,timer}
# Exit on failure, pipe failure
set -e -o pipefail
# Clean up lock if we are killed.
# If killed by systemd, like $(systemctl stop restic), then it kills the whole cgroup and all it's subprocesses.
# However if we kill this script ourselves, we need this trap that kills all subprocesses manually.
exit_hook() {
echo "In exit_hook(), being killed" >&2
jobs -p | xargs kill
restic unlock
}
trap exit_hook INT TERM
source /etc/restic/b2_env.sh
# How many network connections to set up to B2. Default is 5.
B2_CONNECTIONS=50
# Remove locks from other stale processes to keep the automated backup running.
# NOTE nope, don't unlock like restic_backup.sh. restic_backup.sh should take precedence over this script.
#restic unlock &
#wait $!
# Check repository for errors.
restic check \
--option b2.connections=$B2_CONNECTIONS \
--verbose &
wait $!