Modified: CHANGELOG.md

Systemd service sources `$INSTALL_PREFIX/etc/restic/%I.env.sh`: 
https://github.com/erikw/restic-automatic-backup-scheduler/blob/main/usr/lib/systemd/system/restic-backup%40.service#L16

.github/workflows/linter.yml

@@ -3,13 +3,13 @@ name: Lint Code Base
 on:
   workflow_dispatch:
   push:
-    branches: master
+    branches: main
     paths:
       - '**.sh'
       - '.github/workflows/linter.yml'
       - 'bin/**'
   pull_request:
-    branches: master
+    branches: main
     paths:
       - '**.sh'
       - '.github/workflows/linter.yml'
@@ -29,5 +29,5 @@ jobs:
         env:
           VALIDATE_ALL_CODEBASE: true
           VALIDATE_BASH: true
-          DEFAULT_BRANCH: master
+          DEFAULT_BRANCH: main
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

CHANGELOG.md

@@ -5,6 +5,29 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
+### Added
+- Support saving hourly snapshots. [#98](https://github.com/erikw/restic-automatic-backup-scheduler/pull/98)
+
+### Fixed
+- Full path to `/bin/bash` in sytemd services. [#96](https://github.com/erikw/restic-automatic-backup-scheduler/issues/96)
+
+## [7.3.4] - 2022-04-29
+### Fixed
+- Backup stats notifications: fix issue where `restic snapshots --latest 2` will show more than two snapshots due to different backup paths used.
+
+## [7.3.3] - 2022-04-14
+### Fixed
+- Trying to fix broken Homebrew bottles due to GitHub API issues.
+
+## [7.3.2] - 2022-04-11
+### Fixed
+- Trying to fix broken Homebrew bottles
+
+## [7.3.1] - 2022-04-11
+### Fixed
+- `resticw` is now a true wrapper in that it support `--` args to restic.
+- OnFailure no longer masked by the stderr redirect to systemd-cat. [#86](https://github.com/erikw/restic-automatic-backup-scheduler/pull/86)
+
 ## [7.3.0] - 2022-02-15
 ### Added
 - optional user-controlled notification. See `RESTIC_NOTIFY_BACKUP_STATS` and in `backup.sh`.

README.md

@@ -14,6 +14,7 @@
 [![Closed PRs](https://img.shields.io/github/issues-pr-closed/erikw/restic-automatic-backup-scheduler?color=success)](https://github.com/erikw/restic-automatic-backup-scheduler/pulls?q=is%3Apr+is%3Aclosed)
 [![License](https://img.shields.io/badge/license-BSD--3-blue)](LICENSE)
 [![OSS Lifecycle](https://img.shields.io/osslifecycle/erikw/restic-automatic-backup-scheduler)](https://github.com/Netflix/osstracker)
+[![SLOC](https://img.shields.io/tokei/lines/github/erikw/restic-automatic-backup-scheduler?logo=codefactor&logoColor=lightgrey)](#)
 <br>
 
 [![Contributors](https://img.shields.io/github/contributors/erikw/restic-automatic-backup-scheduler)](https://github.com/erikw/restic-automatic-backup-scheduler/graphs/contributors) including these top contributors:
@@ -76,7 +77,7 @@ Many Linux distributions nowadays use [Systemd](https://en.wikipedia.org/wiki/Sy
    ```
 1. Fill out [configuration values](#2-configure-b2-credentials-locally) in `/etc/restic`.
 1. [Initialize](#3-initialize-remote-repo) the remote repo.
-	Source the profile to make all needed configuration available to `restic`. All commands after this assumes the profile is sourced in the current shell.
+	Source the profile to make all needed configuration available to `restic(1)`. All commands after this assumes the profile is sourced in the current shell.
    ```console
 	# source /etc/restic/default.env.sh
 	# restic init
@@ -109,9 +110,9 @@ Many Linux distributions nowadays use [Systemd](https://en.wikipedia.org/wiki/Sy
 ## Setup macOS LaunchAgent
 <img height="64" width="64" src="https://unpkg.com/simple-icons@v6/icons/apple.svg" />
 
-☝ **Note** The macOS setup here will assume an installation to `/usr/local`, as [custom](https://docs.brew.sh/FAQ#why-does-homebrew-say-sudo-is-bad) with Homebrew installations.
+☝ **Note** The macOS setup here will assume a Homebrew installation to the [recommended default location](https://docs.brew.sh/FAQ#why-should-i-install-homebrew-in-the-default-location). This is [`$HOMEBREW_PREFIX` (`brew --prefix`)](https://docs.brew.sh/Formula-Cookbook#variables-for-directory-locations) , which is `/usr/local` on Intel Macs and `/opt/homebrew` on [Apple Silicon](https://docs.brew.sh/FAQ#why-is-the-default-installation-prefix-opthomebrew-on-apple-silicon).
 
-[Launchd](https://www.launchd.info/) is the modern built-in service scheduler in macOS. It has support for running services as root (Daemon) or as a normal user (Agent). Here we set up an LauchAgent to be run as your normal user for starting regular backups.
+[Launchd](https://www.launchd.info/) is the modern built-in service scheduler in macOS. It has support for running services as root (Daemon) or as a normal user (Agent). Here we set up a LaunchAgent to be run as your normal user for starting regular backups.
 
 **TL;DR setup**
 1. [Create](#1-create-backblaze-b2-account-bucket-and-keys) B2 bucket + credentials
@@ -122,13 +123,13 @@ Many Linux distributions nowadays use [Systemd](https://en.wikipedia.org/wiki/Sy
    ```
    * Using `make`:
    ```console
-	$ make PREFIX=/usr/local install-launchagent
+	$ make PREFIX=$(brew --prefix) install-launchagent
    ```
-1. Fill out [configuration values](#2-configure-b2-credentials-locally) in `/usr/local/etc/restic`.
+1. Fill out [configuration values](#2-configure-b2-credentials-locally) in `$(brew --prefix)/etc/restic`.
 1. [Initialize](#3-initialize-remote-repo) the remote repo.
-	Source the profile to make all needed configuration available to `restic`. All commands after this assumes the profile is sourced in the current shell.
+	Source the profile to make all needed configuration available to `restic(1)`. All commands after this assumes the profile is sourced in the current shell.
    ```console
-	$ source /usr/local/etc/restic/default.env.sh
+	$ source $(brew --prefix)/etc/restic/default.env.sh
 	$ restic init
    ```
 1. Configure [how often](https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html#//apple_ref/doc/uid/10000172i-CH1-SW1) backups should be done. If needed, edit `OnCalendar` in
@@ -226,7 +227,7 @@ I describe here one of may ways you can get restic and this backup script workin
 	export RESTIC_BACKUP_PATHS='/c/Users/<username>/My Documents'
    ```
 1. [Initialize](#3-initialize-remote-repo) the remote repo.
-	Source the profile to make all needed configuration available to `restic`. All commands after this assumes the profile is sourced in the current shell.
+	Source the profile to make all needed configuration available to `restic(1)`. All commands after this assumes the profile is sourced in the current shell.
    ```console
     git-bash$ source /etc/restic/default.env.sh
     git-bash$ restic init
@@ -268,9 +269,12 @@ Any system that has a cron-like system can easily setup restic backups as well.
    $ sudo make install-cron
    ```
    * This assumes that your cron supports dropping files into `/etc/cron.d/`. If that is not the case, simply copy the relevant contents of the installed `/etc/cron.d/restic` in to your `/etc/crontab`.
+   ```console
+   # grep "^@.*restic_" /etc/cron.d/restic >> /etc/crontab
+   ```
 1. Fill out [configuration values](#2-configure-b2-credentials-locally) in `/etc/restic`.
 1. [Initialize](#3-initialize-remote-repo) the remote repo.
-	Source the profile to make all needed configuration available to `restic`. All commands after this assumes the profile is sourced in the current shell.
+	Source the profile to make all needed configuration available to `restic(1)`. All commands after this assumes the profile is sourced in the current shell.
    ```console
 	# source /etc/restic/default.env.sh
 	# restic init
@@ -430,11 +434,38 @@ To have different backup jobs having e.g. different buckets, backup path of sche
 To create a different backup and use you can do:
 ```console
 # cp /etc/restic/default.env.sh /etc/restic/other.env.sh
-# vim /etc/restic/default.other.sh  # Set backup path, bucket etc.
-# source /etc/restic/default.other.sh
+# vim /etc/restic/other.env.sh  # Set backup path, bucket etc.
+# source /etc/restic/other.env.sh
 # restic_backup.sh
 ```
 
+### Optional: Desktop Notifications
+<img src="img/macos_notification.png" align="right" />
+
+It's a good idea to be on top of your backups to make sure that they don't increase a lot in size and incur high costs. However it's notoriously tricky to make GUI notifications correctly from a non-user process (e.g. root).
+
+Therefore this project provides a lightweight solution for desktop notifications that works like this: Basically `restic_backup.sh` will append a summary line of the last backup to a user-owned file (the user running your OS's desktop environment) in a fire-and-forget fashion. Then the user has a process that reads this and forward each line as a new message to the desktop environment in use.
+
+To set desktop notifications up:
+1. Create a special FIFO file as your desktop user:
+	```console
+	$ mkfifo /home/user/.cache/notification-queue
+	```
+1. In your profile, e.g. `/etc/restic/default.sh`, set:
+	```bash
+	RESTIC_NOTIFY_BACKUP_STATS=true
+	RESTIC_BACKUP_NOTIFICATION_FILE=/home/user/.cache/notification-queue
+	```
+1. Create a listener on the notification queue file that forwards to desktop notifications
+   * Linux auto start + cross-platform notifier / notify-send
+      * [notification-queue-notifier](https://github.com/gerardbosch/dotfiles/blob/ddc1491056822eab45dedd131f1946308ef62135/home/bin/notification-queue-notifier)
+      * [notification-queue.desktop](https://github.com/gerardbosch/dotfiles-linux/blob/ea0f75bfd7a356945544ecaa42a2fc35c9fab3a1/home/.config/autostart/notification-queue.desktop)
+   * macOS auto start + [terminal-notifier](https://github.com/julienXX/terminal-notifier)
+      * [notification-queue-notifier.sh](https://github.com/erikw/dotfiles/blob/8a942defe268292200b614951cdf433ddccf7170/bin/notification-queue-notifier.sh)
+      * [com.user.notificationqueue.plist](https://github.com/erikw/dotfiles/blob/8a942defe268292200b614951cdf433ddccf7170/.config/LaunchAgents/com.user.notificationqueue.plist)
+
+
+
 ### Optional: Email Notification on Failure
 #### Systemd
 We want to be aware when the automatic backup fails, so we can fix it. Since my laptop does not run a mail server, I went for a solution to set up my laptop to be able to send emails with [postfix via my Gmail](https://easyengine.io/tutorials/linux/ubuntu-postfix-gmail-smtp/). Follow the instructions over there.
@@ -445,7 +476,7 @@ Put this file in `/bin`:
 Put this files in `/etc/systemd/system/`:
 * `status-email-user@.service`: A service that can notify you via email when a systemd service fails. Edit the target email address in this file.
 
-Now edit `restic-backup.service` and `status-email-user.service` to call this service failure.
+Now edit `restic-backup@.service` and `status-email-user@.service` to call this service failure.
 ```
 OnFailure=status-email-user@%n.service
 ```
@@ -459,18 +490,31 @@ To use this, wrap the restic script command with it in your cron file like:
 ```
 
 
-### Optional: No Backup on Metered Connections
+### Optional: No Backup on Metered Connections (Linux/systemd only)
 For a laptop, it can make sense to not do heavy backups when your on a metered connection like a shared connection from you mobile phone. To solve this we can set up a systemd service that is in success state only when a connection is unmetered. Then we can tell our backup service to depend on this service simply! When the unmetered service detects an unmetered connection it will go to failed state. Then our backup service will not run as it requires this other service to be in success state.
 
-Put this file in `/bin`:
-* `nm-unmetered-connection.sh`: Detects metered connections and returns will error code if one is detected. This scripts requires the Gnome [NetworkManager](https://wiki.gnome.org/Projects/NetworkManager) to be installed. Modify this script if your system has a different network manager.
+1. Edit `restic-backup@.service` and `restic-check@.service` to require the new service to be in success state:
+   ```
+   Requires=nm-unmetered-connection.service
+   ```
+1. Copy and paste the command below, it will install the following files and refresh systemd daemon:
+1. Put this file in `/etc/systemd/system/`:
+   * `nm-unmetered-connection.service`: A service that is in success state only if the connection is unmetered.
+1. Install this file in `/bin`:
+   * `nm-unmetered-connection.sh`: Detects metered connections and returns an error code if one is detected. This scripts requires the Gnome [NetworkManager](https://wiki.gnome.org/Projects/NetworkManager) to be installed (modify this script if your system has a different network manager).
+1. Reload systemd with
+	```console
+	# systemctl daemon-reload
+	```
 
-Put this files in `/etc/systemd/system/`:
-* `nm-unmetered-connection.service`: A service that is in success state if the connection is unmetered only.
-
-Now edit `restic-backup.service` and `status-email-user.service` to require the new service to be in success state:
-```
-Requires=nm-unmetered-connection.service
+☝ **Tip**: All steps but the first can be done in one go if you use the Makefile. Set `$PREFIX` as needed or leave empty for install to `/`.
+```bash
+sudo bash -c 'export PREFIX=
+ make build/usr/lib/systemd/system/nm-unmetered-connection.service
+ install -m 0644 build/usr/lib/systemd/system/nm-unmetered-connection.service $PREFIX/etc/systemd/system
+ install -m 0555 bin/nm-unmetered-connection.sh /bin
+ systemctl daemon-reload
+'
 ```
 
 ### Optional: Restic Wrapper Script
@@ -484,8 +528,8 @@ Useful commands:
 | Command                                           | Description                                                       |
 |---------------------------------------------------|-------------------------------------------------------------------|
 | `resticw snapshots`                               | List backup snapshots                                             |
-| `resticw diff <snapshot-id> latest`               | Show the changes from the latest backup                           |
-| `resticw stats` / `resticw stats snapshot-id ...` | Show the statistics for the whole repo or the specified snapshots |
+| `resticw diff <snapshotId-1> <snapshotId-2>`      | Show the changes between backup snapshots                         |
+| `resticw stats` / `resticw stats snapshotId ...`  | Show the statistics for the whole repo or the specified snapshots |
 | `resticw mount /mnt/restic`                       | Mount your remote repository                                      |
 
 
@@ -516,7 +560,7 @@ To make a new release:
    $ git tag vX.Y.Z
    $ git push && git push --tags
    ```
-1. Update version in the AUR [PKGBUILD](https://aur.archlinux.org/packages/restic-automatic-backup-scheduler/).
-1. Update version in the Homebrew Formulas:
-   * [restic-automatic-backup-scheduler](https://github.com/erikw/homebrew-tap/blob/main/Formula/restic-automatic-backup-scheduler.rb).
-   * [restic-automatic-backup-scheduler-check](https://github.com/erikw/homebrew-tap/blob/main/Formula/restic-automatic-backup-scheduler-check.rb).
+1. Update version in the AUR [PKGBUILD](https://aur.archlinux.org/packages/restic-automatic-backup-scheduler/)
+1. Update version in the Homebrew Formulas (see the repo README):
+   * [restic-automatic-backup-scheduler](https://github.com/erikw/homebrew-tap/blob/main/Formula/restic-automatic-backup-scheduler.rb)
+   * [restic-automatic-backup-scheduler-check](https://github.com/erikw/homebrew-tap/blob/main/Formula/restic-automatic-backup-scheduler-check.rb)

bin/restic_backup.sh

@@ -28,7 +28,7 @@ assert_envvars() {
 	local varnames=("$@")
 	for varname in "${varnames[@]}"; do
 		if [ -z ${!varname+x} ]; then
-			printf "%s must be set for this script to work.\n\nDid you forget to source a /etc/restic/*.env.sh profile in the current shell before executing this script?\n" "$varname" >&2
+			printf "%s must be set for this script to work.\n\nDid you forget to source a $INSTALL_PREFIX/etc/restic/*.env.sh profile in the current shell before executing this script?\n" "$varname" >&2
 			exit 1
 		fi
 	done
@@ -93,6 +93,7 @@ restic forget \
 	--option b2.connections="$B2_CONNECTIONS" \
 	--prune \
 	--group-by "paths,tags" \
+	--keep-hourly "$RESTIC_RETENTION_HOURS" \  
 	--keep-daily "$RESTIC_RETENTION_DAYS" \
 	--keep-weekly "$RESTIC_RETENTION_WEEKS" \
 	--keep-monthly "$RESTIC_RETENTION_MONTHS" \
@@ -106,21 +107,7 @@ wait $!
 
 echo "Backup & cleaning is done."
 
-#
 # (optionally) Notify about backup summary stats.
-#
-# How to perform the notification is up to the user; the script only writes the info to a user-owned file in a fire
-# and forget fashion. One option is using a special FIFO file (aka pipe file) on user-side (which will work as a queue)
-# together with an auto-started user process to read from that queue and trigger the notification.
-#
-# Examples of such setup:
-#   - Linux autostart + cross-platform notifier
-#       https://github.com/gerardbosch/dotfiles-linux/blob/ea0f75bfd7a356945544ecaa42a2fc35c9fab3a1/home/.config/autostart/notification-queue.desktop
-#       https://github.com/gerardbosch/dotfiles/blob/ddc1491056822eab45dedd131f1946308ef62135/home/bin/notification-queue-notifier
-#   - MacOS autostart + terminal-notifier
-#       https://github.com/erikw/dotfiles/blob/c25f44db1cad675becf91fc3ff28a5a4dfc4a373/bin/com.user.notificationqueue.plist
-#       https://github.com/erikw/dotfiles/blob/c25f44db1cad675becf91fc3ff28a5a4dfc4a373/bin/notification-queue-notifier.sh
-#
 if [ "$RESTIC_NOTIFY_BACKUP_STATS" = true ]; then
 	if [ -w "$RESTIC_BACKUP_NOTIFICATION_FILE" ]; then
 		echo 'Notifications are enabled: Silently computing backup summary stats...'
@@ -129,6 +116,7 @@ if [ "$RESTIC_NOTIFY_BACKUP_STATS" = true ]; then
 		latest_snapshot_diff=$(restic snapshots --tag "$RESTIC_BACKUP_TAG" --latest 2 --compact \
 			| grep -Ei "^[abcdef0-9]{8} " \
 			| awk '{print $1}' \
+			| tail -2 \
 			| tr '\n' ' ' \
 			| xargs restic diff)
         added=$(echo "$latest_snapshot_diff" | grep -i 'added:' | awk '{print $2 " " $3}')

bin/restic_check.sh

@@ -20,7 +20,7 @@ assert_envvars() {
 	local varnames=("$@")
 	for varname in "${varnames[@]}"; do
 		if [ -z ${!varname+x} ]; then
-			printf "%s must be set for this script to work.\n\nDid you forget to source a /etc/restic/*.env.sh profile in the current shell before executing this script?\n" "$varname" >&2
+			printf "%s must be set for this script to work.\n\nDid you forget to source a $INSTALL_PREFIX/etc/restic/*.env.sh profile in the current shell before executing this script?\n" "$varname" >&2
 			exit 1
 		fi
 	done

bin/resticw

@@ -116,8 +116,9 @@ else eval "${prefix}"'_restic_arguments_line_=()'; fi; local docopt_i=1
 declare -p "${prefix}__profile" "${prefix}_restic_arguments_line_"; done; }
 # docopt parser above, complete command for generating this parser is `docopt.sh resticw`
 
-# Parse arguments
-eval "$(docopt "$@")" # See https://github.com/andsens/docopt.sh for the magic :)
+# Parse arguments - See https://github.com/andsens/docopt.sh for the magic :)
+DOCOPT_OPTIONS_FIRST=true  # treat everything after the first non-option as commands/arguments
+eval "$(docopt "$@")"
 
 # --^^^-- END OF GENERATED COMMAND LINE PARSING STUFF --^^^--
 #

etc/cron.d/restic

@@ -5,9 +5,9 @@ PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin/:$INSTALL_PREFIX/bin/
 # Reference: https://www.freebsd.org/doc/handbook/configtuning-cron.html
 # Reference: crontab(5).
 
-@midnight	root	. $INSTALL_PREFIX/etc/restic/default.sh && restic_backup.sh
-@monthly	root	. $INSTALL_PREFIX/etc/restic/default.sh && restic_check.sh
+@midnight	root	. $INSTALL_PREFIX/etc/restic/default.env.sh && restic_backup.sh
+@monthly	root	. $INSTALL_PREFIX/etc/restic/default.env.sh && restic_check.sh
 
 # Email notification version. Make sure bin/cron_mail is in the above $PATH
-#@midnight	root	. $INSTALL_PREFIX/etc/restic/default.sh && cron_mail restic_backup.sh
-#@monthly	root	. $INSTALL_PREFIX/etc/restic/default.sh && cron_mail restic_check.sh
+#@midnight	root	. $INSTALL_PREFIX/etc/restic/default.env.sh && cron_mail restic_backup.sh
+#@monthly	root	. $INSTALL_PREFIX/etc/restic/default.env.sh && cron_mail restic_check.sh

etc/restic/default.env.sh

@@ -32,6 +32,7 @@ export RESTIC_BACKUP_TAG=systemd.timer
 
 # Retention policy - How many backups to keep.
 # See https://restic.readthedocs.io/en/stable/060_forget.html?highlight=month#removing-snapshots-according-to-a-policy
+export RESTIC_RETENTION_HOURS=1
 export RESTIC_RETENTION_DAYS=14
 export RESTIC_RETENTION_WEEKS=16
 export RESTIC_RETENTION_MONTHS=18

usr/lib/systemd/system/restic-backup@.service

@@ -10,6 +10,7 @@ Type=simple
 Nice=10
 # $HOME or $XDG_CACHE_HOME must be set for restic to find /root/.cache/restic/
 Environment="HOME=/root"
-# The random sleep (in seconds) is in the case of multiple backup profiles. Many restic instances started at the same time could case high load or network bandwith usage.
+# pipefail: so that redirecting stderr from the script to systemd-cat does not hide the failed command from OnFailure above.
+# Random sleep (in seconds): in the case of multiple backup profiles. Many restic instances started at the same time could case high load or network bandwith usage.
 # `systemd-cat` allows showing the restic output to the systemd journal
-ExecStart=bash -c 'ps cax | grep -q restic && sleep $(shuf -i 0-300 -n 1); source $INSTALL_PREFIX/etc/restic/%I.env.sh && $INSTALL_PREFIX/bin/restic_backup.sh | systemd-cat'
+ExecStart=/bin/bash -c 'set -o pipefail; ps cax | grep -q restic && sleep $(shuf -i 0-300 -n 1); source $INSTALL_PREFIX/etc/restic/%I.env.sh && $INSTALL_PREFIX/bin/restic_backup.sh 2>&1 | systemd-cat'

usr/lib/systemd/system/restic-check@.service

@@ -9,5 +9,6 @@ Conflicts=restic-backup.service
 [Service]
 Type=simple
 Nice=10
-# `systemd-cat` allows showing the restic output to the systemd journal
-ExecStart=bash -c 'source $INSTALL_PREFIX/etc/restic/%I.env.sh && $INSTALL_PREFIX/bin/restic_check.sh | systemd-cat'
+# pipefail: so that redirecting stderr from the script to systemd-cat does not hide the failed command from OnFailure above.
+# `systemd-cat`: allows showing the restic output to the systemd journal
+ExecStart=/bin/bash -c 'set -o pipefail; source $INSTALL_PREFIX/etc/restic/%I.env.sh && $INSTALL_PREFIX/bin/restic_check.sh 2>&1 | systemd-cat'