Update CHANGELOG.md

* pre_backup list

* add $INSTALL_PREFIX

* make it executable

* remove pre_backup.txt

* fix quotes

Co-authored-by: Erik Westrup <erik.westrup@gmail.com>

---------

Co-authored-by: Erik Westrup <erik.westrup@gmail.com>

CHANGELOG.md

@@ -6,6 +6,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [7.4.0] - 2023-03-08
+### Added
+- Support saving hourly snapshots. [#98](https://github.com/erikw/restic-automatic-backup-scheduler/pull/98)
+- Support for pre backup script at /etc/restic/pre_backup.sh [107](https://github.com/erikw/restic-automatic-backup-scheduler/pull/107)
+
+### Fixed
+- Full path to `/bin/bash` in sytemd services. [#96](https://github.com/erikw/restic-automatic-backup-scheduler/issues/96)
+
+## [7.3.4] - 2022-04-29
+### Fixed
+- Backup stats notifications: fix issue where `restic snapshots --latest 2` will show more than two snapshots due to different backup paths used.
+
 ## [7.3.3] - 2022-04-14
 ### Fixed
 - Trying to fix broken Homebrew bottles due to GitHub API issues.

README.md

@@ -14,6 +14,7 @@
 [![Closed PRs](https://img.shields.io/github/issues-pr-closed/erikw/restic-automatic-backup-scheduler?color=success)](https://github.com/erikw/restic-automatic-backup-scheduler/pulls?q=is%3Apr+is%3Aclosed)
 [![License](https://img.shields.io/badge/license-BSD--3-blue)](LICENSE)
 [![OSS Lifecycle](https://img.shields.io/osslifecycle/erikw/restic-automatic-backup-scheduler)](https://github.com/Netflix/osstracker)
+[![SLOC](https://img.shields.io/tokei/lines/github/erikw/restic-automatic-backup-scheduler?logo=codefactor&logoColor=lightgrey)](#)
 <br>
 
 [![Contributors](https://img.shields.io/github/contributors/erikw/restic-automatic-backup-scheduler)](https://github.com/erikw/restic-automatic-backup-scheduler/graphs/contributors) including these top contributors:
@@ -35,6 +36,8 @@ The scope for this is not to be a full-fledged super solution that solves all th
 
 Nevertheless the project should work out of the box, be minimal but still open the doors for configuration and extensions by users.
 
+To use a different storage backend than B2, you should only need to tweak a few settings variables in the backup profile as well as some restic arguments inside `restic_backup.sh`.
+
 ## Notes
 * Tip: Navigate this document easily from the Section icon in the top left corner.
    ![README Sections](img/readme_sections.png)
@@ -109,9 +112,9 @@ Many Linux distributions nowadays use [Systemd](https://en.wikipedia.org/wiki/Sy
 ## Setup macOS LaunchAgent
 <img height="64" width="64" src="https://unpkg.com/simple-icons@v6/icons/apple.svg" />
 
-☝ **Note** The macOS setup here will assume an installation to `/usr/local`, as [custom](https://docs.brew.sh/FAQ#why-does-homebrew-say-sudo-is-bad) with Homebrew installations.
+☝ **Note** The macOS setup here will assume a Homebrew installation to the [recommended default location](https://docs.brew.sh/FAQ#why-should-i-install-homebrew-in-the-default-location). This is [`$HOMEBREW_PREFIX` (`brew --prefix`)](https://docs.brew.sh/Formula-Cookbook#variables-for-directory-locations) , which is `/usr/local` on Intel Macs and `/opt/homebrew` on [Apple Silicon](https://docs.brew.sh/FAQ#why-is-the-default-installation-prefix-opthomebrew-on-apple-silicon).
 
-[Launchd](https://www.launchd.info/) is the modern built-in service scheduler in macOS. It has support for running services as root (Daemon) or as a normal user (Agent). Here we set up an LauchAgent to be run as your normal user for starting regular backups.
+[Launchd](https://www.launchd.info/) is the modern built-in service scheduler in macOS. It has support for running services as root (Daemon) or as a normal user (Agent). Here we set up a LaunchAgent to be run as your normal user for starting regular backups.
 
 **TL;DR setup**
 1. [Create](#1-create-backblaze-b2-account-bucket-and-keys) B2 bucket + credentials
@@ -122,13 +125,13 @@ Many Linux distributions nowadays use [Systemd](https://en.wikipedia.org/wiki/Sy
    ```
    * Using `make`:
    ```console
-	$ make PREFIX=/usr/local install-launchagent
+	$ make PREFIX=$(brew --prefix) install-launchagent
    ```
-1. Fill out [configuration values](#2-configure-b2-credentials-locally) in `/usr/local/etc/restic`.
+1. Fill out [configuration values](#2-configure-b2-credentials-locally) in `$(brew --prefix)/etc/restic`.
 1. [Initialize](#3-initialize-remote-repo) the remote repo.
 	Source the profile to make all needed configuration available to `restic(1)`. All commands after this assumes the profile is sourced in the current shell.
    ```console
-	$ source /usr/local/etc/restic/default.env.sh
+	$ source $(brew --prefix)/etc/restic/default.env.sh
 	$ restic init
    ```
 1. Configure [how often](https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html#//apple_ref/doc/uid/10000172i-CH1-SW1) backups should be done. If needed, edit `OnCalendar` in
@@ -433,8 +436,8 @@ To have different backup jobs having e.g. different buckets, backup path of sche
 To create a different backup and use you can do:
 ```console
 # cp /etc/restic/default.env.sh /etc/restic/other.env.sh
-# vim /etc/restic/default.other.sh  # Set backup path, bucket etc.
-# source /etc/restic/default.other.sh
+# vim /etc/restic/other.env.sh  # Set backup path, bucket etc.
+# source /etc/restic/other.env.sh
 # restic_backup.sh
 ```
 
@@ -495,6 +498,7 @@ For a laptop, it can make sense to not do heavy backups when your on a metered c
 1. Edit `restic-backup@.service` and `restic-check@.service` to require the new service to be in success state:
    ```
    Requires=nm-unmetered-connection.service
+   After=nm-unmetered-connection.service
    ```
 1. Copy and paste the command below, it will install the following files and refresh systemd daemon:
 1. Put this file in `/etc/systemd/system/`:

ScheduledTask/install.ps1

@@ -14,6 +14,6 @@ $trigger =  New-ScheduledTaskTrigger -Daily -At 7pm
 Register-ScheduledTask -Action $action -Trigger $trigger -TaskName "restic_backup" -Description "Daily backup to B2 with restic."
 
 # Install restic_check.sh
-$action = New-ScheduledTaskAction -Execute "$(scoop prefix git)\git-bash.exe" -Argument '-l -c "source $INSTALL_PREFIX/etc/restic/default.env.sh && /INSTALL_PREFIX/bin/restic_check.sh"'
+$action = New-ScheduledTaskAction -Execute "$(scoop prefix git)\git-bash.exe" -Argument '-l -c "source $INSTALL_PREFIX/etc/restic/default.env.sh && $INSTALL_PREFIX/bin/restic_check.sh"'
 $trigger =  New-ScheduledTaskTrigger  -Weekly -WeeksInterval 4 -DaysOfWeek Sunday -At 8pm -RandomDelay 128
 Register-ScheduledTask -Action $action -Trigger $trigger -TaskName "restic_check" -Description "Check B2 backups with restic."

bin/restic_backup.sh

@@ -28,7 +28,7 @@ assert_envvars() {
 	local varnames=("$@")
 	for varname in "${varnames[@]}"; do
 		if [ -z ${!varname+x} ]; then
-			printf "%s must be set for this script to work.\n\nDid you forget to source a /etc/restic/*.env.sh profile in the current shell before executing this script?\n" "$varname" >&2
+			printf "%s must be set for this script to work.\n\nDid you forget to source a $INSTALL_PREFIX/etc/restic/*.env.sh profile in the current shell before executing this script?\n" "$varname" >&2
 			exit 1
 		fi
 	done
@@ -44,6 +44,9 @@ assert_envvars \
 IFS=':' read -ra backup_paths <<< "$RESTIC_BACKUP_PATHS"
 IFS=' ' read -ra extra_args <<< "$RESTIC_BACKUP_EXTRA_ARGS"
 
+# If you need to run some commands before performing the backup; create this file, put them there and make the file executable.
+PRE_SCRIPT="${INSTALL_PREFIX}/etc/restic/pre_backup.sh"
+test -x "$PRE_SCRIPT" && "$PRE_SCRIPT"
 
 # Set up exclude files: global + path-specific ones
 # NOTE that restic will fail the backup if not all listed --exclude-files exist. Thus we should only list them if they are really all available.
@@ -93,6 +96,7 @@ restic forget \
 	--option b2.connections="$B2_CONNECTIONS" \
 	--prune \
 	--group-by "paths,tags" \
+	--keep-hourly "$RESTIC_RETENTION_HOURS" \
 	--keep-daily "$RESTIC_RETENTION_DAYS" \
 	--keep-weekly "$RESTIC_RETENTION_WEEKS" \
 	--keep-monthly "$RESTIC_RETENTION_MONTHS" \
@@ -115,6 +119,7 @@ if [ "$RESTIC_NOTIFY_BACKUP_STATS" = true ]; then
 		latest_snapshot_diff=$(restic snapshots --tag "$RESTIC_BACKUP_TAG" --latest 2 --compact \
 			| grep -Ei "^[abcdef0-9]{8} " \
 			| awk '{print $1}' \
+			| tail -2 \
 			| tr '\n' ' ' \
 			| xargs restic diff)
         added=$(echo "$latest_snapshot_diff" | grep -i 'added:' | awk '{print $2 " " $3}')

bin/restic_check.sh

@@ -20,7 +20,7 @@ assert_envvars() {
 	local varnames=("$@")
 	for varname in "${varnames[@]}"; do
 		if [ -z ${!varname+x} ]; then
-			printf "%s must be set for this script to work.\n\nDid you forget to source a /etc/restic/*.env.sh profile in the current shell before executing this script?\n" "$varname" >&2
+			printf "%s must be set for this script to work.\n\nDid you forget to source a $INSTALL_PREFIX/etc/restic/*.env.sh profile in the current shell before executing this script?\n" "$varname" >&2
 			exit 1
 		fi
 	done

etc/restic/default.env.sh

@@ -32,6 +32,7 @@ export RESTIC_BACKUP_TAG=systemd.timer
 
 # Retention policy - How many backups to keep.
 # See https://restic.readthedocs.io/en/stable/060_forget.html?highlight=month#removing-snapshots-according-to-a-policy
+export RESTIC_RETENTION_HOURS=1
 export RESTIC_RETENTION_DAYS=14
 export RESTIC_RETENTION_WEEKS=16
 export RESTIC_RETENTION_MONTHS=18

usr/lib/systemd/system/restic-backup@.service

@@ -4,6 +4,7 @@ Description=Backup with restic to Backblaze B2
 #OnFailure=status-email-user@%n.service
 # Prevent backup on unmetered connection. Needs special setup. See README.md.
 #Requires=nm-unmetered-connection.service
+#After=nm-unmetered-connection.service
 
 [Service]
 Type=simple
@@ -13,4 +14,4 @@ Environment="HOME=/root"
 # pipefail: so that redirecting stderr from the script to systemd-cat does not hide the failed command from OnFailure above.
 # Random sleep (in seconds): in the case of multiple backup profiles. Many restic instances started at the same time could case high load or network bandwith usage.
 # `systemd-cat` allows showing the restic output to the systemd journal
-ExecStart=bash -c 'set -o pipefail; ps cax | grep -q restic && sleep $(shuf -i 0-300 -n 1); source $INSTALL_PREFIX/etc/restic/%I.env.sh && $INSTALL_PREFIX/bin/restic_backup.sh 2>&1 | systemd-cat'
+ExecStart=/bin/bash -c 'set -o pipefail; ps cax | grep -q restic && sleep $(shuf -i 0-300 -n 1); source $INSTALL_PREFIX/etc/restic/%I.env.sh && $INSTALL_PREFIX/bin/restic_backup.sh 2>&1 | systemd-cat'

usr/lib/systemd/system/restic-check@.service

@@ -5,10 +5,11 @@ Description=Check restic backup Backblaze B2 for errors
 Conflicts=restic-backup.service
 # Prevent backup on unmetered connection. Needs special setup. See README.md.
 #Requires=nm-unmetered-connection.service
+#After=nm-unmetered-connection.service
 
 [Service]
 Type=simple
 Nice=10
 # pipefail: so that redirecting stderr from the script to systemd-cat does not hide the failed command from OnFailure above.
 # `systemd-cat`: allows showing the restic output to the systemd journal
-ExecStart=bash -c 'set -o pipefail; source $INSTALL_PREFIX/etc/restic/%I.env.sh && $INSTALL_PREFIX/bin/restic_check.sh 2>&1 | systemd-cat'
+ExecStart=/bin/bash -c 'set -o pipefail; source $INSTALL_PREFIX/etc/restic/%I.env.sh && $INSTALL_PREFIX/bin/restic_check.sh 2>&1 | systemd-cat'