cjones/restic-windows-backup
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'release_1.7.1' into metered-check

Browse Source
This commit is contained in:
Kevin Woley
2025-01-27 15:09:09 -08:00
committed by GitHub
parent 09b9066049 c75bc74724
commit 21e68f4b42
8 changed files with 250 additions and 145 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
.gitignore vendored
View File

@@ -3,4 +3,5 @@ logs
restic.exe restic.exe
secrets.ps1 secrets.ps1
state.xml state.xml
testing testing
restic.exe.bak

27
CHANGELOG.md
View File

@@ -1,5 +1,32 @@
# Changelog # Changelog
## [1.7](https://github.com/kmwoley/restic-windows-backup/tree/1.7) (2025-01-25)
[Full Changelog](https://github.com/kmwoley/restic-windows-backup/compare/1.6...1.7)
*Upgrade Warning - Future Breaking Change*
This release deprecates the following `secrets.ps1` variables:
* `$PSEmailServer` is replaced by `$ResticEmailServer`
* `$ResticEmailConfig` is replaced by `$ResticEmailPort`
In the next release, `$ResticEmailServer` and `$ResticEmailPort` will be required. This release will still work if the deprecated variables are defined.
## What's Changed
* Update README.md by @enzo-g in https://github.com/kmwoley/restic-windows-backup/pull/58
* fix typo in readme by @jonas-hag in https://github.com/kmwoley/restic-windows-backup/pull/74
* fix typo by @Export33 in https://github.com/kmwoley/restic-windows-backup/pull/83
* Added a more detailed example for backup sources by @Export33 in https://github.com/kmwoley/restic-windows-backup/pull/84
* Limit snapshot pruning to the current host by @living180 in https://github.com/kmwoley/restic-windows-backup/pull/94
* Allow for unauthenticated SMTP. by @SeeJayEmm in https://github.com/kmwoley/restic-windows-backup/pull/81
* Replace deprecated Send-MailMessage with Send-MailKitMessage by @innovara in https://github.com/kmwoley/restic-windows-backup/pull/107
* Merge 2024.11 into Main by @kmwoley in https://github.com/kmwoley/restic-windows-backup/pull/110
## New Contributors
* @enzo-g made their first contribution in https://github.com/kmwoley/restic-windows-backup/pull/58
* @jonas-hag made their first contribution in https://github.com/kmwoley/restic-windows-backup/pull/74
* @Export33 made their first contribution in https://github.com/kmwoley/restic-windows-backup/pull/83
* @living180 made their first contribution in https://github.com/kmwoley/restic-windows-backup/pull/94
* @SeeJayEmm made their first contribution in https://github.com/kmwoley/restic-windows-backup/pull/81
* @innovara made their first contribution in https://github.com/kmwoley/restic-windows-backup/pull/107
## [1.6](https://github.com/kmwoley/restic-windows-backup/tree/1.6) (2023-01-14) ## [1.6](https://github.com/kmwoley/restic-windows-backup/tree/1.6) (2023-01-14)
[Full Changelog](https://github.com/kmwoley/restic-windows-backup/compare/1.5...1.6) [Full Changelog](https://github.com/kmwoley/restic-windows-backup/compare/1.5...1.6)

8
README.md
View File

@@ -29,15 +29,15 @@ Simplifies the process of installation and running daily backups.
1. Email sending configuration is also contained with this file. The scripts assume you want to get emails about the success/failure of each backup attempt. 1. Email sending configuration is also contained with this file. The scripts assume you want to get emails about the success/failure of each backup attempt.
1. Run `install.ps1` file 1. Run `install.ps1` file
1. From the elevated (Run as Administrator) Powershell window, run `.\install.ps1` 1. From the elevated (Run as Administrator) Powershell window, run `.\install.ps1`
1. This will initialize the repo, create your logfile directory, and create a scheduled task in Windows Task Scheduler to run the task daily. 1. This will initialize the repo, create your logfile directory, create a scheduled task in Windows Task Scheduler to run the task daily, and install Send-MailKitMessage module.
1. Add your `$BackupSources` to `config.ps1` 1. Add your `$BackupSources` to `config.ps1`
1. By default, all of `C:\` will be backed up. You can add multiple root drives to be backed up. And you can define only specific folders you would like backed up. 1. By default, all of `C:\` will be backed up. You can add multiple root drives to be backed up. And you can define only specific folders you would like backed up.
1. External, removalbe disk drives (i.e. USB hard drives) can be identified by their Volume Label, Serial Number, or Device Name. For example, if you have an external device with the Volume Label "MY BOOK", you can define a backup source as `$BackupSources["MY BOOK"]=@()`. I would recommend using the device serial number to identify external drives to backup, which you can find using the Powershell `get-disk` command. You may also want to set `$IgnoreMissingBackupSources=$true` to avoid seeing errors when the removable drive is not present. 1. External, removable disk drives (i.e. USB hard drives) can be identified by their Volume Label, Serial Number, or Device Name. For example, if you have an external device with the Volume Label "MY BOOK", you can define a backup source as `$BackupSources["MY BOOK"]=@()`. I would recommend using the device serial number to identify external drives to backup, which you can find using the Powershell `get-disk` command. You may also want to set `$IgnoreMissingBackupSources=$true` to avoid seeing errors when the removable drive is not present.
1. Add files/paths not to backup to `local.exclude` 1. Add files/paths not to backup to `local.exclude`
1. If you don't want to modify the included exclude file, you can add any files/paths you want to exclude from the backup to `local.exclude` 1. If you don't want to modify the included exclude file, you can add any files/paths you want to exclude from the backup to `local.exclude`
1. Add `restic.exe` to the Windows Defender / Virus & Threat Detection Exclude list 1. Add `C:\restic\restic.exe` to the Windows Defender / Virus & Threat Detection Exclude list
1. Backups on Windows are really slow if you don't set the Antivirus to ignore restic. 1. Backups on Windows are really slow if you don't set the Antivirus to ignore restic.
1. Navigate from the Start menu to: *Virus & threat protection > Manage Settings > Exclusions (Add or remove exclusions) > Add an exclusion (Process) > Process Name: "restic.exe"* 1. Navigate from the Start menu to: *Virus & threat protection > Manage Settings > Exclusions (Add or remove exclusions) > Add an exclusion (Process) > Process Name: "C:\restic\restic.exe"*
1. *(Recommended)* To a test backup triggered from Task Scheduler 1. *(Recommended)* To a test backup triggered from Task Scheduler
1. It's recommended to open Windows Task Scheduler and trigger the task to run manually to test your first backup. 1. It's recommended to open Windows Task Scheduler and trigger the task to run manually to test your first backup.
1. *Open Task Scheduler > Find "Restic Backup" > Right Click > Run* 1. *Open Task Scheduler > Find "Restic Backup" > Right Click > Run*

250
backup.ps1
View File

@@ -2,12 +2,12 @@
# Restic Windows Backup Script # Restic Windows Backup Script
# #
# =========== start configuration =========== # # =========== start configuration =========== #
# set restic configuration parmeters (destination, passwords, etc.) # load restic configuration parmeters (destination, passwords, etc.)
$SecretsScript = Join-Path $PSScriptRoot "secrets.ps1" $SecretsScript = Join-Path $PSScriptRoot "secrets.ps1"
# backup configuration variables # load backup configuration variables
$ConfigScript = Join-Path $PSScriptRoot "config.ps1" $ConfigScript = Join-Path $PSScriptRoot "config.ps1"
# =========== end configuration =========== # # =========== end configuration =========== #
@@ -19,8 +19,8 @@ $Script:ResticStateLastDeepMaintenance = $null
$Script:ResticStateMaintenanceCounter = $null $Script:ResticStateMaintenanceCounter = $null
$Script:ResticStateLastBackupSuccessful = $true $Script:ResticStateLastBackupSuccessful = $true
$Script:ResticStateLastMaintenanceSuccessful = $true $Script:ResticStateLastMaintenanceSuccessful = $true
# Returns all drive letters which exactly match the serial number, drive label, or drive name of # Returns all drive letters which exactly match the serial number, drive label, or drive name of
# the input parameter. Returns all drives if no input parameter is provided. # the input parameter. Returns all drives if no input parameter is provided.
# inspiration: https://stackoverflow.com/questions/31088930/combine-get-disk-info-and-logicaldisk-info-in-powershell # inspiration: https://stackoverflow.com/questions/31088930/combine-get-disk-info-and-logicaldisk-info-in-powershell
function Get-Drives { function Get-Drives {
@@ -35,16 +35,16 @@ function Get-Drives {
$drives = Get-CimAssociatedInstance -ResultClassName Win32_LogicalDisk -InputObject $partition $drives = Get-CimAssociatedInstance -ResultClassName Win32_LogicalDisk -InputObject $partition
foreach($drive in $drives) { foreach($drive in $drives) {
$volume = Get-Volume | $volume = Get-Volume |
Where-Object { $_.DriveLetter -eq $drive.DeviceID.Trim(":") } | Where-Object { $_.DriveLetter -eq $drive.DeviceID.Trim(":") } |
Select-Object -First 1 Select-Object -First 1
if(($diskMetadata.SerialNumber.trim() -eq $ID) -or if(($diskMetadata.SerialNumber.trim() -eq $ID) -or
($disk.Caption -eq $ID) -or ($disk.Caption -eq $ID) -or
($volume.FileSystemLabel -eq $ID) -or ($volume.FileSystemLabel -eq $ID) -or
($null -eq $ID)) { ($null -eq $ID)) {
[PSCustomObject] @{ [PSCustomObject] @{
DriveLetter = $drive.DeviceID DriveLetter = $drive.DeviceID
Number = $disk.Index Number = $disk.Index
@@ -67,24 +67,24 @@ function Get-Drives {
# restore backup state from disk # restore backup state from disk
function Get-BackupState { function Get-BackupState {
if(Test-Path $StateFile) { if(Test-Path $Script:StateFile) {
Import-Clixml $StateFile | ForEach-Object{ Set-Variable -Scope Script $_.Name $_.Value } Import-Clixml $Script:StateFile | ForEach-Object{ Set-Variable -Scope Script $_.Name $_.Value }
} }
} }
function Set-BackupState { function Set-BackupState {
Get-Variable ResticState* | Export-Clixml $StateFile Get-Variable ResticState* | Export-Clixml $Script:StateFile
} }
# unlock the repository if need be # unlock the repository if need be
function Invoke-Unlock { function Invoke-Unlock {
Param($SuccessLog, $ErrorLog) Param($SuccessLog, $ErrorLog)
$locks = & $ResticExe list locks --no-lock -q 3>&1 2>> $ErrorLog $locks = Invoke-Expression "$Script:ResticExe list locks --no-lock -q 3>&1 2>> $ErrorLog"
if($locks.Length -gt 0) { if($locks.Length -gt 0) {
# unlock the repository (assumes this machine is the only one that will ever use it) # unlock the repository (assumes this machine is the only one that will ever use it)
& $ResticExe unlock 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog Invoke-Expression "$Script:ResticExe unlock 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog"
"[[Unlock]] Repository was locked. Unlocking." | Tee-Object -Append $ErrorLog | Out-File -Append $SuccessLog "[[Unlock]] Repository was locked. Unlocking." | Tee-Object -Append $ErrorLog | Out-File -Append $SuccessLog
Start-Sleep 120 Start-Sleep 120
} }
} }
@@ -112,7 +112,7 @@ function Test-Maintenance {
} }
} }
else { else {
"[[Maintenance]] Running - no past maintenance history known." | Out-File -Append $SuccessLog "[[Maintenance]] Running - no past maintenance history known." | Out-File -Append $SuccessLog
return $true return $true
} }
} }
@@ -120,14 +120,14 @@ function Test-Maintenance {
# run maintenance on the backup set # run maintenance on the backup set
function Invoke-Maintenance { function Invoke-Maintenance {
Param($SuccessLog, $ErrorLog) Param($SuccessLog, $ErrorLog)
"[[Maintenance]] Start $(Get-Date)" | Out-File -Append $SuccessLog "[[Maintenance]] Start $(Get-Date)" | Tee-Object -Append $SuccessLog | Write-Host
$maintenance_success = $true $maintenance_success = $true
Start-Sleep 120 Start-Sleep 120
# forget snapshots based upon the retention policy # forget snapshots based upon the retention policy
"[[Maintenance]] Start forgetting..." | Out-File -Append $SuccessLog "[[Maintenance]] Start forgetting..." | Out-File -Append $SuccessLog
& $ResticExe forget $SnapshotRetentionPolicy 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog Invoke-Expression "$Script:ResticExe forget $SnapshotRetentionPolicy 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog"
if(-not $?) { if(-not $?) {
"[[Maintenance]] Forget operation completed with errors" | Tee-Object -Append $ErrorLog | Out-File -Append $SuccessLog "[[Maintenance]] Forget operation completed with errors" | Tee-Object -Append $ErrorLog | Out-File -Append $SuccessLog
$maintenance_success = $false $maintenance_success = $false
@@ -136,7 +136,7 @@ function Invoke-Maintenance {
# prune (remove) data from the backup step. Running this separate from `forget` because # prune (remove) data from the backup step. Running this separate from `forget` because
# `forget` only prunes when it detects removed snapshots upon invocation, not previously removed # `forget` only prunes when it detects removed snapshots upon invocation, not previously removed
"[[Maintenance]] Start pruning..." | Out-File -Append $SuccessLog "[[Maintenance]] Start pruning..." | Out-File -Append $SuccessLog
& $ResticExe prune $SnapshotPrunePolicy 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog Invoke-Expression "$Script:ResticExe prune $SnapshotPrunePolicy 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog"
if(-not $?) { if(-not $?) {
"[[Maintenance]] Prune operation completed with errors" | Tee-Object -Append $ErrorLog | Out-File -Append $SuccessLog "[[Maintenance]] Prune operation completed with errors" | Tee-Object -Append $ErrorLog | Out-File -Append $SuccessLog
$maintenance_success = $false $maintenance_success = $false
@@ -163,22 +163,26 @@ function Invoke-Maintenance {
$Script:ResticStateLastDeepMaintenance = Get-Date $Script:ResticStateLastDeepMaintenance = Get-Date
} }
& $ResticExe check @data_check 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog Invoke-Expression "$Script:ResticExe check $data_check 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog"
if(-not $?) { if(-not $?) {
"[[Maintenance]] Check completed with errors" | Tee-Object -Append $ErrorLog | Out-File -Append $SuccessLog "[[Maintenance]] Check completed with errors" | Tee-Object -Append $ErrorLog | Tee-Object -Append $SuccessLog | Write-Host
$maintenance_success = $false $maintenance_success = $false
} }
# check for updated restic version # Invoke restic self-update to check for a newer version
"[[Maintenance]] Checking for new version of restic..." | Out-File -Append $SuccessLog # This is enabled by default unless configuration disables self-update
& $ResticExe self-update 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog if ([String]::IsNullOrEmpty($SelfUpdateEnabled) -or ($SelfUpdateEnabled -eq $true)) {
if(-not $?) { # check for updated restic version
"[[Maintenance]] Self-update of restic.exe completed with errors" | Tee-Object -Append $ErrorLog | Out-File -Append $SuccessLog "[[Maintenance]] Checking for new version of restic..." | Out-File -Append $SuccessLog
$maintenance_success = $false Invoke-Expression "$Script:ResticExe self-update 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog"
if(-not $?) {
"[[Maintenance]] Self-update of restic.exe completed with errors" | Tee-Object -Append $ErrorLog | Out-File -Append $SuccessLog
$maintenance_success = $false
}
} }
"[[Maintenance]] End $(Get-Date)" | Out-File -Append $SuccessLog "[[Maintenance]] End $(Get-Date)" | Tee-Object -Append $SuccessLog | Write-Host
if($maintenance_success -eq $true) { if($maintenance_success -eq $true) {
$Script:ResticStateLastMaintenance = Get-Date $Script:ResticStateLastMaintenance = Get-Date
$Script:ResticStateMaintenanceCounter = 0 $Script:ResticStateMaintenanceCounter = 0
@@ -187,11 +191,11 @@ function Invoke-Maintenance {
return $maintenance_success return $maintenance_success
} }
# Run restic backup # Run restic backup
function Invoke-Backup { function Invoke-Backup {
Param($SuccessLog, $ErrorLog) Param($SuccessLog, $ErrorLog)
"[[Backup]] Start $(Get-Date)" | Out-File -Append $SuccessLog "[[Backup]] Start $(Get-Date)" | Tee-Object -Append $SuccessLog | Write-Host
$return_value = $true $return_value = $true
$starting_location = Get-Location $starting_location = Get-Location
ForEach ($item in $BackupSources.GetEnumerator()) { ForEach ($item in $BackupSources.GetEnumerator()) {
@@ -215,7 +219,7 @@ function Invoke-Backup {
$ignore_error = ($null -ne $IgnoreMissingBackupSources) -and $IgnoreMissingBackupSources $ignore_error = ($null -ne $IgnoreMissingBackupSources) -and $IgnoreMissingBackupSources
$warning_message = "[[Backup]] Warning - backup path $root_path not found." $warning_message = "[[Backup]] Warning - backup path $root_path not found."
if($ignore_error) { if($ignore_error) {
$warning_message | Out-File -Append $SuccessLog $warning_message | Out-File -Append $SuccessLog
} }
else { else {
$warning_message | Tee-Object -Append $SuccessLog | Out-File -Append $ErrorLog $warning_message | Tee-Object -Append $SuccessLog | Out-File -Append $ErrorLog
@@ -223,30 +227,30 @@ function Invoke-Backup {
} }
continue continue
} }
$root_path = Join-Path $drives[0].DriveLetter "" $root_path = Join-Path $drives[0].DriveLetter ""
# disable VSS / file system snapshot for external drives # disable VSS / file system snapshot for external drives
# TODO: would be best to just test for VSS compatibility on the drive, rather than assume it won't work # TODO: would be best to just test for VSS compatibility on the drive, rather than assume it won't work
$vss_option = $null $vss_option = $null
} }
"[[Backup]] Start $(Get-Date) [$tag]" | Out-File -Append $SuccessLog "[[Backup]] Start $(Get-Date) [$tag]" | Out-File -Append $SuccessLog
# build the list of folders to backup # build the list of folders to backup
$folder_list = New-Object System.Collections.Generic.List[System.Object] $folder_list = New-Object System.Collections.Generic.List[System.Object]
if ($item.Value.Count -eq 0) { if ($item.Value.Count -eq 0) {
# backup everything in the root if no folders are provided # backup everything in the root if no folders are provided
$folder_list.Add($root_path) $folder_list.Add("`"$root_path`"")
} }
else { else {
# Build the list of folders from settings # Build the list of folders from settings
ForEach ($path in $item.Value) { ForEach ($path in $item.Value) {
$p = '"{0}"' -f ((Join-Path $root_path $path) -replace "\\$") $p = '{0}' -f ((Join-Path $root_path $path) -replace "\\$")
if(Test-Path ($p -replace '"')) { if(Test-Path ($p -replace '"')) {
# add the folder if it exists # add the folder if it exists
$folder_list.Add($p) $folder_list.Add("`"$p`"")
} }
else { else {
# if the folder doesn't exist, log a warning/error # if the folder doesn't exist, log a warning/error
@@ -263,7 +267,7 @@ function Invoke-Backup {
} }
} }
if(-not $folder_list) { if(-not $folder_list) {
# there are no folders to backup # there are no folders to backup
$ignore_error = ($null -ne $IgnoreMissingBackupSources) -and $IgnoreMissingBackupSources $ignore_error = ($null -ne $IgnoreMissingBackupSources) -and $IgnoreMissingBackupSources
@@ -278,18 +282,18 @@ function Invoke-Backup {
} }
else { else {
# Launch Restic # Launch Restic
& $ResticExe backup $folder_list $vss_option --tag "$tag" --exclude-file=$WindowsExcludeFile --exclude-file=$LocalExcludeFile $AdditionalBackupParameters 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog Invoke-Expression "$Script:ResticExe backup $folder_list $vss_option --tag $tag --exclude-file=$WindowsExcludeFile --exclude-file=$LocalExcludeFile $AdditionalBackupParameters 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog"
if(-not $?) { if(-not $?) {
"[[Backup]] Completed with errors" | Tee-Object -Append $ErrorLog | Out-File -Append $SuccessLog "[[Backup]] Completed with errors" | Tee-Object -Append $ErrorLog | Tee-Object -Append $SuccessLog | Write-Host
$return_value = $false $return_value = $false
} }
} }
"[[Backup]] End $(Get-Date) [$tag]" | Out-File -Append $SuccessLog "[[Backup]] End $(Get-Date) [$tag]" | Out-File -Append $SuccessLog
} }
Set-Location $starting_location Set-Location $starting_location
"[[Backup]] End $(Get-Date)" | Out-File -Append $SuccessLog "[[Backup]] End $(Get-Date)" | Tee-Object -Append $SuccessLog | Write-Host
return $return_value return $return_value
} }
@@ -297,13 +301,36 @@ function Invoke-Backup {
function Send-Email { function Send-Email {
Param($SuccessLog, $ErrorLog, $Action) Param($SuccessLog, $ErrorLog, $Action)
Import-Module Send-MailKitMessage
# default the action string to "Backup" # default the action string to "Backup"
if($null -eq $Action) { if($null -eq $Action) {
$Action = "Backup" $Action = "Backup"
} }
$password = ConvertTo-SecureString $ResticEmailPassword -AsPlainText -Force # set email credentials if a username and passsword are provided in configuration
$credentials = New-Object System.Management.Automation.PSCredential ($ResticEmailUsername, $password) $credentials = @{}
if (-not [String]::IsNullOrEmpty($ResticEmailPassword) -and -not [String]::IsNullOrEmpty($ResticEmailUsername)) {
$password = ConvertTo-SecureString -String $ResticEmailPassword -AsPlainText -Force
$credentials = @{
"Credential" = [System.Management.Automation.PSCredential]::new($ResticEmailUsername, $password)
}
}
# Backwards compatability for $ResticEmailConfig port definition:
# $ResticEmailConfig is obsolete and should be replaced with $ResticEmailPort
if ($null -ne $ResticEmailConfig -and $ResticEmailConfig.ContainsKey('Port')) {
if ($null -eq $ResticEmailPort) {
$ResticEmailPort = $ResticEmailConfig['Port']
'[[Email]] Warning - $ResticEmailConfig is deprecated. Define $ResticEmailPort in secrets.ps1 instead.' | Tee-Object -Append $ErrorLog | Tee-Object -Append $SuccessLog | Write-Host
}
}
# Backwards compatibility for $PSEmailServer rename to $ResticEmailServer
if (($null -ne $PSEmailServer) -and ($null -eq $ResticEmailServer)) {
$ResticEmailServer = $PSEmailServer
'[[Email]] Warning - $PSEmailServer is deprecated. Define $ResticEmailServer in secrets.ps1 instead.' | Tee-Object -Append $ErrorLog | Tee-Object -Append $SuccessLog | Write-Host
}
$status = "SUCCESS" $status = "SUCCESS"
$past_failure = $false $past_failure = $false
@@ -311,7 +338,7 @@ function Send-Email {
if (($null -ne $SuccessLog) -and (Test-Path $SuccessLog) -and (Get-Item $SuccessLog).Length -gt 0) { if (($null -ne $SuccessLog) -and (Test-Path $SuccessLog) -and (Get-Item $SuccessLog).Length -gt 0) {
$body = $(Get-Content -Raw $SuccessLog) $body = $(Get-Content -Raw $SuccessLog)
# if previous run contained an error, send the success email confirming that the error has been resolved # if previous run contained an error, send the success email confirming that the error has been resolved
if($Action -eq "Backup") { if($Action -eq "Backup") {
$past_failure = -not $Script:ResticStateLastBackupSuccessful $past_failure = -not $Script:ResticStateLastBackupSuccessful
} }
@@ -320,24 +347,30 @@ function Send-Email {
} }
} }
else { else {
$body = "Crtical Error! Restic $Action log is empty or missing. Check log file path." $body = "Critical Error! Restic $Action log is empty or missing. Check log file path."
$status = "ERROR" $status = "ERROR"
} }
$attachments = @{}
$attachments = [System.Collections.Generic.List[string]]::new()
if (($null -ne $ErrorLog) -and (Test-Path $ErrorLog) -and (Get-Item $ErrorLog).Length -gt 0) { if (($null -ne $ErrorLog) -and (Test-Path $ErrorLog) -and (Get-Item $ErrorLog).Length -gt 0) {
$attachments = @{Attachments = $ErrorLog} $attachments.Add("$ErrorLog")
$status = "ERROR" $status = "ERROR"
} }
if((($status -eq "SUCCESS") -and ($SendEmailOnSuccess -ne $false)) -or ((($status -eq "ERROR") -or $past_failure) -and ($SendEmailOnError -ne $false))) { if((($status -eq "SUCCESS") -and ($SendEmailOnSuccess -ne $false)) -or ((($status -eq "ERROR") -or $past_failure) -and ($SendEmailOnError -ne $false))) {
$subject = "$env:COMPUTERNAME Restic $Action Report [$status]" $subject = "$env:COMPUTERNAME Restic $Action Report [$status]"
# create a temporary error log to log errors; can't write to the same file that Send-MailMessage is reading # create a temporary error log to log errors; can't write to the same file that Send-MailMessage is reading
$temp_error_log = $ErrorLog + "_temp" $temp_error_log = $ErrorLog + "_temp"
Send-MailMessage @ResticEmailConfig -From $ResticEmailFrom -To $ResticEmailTo -Credential $credentials -Subject $subject -Body $body @attachments 3>&1 2>> $temp_error_log $from = [MimeKit.MailboxAddress]$ResticEmailFrom;
$recipients = [MimeKit.InternetAddressList]::new();
$recipients.Add([MimeKit.InternetAddress]$ResticEmailTo);
Send-MailKitMessage -SMTPServer $ResticEmailServer -Port $ResticEmailPort -UseSecureConnectionIfAvailable @credentials -From $from -RecipientList $recipients -Subject $subject -TextBody $body -AttachmentList $attachments 3>&1 2>> $temp_error_log | Out-File -Append $SuccessLog
if(-not $?) { if(-not $?) {
"[[Email]] Sending email completed with errors" | Tee-Object -Append $temp_error_log | Out-File -Append $SuccessLog "[[Email]] Sending email completed with errors" | Tee-Object -Append $temp_error_log | Tee-Object -Append $SuccessLog | Write-Host
} }
# join error logs and remove the temporary # join error logs and remove the temporary
@@ -348,22 +381,22 @@ function Send-Email {
function Invoke-ConnectivityCheck { function Invoke-ConnectivityCheck {
Param($SuccessLog, $ErrorLog) Param($SuccessLog, $ErrorLog)
if($InternetTestAttempts -le 0) { if($InternetTestAttempts -le 0) {
"[[Internet]] Internet connectivity check disabled. Skipping." | Out-File -Append $SuccessLog "[[Internet]] Internet connectivity check disabled. Skipping." | Out-File -Append $SuccessLog
return $true return $true
} }
# skip the internet connectivity check for local repos # skip the internet connectivity check for local repos
if(Test-Path $env:RESTIC_REPOSITORY) { if(Test-Path $env:RESTIC_REPOSITORY) {
"[[Internet]] Local repository. Skipping internet connectivity check." | Out-File -Append $SuccessLog "[[Internet]] Local repository. Skipping internet connectivity check." | Out-File -Append $SuccessLog
return $true return $true
} }
$repository_host = '' $repository_host = ''
# use generic internet service for non-specific repo types (e.g. swift:, rclone:, etc. ) # use generic internet service for non-specific repo types (e.g. swift:, rclone:, etc. )
if(($env:RESTIC_REPOSITORY -match "^swift:") -or if(($env:RESTIC_REPOSITORY -match "^swift:") -or
($env:RESTIC_REPOSITORY -match "^rclone:")) { ($env:RESTIC_REPOSITORY -match "^rclone:")) {
$repository_host = "cloudflare.com" $repository_host = "cloudflare.com"
} }
@@ -409,6 +442,10 @@ function Invoke-ConnectivityCheck {
"[[Internet]] Waiting for connection to repository ($repository_host)... $sleep_count" | Out-File -Append $SuccessLog "[[Internet]] Waiting for connection to repository ($repository_host)... $sleep_count" | Out-File -Append $SuccessLog
Start-Sleep 30 Start-Sleep 30
} }
elseif((-not ([String]::IsNullOrEmpty($BackupOnMeteredNetwork)) -or $BackupOnMeteredNetwork)) -and Invoke-MeteredCheck)) {
"[[Internet]] Waiting for an unmetered network connection... $sleep_count" | Out-File -Append $SuccessLog
Start-Sleep 30
}
else { else {
return $true return $true
} }
@@ -435,16 +472,16 @@ function Invoke-HistoryCheck {
} }
$filter = "*$Action.err.txt".ToLower() $filter = "*$Action.err.txt".ToLower()
$logs = Get-ChildItem $LogPath -Filter $filter | ForEach-Object{$_.Length -gt 0} $logs = Get-ChildItem $Script:LogPath -Filter $filter | ForEach-Object{$_.Length -gt 0}
$logs_with_success = ($logs | Where-Object {($_ -eq $false)}).Count $logs_with_success = ($logs | Where-Object {($_ -eq $false)}).Count
if($logs.Count -gt 0) { if($logs.Count -gt 0) {
Write-Output "[[History]] $Action success rate: $logs_with_success / $($logs.Count) ($(($logs_with_success / $logs.Count).tostring("P")))" | Tee-Object -Append $SuccessLog "[[History]] $Action success rate: $logs_with_success / $($logs.Count) ($(($logs_with_success / $logs.Count).tostring("P")))" | Tee-Object -Append $SuccessLog | Write-Host
} }
} }
# main function # main function
function Invoke-Main { function Invoke-Main {
# check for elevation, required for creation of shadow copy (VSS) # check for elevation, required for creation of shadow copy (VSS)
if (-not (New-Object Security.Principal.WindowsPrincipal([Security.Principal.WindowsIdentity]::GetCurrent())).IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)) if (-not (New-Object Security.Principal.WindowsPrincipal([Security.Principal.WindowsIdentity]::GetCurrent())).IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator))
{ {
@@ -454,14 +491,22 @@ function Invoke-Main {
# initialize secrets # initialize secrets
. $SecretsScript . $SecretsScript
# initialize config # initialize config
. $ConfigScript . $ConfigScript
# apply global configuration
$Script:ResticExe = Join-Path $InstallPath $ExeName
if(-not [String]::IsNullOrEmpty($GlobalParameters)) {
$Script:ResticExe = "$Script:ResticExe $GlobalParameters"
}
$Script:StateFile = Join-Path $InstallPath "state.xml"
$Script:LogPath = Join-Path $InstallPath "logs"
Get-BackupState Get-BackupState
if(!(Test-Path $LogPath)) { if(!(Test-Path $Script:LogPath)) {
Write-Error "[[Backup]] Log file directory $LogPath does not exist. Exiting." Write-Error "[[Backup]] Log file directory $Script:LogPath does not exist. Exiting."
Send-Email Send-Email
exit 1 exit 1
} }
@@ -475,51 +520,44 @@ function Invoke-Main {
while ($attempt_count -gt 0) { while ($attempt_count -gt 0) {
# setup logfiles # setup logfiles
$timestamp = Get-Date -Format FileDateTime $timestamp = Get-Date -Format FileDateTime
$success_log = Join-Path $LogPath ($timestamp + ".backup.log.txt") $success_log = Join-Path $Script:LogPath ($timestamp + ".backup.log.txt")
$error_log = Join-Path $LogPath ($timestamp + ".backup.err.txt") $error_log = Join-Path $Script:LogPath ($timestamp + ".backup.err.txt")
$repository_available = Invoke-ConnectivityCheck $success_log $error_log $repository_available = Invoke-ConnectivityCheck $success_log $error_log
if($repository_available -eq $true) { if($repository_available -eq $true) {
$metered_network = Invoke-MeteredCheck Invoke-Unlock $success_log $error_log
if ($BackupOnMeteredNetwork -eq $true -or $metered_network -eq $false) { $backup_success = Invoke-Backup $success_log $error_log
Invoke-Unlock $success_log $error_log
$backup_success = Invoke-Backup $success_log $error_log
# NOTE: a previously locked repository will cause errors in the log; but backup would be 'successful' # NOTE: a previously locked repository will cause errors in the log; but backup would be 'successful'
# Removing this overly-aggressive test for backup success and relying upon Invoke-Backup to report on success/failure # Removing this overly-aggressive test for backup success and relying upon Invoke-Backup to report on success/failure
# $backup_success = ($backup_success -eq $true) -and (!(Test-Path $error_log) -or ((Get-Item $error_log).Length -eq 0)) # $backup_success = ($backup_success -eq $true) -and (!(Test-Path $error_log) -or ((Get-Item $error_log).Length -eq 0))
$total_attempts = $GlobalRetryAttempts - $attempt_count + 1 $total_attempts = $GlobalRetryAttempts - $attempt_count + 1
if($backup_success -eq $true) { if($backup_success -eq $true) {
# successful backup # successful backup
Write-Output "[[Backup]] Succeeded after $total_attempts attempt(s)" | Tee-Object -Append $success_log "[[Backup]] Succeeded after $total_attempts attempt(s)" | Tee-Object -Append $success_log | Write-Host
# test to see if maintenance is needed if the backup was successful # test to see if maintenance is needed if the backup was successful
$maintenance_needed = Test-Maintenance $success_log $error_log $maintenance_needed = Test-Maintenance $success_log $error_log
}
else {
Write-Output "[[Backup]] Ran with errors on attempt $total_attempts" | Tee-Object -Append $success_log | Tee-Object -Append $error_log
$error_count++
}
} }
else { else {
Write-Output "[[Backup]] Failed - currently on metered connection." | Tee-Object -Append $success_log | Tee-Object -Append $error_log "[[Backup]] Ran with errors on attempt $total_attempts" | Tee-Object -Append $success_log | Tee-Object -Append $error_log | Write-Host
$error_count++ $error_count++
} }
} }
else { else {
Write-Output "[[Backup]] Failed - cannot access repository." | Tee-Object -Append $success_log | Tee-Object -Append $error_log "[[Backup]] Failed - cannot access repository." | Tee-Object -Append $success_log | Tee-Object -Append $error_log | Write-Host
$error_count++ $error_count++
} }
$attempt_count-- $attempt_count--
# update logs prior to sending email # update logs prior to sending email
if($backup_success -eq $false) { if($backup_success -eq $false) {
if($attempt_count -gt 0) { if($attempt_count -gt 0) {
Write-Output "[[Backup]] Sleeping for 15 min and then retrying..." | Tee-Object -Append $success_log "[[Backup]] Sleeping for 15 min and then retrying..." | Tee-Object -Append $success_log | Write-Host
} }
else { else {
Write-Output "[[Backup]] Retry limit has been reached. No more attempts to backup will be made." | Tee-Object -Append $success_log "[[Backup]] Retry limit has been reached. No more attempts to backup will be made." | Tee-Object -Append $success_log | Write-Host
} }
} }
@@ -528,7 +566,7 @@ function Invoke-Main {
# update the state of the last backup success or failure # update the state of the last backup success or failure
$Script:ResticStateLastBackupSuccessful = $backup_success $Script:ResticStateLastBackupSuccessful = $backup_success
# Save state to file # Save state to file
Set-BackupState Set-BackupState
@@ -539,44 +577,44 @@ function Invoke-Main {
else { else {
break break
} }
} }
# only run maintenance if the backup was successful and maintenance is needed # only run maintenance if the backup was successful and maintenance is needed
$attempt_count = $GlobalRetryAttempts $attempt_count = $GlobalRetryAttempts
while (($maintenance_needed -eq $true) -and ($attempt_count -gt 0)) { while (($maintenance_needed -eq $true) -and ($attempt_count -gt 0)) {
# setup logfiles # setup logfiles
$timestamp = Get-Date -Format FileDateTime $timestamp = Get-Date -Format FileDateTime
$success_log = Join-Path $LogPath ($timestamp + ".maintenance.log.txt") $success_log = Join-Path $Script:LogPath ($timestamp + ".maintenance.log.txt")
$error_log = Join-Path $LogPath ($timestamp + ".maintenance.err.txt") $error_log = Join-Path $Script:LogPath ($timestamp + ".maintenance.err.txt")
$repository_available = Invoke-ConnectivityCheck $success_log $error_log $repository_available = Invoke-ConnectivityCheck $success_log $error_log
if($repository_available -eq $true) { if($repository_available -eq $true) {
$maintenance_success = Invoke-Maintenance $success_log $error_log $maintenance_success = Invoke-Maintenance $success_log $error_log
# $maintenance_success = ($maintenance_success -eq $true) -and (!(Test-Path $error_log) -or ((Get-Item $error_log).Length -eq 0)) # $maintenance_success = ($maintenance_success -eq $true) -and (!(Test-Path $error_log) -or ((Get-Item $error_log).Length -eq 0))
$total_attempts = $GlobalRetryAttempts - $attempt_count + 1 $total_attempts = $GlobalRetryAttempts - $attempt_count + 1
if($maintenance_success -eq $true) { if($maintenance_success -eq $true) {
Write-Output "[[Maintenance]] Succeeded after $total_attempts attempt(s)" | Tee-Object -Append $success_log "[[Maintenance]] Succeeded after $total_attempts attempt(s)" | Tee-Object -Append $success_log | Write-Host
} }
else { else {
Write-Output "[[Maintenance]] Ran with errors on attempt $total_attempts" | Tee-Object -Append $success_log | Tee-Object -Append $error_log "[[Maintenance]] Ran with errors on attempt $total_attempts" | Tee-Object -Append $success_log | Tee-Object -Append $error_log | Write-Host
$error_count++ $error_count++
} }
} }
else { else {
Write-Output "[[Maintenance]] Failed - cannot access repository." | Tee-Object -Append $success_log | Tee-Object -Append $error_log "[[Maintenance]] Failed - cannot access repository." | Tee-Object -Append $success_log | Tee-Object -Append $error_log | Write-Host
$error_count++ $error_count++
} }
$attempt_count-- $attempt_count--
# update logs prior to sending email # update logs prior to sending email
if($maintenance_success -eq $false) { if($maintenance_success -eq $false) {
if($attempt_count -gt 0) { if($attempt_count -gt 0) {
Write-Output "[[Maintenance]] Sleeping for 15 min and then retrying..." | Tee-Object -Append $success_log "[[Maintenance]] Sleeping for 15 min and then retrying..." | Tee-Object -Append $success_log | Write-Host
} }
else { else {
Write-Output "[[Maintenance]] Retry limit has been reached. No more attempts to run maintenance will be made." | Tee-Object -Append $success_log "[[Maintenance]] Retry limit has been reached. No more attempts to run maintenance will be made." | Tee-Object -Append $success_log | Write-Host
} }
} }
@@ -585,7 +623,7 @@ function Invoke-Main {
# update the state of the last maintenance success or failure # update the state of the last maintenance success or failure
$Script:ResticStateLastMaintenanceSuccessful = $maintenance_success $Script:ResticStateLastMaintenanceSuccessful = $maintenance_success
# Save state to file # Save state to file
Set-BackupState Set-BackupState
@@ -596,13 +634,13 @@ function Invoke-Main {
else { else {
break break
} }
} }
# Save state to file # Save state to file
Set-BackupState Set-BackupState
# cleanup older log files # cleanup older log files
Get-ChildItem $LogPath | Where-Object {$_.CreationTime -lt $(Get-Date).AddDays(-$LogRetentionDays)} | Remove-Item Get-ChildItem $Script:LogPath | Where-Object {$_.CreationTime -lt $(Get-Date).AddDays(-$LogRetentionDays)} | Remove-Item
exit $error_count exit $error_count
} }

49
config.ps1
View File

@@ -1,38 +1,43 @@
# backup configuration # general configuration
$ExeName = "restic.exe"
$InstallPath = "C:\restic" $InstallPath = "C:\restic"
$ResticExe = Join-Path $InstallPath $ExeName $ExeName = "restic.exe"
$StateFile = Join-Path $InstallPath "state.xml" $GlobalParameters = @()
$WindowsExcludeFile = Join-Path $InstallPath "windows.exclude"
$LocalExcludeFile = Join-Path $InstallPath "local.exclude"
$LogPath = Join-Path $InstallPath "logs"
$LogRetentionDays = 30 $LogRetentionDays = 30
$BackupOnMeteredNetwork = $false $BackupOnMeteredNetwork = $false
$InternetTestAttempts = 10 $InternetTestAttempts = 10
$GlobalRetryAttempts = 4 $GlobalRetryAttempts = 4
$IgnoreMissingBackupSources = $false
$AdditionalBackupParameters = @("--exclude-if-present", ".nobackup")
# maintenance configuration
$SnapshotMaintenanceEnabled = $true
$SnapshotRetentionPolicy = @("--group-by", "host,tags", "--keep-daily", "30", "--keep-weekly", "52", "--keep-monthly", "24", "--keep-yearly", "10")
$SnapshotPrunePolicy = @("--max-unused", "1%")
$SnapshotMaintenanceInterval = 7
$SnapshotMaintenanceDays = 30
$SnapshotDeepMaintenanceDays = 90
# email configuration # email configuration
$SendEmailOnSuccess = $false $SendEmailOnSuccess = $false
$SendEmailOnError = $true $SendEmailOnError = $true
# backup configuration
$WindowsExcludeFile = Join-Path $InstallPath "windows.exclude"
$LocalExcludeFile = Join-Path $InstallPath "local.exclude"
$IgnoreMissingBackupSources = $false
$AdditionalBackupParameters = @("--exclude-if-present", ".nobackup", "--no-scan")
# Paths to backup # Paths to backup
$BackupSources = @{} $BackupSources = @{}
$BackupSources["C:\"] = @( $BackupSources["C:\"] = @(
# 'Users' # "Users\Example\Desktop\Source1",
# "Users\Example\Desktop\Source2"
) )
#$BackupSources["D:\"] = @( # $BackupSources["D:\"] = @(
# 'Software' # "Example\Source3",
#) # "Example\Source4"
# )
#$BackupSources["DRIVE_LABEL_NAME_OR_SERIAL_NUMBER"] = @( #$BackupSources["DRIVE_LABEL_NAME_OR_SERIAL_NUMBER"] = @(
# 'FolderName' # "Example\FolderName"
#) #)
# maintenance configuration
$SnapshotMaintenanceEnabled = $true
$SnapshotRetentionPolicy = @("--host", $env:COMPUTERNAME, "--group-by", "host,tags", "--keep-daily", "30", "--keep-weekly", "52", "--keep-monthly", "24", "--keep-yearly", "10")
$SnapshotPrunePolicy = @("--max-unused", "1%")
$SnapshotMaintenanceInterval = 7
$SnapshotMaintenanceDays = 30
$SnapshotDeepMaintenanceDays = 90
# restic.exe self update configuration
$SelfUpdateEnabled = $true

49
install.ps1
View File

@@ -1,14 +1,35 @@
. .\config.ps1 #
. .\secrets.ps1 # Restic Windows Backup - Installation Script
#
# =========== start configuration =========== #
# load restic configuration parmeters (destination, passwords, etc.)
$SecretsScript = Join-Path $PSScriptRoot "secrets.ps1"
# load backup configuration variables
$ConfigScript = Join-Path $PSScriptRoot "config.ps1"
# initialize secrets
. $SecretsScript
# initialize config
. $ConfigScript
# apply global configuration
$ResticExe = Join-Path $InstallPath $ExeName
$LogPath = Join-Path $InstallPath "logs"
# =========== end configuration =========== #
# download restic # download restic
if(-not (Test-Path $ResticExe)) { if(-not (Test-Path $ResticExe)) {
$url = $null $url = $null
if([Environment]::Is64BitOperatingSystem){ if([Environment]::Is64BitOperatingSystem){
$url = "https://github.com/restic/restic/releases/download/v0.15.0/restic_0.15.0_windows_amd64.zip" $url = "https://github.com/restic/restic/releases/download/v0.17.3/restic_0.17.3_windows_amd64.zip"
} }
else { else {
$url = "https://github.com/restic/restic/releases/download/v0.15.0/restic_0.15.0_windows_386.zip" $url = "https://github.com/restic/restic/releases/download/v0.17.3/restic_0.17.3_windows_386.zip"
} }
$output = Join-Path $InstallPath "restic.zip" $output = Join-Path $InstallPath "restic.zip"
Invoke-WebRequest -Uri $url -OutFile $output Invoke-WebRequest -Uri $url -OutFile $output
@@ -17,13 +38,21 @@ if(-not (Test-Path $ResticExe)) {
Get-ChildItem *.exe | Rename-Item -NewName $ExeName Get-ChildItem *.exe | Rename-Item -NewName $ExeName
} }
# Apply global paramters to $ResticExe, after the $ResticExe has been downloaded/confirmed to exist
if(-not [String]::IsNullOrEmpty($GlobalParameters)) {
$ResticExe = "$ResticExe $GlobalParameters"
}
# Invoke restic self-update to check for a newer version # Invoke restic self-update to check for a newer version
& $ResticExe self-update # This is enabled by default unless configuration disables self-update
if ([String]::IsNullOrEmpty($SelfUpdateEnabled) -or ($SelfUpdateEnabled -eq $true)) {
Invoke-Expression "$ResticExe self-update"
}
# Create log directory if it doesn't exit # Create log directory if it doesn't exit
if(-not (Test-Path $LogPath)) { if(-not (Test-Path $LogPath)) {
New-Item -ItemType Directory -Force -Path $LogPath | Out-Null New-Item -ItemType Directory -Force -Path $LogPath | Out-Null
Write-Output "[[Init]] Repository successfully initialized." Write-Output "[[Init]] Created log directory: $LogPath"
} }
# Create the local exclude file # Create the local exclude file
@@ -32,7 +61,7 @@ if(-not (Test-Path $LocalExcludeFile)) {
} }
# Initialize the restic repository # Initialize the restic repository
& $ResticExe --verbose init Invoke-Expression "$ResticExe --verbose init"
if($?) { if($?) {
Write-Output "[[Init]] Repository successfully initialized." Write-Output "[[Init]] Repository successfully initialized."
} }
@@ -60,4 +89,8 @@ else {
Write-Warning "[[Scheduler]] Backup task not scheduled: there is already a task with the name '$backup_task_name'." Write-Warning "[[Scheduler]] Backup task not scheduled: there is already a task with the name '$backup_task_name'."
} }
# Install NuGet and Send-MailKitMessage module (by force)
if ($PSVersionTable.PSVersion.Major -eq 5) {
Install-PackageProvider -Name NuGet -Force
}
Install-Module Send-MailKitMessage -Repository PSGallery -Scope AllUsers -Force

8
secrets_template.ps1
View File

@@ -9,9 +9,9 @@ $Env:RESTIC_REPOSITORY='<REPO URL>'
$Env:RESTIC_PASSWORD='<BACKUP PASSWORD>' $Env:RESTIC_PASSWORD='<BACKUP PASSWORD>'
# email configuration # email configuration
$PSEmailServer='<SMTP SERVER>' $ResticEmailServer='<SMTP SERVER>'
$ResticEmailConfig=@{UseSsl=$true; Port="587"} $ResticEmailPort='<SMTP PORT NUMBER, i.e. 25 or 587>'
$ResticEmailTo='<DESTINATION EMAIL ADDRESS>' $ResticEmailTo='<DESTINATION EMAIL ADDRESS>'
$ResticEmailFrom='<FROM EMAIL ADDRESS>' $ResticEmailFrom='<FROM EMAIL ADDRESS>'
$ResticEmailUsername='<EMAIL LOGIN USERNAME>' $ResticEmailUsername='<EMAIL LOGIN USERNAME OR EMPTY FOR NO USERNAME>'
$ResticEmailPassword='<EMAIL PASSWORD>' $ResticEmailPassword='<EMAIL PASSWORD OR EMPTY FOR NO PASSWORD>'

1
windows.exclude
View File

@@ -35,6 +35,7 @@ Dropbox
AppData\Local\Google\Drive AppData\Local\Google\Drive
Google Drive\.tmp.drivedownload Google Drive\.tmp.drivedownload
C:\OneDriveTemp C:\OneDriveTemp
Users\**\Nextcloud
# browsers # browsers
Google\Chrome Google\Chrome