Merge pull request #114 from kmwoley/release_1.7.1

Release 1.7.1

.gitignore

@@ -3,4 +3,5 @@ logs
 restic.exe
 secrets.ps1
 state.xml
-testing
+testing
+restic.exe.bak

CHANGELOG.md

@@ -1,5 +1,32 @@
 # Changelog
 
+## [1.7](https://github.com/kmwoley/restic-windows-backup/tree/1.7) (2025-01-25)
+[Full Changelog](https://github.com/kmwoley/restic-windows-backup/compare/1.6...1.7)
+
+*Upgrade Warning - Future Breaking Change*
+This release deprecates the following `secrets.ps1` variables:
+* `$PSEmailServer` is replaced by `$ResticEmailServer`
+* `$ResticEmailConfig` is replaced by `$ResticEmailPort`
+In the next release, `$ResticEmailServer` and `$ResticEmailPort` will be required. This release will still work if the deprecated variables are defined.
+
+## What's Changed
+* Update README.md by @enzo-g in https://github.com/kmwoley/restic-windows-backup/pull/58
+* fix typo in readme by @jonas-hag in https://github.com/kmwoley/restic-windows-backup/pull/74
+* fix typo by @Export33 in https://github.com/kmwoley/restic-windows-backup/pull/83
+* Added a more detailed example for backup sources by @Export33 in https://github.com/kmwoley/restic-windows-backup/pull/84
+* Limit snapshot pruning to the current host by @living180 in https://github.com/kmwoley/restic-windows-backup/pull/94
+* Allow for unauthenticated SMTP. by @SeeJayEmm in https://github.com/kmwoley/restic-windows-backup/pull/81
+* Replace deprecated Send-MailMessage with Send-MailKitMessage by @innovara in https://github.com/kmwoley/restic-windows-backup/pull/107
+* Merge 2024.11 into Main by @kmwoley in https://github.com/kmwoley/restic-windows-backup/pull/110
+
+## New Contributors
+* @enzo-g made their first contribution in https://github.com/kmwoley/restic-windows-backup/pull/58
+* @jonas-hag made their first contribution in https://github.com/kmwoley/restic-windows-backup/pull/74
+* @Export33 made their first contribution in https://github.com/kmwoley/restic-windows-backup/pull/83
+* @living180 made their first contribution in https://github.com/kmwoley/restic-windows-backup/pull/94
+* @SeeJayEmm made their first contribution in https://github.com/kmwoley/restic-windows-backup/pull/81
+* @innovara made their first contribution in https://github.com/kmwoley/restic-windows-backup/pull/107
+
 ## [1.6](https://github.com/kmwoley/restic-windows-backup/tree/1.6) (2023-01-14)
 [Full Changelog](https://github.com/kmwoley/restic-windows-backup/compare/1.5...1.6)
 

README.md

@@ -29,15 +29,15 @@ Simplifies the process of installation and running daily backups.
    1. Email sending configuration is also contained with this file. The scripts assume you want to get emails about the success/failure of each backup attempt.
 1. Run `install.ps1` file
    1. From the elevated (Run as Administrator) Powershell window, run `.\install.ps1`
-   1. This will initialize the repro, create your logfile directory, and create a scheduled task in Windows Task Scheduler to run the task daily.
+   1. This will initialize the repo, create your logfile directory, create a scheduled task in Windows Task Scheduler to run the task daily, and install Send-MailKitMessage module.
 1. Add your `$BackupSources` to `config.ps1`
    1. By default, all of `C:\` will be backed up. You can add multiple root drives to be backed up. And you can define only specific folders you would like backed up.
-   1. External, removalbe disk drives (i.e. USB hard drives) can be identified by their Volume Label, Serial Number, or Device Name. For example, if you have an external device with the Volume Label "MY BOOK", you can define a backup source as `$BackupSources["MY BOOK"]=@()`. I would recommend using the device serial number to identify external drives to backup, which you can find using the Powershell `get-disk` command. You may also want to set `$IgnoreMissingBackupSources=$true` to avoid seeing errors when the removable drive is not present.
+   1. External, removable disk drives (i.e. USB hard drives) can be identified by their Volume Label, Serial Number, or Device Name. For example, if you have an external device with the Volume Label "MY BOOK", you can define a backup source as `$BackupSources["MY BOOK"]=@()`. I would recommend using the device serial number to identify external drives to backup, which you can find using the Powershell `get-disk` command. You may also want to set `$IgnoreMissingBackupSources=$true` to avoid seeing errors when the removable drive is not present.
 1. Add files/paths not to backup to `local.exclude`
    1. If you don't want to modify the included exclude file, you can add any files/paths you want to exclude from the backup to `local.exclude`
-1. Add `restic.exe` to the Windows Defender / Virus & Threat Detection Exclude list
+1. Add `C:\restic\restic.exe` to the Windows Defender / Virus & Threat Detection Exclude list
    1. Backups on Windows are really slow if you don't set the Antivirus to ignore restic.
-   1. Navigate from the Start menu to: *Virus & threat protection > Manage Settings > Exclusions (Add or remove exclusions) > Add an exclusion (Process) > Process Name: "restic.exe"*
+   1. Navigate from the Start menu to: *Virus & threat protection > Manage Settings > Exclusions (Add or remove exclusions) > Add an exclusion (Process) > Process Name: "C:\restic\restic.exe"*
 1. *(Recommended)* To a test backup triggered from Task Scheduler
    1. It's recommended to open Windows Task Scheduler and trigger the task to run manually to test your first backup.
       1. *Open Task Scheduler > Find "Restic Backup" > Right Click > Run*

backup.ps1

@@ -2,12 +2,12 @@
 # Restic Windows Backup Script
 #
 
-# =========== start configuration =========== # 
+# =========== start configuration =========== #
 
-# set restic configuration parmeters (destination, passwords, etc.)
+# load restic configuration parameters (destination, passwords, etc.)
 $SecretsScript = Join-Path $PSScriptRoot "secrets.ps1"
 
-# backup configuration variables
+# load backup configuration variables
 $ConfigScript = Join-Path $PSScriptRoot "config.ps1"
 
 # =========== end configuration =========== #
@@ -19,8 +19,8 @@ $Script:ResticStateLastDeepMaintenance = $null
 $Script:ResticStateMaintenanceCounter = $null
 $Script:ResticStateLastBackupSuccessful = $true
 $Script:ResticStateLastMaintenanceSuccessful = $true
- 
+
-# Returns all drive letters which exactly match the serial number, drive label, or drive name of 
+# Returns all drive letters which exactly match the serial number, drive label, or drive name of
 # the input parameter. Returns all drives if no input parameter is provided.
 # inspiration: https://stackoverflow.com/questions/31088930/combine-get-disk-info-and-logicaldisk-info-in-powershell
 function Get-Drives {
@@ -35,16 +35,16 @@ function Get-Drives {
             $drives = Get-CimAssociatedInstance -ResultClassName Win32_LogicalDisk -InputObject $partition
 
             foreach($drive in $drives) {
-                    
+
                 $volume = Get-Volume |
                           Where-Object { $_.DriveLetter -eq $drive.DeviceID.Trim(":") } |
                           Select-Object -First 1
 
-                if(($diskMetadata.SerialNumber.trim() -eq $ID) -or 
+                if(($diskMetadata.SerialNumber.trim() -eq $ID) -or
                     ($disk.Caption -eq $ID) -or
                     ($volume.FileSystemLabel  -eq $ID) -or
                     ($null -eq $ID)) {
-    
+
                     [PSCustomObject] @{
                         DriveLetter   = $drive.DeviceID
                         Number        = $disk.Index
@@ -67,24 +67,24 @@ function Get-Drives {
 
 # restore backup state from disk
 function Get-BackupState {
-    if(Test-Path $StateFile) {
+    if(Test-Path $Script:StateFile) {
-        Import-Clixml $StateFile | ForEach-Object{ Set-Variable -Scope Script $_.Name $_.Value }
+        Import-Clixml $Script:StateFile | ForEach-Object{ Set-Variable -Scope Script $_.Name $_.Value }
     }
 }
 function Set-BackupState {
-    Get-Variable ResticState* | Export-Clixml $StateFile
+    Get-Variable ResticState* | Export-Clixml $Script:StateFile
 }
 
 # unlock the repository if need be
 function Invoke-Unlock {
     Param($SuccessLog, $ErrorLog)
 
-    $locks = & $ResticExe list locks --no-lock -q 3>&1 2>> $ErrorLog
+    $locks = Invoke-Expression "$Script:ResticExe list locks --no-lock -q 3>&1 2>> $ErrorLog"
     if($locks.Length -gt 0) {
         # unlock the repository (assumes this machine is the only one that will ever use it)
-        & $ResticExe unlock 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog
+        Invoke-Expression "$Script:ResticExe unlock 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog"
         "[[Unlock]] Repository was locked. Unlocking." | Tee-Object -Append $ErrorLog | Out-File -Append $SuccessLog
-        Start-Sleep 120 
+        Start-Sleep 120
     }
 }
 
@@ -112,7 +112,7 @@ function Test-Maintenance {
         }
     }
     else {
-        "[[Maintenance]] Running - no past maintenance history known." | Out-File -Append $SuccessLog        
+        "[[Maintenance]] Running - no past maintenance history known." | Out-File -Append $SuccessLog
         return $true
     }
 }
@@ -120,14 +120,14 @@ function Test-Maintenance {
 # run maintenance on the backup set
 function Invoke-Maintenance {
     Param($SuccessLog, $ErrorLog)
-    
+
-    "[[Maintenance]] Start $(Get-Date)" | Out-File -Append $SuccessLog
+    "[[Maintenance]] Start $(Get-Date)" | Tee-Object -Append $SuccessLog | Write-Host
     $maintenance_success = $true
     Start-Sleep 120
 
     # forget snapshots based upon the retention policy
     "[[Maintenance]] Start forgetting..." | Out-File -Append $SuccessLog
-    & $ResticExe forget $SnapshotRetentionPolicy 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog
+    Invoke-Expression "$Script:ResticExe forget $SnapshotRetentionPolicy 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog"
     if(-not $?) {
         "[[Maintenance]] Forget operation completed with errors" | Tee-Object -Append $ErrorLog | Out-File -Append $SuccessLog
         $maintenance_success = $false
@@ -136,7 +136,7 @@ function Invoke-Maintenance {
     # prune (remove) data from the backup step. Running this separate from `forget` because
     #   `forget` only prunes when it detects removed snapshots upon invocation, not previously removed
     "[[Maintenance]] Start pruning..." | Out-File -Append $SuccessLog
-    & $ResticExe prune $SnapshotPrunePolicy 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog
+    Invoke-Expression "$Script:ResticExe prune $SnapshotPrunePolicy 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog"
     if(-not $?) {
         "[[Maintenance]] Prune operation completed with errors" | Tee-Object -Append $ErrorLog | Out-File -Append $SuccessLog
         $maintenance_success = $false
@@ -163,22 +163,26 @@ function Invoke-Maintenance {
         $Script:ResticStateLastDeepMaintenance = Get-Date
     }
 
-    & $ResticExe check @data_check 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog
+    Invoke-Expression "$Script:ResticExe check $data_check 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog"
     if(-not $?) {
-        "[[Maintenance]] Check completed with errors" | Tee-Object -Append $ErrorLog | Out-File -Append $SuccessLog
+        "[[Maintenance]] Check completed with errors" | Tee-Object -Append $ErrorLog | Tee-Object -Append $SuccessLog | Write-Host
         $maintenance_success = $false
     }
 
-    # check for updated restic version
+    # Invoke restic self-update to check for a newer version
-    "[[Maintenance]] Checking for new version of restic..." | Out-File -Append $SuccessLog
+    # This is enabled by default unless configuration disables self-update
-    & $ResticExe self-update 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog
+    if ([String]::IsNullOrEmpty($SelfUpdateEnabled) -or ($SelfUpdateEnabled -eq $true)) {
-    if(-not $?) {
+        # check for updated restic version
-        "[[Maintenance]] Self-update of restic.exe completed with errors" | Tee-Object -Append $ErrorLog | Out-File -Append $SuccessLog
+        "[[Maintenance]] Checking for new version of restic..." | Out-File -Append $SuccessLog
-        $maintenance_success = $false
+        Invoke-Expression "$Script:ResticExe self-update 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog"
+        if(-not $?) {
+            "[[Maintenance]] Self-update of restic.exe completed with errors" | Tee-Object -Append $ErrorLog | Out-File -Append $SuccessLog
+            $maintenance_success = $false
+        }
     }
 
-    "[[Maintenance]] End $(Get-Date)" | Out-File -Append $SuccessLog
+    "[[Maintenance]] End $(Get-Date)" | Tee-Object -Append $SuccessLog | Write-Host
-    
+
     if($maintenance_success -eq $true) {
         $Script:ResticStateLastMaintenance = Get-Date
         $Script:ResticStateMaintenanceCounter = 0
@@ -187,11 +191,11 @@ function Invoke-Maintenance {
     return $maintenance_success
 }
 
-# Run restic backup 
+# Run restic backup
 function Invoke-Backup {
     Param($SuccessLog, $ErrorLog)
 
-    "[[Backup]] Start $(Get-Date)" | Out-File -Append $SuccessLog
+    "[[Backup]] Start $(Get-Date)" | Tee-Object -Append $SuccessLog | Write-Host
     $return_value = $true
     $starting_location = Get-Location
     ForEach ($item in $BackupSources.GetEnumerator()) {
@@ -215,7 +219,7 @@ function Invoke-Backup {
                 $ignore_error = ($null -ne $IgnoreMissingBackupSources) -and $IgnoreMissingBackupSources
                 $warning_message = "[[Backup]] Warning - backup path $root_path not found."
                 if($ignore_error) {
-                    $warning_message | Out-File -Append $SuccessLog                    
+                    $warning_message | Out-File -Append $SuccessLog
                 }
                 else {
                     $warning_message | Tee-Object -Append $SuccessLog | Out-File -Append $ErrorLog
@@ -223,30 +227,30 @@ function Invoke-Backup {
                 }
                 continue
             }
-            
+
             $root_path = Join-Path $drives[0].DriveLetter ""
-            
+
             # disable VSS / file system snapshot for external drives
             # TODO: would be best to just test for VSS compatibility on the drive, rather than assume it won't work
             $vss_option = $null
         }
 
         "[[Backup]] Start $(Get-Date) [$tag]" | Out-File -Append $SuccessLog
-        
+
         # build the list of folders to backup
         $folder_list = New-Object System.Collections.Generic.List[System.Object]
         if ($item.Value.Count -eq 0) {
             # backup everything in the root if no folders are provided
-            $folder_list.Add($root_path)
+            $folder_list.Add("`"$root_path`"")
         }
         else {
             # Build the list of folders from settings
             ForEach ($path in $item.Value) {
-                $p = '"{0}"' -f ((Join-Path $root_path $path) -replace "\\$")
+                $p = '{0}' -f ((Join-Path $root_path $path) -replace "\\$")
-                
+
                 if(Test-Path ($p -replace '"')) {
                     # add the folder if it exists
-                    $folder_list.Add($p)
+                    $folder_list.Add("`"$p`"")
                 }
                 else {
                     # if the folder doesn't exist, log a warning/error
@@ -263,7 +267,7 @@ function Invoke-Backup {
             }
 
         }
-        
+
         if(-not $folder_list) {
             # there are no folders to backup
             $ignore_error = ($null -ne $IgnoreMissingBackupSources) -and $IgnoreMissingBackupSources
@@ -278,18 +282,18 @@ function Invoke-Backup {
         }
         else {
             # Launch Restic
-            & $ResticExe backup $folder_list $vss_option --tag "$tag" --exclude-file=$WindowsExcludeFile --exclude-file=$LocalExcludeFile $AdditionalBackupParameters 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog
+            Invoke-Expression "$Script:ResticExe backup $folder_list $vss_option --tag $tag --exclude-file=$WindowsExcludeFile --exclude-file=$LocalExcludeFile $AdditionalBackupParameters 3>&1 2>> $ErrorLog | Out-File -Append $SuccessLog"
             if(-not $?) {
-                "[[Backup]] Completed with errors" | Tee-Object -Append $ErrorLog | Out-File -Append $SuccessLog
+                "[[Backup]] Completed with errors" | Tee-Object -Append $ErrorLog | Tee-Object -Append $SuccessLog | Write-Host
                 $return_value = $false
             }
         }
 
         "[[Backup]] End $(Get-Date) [$tag]" | Out-File -Append $SuccessLog
     }
-    
+
     Set-Location $starting_location
-    "[[Backup]] End $(Get-Date)" | Out-File -Append $SuccessLog
+    "[[Backup]] End $(Get-Date)" | Tee-Object -Append $SuccessLog | Write-Host
 
     return $return_value
 }
@@ -297,13 +301,36 @@ function Invoke-Backup {
 function Send-Email {
     Param($SuccessLog, $ErrorLog, $Action)
 
+    Import-Module Send-MailKitMessage
+
     # default the action string to "Backup"
     if($null -eq $Action) {
         $Action = "Backup"
     }
 
-    $password = ConvertTo-SecureString $ResticEmailPassword -AsPlainText -Force
+    # set email credentials if a username and passsword are provided in configuration
-    $credentials = New-Object System.Management.Automation.PSCredential ($ResticEmailUsername, $password)
+    $credentials = @{}
+    if (-not [String]::IsNullOrEmpty($ResticEmailPassword) -and -not [String]::IsNullOrEmpty($ResticEmailUsername)) {
+        $password = ConvertTo-SecureString -String $ResticEmailPassword -AsPlainText -Force
+        $credentials = @{
+            "Credential" = [System.Management.Automation.PSCredential]::new($ResticEmailUsername, $password)
+        }
+    }
+
+    # Backwards compatibility for $ResticEmailConfig port definition:
+    # $ResticEmailConfig is obsolete and should be replaced with $ResticEmailPort
+    if ($null -ne $ResticEmailConfig -and $ResticEmailConfig.ContainsKey('Port')) {
+        if ($null -eq $ResticEmailPort) {
+            $ResticEmailPort = $ResticEmailConfig['Port']
+            '[[Email]] Warning - $ResticEmailConfig is deprecated. Define $ResticEmailPort in secrets.ps1 instead.' | Tee-Object -Append $ErrorLog | Tee-Object -Append $SuccessLog | Write-Host
+        }
+    }
+
+    # Backwards compatibility for $PSEmailServer rename to $ResticEmailServer
+    if (($null -ne $PSEmailServer) -and ($null -eq $ResticEmailServer)) {
+        $ResticEmailServer = $PSEmailServer
+        '[[Email]] Warning - $PSEmailServer is deprecated. Define $ResticEmailServer in secrets.ps1 instead.' | Tee-Object -Append $ErrorLog | Tee-Object -Append $SuccessLog | Write-Host
+    }
 
     $status = "SUCCESS"
     $past_failure = $false
@@ -311,7 +338,7 @@ function Send-Email {
     if (($null -ne $SuccessLog) -and (Test-Path $SuccessLog) -and (Get-Item $SuccessLog).Length -gt 0) {
         $body = $(Get-Content -Raw $SuccessLog)
 
-        # if previous run contained an error, send the success email confirming that the error has been resolved        
+        # if previous run contained an error, send the success email confirming that the error has been resolved
         if($Action -eq "Backup") {
             $past_failure = -not $Script:ResticStateLastBackupSuccessful
         }
@@ -320,24 +347,30 @@ function Send-Email {
         }
     }
     else {
-        $body = "Crtical Error! Restic $Action log is empty or missing. Check log file path."
+        $body = "Critical Error! Restic $Action log is empty or missing. Check log file path."
         $status = "ERROR"
     }
-    $attachments = @{}
+
+    $attachments = [System.Collections.Generic.List[string]]::new()
     if (($null -ne $ErrorLog) -and (Test-Path $ErrorLog) -and (Get-Item $ErrorLog).Length -gt 0) {
-        $attachments = @{Attachments = $ErrorLog}
+        $attachments.Add("$ErrorLog")
         $status = "ERROR"
     }
+
     if((($status -eq "SUCCESS") -and ($SendEmailOnSuccess -ne $false)) -or ((($status -eq "ERROR") -or $past_failure) -and ($SendEmailOnError -ne $false))) {
         $subject = "$env:COMPUTERNAME Restic $Action Report [$status]"
 
         # create a temporary error log to log errors; can't write to the same file that Send-MailMessage is reading
         $temp_error_log = $ErrorLog + "_temp"
 
-        Send-MailMessage @ResticEmailConfig -From $ResticEmailFrom -To $ResticEmailTo -Credential $credentials -Subject $subject -Body $body @attachments 3>&1 2>> $temp_error_log
+        $from = [MimeKit.MailboxAddress]$ResticEmailFrom;
+        $recipients = [MimeKit.InternetAddressList]::new();
+        $recipients.Add([MimeKit.InternetAddress]$ResticEmailTo);
+
+        Send-MailKitMessage -SMTPServer $ResticEmailServer -Port $ResticEmailPort -UseSecureConnectionIfAvailable @credentials -From $from -RecipientList $recipients -Subject $subject -TextBody $body -AttachmentList $attachments 3>&1 2>> $temp_error_log | Out-File -Append $SuccessLog
 
         if(-not $?) {
-            "[[Email]] Sending email completed with errors" | Tee-Object -Append $temp_error_log | Out-File -Append $SuccessLog
+            "[[Email]] Sending email completed with errors" | Tee-Object -Append $temp_error_log | Tee-Object -Append $SuccessLog | Write-Host
         }
 
         # join error logs and remove the temporary
@@ -346,24 +379,46 @@ function Send-Email {
     }
 }
 
+# check if on metered network,
+# returns $true the current connection is a metered network
+function Invoke-MeteredCheck {
+
+    $scriptBlock = {
+        # load NetworkInformation class from the Windows Runtime (WinRT) environment
+        [void][Windows.Networking.Connectivity.NetworkInformation, Windows, ContentType = WindowsRuntime]
+            
+        $cost = [Windows.Networking.Connectivity.NetworkInformation]::GetInternetConnectionProfile().GetConnectionCost()
+        return ($cost.ApproachingDataLimit -or $cost.OverDataLimit -or $cost.Roaming -or $cost.BackgroundDataUsageRestricted -or ($cost.NetworkCostType -ne 'Unrestricted'))
+    }
+    
+    # run this check in PowerShell 5.1
+    # this is a workaround for lack of WinRT support in PowerShell 7
+    $result = powershell.exe -Version 5.1 -Command "$scriptBlock"
+    return ($result -ieq "True")
+}
+
+# check network conditions, retrying a limited number of times until a connection is established
+# returns $true if the repository is accessible and the configuration allows us to use it
 function Invoke-ConnectivityCheck {
     Param($SuccessLog, $ErrorLog)
-    
+
+    $sleep_time = 30
+
     if($InternetTestAttempts -le 0) {
-        "[[Internet]] Internet connectivity check disabled. Skipping." | Out-File -Append $SuccessLog    
+        "[[Internet]] Internet connectivity check disabled. Skipping." | Out-File -Append $SuccessLog
         return $true
     }
 
     # skip the internet connectivity check for local repos
     if(Test-Path $env:RESTIC_REPOSITORY) {
-        "[[Internet]] Local repository. Skipping internet connectivity check." | Out-File -Append $SuccessLog    
+        "[[Internet]] Local repository. Skipping internet connectivity check." | Out-File -Append $SuccessLog
         return $true
     }
 
     $repository_host = ''
 
     # use generic internet service for non-specific repo types (e.g. swift:, rclone:, etc. )
-    if(($env:RESTIC_REPOSITORY -match "^swift:") -or 
+    if(($env:RESTIC_REPOSITORY -match "^swift:") -or
         ($env:RESTIC_REPOSITORY -match "^rclone:")) {
         $repository_host = "cloudflare.com"
     }
@@ -395,19 +450,30 @@ function Invoke-ConnectivityCheck {
     # test for internet connectivity
     $connections = 0
     $sleep_count = $InternetTestAttempts
+    $restricted_by_metered_network = $false
     while($true) {
         $connections = Get-NetRoute | Where-Object DestinationPrefix -eq '0.0.0.0/0' | Get-NetIPInterface | Where-Object ConnectionState -eq 'Connected' | Measure-Object | ForEach-Object{$_.Count}
         if($sleep_count -le 0) {
-            "[[Internet]] Connection to repository ($repository_host) could not be established." | Tee-Object -Append $SuccessLog | Out-File -Append $ErrorLog
+            if($restricted_by_metered_network) {
+                "[[Internet]] Connection to repository ($repository_host) is available but blocked by metered network." | Tee-Object -Append $SuccessLog | Out-File -Append $ErrorLog
+            }
+            else {
+                "[[Internet]] Connection to repository ($repository_host) could not be established." | Tee-Object -Append $SuccessLog | Out-File -Append $ErrorLog
+            }
             return $false
         }
         if(($null -eq $connections) -or ($connections -eq 0)) {
-            "[[Internet]] Waiting for internet connectivity... $sleep_count" | Out-File -Append $SuccessLog
+            "[[Internet]] Waiting $sleep_time seconds for internet connectivity... ($sleep_count/$InternetTestAttempts)" | Out-File -Append $SuccessLog
-            Start-Sleep 30
+            Start-Sleep $sleep_time
         }
         elseif(!(Test-Connection -ComputerName $repository_host -Quiet)) {
-            "[[Internet]] Waiting for connection to repository ($repository_host)... $sleep_count" | Out-File -Append $SuccessLog
+            "[[Internet]] Waiting $sleep_time seconds for connection to repository ($repository_host)... ($sleep_count/$InternetTestAttempts)" | Out-File -Append $SuccessLog
-            Start-Sleep 30
+            Start-Sleep $sleep_time
+        }
+        elseif((-not ([String]::IsNullOrEmpty($BackupOnMeteredNetwork) -or $BackupOnMeteredNetwork)) -and (Invoke-MeteredCheck)) {
+            "[[Internet]] Waiting $sleep_time seconds for an unmetered network connection... ($sleep_count/$InternetTestAttempts)" | Out-File -Append $SuccessLog        
+            $restricted_by_metered_network = $true
+            Start-Sleep $sleep_time
         }
         else {
             return $true
@@ -426,16 +492,16 @@ function Invoke-HistoryCheck {
     }
 
     $filter = "*$Action.err.txt".ToLower()
-    $logs = Get-ChildItem $LogPath -Filter $filter | ForEach-Object{$_.Length -gt 0}
+    $logs = Get-ChildItem $Script:LogPath -Filter $filter | ForEach-Object{$_.Length -gt 0}
     $logs_with_success = ($logs | Where-Object {($_ -eq $false)}).Count
     if($logs.Count -gt 0) {
-        Write-Output "[[History]] $Action success rate: $logs_with_success / $($logs.Count) ($(($logs_with_success / $logs.Count).tostring("P")))" | Tee-Object -Append $SuccessLog
+        "[[History]] $Action success rate: $logs_with_success / $($logs.Count) ($(($logs_with_success / $logs.Count).tostring("P")))" | Tee-Object -Append $SuccessLog  | Write-Host
     }
 }
 
 # main function
 function Invoke-Main {
-    
+
     # check for elevation, required for creation of shadow copy (VSS)
     if (-not (New-Object Security.Principal.WindowsPrincipal([Security.Principal.WindowsIdentity]::GetCurrent())).IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator))
     {
@@ -445,14 +511,22 @@ function Invoke-Main {
 
     # initialize secrets
     . $SecretsScript
-    
+
     # initialize config
     . $ConfigScript
-    
+
+    # apply global configuration
+    $Script:ResticExe = Join-Path $InstallPath $ExeName
+    if(-not [String]::IsNullOrEmpty($GlobalParameters)) {
+        $Script:ResticExe = "$Script:ResticExe $GlobalParameters"
+    }
+    $Script:StateFile = Join-Path $InstallPath "state.xml"
+    $Script:LogPath = Join-Path $InstallPath "logs"
+
     Get-BackupState
 
-    if(!(Test-Path $LogPath)) {
+    if(!(Test-Path $Script:LogPath)) {
-        Write-Error "[[Backup]] Log file directory $LogPath does not exist. Exiting."
+        Write-Error "[[Backup]] Log file directory $Script:LogPath does not exist. Exiting."
         Send-Email
         exit 1
     }
@@ -466,11 +540,11 @@ function Invoke-Main {
     while ($attempt_count -gt 0) {
         # setup logfiles
         $timestamp = Get-Date -Format FileDateTime
-        $success_log = Join-Path $LogPath ($timestamp + ".backup.log.txt")
+        $success_log = Join-Path $Script:LogPath ($timestamp + ".backup.log.txt")
-        $error_log = Join-Path $LogPath ($timestamp + ".backup.err.txt")
+        $error_log = Join-Path $Script:LogPath ($timestamp + ".backup.err.txt")
-        
+
         $repository_available = Invoke-ConnectivityCheck $success_log $error_log
-        if($repository_available -eq $true) { 
+        if($repository_available -eq $true) {
             Invoke-Unlock $success_log $error_log
             $backup_success = Invoke-Backup $success_log $error_log
 
@@ -480,30 +554,30 @@ function Invoke-Main {
             $total_attempts = $GlobalRetryAttempts - $attempt_count + 1
             if($backup_success -eq $true) {
                 # successful backup
-                Write-Output "[[Backup]] Succeeded after $total_attempts attempt(s)" | Tee-Object -Append $success_log
+                "[[Backup]] Succeeded after $total_attempts attempt(s)" | Tee-Object -Append $success_log | Write-Host
 
                 # test to see if maintenance is needed if the backup was successful
                 $maintenance_needed = Test-Maintenance $success_log $error_log
             }
             else {
-                Write-Output "[[Backup]] Ran with errors on attempt $total_attempts" | Tee-Object -Append $success_log | Tee-Object -Append $error_log
+                "[[Backup]] Ran with errors on attempt $total_attempts" | Tee-Object -Append $success_log | Tee-Object -Append $error_log | Write-Host
                 $error_count++
             }
         }
         else {
-            Write-Output "[[Backup]] Failed - cannot access repository." | Tee-Object -Append $success_log | Tee-Object -Append $error_log
+            "[[Backup]] Failed - cannot access repository." | Tee-Object -Append $success_log | Tee-Object -Append $error_log | Write-Host
             $error_count++
         }
 
         $attempt_count--
-        
+
         # update logs prior to sending email
-        if($backup_success -eq $false) { 
+        if($backup_success -eq $false) {
             if($attempt_count -gt 0) {
-                Write-Output "[[Backup]] Sleeping for 15 min and then retrying..." | Tee-Object -Append $success_log
+                "[[Backup]] Sleeping for 15 min and then retrying..." | Tee-Object -Append $success_log  | Write-Host
             }
             else {
-                Write-Output "[[Backup]] Retry limit has been reached. No more attempts to backup will be made." | Tee-Object -Append $success_log
+                "[[Backup]] Retry limit has been reached. No more attempts to backup will be made." | Tee-Object -Append $success_log | Write-Host
             }
         }
 
@@ -512,7 +586,7 @@ function Invoke-Main {
 
         # update the state of the last backup success or failure
         $Script:ResticStateLastBackupSuccessful = $backup_success
-        
+
         # Save state to file
         Set-BackupState
 
@@ -523,44 +597,44 @@ function Invoke-Main {
         else {
             break
         }
-    } 
+    }
 
     # only run maintenance if the backup was successful and maintenance is needed
     $attempt_count = $GlobalRetryAttempts
     while (($maintenance_needed -eq $true) -and ($attempt_count -gt 0)) {
         # setup logfiles
         $timestamp = Get-Date -Format FileDateTime
-        $success_log = Join-Path $LogPath ($timestamp + ".maintenance.log.txt")
+        $success_log = Join-Path $Script:LogPath ($timestamp + ".maintenance.log.txt")
-        $error_log = Join-Path $LogPath ($timestamp + ".maintenance.err.txt")
+        $error_log = Join-Path $Script:LogPath ($timestamp + ".maintenance.err.txt")
-        
+
         $repository_available = Invoke-ConnectivityCheck $success_log $error_log
-        if($repository_available -eq $true) { 
+        if($repository_available -eq $true) {
             $maintenance_success = Invoke-Maintenance $success_log $error_log
 
             # $maintenance_success = ($maintenance_success -eq $true) -and (!(Test-Path $error_log) -or ((Get-Item $error_log).Length -eq 0))
             $total_attempts = $GlobalRetryAttempts - $attempt_count + 1
             if($maintenance_success -eq $true) {
-                Write-Output "[[Maintenance]] Succeeded after $total_attempts attempt(s)" | Tee-Object -Append $success_log
+                "[[Maintenance]] Succeeded after $total_attempts attempt(s)" | Tee-Object -Append $success_log | Write-Host
             }
             else {
-                Write-Output "[[Maintenance]] Ran with errors on attempt $total_attempts" | Tee-Object -Append $success_log | Tee-Object -Append $error_log
+                "[[Maintenance]] Ran with errors on attempt $total_attempts" | Tee-Object -Append $success_log | Tee-Object -Append $error_log | Write-Host
                 $error_count++
             }
         }
         else {
-            Write-Output "[[Maintenance]] Failed - cannot access repository." | Tee-Object -Append $success_log | Tee-Object -Append $error_log
+            "[[Maintenance]] Failed - cannot access repository." | Tee-Object -Append $success_log | Tee-Object -Append $error_log | Write-Host
             $error_count++
         }
-        
+
         $attempt_count--
 
         # update logs prior to sending email
-        if($maintenance_success -eq $false) { 
+        if($maintenance_success -eq $false) {
             if($attempt_count -gt 0) {
-                Write-Output "[[Maintenance]] Sleeping for 15 min and then retrying..." | Tee-Object -Append $success_log
+                "[[Maintenance]] Sleeping for 15 min and then retrying..." | Tee-Object -Append $success_log | Write-Host
             }
             else {
-                Write-Output "[[Maintenance]] Retry limit has been reached. No more attempts to run maintenance will be made." | Tee-Object -Append $success_log
+                "[[Maintenance]] Retry limit has been reached. No more attempts to run maintenance will be made." | Tee-Object -Append $success_log | Write-Host
             }
         }
 
@@ -569,7 +643,7 @@ function Invoke-Main {
 
         # update the state of the last maintenance success or failure
         $Script:ResticStateLastMaintenanceSuccessful = $maintenance_success
-        
+
         # Save state to file
         Set-BackupState
 
@@ -580,13 +654,13 @@ function Invoke-Main {
         else {
             break
         }
-    }    
+    }
 
     # Save state to file
     Set-BackupState
 
     # cleanup older log files
-    Get-ChildItem $LogPath | Where-Object {$_.CreationTime -lt $(Get-Date).AddDays(-$LogRetentionDays)} | Remove-Item
+    Get-ChildItem $Script:LogPath | Where-Object {$_.CreationTime -lt $(Get-Date).AddDays(-$LogRetentionDays)} | Remove-Item
 
     exit $error_count
 }

config.ps1

@@ -1,37 +1,43 @@
-# backup configuration
+# general configuration
-$ExeName = "restic.exe"
 $InstallPath = "C:\restic"
-$ResticExe = Join-Path $InstallPath $ExeName
+$ExeName = "restic.exe"
-$StateFile = Join-Path $InstallPath "state.xml"
+$GlobalParameters = @()
-$WindowsExcludeFile = Join-Path $InstallPath "windows.exclude"
-$LocalExcludeFile = Join-Path $InstallPath "local.exclude"
-$LogPath = Join-Path $InstallPath "logs"
 $LogRetentionDays = 30
+$BackupOnMeteredNetwork = $true
 $InternetTestAttempts = 10
 $GlobalRetryAttempts = 4
-$IgnoreMissingBackupSources = $false
-$AdditionalBackupParameters = @("--exclude-if-present", ".nobackup")
-
-# maintenance configuration
-$SnapshotMaintenanceEnabled = $true
-$SnapshotRetentionPolicy = @("--group-by", "host,tags", "--keep-daily", "30", "--keep-weekly", "52", "--keep-monthly", "24", "--keep-yearly", "10")
-$SnapshotPrunePolicy = @("--max-unused", "1%")
-$SnapshotMaintenanceInterval = 7
-$SnapshotMaintenanceDays = 30
-$SnapshotDeepMaintenanceDays = 90;
 
 # email configuration
 $SendEmailOnSuccess = $false
 $SendEmailOnError = $true
 
+# backup configuration
+$WindowsExcludeFile = Join-Path $InstallPath "windows.exclude"
+$LocalExcludeFile = Join-Path $InstallPath "local.exclude"
+$IgnoreMissingBackupSources = $false
+$AdditionalBackupParameters = @("--exclude-if-present", ".nobackup", "--no-scan")
+
 # Paths to backup
 $BackupSources = @{}
 $BackupSources["C:\"] = @(
-#    'Users'
+#    "Users\Example\Desktop\Source1",
+#    "Users\Example\Desktop\Source2"
 )
-#$BackupSources["D:\"] = @(
+# $BackupSources["D:\"] = @(
-#    'Software'
+#    "Example\Source3",
-#)
+#    "Example\Source4"
+# )
 #$BackupSources["DRIVE_LABEL_NAME_OR_SERIAL_NUMBER"] = @(
-#    'FolderName'
+#    "Example\FolderName"
 #)
+
+# maintenance configuration
+$SnapshotMaintenanceEnabled = $true
+$SnapshotRetentionPolicy = @("--host", $env:COMPUTERNAME, "--group-by", "host,tags", "--keep-daily", "30", "--keep-weekly", "52", "--keep-monthly", "24", "--keep-yearly", "10")
+$SnapshotPrunePolicy = @("--max-unused", "1%")
+$SnapshotMaintenanceInterval = 7
+$SnapshotMaintenanceDays = 30
+$SnapshotDeepMaintenanceDays = 90
+
+# restic.exe self update configuration
+$SelfUpdateEnabled = $true

install.ps1

@@ -1,14 +1,35 @@
-. .\config.ps1
+#
-. .\secrets.ps1
+# Restic Windows Backup - Installation Script
+#
+
+# =========== start configuration =========== #
+
+# load restic configuration parmeters (destination, passwords, etc.)
+$SecretsScript = Join-Path $PSScriptRoot "secrets.ps1"
+
+# load backup configuration variables
+$ConfigScript = Join-Path $PSScriptRoot "config.ps1"
+
+# initialize secrets
+. $SecretsScript
+
+# initialize config
+. $ConfigScript
+
+# apply global configuration
+$ResticExe = Join-Path $InstallPath $ExeName
+$LogPath = Join-Path $InstallPath "logs"
+
+# =========== end configuration =========== #
 
 # download restic
 if(-not (Test-Path $ResticExe)) {
     $url = $null
     if([Environment]::Is64BitOperatingSystem){
-        $url = "https://github.com/restic/restic/releases/download/v0.15.0/restic_0.15.0_windows_amd64.zip"
+        $url = "https://github.com/restic/restic/releases/download/v0.17.3/restic_0.17.3_windows_amd64.zip"
     }
     else {
-        $url = "https://github.com/restic/restic/releases/download/v0.15.0/restic_0.15.0_windows_386.zip"
+        $url = "https://github.com/restic/restic/releases/download/v0.17.3/restic_0.17.3_windows_386.zip"
     }
     $output = Join-Path $InstallPath "restic.zip"
     Invoke-WebRequest -Uri $url -OutFile $output
@@ -17,13 +38,21 @@ if(-not (Test-Path $ResticExe)) {
     Get-ChildItem *.exe | Rename-Item -NewName $ExeName
 }
 
+# Apply global paramters to $ResticExe, after the $ResticExe has been downloaded/confirmed to exist
+if(-not [String]::IsNullOrEmpty($GlobalParameters)) {
+    $ResticExe = "$ResticExe $GlobalParameters"
+}
+
 # Invoke restic self-update to check for a newer version
-& $ResticExe self-update
+# This is enabled by default unless configuration disables self-update
+if ([String]::IsNullOrEmpty($SelfUpdateEnabled) -or ($SelfUpdateEnabled -eq $true)) {
+    Invoke-Expression "$ResticExe self-update"
+}
 
 # Create log directory if it doesn't exit
 if(-not (Test-Path $LogPath)) {
     New-Item -ItemType Directory -Force -Path $LogPath | Out-Null
-    Write-Output "[[Init]] Repository successfully initialized."
+    Write-Output "[[Init]] Created log directory: $LogPath"
 }
 
 # Create the local exclude file
@@ -32,7 +61,7 @@ if(-not (Test-Path $LocalExcludeFile)) {
 }
 
 # Initialize the restic repository
-& $ResticExe --verbose init
+Invoke-Expression "$ResticExe --verbose init"
 if($?) {
     Write-Output "[[Init]] Repository successfully initialized."
 }
@@ -60,4 +89,8 @@ else {
     Write-Warning "[[Scheduler]] Backup task not scheduled: there is already a task with the name '$backup_task_name'."
 }
 
-
+# Install NuGet and Send-MailKitMessage module (by force)
+if ($PSVersionTable.PSVersion.Major -eq 5) {
+    Install-PackageProvider -Name NuGet -Force
+}
+Install-Module Send-MailKitMessage -Repository PSGallery -Scope AllUsers -Force

secrets_template.ps1

@@ -9,9 +9,9 @@ $Env:RESTIC_REPOSITORY='<REPO URL>'
 $Env:RESTIC_PASSWORD='<BACKUP PASSWORD>'
 
 # email configuration
-$PSEmailServer='<SMTP SERVER>'
+$ResticEmailServer='<SMTP SERVER>'
-$ResticEmailConfig=@{UseSsl=$true; Port="587"}
+$ResticEmailPort='<SMTP PORT NUMBER, i.e. 25 or 587>'
 $ResticEmailTo='<DESTINATION EMAIL ADDRESS>'
 $ResticEmailFrom='<FROM EMAIL ADDRESS>'
-$ResticEmailUsername='<EMAIL LOGIN USERNAME>'
+$ResticEmailUsername='<EMAIL LOGIN USERNAME OR EMPTY FOR NO USERNAME>'
-$ResticEmailPassword='<EMAIL PASSWORD>'
+$ResticEmailPassword='<EMAIL PASSWORD OR EMPTY FOR NO PASSWORD>'

windows.exclude

@@ -35,6 +35,7 @@ Dropbox
 AppData\Local\Google\Drive
 Google Drive\.tmp.drivedownload
 C:\OneDriveTemp
+Users\**\Nextcloud
 
 # browsers
 Google\Chrome