minor spelling, typo fixes

Release 1.5

CHANGELOG.md

@@ -1,5 +1,103 @@
 # Changelog
 
+## [1.5](https://github.com/kmwoley/restic-windows-backup/tree/1.5) (2021-09-11)
+[Full Changelog](https://github.com/kmwoley/restic-windows-backup/compare/1.4.1...1.5)
+
+Added support for backing up removable drives (i.e. external USB disks). It's now possible to define a backup source by it's Volume label, device Serial Number, or the hardware Name. 
+
+**WARNING** If you have been previously backing up multiple drives, the default `forget` policy was likely pruning backup sets too aggressively and could lead to data loss. You **must** update your `$SnapshotRetentionPolicy` to include `@("--group-by", "host,tags", ...` to avoid pruning an entire drive's contents inadvertently! 
+
+## Fixes
+- Updated default snapshot forget/prune retention policy to group by "host,tags" to prevent major data loss. Only configurations with multiple `$BackupSources` are impacted by this change.
+- Added tags to each backup source to support grouping by tags. For existing backup sets, this change will result in a slightly longer backup the first time this updated script is run.
+
+## Enhancements
+- External, removable disk drives (i.e. USB hard drives) can be identified by their Volume Label, Serial Number, or Device Name. For example, if you have an external device with the Volume Label "MY BOOK", you can define a backup source as `$BackupSources["MY BOOK"]`. I would recommend using the device serial number to identify external drives to backup, which you can find using the Powershell `get-disk` command.
+- Add the ability to $IgnoreMissingBackupSources. To make sure that errors are not thrown if the device is not present, there is now an option to ignore error reporting when a folder and entire backup source are missing. When `$true`, missing external drives or folders don't produce errors. When `$null` or `$false`, missing drives and/or folders result in an error. The default is set to `$false` as not to silently fail backing up a source.
+- Updated install script to download Restic 12.1
+
+## [1.4.1](https://github.com/kmwoley/restic-windows-backup/tree/1.4.1) (2021-05-29)
+[Full Changelog](https://github.com/kmwoley/restic-windows-backup/compare/1.4...1.4.1)
+
+Bugfix release.
+
+## Fixes
+- Improved URL parsing so that the internet connectivity check works if the URL doesn't provide a protocol
+- Add PowerShell 7.1 support to internet connectivity check
+
+## Enhancements
+- Setting $InternetTestAttempts to 0 will now bypass the internet connectivity checks entirely
+
+## [1.4](https://github.com/kmwoley/restic-windows-backup/tree/1.4) (2021-02-24)
+[Full Changelog](https://github.com/kmwoley/restic-windows-backup/compare/1.3...1.4)
+
+Moved to using Restic's inbuilt filesystem shadow copy creation (VSS).
+
+## Breaking Change
+`local.exclude` file changes that previously referenced the `resticVSS` directory will need to be changed to `C:\` or the relevant root drive letter.
+
+## Other enhancements
+- Future snapshot grouping (and cleanup) will be better since the root-level folders included in the backup won't change (instead, the script targets the root drive letter instead of a list of folders under the drive letter).
+- Added the ability to set prune parameters via `.\config.ps1`, and defaulted the settings to `--group-by host` to clean up the aforementioned snapshot grouping & pruning.
+- Updated the `windows.exclude` to include additional directories (most notably, the Recycle Bin is no longer backed up) 
+
+**Closed issues:**
+
+- Remove VSS Operations, Switch to `--use-fs-snapshot` [\#32](https://github.com/kmwoley/restic-windows-backup/issues/32)
+- powershell execution policy is blocking the scheduled task [\#27](https://github.com/kmwoley/restic-windows-backup/issues/27)
+- VSS Cleanup Upon Errors [\#8](https://github.com/kmwoley/restic-windows-backup/issues/8)
+
+**Merged pull requests:**
+
+- Release 1.4 [\#33](https://github.com/kmwoley/restic-windows-backup/pull/33) ([kmwoley](https://github.com/kmwoley))
+
+## [1.3](https://github.com/kmwoley/restic-windows-backup/tree/1.3) (2021-02-23)
+
+[Full Changelog](https://github.com/kmwoley/restic-windows-backup/compare/1.2.1...1.3)
+
+Improvements for Restic 0.12 and additional error logging.
+
+**Closed issues:**
+
+- backup errors after update to restic 0.12.0 due to --quiet and --verbose being used simultaneously [\#29](https://github.com/kmwoley/restic-windows-backup/issues/29)
+- Restic + rclone errors [\#26](https://github.com/kmwoley/restic-windows-backup/issues/26)
+- E-Mail sending errors are not logged [\#25](https://github.com/kmwoley/restic-windows-backup/issues/25)
+- FYI: Restic now has built-in VSS support [\#23](https://github.com/kmwoley/restic-windows-backup/issues/23)
+- SFTP backup [\#22](https://github.com/kmwoley/restic-windows-backup/issues/22)
+- Dirrectory/Folder Backup [\#21](https://github.com/kmwoley/restic-windows-backup/issues/21)
+- Docker format [\#20](https://github.com/kmwoley/restic-windows-backup/issues/20)
+- Filtering out errors before deciding to retry ? [\#19](https://github.com/kmwoley/restic-windows-backup/issues/19)
+- Backup task stucked [\#18](https://github.com/kmwoley/restic-windows-backup/issues/18)
+
+**Merged pull requests:**
+
+- Release 1.4 [\#31](https://github.com/kmwoley/restic-windows-backup/pull/31) ([kmwoley](https://github.com/kmwoley))
+- Add '-ExecutionPolicy Bypass' to the task scheduler arguments to avoi… [\#28](https://github.com/kmwoley/restic-windows-backup/pull/28) ([scelfo](https://github.com/scelfo))
+- Fix URI parsing [\#24](https://github.com/kmwoley/restic-windows-backup/pull/24) ([Phlogi](https://github.com/Phlogi))
+
+## [1.2.1](https://github.com/kmwoley/restic-windows-backup/tree/1.2.1) (2020-06-08)
+
+[Full Changelog](https://github.com/kmwoley/restic-windows-backup/compare/1.1...1.2.1)
+
+* Internet connectivity test now supports more repository types (s3:, sftp:, rest:, azure:, gs:), and ignores unsupported (swift:, rclone: and local) 
+* Add 32-bit support in the `install.ps1`
+
+* Fix/improve internet connectivity checks for azure: gs: b2:
+
+**Closed issues:**
+
+- azure repo could not be parsed [\#15](https://github.com/kmwoley/restic-windows-backup/issues/15)
+- Need to strip rest: in addition to s3: from RESTIC\_REPOSITORY [\#14](https://github.com/kmwoley/restic-windows-backup/issues/14)
+- Use non-s3 repos [\#10](https://github.com/kmwoley/restic-windows-backup/issues/10)
+- Test-Connection fails [\#9](https://github.com/kmwoley/restic-windows-backup/issues/9)
+- 32bit Windows Support [\#7](https://github.com/kmwoley/restic-windows-backup/issues/7)
+- Add changelog [\#1](https://github.com/kmwoley/restic-windows-backup/issues/1)
+
+**Merged pull requests:**
+
+- Release 1 3 [\#17](https://github.com/kmwoley/restic-windows-backup/pull/17) ([kmwoley](https://github.com/kmwoley))
+- 1.2 Release [\#13](https://github.com/kmwoley/restic-windows-backup/pull/13) ([kmwoley](https://github.com/kmwoley))
+
 ## [1.1](https://github.com/kmwoley/restic-windows-backup/tree/1.1) (2020-02-15)
 
 [Full Changelog](https://github.com/kmwoley/restic-windows-backup/compare/1.0...1.1)
@@ -21,6 +119,7 @@ $SendEmailOnError = $true
 
 **Merged pull requests:**
 
+- add changelog [\#6](https://github.com/kmwoley/restic-windows-backup/pull/6) ([kmwoley](https://github.com/kmwoley))
 - add options to enable/disable email sending  and maintenance [\#4](https://github.com/kmwoley/restic-windows-backup/pull/4) ([kmwoley](https://github.com/kmwoley))
 
 ## [1.0](https://github.com/kmwoley/restic-windows-backup/tree/1.0) (2020-02-09)

README.md

@@ -4,6 +4,7 @@ Simplifies the process of installation and running daily backups.
 
 # Features
 * **VSS (Volume Snapshot Service) support** - backup everything, don't worry about what files are open/in-use
+* **Removable, External Drives** - drives can be identified by their volume labels or serial numbers, making it easy to backup drives that occasionally aren't there or change drive letter.
 * **Easy Installation** - `install.ps1` script downloads Restic, initializes the restic repository, and setups up a Windows Task Scheduler task to run the backup daily
 * **Backup, Maintenance and Monitoring are Automated** - `backup.ps1` script handles
   * Emailing the results of each execution, including log files when there are problems
@@ -28,6 +29,9 @@ Simplifies the process of installation and running daily backups.
 1. Run `install.ps1` file
    1. From the elevated (Run as Administrator) Powershell window, run `.\install.ps1`
    1. This will initialize the repro, create your logfile directory, and create a scheduled task in Windows Task Scheduler to run the task daily.
+1. Add your `$BackupSources` to `config.ps1`
+   1. By default, all of `C:\` will be backed up. You can add multiple root drives to be backed up. And you can define only specific folders you would like backed up.
+   1. External, removalbe disk drives (i.e. USB hard drives) can be identified by their Volume Label, Serial Number, or Device Name. For example, if you have an external device with the Volume Label "MY BOOK", you can define a backup source as `$BackupSources["MY BOOK"]=@()`. I would recommend using the device serial number to identify external drives to backup, which you can find using the Powershell `get-disk` command. You may also want to set `$IgnoreMissingBackupSources=$true` to avoid seeing errors when the removable drive is not present.
 1. Add files/paths not to backup to `local.exclude`
    1. If you don't want to modify the included exclude file, you can add any files/paths you want to exclude from the backup to `local.exclude`
 1. Add `restic.exe` to the Windows Defender / Virus & Threat Detection Exclude list

backup.ps1

@@ -17,6 +17,51 @@ $Script:ResticStateRepositoryInitialized = $null
 $Script:ResticStateLastMaintenance = $null
 $Script:ResticStateLastDeepMaintenance = $null
 $Script:ResticStateMaintenanceCounter = $null
+ 
+# Returns all drive letters which exactly match the serial number, drive label, or drive name of 
+# the input parameter. Returns all drives if no input parameter is provided.
+# inspiration: https://stackoverflow.com/questions/31088930/combine-get-disk-info-and-logicaldisk-info-in-powershell
+function Get-Drives {
+    Param($ID)
+
+    foreach($disk in Get-CimInstance Win32_Diskdrive) {
+        $diskMetadata = Get-Disk | Where-Object { $_.Number -eq $disk.Index } | Select-Object -First 1
+        $partitions = Get-CimAssociatedInstance -ResultClassName Win32_DiskPartition -InputObject $disk
+
+        foreach($partition in $partitions) {
+
+            $drives = Get-CimAssociatedInstance -ResultClassName Win32_LogicalDisk -InputObject $partition
+
+            foreach($drive in $drives) {
+                    
+                $volume = Get-Volume |
+                          Where-Object { $_.DriveLetter -eq $drive.DeviceID.Trim(":") } |
+                          Select-Object -First 1
+
+                if(($diskMetadata.SerialNumber.trim() -eq $ID) -or 
+                    ($disk.Caption -eq $ID) -or
+                    ($volume.FileSystemLabel  -eq $ID) -or
+                    ($null -eq $ID)) {
+    
+                    [PSCustomObject] @{
+                        DriveLetter   = $drive.DeviceID
+                        Number        = $disk.Index
+                        Label         = $volume.FileSystemLabel
+                        Manufacturer  = $diskMetadata.Manufacturer
+                        Model         = $diskMetadata.Model
+                        SerialNumber  = $diskMetadata.SerialNumber.trim()
+                        Name          = $disk.Caption
+                        FileSystem    = $volume.FileSystem
+                        PartitionKind = $diskMetadata.PartitionStyle
+                        Drive         = $drive
+                        Partition     = $partition
+                        Disk          = $disk
+                    }
+                }
+            }
+        }
+    }
+}
 
 # restore backup state from disk
 function Get-BackupState {
@@ -67,7 +112,7 @@ function Invoke-Maintenance {
 
     # forget snapshots based upon the retention policy
     Write-Output "[[Maintenance]] Start forgetting..." | Tee-Object -Append $SuccessLog
-    & $ResticExe --verbose -q forget $SnapshotRetentionPolicy 3>&1 2>> $ErrorLog | Tee-Object -Append $SuccessLog
+    & $ResticExe forget $SnapshotRetentionPolicy 3>&1 2>> $ErrorLog | Tee-Object -Append $SuccessLog
     if(-not $?) {
         Write-Output "[[Maintenance]] Forget operation completed with errors" | Tee-Object -Append $ErrorLog | Tee-Object -Append $SuccessLog
         $maintenance_success = $false
@@ -76,7 +121,7 @@ function Invoke-Maintenance {
     # prune (remove) data from the backup step. Running this separate from `forget` because
     #   `forget` only prunes when it detects removed snapshots upon invocation, not previously removed
     Write-Output "[[Maintenance]] Start pruning..." | Tee-Object -Append $SuccessLog
-    & $ResticExe --verbose -q prune 3>&1 2>> $ErrorLog | Tee-Object -Append $SuccessLog
+    & $ResticExe prune $SnapshotPrunePolicy 3>&1 2>> $ErrorLog | Tee-Object -Append $SuccessLog
     if(-not $?) {
         Write-Output "[[Maintenance]] Prune operation completed with errors" | Tee-Object -Append $ErrorLog | Tee-Object -Append $SuccessLog
         $maintenance_success = $false
@@ -103,7 +148,7 @@ function Invoke-Maintenance {
         $Script:ResticStateLastDeepMaintenance = Get-Date
     }
 
-    & $ResticExe --verbose -q check @data_check 3>&1 2>> $ErrorLog | Tee-Object -Append $SuccessLog
+    & $ResticExe check @data_check 3>&1 2>> $ErrorLog | Tee-Object -Append $SuccessLog
     if(-not $?) {
         Write-Output "[[Maintenance]] Check completed with errors" | Tee-Object -Append $ErrorLog | Tee-Object -Append $SuccessLog
         $maintenance_success = $false
@@ -113,7 +158,7 @@ function Invoke-Maintenance {
     
     if($maintenance_success -eq $true) {
         $Script:ResticStateLastMaintenance = Get-Date
-        $Script:ResticStateMaintenanceCounter = 0;
+        $Script:ResticStateMaintenanceCounter = 0
     }
 }
 
@@ -123,64 +168,102 @@ function Invoke-Backup {
 
     Write-Output "[[Backup]] Start $(Get-Date)" | Tee-Object -Append $SuccessLog
     $return_value = $true
-    $drive_count = $BackupSources.Count
     $starting_location = Get-Location
     ForEach ($item in $BackupSources.GetEnumerator()) {
 
-        $ShadowPath = Join-Path $item.Key 'resticVSS'
+        # Get the source drive letter or identifier and set as the root path
+        $root_path = $item.Key
+        $tag = $item.Key
 
-        # check for existance of previous, orphaned VSS directory (and remove it) before creating the shadow copy
-        if(Test-Path $ShadowPath) {
-            Write-Output "[[Backup]] VSS directory exists: '$ShadowPath' - removing. Past script failure?" | Tee-Object -Append $ErrorLog | Tee-Object -Append $SuccessLog
-            cmd /c rmdir $ShadowPath
-        }
-        
-        # Create the shadow copy
-        $s1 = (Get-WmiObject -List Win32_ShadowCopy).Create($item.Key, "ClientAccessible")
-        $s2 = Get-WmiObject -Class Win32_ShadowCopy | Where-Object { $_.ID -eq $s1.ShadowID }
-        
-        # Create a symbolic link to the shadow copy
-        $device  = $s2.DeviceObject + "\"
-        cmd /c mklink /d $ShadowPath "$device" 3>&1 2>> $ErrorLog | Tee-Object -Append $SuccessLog
+        $vss_option = "--use-fs-snapshot"
 
-        # Build the new list of folders
-        $root_path = $ShadowPath
-        if($drive_count -eq 1) {
-            $root_path = "."
-            Set-Location $ShadowPath
+        # Test if root path is a valid path, if not assume it is an external drive identifier
+        if(-not (Test-Path $root_path)) {
+            # attempt to find a drive letter associated with the identifier provided
+            $drives = Get-Drives $root_path
+            if($drives.Count -gt 1) {
+                Write-Output "[[Backup]] Fatal error - external drives with more than one partition are not currently supported." | Tee-Object -Append $SuccessLog | Tee-Object -Append $ErrorLog
+                return $false
+            }
+            elseif ($drives.Count -eq 0) {
+                $ignore_error = ($null -ne $IgnoreMissingBackupSources) -and $IgnoreMissingBackupSources
+                $warning_message = {Write-Output "[[Backup]] Warning - backup path $root_path not found."}
+                if($ignore_error) {
+                    & $warning_message | Tee-Object -Append $SuccessLog                    
+                }
+                else {
+                    & $warning_message | Tee-Object -Append $SuccessLog | Tee-Object -Append $ErrorLog
+                    $return_value = $false
+                }
+                continue
+            }
+            
+            $root_path = Join-Path $drives[0].DriveLetter ""
+            
+            # disable VSS / file system snapshot for external drives
+            # TODO: would be best to just test for VSS compatibility on the drive, rather than assume it won't work
+            $vss_option = $null
         }
 
+        Write-Output "[[Backup]] Start $(Get-Date) [$tag]" | Tee-Object -Append $SuccessLog
+        
+        # build the list of folders to backup
         $folder_list = New-Object System.Collections.Generic.List[System.Object]
-        ForEach ($path in $item.Value) {
-            $p = Join-Path $root_path $path
-            $folder_list.Add($p)
+        if ($item.Value.Count -eq 0) {
+            # backup everything in the root if no folders are provided
+            $folder_list.Add($root_path)
         }
+        else {
+            # Build the list of folders from settings
+            ForEach ($path in $item.Value) {
+                $p = '"{0}"' -f ((Join-Path $root_path $path) -replace "\\$")
+                
+                if(Test-Path ($p -replace '"')) {
+                    # add the folder if it exists
+                    $folder_list.Add($p)
+                }
+                else {
+                    # if the folder doesn't exist, log a warning/error
+                    $ignore_error = ($null -ne $IgnoreMissingBackupSources) -and $IgnoreMissingBackupSources
+                    $warning_message = {Write-Output "[[Backup]] Warning - backup path $p not found."}
+                    if($ignore_error) {
+                        & $warning_message | Tee-Object -Append $SuccessLog
+                    }
+                    else {
+                        & $warning_message | Tee-Object -Append $SuccessLog | Tee-Object -Append $ErrorLog
+                        $return_value = $false
+                    }
+                }
+            }
 
-        # backup everything in the root if no folders are provided
-        # note this won't select items with hidden attributes (a good thing to avoid)
-        if (-not $folder_list) {
-            ForEach ($path in Get-ChildItem $ShadowPath) {
-                $p = Join-Path $root_path $path
-                $folder_list.Add($p)
+        }
+        
+        if(-not $folder_list) {
+            # there are no folders to backup
+            $ignore_error = ($null -ne $IgnoreMissingBackupSources) -and $IgnoreMissingBackupSources
+            $warning_message = {Write-Output "[[Backup]] Warning - no folders to back up!"}
+            if($ignore_error) {
+                & $warning_message | Tee-Object -Append $SuccessLog
+            }
+            else {
+                & $warning_message | Tee-Object -Append $SuccessLog | Tee-Object -Append $ErrorLog
+                $return_value = $false
+            }
+        }
+        else {
+            # Launch Restic
+            & $ResticExe backup $folder_list $vss_option --tag "$tag" --exclude-file=$WindowsExcludeFile --exclude-file=$LocalExcludeFile 3>&1 2>> $ErrorLog | Tee-Object -Append $SuccessLog
+            if(-not $?) {
+                Write-Output "[[Backup]] Completed with errors" | Tee-Object -Append $ErrorLog | Tee-Object -Append $SuccessLog
+                $return_value = $false
             }
         }
 
-        # Launch Restic
-        & $ResticExe --verbose -q backup $folder_list --exclude-file=$WindowsExcludeFile --exclude-file=$LocalExcludeFile 3>&1 2>> $ErrorLog | Tee-Object -Append $SuccessLog
-        if(-not $?) {
-            Write-Output "[[Backup]] Completed with errors" | Tee-Object -Append $ErrorLog | Tee-Object -Append $SuccessLog
-            $return_value = $false
-        }
-
-        # Delete the shadow copy and remove the symbolic link
-        if($drive_count -eq 1) {
-            Set-Location $starting_location
-        }
-        $s2.Delete()
-        cmd /c rmdir $ShadowPath
-
-        Write-Output "[[Backup]] End $(Get-Date)" | Tee-Object -Append $SuccessLog
+        Write-Output "[[Backup]] End $(Get-Date) [$tag]" | Tee-Object -Append $SuccessLog
     }
+    
+    Set-Location $starting_location
+    Write-Output "[[Backup]] End $(Get-Date)" | Tee-Object -Append $SuccessLog
 
     return $return_value
 }
@@ -191,9 +274,16 @@ function Send-Email {
     $credentials = New-Object System.Management.Automation.PSCredential ($ResticEmailUsername, $password)
 
     $status = "SUCCESS"
+    $success_after_failure = $false
     $body = ""
     if (($null -ne $SuccessLog) -and (Test-Path $SuccessLog) -and (Get-Item $SuccessLog).Length -gt 0) {
         $body = $(Get-Content -Raw $SuccessLog)
+        # if previous run contained an error, send the success email confirming that the error has been resolved
+        # (i.e. get previous error log, if it's not empty, trigger the send of the success-after-failure email)
+        $previous_error_log = Get-ChildItem $LogPath -Filter '*err.txt' | Sort-Object -Descending LastWriteTime | Select-Object -Skip 1 | Select-Object -First 1
+        if(($null -ne $previous_error_log) -and ($previous_error_log.Length -gt 0)){
+            $success_after_failure = $true
+        }
     }
     else {
         $body = "Crtical Error! Restic backup log is empty or missing. Check log file path."
@@ -204,20 +294,69 @@ function Send-Email {
         $attachments = @{Attachments = $ErrorLog}
         $status = "ERROR"
     }
-    if((($status -eq "SUCCESS") -and ($SendEmailOnSuccess -ne $false)) -or (($status -eq "ERROR") -and ($SendEmailOnError -ne $false))) {
+    if((($status -eq "SUCCESS") -and ($SendEmailOnSuccess -ne $false)) -or ((($status -eq "ERROR") -or $success_after_failure) -and ($SendEmailOnError -ne $false))) {
         $subject = "$env:COMPUTERNAME Restic Backup Report [$status]"
-        Send-MailMessage @ResticEmailConfig -From $ResticEmailFrom -To $ResticEmailTo -Credential $credentials -Subject $subject -Body $body @attachments
+
+        # create a temporary error log to log errors; can't write to the same file that Send-MailMessage is reading
+        $temp_error_log = $ErrorLog + "_temp"
+
+        Send-MailMessage @ResticEmailConfig -From $ResticEmailFrom -To $ResticEmailTo -Credential $credentials -Subject $subject -Body $body @attachments 3>&1 2>> $temp_error_log
+
+        if(-not $?) {
+            Write-Output "[[Email]] Sending email completed with errors" | Tee-Object -Append $temp_error_log | Tee-Object -Append $SuccessLog
+        }
+
+        # join error logs and remove the temporary
+        Get-Content $temp_error_log | Add-Content $ErrorLog
+        Remove-Item $temp_error_log
     }
 }
 
 function Invoke-ConnectivityCheck {
     Param($SuccessLog, $ErrorLog)
     
-    # parse connection string for hostname
-    # TODO: handle non-s3 repositories
-    #   Uri parser doesn't handle leading connection type info
-    $connection_string = $env:RESTIC_REPOSITORY -replace "s3:" 
-    $repository_host = ([System.Uri]$connection_string).host
+    if($InternetTestAttempts -le 0) {
+        Write-Output "[[Internet]] Internet connectivity check disabled. Skipping." | Tee-Object -Append $SuccessLog    
+        return $true
+    }
+
+    # skip the internet connectivity check for local repos
+    if(Test-Path $env:RESTIC_REPOSITORY) {
+        Write-Output "[[Internet]] Local repository. Skipping internet connectivity check." | Tee-Object -Append $SuccessLog    
+        return $true
+    }
+
+    $repository_host = ''
+
+    # use generic internet service for non-specific repo types (e.g. swift:, rclone:, etc. )
+    if(($env:RESTIC_REPOSITORY -match "^swift:") -or 
+        ($env:RESTIC_REPOSITORY -match "^rclone:")) {
+        $repository_host = "cloudflare.com"
+    }
+    elseif($env:RESTIC_REPOSITORY -match "^b2:") {
+        $repository_host = "api.backblazeb2.com"
+    }
+    elseif($env:RESTIC_REPOSITORY -match "^azure:") {
+        $repository_host = "azure.microsoft.com"
+    }
+    elseif($env:RESTIC_REPOSITORY -match "^gs:") {
+        $repository_host = "storage.googleapis.com"
+    }
+    else {
+        # parse connection string for hostname
+        # Uri parser doesn't handle leading connection type info (s3:, sftp:, rest:)
+        $connection_string = $env:RESTIC_REPOSITORY -replace "^s3:" -replace "^sftp:" -replace "^rest:"
+        if(-not ($connection_string -match "://")) {
+            # Uri parser expects to have a protocol. Add 'https://' to make it parse correctly.
+            $connection_string = "https://" + $connection_string
+        }
+        $repository_host = ([System.Uri]$connection_string).DnsSafeHost
+    }
+
+    if([string]::IsNullOrEmpty($repository_host)) {
+        Write-Output "[[Internet]] Repository string could not be parsed." | Tee-Object -Append $SuccessLog | Tee-Object -Append $ErrorLog
+        return $false
+    }
 
     # test for internet connectivity
     $connections = 0
@@ -225,14 +364,14 @@ function Invoke-ConnectivityCheck {
     while($true) {
         $connections = Get-NetRoute | Where-Object DestinationPrefix -eq '0.0.0.0/0' | Get-NetIPInterface | Where-Object ConnectionState -eq 'Connected' | Measure-Object | ForEach-Object{$_.Count}
         if($sleep_count -le 0) {
-            Write-Output "[[Internet]] Connection to repository could not be established." | Tee-Object -Append $SuccessLog | Tee-Object -Append $ErrorLog
+            Write-Output "[[Internet]] Connection to repository ($repository_host) could not be established." | Tee-Object -Append $SuccessLog | Tee-Object -Append $ErrorLog
             return $false
         }
         if(($null -eq $connections) -or ($connections -eq 0)) {
             Write-Output "[[Internet]] Waiting for internet connectivity... $sleep_count" | Tee-Object -Append $SuccessLog
             Start-Sleep 30
         }
-        elseif(!(Test-Connection -Server $repository_host -Quiet)) {
+        elseif(!(Test-Connection -ComputerName $repository_host -Quiet)) {
             Write-Output "[[Internet]] Waiting for connection to repository ($repository_host)... $sleep_count" | Tee-Object -Append $SuccessLog
             Start-Sleep 30
         }
@@ -246,7 +385,7 @@ function Invoke-ConnectivityCheck {
 # check previous logs
 function Invoke-HistoryCheck {
     Param($SuccessLog, $ErrorLog)
-    $logs = Get-ChildItem $LogPath -Filter '*err.txt' | %{$_.Length -gt 0}
+    $logs = Get-ChildItem $LogPath -Filter '*err.txt' | ForEach-Object{$_.Length -gt 0}
     $logs_with_success = ($logs | Where-Object {($_ -eq $false)}).Count
     if($logs.Count -gt 0) {
         Write-Output "[[History]] Backup success rate: $logs_with_success / $($logs.Count) ($(($logs_with_success / $logs.Count).tostring("P")))" | Tee-Object -Append $SuccessLog
@@ -303,16 +442,23 @@ function Invoke-Main {
             }
         }
 
-        Write-Warning "Errors found! Error Log: $error_log"
+        Write-Output "[[General]] Errors found. Log: $error_log" | Tee-Object -Append $success_log | Tee-Object -Append $error_log
         $error_count++
         
-        Write-Output "Something went wrong. Sleeping for 15 min and then retrying..." | Tee-Object -Append $success_log
+        $attempt_count--
+        if($attempt_count -gt 0) {
+            Write-Output "[[Retry]] Sleeping for 15 min and then retrying..." | Tee-Object -Append $success_log
+        }
+        else {
+            Write-Output "[[Retry]] Retry limit has been reached. No more attempts to backup will be made." | Tee-Object -Append $success_log
+        }
         if($internet_available -eq $true) {
             Invoke-HistoryCheck $success_log $error_log
             Send-Email $success_log $error_log
         }
-        Start-Sleep (15*60)
-        $attempt_count--
+        if($attempt_count -gt 0) {
+            Start-Sleep (15*60)
+        }
     }    
 
     Set-BackupState
@@ -323,4 +469,4 @@ function Invoke-Main {
     exit $error_count
 }
 
-Invoke-Main
+Invoke-Main

config.ps1

@@ -9,10 +9,12 @@ $LogPath = Join-Path $InstallPath "logs"
 $LogRetentionDays = 30
 $InternetTestAttempts = 10
 $GlobalRetryAttempts = 4
+$IgnoreMissingBackupSources = $false
 
 # maintenance configuration
 $SnapshotMaintenanceEnabled = $true
-$SnapshotRetentionPolicy = @("--keep-daily", "30", "--keep-weekly", "52", "--keep-monthly", "24", "--keep-yearly", "10")
+$SnapshotRetentionPolicy = @("--group-by", "host,tags", "--keep-daily", "30", "--keep-weekly", "52", "--keep-monthly", "24", "--keep-yearly", "10")
+$SnapshotPrunePolicy = @("--max-unused", "1%")
 $SnapshotMaintenanceInterval = 7
 $SnapshotMaintenanceDays = 30
 $SnapshotDeepMaintenanceDays = 90;
@@ -29,3 +31,6 @@ $BackupSources["C:\"] = @(
 #$BackupSources["D:\"] = @(
 #    'Software'
 #)
+#$BackupSources["DRIVE_LABEL_NAME_OR_SERIAL_NUMBER"] = @(
+#    'FolderName'
+#)

install.ps1

@@ -3,7 +3,13 @@
 
 # download restic
 if(-not (Test-Path $ResticExe)) {
-    $url = "https://github.com/restic/restic/releases/download/v0.9.6/restic_0.9.6_windows_amd64.zip"
+    $url = $null
+    if([Environment]::Is64BitOperatingSystem){
+        $url = "https://github.com/restic/restic/releases/download/v0.12.1/restic_0.12.1_windows_386.zip"
+    }
+    else {
+        $url = "https://github.com/restic/restic/releases/download/v0.12.1/restic_0.12.1_windows_amd64.zip"
+    }
     $output = Join-Path $InstallPath "restic.zip"
     Invoke-WebRequest -Uri $url -OutFile $output
     Expand-Archive -LiteralPath $output $InstallPath
@@ -37,7 +43,7 @@ $backup_task_name = "Restic Backup"
 $backup_task = Get-ScheduledTask $backup_task_name -ErrorAction SilentlyContinue
 if($null -eq $backup_task) {
     try {
-        $task_action = New-ScheduledTaskAction -Execute 'powershell.exe' -Argument '-NonInteractive -NoLogo -NoProfile -Command ".\backup.ps1; exit $LASTEXITCODE"' -WorkingDirectory $InstallPath
+        $task_action = New-ScheduledTaskAction -Execute 'powershell.exe' -Argument '-ExecutionPolicy Bypass -NonInteractive -NoLogo -NoProfile -Command ".\backup.ps1; exit $LASTEXITCODE"' -WorkingDirectory $InstallPath
         $task_user = New-ScheduledTaskPrincipal -UserId "NT AUTHORITY\SYSTEM" -RunLevel Highest
         $task_settings = New-ScheduledTaskSettingsSet -RestartCount 4 -RestartInterval (New-TimeSpan -Minutes 15) -ExecutionTimeLimit (New-TimeSpan -Days 3) -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries -DontStopOnIdleEnd -MultipleInstances IgnoreNew -IdleDuration 0 -IdleWaitTimeout 0 -StartWhenAvailable -RestartOnIdle
         $task_trigger = New-ScheduledTaskTrigger -Daily -At 4:00am

windows.exclude

@@ -1,23 +1,25 @@
 # default excludes
 # examples https://github.com/duplicati/duplicati/blob/master/Duplicati/Library/Utility/FilterGroups.cs
-# note, because we're using a VSS directory, we can use that as the root dir for exclude rules (i.e. resticVSS)
-resticVSS\hiberfil.sys
-resticVSS\pagefile.sys
-resticVSS\swapfile.sys
-resticVSS\$Recycle.Bin
-resticVSS\autoexec.bat
-resticVSS\Config.Msi
-resticVSS\Documents and Settings
-resticVSS\Recycled
-resticVSS\Recycler
-resticVSS\System Volume Information
-resticVSS\Recovery
-resticVSS\Program Files
-resticVSS\Program Files (x86)
-resticVSS\ProgramData
-resticVSS\PerfLogs
-resticVSS\Windows
-resticVSS\Windows.old
+C:\hiberfil.sys
+C:\pagefile.sys
+C:\swapfile.sys
+C:\$Recycle.Bin
+C:\autoexec.bat
+C:\Config.Msi
+C:\Documents and Settings
+C:\Recycled
+C:\Recycler
+C:\$$Recycle.Bin
+C:\System Volume Information
+C:\Recovery
+C:\Program Files
+C:\Program Files (x86)
+C:\ProgramData
+C:\PerfLogs
+C:\Windows
+C:\Windows.old
+C:\$$WINDOWS.~BT
+C:\$$WinREAgent
 Microsoft\Windows\Recent
 Microsoft\**\RecoveryStore*
 Microsoft\**\Windows\*.edb
@@ -32,7 +34,7 @@ UsrClass.dat
 Dropbox
 AppData\Local\Google\Drive
 Google Drive\.tmp.drivedownload
-resticVSS\OneDriveTemp
+C:\OneDriveTemp
 
 # browsers
 Google\Chrome
@@ -52,6 +54,12 @@ AppData\Local\Microsoft\Windows Store
 AppData\Local\restic
 AppData\LocalLow\Microsoft\CryptnetUrlCache
 AppData\Local\IsolatedStorage
+AppData\Local\Spotify
+AppData\Local\Programs\signal-desktop
+AppData\Roaming\Code
+AppData\Roaming\Slack
+AppData\Roaming\Spotify
+AppData\Roaming\Zoom
 
 # misc. temporary files
 Temporary Internet Files