Data replication using PowerShell Script + Create log files for your scripts
The below article is focused on data replication from one source to destination - Not a folder synchronization script. It's always been a question to me how to replicate the content of one folder to another for a various purpose. This article also included logging option in Powershell, So we can follow, and see what is happening while the script is executing and copying the data or if the script runs into errors is very useful. Please follow the below article to know how can we achieve this using PowerShell.
Below cmdlets would be the input for the script. We need to define the source location, destination location and log file path for the script.
We have to declare few variable such as Log file name, error handling variables etc.
The next part will be Create\Modify the Log file based on the availability of log and based on the size of the log - If the log file size is more than 1 MB (We can modify the size) log file will be replaced with a new log file.
Checking the source location is accessible or not from the destination location - The script execution will exit in case if the source not accessible.
The System.Security.Cryptography namespace provides cryptographic services, including secure encoding and decoding of data, as well as many other operations, such as hashing, random number generation, and message authentication. For more information. Refer the below link for more details.
https://msdn.microsoft.com/en-us/library/system.security.cryptography(v=vs.110).aspx
Below part is replicating data and writing the respective logs into log file
Combine all the script together and run.
Click here to download the script from Technet
The below article is focused on data replication from one source to destination - Not a folder synchronization script. It's always been a question to me how to replicate the content of one folder to another for a various purpose. This article also included logging option in Powershell, So we can follow, and see what is happening while the script is executing and copying the data or if the script runs into errors is very useful. Please follow the below article to know how can we achieve this using PowerShell.
Below cmdlets would be the input for the script. We need to define the source location, destination location and log file path for the script.
1 2 3 4 5 6 7 8 9 10 11 12 | ################################### # Source Location Details #------------------- $Source = "\\Server2012\ContentFolder" # Destination Details #------------------- $Destination = "D:\ContentFolder" # Log File Location #------------------- $Folderpath = "D:\FolderSync" |
We have to declare few variable such as Log file name, error handling variables etc.
1 2 3 4 5 6 7 8 9 | #################################### # Declaration of Variables $Date1 = Get-Date -UFormat "%m-%d-%y" $time = Get-Date -Format g $Logname = "SyncLog.log" $FilePath = "$Folderpath"+ "\$Logname" $Errormap = $null $errorcheck = '' $error.Clear() |
The next part will be Create\Modify the Log file based on the availability of log and based on the size of the log - If the log file size is more than 1 MB (We can modify the size) log file will be replaced with a new log file.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | ################################### # Setting Log File If ((Test-Path -Path $Folderpath)) { if ((Test-Path -Path $FilePath)) { $filesize = Get-ChildItem $FilePath ; if ($filesize.Length -gt 100kb ) {Remove-Item $filesize.FullName -Force; New-Item $logfolder\$Logname -ItemType file -Force} else {} } else { New-Item $FilePath -ItemType file -Force } } else { New-Item $Folderpath -Type directory -Force New-Item $FilePath -ItemType file -Force } |
Checking the source location is accessible or not from the destination location - The script execution will exit in case if the source not accessible.
1 2 3 4 5 6 7 8 9 10 11 | # Connection to Source If ((Test-Path -Path $Source)) { Add-Content -Path $FilePath -Value "$time Connection to $Source established Successfully`n" } else { Add-Content -Path $FilePath -Value "$time Error : Connection to $Source Failed - Script execution stopped`n" exit } |
The System.Security.Cryptography namespace provides cryptographic services, including secure encoding and decoding of data, as well as many other operations, such as hashing, random number generation, and message authentication. For more information. Refer the below link for more details.
https://msdn.microsoft.com/en-us/library/system.security.cryptography(v=vs.110).aspx
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | ################################## function Get-FileMD5 { $error.Clear() Param([string]$file) $mode = [System.IO.FileMode]("open") $access = [System.IO.FileAccess]("Read") $md5 = New-Object System.Security.Cryptography.MD5CryptoServiceProvider $fs = New-Object System.IO.FileStream($file,$mode,$access) $Hash = $md5.ComputeHash($fs) $fs.Close() [string]$Hash = $Hash Return $Hash } Add-Content -Path $FilePath -Value "$time Data Replication Started`n" |
Below part is replicating data and writing the respective logs into log file
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 | ################################## function Copy-LatestFile{ $error.Clear() Param($File1,$File2,[switch]$whatif) $File1Date = get-Item $File1 | foreach-Object{$_.LastWriteTimeUTC} $File2Date = get-Item $File2 | foreach-Object{$_.LastWriteTimeUTC} if($File1Date -gt $File2Date) { #Write-Host "$File1 is Newer... Copying..." -ForegroundColor Green Add-Content -Path $FilePath -Value "$time $File1 is Newer. Copying...`n" if($whatif){Copy-Item -path $File1 -dest $File2 -force -whatif} else{Copy-Item -path $File1 -dest $File2 -force} $errormap = $error[0] if($error[0] -eq $null) {} else {Add-Content -Path $FilePath -Value "$time Error $errormap"; $errorcheck = "1"} } } # Getting Files/Folders from Source and Destination $SrcEntries = Get-ChildItem $Source -Recurse $DesEntries = Get-ChildItem $Destination -Recurse # Parsing the folders and Files from Collections $Srcfolders = $SrcEntries | Where-Object{$_.PSIsContainer} $SrcFiles = $SrcEntries | Where-Object{!$_.PSIsContainer} $Desfolders = $DesEntries | Where-Object{$_.PSIsContainer} $DesFiles = $DesEntries | Where-Object{!$_.PSIsContainer} foreach($folder in $Srcfolders) { $error.Clear() $SrcFolderPath = $source -replace "\\","\\" -replace "\:","\:" $DesFolder = $folder.Fullname -replace $SrcFolderPath,$Destination if(!(test-path $DesFolder)) { Add-Content -Path $FilePath -Value "$time Folder $DesFolder Missing. Creating it!`n" new-Item $DesFolder -type Directory | out-Null $errormap = $error[0] if($error[0] -eq $null) {} else {Add-Content -Path $FilePath -Value "$time Error $errormap"; $errorcheck = "1"} } } foreach($entry in $SrcFiles) { $error.Clear() $SrcFullname = $entry.fullname $SrcName = $entry.Name $SrcFilePath = $Source -replace "\\","\\" -replace "\:","\:" $DesFile = $SrcFullname -replace $SrcFilePath,$Destination if(test-Path $Desfile) { $SrcMD5 = Get-FileMD5 $SrcFullname $DesMD5 = Get-FileMD5 $DesFile If(Compare-Object $srcMD5 $desMD5) { Add-Content -Path $FilePath -Value "$time The Files MD5's are Different... Checking Write`n" Copy-LatestFile $SrcFullname $DesFile $errormap = $error[0] if($error[0] -eq $null) {} else {Add-Content -Path $FilePath -Value "$time Error $errormap"; $errorcheck = "1"} } } else { $error.Clear() Add-Content -Path $FilePath -Value "$time $Desfile Missing... Copying from $SrcFullname`n" copy-Item -path $SrcFullName -dest $DesFile -force $errormap = $error[0] if($errormap -eq $null) {} else {Add-Content -Path $FilePath -Value "$time Error $errormap"; $errorcheck = "1"} } } # Checking for Files that are in the Destinatino, but not in Source foreach($entry in $DesFiles) { $error.Clear() $DesFullname = $entry.fullname $DesName = $entry.Name $DesFilePath = $Destination -replace "\\","\\" -replace "\:","\:" $SrcFile = $DesFullname -replace $DesFilePath,$Source if(!(Test-Path $SrcFile)) { Add-Content -Path $FilePath -Value "$time $SrcFile is deleted from Source... Deleting it from Destination`n" Remove-Item -path $DesFullname -Recurse -force $errormap = $error[0] if($error[0] -eq $null) {} else {Add-Content -Path $FilePath -Value "$time Error $errormap"; $errorcheck = "1"} } } $error.Clear() $Source = Get-ChildItem $Destination -Recurse $dirs = $Source | Where { (gci $_.fullName).count -eq 0 } | select -expandproperty FullName $dirs | Foreach-Object { Remove-Item $_ } if ($errorcheck -ne '1') { Add-Content -Path $FilePath -Value "$time Data Replication completed successfully`n"} else {Add-Content -Path $FilePath -Value "$time Data Replication completed with some error - Please check the log for details`n"} ################################### |
Combine all the script together and run.
Click here to download the script from Technet