How to Change a Disk Signature using Diskpart

Application: to potentially resolve issues with duplicate disk signature in a Windows Virtual Machine

Symptom:

Log Name:      System
Source: partmgr
Date: 8/29/2019 9:01:23 PM
Event ID: 58
Task Category: None
Level: Warning
Keywords: Classic
User: N/A
Computer: virtualmachine007
Description:
The disk signature of disk 4 is equal to the disk signature of disk 0.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="partmgr" />
<EventID Qualifiers="32772">58</EventID>
<Level>3</Level>
<Task>0</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2019-08-30T04:01:23.312782700Z" />
<EventRecordID>1847939</EventRecordID>
<Channel>System</Channel>
<Computer>virtualmachine007</Computer>
<Security />
</System>
<EventData>
<Data>
</Data>
<Data>4</Data>
<Data>0</Data>
<Binary>0000000003003000000000003A000480000000000000000000000000000000000000000000000000</Binary>
</EventData>
</Event>

1. Boot from the Windows install disc
2. Click on Repair your computer after selecting proper language, time and keyboard input.
3. Choose Command Prompt when the System Recovery Options box appears
4. Change into C:
5. Run diskpart

PS C:\> diskpart

Microsoft DiskPart version 6.3.9600

Copyright (C) 1999-2013 Microsoft Corporation.
On computer: TOR-FS01

DISKPART> list disk

Disk ### Status Size Free Dyn Gpt
-------- ------------- ------- ------- --- ---
Disk 0 Online 127 GB 0 B *
Disk 1 Reserved 10 TB 0 B *
Disk 2 Reserved 5120 MB 0 B *
Disk 3 Reserved 2048 GB 0 B *

DISKPART> select disk 0

Disk 0 is now the selected disk.

DISKPART> OFFLINE DISK
Diskpart successfully offlined the selected disk

DISKPART> uniqueid disk

Disk ID: {EA2ECF55-2B5C-41F6-A29D-1A1BC527FCF1}

DISKPART> UNIQUEID DISK ID=1234ABCD

DISKPART> ONLINE DISK
Diskpart successfully onlined the selected disk

DISKPART> LIST DISK

Disk ### Status Size Free Dyn Gpt
-------- ------------- ------- ------- --- ---
Disk 0 Online 127 GB 0 B *
Disk 1 Reserved 10 TB 0 B *
Disk 2 Reserved 5120 MB 0 B *
Disk 3 Reserved 2048 GB 0 B *

Some Useful Windows Commands to Troubleshoot Networking on Windoze

# Check this computer's trust relationship to its domain controllers
$domainName="intranet.kimconnect.com"
PS C:\Windows\system32> nltest /SC_QUERY:$domainName
Flags: 0
Trusted DC Name
Trusted DC Connection Status Status = 1311 0x51f ERROR_NO_LOGON_SERVERS
The command completed successfully

# Reset computer account password
$domainController="DC01.intranet.kimconnect.com"
$domainAdmin="domainAdmin"
netdom.exe resetpwd /s:$domainController /ud:$domainAdmin /pd:*

Output:
PS C:\Windows\system32> netdom.exe resetpwd /s:$domainController /ud:$domainAdmin /pd:*
Type the password associated with the domain user:
The machine account password for the local machine could not be reset.
The network path was not found.
The command failed to complete successfully.

# Check network interfaces
PS C:\Windows\system32> get-netadapter *
Name InterfaceDescription ifIndex Status MacAddress LinkSpeed
---- -------------------- ------- ------ ---------- ---------
DEV Cisco VIC Ethernet Interface #4 15 Up 00-25-B5 10 Gbps
iSCSI-A Cisco VIC Ethernet Interface #3 14 Up 00-25-B5 10 Gbps
CLUSTER Cisco VIC Ethernet Interface #2 8 Up 00-25-B5 10 Gbps
iSCSI-B Cisco VIC Ethernet Interface 2 Up 00-25-B5 10 Gbps

# Restart a NIC
PS C:\Windows\system32> get-netadapter -Name "PROD" | Restart-NetAdapter

# Overload a NIC with additional IP address(es)
$availableIP="x.x.x.x"
$cidrMask=24
$adapterName="DEV"
New-NetIPAddress –IPAddress $availableIP –PrefixLength $cidrMask –InterfaceAlias $adapterName –SkipAsSource $True

PS C:\Windows\system32> New-NetIPAddress –IPAddress $availableIP –PrefixLength $cidrMask –InterfaceAlias $adapterName –SkipAsSource $True
IPAddress : x.x.x.x
InterfaceIndex : 15
InterfaceAlias : PROD
AddressFamily : IPv4
Type : Unicast
PrefixLength : 24
PrefixOrigin : Manual
SuffixOrigin : Manual
AddressState : Tentative
ValidLifetime : Infinite ([TimeSpan]::MaxValue)
PreferredLifetime : Infinite ([TimeSpan]::MaxValue)
SkipAsSource : True
PolicyStore : ActiveStore

# Attempt to ping DNS1 from the new IP - failure means no ICMP outbound allowed
PS C:\Windows\system32> ping 1.1.1.1 -i x.x.x.x
Pinging 1.1.1.1 with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Ping statistics for 1.1.1.1:
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss)

# Set static route. Warning: don't do this unless you really know routing and switching!
$subnet="1.1.1.0"
$cidrMask=24
$ifIndex=15
$nextHop="1.1.1.1"
New-NetRoute -DestinationPrefix "$subnet`/$cidrMask" -InterfaceIndex $ifIndex -NextHop $nextHop
Get-NetRoute

PowerShell: Quickly Start Services that were Set to Auto

Quick Liners:
# Command to check
Get-WmiObject -ClassName Win32_Service -Filter "StartMode='Auto' AND State<>'Running'" | Format-Table -Auto DisplayName,Name,StartMode,State

# Command to start those non-running services
Get-WmiObject -ClassName Win32_Service -Filter "StartMode='Auto' AND State<>'Running'" | Start-Service
Sample Output:
[sherver007.something.loco]: PS C:\Users\176233\Documents> Get-WmiObject -ClassName Win32_Service -Filter "StartMode='Auto' AND State<>'Running'" | Format-Table -Auto DisplayName,Name,StartMode,State

DisplayName Name StartMode State
----------- ---- --------- -----
Downloaded Maps Manager MapsBroker Auto Stopped
Remote Registry RemoteRegistry Auto Stopped
Software Protection sppsvc Auto Stopped
Windows Biometric Service WbioSrvc Auto Stopped

[sherver007.something.loco]: PS C:\Users\176233\Documents> Get-WmiObject -ClassName Win32_Service -Filter "StartMode='Auto' AND State<>'Running'" | Start-Service
# output = silent

[sherver007.something.loco]: PS C:\Users\176233\Documents> Get-WmiObject -ClassName Win32_Service -Filter "StartMode='Auto' AND State<>'Running'" | Format-Table -Auto DisplayName,Name,StartMode,State
# output = de nata

PowerShell: Trigger Azure AD Sync on Remote Server as a Domain Admin

# Check whether a given username matches the list of Domain Admins
function validateDomainAdminMembership{
param (
[string]$username
)
$matchedAdmin=$username -in $domainAdmins
if($matchedAdmin){
Write-Host "$username is a Domain Admin";
return $True;
}else{
Write-Host "$username not a Domain Admin.";
return $False;
}
}

function testCredential{
param (
[string]$username,
[string]$password
)
$plaintextPassword = (New-Object System.Management.Automation.PSCredential 'N/A',$providedPassword).GetNetworkCredential().Password
$domainBindTest = (New-Object System.DirectoryServices.DirectoryEntry($domainObject,$username,$plaintextPassword)).DistinguishedName
if ($domainBindTest){return $True;} else{Return $False;}
}

function obtainDomainAdminCred{
$domainAdmins=(Get-ADGroupMember -Identity "Domain Admins" -Recursive | %{Get-ADUser -Identity $_.distinguishedName} | Where-Object {$_.Enabled -eq $True}).SamAccountName
$global:cred=$False
do {
$providedID=Read-Host -Prompt 'Input a domain admin username'
if (validateDomainAdminMembership $providedID){
$providedPassword = Read-Host -assecurestring "Please enter the password"
#$providedPassword = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto([System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($password))
#$providedCredential = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $providedID,$providedPassword
$goodCredential=testCredential -username $providedID -password $providedPassword
if($goodCredential){
"Domain Admin Credential validated!";
$global:cred=New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $providedID,$providedPassword;
#return $True;
}
else{
"Password doesn't match.";
$global:cred=$False;
#return $False;
}
}else{
"Try again..."
#return $False;
}
} until ($cred)
}

function validateCurrentAccountAsDomainAdmin{
if((whoami /groups) -match 'domain admins'){
"This account is a Domain Admins member";
return $True;
}else{"This account is NOT a Domain Admins member";return $False;}
}

If(!(validateCurrentAccountAsDomainAdmin)){obtainDomainAdminCred;}

# Query Active Directory to obtain the Azure AD Connect servername
$adConnectServers=(Get-ADUser -filter 'name -like "Msol*"' -Properties Description).Description | %{[void]($_ -match "computer\s'(.+)'\sconfigured");$matches[1]}
#if ($adConnectServers.GetType().Name -eq "String"){$serverName = $adConnectServers}else{$serverName = $adConnectServers[0]}

foreach ($servename in $adConnectServers){
"Invoking command on $servename`r`nStart-ADSyncSyncCycle -PolicyType Delta...";

$result=Invoke-Command -computername $serverName -scriptblock {
$value=Start-ADSyncSyncCycle -PolicyType Delta;
return $value.Result;
} -credential $cred

if ($result.Value -like 'Success'){break;}
}

"`r`nDone. Press Enter to close window."
pause;

Sample output:

Invoking command on ADKONNECTSHERVER01
Start-ADSyncSyncCycle -PolicyType Delta...

PSComputerName RunspaceId Result
-------------- ---------- ------
ADKONNECTSHERVER01 031af674-3bca-407c-bfab-d9ffb94123ab Success

PowerShell: How to Call a Batch File to Run-As Administrator

Step 1: Create a Batch File and place it inside C:\scripts

@echo off
powercfg.exe -x -monitor-timeout-ac 0
powercfg.exe -x -monitor-timeout-dc 0
powercfg.exe -x -disk-timeout-ac 0
powercfg.exe -x -disk-timeout-dc 0
powercfg.exe -x -standby-timeout-ac 0
powercfg.exe -x -standby-timeout-dc 0
powercfg.exe -x -hibernate-timeout-ac 0
powercfg.exe -x -hibernate-timeout-dc 0

Step 2: Create a PowerShell file in the same directory and place a shortcut on the Desktop

var startProcess = new System.Diagnostics.ProcessStartInfo();
        newProcessInfo.FileName = @"C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe";
        startProcess.WindowStyle = System.Diagnostics.ProcessWindowStyle.Hidden; # run in the background
        startProcess.Verb = "runas"; # run as [administrator]
        startProcess.Arguments = @"-executionpolicy unrestricted -Command ""C:\scripts\powercfg.bat -noexit"""; # set powershell switches to call the batch files to run unrestricted. Remove the -noexit item to close the window automatically after each run time
        System.Diagnostics.Process.Start(startProcess); # Trigger the start command to instantiate and bowow

PowerShell: Manually Create Failover Cluster SMB Shares

# Set three variables
$shareName="someShare"
$smbServer="someClusteredVServer"
$sharePath="S:\Shares\someShare"

# Generate Domain Admins label
$subdomain=(net config workstation) -match 'Workstation domain\s+\S+$' -replace '.+?(\S+)$','$1'
$domainadmins="$subdomain`\Domain Admins";

# Create directory and its parents if they do not exists
New-Item -ItemType Directory -Force -Path $sharePath;

# Set NTFS Permissions on folder
Add-NTFSAccess –Path $sharePath –Account $domainadmins –AccessRights Full -ErrorAction SilentlyContinue

# Create the share with associated share permissions
New-SmbShare -ScopeName $smbServer -Name $shareName -Path $sharePath -FullAccess $domainadmins -FolderEnumerationMode AccessBased

PowerShell: Grant Domain Admins Access to Directories

Snippet:
# Messing around
$shareDrives=@("E","F","G","I","J","N",'O','Q','R','S','T','U','V','W','Z','Y');
$subdomain=(net config workstation) -match 'Workstation domain\s+\S+$' -replace '.+?(\S+)$','$1';
$domainadmins="$subdomain`\Domain Admins";
$shareDrives | %{"Add-NTFSAccess –Path '$_`:\' –Account $domainadmins –AccessRights Full -ErrorAction SilentlyContinue";}

# The real deal (will take a long time to process)
# Limitations: a) it will not modify ACLs of root volumes b) will not modify items where "SYSTEM" or "Administrators" have no access
$shareDrives=@("E","F","G","I","J","N",'O','Q','R','S','T','U','V','W','Z','Y');
$subdomain=(net config workstation) -match 'Workstation domain\s+\S+$' -replace '.+?(\S+)$','$1';
$domainadmins="$subdomain`\Domain Admins";
$shareDrives | %{Add-NTFSAccess –Path "$_`:\Shares" –Account $domainadmins –AccessRights Full -ErrorAction SilentlyContinue;}
Sample Output:
Add-NTFSAccess –Path 'E:\' –Account WORKGROUP\Domain Admins –AccessRights Full -ErrorAction SilentlyContinue
Add-NTFSAccess –Path 'F:\' –Account WORKGROUP\Domain Admins –AccessRights Full -ErrorAction SilentlyContinue
Add-NTFSAccess –Path 'G:\' –Account WORKGROUP\Domain Admins –AccessRights Full -ErrorAction SilentlyContinue
Add-NTFSAccess –Path 'I:\' –Account WORKGROUP\Domain Admins –AccessRights Full -ErrorAction SilentlyContinue
Add-NTFSAccess –Path 'J:\' –Account WORKGROUP\Domain Admins –AccessRights Full -ErrorAction SilentlyContinue
Add-NTFSAccess –Path 'N:\' –Account WORKGROUP\Domain Admins –AccessRights Full -ErrorAction SilentlyContinue
Add-NTFSAccess –Path 'O:\' –Account WORKGROUP\Domain Admins –AccessRights Full -ErrorAction SilentlyContinue
Add-NTFSAccess –Path 'Q:\' –Account WORKGROUP\Domain Admins –AccessRights Full -ErrorAction SilentlyContinue
Add-NTFSAccess –Path 'R:\' –Account WORKGROUP\Domain Admins –AccessRights Full -ErrorAction SilentlyContinue
Add-NTFSAccess –Path 'S:\' –Account WORKGROUP\Domain Admins –AccessRights Full -ErrorAction SilentlyContinue
Add-NTFSAccess –Path 'T:\' –Account WORKGROUP\Domain Admins –AccessRights Full -ErrorAction SilentlyContinue
Add-NTFSAccess –Path 'U:\' –Account WORKGROUP\Domain Admins –AccessRights Full -ErrorAction SilentlyContinue
Add-NTFSAccess –Path 'V:\' –Account WORKGROUP\Domain Admins –AccessRights Full -ErrorAction SilentlyContinue
Add-NTFSAccess –Path 'W:\' –Account WORKGROUP\Domain Admins –AccessRights Full -ErrorAction SilentlyContinue
Add-NTFSAccess –Path 'Z:\' –Account WORKGROUP\Domain Admins –AccessRights Full -ErrorAction SilentlyContinue
Add-NTFSAccess –Path 'Y:\' –Account WORKGROUP\Domain Admins –AccessRights Full -ErrorAction SilentlyContinue

PowerShell: Microsoft Clustered Disks Creation Script v 0.10

Code:
<# Microsoft_Clustered_Disks_Creation_V0.10.ps1

Purpose:
This snippet streamlines the process of creating physical disk volumes on a Windows host and automatically add them into an existing Microsoft cluster

What this script does:
- Automatically formats a list of pre-defined disks with user-specified cluster sizes
- Assigns available drive letters as paths to newly created volumes
- Systematically adds new disks into a Microsoft Cluster as Physical Disks with correct labels

Limitations:
- Currently, this program cannot reliably sets unitialized disks as online or offline. Thus, prior to starting this script,
the admin must set disks as online MANUALLY; otherwise, such disks will be skipped
- This script doesn't fix weird problems at the LUN protocol level, as fixing that requires a re-creation of the LUN volumes.

Additional troubleshooting documentation:

using diskpart (manual):
SELECT DISK 2
ONLINE DISK
CLEAN
CREATE PARTITION PRIMARY
FORMAT FS=NTFS UNIT=16K QUICK
ASSIGN LETTER=B #Change B to any other available access-path letter
ACTIVE
EXIT

Add to Cluster (manual):
$numero=7
$etiqueta="LUN007"
Get-Disk -Number $numero | Add-ClusterDisk | %{$_.Name=$etiqueta }

Report a list of physical disks available on the host:
PS C:\Windows\system32> Get-WmiObject -Class win32_volume | select name,blocksize
name blocksize
---- ---------
C:\ 4096
E:\ 16384
F:\ 16384
G:\ 16384
I:\ 16384
J:\ 16384
N:\ 16384
\\?\Volume{3b1730ae-2559-40e3-a547-ae6a91d2d540}\ 4096

View all available disks:
Get-WmiObject -Class win32_volume | ?{$_.Name -match "\\\\?\\";}

PS C:\Windows\system32> Format-Volume -DriveLetter $driveLetter -FileSystem NTFS -AllocationUnitSize $sectorSize -NewFil
eSystemLabel $diskLabel -Confirm:$false -Force

DriveLetter FileSystemLabel FileSystem DriveType HealthStatus OperationalStatus SizeRemaining Size
----------- --------------- ---------- --------- ------------ ----------------- ------------- ----
F SIMONICULA-007 NTFS Fixed Healthy OK 12 TB 12 TB

Errors:

Add-ClusterDisk : The disk with Id {1}\\TestCluster007\root/Microsoft/Windows/Storage/Providers_v2\WSP_Disk.ObjectId="{4ef8972
8-9924-414d-960a-f09d2ab2155a}:DI:\\?\Disk{6fca7305-68a9-5804-84ed-bf26bd6bfc62}" is unable to be clustered.
At line:1 char:22
+ Get-Disk -Number 7 | Add-ClusterDisk | %{$_.name="PRDSHOME40-41" }
+ ~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (MSFT_Disk (Obje...Windows/Sto...):CimInstance) [Add-ClusterDisk], Invalid
OperationException
+ FullyQualifiedErrorId : Add-ClusterDisk,Microsoft.FailoverClusters.PowerShell.AddClusterDiskCommand

Resolution:
- Delete the LUNs at the SAN Volumes Manager
- Online disk > partition GPT > format disk as NTFS > assign drive letter > add disk to Cluster

Miscellaneous clustering commands:
# $clusterDiskObject=Get-ClusterResource -Cluster $cluster -Name $clusterDiskName
# Put disk in cluster maintenance mode
# $clusterDiskObject | Suspend-ClusterResource
# $clusterDiskObject | Stop-ClusterResource
<# avoid this error:
Format-Volume : The specified object is managed by the Microsoft Failover Clustering component. The disk must be in
cluster maintenance mode and the cluster resource status must be online to perform this operation.

#Clear-ClusterDiskReservation -Disk $diskNumber
#$clusterDiskObject | Start-ClusterResource | Resume-ClusterResource
# Put clustered disk in maintenance mode
# Get volume
# get-partition -DriveLetter C
#New-FileShare -Name $shareName -FileServerFriendlyName $serverName -SourceVolume $Volume -IsContinuouslyAvailable $True -Protocol SMB

# Set-Partition -DriveLetter $driveLetter -IsActive $true

# Create report after volumes have been created
Get-WmiObject -Class win32_volume | select name,blocksize

Error:
PS C:\Windows\system32> Set-Partition -DriveLetter $driveLetter -IsActive $true
Set-Partition : A parameter is not valid for this type of partition.

Extended information:
The parameters MbrType and IsActive cannot be used on a GPT disk.

Activity ID: {9b225291-2b3f-43e5-b62f-74a96d8e5dd6}
At line:1 char:1
+ Set-Partition -DriveLetter $driveLetter -IsActive $true
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (StorageWMI:ROOT/Microsoft/..._StorageCmdlets) [Set-Partition], CimExce
ption
+ FullyQualifiedErrorId : StorageWMI 41006,Set-Partition

Resolution:
The command only works with MBR disks - cannot issue it against at GTP type.
#>


################################## Excuting Program as an Administrator ####################################
# Get the ID and security principal of the current user account
$myWindowsID=[System.Security.Principal.WindowsIdentity]::GetCurrent()
$myWindowsPrincipal=new-object System.Security.Principal.WindowsPrincipal($myWindowsID)

# Get the security principal for the Administrator role
$adminRole=[System.Security.Principal.WindowsBuiltInRole]::Administrator

# Check to see if we are currently running "as Administrator"
if ($myWindowsPrincipal.IsInRole($adminRole))
{
# We are running "as Administrator" - so change the title and background color to indicate this
$Host.UI.RawUI.WindowTitle = $myInvocation.MyCommand.Definition + "(Elevated)"
$Host.UI.RawUI.BackgroundColor = "Black"
clear-host
}
else
{
# We are not running "as Administrator" - so relaunch as administrator

# Create a new process object that starts PowerShell
$newProcess = new-object System.Diagnostics.ProcessStartInfo "PowerShell";

# Specify the current script path and name as a parameter
$newProcess.Arguments = $myInvocation.MyCommand.Definition;

# Indicate that the process should be elevated
$newProcess.Verb = "runas";

# Start the new process
[System.Diagnostics.Process]::Start($newProcess);

# Exit from the current, unelevated, process
exit
}

Write-Host -NoNewLine "Running as Administrator..."
################################## Excuting Program as an Administrator ####################################

# Set global variables
$sectorSize=16384 #16K which translates to 64TB max per volume
$driveLettersExclusion="[ABCDHKLMPZ]"
$serverName="SHERVER007"
$cluster="CLUSTER007"

# Set cluster disk label and disk number
$clusterDisks=@(
@("DISK-008",8),
@("DISK-009",9),
@("DISK-010",10),
@("DISK-011",11),
@("DISK-012",12),
@("DISK-013",13),
@("DISK-014",14),
@("DISK-015",15)
)

# Scan for next available drive letter, excluding D "CD Rom" and H "Home"
$availableDriveLetters=ls function:[A-Z]: -n|?{!(test-path $_)}|%{$_[0]}|?{!($_ -match $driveLettersExclusion)}

function createVolume($name,$number,$letter){
# Set variables
$diskNumber=$number
$diskLabel=$name
$driveLetter=$letter
$drivePath=$driveLetter+":"
$tempDriveLetter=$availableDriveLetters[$availableDriveLetters.Length -1]
$tempPath=$tempDriveLetter+":"

"Processing disk number $diskNumber`: set drive letter as $driveLetter & label equals $diskLabel"

# specify disk object
# $disk = Get-Disk -Number $diskNumber

# Set disk as online
Set-Disk $diskNumber -IsOffline $false -InformationAction SilentlyContinue # This PowerShell command doesn't work

# Clean disk
Clear-Disk -Number $diskNumber -RemoveData -Confirm:$False

# Set partition as GPT
Initialize-Disk -Number $diskNumber -PartitionStyle GPT -InformationAction SilentlyContinue

# Create a new partition and format it
#Add-PartitionAccessPath -DiskNumber $diskNumber -PartitionNumber 2 -AccessPath $drivePath
New-Partition $diskNumber -UseMaximumSize -DriveLetter $tempDriveLetter
Format-Volume -DriveLetter $tempDriveLetter -FileSystem NTFS -AllocationUnitSize $sectorSize -NewFileSystemLabel $diskLabel -Confirm:$false -Force
Set-Disk $diskNumber -isOffline $false
Set-Disk $diskNumber -isReadOnly $false

# Use WMI to relabel volumes as PowerShell currently doesn't have this capability
$disk = Get-WmiObject -Class win32_volume -Filter "Label = '$diskLabel'"
Set-WmiInstance -input $disk -Arguments @{DriveLetter=$drivePath; Label="$diskLabel"}

Remove-PartitionAccessPath -DiskNumber $diskNumber -PartitionNumber 2 -Accesspath $tempPath
}

function addDiskToCluster($name,$number){
Get-Disk -Number $number | Add-ClusterDisk | %{$_.name=$name }
}

function processDisks{

$i=0;
foreach ($disk in $clusterDisks){
createVolume $($disk[0]) $($disk[1]) $availableDriveLetters[$i++];
#pause;
}

"Please verify that all disks have been created prior to proceeding to add disks to Clusters."
pause;

$clusterDisks | %{
"Please verify the accuracy of this statement and press Enter to accept:`r`n"
"Name: $($_[0]) | Localhost Physical Disk Number: $($_[1])";
#pause;
addDiskToCluster $($_[0]) $($_[1]);
}
}

processDisks;
Sample Output:
Running as Administrator...Processing disk number 8: set drive letter as Q & label equals DISK-008

DiskPath: \\?\Disk{13efa5dc-bfb7-4267-4b38-a7bfc5755a61}

PartitionNumber DriveLetter Offset Size Type
--------------- ----------- ------ ---- ----
2 Y 135266304 74.87 GB Basic

ObjectId : {1}\\PRDFCL2\root/Microsoft/Windows/Storage/Providers_v2\WSP_Volume.ObjectId="{4ef89728-9924-414d-960a-f09d2ab2155a}:VO:\\?\V
olume{6b96c857-e7e7-41d9-ad49-c83be3c7ffcc}\"
PassThroughClass :
PassThroughIds :
PassThroughNamespace :
PassThroughServer :
UniqueId : \\?\Volume{6b96c857-e7e7-41d9-ad49-c83be3c7ffcc}\
AllocationUnitSize : 16384
DedupMode : NotAvailable
DriveLetter : Y
DriveType : Fixed
FileSystem : NTFS
FileSystemLabel : DISK-008
FileSystemType : NTFS
HealthStatus : Healthy
OperationalStatus : OK
Path : \\?\Volume{6b96c857-e7e7-41d9-ad49-c83be3c7ffcc}\
Size : 80394305536
SizeRemaining : 79481651200
PSComputerName :


__GENUS : 2
__CLASS : Win32_Volume
__SUPERCLASS : CIM_StorageVolume
__DYNASTY : CIM_ManagedSystemElement
__RELPATH : Win32_Volume.DeviceID="\\\\?\\Volume{6b96c857-e7e7-41d9-ad49-c83be3c7ffcc}\\"
__PROPERTY_COUNT : 44
__DERIVATION : {CIM_StorageVolume, CIM_StorageExtent, CIM_LogicalDevice, CIM_LogicalElement...}
__SERVER : SHERVER007
__NAMESPACE : root\cimv2
__PATH : \\SHERVER007\root\cimv2:Win32_Volume.DeviceID="\\\\?\\Volume{6b96c857-e7e7-41d9-ad49-c83be3c7ffcc}\\"
Access :
Automount : True
Availability :
BlockSize : 16384
BootVolume : False
Capacity : 80394305536
Caption : Q:\
Compressed : False
ConfigManagerErrorCode :
ConfigManagerUserConfig :
CreationClassName :
Description :
DeviceID : \\?\Volume{6b96c857-e7e7-41d9-ad49-c83be3c7ffcc}\
DirtyBitSet : False
DriveLetter : Q:
DriveType : 3
ErrorCleared :
ErrorDescription :
ErrorMethodology :
FileSystem : NTFS
FreeSpace : 79481667584
IndexingEnabled : True
InstallDate :
Label : DISK-008
LastErrorCode :
MaximumFileNameLength : 255
Name : Q:\
NumberOfBlocks :
PageFilePresent : False
PNPDeviceID :
PowerManagementCapabilities :
PowerManagementSupported :
Purpose :
QuotasEnabled : False
QuotasIncomplete : False
QuotasRebuilding : False
SerialNumber : 2190170871
Status :
StatusInfo :
SupportsDiskQuotas : True
SupportsFileBasedCompression : False
SystemCreationClassName :
SystemName : SHERVER007
SystemVolume : False
PSComputerName : SHERVER007

Press Enter to continue...:

### Truncated for brevity ###

Please verify that all disks have been created prior to proceeding to add disks to Clusters.
Press Enter to continue...:
Please verify the accuracy of this statement and press Enter to accept:

Name: DISK-008 | Localhost Physical Disk Number: 8
Press Enter to continue...:
Please verify the accuracy of this statement and press Enter to accept:

### Truncated for brevity ###

Name: DISK-015 | Localhost Physical Disk Number: 15
Press Enter to continue...:

PowerShell: Deploy Certs on Remote Windows Servers

# Purpose: manually deploy certificates onto remote servers at the "Personal" Certificates store

# Set variables
$sourceCert="\\FILESHERVER01\SOMECERT.pfx "
$certPassword=ConvertTo-SecureString "CERT_PASSWORT" -AsPlainText -Force
$servers="SHERVER01","SHERVER02"

# Function to copy cert to remote servers prior to accessing WinRM to apply them
function copyCertsToServers{
$servers |%{Copy-Item $sourceCert -Destination "\\$_`\c$"}
}
copyCertsToServers;

# Apply certs on remote machines
$servers | %{ Invoke-Command -ComputerName $_ -ScriptBlock {
param($x)
$env:computername;
Import-PfxCertificate -CertStoreLocation Cert:\LocalMachine\My -FilePath "C:\WildCard.pfx" -Password $x;
} -ArgumentList $certPassword
}

PowerShell: Microsoft Failover Cluster Discovery Version 0.10

<#  .Sypnosis: Microsoft_Clusters_Discovery_v.0.10.ps1
What it does:
- Installs Microsoft Failover Cluster PowerShell module on the local machine (Jump Box).
This can work behind a proxy if those variables are specified.
Be advised that RSAT-Clustering-PowerShell currently has known conflicts with VMWare.PowerCLI.
Thus, this must be ran on machines that does not already have VMWare.PowerCLI installed.
- Collects all FailOver Clusters in the current domain
- Lists all nodes of a selected cluster
- Checks whether Remote PowerShell is enabled on a selected node
- Connects to a user-selected node via WinRM and queries for clustering information: resource names and current owners

Requirements:
- Script must be ran using a Domain Admin account
- It can only query remote machines with WinRM being enabled
- Requires a manual trigger of PowerShell Execution Policy: Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Confirm:$false -Force

Future development:
- Runs as a Domain Admin
- Uses PSExec to enable WinRM on remote machines prior to executing Remote Powershell invoke-command
- Describes the clustering types and trigger selected functions to match the type of resource (SQL, Hyper-V, FileServer, etc.)
#>

$proxy="http://proxy:8080"
$exclusionList="localhost;*.sdcs.local"

<# Future development: Domain Admin credential prompts
# Credential for PS-Remoting
$usernameInput=Read-Host -Prompt "Input the username";
$username="$($env:USERDNSDOMAIN)\$usernameInput";
$password = Read-Host -Prompt "Input the password for account $username" -AsSecureString
#$password=convertto-securestring "PASSWORD" -AsPlainText -Force
$cred = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $username,$password
#>

################################## Excuting Program as an Administrator ####################################
# Get the ID and security principal of the current user account
$myWindowsID=[System.Security.Principal.WindowsIdentity]::GetCurrent()
$myWindowsPrincipal=new-object System.Security.Principal.WindowsPrincipal($myWindowsID)

# Get the security principal for the Administrator role
$adminRole=[System.Security.Principal.WindowsBuiltInRole]::Administrator

# Check to see if we are currently running "as Administrator"
if ($myWindowsPrincipal.IsInRole($adminRole))
{
# We are running "as Administrator" - so change the title and background color to indicate this
$Host.UI.RawUI.WindowTitle = $myInvocation.MyCommand.Definition + "(Elevated)"
$Host.UI.RawUI.BackgroundColor = "Black"
clear-host
}
else
{
# We are not running "as Administrator" - so relaunch as administrator

# Create a new process object that starts PowerShell
$newProcess = new-object System.Diagnostics.ProcessStartInfo "PowerShell";

# Specify the current script path and name as a parameter
$newProcess.Arguments = $myInvocation.MyCommand.Definition;

# Indicate that the process should be elevated
$newProcess.Verb = "runas";

# Start the new process
[System.Diagnostics.Process]::Start($newProcess);

# Exit from the current, unelevated, process
exit
}

Write-Host -NoNewLine "Running as Administrator..."
################################## Excuting Program as an Administrator ####################################

function checkProxy{
try{
$connectionTest=iwr download.microsoft.com -UseBasicParsing
#$connectionSucceeds=Test-NetConnection -Computername download.microsoft.com -Port 443 -InformationLevel Quiet
if ($connectionTest){
return $True;
}
}
catch{
return $False
}
}

function fixProxy{
# Check if proxy is enabled on the system and fix it
$proxyKey=(Get-ItemProperty -Path "Registry::HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings")
if ($proxyKey.ProxyEnable){
# Set http proxy for browsers
Set-Itemproperty -path "Registry::HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings" -Name 'ProxyServer' -value $proxy

# Set winhttp proxy for PowerShell
netsh winhttp set proxy $proxy $exclusionList

[system.net.webrequest]::defaultwebproxy = New-Object system.net.webproxy($proxy)
[system.net.webrequest]::defaultwebproxy.credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials
[system.net.webrequest]::defaultwebproxy.BypassProxyOnLocal = $true
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
}

if (checkProxy){
"Proxy is now good to go..."
}
else{
"Proxy problems..."
break;
}
}

# Set PowerShell Gallery as Trusted to bypass prompts
Function setPSGalleryTrust{
$trustPSGallery=(Get-psrepository -Name 'PSGallery').InstallationPolicy
If($trustPSGallery -ne 'Trusted'){
Set-PSRepository -Name 'PSGallery' -InstallationPolicy Trusted
}
}

# Adding Prerequisite Active Directory Module
Function installADModule{
if (!(get-module -name "ActiveDirectory") ){
Add-WindowsFeature RSAT-AD-PowerShell | out-null;
import-module -name "ActiveDirectory" -DisableNameChecking | out-null;
}
$domain=get-addomain
}

# Adding Prerequisite Microsoft Cluster
Function installFailoverClustersModule{
if (!(get-module -Name "FailoverClusters") ){
#Install-WindowsFeature Failover-Clustering | out-null;
Install-WindowsFeature RSAT-Clustering-MGMT | out-null;
Install-WindowsFeature RSAT-Clustering-PowerShell | out-null;
Import-Module FailoverClusters | out-null;
}
}

Function installPrerequisites{
setPSGalleryTrust;
#installADModule;
installFailoverClustersModule;
}

function listClusters{
"`n`nListing all Clusters in $($env:USERDNSDOMAIN)...`n"
$GLOBAL:clusters=get-cluster -domain $env:USERDNSDOMAIN
}

function listNodes{
"`n`nAttempting to collect a list of all Nodes in this Cluster...`n"
try{
$global:nodes=(Get-ClusterNode -Cluster "$pickedItem.$env:USERDNSDOMAIN").Name
$global:pickedCluster=$pickedItem
}
catch{
$_.Exception.Message;
}
#$global:headLine="Available nodes in $pickedItem`:"
}

function showMenu{
Param($callFunction)
invoke-command (Get-Item "function:$callFunction").ScriptBlock

switch($callFunction){
"listClusters"{$count=$clusters.count; $global:list=$clusters;}
"listNodes" {$count=$nodes.count; $global:list=$nodes;}
}

$show="`n--------------------------------------------------------`nThere are $count items on this list:`n--------------------------------------------------------`n"
for ($row=0;$row -le $count-1;$row++){
$server=$list[$row]
$show += "$row" + ": " + "$server" + "`n"
}
return $show
}

function pickItem{
do {
try {
$flag = $true
[int]$pick=Read-Host -Prompt "`n---------------------------------------`nPlease pick an item from the above list`n---------------------------------------"
} # end try
catch {$flag = $false}
} # end do
until ($pick -lt $list.count -and $flag)

$global:pickIndex=$pick
$global:pickedItem=$list[$pick]
$clusters=""
$nodes=""
"$pick`: $pickedItem is picked"
}

# return true or false as determinant on whether program proceeds
function checkNodesForWinRM{
$nodes | foreach {
$winRM=Test-NetConnection -ComputerName "$_`.$($env:USERDNSDOMAIN)" -CommonTCPPort WINRM -InformationLevel Quiet
if ($winRM){"$_ is reachable via WinRM.";}
}
}

function queryFailoverClusters{

# Pick Cluster
do {
try{
$flag1=$true
showMenu -callFunction listClusters
pickItem
}
catch{$flag1=$false}
} until ($flag1)

# Pick Node
do {
try{
$flag2=$true
showMenu -callFunction listNodes
if (!($nodes)){queryFailoverClusters;}
checkNodesForWinRM;
pickItem
}
catch{$flag2=$false}
} until($flag2)
}

$failoverClusterModuleAvailable=get-cluster
if (!($failoverClusterModuleAvailable)){
if(!(checkProxy)){"Internet issues detected. Attempting a fix now..."; fixProxy;}
installPrerequisites;
}

function failoverClusterDiscovery{
queryFailoverClusters;

$pickedItemWinRM=Test-NetConnection -ComputerName "$pickedItem`.$($env:USERDNSDOMAIN)" -CommonTCPPort WINRM -InformationLevel Quiet
if($pickedItemWinRM){
"Obtaining Cluster Resource from node $pickedItem..."

# Triggering invoke-command as multi-threaded jobs so that the program will wait until these jobs are done before moving to the Pause command
$jobs = Invoke-Command -ComputerName $pickedItem -ScriptBlock {Get-ClusterResource | format-list Name,OwnerNode;} -AsJob
While ($Jobs.State -Contains "Running"){
$Jobs | Wait-Job -Any;
if ($jobs.State -contains "Completed") {
Receive-Job -Job $jobs;
Remove-Job -Job $jobs;
}
}

#Invoke-Command -ComputerName $pickedItem -ScriptBlock{Get-ClusterResource;}
}else{
"$pickedItem is not reachable via Remote PowerShell to query Cluster Resources.";
pause;
}
}

do {
failoverClusterDiscovery;
[string]$response=Read-Host -Prompt "`nPress any key to repeat process. Type 'exit' or use Ctrl+C to terminate program"
} until ($response -like "exit")

Disk Partitioning & Formatting Reference

Refer to this table as a reference on sector/cluster size leading to maximum storage per LUN:
 

Cluster size

NTFS Max Size

  512 bytes             

  2,199,023,255,040 (2TB)

 1024 bytes

  4,398,046,510,080 (4TB)

 2048 bytes

  8,796,093,020,160 (8TB)

 4096 bytes

 17,592,186,040,320 (16TB) Default cluster size

 8192 bytes

 35,184,372,080,640 (32TB)

16384 bytes  

 70,368,744,161,280 (64TB)

32768 bytes

140,737,488,322,560 (128TB)

65536 bytes            

281,474,976,654,120 (256TB)

On Partitioning, the caveat is that we cannot use MBR as its limitation is (2^32)-1 sectors * 512 bytes/cluster = 2,199,023,255,040 bytes or 2TB. Instead, we use GPT (Guid Partition Table). GPT uses 64 bits for logical block addresses, allowing a maximum disk size of 264 sectors. For disks with 512-byte sectors, the maximum size is 9.4 ZB (9.4 × 10²¹ bytes) or 8 ZiB (264 sectors × 29bytes per sector). Booting on a GPT disk may require UEFI and 64bit OS. If such disk is dedicated as storage, then GTP trumps MBR.
 
On Formatting, “NTFS maximum theoretical limit on the size of individual files is 16 EiB (16 × 10246 or 264 bytes) minus 1 KB, which totals 18,446,744,073,709,550,592 bytes. With Windows 10 version 1709 and Windows Server 2019, the maximum implemented file size is 8 PB minus 2 MB or 9,007,199,252,643,840 bytes.”  Please note that Windows 2016 Server are having these limitations, also. (https://en.wikipedia.org/wiki/NTFS)
 
Reference table from http://www.ntfs.com/ntfs_vs_fat.htm:
 

Criteria

NTFS5

NTFS

exFAT

FAT32

Operating System Windows 2000
Windows XP
Windows 2003 Server
Windows 2008
Windows Vista
Windows 7
Windows NT
Windows 2000
Windows XP
Windows 2003 Server
Windows 2008Windows Vista
Windows 7
Windows CE 6.0
Windows Vista SP1
Windows 7
WinXP+KB955704
DOS v7 and higher
Windows 98
Windows ME
Windows 2000
Windows XP
Windows 2003 Server
Windows Vista
Windows 7
 
Max Volume Size 2 ^ 64 clusters – 1 cluster 2 ^ 32 clusters – 1 cluster 128PB 32GB for all OS. 2TB for some OS
Max Files on Volume 4,294,967,295
2 ^ 32 -1
4,294,967,295
2 ^ 32 -1
Nearly Unlimited 4194304
Max File Size 2 ^ 64 bytes 
(16 ExaBytes) minus 1KB
2 ^ 44 bytes 
(16 TeraBytes) minus 64KB
16EB 4GB minus 2 Bytes
Max Clusters Number 2 ^ 64 clusters – 1 cluster 2 ^ 32 clusters – 1 cluster 4294967295 4177918
Max File Name Length Up to 255 Up to 255 Up to 255 Up to 255

Exchange Email Sending Status code: 550 5.7.133

Error Message:

The group <group name> only accepts messages from people in its organization or on its allowed senders list, and your email address isn’t on the list.

More Info for Email Admins
Status code: 550 5.7.133

This error occurs when the distribution group, security group, or Office 365 group is configured to accept messages only from authenticated senders (senders in the same organization or those added to the group's allowed senders list).

To fix the issue, the recipient's email admin or the group owner must add the sender's email address to the group's allowed senders list or change the group's delivery management setting to accept messages from senders inside and outside of the organization.

Usually this issue can only be fixed by the recipient's email admin or the group owner.

For more information and steps to fix this error, see Fix email delivery issues for error code 5.7.133 in Office 365.

Resolution:

Assuming that the Office365 to On-Premise Exchange connect have been validated and functional, this command would fix the problem.

$groupName="<group name>@kimconnect.com"
Set-DistributionGroup $groupName -RequireSenderAuthenticationEnabled $false

VMware: How To Mount a USB Thumb Drive as a Data Store

Although USB is not a recommended data store type, it is still possible to mount this type of storage in VMWare. It should also be noted that VMWare will not provide support for this work around. Caveat emptor to those who choose to follow my mickey mouse methods. Also, one must be advised that a change in “USB Arbitrator” setting will affect all guests on such particular host.

Some benefits of using USB media are for storing: ISOs, infrequently used bulk files, and backup repositories. This is to save precious SAN volumes for other redundancy required production workloads.

# Get a list of existing disks

[rambo@testbox:~] ls /dev/disks/
mpx.vmhba33:C0:T0:L0
mpx.vmhba33:C0:T0:L0:1
mpx.vmhba33:C0:T0:L0:5
mpx.vmhba33:C0:T0:L0:6
mpx.vmhba33:C0:T0:L0:7
mpx.vmhba33:C0:T0:L0:8
mpx.vmhba33:C0:T0:L0:9
t10.ATA_____Samsung_SSD_850_PRO_512GB_______________S250NX0H835192J_____
t10.ATA_____Samsung_SSD_850_PRO_512GB_______________S250NX0H835192J_____:1
vml.0000000000766d68626133333a303a30
vml.0000000000766d68626133333a303a30:1
vml.0000000000766d68626133333a303a30:5
vml.0000000000766d68626133333a303a30:6
vml.0000000000766d68626133333a303a30:7
vml.0000000000766d68626133333a303a30:8
vml.0000000000766d68626133333a303a30:9
vml.0100000000533235304e5830483833353139324a202020202053616d73756e
vml.0100000000533235304e5830483833353139324a202020202053616d73756e:1

# Stop USB Arbitrator (pass-through) service
/etc/init.d/usbarbitrator stop

# Disable USB Arbitrator
chkconfig usbarbitrator off

# Plugin USB thumb drive

# Run list disks command again to locate the newly detected item
[rambo@testbox:~] ls /dev/disks/
mpx.vmhba32:C0:T0:L0
mpx.vmhba32:C0:T0:L0:1
mpx.vmhba33:C0:T0:L0
mpx.vmhba33:C0:T0:L0:1
mpx.vmhba33:C0:T0:L0:5
mpx.vmhba33:C0:T0:L0:6
mpx.vmhba33:C0:T0:L0:7
mpx.vmhba33:C0:T0:L0:8
mpx.vmhba33:C0:T0:L0:9
t10.ATA_____Samsung_SSD_850_PRO_512GB_______________S250NX0H835192J_____
t10.ATA_____Samsung_SSD_850_PRO_512GB_______________S250NX0H835192J_____:1
vml.0000000000766d68626133323a303a30
vml.0000000000766d68626133323a303a30:1
vml.0000000000766d68626133333a303a30
vml.0000000000766d68626133333a303a30:1
vml.0000000000766d68626133333a303a30:5
vml.0000000000766d68626133333a303a30:6
vml.0000000000766d68626133333a303a30:7
vml.0000000000766d68626133333a303a30:8
vml.0000000000766d68626133333a303a30:9
vml.0100000000533235304e5830483833353139324a202020202053616d73756e
vml.0100000000533235304e5830483833353139324a202020202053616d73756e:1

# Make partition table for new disk
partedUtil mklabel /dev/disks/vml.0000000000766d68626133323a303a30 gpt

# Find end sector
partedUtil getptbl /dev/disks/vml.0000000000766d68626133323a303a30

[rambo@testbox:~] partedUtil getptbl /dev/disks/vml.0000000000766d68626133323a303a30
Error: The primary GPT table on '/dev/disks/mpx.vmhba32:C0:T0:L0' is OK, but secondary is corrupt. Fix secondary table? This will move secondary at the end in case it is not at the end already. It will also set LastUsableLBA to use all the space at the end. diskSize (1048576000) AlternateLBA (1048575999) LastUsableLBA (1048575966)
gpt
65270 255 63 1048576000

# Calculate
# 65270 * 255 * 63 - 1 = 1048562549

# Create VMFS partition
# The start sector is always 2048
# The GUID for VMFS is AA31E02A400F11DB9590000C2911D1B8
partedUtil setptbl /dev/disks/vml.0000000000766d68626133323a303a30 gpt "1 2048 1048562549 AA31E02A400F11DB9590000C2911D1B8 0"

# An example of command being successful
[rambo@testbox:~] partedUtil setptbl /dev/disks/vml.01000000003443353330303030323430
343230313035353335556c74726120 gpt "1 2048 240252074 AA31E02A400F11DB9590000C291
1D1B8 0"
gpt
0 0 0 0
1 2048 240252074 AA31E02A400F11DB9590000C2911D1B8 0

# Run this command if the existing gpt disk has errors
[rambo@testbox:~] partedUtil fixGpt /dev/disks/vml.0000000000766d68626133323a303a30
FixGpt tries to fix any problems detected in GPT table.
Please ensure that you don't run this on any RDM (Raw Device Mapping) disk.
Are you sure you want to continue (Y/N): Y
Error: The primary GPT table on '/dev/disks/mpx.vmhba32:C0:T0:L0' is OK, but secondary is corrupt. Fix secondary table? This will move secondary at the end in case it is not at the end already. It will also set LastUsableLBA to use all the space at the end. diskSize (1048576000) AlternateLBA (1048575999) LastUsableLBA (1048575966)
Fix/Ignore/Cancel? Fix
gpt
65270 255 63 1048576000
1 2048 1048562549 AA31E02A400F11DB9590000C2911D1B8 vmfs 0

# Format partition with vmfs6
vmkfstools -C vmfs6 -S USB-Datastore1 /dev/disks/vml.0000000000766d68626133323a303a30:1

# Example of successful command
[rambo@testbox:~] vmkfstools -C vmfs6 -S USB-Datastore /dev/disks/vml.01000000003443
353330303030323430343230313035353335556c74726120:1
create fs deviceName:'/dev/disks/vml.01000000003443353330303030323430343230313035353335556c74726120:1', fsShortName:'vmfs6', fsName:'USB-Datastore'
deviceFullPath:/dev/disks/t10.SanDisk00Ultra00000000000000000000004C530000240420105535:1 deviceFile:t10.SanDisk00Ultra00000000000000000000004C530000240420105535:1
ATS on device /dev/disks/t10.SanDisk00Ultra00000000000000000000004C530000240420105535:1: not supported
.
Checking if remote hosts are using this device as a valid file system. This may take a few seconds...
Creating vmfs6 file system on "t10.SanDisk00Ultra00000000000000000000004C530000240420105535:1" with blockSize 1048576, unmapGranularity 1048576, unmapPriority default and volume label "USB-Datastore".
Successfully created new volume: 5d5ab29d-2aa0b031-7cbf-94c691abb6e2

# Check mounts
esxcfg-scsidevs -m

[rambo@testbox:~] esxcfg-scsidevs -m
t10.ATA_____Samsung_SSD_850_PRO_512GB_______________S250NX0H835192J_____:1 /vmfs/devices/disks/t10.ATA_____Samsung_SSD_850_PRO_512GB_______________S250NX0H835192J_____:1 5c8206f0-49fe1606-76b6-94c691abb6e2 0 SamsungSSD
mpx.vmhba32:C0:T0:L0:1 /vmfs/devices/disks/mpx.vmhba32:C0:T0:L0:1 5d59f580-2570f44c-5b05-94c691abb6e2 0 USB-Datastore1

[root@esx1:~] esxcfg-scsidevs -m
t10.ATA_____Samsung_SSD_850_PRO_512GB_______________S250NX0H835192J_____:1 /vmfs/devices/disks/t10.ATA_____Samsung_SSD_850_PRO_512GB_______________S250NX0H835192J_____:1 5c8206f0-49fe1606-76b6-94c691abb6e2 0 SamsungSSD
t10.SanDisk00Ultra00000000000000000000004C530000240420105535:1 /vmfs/devices/disks/t10.SanDisk00Ultra00000000000000000000004C530000240420105535:1 5d5ab29d-2aa0b031-7cbf-94c691abb6e2 0 USB-Datastore

Windows File System Symlink Settings

Issue: Shortcuts or symbolic links are inaccessible at the destination, after being synchronized (using robocopy, xcopy, emcopy, etc.)

---------------------------
Open Folder
---------------------------
\\FILESHERVER01\Users\tester\Music is not accessible. You might not have permission to use this network resource. Contact the administrator of this server to find out if you have access permissions.

The symbolic link cannot be followed because its type is disabled.
---------------------------
OK
---------------------------

Check local file system settings on Symlink behavior:

PS C:\Users\admindude> fsutil behavior query SymlinkEvaluation
Local to local symbolic links are enabled
Local to remote symbolic links are enabled.
Remote to local symbolic links are disabled.
Remote to Remote symbolic links are disabled.

Enable Remote to remote symlink following option

fsutil behavior set SymlinkEvaluation R2R:1

PowerShell: Scheduled Task to Backup Local Files to remote UNC Path

Set Script Execution Policy on Host
PS H:\> Set-ExecutionPolicy RemoteSigned

Execution Policy Change
The execution policy helps protect you from scripts that you do not trust. Changing the execution policy might expose
you to the security risks described in the about_Execution_Policies help topic. Do you want to change the execution
policy?
[Y] Yes [N] No [S] Suspend [?] Help (default is "Y"): Y
Create a Windows Scheduled task with this setting
Create a C:\Scripts\File_Copy_Script_V0.15.ps1 file with this content:
<#
.Description File_Copy_Script Version: 0.15

Purpose: this PowerShell Script is to efficiently mirror large batches of files using Emcopy in conjunction with Volume Shadow Services

Current Features:
1. Check for any errors on the Sources or Destinations and generate a report of any extra spaces in UNC
2. Create a snapshot of the source volume using Shadow Copy to capture any locked files
3. Execute robocopy to mirror the source to its corresponding destination with a time stamp variance allowance of 2 seconds for speed and resiliency
4. Sample the copied files to 'spot check' any time stamp variances
5. Execute in the context of an Administrator

Features planned for development:
6. Enable Volume Shadow Copy (VSS) at Source machines if it has been disabled, and reverse the action when done with copying
7. Trigger Remote Powershell to launch execution from a middle server (a "jump box" that is not a source nor destination)
if the provided Source is detected as a Universal Naming Convention (UNC) path instead of a local file system (LFS) path

Limitations:
1. This iteration requires that script is triggered from a local Windows machine with Internet access (no proxies)
2. Source must be LFS and Destination could either be LFS or UNC
#>

# Specify Source and Destination
$source="C:\Users\brucelee\Desktop\Clients" # Must be a LFS path
$destination="C:\Users\brucelee\Desktop\Test"
$block="$source $destination"

# Emcopy switches
$switches="/o /secforce /de /sd /c /r:0 /w:0 /th 128 /s /purge /sdd /stream"
<# Switch explanations
/s copies sub directories
/purge removes files and directories from the destination that do not exist in the source.
/sdd forces the target directories dates to be synchronized with the source directory.
/de Compares both file size and last modification time when deciding to update a file, updates it if either have been changed.
/cm md5 - checks the file content after copying using and md5 comparison of the source and destination.
/o copies the files owner, without this the account used for the copy will be the owner
/secforce overwrites the destination security settings with the source security settings (no merging, actual update of security settings at the destination)
/sd preserves security, the file isn't copied if an error occurs during security settings.
/th 128 - Uses 128 threads, default is 64
/r:0 retries zero times
/w:0 is the wait time in seconds between retries
/c will allow the process to continue after the retries
/log:filename option allows to redirect the console messages to a new file.
/log+:filename option appends the new messages to an existing file.
/stream option enables the copie of files and directories datastreams. Without that option only the main datastream of files are copied.
#>

# Initialize log files
$dateStamp = Get-Date -Format "yyyy-MM-dd-hhmmss"
$scriptName=$MyInvocation.MyCommand.Path
$scriptPath=Split-Path -Path $scriptName
$logPath="$scriptPath\emcopy_logs"
$logFile="$logPath\emcopy-log-$dateStamp.txt"
$log=" /LOG+:$logFile"
$lockedFilesReport="$logPath\_locked-files-log-$dateStamp.txt"
$pathErrorsLog="$logPath`\_path-errors-log-$dateStamp.txt"

# Init other variables
$sampleSize=1000;
$GLOBAL:shadowMount="C:\shadowcopy"

################################## Excuting Program as an Administrator ####################################
# Get the ID and security principal of the current user account
$myWindowsID=[System.Security.Principal.WindowsIdentity]::GetCurrent()
$myWindowsPrincipal=new-object System.Security.Principal.WindowsPrincipal($myWindowsID)

# Get the security principal for the Administrator role
$adminRole=[System.Security.Principal.WindowsBuiltInRole]::Administrator

# Check to see if we are currently running "as Administrator"
if ($myWindowsPrincipal.IsInRole($adminRole))
{
# We are running "as Administrator" - so change the title and background color to indicate this
$Host.UI.RawUI.WindowTitle = $myInvocation.MyCommand.Definition + "(Elevated)"
$Host.UI.RawUI.BackgroundColor = "White"
clear-host
}
else
{
# We are not running "as Administrator" - so relaunch as administrator

# Create a new process object that starts PowerShell
$newProcess = new-object System.Diagnostics.ProcessStartInfo "PowerShell";

# Specify the current script path and name as a parameter
$newProcess.Arguments = $myInvocation.MyCommand.Definition;

# Indicate that the process should be elevated
$newProcess.Verb = "runas";

# Start the new process
[System.Diagnostics.Process]::Start($newProcess);

# Exit from the current, unelevated, process
exit
}

Write-Host -NoNewLine "Running as Administrator..."
################################## Excuting Program as an Administrator ####################################

################################## Commence Programming Sequence ####################################
New-Item -ItemType Directory -Force -Path $logpath;
"Executing copying tasks..."

function createShadow(){
[cmdletbinding()]
param(
[string]$targetVolume="C:\"
)
if (!($targetVolume -like "*\")){$targetVolume+="\"}
$shadowCopyClass=[WMICLASS]"root\cimv2:win32_shadowcopy"
$thisSnapshot = $shadowCopyClass.Create($targetVolume, "ClientAccessible")
$thisShadow = Get-WmiObject Win32_ShadowCopy | Where-Object { $_.ID -eq $thisSnapshot.ShadowID }
$thisShadowPath = $thisShadow.DeviceObject + "\"
C:
cmd /c mklink /d $shadowMount $thisShadowPath
"Shadow of $targetVolume has been made and it's accessible at this local file system (LFS): $shadowMount."
#copyLockedFiles; # this function is to be developed: retrieve lock files list, copy each item on list
#deleteShadow $thisShadow $thisShadowMount

# Export variables
$GLOBAL:shadow=$thisShadow;
}

function deleteShadow(){
# Remove symlink
(Get-Item $shadowMount).Delete()

# delete single instance of volume snapshots
$shadow.Delete()

# Delete all instances of volume snapshots
#Get-WmiObject Win32_ShadowCopy | % {$_.delete()}

"Shadow link $shadowMount has been removed."
}

function logPathError($pathError){
Add-Content $pathErrorsLog "$pathError";
}

function validateDirectory($dirToValidate){
if(Test-Path -Path $dirToValidate){return $True}
else{return $False;}
}

function createDirectory($dir){
# Create folder if it doesn't exist
if(!(validateDirectory $dir)){
New-Item -path $dir -type directory
}
}

function validateSourceAndDestination($thisblock){
$spacesCount=($thisblock.Split(' ')).Count-1
if ($spacesCount -eq 1){
$GLOBAL:source,$GLOBAL:destination=$thisblock.split(' ');
$sourceTest=validateDirectory $source
$destinationTest=validateDirectory $destination
if ($sourceTest -and $destinationTest){
return $True;
}
else {
if (!($sourceTest)){
logPathError "Source: $source";
return $False;
}
if (!($destinationTest)){
$createDestinationPath=(Read-Host -Prompt "Destination: $destination does not exist.`nType 'y' or 'yes' to create.");
if ($createDestinationPath -like 'yes' -or $createDestinationPath -like 'y'){
createDirectory $destination;
return $True;
}
else{
logPathError "Destination: $destination";
return $False
}
}
return $False;
}
}
else {
logPathError $thisblock;
return $False;
}
}

function translateSource{
param(
[string]$uncPath
)
$uri = new-object System.Uri($uncPath)
$thisLocalPath=$uri.LocalPath
#$thisHost=$uri.Host
$GLOBAL:sourceVolume="$((Get-Item $thisLocalPath).PSDrive.Name)`:"
$GLOBAL:translatedSource=$thisLocalPath -replace "$sourceVolume", $shadowMount
}

function sampleTimeStamp{
# Enable Remote to remote symlink following if it's not already set
$r2rEnabled=fsutil behavior query SymlinkEvaluation | select-string -Pattern "Remote to remote symbolic links are enabled."
if (!($r2rEnabled)){fsutil behavior set SymlinkEvaluation R2R:1;}

#$sourceFiles=Get-ChildItem $source -recurse | ? { !($_.PsIsContainer -and $_.FullName -notmatch 'archive' -and $_.FullName.Length -lt 260) } | get-random -count $sampleSize | % {$_.FullName}
$sourceFiles=Get-ChildItem $source -recurse | get-random -count $sampleSize | % {$_.FullName}
$commonDenominatingPaths=$sourceFiles | %{$_.replace($source,'')}
$destinationFiles=$commonDenominatingPaths | %{"$destination"+"$_";}
$badStamps=0;

"Checking a sample of $sampleSize for any time stamp inaccuracy..."

for ($i=0;$i -lt $sourceFiles.length;$i++){
$sourceFile=$sourceFiles[$i];
$destinationFile=$destinationFiles[$i];
if ($destinationFile.Length -lt 260){
$sourceTimeStamp=(gi $sourceFile).LastWriteTime;
$destinationTimeStamp=(gi $destinationFile).LastWriteTime;
if ($sourceTimeStamp -eq $destinationTimeStamp){
#$output+="`r`n$destinationFile is GOOD";
} else {
$output+="`r`n$destinationFile timestamp of $destinationTimeStamp DOES NOT MATCH its source $sourceFile timestamp of $sourceTimeStamp";
$badStamps++;
}
} else {
$output+="`r`nException: $destinationFile length is $($destinationFile.Length)";
}
}
$output="`r`n`r`n------------$((($sourceFiles.Length-$badStamps)/$sourceFiles.Length).tostring('P')) of the files in a sample of $($sourceFiles.Length) are having accurate time stamps--------------`n"+$output;
Add-Content $logFile $output;
}

function checkDiskFree{
<#
Excerpt from http://technet.microsoft.com/en-us/library/ee692290(WS.10).aspx:

"For volumes less than 500 megabytes, the minimum is 50 megabytes of free space.
For volumes more than 500 megabytes, the minimum is 320 megabytes of free space.
It is recommended that least 1 gigabyte of free disk space on each volume if the volume size is more than 1 gigabyte."

#>

# Import variables
$thisNode="localhost"
$thisVolume=$sourceVolume

# Obtain disk information
$diskObject = Get-WmiObject Win32_LogicalDisk -ComputerName $thisNode -Filter "DeviceID='$thisVolume'"
$diskFree=[Math]::Round($diskObject.FreeSpace / 1MB)
$diskSize=[Math]::Round($diskObject.Size / 1MB)

switch ($diskSize){
{$diskSize -ge 1024} {if ($diskFree -gt 1024){$feasible=$True;}else{$feasible=$False;};;break;}
{$diskSize -ge 500} {if ($diskFree -gt 320){$feasible=$True;}else{$feasible=$False;};;break;}
{$diskSize -lt 500} {if ($diskFree -gt 50){$feasible=$True;}else{$feasible=$False;};break;}
}

return $feasible
}

function expandZipfile($file, $destination){
$shell = new-object -com shell.application
$zip = $shell.NameSpace($file)

foreach($item in $zip.items()){
$shell.Namespace($destination).copyhere($item)
}
}

function installEmcopy{
$emcopyIsInstalled=(Get-Command emcopy.exe -ErrorAction SilentlyContinue) # Deterministic check on whether emcopy is already available on this system
if (!($emcopyIsInstalled)){
$tempDir="C:\Temp";
$extractionDir="C:\Windows"
$source = "https://kimconnect.com/wp-content/uploads/2019/08/emcopy.zip";
$destinationFile = "$tempDir\emcopy.zip";
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
New-Item -ItemType Directory -Force -Path $tempDir
New-Item -ItemType Directory -Force -Path $extractionDir
$webclient = New-Object System.Net.WebClient;
$WebClient.DownloadFile($source,$destinationFile);
expandZipfile $destinationFile -Destination $extractionDir
}else{
"EMCOPY is currently available in this system.`n";
}
}

function setACL{
get-acl -Path $source | Set-Acl -Path $destination # This command requires PowerShell 5
}

function startEmcopy{
param(
[string]$sourceAndDestination
)
"Emcopy has started..."
try{
invoke-expression "emcopy.exe $sourceAndDestination $switches $log";
}
catch{
# Record any errors into log and continue to next item
continue;
}
"`r`nEmcopy process has finished. Log is now generated at: $log";
}

Function proceed{
# Start the timer
Add-Content $logFile "`n`n`n---------------------------------Job Started: $dateStamp---------------------------------";
$totalTime=0;
$stopWatch= [System.Diagnostics.Stopwatch]::StartNew()
"Powershell version detected: $($PSVersionTable.PSVersion.Major)`.$($PSVersionTable.PSVersion.Minor)"
if (validateSourceAndDestination $block){
translateSource $source;
#"Shadow Copy source: $tranlatedSource`nDestination: $destination";
$translatedBlock="$translatedSource $destination";
if (checkDiskFree){
createShadow $sourceVolume;
installEmcopy;
startEmcopy $translatedBlock;
deleteShadow;
sampleTimeStamp;
"Program is completed."
"Log for this activity has been generated at $logFile"
}else{"Not enough disk space to create a VSS snapshot.`r`Program is aborted."}
}

# Getting storage estimate
$storage=(Get-ChildItem $source -Recurse | Measure-Object -Property Length -Sum -ErrorAction Stop).Sum / 1GB

# Stop the timer
$elapsedSeconds=$stopWatch.Elapsed.TotalSeconds;
$elapsedDisplay=([timespan]::fromseconds($elapsedSeconds)).ToString().Split('.')[0]
$totalTime+=$elapsedSeconds;

# Add total time display to log
$timeDisplay = ([timespan]::fromseconds($totalTime)).ToString()
Add-Content $logFile "`r`n------------------------------Total Time Elapsed: $timeDisplay------------------------------";

# Record overall speed to log
$speed=($totalTime/360)/$storage
Add-Content $logFile "`r`n---------------------------Aggregate speed: $speed GiB per hour---------------------------";
}

proceed;
################################## Main Programming Sequence ####################################

# cmd /c pause | out-null;

Current issues

Get-Item : The specified path, file name, or both are too long. The fully qualified file name must be less than 260 cha
racters, and the directory name must be less than 248 characters.
At line:1 char:3
+ gi <<<< $file
+ CategoryInfo : InvalidArgument: (\\FILESHERVER01...t_40_labels.doc:String) [Get-Item], PathTooLongExcep
tion
+ FullyQualifiedErrorId : ItemExistsPathTooLongError,Microsoft.PowerShell.Commands.GetItemCommand

Get-Item : Cannot find path '\\FILESHERVER01\My Documents\i21 Year 3\i21 Netbook Labels\2010 - 2011 Cla
ssroom Delivery Labels\2011 Printed Labels for sites for 2011 YR2\Secondary - Middle High K-8 & Alternative\DePortola
MS\DePortola MS B13 - avery_6576_cart_40_labels.doc' because it does not exist.
At line:1 char:3
+ gi <<<< $file
+ CategoryInfo : ObjectNotFound: (\\FILESHERVER01...t_40_labels.doc:String) [Get-Item], ItemNotFoundExcep
tion
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetItemCommand
Version 0.16
<#
.Description File_Copy_Script Version: 0.16

Purpose: this PowerShell Script is to efficiently mirror large batches of files using Emcopy in conjunction with Volume Shadow Services

Current Features:
1. Check for any errors on the Sources or Destinations and generate a report of any extra spaces in UNC
2. Create a snapshot of the source volume using Shadow Copy to capture any locked files
3. Execute robocopy to mirror the source to its corresponding destination with a time stamp variance allowance of 2 seconds for speed and resiliency
4. Sample the copied files to 'spot check' any time stamp variances
5. Execute in the context of an Administrator

Features planned for development:
6. Enable Volume Shadow Copy (VSS) at Source machines if it has been disabled, and reverse the action when done with copying
7. Trigger Remote Powershell to launch execution from a middle server (a "jump box" that is not a source nor destination)
if the provided Source is detected as a Universal Naming Convention (UNC) path instead of a local file system (LFS) path

Limitations:
1. This iteration requires that script is triggered from a local Windows machine with Internet access (no proxies) to download dependencies
2. Source must be LFS and Destination could either be LFS or UNC
#>

# Specify Sources (LFS) and Destinations (UNC)
# Using brackets to create a two dimensional array
$arr=@{}
$arr["from"] = @{}; $arr["to"] = @{}
$arr["from"] = @("C:\Users\Rambo\Desktop\Clients"); $arr["to"]=@("C:\Users\Rambo\Desktop\Test")
$arr["from"] += "C:\Users\Rambo\Desktop\Clients"; $arr["to"] += "C:\Users\Rambo\Desktop\Test1"

# Emcopy switches
$switches="/o /secforce /de /sd /c /r:0 /w:0 /th 128 /s /purge /sdd /stream"
<# Switch explanations
/s copies sub directories
/purge removes files and directories from the destination that do not exist in the source.
/sdd forces the target directories dates to be synchronized with the source directory.
/de Compares both file size and last modification time when deciding to update a file, updates it if either have been changed.
/cm md5 - checks the file content after copying using and md5 comparison of the source and destination.
/o copies the files owner, without this the account used for the copy will be the owner
/secforce overwrites the destination security settings with the source security settings (no merging, actual update of security settings at the destination)
/sd preserves security, the file isn't copied if an error occurs during security settings.
/th 128 - Uses 128 threads, default is 64
/r:0 retries zero times
/w:0 is the wait time in seconds between retries
/c will allow the process to continue after the retries
/log:filename option allows to redirect the console messages to a new file.
/log+:filename option appends the new messages to an existing file.
/stream option enables the copie of files and directories datastreams. Without that option only the main datastream of files are copied.
#>

# Init global variables
$GLOBAL:totalTime=0
$GLOBAL:totalStorage=0
$GLOBAL:previousDriveLetter="none"
$GLOBAL:snapshotDriveLetterSameAsNext=$False;
$GLOBAL:sampleSize=1000;
$GLOBAL:shadowMount="C:\shadowcopy"

# Initialize log files
$dateStamp = Get-Date -Format "yyyy-MM-dd-hhmmss"
$scriptName=$MyInvocation.MyCommand.Path
$scriptPath=Split-Path -Path $scriptName
$logPath="$scriptPath\filecopy_logs"
$logFile="$logPath\filecopy-log-$dateStamp.txt"
$log=" /LOG+:$logFile"
$lockedFilesReport="$logPath\_locked-files-log-$dateStamp.txt"
$pathErrorsLog="$logPath`\_path-errors-log-$dateStamp.txt"

################################## Excuting Program as an Administrator ####################################
# Get the ID and security principal of the current user account
$myWindowsID=[System.Security.Principal.WindowsIdentity]::GetCurrent()
$myWindowsPrincipal=new-object System.Security.Principal.WindowsPrincipal($myWindowsID)

# Get the security principal for the Administrator role
$adminRole=[System.Security.Principal.WindowsBuiltInRole]::Administrator

# Check to see if we are currently running "as Administrator"
if ($myWindowsPrincipal.IsInRole($adminRole))
{
# We are running "as Administrator" - so change the title and background color to indicate this
$Host.UI.RawUI.WindowTitle = $myInvocation.MyCommand.Definition + "(Elevated)"
$Host.UI.RawUI.BackgroundColor = "Black"
clear-host
}
else
{
# We are not running "as Administrator" - so relaunch as administrator

# Create a new process object that starts PowerShell
$newProcess = new-object System.Diagnostics.ProcessStartInfo "PowerShell";

# Specify the current script path and name as a parameter
$newProcess.Arguments = $myInvocation.MyCommand.Definition;

# Indicate that the process should be elevated
$newProcess.Verb = "runas";

# Start the new process
[System.Diagnostics.Process]::Start($newProcess);

# Exit from the current, unelevated, process
exit
}

Write-Host -NoNewLine "Running as Administrator..."
################################## Excuting Program as an Administrator ####################################

################################## Commence Programming Sequence ####################################
New-Item -ItemType Directory -Force -Path $logpath;
"Executing copying tasks..."

function createShadow(){
[cmdletbinding()]
param(
[string]$targetVolume="C:\"
)
if (!($targetVolume -like "*\")){$targetVolume+="\"}

$GLOBAL:thisDriveLetter=(Get-Item $targetVolume).PSDrive.Name

"Comparing $previousDriveLetter to $thisDriveLetter..."

if ($thisDriveLetter -eq $previousDriveLetter){
"Processing same volume. No VSS snapshop required.";
}else{
$GLOBAL:previousDriveLetter=$thisDriveLetter;

$shadowCopyClass=[WMICLASS]"root\cimv2:win32_shadowcopy"
$thisSnapshot = $shadowCopyClass.Create($targetVolume, "ClientAccessible")
$thisShadow = Get-WmiObject Win32_ShadowCopy | Where-Object { $_.ID -eq $thisSnapshot.ShadowID }
$thisShadowPath = $thisShadow.DeviceObject + "\"
C:
cmd /c mklink /d $shadowMount $thisShadowPath
"Shadow of $targetVolume has been made and it's accessible at this local file system (LFS): $shadowMount."
#copyLockedFiles; # this function is to be developed: retrieve lock files list, copy each item on list
#deleteShadow $thisShadow $thisShadowMount

# Export variables
$GLOBAL:shadow=$thisShadow;
}
}

function deleteShadow(){
if (!($snapshotDriveLetterSameAsNext)){
# Remove symlink
(Get-Item $shadowMount).Delete()

# delete single instance of volume snapshots
$shadow.Delete()

# Delete all instances of volume snapshots
#Get-WmiObject Win32_ShadowCopy | % {$_.delete()}

"Shadow link $shadowMount has been removed."
}else{
"No snapshots to remove.";
}
}

function logPathError($pathError){
Add-Content $pathErrorsLog "$pathError";
}

function validateDirectory($dirToValidate){
if(Test-Path -Path $dirToValidate){return $True}
else{return $False;}
}

function createDirectory($dir){
# Create folder if it doesn't exist
if(!(validateDirectory $dir)){
New-Item -path $dir -type directory
}
}

function validateSourceAndDestination($thisblock){
$spacesCount=($thisblock.Split(' ')).Count-1
if ($spacesCount -eq 1){
$GLOBAL:source,$GLOBAL:destination=$thisblock.split(' ');
$sourceTest=validateDirectory $source
$destinationTest=validateDirectory $destination
if ($sourceTest -and $destinationTest){
return $True;
}
else {
if (!($sourceTest)){
logPathError "Source: $source";
return $False;
}
if (!($destinationTest)){
$createDestinationPath=(Read-Host -Prompt "Destination: $destination does not exist.`nType 'y' or 'yes' to create.");
if ($createDestinationPath -like 'yes' -or $createDestinationPath -like 'y'){
createDirectory $destination;
return $True;
}
else{
logPathError "Destination: $destination";
return $False
}
}
return $False;
}
}
else {
logPathError $thisblock;
return $False;
}
}

function translateSource{
param(
[string]$uncPath
)
$uri = new-object System.Uri($uncPath)
$thisLocalPath=$uri.LocalPath
#$thisHost=$uri.Host
$GLOBAL:sourceVolume="$((Get-Item $thisLocalPath).PSDrive.Name)`:"
$GLOBAL:translatedSource=$thisLocalPath -replace "$sourceVolume", $shadowMount
}

function sampleTimeStamp{
# Enable Remote to remote symlink following if it's not already set
$r2rEnabled=fsutil behavior query SymlinkEvaluation | select-string -Pattern "Remote to remote symbolic links are enabled."
if (!($r2rEnabled)){fsutil behavior set SymlinkEvaluation R2R:1;}

#$sourceFiles=Get-ChildItem $source -recurse | ? { !($_.PsIsContainer -and $_.FullName -notmatch 'archive' -and $_.FullName.Length -lt 260) } | get-random -count $sampleSize | % {$_.FullName}
$sourceFiles=Get-ChildItem $source -recurse | get-random -count $sampleSize | % {$_.FullName}
$commonDenominatingPaths=$sourceFiles | %{$_.replace($source,'')}
$destinationFiles=$commonDenominatingPaths | %{"$destination"+"$_";}
$badStamps=0;

"Checking a sample of $sampleSize for any time stamp inaccuracy..."

for ($i=0;$i -lt $sourceFiles.length;$i++){
$sourceFile=$sourceFiles[$i];
$destinationFile=$destinationFiles[$i];
if ($destinationFile.Length -lt 260){
$sourceTimeStamp=(gi $sourceFile).LastWriteTime;
$destinationTimeStamp=(gi $destinationFile).LastWriteTime;
if ($sourceTimeStamp -eq $destinationTimeStamp){
#$output+="`r`n$destinationFile is GOOD";
} else {
$output+="`r`n$destinationFile timestamp of $destinationTimeStamp DOES NOT MATCH its source $sourceFile timestamp of $sourceTimeStamp";
$badStamps++;
}
} else {
$output+="`r`nException: $destinationFile length is $($destinationFile.Length)";
}
}
$output="`r`n`r`n------------$((($sourceFiles.Length-$badStamps)/$sourceFiles.Length).tostring('P')) of the files in a sample of $($sourceFiles.Length) are having accurate time stamps--------------`n"+$output;
Add-Content $logFile $output;
}

function checkDiskFree{
<#
Excerpt from http://technet.microsoft.com/en-us/library/ee692290(WS.10).aspx:

"For volumes less than 500 megabytes, the minimum is 50 megabytes of free space.
For volumes more than 500 megabytes, the minimum is 320 megabytes of free space.
It is recommended that least 1 gigabyte of free disk space on each volume if the volume size is more than 1 gigabyte."

#>

# Import variables
$thisNode="localhost"
$thisVolume=$sourceVolume

# Obtain disk information
$diskObject = Get-WmiObject Win32_LogicalDisk -ComputerName $thisNode -Filter "DeviceID='$thisVolume'"
$diskFree=[Math]::Round($diskObject.FreeSpace / 1MB)
$diskSize=[Math]::Round($diskObject.Size / 1MB)

switch ($diskSize){
{$diskSize -ge 1024} {if ($diskFree -gt 1024){$feasible=$True;}else{$feasible=$False;};;break;}
{$diskSize -ge 500} {if ($diskFree -gt 320){$feasible=$True;}else{$feasible=$False;};;break;}
{$diskSize -lt 500} {if ($diskFree -gt 50){$feasible=$True;}else{$feasible=$False;};break;}
}

return $feasible
}

function expandZipfile($file, $destination){
$shell = new-object -com shell.application
$zip = $shell.NameSpace($file)

foreach($item in $zip.items()){
$shell.Namespace($destination).copyhere($item)
}
}

function installEmcopy{
$emcopyIsInstalled=(Get-Command emcopy.exe -ErrorAction SilentlyContinue) # Deterministic check on whether emcopy is already available on this system
if (!($emcopyIsInstalled)){
$tempDir="C:\Temp";
$extractionDir="C:\Windows"
$source = "https://kimconnect.com/wp-content/uploads/2019/08/emcopy.zip";
$destinationFile = "$tempDir\emcopy.zip";
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
New-Item -ItemType Directory -Force -Path $tempDir
New-Item -ItemType Directory -Force -Path $extractionDir
$webclient = New-Object System.Net.WebClient;
$WebClient.DownloadFile($source,$destinationFile);
expandZipfile $destinationFile -Destination $extractionDir
}else{
"EMCOPY is currently available in this system.`n";
}
}

function setACL{
get-acl -Path $source | Set-Acl -Path $destination # This command requires PowerShell 5
}

function startEmcopy{
param(
[string]$sourceAndDestination
)
"Emcopy has started..."
try{
invoke-expression "emcopy.exe $sourceAndDestination $switches $log";
}
catch{
# Record any errors into log and continue to next item
continue;
}
"`r`nEmcopy process has finished. Log is now generated at: $log";
}

Function startCopy($from,$to){

$GLOBAL:source=$from;
$GLOBAL:destination=$to;
$GLOBAL:block="$source $destination"
$time=0;
$stopWatch= [System.Diagnostics.Stopwatch]::StartNew()

if (validateSourceAndDestination $block){
translateSource $source;
#"Shadow Copy source: $tranlatedSource`nDestination: $destination";
$translatedBlock="$translatedSource $destination";
if (checkDiskFree){
createShadow $sourceVolume;
installEmcopy;
startEmcopy $translatedBlock;
deleteShadow;
sampleTimeStamp;
"Program is completed."
"Log for this activity has been generated at $logFile"
}else{"Not enough disk space to create a VSS snapshot.`r`Skipping this iteration..."}
}

# Getting storage estimate
$storage=(Get-ChildItem $source -Recurse | Measure-Object -Property Length -Sum -ErrorAction Stop).Sum / 1GB;
$storageDestination=(Get-ChildItem $destination -Recurse | Measure-Object -Property Length -Sum -ErrorAction Stop).Sum / 1GB;
$GLOBAL:totalStorage+=$storage;

# Stop the timer
$time=$stopWatch.Elapsed.TotalSeconds;
#$elapsedDisplay=([timespan]::fromseconds($elapsedSeconds)).ToString().Split('.')[0]
$GLOBAL:totalTime+=$time;

# Add total time display to log
$timeDisplay = ([timespan]::fromseconds($time)).ToString()
Add-Content $logFile "`r`n------------------------------Time Elapsed: $timeDisplay------------------------------";

# Record overall speed to log
$speed=($storage/$time)*3600
Add-Content $logFile "`r`n---------------------------Speed: $([math]::Round($speed,4)) GiB per hour---------------------------";

# Compare storage results
Add-Content $logFile "`r`n---------------------------Comparing: Source $([math]::Round($storage,4)) GiB vs Destination $([math]::Round($storageDestination,4)) GiB---------------------------";
}

function proceed{
$initInfo="=============================================Job Started: $dateStamp=============================================`r`n";
$initInfo+="Powershell version detected: $($PSVersionTable.PSVersion.Major)`.$($PSVersionTable.PSVersion.Minor)`r`n"
$initInfo+="Processing the following operations:`r";
$initInfo+=for ($i=0;$i -lt $arr.from.count; $i++){"$($arr.from[$i]) => $($arr.to[$i])`r";}
$initInfo;
Add-Content $logFile $initInfo;

# Process the copying operations
for ($i=0;$i -lt $arr.from.count; $i++){
$processDisplay="=============================================$($arr.from[$i]) => $($arr.to[$i])=============================================`r";
Add-Content $logFile $processDisplay;
$thisSourceVolume=Split-Path -Path $arr.from[$i] -Qualifier
$nextSourceVolume=Split-Path -Path $arr.from[$i+1] -Qualifier
if ($thisSourceVolume-eq $nextSourceVolume){$snapshotDriveLetterSameAsNext=$True;}else{$snapshotDriveLetterSameAsNext=$False;}
startCopy $arr.from[$i] $arr.to[$i];
}

# Record overall speed to log
$aggregateSpeed=($totalStorage/$totalTime)*3600
$summary="`r`n==============================$([math]::Round($totalStorage,4)) GiB has been processed in $([math]::Round($totalTime,4)) second(s)==============================`r"
$summary+="`r`n==============================Jobs Completed with Total Aggregate Speed: $([math]::Round($aggregateSpeed,4)) GiB per hour=============================="
Add-Content $logFile $summary;
$nextSourceVolume=$null;
}

proceed;
################################## Main Programming Sequence ####################################
# pause;
#cmd /c pause | out-null;

PowerShell: Use EMCOPY to Mirror a Directory

# Purpose: this PowerShell snippet is to demonstrate the use of Emcopy

$source="C:\Users\tester\Desktop\Clients"
$destination="C:\Users\tester\Desktop\Test"
#$switches="/o /secforce /s /de /sd /c /r:0 /th 32 /cm md5 /purge /sdd"
$switches="/o /secforce /de /sd /c /r:0 /th 32 /s /purge /sdd"
<# Switch explanations
/s copies sub directories
/purge removes files and directories from the destination that do not exist in the source.
/sdd forces the target directories dates to be synchronized with the source directory.
/de Compares both file size and last modification time when deciding to update a file, updates it if either have been changed.
/cm md5 - checks the file content after copying using and md5 comparison of the source and destination.
/o copies the files owner, without this the account used for the copy will be the owner
/secforce overwrites the destination security settings with the source security settings (copies security settings)
/sd preserves security, the file isn't copied if an error occurs during security settings.
/th 32 - Uses 32 threads, default is 64
/r:0 retries zero times
/w:0 is the wait time in seconds between retries
/c will allow the process to continue after the retries
/log:filename option allows to redirect the console messages to a new file.
/log+:filename option appends the new messages to an existing file.
#>

$dateStamp = Get-Date -Format "yyyy-MM-dd-hhmmss"
$scriptName=$MyInvocation.MyCommand.Path
$scriptPath=Split-Path -Path $scriptName
$logPath="$scriptPath\emcopy_logs"
$logFile="$logPath\robocopy-log-$dateStamp.txt"
$log=" /LOG+:$logFile"


# Get the ID and security principal of the current user account
$myWindowsID=[System.Security.Principal.WindowsIdentity]::GetCurrent()
$myWindowsPrincipal=new-object System.Security.Principal.WindowsPrincipal($myWindowsID)

# Get the security principal for the Administrator role
$adminRole=[System.Security.Principal.WindowsBuiltInRole]::Administrator

# Check to see if we are currently running "as Administrator"
if ($myWindowsPrincipal.IsInRole($adminRole))
{
# We are running "as Administrator" - so change the title and background color to indicate this
$Host.UI.RawUI.WindowTitle = $myInvocation.MyCommand.Definition + "(Elevated)"
$Host.UI.RawUI.BackgroundColor = "Black"
clear-host
}
else
{
"Relaunching as an Administrator.";

# Create a new process object that starts PowerShell
$newProcess = new-object System.Diagnostics.ProcessStartInfo "PowerShell";

# Specify the current script path and name as a parameter
$newProcess.Arguments = $myInvocation.MyCommand.Definition;

# Indicate that the process should be elevated
$newProcess.Verb = "runas";

# Start the new process
[System.Diagnostics.Process]::Start($newProcess);

# Exit from the current, unelevated, process
exit
}

# Run your code that needs to be elevated here
Write-Host -NoNewLine "Running as Administrator..."
#$null = $Host.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown")

function installEmcopy{
$emcopyIsInstalled=(Get-Command emcopy.exe -ErrorAction SilentlyContinue) # Deterministic check on whether emcopy is already available on this system
if (!($emcopyIsInstalled)){
$tempDir="C:\Temp";
$extractionDir="C:\Windows"
$source = "https://kimconnect.com/wp-content/uploads/2019/08/emcopy.zip";
$destinationFile = "$tempDir\emcopy.zip";
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
New-Item -ItemType Directory -Force -Path $tempDir
New-Item -ItemType Directory -Force -Path $extractionDir
$webclient = New-Object System.Net.WebClient;
$WebClient.DownloadFile($source,$destinationFile);
Expand-Archive -LiteralPath $destinationFile -DestinationPath $extractionDir
}else{
"EMCOPY is currently available in this system.`n";
}
}

function startEmcopy{
New-Item -ItemType Directory -Force -Path $logPath;
"Confirm this statement:`nemcopy64.exe $source $destination $switches $log";
pause;
invoke-expression "emcopy64.exe $source $destination $switches $log"
"emcopy process is finished. Log is generated here: $log"
pause;
}

installEmcopy;
startEmcopy;

A typical error:

TH000 : 20:32:21 : WARNING : Unable to retrieve the PDC of the domain intranet nearby \\FILESHERVER007, error : 1210
TH000 : 20:32:21 : WARNING : The default primary group is set to "domain users"

Resolution:

  • Identify the user account used for running EMCopy (admin account you are using)
  • Add that account to the local administrator group and local backup group on the Windows file server
  • Allow that account to “run as root” on the Isilon share

PowerShell: File Copying Operation

Version 0.16
<#
.Description File_Copy_Script Version: 0.16

Purpose: this PowerShell Script is to efficiently mirror large batches of files using Emcopy in conjunction with Volume Shadow Services

Current Features:
1. Check for any errors on the Sources or Destinations and generate a report of any extra spaces in UNC
2. Create a snapshot of the source volume using Shadow Copy to capture any locked files
3. Execute robocopy to mirror the source to its corresponding destination with a time stamp variance allowance of 2 seconds for speed and resiliency
4. Sample the copied files to 'spot check' any time stamp variances
5. Execute in the context of an Administrator

Features planned for development:
6. Enable Volume Shadow Copy (VSS) at Source machines if it has been disabled, and reverse the action when done with copying
7. Trigger Remote Powershell to launch execution from a middle server (a "jump box" that is not a source nor destination)
if the provided Source is detected as a Universal Naming Convention (UNC) path instead of a local file system (LFS) path

Limitations:
1. This iteration requires that script is triggered from a local Windows machine with Internet access (no proxies) to download dependencies
2. Source must be LFS and Destination could either be LFS or UNC

Troubleshooting:

# Create snapshot
$targetVolume="M:\"
$shadowCopyClass=[WMICLASS]"root\cimv2:win32_shadowcopy";
$thisSnapshot = $shadowCopyClass.Create($targetVolume, "ClientAccessible");
$thisShadow = Get-WmiObject Win32_ShadowCopy | Where-Object { $_.ID -eq $thisSnapshot.ShadowID };
$thisShadowPath = $thisShadow.DeviceObject + "\";
C:
cmd /c mklink /d "C:\shadowcopy" $thisShadowPath

# Remove snapshot
$thisShadow.Delete()
$thisSnapshot.Dispose()
(Get-Item "C:\shadowcopy").Delete()

# Delete all shadows
vssadmin delete shadows /all /Quiet
#>

# Specify Sources (LFS) and Destinations (UNC)
# Using brackets notation to create a two dimensional array, overcoming PowerShell's limitations
$arr=@{}
$arr["from"] = @{}; $arr["to"] = @{}
$arr["from"] = @("C:\Users\Kim\Desktop\Tech_Documentation"); $arr["to"]=@("C:\Users\Kim\Desktop\Test")
$arr["from"] += "C:\Users\Kim\Desktop\Test"; $arr["to"]+="C:\Users\Kim\Desktop\Test1"

# Init global variables
$GLOBAL:totalTime=0;
$GLOBAL:totalStorage=0;
$GLOBAL:sampleSize=1000;
$GLOBAL:shadowMount="C:\shadowcopy"
$GLOBAL:toCreateShadow=$True
$GLOBAL:previousDriveLetter="none"
$GLOBAL:thisDriveLetter="none"
$GLOBAL:nextDriveLetter="none"
$clusterSizeOfNetworkShare=16384

# Initialize log files
$dateStamp = Get-Date -Format "yyyy-MM-dd-hhmmss"
$scriptName=$MyInvocation.MyCommand.Path
$scriptPath=Split-Path -Path $scriptName
$logPath="$scriptPath\filecopy_logs"
$logFile="$logPath\filecopy-log-$dateStamp.txt"
$log=" /LOG+:$logFile"
$lockedFilesReport="$logPath\_locked-files-log-$dateStamp.txt"
$pathErrorsLog="$logPath`\_path-errors-log-$dateStamp.txt"

# Emcopy switches
$switches="/o /secforce /de /sd /c /r:0 /w:0 /th 128 /s /purge /sdd /stream"
<# Switch explanations
/s copies sub directories
/purge removes files and directories from the destination that do not exist in the source.
/sdd forces the target directories dates to be synchronized with the source directory.
/de Compares both file size and last modification time when deciding to update a file, updates it if either have been changed.
/cm md5 - checks the file content after copying using and md5 comparison of the source and destination.
/o copies the files owner, without this the account used for the copy will be the owner
/secforce overwrites the destination security settings with the source security settings (no merging, actual update of security settings at the destination)
/sd preserves security, the file isn't copied if an error occurs during security settings.
/th 128 - Uses 128 threads, default is 64
/r:0 retries zero times
/w:0 is the wait time in seconds between retries
/c will allow the process to continue after the retries
/log:filename option allows to redirect the console messages to a new file.
/log+:filename option appends the new messages to an existing file.
/stream option enables the copie of files and directories datastreams. Without that option only the main datastream of files are copied.
#>

################################## Excuting Program as an Administrator ####################################
# Get the ID and security principal of the current user account
$myWindowsID=[System.Security.Principal.WindowsIdentity]::GetCurrent()
$myWindowsPrincipal=new-object System.Security.Principal.WindowsPrincipal($myWindowsID)

# Get the security principal for the Administrator role
$adminRole=[System.Security.Principal.WindowsBuiltInRole]::Administrator

# Check to see if we are currently running "as Administrator"
if ($myWindowsPrincipal.IsInRole($adminRole))
{
# We are running "as Administrator" - so change the title and background color to indicate this
$Host.UI.RawUI.WindowTitle = $myInvocation.MyCommand.Definition + "(Elevated)"
$Host.UI.RawUI.BackgroundColor = "Black"
clear-host
}
else
{
# We are not running "as Administrator" - so relaunch as administrator

# Create a new process object that starts PowerShell
$newProcess = new-object System.Diagnostics.ProcessStartInfo "PowerShell";

# Specify the current script path and name as a parameter
$newProcess.Arguments = $myInvocation.MyCommand.Definition;

# Indicate that the process should be elevated
$newProcess.Verb = "runas";

# Start the new process
[System.Diagnostics.Process]::Start($newProcess);

# Exit from the current, unelevated, process
exit
}

Write-Host -NoNewLine "Running as Administrator..."
################################## Excuting Program as an Administrator ####################################

################################## Commence Programming Sequence ####################################
New-Item -ItemType Directory -Force -Path $logpath;
"Executing copying tasks..."

function createShadow(){
[cmdletbinding()]
param(
[string]$targetVolume="C:\"
)
if (!($targetVolume -like "*\")){$targetVolume+="\"}

if ($toCreateShadow){
$shadowCopyClass=[WMICLASS]"root\cimv2:win32_shadowcopy"
$thisSnapshot = $shadowCopyClass.Create($targetVolume, "ClientAccessible")
$thisShadow = Get-WmiObject Win32_ShadowCopy | Where-Object { $_.ID -eq $thisSnapshot.ShadowID }
$thisShadowPath = $thisShadow.DeviceObject + "\"
C:
cmd /c mklink /d $shadowMount $thisShadowPath
"Shadow of $targetVolume has been made and it's accessible at this local file system (LFS): $shadowMount."
#copyLockedFiles; # this function is to be developed: retrieve lock files list, copy each item on list
#deleteShadow $thisShadow $thisShadowMount

# Export variables
$GLOBAL:shadow=$thisShadow;
$GLOBAL:snapshot=$thisSnapshot;

}else{
"Not creating a duplicate shadow in this iteration.";
}
}

function deleteShadow(){
if ($nextDriveLetter -eq $thisDriveLetter){
"Not removing snapshot because next one is on the same drive letter.";
}else{
# Remove symlink
(Get-Item $shadowMount).Delete()

# delete single instance of volume snapshots
$shadow.Delete()
$snapshot.Dispose()

# Delete all instances of volume snapshots
#Get-WmiObject Win32_ShadowCopy | % {$_.delete()}

"Shadow link $shadowMount has been removed."
}
}

function logPathError($pathError){
Add-Content $pathErrorsLog "$pathError";
}

function validateDirectory($dirToValidate){
if(Test-Path -Path $dirToValidate){return $True}
else{return $False;}
}

function createDirectory($dir){
# Create folder if it doesn't exist
if(!(validateDirectory $dir)){
New-Item -path $dir -type directory
}
}

function validateSourceAndDestination($thisblock){
$spacesCount=($thisblock.Split(' ')).Count-1
if ($spacesCount -eq 1){
$GLOBAL:source,$GLOBAL:destination=$thisblock.split(' ');
$sourceTest=validateDirectory $source
$destinationTest=validateDirectory $destination
if ($sourceTest -and $destinationTest){
return $True;
}
else {
if (!($sourceTest)){
logPathError "Source: $source";
return $False;
}
if (!($destinationTest)){
$createDestinationPath=(Read-Host -Prompt "Destination: $destination does not exist.`nType 'y' or 'yes' to create.");
if ($createDestinationPath -like 'yes' -or $createDestinationPath -like 'y'){
createDirectory $destination;
return $True;
}
else{
logPathError "Destination: $destination";
return $False
}
}
return $False;
}
}
else {
logPathError $thisblock;
return $False;
}
}

function translateSource{
param(
[string]$uncPath
)
$uri = new-object System.Uri($uncPath)
$thisLocalPath=$uri.LocalPath
#$thisHost=$uri.Host
$GLOBAL:sourceVolume="$((Get-Item $thisLocalPath).PSDrive.Name)`:"
$GLOBAL:translatedSource=$thisLocalPath -replace "$sourceVolume", $shadowMount
}

function sampleTimeStamp{
# Enable Remote to remote symlink following if it's not already set
$r2rEnabled=fsutil behavior query SymlinkEvaluation | select-string -Pattern "Remote to remote symbolic links are enabled."
if (!($r2rEnabled)){fsutil behavior set SymlinkEvaluation R2R:1;}

#$sourceFiles=Get-ChildItem $source -recurse | ? { !($_.PsIsContainer -and $_.FullName -notmatch 'archive' -and $_.FullName.Length -lt 260) } | get-random -count $sampleSize | % {$_.FullName}
$sourceFiles=Get-ChildItem $source -recurse | get-random -count $sampleSize | % {$_.FullName}
$commonDenominatingPaths=$sourceFiles | %{$_.replace($source,'')}
$destinationFiles=$commonDenominatingPaths | %{"$destination"+"$_";}
$badStamps=0;

if ($sourceFiles[0].Length -gt 1){
"Checking a sample of $sampleSize for any time stamp inaccuracy..."
for ($i=0;$i -lt $sourceFiles.length;$i++){
$sourceFile=$sourceFiles[$i];
$destinationFile=$destinationFiles[$i];
if ($destinationFile.Length -lt 260){
$sourceTimeStamp=(gi $sourceFile).LastWriteTime;
$destinationTimeStamp=(gi $destinationFile).LastWriteTime;
if ($sourceTimeStamp -eq $destinationTimeStamp){
#$output+="`r`n$destinationFile is GOOD";
} else {
$output+="`r`n$destinationFile timestamp of $destinationTimeStamp DOES NOT MATCH its source $sourceFile timestamp of $sourceTimeStamp";
$badStamps++;
}
} else {
$output+="`r`nException: $destinationFile length is $($destinationFile.Length)";
}
}
$output="`r`n`r`n------------$((($sourceFiles.Length-$badStamps)/$sourceFiles.Length).tostring('P')) of the files in a sample of $($sourceFiles.Length) are having accurate time stamps--------------`n"+$output;
}else{$output="`r`n`r`n------------Insufficient number of files to process.------------`r`n";}

$output;
Add-Content $logFile $output;
}

function checkDiskFree{
<#
Excerpt from http://technet.microsoft.com/en-us/library/ee692290(WS.10).aspx:

"For volumes less than 500 megabytes, the minimum is 50 megabytes of free space.
For volumes more than 500 megabytes, the minimum is 320 megabytes of free space.
It is recommended that least 1 gigabyte of free disk space on each volume if the volume size is more than 1 gigabyte."

#>

# Import variables
$thisNode="localhost"
$thisVolume=$sourceVolume

# Obtain disk information
$diskObject = Get-WmiObject Win32_LogicalDisk -ComputerName $thisNode -Filter "DeviceID='$thisVolume'"
$diskFree=[Math]::Round($diskObject.FreeSpace / 1MB)
$diskSize=[Math]::Round($diskObject.Size / 1MB)

switch ($diskSize){
{$diskSize -ge 1024} {if ($diskFree -gt 1024){$feasible=$True;}else{$feasible=$False;};;break;}
{$diskSize -ge 500} {if ($diskFree -gt 320){$feasible=$True;}else{$feasible=$False;};;break;}
{$diskSize -lt 500} {if ($diskFree -gt 50){$feasible=$True;}else{$feasible=$False;};break;}
}

return $feasible
}

function expandZipfile($file, $destination){
$shell = new-object -com shell.application
$zip = $shell.NameSpace($file)

foreach($item in $zip.items()){
$shell.Namespace($destination).copyhere($item)
}
}

function installEmcopy{
$emcopyIsInstalled=(Get-Command emcopy.exe -ErrorAction SilentlyContinue) # Deterministic check on whether emcopy is already available on this system
if (!($emcopyIsInstalled)){
$tempDir="C:\Temp";
$extractionDir="C:\Windows"
$source = "https://kimconnect.com/wp-content/uploads/2019/08/emcopy.zip";
$destinationFile = "$tempDir\emcopy.zip";
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
New-Item -ItemType Directory -Force -Path $tempDir
New-Item -ItemType Directory -Force -Path $extractionDir
$webclient = New-Object System.Net.WebClient;
$WebClient.DownloadFile($source,$destinationFile);
expandZipfile $destinationFile -Destination $extractionDir
}else{
"EMCOPY is currently available in this system.`n";
}
}

function setACL{
get-acl -Path $source | Set-Acl -Path $destination # This command requires PowerShell 5
}

function startEmcopy{
param(
[string]$sourceAndDestination
)
"Emcopy has started..."
try{
invoke-expression "emcopy.exe $sourceAndDestination $switches $log";
}
catch{
# Record any errors into log and continue to next item
continue;
}
"`r`nEmcopy process has finished. Log is now generated at: $log";
}

Function startCopy($from,$to){
$GLOBAL:source=$from;
$GLOBAL:destination=$to;
$GLOBAL:block="$source $destination"
$time=0;
$stopWatch= [System.Diagnostics.Stopwatch]::StartNew()

if (validateSourceAndDestination $block){
translateSource $source;
#"Shadow Copy source: $tranlatedSource`nDestination: $destination";
$translatedBlock="$translatedSource $destination";
if (checkDiskFree){
createShadow $sourceVolume;
installEmcopy;
Try{
startEmcopy $translatedBlock;
}
catch{
continue;
}
deleteShadow;
sampleTimeStamp;
"Log for this activity has been generated at $logFile"
}else{"Not enough disk space to create a VSS snapshot.`r`Skipping this iteration..."}
}

# Getting storage estimate
#$storage=(Get-ChildItem $source -Recurse | Measure-Object -Property Length -Sum -ErrorAction Stop).Sum / 1GB;
#$storageDestination=(Get-ChildItem $destination -Recurse | Measure-Object -Property Length -Sum -ErrorAction Stop).Sum / 1GB;
$storage=(getSizeOnDisk -path $source) / 1GB;
$storageDestination=(getSizeOnDisk -path $destination) / 1GB;
$percentDifference=[math]::Round((($storageDestination-$storage)/$storage)*100),2)
$GLOBAL:totalStorage+=$storage;
$totalStorage;

# Stop the timer
$time=$stopWatch.Elapsed.TotalSeconds;
#$elapsedDisplay=([timespan]::fromseconds($elapsedSeconds)).ToString().Split('.')[0]
$GLOBAL:totalTime+=$time;
$totalTime;

# Add total time display to log
$timeDisplay = ([timespan]::fromseconds($time)).ToString()
Add-Content $logFile "`r`n------------------------------Time Elapsed: $timeDisplay------------------------------";

# Record overall speed to log
$speed=($storage/$time)*3600
Add-Content $logFile "`r`n---------------------------Speed: $([math]::Round($speed,4)) GiB per hour---------------------------";

# Compare storage results
Add-Content $logFile "`r`n---------------------------$(if($percentDifference -ge 0){"$percentDifference`% loss";}else{"$([math]::abs($percentDifference))`% gain";})`: Source $([math]::Round($storage,4)) GiB vs Destination $([math]::Round($storageDestination,4)) GiB---------------------------";
}

function getSizeOnDisk{
param (
[string]$path='.'
)

$source = @"
using System;
using System.Runtime.InteropServices;
using System.ComponentModel;
using System.IO;

namespace Win32
{

public class Disk {

[DllImport("kernel32.dll")]
static extern uint GetCompressedFileSizeW([In, MarshalAs(UnmanagedType.LPWStr)] string lpFileName,
[Out, MarshalAs(UnmanagedType.U4)] out uint lpFileSizeHigh);

public static ulong GetSizeOnDisk(string filename)
{
uint HighOrderSize;
uint LowOrderSize;
ulong size;

FileInfo file = new FileInfo(filename);
LowOrderSize = GetCompressedFileSizeW(file.FullName, out HighOrderSize);

if (HighOrderSize == 0 && LowOrderSize == 0xffffffff)
{
throw new Win32Exception(Marshal.GetLastWin32Error());
}
else {
size = ((ulong)HighOrderSize << 32) + LowOrderSize;
return size;
}
}
}
}

"@

Add-Type -TypeDefinition $source

$totalSize=0;
$count=0;
$driveLetter=(Get-Item $path).PSDrive.Root
$clusterSize=(Get-WmiObject -Class Win32_Volume | Where-Object {$_.Name -eq $driveLetter}).BlockSize
if(!($clusterSize)){$clusterSize=$clusterSizeOfNetworkShare}

Get-ChildItem $path -Recurse -Force | where { !$_.PSisContainer }|% {
$size = [Win32.Disk]::GetSizeOnDisk($_.FullName)
#$AFZ=[MidpointRounding]::AwayFromZero
#$roundedSize=[math]::Round($size/$ClusterSize+0.5,$AFZ)*$ClusterSize
if($size -gt $clusterSize){$roundedSize=[math]::ceiling($size/$clusterSize)*$clusterSize}else{$roundedSize=$clusterSize;}
$count=$count+1
#"$count`. $($_.FullName): $size vs $roundedSize"
$totalSize+=$roundedSize
}

#$foldersCount=(Get-ChildItem $path -Recurse -Force | where { $_.PSisContainer }).Count
#write-output "$count files $foldersCount folders total size: $totalSize"
return $totalSize;
}

function proceed{
$initInfo="=============================================Job Started: $dateStamp=============================================`r`n";
$initInfo+="Powershell version detected: $($PSVersionTable.PSVersion.Major)`.$($PSVersionTable.PSVersion.Minor)`r`n"
$initInfo+="Processing the following operations:`r";
$initInfo+=for ($i=0;$i -lt $arr.from.count; $i++){"$($arr.from[$i]) => $($arr.to[$i])`r";}
$initInfo;
Add-Content $logFile $initInfo;

# Process the copying operations
for ($i=0;$i -lt $arr.from.count; $i++){
$processDisplay="=======================================Pass $($i+1)`: $($arr.from[$i]) => $($arr.to[$i])=======================================`r";
Add-Content $logFile $processDisplay;
$GLOBAL:thisDriveLetter=Split-Path -Path $arr.from[$i] -Qualifier
$GLOBAL:nextDriveLetter=try{Split-Path -Path $arr.from[$i+1] -Qualifier}catch{$false;}
"Previous drive $previousDriveLetter vs current $thisDriveLetter vs next $nextDriveLetter"
if($previousDriveLetter -ne $thisDriveLetter){$GLOBAL:toCreateShadow=$True}else{$GLOBAL:toCreateShadow=$False}
startCopy $arr.from[$i] $arr.to[$i];
$GLOBAL:previousDriveLetter=$thisDriveLetter;
# cmd /c pause | out-null;
}

# Record overall speed to log
$aggregateSpeed=($totalStorage/$totalTime)*3600
$summary="`r`n==============================Program has completed.==============================`r";
$summary+="`r`n==============================$([math]::Round($totalStorage,4)) GiB has been processed in $([math]::Round($totalTime/3600,4)) hours(s)==============================`r"
$summary+="`r`n==============================Jobs Completed with Total Aggregate Speed: $([math]::Round($aggregateSpeed,4)) GiB per hour=============================="
$summary;
Add-Content $logFile $summary;
}

function checkInputs{
"Checking inputs for bad paths..."
$GLOBAL:proceed=$True
for ($i=0;$i -lt $arr.from.count; $i++){
$from=$arr.from[$i]
$to=$arr.to[$i]
if(!(test-path $from)){$from;$GLOBAL:proceed=$False}
if(!(test-path $to)){$to;$GLOBAL:proceed=$False}
}
}

checkInputs;
$proceed
if($proceed){proceed;}else{"Paths validation has failed. Program will not proceed."}
################################## Main Programming Sequence ####################################
pause;
#cmd /c pause | out-null;
Version 0.15
<# .Description File_Copy_Script Version: 0.15

Purpose: this PowerShell Script is to efficiently mirror large batches of files using Emcopy in conjunction with Volume Shadow Services

Current Features:
1. Check for any errors on the Sources or Destinations and generate a report of any extra spaces in UNC
2. Create a snapshot of the source volume using Shadow Copy to capture any locked files
3. Execute emcopy to mirror the source to its corresponding destination with a time stamp variance allowance of 2 seconds for speed and resiliency
4. Sample the copied files to 'spot check' any time stamp variances
5. Execute in the context of an Administrator

Features planned for development:
6. Enable Volume Shadow Copy (VSS) at Source machines if it has been disabled, and reverse the action when done with copying
7. Trigger Remote Powershell to launch execution from a middle server (a "jump box" that is not a source nor destination)
if the provided Source is detected as a Universal Naming Convention (UNC) path instead of a local file system (LFS) path

Limitations:
1. This iteration requires that script is triggered from a local Windows machine with Internet access (no proxies)
2. Source must be LFS and Destination could either be LFS or UNC
#>

# Specify Source and Destination
$source="C:\Users\Test\Desktop\Clients" # Must be a LFS path
$destination="C:\Test\Kim\Desktop\Test"
$block="$source $destination"

# Emcopy switches
$switches="/o /secforce /de /sd /c /r:0 /w:0 /th 128 /s /purge /sdd /stream"
<# Switch explanations
/s copies sub directories
/purge removes files and directories from the destination that do not exist in the source.
/sdd forces the target directories dates to be synchronized with the source directory.
/de Compares both file size and last modification time when deciding to update a file, updates it if either have been changed.
/cm md5 - checks the file content after copying using and md5 comparison of the source and destination.
/o copies the files owner, without this the account used for the copy will be the owner
/secforce overwrites the destination security settings with the source security settings (no merging, actual update of security settings at the destination)
/sd preserves security, the file isn't copied if an error occurs during security settings.
/th 128 - Uses 128 threads, default is 64
/r:0 retries zero times
/w:0 is the wait time in seconds between retries
/c will allow the process to continue after the retries
/log:filename option allows to redirect the console messages to a new file.
/log+:filename option appends the new messages to an existing file.
/stream option enables the copie of files and directories datastreams. Without that option only the main datastream of files are copied.
#>

# Initialize log files
$dateStamp = Get-Date -Format "yyyy-MM-dd-hhmmss"
$scriptName=$MyInvocation.MyCommand.Path
$scriptPath=Split-Path -Path $scriptName
$logPath="$scriptPath\emcopy_logs"
$logFile="$logPath\emcopy-log-$dateStamp.txt"
$log=" /LOG+:$logFile"
$lockedFilesReport="$logPath\_locked-files-log-$dateStamp.txt"
$pathErrorsLog="$logPath`\_path-errors-log-$dateStamp.txt"

# Init other variables
$sampleSize=1000;
$GLOBAL:shadowMount="C:\shadowcopy"

################################## Excuting Program as an Administrator ####################################
# Get the ID and security principal of the current user account
$myWindowsID=[System.Security.Principal.WindowsIdentity]::GetCurrent()
$myWindowsPrincipal=new-object System.Security.Principal.WindowsPrincipal($myWindowsID)

# Get the security principal for the Administrator role
$adminRole=[System.Security.Principal.WindowsBuiltInRole]::Administrator

# Check to see if we are currently running "as Administrator"
if ($myWindowsPrincipal.IsInRole($adminRole))
{
# We are running "as Administrator" - so change the title and background color to indicate this
$Host.UI.RawUI.WindowTitle = $myInvocation.MyCommand.Definition + "(Elevated)"
$Host.UI.RawUI.BackgroundColor = "White"
clear-host
}
else
{
# We are not running "as Administrator" - so relaunch as administrator

# Create a new process object that starts PowerShell
$newProcess = new-object System.Diagnostics.ProcessStartInfo "PowerShell";

# Specify the current script path and name as a parameter
$newProcess.Arguments = $myInvocation.MyCommand.Definition;

# Indicate that the process should be elevated
$newProcess.Verb = "runas";

# Start the new process
[System.Diagnostics.Process]::Start($newProcess);

# Exit from the current, unelevated, process
exit
}

Write-Host -NoNewLine "Running as Administrator..."
################################## Excuting Program as an Administrator ####################################

################################## Commence Programming Sequence ####################################
New-Item -ItemType Directory -Force -Path $logpath;
"Executing copying tasks..."

function createShadow(){
[cmdletbinding()]
param(
[string]$targetVolume="C:\"
)
if (!($targetVolume -like "*\")){$targetVolume+="\"}
$shadowCopyClass=[WMICLASS]"root\cimv2:win32_shadowcopy"
$thisSnapshot = $shadowCopyClass.Create($targetVolume, "ClientAccessible")
$thisShadow = Get-WmiObject Win32_ShadowCopy | Where-Object { $_.ID -eq $thisSnapshot.ShadowID }
$thisShadowPath = $thisShadow.DeviceObject + "\"
C:
cmd /c mklink /d $shadowMount $thisShadowPath
"Shadow of $targetVolume has been made and it's accessible at this local file system (LFS): $shadowMount."
#copyLockedFiles; # this function is to be developed: retrieve lock files list, copy each item on list
#deleteShadow $thisShadow $thisShadowMount

# Export variables
$GLOBAL:shadow=$thisShadow;
}

function deleteShadow(){
# Remove symlink
(Get-Item $shadowMount).Delete()

# delete single instance of volume snapshots
$shadow.Delete()

# Delete all instances of volume snapshots
#Get-WmiObject Win32_ShadowCopy | % {$_.delete()}

"Shadow link $shadowMount has been removed."
}

function logPathError($pathError){
Add-Content $pathErrorsLog "$pathError";
}

function validateDirectory($dirToValidate){
if(Test-Path -Path $dirToValidate){return $True}
else{return $False;}
}

function createDirectory($dir){
# Create folder if it doesn't exist
if(!(validateDirectory $dir)){
New-Item -path $dir -type directory
}
}

function validateSourceAndDestination($thisblock){
$spacesCount=($thisblock.Split(' ')).Count-1
if ($spacesCount -eq 1){
$GLOBAL:source,$GLOBAL:destination=$thisblock.split(' ');
$sourceTest=validateDirectory $source
$destinationTest=validateDirectory $destination
if ($sourceTest -and $destinationTest){
return $True;
}
else {
if (!($sourceTest)){
logPathError "Source: $source";
return $False;
}
if (!($destinationTest)){
$createDestinationPath=(Read-Host -Prompt "Destination: $destination does not exist.`nType 'y' or 'yes' to create.");
if ($createDestinationPath -like 'yes' -or $createDestinationPath -like 'y'){
createDirectory $destination;
return $True;
}
else{
logPathError "Destination: $destination";
return $False
}
}
return $False;
}
}
else {
logPathError $thisblock;
return $False;
}
}

function translateSource{
param(
[string]$uncPath
)
$uri = new-object System.Uri($uncPath)
$thisLocalPath=$uri.LocalPath
#$thisHost=$uri.Host
$GLOBAL:sourceVolume="$((Get-Item $thisLocalPath).PSDrive.Name)`:"
$GLOBAL:translatedSource=$thisLocalPath -replace "$sourceVolume", $shadowMount
}

function sampleTimeStamp{
# Enable Remote to remote symlink following if it's not already set
$r2rEnabled=fsutil behavior query SymlinkEvaluation | select-string -Pattern "Remote to remote symbolic links are enabled."
if (!($r2rEnabled)){fsutil behavior set SymlinkEvaluation R2R:1;}

#$sourceFiles=Get-ChildItem $source -recurse | ? { !($_.PsIsContainer -and $_.FullName -notmatch 'archive' -and $_.FullName.Length -lt 260) } | get-random -count $sampleSize | % {$_.FullName}
$sourceFiles=Get-ChildItem $source -recurse | get-random -count $sampleSize | % {$_.FullName}
$commonDenominatingPaths=$sourceFiles | %{$_.replace($source,'')}
$destinationFiles=$commonDenominatingPaths | %{"$destination"+"$_";}
$badStamps=0;

"Checking a sample of $sampleSize for any time stamp inaccuracy..."

for ($i=0;$i -lt $sourceFiles.length;$i++){
$sourceFile=$sourceFiles[$i];
$destinationFile=$destinationFiles[$i];
if ($destinationFile.Length -lt 260){
$sourceTimeStamp=(gi $sourceFile).LastWriteTime;
$destinationTimeStamp=(gi $destinationFile).LastWriteTime;
if ($sourceTimeStamp -eq $destinationTimeStamp){
#$output+="`r`n$destinationFile is GOOD";
} else {
$output+="`r`n$destinationFile timestamp of $destinationTimeStamp DOES NOT MATCH its source $sourceFile timestamp of $sourceTimeStamp";
$badStamps++;
}
} else {
$output+="`r`nException: $destinationFile length is $($destinationFile.Length)";
}
}
$output="`r`n`r`n------------$((($sourceFiles.Length-$badStamps)/$sourceFiles.Length).tostring('P')) of the files in a sample of $($sourceFiles.Length) are having accurate time stamps--------------`n"+$output;
Add-Content $logFile $output;
}

function checkDiskFree{
<#
Excerpt from http://technet.microsoft.com/en-us/library/ee692290(WS.10).aspx:

"For volumes less than 500 megabytes, the minimum is 50 megabytes of free space.
For volumes more than 500 megabytes, the minimum is 320 megabytes of free space.
It is recommended that least 1 gigabyte of free disk space on each volume if the volume size is more than 1 gigabyte."

#>

# Import variables
$thisNode="localhost"
$thisVolume=$sourceVolume

# Obtain disk information
$diskObject = Get-WmiObject Win32_LogicalDisk -ComputerName $thisNode -Filter "DeviceID='$thisVolume'"
$diskFree=[Math]::Round($diskObject.FreeSpace / 1MB)
$diskSize=[Math]::Round($diskObject.Size / 1MB)

switch ($diskSize){
{$diskSize -ge 1024} {if ($diskFree -gt 1024){$feasible=$True;}else{$feasible=$False;};;break;}
{$diskSize -ge 500} {if ($diskFree -gt 320){$feasible=$True;}else{$feasible=$False;};;break;}
{$diskSize -lt 500} {if ($diskFree -gt 50){$feasible=$True;}else{$feasible=$False;};break;}
}

return $feasible
}

function expandZipfile($file, $destination){
$shell = new-object -com shell.application
$zip = $shell.NameSpace($file)

foreach($item in $zip.items()){
$shell.Namespace($destination).copyhere($item)
}
}

function installEmcopy{
$emcopyIsInstalled=(Get-Command emcopy.exe -ErrorAction SilentlyContinue) # Deterministic check on whether emcopy is already available on this system
if (!($emcopyIsInstalled)){
$tempDir="C:\Temp";
$extractionDir="C:\Windows"
$source = "https://kimconnect.com/wp-content/uploads/2019/08/emcopy.zip";
$destinationFile = "$tempDir\emcopy.zip";
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
New-Item -ItemType Directory -Force -Path $tempDir
New-Item -ItemType Directory -Force -Path $extractionDir
$webclient = New-Object System.Net.WebClient;
$WebClient.DownloadFile($source,$destinationFile);
expandZipfile $destinationFile -Destination $extractionDir
}else{
"EMCOPY is currently available in this system.`n";
}
}

function setACL{
get-acl -Path $source | Set-Acl -Path $destination # This command requires PowerShell 5
}

function startEmcopy{
param(
[string]$sourceAndDestination
)
"Emcopy has started..."
try{
invoke-expression "emcopy.exe $sourceAndDestination $switches $log";
}
catch{
# Record any errors into log and continue to next item
continue;
}
"`r`nEmcopy process has finished. Log is now generated at: $log";
}

Function proceed{
# Start the timer
Add-Content $logFile "`n`n`n---------------------------------Job Started: $dateStamp---------------------------------";
$totalTime=0;
$stopWatch= [System.Diagnostics.Stopwatch]::StartNew()
"Powershell version detected: $($PSVersionTable.PSVersion.Major)`.$($PSVersionTable.PSVersion.Minor)"
if (validateSourceAndDestination $block){
translateSource $source;
#"Shadow Copy source: $tranlatedSource`nDestination: $destination";
$translatedBlock="$translatedSource $destination";
if (checkDiskFree){
createShadow $sourceVolume;
installEmcopy;
startEmcopy $translatedBlock;
deleteShadow;
sampleTimeStamp;
"Program is completed."
"Log for this activity has been generated at $logFile"
}else{"Not enough disk space to create a VSS snapshot.`r`Program is aborted."}
}

# Getting storage estimate
$storage=(Get-ChildItem $source -Recurse | Measure-Object -Property Length -Sum -ErrorAction Stop).Sum / 1GB;
$storageDestination=(Get-ChildItem $destination -Recurse | Measure-Object -Property Length -Sum -ErrorAction Stop).Sum / 1GB;

# Stop the timer
#$elapsedSeconds=$stopWatch.Elapsed.TotalSeconds;
$totalTime=$stopWatch.Elapsed.TotalSeconds;
#$elapsedDisplay=([timespan]::fromseconds($elapsedSeconds)).ToString().Split('.')[0]
#$totalTime+=$elapsedSeconds;

# Add total time display to log
$timeDisplay = ([timespan]::fromseconds($totalTime)).ToString()
Add-Content $logFile "`r`n------------------------------Total Time Elapsed: $timeDisplay------------------------------";

# Record overall speed to log
$speed=$storage/($totalTime/360)
Add-Content $logFile "`r`n---------------------------Aggregate speed: $speed GB per hour---------------------------";

# Compare storage results
Add-Content $logFile "`r`n---------------------------Comparing: Source $storage GB vs Destination $storageDestination GB---------------------------";
}

proceed;
################################## Main Programming Sequence ####################################

# cmd /c pause | out-null;
Version 0.14
<#
.Description: File_Copy_Script Version: 0.14

Purpose: this PowerShell Script is to efficiently mirror large batches of files using Robocopy in conjunction with Volume Shadow Services

Current Features:
1. Check for any errors on the Sources or Destinations and generate a report of any extra spaces in UNC
2. Create a snapshot of the source volume using Shadow Copy to capture any locked files
3. Execute robocopy to mirror the source to its corresponding destination with a time stamp variance allowance of 2 seconds for speed and resiliency
4. Sample the copied files to 'spot check' any time stamp variances

Features planned for development:
5. Execute in the context of a Domain Administrator
6. Enable Volume Shadow Copy (VSS) at Source machines if it has been disabled, and reverse the action when done with copying
7. Trigger Remote Powershell to launch execution from a middle server (a "jump box" that is not a source nor destination)
if the provided Source is detected as a Universal Naming Convention (UNC) path instead of a local file system (LFS) path

Limitations:
- Require robocopy version XP016 or higher
- Only works if the source is LFS and destination is UNC
#>


$source="C:\Users\yamama\Desktop\test" # LFS path, only
$destination="\\FILESHERVER01\test" # UNC path, only
$block="$source $destination"

$switches="/MIR /SEC /DCOPY:T /R:0 /W:0 /XO /FFT /TBD /NP "
$dateStamp = Get-Date -Format "yyyy-MM-dd-hhmmss"
$scriptName=$MyInvocation.MyCommand.Path
$scriptPath=Split-Path -Path $scriptName
$logPath="$scriptPath\robocopy_logs"
$logFile="$logPath\robocopy-log-$dateStamp.txt"
$log="/LOG+:$logFile"
$lockedFilesReport="$logPath\_locked-files-log-$dateStamp.txt"
$pathErrorsLog="$logPath`\_path-errors-log-$dateStamp.txt"
$sampleSize=1000;
$GLOBAL:shadowMount="C:\shadowcopy"

# Get the ID and security principal of the current user account
$myWindowsID=[System.Security.Principal.WindowsIdentity]::GetCurrent()
$myWindowsPrincipal=new-object System.Security.Principal.WindowsPrincipal($myWindowsID)

# Get the security principal for the Administrator role
$adminRole=[System.Security.Principal.WindowsBuiltInRole]::Administrator

# Check to see if we are currently running "as Administrator"
if ($myWindowsPrincipal.IsInRole($adminRole))
{
# We are running "as Administrator" - so change the title and background color to indicate this
$Host.UI.RawUI.WindowTitle = $myInvocation.MyCommand.Definition + "(Elevated)"
$Host.UI.RawUI.BackgroundColor = "Black"
clear-host
}
else
{
# We are not running "as Administrator" - so relaunch as administrator

# Create a new process object that starts PowerShell
$newProcess = new-object System.Diagnostics.ProcessStartInfo "PowerShell";

# Specify the current script path and name as a parameter
$newProcess.Arguments = $myInvocation.MyCommand.Definition;

# Indicate that the process should be elevated
$newProcess.Verb = "runas";

# Start the new process
[System.Diagnostics.Process]::Start($newProcess);

# Exit from the current, unelevated, process
exit
}

# Run your code that needs to be elevated here
Write-Host -NoNewLine "Now Running as Administrator..."
#$null = $Host.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown")

New-Item -ItemType Directory -Force -Path $logpath;
"Executing copying tasks..."

function createShadow(){
[cmdletbinding()]
param(
[string]$targetVolume="C:\"
)
if (!($targetVolume -like "*\")){$targetVolume+="\"}
$shadowCopyClass=[WMICLASS]"root\cimv2:win32_shadowcopy"
$thisSnapshot = $shadowCopyClass.Create($targetVolume, "ClientAccessible")
$thisShadow = Get-WmiObject Win32_ShadowCopy | Where-Object { $_.ID -eq $thisSnapshot.ShadowID }
$thisShadowPath = $thisShadow.DeviceObject + "\"
cmd /c mklink /d $shadowMount $thisShadowPath
"Shadow of $targetVolume has been made and it's accessible at this local file system (LFS): $shadowMount."
#copyLockedFiles; # this function is to be developed: retrieve lock files list, copy each item on list
#deleteShadow $thisShadow $thisShadowMount

# Export variables
$GLOBAL:shadow=$thisShadow;
}

function deleteShadow(){
# Remove symlink
(Get-Item $shadowMount).Delete()

# delete single instance of volume snapshots
$shadow.Delete()

# Delete all instances of volume snapshots
#Get-WmiObject Win32_ShadowCopy | % {$_.delete()}

"Shadow link $shadowMount has been removed."
}

function logPathError($pathError){
Add-Content $pathErrorsLog "$pathError";
}

function validateDirectory($dirToValidate){
if(Test-Path -Path $dirToValidate){return $True}
else{return $False;}
}

function createDirectory($dir){
# Create folder if it doesn't exist
if(!(validateDirectory $dir)){
New-Item -path $dir -type directory
}
}

function validateSourceAndDestination($thisblock){
$spacesCount=($thisblock.Split(' ')).Count-1
if ($spacesCount -eq 1){
$GLOBAL:source,$GLOBAL:destination=$thisblock.split(' ');
$sourceTest=validateDirectory $source
$destinationTest=validateDirectory $destination
if ($sourceTest -and $destinationTest){
return $True;
}
else {
if (!($sourceTest)){
logPathError "Source: $source";
return $False;
}
if (!($destinationTest)){
$createDestinationPath=(Read-Host -Prompt "Destination: $destination does not exist.`nType 'y' or 'yes' to create.");
if ($createDestinationPath -like 'yes' -or $createDestinationPath -like 'y'){
createDirectory $destination;
return $True;
}
else{
logPathError "Destination: $destination";
return $False
}
}
return $False;
}
}
else {
logPathError $thisblock;
return $False;
}
}

function translateSource{
param(
[string]$uncPath
)
$uri = new-object System.Uri($uncPath)
$thisLocalPath=$uri.LocalPath
#$thisHost=$uri.Host
$GLOBAL:sourceVolume="$((Get-Item $thisLocalPath).PSDrive.Name)`:"
$GLOBAL:translatedSource=$thisLocalPath -replace "$sourceVolume", $shadowMount
}

function reportLockedFiles($unc){
$files=Get-ChildItem -Path $unc -Recurse | where { ! $_.PSIsContainer }| % {$_.FullName}
#$x=$files[0]
#simulateLockedFile "$x"
$lockedFiles=$files | % { if(isFileLocked $_){$_; } }
if ($lockedFiles){
createDirectory $logPath;
Add-Content $lockedFilesReport $lockedFiles;
}
#$lockedFile.Close();
}

function startRobocopy{
param(
[string]$sourceAndDestination
)
Add-Content $logFile "`n`n`n---------------------------------Job Started: $dateStamp---------------------------------";
$totalTime=0;
"Robocopy has started..."
$stopWatch= [System.Diagnostics.Stopwatch]::StartNew()
try{
invoke-expression "robocopy $sourceAndDestination $switches $log | Out-Null";
#invoke-expression "robocopy $item $switches $log | Out-Null";
}
catch{
# Record any errors into log and continue to next item
continue;
}
#"Checking locked files in $source..."
#reportLockedFiles $source
$elapsedSeconds=$stopWatch.Elapsed.TotalSeconds;
$elapsedDisplay=([timespan]::fromseconds($elapsedSeconds)).ToString().Split('.')[0]
#Add-Content $logFile "Block Time Elapsed: $elapsedDisplay";
$totalTime+=$elapsedSeconds;
"`r`nRobocopy is done."
$timeDisplay = ([timespan]::fromseconds($totalTime)).ToString()
#$timeDisplay = $timeString.substring(0, $timeString.lastIndexOf("."));
Add-Content $logFile "`r`n------------------------------Total Time Elapsed: $timeDisplay------------------------------";
}

function sampleTimeStamp{
$sourceFiles=Get-ChildItem $source -recurse | ? { !($_.PsIsContainer -and $_.FullName -notmatch 'archive') } | get-random -count $sampleSize | % {$_.FullName}
$commonDenominatingPaths=$sourceFiles | %{$_.replace($source,'')}
$destinationFiles=$commonDenominatingPaths | %{"$destination"+"$_";}
$badStamps=0;

"Checking a sample of $sampleSize for any time stamp inaccuracy..."

for ($i=0;$i -lt $sourceFiles.length;$i++){
$sourceFile=$sourceFiles[$i];
$destinationFile=$destinationFiles[$i];
$sourceTimeStamp=(gi $sourceFile).LastWriteTime;
$destinationTimeStamp=(gi $destinationFile).LastWriteTime;
if ($sourceTimeStamp -eq $destinationTimeStamp){
#$output+="`r`n$destinationFile is GOOD";
}
else {
$output+="`r`n$destinationFile is BAD";
$badStamps++;
}
}
$output="`r`n`r`n------------$((($sourceFiles.Length-$badStamps)/$sourceFiles.Length).tostring('P')) of the files in a sample of $($sourceFiles.Length) are having accurate time stamps--------------`n"+$output;
Add-Content $logFile $output;
}

function checkDiskFree{
<#
Excerpt from http://technet.microsoft.com/en-us/library/ee692290(WS.10).aspx:

"For volumes less than 500 megabytes, the minimum is 50 megabytes of free space.
For volumes more than 500 megabytes, the minimum is 320 megabytes of free space.
It is recommended that least 1 gigabyte of free disk space on each volume if the volume size is more than 1 gigabyte."

#>

# Import variables
$thisNode="localhost"
$thisVolume=$sourceVolume

# Obtain disk information
$diskObject = Get-WmiObject Win32_LogicalDisk -ComputerName $thisNode -Filter "DeviceID='$thisVolume'"
$diskFree=[Math]::Round($diskObject.FreeSpace / 1MB)
$diskSize=[Math]::Round($diskObject.Size / 1MB)

switch ($diskSize){
{$diskSize -ge 1024} {if ($diskFree -gt 1024){$feasible=$True;}else{$feasible=$False;};;break;}
{$diskSize -ge 500} {if ($diskFree -gt 320){$feasible=$True;}else{$feasible=$False;};;break;}
{$diskSize -lt 500} {if ($diskFree -gt 50){$feasible=$True;}else{$feasible=$False;};break;}
}

return $feasible
}

Function proceed{
if (validateSourceAndDestination $block){
translateSource $source;
#"Shadow Copy source: $tranlatedSource`nDestination: $destination";
$translatedBlock="$translatedSource $destination";
if (checkDiskFree){
createShadow $sourceVolume;
startRobocopy $translatedBlock;
deleteShadow;
sampleTimeStamp;
"Program is completed."
"Log for this activity has been generated at $logFile"
}else{"Not enough disk space to create a VSS snapshot.`r`Program is aborted."}
}
}

proceed;
pause;

Securing Windows Remote Desktop Services

secpol.msc > Local Policies > User Rights Assignments > double-click “Allow Log on through Remote Desktop Services” > remove Administrators and Remote Desktop Users > Add a customized group and/or users

gpedit.msc > Computer Configuration > Adminstrative Templates > Windows Components > Remote Desktop Services > Remote Desktop Session host > security > change these settings:
– Set client encryption level = High
– Require secure RPC communication = Enabled
– Require use of specific security layer for remote (RDP) connections = SSL
– Require user authentication for remote connections by using Network Level Authentication = Enabled