HowTo: Mirror Project Documents between Datasources

Be sure to check out my Scripting4Crypto initiative. It’s a fun way to get into using cryptocurrencies all while getting your PowerShell needs met.

In this post, I am going to expand on my previous post, “HowTo: Mirror Project Folders between Datasources“,  and mirror the documents contained in the folders of a Work Area in one datasource to a corresponding Work Area in a second datasource. Once again, we will take advantage of the ProjectWise session capabilities. This will allow us to be logged into both datasources simultaneously, and switch back and forth.

A few caveats regarding this post; it assumes you are logged into the source and target datasources and that the folders have been mirrored per my previous post. Also, we will not be bringing over document versions, and we will not be recreating and populating flat sets.

NOTE: We will be using the source folder objects ($SourceFolders) for this process.

One final note, I have added a lot of Write-Host statements taking advantage of the ability to set the color of the text. This is for demonstration purposed only. These would be Write-Verbose, Write-Warning and Write-Error for a production script.

We will be using the following cmdlets to accomplish this task. All of the ProjectWise related cmdlets are available using the PWPS_DAB module. At the time of this post, I am using version 1.21.7.0. Take a look at the help for each of the cmdlets to become familiar with their functionality, available parameters, etc.

• Get-PWSessions
• Get-PWCurrentDatasource
• Set-PWSession
• Get-PWDocumentsBySearch
• CheckOut-PWDocuments
• Remove-PWDocuments
• Get-FileHash
• Copy-PWDocumentsBetweenDatasources
• Update-PWDocumentFile

Get ALL Documents for Each Work Area

Here we will be obtaining a list of ALL documents within both the Source and Target Work Areas. We will use the Set-PWSession to switch between the datasources as we go. Getting all of the documents at one time eliminates the need to make many calls to the database to continually request this data. Once we obtain all of the document objects, we can easily select only the document objects we want to work with from either array.

Switching Between Datasources

We can use the Get-PWCurrentDatasource to determine which datasource is active. Then we will use the Set-PWSession to set the one we want active.  In this instance we are logged into the Target datasource but we want to be in the Source datasource.

Get Source Documents

The first thing we need to do is ensure the Source datasource is active by switching to it using the Set-PWSession cmdlet. Then we will use the Get-PWDocumentsBySearch cmdlet to retrieve all of the documents from the Source Work Area. Again, using Write-Host to return information to the console.  As you can see, we are copying out all of the source  documents to the local working directory. This is to facilitate the retrieval of the file checksum later in the script.

#region Get ALL Source documents.
try {
    # Set the Source datasource active.
    if( -not ((Get-PWCurrentDatasource) -eq $SourceDatasource)) {
        Write-Host "Switching to '$SourceDatasource' datasource ProjectWise Session." -ForegroundColor Green
        $null = Set-PWSession $SourceDatasource
    }

    # Get ALL documents from source Work Area / folder.
    Write-Host "Getting source documents from '$SourceFolderRoot'." -ForegroundColor Cyan
    $pwDocuments_Source = Get-PWDocumentsBySearch -FolderPath $SourceFolderRoot -GetAttributes -WarningAction SilentlyContinue |
    CheckOut-PWDocuments -CopyOut -WarningAction SilentlyContinue

    # If no source documents are found, ensure the corresponding Work Area / folder is empty.
    if( -not ($pwDocuments_Source)) {
        Write-Host " - No source documents returned for folder." -ForegroundColor Magenta 
    } else {
        Write-Host " - '$($pwDocuments_Source.Count)' source documents returned." -ForegroundColor Cyan
    }
} catch {
    Write-Warning -Message "Error occurred while getting source documents."
    throw $($Error[0].Exception.Message)
}
#endregion Get ALL Source documents.

Get Target Documents

Now, we simply repeat the process for the Target Work Area.

#region Get ALL Target documents.
try {
    # Set the Target datasource active.
    if( -not ((Get-PWCurrentDatasource) -eq $TargetDatasource)) {
        Write-Host "Switching to '$TargetDatasource' datasource ProjectWise Session." -ForegroundColor Green
        $null = Set-PWSession $TargetDatasource
    }

    # Get ALL documents from target Work Area / folder.
    Write-Host "Getting target documents from '$DestinationFolderRoot'." -ForegroundColor Cyan
    $pwDocuments_Target = Get-PWDocumentsBySearch -FolderPath $DestinationFolderRoot -GetAttributes -WarningAction SilentlyContinue

    # If no source documents are found, ensure the corresponding folder is empty.
    if( -not ($pwDocuments_Target)) {
        Write-Host " - No target documents returned for folder." -ForegroundColor Magenta 
    } else {
        Write-Host " - '$($pwDocuments_Target.Count)' target documents returned." -ForegroundColor Cyan
    }
} catch {
    Write-Warning -Message "Error occurred while getting target documents."
    throw $($Error[0].Exception.Message)
}
#endregion Get ALL Target documents.

The following shows the document counts retrieved for both the source and target projects.

Compare Source to Target

During this process we will focus on a few folders to demonstrate the functionality.

First, the BIM folder contains documents in the source but not the target. So, the documents will need to be copied over.

Second, the ‘BSI900 – Request for Bid.docx’ was updated within the source. The document within the target will be updated, by replacing the associated physical file.

Third, the ‘Lynn Strunk Mechanical Consultants’ folder no longer contains any documents within the source datasource. The corresponding target folder will need to be emptied.

---

Here we will loop through each of the source folders and retrieve the documents from the $pwDocuments_Source array which reside in each folder. We will need to switch to the source datasource.  Also, for each time through the loop, we will create two arraylists; one to store documents to be copied and one to store documents to be updated.

<# Loop through each source folder to obtain all document objects.
Switch to target datasource and do the same for the corresponding folder.
If the target folder does not contain any documents, copy all.
Otherwise, compare each document to determine if it is up to date
by comparing date and size information. #>
foreach($sourceFolder in $SourceFolders){

    try { 
        # Set the Source datasource active.
        if( -not ((Get-PWCurrentDatasource) -eq $SourceDatasource)) {
            Write-Host "Switching to '$SourceDatasource' datasource ProjectWise Session." -ForegroundColor Green
            $null = Set-PWSession $SourceDatasource
        }

        # ArrayList to store documents to copy.
        $pwDocsToCopy = [Collections.ArrayList]::New()
        $pwDocsToUpdate = [Collections.ArrayList]::New()
        $SourceFolderIsEmpty = $false

        Write-Host "Source folder: '$($sourceFolder.Name)' ; FullPath: $($sourceFolder.FullPath)" -ForegroundColor Cyan

        try {
            # Get documents from source for current folder.
            $currentDocs_Source = $pwDocuments_Source | Where-Object FolderPath -eq $SourceFolder.FullPath

If no source documents are returned, we need to ensure the corresponding target folder is empty as well. We flag the folder to be emptied by setting variable $SourceFolderIsEmpty to true.

            # If no source documents are found, ensure the corresponding folder is empty. 
            if( -not ($currentDocs_Source)) {
                Write-Host " - No source documents returned for folder." -ForegroundColor Magenta
                $SourceFolderIsEmpty = $true 
            } else {
                Write-Host " - '$($currentDocs_Source.Count)' source documents returned." -ForegroundColor Cyan
            }
        } catch {
            Write-Warning -Message "Error occurred while getting source documents."
            throw $($Error[0].Exception.Message)
        }

Now, we need to switch to the target datasource. We will do the same thing and retrieve all documents in the corresponding target folder. If the target folder is empty, we will add each of the source documents to the $pwDocsToCopy arraylist. Otherwise, we will start the process of comparing each document.

        # Set the Target datasource active.
        if( -not ((Get-PWCurrentDatasource) -eq $TargetDatasource)) {
            Write-Host "Switching to '$TargetDatasource' datasource ProjectWise Session." -ForegroundColor Green
            $null = Set-PWSession $TargetDatasource
        } 

        # Get the Target Work Area folder name. 
        $tempDestination = $SourceFolder.FullPath.Replace($SourceFolderRoot, $DestinationFolderRoot)
        Write-Host "Target folder: '$($tempDestination)'" -ForegroundColor Cyan

        try {
            # Get documents from target for current folder.
            $currentDocs_Target = $pwDocuments_Target | Where-Object FolderPath -eq $tempDestination

            if( -not ($currentDocs_Target)) {
                Write-Host " - is empty. Copy all documents." -ForegroundColor Cyan

                # Add each document to the pwDocsToCopy ArrayList.
                foreach($d in $currentDocs_Source) {
                    $null = $pwDocsToCopy.add($d)
                }
            } else {
                Write-Host " - '$($currentDocs_Target.Count)' target documents returned." -ForegroundColor Cyan

The following shows the BIM folder requiring all documents to be copied over to the target datasource.

If the source folder was empty, here is where we will remove any documents in the target folder using the Remove-PWDocuments cmdlet. Then we will continue to the next source folder in the loop.

                #region EMPTY Target Folder
                # If source folder is empty, remove any documents from the corresponding target folder.
                if($SourceFolderIsEmpty){
                    Write-Warning -Message "'$($sourceFolder.Name)' is empty. Removing all documents from corresponding target folder."
                    try{
                         Write-Host "'$($sourceFolder.Name)' is empty. Removing all documents from corresponding target folder." -ForegroundColor DarkCyan
                        $null = Remove-PWDocuments -InputDocument $currentDocs_Target -ErrorAction Stop
                    } catch {
                        Write-Warning -Message "Error occurred while attempting to remove documents from '$tempDestination'. $($Error[0].Exception.Message)"
                    }
                    continue
                } 
                #endregion EMPTY Target Folder

The following shows the ‘Lynn Strunk Mechanical Consultants’ folder no longer contains any documents within the source. Therefore, all documents are removed from the target folder.

Let’s get into the compare process.  I am going to compare each file three different ways, by file updated date, file size, and file checksum. This is to demonstrate each of the methods.   If this was in a function, you could specify which method to use. Keep in mind these are the methods I developed. You may have a better way. If so, please share.

Here we will loop through each of the source documents returned for the current folder. For each document we will capture the file updated date and the file size. We will also determine the file checksum using the Get-FileHash cmldet. This does require that the document be copied out to the local working directory.

                #region COMPARE DOCUMENTS
                foreach($cds in $currentDocs_Source) {
                    # Source document info
                    Write-Host "Source document: '$($cds.FullPath)'" -ForegroundColor DarkCyan

                    # If current document is a flat set file, skip.
                    if($cds.IsSet -eq $true) { 
                        Write-Host " - '$($cds.Name)' is a set file. Skipping.'" -ForegroundColor Magenta 
                        continue
                    }

                    $sourceFileUpdateDate = $cds.FileUpdateDate
                    $sourceFileSize = $cds.FileSize
                    $sourceFileCheckSum = Get-FileHash -Path $cds.CheckedOutLocalFileName -Algorithm SHA256
                    
                    Write-Host "File Name: '$($cds.Name)'" -ForegroundColor DarkCyan
                    Write-Host " - file updated date '$sourceFileUpdateDate'." -ForegroundColor DarkCyan 
                    Write-Host " - file size '$sourceFileSize'." -ForegroundColor DarkCyan
                    Write-Host " - file checksum '$($sourceFileCheckSum.Hash)'." -ForegroundColor DarkCyan

The following is the message received when a document is a set file.

For each source document we will determine if a corresponding target document exists. If it does, we will copy out the document to the local working directory to be used to get the checksum. If the source document is a set file, we will skip it and continue to the next document in the loop.  If there isn’t a target document found, we will add the current source document to the $pwDocsToCopy arraylist. And finally, if the source document does not have a physical file associated with it, we will continue to the next source document in the loop.

                   # Target document info
                   $tempFolder = Split-Path $cds.FolderPath -Leaf
                   $tempFolder = $tempFolder.Replace($SourceFolderRoot, $DestinationFolderRoot)

                   $cdt = $currentDocs_Target | 
                   Where-Object { ($_.FolderPath).contains($tempFolder) -and $_.Name -eq $cds.Name } |
                   CheckOut-PWDocuments -CopyOut -WarningAction SilentlyContinue

                   # If document is not found within the target folder, add to the pwDocsToCopy ArrayList.
                   if(-not ($cdt)) {
                       if($cds.FileName){ 
                           Write-Host " - '$($cds.Name)' not found in target folder. Adding to documents to copy arraylist." -ForegroundColor Magenta 
                           $null = $pwDocsToCopy.add($cds)
                       } else {
                           Write-Host " - '$($cds.Name)' does not have a file associated with it.'" -ForegroundColor Magenta
                       }
                       continue
                   }

If a target document is returned we will capture the file updated date, the file size and the file checksum.

                $targetFileUpdateDate = $cdt.FileUpdateDate
                $targetFileSize = $cdt.FileSize
                $targetFileCheckSum = Get-FileHash -Path $cdt.CheckedOutLocalFileName -Algorithm SHA256

                Write-Host "Target document: '$($cdt.FullPath)'" -ForegroundColor DarkCyan
                Write-Host "File Name: '$($cdt.Name)'" -ForegroundColor DarkCyan
                Write-Host " - file updated date '$targetFileUpdateDate'." -ForegroundColor DarkCyan
                Write-Host " - file size '$targetFileSize'." -ForegroundColor DarkCyan
                Write-Host " - file checksum: '$($targetFileCheckSum.Hash)'." -ForegroundColor DarkCyan

Compare Files

Here we will do simple comparisons. If one fails, an error is thrown and the current source document will be added to the $pwDocsToUpdate arraylist.

                try { 
                    # Compare File Update Dates
                    if($sourceFileUpdateDate -gt $targetFileUpdateDate){ 
                        throw "Source file is newer than the Target file."
                    } else {
                        Write-Host "Source file is older than the target file." -ForegroundColor DarkCyan
                    }
                    # Compare File Size
                    if($sourceFileSize -ne $targetFileSize){
                        throw "File sizes are different."
                    } else {
                        Write-Host "Files are the same size." -ForegroundColor DarkCyan
                    }
                    # Compare File Checksum 
                    if($sourceFileCheckSum.Hash -ne $targetFileCheckSum.Hash){
                        throw "File checksums are different."
                    } else {
                        Write-Host "File checksums are the same." -ForegroundColor DarkCyan
                    }
                } catch {
                    Write-Host "$($Error[0].Exception.Message) Adding to arraylist to update file." -ForegroundColor Magenta

                    if( -not ($pwDocsToUpdate.Contains($cds))) {
                        $null = $pwDocsToUpdate.Add($cds) 
                    }
                    Continue
                }
            } # end foreach($cds in $currentDocs_Source...
            #endregion COMPARE DOCUMENTS
        }
    } catch {
        Write-Warning -Message "Error occurred while getting target documents."
        throw $($Error[0].Exception.Message)
    }

The following shows the ‘BSI900 – Request for Bid.docx’ needs to be updated because the source file is newer than the target.

What we should have ended up with are two arraylists containing source document objects to be either used to update the physical files associated with the corresponding target documents or copied over to the target folder.

Copy Source Documents To Target

Here will will copy the documents to the target folder using the Copy-PWDocumentsBetweenDatasources cmdlet. We will need to switch to the source datasource. We will capture any errors using a try / catch.

    #region Copy documents.
    if($pwDocsToCopy.Count -gt 0){
        try{ 
            # Set the Source datasource active.
            if( -not ((Get-PWCurrentDatasource) -eq $SourceDatasource)) {
                Write-Host "Switching to '$SourceDatasource' datasource ProjectWise Session." -ForegroundColor Green
                $null = Set-PWSession $SourceDatasource
            }
            Write-Host "Copying $($pwDocsToCopy.Count) documents." -ForegroundColor Cyan

            foreach($doc in $pwDocsToCopy){
                $tempFolder = $doc.FolderPath.Replace($SourceFolderRoot, $DestinationFolderRoot)
                $Splat_CopyDocument = @{
                    InputDocument = $doc
                    TargetDatasource = $TargetDatasource
                    TargetFolderPath = $tempFolder
                }
                $results = Copy-PWDocumentsBetweenDatasources @Splat_CopyDocument -ErrorAction Stop -WarningAction Stop
                Write-Host "Copied '$($doc.Name)'." -ForegroundColor Yellow
            }
        } catch {
            Write-Warning -Message "Error occurred while copying documents."
            throw $($Error[0].Exception.Message)
        }
    }
    #endregion Copy documents.

Update Physical Files

Here will will update the target documents with the corresponding physical files from the source documents. We will be using the Update-PWDocumentFile cmdlet. We will need to switch to the target datasource. We will capture any errors using a try / catch.

    #region Update documents.
    if($pwDocsToUpdate.Count -gt 0){
        try { 

            # Set the Target datasource active.
            if( -not ((Get-PWCurrentDatasource) -eq $TargetDatasource)) {
                Write-Host "Switching to '$TargetDatasource' datasource ProjectWise Session." -ForegroundColor Green
                $null = Set-PWSession $TargetDatasource
            } 
            Write-Host "Updating $($pwDocsToUpdate.Count) documents." -ForegroundColor Cyan

            foreach($doc in $pwDocsToUpdate){
                $tempFolder = $doc.FolderPath.Replace($SourceFolderRoot, $DestinationFolderRoot)
                $InputDocument = $currentDocs_Target | Where-Object { ($_.FolderPath).contains($tempFolder) -and $_.Name -eq $doc.Name }
                $NewFilePathName = $doc.CheckedOutLocalFileName

                $Splat_UpdateDocument = @{
                    InputDocuments = $InputDocument
                    NewFilePathName = $NewFilePathName
                    KeepExistingFileName = $true
                }
                $results = Update-PWDocumentFile @Splat_UpdateDocument -ErrorAction Stop
                Write-Host "Updated '$($InputDocument.Name)'." -ForegroundColor Yellow
            }
        } catch {
            Write-Warning -Message "Error occurred while copying documents."
            throw $($Error[0].Exception.Message)
        }
    }
    #endregion Update documents.
    } catch {
        Write-Warning -Message "Error occurred while attempting to copy documents from '$($sourceFolder.FullPath)'. $($Error[0].Exception.Message)"
    }   
}
#endregion DOCUMENTS

Results

The following shows the results of the process.

First, all documents within the source ‘BIM_Models’ folder were copied over to the target datasource.HowTo-MirrorPWFoldersAndDocuments

Second, you can see that the physical file associated with the ‘BSI900 – Request for Bid.docx’ file was replaced in the target datasource.

Third, all documents have been removed from the ‘Lynn Strunk Mechanical Consultants’ folder within the target datasource.

Summary

In this post, we mirrored (copied or updated) the source documents to the corresponding documents within the target datasource. We used the file updated date, file size and file checksum values to compare each document to determine whether or not to update the existing physical file. Again, there are probably other ways to accomplish this task. If you have one, please share.

---

Experiment with it and have fun. And please, let me know if there is a topic you would like to see a post for. Or share a solution you have developed.

Below is a link to the MirrorPWFoldersAndDocuments.ps1 file.

HowTo-MirrorPWFoldersAndDocuments

Hopefully, you find this useful. Please let me know if you have any questions or comments.  If you like this post, please click the Like button at the bottom of the page. And thank you for checking out my blog.