diff --git a/AGENTS.md b/AGENTS.md index dc6aee2..f8d4e02 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -29,11 +29,13 @@ The CI treats any **Error**-severity finding as a failure; warnings are logged b - Framework: **Pester v5** (`#Requires -Modules @{ModuleName='Pester'; ModuleVersion='5.0.0'}` in every test file — a floor, not an exact pin) - Test files: `Tests/*.Tests.ps1`, one per exported function plus 4 meta files (`Help.Tests.ps1`, `Module.Tests.ps1`, `Pester.Tests.ps1`, `ScriptAnalyzer.Tests.ps1`) -- Tags: `Build` (the 4 meta files), `UnitTests` (everything else — this module has no background jobs/network/filesystem I/O to integration-test) +- Tags: `Build` (the 4 meta files), `UnitTests` (mocked `Write-Progress`, precise parameter-shape assertions), `IntegrationTests` (real `Write-Progress`, no mocking at all) +- `IntegrationTests` (in `Write-xProgress.Tests.ps1` and `Write-xJobProgress.Tests.ps1`) run the module's real functions inside real `Start-Job` background jobs over a real temp file-tree fixture (`New-TestFileTree` in `ModuleUnderTest.ps1`, built under Pester's `$TestDrive`), then assert on the job's own real, unmocked `.Progress` collection of `ProgressRecord` objects. `Write-xJobProgress`'s integration test nests two real jobs (an inner one doing the real traversal, an outer one making the real, unmocked `Write-xJobProgress` calls) to capture its real mirrored output. - Run all: `Invoke-Pester -Path ./Tests -Output Detailed` - Run one function's tests: `Invoke-Pester -Path ./Tests/New-xProgress.Tests.ps1 -Output Detailed` +- Run just the fast suite: `Invoke-Pester -Path ./Tests -ExcludeTag IntegrationTests -Output Detailed` - Each test file independently locates and imports the manifest (`Import-Module ...\xProgress.psd1 -Force`) in its own `BeforeAll`, so files can run standalone or in any order -- `Write-Progress`/`Write-Information` are mocked (`Mock -ModuleName xProgress Write-Progress { }`) where the test needs to assert *what* was passed to them (e.g. a non-null `-Id`); everywhere else tests assert on real return values/state +- `Write-Progress`/`Write-Information` are mocked (`Mock -ModuleName xProgress Write-Progress { }`) in `UnitTests` blocks where the test needs to assert *what* was passed to them (e.g. a non-null `-Id`); `IntegrationTests` blocks never mock `Write-Progress` — everywhere else tests assert on real return values/state - Module-private/script-scoped state (`$script:ProgressTracker`, `$script:WriteProgressID`) can be inspected directly via `& (Get-Module xProgress) { $script:ProgressTracker }` if a future private helper needs it — not currently used since all functions are public and `Get-xProgress` already exposes instance state - CI runs the suite on every push via the `test-with-pester` job in `.github/workflows/main.yml` @@ -56,6 +58,8 @@ All progress instances live in two module-scoped variables in `xProgress.psm1`: - `$script:ProgressTracker` — hashtable keyed by GUID string; each value is a `PSCustomObject` representing one progress instance. - `$script:WriteProgressID` — integer counter starting at 628, auto-incremented to assign unique `Write-Progress -Id` values. +- `$script:JobProgressMap` — used only by `Write-xJobProgress`, entirely separate from `$script:ProgressTracker`. Nested hashtable keyed by `Job.InstanceId.Guid` -> `ChildJob.InstanceId.Guid` -> `ActivityId (int)` -> assigned `Write-Progress -Id` (drawn from the same `$script:WriteProgressID` counter, so job-mirrored bars can't collide with regular xProgress instance IDs). +- `$script:JobProgressRetired` — set (hashtable of `Job.InstanceId.Guid` -> `$true`) of jobs `Write-xJobProgress` has already completed/cleaned up, so leftover `.Progress` records on a finished job are never reprocessed. ### xProgress instance object shape @@ -96,6 +100,7 @@ xParentIdentity # GUID of parent xProgress instance (if nested) | `Start-xProgress` | Manually starts Stopwatch | | `Suspend-xProgress` | Stops Stopwatch without resetting (to exclude wait time from elapsed) | | `Resume-xProgress` | Restarts a suspended Stopwatch | +| `Write-xJobProgress` | Mirrors progress from a background job's `ChildJobs[*].Progress` (or the job's own `.Progress` if it has no ChildJobs) into `Write-Progress`, one bar per distinct `ActivityId`, preserving `ParentActivityId` nesting. Lightweight write-only passthrough - does not use `$script:ProgressTracker` | `Initialize-xProgress` is an alias for `New-xProgress`. @@ -106,6 +111,8 @@ Parent/child `Write-Progress` nesting is supported two ways: - **Manual:** Pass `-Id` / `-ParentId` integers directly. - **xProgress-managed:** Pass `-xParentIdentity` (alias `xPPID`) with the parent's GUID; the module resolves the integer IDs automatically. +**Fast-follow note:** `Write-xJobProgress`'s mirrored job bars are not yet nestable under a caller's own xProgress instance. A future `-xParentIdentity` parameter on `Write-xJobProgress` is planned - `$script:JobProgressMap` is already independent of `$script:ProgressTracker`, so adding it would only require resolving the parent's `Get-xProgress` `.ID` once and using it as the `ParentId` fallback for a job's top-level activities. + ## CI/CD - **On every push** → `.github/workflows/main.yml` runs PSScriptAnalyzer and the Pester test suite (two jobs) on ubuntu-latest. @@ -118,4 +125,4 @@ Parent/child `Write-Progress` nesting is supported two ways: ## WIP -`WIP/JobProgress.ps1` is a stub for a future feature: displaying progress from PowerShell background jobs. It is not exported or functional yet. +The former `WIP/JobProgress.ps1` stub has graduated into `Write-xJobProgress` in `xProgress.psm1` and is exported. See the "Nesting" fast-follow note above for the one deliberately deferred piece (nesting job progress under a caller's own xProgress instance). diff --git a/README.md b/README.md index a02e399..1dfd6fd 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,24 @@ # xProgress Powershell Module -xProgress makes the complexity of using progress bars (including some of the more advanced features like time remaining) in Powershell simple while minimizing the performance impact of Write-Progress for processing of large numbers of actions when iterating through an array. +xProgress makes the complexity of using progress bars in Powershell simple: throttled updates, accurate time-remaining, nested parent/child bars, and, uniquely, progress reported from *inside a background job* — while minimizing the performance impact of calling Write-Progress on every iteration of a large loop. -Write-Progress is expensive to call on every iteration of a large loop and is complex to manage when fully using it's capabilities. -xProgress solves these problems. +Write-Progress is expensive to call on every iteration of a large loop, complex to manage when fully using its capabilities, and in the case of background jobs, invisible to the calling session. -Performance - xProgress throttles Write-Progress calls to configurable intervals (e.g. every 1%, every 10 items) while still calculating accurate percentage complete and estimated time remaining for every item processed. +xProgress solves all three problems. -Complexity - Managing progress bar calculations, parent/child relationships, and timer state is handled automatically by xProgress so you do not need to write custom tracking code for each scenario where progress output is needed. +## Performance + +xProgress throttles Write-Progress calls to configurable intervals (e.g. every 1%, every 10 items) while still calculating accurate percentage complete and estimated time remaining for every item processed. + +## Complexity + +Managing progress bar calculations, parent/child relationships, and timer state is handled automatically by xProgress so you do not need to write custom tracking code for each scenario where progress output is desired. + +A hand-rolled equivalent typically means: a counter and a modulus check to throttle calls, a stopwatch plus elapsed/remaining-time math, percent-complete capping so a rounding error never reports 101%, and — the moment a second progress bar nests under the first — a hand-maintained scheme for Id/ParentId bookkeeping. Roughly 40-80 lines of that plumbing per progress bar, copy-pasted and re-adapted at every callsite, versus one `New-xProgress` line plus a `Write-xProgress` call per iteration. It's also easy to get subtly wrong: a percent-complete that could exceed 100, a divide-by-zero on the first call, an auto-assigned Id that silently came out `$null`, an off-by-one in a batch-count message. All bugs you no longer have to worry about. + +## Background Jobs + +Write-Progress calls inside a `Start-Job` scriptblock only reach that job's own Progress stream — they are invisible in the calling session without tooling like xProgress. As far as we're aware, no other PowerShell module does this for you. Getting it right by hand (tracking each activity separately instead of just the last message received, preserving parent/child nesting reported inside the job, avoiding Id collisions across concurrent jobs, cleaning up once a job finishes) easily runs past 100 lines. This why most scripts and modules either skip job progress entirely or settle for a naive "show whatever came in last" readout. `Write-xJobProgress` reduces all of that to one cmdlet call in your own polling loop. ```Powershell New-xProgress @@ -20,6 +29,7 @@ Complete-xProgress Start-xProgress Suspend-xProgress Resume-xProgress +Write-xJobProgress ``` ## Examples @@ -142,8 +152,42 @@ foreach ($i in $MyListOfItems) Complete-xProgress -Identity $xProgressID ``` +### Job Progress + +`Write-xJobProgress` mirrors `Write-Progress` calls happening *inside* a background job's scriptblock into your own session. It's a lightweight, write-only passthrough — unlike the rest of xProgress, it does not register anything in xProgress's own tracker and has no throttling, Suspend/Resume, or timer semantics. Call it repeatedly from your own polling loop; it never blocks or polls on its own. + +```powershell +$job = Start-Job -ScriptBlock { + 1..10 | ForEach-Object { + Write-Progress -Activity 'Work' -PercentComplete ($_ * 10) + Start-Sleep -Milliseconds 200 + } +} + +while ($job.State -eq 'Running') +{ + Write-xJobProgress -Job $job + Start-Sleep -Milliseconds 250 +} +Write-xJobProgress -Job $job +# the final call shows the last update and completes/clears the progress bar + +# Or, for every job in the session: +Get-Job | Write-xJobProgress +``` + +If the job's scriptblock reports more than one activity (including nested activities via `-ParentId`), `Write-xJobProgress` mirrors each one with a distinct, stable progress bar and preserves the parent/child nesting. Progress `-Id` values are drawn from the same pool xProgress itself uses, so mirrored job bars never collide with your own xProgress instances. + ## Releases +1.1.0 New Functionality + +- `Write-xJobProgress`: mirrors `Write-Progress` calls happening inside a + background job's scriptblock into the caller's session, one bar per + distinct activity, preserving parent/child nesting reported inside the + job. Lightweight write-only passthrough - no throttling/timer/tracker + integration (see the Job Progress section above). + 1.0.1 Bug Fix - `New-xProgress`: auto-assigned `-Id` (and any child's `ParentID` set via @@ -197,5 +241,5 @@ Complete-xProgress -Identity $xProgressID ## Development Plans -- add/extend functions for Job Progress display -- possibly incorporate some gui progress bars like this: https://key2consulting.com/powershell-how-to-display-job-progress/ or https://github.com/Tiberriver256/PoshProgressBar \ No newline at end of file +- `Write-xJobProgress`: add an `-xParentIdentity` parameter to nest mirrored job progress bars under a caller's own xProgress instance +- possibly incorporate new progress bar ideas such as this: https://github.com/Tiberriver256/PoshProgressBar or https://github.com/rsalmei/alive-progress or https://github.com/tqdm/tqdm?tab=readme-ov-file \ No newline at end of file diff --git a/Tests/ModuleUnderTest.ps1 b/Tests/ModuleUnderTest.ps1 index 9768890..00ce858 100644 --- a/Tests/ModuleUnderTest.ps1 +++ b/Tests/ModuleUnderTest.ps1 @@ -19,3 +19,27 @@ if (-not $manifest) $projectRoot = $manifest.DirectoryName $moduleName = $manifest.BaseName $manifestPath = $manifest.FullName + +# Builds a small real directory/file tree under $Root and returns the created file paths. +# $Root must be a plain OS path (e.g. Pester's $TestDrive), not the TestDrive: PSDrive - a +# separate Start-Job process can't resolve the caller's PSDrives. +function New-TestFileTree +{ + param( + [Parameter(Mandatory)] + [string]$Root + ) + + $files = foreach ($sub in 'Alpha', 'Beta', 'Gamma') + { + $dir = Join-Path -Path $Root -ChildPath $sub + New-Item -Path $dir -ItemType Directory -Force | Out-Null + foreach ($i in 1..4) + { + $filePath = Join-Path -Path $dir -ChildPath "file$i.txt" + Set-Content -Path $filePath -Value "$sub-$i-$(Get-Random)" + $filePath + } + } + return $files +} diff --git a/Tests/Write-xJobProgress.Tests.ps1 b/Tests/Write-xJobProgress.Tests.ps1 new file mode 100644 index 0000000..c23faf8 --- /dev/null +++ b/Tests/Write-xJobProgress.Tests.ps1 @@ -0,0 +1,273 @@ +#Requires -Modules @{ModuleName = 'Pester'; ModuleVersion = '5.0.0' } + +$CommandName = $MyInvocation.MyCommand.Name.Replace('.Tests.ps1', '') + +BeforeAll { + . "$PSScriptRoot/ModuleUnderTest.ps1" + Import-Module -Name $manifestPath -Force + + # Polls until the job's own or first child job's Progress collection has at least + # $Count records, or $TimeoutSeconds elapses. Real background jobs are async, so tests + # poll instead of relying on a fixed sleep. + function Wait-TestJobProgress + { + param( + [System.Management.Automation.Job]$Job, + [int]$Count = 1, + [int]$TimeoutSeconds = 20 + ) + $sw = [System.Diagnostics.Stopwatch]::StartNew() + while ($sw.Elapsed.TotalSeconds -lt $TimeoutSeconds) + { + $progress = if ($Job.ChildJobs.Count -gt 0) { $Job.ChildJobs[0].Progress } else { $Job.Progress } + if ($progress.Count -ge $Count) + { + return + } + Start-Sleep -Milliseconds 100 + } + } +} + +Describe "$CommandName Unit Tests" -Tag 'UnitTests' { + BeforeEach { + Mock -ModuleName xProgress Write-Progress { } + } + + Context 'Validate parameters' { + It 'Should have the expected parameters' { + [object[]]$params = (Get-ChildItem "function:\$CommandName").Parameters.Keys + $knownParameters = @('Job') + foreach ($kp in $knownParameters) + { + $kp | Should -BeIn $params + } + } + } + + Context 'Single job, single activity' { + BeforeEach { + $script:job = Start-Job -ScriptBlock { + Write-Progress -Activity 'Work' -PercentComplete 50 + Start-Sleep -Seconds 5 + } + Wait-TestJobProgress -Job $script:job -Count 1 + } + + AfterEach { + Remove-Job -Job $script:job -Force -ErrorAction SilentlyContinue + } + + It 'Mirrors PercentComplete and assigns a non-null Id' { + Write-xJobProgress -Job $script:job + Should -Invoke Write-Progress -ModuleName xProgress -Times 1 -ParameterFilter { + $PercentComplete -eq 50 -and $null -ne $Id + } + } + + It 'Uses a stable Id across repeated calls while the job is unfinished' { + $script:capturedIds = @() + Mock -ModuleName xProgress Write-Progress { $script:capturedIds += $Id } + Write-xJobProgress -Job $script:job + Write-xJobProgress -Job $script:job + $script:capturedIds.Count | Should -Be 2 + ($script:capturedIds | Select-Object -Unique).Count | Should -Be 1 + } + } + + Context 'Concurrent activities with parent/child nesting' { + BeforeEach { + $script:job = Start-Job -ScriptBlock { + Write-Progress -Id 1 -Activity 'Outer' -PercentComplete 10 + Write-Progress -Id 2 -ParentId 1 -Activity 'Inner' -PercentComplete 20 + Start-Sleep -Seconds 5 + } + Wait-TestJobProgress -Job $script:job -Count 2 + } + + AfterEach { + Remove-Job -Job $script:job -Force -ErrorAction SilentlyContinue + } + + It 'Mirrors both activities with distinct ids and the inner one parented to the outer one' { + $script:capturedCalls = @() + Mock -ModuleName xProgress Write-Progress { + $script:capturedCalls += [pscustomobject]@{ Activity = $Activity; Id = $Id; ParentId = $ParentId } + } + Write-xJobProgress -Job $script:job + $script:capturedCalls.Count | Should -Be 2 + + $outer = $script:capturedCalls | Where-Object Activity -EQ 'Outer' + $inner = $script:capturedCalls | Where-Object Activity -EQ 'Inner' + $outer | Should -Not -BeNullOrEmpty + $inner | Should -Not -BeNullOrEmpty + $inner.Id | Should -Not -Be $outer.Id + $inner.ParentId | Should -Be $outer.Id + } + } + + Context 'Zero-ChildJobs fallback' { + It 'Uses the job''s own Progress stream when it has no ChildJobs' { + $job = Start-Job -ScriptBlock { Write-Progress -Activity 'Work' -PercentComplete 75 } + try + { + # Job.Progress can only be set once the job has left the Running state, so let + # it finish naturally, then simulate a job type with no ChildJobs by moving the + # child's progress up to the job itself and clearing ChildJobs (both are + # supported, mutable members of System.Management.Automation.Job). + Wait-Job -Job $job -Timeout 20 | Out-Null + $job.Progress = $job.ChildJobs[0].Progress + $job.ChildJobs.Clear() + + Write-xJobProgress -Job $job + Should -Invoke Write-Progress -ModuleName xProgress -Times 1 -ParameterFilter { $PercentComplete -eq 75 } + } + finally + { + Remove-Job -Job $job -Force -ErrorAction SilentlyContinue + } + } + } + + Context 'Empty progress' { + It 'Does not throw and does not call Write-Progress for a job that has not reported yet' { + $job = Start-Job -ScriptBlock { Start-Sleep -Seconds 5 } + try + { + { Write-xJobProgress -Job $job } | Should -Not -Throw + Should -Invoke Write-Progress -ModuleName xProgress -Times 0 + } + finally + { + Remove-Job -Job $job -Force -ErrorAction SilentlyContinue + } + } + } + + Context 'Completion cleanup' { + It 'Completes the mirrored bar once the job finishes and does not re-emit on a later call' { + $job = Start-Job -ScriptBlock { Write-Progress -Activity 'Almost done' -PercentComplete 99 } + try + { + Wait-Job -Job $job -Timeout 20 | Out-Null + $job.State | Should -Be 'Completed' + + Write-xJobProgress -Job $job + Should -Invoke Write-Progress -ModuleName xProgress -Times 1 -ParameterFilter { $Completed -eq $true } + + { Write-xJobProgress -Job $job } | Should -Not -Throw + Should -Invoke Write-Progress -ModuleName xProgress -Times 1 -ParameterFilter { $Completed -eq $true } + + $retired = & (Get-Module xProgress) { $script:JobProgressRetired.ContainsKey($args[0]) } $job.InstanceId.Guid + $retired | Should -BeTrue + } + finally + { + Remove-Job -Job $job -Force -ErrorAction SilentlyContinue + } + } + } + + Context 'ID isolation from xProgress instances' { + It 'Never assigns a job progress Id that collides with a live xProgress instance Id' { + $xpId = New-xProgress -ArrayToProcess (1..10) -ExplicitProgressInterval 1 -Activity 'Caller-driven' + $xpAssignedId = (Get-xProgress -Identity $xpId).ID + + $job = Start-Job -ScriptBlock { + Write-Progress -Activity 'Work' -PercentComplete 50 + Start-Sleep -Seconds 5 + } + try + { + Wait-TestJobProgress -Job $job -Count 1 + $script:capturedIds = @() + Mock -ModuleName xProgress Write-Progress { $script:capturedIds += $Id } + Write-xJobProgress -Job $job + $script:capturedIds | Should -Not -Contain $xpAssignedId + } + finally + { + Remove-Job -Job $job -Force -ErrorAction SilentlyContinue + } + } + } +} + +Describe "$CommandName Integration Tests" -Tag 'IntegrationTests' { + # No Mock anywhere in this Describe. Two real, nested background jobs are used: an inner job + # does real file-system traversal and calls the real built-in Write-Progress itself (the + # "someone else's job I want to mirror" case Write-xJobProgress exists for); an outer job + # imports the module and makes real, unmocked Write-xJobProgress calls while polling the + # inner job. The outer job's own Progress collection captures Write-xJobProgress's real, + # mirrored Write-Progress output. + BeforeAll { + $script:integrationFiles = New-TestFileTree -Root $TestDrive + } + + Context 'Mirroring a real background job''s real Write-Progress output' { + BeforeAll { + $script:outerJob = Start-Job -ScriptBlock { + param($ManifestPath, $Files) + Import-Module -Name $ManifestPath -Force + + $innerJob = Start-Job -ScriptBlock { + param($Files) + $total = $Files.Count + $count = 0 + foreach ($f in $Files) + { + $count++ + Get-FileHash -Path $f -Algorithm SHA1 | Out-Null + Write-Progress -Activity 'Real file traversal' -PercentComplete ([math]::Round($count / $total * 100)) + Start-Sleep -Milliseconds 150 + } + } -ArgumentList (, $Files) + + try + { + $timeout = [System.Diagnostics.Stopwatch]::StartNew() + while ($innerJob.State -eq 'Running' -and $timeout.Elapsed.TotalSeconds -lt 30) + { + Write-xJobProgress -Job $innerJob + Start-Sleep -Milliseconds 150 + } + Wait-Job -Job $innerJob -Timeout 30 | Out-Null + Write-xJobProgress -Job $innerJob + } + finally + { + Remove-Job -Job $innerJob -Force -ErrorAction SilentlyContinue + } + } -ArgumentList $manifestPath, (, $script:integrationFiles) + Wait-Job -Job $script:outerJob -Timeout 60 | Out-Null + $script:mirrored = $script:outerJob.ChildJobs[0].Progress + } + + AfterAll { + Remove-Job -Job $script:outerJob -Force -ErrorAction SilentlyContinue + } + + It 'Completes the outer job without error' { + $script:outerJob.State | Should -Be 'Completed' + $script:outerJob.ChildJobs[0].Error | Should -BeNullOrEmpty + } + + It 'Mirrors at least one real, non-completed progress update' { + $processing = $script:mirrored | Where-Object RecordType -EQ 'Processing' + $processing | Should -Not -BeNullOrEmpty + ($processing.PercentComplete | Where-Object { $_ -gt 0 -and $_ -le 100 }) | Should -Not -BeNullOrEmpty + } + + It 'Uses a single, stable, non-null mirrored Id throughout' { + $processing = $script:mirrored | Where-Object RecordType -EQ 'Processing' + $processing.ActivityId | Should -Not -Contain $null + ($processing.ActivityId | Select-Object -Unique).Count | Should -Be 1 + } + + It 'Ends with exactly one real Completed record, once the inner job finishes' { + $completed = $script:mirrored | Where-Object RecordType -EQ 'Completed' + $completed.Count | Should -Be 1 + $script:mirrored[-1].RecordType | Should -Be 'Completed' + } + } +} diff --git a/Tests/Write-xProgress.Tests.ps1 b/Tests/Write-xProgress.Tests.ps1 index bb34d61..9884d38 100644 --- a/Tests/Write-xProgress.Tests.ps1 +++ b/Tests/Write-xProgress.Tests.ps1 @@ -120,3 +120,65 @@ Describe "$CommandName Unit Tests" -Tag 'UnitTests' { } } } + +Describe "$CommandName Integration Tests" -Tag 'IntegrationTests' { + # No Mock anywhere in this Describe - Write-Progress runs for real, inside a background job, + # and its real ProgressRecord output is inspected via the job's own Progress collection. + BeforeAll { + $script:integrationFiles = New-TestFileTree -Root $TestDrive + } + + Context 'Real New-xProgress -> Write-xProgress -> Complete-xProgress lifecycle over real files' { + BeforeAll { + $script:job = Start-Job -ScriptBlock { + param($ManifestPath, $Files) + Import-Module -Name $ManifestPath -Force + $id = New-xProgress -ArrayToProcess $Files -ExplicitProgressInterval 1 -Activity 'Hashing files' + foreach ($f in $Files) + { + Get-FileHash -Path $f -Algorithm SHA1 | Out-Null + Write-xProgress -Identity $id + } + Complete-xProgress -Identity $id + } -ArgumentList $manifestPath, (, $script:integrationFiles) + Wait-Job -Job $script:job -Timeout 30 | Out-Null + $script:progress = $script:job.ChildJobs[0].Progress + } + + AfterAll { + Remove-Job -Job $script:job -Force -ErrorAction SilentlyContinue + } + + It 'Completes the job without error' { + $script:job.State | Should -Be 'Completed' + $script:job.ChildJobs[0].Error | Should -BeNullOrEmpty + } + + It 'Emits real progress records: at least one update plus the final real Completed record' { + # Real Write-Progress calls fired in a tight loop can be coalesced in transit from a + # job to the parent session (only the latest state survives between polls), so this + # asserts on real properties rather than an exact per-file count. + $script:progress.Count | Should -BeGreaterOrEqual 2 + } + + It 'Uses a single, non-null ActivityId for every record' { + $script:progress.ActivityId | Should -Not -Contain $null + ($script:progress.ActivityId | Select-Object -Unique).Count | Should -Be 1 + } + + It 'Reports PercentComplete that never decreases and reaches 100' { + $percents = $script:progress.PercentComplete + for ($i = 1; $i -lt $percents.Count; $i++) + { + $percents[$i] | Should -BeGreaterOrEqual $percents[$i - 1] + } + $percents[-1] | Should -Be 100 + } + + It 'Ends with a real Completed record from Complete-xProgress' { + $last = $script:progress[-1] + $last.RecordType | Should -Be 'Completed' + $last.PercentComplete | Should -Be 100 + } + } +} diff --git a/WIP/JobProgress.ps1 b/WIP/JobProgress.ps1 deleted file mode 100644 index b3e3449..0000000 --- a/WIP/JobProgress.ps1 +++ /dev/null @@ -1,23 +0,0 @@ - -Function Write-xJobProgress -{ - param( - [System.Management.Automation.Job[]]$Job - ) - - process - { - foreach ($j in $Job) - { - #Extracts the latest progress of the job and writes the progress - $jobProgressHistory = $j.ChildJobs[0].Progress - $latestProgress = $jobProgressHistory[$jobProgressHistory.Count - 1] - $latestPercentComplete = $latestProgress.PercentComplete - $latestActivity = $latestProgress.Activity - $latestStatus = $latestProgress.StatusDescription - - #When adding multiple progress bars, a unique ID must be provided. Here I am providing the JobID as this - Write-Progress -Id $j.Id -Activity $latestActivity -Status $latestStatus -PercentComplete $latestPercentComplete; - } - } -} \ No newline at end of file diff --git a/en-us/about_xProgress.help.txt b/en-us/about_xProgress.help.txt index be22d67..4b05974 100644 --- a/en-us/about_xProgress.help.txt +++ b/en-us/about_xProgress.help.txt @@ -108,6 +108,12 @@ FUNCTIONS Resume-xProgress Resume a paused stopwatch. + Write-xJobProgress + Mirror progress reported inside a background job's scriptblock + (via Write-Progress) into the caller's session. Lightweight, + write-only passthrough - does not use module state from the + functions above. Call repeatedly from your own polling loop. + Initialize-xProgress is an alias for New-xProgress. EXAMPLES @@ -178,6 +184,23 @@ EXAMPLES Complete-xProgress -Identity $id + Displaying progress from a background job + + $job = Start-Job -ScriptBlock { + 1..10 | ForEach-Object { + Write-Progress -Activity 'Work' -PercentComplete ($_ * 10) + Start-Sleep -Milliseconds 200 + } + } + + while ($job.State -eq 'Running') + { + Write-xJobProgress -Job $job + Start-Sleep -Milliseconds 250 + } + Write-xJobProgress -Job $job + # final call shows the last update and completes/clears the bar + SEE ALSO Get-Help New-xProgress Get-Help Write-xProgress @@ -186,4 +209,5 @@ SEE ALSO Get-Help Start-xProgress Get-Help Suspend-xProgress Get-Help Resume-xProgress + Get-Help Write-xJobProgress Get-Help Write-Progress diff --git a/xProgress.psd1 b/xProgress.psd1 index 701c432..236bfed 100644 --- a/xProgress.psd1 +++ b/xProgress.psd1 @@ -12,7 +12,7 @@ RootModule = 'xProgress.psm1' # Version number of this module. - ModuleVersion = '1.0.1' + ModuleVersion = '1.1.0' # Supported PSEditions # CompatiblePSEditions = @() @@ -69,7 +69,7 @@ # NestedModules = @() # Functions to export from this module, for best performance, do not use wildcards and do not delete the entry, use an empty array if there are no functions to export. - FunctionsToExport = 'New-xProgress', 'Write-xProgress', 'Complete-xProgress', 'Get-xProgress', 'Set-xProgress', 'Start-xProgress', 'Suspend-xProgress', 'Resume-xProgress' + FunctionsToExport = 'New-xProgress', 'Write-xProgress', 'Complete-xProgress', 'Get-xProgress', 'Set-xProgress', 'Start-xProgress', 'Suspend-xProgress', 'Resume-xProgress', 'Write-xJobProgress' # Cmdlets to export from this module, for best performance, do not use wildcards and do not delete the entry, use an empty array if there are no cmdlets to export. #CmdletsToExport = '*' @@ -107,7 +107,7 @@ # IconUri = '' # ReleaseNotes of this module - ReleaseNotes = '1.0.1 - Bug fix: New-xProgress auto-assigned -Id (and any child ParentID via -xParentIdentity) could come out null/0 instead of an incrementing integer, causing Write-Progress to throw. Fixed.' + ReleaseNotes = '1.1.0 - Added Write-xJobProgress: mirrors progress reported by PowerShell background jobs (Write-Progress calls inside a job scriptblock) into the caller''s session via Write-Progress, preserving concurrent/nested activities.' # Prerelease string of this module # Prerelease = '' diff --git a/xProgress.psm1 b/xProgress.psm1 index 165e424..03772ce 100644 --- a/xProgress.psm1 +++ b/xProgress.psm1 @@ -1,5 +1,7 @@ $script:ProgressTracker = @{} $script:WriteProgressID = 628 +$script:JobProgressMap = @{} +$script:JobProgressRetired = @{} Function New-xProgress @@ -755,4 +757,156 @@ Function Resume-xProgress } } +Function Write-xJobProgress +{ + <# + .SYNOPSIS + Mirrors progress reported by a background job's ChildJobs into the caller's session using + Write-Progress + .DESCRIPTION + Inspects the Progress stream of every ChildJob under one or more + System.Management.Automation.Job objects (falling back to the job's own Progress stream + if it has no ChildJobs) and re-emits the latest record for each distinct ActivityId via + Write-Progress in the caller's session, preserving ParentActivityId nesting. + + This is a lightweight, write-only passthrough: unlike New-xProgress/Write-xProgress, it + does not register anything in xProgress's own instance tracker and has no throttling, + Suspend/Resume, or timer semantics. Call it repeatedly from your own polling loop (see + examples); it does not block or poll on its own. + + Write-Progress IDs are assigned from the same shared counter New-xProgress uses, so + mirrored job progress bars never collide with your own xProgress instance IDs. The + mapping is stable across repeated calls for the life of the job and is cleaned up (a + final Write-Progress -Completed plus removal from internal state) once the job's State + leaves Running. + + Known limitation: if a job is removed or force-stopped before it leaves Running (and + before a final Write-xJobProgress call sees that), its internal id mapping is not + cleaned up and leaks for the life of the session. + .EXAMPLE + $job = Start-Job -ScriptBlock { 1..10 | ForEach-Object { Write-Progress -Activity 'Work' -PercentComplete ($_ * 10); Start-Sleep -Milliseconds 200 } } + while ($job.State -eq 'Running') + { + Write-xJobProgress -Job $job + Start-Sleep -Milliseconds 250 + } + Write-xJobProgress -Job $job + + Polls a single background job and mirrors its progress until it finishes. The final call + after the loop shows the last update and completes/clears the progress bar. + .EXAMPLE + Get-Job | Write-xJobProgress + + Mirrors progress for every job currently tracked in the session, via the pipeline. + #> + [CmdletBinding()] + param( + [parameter(Mandatory, ValueFromPipeline, ValueFromPipelineByPropertyName, + HelpMessage = 'One or more background jobs whose progress should be mirrored')] + [System.Management.Automation.Job[]]$Job #One or more background jobs whose progress should be mirrored + ) + + process + { + foreach ($j in $Job) + { + $jobKey = $j.InstanceId.Guid + + #a retired job is fully done - never reprocess leftover Progress records for it + if ($script:JobProgressRetired.ContainsKey($jobKey)) + { + continue + } + + $childJobs = if ($j.ChildJobs -and $j.ChildJobs.Count -gt 0) { $j.ChildJobs } else { , $j } + + foreach ($child in $childJobs) + { + if (-not $child.Progress -or $child.Progress.Count -eq 0) + { + continue + } + + #collapse the child's whole progress history down to the latest record per ActivityId + $latestByActivity = @{} + foreach ($record in $child.Progress) + { + $latestByActivity[$record.ActivityId] = $record + } + + $childKey = $child.InstanceId.Guid + if (-not $script:JobProgressMap.ContainsKey($jobKey)) + { + $script:JobProgressMap[$jobKey] = @{} + } + if (-not $script:JobProgressMap[$jobKey].ContainsKey($childKey)) + { + $script:JobProgressMap[$jobKey][$childKey] = @{} + } + $activityMap = $script:JobProgressMap[$jobKey][$childKey] #ActivityId (int) -> WriteProgressId (int) + + #assign stable Write-Progress ids for any ActivityId seen for the first time + foreach ($activityId in $latestByActivity.Keys) + { + if (-not $activityMap.ContainsKey($activityId)) + { + $activityMap[$activityId] = (++$script:WriteProgressID) + } + } + + foreach ($activityId in $latestByActivity.Keys) + { + $record = $latestByActivity[$activityId] + $wpParams = @{ + Id = $activityMap[$activityId] + ParentId = + if ($record.ParentActivityId -ge 0 -and $activityMap.ContainsKey($record.ParentActivityId)) + { $activityMap[$record.ParentActivityId] } + else { -1 } + Activity = if ($record.Activity) { $record.Activity } else { 'Job Progress' } + PercentComplete = $record.PercentComplete + SecondsRemaining = $record.SecondsRemaining + } + if ($record.StatusDescription) + { + $wpParams.Status = $record.StatusDescription + } + if ($record.CurrentOperation) + { + $wpParams.CurrentOperation = $record.CurrentOperation + } + + if ($record.RecordType -eq [System.Management.Automation.ProgressRecordType]::Completed) + { + Write-Progress @wpParams -Completed + $activityMap.Remove($activityId) + } + else + { + Write-Progress @wpParams + } + } + } + + #once the job has left Running/NotStarted, complete and retire any remaining mirrored bars + if ($j.State -notin @([System.Management.Automation.JobState]::Running, [System.Management.Automation.JobState]::NotStarted)) + { + if ($script:JobProgressMap.ContainsKey($jobKey)) + { + foreach ($childMap in $script:JobProgressMap[$jobKey].Values) + { + foreach ($wpId in $childMap.Values) + { + Write-Progress -Id $wpId -Activity 'Job Progress' -Completed + } + } + $script:JobProgressMap.Remove($jobKey) + } + $script:JobProgressRetired[$jobKey] = $true + } + } + } +} + + New-Alias -Name Initialize-xProgress -Value New-xProgress \ No newline at end of file