Skip to content

test: make Example_* output deterministic - #69

Merged
MasterOfBinary merged 2 commits into
masterfrom
fix/example-determinism
Jun 12, 2026
Merged

test: make Example_* output deterministic#69
MasterOfBinary merged 2 commits into
masterfrom
fix/example-determinism

Conversation

@MasterOfBinary

Copy link
Copy Markdown
Owner

Summary

Several Example_* tests printed from inside Process and relied on a hardcoded // Output: block that assumed batches print in order. The engine dispatches one goroutine per batch with no cross-batch happens-before, so ordering was held only by time.Sleep in the sources — latent CI flakiness that could fail the build non-deterministically on a loaded runner.

Fix

Each affected example made deterministic with the least-invasive technique:

  • Single-batch config (MinItems == MaxItems >= N) so there is exactly one printing goroutine — Example, Example_simpleProcessor, Example_processorChain, Example_errorHandling.
  • Collect-then-print after <-b.Done() for Example_dynamicConfig (which inherently demonstrates runtime resizing); asserts invariants rather than interleaved order.
  • Example_customConfig left unchanged — every batch prints an identical line, so it is already order-independent.

No examples were downgraded to non-verified (// no output) form.

Testing

go test -run Example -count=20 ./batch/ and the same at GOMAXPROCS=1 pass on every run (also verified under CPU-burner load). Full go test -race ./... green.

Backwards compatibility

Test-only; no library code touched.


Part of a 5-PR set from a full-repo review; file-disjoint and independently mergeable in any order.

🤖 Generated with Claude Code

@codecov

codecov Bot commented May 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.76%. Comparing base (ba2a756) to head (4785ea8).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master      #69      +/-   ##
==========================================
+ Coverage   96.73%   96.76%   +0.02%     
==========================================
  Files          12       12              
  Lines         368      371       +3     
==========================================
+ Hits          356      359       +3     
  Misses          9        9              
  Partials        3        3              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors several example tests in the batch package to ensure deterministic and stable outputs. This is achieved by removing time.Sleep calls, eliminating non-deterministic console prints from concurrent goroutines, and configuring the batchers to process all items in a single batch. The review feedback is highly constructive, suggesting a minor wording improvement in batch/example_error_handling_test.go to clarify that the batch is flushed due to EOF rather than reaching the MaxItems cap, which ensures technical accuracy.

Comment thread batch/example_error_handling_test.go Outdated
Comment on lines +141 to +143
// processor errors. MaxItems caps the batch; because two source reads are
// reported as errors instead of items, the batch holds the 13 emitted
// items and is flushed at end-of-input.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The comment states that MaxItems caps the batch, but since MaxItems is set to 15 and only 13 items are successfully emitted by the source (due to 2 source errors), the batch is actually flushed by EOF (end-of-input) rather than reaching the MaxItems cap. Clarifying this makes the explanation technically precise.

Suggested change
// processor errors. MaxItems caps the batch; because two source reads are
// reported as errors instead of items, the batch holds the 13 emitted
// items and is flushed at end-of-input.
// processor errors. Because two source reads are reported as errors instead
// of items, the batch holds the 13 emitted items and is flushed at
// end-of-input (EOF) rather than reaching the MaxItems cap.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — applied in 4785ea8. The batch holds 13 items (two reads surface as source errors), so EOF is what flushes it, not the MaxItems cap.

Rebase the deterministic-example work onto the single-use Batch API (#64).
Batch.Go now returns (<-chan error, error) and a Batch is single-use, so
the examples that call Go directly use `errs, err := b.Go(...)` and check
the start error.

Examples remain deterministic under both normal scheduling and
GOMAXPROCS=1: single-batch configs (MinItems==MaxItems) collapse the
pipeline into one goroutine for the chained/per-batch-print examples, and
Example_dynamicConfig counts items under a mutex and prints an invariant
summary after <-b.Done() instead of racing per-batch output.

Example_customConfig and Example_simpleProcessor-style helpers (RunBatchAndWait)
were already API-compatible and are left as master's where appropriate.

Verified: gofmt -l (clean), go vet, go build, golangci-lint (0 issues),
go test -race, and go test -run Example -count=20 both normally and with
GOMAXPROCS=1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The batch holds 13 items (two source reads become errors), so it is
flushed at end-of-input rather than by the MaxItems cap. Reword the
comment so the lead clause doesn't suggest MaxItems triggers the flush.

Suggested by gemini-code-assist on #69.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@MasterOfBinary
MasterOfBinary merged commit b09f00e into master Jun 12, 2026
4 checks passed
@MasterOfBinary
MasterOfBinary deleted the fix/example-determinism branch June 12, 2026 05:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant