Epic: Migrate copick-utils to OME-Zarr 0.5 and Zarr v3

[uermel]

Goal

Migrate copick-utils to the published copick OME-Zarr 0.5 / Zarr v3 contract without duplicating storage policy or changing feature-calculation semantics beyond correcting invalid overlap slice bounds.

This epic targets the copick-utils 2.0 release line.

Project: https://github.com/orgs/copick/projects/1

Current implementation

The implementation is available as a linear stack against v2.0:

• test: establish Zarr migration baseline #94 — U0 test and packaging gate
• feat: resolve tomogram levels from OME metadata #93 — U1 runtime and metadata-driven reads, based on test: establish Zarr migration baseline #94
• test: verify delegated canonical writers #96 — U2 delegated writers, based on feat: resolve tomogram levels from OME metadata #93
• feat: delegate feature persistence to copick #95 — U3 retained feature writer, based on test: verify delegated canonical writers #96
• ci: gate releases on validated artifacts #97 — U4 exact-commit release gates, based on feat: delegate feature persistence to copick #95

Local validation passes 63 tests on the final lock with Pydantic 2.13.4. The final suite also passes on Python 3.11.15/Zarr 3.1.6 and Python 3.12.10/Zarr 3.3.0. Both workflows validate, all pinned pre-commit hooks pass, a wheel plus source distribution build and inspect successfully, and every check is green across all five stacked pull requests.

Implemented scope

• Add an unconditional pull-request matrix on stable Python 3.11 through 3.14, including documentation-only changes.
• Preserve a deterministic pre-migration feature result as a numerical compatibility check.
• Add one explicitly read-only metadata-based level accessor and replace all eight literal-path tomogram readers.
• Keep tomogram and segmentation persistence delegated to copick entities and verify representative canonical Zarr v3 output behavior.
• Preserve all existing compute_skimage_features(...) arguments and append optional keyword-only chunks=None and shards=None.
• Preserve existing chunk subdivision, scalar overlap, transpose, edge placement, entity-creation timing, and failure behavior.
• Correct the pre-existing trailing-overlap broadcast failure with bounded read/crop/output slices on every axis.
• Assemble one temporary disk-backed feature-major float32 tensor and delegate exactly one final write to CopickFeatures.from_numpy().
• Reuse the exact-SHA test workflow for releases, build artifacts once, inspect them, and publish those same artifacts with matching artifact-action majors.

Tests are behavior-focused. They do not parse source syntax or assert dependency declaration strings. Broader converter, input-layout, and read-no-mutation coverage remains deferred and is not claimed by this stack.

Remaining final-cutover work

• Complete live local/S3-compatible/Secure Shell/ML Croissant and portal validation as the surrounding ecosystem reaches the same migration point.
• Complete the deferred converter/input-layout coverage matrix.
• Keep Server Message Block optional and non-gating.
• Perform the final README/changelog/ecosystem documentation pass immediately before promotion from v2.0 to stable.
• Configure the final v2.0 required checks and validate the published artifacts.

These documentation, ecosystem, and broader coverage checks do not hold up review of the implementation stack.

Acceptance criteria

• The stacked pull requests merge in order and their locked Python matrices pass.
• All direct reads are metadata-driven and explicitly read-only.
• New outputs delegate to core's writer policy.
• Existing feature calls remain valid and previously valid decoded output matches the frozen chunk-wise result.
• Trailing-overlap inputs no longer crash or leave unwritten output.
• Feature staging has bounded process memory and is cleaned after success or failure.
• All 32 entry points load and render their help paths.
• Final backend, documentation, ruleset, coverage, and published-artifact checks pass before stable cutover.

Explicit non-goals

• No in-place conversion of existing stores.
• No private writer or compatibility layer.
• No equivalence requirement with a whole-volume scikit-image call.
• No support guarantee beyond available temporary-disk capacity.
• No rollback of partially created feature entities.
• No change to command-line partial-failure exit semantics.
• No AST/source-inventory tests or dependency-string tests.