Skip to content

feat: delegate feature persistence to copick - #95

Merged
uermel merged 3 commits into
uermel/zarr-v3-u2-delegated-writersfrom
uermel/zarr-v3-u3-feature-writer
Aug 18, 2026
Merged

feat: delegate feature persistence to copick#95
uermel merged 3 commits into
uermel/zarr-v3-u2-delegated-writersfrom
uermel/zarr-v3-u3-feature-writer

Conversation

@uermel

@uermel uermel commented Aug 18, 2026

Copy link
Copy Markdown
Member

Tracks #86 and is the fourth PR in the copick-utils Zarr v3 migration stack.

Base: uermel/zarr-v3-u2-delegated-writers

This replaces the legacy root-array/Blosc feature writer while preserving its public API and its frozen result for inputs that already worked:

  • keeps every existing parameter, order, default, and positional call valid;
  • adds optional keyword-only chunks=None and shards=None controls;
  • preserves chunk subdivision, scalar overlap, transpose, and edge placement;
  • corrects the pre-existing trailing-overlap slice bug by deriving bounded read, crop, and output slices independently on every axis;
  • stages the complete feature-major float32 tensor in a temporary np.memmap instead of process memory;
  • releases and removes staging after success or failure;
  • performs exactly one final CopickFeatures.from_numpy() write; and
  • delegates metadata, chunks, shards, codecs, overwrite, and validation to copick core.

Targeted tests preserve the frozen pre-migration digest, cover the formerly crashing boundary geometry on all three axes, detect unwritten cells, verify staging cleanup and one-write behavior, and cap peak Python memory below both 64 MiB and 75% of the logical 30-channel output size. Broader converter and layout coverage remains deferred.

Validation: all 63 behavior tests and pinned pre-commit hooks pass. The full suite passes on Python 3.11.15/Zarr 3.1.6 and Python 3.12.10/Zarr 3.3.0.

The next stack layer is uermel/zarr-v3-u4-release-gates.

@uermel
uermel force-pushed the uermel/zarr-v3-u3-feature-writer branch from 21cdb86 to a91bb84 Compare August 18, 2026 21:16
@uermel
uermel merged commit 71b8290 into v2.0 Aug 18, 2026
7 checks passed
uermel pushed a commit that referenced this pull request Aug 18, 2026
🤖 I have created a release *beep* *boop*
---


##
[2.0.0-alpha.1](copick-utils-v1.8.0...copick-utils-v2.0.0-alpha.1)
(2026-08-18)


### ✨ Features

* delegate feature persistence to copick
([#95](#95))
([71b8290](71b8290))
* resolve tomogram levels from OME metadata
([#93](#93))
([63e6273](63e6273))


### 🐞 Bug Fixes

* bump actions/setup-python from 6 to 7
([#80](#80))
([45aed3d](45aed3d))
* bump chanzuckerberg/github-actions from 6.29.4 to 6.32.0
([#82](#82))
([9e5da98](9e5da98))


### 🧪 Tests

* establish Zarr migration baseline
([#94](#94))
([d55fc36](d55fc36))
* verify delegated canonical writers
([#96](#96))
([0c9e5cd](0c9e5cd))


### ⚙ Continuous Integration

* configure v2 alpha releases
([0696b9b](0696b9b))
* gate releases on validated artifacts
([#97](#97))
([ae873d3](ae873d3))

---
Merging this PR will publish a copick-utils alpha prerelease to PyPI.

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant