Skip to content

feat: alternates and transcode operations, with a hierarchical brick store - #130

Open
wayfarer3130 wants to merge 2 commits into
masterfrom
feat/generate-brick-recompress
Open

feat: alternates and transcode operations, with a hierarchical brick store#130
wayfarer3130 wants to merge 2 commits into
masterfrom
feat/generate-brick-recompress

Conversation

@wayfarer3130

Copy link
Copy Markdown
Collaborator

Adds two secondary createdicomweb operations that act on an already created DICOMweb tree, addressed by study UID. Neither touches the create operation, both are single threaded, and both are idempotent and resumable. The brick store this generates has been tested against CS3D.

alternates

Writes renditions beside frames/, which it never writes or deletes, so the primary rendition stays byte identical.

createdicomweb alternates <studyUID> \
  [--dicomdir <path>] [--series-uid <uid>] \
  [--jls] [--jls-thumbnail] [--htj2k] [--htj2k-lossy] [--brick] \
  [--brick-order <z-minor|plane-major>] [--brick-codec <jls|htj2k>] [--brick-size <N>] \
  [--force] [--json]
Flag Directory Transfer syntax
--jls jls/ 1.2.840.10008.1.2.4.80
--jls-thumbnail jlsThumbnail/ 1.2.840.10008.1.2.4.80, quarter resolution
--htj2k htj2k/ 1.2.840.10008.1.2.4.201
--htj2k-lossy htj2kLossy/ 1.2.840.10008.1.2.4.203
--brick brick/ per --brick-codec, series level

Every requested rendition is built from the same decode of the same frames, so the size and compression summary it finishes with is a fair comparison across them. Ratios are measured from bytes actually on disk against each rendition's own dimensions, so a quarter resolution thumbnail is not credited with a 16x ratio it did not earn, and the brick store reports its cost as a percentage of frames/. --json emits the same figures on stdout with progress on stderr.

Renditions are grayscale only. Ineligible instances and series are skipped with a logged reason and the command still exits 0; a non-zero exit means a series errored.

The brick store

  • Levels reduce per axis by voxel spacing rather than by a single factor - an axis is halved when its samples are more than half an octave (√2) closer together than the coarsest axis'. Isotropic data gets the familiar uniform d1, d2, d4 ladder; 5 mm slice data brings the in-plane axes in first, so coarse levels are physically rather than numerically cubic and carry all three factors in the name (d8_8_2).
  • Box averaging, never decimation. Decimation aliases, which fabricates structure rather than blurring it.
  • Bricks are stored at their true extent rather than zero padded, and a level small enough to be worth one request is stored as a single brick shaped like the level. Each is one codestream, packed as a 2D image whose row ordering (z-minor by default, or plane-major for thick slices) is chosen to feed the JPEG-LS MED predictor.
  • Layout is series/{uid}/brick/{level}/{t###}/{k###}/y{ky}x{kx}.jls, with the {t###} component omitted for a plain 3D series. Non-spatial axes are indexed, never subsampled.
  • manifest.json carries axes, per-level sizes, factors, brick pitch and counts, order, spacing and transfer syntax, and is written last, so its presence marks the store complete and an interrupted run rebuilds rather than trusting a partial store.
  • Private tag (0009,10E0) UR BrickManifestURI under creator RadicalImaging, extending the group 0009 block already used for Content-Location. Layout details stay in the manifest, so they can change without a tag change.
  • Ineligible series are skipped with a reason, not failed: colour, single frame, fewer than 16 spatial slices, all frames at one ImagePositionPatient (third index is time, not space), or anything off a regular rectilinear grid - irregular spacing, gantry tilt, non-coplanar frames, ragged sampling.

Memory is bounded by a per-level slab rather than the volume: planes are reduced and handed down as they arrive, so the whole pyramid costs a little over one full-resolution slab.

transcode

createdicomweb transcode <studyUID> [--dicomdir <path>] [--series-uid <uid>] [--to jls] [--force]

Rewrites uncompressed grayscale frames/ to JPEG-LS lossless and updates AvailableTransferSyntaxUID. Colour and already compressed instances are untouched. Frames are staged beside the instance and moved into place only once the whole instance has encoded, so a failure leaves the originals intact.

Behaviour change to existing output

Thumbnail reduction is now a box average rather than the nearest-neighbour replicate, which changes existing jlsThumbnail and alternateThumbnail output. Where the attributes are available the average is taken over true pixel values rather than stored words, pixel padding is left out, and segmentation label maps take the first occupied sample of each box - the mean of two labels is a third segment that is in neither place.

Fixes carried along

  • readBulkData now locates the closing multipart boundary instead of assuming the footer's length. The two writers in this repository disagree about the trailing CRLF, and for a JPEG family codestream those two bytes are the EOI marker, so the frame failed to decode rather than merely looking odd.
  • getImageInfo carries BitsStored, HighBit, the pixel padding attributes and the SOP class through, since dicom-codec's adaptImageInfo drops them before the decoder sees them.
  • static-wado-creator exports codecFrame, so packages that already depend on it can transcode frames without adding a codec dependency.
  • The root jest config's comment no longer contains a glob whose */ closed the block comment early.

Docs

Root README.md gains an "Alternate Renditions and Brick Stores" section covering the CLI, the pyramid, the manifest and the eligibility rules; file-structure.md gains the new paths; the create-dicomweb README gains the command usage.

Testing

  • 133 jest tests across 10 suites pass (jest --config packages/create-dicomweb/jest.config.cjs), including 5 new suites: box filtering, brick packing round-trips under both orderings, brick eligibility, the pyramid end to end, and frame renditions.
  • The brick output has been exercised in CS3D.
  • New and changed source files are eslint/prettier clean. Note that bin/createdicomweb.mjs and static-wado-util/lib/index.js already fail prettier on master; the new CLI options follow the file's existing style rather than reformatting unrelated lines.

🤖 Generated with Claude Code

…store

Adds two secondary createdicomweb operations that act on an already created
DICOMweb tree, addressed by study UID. Neither touches the create operation,
both are single threaded, and both are idempotent and resumable.

alternates writes renditions beside frames/, which it never writes or deletes,
so the primary rendition stays byte identical:

  --jls / --htj2k / --htj2k-lossy  full resolution copies of every frame
  --jls-thumbnail                  quarter resolution copy of every frame
  --brick                          hierarchical brick pyramid, series level

Every requested rendition is built from the same decode of the same frames, so
the size and compression report it finishes with is a fair comparison across
them. Ratios are measured against each rendition's own dimensions, from bytes
actually on disk, and --json emits the same figures for collection across a
corpus.

The brick store is the part CS3D consumes. Levels reduce per axis by voxel
spacing rather than by a single factor, so coarse levels of anisotropic data
stay physically isotropic, and every level is a box average of the one above
rather than a decimation. Bricks are stored at their true extent, packed into
one codestream per brick with the row ordering chosen to feed the JPEG-LS
predictor, and described by a manifest written last so its presence marks the
store complete. Ineligible series - colour, too few slices, a temporal third
axis, or anything off a regular rectilinear grid - are skipped with a reason
rather than failing the run.

transcode rewrites uncompressed grayscale frames/ to JPEG-LS lossless, staging
frames and swapping them in only once the whole instance has encoded.

Also in this change:

- thumbnail reduction is now a box average rather than a nearest-neighbour
  replicate, which changes existing jlsThumbnail and alternateThumbnail output.
  Decimation aliases; where the attributes are available the average is taken
  over true pixel values, excludes pixel padding, and takes the first occupied
  sample for segmentation label maps.
- readBulkData locates the closing multipart boundary instead of assuming the
  footer's length. The two writers in this repository disagree about the
  trailing CRLF, and for a JPEG family codestream those two bytes are the EOI
  marker, so the frame failed to decode rather than merely looking odd.
- getImageInfo carries BitsStored, HighBit, the padding attributes and the SOP
  class through, since dicom-codec drops them before the decoder sees them.
- static-wado-creator exports codecFrame, so packages that already depend on it
  can transcode frames without adding a codec dependency.
- the root jest config comment no longer contains a glob that closes the block
  comment early.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Picks up @cornerstonejs/codec-openjph 2.4.10 and dicom-codec 1.0.12,
which carry the OpenJPH 0.30.1 bump, and drops the local file links into
/src/codecs that were standing in while those were unreleased.

Both package managers refuse a version published inside the two day
minimumReleaseAge soak, so the codecs need an exclusion to be installable
on release day. Unlike pnpm's, bun's list matches on package name only,
which is why the exact-version pins in the package.json files are what
bound the exposure here. The list has to be repeated in
bunfig.update-lockfile.toml because --config replaces bunfig.toml rather
than merging with it, while the global ~/.bunfig.toml still supplies
minimumReleaseAge.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant