feat: alternates and transcode operations, with a hierarchical brick store - #130
Open
wayfarer3130 wants to merge 2 commits into
Open
feat: alternates and transcode operations, with a hierarchical brick store#130wayfarer3130 wants to merge 2 commits into
wayfarer3130 wants to merge 2 commits into
Conversation
…store Adds two secondary createdicomweb operations that act on an already created DICOMweb tree, addressed by study UID. Neither touches the create operation, both are single threaded, and both are idempotent and resumable. alternates writes renditions beside frames/, which it never writes or deletes, so the primary rendition stays byte identical: --jls / --htj2k / --htj2k-lossy full resolution copies of every frame --jls-thumbnail quarter resolution copy of every frame --brick hierarchical brick pyramid, series level Every requested rendition is built from the same decode of the same frames, so the size and compression report it finishes with is a fair comparison across them. Ratios are measured against each rendition's own dimensions, from bytes actually on disk, and --json emits the same figures for collection across a corpus. The brick store is the part CS3D consumes. Levels reduce per axis by voxel spacing rather than by a single factor, so coarse levels of anisotropic data stay physically isotropic, and every level is a box average of the one above rather than a decimation. Bricks are stored at their true extent, packed into one codestream per brick with the row ordering chosen to feed the JPEG-LS predictor, and described by a manifest written last so its presence marks the store complete. Ineligible series - colour, too few slices, a temporal third axis, or anything off a regular rectilinear grid - are skipped with a reason rather than failing the run. transcode rewrites uncompressed grayscale frames/ to JPEG-LS lossless, staging frames and swapping them in only once the whole instance has encoded. Also in this change: - thumbnail reduction is now a box average rather than a nearest-neighbour replicate, which changes existing jlsThumbnail and alternateThumbnail output. Decimation aliases; where the attributes are available the average is taken over true pixel values, excludes pixel padding, and takes the first occupied sample for segmentation label maps. - readBulkData locates the closing multipart boundary instead of assuming the footer's length. The two writers in this repository disagree about the trailing CRLF, and for a JPEG family codestream those two bytes are the EOI marker, so the frame failed to decode rather than merely looking odd. - getImageInfo carries BitsStored, HighBit, the padding attributes and the SOP class through, since dicom-codec drops them before the decoder sees them. - static-wado-creator exports codecFrame, so packages that already depend on it can transcode frames without adding a codec dependency. - the root jest config comment no longer contains a glob that closes the block comment early. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Picks up @cornerstonejs/codec-openjph 2.4.10 and dicom-codec 1.0.12, which carry the OpenJPH 0.30.1 bump, and drops the local file links into /src/codecs that were standing in while those were unreleased. Both package managers refuse a version published inside the two day minimumReleaseAge soak, so the codecs need an exclusion to be installable on release day. Unlike pnpm's, bun's list matches on package name only, which is why the exact-version pins in the package.json files are what bound the exposure here. The list has to be repeated in bunfig.update-lockfile.toml because --config replaces bunfig.toml rather than merging with it, while the global ~/.bunfig.toml still supplies minimumReleaseAge. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds two secondary
createdicomweboperations that act on an already created DICOMweb tree, addressed by study UID. Neither touches thecreateoperation, both are single threaded, and both are idempotent and resumable. The brick store this generates has been tested against CS3D.alternatesWrites renditions beside
frames/, which it never writes or deletes, so the primary rendition stays byte identical.--jlsjls/1.2.840.10008.1.2.4.80--jls-thumbnailjlsThumbnail/1.2.840.10008.1.2.4.80, quarter resolution--htj2khtj2k/1.2.840.10008.1.2.4.201--htj2k-lossyhtj2kLossy/1.2.840.10008.1.2.4.203--brickbrick/--brick-codec, series levelEvery requested rendition is built from the same decode of the same frames, so the size and compression summary it finishes with is a fair comparison across them. Ratios are measured from bytes actually on disk against each rendition's own dimensions, so a quarter resolution thumbnail is not credited with a 16x ratio it did not earn, and the brick store reports its cost as a percentage of
frames/.--jsonemits the same figures on stdout with progress on stderr.Renditions are grayscale only. Ineligible instances and series are skipped with a logged reason and the command still exits 0; a non-zero exit means a series errored.
The brick store
d1, d2, d4ladder; 5 mm slice data brings the in-plane axes in first, so coarse levels are physically rather than numerically cubic and carry all three factors in the name (d8_8_2).z-minorby default, orplane-majorfor thick slices) is chosen to feed the JPEG-LS MED predictor.series/{uid}/brick/{level}/{t###}/{k###}/y{ky}x{kx}.jls, with the{t###}component omitted for a plain 3D series. Non-spatial axes are indexed, never subsampled.manifest.jsoncarries axes, per-level sizes, factors, brick pitch and counts, order, spacing and transfer syntax, and is written last, so its presence marks the store complete and an interrupted run rebuilds rather than trusting a partial store.(0009,10E0) UR BrickManifestURIunder creatorRadicalImaging, extending the group 0009 block already used for Content-Location. Layout details stay in the manifest, so they can change without a tag change.ImagePositionPatient(third index is time, not space), or anything off a regular rectilinear grid - irregular spacing, gantry tilt, non-coplanar frames, ragged sampling.Memory is bounded by a per-level slab rather than the volume: planes are reduced and handed down as they arrive, so the whole pyramid costs a little over one full-resolution slab.
transcodeRewrites uncompressed grayscale
frames/to JPEG-LS lossless and updatesAvailableTransferSyntaxUID. Colour and already compressed instances are untouched. Frames are staged beside the instance and moved into place only once the whole instance has encoded, so a failure leaves the originals intact.Behaviour change to existing output
Thumbnail reduction is now a box average rather than the nearest-neighbour
replicate, which changes existingjlsThumbnailandalternateThumbnailoutput. Where the attributes are available the average is taken over true pixel values rather than stored words, pixel padding is left out, and segmentation label maps take the first occupied sample of each box - the mean of two labels is a third segment that is in neither place.Fixes carried along
readBulkDatanow locates the closing multipart boundary instead of assuming the footer's length. The two writers in this repository disagree about the trailing CRLF, and for a JPEG family codestream those two bytes are the EOI marker, so the frame failed to decode rather than merely looking odd.getImageInfocarriesBitsStored,HighBit, the pixel padding attributes and the SOP class through, since dicom-codec'sadaptImageInfodrops them before the decoder sees them.static-wado-creatorexportscodecFrame, so packages that already depend on it can transcode frames without adding a codec dependency.*/closed the block comment early.Docs
Root README.md gains an "Alternate Renditions and Brick Stores" section covering the CLI, the pyramid, the manifest and the eligibility rules; file-structure.md gains the new paths; the create-dicomweb README gains the command usage.
Testing
jest --config packages/create-dicomweb/jest.config.cjs), including 5 new suites: box filtering, brick packing round-trips under both orderings, brick eligibility, the pyramid end to end, and frame renditions.bin/createdicomweb.mjsandstatic-wado-util/lib/index.jsalready fail prettier onmaster; the new CLI options follow the file's existing style rather than reformatting unrelated lines.🤖 Generated with Claude Code