fix: Decode partial htj2k stream - #68
Closed
wayfarer3130 wants to merge 2 commits into
Closed
Conversation
Merging this PR will regress 3 benchmarks
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Simulation | decode CT1.j2c (.201 lossless, 512x512x16bit) — warm |
36.8 ms | 40.3 ms | -8.69% |
| ❌ | Simulation | decode CT2.j2c (.201 lossless, 512x512x16bit) — warm |
36.6 ms | 39.2 ms | -6.76% |
| ❌ | Simulation | decode CT2.j2c (.201 lossless, 512x512x16bit) — cold |
36.6 ms | 39.2 ms | -6.74% |
| ⚡ | WallTime | HTJ2K Lossless (.201) |
130.5 ms | 31.5 ms | ×4.1 |
| ⚡ | Simulation | HTJ2K Lossless (.201) |
141.4 ms | 40.6 ms | ×3.5 |
| ⚡ | WallTime | instantiate+destroy HTJ2KEncoder x50 |
256.8 µs | 232 µs | +10.69% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing fix/htj2k-partial (9866b66) with main (a88a461)2
Footnotes
-
79 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
-
No successful run was found on
main(9c086c9) during the generation of this report, so a88a461 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩
4 tasks
📝 WalkthroughWalkthroughChangesHTJ2K decoder reuse and resilience
Estimated code review effort: 3 (Moderate) | ~25 minutes Suggested reviewers: Sequence Diagram(s)sequenceDiagram
participant HTJ2KDecode
participant CodecFactory
participant CodecConfig
participant Decoder
HTJ2KDecode->>CodecFactory: request decode with reuseDecoder true
CodecFactory->>CodecConfig: read or create reusedDecoder
CodecFactory->>Decoder: decode image frame
Decoder-->>CodecFactory: decoded image data
CodecFactory-->>HTJ2KDecode: return decoded image data
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/openjphjs/src/HTJ2KDecoder.hpp`:
- Around line 163-169: Update decode() at
packages/openjphjs/src/HTJ2KDecoder.hpp#L156-L170 and decodeSubResolution() at
packages/openjphjs/src/HTJ2KDecoder.hpp#L178-L192 to clear frameInfo_, metadata,
and pDecoded_ before each attempt, initialize the destination buffer before
pulling lines, and return or expose an explicit failure/completion status when
exceptions occur so callers cannot observe partial pixels or stale results.
In `@packages/openjphjs/test/node/index.js`:
- Around line 90-91: Ensure the encoded lossless fixture exceeds the truncation
limit before creating partial streams, then always slice at the fixed limit. In
packages/openjphjs/test/node/index.js lines 90-91, assert encodedLossless.length
exceeds 10 KiB and replace the Math.min-based slice; apply the same assertion
and fixed TRUNCATED_BYTE_LIMIT slice in
packages/openjphjs/test/truncated.test.js lines 112-113 and 164-165 before the
benchmark.
In `@packages/openjphjs/test/truncated.test.js`:
- Around line 264-284: Update the benchmark around reusedDecoder and fresh to
use medianDecodeMs for both paths: warm up reusedDecoder before collecting
samples, keep it alive through all reused samples, and create and delete a fresh
HTJ2KDecoder within each fresh-path sample. Compare the resulting median timings
and retain cleanup after sampling.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 9b2492c5-07ef-48e5-869c-3388dc300beb
📒 Files selected for processing (8)
packages/dicom-codec/src/codecs/codecFactory.jspackages/dicom-codec/src/codecs/htj2k.jspackages/openjphjs/bench/decode.bench.jspackages/openjphjs/extern/openjphpackages/openjphjs/src/CMakeLists.txtpackages/openjphjs/src/HTJ2KDecoder.hpppackages/openjphjs/test/node/index.jspackages/openjphjs/test/truncated.test.js
Comment on lines
+163
to
+169
| readHeader_(codestream, mem_file); | ||
| decode_(codestream, frameInfo_, 0); | ||
| } | ||
| catch (const std::exception &e) | ||
| { | ||
| OJPH_INFO(0x00010021, "decode failed (likely truncated stream): %s", e.what()); | ||
| } |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== files =="
git ls-files | rg '(^|/)HTJ2KDecoder\.hpp$|openjphjs'
echo "== HTJ2KDecoder outline =="
ast-grep outline packages/openjphjs/src/HTJ2KDecoder.hpp || true
echo "== relevant HTJ2KDecoder.hpp lines =="
wc -l packages/openjphjs/src/HTJ2KDecoder.hpp
cat -n packages/openjphjs/src/HTJ2KDecoder.hpp | sed -n '1,240p'
echo "== search result state methods/usages =="
rg -n "pDecoded_|decoded|resize\\(|decode\\(|decodeSubResolution|readHeader_|OJPH_INFO\\(0x0001002" packages/openjphjs/src packages/openjphjs -g '!**/node_modules/**' || trueRepository: cornerstonejs/codecs
Length of output: 31773
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== readHeader_/decode_ implementation =="
cat -n packages/openjphjs/src/HTJ2KDecoder.hpp | sed -n '270,455p'
echo "== decoder class members before implementation =="
cat -n packages/openjphjs/src/HTJ2KDecoder.hpp | sed -n '236,290p'
echo "== tests using reuse/failure paths =="
cat -n packages/openjphjs/test/truncated.test.js | sed -n '140,285p'
cat -n packages/openjphjs/test/node/index.js | sed -n '84,105p'Repository: cornerstonejs/codecs
Length of output: 16378
🌐 Web query:
C++ std::vector resize preserves elements unchanged when size stays same documentation
💡 Result:
Yes, the C++ standard explicitly states that std::vector::resize does nothing if the requested size equals the current size of the vector [1][2][3]. Because the function performs no operations when the size remains the same, all existing elements are preserved unchanged, and no iterators, references, or pointers to the elements are invalidated [4]. The C++ standard defines the effects of resize as follows: if the new size is less than the current size, it erases the excess elements at the end; if it is greater, it appends additional default-inserted or specified elements [5]. When the new size is identical to the current size, no elements are erased or appended, resulting in no changes to the container's state [5][4].
Citations:
- 1: https://en.cppreference.com/cpp/container/vector/resize
- 2: https://cppreference.dev/w/cpp/container/vector/resize
- 3: https://en.cppreference.com/Template:cpp/container/resize
- 4: https://stackoverflow.com/questions/53447386/what-if-size-argument-for-stdvectorresize-is-equal-to-the-current-size
- 5: https://timsong-cpp.github.io/cppwp/std23/vector
Reset decoder result state after incomplete decode.
decode() and decodeSubResolution() catch failures and return normally, but they leave frameInfo_, metadata, and pDecoded_ from previous successful decodes unchanged or only resized. Reused decoders must clear result state before each decode attempt, initialize the destination buffer before line pulls, and report a failure/completion status so callers do not return partial pixels or stale metadata.
This applies to:
packages/openjphjs/src/HTJ2KDecoder.hpp#L156-L170packages/openjphjs/src/HTJ2KDecoder.hpp#L178-L192
📍 Affects 1 file
packages/openjphjs/src/HTJ2KDecoder.hpp#L163-L169(this comment)packages/openjphjs/src/HTJ2KDecoder.hpp#L185-L191
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/openjphjs/src/HTJ2KDecoder.hpp` around lines 163 - 169, Update
decode() at packages/openjphjs/src/HTJ2KDecoder.hpp#L156-L170 and
decodeSubResolution() at packages/openjphjs/src/HTJ2KDecoder.hpp#L178-L192 to
clear frameInfo_, metadata, and pDecoded_ before each attempt, initialize the
destination buffer before pulling lines, and return or expose an explicit
failure/completion status when exceptions occur so callers cannot observe
partial pixels or stale results.
Comment on lines
+90
to
+91
| const truncatedSize = Math.min(10 * 1024, encodedLossless.length) | ||
| const truncatedBitstream = encodedLossless.slice(0, truncatedSize) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Guarantee partial-stream coverage.
Math.min(limit, encodedLength) permits a full stream when the fixture is small. Require an encoded fixture larger than the limit, then slice at the fixed limit.
packages/openjphjs/test/node/index.js#L90-L91: assert thatencodedLossless.lengthexceeds 10 KiB before slicing at 10 KiB.packages/openjphjs/test/truncated.test.js#L112-L113: assert thatencodedLossless.lengthexceedsTRUNCATED_BYTE_LIMITbefore slicing.packages/openjphjs/test/truncated.test.js#L164-L165: apply the same assertion and fixed slice before the truncated performance benchmark.
📍 Affects 2 files
packages/openjphjs/test/node/index.js#L90-L91(this comment)packages/openjphjs/test/truncated.test.js#L112-L113packages/openjphjs/test/truncated.test.js#L164-L165
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/openjphjs/test/node/index.js` around lines 90 - 91, Ensure the
encoded lossless fixture exceeds the truncation limit before creating partial
streams, then always slice at the fixed limit. In
packages/openjphjs/test/node/index.js lines 90-91, assert encodedLossless.length
exceeds 10 KiB and replace the Math.min-based slice; apply the same assertion
and fixed TRUNCATED_BYTE_LIMIT slice in
packages/openjphjs/test/truncated.test.js lines 112-113 and 164-165 before the
benchmark.
Comment on lines
+264
to
+284
| const reusedDecoder = new codec.HTJ2KDecoder() | ||
| const t0 = performance.now() | ||
| reusedDecoder.getEncodedBuffer(ct1Encoded.length).set(ct1Encoded) | ||
| reusedDecoder.decode() | ||
| reusedDecoder.getDecodedBuffer() | ||
| const reusedMs = performance.now() - t0 | ||
| reusedDecoder.delete() | ||
|
|
||
| const t1 = performance.now() | ||
| const fresh = new codec.HTJ2KDecoder() | ||
| fresh.getEncodedBuffer(ct1Encoded.length).set(ct1Encoded) | ||
| fresh.decode() | ||
| fresh.getDecodedBuffer() | ||
| fresh.delete() | ||
| const freshMs = performance.now() - t1 | ||
|
|
||
| console.log( | ||
| `Single decode — reused decoder: ${reusedMs.toFixed(2)} ms, fresh decoder: ${freshMs.toFixed(2)} ms` | ||
| ) | ||
|
|
||
| expect(reusedMs).toBeLessThan(freshMs) |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
Measure steady-state decoder reuse.
The timed reusedDecoder path performs its first decode after construction. It does not reuse the decoder before measurement. A single wall-clock sample can also fail due to scheduler and WebAssembly warm-up noise.
Use medianDecodeMs for both paths. Keep the reused decoder alive across its warm-up and samples. Create and delete a fresh decoder inside each fresh-path sample.
Proposed change
const reusedDecoder = new codec.HTJ2KDecoder()
-const t0 = performance.now()
-reusedDecoder.getEncodedBuffer(ct1Encoded.length).set(ct1Encoded)
-reusedDecoder.decode()
-reusedDecoder.getDecodedBuffer()
-const reusedMs = performance.now() - t0
-reusedDecoder.delete()
+let reusedMs
+try {
+ reusedMs = medianDecodeMs(() => {
+ reusedDecoder.getEncodedBuffer(ct1Encoded.length).set(ct1Encoded)
+ reusedDecoder.decode()
+ reusedDecoder.getDecodedBuffer()
+ })
+} finally {
+ reusedDecoder.delete()
+}
-const t1 = performance.now()
-const fresh = new codec.HTJ2KDecoder()
-fresh.getEncodedBuffer(ct1Encoded.length).set(ct1Encoded)
-fresh.decode()
-fresh.getDecodedBuffer()
-fresh.delete()
-const freshMs = performance.now() - t1
+const freshMs = medianDecodeMs(() => {
+ const fresh = new codec.HTJ2KDecoder()
+ try {
+ fresh.getEncodedBuffer(ct1Encoded.length).set(ct1Encoded)
+ fresh.decode()
+ fresh.getDecodedBuffer()
+ } finally {
+ fresh.delete()
+ }
+})📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| const reusedDecoder = new codec.HTJ2KDecoder() | |
| const t0 = performance.now() | |
| reusedDecoder.getEncodedBuffer(ct1Encoded.length).set(ct1Encoded) | |
| reusedDecoder.decode() | |
| reusedDecoder.getDecodedBuffer() | |
| const reusedMs = performance.now() - t0 | |
| reusedDecoder.delete() | |
| const t1 = performance.now() | |
| const fresh = new codec.HTJ2KDecoder() | |
| fresh.getEncodedBuffer(ct1Encoded.length).set(ct1Encoded) | |
| fresh.decode() | |
| fresh.getDecodedBuffer() | |
| fresh.delete() | |
| const freshMs = performance.now() - t1 | |
| console.log( | |
| `Single decode — reused decoder: ${reusedMs.toFixed(2)} ms, fresh decoder: ${freshMs.toFixed(2)} ms` | |
| ) | |
| expect(reusedMs).toBeLessThan(freshMs) | |
| const reusedDecoder = new codec.HTJ2KDecoder() | |
| let reusedMs | |
| try { | |
| reusedMs = medianDecodeMs(() => { | |
| reusedDecoder.getEncodedBuffer(ct1Encoded.length).set(ct1Encoded) | |
| reusedDecoder.decode() | |
| reusedDecoder.getDecodedBuffer() | |
| }) | |
| } finally { | |
| reusedDecoder.delete() | |
| } | |
| const freshMs = medianDecodeMs(() => { | |
| const fresh = new codec.HTJ2KDecoder() | |
| try { | |
| fresh.getEncodedBuffer(ct1Encoded.length).set(ct1Encoded) | |
| fresh.decode() | |
| fresh.getDecodedBuffer() | |
| } finally { | |
| fresh.delete() | |
| } | |
| }) | |
| console.log( | |
| `Single decode — reused decoder: ${reusedMs.toFixed(2)} ms, fresh decoder: ${freshMs.toFixed(2)} ms` | |
| ) | |
| expect(reusedMs).toBeLessThan(freshMs) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/openjphjs/test/truncated.test.js` around lines 264 - 284, Update the
benchmark around reusedDecoder and fresh to use medianDecodeMs for both paths:
warm up reusedDecoder before collecting samples, keep it alive through all
reused samples, and create and delete a fresh HTJ2KDecoder within each
fresh-path sample. Compare the resulting median timings and retain cleanup after
sampling.
wayfarer3130
added a commit
that referenced
this pull request
Aug 19, 2026
Single squashed commit of ci/pnpm-trusted-publishing (#87), on the assumption that #87 lands on main before this PR. Purpose is measurement: the pnpm migration shifts CodSpeed's baseline on its own, and the HTJ2K work in this branch shifts it again, so carrying both here lets one report show the combined effect instead of attributing the sum to whichever merges second. Expect this commit to become a no-op the moment #87 merges -- it should then either drop out of the diff or merge cleanly against itself. It is NOT a second copy of that work to review; review it in #87. Merged with no conflicts. Two things worth noting about the overlap: - The submodule gitlink stayed at this branch's 0748112b rather than taking #87's e01c7b7, because #87 only reverted its own accidental bump back to the value main already had. Updated separately in the next commit. - dicom-codec/src/codecs/codecFactory.js is touched by both branches and did not conflict: #87 changes initialize() (routing emscripten's print through the logger) while this branch's carried work from #68 changes decode() (decoder reuse). They are independent edits to the same file. Includes the pnpm.overrides pinning esbuild/rollup/webpack/terser to the versions yarn.lock resolved, so build output does not drift across the migration -- relevant here because this PR is measured against those baselines.
wayfarer3130
added a commit
that referenced
this pull request
Aug 19, 2026
…ength Carried from #68, which this PR supersedes. Previously the decoder could handle a partial HTJ2K stream only if the caller already knew the full length; now a truncated buffer decodes as far as its data allows. readHeader, decode and decodeSubResolution wrap their codestream work in try/catch and report instead of propagating, so resilient mode's throw at the end of the available data yields a partial image rather than a failed decode. frameInfo_ keeps whatever the header established, so dimensions survive. Two deliberate changes from #68's version: - The diagnostics are OJPH_WARN, not OJPH_INFO. jslib.cpp raises OpenJPH's threshold to WARN to kill the per-construction banner, so INFO here would be dropped exactly when a decode failed. - DISABLE_EXCEPTION_CATCHING flips 1 -> 0 (double negative: catching ENABLED). This is required, not stylistic: with catching disabled emscripten compiles the handlers out and the throw terminates the module instead of being caught. It costs wasm size, so dist-size may need re-baselining. test/truncated.test.js covers truncated and lossy decodes, and decoder reuse across 500 decodes. NOTE: its performance assertions are wall-clock (reused-faster-than-fresh, and a min/max ratio across milestones), so they are inherently softer than the pixel-exactness tests and may prove flaky on shared CI runners. Worth watching, and worth converting to a looser bound or dropping if they turn noisy. The core-side work is upstream as of aous72#331, so this is only the emscripten wrapper plus tests -- the corresponding fork patches are gone.
wayfarer3130
added a commit
that referenced
this pull request
Aug 19, 2026
Carried from #68. codecFactory.decode gains an opt-in reuseDecoder option: the decoder is held on codecConfig (the per-codec singleton the wrapper modules already share) and not deleted after each call. htj2k.js opts in; every other codec keeps the construct-and-delete behaviour. This is very likely the bulk of #68's measured 3.5x speedup on the dicom-codec dispatch bench for HTJ2K -- CodSpeed reported 141.4ms -> 40.6ms there, and 40.6ms is about what openjphjs' own direct decode benches cost, i.e. reuse closes the gap between dispatching through this factory and calling the codec directly. Constructing a wasm decoder per frame allocates heap and registers embind bindings each time; for openjph it also ran the constructor banner through the console on every frame. Opt-in rather than default on purpose: a decoder that carries state between decodes, or whose retained buffers grow without bound, must not enable it. openjphjs' reuse test covers the consequence that matters for HTJ2K -- 500 successive decodes on one instance without progressive slowdown. Independent of #87's change to the same file: that one routes emscripten's print through the logger in initialize(); this one changes decode(). They merged with no conflict.
Contributor
Author
|
Superseded by #76, which now carries all of this work. Closing. Everything here was folded in, with two deliberate changes:
Carried across verbatim: Also worth recording here: the decoder-reuse change is very likely the bulk of the 3.5× dispatch speedup this PR measured (141.4 ms → 40.6 ms on Your |
wayfarer3130
added a commit
that referenced
this pull request
Aug 27, 2026
…ersedes #68) (#76) codspeed regression is due to parallelization not being consistent in the simulation, not due to code speed changes * chore(openjphjs): bump extern/openjph submodule to upstream 0.30.1 Points the openjph submodule at cornerstonejs/OpenJPH#5, which rebases the fork onto upstream OpenJPH 0.30.1 (was ~22 months behind) and re-applies our custom patches. Net cornerstonejs delta from stock 0.30.1 is 3 lines in ojph_codestream_local.cpp: resilient=true (tolerate truncated streams) + suppressed 'File terminated early' log. Dropped the cosmetic SIZ-marker message renames (conflicted with upstream's swap_byte rename) and the temporary debug-build toggle. CI is the first real build/validation of this bump (not built locally). After OpenJPH#5 merges, re-point this submodule at the merge commit. * fix(openjphjs): add OpenJPH 0.30.1 header dirs to the wasm target include path 0.30.1 relocated its public headers under src/core/openjph (+ src/core/shared); our glue's bare <ojph_arch.h> include no longer resolved and the openjphjs wasm build failed with 'ojph_arch.h file not found'. Add both 0.30.1 header roots to the openjphjs target's include path. * fix(openjphjs): link the OpenJPH 0.30.1 'openjph' target (was renamed from openjphsimd) 0.30.1 builds a single architecture-agnostic 'openjph' library; the old 'openjphsimd' target no longer exists, so wasm-ld failed with 'unable to find library -lopenjphsimd'. Link 'openjph', matching upstream's own wasm wrapper (subprojects/js). * fix(openjphjs): keep WASM SIMD enabled for OpenJPH 0.30.1 0.30.1 deprecated OJPH_DISABLE_INTEL_SIMD and bridges it onto the new OJPH_DISABLE_SIMD; our old 'OJPH_DISABLE_INTEL_SIMD=ON' therefore disabled ALL SIMD (OJPH_ENABLE_WASM_SIMD=OFF), shipping a scalar wasm ~2x slower on decode/ encode. Stop setting the deprecated option and force OJPH_DISABLE_SIMD=OFF so 0.30.1's Emscripten path builds the WASM SIMD kernels (-msimd128). * fix(openjphjs): build the wasm in Release, not Debug build.sh forced CMAKE_BUILD_TYPE=Debug, so the shipped openjph wasm was built -O0 with unoptimized SIMD intrinsics — the reason decode/encode benched far slower (SIMD-on was even slower than scalar under -O0) and the binary was oversized. Release (-O3) is the correct artifact and is what makes the 0.30.1 SIMD kernels fast and the wasm small. * ci: squash in the pnpm migration from #87 Single squashed commit of ci/pnpm-trusted-publishing (#87), on the assumption that #87 lands on main before this PR. Purpose is measurement: the pnpm migration shifts CodSpeed's baseline on its own, and the HTJ2K work in this branch shifts it again, so carrying both here lets one report show the combined effect instead of attributing the sum to whichever merges second. Expect this commit to become a no-op the moment #87 merges -- it should then either drop out of the diff or merge cleanly against itself. It is NOT a second copy of that work to review; review it in #87. Merged with no conflicts. Two things worth noting about the overlap: - The submodule gitlink stayed at this branch's 0748112b rather than taking #87's e01c7b7, because #87 only reverted its own accidental bump back to the value main already had. Updated separately in the next commit. - dicom-codec/src/codecs/codecFactory.js is touched by both branches and did not conflict: #87 changes initialize() (routing emscripten's print through the logger) while this branch's carried work from #68 changes decode() (decoder reuse). They are independent edits to the same file. Includes the pnpm.overrides pinning esbuild/rollup/webpack/terser to the versions yarn.lock resolved, so build output does not drift across the migration -- relevant here because this PR is measured against those baselines. * build(openjphjs): track upstream OpenJPH master for the streaming fix Moves extern/openjph from 0.30.1 + carried patches to cornerstonejs/OpenJPH#6, which merges upstream master (6f3caf3) with ZERO fork delta. Why master rather than a release: the streaming/truncated-decode fix landed upstream as 638ccb4 "Cs3d/truncated decode graceful 0.30.1 (aous72#331)" on 2026-08-08, and the newest upstream release 0.31.0 was published 2026-07-27 -- twelve days earlier. `git tag --contains 638ccb4` is empty, so no tagged release carries it yet. master is four commits past the fix (three dependabot codeql bumps and a warning fix). Re-pin to a tag once upstream cuts one with #331. Both patches the fork used to carry are gone, replaced by public API: resilient = true codestream::enable_resilience() -- already called by HTJ2KDecoder on main (line 270), so this patch was redundant before this PR. commented-out OJPH_INFO "File terminated early" ojph::set_message_level(OJPH_MSG_WARN), set here in jslib.cpp. The message level is worth its own note. OpenJPH INFO goes to stdout, which emscripten forwards to console.log, and there were two sources of per-decode noise: HTJ2KDecoder's constructor banner ("v06 HTJ2K Decoder") on every construction, and "File terminated early" on every resilient decode of a truncated stream -- which with streaming support is the normal case. Raising the threshold to WARN drops both and keeps warnings and errors, so it replaces a source patch with a supported call and makes future upstream bumps fast-forwards. Verified: OJPH_DISABLE_SIMD still exists upstream (this branch's FORCE OFF is still correct), and every header HTJ2KDecoder.hpp includes is present under src/core/openjph on master. * fix(openjphjs): decode truncated HTJ2K streams without a known full length Carried from #68, which this PR supersedes. Previously the decoder could handle a partial HTJ2K stream only if the caller already knew the full length; now a truncated buffer decodes as far as its data allows. readHeader, decode and decodeSubResolution wrap their codestream work in try/catch and report instead of propagating, so resilient mode's throw at the end of the available data yields a partial image rather than a failed decode. frameInfo_ keeps whatever the header established, so dimensions survive. Two deliberate changes from #68's version: - The diagnostics are OJPH_WARN, not OJPH_INFO. jslib.cpp raises OpenJPH's threshold to WARN to kill the per-construction banner, so INFO here would be dropped exactly when a decode failed. - DISABLE_EXCEPTION_CATCHING flips 1 -> 0 (double negative: catching ENABLED). This is required, not stylistic: with catching disabled emscripten compiles the handlers out and the throw terminates the module instead of being caught. It costs wasm size, so dist-size may need re-baselining. test/truncated.test.js covers truncated and lossy decodes, and decoder reuse across 500 decodes. NOTE: its performance assertions are wall-clock (reused-faster-than-fresh, and a min/max ratio across milestones), so they are inherently softer than the pixel-exactness tests and may prove flaky on shared CI runners. Worth watching, and worth converting to a looser bound or dropping if they turn noisy. The core-side work is upstream as of aous72#331, so this is only the emscripten wrapper plus tests -- the corresponding fork patches are gone. * perf(dicom-codec): reuse the HTJ2K decoder instead of one per frame Carried from #68. codecFactory.decode gains an opt-in reuseDecoder option: the decoder is held on codecConfig (the per-codec singleton the wrapper modules already share) and not deleted after each call. htj2k.js opts in; every other codec keeps the construct-and-delete behaviour. This is very likely the bulk of #68's measured 3.5x speedup on the dicom-codec dispatch bench for HTJ2K -- CodSpeed reported 141.4ms -> 40.6ms there, and 40.6ms is about what openjphjs' own direct decode benches cost, i.e. reuse closes the gap between dispatching through this factory and calling the codec directly. Constructing a wasm decoder per frame allocates heap and registers embind bindings each time; for openjph it also ran the constructor banner through the console on every frame. Opt-in rather than default on purpose: a decoder that carries state between decodes, or whose retained buffers grow without bound, must not enable it. openjphjs' reuse test covers the consequence that matters for HTJ2K -- 500 successive decodes on one instance without progressive slowdown. Independent of #87's change to the same file: that one routes emscripten's print through the logger in initialize(); this one changes decode(). They merged with no conflict. * test(openjphjs): make the decoder-reuse perf assertion measure something real Built openjphjs locally via tools/docker/build.sh and ran the suite, which is how this surfaced: "reused decoder is faster than instantiate+decode+destroy per frame" FAILED locally (3.34 ms vs 2.72 ms) while passing CI by 5% (2.38 vs 2.50). The original had a structural flaw, not bad luck. It took ONE sample per path with no warmup, and measured the reused path FIRST -- so V8's JIT warmup was charged to exactly the side the assertion expects to win. Construction costs well under a millisecond against a ~2.5 ms decode, so a single cold sample measures warmup rather than the difference under test. Fixed the measurement: warm both paths, then compare medians of 25 iterations. That removed the order bias but showed the assertion itself is not sound at this granularity -- warmed, construct+decode+destroy costs about the same as decode alone (~1.6 ms each), so the medians sit inside each other's noise. Eight observed runs produced two failures on unchanged code. So the assertion is now a bound in the useful direction: reuse must not be materially SLOWER (the real risk, e.g. retained state degrading each decode) rather than provably faster. The medians are still logged. This does not weaken the perf claim, it relocates it to the tool that can actually measure it. CodSpeed on this branch reports the dispatch bench 141.5 ms -> 24.1 ms and instantiate+destroy HTJ2KDecoder x50 2315 us -> 458 us, because Simulation counts instructions where wall-clock at ~3% of a decode cannot resolve it. The 500-decode stability test is untouched and still guards the thing that matters for reuse: no progressive slowdown from retained buffers. Verified: 5 consecutive local runs stable, full openjphjs suite 30 passed. * fix(htj2k): own the decoded buffer and surface swallowed decode failures Addresses the review findings on #76, all of which stem from the same two changes in this PR: reusing one HTJ2K decoder across a series, and swallowing OpenJPH's exceptions so a partial codestream degrades to a partial image. Buffer ownership (the critical one). getDecodedBuffer()/getEncodedBuffer() return an emscripten typed_memory_view -- a live window onto the wasm heap owned by the codec instance. Returning it as `imageFrame` was wrong three ways: delete() frees the memory it points at, the next decode on a reused instance overwrites it, and heap growth detaches it outright. Frames 1..n of a series all showed frame n. copyFromWasm() now copies on both the decode and encode paths -- unconditionally, because the non-reuse path was already handing back memory delete() had just freed. Measured side effect: the raw view's .buffer is the whole 50 MB heap, so callers passing imageFrame.buffer to a worker were transferring the heap rather than the frame. Failure reporting. decode()/readHeader()/decodeSubResolution() returned normally after swallowing an exception, so codecFactory reported success and the OJPH_WARN went to a logger that is silent unless setVerbose. HTJ2KDecoder now exposes getIsHeaderValid()/getLastErrorMessage(), reset per call, and codecFactory throws on an invalid header while flagging processInfo.partial otherwise. Verified: before this, garbage input on a reused decoder resolved successfully with the previous slice's pixels under the new frame's metadata. Stale pixels. decode_ used resize(), which only value-initialises NEW elements, so anything the decoder did not write kept the previous frame's pixels. Now assign(size, 0), which zero-fills without giving up the capacity reuse depends on. readHeader_ likewise resets every header-derived field before parsing, and the previously uninitialised members (numDecompositions_, numLayers_, ...) get initialisers -- a failed parse on a fresh decoder was doing arithmetic on heap garbage. Two comments in this PR claimed things that turned out to be false, corrected in place. Truncation does NOT throw into decode()'s catch: swept CT1.j2c at every length from 60 bytes up plus 875 single-byte corruptions and not one input aborts mid-decode -- resilient mode absorbs a short codestream as zero coefficients and reports success. The reachable stale-buffer window is restrict_input_resolution() throwing for a decomposition level the codestream does not carry, which is what the new regression test uses (it fails against resize() with 128 stale bytes; a truncation-based test passes either way and tests nothing). Also: - unset(OJPH_DISABLE_INTEL_SIMD CACHE): deleting the option() line does not remove it from an existing CMakeCache.txt, and upstream's `DEFINED` bridge shadows the forced OJPH_DISABLE_SIMD=OFF, so incremental local builds kept shipping the scalar wasm this PR's own comment warns about. - releaseDecoder()/htj2k.release()/dicomCodec.release(): a reused decoder held its largest frame's buffers for the module's lifetime with no way to free. - dist-size baseline for openjphjs: stale at 2241 KiB against a 293 KiB artifact, because THIS PR switched the build Debug -> Release. The gate only fails on growth, so it would have tolerated a 7.7x regression. Rebuilt with tools/docker/build.sh (emsdk 3.1.74, SIMD confirmed intact); openjphjs 34 passed, dicom-codec 41 passed, other codec suites unaffected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Bill Wallace <wayfarer3130@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The previous version of the htj2k decoder could decode a partial htj2k stream IF it knew the exact full length beforehand. This change allows just giving the htj2k a partial input and allowing it to decode. Most of hte changes are in cornerstonejs/OpenJPH#3
Also added new tests for this and change the decoder to work repeatedly rather than getting slower and slower.
Summary by CodeRabbit
Performance
Reliability
Compatibility