Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .changeset/olive-mountains-install.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
"@cosyte/mllp": patch
---

The `@cosyte/hl7` dev/test peer now installs from the npm registry instead of a
vendored packed tarball, and `vendor/` is deleted.

The tarball held `@cosyte/hl7` at `0.0.0` and never moved while that package
shipped ten releases. One of them made the MSA-2 control-id echo byte-verbatim
across the full escape alphabet. Because the pinned copy never picked it up,
this package's own test suite went on asserting the old behaviour as a
guarantee: an inbound `MSH-10` of `ID\X` echoed back as `ID\E\X`, which is a
different control id on the wire, so an ACK the sender cannot correlate, so a
resend, so a duplicate clinical message. That case is now pinned inverted, as a
round-trip through the parser path that must produce the same bytes as the
byte-copy path and must not warn.

No runtime behaviour of this package changes: `@cosyte/hl7` remains an optional
peer dependency referenced only from the `@cosyte/mllp/ack-from-hl7` subpath,
and the emitted framing, ACK and warning-code surfaces are untouched. What
changes is which version of the peer the tests run against, and therefore what
they can prove.
6 changes: 4 additions & 2 deletions .github/workflows/no-emdash.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,15 @@ name: Em-dash gate
# the script it calls refuses to print OK from a scan that did not read everything it
# was handed, and why every one of those refusals was checked red before this landed.
#
# mllp tracks ONE binary, `vendor/cosyte-hl7-0.0.0.tgz` (the packed `@cosyte/hl7`
# mllp used to track ONE binary, `vendor/cosyte-hl7-0.0.0.tgz` (the packed `@cosyte/hl7`
# devDependency, 472,097 bytes, 1,774 NUL bytes). It is why this repo runs website's
# NUL-exclusion variant of the script rather than the text-only one hl7, fhir,
# knowledgebase, pathways, x12, ncpdp and dicom run. The reasoning, the cost, and the
# measurements behind that choice are in the script header; the short version is that
# a compressed stream can contain the em dash bytes by coincidence, and that red would
# have no fix.
# have no fix. The tarball left on 2026-08-10 (`@cosyte/hl7` installs from npm) and the
# VARIANT STAYS: the argument is about tracked binaries as a class, and the script header
# says why the specimen's departure does not retire it.
#
# WHY A SEPARATE WORKFLOW, and not a job in ci.yml:
#
Expand Down
18 changes: 9 additions & 9 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,10 @@ deleting the pointer or the heading.** Why, and every disclosed miss:
- **Phase 9 of 11.** Client / server / framing / connection / transport shipped; Phases 6, 7, 8
and 9 are done. Next: `operations/roadmaps/mllp.md`. What each phase actually shipped:
`documentation/agent-notes.md#shipped-phases-and-the-vendored-hl7-peer-tarball`
- **For dev/test the `@cosyte/hl7` peer is a vendored packed tarball**
(`vendor/cosyte-hl7-0.0.0.tgz`, a devDependency), interim until the cross-repo consumption
decision lands. **Re-vendoring strips the `peerDependencies` entry: restore it afterwards.** The
exact command sequence: `documentation/agent-notes.md#shipped-phases-and-the-vendored-hl7-peer-tarball`
- **The `@cosyte/hl7` peer installs from npm**, like every other `@cosyte/*` dep. The vendored
tarball is GONE: it pinned dev/test to a `0.0.0` snapshot and hid a FIXED MSA-2 correlation bug
for ten releases. **Never re-vendor one:**
`documentation/agent-notes.md#shipped-phases-and-the-vendored-hl7-peer-tarball`
- **This package is on the npm registry, and this file names no version, deliberately** (derive it:
`npm view @cosyte/mllp version`). Never quote a version here, never move a published version
backwards, and never infer repo visibility from publish state or the reverse: they are
Expand All @@ -48,12 +48,12 @@ Full rationale, measurements and residuals: `documentation/agent-notes.md#the-em
parentheses.
- **Never add `grep -I`.** Measured on GNU grep 3.8: it skips a text file whose bad byte shares a
line with the em dash, in total silence, and the gate prints OK.
- **Never switch this copy to the text-only shape the other parsers run.** The vendored tarball is a
compressed stream that can carry `E2 80 94` by coincidence, and a red with no possible fix is a
gate someone disables.
- **Never switch this copy to the text-only shape the other parsers run.** The tarball that forced
it is gone; the shape stays. A binary carrying `E2 80 94` by coincidence is a red with no possible
fix, which is a gate someone disables.
- **The NUL exclusion is a disclosed miss, not a pass**, and the tell is the excluded count on the OK
line: **it reads 1 today**. If a NUL-bearing text fixture ever lands, revisit the partition, never
the ban. The at-risk fixture class already exists, so do not round this off to hypothetical.
line: **it reads 0 today**, 1 until the tarball left. If a NUL-bearing text fixture ever lands,
revisit the partition, never the ban. The at-risk class exists, so do not call this hypothetical.
- **Never count over markdown alone.** A markdown-only count is what wrongly cleared `dicom`.
- **The script is composed from three sibling copies; understand the composition before editing it**,
and fix shared limits in the script header, not here.
Expand Down
29 changes: 16 additions & 13 deletions documentation/agent-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,15 @@ thing a rule guards, or when you are tempted to relax one. Every paragraph here
Mirth/NextGen (`test/differential/`, `MLLP_DIFF_ADAPTER`-gated live tier), the §3 quirk corpus
(`test/conformance/`), and a PHI/observability audit that closed the `MLLP_FRAME_TOO_LARGE`
`snippet` payload-slice leak) done. Next: see `operations/roadmaps/mllp.md` for what follows Phase 9.
For dev/test the
`@cosyte/hl7` peer is consumed as a **vendored packed
tarball** (`vendor/cosyte-hl7-0.0.0.tgz`, a devDependency): an interim mechanism until the
cross-repo consumption decision (umbrella `PW-5` gate) lands; refresh it by re-running
`pnpm -C ../hl7 build && pnpm -C ../hl7 pack --out ../mllp/vendor/cosyte-hl7-0.0.0.tgz`
(`--out` resolves relative to the `-C` directory) then `pnpm remove @cosyte/hl7 &&
pnpm add -D @cosyte/hl7@file:vendor/cosyte-hl7-0.0.0.tgz`. Note `pnpm remove` also
strips the `peerDependencies` entry; restore it (`"@cosyte/hl7": ">=0.0.0"`) after.
- **The vendored tarball is GONE. The `@cosyte/hl7` peer installs from npm** at `^0.0.N` (ADR 0032,
which superseded the interim ADR 0008 five weeks after its own `PUB-FLIP` trigger fired).
**Why you never re-vendor one.** The tarball pinned dev/test to `0.0.0` while `@cosyte/hl7`
shipped ten releases. One made the MSA-2 echo byte-verbatim across the escape alphabet, naming
THIS package's correlation as the reason. The fix never arrived, so
`test/ack-from-hl7/control-id-verbatim.test.ts` asserted the DEFECT as a guarantee: `ID\X` echoed
back `ID\E\X`, a different control id on the wire, so an unmatched ACK, a resend, a duplicate
clinical message. **A pinned blob does not read as stale, it reads as the contract** -- which is
what makes it worse than an out-of-date range, since a range announces its age.

## The em-dash brand gate

Expand All @@ -53,11 +54,13 @@ thing a rule guards, or when you are tempted to relax one. Every paragraph here
applies the `./` prefix in the list-building loop instead of through `sed -z`, so the scan is a
single command with the stderr capture bound to all of it, which closes a residual the `ncpdp`
and `dicom` copies still carry.
**Why not the text-only shape the other parsers run, since `vendor/cosyte-hl7-0.0.0.tgz` has no
em dash in it today:** the reason is durability, not a present-day red. That tarball is a
compressed stream, it is re-vendored by hand (see the Phase 9 note above), and a compressed
stream can contain `E2 80 94` by coincidence. Measured both ways against the real file: `dicom`'s
copy is green on today's bytes and **red, unremediably, on a copy seeded with those three bytes**.
**Why not the text-only shape the other parsers run:** durability, not a present-day red, and the
reason OUTLIVED the file that prompted it. The argument was about `vendor/cosyte-hl7-0.0.0.tgz`, a
hand-refreshed compressed stream that can contain `E2 80 94` by coincidence. Measured both ways
against the real file at the time: `dicom`'s copy was green on those bytes and **red, unremediably,
on a copy seeded with those three bytes**. **The tarball is gone and the shape stays**, because the
property belongs to binaries in general, not to that one file. Read the live excluded count off the
gate's OK line.
You cannot rewrite a compressed byte stream with a period, and a red with no fix is a gate
someone disables.
**The disclosed cost, said plainly: a tracked TEXT file holding a NUL byte is silently exempt,
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@
"@arethetypeswrong/cli": "0.18.4",
"@changesets/cli": "2.31.0",
"@cosyte/eslint-config": "^0.0.3",
"@cosyte/hl7": "file:vendor/cosyte-hl7-0.0.0.tgz",
"@cosyte/hl7": "^0.0.10",
"@cosyte/prettier-config": "^0.0.2",
"@cosyte/test-utils": "^0.0.2",
"@cosyte/tsconfig": "^0.0.2",
Expand Down
11 changes: 5 additions & 6 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 5 additions & 4 deletions scripts/check-agent-notes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -116,10 +116,11 @@
* (v) [PINNED] A NUL-BEARING FILE IS SKIPPED WHOLE, SO ITS POINTERS ARE NEVER READ. This is
* the one that can print `all resolving` over a dangling pointer, and it is a disclosed
* miss rather than a pass. THE TELL IS THE SKIPPED COUNT ON THE OK LINE, exactly as
* `check-no-emdash.sh`'s NUL exclusion works and for the same reason: the one file it
* excludes today is the vendored `@cosyte/hl7` tarball, a compressed stream that cannot
* be read as markdown and cannot be edited to fix a red. If a NUL-bearing TEXT file ever
* lands here, revisit the partition rather than the rule. `CLAUDE.md` requires that
* `check-no-emdash.sh`'s NUL exclusion works and for the same reason. It excludes NO file
* today: the one it used to exclude was the vendored `@cosyte/hl7` tarball, a compressed
* stream that could not be read as markdown and could not be edited to fix a red, and that
* tarball is gone (the peer installs from npm). The rule stays. If a NUL-bearing TEXT file
* ever lands here, revisit the partition rather than the rule. `CLAUDE.md` requires that
* exclusion be carried as a disclosed miss and says the at-risk class already exists, so
* do not round this off to hypothetical.
* (vi) [PINNED] AN ATX HEADING INSIDE A FENCED CODE BLOCK IS NOT AN ANCHOR, and the fence
Expand Down
59 changes: 34 additions & 25 deletions scripts/check-no-emdash.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,19 +25,21 @@
# * ZERO tracked files carry U+2014, as the literal character or in any encoded form.
# So this gate changed no content in this repo. It exists purely to stop a regression,
# which is the only reason to add a gate to a clean tree.
# * ONE file holds a NUL byte: `vendor/cosyte-hl7-0.0.0.tgz` (472,097 bytes, 1,774 NULs),
# a real gzip stream and the only binary this repo tracks. See BINARY EXCLUSION.
# * FOUR files are what GIT calls binary (`git ls-files --eol` reports `i/-text`), which is
# ZERO files hold a NUL byte. This read ONE until ADR 0032 deleted
# `vendor/cosyte-hl7-0.0.0.tgz`, a real gzip stream and the only binary this repo tracked.
# THE BINARY EXCLUSION STAYS; its section explains why the specimen leaving does not
# retire the shape. The live count is on the OK line, not here.
# * THREE files are what GIT calls binary (`git ls-files --eol` reports `i/-text`), which is
# a WIDER set than the NUL rule and is the one worth naming, because it is where the
# exclusion could start biting without anyone noticing. The other three are
# exclusion could start biting without anyone noticing. They are
# `test/differential/fixtures/r1-ack-aa.frame.bin`, `r1-adt-a01.frame.bin` and
# `r1-oru-r01.frame.bin`: git classifies them binary on its LONE-CR branch, since an
# HL7 v2 segment terminator is CR with no LF. They hold ZERO NUL bytes, so `has_nul`
# keeps them IN scope and they are scanned as text. Proved rather than assumed, the way
# ncpdp proved its `.xml` fixtures are read: each of the three was seeded with a live em
# dash in turn and the gate went RED on each, naming the file.
# * ZERO tracked gitlinks (no mode 160000 entry) and ZERO tracked symlinks.
# * Every tracked file except that tarball decodes as UTF-8.
# * Every tracked file decodes as UTF-8 (the one exception was that tarball).
#
# The fix is never to re-encode the character: rewrite the sentence with a
# period, a colon, a comma, or parentheses.
Expand All @@ -62,19 +64,25 @@
# stderr capture instead of being skipped. That is fail-closed and it is the better shape
# where it applies.
#
# It does not apply here, and the reason is DURABILITY, not a present-day red. Measured:
# `vendor/cosyte-hl7-0.0.0.tgz` does NOT currently contain the byte sequence `E2 80 94`,
# so a knowledgebase-shape script would pass over it today, and dicom's copy was run
# against this exact tree to confirm that rather than to assume it (it printed OK, exit 0).
# But that tarball is a compressed byte stream, and a compressed stream contains any given
# three-byte sequence by coincidence with real probability. The tarball is re-vendored by
# hand whenever `@cosyte/hl7` is rebuilt (see this repo's CLAUDE.md), and the next re-vendor
# could land a copy that happens to contain those bytes. Measured, not argued: appending
# those three bytes to the real tarball and re-running dicom's copy makes it exit 1 with
# `binary file matches` against `vendor/cosyte-hl7-0.0.0.tgz`. That red has NO REMEDIATION
# AVAILABLE: you cannot rewrite a compressed byte stream with a period, and the tarball's
# contents are not prose anyone wrote. A gate whose red state has no defined fix is a gate
# someone disables, which is strictly worse than the hole it closed.
# It does not apply here, and the reason is DURABILITY, not a present-day red. The argument was
# made against a specimen that has since left the repo, and it OUTLIVES it; read to the end
# before concluding the shape can now be relaxed.
#
# Measured at the time: `vendor/cosyte-hl7-0.0.0.tgz` did NOT contain the byte sequence
# `E2 80 94`, so a knowledgebase-shape script would have passed over it, and dicom's copy was
# run against that exact tree to confirm it rather than assume it (it printed OK, exit 0).
# But that tarball was a compressed byte stream, and a compressed stream contains any given
# three-byte sequence by coincidence with real probability. It was re-vendored by hand
# whenever `@cosyte/hl7` was rebuilt, and the next re-vendor could have landed a copy holding
# those bytes. Measured, not argued: appending those three bytes to the real tarball and
# re-running dicom's copy made it exit 1 with `binary file matches` against the tarball. That
# red has NO REMEDIATION AVAILABLE: you cannot rewrite a compressed byte stream with a period,
# and the tarball's contents are not prose anyone wrote. A gate whose red state has no defined
# fix is a gate someone disables, which is strictly worse than the hole it closed.
#
# THE TARBALL IS GONE (ADR 0032) AND THIS SHAPE STAYS. The property that forced it, an
# unremediable red on bytes nobody authored, belongs to tracked binaries as a class, not to
# that one file. Revisit the PARTITION if you must, never the ban.
#
# Do not read website's stated reason across to here. website regenerates 7 PNGs on every
# PR, so it quoted a per-PR probability. mllp has exactly one binary, it is static, and it
Expand Down Expand Up @@ -105,13 +113,14 @@
# THE COST, stated rather than left implicit, because it is a real hole in a ban whose own
# wording has no exceptions: a tracked TEXT file that happens to hold a NUL byte (a UTF-16
# document, a fixture carrying framing bytes) is excluded here and its em dash would be
# missed. Say what that means for the tarball without softening it: seed the real
# `vendor/cosyte-hl7-0.0.0.tgz` with a live em dash and THIS GATE PRINTS OK AND EXITS 0.
# That is a MISS, not a pass. It is the deliberate price of the durability argument above,
# and it was checked rather than reasoned, along with the control that proves the NUL rule
# is what causes it: the same tarball bytes with every NUL replaced, carrying the same em
# dash, goes RED. mllp has NO tracked text file with a NUL today, so the exclusion currently
# exempts exactly one file and that file is a genuine binary.
# missed. It was said for the tarball without softening, and the measurement stands as the
# demonstration even though the file is gone: seeding the real `vendor/cosyte-hl7-0.0.0.tgz`
# with a live em dash made THIS GATE PRINT OK AND EXIT 0. That is a MISS, not a pass. It is
# the deliberate price of the durability argument above, and it was checked rather than
# reasoned, along with the control that proves the NUL rule is what causes it: the same
# tarball bytes with every NUL replaced, carrying the same em dash, went RED.
# mllp has NO tracked file with a NUL today, so the exclusion currently exempts nothing.
# Read the count off the OK line rather than from here.
#
# Do NOT round that off to "the hole is hypothetical here", which an earlier draft of this
# header did. The at-risk fixture class ALREADY EXISTS: the three `.frame.bin` captures under
Expand Down
9 changes: 5 additions & 4 deletions scripts/check-no-internal-refs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -136,8 +136,9 @@
# the audit log for fixture-level PHI-scan bypasses. Internal
# compliance bookkeeping, not consumer documentation, and not in
# package.json `files`.
# * CLAUDE.md, .github/, .changeset/, scripts/, test/, vendor/
# internal by definition, or code rather than prose.
# * CLAUDE.md, .github/, .changeset/, scripts/, test/
# internal by definition, or code rather than prose. (`vendor/` was
# in this list until it was deleted with the hl7 tarball.)
# * src/ DOC COMMENTS IN SCOPE, as a THIRD PASS at the bottom of this file, with its
# own rule array (SRC_RULE_PATTERN), its own self-tests, and its
# own extractor. `src/` JSDoc IS public: it is compiled into the
Expand Down Expand Up @@ -1047,8 +1048,8 @@ fi
# heuristic calls binary, which includes a genuine TEXT file with a broken encoding, so a
# violation inside one would be skipped in silence. This gate's surface is markdown, JSON and
# TypeScript with no binaries (CHECKED ON THIS TREE: no tracked file under any scanned path
# holds a NUL byte; the repo's one vendored tarball lives under `vendor/`, which this gate
# does not scan), so losing `-I` makes a future binary a loud red instead of a silent miss.
# holds a NUL byte, and since the hl7 tarball was deleted no tracked file anywhere in the repo
# does), so losing `-I` makes a future binary a loud red instead of a silent miss.
# Fail closed, not open. `-H` is set so every hit carries its filename: grep omits the name
# when handed exactly one file, which an xargs batch boundary can produce.
: > "$SCANLIST"
Expand Down
Loading
Loading