Say "no timestamp" instead of inventing 1990-01-01 - #631
Merged
Conversation
EPICS separates a value that is fine while the process it describes is in alarm (MINOR, MAJOR) from a value that cannot be trusted at all (INVALID). All three EPICS adapters collapsed MAJOR onto Bad, which threw away a reading the IOC and the floor both consider authoritative. At 2-BM the everyday case is an interlocked hutch left secured with a designed MAJOR STATE alarm on SecureM: the position is known, the alarm is the design, and CORA was reading it as a value it could not vouch for. Mapping MAJOR to Uncertain restores the standard semantics and puts the choice back where it belongs, at each consumer. No behaviour moves with this commit. Every consumer today tests quality == "Good", so Uncertain fails exactly where Bad failed; the only visible change is the reason string on a halted check. Consumers that mean "can I believe this value" rather than "can I act on this value" opt in to != "Bad" one at a time, on their own merits. The port docstring claimed MINOR collapsed to Bad as well, which no adapter has done; it now states the mapping the code implements. The softIOC fixture gains a second alarm record, because MAJOR and INVALID now land on opposite sides of the trichotomy and one record cannot pin both arms. A unit-tier table pins all three adapters together so a drift between them fails in the fast lane rather than waiting for a live IOC. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
EPICS starts counting at 1990-01-01, so a record that never processed carries stamp zero, and `datetime.fromtimestamp` turned that into a date that parses, sorts and filters exactly like a real reading. Nothing marked it synthetic, and the port docstring positively asserted the opposite. `Measurement.produced_at` is now `datetime | None`, and each adapter decides absence for itself because every substrate spells it differently. Measured at APS 2-BM through CORA's own adapter on the subscribe path: both PSS permit signals report an undefined stamp on every update, so this was the everyday case there rather than an edge. S02BM-PSS:StaA:SecureM 'ON' produced_at = 1990-01-01T00:00:00Z S02BM-PSS:StaB:SecureM 'OFF' produced_at = 1990-01-01T00:00:00Z Not a latent problem either: the conductor already persisted the field into run records twice, so the sentinel was reaching stored history. The enclosure seam is the one place it was dropped, which is why nobody noticed. The boundary is each substrate's own epoch, since no reading can predate the clock that stamps it. EPICS adapters cut at 1990, Tango at 1970. This catches a MISSING time, never a WRONG one: an IOC whose clock is set to the wrong year still reports a plausible stamp that no adapter can detect. caproto additionally substituted `datetime.now()` when metadata carried no stamp. That made an ingest time indistinguishable from a substrate time once written down, which is the confusion this change exists to remove; it is gone and pinned so it cannot return. Two docstrings claimed the enclosure handler preserved the substrate's time and two claimed it used the Clock. The code did the latter and the seam drops the field entirely. Both now say so, and Supply's said its time was "captured on the SUBSCRIPTION side" when Supply has no observer port at all. Strengthening the existing assertions from "tzinfo is set" to "a stamp is present" exposed a separate bug in the softIOC fixture: its NTNDArray Q:group aimed `+type:"meta"` at named subfields instead of the structure root, nesting alarm and timeStamp one level too deep for p4p's unwrap. Every image read had been reporting NO_ALARM and 1970-01-01 regardless of the record. The alarm half passed unnoticed because the default matched the expected value. An assertion weak enough to pass on a synthesized default pins nothing. Threading the substrate time through the command, event payload and projection is deliberately NOT here; it is the next slice. Verified: no-IO lane 46581 passed (parent 46567 plus the 14 new unit tests, exactly), architecture 30313 passed against the final tree, pyright clean, and live-softIOC integration green on all three wire protocols (34 CA + caproto, 18 PVA). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ible-timestamps # Conflicts: # apps/api/tests/integration/_softioc.py # apps/api/tests/integration/test_caproto_control_port.py # apps/api/tests/integration/test_epics_ca_control_port.py # apps/api/tests/integration/test_epics_pva_control_port.py
Coverage report
Click to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
This was referenced Aug 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #629. Merge that first, then retarget this to
mainso CI runs(
ci.ymlonly triggers on PRs targeting main, so a stacked PR gets no checks).EPICS starts counting at 1990-01-01, so a record that never processed carries
stamp zero, and
datetime.fromtimestampturned that into a date that parses,sorts and filters exactly like a real reading.
Measurement.produced_atis nowdatetime | None, and each adapter decides absence for itself because everysubstrate spells it differently.
Measured, not inferred
Through CORA's own adapter on the subscribe path, against the live 2-BM IOC:
Every update, both signals. The everyday case there, not an edge.
This was also not latent:
conductor.pyalready persisted the field into runrecords in two places, so the sentinel was reaching stored history. The
enclosure seam is the one place it was dropped, which is why it went unnoticed
(2-BM-A's recorded observation looked sane because it was stamped from CORA's
clock, not the IOC's).
The rule
Each substrate's own epoch is the boundary, since no reading can predate the
clock that stamps it. EPICS adapters cut at 1990, Tango at 1970.
This catches a MISSING time, never a WRONG one. An IOC whose clock is set to
the wrong year reports a plausible stamp that no adapter can detect, and the
docstrings say so rather than implying broader coverage.
Two things found along the way
caprotosubstituteddatetime.now()when metadata carried no stamp, whichmade an ingest time indistinguishable from a substrate time once written down.
That is the exact fallback this design rejects. Removed and pinned.
Strengthening the existing assertions from "tzinfo is set" to "a stamp is
present" exposed a bug in the softIOC fixture: its NTNDArray
Q:groupaimed+type:"meta"at named subfields instead of the structure root, nesting alarmand timeStamp one level too deep for p4p's unwrap. Every image read had been
reporting NO_ALARM and 1970-01-01 regardless of the record. The alarm half
passed unnoticed because the default matched the expected value. An assertion
weak enough to pass on a synthesized default pins nothing.
Not in scope
Threading the substrate time through the command, event payload and projection
is the next slice, and it carries the schema change. This PR only makes absence
representable and stops the invented dates. 2-BM's permit records are unchanged
by it.
Verification
🤖 Generated with Claude Code