Skip to content

Record that 2-BM's start_date is the previous scan's end_date #653

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
xmap merged 1 commit into from
Aug 12, 2026
Merged

Record that 2-BM's start_date is the previous scan's end_date #653

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 33 additions & 7 deletions docs/deployments/2-bm/operations.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,11 +80,31 @@ dataset paths and lifecycle timing this section assumes. Two of its facts matter
commanded scan geometry (`/process/acquisition/rotation/num_angles`, the flat and dark field
mode-and-count groups) is written `OnFileClose`, so it exists only in cleanly closed files, and it is
what a shortfall check compares captured frames against; the `/defaults` frame ids alone can never show
tail truncation, because a missed trigger never receives an id. The acquisition timestamp
(`/process/acquisition/start_date`, from the PV `S:IOC:timeOfDayISO8601`) is written `OnFileOpen`, so a
crashed file keeps its timestamp while losing its geometry. That PV carries an explicit UTC offset
(confirmed 2026-08-11, DATA-10: `2026-08-11T12:01:05-0500`), not naive local time, so a reader parses it
as an unambiguous instant with no site timezone rule needed.
tail truncation, because a missed trigger never receives an id. The two acquisition timestamps
(`/process/acquisition/start_date` and `end_date`, both from the PV `S:IOC:timeOfDayISO8601`) are written
`OnFileOpen` and `OnFileClose`, so a crashed file keeps a start time while losing its geometry. The PV
carries an explicit UTC offset (confirmed 2026-08-11, DATA-10: `2026-08-11T12:01:05-0500`), not naive
local time, so a reader parses either as an unambiguous instant with no site timezone rule needed.

**`start_date` does not say when the scan started, and `end_date` does say when it ended.** Measured
2026-08-12 across the six files in `2026-08-DeCarlo-1015116`: every file's `end_date` falls within five
seconds of its own close, and in three consecutive cases the `start_date` is the PREVIOUS file's
`end_date` to the second (`test_003` carries `test_002`'s, `test_004` carries `test_003`'s, `test_005`
carries `test_004`'s). `test_005` ran on the morning of 2026-08-12 and claims a start on the evening of
2026-08-11, wrong by about twelve hours and across a day boundary. Two early smoke tests carry a value
five days older than the file. One file, `test_002`, does look correct, which is part of why the defect
is easy to miss.

The clock PV is healthy: read live on 2026-08-12 it returned the correct instant. The reading consistent
with the evidence is that the areaDetector timestamp attribute refreshes only while frames are flowing,
so a file opened before any frame of the new scan inherits the value the previous scan left, while the
value written on close has been refreshed by that scan's own frames. That mechanism is inferred from the
measurements rather than read out of the IOC configuration, and staff are the authority on it.

Two consequences. A reader wanting when a scan happened uses `end_date` here, which is why the descriptor
declares `captured_at_source: end_date` and CORA records which timestamp it used rather than leaving a
reader to assume. And the defect is upstream of CORA: it is in every scan file 2-BM writes, so anything
else reading `start_date` inherits it silently.

**A finished capture is not a finished file, and this is the fact an ingest reader must respect.** The
end-of-scan sequence stops the file plugin (`FPCapture` to `Done`, then waits for `Capture_RBV` to reach
Expand All @@ -102,8 +122,14 @@ naming the missing ones when they disagree. Dropped frames are therefore a known
non-fatal condition: a reader that records only what landed will silently under-describe such a scan.
A genuine instance was found 2026-08-11 reading a real file from the beamline (`test_000.h5`, an early
smoke test rather than a production scan): 3601 angles commanded, one frame captured, `theta` absent.
`DataExchangeScanReader` handled it exactly as designed; DATA-8's frequency question (rare-and-alarming
versus routine for an actual production scan) is still open.
`DataExchangeScanReader` handled it exactly as designed.

The first production scan CORA read end to end (`test_005.h5`, 2026-08-12) dropped nothing: 1501
commanded, 1501 captured, 40 flats, 20 darks, angles 0 to 180.013. That is one scan, so it does not
settle DATA-8's frequency question, but it is the first data point and it comes from the record rather
than from a log line. Until 2026-08-12 the question could not be asked at all: the reader could not read
the commanded counts, because 2-BM writes them as one-element arrays and the reader only understood
plain scalars, so every shortfall check compared against nothing.

The experiment folder is computed, not conventional, which is what makes it derivable rather than
guessable. `dmagic`'s `dm.py` formats it as `{year_month}-{pi_last_name}-{gup_number}` from APS
Expand Down
12 changes: 8 additions & 4 deletions docs/deployments/2-bm/questions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,8 @@ answer is left here.

| ID | Priority | Question | CORA assumes | Already done? | Resolves |
| --- | --- | --- | --- | --- | --- |
| DATA-8 | `Nice-to-have` | How often do scans finish with dropped frames? `add_theta()` compares written frames against commanded angles and logs a warning when they disagree, so the condition is detected but not fatal. Knowing whether this is rare-and-alarming or routine decides whether a record of the scan should refuse to be written, or carry the shortfall as an ordinary recorded fact. | rare enough to treat as an exception worth surfacing, not a routine outcome to normalise | not yet (a genuine instance was found 2026-08-11 reading a live 2bmb test file, `test_000.h5`: 3601 angles commanded, 1 frame captured, `theta` absent; `DataExchangeScanReader` handled it exactly as designed, but the file was an early smoke test, not evidence about routine production frequency) | [Operations](operations.md) |
| DATA-8 | `Nice-to-have` | How often do scans finish with dropped frames? `add_theta()` compares written frames against commanded angles and logs a warning when they disagree, so the condition is detected but not fatal. Knowing whether this is rare-and-alarming or routine decides whether a record of the scan should refuse to be written, or carry the shortfall as an ordinary recorded fact. | rare enough to treat as an exception worth surfacing, not a routine outcome to normalise | not yet, but the question is now askable: CORA could not read the commanded counts at all until 2026-08-12 (2-BM writes them as one-element arrays and the reader understood only plain scalars), so every shortfall check silently compared against nothing. Two data points since: `test_000.h5`, an early smoke test, 3601 commanded and 1 captured with `theta` absent; and `test_005.h5`, the first production scan CORA read end to end, 1501 commanded and 1501 captured, nothing dropped | [Operations](operations.md) |
| DATA-11 | `Blocks-go-live` | Can the detector IOC's timestamp attribute be made to refresh when a file opens? `start_date` is measurably the PREVIOUS scan's `end_date` (see [Operations](operations.md#inside-the-scan-file) for the six-file evidence), so every scan file 2-BM writes carries a wrong acquisition start, silently, for every consumer and not only CORA. CORA reads `end_date` here and is unblocked; the question is whether the file contract itself can be repaired at the source, and whether the inferred mechanism matches what the IOC configuration actually does. | the attribute can be refreshed at file open, making `start_date` true rather than inherited | not yet | [Operations](operations.md#inside-the-scan-file) |

## Where CORA runs

Expand All @@ -140,9 +141,12 @@ interim choice for testing and development while nothing structured is decided y
move later. Two consequences follow. `arcturus`'s own `EPICS_CA_ADDR_LIST` (an explicit address list,
not broadcast) is simply what CORA now uses, which is what the former HOST-1 asked. And read access to a
finished scan file has a working interim answer that needs no mount and never touches the beamline's
write paths: read the file in place over SSH to `tomdet`, the same way `test_003.h5` was inspected on
2026-08-11 (see [Operations](operations.md#inside-the-scan-file)). The durable version of that question,
a mount or a supported fetch path, stays open below.
write paths: copy the finished file from `tomdet` to a staging directory on the host and read it there.
That was exercised end to end on 2026-08-12, staging `test_005.h5` (24.5 GB) to `/local/cora-scans` on
`arcturus` and ingesting it. Two costs the durable answer should weigh: the copy duplicates the file, and
digesting it takes real time (`sha256sum` alone is 82 seconds on this hardware, about 300 MB/s), which is
why the deployment raises the digest walk budget rather than accepting the 60-second default. The durable
version of the question, a mount or a supported fetch path, stays open below.

These are likely controls, networking, or IT questions rather than floor questions. Routing them to the
right person, or naming who that person is, is a complete answer to any row here.
Expand Down
Loading