feat: local case history and comparison - #35
Merged
Conversation
An operator can revisit a previous case and see what changed between two cases without browsing output directories or diffing large JSON. - `aas cases` lists bounded summaries (outcome, policy reference, review verdict, evidence digest, component revisions) and `aas compare <a> <b>` classifies a pair as identical, different, or not comparable, listing the fields that differ. Summaries never include raw evidence, and comparison states that differences do not establish causation and that matching metadata does not prove matching evidence. - GUI mirrors this with `Load history` and `Compare selected cases`, including loading, error, and not-comparable states, and per-selection tokens so switching cases mid-load cannot show another pair's result. - Run ids are validated as real run ids: "." and ".." matched the old pattern and resolved outside the runs directory. Filesystem errors are reported without absolute paths. An unreadable stage artifact now fails closed as not-comparable instead of silently reading as absent, and `stages` is compared. Comparison exits nonzero when not comparable. - History scans newest-first before reading manifests, so a large store cannot make the scan or the summaries unbounded.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
There was a problem hiding this comment.
Devin Review found 2 potential issues.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
Comment on lines
+1695
to
+1698
| function printComparison(result, asJson) { | ||
| if (asJson) { | ||
| process.stdout.write(`${JSON.stringify({ ok: true, ...result }, null, 2)}\n`); | ||
| return; |
There was a problem hiding this comment.
🟡 JSON comparisons hide unavailable cases
With --json, printComparison returns before assigning the failure status for a not-comparable pair. Automation therefore accepts missing, unreadable, or unsupported cases as successful comparisons.
Suggested change
| function printComparison(result, asJson) { | |
| if (asJson) { | |
| process.stdout.write(`${JSON.stringify({ ok: true, ...result }, null, 2)}\n`); | |
| return; | |
| function printComparison(result, asJson) { | |
| if (asJson) { | |
| process.stdout.write(`${JSON.stringify({ ok: true, ...result }, null, 2)}\n`); | |
| process.exitCode = result.classification === "not-comparable" ? 1 : 0; | |
| return; |
Was this helpful? React with 👍 or 👎 to provide feedback.
| const response=await fetch('/api/compare?a='+encodeURIComponent(leftCase.value)+'&b='+encodeURIComponent(rightCase.value)); | ||
| body=await response.json(); | ||
| } catch(error){ if(token!==compareToken) return; compareStatus.textContent='Comparison failed.'; compareButton.disabled=false; return; } | ||
| if(token!==compareToken) return; |
There was a problem hiding this comment.
🟡 History refresh freezes comparison button
When history refresh starts during compareButton's request, the stale-response guard returns without re-enabling the button. The interface remains stuck on “Comparing...” and blocks every later comparison.
Prompt for agents
The GUI uses one compareToken for both history refreshes and comparison requests. Starting refreshHistory while a comparison fetch is pending invalidates the comparison token. Its stale-response return at bin/aas-gui.mjs leaves compareButton disabled and compareStatus at “Comparing...”. The reverse overlap can also leave historyList at “Loading history...”. Separate operation tokens or ensure each stale path restores only the UI state owned by that operation without allowing stale results to overwrite newer output.
Was this helpful? React with 👍 or 👎 to provide feedback.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Capability (approved milestone 3)
Revisit a previous case and understand what changed between two cases
without browsing output directories or diffing large JSON.
aas cases: bounded, summary-only history (outcome, policy reference,review verdict, evidence digest, component revisions).
aas compare <a> <b>: classifies identical / different / not comparableand lists the fields that differ.
Load history+Compare selected caseswith loading, error, andnot-comparable states; selection tokens prevent cross-pair results.
Semantics are explicit: no causation from differences, matching metadata is
not proof of matching evidence, and only the listed compared fields are
checked. Raw evidence never enters summary views; nothing mutates,
overwrites, or deletes a case, and cleanup stays the explicit
aas prune --keep.Fixes found during review
.and..satisfied the run-id pattern and resolved outside the runsdirectory (confirmed by a canary read over HTTP and CLI); run ids must now
look like real run ids, validated identically on both entry points.
null, so a corruptedprove artifact made two different cases read as identical. Now tracked and
forced to not-comparable.
stageswas not compared, so prove skipped vs prove failed read asidentical; now compared, and the CLI prints both caveats (previously it
printed only the causation one, contradicting the README).
comparable and reports filesystem errors without paths.
Verification
npm test: 101/101, including traversal rejection (./../...),unreadable-artifact fail-closed, distinct-run stage differences, bounded
history that skips unreadable cases, CLI exit codes, and endpoint
validation with escaping checks.
npm run test:browser: 9/9 real browser journeys, including history loadand UI comparison.
cases/compareagainst pinned components.