Skip to content

fix(libsy): stop reading source text as a runtime failure - #568

Draft
ryan-lempka wants to merge 4 commits into
mainfrom
fix/tool-signal-false-positives
Draft

fix(libsy): stop reading source text as a runtime failure#568
ryan-lempka wants to merge 4 commits into
mainfrom
fix/tool-signal-false-positives

Conversation

@ryan-lempka

@ryan-lempka ryan-lempka commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator

Background: Stage routing scores severity by pattern-matching the text a tool returns. Three matches fire on text that is not an error. Severity forces the capable tier ahead of every other rule, and a failure literal vetoes test-pass detection (e.g. exit code 0), so both block de-escalation.

  • Reading a file containing "out of memory" scored as a critical failure.
  • A streaming chunk echoing a diff scored on the diff, so writing a test fixture with "Traceback" looked like a crash.
  • libsy has a module named error, so passing tests print test error::tests::.... Any repo with an error module cannot detect a green cargo test.

Both failures are silent: routing returns 200 and the cheap tier stops being used.

What: Both in crates/libsy/src/algorithms/util/tool_signals.rs.

  • New reports_no_failure skips severity scoring when a tool result reports exit 0, or the command is still running.
  • New contains_failure_literal stops error: matching the path error::.

Why: Status code should be trusted over text matching and we should not treat messages from source text reading as a runtime error.

Signed-off-by: Ryan Lempka <rlempka@nvidia.com>
@github-actions

github-actions Bot commented Aug 27, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1

🚀 View preview at
https://NVIDIA-NeMo.github.io/Switchyard/pr-preview/pr-568/

Built to branch gh-pages at 2026-08-27 18:37 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

…detection

Signed-off-by: Ryan Lempka <rlempka@nvidia.com>
Signed-off-by: Ryan Lempka <rlempka@nvidia.com>
Signed-off-by: Ryan Lempka <rlempka@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant