From 9a3c4cbd49afb7f70724a5a679cffc3a42cda256 Mon Sep 17 00:00:00 2001 From: Oleksii Dolhov Date: Mon, 3 Aug 2026 15:17:30 +0300 Subject: [PATCH 1/4] ci: give the live-instance integration suite a nightly CI home (#1896) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ~93 of the test files in `tests/` root need a RUNNING Trinity — the fixtures create and delete real agents against `TRINITY_API_URL` — and no CI job ran them, so they gated nothing. `integration-nightly.yml` boots a stack per open PR and runs the suite on both sides, reusing the pieces that already exist rather than inventing any: `scripts/deploy/start.sh` for the stack (as frontend-e2e.yml does), the discover -> test -> comment job split and sticky-comment machinery from backend-unit-nightly.yml, and `diff-pytest-failures.py` for base-vs-head attribution. NOT BLOCKED ON #1895, despite that issue's sequencing note. Membership is directory-based — `--ignore=unit --ignore=process_engine`, exactly as tests/run-core.sh:22 already defines the same suite — so every file #1895 relocates leaves this job's scope automatically and the workflow needs no edit. There is no "~60 files" target to chase and no list to drift. Nightly rather than per-PR (AC 2): the suite MUTATES its target and needs ~3-5 min on top of a ~2-3 min boot, twice. In the merge path that is ~10 min on every backend PR plus a mutating flake surface in front of every merge. Three things measured rather than assumed, each of which would have shipped this broken: 1. `--continue-on-collection-errors` is load-bearing. Three root modules currently fail to IMPORT (two git-sync files wanting `utils.credential_sanitizer`, and tests/integration's conftest). Without the flag pytest aborts the session with "Interrupted: 3 errors during collection" and writes NO JUnit XML — the job would have failed every night for every PR while reporting nothing. With it the errors land on both sides, cancel in the diff, and a NEW collection error still surfaces. 2. `src/cli` must be pip-installed. `test_cli_*.py` imports `trinity_cli`, which is deliberately kept out of requirements-test.txt (an `-e ./src/cli` line there breaks GitHub's dependency-graph updater). tests/setup-env.sh installs it on demand; CI has to do the same or those modules silently never run — the very no-coverage this issue is about. 3. A missing JUnit must fail the job LOUDLY. `diff-pytest-failures.py` is fail-closed on a missing input, so a stack that never booted would have been reported to the author as "this PR introduced regressions" — an infrastructure failure wearing a PR's name, which is #1941's exact class. Also inherited from #1941: full-depth fetch (two shallow sides share no ancestor and every PR reads as conflicting) and a conflict verdict gated on actual unmerged paths. Teardown (AC 4): `docker compose down -v` between the two sides so head never inherits base's agents, volumes or DB, and the runner VM is destroyed afterwards regardless — the instance is disposable by construction. Runtime (AC 5): 45-minute ceiling, expected ~15-20. The escalation if it is hit is written into the workflow header — shard by marker or reduce the PR limit BEFORE raising the ceiling. `tests/run-core.sh` is untouched and remains the supported local path (AC 6); a guard test asserts the two keep defining the same suite. Related to #1896 Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/integration-nightly.yml | 399 ++++++++++++++++++ tests/registry.json | 89 ++-- .../test_1896_integration_nightly_workflow.py | 200 +++++++++ 3 files changed, 649 insertions(+), 39 deletions(-) create mode 100644 .github/workflows/integration-nightly.yml create mode 100644 tests/unit/test_1896_integration_nightly_workflow.py diff --git a/.github/workflows/integration-nightly.yml b/.github/workflows/integration-nightly.yml new file mode 100644 index 000000000..6c4d0a773 --- /dev/null +++ b/.github/workflows/integration-nightly.yml @@ -0,0 +1,399 @@ +name: integration-nightly + +# Nightly live-instance integration sweep over open PRs targeting `dev` (#1896). +# +# ~93 of the test files in `tests/` root need a RUNNING Trinity: the fixtures +# create and delete real agents against `TRINITY_API_URL`. No CI job ran them, +# so they gated nothing — the sibling of #1895 (which relocates the files that +# can pass under `tests/unit/`). +# +# Membership is directory-based, not a file list: `--ignore=unit +# --ignore=process_engine`, exactly as `tests/run-core.sh:22` already defines +# it. So every file #1895 relocates leaves this job's scope automatically and +# this workflow needs no edit — which is also why this did not have to wait for +# #1895 to land, despite the issue's sequencing note. +# +# Architecture is inherited verbatim from backend-unit-nightly.yml, because the +# same rule applies (Codex finding 5 — untrusted PR code must never share a job +# with a write-capable token): +# +# discover (RO, lists open PRs) +# └─> test (RO, no creds, boots a stack, runs untrusted PR code) +# └─> comment (write, never checks out PR code) +# +# Artifacts (diff Markdown + status JSON) are how `test` hands work to +# `comment` without ever sharing a write token with the PR code. +# +# WHY NIGHTLY rather than per-PR (#1896 AC 2): the suite MUTATES its target — +# it creates and deletes real agents — and needs ~3-5 min of runtime on top of +# a ~2-3 min stack boot, twice (base and head). Putting that in the merge path +# would add ~10 min to every backend PR and park a mutating flake surface in +# front of every merge. Nightly buys the base-vs-head attribution the ACs +# require for free, because that tooling already exists for this exact shape. +# +# RUNTIME BUDGET (#1896 AC 5): `timeout-minutes: 45` per PR, expected ~15-20 +# (boot + suite, twice). If a PR routinely hits the ceiling the documented +# escalation is, in order: (1) drop to `-m "smoke"` for the head run, (2) shard +# the suite by marker across two jobs, (3) reduce `discover`'s PR limit. Do NOT +# raise the timeout without doing one of those first — a 45-minute mutating job +# that nobody reads is the failure mode this issue exists to fix. + +on: + schedule: + - cron: '30 6 * * *' # 06:30 UTC — after backend-unit-nightly (06:00) so + # the two nightlies do not contend for runners + workflow_dispatch: + inputs: + pr_number: + description: 'Run for a single PR number (blank = all open PRs)' + required: false + type: string + +# No top-level permissions — each job declares its own narrow scope. +permissions: {} + +concurrency: + group: integration-nightly + cancel-in-progress: false + +jobs: + discover: + name: discover open PRs + runs-on: ubuntu-latest + timeout-minutes: 5 + permissions: + contents: read + pull-requests: read + outputs: + matrix: ${{ steps.list.outputs.matrix }} + has_prs: ${{ steps.list.outputs.has_prs }} + steps: + - id: list + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + SINGLE_PR: ${{ inputs.pr_number }} + run: | + set -euo pipefail + if [ -n "${SINGLE_PR:-}" ]; then + prs=$(gh pr view "$SINGLE_PR" \ + --repo "${{ github.repository }}" \ + --json number,headRefName,headRefOid | jq '[.]') + else + # Lower limit than the unit nightly (50): each entry here boots two + # full stacks, so the fleet cost is an order of magnitude higher. + prs=$(gh pr list \ + --repo "${{ github.repository }}" \ + --base dev \ + --state open \ + --limit 20 \ + --json number,headRefName,headRefOid) + fi + count=$(echo "$prs" | jq 'length') + if [ "$count" = "0" ]; then + echo "has_prs=false" >> "$GITHUB_OUTPUT" + echo "matrix={\"include\":[]}" >> "$GITHUB_OUTPUT" + exit 0 + fi + echo "has_prs=true" >> "$GITHUB_OUTPUT" + matrix=$(echo "$prs" | jq -c '{include: [.[] | {pr_number: .number, head_sha: .headRefOid}]}') + echo "matrix=$matrix" >> "$GITHUB_OUTPUT" + + test: + name: integration (PR #${{ matrix.pr_number }}) + needs: discover + if: needs.discover.outputs.has_prs == 'true' + runs-on: ubuntu-latest + # See RUNTIME BUDGET above before changing this. + timeout-minutes: 45 + permissions: + contents: read # deliberately no write scope — this job runs PR code + strategy: + fail-fast: false + # Each entry boots two stacks; keep the fleet from saturating the runners. + max-parallel: 3 + matrix: ${{ fromJSON(needs.discover.outputs.matrix) }} + + steps: + - uses: actions/checkout@v7 + with: + fetch-depth: 0 # #1941: the merge below needs a common ancestor + persist-credentials: false + + - name: Stash trusted diff script (pre-merge) + # Workspace is unmodified `dev` at this point. Copy the diff utility to + # a stable path BEFORE the merge, so a malicious PR editing + # scripts/ci/diff-pytest-failures.py cannot make the gate green. + run: | + mkdir -p "$HOME/.ci-tools" + cp scripts/ci/diff-pytest-failures.py "$HOME/.ci-tools/diff-pytest-failures.py" + + - uses: actions/setup-python@v7 + with: + python-version: '3.11' + cache: 'pip' + cache-dependency-path: tests/requirements-test.txt + + - name: Install test dependencies + run: | + python -m pip install --upgrade pip + pip install -r tests/requirements-test.txt + # `test_cli_*.py` imports `trinity_cli`, which is deliberately NOT in + # requirements-test.txt (an `-e ./src/cli` line there breaks GitHub's + # dependency-graph updater — see tests/setup-env.sh). Install it the + # same on-demand way the local wrapper does, or those files fail to + # import and their tests silently never run. + pip install -e ./src/cli + + - name: Fetch PR head and attempt the merge + id: merge + run: | + set -euo pipefail + git config user.email "ci@trinity.local" + git config user.name "trinity-ci" + # #1941: no --depth. Both sides need enough history to reach the + # ancestor they share, or the merge fails as "unrelated histories" + # and every PR reads as conflicting. + git fetch origin "pull/${{ matrix.pr_number }}/head:pr-head" + git switch --detach origin/dev + git branch -f base-side HEAD + if git merge --no-edit --no-ff pr-head; then + echo "merge_conflict=false" >> "$GITHUB_OUTPUT" + git branch -f head-side HEAD + else + # #1941: only unmerged paths prove a real conflict; any other git + # failure is infrastructure and must not be reported as one. + if [ -n "$(git diff --name-only --diff-filter=U)" ]; then + echo "merge_conflict=true" >> "$GITHUB_OUTPUT" + git merge --abort || true + else + git merge --abort || true + echo "::error::git merge failed with no conflicted paths — infrastructure failure, not a PR conflict" + exit 1 + fi + fi + + - name: Generate CI admin password + if: steps.merge.outputs.merge_conflict == 'false' + # Mirrors frontend-e2e.yml: a per-run random value so the repo never + # publishes a default admin credential, even one only reachable from + # the runner sandbox. The `Aa1!` suffix guarantees the OWASP ASVS 2.1 + # complexity classes on the random branch. + run: | + set -euo pipefail + PASS="$(openssl rand -base64 24 | tr -d '/+=\n')Aa1!" + echo "::add-mask::$PASS" + echo "ADMIN_PASSWORD=$PASS" >> "$GITHUB_ENV" + + - name: Run the live suite on BOTH sides + if: steps.merge.outputs.merge_conflict == 'false' + env: + ANTHROPIC_API_KEY: ${{ secrets.E2E_ANTHROPIC_API_KEY || 'placeholder' }} + # `tests/conftest.py` aliases TRINITY_TEST_PASSWORD from + # ADMIN_PASSWORD, so the credential does not need restating. + TRINITY_API_URL: http://localhost:8000 + # Safe here and nowhere else: the sweep refuses a non-localhost + # target, and this instance is a throwaway (#1558). + TRINITY_TEST_CLEANUP_SWEEP: '1' + run: | + set -uo pipefail + + boot_stack() { + cp .env.example .env + echo "ADMIN_PASSWORD=$ADMIN_PASSWORD" >> .env + echo "ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY" >> .env + echo "SECRET_KEY=$(openssl rand -hex 32)" >> .env + # Zero user agents at baseline (learnings 2026-07-08) so the suite's + # own fixtures are the only agents on the instance. + echo "TRINITY_DEFAULT_SYSTEM_MANIFEST=disabled" >> .env + ./scripts/deploy/start.sh + for i in $(seq 1 90); do + if curl -fsS http://localhost:8000/health >/dev/null 2>&1; then + echo "backend healthy"; return 0 + fi + sleep 2 + done + echo "::error::backend never became healthy"; return 1 + } + + teardown_stack() { + # `-v` is the point: the suite mutates its instance, so the next + # side must not inherit agents, volumes or a dirty DB from the + # previous one. The runner VM is destroyed afterwards regardless, + # which is what makes the instance disposable (#1896 AC 4). + docker compose down -v --remove-orphans || true + } + + run_side() { + side="$1"; ref="$2" + git switch --detach "$ref" + if ! boot_stack; then + # Tear down whatever half-came-up, so the OTHER side still gets a + # clean instance instead of inheriting this one's wreckage. + teardown_stack + return 1 + fi + # Membership matches tests/run-core.sh:22 — directory-based, so + # #1895's relocations shrink this automatically. + # `-m "not slow"`: the 99 slow-marked tests are excluded to keep the + # budget above honest; they are a documented gap, not an oversight. + # `--continue-on-collection-errors` is load-bearing, not defensive + # tidiness: the root suite currently has 3 modules that fail to + # IMPORT (two git-sync files wanting `utils.credential_sanitizer`, + # and tests/integration's conftest). Without the flag pytest aborts + # the whole session with "Interrupted: N errors during collection" + # and writes NO JUnit XML at all — so this job would fail every + # night for every PR while reporting nothing useful. With it, the + # errors are recorded on BOTH sides, cancel out in the base-vs-head + # diff, and a NEW collection error introduced by a PR still shows up + # as a new failure. Those 3 are pre-existing and belong to the + # #1895/#1896 cleanup, not to whichever PR runs first. + ( cd tests && python -m pytest -m "not slow" \ + --ignore=unit --ignore=process_engine \ + --continue-on-collection-errors \ + --junit-xml="$GITHUB_WORKSPACE/junit-${side}-pr${{ matrix.pr_number }}.xml" \ + --tb=no -q ) || true + teardown_stack + } + + # Base first, then head, each against a PRISTINE stack. + run_side base "$(git rev-parse base-side)" + run_side head "$(git rev-parse head-side)" + + # #1941's lesson, applied before it can bite: the diff script is + # fail-closed on a missing input XML, so a stack that never booted + # would surface as "this PR introduced regressions" — an + # infrastructure failure wearing a PR's name. Fail the job loudly + # instead; the `comment` step then posts nothing for this PR rather + # than something false. + for side in base head; do + xml="$GITHUB_WORKSPACE/junit-${side}-pr${{ matrix.pr_number }}.xml" + if [ ! -s "$xml" ]; then + echo "::error::no JUnit XML for the $side side — the stack never ran the suite. Not reporting a regression." + exit 1 + fi + done + + - name: Run regression diff (trusted copy) + if: steps.merge.outputs.merge_conflict == 'false' + id: diff + run: | + set +e + python "$HOME/.ci-tools/diff-pytest-failures.py" \ + --base junit-base-pr${{ matrix.pr_number }}.xml \ + --head junit-head-pr${{ matrix.pr_number }}.xml \ + --out "diff-pr${{ matrix.pr_number }}.md" + rc=$? + if [ $rc -eq 0 ]; then + echo "regression=false" >> "$GITHUB_OUTPUT" + else + echo "regression=true" >> "$GITHUB_OUTPUT" + fi + + - name: Collect stack logs on failure + if: failure() + run: docker compose logs --no-color --tail=300 > trinity-logs-pr${{ matrix.pr_number }}.txt || true + + - name: Write status JSON + if: always() + run: | + merge_conflict='${{ steps.merge.outputs.merge_conflict }}' + regression='${{ steps.diff.outputs.regression }}' + if [ -z "$regression" ]; then regression="false"; fi + jq -n \ + --arg pr "${{ matrix.pr_number }}" \ + --arg head_sha "${{ matrix.head_sha }}" \ + --arg merge_conflict "$merge_conflict" \ + --arg regression "$regression" \ + '{pr_number: ($pr|tonumber), head_sha: $head_sha, + merge_conflict: ($merge_conflict == "true"), + regression: ($regression == "true")}' \ + > "status-pr${{ matrix.pr_number }}.json" + + - name: Upload artifacts + if: always() + uses: actions/upload-artifact@v7 + with: + name: integration-pr-${{ matrix.pr_number }} + path: | + diff-pr${{ matrix.pr_number }}.md + status-pr${{ matrix.pr_number }}.json + junit-*-pr${{ matrix.pr_number }}.xml + trinity-logs-pr${{ matrix.pr_number }}.txt + if-no-files-found: warn + retention-days: 14 + + comment: + name: post sticky comments + needs: [discover, test] + if: always() && needs.discover.outputs.has_prs == 'true' + runs-on: ubuntu-latest + timeout-minutes: 10 + permissions: + issues: write # PR conversation comments are issue comments + pull-requests: write + steps: + - name: Download all integration artifacts + uses: actions/download-artifact@v8 + with: + pattern: integration-pr-* + merge-multiple: true + + - name: Post or update sticky comment per PR + uses: actions/github-script@v9 + env: + MARKER: '' + with: + script: | + const fs = require('fs'); + const marker = process.env.MARKER; + const paths = fs.readdirSync('.').filter(f => /^status-pr\d+\.json$/.test(f)); + if (paths.length === 0) { + core.info('No status JSONs found — nothing to comment on.'); + return; + } + + for (const p of paths) { + try { + const status = JSON.parse(fs.readFileSync(p, 'utf8')); + const pr = status.pr_number; + const diffPath = `diff-pr${pr}.md`; + + let body; + if (status.merge_conflict) { + body = `${marker}\n⚠️ **Live-instance suite skipped — merge conflict against \`dev\`.**\n\nResolve by merging \`dev\` locally and pushing the result; the next nightly re-tests.`; + } else if (status.regression) { + let diffMd = '_diff artifact missing_'; + if (fs.existsSync(diffPath)) { + diffMd = fs.readFileSync(diffPath, 'utf8'); + } + body = `${marker}\n⚠️ **Live-instance integration suite found regressions when this PR is merged into \`dev\`.**\n\n
\nRegression details (head_sha: \`${status.head_sha}\`)\n\n${diffMd}\n\n
\n\n_Reproduce locally against a running stack:_ \`tests/run-core.sh\``; + } else { + body = `${marker}\n✅ **Live-instance integration suite clean** when this PR is merged into \`dev\` (head_sha: \`${status.head_sha}\`).`; + } + + // Find the existing sticky comment by marker (paginated). The + // bot-author filter is required: without it a user comment + // quoting the marker matches, updateComment 403s, and no fresh + // sticky is ever created. + const existing = await github.paginate( + github.rest.issues.listComments, + { owner: context.repo.owner, repo: context.repo.repo, issue_number: pr, per_page: 100 } + ); + const mine = existing.find(c => + c.body && c.body.includes(marker) && c.user && c.user.type === 'Bot'); + + if (mine) { + await github.rest.issues.updateComment({ + owner: context.repo.owner, repo: context.repo.repo, + comment_id: mine.id, body, + }); + } else { + await github.rest.issues.createComment({ + owner: context.repo.owner, repo: context.repo.repo, + issue_number: pr, body, + }); + } + } catch (e) { + core.warning(`Failed to comment for ${p}: ${e.message}`); + } + } diff --git a/tests/registry.json b/tests/registry.json index eb177c29f..dff521d02 100644 --- a/tests/registry.json +++ b/tests/registry.json @@ -60,7 +60,7 @@ "brain-orb", "feature-flags" ], - "description": "Admin-configurable Brain Orb flags (trinity-enterprise#85): _resolve_bool_flag order (stored wins both directions, env opt-in, default OFF, junk-value fail-safe, fail-open on DB error), GET/PUT /api/settings/brain-orb (per-flag source, partial update, clear revert-to-env, set+clear conflict 400, 403 non-admin, audit old→new, /{key} route ordering), generic PUT/DELETE compatibility, brain-orb route gate honoring a real DB flip without restart, and feature-flags composition (voice = base ∧ voice ∧ key; 200 despite brain-orb DB failure)." + "description": "Admin-configurable Brain Orb flags (trinity-enterprise#85): _resolve_bool_flag order (stored wins both directions, env opt-in, default OFF, junk-value fail-safe, fail-open on DB error), GET/PUT /api/settings/brain-orb (per-flag source, partial update, clear revert-to-env, set+clear conflict 400, 403 non-admin, audit old\u2192new, /{key} route ordering), generic PUT/DELETE compatibility, brain-orb route gate honoring a real DB flip without restart, and feature-flags composition (voice = base \u2227 voice \u2227 key; 200 despite brain-orb DB failure)." }, { "file": "unit/test_1332_cancelled_activity_state.py", @@ -72,7 +72,7 @@ "activities", "observability" ], - "description": "A user-cancelled execution's dispatch activity is recorded as ActivityState.CANCELLED, not FAILED. Covers the enum value + activity_state_for_terminal mapping helper, the operator-terminate handler (Path B) closing the open dispatch activity as CANCELLED (terminated→close, already_finished→no-op, no-activity→no-op, close-raises→swallowed), and the collaboration/self-task close helpers mapping a cancelled result to CANCELLED (#1332)." + "description": "A user-cancelled execution's dispatch activity is recorded as ActivityState.CANCELLED, not FAILED. Covers the enum value + activity_state_for_terminal mapping helper, the operator-terminate handler (Path B) closing the open dispatch activity as CANCELLED (terminated\u2192close, already_finished\u2192no-op, no-activity\u2192no-op, close-raises\u2192swallowed), and the collaboration/self-task close helpers mapping a cancelled result to CANCELLED (#1332)." }, { "file": "unit/test_voip_audio.py", @@ -84,7 +84,7 @@ "voip", "audio" ], - "description": "Audio codec round-trip + stateful ratecv continuity (anti-click) + 160-byte framing for the Twilio↔Gemini bridge (#1056). Skips where audioop/audioop-lts is unavailable." + "description": "Audio codec round-trip + stateful ratecv continuity (anti-click) + 160-byte framing for the Twilio\u2194Gemini bridge (#1056). Skips where audioop/audioop-lts is unavailable." }, { "file": "unit/test_voip_db.py", @@ -467,7 +467,7 @@ "lifecycle", "file-sharing" ], - "description": "check_public_folder_mount_matches truth table: enabled+mounted → True, enabled+unmounted → False (needs recreation to attach), disabled+mounted → False (needs recreation to detach), disabled+unmounted → True. Adversarial cases: similar paths (/public-backup, /public/inner) don't match, missing 'Mounts' key handled, flag re-read each call, other mounts (shared-out, shared-in/*, workspace) don't interfere (9 tests)" + "description": "check_public_folder_mount_matches truth table: enabled+mounted \u2192 True, enabled+unmounted \u2192 False (needs recreation to attach), disabled+mounted \u2192 False (needs recreation to detach), disabled+unmounted \u2192 True. Adversarial cases: similar paths (/public-backup, /public/inner) don't match, missing 'Mounts' key handled, flag re-read each call, other mounts (shared-out, shared-in/*, workspace) don't interfere (9 tests)" }, { "file": "unit/test_slack_dm_default.py", @@ -479,7 +479,7 @@ "db", "slack" ], - "description": "set_dm_default + unbind_agent contract: setter is single-tx clear-then-set, idempotent, exclusive (exactly one default per workspace), per-workspace isolation, returns False when agent not bound. Unbind is pure delete (does NOT auto-promote — router enforces the guard), works on non-default and last-agent paths, unknown agent returns False (10 tests)" + "description": "set_dm_default + unbind_agent contract: setter is single-tx clear-then-set, idempotent, exclusive (exactly one default per workspace), per-workspace isolation, returns False when agent not bound. Unbind is pure delete (does NOT auto-promote \u2014 router enforces the guard), works on non-default and last-agent paths, unknown agent returns False (10 tests)" }, { "file": "test_public_chat_history.py", @@ -491,7 +491,7 @@ "public", "chat" ], - "description": "Tests for GET /api/public/sessions/{token} and GET /api/public/sessions/{token}/{session_id} — auth requirements, 404 on invalid tokens, response shape, limit param" + "description": "Tests for GET /api/public/sessions/{token} and GET /api/public/sessions/{token}/{session_id} \u2014 auth requirements, 404 on invalid tokens, response shape, limit param" }, { "file": "unit/test_voice_tools.py", @@ -504,7 +504,7 @@ "gemini", "tools" ], - "description": "Unit tests for voice tool call support (#581): _execute_tool (success, empty prompt, truncation to 2000 chars, agent not reachable, task error), _execute_and_respond (success path with callbacks, timeout → error response, inactive session skips send), tool declaration (_RUN_TASK_TOOL name + required prompt), end_session cancels pending tool tasks (12 tests)" + "description": "Unit tests for voice tool call support (#581): _execute_tool (success, empty prompt, truncation to 2000 chars, agent not reachable, task error), _execute_and_respond (success path with callbacks, timeout \u2192 error response, inactive session skips send), tool declaration (_RUN_TASK_TOOL name + required prompt), end_session cancels pending tool tasks (12 tests)" }, { "file": "test_files_guardrail_bypass.py", @@ -543,7 +543,7 @@ "files", "chat" ], - "description": "Unit tests for web chat file upload (#364): sanitize_filename (path traversal, unicode, dedup, truncation), decode_web_file (data: URI prefix stripping, raw base64, empty/bad input), process_file_uploads (empty list, download failure, unsupported MIME, oversized file, image vision block collection, text file container write, max_files cap) — 14 tests" + "description": "Unit tests for web chat file upload (#364): sanitize_filename (path traversal, unicode, dedup, truncation), decode_web_file (data: URI prefix stripping, raw base64, empty/bad input), process_file_uploads (empty list, download failure, unsupported MIME, oversized file, image vision block collection, text file container write, max_files cap) \u2014 14 tests" }, { "file": "unit/test_slack_mrkdwn.py", @@ -569,7 +569,7 @@ "webhooks", "lint" ], - "description": "AST-based lint guard (no backend deps required) that asserts every db.(...) call in src/backend/routers/ and src/backend/services/ resolves to a real method on DatabaseManager. Two tests: strict regression check for the four WEBHOOK-001 methods (#647: generate_webhook_token, get_schedule_by_webhook_token, revoke_webhook_token, get_webhook_status) and a broad facade-resolution scan guarded by a KNOWN_FACADE_GAPS allowlist for eight unrelated pre-existing gaps. Catches AttributeError-at-runtime regressions that integration-only tests miss in CI — would have caught WEBHOOK-001 before #291 landed." + "description": "AST-based lint guard (no backend deps required) that asserts every db.(...) call in src/backend/routers/ and src/backend/services/ resolves to a real method on DatabaseManager. Two tests: strict regression check for the four WEBHOOK-001 methods (#647: generate_webhook_token, get_schedule_by_webhook_token, revoke_webhook_token, get_webhook_status) and a broad facade-resolution scan guarded by a KNOWN_FACADE_GAPS allowlist for eight unrelated pre-existing gaps. Catches AttributeError-at-runtime regressions that integration-only tests miss in CI \u2014 would have caught WEBHOOK-001 before #291 landed." }, { "file": "unit/test_slack_token_encryption.py", @@ -597,7 +597,7 @@ "credentials", "encryption" ], - "description": "Unit tests for Telegram bot token encryption in db/telegram_channels.py (#664): round-trip via TelegramChannelOperations.create_binding + get_decrypted_bot_token, raw DB value is AES-256-GCM JSON envelope, get_binding_by_agent returns the encrypted blob (plaintext only via accessor), corrupt envelope returns None with ERROR log, wrong-key envelope returns None, re-encryption on update produces a fresh nonce (and token rotation reflects the new plaintext), encrypt raises on missing CREDENTIAL_ENCRYPTION_KEY, decrypt swallows missing-key error and returns None. No plaintext fallback path — Telegram never shipped plaintext." + "description": "Unit tests for Telegram bot token encryption in db/telegram_channels.py (#664): round-trip via TelegramChannelOperations.create_binding + get_decrypted_bot_token, raw DB value is AES-256-GCM JSON envelope, get_binding_by_agent returns the encrypted blob (plaintext only via accessor), corrupt envelope returns None with ERROR log, wrong-key envelope returns None, re-encryption on update produces a fresh nonce (and token rotation reflects the new plaintext), encrypt raises on missing CREDENTIAL_ENCRYPTION_KEY, decrypt swallows missing-key error and returns None. No plaintext fallback path \u2014 Telegram never shipped plaintext." }, { "file": "unit/test_whatsapp_token_encryption.py", @@ -612,7 +612,7 @@ "credentials", "encryption" ], - "description": "Unit tests for Twilio AuthToken encryption in db/whatsapp_channels.py (#664): round-trip via WhatsAppChannelOperations.create_binding + get_decrypted_auth_token, raw DB value is AES-256-GCM JSON envelope, account_sid stays plaintext (public Twilio identifier — pinned so future refactors don't accidentally encrypt or strip it), corrupt envelope returns None, wrong-key envelope returns None, re-encryption on update produces a fresh nonce, encrypt raises on missing CREDENTIAL_ENCRYPTION_KEY, decrypt returns None on missing key." + "description": "Unit tests for Twilio AuthToken encryption in db/whatsapp_channels.py (#664): round-trip via WhatsAppChannelOperations.create_binding + get_decrypted_auth_token, raw DB value is AES-256-GCM JSON envelope, account_sid stays plaintext (public Twilio identifier \u2014 pinned so future refactors don't accidentally encrypt or strip it), corrupt envelope returns None, wrong-key envelope returns None, re-encryption on update produces a fresh nonce, encrypt raises on missing CREDENTIAL_ENCRYPTION_KEY, decrypt returns None on missing key." }, { "file": "unit/test_slack_workspaces_encryption.py", @@ -671,7 +671,7 @@ "resilience", "circuit-breaker" ], - "description": "Tests that the AgentClient circuit breaker only counts TCP unreachability (connect errors/timeouts) toward opening — HTTP errors, read/write/pool failures, and protocol errors are excluded" + "description": "Tests that the AgentClient circuit breaker only counts TCP unreachability (connect errors/timeouts) toward opening \u2014 HTTP errors, read/write/pool failures, and protocol errors are excluded" }, { "file": "integration/test_circuit_breaker.py", @@ -685,7 +685,7 @@ "circuit-breaker", "redis" ], - "description": "Drives the real AgentClient through httpx.MockTransport against real Redis circuit state — hard vs soft failures, mixed interleave, open-circuit fast-fail, recovery on 200, and the deferred half-open soft-failure probe-lock behaviour" + "description": "Drives the real AgentClient through httpx.MockTransport against real Redis circuit state \u2014 hard vs soft failures, mixed interleave, open-circuit fast-fail, recovery on 200, and the deferred half-open soft-failure probe-lock behaviour" }, { "file": "integration/test_monitoring_service.py", @@ -698,7 +698,7 @@ "monitoring", "circuit-breaker" ], - "description": "Mirrors the #474 classification rule on the /health probe path and pins the `status_code >= 500 → UNHEALTHY` aggregator branch" + "description": "Mirrors the #474 classification rule on the /health probe path and pins the `status_code >= 500 \u2192 UNHEALTHY` aggregator branch" }, { "file": "unit/test_platform_default_model_regression.py", @@ -769,7 +769,7 @@ "migrations", "reliability" ], - "description": "Migration runner atomicity + cross-process lock (#1160). _atomic_rebuild rename-swap inside an explicit transaction closes the DROP-rebuild data-loss window (crash mid-rebuild rolls back with no data loss; verified for agent_sharing + agent_skills incl. lowercase/NULL-email behavior preservation and index recreation); replay-after-partial-apply completes cleanly; a failed migration is named on its traceback via add_note (original exception type preserved). Cross-process flock (db/migration_lock.py) serialises concurrent boots — multiprocessing tests prove mutual exclusion and a safe concurrent rebuild." + "description": "Migration runner atomicity + cross-process lock (#1160). _atomic_rebuild rename-swap inside an explicit transaction closes the DROP-rebuild data-loss window (crash mid-rebuild rolls back with no data loss; verified for agent_sharing + agent_skills incl. lowercase/NULL-email behavior preservation and index recreation); replay-after-partial-apply completes cleanly; a failed migration is named on its traceback via add_note (original exception type preserved). Cross-process flock (db/migration_lock.py) serialises concurrent boots \u2014 multiprocessing tests prove mutual exclusion and a safe concurrent rebuild." }, { "file": "unit/test_compatibility_checks.py", @@ -794,7 +794,7 @@ "voip", "voice" ], - "description": "Per-agent persisted voice + VoIP enable/disable toggle (#28). db/agents.py get_voice_name/set_voice_name: unset->'Kore' fallback, set/get roundtrip, invalid-persisted-value->default (reviewer M1 read-path validation), clear->default, set-on-missing-agent->False. db/voip.py set_enabled toggle + the create_binding 'preserve enabled on re-PUT' fix (reviewer H3 — re-saving credentials on a disabled binding must not silently re-enable it) + set_enabled on a missing binding returns False (router 404s). config.GEMINI_VOICE_NAMES default + parity guard asserting the frontend src/constants/voices.js VOICE ids and DEFAULT_VOICE_NAME mirror the backend constants (reviewer M2 cross-language drift guard). Runs on db_harness backends (SQLite always, PostgreSQL when TEST_POSTGRES_URL set); no Docker/API/Redis." + "description": "Per-agent persisted voice + VoIP enable/disable toggle (#28). db/agents.py get_voice_name/set_voice_name: unset->'Kore' fallback, set/get roundtrip, invalid-persisted-value->default (reviewer M1 read-path validation), clear->default, set-on-missing-agent->False. db/voip.py set_enabled toggle + the create_binding 'preserve enabled on re-PUT' fix (reviewer H3 \u2014 re-saving credentials on a disabled binding must not silently re-enable it) + set_enabled on a missing binding returns False (router 404s). config.GEMINI_VOICE_NAMES default + parity guard asserting the frontend src/constants/voices.js VOICE ids and DEFAULT_VOICE_NAME mirror the backend constants (reviewer M2 cross-language drift guard). Runs on db_harness backends (SQLite always, PostgreSQL when TEST_POSTGRES_URL set); no Docker/API/Redis." }, { "file": "unit/test_28_voip_voice_endpoints.py", @@ -878,7 +878,7 @@ "infrastructure", "brain-orb" ], - "description": "Cornelius first-run seeder (ent#107): ensure_seeded first-run gating via durable cornelius_seeded flag (deleted agent not resurrected); fresh-install scoping (existing non-system agents -> skip + converge flag, orb flag untouched); owner-must-exist deferral (pre-setup skip without burning the flag); Docker-unavailable skip; Brain Orb flag defaulted ON existence-guarded (admin OFF preserved); 409-on-exists convergence vs generic-failure retry (flag not burned, never raises); --workers 2 Redis SETNX provision lock (held -> skip, winner provisions+releases with TTL, Redis-down fail-open); _provision builds a github:Abilityai/cornelius create with request=None and source_mode default True (#1656 — the trinity-enterprise#123 tokenless public-clone path is source-mode only); real-DB smoke of db.count_non_system_agents() facade delegation. create_agent_internal/db/redis/docker seams patched; no Docker/backend. ent#124 addition: precomputed freshness verdict — ensure_seeded(fresh=True) skips the internal count, fresh=False converges the flag without provisioning, fresh=None preserves the legacy count path." + "description": "Cornelius first-run seeder (ent#107): ensure_seeded first-run gating via durable cornelius_seeded flag (deleted agent not resurrected); fresh-install scoping (existing non-system agents -> skip + converge flag, orb flag untouched); owner-must-exist deferral (pre-setup skip without burning the flag); Docker-unavailable skip; Brain Orb flag defaulted ON existence-guarded (admin OFF preserved); 409-on-exists convergence vs generic-failure retry (flag not burned, never raises); --workers 2 Redis SETNX provision lock (held -> skip, winner provisions+releases with TTL, Redis-down fail-open); _provision builds a github:Abilityai/cornelius create with request=None and source_mode default True (#1656 \u2014 the trinity-enterprise#123 tokenless public-clone path is source-mode only); real-DB smoke of db.count_non_system_agents() facade delegation. create_agent_internal/db/redis/docker seams patched; no Docker/backend. ent#124 addition: precomputed freshness verdict \u2014 ensure_seeded(fresh=True) skips the internal count, fresh=False converges the flag without provisioning, fresh=None preserves the legacy count path." }, { "file": "unit/test_1557_autonomy_breaker_decoupled.py", @@ -890,7 +890,7 @@ "circuit-breaker", "autonomy" ], - "description": "Disabling autonomy must not touch the circuit breaker (#1557). Structural guards: autonomy.py no longer references force_circuit_dormant/reset_circuit (regression guard — fails on pre-#1557 source) and still calls set_schedule_enabled (proactive suppression intact). Message honesty: _circuit_breaker_error names transport-unreachable vs dispatch-auth-dead, and every branch keeps the 'circuit breaker open' substring pinned by the #1560 integration test." + "description": "Disabling autonomy must not touch the circuit breaker (#1557). Structural guards: autonomy.py no longer references force_circuit_dormant/reset_circuit (regression guard \u2014 fails on pre-#1557 source) and still calls set_schedule_enabled (proactive suppression intact). Message honesty: _circuit_breaker_error names transport-unreachable vs dispatch-auth-dead, and every branch keeps the 'circuit breaker open' substring pinned by the #1560 integration test." }, { "file": "integration/test_1557_autonomy_inbound.py", @@ -951,7 +951,7 @@ "proactive-messaging", "regression" ], - "description": "Proactive messages persist to channel session history (#1600). Core property is session-identifier EQUALITY: the key the proactive path derives must equal the key an inbound DM resolves to, per channel (telegram/whatsapp/slack) — persisting into a different session looks like a fix but leaves the agent unaware of its own outreach. Also: #903 attribution (assistant role, agent sender_label, recipient sender_email for the single-participant DM), persist only on confirmed delivery (no phantom turn on failure), fail-soft on DB error (message already sent), session created when absent (the chat-link session_id column is never written by any code path), and access-grant notifications deliberately NOT persisted (#951 out of scope)." + "description": "Proactive messages persist to channel session history (#1600). Core property is session-identifier EQUALITY: the key the proactive path derives must equal the key an inbound DM resolves to, per channel (telegram/whatsapp/slack) \u2014 persisting into a different session looks like a fix but leaves the agent unaware of its own outreach. Also: #903 attribution (assistant role, agent sender_label, recipient sender_email for the single-participant DM), persist only on confirmed delivery (no phantom turn on failure), fail-soft on DB error (message already sent), session created when absent (the chat-link session_id column is never written by any code path), and access-grant notifications deliberately NOT persisted (#951 out of scope)." }, { "file": "unit/test_1649_group_message_history.py", @@ -963,7 +963,7 @@ "proactive-messaging", "regression" ], - "description": "Proactive GROUP messages persist to channel session history (#1649). Slack is a real recall fix: the broadcast is filed at the posted message's own ts, which IS the thread key an in-thread reply resolves to (asserted against the adapter). Telegram is bookkeeping only — group sessions are per-(sender, chat) with no group branch, so a broadcast uses a synthetic agent-sender key; a test pins that it deliberately does NOT match a participant's session, so the accepted trade-off can't be mistaken for a bug and a future adapter group-branch forces a re-decision. Router-level tests drive the real endpoints (the #1600 lesson: helper-only tests passed with the persistence call deleted outright). Also: #903 shared-thread attribution (sender_email=None), persist only on confirmed delivery, fail-soft, and send_message_detailed's ts capture keeping send_message's 2-tuple contract for its ~7 callers." + "description": "Proactive GROUP messages persist to channel session history (#1649). Slack is a real recall fix: the broadcast is filed at the posted message's own ts, which IS the thread key an in-thread reply resolves to (asserted against the adapter). Telegram is bookkeeping only \u2014 group sessions are per-(sender, chat) with no group branch, so a broadcast uses a synthetic agent-sender key; a test pins that it deliberately does NOT match a participant's session, so the accepted trade-off can't be mistaken for a bug and a future adapter group-branch forces a re-decision. Router-level tests drive the real endpoints (the #1600 lesson: helper-only tests passed with the persistence call deleted outright). Also: #903 shared-thread attribution (sender_email=None), persist only on confirmed delivery, fail-soft, and send_message_detailed's ts capture keeping send_message's 2-tuple contract for its ~7 callers." }, { "file": "unit/test_1632_operator_queue_caps.py", @@ -976,7 +976,7 @@ "security", "reliability" ], - "description": "Operator-queue create-path ingestion caps (#1632): the agent-authored sync boundary is bounded by a DB-measured pending-DEPTH cap (primary, Redis-independent), a per-agent + fleet RATE cap (fail-open, break on deny, real in-process-fallback exercise), a total truncate-with-marker field-hygiene clamp inside the #1525 try/except (title/question/context/options/execution_id/created_at/priority, non-dict context, boundary lengths, never-raises), the reserved-platform-id + malformed-id guard, the opqueue:leader cross-worker lock (non-leader poll cycle is a no-op), the one-per-episode flood alert (un-guessable id, cooldown, emit-failure-safe, platform-exempt), the oversize-file skip, and the generous DB-sink belt + count_pending_for_agent helper. Also pins the concrete platform producer validation_service._notify_operator_on_failure, converted to a direct db.create_operator_queue_item (#1632): reserved `val_` id (an agent can't pre-create and thereby suppress its own validation alarm), bypasses the agent-file sync caps, and stays best-effort (a create failure never fails validation) — the conversion also fixed a latent bug where the notification appended to a bare list and was never ingested. Also pins the _leader_ttl 30s floor (F1: lease outlasts one worst-case cycle + sleep, so leadership doesn't flap and the flood alert doesn't double-emit under --workers 2). Pure/mocked (rate_limiter stubbed + one real in-process test; DB engine stubbed)." + "description": "Operator-queue create-path ingestion caps (#1632): the agent-authored sync boundary is bounded by a DB-measured pending-DEPTH cap (primary, Redis-independent), a per-agent + fleet RATE cap (fail-open, break on deny, real in-process-fallback exercise), a total truncate-with-marker field-hygiene clamp inside the #1525 try/except (title/question/context/options/execution_id/created_at/priority, non-dict context, boundary lengths, never-raises), the reserved-platform-id + malformed-id guard, the opqueue:leader cross-worker lock (non-leader poll cycle is a no-op), the one-per-episode flood alert (un-guessable id, cooldown, emit-failure-safe, platform-exempt), the oversize-file skip, and the generous DB-sink belt + count_pending_for_agent helper. Also pins the concrete platform producer validation_service._notify_operator_on_failure, converted to a direct db.create_operator_queue_item (#1632): reserved `val_` id (an agent can't pre-create and thereby suppress its own validation alarm), bypasses the agent-file sync caps, and stays best-effort (a create failure never fails validation) \u2014 the conversion also fixed a latent bug where the notification appended to a bare list and was never ingested. Also pins the _leader_ttl 30s floor (F1: lease outlasts one worst-case cycle + sleep, so leadership doesn't flap and the flood alert doesn't double-emit under --workers 2). Pure/mocked (rate_limiter stubbed + one real in-process test; DB engine stubbed)." }, { "file": "unit/test_1615_ssh_password_removed.py", @@ -988,7 +988,7 @@ "security", "regression" ], - "description": "Password SSH auth removed (#1615, router surface): an explicit auth_method='password' returns 400 naming key auth as the alternative — not the pre-fix 500 (ModuleNotFoundError: crypt, removed from the stdlib in Python 3.13) and not a silent fall-through; refusal is case-insensitive and covers unknown methods; the request never reaches the container; key auth still works and remains the default; the response carries neither private_key (#175) nor password. Complements test_ssh_service.py, which guards the service layer (helpers deleted, no crypt import)." + "description": "Password SSH auth removed (#1615, router surface): an explicit auth_method='password' returns 400 naming key auth as the alternative \u2014 not the pre-fix 500 (ModuleNotFoundError: crypt, removed from the stdlib in Python 3.13) and not a silent fall-through; refusal is case-insensitive and covers unknown methods; the request never reaches the container; key auth still works and remains the default; the response carries neither private_key (#175) nor password. Complements test_ssh_service.py, which guards the service layer (helpers deleted, no crypt import)." }, { "file": "unit/test_ent162_per_user_github_pat.py", @@ -1014,7 +1014,7 @@ "live", "integration" ], - "description": "Live SSH-access API tests (#1615). Password auth is refused with 400 and never 500 (the pre-fix ModuleNotFoundError: crypt on Python 3.13) — including with a valid public_key supplied, which is what distinguishes the guard from an incidental missing-key 400; refusal is case-insensitive and covers unknown methods. Key auth (BYOK) returns connection details, is the default, clamps TTL, and leaks neither private_key (#175) nor password. Marked `integration`: an ed25519 key injected through the API is used for a REAL ssh login into the container (proves key auth works — mocks cannot), and a password login is proven impossible against the agent sshd's own PasswordAuthentication=no. sshd readiness is polled via the SSH banner so the e2e tests don't race container boot." + "description": "Live SSH-access API tests (#1615). Password auth is refused with 400 and never 500 (the pre-fix ModuleNotFoundError: crypt on Python 3.13) \u2014 including with a valid public_key supplied, which is what distinguishes the guard from an incidental missing-key 400; refusal is case-insensitive and covers unknown methods. Key auth (BYOK) returns connection details, is the default, clamps TTL, and leaks neither private_key (#175) nor password. Marked `integration`: an ed25519 key injected through the API is used for a REAL ssh login into the container (proves key auth works \u2014 mocks cannot), and a password login is proven impossible against the agent sshd's own PasswordAuthentication=no. sshd readiness is polled via the SSH banner so the e2e tests don't race container boot." }, { "file": "unit/test_1673_execution_error_not_success.py", @@ -1061,7 +1061,7 @@ "agents", "unit" ], - "description": "Per-agent display label (ent#181): a human-facing name that is rendered, never resolved — the slug (agent_name) stays the identity every route, container, volume, MCP key and A2A card keys on. Pins the point of the feature: setting a label leaves the slug AND its #1664 volume identity untouched, does not reserve a name, and two agents may share one (labels aren't identities). NULL = render the slug (no backfill; clearing reverts rather than blanking); a blank/whitespace label stores NULL, not an empty string that would render a nameless agent. Soft-deleted agents are not editable (deleted_at guard, mirroring the other settings setters). Batch read for the fleet list (absent = no label = slug) so the hottest endpoint stays 1 query. Router: owner-gated PUT, `label` never coerced to the slug on read (the UI must tell 'no label' from 'label equals slug'), null clears, 404 when the row vanished, WS agent_label_changed broadcast." + "description": "Per-agent display label (ent#181): a human-facing name that is rendered, never resolved \u2014 the slug (agent_name) stays the identity every route, container, volume, MCP key and A2A card keys on. Pins the point of the feature: setting a label leaves the slug AND its #1664 volume identity untouched, does not reserve a name, and two agents may share one (labels aren't identities). NULL = render the slug (no backfill; clearing reverts rather than blanking); a blank/whitespace label stores NULL, not an empty string that would render a nameless agent. Soft-deleted agents are not editable (deleted_at guard, mirroring the other settings setters). Batch read for the fleet list (absent = no label = slug) so the hottest endpoint stays 1 query. Router: owner-gated PUT, `label` never coerced to the slug on read (the UI must tell 'no label' from 'label equals slug'), null clears, 404 when the row vanished, WS agent_label_changed broadcast." }, { "file": "unit/test_1484_create_agent_characterization.py", @@ -1084,7 +1084,7 @@ "skills", "unit" ], - "description": "Full-directory skill packages (ent#183): pure packaging primitives (hardened frontmatter contract parse — alias-bomb refused, garbage requires typed-guarded, malicious dep names regex-gated; git-archive member vetting — REGTYPE only, symlinks/litter/protected basenames dropped with named warnings; injection tar with generated .trinity-skill.json meta appended LAST; manifest-based prune diff capped 200/skill); round-trip against the REAL agent-server restore_from_tar incl. allowlist confinement; real-git end-to-end (repo -> archive -> filter -> restore, tree-SHA determinism across clones, exec bits from git modes); injection orchestration (skip-if-unchanged vs force, old-image 404 fallback with multi_file_dropped_old_image only for multi-file skills, restore-failure repair path = delete-dir + one re-restore, prune deletes ONLY previous-manifest files, unmanaged same-named dir overwritten but never pruned, per-skill + total caps named errors, dep-probe warnings missing_binary/missing_env fail-open and run for unchanged skills too, SkillInjectionBusy on lock contention, CLAUDE.md section rebuilt from ALL present skills with line-anchored replacement that spares ### lookalikes)." + "description": "Full-directory skill packages (ent#183): pure packaging primitives (hardened frontmatter contract parse \u2014 alias-bomb refused, garbage requires typed-guarded, malicious dep names regex-gated; git-archive member vetting \u2014 REGTYPE only, symlinks/litter/protected basenames dropped with named warnings; injection tar with generated .trinity-skill.json meta appended LAST; manifest-based prune diff capped 200/skill); round-trip against the REAL agent-server restore_from_tar incl. allowlist confinement; real-git end-to-end (repo -> archive -> filter -> restore, tree-SHA determinism across clones, exec bits from git modes); injection orchestration (skip-if-unchanged vs force, old-image 404 fallback with multi_file_dropped_old_image only for multi-file skills, restore-failure repair path = delete-dir + one re-restore, prune deletes ONLY previous-manifest files, unmanaged same-named dir overwritten but never pruned, per-skill + total caps named errors, dep-probe warnings missing_binary/missing_env fail-open and run for unchanged skills too, SkillInjectionBusy on lock contention, CLAUDE.md section rebuilt from ALL present skills with line-anchored replacement that spares ### lookalikes)." }, { "file": "unit/test_ent123_tokenless_clone.py", @@ -1096,7 +1096,7 @@ "git", "unit" ], - "description": "PAT-free clone of public github: templates (ent#123). 50 tests over the 1484-shape purge-and-mock harness: _gate_tokenless_request (\"\"->None normalization, source-mode-only 400 incl. explicit None, fork passthrough), _parse_github_ref charset guard (eval-interpolation hardening), _validate_github_access tokenless ls-remote probe (ok/unavailable->400/transient->502 fail-closed, anonymous branch check, PAT-ful REST regression), _apply_github_env token-var gating (+GIT_SYNC_AUTO belt), lifecycle._apply_persisted_auth_env rebuild seam (repo-only gate, source-mode re-derivation), git_service.probe_anonymous_repo_access stderr classification, the no_write_credentials sync/reset guard (baked env OR per-agent PAT row — #1264 live-injection window, fail-open), and startup.sh static guards (repo-only clone gate, credential-less CLONE_URL, GIT_TERMINAL_PROMPT=0, push blackhole, .env PAT fallback, private-repo failure cause)." + "description": "PAT-free clone of public github: templates (ent#123). 50 tests over the 1484-shape purge-and-mock harness: _gate_tokenless_request (\"\"->None normalization, source-mode-only 400 incl. explicit None, fork passthrough), _parse_github_ref charset guard (eval-interpolation hardening), _validate_github_access tokenless ls-remote probe (ok/unavailable->400/transient->502 fail-closed, anonymous branch check, PAT-ful REST regression), _apply_github_env token-var gating (+GIT_SYNC_AUTO belt), lifecycle._apply_persisted_auth_env rebuild seam (repo-only gate, source-mode re-derivation), git_service.probe_anonymous_repo_access stderr classification, the no_write_credentials sync/reset guard (baked env OR per-agent PAT row \u2014 #1264 live-injection window, fail-open), and startup.sh static guards (repo-only clone gate, credential-less CLONE_URL, GIT_TERMINAL_PROMPT=0, push blackhole, .env PAT fallback, private-repo failure cause)." }, { "file": "unit/test_ent125_resilient_system_deploy.py", @@ -1107,7 +1107,7 @@ "systems", "unit" ], - "description": "Resilient system-manifest deploy (ent#125): best-effort default — partial deploy continues past a failed agent-create and reports failed[] with {name, short_name, template, reason, status_code}; post-create config (folders/permissions/schedules/tags) scoped to the survivor map; config-phase failures degrade to warnings; total failure returns HTTP 500 with the full report body, skips config/start, and never writes trinity_prompt; partial still writes it; strict=True aborts preserving the original status code (dict-detail 429 not flattened to 500); reason normalization (dict detail → error field), PAT-bearing git-URL userinfo redaction (learnings 2026-07-14), and 500-char truncation; orchestrator-workers preset with failed orchestrator warns 'non-functional' and configures permissions over survivors only; dry_run and all-success responses unchanged (failed == []). Router mounted alone; since the ent#124 extraction the deploy orchestration lives in services.system_service.deploy_manifest, so config fns/db are patched on services.system_service and agent creation via the _default_create_agent_fn seam." + "description": "Resilient system-manifest deploy (ent#125): best-effort default \u2014 partial deploy continues past a failed agent-create and reports failed[] with {name, short_name, template, reason, status_code}; post-create config (folders/permissions/schedules/tags) scoped to the survivor map; config-phase failures degrade to warnings; total failure returns HTTP 500 with the full report body, skips config/start, and never writes trinity_prompt; partial still writes it; strict=True aborts preserving the original status code (dict-detail 429 not flattened to 500); reason normalization (dict detail \u2192 error field), PAT-bearing git-URL userinfo redaction (learnings 2026-07-14), and 500-char truncation; orchestrator-workers preset with failed orchestrator warns 'non-functional' and configures permissions over survivors only; dry_run and all-success responses unchanged (failed == []). Router mounted alone; since the ent#124 extraction the deploy orchestration lives in services.system_service.deploy_manifest, so config fns/db are patched on services.system_service and agent creation via the _default_create_agent_fn seam." }, { "file": "unit/test_ent124_default_system_seed.py", @@ -1119,7 +1119,7 @@ "unit", "infrastructure" ], - "description": "First-run default-system seeder (ent#124): the REAL bundled config/manifests/default-system.yaml validates through the executor's own parse_manifest/validate_manifest (acme trio, full-mesh, no schedules, no prompt) and every local: template it references exists in-tree (learnings 2026-07-23 blank-agent trap); durable default_system_seeded flag gating (deleted fleet never resurrected); persisted first_run_fresh verdict — computed once BEFORE Cornelius runs, persisted, reused on later passes (cross-pass count-poisoning bug), cornelius_seeded=true forces not-fresh (ent#107-era installs get no fleet), count failure defers without persisting; TRINITY_DEFAULT_SYSTEM_MANIFEST disable sentinels skip without burning the flag; override path used when set, unreadable override fails loudly (operator-queue alert, no bundled fallback, no flag); flag policy per deploy outcome: deployed/partial set flag (+partial alert), failed (0 created)/exception do NOT (retry-safe); existence backstop converges the flag without deploying when any {system}-{short} name is reserved (suffix double-seed guard for the fail-open lock); SETNX lock held->skip / winner deploys+releases; orchestrator never raises when Cornelius explodes. db/docker/redis/deploy_manifest seams patched; no Docker/backend." + "description": "First-run default-system seeder (ent#124): the REAL bundled config/manifests/default-system.yaml validates through the executor's own parse_manifest/validate_manifest (acme trio, full-mesh, no schedules, no prompt) and every local: template it references exists in-tree (learnings 2026-07-23 blank-agent trap); durable default_system_seeded flag gating (deleted fleet never resurrected); persisted first_run_fresh verdict \u2014 computed once BEFORE Cornelius runs, persisted, reused on later passes (cross-pass count-poisoning bug), cornelius_seeded=true forces not-fresh (ent#107-era installs get no fleet), count failure defers without persisting; TRINITY_DEFAULT_SYSTEM_MANIFEST disable sentinels skip without burning the flag; override path used when set, unreadable override fails loudly (operator-queue alert, no bundled fallback, no flag); flag policy per deploy outcome: deployed/partial set flag (+partial alert), failed (0 created)/exception do NOT (retry-safe); existence backstop converges the flag without deploying when any {system}-{short} name is reserved (suffix double-seed guard for the fail-open lock); SETNX lock held->skip / winner deploys+releases; orchestrator never raises when Cornelius explodes. db/docker/redis/deploy_manifest seams patched; no Docker/backend." }, { "file": "unit/test_1809_image_drift_recreate.py", @@ -1131,7 +1131,7 @@ "unit", "reliability" ], - "description": "Image-drift recreate predicate (#1809): check_base_image_matches compares the container's resolved image id against what its OWN Config.Image reference currently resolves to — drift→False (recreate on next cold start), match→True; fail-open True on falsy attrs (Config.Image can be \"\"), ImageNotFound (recreate would fail on the same missing tag), and generic daemon errors (2am case: start proceeds on the old image, never a fleet recreate); ID-pinned Config.Image is a documented tautological no-op; version-pinned tags compare their own tag only (a :latest-only rebuild leaves 0.8.0-pinned agents untouched). Source-pins the lifecycle wiring: lazy evaluation gated on `not needs_recreation and not was_already_running` (cold start only — start-on-running stays an idempotent no-op), ephemeral-ghost exclusion, image-check assignment ABOVE the #1560 clear_agent_breakers gate, recreate delivering the upgrade via the container's own tag, trinity.base-image-version label refresh, recreate-race hardening (remove NotFound tolerated, run 409 adopts the winner), start-response recreated/recreate_reason surfacing, and the skip-inject _reset fixture stubbing the new predicate (Mock-auto-child trap)." + "description": "Image-drift recreate predicate (#1809): check_base_image_matches compares the container's resolved image id against what its OWN Config.Image reference currently resolves to \u2014 drift\u2192False (recreate on next cold start), match\u2192True; fail-open True on falsy attrs (Config.Image can be \"\"), ImageNotFound (recreate would fail on the same missing tag), and generic daemon errors (2am case: start proceeds on the old image, never a fleet recreate); ID-pinned Config.Image is a documented tautological no-op; version-pinned tags compare their own tag only (a :latest-only rebuild leaves 0.8.0-pinned agents untouched). Source-pins the lifecycle wiring: lazy evaluation gated on `not needs_recreation and not was_already_running` (cold start only \u2014 start-on-running stays an idempotent no-op), ephemeral-ghost exclusion, image-check assignment ABOVE the #1560 clear_agent_breakers gate, recreate delivering the upgrade via the container's own tag, trinity.base-image-version label refresh, recreate-race hardening (remove NotFound tolerated, run 409 adopts the winner), start-response recreated/recreate_reason surfacing, and the skip-inject _reset fixture stubbing the new predicate (Mock-auto-child trap)." }, { "file": "unit/test_1759_local_template_not_found.py", @@ -1143,7 +1143,7 @@ "unit", "reliability" ], - "description": "Local-template create gate (#1759, companion to #1793). Drives the FULL create_agent_internal, so unlike test_1793_unknown_local_template.py it also proves the reject is pre-side-effect (no container, volume, MCP key, ownership row or ephemeral slot). Covers the absent case at #1793's 404 UNKNOWN_LOCAL_TEMPLATE (both roots missing, dir without template.yaml, name resolving to a regular file), the #1759 400 LOCAL_TEMPLATE_INVALID band (empty / whitespace / comment-only / scalar / list / unparseable template.yaml), INVALID_LOCAL_TEMPLATE_NAME precedence, Blank Agent passthrough, the /template bind source under set/unset/EMPTY HOST_TEMPLATES_PATH, and the disclosure rule — one identical message whichever root missed, echoing no filesystem path (deploy-local template dirs are named after AGENT names, so 'which root' is a #186 enumeration oracle)." + "description": "Local-template create gate (#1759, companion to #1793). Drives the FULL create_agent_internal, so unlike test_1793_unknown_local_template.py it also proves the reject is pre-side-effect (no container, volume, MCP key, ownership row or ephemeral slot). Covers the absent case at #1793's 404 UNKNOWN_LOCAL_TEMPLATE (both roots missing, dir without template.yaml, name resolving to a regular file), the #1759 400 LOCAL_TEMPLATE_INVALID band (empty / whitespace / comment-only / scalar / list / unparseable template.yaml), INVALID_LOCAL_TEMPLATE_NAME precedence, Blank Agent passthrough, the /template bind source under set/unset/EMPTY HOST_TEMPLATES_PATH, and the disclosure rule \u2014 one identical message whichever root missed, echoing no filesystem path (deploy-local template dirs are named after AGENT names, so 'which root' is a #186 enumeration oracle)." }, { "file": "unit/test_1759_template_root_parity.py", @@ -1155,7 +1155,7 @@ "unit", "reliability" ], - "description": "Curated-template root parity (#1759). The create resolver's repo-relative fallback is hand-rolled rather than imported from template_service (which the #1484 harness MagicMocks, so an imported gate would be satisfied by a truthy mock), which makes drift between the two surfaces invisible — these tests pin them equal against both REAL modules. Also pins that the fallback resolves to /config/agent-templates and not /src/config/agent-templates (crud.py sits one directory deeper than template_service.py), that the deploy-local root is unchanged, that the container branch returns the pre-#1759 literal byte-identically, that the host branch is absolute so Docker accepts it as a bind source, and — the guard that would have caught the missing local:default — that EVERY shipped in-tree template resolves through the create gate." + "description": "Curated-template root parity (#1759). The create resolver's repo-relative fallback is hand-rolled rather than imported from template_service (which the #1484 harness MagicMocks, so an imported gate would be satisfied by a truthy mock), which makes drift between the two surfaces invisible \u2014 these tests pin them equal against both REAL modules. Also pins that the fallback resolves to /config/agent-templates and not /src/config/agent-templates (crud.py sits one directory deeper than template_service.py), that the deploy-local root is unchanged, that the container branch returns the pre-#1759 literal byte-identically, that the host branch is absolute so Docker accepts it as a bind source, and \u2014 the guard that would have caught the missing local:default \u2014 that EVERY shipped in-tree template resolves through the create gate." }, { "file": "unit/test_1759_export_manifest_template.py", @@ -1166,7 +1166,7 @@ "systems", "unit" ], - "description": "export_manifest template round-trip (#1759). Blank Agents carry template: None (key PRESENT), and dict.get(key, default) returns the default only when the key is ABSENT — so the old local:business-assistant fallback was unreachable dead code and every template-less agent exported template: null, which SystemAgentConfig.template (non-Optional str) rejects on redeploy. Pins the `or` fix emitting local:default, pass-through of a real template, that the exported manifest both validates and names a template that exists in-tree, and that inferred agents are logged." + "description": "export_manifest template round-trip (#1759). Blank Agents carry template: None (key PRESENT), and dict.get(key, default) returns the default only when the key is ABSENT \u2014 so the old local:business-assistant fallback was unreachable dead code and every template-less agent exported template: null, which SystemAgentConfig.template (non-Optional str) rejects on redeploy. Pins the `or` fix emitting local:default, pass-through of a real template, that the exported manifest both validates and names a template that exists in-tree, and that inferred agents are logged." }, { "file": "unit/test_1771c_schedules_cas_edges.py", @@ -1206,7 +1206,7 @@ "analytics", "scheduling" ], - "description": "Edge-case matrix (sub-area B) for db/schedules/analytics.py on db_harness (#300): _TRIGGER_BUCKETS subset of _BUCKET_ORDER regression guard (the existing literal assertion omits 'Reminders'), unknown/empty/None trigger bucketing, zero-terminal day reports None not 0%, headline-0.0 spec gap (UNSPEC), terminal-based success_rate incl. the legacy 'error' alias, NULL-skipping context AVG, percentile-cap boundary, full-set avg vs sampled p95 (the locked data-source discipline), percentile pool 0/1/2 rows, offset-bearing day bucketing [SQLITE-ONLY], strict '>' window boundary with a frozen iso_cutoff, gap-filled contiguous timeline, and _schedule_command_label unicode/length boundaries. Also covers get_schedule_analytics end-to-end (agent-written tool_calls SHAPE guards — valid-JSON-not-a-list, list-of-scalars, dict-without-name, non-numeric duration — on BOTH that surface and get_agent_schedules_summary; the FAILED arm of the per-schedule timeline; and its own 0/1/2/3-row percentile ladder, a second implementation of the same statistics.quantiles arithmetic get_agent_analytics has)." + "description": "Edge-case matrix (sub-area B) for db/schedules/analytics.py on db_harness (#300): _TRIGGER_BUCKETS subset of _BUCKET_ORDER regression guard (the existing literal assertion omits 'Reminders'), unknown/empty/None trigger bucketing, zero-terminal day reports None not 0%, headline-0.0 spec gap (UNSPEC), terminal-based success_rate incl. the legacy 'error' alias, NULL-skipping context AVG, percentile-cap boundary, full-set avg vs sampled p95 (the locked data-source discipline), percentile pool 0/1/2 rows, offset-bearing day bucketing [SQLITE-ONLY], strict '>' window boundary with a frozen iso_cutoff, gap-filled contiguous timeline, and _schedule_command_label unicode/length boundaries. Also covers get_schedule_analytics end-to-end (agent-written tool_calls SHAPE guards \u2014 valid-JSON-not-a-list, list-of-scalars, dict-without-name, non-numeric duration \u2014 on BOTH that surface and get_agent_schedules_summary; the FAILED arm of the per-schedule timeline; and its own 0/1/2/3-row percentile ladder, a second implementation of the same statistics.quantiles arithmetic get_agent_analytics has)." }, { "file": "unit/test_1771c_schedules_analytics_properties.py", @@ -1292,7 +1292,7 @@ "scheduling", "agents" ], - "description": "Creation-time schedule materialization (ent#89) against a REAL DB with row read-back: both resolver branches populate declared_schedules (the §0/R2 regression — the github: branch never populated template_data), the github fetch uses the creation-resolved PAT and the parsed @branch ref, enabled honored/defaulted, cron->cron_expression, name-match and intra-block idempotency, the ghost skip, a falsy create_schedule return counted as failed (R6), and non-fatality of a raising create_schedule / list_agent_schedules inside the rollback fence." + "description": "Creation-time schedule materialization (ent#89) against a REAL DB with row read-back: both resolver branches populate declared_schedules (the \u00a70/R2 regression \u2014 the github: branch never populated template_data), the github fetch uses the creation-resolved PAT and the parsed @branch ref, enabled honored/defaulted, cron->cron_expression, name-match and intra-block idempotency, the ghost skip, a falsy create_schedule return counted as failed (R6), and non-fatality of a raising create_schedule / list_agent_schedules inside the rollback fence." }, { "file": "unit/test_ent89_manifest_no_duplicate.py", @@ -1398,7 +1398,7 @@ "lifecycle", "credentials" ], - "description": "Agent MCP key detect/self-heal/rotate (#1854). DB: create_agent_mcp_api_key born ACTIVE (tables.py has no column default -> NULL reads as revoked and cannot even deserialize); captured-id delete (not `id != new_id`, so a racing recreate_missing_container mint is not collateral), connector key survives, keep_id never removed, other agents unreachable; DELETE-not-deactivate proven durable across soft-delete -> recover (recover_agent_ownership reactivates every inactive per-agent row). spawned_by_key_id reconcile: `!= current` repairs a child stranded on an OLDER superseded id, leaves a foreign parent alone, idempotent, and enforce_agent_spawn_scope never 403s at any step of a rotation. Drift predicate: absent env, URL-missing (injection needs BOTH), hash-mismatch, system/ghost exempt, fail-safe on DB error. Headline AC: env_overrides carry BOTH TRINITY_MCP_API_KEY and TRINITY_MCP_URL, proven end-to-end through recreate_container_with_updated_config on an old container that had NEITHER, plus a source-order guard that overrides are applied LAST. Probe: ok/foreign_user_key/foreign_agent_key/unknown_key/not_configured/shadow_entry, stopped container -> unavailable not 500, only digests leave the container (no token, no .mcp.json body) plus a non-vacuous SOURCE guard that no future edit of the in-container script can start leaking one. Health `stale` fires when last_used_at predates recent executions; trinity-system reports exempt not a false missing. Rotation: system/ghost 409 BEFORE any mutation, Redis-down 503 (fail-CLOSED) and contention 409, stopped agent stays stopped on the DB-only path, post-removal failure keeps superseded keys and names the real state with no raw exception text, 409-adoption post-condition blocks deletion, clear_agent_breakers before the recreate, no plaintext or key hash in response/log/audit, rate limit 429. Plus the FR-7 guards on MCP key revoke/delete and connector key mint. Review additions: the self-heal BODY is pinned — it takes the SAME agent:mcp_key_regen lock as rotation and does NOTHING (no mint, no delete) when Redis is down or another heal/rotation holds it, because there is no per-agent start lock and an unserialised second heal deletes the key the first is about to bake in; it audits its own mint+DELETE; container-env health (env_absent/env_mismatch) outranks the usage-derived states and is fail-soft; `stale` survives a naive legacy timestamp; the 409-adoption post-condition is an EXACT constant-time key match, proven against a container carrying a same-20-char-prefix twin; and any non-`agent` scope (connector, portal_delegate, a future sixth) reads as foreign, never `unknown_key`." + "description": "Agent MCP key detect/self-heal/rotate (#1854). DB: create_agent_mcp_api_key born ACTIVE (tables.py has no column default -> NULL reads as revoked and cannot even deserialize); captured-id delete (not `id != new_id`, so a racing recreate_missing_container mint is not collateral), connector key survives, keep_id never removed, other agents unreachable; DELETE-not-deactivate proven durable across soft-delete -> recover (recover_agent_ownership reactivates every inactive per-agent row). spawned_by_key_id reconcile: `!= current` repairs a child stranded on an OLDER superseded id, leaves a foreign parent alone, idempotent, and enforce_agent_spawn_scope never 403s at any step of a rotation. Drift predicate: absent env, URL-missing (injection needs BOTH), hash-mismatch, system/ghost exempt, fail-safe on DB error. Headline AC: env_overrides carry BOTH TRINITY_MCP_API_KEY and TRINITY_MCP_URL, proven end-to-end through recreate_container_with_updated_config on an old container that had NEITHER, plus a source-order guard that overrides are applied LAST. Probe: ok/foreign_user_key/foreign_agent_key/unknown_key/not_configured/shadow_entry, stopped container -> unavailable not 500, only digests leave the container (no token, no .mcp.json body) plus a non-vacuous SOURCE guard that no future edit of the in-container script can start leaking one. Health `stale` fires when last_used_at predates recent executions; trinity-system reports exempt not a false missing. Rotation: system/ghost 409 BEFORE any mutation, Redis-down 503 (fail-CLOSED) and contention 409, stopped agent stays stopped on the DB-only path, post-removal failure keeps superseded keys and names the real state with no raw exception text, 409-adoption post-condition blocks deletion, clear_agent_breakers before the recreate, no plaintext or key hash in response/log/audit, rate limit 429. Plus the FR-7 guards on MCP key revoke/delete and connector key mint. Review additions: the self-heal BODY is pinned \u2014 it takes the SAME agent:mcp_key_regen lock as rotation and does NOTHING (no mint, no delete) when Redis is down or another heal/rotation holds it, because there is no per-agent start lock and an unserialised second heal deletes the key the first is about to bake in; it audits its own mint+DELETE; container-env health (env_absent/env_mismatch) outranks the usage-derived states and is fail-soft; `stale` survives a naive legacy timestamp; the 409-adoption post-condition is an EXACT constant-time key match, proven against a container carrying a same-20-char-prefix twin; and any non-`agent` scope (connector, portal_delegate, a future sixth) reads as foreign, never `unknown_key`." }, { "file": "unit/test_ent313_failed_creation_container_reclaim.py", @@ -1409,7 +1409,7 @@ "unit", "reliability" ], - "description": "Failed-creation container reclaim (ent#313): a create that fails after containers.run must leave no orphan container, no unreclaimable volume and no name-keyed Redis state. Covers both arrival shapes — handle in hand (later step raised) and NO handle (the reported 60s Docker read timeout, where the daemon created the container and the client never got it, so it is re-derived by name). Pins every fail-closed gate on that re-derived path, because a name resolves to another install's live agent on a shared Docker daemon: 409 name conflict (daemon created nothing) → refuse; existing agent_ownership row (a concurrent creation won the name) → refuse; trinity.created label older than the attempt's floor, missing, or unparseable → refuse; missing floor → refuse; DB or Docker lookup error → refuse. Also: Redis cleared when the container is provably gone (removed or never created) and NOT when removal failed (the slot ZSET isn't idle); every failure swallowed so the original creation error is what the caller reports; and an AST guard that create_agent_internal actually AWAITS the reclaim, so the unit tests can't stay green while nothing calls it. Also pins the never-raises contract under a stubbed docker module: the first version used isinstance(exc, docker.errors.APIError), which raises TypeError wherever docker is a test double and replaced the real creation error with an unrelated one (caught by the full suite via test_fork_to_own / test_1484); the 409 check is duck-typed on .response.status_code now." + "description": "Failed-creation container reclaim (ent#313): a create that fails after containers.run must leave no orphan container, no unreclaimable volume and no name-keyed Redis state. Covers both arrival shapes \u2014 handle in hand (later step raised) and NO handle (the reported 60s Docker read timeout, where the daemon created the container and the client never got it, so it is re-derived by name). Pins every fail-closed gate on that re-derived path, because a name resolves to another install's live agent on a shared Docker daemon: 409 name conflict (daemon created nothing) \u2192 refuse; existing agent_ownership row (a concurrent creation won the name) \u2192 refuse; trinity.created label older than the attempt's floor, missing, or unparseable \u2192 refuse; missing floor \u2192 refuse; DB or Docker lookup error \u2192 refuse. Also: Redis cleared when the container is provably gone (removed or never created) and NOT when removal failed (the slot ZSET isn't idle); every failure swallowed so the original creation error is what the caller reports; and an AST guard that create_agent_internal actually AWAITS the reclaim, so the unit tests can't stay green while nothing calls it. Also pins the never-raises contract under a stubbed docker module: the first version used isinstance(exc, docker.errors.APIError), which raises TypeError wherever docker is a test double and replaced the real creation error with an unrelated one (caught by the full suite via test_fork_to_own / test_1484); the 409 check is duck-typed on .response.status_code now." }, { "file": "unit/test_1951_slack_inbound_media.py", @@ -1420,7 +1420,7 @@ "unit", "security" ], - "description": "Slack inbound file-download SSRF gate (#1951) — the third channel to get the #1932 two-tier treatment. download_file previously fetched url_private_download with no host allowlist, no scheme check and follow_redirects=True. Covers: the legitimate files.slack.com download still succeeds (the #1932 lesson — an allowlist that rejects the real host reads as 'allowlist present' while being 100% broken); a validated CDN redirect is followed WITHOUT the bot token; hop-1 refusals for unrelated/lookalike/apex-lookalike hosts, the dotless-suffix bypass (evil-slack-files.com), plaintext http, file://, loopback and a platform-network address; off-domain and downgrade redirects refused after exactly one hop; cloud metadata (169.254.169.254) as a redirect target; bounded redirect budget; refusal logged at ERROR not WARNING; source tier proven narrower than the redirect tier; and a static backstop that follow_redirects=True never returns. EVERY negative test asserts the recorded call log, not `is None` — download_file returns None on every failure path and its bare except swallows AssertionError, so a return-value assertion passes against a completely unpatched seam." + "description": "Slack inbound file-download SSRF gate (#1951) \u2014 the third channel to get the #1932 two-tier treatment. download_file previously fetched url_private_download with no host allowlist, no scheme check and follow_redirects=True. Covers: the legitimate files.slack.com download still succeeds (the #1932 lesson \u2014 an allowlist that rejects the real host reads as 'allowlist present' while being 100% broken); a validated CDN redirect is followed WITHOUT the bot token; hop-1 refusals for unrelated/lookalike/apex-lookalike hosts, the dotless-suffix bypass (evil-slack-files.com), plaintext http, file://, loopback and a platform-network address; off-domain and downgrade redirects refused after exactly one hop; cloud metadata (169.254.169.254) as a redirect target; bounded redirect budget; refusal logged at ERROR not WARNING; source tier proven narrower than the redirect tier; and a static backstop that follow_redirects=True never returns. EVERY negative test asserts the recorded call log, not `is None` \u2014 download_file returns None on every failure path and its bare except swallows AssertionError, so a return-value assertion passes against a completely unpatched seam." }, { "file": "unit/test_1917_stack_trace_exposure.py", @@ -1431,7 +1431,7 @@ "unit", "security" ], - "description": "py/stack-trace-exposure leak guards for the ops / system-agent siblings PR #1912 left untouched (#1917), including open CodeQL alert #231. Each test drives the REAL router function with a sentinel planted in the exception message and asserts both halves: the sentinel is absent from the response AND the exception class name is present (a response that dropped the error entirely would pass a sentinel-only check while making the failure undiagnosable). Covers stop_fleet per-agent, _stop_agent_container (feeds emergency_stop), fleet health (previously str(e)[:50] — truncation bounds a leak, it does not remove one, and docker/httpx messages put the host FIRST), ops costs (the OTel collector URL), and system-agent health. Plus a static ban on str(e) in ops.py/system_agent.py and a window check on agents.py's three lifecycle details. All 8 fail against pre-fix code — verified, because the fleet-health test initially patched a get_agent_context_info that does not exist (raising=False made it silent), so it passed against the unpatched router until the pre-fix run exposed it; it now patches get_agent_client and asserts the probe ran." + "description": "py/stack-trace-exposure leak guards for the ops / system-agent siblings PR #1912 left untouched (#1917), including open CodeQL alert #231. Each test drives the REAL router function with a sentinel planted in the exception message and asserts both halves: the sentinel is absent from the response AND the exception class name is present (a response that dropped the error entirely would pass a sentinel-only check while making the failure undiagnosable). Covers stop_fleet per-agent, _stop_agent_container (feeds emergency_stop), fleet health (previously str(e)[:50] \u2014 truncation bounds a leak, it does not remove one, and docker/httpx messages put the host FIRST), ops costs (the OTel collector URL), and system-agent health. Plus a static ban on str(e) in ops.py/system_agent.py and a window check on agents.py's three lifecycle details. All 8 fail against pre-fix code \u2014 verified, because the fleet-health test initially patched a get_agent_context_info that does not exist (raising=False made it silent), so it passed against the unpatched router until the pre-fix run exposed it; it now patches get_agent_client and asserts the probe ran." }, { "file": "unit/test_ent314_hardened_yaml.py", @@ -1442,7 +1442,7 @@ "unit", "security" ], - "description": "Hardened parsing of author-controlled YAML (ent#314). template.yaml was on bare yaml.safe_load, leaving alias amplification (measured 416 B -> 110 MB at json.dumps time, parse itself 0.0011 s, so an input-size cap cannot close it) and silent last-wins duplicate keys, which let a template show one credentials: block to a human and declare another to Trinity. Reachable by any creator-role user via a public repo (ent#123 tokenless) since _build_template runs unfenced inside get_all_templates(). Covers: the fixture itself still amplifies under bare safe_load (so the suite cannot pass vacuously); level 4/5/6 bombs rejected under both policies; a small honest anchor STILL parses under BUDGET (the #1932 lesson — a guard that rejects the legitimate document is an outage that reads as hardened); REJECT refuses one alias and also gates at the scanner (skill_packaging's copy had both hooks); the issue's exact duplicate-credentials case and a nested duplicate; size cap; reject-not-truncate; ManifestError still a subclass so routers/systems.py's named 400 is unchanged and the manifest's published codes survive; and two consolidation guards — no module may grow a fourth SafeLoader subclass, and all four author-controlled readers must call the shared loader with no bare safe_load left. The four product-level guards fail against pre-fix code." + "description": "Hardened parsing of author-controlled YAML (ent#314). template.yaml was on bare yaml.safe_load, leaving alias amplification (measured 416 B -> 110 MB at json.dumps time, parse itself 0.0011 s, so an input-size cap cannot close it) and silent last-wins duplicate keys, which let a template show one credentials: block to a human and declare another to Trinity. Reachable by any creator-role user via a public repo (ent#123 tokenless) since _build_template runs unfenced inside get_all_templates(). Covers: the fixture itself still amplifies under bare safe_load (so the suite cannot pass vacuously); level 4/5/6 bombs rejected under both policies; a small honest anchor STILL parses under BUDGET (the #1932 lesson \u2014 a guard that rejects the legitimate document is an outage that reads as hardened); REJECT refuses one alias and also gates at the scanner (skill_packaging's copy had both hooks); the issue's exact duplicate-credentials case and a nested duplicate; size cap; reject-not-truncate; ManifestError still a subclass so routers/systems.py's named 400 is unchanged and the manifest's published codes survive; and two consolidation guards \u2014 no module may grow a fourth SafeLoader subclass, and all four author-controlled readers must call the shared loader with no bare safe_load left. The four product-level guards fail against pre-fix code." }, { "file": "unit/test_1969_lock_denied_tick_audit.py", @@ -1553,6 +1553,17 @@ "parity" ], "description": "The agent server was outside ent#314's YAML sweep (#1965). utils/safe_yaml.py (PR #1961) put every author-controlled YAML reader in the backend behind one hardened loader, and its AST guard walks the whole backend with an EMPTY allowlist - but it walked _BACKEND.rglob only, so docker/base-image/agent_server/ kept six bare yaml.safe_load calls on documents the backend itself assigns REJECT: template.yaml (x2, credential_requirements_service), skill frontmatter (skill_packaging), dashboard.yaml (compatibility/static_checks) and .trinity/persistent-state.yaml. The vector is amplification at SERIALIZATION, not parse - a 416 B level-6 anchor bomb resolves in ~0.001 s and blows up to ~110 MB when something walks the graph - and the backend proxies /info and /dashboard, so the walk happens in-container then again across the wire. Covers: byte-parity of the vendored loader (the credential_paths.py shape, Invariant #5) plus proof the vendored COPY actually behaves - refuses a level-6 bomb under BUDGET, any alias under REJECT, duplicate keys, and still parses an honest document (byte parity is not behaviour parity if the file never imports); each of the four agent-authored sites on the shared loader with its backend counterpart's kind AND policy; /config/agent-config.yaml deliberately BUDGET not REJECT, stated as an exception because the platform writes it and bind-mounts it mode:'ro' so the agent cannot author it, and yaml.dump emits an anchor for any shared object reference - REJECT there would be a self-inflicted outage for no security; no bare safe_load left anywhere in the tree; and HardenedYamlError named in the except arms that previously caught only yaml.YAMLError (it is a ValueError, so without its own arm a refused bomb escapes to the generic handler and surfaces as the unnamed 500 the AC rules out - the trap static_checks._parse_yaml records backend-side). AC #4 end-to-end: a level-6 bomb in a container's template.yaml is refused by BOTH template.yaml readers with the expanded graph never reaching the response, an honest template still serves, and the metrics assertion checks the NAMED refusal rather than has_metrics:False - a bomb parses fine under bare safe_load and yields no metrics: key, so the flag alone passes against the very tree this issue reports. The AST-guard widening itself lives in test_ent314_hardened_yaml.py (both trees, still empty allowlist) rather than here, because splitting a guard across two files is how the second copy stops being run." + }, + { + "file": "unit/test_1896_integration_nightly_workflow.py", + "feature": "#1896", + "added": "2026-08-03", + "categories": [ + "ci", + "unit", + "guard" + ], + "description": "Static guards on integration-nightly.yml (#1896), the CI home for the live-instance root suite (~93 files needing a running Trinity; nothing ran them, so they gated nothing). Pins the properties whose absence is invisible until the night it matters: membership is DIRECTORY-based (--ignore=unit, matching tests/run-core.sh) so #1895's relocations shrink it with no workflow edit and no file list to drift; base and head both run for attribution; a missing JUnit fails the job LOUDLY instead of being reported to the author as a regression (the diff script is fail-closed on missing XML, so a failed stack boot would otherwise wear a PR's name \u2014 #1941's class); each side gets a pristine stack via `down -v` since the suite mutates its instance; the merge is full-depth with the conflict verdict gated on unmerged paths (#1941 again); the job running untrusted PR code holds only contents:read while the write-scoped comment job never checks out PR code; it is not on the pull_request path; the runtime budget and its escalation are written down; --continue-on-collection-errors is present (measured: 3 root modules currently fail to import, and without the flag pytest writes no XML at all); and src/cli is installed, since test_cli_*.py imports trinity_cli which is deliberately absent from requirements-test.txt." } ] } diff --git a/tests/unit/test_1896_integration_nightly_workflow.py b/tests/unit/test_1896_integration_nightly_workflow.py new file mode 100644 index 000000000..5f41e88cb --- /dev/null +++ b/tests/unit/test_1896_integration_nightly_workflow.py @@ -0,0 +1,200 @@ +"""#1896 — the live-instance suite has a CI home, and it stays honest. + +~93 of the `test_*.py` files in `tests/` root need a RUNNING Trinity (the +fixtures create and delete real agents against `TRINITY_API_URL`), and no CI +job ran them, so they gated nothing. `integration-nightly.yml` is that home. + +These are static guards on the workflow, in the shape of +`test_1941_nightly_merge_depth.py` — the properties below are ones whose +absence is invisible until the night it matters: + +* the suite membership is DIRECTORY-based, so #1895's relocations shrink it + automatically and nobody has to remember to edit a file list; +* the job that runs untrusted PR code holds no write token; +* an infrastructure failure cannot be reported to an author as "your PR + introduced regressions" (#1941's class, which this workflow could reproduce + through the diff script's fail-closed-on-missing-XML behaviour); +* the merge is not shallow (#1941 again — the same trap, a second workflow). +""" + +from __future__ import annotations + +from pathlib import Path + +import pytest + +yaml = pytest.importorskip("yaml") + +WORKFLOW = ( + Path(__file__).resolve().parents[2] + / ".github" + / "workflows" + / "integration-nightly.yml" +) + + +def _doc(): + return yaml.safe_load(WORKFLOW.read_text(encoding="utf-8")) + + +def _test_job(): + return _doc()["jobs"]["test"] + + +def _runs(job) -> str: + return "\n".join( + str(s.get("run", "")) for s in job["steps"] if isinstance(s, dict) + ) + + +def _commands(text: str) -> str: + """Shell body minus comment lines — assert on what executes, not on prose.""" + return "\n".join( + line for line in text.splitlines() if not line.strip().startswith("#") + ) + + +def test_the_workflow_exists_and_parses(): + assert WORKFLOW.exists(), "the live suite lost its CI home again (#1896)" + assert _doc()["jobs"].keys() >= {"discover", "test", "comment"} + + +def test_membership_is_directory_based_not_a_file_list(): + """The AC that keeps this from rotting: `--ignore=unit`, exactly as + `tests/run-core.sh` defines the same suite. A hand-listed set would drift + the moment #1895 moves a file, and the drift is silent.""" + cmds = _commands(_runs(_test_job())) + assert "--ignore=unit" in cmds + assert "--ignore=process_engine" in cmds + assert "pytest" in cmds + # No per-file enumeration: a `test_*.py` literal in the invocation would + # mean the membership is hand-maintained again. + import re + + assert not re.search(r"test_\w+\.py", cmds), ( + "the suite is being enumerated file by file; keep it directory-based" + ) + + +def test_the_suite_runs_on_both_sides_for_attribution(): + """AC 3: base and head, so a pre-existing failure does not read as new.""" + cmds = _commands(_runs(_test_job())) + assert "run_side base" in cmds and "run_side head" in cmds + assert "diff-pytest-failures.py" in cmds + + +def test_missing_junit_fails_the_job_instead_of_blaming_the_pr(): + """`diff-pytest-failures.py` is fail-closed on a missing input XML, so a + stack that never booted would otherwise surface to the author as "this PR + introduced regressions" — an infrastructure failure wearing a PR's name, + which is exactly what #1941 was filed about.""" + cmds = _commands(_runs(_test_job())) + assert "-s \"$xml\"" in cmds or "! -s \"$xml\"" in cmds, ( + "the JUnit-existence check is gone; a failed boot would be reported as " + "a regression (#1896 / #1941)" + ) + assert "Not reporting a regression" in _runs(_test_job()) + + +def test_collection_errors_do_not_zero_out_the_run(): + """Measured, not defensive: the root suite currently has 3 modules that fail + to IMPORT. Without `--continue-on-collection-errors` pytest aborts the whole + session ("Interrupted: N errors during collection") and writes NO JUnit XML, + so this job would fail every night for every PR and report nothing. With it, + the errors land on both sides and cancel in the diff, while a NEW collection + error a PR introduces still surfaces.""" + cmds = _commands(_runs(_test_job())) + assert "--continue-on-collection-errors" in cmds + + +def test_the_cli_package_is_installed_for_test_cli_modules(): + """`test_cli_*.py` imports `trinity_cli`, which is deliberately kept out of + requirements-test.txt (an `-e ./src/cli` line there breaks GitHub's + dependency-graph updater — tests/setup-env.sh explains it). Without the + on-demand install those modules fail to import and their tests never run, + which is the same silent-no-coverage this issue exists to end.""" + cmds = _commands(_runs(_test_job())) + assert "src/cli" in cmds + + +def test_each_side_gets_a_pristine_stack(): + """AC 4: the suite MUTATES its instance, so the head side must not inherit + agents, volumes or a dirty DB from the base side.""" + cmds = _commands(_runs(_test_job())) + assert "docker compose down -v" in cmds, ( + "without `-v` the second side inherits the first side's volumes" + ) + + +def test_the_merge_is_not_shallow(): + """#1941, in a second workflow: two shallow fetches share no ancestor, and + every PR reads as conflicting.""" + job = _test_job() + checkouts = [ + s for s in job["steps"] + if isinstance(s, dict) and str(s.get("uses", "")).startswith("actions/checkout") + ] + assert checkouts, "the merge job no longer checks out a base" + for step in checkouts: + assert (step.get("with") or {}).get("fetch-depth") == 0 + + for line in _commands(_runs(job)).splitlines(): + if "git fetch" in line and "pull/" in line: + assert "--depth" not in line, f"shallow PR-head fetch: {line.strip()}" + + +def test_conflict_verdict_requires_unmerged_paths(): + """Same rule as #1941: a non-zero `git merge` is not proof of a conflict.""" + cmds = _commands(_runs(_test_job())) + assert "merge_conflict=true" in cmds + before = cmds.split("merge_conflict=true")[0] + assert "--diff-filter=U" in before + + +def test_the_job_running_pr_code_has_no_write_token(): + """The inherited security architecture: untrusted PR code must never share + a job with a write-capable token. `comment` holds the write scope and never + checks out PR code.""" + test_perms = _test_job().get("permissions") or {} + assert test_perms == {"contents": "read"}, ( + f"the PR-code job gained scope beyond contents:read: {test_perms}" + ) + + comment = _doc()["jobs"]["comment"] + steps = str(comment["steps"]) + assert "actions/checkout" not in steps, ( + "the write-scoped comment job must not check out PR code" + ) + + +def test_it_is_not_wired_into_the_pull_request_merge_path(): + """AC 2: nightly, not per-PR. A mutating ~93-file suite in the merge path + is a cost and a flake source; `workflow_dispatch` keeps it runnable on + demand for a single PR.""" + triggers = _doc()[True] if True in _doc() else _doc()["on"] + assert "schedule" in triggers + assert "workflow_dispatch" in triggers + assert "pull_request" not in triggers + + +def test_runtime_is_bounded_and_the_escalation_is_written_down(): + """AC 5: a stated budget, and a documented answer for exceeding it — so the + next person raises the ceiling only after sharding, not instead of it.""" + job = _test_job() + assert isinstance(job.get("timeout-minutes"), int) + assert job["timeout-minutes"] <= 60, "an unbounded mutating nightly is the bug" + header = WORKFLOW.read_text(encoding="utf-8") + assert "RUNTIME BUDGET" in header + assert "shard" in header.lower() + + +def test_local_workflow_is_untouched(): + """AC 6: `tests/run-core.sh` remains the supported local path. This issue + adds a CI home; it does not migrate the local one.""" + run_core = WORKFLOW.parents[2] / "tests" / "run-core.sh" + assert run_core.exists() + body = run_core.read_text(encoding="utf-8") + assert "--ignore=unit" in body, ( + "run-core.sh no longer defines the same suite the nightly runs — the " + "two must agree or the local path stops reproducing CI (#1896)" + ) From d7938b6275bec0717a96213e1a1ffc7de40c61c6 Mon Sep 17 00:00:00 2001 From: Oleksii Dolhov Date: Mon, 3 Aug 2026 17:55:33 +0300 Subject: [PATCH 2/4] ci: pin the nightly integration job to the image's Python (#1891) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The new workflow shipped `python-version: '3.11'` while every image is `FROM python:3.13`. #1891's parity guard caught it before merge — and this is exactly the case its docstring predicts: "a seventh workflow added next month with a stale pin has to fail this test, or the guard only documents today's drift instead of closing the class." It matters more here than in a unit job: this suite boots real containers, so running the client on 3.11 against 3.13 services reintroduces the stdlib-removal blind spot (`crypt`, `audioop`) that #1891 exists to close. Related to #1896 --- .github/workflows/integration-nightly.yml | 8 ++- tests/registry.json | 80 +++++++++++------------ 2 files changed, 47 insertions(+), 41 deletions(-) diff --git a/.github/workflows/integration-nightly.yml b/.github/workflows/integration-nightly.yml index 6c4d0a773..5c7c0eedf 100644 --- a/.github/workflows/integration-nightly.yml +++ b/.github/workflows/integration-nightly.yml @@ -129,7 +129,13 @@ jobs: - uses: actions/setup-python@v7 with: - python-version: '3.11' + # Must equal the `FROM python:` pin in the image Dockerfiles (#1891) — + # a suite validated on an older minor cannot catch the stdlib-removal + # class of defect that has already shipped twice here. Guarded by + # tests/unit/test_1891_python_version_parity.py, which scans EVERY + # workflow precisely so a newly added one cannot drift in unnoticed. + # It caught this file at '3.11' before merge. + python-version: '3.13' cache: 'pip' cache-dependency-path: tests/requirements-test.txt diff --git a/tests/registry.json b/tests/registry.json index dff521d02..a4f5bd9ef 100644 --- a/tests/registry.json +++ b/tests/registry.json @@ -60,7 +60,7 @@ "brain-orb", "feature-flags" ], - "description": "Admin-configurable Brain Orb flags (trinity-enterprise#85): _resolve_bool_flag order (stored wins both directions, env opt-in, default OFF, junk-value fail-safe, fail-open on DB error), GET/PUT /api/settings/brain-orb (per-flag source, partial update, clear revert-to-env, set+clear conflict 400, 403 non-admin, audit old\u2192new, /{key} route ordering), generic PUT/DELETE compatibility, brain-orb route gate honoring a real DB flip without restart, and feature-flags composition (voice = base \u2227 voice \u2227 key; 200 despite brain-orb DB failure)." + "description": "Admin-configurable Brain Orb flags (trinity-enterprise#85): _resolve_bool_flag order (stored wins both directions, env opt-in, default OFF, junk-value fail-safe, fail-open on DB error), GET/PUT /api/settings/brain-orb (per-flag source, partial update, clear revert-to-env, set+clear conflict 400, 403 non-admin, audit old→new, /{key} route ordering), generic PUT/DELETE compatibility, brain-orb route gate honoring a real DB flip without restart, and feature-flags composition (voice = base ∧ voice ∧ key; 200 despite brain-orb DB failure)." }, { "file": "unit/test_1332_cancelled_activity_state.py", @@ -72,7 +72,7 @@ "activities", "observability" ], - "description": "A user-cancelled execution's dispatch activity is recorded as ActivityState.CANCELLED, not FAILED. Covers the enum value + activity_state_for_terminal mapping helper, the operator-terminate handler (Path B) closing the open dispatch activity as CANCELLED (terminated\u2192close, already_finished\u2192no-op, no-activity\u2192no-op, close-raises\u2192swallowed), and the collaboration/self-task close helpers mapping a cancelled result to CANCELLED (#1332)." + "description": "A user-cancelled execution's dispatch activity is recorded as ActivityState.CANCELLED, not FAILED. Covers the enum value + activity_state_for_terminal mapping helper, the operator-terminate handler (Path B) closing the open dispatch activity as CANCELLED (terminated→close, already_finished→no-op, no-activity→no-op, close-raises→swallowed), and the collaboration/self-task close helpers mapping a cancelled result to CANCELLED (#1332)." }, { "file": "unit/test_voip_audio.py", @@ -84,7 +84,7 @@ "voip", "audio" ], - "description": "Audio codec round-trip + stateful ratecv continuity (anti-click) + 160-byte framing for the Twilio\u2194Gemini bridge (#1056). Skips where audioop/audioop-lts is unavailable." + "description": "Audio codec round-trip + stateful ratecv continuity (anti-click) + 160-byte framing for the Twilio↔Gemini bridge (#1056). Skips where audioop/audioop-lts is unavailable." }, { "file": "unit/test_voip_db.py", @@ -467,7 +467,7 @@ "lifecycle", "file-sharing" ], - "description": "check_public_folder_mount_matches truth table: enabled+mounted \u2192 True, enabled+unmounted \u2192 False (needs recreation to attach), disabled+mounted \u2192 False (needs recreation to detach), disabled+unmounted \u2192 True. Adversarial cases: similar paths (/public-backup, /public/inner) don't match, missing 'Mounts' key handled, flag re-read each call, other mounts (shared-out, shared-in/*, workspace) don't interfere (9 tests)" + "description": "check_public_folder_mount_matches truth table: enabled+mounted → True, enabled+unmounted → False (needs recreation to attach), disabled+mounted → False (needs recreation to detach), disabled+unmounted → True. Adversarial cases: similar paths (/public-backup, /public/inner) don't match, missing 'Mounts' key handled, flag re-read each call, other mounts (shared-out, shared-in/*, workspace) don't interfere (9 tests)" }, { "file": "unit/test_slack_dm_default.py", @@ -479,7 +479,7 @@ "db", "slack" ], - "description": "set_dm_default + unbind_agent contract: setter is single-tx clear-then-set, idempotent, exclusive (exactly one default per workspace), per-workspace isolation, returns False when agent not bound. Unbind is pure delete (does NOT auto-promote \u2014 router enforces the guard), works on non-default and last-agent paths, unknown agent returns False (10 tests)" + "description": "set_dm_default + unbind_agent contract: setter is single-tx clear-then-set, idempotent, exclusive (exactly one default per workspace), per-workspace isolation, returns False when agent not bound. Unbind is pure delete (does NOT auto-promote — router enforces the guard), works on non-default and last-agent paths, unknown agent returns False (10 tests)" }, { "file": "test_public_chat_history.py", @@ -491,7 +491,7 @@ "public", "chat" ], - "description": "Tests for GET /api/public/sessions/{token} and GET /api/public/sessions/{token}/{session_id} \u2014 auth requirements, 404 on invalid tokens, response shape, limit param" + "description": "Tests for GET /api/public/sessions/{token} and GET /api/public/sessions/{token}/{session_id} — auth requirements, 404 on invalid tokens, response shape, limit param" }, { "file": "unit/test_voice_tools.py", @@ -504,7 +504,7 @@ "gemini", "tools" ], - "description": "Unit tests for voice tool call support (#581): _execute_tool (success, empty prompt, truncation to 2000 chars, agent not reachable, task error), _execute_and_respond (success path with callbacks, timeout \u2192 error response, inactive session skips send), tool declaration (_RUN_TASK_TOOL name + required prompt), end_session cancels pending tool tasks (12 tests)" + "description": "Unit tests for voice tool call support (#581): _execute_tool (success, empty prompt, truncation to 2000 chars, agent not reachable, task error), _execute_and_respond (success path with callbacks, timeout → error response, inactive session skips send), tool declaration (_RUN_TASK_TOOL name + required prompt), end_session cancels pending tool tasks (12 tests)" }, { "file": "test_files_guardrail_bypass.py", @@ -543,7 +543,7 @@ "files", "chat" ], - "description": "Unit tests for web chat file upload (#364): sanitize_filename (path traversal, unicode, dedup, truncation), decode_web_file (data: URI prefix stripping, raw base64, empty/bad input), process_file_uploads (empty list, download failure, unsupported MIME, oversized file, image vision block collection, text file container write, max_files cap) \u2014 14 tests" + "description": "Unit tests for web chat file upload (#364): sanitize_filename (path traversal, unicode, dedup, truncation), decode_web_file (data: URI prefix stripping, raw base64, empty/bad input), process_file_uploads (empty list, download failure, unsupported MIME, oversized file, image vision block collection, text file container write, max_files cap) — 14 tests" }, { "file": "unit/test_slack_mrkdwn.py", @@ -569,7 +569,7 @@ "webhooks", "lint" ], - "description": "AST-based lint guard (no backend deps required) that asserts every db.(...) call in src/backend/routers/ and src/backend/services/ resolves to a real method on DatabaseManager. Two tests: strict regression check for the four WEBHOOK-001 methods (#647: generate_webhook_token, get_schedule_by_webhook_token, revoke_webhook_token, get_webhook_status) and a broad facade-resolution scan guarded by a KNOWN_FACADE_GAPS allowlist for eight unrelated pre-existing gaps. Catches AttributeError-at-runtime regressions that integration-only tests miss in CI \u2014 would have caught WEBHOOK-001 before #291 landed." + "description": "AST-based lint guard (no backend deps required) that asserts every db.(...) call in src/backend/routers/ and src/backend/services/ resolves to a real method on DatabaseManager. Two tests: strict regression check for the four WEBHOOK-001 methods (#647: generate_webhook_token, get_schedule_by_webhook_token, revoke_webhook_token, get_webhook_status) and a broad facade-resolution scan guarded by a KNOWN_FACADE_GAPS allowlist for eight unrelated pre-existing gaps. Catches AttributeError-at-runtime regressions that integration-only tests miss in CI — would have caught WEBHOOK-001 before #291 landed." }, { "file": "unit/test_slack_token_encryption.py", @@ -597,7 +597,7 @@ "credentials", "encryption" ], - "description": "Unit tests for Telegram bot token encryption in db/telegram_channels.py (#664): round-trip via TelegramChannelOperations.create_binding + get_decrypted_bot_token, raw DB value is AES-256-GCM JSON envelope, get_binding_by_agent returns the encrypted blob (plaintext only via accessor), corrupt envelope returns None with ERROR log, wrong-key envelope returns None, re-encryption on update produces a fresh nonce (and token rotation reflects the new plaintext), encrypt raises on missing CREDENTIAL_ENCRYPTION_KEY, decrypt swallows missing-key error and returns None. No plaintext fallback path \u2014 Telegram never shipped plaintext." + "description": "Unit tests for Telegram bot token encryption in db/telegram_channels.py (#664): round-trip via TelegramChannelOperations.create_binding + get_decrypted_bot_token, raw DB value is AES-256-GCM JSON envelope, get_binding_by_agent returns the encrypted blob (plaintext only via accessor), corrupt envelope returns None with ERROR log, wrong-key envelope returns None, re-encryption on update produces a fresh nonce (and token rotation reflects the new plaintext), encrypt raises on missing CREDENTIAL_ENCRYPTION_KEY, decrypt swallows missing-key error and returns None. No plaintext fallback path — Telegram never shipped plaintext." }, { "file": "unit/test_whatsapp_token_encryption.py", @@ -612,7 +612,7 @@ "credentials", "encryption" ], - "description": "Unit tests for Twilio AuthToken encryption in db/whatsapp_channels.py (#664): round-trip via WhatsAppChannelOperations.create_binding + get_decrypted_auth_token, raw DB value is AES-256-GCM JSON envelope, account_sid stays plaintext (public Twilio identifier \u2014 pinned so future refactors don't accidentally encrypt or strip it), corrupt envelope returns None, wrong-key envelope returns None, re-encryption on update produces a fresh nonce, encrypt raises on missing CREDENTIAL_ENCRYPTION_KEY, decrypt returns None on missing key." + "description": "Unit tests for Twilio AuthToken encryption in db/whatsapp_channels.py (#664): round-trip via WhatsAppChannelOperations.create_binding + get_decrypted_auth_token, raw DB value is AES-256-GCM JSON envelope, account_sid stays plaintext (public Twilio identifier — pinned so future refactors don't accidentally encrypt or strip it), corrupt envelope returns None, wrong-key envelope returns None, re-encryption on update produces a fresh nonce, encrypt raises on missing CREDENTIAL_ENCRYPTION_KEY, decrypt returns None on missing key." }, { "file": "unit/test_slack_workspaces_encryption.py", @@ -671,7 +671,7 @@ "resilience", "circuit-breaker" ], - "description": "Tests that the AgentClient circuit breaker only counts TCP unreachability (connect errors/timeouts) toward opening \u2014 HTTP errors, read/write/pool failures, and protocol errors are excluded" + "description": "Tests that the AgentClient circuit breaker only counts TCP unreachability (connect errors/timeouts) toward opening — HTTP errors, read/write/pool failures, and protocol errors are excluded" }, { "file": "integration/test_circuit_breaker.py", @@ -685,7 +685,7 @@ "circuit-breaker", "redis" ], - "description": "Drives the real AgentClient through httpx.MockTransport against real Redis circuit state \u2014 hard vs soft failures, mixed interleave, open-circuit fast-fail, recovery on 200, and the deferred half-open soft-failure probe-lock behaviour" + "description": "Drives the real AgentClient through httpx.MockTransport against real Redis circuit state — hard vs soft failures, mixed interleave, open-circuit fast-fail, recovery on 200, and the deferred half-open soft-failure probe-lock behaviour" }, { "file": "integration/test_monitoring_service.py", @@ -698,7 +698,7 @@ "monitoring", "circuit-breaker" ], - "description": "Mirrors the #474 classification rule on the /health probe path and pins the `status_code >= 500 \u2192 UNHEALTHY` aggregator branch" + "description": "Mirrors the #474 classification rule on the /health probe path and pins the `status_code >= 500 → UNHEALTHY` aggregator branch" }, { "file": "unit/test_platform_default_model_regression.py", @@ -769,7 +769,7 @@ "migrations", "reliability" ], - "description": "Migration runner atomicity + cross-process lock (#1160). _atomic_rebuild rename-swap inside an explicit transaction closes the DROP-rebuild data-loss window (crash mid-rebuild rolls back with no data loss; verified for agent_sharing + agent_skills incl. lowercase/NULL-email behavior preservation and index recreation); replay-after-partial-apply completes cleanly; a failed migration is named on its traceback via add_note (original exception type preserved). Cross-process flock (db/migration_lock.py) serialises concurrent boots \u2014 multiprocessing tests prove mutual exclusion and a safe concurrent rebuild." + "description": "Migration runner atomicity + cross-process lock (#1160). _atomic_rebuild rename-swap inside an explicit transaction closes the DROP-rebuild data-loss window (crash mid-rebuild rolls back with no data loss; verified for agent_sharing + agent_skills incl. lowercase/NULL-email behavior preservation and index recreation); replay-after-partial-apply completes cleanly; a failed migration is named on its traceback via add_note (original exception type preserved). Cross-process flock (db/migration_lock.py) serialises concurrent boots — multiprocessing tests prove mutual exclusion and a safe concurrent rebuild." }, { "file": "unit/test_compatibility_checks.py", @@ -794,7 +794,7 @@ "voip", "voice" ], - "description": "Per-agent persisted voice + VoIP enable/disable toggle (#28). db/agents.py get_voice_name/set_voice_name: unset->'Kore' fallback, set/get roundtrip, invalid-persisted-value->default (reviewer M1 read-path validation), clear->default, set-on-missing-agent->False. db/voip.py set_enabled toggle + the create_binding 'preserve enabled on re-PUT' fix (reviewer H3 \u2014 re-saving credentials on a disabled binding must not silently re-enable it) + set_enabled on a missing binding returns False (router 404s). config.GEMINI_VOICE_NAMES default + parity guard asserting the frontend src/constants/voices.js VOICE ids and DEFAULT_VOICE_NAME mirror the backend constants (reviewer M2 cross-language drift guard). Runs on db_harness backends (SQLite always, PostgreSQL when TEST_POSTGRES_URL set); no Docker/API/Redis." + "description": "Per-agent persisted voice + VoIP enable/disable toggle (#28). db/agents.py get_voice_name/set_voice_name: unset->'Kore' fallback, set/get roundtrip, invalid-persisted-value->default (reviewer M1 read-path validation), clear->default, set-on-missing-agent->False. db/voip.py set_enabled toggle + the create_binding 'preserve enabled on re-PUT' fix (reviewer H3 — re-saving credentials on a disabled binding must not silently re-enable it) + set_enabled on a missing binding returns False (router 404s). config.GEMINI_VOICE_NAMES default + parity guard asserting the frontend src/constants/voices.js VOICE ids and DEFAULT_VOICE_NAME mirror the backend constants (reviewer M2 cross-language drift guard). Runs on db_harness backends (SQLite always, PostgreSQL when TEST_POSTGRES_URL set); no Docker/API/Redis." }, { "file": "unit/test_28_voip_voice_endpoints.py", @@ -878,7 +878,7 @@ "infrastructure", "brain-orb" ], - "description": "Cornelius first-run seeder (ent#107): ensure_seeded first-run gating via durable cornelius_seeded flag (deleted agent not resurrected); fresh-install scoping (existing non-system agents -> skip + converge flag, orb flag untouched); owner-must-exist deferral (pre-setup skip without burning the flag); Docker-unavailable skip; Brain Orb flag defaulted ON existence-guarded (admin OFF preserved); 409-on-exists convergence vs generic-failure retry (flag not burned, never raises); --workers 2 Redis SETNX provision lock (held -> skip, winner provisions+releases with TTL, Redis-down fail-open); _provision builds a github:Abilityai/cornelius create with request=None and source_mode default True (#1656 \u2014 the trinity-enterprise#123 tokenless public-clone path is source-mode only); real-DB smoke of db.count_non_system_agents() facade delegation. create_agent_internal/db/redis/docker seams patched; no Docker/backend. ent#124 addition: precomputed freshness verdict \u2014 ensure_seeded(fresh=True) skips the internal count, fresh=False converges the flag without provisioning, fresh=None preserves the legacy count path." + "description": "Cornelius first-run seeder (ent#107): ensure_seeded first-run gating via durable cornelius_seeded flag (deleted agent not resurrected); fresh-install scoping (existing non-system agents -> skip + converge flag, orb flag untouched); owner-must-exist deferral (pre-setup skip without burning the flag); Docker-unavailable skip; Brain Orb flag defaulted ON existence-guarded (admin OFF preserved); 409-on-exists convergence vs generic-failure retry (flag not burned, never raises); --workers 2 Redis SETNX provision lock (held -> skip, winner provisions+releases with TTL, Redis-down fail-open); _provision builds a github:Abilityai/cornelius create with request=None and source_mode default True (#1656 — the trinity-enterprise#123 tokenless public-clone path is source-mode only); real-DB smoke of db.count_non_system_agents() facade delegation. create_agent_internal/db/redis/docker seams patched; no Docker/backend. ent#124 addition: precomputed freshness verdict — ensure_seeded(fresh=True) skips the internal count, fresh=False converges the flag without provisioning, fresh=None preserves the legacy count path." }, { "file": "unit/test_1557_autonomy_breaker_decoupled.py", @@ -890,7 +890,7 @@ "circuit-breaker", "autonomy" ], - "description": "Disabling autonomy must not touch the circuit breaker (#1557). Structural guards: autonomy.py no longer references force_circuit_dormant/reset_circuit (regression guard \u2014 fails on pre-#1557 source) and still calls set_schedule_enabled (proactive suppression intact). Message honesty: _circuit_breaker_error names transport-unreachable vs dispatch-auth-dead, and every branch keeps the 'circuit breaker open' substring pinned by the #1560 integration test." + "description": "Disabling autonomy must not touch the circuit breaker (#1557). Structural guards: autonomy.py no longer references force_circuit_dormant/reset_circuit (regression guard — fails on pre-#1557 source) and still calls set_schedule_enabled (proactive suppression intact). Message honesty: _circuit_breaker_error names transport-unreachable vs dispatch-auth-dead, and every branch keeps the 'circuit breaker open' substring pinned by the #1560 integration test." }, { "file": "integration/test_1557_autonomy_inbound.py", @@ -951,7 +951,7 @@ "proactive-messaging", "regression" ], - "description": "Proactive messages persist to channel session history (#1600). Core property is session-identifier EQUALITY: the key the proactive path derives must equal the key an inbound DM resolves to, per channel (telegram/whatsapp/slack) \u2014 persisting into a different session looks like a fix but leaves the agent unaware of its own outreach. Also: #903 attribution (assistant role, agent sender_label, recipient sender_email for the single-participant DM), persist only on confirmed delivery (no phantom turn on failure), fail-soft on DB error (message already sent), session created when absent (the chat-link session_id column is never written by any code path), and access-grant notifications deliberately NOT persisted (#951 out of scope)." + "description": "Proactive messages persist to channel session history (#1600). Core property is session-identifier EQUALITY: the key the proactive path derives must equal the key an inbound DM resolves to, per channel (telegram/whatsapp/slack) — persisting into a different session looks like a fix but leaves the agent unaware of its own outreach. Also: #903 attribution (assistant role, agent sender_label, recipient sender_email for the single-participant DM), persist only on confirmed delivery (no phantom turn on failure), fail-soft on DB error (message already sent), session created when absent (the chat-link session_id column is never written by any code path), and access-grant notifications deliberately NOT persisted (#951 out of scope)." }, { "file": "unit/test_1649_group_message_history.py", @@ -963,7 +963,7 @@ "proactive-messaging", "regression" ], - "description": "Proactive GROUP messages persist to channel session history (#1649). Slack is a real recall fix: the broadcast is filed at the posted message's own ts, which IS the thread key an in-thread reply resolves to (asserted against the adapter). Telegram is bookkeeping only \u2014 group sessions are per-(sender, chat) with no group branch, so a broadcast uses a synthetic agent-sender key; a test pins that it deliberately does NOT match a participant's session, so the accepted trade-off can't be mistaken for a bug and a future adapter group-branch forces a re-decision. Router-level tests drive the real endpoints (the #1600 lesson: helper-only tests passed with the persistence call deleted outright). Also: #903 shared-thread attribution (sender_email=None), persist only on confirmed delivery, fail-soft, and send_message_detailed's ts capture keeping send_message's 2-tuple contract for its ~7 callers." + "description": "Proactive GROUP messages persist to channel session history (#1649). Slack is a real recall fix: the broadcast is filed at the posted message's own ts, which IS the thread key an in-thread reply resolves to (asserted against the adapter). Telegram is bookkeeping only — group sessions are per-(sender, chat) with no group branch, so a broadcast uses a synthetic agent-sender key; a test pins that it deliberately does NOT match a participant's session, so the accepted trade-off can't be mistaken for a bug and a future adapter group-branch forces a re-decision. Router-level tests drive the real endpoints (the #1600 lesson: helper-only tests passed with the persistence call deleted outright). Also: #903 shared-thread attribution (sender_email=None), persist only on confirmed delivery, fail-soft, and send_message_detailed's ts capture keeping send_message's 2-tuple contract for its ~7 callers." }, { "file": "unit/test_1632_operator_queue_caps.py", @@ -976,7 +976,7 @@ "security", "reliability" ], - "description": "Operator-queue create-path ingestion caps (#1632): the agent-authored sync boundary is bounded by a DB-measured pending-DEPTH cap (primary, Redis-independent), a per-agent + fleet RATE cap (fail-open, break on deny, real in-process-fallback exercise), a total truncate-with-marker field-hygiene clamp inside the #1525 try/except (title/question/context/options/execution_id/created_at/priority, non-dict context, boundary lengths, never-raises), the reserved-platform-id + malformed-id guard, the opqueue:leader cross-worker lock (non-leader poll cycle is a no-op), the one-per-episode flood alert (un-guessable id, cooldown, emit-failure-safe, platform-exempt), the oversize-file skip, and the generous DB-sink belt + count_pending_for_agent helper. Also pins the concrete platform producer validation_service._notify_operator_on_failure, converted to a direct db.create_operator_queue_item (#1632): reserved `val_` id (an agent can't pre-create and thereby suppress its own validation alarm), bypasses the agent-file sync caps, and stays best-effort (a create failure never fails validation) \u2014 the conversion also fixed a latent bug where the notification appended to a bare list and was never ingested. Also pins the _leader_ttl 30s floor (F1: lease outlasts one worst-case cycle + sleep, so leadership doesn't flap and the flood alert doesn't double-emit under --workers 2). Pure/mocked (rate_limiter stubbed + one real in-process test; DB engine stubbed)." + "description": "Operator-queue create-path ingestion caps (#1632): the agent-authored sync boundary is bounded by a DB-measured pending-DEPTH cap (primary, Redis-independent), a per-agent + fleet RATE cap (fail-open, break on deny, real in-process-fallback exercise), a total truncate-with-marker field-hygiene clamp inside the #1525 try/except (title/question/context/options/execution_id/created_at/priority, non-dict context, boundary lengths, never-raises), the reserved-platform-id + malformed-id guard, the opqueue:leader cross-worker lock (non-leader poll cycle is a no-op), the one-per-episode flood alert (un-guessable id, cooldown, emit-failure-safe, platform-exempt), the oversize-file skip, and the generous DB-sink belt + count_pending_for_agent helper. Also pins the concrete platform producer validation_service._notify_operator_on_failure, converted to a direct db.create_operator_queue_item (#1632): reserved `val_` id (an agent can't pre-create and thereby suppress its own validation alarm), bypasses the agent-file sync caps, and stays best-effort (a create failure never fails validation) — the conversion also fixed a latent bug where the notification appended to a bare list and was never ingested. Also pins the _leader_ttl 30s floor (F1: lease outlasts one worst-case cycle + sleep, so leadership doesn't flap and the flood alert doesn't double-emit under --workers 2). Pure/mocked (rate_limiter stubbed + one real in-process test; DB engine stubbed)." }, { "file": "unit/test_1615_ssh_password_removed.py", @@ -988,7 +988,7 @@ "security", "regression" ], - "description": "Password SSH auth removed (#1615, router surface): an explicit auth_method='password' returns 400 naming key auth as the alternative \u2014 not the pre-fix 500 (ModuleNotFoundError: crypt, removed from the stdlib in Python 3.13) and not a silent fall-through; refusal is case-insensitive and covers unknown methods; the request never reaches the container; key auth still works and remains the default; the response carries neither private_key (#175) nor password. Complements test_ssh_service.py, which guards the service layer (helpers deleted, no crypt import)." + "description": "Password SSH auth removed (#1615, router surface): an explicit auth_method='password' returns 400 naming key auth as the alternative — not the pre-fix 500 (ModuleNotFoundError: crypt, removed from the stdlib in Python 3.13) and not a silent fall-through; refusal is case-insensitive and covers unknown methods; the request never reaches the container; key auth still works and remains the default; the response carries neither private_key (#175) nor password. Complements test_ssh_service.py, which guards the service layer (helpers deleted, no crypt import)." }, { "file": "unit/test_ent162_per_user_github_pat.py", @@ -1014,7 +1014,7 @@ "live", "integration" ], - "description": "Live SSH-access API tests (#1615). Password auth is refused with 400 and never 500 (the pre-fix ModuleNotFoundError: crypt on Python 3.13) \u2014 including with a valid public_key supplied, which is what distinguishes the guard from an incidental missing-key 400; refusal is case-insensitive and covers unknown methods. Key auth (BYOK) returns connection details, is the default, clamps TTL, and leaks neither private_key (#175) nor password. Marked `integration`: an ed25519 key injected through the API is used for a REAL ssh login into the container (proves key auth works \u2014 mocks cannot), and a password login is proven impossible against the agent sshd's own PasswordAuthentication=no. sshd readiness is polled via the SSH banner so the e2e tests don't race container boot." + "description": "Live SSH-access API tests (#1615). Password auth is refused with 400 and never 500 (the pre-fix ModuleNotFoundError: crypt on Python 3.13) — including with a valid public_key supplied, which is what distinguishes the guard from an incidental missing-key 400; refusal is case-insensitive and covers unknown methods. Key auth (BYOK) returns connection details, is the default, clamps TTL, and leaks neither private_key (#175) nor password. Marked `integration`: an ed25519 key injected through the API is used for a REAL ssh login into the container (proves key auth works — mocks cannot), and a password login is proven impossible against the agent sshd's own PasswordAuthentication=no. sshd readiness is polled via the SSH banner so the e2e tests don't race container boot." }, { "file": "unit/test_1673_execution_error_not_success.py", @@ -1061,7 +1061,7 @@ "agents", "unit" ], - "description": "Per-agent display label (ent#181): a human-facing name that is rendered, never resolved \u2014 the slug (agent_name) stays the identity every route, container, volume, MCP key and A2A card keys on. Pins the point of the feature: setting a label leaves the slug AND its #1664 volume identity untouched, does not reserve a name, and two agents may share one (labels aren't identities). NULL = render the slug (no backfill; clearing reverts rather than blanking); a blank/whitespace label stores NULL, not an empty string that would render a nameless agent. Soft-deleted agents are not editable (deleted_at guard, mirroring the other settings setters). Batch read for the fleet list (absent = no label = slug) so the hottest endpoint stays 1 query. Router: owner-gated PUT, `label` never coerced to the slug on read (the UI must tell 'no label' from 'label equals slug'), null clears, 404 when the row vanished, WS agent_label_changed broadcast." + "description": "Per-agent display label (ent#181): a human-facing name that is rendered, never resolved — the slug (agent_name) stays the identity every route, container, volume, MCP key and A2A card keys on. Pins the point of the feature: setting a label leaves the slug AND its #1664 volume identity untouched, does not reserve a name, and two agents may share one (labels aren't identities). NULL = render the slug (no backfill; clearing reverts rather than blanking); a blank/whitespace label stores NULL, not an empty string that would render a nameless agent. Soft-deleted agents are not editable (deleted_at guard, mirroring the other settings setters). Batch read for the fleet list (absent = no label = slug) so the hottest endpoint stays 1 query. Router: owner-gated PUT, `label` never coerced to the slug on read (the UI must tell 'no label' from 'label equals slug'), null clears, 404 when the row vanished, WS agent_label_changed broadcast." }, { "file": "unit/test_1484_create_agent_characterization.py", @@ -1084,7 +1084,7 @@ "skills", "unit" ], - "description": "Full-directory skill packages (ent#183): pure packaging primitives (hardened frontmatter contract parse \u2014 alias-bomb refused, garbage requires typed-guarded, malicious dep names regex-gated; git-archive member vetting \u2014 REGTYPE only, symlinks/litter/protected basenames dropped with named warnings; injection tar with generated .trinity-skill.json meta appended LAST; manifest-based prune diff capped 200/skill); round-trip against the REAL agent-server restore_from_tar incl. allowlist confinement; real-git end-to-end (repo -> archive -> filter -> restore, tree-SHA determinism across clones, exec bits from git modes); injection orchestration (skip-if-unchanged vs force, old-image 404 fallback with multi_file_dropped_old_image only for multi-file skills, restore-failure repair path = delete-dir + one re-restore, prune deletes ONLY previous-manifest files, unmanaged same-named dir overwritten but never pruned, per-skill + total caps named errors, dep-probe warnings missing_binary/missing_env fail-open and run for unchanged skills too, SkillInjectionBusy on lock contention, CLAUDE.md section rebuilt from ALL present skills with line-anchored replacement that spares ### lookalikes)." + "description": "Full-directory skill packages (ent#183): pure packaging primitives (hardened frontmatter contract parse — alias-bomb refused, garbage requires typed-guarded, malicious dep names regex-gated; git-archive member vetting — REGTYPE only, symlinks/litter/protected basenames dropped with named warnings; injection tar with generated .trinity-skill.json meta appended LAST; manifest-based prune diff capped 200/skill); round-trip against the REAL agent-server restore_from_tar incl. allowlist confinement; real-git end-to-end (repo -> archive -> filter -> restore, tree-SHA determinism across clones, exec bits from git modes); injection orchestration (skip-if-unchanged vs force, old-image 404 fallback with multi_file_dropped_old_image only for multi-file skills, restore-failure repair path = delete-dir + one re-restore, prune deletes ONLY previous-manifest files, unmanaged same-named dir overwritten but never pruned, per-skill + total caps named errors, dep-probe warnings missing_binary/missing_env fail-open and run for unchanged skills too, SkillInjectionBusy on lock contention, CLAUDE.md section rebuilt from ALL present skills with line-anchored replacement that spares ### lookalikes)." }, { "file": "unit/test_ent123_tokenless_clone.py", @@ -1096,7 +1096,7 @@ "git", "unit" ], - "description": "PAT-free clone of public github: templates (ent#123). 50 tests over the 1484-shape purge-and-mock harness: _gate_tokenless_request (\"\"->None normalization, source-mode-only 400 incl. explicit None, fork passthrough), _parse_github_ref charset guard (eval-interpolation hardening), _validate_github_access tokenless ls-remote probe (ok/unavailable->400/transient->502 fail-closed, anonymous branch check, PAT-ful REST regression), _apply_github_env token-var gating (+GIT_SYNC_AUTO belt), lifecycle._apply_persisted_auth_env rebuild seam (repo-only gate, source-mode re-derivation), git_service.probe_anonymous_repo_access stderr classification, the no_write_credentials sync/reset guard (baked env OR per-agent PAT row \u2014 #1264 live-injection window, fail-open), and startup.sh static guards (repo-only clone gate, credential-less CLONE_URL, GIT_TERMINAL_PROMPT=0, push blackhole, .env PAT fallback, private-repo failure cause)." + "description": "PAT-free clone of public github: templates (ent#123). 50 tests over the 1484-shape purge-and-mock harness: _gate_tokenless_request (\"\"->None normalization, source-mode-only 400 incl. explicit None, fork passthrough), _parse_github_ref charset guard (eval-interpolation hardening), _validate_github_access tokenless ls-remote probe (ok/unavailable->400/transient->502 fail-closed, anonymous branch check, PAT-ful REST regression), _apply_github_env token-var gating (+GIT_SYNC_AUTO belt), lifecycle._apply_persisted_auth_env rebuild seam (repo-only gate, source-mode re-derivation), git_service.probe_anonymous_repo_access stderr classification, the no_write_credentials sync/reset guard (baked env OR per-agent PAT row — #1264 live-injection window, fail-open), and startup.sh static guards (repo-only clone gate, credential-less CLONE_URL, GIT_TERMINAL_PROMPT=0, push blackhole, .env PAT fallback, private-repo failure cause)." }, { "file": "unit/test_ent125_resilient_system_deploy.py", @@ -1107,7 +1107,7 @@ "systems", "unit" ], - "description": "Resilient system-manifest deploy (ent#125): best-effort default \u2014 partial deploy continues past a failed agent-create and reports failed[] with {name, short_name, template, reason, status_code}; post-create config (folders/permissions/schedules/tags) scoped to the survivor map; config-phase failures degrade to warnings; total failure returns HTTP 500 with the full report body, skips config/start, and never writes trinity_prompt; partial still writes it; strict=True aborts preserving the original status code (dict-detail 429 not flattened to 500); reason normalization (dict detail \u2192 error field), PAT-bearing git-URL userinfo redaction (learnings 2026-07-14), and 500-char truncation; orchestrator-workers preset with failed orchestrator warns 'non-functional' and configures permissions over survivors only; dry_run and all-success responses unchanged (failed == []). Router mounted alone; since the ent#124 extraction the deploy orchestration lives in services.system_service.deploy_manifest, so config fns/db are patched on services.system_service and agent creation via the _default_create_agent_fn seam." + "description": "Resilient system-manifest deploy (ent#125): best-effort default — partial deploy continues past a failed agent-create and reports failed[] with {name, short_name, template, reason, status_code}; post-create config (folders/permissions/schedules/tags) scoped to the survivor map; config-phase failures degrade to warnings; total failure returns HTTP 500 with the full report body, skips config/start, and never writes trinity_prompt; partial still writes it; strict=True aborts preserving the original status code (dict-detail 429 not flattened to 500); reason normalization (dict detail → error field), PAT-bearing git-URL userinfo redaction (learnings 2026-07-14), and 500-char truncation; orchestrator-workers preset with failed orchestrator warns 'non-functional' and configures permissions over survivors only; dry_run and all-success responses unchanged (failed == []). Router mounted alone; since the ent#124 extraction the deploy orchestration lives in services.system_service.deploy_manifest, so config fns/db are patched on services.system_service and agent creation via the _default_create_agent_fn seam." }, { "file": "unit/test_ent124_default_system_seed.py", @@ -1119,7 +1119,7 @@ "unit", "infrastructure" ], - "description": "First-run default-system seeder (ent#124): the REAL bundled config/manifests/default-system.yaml validates through the executor's own parse_manifest/validate_manifest (acme trio, full-mesh, no schedules, no prompt) and every local: template it references exists in-tree (learnings 2026-07-23 blank-agent trap); durable default_system_seeded flag gating (deleted fleet never resurrected); persisted first_run_fresh verdict \u2014 computed once BEFORE Cornelius runs, persisted, reused on later passes (cross-pass count-poisoning bug), cornelius_seeded=true forces not-fresh (ent#107-era installs get no fleet), count failure defers without persisting; TRINITY_DEFAULT_SYSTEM_MANIFEST disable sentinels skip without burning the flag; override path used when set, unreadable override fails loudly (operator-queue alert, no bundled fallback, no flag); flag policy per deploy outcome: deployed/partial set flag (+partial alert), failed (0 created)/exception do NOT (retry-safe); existence backstop converges the flag without deploying when any {system}-{short} name is reserved (suffix double-seed guard for the fail-open lock); SETNX lock held->skip / winner deploys+releases; orchestrator never raises when Cornelius explodes. db/docker/redis/deploy_manifest seams patched; no Docker/backend." + "description": "First-run default-system seeder (ent#124): the REAL bundled config/manifests/default-system.yaml validates through the executor's own parse_manifest/validate_manifest (acme trio, full-mesh, no schedules, no prompt) and every local: template it references exists in-tree (learnings 2026-07-23 blank-agent trap); durable default_system_seeded flag gating (deleted fleet never resurrected); persisted first_run_fresh verdict — computed once BEFORE Cornelius runs, persisted, reused on later passes (cross-pass count-poisoning bug), cornelius_seeded=true forces not-fresh (ent#107-era installs get no fleet), count failure defers without persisting; TRINITY_DEFAULT_SYSTEM_MANIFEST disable sentinels skip without burning the flag; override path used when set, unreadable override fails loudly (operator-queue alert, no bundled fallback, no flag); flag policy per deploy outcome: deployed/partial set flag (+partial alert), failed (0 created)/exception do NOT (retry-safe); existence backstop converges the flag without deploying when any {system}-{short} name is reserved (suffix double-seed guard for the fail-open lock); SETNX lock held->skip / winner deploys+releases; orchestrator never raises when Cornelius explodes. db/docker/redis/deploy_manifest seams patched; no Docker/backend." }, { "file": "unit/test_1809_image_drift_recreate.py", @@ -1131,7 +1131,7 @@ "unit", "reliability" ], - "description": "Image-drift recreate predicate (#1809): check_base_image_matches compares the container's resolved image id against what its OWN Config.Image reference currently resolves to \u2014 drift\u2192False (recreate on next cold start), match\u2192True; fail-open True on falsy attrs (Config.Image can be \"\"), ImageNotFound (recreate would fail on the same missing tag), and generic daemon errors (2am case: start proceeds on the old image, never a fleet recreate); ID-pinned Config.Image is a documented tautological no-op; version-pinned tags compare their own tag only (a :latest-only rebuild leaves 0.8.0-pinned agents untouched). Source-pins the lifecycle wiring: lazy evaluation gated on `not needs_recreation and not was_already_running` (cold start only \u2014 start-on-running stays an idempotent no-op), ephemeral-ghost exclusion, image-check assignment ABOVE the #1560 clear_agent_breakers gate, recreate delivering the upgrade via the container's own tag, trinity.base-image-version label refresh, recreate-race hardening (remove NotFound tolerated, run 409 adopts the winner), start-response recreated/recreate_reason surfacing, and the skip-inject _reset fixture stubbing the new predicate (Mock-auto-child trap)." + "description": "Image-drift recreate predicate (#1809): check_base_image_matches compares the container's resolved image id against what its OWN Config.Image reference currently resolves to — drift→False (recreate on next cold start), match→True; fail-open True on falsy attrs (Config.Image can be \"\"), ImageNotFound (recreate would fail on the same missing tag), and generic daemon errors (2am case: start proceeds on the old image, never a fleet recreate); ID-pinned Config.Image is a documented tautological no-op; version-pinned tags compare their own tag only (a :latest-only rebuild leaves 0.8.0-pinned agents untouched). Source-pins the lifecycle wiring: lazy evaluation gated on `not needs_recreation and not was_already_running` (cold start only — start-on-running stays an idempotent no-op), ephemeral-ghost exclusion, image-check assignment ABOVE the #1560 clear_agent_breakers gate, recreate delivering the upgrade via the container's own tag, trinity.base-image-version label refresh, recreate-race hardening (remove NotFound tolerated, run 409 adopts the winner), start-response recreated/recreate_reason surfacing, and the skip-inject _reset fixture stubbing the new predicate (Mock-auto-child trap)." }, { "file": "unit/test_1759_local_template_not_found.py", @@ -1143,7 +1143,7 @@ "unit", "reliability" ], - "description": "Local-template create gate (#1759, companion to #1793). Drives the FULL create_agent_internal, so unlike test_1793_unknown_local_template.py it also proves the reject is pre-side-effect (no container, volume, MCP key, ownership row or ephemeral slot). Covers the absent case at #1793's 404 UNKNOWN_LOCAL_TEMPLATE (both roots missing, dir without template.yaml, name resolving to a regular file), the #1759 400 LOCAL_TEMPLATE_INVALID band (empty / whitespace / comment-only / scalar / list / unparseable template.yaml), INVALID_LOCAL_TEMPLATE_NAME precedence, Blank Agent passthrough, the /template bind source under set/unset/EMPTY HOST_TEMPLATES_PATH, and the disclosure rule \u2014 one identical message whichever root missed, echoing no filesystem path (deploy-local template dirs are named after AGENT names, so 'which root' is a #186 enumeration oracle)." + "description": "Local-template create gate (#1759, companion to #1793). Drives the FULL create_agent_internal, so unlike test_1793_unknown_local_template.py it also proves the reject is pre-side-effect (no container, volume, MCP key, ownership row or ephemeral slot). Covers the absent case at #1793's 404 UNKNOWN_LOCAL_TEMPLATE (both roots missing, dir without template.yaml, name resolving to a regular file), the #1759 400 LOCAL_TEMPLATE_INVALID band (empty / whitespace / comment-only / scalar / list / unparseable template.yaml), INVALID_LOCAL_TEMPLATE_NAME precedence, Blank Agent passthrough, the /template bind source under set/unset/EMPTY HOST_TEMPLATES_PATH, and the disclosure rule — one identical message whichever root missed, echoing no filesystem path (deploy-local template dirs are named after AGENT names, so 'which root' is a #186 enumeration oracle)." }, { "file": "unit/test_1759_template_root_parity.py", @@ -1155,7 +1155,7 @@ "unit", "reliability" ], - "description": "Curated-template root parity (#1759). The create resolver's repo-relative fallback is hand-rolled rather than imported from template_service (which the #1484 harness MagicMocks, so an imported gate would be satisfied by a truthy mock), which makes drift between the two surfaces invisible \u2014 these tests pin them equal against both REAL modules. Also pins that the fallback resolves to /config/agent-templates and not /src/config/agent-templates (crud.py sits one directory deeper than template_service.py), that the deploy-local root is unchanged, that the container branch returns the pre-#1759 literal byte-identically, that the host branch is absolute so Docker accepts it as a bind source, and \u2014 the guard that would have caught the missing local:default \u2014 that EVERY shipped in-tree template resolves through the create gate." + "description": "Curated-template root parity (#1759). The create resolver's repo-relative fallback is hand-rolled rather than imported from template_service (which the #1484 harness MagicMocks, so an imported gate would be satisfied by a truthy mock), which makes drift between the two surfaces invisible — these tests pin them equal against both REAL modules. Also pins that the fallback resolves to /config/agent-templates and not /src/config/agent-templates (crud.py sits one directory deeper than template_service.py), that the deploy-local root is unchanged, that the container branch returns the pre-#1759 literal byte-identically, that the host branch is absolute so Docker accepts it as a bind source, and — the guard that would have caught the missing local:default — that EVERY shipped in-tree template resolves through the create gate." }, { "file": "unit/test_1759_export_manifest_template.py", @@ -1166,7 +1166,7 @@ "systems", "unit" ], - "description": "export_manifest template round-trip (#1759). Blank Agents carry template: None (key PRESENT), and dict.get(key, default) returns the default only when the key is ABSENT \u2014 so the old local:business-assistant fallback was unreachable dead code and every template-less agent exported template: null, which SystemAgentConfig.template (non-Optional str) rejects on redeploy. Pins the `or` fix emitting local:default, pass-through of a real template, that the exported manifest both validates and names a template that exists in-tree, and that inferred agents are logged." + "description": "export_manifest template round-trip (#1759). Blank Agents carry template: None (key PRESENT), and dict.get(key, default) returns the default only when the key is ABSENT — so the old local:business-assistant fallback was unreachable dead code and every template-less agent exported template: null, which SystemAgentConfig.template (non-Optional str) rejects on redeploy. Pins the `or` fix emitting local:default, pass-through of a real template, that the exported manifest both validates and names a template that exists in-tree, and that inferred agents are logged." }, { "file": "unit/test_1771c_schedules_cas_edges.py", @@ -1206,7 +1206,7 @@ "analytics", "scheduling" ], - "description": "Edge-case matrix (sub-area B) for db/schedules/analytics.py on db_harness (#300): _TRIGGER_BUCKETS subset of _BUCKET_ORDER regression guard (the existing literal assertion omits 'Reminders'), unknown/empty/None trigger bucketing, zero-terminal day reports None not 0%, headline-0.0 spec gap (UNSPEC), terminal-based success_rate incl. the legacy 'error' alias, NULL-skipping context AVG, percentile-cap boundary, full-set avg vs sampled p95 (the locked data-source discipline), percentile pool 0/1/2 rows, offset-bearing day bucketing [SQLITE-ONLY], strict '>' window boundary with a frozen iso_cutoff, gap-filled contiguous timeline, and _schedule_command_label unicode/length boundaries. Also covers get_schedule_analytics end-to-end (agent-written tool_calls SHAPE guards \u2014 valid-JSON-not-a-list, list-of-scalars, dict-without-name, non-numeric duration \u2014 on BOTH that surface and get_agent_schedules_summary; the FAILED arm of the per-schedule timeline; and its own 0/1/2/3-row percentile ladder, a second implementation of the same statistics.quantiles arithmetic get_agent_analytics has)." + "description": "Edge-case matrix (sub-area B) for db/schedules/analytics.py on db_harness (#300): _TRIGGER_BUCKETS subset of _BUCKET_ORDER regression guard (the existing literal assertion omits 'Reminders'), unknown/empty/None trigger bucketing, zero-terminal day reports None not 0%, headline-0.0 spec gap (UNSPEC), terminal-based success_rate incl. the legacy 'error' alias, NULL-skipping context AVG, percentile-cap boundary, full-set avg vs sampled p95 (the locked data-source discipline), percentile pool 0/1/2 rows, offset-bearing day bucketing [SQLITE-ONLY], strict '>' window boundary with a frozen iso_cutoff, gap-filled contiguous timeline, and _schedule_command_label unicode/length boundaries. Also covers get_schedule_analytics end-to-end (agent-written tool_calls SHAPE guards — valid-JSON-not-a-list, list-of-scalars, dict-without-name, non-numeric duration — on BOTH that surface and get_agent_schedules_summary; the FAILED arm of the per-schedule timeline; and its own 0/1/2/3-row percentile ladder, a second implementation of the same statistics.quantiles arithmetic get_agent_analytics has)." }, { "file": "unit/test_1771c_schedules_analytics_properties.py", @@ -1292,7 +1292,7 @@ "scheduling", "agents" ], - "description": "Creation-time schedule materialization (ent#89) against a REAL DB with row read-back: both resolver branches populate declared_schedules (the \u00a70/R2 regression \u2014 the github: branch never populated template_data), the github fetch uses the creation-resolved PAT and the parsed @branch ref, enabled honored/defaulted, cron->cron_expression, name-match and intra-block idempotency, the ghost skip, a falsy create_schedule return counted as failed (R6), and non-fatality of a raising create_schedule / list_agent_schedules inside the rollback fence." + "description": "Creation-time schedule materialization (ent#89) against a REAL DB with row read-back: both resolver branches populate declared_schedules (the §0/R2 regression — the github: branch never populated template_data), the github fetch uses the creation-resolved PAT and the parsed @branch ref, enabled honored/defaulted, cron->cron_expression, name-match and intra-block idempotency, the ghost skip, a falsy create_schedule return counted as failed (R6), and non-fatality of a raising create_schedule / list_agent_schedules inside the rollback fence." }, { "file": "unit/test_ent89_manifest_no_duplicate.py", @@ -1398,7 +1398,7 @@ "lifecycle", "credentials" ], - "description": "Agent MCP key detect/self-heal/rotate (#1854). DB: create_agent_mcp_api_key born ACTIVE (tables.py has no column default -> NULL reads as revoked and cannot even deserialize); captured-id delete (not `id != new_id`, so a racing recreate_missing_container mint is not collateral), connector key survives, keep_id never removed, other agents unreachable; DELETE-not-deactivate proven durable across soft-delete -> recover (recover_agent_ownership reactivates every inactive per-agent row). spawned_by_key_id reconcile: `!= current` repairs a child stranded on an OLDER superseded id, leaves a foreign parent alone, idempotent, and enforce_agent_spawn_scope never 403s at any step of a rotation. Drift predicate: absent env, URL-missing (injection needs BOTH), hash-mismatch, system/ghost exempt, fail-safe on DB error. Headline AC: env_overrides carry BOTH TRINITY_MCP_API_KEY and TRINITY_MCP_URL, proven end-to-end through recreate_container_with_updated_config on an old container that had NEITHER, plus a source-order guard that overrides are applied LAST. Probe: ok/foreign_user_key/foreign_agent_key/unknown_key/not_configured/shadow_entry, stopped container -> unavailable not 500, only digests leave the container (no token, no .mcp.json body) plus a non-vacuous SOURCE guard that no future edit of the in-container script can start leaking one. Health `stale` fires when last_used_at predates recent executions; trinity-system reports exempt not a false missing. Rotation: system/ghost 409 BEFORE any mutation, Redis-down 503 (fail-CLOSED) and contention 409, stopped agent stays stopped on the DB-only path, post-removal failure keeps superseded keys and names the real state with no raw exception text, 409-adoption post-condition blocks deletion, clear_agent_breakers before the recreate, no plaintext or key hash in response/log/audit, rate limit 429. Plus the FR-7 guards on MCP key revoke/delete and connector key mint. Review additions: the self-heal BODY is pinned \u2014 it takes the SAME agent:mcp_key_regen lock as rotation and does NOTHING (no mint, no delete) when Redis is down or another heal/rotation holds it, because there is no per-agent start lock and an unserialised second heal deletes the key the first is about to bake in; it audits its own mint+DELETE; container-env health (env_absent/env_mismatch) outranks the usage-derived states and is fail-soft; `stale` survives a naive legacy timestamp; the 409-adoption post-condition is an EXACT constant-time key match, proven against a container carrying a same-20-char-prefix twin; and any non-`agent` scope (connector, portal_delegate, a future sixth) reads as foreign, never `unknown_key`." + "description": "Agent MCP key detect/self-heal/rotate (#1854). DB: create_agent_mcp_api_key born ACTIVE (tables.py has no column default -> NULL reads as revoked and cannot even deserialize); captured-id delete (not `id != new_id`, so a racing recreate_missing_container mint is not collateral), connector key survives, keep_id never removed, other agents unreachable; DELETE-not-deactivate proven durable across soft-delete -> recover (recover_agent_ownership reactivates every inactive per-agent row). spawned_by_key_id reconcile: `!= current` repairs a child stranded on an OLDER superseded id, leaves a foreign parent alone, idempotent, and enforce_agent_spawn_scope never 403s at any step of a rotation. Drift predicate: absent env, URL-missing (injection needs BOTH), hash-mismatch, system/ghost exempt, fail-safe on DB error. Headline AC: env_overrides carry BOTH TRINITY_MCP_API_KEY and TRINITY_MCP_URL, proven end-to-end through recreate_container_with_updated_config on an old container that had NEITHER, plus a source-order guard that overrides are applied LAST. Probe: ok/foreign_user_key/foreign_agent_key/unknown_key/not_configured/shadow_entry, stopped container -> unavailable not 500, only digests leave the container (no token, no .mcp.json body) plus a non-vacuous SOURCE guard that no future edit of the in-container script can start leaking one. Health `stale` fires when last_used_at predates recent executions; trinity-system reports exempt not a false missing. Rotation: system/ghost 409 BEFORE any mutation, Redis-down 503 (fail-CLOSED) and contention 409, stopped agent stays stopped on the DB-only path, post-removal failure keeps superseded keys and names the real state with no raw exception text, 409-adoption post-condition blocks deletion, clear_agent_breakers before the recreate, no plaintext or key hash in response/log/audit, rate limit 429. Plus the FR-7 guards on MCP key revoke/delete and connector key mint. Review additions: the self-heal BODY is pinned — it takes the SAME agent:mcp_key_regen lock as rotation and does NOTHING (no mint, no delete) when Redis is down or another heal/rotation holds it, because there is no per-agent start lock and an unserialised second heal deletes the key the first is about to bake in; it audits its own mint+DELETE; container-env health (env_absent/env_mismatch) outranks the usage-derived states and is fail-soft; `stale` survives a naive legacy timestamp; the 409-adoption post-condition is an EXACT constant-time key match, proven against a container carrying a same-20-char-prefix twin; and any non-`agent` scope (connector, portal_delegate, a future sixth) reads as foreign, never `unknown_key`." }, { "file": "unit/test_ent313_failed_creation_container_reclaim.py", @@ -1409,7 +1409,7 @@ "unit", "reliability" ], - "description": "Failed-creation container reclaim (ent#313): a create that fails after containers.run must leave no orphan container, no unreclaimable volume and no name-keyed Redis state. Covers both arrival shapes \u2014 handle in hand (later step raised) and NO handle (the reported 60s Docker read timeout, where the daemon created the container and the client never got it, so it is re-derived by name). Pins every fail-closed gate on that re-derived path, because a name resolves to another install's live agent on a shared Docker daemon: 409 name conflict (daemon created nothing) \u2192 refuse; existing agent_ownership row (a concurrent creation won the name) \u2192 refuse; trinity.created label older than the attempt's floor, missing, or unparseable \u2192 refuse; missing floor \u2192 refuse; DB or Docker lookup error \u2192 refuse. Also: Redis cleared when the container is provably gone (removed or never created) and NOT when removal failed (the slot ZSET isn't idle); every failure swallowed so the original creation error is what the caller reports; and an AST guard that create_agent_internal actually AWAITS the reclaim, so the unit tests can't stay green while nothing calls it. Also pins the never-raises contract under a stubbed docker module: the first version used isinstance(exc, docker.errors.APIError), which raises TypeError wherever docker is a test double and replaced the real creation error with an unrelated one (caught by the full suite via test_fork_to_own / test_1484); the 409 check is duck-typed on .response.status_code now." + "description": "Failed-creation container reclaim (ent#313): a create that fails after containers.run must leave no orphan container, no unreclaimable volume and no name-keyed Redis state. Covers both arrival shapes — handle in hand (later step raised) and NO handle (the reported 60s Docker read timeout, where the daemon created the container and the client never got it, so it is re-derived by name). Pins every fail-closed gate on that re-derived path, because a name resolves to another install's live agent on a shared Docker daemon: 409 name conflict (daemon created nothing) → refuse; existing agent_ownership row (a concurrent creation won the name) → refuse; trinity.created label older than the attempt's floor, missing, or unparseable → refuse; missing floor → refuse; DB or Docker lookup error → refuse. Also: Redis cleared when the container is provably gone (removed or never created) and NOT when removal failed (the slot ZSET isn't idle); every failure swallowed so the original creation error is what the caller reports; and an AST guard that create_agent_internal actually AWAITS the reclaim, so the unit tests can't stay green while nothing calls it. Also pins the never-raises contract under a stubbed docker module: the first version used isinstance(exc, docker.errors.APIError), which raises TypeError wherever docker is a test double and replaced the real creation error with an unrelated one (caught by the full suite via test_fork_to_own / test_1484); the 409 check is duck-typed on .response.status_code now." }, { "file": "unit/test_1951_slack_inbound_media.py", @@ -1420,7 +1420,7 @@ "unit", "security" ], - "description": "Slack inbound file-download SSRF gate (#1951) \u2014 the third channel to get the #1932 two-tier treatment. download_file previously fetched url_private_download with no host allowlist, no scheme check and follow_redirects=True. Covers: the legitimate files.slack.com download still succeeds (the #1932 lesson \u2014 an allowlist that rejects the real host reads as 'allowlist present' while being 100% broken); a validated CDN redirect is followed WITHOUT the bot token; hop-1 refusals for unrelated/lookalike/apex-lookalike hosts, the dotless-suffix bypass (evil-slack-files.com), plaintext http, file://, loopback and a platform-network address; off-domain and downgrade redirects refused after exactly one hop; cloud metadata (169.254.169.254) as a redirect target; bounded redirect budget; refusal logged at ERROR not WARNING; source tier proven narrower than the redirect tier; and a static backstop that follow_redirects=True never returns. EVERY negative test asserts the recorded call log, not `is None` \u2014 download_file returns None on every failure path and its bare except swallows AssertionError, so a return-value assertion passes against a completely unpatched seam." + "description": "Slack inbound file-download SSRF gate (#1951) — the third channel to get the #1932 two-tier treatment. download_file previously fetched url_private_download with no host allowlist, no scheme check and follow_redirects=True. Covers: the legitimate files.slack.com download still succeeds (the #1932 lesson — an allowlist that rejects the real host reads as 'allowlist present' while being 100% broken); a validated CDN redirect is followed WITHOUT the bot token; hop-1 refusals for unrelated/lookalike/apex-lookalike hosts, the dotless-suffix bypass (evil-slack-files.com), plaintext http, file://, loopback and a platform-network address; off-domain and downgrade redirects refused after exactly one hop; cloud metadata (169.254.169.254) as a redirect target; bounded redirect budget; refusal logged at ERROR not WARNING; source tier proven narrower than the redirect tier; and a static backstop that follow_redirects=True never returns. EVERY negative test asserts the recorded call log, not `is None` — download_file returns None on every failure path and its bare except swallows AssertionError, so a return-value assertion passes against a completely unpatched seam." }, { "file": "unit/test_1917_stack_trace_exposure.py", @@ -1431,7 +1431,7 @@ "unit", "security" ], - "description": "py/stack-trace-exposure leak guards for the ops / system-agent siblings PR #1912 left untouched (#1917), including open CodeQL alert #231. Each test drives the REAL router function with a sentinel planted in the exception message and asserts both halves: the sentinel is absent from the response AND the exception class name is present (a response that dropped the error entirely would pass a sentinel-only check while making the failure undiagnosable). Covers stop_fleet per-agent, _stop_agent_container (feeds emergency_stop), fleet health (previously str(e)[:50] \u2014 truncation bounds a leak, it does not remove one, and docker/httpx messages put the host FIRST), ops costs (the OTel collector URL), and system-agent health. Plus a static ban on str(e) in ops.py/system_agent.py and a window check on agents.py's three lifecycle details. All 8 fail against pre-fix code \u2014 verified, because the fleet-health test initially patched a get_agent_context_info that does not exist (raising=False made it silent), so it passed against the unpatched router until the pre-fix run exposed it; it now patches get_agent_client and asserts the probe ran." + "description": "py/stack-trace-exposure leak guards for the ops / system-agent siblings PR #1912 left untouched (#1917), including open CodeQL alert #231. Each test drives the REAL router function with a sentinel planted in the exception message and asserts both halves: the sentinel is absent from the response AND the exception class name is present (a response that dropped the error entirely would pass a sentinel-only check while making the failure undiagnosable). Covers stop_fleet per-agent, _stop_agent_container (feeds emergency_stop), fleet health (previously str(e)[:50] — truncation bounds a leak, it does not remove one, and docker/httpx messages put the host FIRST), ops costs (the OTel collector URL), and system-agent health. Plus a static ban on str(e) in ops.py/system_agent.py and a window check on agents.py's three lifecycle details. All 8 fail against pre-fix code — verified, because the fleet-health test initially patched a get_agent_context_info that does not exist (raising=False made it silent), so it passed against the unpatched router until the pre-fix run exposed it; it now patches get_agent_client and asserts the probe ran." }, { "file": "unit/test_ent314_hardened_yaml.py", @@ -1442,7 +1442,7 @@ "unit", "security" ], - "description": "Hardened parsing of author-controlled YAML (ent#314). template.yaml was on bare yaml.safe_load, leaving alias amplification (measured 416 B -> 110 MB at json.dumps time, parse itself 0.0011 s, so an input-size cap cannot close it) and silent last-wins duplicate keys, which let a template show one credentials: block to a human and declare another to Trinity. Reachable by any creator-role user via a public repo (ent#123 tokenless) since _build_template runs unfenced inside get_all_templates(). Covers: the fixture itself still amplifies under bare safe_load (so the suite cannot pass vacuously); level 4/5/6 bombs rejected under both policies; a small honest anchor STILL parses under BUDGET (the #1932 lesson \u2014 a guard that rejects the legitimate document is an outage that reads as hardened); REJECT refuses one alias and also gates at the scanner (skill_packaging's copy had both hooks); the issue's exact duplicate-credentials case and a nested duplicate; size cap; reject-not-truncate; ManifestError still a subclass so routers/systems.py's named 400 is unchanged and the manifest's published codes survive; and two consolidation guards \u2014 no module may grow a fourth SafeLoader subclass, and all four author-controlled readers must call the shared loader with no bare safe_load left. The four product-level guards fail against pre-fix code." + "description": "Hardened parsing of author-controlled YAML (ent#314). template.yaml was on bare yaml.safe_load, leaving alias amplification (measured 416 B -> 110 MB at json.dumps time, parse itself 0.0011 s, so an input-size cap cannot close it) and silent last-wins duplicate keys, which let a template show one credentials: block to a human and declare another to Trinity. Reachable by any creator-role user via a public repo (ent#123 tokenless) since _build_template runs unfenced inside get_all_templates(). Covers: the fixture itself still amplifies under bare safe_load (so the suite cannot pass vacuously); level 4/5/6 bombs rejected under both policies; a small honest anchor STILL parses under BUDGET (the #1932 lesson — a guard that rejects the legitimate document is an outage that reads as hardened); REJECT refuses one alias and also gates at the scanner (skill_packaging's copy had both hooks); the issue's exact duplicate-credentials case and a nested duplicate; size cap; reject-not-truncate; ManifestError still a subclass so routers/systems.py's named 400 is unchanged and the manifest's published codes survive; and two consolidation guards — no module may grow a fourth SafeLoader subclass, and all four author-controlled readers must call the shared loader with no bare safe_load left. The four product-level guards fail against pre-fix code." }, { "file": "unit/test_1969_lock_denied_tick_audit.py", @@ -1563,7 +1563,7 @@ "unit", "guard" ], - "description": "Static guards on integration-nightly.yml (#1896), the CI home for the live-instance root suite (~93 files needing a running Trinity; nothing ran them, so they gated nothing). Pins the properties whose absence is invisible until the night it matters: membership is DIRECTORY-based (--ignore=unit, matching tests/run-core.sh) so #1895's relocations shrink it with no workflow edit and no file list to drift; base and head both run for attribution; a missing JUnit fails the job LOUDLY instead of being reported to the author as a regression (the diff script is fail-closed on missing XML, so a failed stack boot would otherwise wear a PR's name \u2014 #1941's class); each side gets a pristine stack via `down -v` since the suite mutates its instance; the merge is full-depth with the conflict verdict gated on unmerged paths (#1941 again); the job running untrusted PR code holds only contents:read while the write-scoped comment job never checks out PR code; it is not on the pull_request path; the runtime budget and its escalation are written down; --continue-on-collection-errors is present (measured: 3 root modules currently fail to import, and without the flag pytest writes no XML at all); and src/cli is installed, since test_cli_*.py imports trinity_cli which is deliberately absent from requirements-test.txt." + "description": "Static guards on integration-nightly.yml (#1896), the CI home for the live-instance root suite (~93 files needing a running Trinity; nothing ran them, so they gated nothing). Pins the properties whose absence is invisible until the night it matters: membership is DIRECTORY-based (--ignore=unit, matching tests/run-core.sh) so #1895's relocations shrink it with no workflow edit and no file list to drift; base and head both run for attribution; a missing JUnit fails the job LOUDLY instead of being reported to the author as a regression (the diff script is fail-closed on missing XML, so a failed stack boot would otherwise wear a PR's name — #1941's class); each side gets a pristine stack via `down -v` since the suite mutates its instance; the merge is full-depth with the conflict verdict gated on unmerged paths (#1941 again); the job running untrusted PR code holds only contents:read while the write-scoped comment job never checks out PR code; it is not on the pull_request path; the runtime budget and its escalation are written down; --continue-on-collection-errors is present (measured: 3 root modules currently fail to import, and without the flag pytest writes no XML at all); and src/cli is installed, since test_cli_*.py imports trinity_cli which is deliberately absent from requirements-test.txt." } ] } From 3405002f8f22be4cad4075cb38595273226e1c47 Mon Sep 17 00:00:00 2001 From: Oleksii Dolhov Date: Wed, 5 Aug 2026 16:58:13 +0300 Subject: [PATCH 3/4] fix(ci): stop the integration nightly posting a clean verdict it never computed (#2029) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Same defect as the backend nightly, and this workflow reaches it by design: the missing-JUnit guard added here exits 1 when a stack never booted, `Write status JSON` still runs under `if: always()`, the unset merge output stringifies to a clean verdict, and the sticky comment says ✅ for a suite that never ran. The comment above that guard claimed "the `comment` step then posts nothing for this PR rather than something false". That was not true when I wrote it — it posts something false. Corrected in place rather than quietly deleted, because a comment asserting a property the code does not have is worth more as a recorded mistake than as a tidy line. The status step now refuses to write when the merge verdict is unknown, and only defaults `regression` to false when the merge actually conflicted — the one case where the diff legitimately never ran. The comment job enumerates the status files that exist, so a missing one leaves that PR's sticky untouched. 3 guard tests, each mutation-verified. Refs #2029 Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/integration-nightly.yml | 41 +++++++++++++++-- .../test_1896_integration_nightly_workflow.py | 46 +++++++++++++++++++ 2 files changed, 84 insertions(+), 3 deletions(-) diff --git a/.github/workflows/integration-nightly.yml b/.github/workflows/integration-nightly.yml index 5c7c0eedf..9411c4f73 100644 --- a/.github/workflows/integration-nightly.yml +++ b/.github/workflows/integration-nightly.yml @@ -269,8 +269,14 @@ jobs: # fail-closed on a missing input XML, so a stack that never booted # would surface as "this PR introduced regressions" — an # infrastructure failure wearing a PR's name. Fail the job loudly - # instead; the `comment` step then posts nothing for this PR rather - # than something false. + # instead. + # + # This comment used to end "the `comment` step then posts nothing for + # this PR rather than something false", and that was not true when it + # was written — #2029. Exiting here skipped the verdict, the status + # step still ran under `if: always()`, the unset output stringified to + # a clean result, and the sticky said ✅. The guard in `Write status + # JSON` below is what finally makes the sentence accurate. for side in base head; do xml="$GITHUB_WORKSPACE/junit-${side}-pr${{ matrix.pr_number }}.xml" if [ ! -s "$xml" ]; then @@ -304,7 +310,36 @@ jobs: run: | merge_conflict='${{ steps.merge.outputs.merge_conflict }}' regression='${{ steps.diff.outputs.regression }}' - if [ -z "$regression" ]; then regression="false"; fi + + # #2029: absence of a verdict is its own state, not "clean". + # + # This step is `if: always()`, so it also runs when the job died + # BEFORE a verdict existed — a failed checkout, a failed PR-head + # fetch, a non-conflict merge failure, or the missing-JUnit guard + # above. `merge_conflict` is then '', `'' == "true"` is false, and the + # JSON says `{merge_conflict: false, regression: false}` — identical + # to a genuinely clean run, which the comment job renders as a green + # tick for a suite that never ran. + # + # Writing no file is the honest answer: the comment job enumerates + # `status-pr*.json`, so a missing one leaves that PR's sticky exactly + # as it was. + if [ -z "$merge_conflict" ]; then + echo "::warning::merge verdict unknown for PR ${{ matrix.pr_number }} — no status written, sticky left untouched" + exit 0 + fi + + # `regression` is legitimately unset when the merge conflicted (the + # diff never ran). Any other empty value means the diff step was + # skipped by a job failure — the same unknown verdict as above. + if [ -z "$regression" ]; then + if [ "$merge_conflict" = "true" ]; then + regression="false" + else + echo "::warning::regression verdict unknown for PR ${{ matrix.pr_number }} — no status written, sticky left untouched" + exit 0 + fi + fi jq -n \ --arg pr "${{ matrix.pr_number }}" \ --arg head_sha "${{ matrix.head_sha }}" \ diff --git a/tests/unit/test_1896_integration_nightly_workflow.py b/tests/unit/test_1896_integration_nightly_workflow.py index 5f41e88cb..24429fda5 100644 --- a/tests/unit/test_1896_integration_nightly_workflow.py +++ b/tests/unit/test_1896_integration_nightly_workflow.py @@ -198,3 +198,49 @@ def test_local_workflow_is_untouched(): "run-core.sh no longer defines the same suite the nightly runs — the " "two must agree or the local path stops reproducing CI (#1896)" ) + + +# --------------------------------------------------------------------------- +# #2029 — an absent verdict must not render as "clean" +# --------------------------------------------------------------------------- + + +def _status_step(): + """The step that writes `status-pr*.json`, located by what it writes rather + than by name, so renaming it doesn't silently skip these assertions.""" + for _name, job in _doc()["jobs"].items(): + for step in job.get("steps") or []: + if isinstance(step, dict) and "status-pr" in str(step.get("run", "")): + return step + pytest.fail("no step writes status-pr*.json any more") + + +def test_an_unknown_verdict_writes_no_status_file(): + """This workflow's own missing-JUnit guard (`exit 1`) routes straight into + the #2029 defect: the job dies, `Write status JSON` still runs under + `if: always()`, the unset merge output stringifies to a clean verdict, and + the sticky comment says ✅ for a suite that never ran. + + The guard makes the earlier comment's claim — "posts nothing for this PR + rather than something false" — actually true. + """ + body = _commands(_status_step()["run"]) + assert 'if [ -z "$merge_conflict" ]' in body, ( + "the status step writes a verdict even when none was produced (#2029)" + ) + assert "exit 0" in body[body.index('if [ -z "$merge_conflict" ]'):].split("fi")[0] + + +def test_regression_defaults_to_false_only_on_a_real_conflict(): + body = _commands(_status_step()["run"]) + branch = body[body.index('if [ -z "$regression" ]'):] + assert '"$merge_conflict" = "true"' in branch[:400], ( + "regression falls back to false without checking the merge actually " + "conflicted, so a skipped diff reads as clean (#2029)" + ) + + +def test_the_status_step_still_runs_on_failure(): + """`if: always()` is what lets the step see the failed case at all — + removing it would suppress genuine conflict verdicts too.""" + assert _status_step().get("if") == "always()" From 53308fabc5411e1a0aa01dbe59dd957997d93579 Mon Sep 17 00:00:00 2001 From: Oleksii Dolhov Date: Mon, 10 Aug 2026 11:24:57 +0300 Subject: [PATCH 4/4] ci(1896): keep secrets out of the job that runs PR code, and skip fork PRs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `ANTHROPIC_API_KEY: ${{ secrets.E2E_ANTHROPIC_API_KEY || 'placeholder' }}` sat on the step that runs the merged PR tree — `boot_stack` executes the PR's own scripts/deploy/start.sh, so that is arbitrary shell, not just test files. And `schedule` (unlike `pull_request`) exposes repository secrets to fork PRs. Nothing leaks today because that secret does not exist on the repo, so the expression resolves to 'placeholder' — but the workflow was DESIGNED around it existing, and whoever adds it will not re-review this file. This job's whole premise, inherited from backend-unit-nightly.yml, is that it holds no credentials. Two changes, and neither alters what runs today: 1. The step commits to the literal 'placeholder', with the reasoning inline so the next person adding a live-LLM key sees why this line is not the place. Tests needing a real key are skip-marked instead — which is the AC anyway. 2. `discover` filters `isCrossRepository` PRs out of the matrix, so untrusted code never runs in a scheduled job at all. Logged with ::warning:: rather than dropped silently: a skipped fork is a coverage gap, and a silent skip looks exactly like a pass. #2009 is an open fork PR against dev right now. The guard is extended to match. `test_the_job_running_pr_code_has_no_write_token` asserted only on `permissions:` — token scope was right, and a `secrets.*` reference in the same job was unguarded, which is how this would have survived review a second time. Now: no `secrets.*` reference of any name in the test job, and the fork filter must be present, filtered on, AND logged. secret restored -> 1 failed, 17 passed fork filter removed -> 1 failed, 17 passed as shipped -> 18 passed Blocking item 1 (the false green tick) was already fixed in 3405002f before this round: the status step refuses to write when the verdict is unknown, so the comment job skips that PR instead of rendering a clean verdict it never computed. Related to #1896 Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/integration-nightly.yml | 26 ++++++++++++-- .../test_1896_integration_nightly_workflow.py | 34 +++++++++++++++++++ 2 files changed, 58 insertions(+), 2 deletions(-) diff --git a/.github/workflows/integration-nightly.yml b/.github/workflows/integration-nightly.yml index 9411c4f73..86e37e0bc 100644 --- a/.github/workflows/integration-nightly.yml +++ b/.github/workflows/integration-nightly.yml @@ -86,8 +86,18 @@ jobs: --base dev \ --state open \ --limit 20 \ - --json number,headRefName,headRefOid) + --json number,headRefName,headRefOid,isCrossRepository) fi + # Defence in depth for the same exposure: fork PRs are excluded from + # the matrix, so PR code from outside the org never runs in a + # scheduled job that can see repository secrets or the runner's + # environment. Logged, not silent — a skipped fork must be visible as + # a coverage gap rather than look like a passing run. + forks=$(echo "$prs" | jq -r '[.[] | select(.isCrossRepository)] | map(.number|tostring) | join(", ")') + if [ -n "$forks" ]; then + echo "::warning::skipping fork PR(s) $forks — untrusted code is not run in the scheduled integration job" + fi + prs=$(echo "$prs" | jq '[.[] | select(.isCrossRepository | not)]') count=$(echo "$prs" | jq 'length') if [ "$count" = "0" ]; then echo "has_prs=false" >> "$GITHUB_OUTPUT" @@ -193,7 +203,19 @@ jobs: - name: Run the live suite on BOTH sides if: steps.merge.outputs.merge_conflict == 'false' env: - ANTHROPIC_API_KEY: ${{ secrets.E2E_ANTHROPIC_API_KEY || 'placeholder' }} + # NO `secrets.*` here, deliberately (#1896 review). `schedule` exposes + # repository secrets to fork PRs, and this step runs arbitrary shell + # from the merged PR tree (`boot_stack` runs the PR's own + # scripts/deploy/start.sh), so any secret named here is readable by + # anyone who opens a PR. `backend-unit-nightly.yml` — the workflow + # whose security architecture this one inherits — keeps its only + # secret in `discover`, and its test job is genuinely credential-free; + # this one is too, now. Tests needing a real key are skip-marked + # rather than paid for with a fleet-wide exposure. If a live-LLM + # nightly is ever wanted, it needs a different shape (a + # `pull_request_target`-free, same-repo-only workflow with an + # environment approval), not a secret added to this line. + ANTHROPIC_API_KEY: placeholder # `tests/conftest.py` aliases TRINITY_TEST_PASSWORD from # ADMIN_PASSWORD, so the credential does not need restating. TRINITY_API_URL: http://localhost:8000 diff --git a/tests/unit/test_1896_integration_nightly_workflow.py b/tests/unit/test_1896_integration_nightly_workflow.py index 24429fda5..57f23d40e 100644 --- a/tests/unit/test_1896_integration_nightly_workflow.py +++ b/tests/unit/test_1896_integration_nightly_workflow.py @@ -167,6 +167,40 @@ def test_the_job_running_pr_code_has_no_write_token(): ) +def test_the_job_running_pr_code_holds_no_secrets(): + """Token SCOPE was guarded; secret REFERENCES were not. + + The original version of this test asserted only on `permissions:`, so + `ANTHROPIC_API_KEY: ${{ secrets.E2E_ANTHROPIC_API_KEY || 'placeholder' }}` + sat in the same job unremarked — and `schedule` (unlike `pull_request`) + exposes repository secrets to fork PRs, while this job runs arbitrary shell + from the merged PR tree. The guard read as if it covered that and did not. + """ + import re + + job = str(_test_job()) + referenced = set(re.findall(r"secrets\.([A-Za-z_][A-Za-z0-9_]*)", job)) + assert not referenced, ( + f"the job that runs untrusted PR code references secret(s): " + f"{sorted(referenced)} — a scheduled run hands those to any fork PR" + ) + + +def test_fork_prs_are_not_run(): + """Defence in depth for the same exposure, and visible when it bites.""" + discover = str(_doc()["jobs"]["discover"]) + assert "isCrossRepository" in discover, ( + "discover no longer filters fork PRs — untrusted code would run in a " + "scheduled job with access to the repository's environment" + ) + assert "select(.isCrossRepository | not)" in discover, ( + "the fork field is fetched but not filtered on" + ) + assert "::warning::skipping fork PR" in discover, ( + "a skipped fork must be logged — a silent skip looks like a pass" + ) + + def test_it_is_not_wired_into_the_pull_request_merge_path(): """AC 2: nightly, not per-PR. A mutating ~93-file suite in the merge path is a cost and a flake source; `workflow_dispatch` keeps it runnable on