From 638e86b70384750c396807bc7bd856c574f23b42 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:18:48 +0300 Subject: [PATCH 01/62] ci(af01): add pinned Scorecard posture evidence --- .github/workflows/af01-scorecard.yml | 36 ++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 .github/workflows/af01-scorecard.yml diff --git a/.github/workflows/af01-scorecard.yml b/.github/workflows/af01-scorecard.yml new file mode 100644 index 00000000..9609a344 --- /dev/null +++ b/.github/workflows/af01-scorecard.yml @@ -0,0 +1,36 @@ +name: af01-scorecard + +on: + pull_request: + push: + branches: + - main + schedule: + - cron: "17 3 * * 6" + workflow_dispatch: + +permissions: + contents: read + +jobs: + scorecard: + runs-on: ubuntu-24.04 + timeout-minutes: 15 + steps: + - uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5 / Node 24 + with: + ref: ${{ github.event.pull_request.head.sha || github.sha }} + persist-credentials: false + - name: Run OpenSSF Scorecard + uses: ossf/scorecard-action@2d1146689b8cda280b9bc96326124645441f03bc # v2.4.4 / Scorecard v5.5.0 + with: + results_file: af01-scorecard.json + results_format: json + publish_results: false + - name: Retain per-check Scorecard evidence + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 + with: + name: af01-scorecard + path: af01-scorecard.json + if-no-files-found: error + retention-days: 7 From fc3bf3cb590c1b82e225ee5b4fd1ed299aadc138 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:19:05 +0300 Subject: [PATCH 02/62] ci(af01): authorize Scorecard least-privilege job --- .github/workflow-trust-policy.json | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/.github/workflow-trust-policy.json b/.github/workflow-trust-policy.json index 407a3f21..151fdfd8 100644 --- a/.github/workflow-trust-policy.json +++ b/.github/workflow-trust-policy.json @@ -22,6 +22,17 @@ "require_external_uses_full_sha": "External Actions and reusable workflows must be bound to an immutable 40-hex commit so mutable tags or branches cannot change executed code without a repository diff." }, "workflows": { + ".github/workflows/af01-scorecard.yml": { + "jobs": { + "scorecard": { + "permissions": { + "contents": "read" + }, + "runner": "ubuntu-24.04", + "timeout_minutes": 15 + } + } + }, ".github/workflows/af01-security.yml": { "jobs": { "cargo-audit": { @@ -145,4 +156,4 @@ } }, "exceptions": [] -} \ No newline at end of file +} From db3fc33fbd42f650c610e40abec794828a68b725 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:25:55 +0300 Subject: [PATCH 03/62] docs(security): add vulnerability reporting policy --- SECURITY.md | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) create mode 100644 SECURITY.md diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 00000000..793be13b --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,30 @@ +# Security Policy + +## Supported versions + +commandF is currently developed from the canonical `main` branch. Until a stable release policy is published, security fixes target `main`; historical commits and experimental branches are not separately supported security lines. + +## Reporting a vulnerability + +Please do not publish exploit details, credentials, private data, or a working proof of concept in a public issue. + +Prefer GitHub's private vulnerability-reporting flow for this repository when the **Report a vulnerability** option is available under the repository Security tab. Include: + +- the affected commandF component and exact commit or release identity; +- the security property that can be violated; +- minimal reproduction conditions and expected versus observed behavior; +- impact and required attacker capabilities; +- any known workaround or containment; +- whether the report contains embargo-sensitive details. + +If the private reporting control is not available, open a public issue containing only a request for a private security contact and non-sensitive routing information. Do not include exploit steps or sensitive evidence in that issue. + +## Scope + +Security reports are especially useful for dependency or workflow supply-chain compromise, unsafe archive or path handling, command execution, credential exposure, source-map or annotation injection, artifact/proof integrity failures, cache or lockfile trust violations, and bypasses of repository assurance gates. + +commandF is not a clinical decision system. Reports about clinical interpretation should distinguish a commandF implementation defect from behavior of external HL7/FHIR tooling or source artifacts. + +## Disclosure and fixes + +Validated reports should receive a bounded remediation plan before public disclosure. Security exceptions must follow the checked-in AF-01 waiver policy; an advisory or scanner finding is not silently ignored and a passing aggregate score is not treated as evidence that a specific vulnerability is resolved. From 8d964dcbaa54dd48f7754651f12fd6c08a4cc1e4 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:26:05 +0300 Subject: [PATCH 04/62] ci(security): add bounded dependency update automation --- .github/dependabot.yml | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 .github/dependabot.yml diff --git a/.github/dependabot.yml b/.github/dependabot.yml new file mode 100644 index 00000000..04535338 --- /dev/null +++ b/.github/dependabot.yml @@ -0,0 +1,22 @@ +version: 2 +updates: + - package-ecosystem: cargo + directory: / + schedule: + interval: weekly + day: monday + open-pull-requests-limit: 5 + + - package-ecosystem: maven + directory: /tools/hl7-oracle + schedule: + interval: weekly + day: monday + open-pull-requests-limit: 5 + + - package-ecosystem: github-actions + directory: / + schedule: + interval: weekly + day: monday + open-pull-requests-limit: 5 From fe302ac49122cb68ddc68f18c51b6acd14a4c329 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:27:20 +0300 Subject: [PATCH 05/62] ci(af01): retain repository-aware Scorecard posture --- .github/workflows/af01-scorecard.yml | 54 ++++++++++++++++++++++++++-- 1 file changed, 51 insertions(+), 3 deletions(-) diff --git a/.github/workflows/af01-scorecard.yml b/.github/workflows/af01-scorecard.yml index 9609a344..04c67589 100644 --- a/.github/workflows/af01-scorecard.yml +++ b/.github/workflows/af01-scorecard.yml @@ -21,16 +21,64 @@ jobs: with: ref: ${{ github.event.pull_request.head.sha || github.sha }} persist-credentials: false - - name: Run OpenSSF Scorecard + - name: Run OpenSSF Scorecard against exact checked-out source uses: ossf/scorecard-action@2d1146689b8cda280b9bc96326124645441f03bc # v2.4.4 / Scorecard v5.5.0 with: - results_file: af01-scorecard.json + results_file: af01-scorecard-local.json results_format: json publish_results: false + - name: Run repository-aware Scorecard posture scan + env: + GITHUB_AUTH_TOKEN: ${{ github.token }} + run: | + set -euo pipefail + archive=/tmp/scorecard_5.5.0_linux_amd64.tar.gz + install_dir=/tmp/af01-scorecard-cli + curl --fail --location --silent --show-error \ + https://github.com/ossf/scorecard/releases/download/v5.5.0/scorecard_5.5.0_linux_amd64.tar.gz \ + --output "$archive" + printf '%s %s\n' \ + '83b90a05c1540ef1390db1cd5711e5fd04be9c1d8537fb84d39d02092d6a8dff' \ + "$archive" | sha256sum --check --strict + mkdir -p "$install_dir" + tar -xzf "$archive" -C "$install_dir" + "$install_dir/scorecard" \ + --repo="github.com/${GITHUB_REPOSITORY}" \ + --checks=Branch-Protection,Dangerous-Workflow,Dependency-Update-Tool,Pinned-Dependencies,Security-Policy,Token-Permissions,Vulnerabilities \ + --format=json \ + --show-details \ + > af01-scorecard-repository.json + python3 - <<'PY' + import json + from pathlib import Path + + path = Path("af01-scorecard-repository.json") + result = json.loads(path.read_text(encoding="utf-8")) + expected = { + "Branch-Protection", + "Dangerous-Workflow", + "Dependency-Update-Tool", + "Pinned-Dependencies", + "Security-Policy", + "Token-Permissions", + "Vulnerabilities", + } + repo = result.get("repo", {}) + checks = result.get("checks") + if repo.get("name") != "github.com/TheHalfMoon/commandF": + raise SystemExit(f"unexpected repository identity: {repo!r}") + if not isinstance(checks, list): + raise SystemExit("repository Scorecard output has no checks array") + names = {item.get("name") for item in checks if isinstance(item, dict)} + if names != expected: + raise SystemExit(f"repository Scorecard checks mismatch: {sorted(names)!r}") + PY - name: Retain per-check Scorecard evidence uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: af01-scorecard - path: af01-scorecard.json + path: | + af01-scorecard-local.json + af01-scorecard-repository.json if-no-files-found: error retention-days: 7 From ed00c8ab67fe527fb01eca1d4264e65681d4dc96 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:28:44 +0300 Subject: [PATCH 06/62] ci(af01): add deterministic assurance summary builder --- .../scripts/build_af01_assurance_summary.py | 264 ++++++++++++++++++ 1 file changed, 264 insertions(+) create mode 100644 .github/scripts/build_af01_assurance_summary.py diff --git a/.github/scripts/build_af01_assurance_summary.py b/.github/scripts/build_af01_assurance_summary.py new file mode 100644 index 00000000..fbe3be91 --- /dev/null +++ b/.github/scripts/build_af01_assurance_summary.py @@ -0,0 +1,264 @@ +#!/usr/bin/env python3 +"""Build the deterministic AF-01 assurance summary from exact-source evidence.""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import re +import subprocess +from pathlib import Path +from typing import Any + +CHECKOUT_ACTION = "fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09" +CARGO_DENY_ACTION = "3c6349835b2b7b196a839186cb8b78e02f7b5f25" +CARGO_DENY_VERSION = "0.20.2" +CARGO_AUDIT_VERSION = "0.22.2" +ZIZMOR_ACTION = "3dc1ecc9bcb9e94e9b2c709687979e1298497054" +ZIZMOR_VERSION = "1.29.0" +RUSTSEC_ORIGIN = "https://github.com/RustSec/advisory-db.git" +CONFIG_PATHS = ( + ".github/workflow-trust-policy.json", + "Cargo.lock", + "deny.toml", +) +EVIDENCE_FILES = { + "workflow_trust": "workflow-trust.json", + "dependency_inventory": "dependency-inventory.json", + "cargo_deny": "cargo-deny-proof.json", + "cargo_audit": "cargo-audit-proof.json", + "cargo_audit_result": "cargo-audit.json", + "zizmor": "zizmor-proof.json", +} +HEX40 = re.compile(r"^[0-9a-f]{40}$") + + +class AssuranceError(ValueError): + """Raised when evidence cannot prove the requested exact source state.""" + + +def sha256_bytes(data: bytes) -> str: + return hashlib.sha256(data).hexdigest() + + +def sha256_file(path: Path) -> str: + if not path.is_file(): + raise AssuranceError(f"required file is missing: {path}") + return sha256_bytes(path.read_bytes()) + + +def read_json(path: Path) -> dict[str, Any]: + if not path.is_file(): + raise AssuranceError(f"required evidence is missing: {path.name}") + try: + value = json.loads(path.read_text(encoding="utf-8")) + except (OSError, UnicodeError, json.JSONDecodeError) as error: + raise AssuranceError(f"invalid JSON evidence {path.name}: {error}") from error + if not isinstance(value, dict): + raise AssuranceError(f"evidence root must be an object: {path.name}") + return value + + +def git(root: Path, *args: str) -> str: + try: + return subprocess.check_output( + ["git", "-C", str(root), *args], text=True, stderr=subprocess.STDOUT + ).strip() + except subprocess.CalledProcessError as error: + raise AssuranceError(f"git {' '.join(args)} failed: {error.output.strip()}") from error + + +def tracked_files(root: Path) -> list[str]: + rendered = git(root, "ls-files", "-z") + return sorted(path for path in rendered.split("\0") if path) + + +def security_surfaces(paths: list[str]) -> tuple[list[str], list[str]]: + workflows = sorted( + path + for path in paths + if path.startswith(".github/workflows/") and path.endswith((".yml", ".yaml")) + ) + actions = sorted( + path for path in paths if Path(path).name in {"action.yml", "action.yaml"} + ) + return workflows, actions + + +def surface_digest(root: Path, paths: list[str]) -> str: + digest = hashlib.sha256() + for relative in paths: + encoded = relative.encode("utf-8") + data = (root / relative).read_bytes() + digest.update(len(encoded).to_bytes(4, "big")) + digest.update(encoded) + digest.update(len(data).to_bytes(8, "big")) + digest.update(data) + return digest.hexdigest() + + +def require_exact_source(root: Path, source_sha: str, tree_sha: str) -> None: + if not HEX40.fullmatch(source_sha) or not HEX40.fullmatch(tree_sha): + raise AssuranceError("source and tree identities must be lowercase 40-hex SHA-1 values") + actual_source = git(root, "rev-parse", "HEAD") + actual_tree = git(root, "rev-parse", "HEAD^{tree}") + if actual_source != source_sha: + raise AssuranceError(f"source SHA mismatch: expected {source_sha}, found {actual_source}") + if actual_tree != tree_sha: + raise AssuranceError(f"tree SHA mismatch: expected {tree_sha}, found {actual_tree}") + dirty = git(root, "status", "--porcelain=v1", "--untracked-files=all") + if dirty: + raise AssuranceError(f"source worktree is dirty or has unexpected files: {dirty.splitlines()[0]}") + + +def validate_workflow_trust( + evidence: dict[str, Any], workflows: list[str], actions: list[str] +) -> None: + if evidence.get("schema") != 1 or evidence.get("ok") is not True: + raise AssuranceError("workflow trust evidence is not a successful schema-1 audit") + if evidence.get("findings") != []: + raise AssuranceError("workflow trust evidence contains findings") + if evidence.get("workflows") != workflows: + raise AssuranceError("workflow trust evidence does not cover the exact workflow set") + if evidence.get("action_metadata") != actions: + raise AssuranceError("workflow trust evidence does not cover both exact Action metadata forms") + + +def validate_dependency_inventory(evidence: dict[str, Any]) -> None: + if evidence.get("schema") != 2 or evidence.get("ok") is not True: + raise AssuranceError("dependency inventory is not a successful schema-2 exact graph") + if evidence.get("unknown_license") != []: + raise AssuranceError("dependency inventory contains unknown third-party license metadata") + if not isinstance(evidence.get("packages"), list) or evidence.get("package_count") != len( + evidence["packages"] + ): + raise AssuranceError("dependency inventory package count is inconsistent") + + +def require_head(proof: dict[str, Any], source_sha: str, label: str) -> None: + if proof.get("schema") != 1 or proof.get("head_sha") != source_sha: + raise AssuranceError(f"{label} proof is not bound to the exact source SHA") + + +def build_summary(root: Path, evidence_dir: Path, source_sha: str, tree_sha: str) -> dict[str, Any]: + root = root.resolve() + evidence_dir = evidence_dir.resolve() + require_exact_source(root, source_sha, tree_sha) + + paths = tracked_files(root) + workflows, actions = security_surfaces(paths) + workflow_trust = read_json(evidence_dir / EVIDENCE_FILES["workflow_trust"]) + dependency_inventory = read_json(evidence_dir / EVIDENCE_FILES["dependency_inventory"]) + cargo_deny = read_json(evidence_dir / EVIDENCE_FILES["cargo_deny"]) + cargo_audit = read_json(evidence_dir / EVIDENCE_FILES["cargo_audit"]) + cargo_audit_result = read_json(evidence_dir / EVIDENCE_FILES["cargo_audit_result"]) + zizmor = read_json(evidence_dir / EVIDENCE_FILES["zizmor"]) + + validate_workflow_trust(workflow_trust, workflows, actions) + validate_dependency_inventory(dependency_inventory) + + cargo_lock_sha = sha256_file(root / "Cargo.lock") + deny_sha = sha256_file(root / "deny.toml") + + require_head(cargo_deny, source_sha, "cargo-deny") + if ( + cargo_deny.get("action_commit") != CARGO_DENY_ACTION + or cargo_deny.get("cargo_deny_version") != CARGO_DENY_VERSION + or cargo_deny.get("cargo_lock_sha256") != cargo_lock_sha + or cargo_deny.get("deny_toml_sha256") != deny_sha + or cargo_deny.get("checks") != ["advisories", "bans", "licenses", "sources"] + ): + raise AssuranceError("cargo-deny proof identity/configuration mismatch") + + require_head(cargo_audit, source_sha, "cargo-audit") + if ( + cargo_audit.get("cargo_audit_version") != CARGO_AUDIT_VERSION + or cargo_audit.get("cargo_lock_sha256") != cargo_lock_sha + or cargo_audit.get("exit_code") != 0 + or cargo_audit.get("advisory_db_origin") != RUSTSEC_ORIGIN + or not HEX40.fullmatch(str(cargo_audit.get("advisory_db_commit", ""))) + ): + raise AssuranceError("cargo-audit proof identity/result mismatch") + if cargo_audit_result.get("vulnerabilities", {}).get("found") is True: + raise AssuranceError("cargo-audit result reports RustSec vulnerabilities") + + require_head(zizmor, source_sha, "zizmor") + if ( + zizmor.get("action_commit") != ZIZMOR_ACTION + or zizmor.get("zizmor_version") != ZIZMOR_VERSION + or zizmor.get("min_severity") != "medium" + or zizmor.get("online_audits") is not False + or zizmor.get("advanced_security") is not False + ): + raise AssuranceError("zizmor proof identity/policy mismatch") + + config = { + path: sha256_file(root / path) + for path in CONFIG_PATHS + } + surface_paths = sorted(set(workflows + actions)) + evidence_sha256 = { + key: sha256_file(evidence_dir / filename) + for key, filename in sorted(EVIDENCE_FILES.items()) + } + + return { + "config_sha256": config, + "dependency_graph": { + "package_count": dependency_inventory["package_count"], + "schema": dependency_inventory["schema"], + }, + "evidence_sha256": evidence_sha256, + "execution_identities": { + "actions": { + "actions/checkout": CHECKOUT_ACTION, + "EmbarkStudios/cargo-deny-action": CARGO_DENY_ACTION, + "zizmorcore/zizmor-action": ZIZMOR_ACTION, + }, + "containers": [], + "tools": { + "cargo-audit": CARGO_AUDIT_VERSION, + "cargo-deny": CARGO_DENY_VERSION, + "zizmor": ZIZMOR_VERSION, + }, + }, + "rustsec_advisory_db_commit": cargo_audit["advisory_db_commit"], + "schema": 1, + "source": {"sha": source_sha, "tree": tree_sha}, + "workflow_surface": { + "action_metadata": actions, + "sha256": surface_digest(root, surface_paths), + "workflows": workflows, + }, + } + + +def render_summary(summary: dict[str, Any]) -> bytes: + return (json.dumps(summary, indent=2, sort_keys=True, separators=(",", ": ")) + "\n").encode( + "utf-8" + ) + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--root", type=Path, default=Path(".")) + parser.add_argument("--evidence-dir", type=Path, required=True) + parser.add_argument("--source-sha", required=True) + parser.add_argument("--tree-sha", required=True) + parser.add_argument("--output", type=Path, required=True) + args = parser.parse_args() + try: + summary = build_summary(args.root, args.evidence_dir, args.source_sha, args.tree_sha) + rendered = render_summary(summary) + args.output.write_bytes(rendered) + except (AssuranceError, OSError, UnicodeError) as error: + print(json.dumps({"error": str(error), "ok": False}, sort_keys=True)) + return 1 + digest = sha256_bytes(rendered) + print(f"AF01_ASSURANCE_SHA256={digest}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) From 4561eb5af464e2fed3bc10d39fa6e4e6e1134302 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:29:18 +0300 Subject: [PATCH 07/62] test(af01): cover assurance proof counterexamples --- .../test_build_af01_assurance_summary.py | 203 ++++++++++++++++++ 1 file changed, 203 insertions(+) create mode 100644 .github/scripts/test_build_af01_assurance_summary.py diff --git a/.github/scripts/test_build_af01_assurance_summary.py b/.github/scripts/test_build_af01_assurance_summary.py new file mode 100644 index 00000000..2057a461 --- /dev/null +++ b/.github/scripts/test_build_af01_assurance_summary.py @@ -0,0 +1,203 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import hashlib +import importlib.util +import json +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path + +MODULE_PATH = Path(__file__).with_name("build_af01_assurance_summary.py") +SPEC = importlib.util.spec_from_file_location("af01_assurance_summary", MODULE_PATH) +assert SPEC is not None and SPEC.loader is not None +SUMMARY = importlib.util.module_from_spec(SPEC) +sys.modules[SPEC.name] = SUMMARY +SPEC.loader.exec_module(SUMMARY) + + +def run(root: Path, *args: str) -> str: + return subprocess.check_output(["git", "-C", str(root), *args], text=True).strip() + + +def write_json(path: Path, value: object) -> None: + path.write_text(json.dumps(value, indent=2, sort_keys=True) + "\n", encoding="utf-8") + + +class AssuranceSummaryTests(unittest.TestCase): + def setUp(self) -> None: + self.temp = tempfile.TemporaryDirectory() + base = Path(self.temp.name) + self.root = base / "repo" + self.evidence = base / "evidence" + self.root.mkdir() + self.evidence.mkdir() + + files = { + ".github/workflows/ci.yml": "name: ci\non: [pull_request]\n", + ".github/workflow-trust-policy.json": "{\"schema\":1}\n", + "action.yaml": "name: fixture\nruns:\n using: composite\n steps: []\n", + "Cargo.lock": "# fixture lock\n", + "deny.toml": "[bans]\nwildcards = \"deny\"\n", + } + for relative, content in files.items(): + path = self.root / relative + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(content, encoding="utf-8") + + subprocess.check_call(["git", "init", "-b", "main", str(self.root)], stdout=subprocess.DEVNULL) + subprocess.check_call(["git", "-C", str(self.root), "config", "user.email", "af01@example.invalid"]) + subprocess.check_call(["git", "-C", str(self.root), "config", "user.name", "AF-01 Test"]) + subprocess.check_call(["git", "-C", str(self.root), "add", "."]) + subprocess.check_call(["git", "-C", str(self.root), "commit", "-m", "fixture"], stdout=subprocess.DEVNULL) + self.source = run(self.root, "rev-parse", "HEAD") + self.tree = run(self.root, "rev-parse", "HEAD^{tree}") + self._write_valid_evidence() + + def tearDown(self) -> None: + self.temp.cleanup() + + def _sha(self, relative: str) -> str: + return hashlib.sha256((self.root / relative).read_bytes()).hexdigest() + + def _write_valid_evidence(self) -> None: + write_json( + self.evidence / "workflow-trust.json", + { + "schema": 1, + "ok": True, + "workflows": [".github/workflows/ci.yml"], + "action_metadata": ["action.yaml"], + "findings": [], + }, + ) + write_json( + self.evidence / "dependency-inventory.json", + { + "schema": 2, + "ok": True, + "package_count": 0, + "packages": [], + "unknown_license": [], + }, + ) + write_json( + self.evidence / "cargo-deny-proof.json", + { + "schema": 1, + "head_sha": self.source, + "action_commit": SUMMARY.CARGO_DENY_ACTION, + "cargo_deny_version": SUMMARY.CARGO_DENY_VERSION, + "cargo_lock_sha256": self._sha("Cargo.lock"), + "deny_toml_sha256": self._sha("deny.toml"), + "checks": ["advisories", "bans", "licenses", "sources"], + }, + ) + write_json( + self.evidence / "cargo-audit-proof.json", + { + "schema": 1, + "head_sha": self.source, + "cargo_audit_version": SUMMARY.CARGO_AUDIT_VERSION, + "cargo_lock_sha256": self._sha("Cargo.lock"), + "exit_code": 0, + "advisory_db_origin": SUMMARY.RUSTSEC_ORIGIN, + "advisory_db_commit": "a" * 40, + }, + ) + write_json( + self.evidence / "cargo-audit.json", + {"vulnerabilities": {"found": False, "count": 0, "list": []}}, + ) + write_json( + self.evidence / "zizmor-proof.json", + { + "schema": 1, + "head_sha": self.source, + "action_commit": SUMMARY.ZIZMOR_ACTION, + "zizmor_version": SUMMARY.ZIZMOR_VERSION, + "min_severity": "medium", + "online_audits": False, + "advanced_security": False, + }, + ) + + def build(self) -> dict[str, object]: + return SUMMARY.build_summary(self.root, self.evidence, self.source, self.tree) + + def test_repeated_summary_is_byte_identical(self) -> None: + first = SUMMARY.render_summary(self.build()) + second = SUMMARY.render_summary(self.build()) + self.assertEqual(first, second) + self.assertEqual(SUMMARY.sha256_bytes(first), SUMMARY.sha256_bytes(second)) + + def test_source_sha_mismatch_fails_closed(self) -> None: + with self.assertRaisesRegex(SUMMARY.AssuranceError, "source SHA mismatch"): + SUMMARY.build_summary(self.root, self.evidence, "0" * 40, self.tree) + + def test_tree_sha_mismatch_fails_closed(self) -> None: + with self.assertRaisesRegex(SUMMARY.AssuranceError, "tree SHA mismatch"): + SUMMARY.build_summary(self.root, self.evidence, self.source, "0" * 40) + + def test_missing_required_evidence_fails_closed(self) -> None: + (self.evidence / "cargo-deny-proof.json").unlink() + with self.assertRaisesRegex(SUMMARY.AssuranceError, "required evidence is missing"): + self.build() + + def test_malformed_evidence_fails_closed(self) -> None: + (self.evidence / "dependency-inventory.json").write_text("{", encoding="utf-8") + with self.assertRaisesRegex(SUMMARY.AssuranceError, "invalid JSON evidence"): + self.build() + + def test_permission_policy_mismatch_fails_closed(self) -> None: + write_json( + self.evidence / "workflow-trust.json", + { + "schema": 1, + "ok": False, + "workflows": [".github/workflows/ci.yml"], + "action_metadata": ["action.yaml"], + "findings": [{"code": "permission_mismatch"}], + }, + ) + with self.assertRaisesRegex(SUMMARY.AssuranceError, "not a successful schema-1 audit"): + self.build() + + def test_mutable_proof_container_fails_closed(self) -> None: + write_json( + self.evidence / "workflow-trust.json", + { + "schema": 1, + "ok": False, + "workflows": [".github/workflows/ci.yml"], + "action_metadata": ["action.yaml"], + "findings": [{"code": "mutable_container", "path": ".github/workflows/ci.yml"}], + }, + ) + with self.assertRaisesRegex(SUMMARY.AssuranceError, "not a successful schema-1 audit"): + self.build() + + def test_missing_action_yaml_coverage_fails_closed(self) -> None: + write_json( + self.evidence / "workflow-trust.json", + { + "schema": 1, + "ok": True, + "workflows": [".github/workflows/ci.yml"], + "action_metadata": [], + "findings": [], + }, + ) + with self.assertRaisesRegex(SUMMARY.AssuranceError, "does not cover both exact Action metadata forms"): + self.build() + + def test_dirty_or_unexpected_source_fails_closed(self) -> None: + (self.root / "unexpected.txt").write_text("dirty\n", encoding="utf-8") + with self.assertRaisesRegex(SUMMARY.AssuranceError, "dirty or has unexpected files"): + self.build() + + +if __name__ == "__main__": + unittest.main() From 606ba16c974728e646b806a5f5d486aa8d6c2632 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:30:10 +0300 Subject: [PATCH 08/62] ci(af01): add deterministic assurance proof workflow --- .github/workflows/af01-assurance-proof.yml | 180 +++++++++++++++++++++ 1 file changed, 180 insertions(+) create mode 100644 .github/workflows/af01-assurance-proof.yml diff --git a/.github/workflows/af01-assurance-proof.yml b/.github/workflows/af01-assurance-proof.yml new file mode 100644 index 00000000..14d8c5e6 --- /dev/null +++ b/.github/workflows/af01-assurance-proof.yml @@ -0,0 +1,180 @@ +name: af01-assurance-proof + +on: + pull_request: + push: + branches: + - main + workflow_dispatch: + +permissions: + contents: read + +env: + AF01_SOURCE_SHA: ${{ github.event.pull_request.head.sha || github.sha }} + AF01_EVIDENCE_DIR: /tmp/af01-assurance-evidence + +jobs: + assurance-proof: + runs-on: ubuntu-24.04 + timeout-minutes: 35 + steps: + - uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5 / Node 24 + with: + ref: ${{ env.AF01_SOURCE_SHA }} + persist-credentials: false + - uses: dtolnay/rust-toolchain@032958afbdc797a9164d3bc0b56325c1308924a5 # 1.97.1 + - name: Assert exact source and run assurance counterexamples + run: | + set -euo pipefail + test "$(git rev-parse HEAD)" = "$AF01_SOURCE_SHA" + python3 .github/scripts/test_build_af01_assurance_summary.py + mkdir -p "$AF01_EVIDENCE_DIR" + - name: Generate exact workflow-trust evidence + run: | + set -euo pipefail + python3 .github/scripts/audit_workflow_trust.py > "$AF01_EVIDENCE_DIR/workflow-trust.json" + - name: Generate exact locked dependency evidence + run: | + set -euo pipefail + cargo metadata --locked --format-version 1 > /tmp/af01-cargo-metadata.json + python3 .github/scripts/summarize_cargo_metadata.py \ + < /tmp/af01-cargo-metadata.json \ + > "$AF01_EVIDENCE_DIR/dependency-inventory.json" + - name: Record cargo-deny proof identity + run: | + set -euo pipefail + python3 - <<'PY' + import hashlib + import json + import os + from pathlib import Path + + def sha256(path: str) -> str: + return hashlib.sha256(Path(path).read_bytes()).hexdigest() + + proof = { + "action_commit": "3c6349835b2b7b196a839186cb8b78e02f7b5f25", + "cargo_deny_version": "0.20.2", + "cargo_lock_sha256": sha256("Cargo.lock"), + "checks": ["advisories", "bans", "licenses", "sources"], + "deny_toml_sha256": sha256("deny.toml"), + "head_sha": os.environ["AF01_SOURCE_SHA"], + "schema": 1, + } + Path(os.environ["AF01_EVIDENCE_DIR"], "cargo-deny-proof.json").write_text( + json.dumps(proof, indent=2, sort_keys=True) + "\n", encoding="utf-8" + ) + PY + - name: Enforce cargo-deny dependency policy + uses: EmbarkStudios/cargo-deny-action@3c6349835b2b7b196a839186cb8b78e02f7b5f25 # v2.1.1 / cargo-deny 0.20.2 + with: + command: check + arguments: --all-features + command-arguments: advisories bans licenses sources + log-level: warn + - name: Install pinned cargo-audit + run: cargo install cargo-audit --version 0.22.2 --locked + - name: Audit exact Cargo.lock and record RustSec identity + run: | + set -euo pipefail + set +e + cargo audit --file Cargo.lock --json > "$AF01_EVIDENCE_DIR/cargo-audit.json" + audit_status=$? + set -e + python3 - "$audit_status" <<'PY' + import hashlib + import json + import os + import subprocess + import sys + from pathlib import Path + + status = int(sys.argv[1]) + db = Path.home() / ".cargo" / "advisory-db" + if not (db / ".git").is_dir(): + raise SystemExit("RustSec advisory database identity is missing") + proof = { + "advisory_db_commit": subprocess.check_output( + ["git", "-C", str(db), "rev-parse", "HEAD"], text=True + ).strip(), + "advisory_db_origin": subprocess.check_output( + ["git", "-C", str(db), "remote", "get-url", "origin"], text=True + ).strip(), + "cargo_audit_version": "0.22.2", + "cargo_lock_sha256": hashlib.sha256(Path("Cargo.lock").read_bytes()).hexdigest(), + "exit_code": status, + "head_sha": os.environ["AF01_SOURCE_SHA"], + "schema": 1, + } + Path(os.environ["AF01_EVIDENCE_DIR"], "cargo-audit-proof.json").write_text( + json.dumps(proof, indent=2, sort_keys=True) + "\n", encoding="utf-8" + ) + PY + test "$audit_status" -eq 0 + - name: Record zizmor proof identity + run: | + set -euo pipefail + python3 - <<'PY' + import json + import os + from pathlib import Path + + proof = { + "action_commit": "3dc1ecc9bcb9e94e9b2c709687979e1298497054", + "advanced_security": False, + "head_sha": os.environ["AF01_SOURCE_SHA"], + "min_severity": "medium", + "online_audits": False, + "schema": 1, + "zizmor_version": "1.29.0", + } + Path(os.environ["AF01_EVIDENCE_DIR"], "zizmor-proof.json").write_text( + json.dumps(proof, indent=2, sort_keys=True) + "\n", encoding="utf-8" + ) + PY + - name: Enforce workflow audit with zizmor + uses: zizmorcore/zizmor-action@3dc1ecc9bcb9e94e9b2c709687979e1298497054 # v0.6.2 / zizmor 1.29.0 + with: + inputs: . + collect: all + online-audits: false + persona: regular + min-severity: medium + version: 1.29.0 + advanced-security: false + color: false + annotations: false + fail-on-no-inputs: true + - name: Build deterministic exact-head assurance summary + run: | + set -euo pipefail + tree_sha="$(git rev-parse 'HEAD^{tree}')" + first=/tmp/assurance-summary.json + second=/tmp/assurance-summary-repeat.json + first_line="$(python3 .github/scripts/build_af01_assurance_summary.py \ + --evidence-dir "$AF01_EVIDENCE_DIR" \ + --source-sha "$AF01_SOURCE_SHA" \ + --tree-sha "$tree_sha" \ + --output "$first")" + second_line="$(python3 .github/scripts/build_af01_assurance_summary.py \ + --evidence-dir "$AF01_EVIDENCE_DIR" \ + --source-sha "$AF01_SOURCE_SHA" \ + --tree-sha "$tree_sha" \ + --output "$second")" + test "$first_line" = "$second_line" + cmp --silent "$first" "$second" + printf '%s\n' "$first_line" | tee /tmp/AF01_ASSURANCE_SHA256.txt + git status --porcelain=v1 --untracked-files=all | tee /tmp/af01-source-status.txt + test ! -s /tmp/af01-source-status.txt + - name: Retain AF-01 assurance proof + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 + with: + name: af01-assurance-proof + path: | + /tmp/assurance-summary.json + /tmp/AF01_ASSURANCE_SHA256.txt + /tmp/af01-source-status.txt + /tmp/af01-assurance-evidence/*.json + if-no-files-found: error + retention-days: 7 From 35999a43bc2907c526fbe83f0f81c8b5ccd30363 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:32:15 +0300 Subject: [PATCH 09/62] ci(af01): authorize assurance proof job --- .github/workflow-trust-policy.json | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/.github/workflow-trust-policy.json b/.github/workflow-trust-policy.json index 151fdfd8..e8fef6bc 100644 --- a/.github/workflow-trust-policy.json +++ b/.github/workflow-trust-policy.json @@ -22,6 +22,17 @@ "require_external_uses_full_sha": "External Actions and reusable workflows must be bound to an immutable 40-hex commit so mutable tags or branches cannot change executed code without a repository diff." }, "workflows": { + ".github/workflows/af01-assurance-proof.yml": { + "jobs": { + "assurance-proof": { + "permissions": { + "contents": "read" + }, + "runner": "ubuntu-24.04", + "timeout_minutes": 35 + } + } + }, ".github/workflows/af01-scorecard.yml": { "jobs": { "scorecard": { From 5098361808042982dc193fd274734a6e4f366c9d Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:32:51 +0300 Subject: [PATCH 10/62] ci(af01): bound assurance proof to timeout policy --- .github/workflows/af01-assurance-proof.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/af01-assurance-proof.yml b/.github/workflows/af01-assurance-proof.yml index 14d8c5e6..ecefb5c5 100644 --- a/.github/workflows/af01-assurance-proof.yml +++ b/.github/workflows/af01-assurance-proof.yml @@ -17,7 +17,7 @@ env: jobs: assurance-proof: runs-on: ubuntu-24.04 - timeout-minutes: 35 + timeout-minutes: 30 steps: - uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5 / Node 24 with: From 35cfe84118969b9c1fb3ca4d8cd3faef8e1d0918 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:33:14 +0300 Subject: [PATCH 11/62] ci(af01): keep assurance timeout within baseline --- .github/workflow-trust-policy.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflow-trust-policy.json b/.github/workflow-trust-policy.json index e8fef6bc..68fead8f 100644 --- a/.github/workflow-trust-policy.json +++ b/.github/workflow-trust-policy.json @@ -29,7 +29,7 @@ "contents": "read" }, "runner": "ubuntu-24.04", - "timeout_minutes": 35 + "timeout_minutes": 30 } } }, From 5919753ff680e35f6906b2c9d06ba99824dbaa41 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:34:12 +0300 Subject: [PATCH 12/62] ci(af01): define universal required-check topology --- .github/required-checks.json | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 .github/required-checks.json diff --git a/.github/required-checks.json b/.github/required-checks.json new file mode 100644 index 00000000..d7006976 --- /dev/null +++ b/.github/required-checks.json @@ -0,0 +1,21 @@ +{ + "schema": 1, + "protected_branch": "main", + "checks": [ + { + "context": "rust", + "workflow": ".github/workflows/ci.yml", + "job": "rust" + }, + { + "context": "assurance-proof", + "workflow": ".github/workflows/af01-assurance-proof.yml", + "job": "assurance-proof" + }, + { + "context": "scorecard", + "workflow": ".github/workflows/af01-scorecard.yml", + "job": "scorecard" + } + ] +} From 075c2a152d2557710eb7f73d31e74a67153f3bcd Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:34:32 +0300 Subject: [PATCH 13/62] test(af01): reject nonterminal required-check topology --- .../scripts/test_required_check_topology.py | 124 ++++++++++++++++++ 1 file changed, 124 insertions(+) create mode 100644 .github/scripts/test_required_check_topology.py diff --git a/.github/scripts/test_required_check_topology.py b/.github/scripts/test_required_check_topology.py new file mode 100644 index 00000000..657d8749 --- /dev/null +++ b/.github/scripts/test_required_check_topology.py @@ -0,0 +1,124 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import json +import unittest +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[2] +CONFIG = ROOT / ".github" / "required-checks.json" + + +def pull_request_children(lines: list[str]) -> list[str]: + try: + start = lines.index(" pull_request:") + except ValueError as error: + raise AssertionError("required-check workflow must use a mapping-form pull_request trigger") from error + children: list[str] = [] + for line in lines[start + 1 :]: + if line and not line.startswith(" "): + break + if line.startswith(" ") and not line.startswith(" ") and line.strip(): + break + if line.strip() and not line.lstrip().startswith("#"): + children.append(line.strip()) + return children + + +def job_range(lines: list[str], job: str) -> tuple[int, int]: + try: + jobs_index = lines.index("jobs:") + except ValueError as error: + raise AssertionError("required-check workflow has no jobs mapping") from error + target = f" {job}:" + try: + start = lines.index(target, jobs_index + 1) + except ValueError as error: + raise AssertionError(f"required-check workflow has no job {job!r}") from error + end = len(lines) + for index in range(start + 1, len(lines)): + line = lines[index] + if line and not line.startswith(" "): + end = index + break + if line.startswith(" ") and not line.startswith(" ") and line.strip(): + end = index + break + return start, end + + +def assert_universal_required_job(path: Path, job: str) -> None: + lines = path.read_text(encoding="utf-8").splitlines() + children = pull_request_children(lines) + if children: + raise AssertionError( + f"{path.relative_to(ROOT)} pull_request trigger is filtered or narrowed: {children!r}" + ) + start, end = job_range(lines, job) + for line in lines[start + 1 : end]: + if line.startswith(" if:"): + raise AssertionError( + f"{path.relative_to(ROOT)} job {job!r} has a job-level conditional and is not universally terminal" + ) + + +class RequiredCheckTopologyTests(unittest.TestCase): + def load_config(self) -> dict[str, object]: + value = json.loads(CONFIG.read_text(encoding="utf-8")) + self.assertIsInstance(value, dict) + return value + + def test_selected_checks_are_unique_and_universal(self) -> None: + config = self.load_config() + self.assertEqual(config.get("schema"), 1) + self.assertEqual(config.get("protected_branch"), "main") + checks = config.get("checks") + self.assertIsInstance(checks, list) + self.assertGreater(len(checks), 0) + contexts: set[str] = set() + pairs: set[tuple[str, str]] = set() + for index, item in enumerate(checks): + with self.subTest(index=index): + self.assertIsInstance(item, dict) + self.assertEqual(set(item), {"context", "workflow", "job"}) + context = item["context"] + workflow = item["workflow"] + job = item["job"] + self.assertIsInstance(context, str) + self.assertIsInstance(workflow, str) + self.assertIsInstance(job, str) + self.assertNotIn(context, contexts) + self.assertNotIn((workflow, job), pairs) + contexts.add(context) + pairs.add((workflow, job)) + path = ROOT / workflow + self.assertTrue(path.is_file(), workflow) + assert_universal_required_job(path, job) + + def test_counterexample_path_filtered_workflow_is_rejected(self) -> None: + with self.subTest("paths"): + lines = [ + "name: example", + "on:", + " pull_request:", + " paths:", + " - src/**", + "jobs:", + " gate:", + " runs-on: ubuntu-24.04", + ] + self.assertEqual(pull_request_children(lines), ["paths:", "- src/**"]) + + def test_counterexample_job_level_if_is_rejected(self) -> None: + with self.assertRaisesRegex(AssertionError, "job-level conditional"): + with __import__("tempfile").TemporaryDirectory() as directory: + path = Path(directory) / "workflow.yml" + path.write_text( + "name: example\non:\n pull_request:\njobs:\n gate:\n if: github.actor != 'x'\n runs-on: ubuntu-24.04\n", + encoding="utf-8", + ) + assert_universal_required_job(path, "gate") + + +if __name__ == "__main__": + unittest.main() From 55df56ea9e94ea9b54d78242a4a8440cc44d1a9b Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:34:59 +0300 Subject: [PATCH 14/62] test(af01): make topology counterexample deterministic --- .../scripts/test_required_check_topology.py | 49 +++++++++++-------- 1 file changed, 28 insertions(+), 21 deletions(-) diff --git a/.github/scripts/test_required_check_topology.py b/.github/scripts/test_required_check_topology.py index 657d8749..dbf47f07 100644 --- a/.github/scripts/test_required_check_topology.py +++ b/.github/scripts/test_required_check_topology.py @@ -2,6 +2,7 @@ from __future__ import annotations import json +import tempfile import unittest from pathlib import Path @@ -9,6 +10,13 @@ CONFIG = ROOT / ".github" / "required-checks.json" +def display_path(path: Path) -> str: + try: + return str(path.relative_to(ROOT)) + except ValueError: + return str(path) + + def pull_request_children(lines: list[str]) -> list[str]: try: start = lines.index(" pull_request:") @@ -52,13 +60,13 @@ def assert_universal_required_job(path: Path, job: str) -> None: children = pull_request_children(lines) if children: raise AssertionError( - f"{path.relative_to(ROOT)} pull_request trigger is filtered or narrowed: {children!r}" + f"{display_path(path)} pull_request trigger is filtered or narrowed: {children!r}" ) start, end = job_range(lines, job) for line in lines[start + 1 : end]: if line.startswith(" if:"): raise AssertionError( - f"{path.relative_to(ROOT)} job {job!r} has a job-level conditional and is not universally terminal" + f"{display_path(path)} job {job!r} has a job-level conditional and is not universally terminal" ) @@ -96,27 +104,26 @@ def test_selected_checks_are_unique_and_universal(self) -> None: assert_universal_required_job(path, job) def test_counterexample_path_filtered_workflow_is_rejected(self) -> None: - with self.subTest("paths"): - lines = [ - "name: example", - "on:", - " pull_request:", - " paths:", - " - src/**", - "jobs:", - " gate:", - " runs-on: ubuntu-24.04", - ] - self.assertEqual(pull_request_children(lines), ["paths:", "- src/**"]) + lines = [ + "name: example", + "on:", + " pull_request:", + " paths:", + " - src/**", + "jobs:", + " gate:", + " runs-on: ubuntu-24.04", + ] + self.assertEqual(pull_request_children(lines), ["paths:", "- src/**"]) def test_counterexample_job_level_if_is_rejected(self) -> None: - with self.assertRaisesRegex(AssertionError, "job-level conditional"): - with __import__("tempfile").TemporaryDirectory() as directory: - path = Path(directory) / "workflow.yml" - path.write_text( - "name: example\non:\n pull_request:\njobs:\n gate:\n if: github.actor != 'x'\n runs-on: ubuntu-24.04\n", - encoding="utf-8", - ) + with tempfile.TemporaryDirectory() as directory: + path = Path(directory) / "workflow.yml" + path.write_text( + "name: example\non:\n pull_request:\njobs:\n gate:\n if: github.actor != 'x'\n runs-on: ubuntu-24.04\n", + encoding="utf-8", + ) + with self.assertRaisesRegex(AssertionError, "job-level conditional"): assert_universal_required_job(path, "gate") From 341b1fd8f6241b30f0fd6399faadb6c2de321e87 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:35:39 +0300 Subject: [PATCH 15/62] docs(af01): prepare exact main ruleset contract --- .github/main-ruleset.json | 53 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) create mode 100644 .github/main-ruleset.json diff --git a/.github/main-ruleset.json b/.github/main-ruleset.json new file mode 100644 index 00000000..24bf9515 --- /dev/null +++ b/.github/main-ruleset.json @@ -0,0 +1,53 @@ +{ + "name": "commandF main assurance", + "target": "branch", + "enforcement": "active", + "bypass_actors": [], + "conditions": { + "ref_name": { + "include": [ + "refs/heads/main" + ], + "exclude": [] + } + }, + "rules": [ + { + "type": "deletion" + }, + { + "type": "non_fast_forward" + }, + { + "type": "pull_request", + "parameters": { + "allowed_merge_methods": [ + "merge" + ], + "dismiss_stale_reviews_on_push": true, + "require_code_owner_review": false, + "require_last_push_approval": true, + "required_approving_review_count": 1, + "required_review_thread_resolution": true + } + }, + { + "type": "required_status_checks", + "parameters": { + "do_not_enforce_on_create": false, + "required_status_checks": [ + { + "context": "rust" + }, + { + "context": "assurance-proof" + }, + { + "context": "scorecard" + } + ], + "strict_required_status_checks_policy": true + } + } + ] +} From 8a740a8ce99cea51a3fdc5d63a2a800cea77fa7e Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:36:22 +0300 Subject: [PATCH 16/62] test(af01): lock main ruleset contract to topology --- .github/scripts/test_main_ruleset_contract.py | 64 +++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 .github/scripts/test_main_ruleset_contract.py diff --git a/.github/scripts/test_main_ruleset_contract.py b/.github/scripts/test_main_ruleset_contract.py new file mode 100644 index 00000000..2bca6288 --- /dev/null +++ b/.github/scripts/test_main_ruleset_contract.py @@ -0,0 +1,64 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import json +import unittest +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[2] +TOPOLOGY = ROOT / ".github" / "required-checks.json" +RULESET = ROOT / ".github" / "main-ruleset.json" + + +class MainRulesetContractTests(unittest.TestCase): + def test_ruleset_matches_required_check_topology_and_governance(self) -> None: + topology = json.loads(TOPOLOGY.read_text(encoding="utf-8")) + ruleset = json.loads(RULESET.read_text(encoding="utf-8")) + + self.assertEqual(ruleset.get("name"), "commandF main assurance") + self.assertEqual(ruleset.get("target"), "branch") + self.assertEqual(ruleset.get("enforcement"), "active") + self.assertEqual(ruleset.get("bypass_actors"), []) + self.assertEqual( + ruleset.get("conditions"), + {"ref_name": {"include": ["refs/heads/main"], "exclude": []}}, + ) + + rules = ruleset.get("rules") + self.assertIsInstance(rules, list) + by_type = {rule.get("type"): rule for rule in rules if isinstance(rule, dict)} + self.assertEqual(set(by_type), {"deletion", "non_fast_forward", "pull_request", "required_status_checks"}) + self.assertEqual(by_type["deletion"], {"type": "deletion"}) + self.assertEqual(by_type["non_fast_forward"], {"type": "non_fast_forward"}) + + pull_request = by_type["pull_request"].get("parameters") + self.assertEqual( + pull_request, + { + "allowed_merge_methods": ["merge"], + "dismiss_stale_reviews_on_push": True, + "require_code_owner_review": False, + "require_last_push_approval": True, + "required_approving_review_count": 1, + "required_review_thread_resolution": True, + }, + ) + + required = by_type["required_status_checks"].get("parameters") + self.assertIsInstance(required, dict) + self.assertFalse(required.get("do_not_enforce_on_create")) + self.assertTrue(required.get("strict_required_status_checks_policy")) + contexts = [item.get("context") for item in required.get("required_status_checks", [])] + expected = [item["context"] for item in topology["checks"]] + self.assertEqual(contexts, expected) + self.assertEqual(contexts, ["rust", "assurance-proof", "scorecard"]) + + def test_ruleset_has_no_unreviewed_integration_binding(self) -> None: + ruleset = json.loads(RULESET.read_text(encoding="utf-8")) + required_rule = next(rule for rule in ruleset["rules"] if rule["type"] == "required_status_checks") + for check in required_rule["parameters"]["required_status_checks"]: + self.assertEqual(set(check), {"context"}) + + +if __name__ == "__main__": + unittest.main() From bb3ba26e39f83bbbe71fcfc1b81342fe59b6ee63 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:37:39 +0300 Subject: [PATCH 17/62] docs(security): link private vulnerability reporting --- SECURITY.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/SECURITY.md b/SECURITY.md index 793be13b..b578dd12 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -8,7 +8,11 @@ commandF is currently developed from the canonical `main` branch. Until a stable Please do not publish exploit details, credentials, private data, or a working proof of concept in a public issue. -Prefer GitHub's private vulnerability-reporting flow for this repository when the **Report a vulnerability** option is available under the repository Security tab. Include: +Prefer GitHub's private vulnerability-reporting flow for this repository when available: + +https://github.com/TheHalfMoon/commandF/security/advisories/new + +Include: - the affected commandF component and exact commit or release identity; - the security property that can be violated; @@ -17,7 +21,7 @@ Prefer GitHub's private vulnerability-reporting flow for this repository when th - any known workaround or containment; - whether the report contains embargo-sensitive details. -If the private reporting control is not available, open a public issue containing only a request for a private security contact and non-sensitive routing information. Do not include exploit steps or sensitive evidence in that issue. +If GitHub does not offer the private reporting control for this repository, open a public issue containing only a request for a private security contact and non-sensitive routing information. Do not include exploit steps or sensitive evidence in that issue. ## Scope From b77c7c478f55bf0175bb4d6d3f0dd1f3fe0dadd5 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:38:51 +0300 Subject: [PATCH 18/62] ci(af01): remove mutable Scorecard container runtime --- .github/workflows/af01-scorecard.yml | 55 ++++++++++++++++------------ 1 file changed, 32 insertions(+), 23 deletions(-) diff --git a/.github/workflows/af01-scorecard.yml b/.github/workflows/af01-scorecard.yml index 04c67589..69aa7d80 100644 --- a/.github/workflows/af01-scorecard.yml +++ b/.github/workflows/af01-scorecard.yml @@ -21,15 +21,7 @@ jobs: with: ref: ${{ github.event.pull_request.head.sha || github.sha }} persist-credentials: false - - name: Run OpenSSF Scorecard against exact checked-out source - uses: ossf/scorecard-action@2d1146689b8cda280b9bc96326124645441f03bc # v2.4.4 / Scorecard v5.5.0 - with: - results_file: af01-scorecard-local.json - results_format: json - publish_results: false - - name: Run repository-aware Scorecard posture scan - env: - GITHUB_AUTH_TOKEN: ${{ github.token }} + - name: Install digest-verified OpenSSF Scorecard 5.5.0 run: | set -euo pipefail archive=/tmp/scorecard_5.5.0_linux_amd64.tar.gz @@ -42,7 +34,21 @@ jobs: "$archive" | sha256sum --check --strict mkdir -p "$install_dir" tar -xzf "$archive" -C "$install_dir" - "$install_dir/scorecard" \ + "$install_dir/scorecard" version + - name: Run Scorecard against exact checked-out source + run: | + set -euo pipefail + /tmp/af01-scorecard-cli/scorecard \ + --local=. \ + --format=json \ + --show-details \ + > af01-scorecard-local.json + - name: Run repository-aware Scorecard posture scan + env: + GITHUB_AUTH_TOKEN: ${{ github.token }} + run: | + set -euo pipefail + /tmp/af01-scorecard-cli/scorecard \ --repo="github.com/${GITHUB_REPOSITORY}" \ --checks=Branch-Protection,Dangerous-Workflow,Dependency-Update-Tool,Pinned-Dependencies,Security-Policy,Token-Permissions,Vulnerabilities \ --format=json \ @@ -52,10 +58,9 @@ jobs: import json from pathlib import Path - path = Path("af01-scorecard-repository.json") - result = json.loads(path.read_text(encoding="utf-8")) - expected = { - "Branch-Protection", + local = json.loads(Path("af01-scorecard-local.json").read_text(encoding="utf-8")) + remote = json.loads(Path("af01-scorecard-repository.json").read_text(encoding="utf-8")) + local_required = { "Dangerous-Workflow", "Dependency-Update-Tool", "Pinned-Dependencies", @@ -63,15 +68,19 @@ jobs: "Token-Permissions", "Vulnerabilities", } - repo = result.get("repo", {}) - checks = result.get("checks") - if repo.get("name") != "github.com/TheHalfMoon/commandF": - raise SystemExit(f"unexpected repository identity: {repo!r}") - if not isinstance(checks, list): - raise SystemExit("repository Scorecard output has no checks array") - names = {item.get("name") for item in checks if isinstance(item, dict)} - if names != expected: - raise SystemExit(f"repository Scorecard checks mismatch: {sorted(names)!r}") + remote_expected = local_required | {"Branch-Protection"} + local_checks = local.get("checks") + remote_checks = remote.get("checks") + if not isinstance(local_checks, list) or not isinstance(remote_checks, list): + raise SystemExit("Scorecard output has no checks array") + local_names = {item.get("name") for item in local_checks if isinstance(item, dict)} + remote_names = {item.get("name") for item in remote_checks if isinstance(item, dict)} + if not local_required.issubset(local_names): + raise SystemExit(f"local Scorecard checks missing: {sorted(local_required - local_names)!r}") + if remote_names != remote_expected: + raise SystemExit(f"repository Scorecard checks mismatch: {sorted(remote_names)!r}") + if remote.get("repo", {}).get("name") != "github.com/TheHalfMoon/commandF": + raise SystemExit(f"unexpected repository identity: {remote.get('repo')!r}") PY - name: Retain per-check Scorecard evidence uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 From b5d813477b5b45f1a396bab1cea1c2bc10ad568f Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:39:59 +0300 Subject: [PATCH 19/62] docs(af01): disposition Stack C Scorecard findings --- .../stack-c-scorecard-posture.md | 133 ++++++++++++++++++ 1 file changed, 133 insertions(+) create mode 100644 specs/015-af-01-trusted-development-baseline/stack-c-scorecard-posture.md diff --git a/specs/015-af-01-trusted-development-baseline/stack-c-scorecard-posture.md b/specs/015-af-01-trusted-development-baseline/stack-c-scorecard-posture.md new file mode 100644 index 00000000..25d17a0f --- /dev/null +++ b/specs/015-af-01-trusted-development-baseline/stack-c-scorecard-posture.md @@ -0,0 +1,133 @@ +# AF-01 Stack C Scorecard Posture and Finding Disposition + +Status: `T031_DISPOSITIONED_DEVELOPMENT_EVIDENCE_FINAL_T041_PENDING` + +This record is posture evidence. OpenSSF Scorecard's aggregate score is not commandF correctness authority and is not used as an AF-01 semantic PASS signal. + +## Canonical repository state inspected + +Repository-aware posture scan: + +```text +canonical main at scan: 301aa5e66089859e938145870dc4a9300a25692a +canonical main tree: d8557e6992ea82c0d2bb36178cf85961243e0691 +Stack C development head: 35cfe84118969b9c1fb3ca4d8cd3faef8e1d0918 +workflow: af01-scorecard +run: 33054545670 +artifact: 9639039033 +artifact name: af01-scorecard +artifact digest: sha256:26c415bf0b08e03e0b55a2bbf6336c0c4363638485e8f89194ab625e0032dd6c +``` + +The artifact contains two independent views: + +- `af01-scorecard-local.json`: the exact checked-out Stack C source tree, used for source-file posture such as workflow policy, `SECURITY.md`, and Dependabot configuration. +- `af01-scorecard-repository.json`: a repository-aware scan of live `github.com/TheHalfMoon/commandF`, used for live GitHub posture such as Branch-Protection. + +The development run above used Scorecard v5.5.0. Stack C subsequently hardened execution further: instead of relying on `ossf/scorecard-action`'s internally mutable `docker://ghcr.io/ossf/scorecard-action:v2.4.4` image reference, the workflow downloads the official Scorecard v5.5.0 Linux amd64 release archive and verifies GitHub's published SHA-256 before execution: + +```text +scorecard version: 5.5.0 +scorecard source commit reported by result: c395761df6afe1a69e476bc60a013a94bcbc153f +release asset: scorecard_5.5.0_linux_amd64.tar.gz +release asset sha256: 83b90a05c1540ef1390db1cd5711e5fd04be9c1d8537fb84d39d02092d6a8dff +``` + +Final T041 evidence must come from the hardened exact final Stack C head; this development artifact is retained as the T031 finding/disposition input. + +## Required T031 checks + +### Branch-Protection — `0/10` on live canonical main + +Observed detail: + +```text +Warn: branch protection not enabled for branch 'main' +``` + +Disposition: `BLOCKING_T038`. + +This is not waived. T037 defines the exact checked-in `main` ruleset contract; T038 must apply it through an administrator-authorized GitHub path; T039 must read live GitHub back; and T040 must prove the effective negative-governance properties. AF-01 cannot close while this finding remains true. + +### Dangerous-Workflow — `10/10` + +Observed result: no dangerous workflow patterns detected. + +Disposition: `NO_FINDING`. + +This is complementary to, not a replacement for, the repository-owned AF-01 workflow-trust audit and zizmor gate. + +### Pinned-Dependencies — `10/10` + +Observed result: all dependencies inspected by Scorecard were pinned. + +Disposition: `NO_FINDING`. + +The AF-01 repository-owned workflow audit remains stricter authority for external Action full-SHA references, credentialless checkout, proof-container identity, and the exact tracked Action-metadata surface. + +### Token-Permissions — `10/10` + +Observed result: GitHub workflow tokens follow least privilege. + +Disposition: `NO_FINDING`. + +The checked-in workflow-trust policy remains the machine-checkable permission authority. + +### Security-Policy + +Live canonical main at the T031 scan scored `0/10` because `SECURITY.md` did not yet exist there. The exact Stack C local tree already detected `SECURITY.md` and scored `4/10`; the initial policy lacked direct linked reporting content. Stack C then added a direct GitHub private security-advisory reporting link while retaining fail-safe public routing instructions that forbid posting exploit material. + +Disposition: `FIX_IMPLEMENTED_FINAL_SCORECARD_RECHECK_REQUIRED`. + +The final exact-head Scorecard run must inspect the hardened policy. No aggregate-score target is required; the substantive acceptance criterion is a present, usable vulnerability-reporting policy without weakening security handling. + +### Dependency-Update-Tool + +Live canonical main at the T031 scan scored `0/10`. The exact Stack C local tree scored `10/10` after adding `.github/dependabot.yml` for the Cargo workspace, the Maven HL7 oracle, and GitHub Actions, each on a bounded weekly schedule. + +Disposition: `FIX_IMPLEMENTED_FINAL_SCORECARD_RECHECK_REQUIRED`. + +Dependabot does not authorize automatic dependency merges or semantic oracle upgrades; each resulting PR remains subject to repository qualification. + +### Vulnerabilities — `8/10` + +Scorecard reported two repository-level OSV/GHSA findings: + +```text +GHSA-rcgg-9c38-7xpx / CVE-2026-45292 +GHSA-269g-pwp5-87pp / CVE-2020-15250 +``` + +The first affects OpenTelemetry Java baggage propagation versions before the fixed line; the second affects JUnit4 `TemporaryFolder` before its fixed line. commandF does not declare either package directly in the Rust workspace. RustSec `cargo-audit 0.22.2` on the exact Cargo lockfile is independently enforced and was clean during Stack B qualification. + +The repository contains a separate Maven-based HL7/FHIR oracle at `tools/hl7-oracle/pom.xml`, currently pinned to the existing qualified HL7 FHIR toolchain. AF-01 freezes product/oracle semantics and therefore does not authorize blindly changing that oracle dependency graph merely to improve an aggregate posture score. + +Disposition: `BOUNDED_EXTERNAL_ORACLE_DEPENDENCY_FINDING_REQUIRES_SEPARATE_REQUALIFICATION`. + +Scope and controls: + +- no Rust/Cargo advisory waiver is created; +- no Scorecard threshold is lowered or finding hidden; +- the exact advisory identities remain visible in retained Scorecard evidence; +- the Maven oracle is not application runtime authority and remains bounded to the existing proof/oracle workflows; +- weekly Maven Dependabot discovery is enabled, but no update is automatically trusted or merged; +- a future oracle dependency update that proves the affected transitive package is removed/fixed must run the full applicable oracle/quality proof set before becoming canonical. + +Revisit/removal condition: remove this disposition only when exact dependency-tree evidence plus qualified oracle workflows prove the affected package versions are no longer present, or when authoritative vulnerability data establishes the finding is inapplicable to the pinned oracle graph. + +## Additional local Scorecard checks + +The local development view also reported: + +- License: `0/10` — no repository license file has been selected. AF-01 has no authority to choose a legal license on the founder's behalf; this is not silently treated as completed assurance. +- SAST: `0/10` — broader static-analysis program remains outside AF-01 and is retained for later assurance work. +- Fuzzing: `0/10` — explicitly retained for AF-02 adversarial test strength. +- Packaging: not applicable in this development view — stable release assurance belongs to AF-03. + +These values are not converted into fictitious AF-01 PASS claims. + +## T031 decision + +`T031 = DISPOSITIONED` + +Material Scorecard findings have explicit bounded treatment. Branch protection remains a hard Stack C blocker until T038–T040. Security-policy and dependency-update posture fixes are implemented and require final exact-head evidence. The two Java vulnerability findings remain visible and bounded to separate oracle dependency requalification rather than causing an unauthorized AF-01 semantic dependency mutation. From 8d2d1bd62a660733613becdb408b3861c77dd8f0 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:40:28 +0300 Subject: [PATCH 20/62] test(af01): include required-check topology in universal CI --- ...udit_workflow_trust_required_check_topology.py | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 .github/scripts/test_audit_workflow_trust_required_check_topology.py diff --git a/.github/scripts/test_audit_workflow_trust_required_check_topology.py b/.github/scripts/test_audit_workflow_trust_required_check_topology.py new file mode 100644 index 00000000..3c53ec67 --- /dev/null +++ b/.github/scripts/test_audit_workflow_trust_required_check_topology.py @@ -0,0 +1,15 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import importlib.util +import sys +from pathlib import Path + +TARGET = Path(__file__).with_name("test_required_check_topology.py") +SPEC = importlib.util.spec_from_file_location("af01_required_check_topology_tests", TARGET) +assert SPEC is not None and SPEC.loader is not None +MODULE = importlib.util.module_from_spec(SPEC) +sys.modules[SPEC.name] = MODULE +SPEC.loader.exec_module(MODULE) + +RequiredCheckTopologyTests = MODULE.RequiredCheckTopologyTests From 8a8591158544b8d066d54af6be19903f51f04aef Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:40:35 +0300 Subject: [PATCH 21/62] test(af01): include ruleset contract in universal CI --- ..._audit_workflow_trust_main_ruleset_contract.py | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 .github/scripts/test_audit_workflow_trust_main_ruleset_contract.py diff --git a/.github/scripts/test_audit_workflow_trust_main_ruleset_contract.py b/.github/scripts/test_audit_workflow_trust_main_ruleset_contract.py new file mode 100644 index 00000000..beb86934 --- /dev/null +++ b/.github/scripts/test_audit_workflow_trust_main_ruleset_contract.py @@ -0,0 +1,15 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import importlib.util +import sys +from pathlib import Path + +TARGET = Path(__file__).with_name("test_main_ruleset_contract.py") +SPEC = importlib.util.spec_from_file_location("af01_main_ruleset_contract_tests", TARGET) +assert SPEC is not None and SPEC.loader is not None +MODULE = importlib.util.module_from_spec(SPEC) +sys.modules[SPEC.name] = MODULE +SPEC.loader.exec_module(MODULE) + +MainRulesetContractTests = MODULE.MainRulesetContractTests From 4aef8ae8d28d719b1874df042964c84284b079c1 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 11:40:45 +0300 Subject: [PATCH 22/62] test(af01): include assurance summary counterexamples in CI --- ...test_audit_workflow_trust_assurance_summary.py | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 .github/scripts/test_audit_workflow_trust_assurance_summary.py diff --git a/.github/scripts/test_audit_workflow_trust_assurance_summary.py b/.github/scripts/test_audit_workflow_trust_assurance_summary.py new file mode 100644 index 00000000..e4885101 --- /dev/null +++ b/.github/scripts/test_audit_workflow_trust_assurance_summary.py @@ -0,0 +1,15 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import importlib.util +import sys +from pathlib import Path + +TARGET = Path(__file__).with_name("test_build_af01_assurance_summary.py") +SPEC = importlib.util.spec_from_file_location("af01_assurance_summary_tests", TARGET) +assert SPEC is not None and SPEC.loader is not None +MODULE = importlib.util.module_from_spec(SPEC) +sys.modules[SPEC.name] = MODULE +SPEC.loader.exec_module(MODULE) + +AssuranceSummaryTests = MODULE.AssuranceSummaryTests From 425756762ebf07a1d663ab7cbffbc791d9d7b359 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 12:02:51 +0300 Subject: [PATCH 23/62] fix(af01): avoid ambiguous cargo artifact naming --- .github/workflows/af01-assurance-proof.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/af01-assurance-proof.yml b/.github/workflows/af01-assurance-proof.yml index ecefb5c5..8f8ebed8 100644 --- a/.github/workflows/af01-assurance-proof.yml +++ b/.github/workflows/af01-assurance-proof.yml @@ -37,9 +37,9 @@ jobs: - name: Generate exact locked dependency evidence run: | set -euo pipefail - cargo metadata --locked --format-version 1 > /tmp/af01-cargo-metadata.json + cargo metadata --locked --format-version 1 > /tmp/af01-dependency-metadata.json python3 .github/scripts/summarize_cargo_metadata.py \ - < /tmp/af01-cargo-metadata.json \ + < /tmp/af01-dependency-metadata.json \ > "$AF01_EVIDENCE_DIR/dependency-inventory.json" - name: Record cargo-deny proof identity run: | From 3183f9b06e393398b3a7baf02e5255676d91d583 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 12:05:35 +0300 Subject: [PATCH 24/62] fix(af01): require Dependabot update cooldown --- .github/dependabot.yml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 04535338..e9cba92b 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -5,6 +5,8 @@ updates: schedule: interval: weekly day: monday + cooldown: + default-days: 7 open-pull-requests-limit: 5 - package-ecosystem: maven @@ -12,6 +14,8 @@ updates: schedule: interval: weekly day: monday + cooldown: + default-days: 7 open-pull-requests-limit: 5 - package-ecosystem: github-actions @@ -19,4 +23,6 @@ updates: schedule: interval: weekly day: monday + cooldown: + default-days: 7 open-pull-requests-limit: 5 From 06df97223a1551ca039116a277292e3213811e68 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 12:12:51 +0300 Subject: [PATCH 25/62] fix(af01): surface assurance summary failures --- .github/workflows/af01-assurance-proof.yml | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/.github/workflows/af01-assurance-proof.yml b/.github/workflows/af01-assurance-proof.yml index 8f8ebed8..ab4cf16f 100644 --- a/.github/workflows/af01-assurance-proof.yml +++ b/.github/workflows/af01-assurance-proof.yml @@ -152,16 +152,20 @@ jobs: tree_sha="$(git rev-parse 'HEAD^{tree}')" first=/tmp/assurance-summary.json second=/tmp/assurance-summary-repeat.json - first_line="$(python3 .github/scripts/build_af01_assurance_summary.py \ + first_log=/tmp/assurance-summary-first.log + second_log=/tmp/assurance-summary-second.log + python3 .github/scripts/build_af01_assurance_summary.py \ --evidence-dir "$AF01_EVIDENCE_DIR" \ --source-sha "$AF01_SOURCE_SHA" \ --tree-sha "$tree_sha" \ - --output "$first")" - second_line="$(python3 .github/scripts/build_af01_assurance_summary.py \ + --output "$first" | tee "$first_log" + first_line="$(cat "$first_log")" + python3 .github/scripts/build_af01_assurance_summary.py \ --evidence-dir "$AF01_EVIDENCE_DIR" \ --source-sha "$AF01_SOURCE_SHA" \ --tree-sha "$tree_sha" \ - --output "$second")" + --output "$second" | tee "$second_log" + second_line="$(cat "$second_log")" test "$first_line" = "$second_line" cmp --silent "$first" "$second" printf '%s\n' "$first_line" | tee /tmp/AF01_ASSURANCE_SHA256.txt From 20e66e5dba940040d137a97f807f33c62a4bcedc Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 12:17:13 +0300 Subject: [PATCH 26/62] fix(af01): prevent proof bytecode worktree pollution --- .github/workflows/af01-assurance-proof.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/af01-assurance-proof.yml b/.github/workflows/af01-assurance-proof.yml index ab4cf16f..4c2c0361 100644 --- a/.github/workflows/af01-assurance-proof.yml +++ b/.github/workflows/af01-assurance-proof.yml @@ -13,6 +13,7 @@ permissions: env: AF01_SOURCE_SHA: ${{ github.event.pull_request.head.sha || github.sha }} AF01_EVIDENCE_DIR: /tmp/af01-assurance-evidence + PYTHONDONTWRITEBYTECODE: "1" jobs: assurance-proof: From b3f51347868dda9a5e7f57f67233ad61bdcf3f91 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 12:27:40 +0300 Subject: [PATCH 27/62] fix(af01): hash complete dependency graph evidence --- .github/scripts/summarize_cargo_metadata.py | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/.github/scripts/summarize_cargo_metadata.py b/.github/scripts/summarize_cargo_metadata.py index 7947dd74..49bac368 100644 --- a/.github/scripts/summarize_cargo_metadata.py +++ b/.github/scripts/summarize_cargo_metadata.py @@ -3,6 +3,7 @@ from __future__ import annotations +import hashlib import json import sys from collections import defaultdict @@ -36,6 +37,16 @@ def validate_manifest_dependencies(package: dict[str, Any], identity: str) -> No ) +def graph_sha256(packages: list[dict[str, object]]) -> str: + rendered = json.dumps( + packages, + sort_keys=True, + separators=(",", ":"), + ensure_ascii=False, + ).encode("utf-8") + return hashlib.sha256(rendered).hexdigest() + + def summarize(metadata: object) -> dict[str, object]: if not isinstance(metadata, dict): raise InventoryError("cargo metadata root is not an object") @@ -214,6 +225,7 @@ def summarize(metadata: object) -> dict[str, object]: } return { "duplicates": duplicates, + "graph_sha256": graph_sha256(inventory), "licenses": licenses, "non_crates_io": sorted(non_crates_io), "ok": not unknown_license, From 0f3b6df163b258f36df7806a38865987e98c1508 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 13:32:28 +0300 Subject: [PATCH 28/62] fix(af01): validate exact dependency and RustSec evidence --- .../scripts/build_af01_assurance_summary.py | 227 +++++++++++++++++- 1 file changed, 216 insertions(+), 11 deletions(-) diff --git a/.github/scripts/build_af01_assurance_summary.py b/.github/scripts/build_af01_assurance_summary.py index fbe3be91..57ae34b9 100644 --- a/.github/scripts/build_af01_assurance_summary.py +++ b/.github/scripts/build_af01_assurance_summary.py @@ -8,6 +8,7 @@ import json import re import subprocess +from collections import Counter from pathlib import Path from typing import Any @@ -18,6 +19,8 @@ ZIZMOR_ACTION = "3dc1ecc9bcb9e94e9b2c709687979e1298497054" ZIZMOR_VERSION = "1.29.0" RUSTSEC_ORIGIN = "https://github.com/RustSec/advisory-db.git" +CRATES_IO_SOURCE = "registry+https://github.com/rust-lang/crates.io-index" +INVENTORY_COMMAND = ["cargo", "metadata", "--locked", "--format-version", "1"] CONFIG_PATHS = ( ".github/workflow-trust-policy.json", "Cargo.lock", @@ -26,12 +29,14 @@ EVIDENCE_FILES = { "workflow_trust": "workflow-trust.json", "dependency_inventory": "dependency-inventory.json", + "dependency_inventory_proof": "dependency-inventory-proof.json", "cargo_deny": "cargo-deny-proof.json", "cargo_audit": "cargo-audit-proof.json", "cargo_audit_result": "cargo-audit.json", "zizmor": "zizmor-proof.json", } HEX40 = re.compile(r"^[0-9a-f]{40}$") +HEX64 = re.compile(r"^[0-9a-f]{64}$") class AssuranceError(ValueError): @@ -98,6 +103,22 @@ def surface_digest(root: Path, paths: list[str]) -> str: return digest.hexdigest() +def canonical_graph_sha256(packages: list[dict[str, object]]) -> str: + rendered = json.dumps( + packages, + sort_keys=True, + separators=(",", ":"), + ensure_ascii=False, + ).encode("utf-8") + return sha256_bytes(rendered) + + +def require_string(value: object, message: str) -> str: + if not isinstance(value, str) or not value: + raise AssuranceError(message) + return value + + def require_exact_source(root: Path, source_sha: str, tree_sha: str) -> None: if not HEX40.fullmatch(source_sha) or not HEX40.fullmatch(tree_sha): raise AssuranceError("source and tree identities must be lowercase 40-hex SHA-1 values") @@ -125,22 +146,196 @@ def validate_workflow_trust( raise AssuranceError("workflow trust evidence does not cover both exact Action metadata forms") -def validate_dependency_inventory(evidence: dict[str, Any]) -> None: +def validate_dependency_inventory(evidence: dict[str, Any]) -> str: if evidence.get("schema") != 2 or evidence.get("ok") is not True: raise AssuranceError("dependency inventory is not a successful schema-2 exact graph") if evidence.get("unknown_license") != []: raise AssuranceError("dependency inventory contains unknown third-party license metadata") - if not isinstance(evidence.get("packages"), list) or evidence.get("package_count") != len( - evidence["packages"] - ): + + packages = evidence.get("packages") + if not isinstance(packages, list) or evidence.get("package_count") != len(packages): raise AssuranceError("dependency inventory package count is inconsistent") + required_package_keys = { + "checksum", + "dependencies", + "license", + "name", + "package_id", + "source", + "source_class", + "version", + "workspace", + } + required_edge_keys = {"name", "package_id", "package_name", "source", "version"} + package_by_id: dict[str, dict[str, object]] = {} + source_classes: Counter[str] = Counter() + + for index, package in enumerate(packages): + if not isinstance(package, dict) or set(package) != required_package_keys: + raise AssuranceError(f"dependency inventory package {index} has an invalid schema") + package_id = require_string( + package.get("package_id"), f"dependency inventory package {index} has invalid identity" + ) + require_string(package.get("name"), f"dependency inventory package {index} has invalid name") + require_string(package.get("version"), f"dependency inventory package {index} has invalid version") + if package_id in package_by_id: + raise AssuranceError(f"dependency inventory contains duplicate package id: {package_id}") + + source = package.get("source") + if source is not None and not isinstance(source, str): + raise AssuranceError(f"dependency inventory package {package_id} has invalid source") + license_expr = package.get("license") + if license_expr is not None and not isinstance(license_expr, str): + raise AssuranceError(f"dependency inventory package {package_id} has invalid license") + checksum = package.get("checksum") + if checksum is not None and ( + not isinstance(checksum, str) or HEX64.fullmatch(checksum) is None + ): + raise AssuranceError(f"dependency inventory package {package_id} has invalid checksum") + workspace = package.get("workspace") + if not isinstance(workspace, bool): + raise AssuranceError(f"dependency inventory package {package_id} has invalid workspace flag") + source_class = package.get("source_class") + if source_class not in {"workspace", "crates.io", "other"}: + raise AssuranceError(f"dependency inventory package {package_id} has invalid source class") + if workspace != (source_class == "workspace"): + raise AssuranceError(f"dependency inventory package {package_id} has inconsistent workspace class") + if source_class == "crates.io" and source != CRATES_IO_SOURCE: + raise AssuranceError(f"dependency inventory package {package_id} has inconsistent crates.io source") + if source_class == "workspace" and checksum is not None: + raise AssuranceError(f"workspace package {package_id} unexpectedly carries a registry checksum") + if source_class == "crates.io" and checksum is None: + raise AssuranceError(f"crates.io package {package_id} is missing its registry checksum") + + dependencies = package.get("dependencies") + if not isinstance(dependencies, list): + raise AssuranceError(f"dependency inventory package {package_id} has invalid dependency edges") + package_by_id[package_id] = package + source_classes[str(source_class)] += 1 + + ordered_packages = sorted( + packages, + key=lambda item: ( + str(item["name"]), + str(item["version"]), + str(item["source"]), + str(item["package_id"]), + ), + ) + if packages != ordered_packages: + raise AssuranceError("dependency inventory package ordering is not canonical") + + for package in packages: + package_id = str(package["package_id"]) + dependencies = package["dependencies"] + seen_edges: set[tuple[str, str]] = set() + for edge_index, edge in enumerate(dependencies): + if not isinstance(edge, dict) or set(edge) != required_edge_keys: + raise AssuranceError( + f"dependency edge {edge_index} for {package_id} has an invalid schema" + ) + edge_name = require_string( + edge.get("name"), f"dependency edge {edge_index} for {package_id} has invalid name" + ) + target_id = require_string( + edge.get("package_id"), + f"dependency edge {edge_index} for {package_id} has invalid target id", + ) + target = package_by_id.get(target_id) + if target is None: + raise AssuranceError( + f"dependency edge {edge_index} for {package_id} references unknown package id" + ) + if ( + edge.get("package_name") != target["name"] + or edge.get("version") != target["version"] + or edge.get("source") != target["source"] + ): + raise AssuranceError( + f"dependency edge {edge_index} for {package_id} disagrees with target package identity" + ) + identity = (edge_name, target_id) + if identity in seen_edges: + raise AssuranceError(f"dependency inventory contains duplicate edge for {package_id}") + seen_edges.add(identity) + ordered_edges = sorted( + dependencies, + key=lambda edge: ( + str(edge["name"]), + str(edge["package_name"]), + str(edge["version"]), + str(edge["source"]), + str(edge["package_id"]), + ), + ) + if dependencies != ordered_edges: + raise AssuranceError(f"dependency edges for {package_id} are not canonically ordered") + + expected_source_classes = dict(sorted(source_classes.items())) + if evidence.get("source_classes") != expected_source_classes: + raise AssuranceError("dependency inventory source-class counts are inconsistent") + + graph_sha = evidence.get("graph_sha256") + expected_graph_sha = canonical_graph_sha256(packages) + if not isinstance(graph_sha, str) or not HEX64.fullmatch(graph_sha): + raise AssuranceError("dependency inventory graph digest is missing or malformed") + if graph_sha != expected_graph_sha: + raise AssuranceError("dependency inventory graph digest does not match exact package records") + return graph_sha + def require_head(proof: dict[str, Any], source_sha: str, label: str) -> None: if proof.get("schema") != 1 or proof.get("head_sha") != source_sha: raise AssuranceError(f"{label} proof is not bound to the exact source SHA") +def validate_dependency_inventory_proof( + proof: dict[str, Any], + source_sha: str, + cargo_lock_sha: str, + inventory_sha: str, + graph_sha: str, +) -> None: + require_head(proof, source_sha, "dependency inventory") + if ( + proof.get("command") != INVENTORY_COMMAND + or proof.get("cargo_lock_sha256") != cargo_lock_sha + or proof.get("inventory_sha256") != inventory_sha + or proof.get("graph_sha256") != graph_sha + ): + raise AssuranceError("dependency inventory proof identity/graph mismatch") + + +def validate_cargo_audit_result( + result: dict[str, Any], proof: dict[str, Any], package_count: int +) -> None: + vulnerabilities = result.get("vulnerabilities") + if not isinstance(vulnerabilities, dict): + raise AssuranceError("cargo-audit result is missing vulnerabilities evidence") + if vulnerabilities.get("found") is not False: + raise AssuranceError("cargo-audit result does not explicitly prove zero vulnerabilities") + if vulnerabilities.get("count") != 0 or vulnerabilities.get("list") != []: + raise AssuranceError("cargo-audit zero-vulnerability fields are inconsistent") + + lockfile = result.get("lockfile") + if not isinstance(lockfile, dict) or lockfile.get("dependency-count") != package_count: + raise AssuranceError("cargo-audit lockfile dependency count does not match exact inventory") + + database = result.get("database") + if not isinstance(database, dict): + raise AssuranceError("cargo-audit result is missing advisory database evidence") + advisory_count = database.get("advisory-count") + if not isinstance(advisory_count, int) or isinstance(advisory_count, bool) or advisory_count < 0: + raise AssuranceError("cargo-audit advisory database count is invalid") + if database.get("last-commit") != proof.get("advisory_db_commit"): + raise AssuranceError("cargo-audit result advisory database commit does not match proof") + if not isinstance(database.get("last-updated"), str) or not database["last-updated"]: + raise AssuranceError("cargo-audit result advisory database timestamp is invalid") + if not isinstance(result.get("settings"), dict) or not isinstance(result.get("warnings"), dict): + raise AssuranceError("cargo-audit result is missing settings/warnings objects") + + def build_summary(root: Path, evidence_dir: Path, source_sha: str, tree_sha: str) -> dict[str, Any]: root = root.resolve() evidence_dir = evidence_dir.resolve() @@ -150,16 +345,27 @@ def build_summary(root: Path, evidence_dir: Path, source_sha: str, tree_sha: str workflows, actions = security_surfaces(paths) workflow_trust = read_json(evidence_dir / EVIDENCE_FILES["workflow_trust"]) dependency_inventory = read_json(evidence_dir / EVIDENCE_FILES["dependency_inventory"]) + dependency_inventory_proof = read_json( + evidence_dir / EVIDENCE_FILES["dependency_inventory_proof"] + ) cargo_deny = read_json(evidence_dir / EVIDENCE_FILES["cargo_deny"]) cargo_audit = read_json(evidence_dir / EVIDENCE_FILES["cargo_audit"]) cargo_audit_result = read_json(evidence_dir / EVIDENCE_FILES["cargo_audit_result"]) zizmor = read_json(evidence_dir / EVIDENCE_FILES["zizmor"]) validate_workflow_trust(workflow_trust, workflows, actions) - validate_dependency_inventory(dependency_inventory) + graph_sha = validate_dependency_inventory(dependency_inventory) cargo_lock_sha = sha256_file(root / "Cargo.lock") deny_sha = sha256_file(root / "deny.toml") + inventory_sha = sha256_file(evidence_dir / EVIDENCE_FILES["dependency_inventory"]) + validate_dependency_inventory_proof( + dependency_inventory_proof, + source_sha, + cargo_lock_sha, + inventory_sha, + graph_sha, + ) require_head(cargo_deny, source_sha, "cargo-deny") if ( @@ -180,8 +386,9 @@ def build_summary(root: Path, evidence_dir: Path, source_sha: str, tree_sha: str or not HEX40.fullmatch(str(cargo_audit.get("advisory_db_commit", ""))) ): raise AssuranceError("cargo-audit proof identity/result mismatch") - if cargo_audit_result.get("vulnerabilities", {}).get("found") is True: - raise AssuranceError("cargo-audit result reports RustSec vulnerabilities") + validate_cargo_audit_result( + cargo_audit_result, cargo_audit, int(dependency_inventory["package_count"]) + ) require_head(zizmor, source_sha, "zizmor") if ( @@ -193,10 +400,7 @@ def build_summary(root: Path, evidence_dir: Path, source_sha: str, tree_sha: str ): raise AssuranceError("zizmor proof identity/policy mismatch") - config = { - path: sha256_file(root / path) - for path in CONFIG_PATHS - } + config = {path: sha256_file(root / path) for path in CONFIG_PATHS} surface_paths = sorted(set(workflows + actions)) evidence_sha256 = { key: sha256_file(evidence_dir / filename) @@ -206,6 +410,7 @@ def build_summary(root: Path, evidence_dir: Path, source_sha: str, tree_sha: str return { "config_sha256": config, "dependency_graph": { + "graph_sha256": graph_sha, "package_count": dependency_inventory["package_count"], "schema": dependency_inventory["schema"], }, From e3acbc8c0f5fb679ad793537c497c54b1a08f1e8 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 13:33:52 +0300 Subject: [PATCH 29/62] fix(af01): bind dependency inventory to Cargo.lock --- .../scripts/build_af01_assurance_summary.py | 83 ++++++++++++++++--- 1 file changed, 71 insertions(+), 12 deletions(-) diff --git a/.github/scripts/build_af01_assurance_summary.py b/.github/scripts/build_af01_assurance_summary.py index 57ae34b9..5bdb698f 100644 --- a/.github/scripts/build_af01_assurance_summary.py +++ b/.github/scripts/build_af01_assurance_summary.py @@ -8,6 +8,7 @@ import json import re import subprocess +import tomllib from collections import Counter from pathlib import Path from typing import Any @@ -103,9 +104,9 @@ def surface_digest(root: Path, paths: list[str]) -> str: return digest.hexdigest() -def canonical_graph_sha256(packages: list[dict[str, object]]) -> str: +def canonical_json_sha256(value: object) -> str: rendered = json.dumps( - packages, + value, sort_keys=True, separators=(",", ":"), ensure_ascii=False, @@ -113,6 +114,10 @@ def canonical_graph_sha256(packages: list[dict[str, object]]) -> str: return sha256_bytes(rendered) +def canonical_graph_sha256(packages: list[dict[str, object]]) -> str: + return canonical_json_sha256(packages) + + def require_string(value: object, message: str) -> str: if not isinstance(value, str) or not value: raise AssuranceError(message) @@ -157,7 +162,6 @@ def validate_dependency_inventory(evidence: dict[str, Any]) -> str: raise AssuranceError("dependency inventory package count is inconsistent") required_package_keys = { - "checksum", "dependencies", "license", "name", @@ -188,11 +192,6 @@ def validate_dependency_inventory(evidence: dict[str, Any]) -> str: license_expr = package.get("license") if license_expr is not None and not isinstance(license_expr, str): raise AssuranceError(f"dependency inventory package {package_id} has invalid license") - checksum = package.get("checksum") - if checksum is not None and ( - not isinstance(checksum, str) or HEX64.fullmatch(checksum) is None - ): - raise AssuranceError(f"dependency inventory package {package_id} has invalid checksum") workspace = package.get("workspace") if not isinstance(workspace, bool): raise AssuranceError(f"dependency inventory package {package_id} has invalid workspace flag") @@ -203,10 +202,6 @@ def validate_dependency_inventory(evidence: dict[str, Any]) -> str: raise AssuranceError(f"dependency inventory package {package_id} has inconsistent workspace class") if source_class == "crates.io" and source != CRATES_IO_SOURCE: raise AssuranceError(f"dependency inventory package {package_id} has inconsistent crates.io source") - if source_class == "workspace" and checksum is not None: - raise AssuranceError(f"workspace package {package_id} unexpectedly carries a registry checksum") - if source_class == "crates.io" and checksum is None: - raise AssuranceError(f"crates.io package {package_id} is missing its registry checksum") dependencies = package.get("dependencies") if not isinstance(dependencies, list): @@ -285,6 +280,66 @@ def validate_dependency_inventory(evidence: dict[str, Any]) -> str: return graph_sha +def validate_inventory_against_cargo_lock( + root: Path, packages: list[dict[str, object]] +) -> str: + try: + lock = tomllib.loads((root / "Cargo.lock").read_text(encoding="utf-8")) + except (OSError, UnicodeError, tomllib.TOMLDecodeError) as error: + raise AssuranceError(f"Cargo.lock cannot be parsed for package identity proof: {error}") from error + lock_packages = lock.get("package") + if not isinstance(lock_packages, list): + raise AssuranceError("Cargo.lock is missing package records") + + canonical_lock: list[dict[str, object]] = [] + lock_by_identity: dict[tuple[str, str, object], dict[str, object]] = {} + for index, package in enumerate(lock_packages): + if not isinstance(package, dict): + raise AssuranceError(f"Cargo.lock package {index} is not an object") + name = require_string(package.get("name"), f"Cargo.lock package {index} has invalid name") + version = require_string( + package.get("version"), f"Cargo.lock package {index} has invalid version" + ) + source = package.get("source") + if source is not None and not isinstance(source, str): + raise AssuranceError(f"Cargo.lock package {name}@{version} has invalid source") + checksum = package.get("checksum") + if checksum is not None and ( + not isinstance(checksum, str) or HEX64.fullmatch(checksum) is None + ): + raise AssuranceError(f"Cargo.lock package {name}@{version} has invalid checksum") + if source == CRATES_IO_SOURCE and checksum is None: + raise AssuranceError(f"Cargo.lock crates.io package {name}@{version} is missing checksum") + identity = (name, version, source) + if identity in lock_by_identity: + raise AssuranceError(f"Cargo.lock contains duplicate package identity: {name}@{version}") + record = { + "checksum": checksum, + "name": name, + "source": source, + "version": version, + } + lock_by_identity[identity] = record + canonical_lock.append(record) + + inventory_identities = { + (str(package["name"]), str(package["version"]), package["source"]) + for package in packages + } + lock_identities = set(lock_by_identity) + if inventory_identities != lock_identities: + missing = sorted(str(value) for value in lock_identities - inventory_identities) + extra = sorted(str(value) for value in inventory_identities - lock_identities) + raise AssuranceError( + f"dependency inventory does not match Cargo.lock package identities: missing={missing[:1]} extra={extra[:1]}" + ) + + canonical_lock.sort( + key=lambda item: (str(item["name"]), str(item["version"]), str(item["source"])) + ) + return canonical_json_sha256(canonical_lock) + + def require_head(proof: dict[str, Any], source_sha: str, label: str) -> None: if proof.get("schema") != 1 or proof.get("head_sha") != source_sha: raise AssuranceError(f"{label} proof is not bound to the exact source SHA") @@ -355,6 +410,9 @@ def build_summary(root: Path, evidence_dir: Path, source_sha: str, tree_sha: str validate_workflow_trust(workflow_trust, workflows, actions) graph_sha = validate_dependency_inventory(dependency_inventory) + lock_packages_sha = validate_inventory_against_cargo_lock( + root, dependency_inventory["packages"] + ) cargo_lock_sha = sha256_file(root / "Cargo.lock") deny_sha = sha256_file(root / "deny.toml") @@ -410,6 +468,7 @@ def build_summary(root: Path, evidence_dir: Path, source_sha: str, tree_sha: str return { "config_sha256": config, "dependency_graph": { + "cargo_lock_packages_sha256": lock_packages_sha, "graph_sha256": graph_sha, "package_count": dependency_inventory["package_count"], "schema": dependency_inventory["schema"], From 772b972b9e912480cd80d579f7b64e0e349e9344 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 13:34:33 +0300 Subject: [PATCH 30/62] test(af01): add malformed evidence counterexamples --- .../test_build_af01_assurance_summary.py | 148 ++++++++++++++++-- 1 file changed, 135 insertions(+), 13 deletions(-) diff --git a/.github/scripts/test_build_af01_assurance_summary.py b/.github/scripts/test_build_af01_assurance_summary.py index 2057a461..7b71bd3b 100644 --- a/.github/scripts/test_build_af01_assurance_summary.py +++ b/.github/scripts/test_build_af01_assurance_summary.py @@ -39,7 +39,17 @@ def setUp(self) -> None: ".github/workflows/ci.yml": "name: ci\non: [pull_request]\n", ".github/workflow-trust-policy.json": "{\"schema\":1}\n", "action.yaml": "name: fixture\nruns:\n using: composite\n steps: []\n", - "Cargo.lock": "# fixture lock\n", + "Cargo.lock": ( + "version = 4\n\n" + "[[package]]\n" + "name = \"commandf\"\n" + "version = \"0.1.0\"\n\n" + "[[package]]\n" + "name = \"dep\"\n" + "version = \"1.0.0\"\n" + f"source = \"{SUMMARY.CRATES_IO_SOURCE}\"\n" + f"checksum = \"{'b' * 64}\"\n" + ), "deny.toml": "[bans]\nwildcards = \"deny\"\n", } for relative, content in files.items(): @@ -62,6 +72,66 @@ def tearDown(self) -> None: def _sha(self, relative: str) -> str: return hashlib.sha256((self.root / relative).read_bytes()).hexdigest() + def _inventory_packages(self) -> list[dict[str, object]]: + dep_id = f"{SUMMARY.CRATES_IO_SOURCE}#dep@1.0.0" + root_id = "path+file:///workspace/commandf#0.1.0" + return [ + { + "dependencies": [ + { + "name": "dep", + "package_id": dep_id, + "package_name": "dep", + "source": SUMMARY.CRATES_IO_SOURCE, + "version": "1.0.0", + } + ], + "license": None, + "name": "commandf", + "package_id": root_id, + "source": None, + "source_class": "workspace", + "version": "0.1.0", + "workspace": True, + }, + { + "dependencies": [], + "license": "MIT", + "name": "dep", + "package_id": dep_id, + "source": SUMMARY.CRATES_IO_SOURCE, + "source_class": "crates.io", + "version": "1.0.0", + "workspace": False, + }, + ] + + def _write_inventory(self, packages: list[dict[str, object]] | None = None) -> None: + packages = self._inventory_packages() if packages is None else packages + inventory = { + "schema": 2, + "ok": True, + "package_count": len(packages), + "packages": packages, + "graph_sha256": SUMMARY.canonical_graph_sha256(packages), + "source_classes": {"crates.io": 1, "workspace": 1}, + "unknown_license": [], + } + write_json(self.evidence / "dependency-inventory.json", inventory) + write_json( + self.evidence / "dependency-inventory-proof.json", + { + "schema": 1, + "head_sha": self.source, + "command": SUMMARY.INVENTORY_COMMAND, + "cargo_lock_sha256": self._sha("Cargo.lock"), + "inventory_sha256": hashlib.sha256( + (self.evidence / "dependency-inventory.json").read_bytes() + ).hexdigest(), + "graph_sha256": inventory["graph_sha256"], + }, + ) + def _write_valid_evidence(self) -> None: write_json( self.evidence / "workflow-trust.json", @@ -73,16 +143,7 @@ def _write_valid_evidence(self) -> None: "findings": [], }, ) - write_json( - self.evidence / "dependency-inventory.json", - { - "schema": 2, - "ok": True, - "package_count": 0, - "packages": [], - "unknown_license": [], - }, - ) + self._write_inventory() write_json( self.evidence / "cargo-deny-proof.json", { @@ -109,7 +170,17 @@ def _write_valid_evidence(self) -> None: ) write_json( self.evidence / "cargo-audit.json", - {"vulnerabilities": {"found": False, "count": 0, "list": []}}, + { + "database": { + "advisory-count": 1, + "last-commit": "a" * 40, + "last-updated": "2026-08-27T00:00:00Z", + }, + "lockfile": {"dependency-count": 2}, + "settings": {}, + "vulnerabilities": {"found": False, "count": 0, "list": []}, + "warnings": {}, + }, ) write_json( self.evidence / "zizmor-proof.json", @@ -142,7 +213,7 @@ def test_tree_sha_mismatch_fails_closed(self) -> None: SUMMARY.build_summary(self.root, self.evidence, self.source, "0" * 40) def test_missing_required_evidence_fails_closed(self) -> None: - (self.evidence / "cargo-deny-proof.json").unlink() + (self.evidence / "dependency-inventory-proof.json").unlink() with self.assertRaisesRegex(SUMMARY.AssuranceError, "required evidence is missing"): self.build() @@ -151,6 +222,57 @@ def test_malformed_evidence_fails_closed(self) -> None: with self.assertRaisesRegex(SUMMARY.AssuranceError, "invalid JSON evidence"): self.build() + def test_dependency_graph_digest_mismatch_fails_closed(self) -> None: + inventory = json.loads( + (self.evidence / "dependency-inventory.json").read_text(encoding="utf-8") + ) + inventory["packages"][1]["version"] = "9.9.9" + write_json(self.evidence / "dependency-inventory.json", inventory) + with self.assertRaisesRegex(SUMMARY.AssuranceError, "graph digest does not match"): + self.build() + + def test_dependency_inventory_proof_must_bind_cargo_lock(self) -> None: + proof = json.loads( + (self.evidence / "dependency-inventory-proof.json").read_text(encoding="utf-8") + ) + proof["cargo_lock_sha256"] = "0" * 64 + write_json(self.evidence / "dependency-inventory-proof.json", proof) + with self.assertRaisesRegex(SUMMARY.AssuranceError, "identity/graph mismatch"): + self.build() + + def test_dependency_inventory_must_match_cargo_lock_identities(self) -> None: + packages = self._inventory_packages() + packages[1]["version"] = "2.0.0" + self._write_inventory(packages) + with self.assertRaisesRegex(SUMMARY.AssuranceError, "does not match Cargo.lock"): + self.build() + + def test_cargo_audit_empty_object_fails_closed(self) -> None: + write_json(self.evidence / "cargo-audit.json", {}) + with self.assertRaisesRegex(SUMMARY.AssuranceError, "missing vulnerabilities evidence"): + self.build() + + def test_cargo_audit_missing_found_fails_closed(self) -> None: + result = json.loads((self.evidence / "cargo-audit.json").read_text(encoding="utf-8")) + result["vulnerabilities"].pop("found") + write_json(self.evidence / "cargo-audit.json", result) + with self.assertRaisesRegex(SUMMARY.AssuranceError, "explicitly prove zero"): + self.build() + + def test_cargo_audit_zero_fields_must_be_consistent(self) -> None: + result = json.loads((self.evidence / "cargo-audit.json").read_text(encoding="utf-8")) + result["vulnerabilities"]["count"] = 1 + write_json(self.evidence / "cargo-audit.json", result) + with self.assertRaisesRegex(SUMMARY.AssuranceError, "zero-vulnerability fields"): + self.build() + + def test_cargo_audit_database_commit_must_match_proof(self) -> None: + result = json.loads((self.evidence / "cargo-audit.json").read_text(encoding="utf-8")) + result["database"]["last-commit"] = "b" * 40 + write_json(self.evidence / "cargo-audit.json", result) + with self.assertRaisesRegex(SUMMARY.AssuranceError, "database commit"): + self.build() + def test_permission_policy_mismatch_fails_closed(self) -> None: write_json( self.evidence / "workflow-trust.json", From 7670a61e51f1f0f35ab4f6e0349d649d0c5fe3ce Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 13:35:00 +0300 Subject: [PATCH 31/62] fix(af01): bind deterministic dependency inventory proof --- .github/workflows/af01-assurance-proof.yml | 39 +++++++++++++++++++--- 1 file changed, 35 insertions(+), 4 deletions(-) diff --git a/.github/workflows/af01-assurance-proof.yml b/.github/workflows/af01-assurance-proof.yml index 4c2c0361..57afa447 100644 --- a/.github/workflows/af01-assurance-proof.yml +++ b/.github/workflows/af01-assurance-proof.yml @@ -35,13 +35,44 @@ jobs: run: | set -euo pipefail python3 .github/scripts/audit_workflow_trust.py > "$AF01_EVIDENCE_DIR/workflow-trust.json" - - name: Generate exact locked dependency evidence + - name: Generate deterministic exact locked dependency evidence run: | set -euo pipefail - cargo metadata --locked --format-version 1 > /tmp/af01-dependency-metadata.json + first_metadata=/tmp/af01-dependency-metadata.json + second_metadata=/tmp/af01-dependency-metadata-repeat.json + first_inventory="$AF01_EVIDENCE_DIR/dependency-inventory.json" + second_inventory=/tmp/dependency-inventory-repeat.json + cargo metadata --locked --format-version 1 > "$first_metadata" python3 .github/scripts/summarize_cargo_metadata.py \ - < /tmp/af01-dependency-metadata.json \ - > "$AF01_EVIDENCE_DIR/dependency-inventory.json" + < "$first_metadata" \ + > "$first_inventory" + cargo metadata --locked --format-version 1 > "$second_metadata" + python3 .github/scripts/summarize_cargo_metadata.py \ + < "$second_metadata" \ + > "$second_inventory" + cmp --silent "$first_inventory" "$second_inventory" + python3 - <<'PY' + import hashlib + import json + import os + from pathlib import Path + + evidence_dir = Path(os.environ["AF01_EVIDENCE_DIR"]) + inventory_path = evidence_dir / "dependency-inventory.json" + inventory = json.loads(inventory_path.read_text(encoding="utf-8")) + + proof = { + "cargo_lock_sha256": hashlib.sha256(Path("Cargo.lock").read_bytes()).hexdigest(), + "command": ["cargo", "metadata", "--locked", "--format-version", "1"], + "graph_sha256": inventory["graph_sha256"], + "head_sha": os.environ["AF01_SOURCE_SHA"], + "inventory_sha256": hashlib.sha256(inventory_path.read_bytes()).hexdigest(), + "schema": 1, + } + (evidence_dir / "dependency-inventory-proof.json").write_text( + json.dumps(proof, indent=2, sort_keys=True) + "\n", encoding="utf-8" + ) + PY - name: Record cargo-deny proof identity run: | set -euo pipefail From 0fb52c1749cade8317890123dc06fa62d8c86711 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 13:35:15 +0300 Subject: [PATCH 32/62] fix(af01): bind required checks to GitHub Actions --- .github/required-checks.json | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/required-checks.json b/.github/required-checks.json index d7006976..5b50cee4 100644 --- a/.github/required-checks.json +++ b/.github/required-checks.json @@ -4,16 +4,19 @@ "checks": [ { "context": "rust", + "integration_id": 15368, "workflow": ".github/workflows/ci.yml", "job": "rust" }, { "context": "assurance-proof", + "integration_id": 15368, "workflow": ".github/workflows/af01-assurance-proof.yml", "job": "assurance-proof" }, { "context": "scorecard", + "integration_id": 15368, "workflow": ".github/workflows/af01-scorecard.yml", "job": "scorecard" } From a2cd249baa3b8fa5209a10ae30d6dc3a5629afc0 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 13:35:25 +0300 Subject: [PATCH 33/62] fix(af01): require GitHub Actions check source --- .github/main-ruleset.json | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/.github/main-ruleset.json b/.github/main-ruleset.json index 24bf9515..16cdd235 100644 --- a/.github/main-ruleset.json +++ b/.github/main-ruleset.json @@ -37,13 +37,16 @@ "do_not_enforce_on_create": false, "required_status_checks": [ { - "context": "rust" + "context": "rust", + "integration_id": 15368 }, { - "context": "assurance-proof" + "context": "assurance-proof", + "integration_id": 15368 }, { - "context": "scorecard" + "context": "scorecard", + "integration_id": 15368 } ], "strict_required_status_checks_policy": true From 3f35c1051c3c2303893a18812716a57575d6d673 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 13:35:42 +0300 Subject: [PATCH 34/62] test(af01): require integration-bound status checks --- .github/scripts/test_main_ruleset_contract.py | 35 +++++++++++++++---- 1 file changed, 28 insertions(+), 7 deletions(-) diff --git a/.github/scripts/test_main_ruleset_contract.py b/.github/scripts/test_main_ruleset_contract.py index 2bca6288..dae313d3 100644 --- a/.github/scripts/test_main_ruleset_contract.py +++ b/.github/scripts/test_main_ruleset_contract.py @@ -8,6 +8,7 @@ ROOT = Path(__file__).resolve().parents[2] TOPOLOGY = ROOT / ".github" / "required-checks.json" RULESET = ROOT / ".github" / "main-ruleset.json" +GITHUB_ACTIONS_INTEGRATION_ID = 15368 class MainRulesetContractTests(unittest.TestCase): @@ -48,16 +49,36 @@ def test_ruleset_matches_required_check_topology_and_governance(self) -> None: self.assertIsInstance(required, dict) self.assertFalse(required.get("do_not_enforce_on_create")) self.assertTrue(required.get("strict_required_status_checks_policy")) - contexts = [item.get("context") for item in required.get("required_status_checks", [])] - expected = [item["context"] for item in topology["checks"]] - self.assertEqual(contexts, expected) - self.assertEqual(contexts, ["rust", "assurance-proof", "scorecard"]) + actual = required.get("required_status_checks", []) + expected = [ + { + "context": item["context"], + "integration_id": item["integration_id"], + } + for item in topology["checks"] + ] + self.assertEqual(actual, expected) + self.assertEqual( + actual, + [ + {"context": "rust", "integration_id": GITHUB_ACTIONS_INTEGRATION_ID}, + {"context": "assurance-proof", "integration_id": GITHUB_ACTIONS_INTEGRATION_ID}, + {"context": "scorecard", "integration_id": GITHUB_ACTIONS_INTEGRATION_ID}, + ], + ) + + def test_every_required_check_is_bound_to_github_actions(self) -> None: + topology = json.loads(TOPOLOGY.read_text(encoding="utf-8")) + for check in topology["checks"]: + self.assertEqual(check.get("integration_id"), GITHUB_ACTIONS_INTEGRATION_ID) - def test_ruleset_has_no_unreviewed_integration_binding(self) -> None: ruleset = json.loads(RULESET.read_text(encoding="utf-8")) - required_rule = next(rule for rule in ruleset["rules"] if rule["type"] == "required_status_checks") + required_rule = next( + rule for rule in ruleset["rules"] if rule["type"] == "required_status_checks" + ) for check in required_rule["parameters"]["required_status_checks"]: - self.assertEqual(set(check), {"context"}) + self.assertEqual(set(check), {"context", "integration_id"}) + self.assertEqual(check["integration_id"], GITHUB_ACTIONS_INTEGRATION_ID) if __name__ == "__main__": From a587a7ca869168fe4111d879acd52616f11e3322 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 13:36:39 +0300 Subject: [PATCH 35/62] test(af01): reject duplicate required check producers --- .../scripts/test_required_check_topology.py | 101 +++++++++++++++++- 1 file changed, 99 insertions(+), 2 deletions(-) diff --git a/.github/scripts/test_required_check_topology.py b/.github/scripts/test_required_check_topology.py index dbf47f07..9adceb6d 100644 --- a/.github/scripts/test_required_check_topology.py +++ b/.github/scripts/test_required_check_topology.py @@ -8,6 +8,8 @@ ROOT = Path(__file__).resolve().parents[2] CONFIG = ROOT / ".github" / "required-checks.json" +WORKFLOWS = ROOT / ".github" / "workflows" +GITHUB_ACTIONS_INTEGRATION_ID = 15368 def display_path(path: Path) -> str: @@ -70,13 +72,69 @@ def assert_universal_required_job(path: Path, job: str) -> None: ) +def workflow_paths(root: Path = WORKFLOWS) -> list[Path]: + return sorted( + path + for path in root.iterdir() + if path.is_file() and path.suffix in {".yml", ".yaml"} + ) + + +def literal_job_name(lines: list[str], start: int, end: int) -> str | None: + names = [line[9:].strip() for line in lines[start + 1 : end] if line.startswith(" name:")] + if len(names) > 1: + raise AssertionError("workflow job has multiple top-level name fields") + if not names: + return None + value = names[0] + if len(value) >= 2 and value[0] == value[-1] and value[0] in {"'", '"'}: + value = value[1:-1] + if not value: + raise AssertionError("workflow job has an empty name") + if "${{" in value: + return None + return value + + +def job_check_contexts(path: Path) -> list[tuple[str, str | None]]: + lines = path.read_text(encoding="utf-8").splitlines() + try: + jobs_index = lines.index("jobs:") + except ValueError: + return [] + job_ids: list[str] = [] + for line in lines[jobs_index + 1 :]: + if line and not line.startswith(" "): + break + if line.startswith(" ") and not line.startswith(" ") and line.rstrip().endswith(":"): + job_ids.append(line.strip()[:-1]) + result: list[tuple[str, str | None]] = [] + for job_id in job_ids: + start, end = job_range(lines, job_id) + explicit_name = literal_job_name(lines, start, end) + result.append((job_id, explicit_name or job_id)) + return result + + +def required_context_producers( + workflow_root: Path, contexts: set[str] +) -> dict[str, list[tuple[str, str]]]: + producers = {context: [] for context in contexts} + for path in workflow_paths(workflow_root): + relative = display_path(path) + for job_id, context in job_check_contexts(path): + if context in contexts: + producers[str(context)].append((relative, job_id)) + return producers + + class RequiredCheckTopologyTests(unittest.TestCase): def load_config(self) -> dict[str, object]: value = json.loads(CONFIG.read_text(encoding="utf-8")) self.assertIsInstance(value, dict) return value - def test_selected_checks_are_unique_and_universal(self) -> None: + def test_selected_checks_are_unique_universal_and_integration_bound(self) -> None: config = self.load_config() self.assertEqual(config.get("schema"), 1) self.assertEqual(config.get("protected_branch"), "main") @@ -85,23 +143,41 @@ def test_selected_checks_are_unique_and_universal(self) -> None: self.assertGreater(len(checks), 0) contexts: set[str] = set() pairs: set[tuple[str, str]] = set() + expected_producers: dict[str, tuple[str, str]] = {} for index, item in enumerate(checks): with self.subTest(index=index): self.assertIsInstance(item, dict) - self.assertEqual(set(item), {"context", "workflow", "job"}) + self.assertEqual(set(item), {"context", "integration_id", "workflow", "job"}) context = item["context"] + integration_id = item["integration_id"] workflow = item["workflow"] job = item["job"] self.assertIsInstance(context, str) + self.assertEqual(integration_id, GITHUB_ACTIONS_INTEGRATION_ID) self.assertIsInstance(workflow, str) self.assertIsInstance(job, str) self.assertNotIn(context, contexts) self.assertNotIn((workflow, job), pairs) contexts.add(context) pairs.add((workflow, job)) + expected_producers[context] = (workflow, job) path = ROOT / workflow self.assertTrue(path.is_file(), workflow) assert_universal_required_job(path, job) + actual_contexts = dict(job_check_contexts(path)) + self.assertEqual( + actual_contexts.get(job), + context, + f"{workflow} job {job!r} does not emit required context {context!r}", + ) + + producers = required_context_producers(WORKFLOWS, contexts) + for context, expected in expected_producers.items(): + self.assertEqual( + producers[context], + [expected], + f"required context {context!r} must have exactly one authoritative workflow/job producer", + ) def test_counterexample_path_filtered_workflow_is_rejected(self) -> None: lines = [ @@ -126,6 +202,27 @@ def test_counterexample_job_level_if_is_rejected(self) -> None: with self.assertRaisesRegex(AssertionError, "job-level conditional"): assert_universal_required_job(path, "gate") + def test_counterexample_duplicate_or_named_spoof_context_is_detected(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + (root / "authoritative.yml").write_text( + "name: authoritative\njobs:\n rust:\n runs-on: ubuntu-24.04\n", + encoding="utf-8", + ) + (root / "spoof.yml").write_text( + "name: spoof\njobs:\n harmless-id:\n name: rust\n runs-on: ubuntu-24.04\n", + encoding="utf-8", + ) + producers = required_context_producers(root, {"rust"}) + self.assertEqual(len(producers["rust"]), 2) + self.assertEqual( + producers["rust"], + [ + (str(root / "authoritative.yml"), "rust"), + (str(root / "spoof.yml"), "harmless-id"), + ], + ) + if __name__ == "__main__": unittest.main() From f5878260f6d5419c07899e35f1999120062a33d3 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 13:38:04 +0300 Subject: [PATCH 36/62] test(af01): isolate dependency proof counterexamples --- .github/scripts/test_build_af01_assurance_summary.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.github/scripts/test_build_af01_assurance_summary.py b/.github/scripts/test_build_af01_assurance_summary.py index 7b71bd3b..865ed47f 100644 --- a/.github/scripts/test_build_af01_assurance_summary.py +++ b/.github/scripts/test_build_af01_assurance_summary.py @@ -226,7 +226,7 @@ def test_dependency_graph_digest_mismatch_fails_closed(self) -> None: inventory = json.loads( (self.evidence / "dependency-inventory.json").read_text(encoding="utf-8") ) - inventory["packages"][1]["version"] = "9.9.9" + inventory["packages"][1]["license"] = "Apache-2.0" write_json(self.evidence / "dependency-inventory.json", inventory) with self.assertRaisesRegex(SUMMARY.AssuranceError, "graph digest does not match"): self.build() @@ -243,6 +243,7 @@ def test_dependency_inventory_proof_must_bind_cargo_lock(self) -> None: def test_dependency_inventory_must_match_cargo_lock_identities(self) -> None: packages = self._inventory_packages() packages[1]["version"] = "2.0.0" + packages[0]["dependencies"][0]["version"] = "2.0.0" self._write_inventory(packages) with self.assertRaisesRegex(SUMMARY.AssuranceError, "does not match Cargo.lock"): self.build() From 900220bc12af5c3da288e91fa3faeabe1bddb9fc Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 13:38:37 +0300 Subject: [PATCH 37/62] test(af01): cover inventory status and graph digest --- .github/scripts/test_summarize_cargo_metadata.py | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/.github/scripts/test_summarize_cargo_metadata.py b/.github/scripts/test_summarize_cargo_metadata.py index 27461a4c..cfd4c8f6 100644 --- a/.github/scripts/test_summarize_cargo_metadata.py +++ b/.github/scripts/test_summarize_cargo_metadata.py @@ -100,6 +100,22 @@ def test_resolved_edges_preserve_exact_selected_package_identity(self) -> None: ) self.assertEqual(result["duplicates"], {"getrandom": ["0.2.17", "0.4.3"]}) + def test_valid_metadata_reports_clean_status_and_graph_digest(self) -> None: + first = SUMMARY.summarize(valid_metadata()) + second = SUMMARY.summarize(valid_metadata()) + self.assertTrue(first["ok"]) + self.assertEqual(first["unknown_license"], []) + self.assertEqual(first["non_crates_io"], []) + self.assertEqual(first["graph_sha256"], SUMMARY.graph_sha256(first["packages"])) + self.assertEqual(first["graph_sha256"], second["graph_sha256"]) + + def test_missing_license_on_external_package_fails_closed(self) -> None: + metadata = valid_metadata() + metadata["packages"][1]["license"] = None + result = SUMMARY.summarize(metadata) + self.assertFalse(result["ok"]) + self.assertEqual(result["unknown_license"], ["getrandom@0.2.17"]) + def test_manifest_dependencies_must_be_an_array(self) -> None: metadata = valid_metadata() metadata["packages"][0]["dependencies"] = {"name": "hidden"} From 9bd66c026e3354fb9a86a3206ec7730d965bf0f2 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 13:45:54 +0300 Subject: [PATCH 38/62] test(af01): reject dynamic required-check names --- .../scripts/test_required_check_topology.py | 21 +++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/.github/scripts/test_required_check_topology.py b/.github/scripts/test_required_check_topology.py index 9adceb6d..4d88e84d 100644 --- a/.github/scripts/test_required_check_topology.py +++ b/.github/scripts/test_required_check_topology.py @@ -92,11 +92,13 @@ def literal_job_name(lines: list[str], start: int, end: int) -> str | None: if not value: raise AssertionError("workflow job has an empty name") if "${{" in value: - return None + raise AssertionError( + "dynamic workflow job names are forbidden because they can spoof a required check context" + ) return value -def job_check_contexts(path: Path) -> list[tuple[str, str | None]]: +def job_check_contexts(path: Path) -> list[tuple[str, str]]: lines = path.read_text(encoding="utf-8").splitlines() try: jobs_index = lines.index("jobs:") @@ -108,7 +110,7 @@ def job_check_contexts(path: Path) -> list[tuple[str, str | None]]: break if line.startswith(" ") and not line.startswith(" ") and line.rstrip().endswith(":"): job_ids.append(line.strip()[:-1]) - result: list[tuple[str, str | None]] = [] + result: list[tuple[str, str]] = [] for job_id in job_ids: start, end = job_range(lines, job_id) explicit_name = literal_job_name(lines, start, end) @@ -124,7 +126,7 @@ def required_context_producers( relative = display_path(path) for job_id, context in job_check_contexts(path): if context in contexts: - producers[str(context)].append((relative, job_id)) + producers[context].append((relative, job_id)) return producers @@ -223,6 +225,17 @@ def test_counterexample_duplicate_or_named_spoof_context_is_detected(self) -> No ], ) + def test_counterexample_dynamic_job_name_is_rejected(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + path = root / "dynamic.yml" + path.write_text( + "name: dynamic\njobs:\n harmless-id:\n name: ${{ github.event.pull_request.title }}\n runs-on: ubuntu-24.04\n", + encoding="utf-8", + ) + with self.assertRaisesRegex(AssertionError, "dynamic workflow job names are forbidden"): + job_check_contexts(path) + if __name__ == "__main__": unittest.main() From 6ede75d0b964a068006408757219e1edd156a88b Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 13:52:48 +0300 Subject: [PATCH 39/62] feat(af01): verify fetched crate checksums --- .github/scripts/verify_crate_checksums.py | 158 ++++++++++++++++++++++ 1 file changed, 158 insertions(+) create mode 100644 .github/scripts/verify_crate_checksums.py diff --git a/.github/scripts/verify_crate_checksums.py b/.github/scripts/verify_crate_checksums.py new file mode 100644 index 00000000..3b7429e8 --- /dev/null +++ b/.github/scripts/verify_crate_checksums.py @@ -0,0 +1,158 @@ +#!/usr/bin/env python3 +"""Verify fetched crates.io archive bytes against exact Cargo.lock checksums.""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import os +import re +import sys +import tomllib +from pathlib import Path +from typing import Any + +CRATES_IO_SOURCE = "registry+https://github.com/rust-lang/crates.io-index" +HEX64 = re.compile(r"^[0-9a-f]{64}$") + + +class ChecksumError(ValueError): + """Raised when fetched crate bytes cannot prove the locked checksum.""" + + +def require_string(value: object, message: str) -> str: + if not isinstance(value, str) or not value: + raise ChecksumError(message) + return value + + +def sha256_file(path: Path) -> str: + digest = hashlib.sha256() + with path.open("rb") as handle: + for chunk in iter(lambda: handle.read(1024 * 1024), b""): + digest.update(chunk) + return digest.hexdigest() + + +def load_json(path: Path) -> dict[str, Any]: + try: + value = json.loads(path.read_text(encoding="utf-8")) + except (OSError, UnicodeError, json.JSONDecodeError) as error: + raise ChecksumError(f"invalid dependency inventory: {error}") from error + if not isinstance(value, dict): + raise ChecksumError("dependency inventory root is not an object") + return value + + +def load_lock(path: Path) -> list[dict[str, Any]]: + try: + value = tomllib.loads(path.read_text(encoding="utf-8")) + except (OSError, UnicodeError, tomllib.TOMLDecodeError) as error: + raise ChecksumError(f"invalid Cargo.lock: {error}") from error + packages = value.get("package") + if not isinstance(packages, list) or not all(isinstance(item, dict) for item in packages): + raise ChecksumError("Cargo.lock package records are missing or malformed") + return packages + + +def archive_candidates(cargo_home: Path, name: str, version: str) -> list[Path]: + filename = f"{name}-{version}.crate" + cache_root = cargo_home / "registry" / "cache" + if not cache_root.is_dir(): + raise ChecksumError(f"Cargo registry cache is missing: {cache_root}") + return sorted(path for path in cache_root.glob(f"*/{filename}") if path.is_file()) + + +def verify(inventory: dict[str, Any], lock_packages: list[dict[str, Any]], cargo_home: Path) -> dict[str, object]: + packages = inventory.get("packages") + if inventory.get("schema") != 2 or inventory.get("ok") is not True or not isinstance(packages, list): + raise ChecksumError("dependency inventory is not a successful schema-2 inventory") + + lock_by_identity: dict[tuple[str, str, str], str] = {} + for index, package in enumerate(lock_packages): + name = require_string(package.get("name"), f"Cargo.lock package {index} has invalid name") + version = require_string(package.get("version"), f"Cargo.lock package {index} has invalid version") + source = package.get("source") + if source != CRATES_IO_SOURCE: + continue + checksum = package.get("checksum") + if not isinstance(checksum, str) or HEX64.fullmatch(checksum) is None: + raise ChecksumError(f"Cargo.lock crates.io package {name}@{version} has invalid checksum") + identity = (name, version, source) + if identity in lock_by_identity: + raise ChecksumError(f"duplicate crates.io Cargo.lock identity: {name}@{version}") + lock_by_identity[identity] = checksum + + checksums: dict[str, str] = {} + seen_lock_identities: set[tuple[str, str, str]] = set() + for index, package in enumerate(packages): + if not isinstance(package, dict): + raise ChecksumError(f"dependency inventory package {index} is not an object") + if package.get("source_class") != "crates.io": + continue + package_id = require_string( + package.get("package_id"), f"dependency inventory package {index} has invalid package id" + ) + name = require_string(package.get("name"), f"dependency inventory package {index} has invalid name") + version = require_string( + package.get("version"), f"dependency inventory package {index} has invalid version" + ) + source = package.get("source") + if source != CRATES_IO_SOURCE: + raise ChecksumError(f"inventory crates.io package {package_id} has inconsistent source") + identity = (name, version, source) + expected = lock_by_identity.get(identity) + if expected is None: + raise ChecksumError(f"inventory crates.io package is absent from Cargo.lock: {name}@{version}") + if identity in seen_lock_identities: + raise ChecksumError(f"duplicate crates.io inventory identity: {name}@{version}") + seen_lock_identities.add(identity) + + archives = archive_candidates(cargo_home, name, version) + if not archives: + raise ChecksumError(f"fetched crate archive is missing: {name}@{version}") + digests = sorted({sha256_file(path) for path in archives}) + if len(digests) != 1: + raise ChecksumError(f"fetched crate archives disagree: {name}@{version}") + actual = digests[0] + if actual != expected: + raise ChecksumError( + f"fetched crate checksum mismatch for {name}@{version}: expected {expected}, found {actual}" + ) + checksums[package_id] = actual + + if seen_lock_identities != set(lock_by_identity): + missing = sorted(f"{name}@{version}" for name, version, _ in set(lock_by_identity) - seen_lock_identities) + raise ChecksumError(f"Cargo.lock crates.io packages missing from inventory: {missing[:1]}") + + ordered = dict(sorted(checksums.items())) + return { + "checksums": ordered, + "ok": True, + "package_count": len(ordered), + "schema": 1, + } + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--inventory", type=Path, required=True) + parser.add_argument("--cargo-lock", type=Path, default=Path("Cargo.lock")) + parser.add_argument( + "--cargo-home", + type=Path, + default=Path(os.environ.get("CARGO_HOME", str(Path.home() / ".cargo"))), + ) + args = parser.parse_args() + try: + result = verify(load_json(args.inventory), load_lock(args.cargo_lock), args.cargo_home) + except ChecksumError as error: + print(json.dumps({"error": str(error), "ok": False}, sort_keys=True)) + return 1 + print(json.dumps(result, indent=2, sort_keys=True)) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) From fe2d19281e601c3dd373956779c901deb07dbb55 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 13:53:09 +0300 Subject: [PATCH 40/62] test(af01): cover fetched crate checksum proof --- .../scripts/test_verify_crate_checksums.py | 110 ++++++++++++++++++ 1 file changed, 110 insertions(+) create mode 100644 .github/scripts/test_verify_crate_checksums.py diff --git a/.github/scripts/test_verify_crate_checksums.py b/.github/scripts/test_verify_crate_checksums.py new file mode 100644 index 00000000..1df26ef9 --- /dev/null +++ b/.github/scripts/test_verify_crate_checksums.py @@ -0,0 +1,110 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import hashlib +import importlib.util +import sys +import tempfile +import unittest +from pathlib import Path + +MODULE_PATH = Path(__file__).with_name("verify_crate_checksums.py") +SPEC = importlib.util.spec_from_file_location("verify_crate_checksums_target", MODULE_PATH) +assert SPEC is not None and SPEC.loader is not None +VERIFY = importlib.util.module_from_spec(SPEC) +sys.modules[SPEC.name] = VERIFY +SPEC.loader.exec_module(VERIFY) + + +def fixture(archive_bytes: bytes = b"crate archive bytes") -> tuple[dict[str, object], list[dict[str, object]], bytes]: + digest = hashlib.sha256(archive_bytes).hexdigest() + package_id = f"{VERIFY.CRATES_IO_SOURCE}#dep@1.0.0" + inventory = { + "schema": 2, + "ok": True, + "packages": [ + { + "package_id": "path+file:///workspace/root#0.1.0", + "name": "root", + "version": "0.1.0", + "source": None, + "source_class": "workspace", + }, + { + "package_id": package_id, + "name": "dep", + "version": "1.0.0", + "source": VERIFY.CRATES_IO_SOURCE, + "source_class": "crates.io", + }, + ], + } + lock = [ + {"name": "root", "version": "0.1.0"}, + { + "name": "dep", + "version": "1.0.0", + "source": VERIFY.CRATES_IO_SOURCE, + "checksum": digest, + }, + ] + return inventory, lock, archive_bytes + + +class CrateChecksumVerificationTests(unittest.TestCase): + def write_archive(self, cargo_home: Path, content: bytes, registry: str = "index.crates.io-test") -> Path: + path = cargo_home / "registry" / "cache" / registry / "dep-1.0.0.crate" + path.parent.mkdir(parents=True, exist_ok=True) + path.write_bytes(content) + return path + + def test_fetched_archive_matches_locked_checksum(self) -> None: + inventory, lock, archive = fixture() + with tempfile.TemporaryDirectory() as directory: + cargo_home = Path(directory) / "cargo" + self.write_archive(cargo_home, archive) + result = VERIFY.verify(inventory, lock, cargo_home) + package_id = f"{VERIFY.CRATES_IO_SOURCE}#dep@1.0.0" + digest = hashlib.sha256(archive).hexdigest() + self.assertEqual( + result, + {"checksums": {package_id: digest}, "ok": True, "package_count": 1, "schema": 1}, + ) + + def test_checksum_mismatch_fails_closed(self) -> None: + inventory, lock, _ = fixture() + with tempfile.TemporaryDirectory() as directory: + cargo_home = Path(directory) / "cargo" + self.write_archive(cargo_home, b"tampered archive") + with self.assertRaisesRegex(VERIFY.ChecksumError, "checksum mismatch"): + VERIFY.verify(inventory, lock, cargo_home) + + def test_missing_archive_fails_closed(self) -> None: + inventory, lock, _ = fixture() + with tempfile.TemporaryDirectory() as directory: + cargo_home = Path(directory) / "cargo" + (cargo_home / "registry" / "cache" / "index.crates.io-test").mkdir(parents=True) + with self.assertRaisesRegex(VERIFY.ChecksumError, "archive is missing"): + VERIFY.verify(inventory, lock, cargo_home) + + def test_conflicting_cached_archives_fail_closed(self) -> None: + inventory, lock, archive = fixture() + with tempfile.TemporaryDirectory() as directory: + cargo_home = Path(directory) / "cargo" + self.write_archive(cargo_home, archive, "index.crates.io-a") + self.write_archive(cargo_home, b"other bytes", "index.crates.io-b") + with self.assertRaisesRegex(VERIFY.ChecksumError, "archives disagree"): + VERIFY.verify(inventory, lock, cargo_home) + + def test_missing_inventory_package_fails_closed(self) -> None: + inventory, lock, archive = fixture() + inventory["packages"] = inventory["packages"][:1] + with tempfile.TemporaryDirectory() as directory: + cargo_home = Path(directory) / "cargo" + self.write_archive(cargo_home, archive) + with self.assertRaisesRegex(VERIFY.ChecksumError, "missing from inventory"): + VERIFY.verify(inventory, lock, cargo_home) + + +if __name__ == "__main__": + unittest.main() From f0c07e6e5c73e1c5bbb87090cc3697c5a8535d9d Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 13:53:19 +0300 Subject: [PATCH 41/62] test(af01): expose crate checksum tests to CI trust suite --- ..._workflow_trust_crate_checksum_verification.py | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 .github/scripts/test_audit_workflow_trust_crate_checksum_verification.py diff --git a/.github/scripts/test_audit_workflow_trust_crate_checksum_verification.py b/.github/scripts/test_audit_workflow_trust_crate_checksum_verification.py new file mode 100644 index 00000000..e2764086 --- /dev/null +++ b/.github/scripts/test_audit_workflow_trust_crate_checksum_verification.py @@ -0,0 +1,15 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import importlib.util +import sys +from pathlib import Path + +TARGET = Path(__file__).with_name("test_verify_crate_checksums.py") +SPEC = importlib.util.spec_from_file_location("af01_crate_checksum_tests", TARGET) +assert SPEC is not None and SPEC.loader is not None +MODULE = importlib.util.module_from_spec(SPEC) +sys.modules[SPEC.name] = MODULE +SPEC.loader.exec_module(MODULE) + +CrateChecksumVerificationTests = MODULE.CrateChecksumVerificationTests From 9d30d163a7218831683252c6f0e883337ef82f5a Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 13:55:09 +0300 Subject: [PATCH 42/62] fix(af01): verify fetched crate archive checksums --- .github/workflows/af01-assurance-proof.yml | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/.github/workflows/af01-assurance-proof.yml b/.github/workflows/af01-assurance-proof.yml index 57afa447..ffc806b0 100644 --- a/.github/workflows/af01-assurance-proof.yml +++ b/.github/workflows/af01-assurance-proof.yml @@ -30,6 +30,7 @@ jobs: set -euo pipefail test "$(git rev-parse HEAD)" = "$AF01_SOURCE_SHA" python3 .github/scripts/test_build_af01_assurance_summary.py + python3 .github/scripts/test_verify_crate_checksums.py mkdir -p "$AF01_EVIDENCE_DIR" - name: Generate exact workflow-trust evidence run: | @@ -42,6 +43,8 @@ jobs: second_metadata=/tmp/af01-dependency-metadata-repeat.json first_inventory="$AF01_EVIDENCE_DIR/dependency-inventory.json" second_inventory=/tmp/dependency-inventory-repeat.json + checksum_evidence="$AF01_EVIDENCE_DIR/crate-checksums.json" + cargo metadata --locked --format-version 1 > "$first_metadata" python3 .github/scripts/summarize_cargo_metadata.py \ < "$first_metadata" \ @@ -51,6 +54,14 @@ jobs: < "$second_metadata" \ > "$second_inventory" cmp --silent "$first_inventory" "$second_inventory" + + cargo fetch --locked + python3 .github/scripts/verify_crate_checksums.py \ + --inventory "$first_inventory" \ + --cargo-lock Cargo.lock \ + --cargo-home "$CARGO_HOME" \ + > "$checksum_evidence" + python3 - <<'PY' import hashlib import json @@ -59,11 +70,18 @@ jobs: evidence_dir = Path(os.environ["AF01_EVIDENCE_DIR"]) inventory_path = evidence_dir / "dependency-inventory.json" + checksum_path = evidence_dir / "crate-checksums.json" inventory = json.loads(inventory_path.read_text(encoding="utf-8")) + checksum_evidence = json.loads(checksum_path.read_text(encoding="utf-8")) + if checksum_evidence.get("schema") != 1 or checksum_evidence.get("ok") is not True: + raise SystemExit("crate checksum evidence is not a successful schema-1 proof") proof = { "cargo_lock_sha256": hashlib.sha256(Path("Cargo.lock").read_bytes()).hexdigest(), - "command": ["cargo", "metadata", "--locked", "--format-version", "1"], + "command": ["cargo", "metadata", "--locked", "--format-version", 1], + "crate_checksum_package_count": checksum_evidence["package_count"], + "crate_checksums_sha256": hashlib.sha256(checksum_path.read_bytes()).hexdigest(), + "fetch_command": ["cargo", "fetch", "--locked"], "graph_sha256": inventory["graph_sha256"], "head_sha": os.environ["AF01_SOURCE_SHA"], "inventory_sha256": hashlib.sha256(inventory_path.read_bytes()).hexdigest(), From b5bdf62dae2131e3a4222afc486c7d97330a7eb3 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 14:06:07 +0300 Subject: [PATCH 43/62] fix(af01): bind fetched crate checksums into final assurance --- .../build_af01_assurance_summary_verified.py | 143 ++++++++++++++++++ 1 file changed, 143 insertions(+) create mode 100644 .github/scripts/build_af01_assurance_summary_verified.py diff --git a/.github/scripts/build_af01_assurance_summary_verified.py b/.github/scripts/build_af01_assurance_summary_verified.py new file mode 100644 index 00000000..9496a5f3 --- /dev/null +++ b/.github/scripts/build_af01_assurance_summary_verified.py @@ -0,0 +1,143 @@ +#!/usr/bin/env python3 +"""Build the final AF-01 assurance summary with independent fetched-crate checksum binding.""" + +from __future__ import annotations + +import argparse +import importlib.util +import json +import os +import sys +from pathlib import Path +from typing import Any + +SCRIPT_DIR = Path(__file__).resolve().parent +FETCH_COMMAND = ["cargo", "fetch", "--locked"] +CHECKSUM_EVIDENCE = "crate-checksums.json" + + +def _load_module(name: str, filename: str): + spec = importlib.util.spec_from_file_location(name, SCRIPT_DIR / filename) + if spec is None or spec.loader is None: + raise RuntimeError(f"cannot load {filename}") + module = importlib.util.module_from_spec(spec) + sys.modules[name] = module + spec.loader.exec_module(module) + return module + + +BASE = _load_module("af01_assurance_summary_core", "build_af01_assurance_summary.py") +VERIFY = _load_module("af01_crate_checksum_verifier", "verify_crate_checksums.py") + + +def validate_checksum_binding( + root: Path, + evidence_dir: Path, + cache_root: Path, +) -> tuple[str, int]: + root = root.resolve() + evidence_dir = evidence_dir.resolve() + cache_root = cache_root.resolve() + + inventory_path = evidence_dir / BASE.EVIDENCE_FILES["dependency_inventory"] + proof_path = evidence_dir / BASE.EVIDENCE_FILES["dependency_inventory_proof"] + checksum_path = evidence_dir / CHECKSUM_EVIDENCE + + inventory = BASE.read_json(inventory_path) + proof = BASE.read_json(proof_path) + recorded = BASE.read_json(checksum_path) + + try: + regenerated = VERIFY.verify( + inventory, + VERIFY.load_lock(root / "Cargo.lock"), + cache_root, + ) + except VERIFY.ChecksumError as error: + raise BASE.AssuranceError(f"fetched crate checksum verification failed: {error}") from error + + if recorded != regenerated: + raise BASE.AssuranceError( + "recorded crate checksum evidence does not match independently reverified archive bytes" + ) + + checksums = recorded.get("checksums") + package_count = recorded.get("package_count") + if ( + recorded.get("schema") != 1 + or recorded.get("ok") is not True + or not isinstance(checksums, dict) + or not isinstance(package_count, int) + or isinstance(package_count, bool) + or package_count != len(checksums) + ): + raise BASE.AssuranceError("crate checksum evidence is not a successful canonical schema-1 proof") + + evidence_sha = BASE.sha256_file(checksum_path) + if ( + proof.get("fetch_command") != FETCH_COMMAND + or proof.get("crate_checksums_sha256") != evidence_sha + or proof.get("crate_checksum_package_count") != package_count + ): + raise BASE.AssuranceError("dependency inventory proof does not bind exact crate checksum evidence") + + return evidence_sha, package_count + + +def build_verified_summary( + root: Path, + evidence_dir: Path, + source_sha: str, + tree_sha: str, + cache_root: Path, +) -> dict[str, Any]: + summary = BASE.build_summary(root, evidence_dir, source_sha, tree_sha) + checksum_sha, package_count = validate_checksum_binding(root, evidence_dir, cache_root) + + dependency_graph = summary.get("dependency_graph") + evidence_sha256 = summary.get("evidence_sha256") + if not isinstance(dependency_graph, dict) or not isinstance(evidence_sha256, dict): + raise BASE.AssuranceError("core assurance summary has an invalid dependency/evidence schema") + + dependency_graph["crate_checksum_package_count"] = package_count + dependency_graph["crate_checksums_sha256"] = checksum_sha + dependency_graph["fetch_command"] = FETCH_COMMAND + evidence_sha256["crate_checksums"] = checksum_sha + return summary + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--root", type=Path, default=Path(".")) + parser.add_argument("--evidence-dir", type=Path, required=True) + parser.add_argument("--source-sha", required=True) + parser.add_argument("--tree-sha", required=True) + parser.add_argument("--output", type=Path, required=True) + parser.add_argument( + "--cache-root", + type=Path, + default=Path(os.environ.get("CARGO_HOME", str(Path.home() / ".cargo"))), + ) + args = parser.parse_args() + + try: + summary = build_verified_summary( + args.root, + args.evidence_dir, + args.source_sha, + args.tree_sha, + args.cache_root, + ) + rendered = BASE.render_summary(summary) + args.output.write_bytes(rendered) + except (BASE.AssuranceError, OSError, UnicodeError) as error: + print(json.dumps({"error": str(error), "ok": False}, sort_keys=True)) + return 1 + + digest = BASE.sha256_bytes(rendered) + print(f"AF01_ASSURANCE_SHA256={digest}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) From d09ba4cf2b80b65bec120adfbef27fb9b9b7dc5b Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 14:06:29 +0300 Subject: [PATCH 44/62] test(af01): cover final crate checksum assurance binding --- ...t_build_af01_assurance_summary_verified.py | 150 ++++++++++++++++++ 1 file changed, 150 insertions(+) create mode 100644 .github/scripts/test_build_af01_assurance_summary_verified.py diff --git a/.github/scripts/test_build_af01_assurance_summary_verified.py b/.github/scripts/test_build_af01_assurance_summary_verified.py new file mode 100644 index 00000000..b6823aee --- /dev/null +++ b/.github/scripts/test_build_af01_assurance_summary_verified.py @@ -0,0 +1,150 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import hashlib +import importlib.util +import json +import sys +import tempfile +import unittest +from pathlib import Path + +MODULE_PATH = Path(__file__).with_name("build_af01_assurance_summary_verified.py") +SPEC = importlib.util.spec_from_file_location("af01_assurance_verified", MODULE_PATH) +assert SPEC is not None and SPEC.loader is not None +VERIFIED = importlib.util.module_from_spec(SPEC) +sys.modules[SPEC.name] = VERIFIED +SPEC.loader.exec_module(VERIFIED) + + +def write_json(path: Path, value: object) -> None: + path.write_text(json.dumps(value, indent=2, sort_keys=True) + "\n", encoding="utf-8") + + +class VerifiedAssuranceSummaryTests(unittest.TestCase): + def setUp(self) -> None: + self.temp = tempfile.TemporaryDirectory() + base = Path(self.temp.name) + self.root = base / "repo" + self.evidence = base / "evidence" + self.cache = base / "cache-home" + self.root.mkdir() + self.evidence.mkdir() + + self.archive = b"independently fetched crate archive bytes" + self.digest = hashlib.sha256(self.archive).hexdigest() + self.package_id = f"{VERIFIED.VERIFY.CRATES_IO_SOURCE}#dep@1.0.0" + + (self.root / "Cargo.lock").write_text( + "version = 4\n\n" + "[[package]]\n" + "name = \"root\"\n" + "version = \"0.1.0\"\n\n" + "[[package]]\n" + "name = \"dep\"\n" + "version = \"1.0.0\"\n" + f"source = \"{VERIFIED.VERIFY.CRATES_IO_SOURCE}\"\n" + f"checksum = \"{self.digest}\"\n", + encoding="utf-8", + ) + + inventory = { + "schema": 2, + "ok": True, + "packages": [ + { + "package_id": "path+file:///workspace/root#0.1.0", + "name": "root", + "version": "0.1.0", + "source": None, + "source_class": "workspace", + }, + { + "package_id": self.package_id, + "name": "dep", + "version": "1.0.0", + "source": VERIFIED.VERIFY.CRATES_IO_SOURCE, + "source_class": "crates.io", + }, + ], + } + write_json(self.evidence / "dependency-inventory.json", inventory) + + archive_path = self.cache / "registry" / "cache" / "index.crates.io-test" / "dep-1.0.0.crate" + archive_path.parent.mkdir(parents=True) + archive_path.write_bytes(self.archive) + + checksum_evidence = VERIFIED.VERIFY.verify( + inventory, + VERIFIED.VERIFY.load_lock(self.root / "Cargo.lock"), + self.cache, + ) + checksum_path = self.evidence / VERIFIED.CHECKSUM_EVIDENCE + write_json(checksum_path, checksum_evidence) + checksum_sha = hashlib.sha256(checksum_path.read_bytes()).hexdigest() + + write_json( + self.evidence / "dependency-inventory-proof.json", + { + "schema": 1, + "head_sha": "a" * 40, + "fetch_command": VERIFIED.FETCH_COMMAND, + "crate_checksum_package_count": 1, + "crate_checksums_sha256": checksum_sha, + }, + ) + + def tearDown(self) -> None: + self.temp.cleanup() + + def test_exact_archive_bytes_are_bound_into_final_proof(self) -> None: + checksum_sha, count = VERIFIED.validate_checksum_binding( + self.root, self.evidence, self.cache + ) + self.assertEqual(count, 1) + self.assertEqual( + checksum_sha, + hashlib.sha256((self.evidence / VERIFIED.CHECKSUM_EVIDENCE).read_bytes()).hexdigest(), + ) + + def test_tampered_recorded_checksum_evidence_fails_closed(self) -> None: + path = self.evidence / VERIFIED.CHECKSUM_EVIDENCE + value = json.loads(path.read_text(encoding="utf-8")) + value["checksums"][self.package_id] = "0" * 64 + write_json(path, value) + with self.assertRaisesRegex( + VERIFIED.BASE.AssuranceError, "does not match independently reverified archive bytes" + ): + VERIFIED.validate_checksum_binding(self.root, self.evidence, self.cache) + + def test_tampered_fetched_archive_fails_closed(self) -> None: + archive_path = self.cache / "registry" / "cache" / "index.crates.io-test" / "dep-1.0.0.crate" + archive_path.write_bytes(b"tampered") + with self.assertRaisesRegex( + VERIFIED.BASE.AssuranceError, "fetched crate checksum verification failed" + ): + VERIFIED.validate_checksum_binding(self.root, self.evidence, self.cache) + + def test_dependency_proof_must_bind_checksum_evidence_digest(self) -> None: + proof_path = self.evidence / "dependency-inventory-proof.json" + proof = json.loads(proof_path.read_text(encoding="utf-8")) + proof["crate_checksums_sha256"] = "0" * 64 + write_json(proof_path, proof) + with self.assertRaisesRegex( + VERIFIED.BASE.AssuranceError, "does not bind exact crate checksum evidence" + ): + VERIFIED.validate_checksum_binding(self.root, self.evidence, self.cache) + + def test_dependency_proof_must_bind_fetch_command(self) -> None: + proof_path = self.evidence / "dependency-inventory-proof.json" + proof = json.loads(proof_path.read_text(encoding="utf-8")) + proof["fetch_command"] = ["cargo", "fetch"] + write_json(proof_path, proof) + with self.assertRaisesRegex( + VERIFIED.BASE.AssuranceError, "does not bind exact crate checksum evidence" + ): + VERIFIED.validate_checksum_binding(self.root, self.evidence, self.cache) + + +if __name__ == "__main__": + unittest.main() From a042df08cfde2651c7a2cae00eb2075890e60b3e Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 14:06:40 +0300 Subject: [PATCH 45/62] test(af01): expose verified assurance tests to trust suite --- ...t_workflow_trust_verified_assurance_summary.py | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 .github/scripts/test_audit_workflow_trust_verified_assurance_summary.py diff --git a/.github/scripts/test_audit_workflow_trust_verified_assurance_summary.py b/.github/scripts/test_audit_workflow_trust_verified_assurance_summary.py new file mode 100644 index 00000000..349399fa --- /dev/null +++ b/.github/scripts/test_audit_workflow_trust_verified_assurance_summary.py @@ -0,0 +1,15 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import importlib.util +import sys +from pathlib import Path + +TARGET = Path(__file__).with_name("test_build_af01_assurance_summary_verified.py") +SPEC = importlib.util.spec_from_file_location("verified_assurance_summary_tests", TARGET) +assert SPEC is not None and SPEC.loader is not None +MODULE = importlib.util.module_from_spec(SPEC) +sys.modules[SPEC.name] = MODULE +SPEC.loader.exec_module(MODULE) + +VerifiedAssuranceSummaryTests = MODULE.VerifiedAssuranceSummaryTests From 0f3e3d975f13ed1e67b323793210620dd4ff0748 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 14:07:15 +0300 Subject: [PATCH 46/62] fix(af01): make crate checksum proof terminal assurance evidence --- .github/workflows/af01-assurance-proof.yml | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/.github/workflows/af01-assurance-proof.yml b/.github/workflows/af01-assurance-proof.yml index ffc806b0..1321a7f8 100644 --- a/.github/workflows/af01-assurance-proof.yml +++ b/.github/workflows/af01-assurance-proof.yml @@ -31,6 +31,7 @@ jobs: test "$(git rev-parse HEAD)" = "$AF01_SOURCE_SHA" python3 .github/scripts/test_build_af01_assurance_summary.py python3 .github/scripts/test_verify_crate_checksums.py + python3 .github/scripts/test_build_af01_assurance_summary_verified.py mkdir -p "$AF01_EVIDENCE_DIR" - name: Generate exact workflow-trust evidence run: | @@ -58,8 +59,6 @@ jobs: cargo fetch --locked python3 .github/scripts/verify_crate_checksums.py \ --inventory "$first_inventory" \ - --cargo-lock Cargo.lock \ - --cargo-home "$CARGO_HOME" \ > "$checksum_evidence" python3 - <<'PY' @@ -204,13 +203,13 @@ jobs: second=/tmp/assurance-summary-repeat.json first_log=/tmp/assurance-summary-first.log second_log=/tmp/assurance-summary-second.log - python3 .github/scripts/build_af01_assurance_summary.py \ + python3 .github/scripts/build_af01_assurance_summary_verified.py \ --evidence-dir "$AF01_EVIDENCE_DIR" \ --source-sha "$AF01_SOURCE_SHA" \ --tree-sha "$tree_sha" \ --output "$first" | tee "$first_log" first_line="$(cat "$first_log")" - python3 .github/scripts/build_af01_assurance_summary.py \ + python3 .github/scripts/build_af01_assurance_summary_verified.py \ --evidence-dir "$AF01_EVIDENCE_DIR" \ --source-sha "$AF01_SOURCE_SHA" \ --tree-sha "$tree_sha" \ From 2477a06baddcfed14a5bcf02877320d2030b3607 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 14:09:20 +0300 Subject: [PATCH 47/62] fix(af01): preserve literal metadata command identity --- .github/workflows/af01-assurance-proof.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/af01-assurance-proof.yml b/.github/workflows/af01-assurance-proof.yml index 1321a7f8..f4104541 100644 --- a/.github/workflows/af01-assurance-proof.yml +++ b/.github/workflows/af01-assurance-proof.yml @@ -77,7 +77,7 @@ jobs: proof = { "cargo_lock_sha256": hashlib.sha256(Path("Cargo.lock").read_bytes()).hexdigest(), - "command": ["cargo", "metadata", "--locked", "--format-version", 1], + "command": ["cargo", "metadata", "--locked", "--format-version", "1"], "crate_checksum_package_count": checksum_evidence["package_count"], "crate_checksums_sha256": hashlib.sha256(checksum_path.read_bytes()).hexdigest(), "fetch_command": ["cargo", "fetch", "--locked"], From d5ac1daca7a2e82299bd087cdea433c585c60b9c Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 14:16:22 +0300 Subject: [PATCH 48/62] fix(af01): mechanically bind scanner invocations to assurance --- .../validate_af01_scanner_invocations.py | 196 ++++++++++++++++++ 1 file changed, 196 insertions(+) create mode 100644 .github/scripts/validate_af01_scanner_invocations.py diff --git a/.github/scripts/validate_af01_scanner_invocations.py b/.github/scripts/validate_af01_scanner_invocations.py new file mode 100644 index 00000000..2d7a3909 --- /dev/null +++ b/.github/scripts/validate_af01_scanner_invocations.py @@ -0,0 +1,196 @@ +#!/usr/bin/env python3 +"""Validate exact AF-01 scanner action refs, inputs, and direct cargo-audit commands.""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import re +import sys +from pathlib import Path +from typing import Any + +CARGO_DENY_USES = "EmbarkStudios/cargo-deny-action@3c6349835b2b7b196a839186cb8b78e02f7b5f25" +CARGO_DENY_INPUTS = { + "arguments": "--all-features", + "command": "check", + "command-arguments": "advisories bans licenses sources", + "log-level": "warn", +} +ZIZMOR_USES = "zizmorcore/zizmor-action@3dc1ecc9bcb9e94e9b2c709687979e1298497054" +ZIZMOR_INPUTS = { + "advanced-security": "false", + "annotations": "false", + "collect": "all", + "color": "false", + "fail-on-no-inputs": "true", + "inputs": ".", + "min-severity": "medium", + "online-audits": "false", + "persona": "regular", + "version": "1.29.0", +} +CARGO_AUDIT_INSTALL = "cargo install cargo-audit --version 0.22.2 --locked" +CARGO_AUDIT_RUN = 'cargo audit --file Cargo.lock --json > "$AF01_EVIDENCE_DIR/cargo-audit.json"' +FULL_SHA = re.compile(r"^[0-9a-f]{40}$") + + +class ScannerContractError(ValueError): + """Raised when the workflow scanner invocation no longer matches the assurance contract.""" + + +def _indent(line: str) -> int: + return len(line) - len(line.lstrip(" ")) + + +def _scalar(value: str) -> str: + value = value.strip() + if " #" in value: + value = value.split(" #", 1)[0].rstrip() + if len(value) >= 2 and value[0] == value[-1] and value[0] in {"'", '"'}: + value = value[1:-1] + return value + + +def _step_bounds(lines: list[str], uses_index: int) -> tuple[int, int, int]: + uses_indent = _indent(lines[uses_index]) + step_indent = uses_indent - 2 + start = uses_index + for index in range(uses_index, -1, -1): + line = lines[index] + if _indent(line) == step_indent and line.lstrip().startswith("- "): + start = index + break + else: + raise ScannerContractError("scanner uses entry is not inside a static workflow step") + + end = len(lines) + for index in range(start + 1, len(lines)): + line = lines[index] + if line.strip() and _indent(line) < step_indent: + end = index + break + if _indent(line) == step_indent and line.lstrip().startswith("- "): + end = index + break + return start, end, step_indent + + +def _step_inputs(lines: list[str], uses_index: int) -> dict[str, str]: + start, end, step_indent = _step_bounds(lines, uses_index) + with_indexes = [ + index + for index in range(start + 1, end) + if _indent(lines[index]) == step_indent + 2 and lines[index].strip() == "with:" + ] + if len(with_indexes) != 1: + raise ScannerContractError("scanner step must contain exactly one static with mapping") + + result: dict[str, str] = {} + with_index = with_indexes[0] + for index in range(with_index + 1, end): + line = lines[index] + if not line.strip() or line.lstrip().startswith("#"): + continue + indent = _indent(line) + if indent <= step_indent + 2: + break + if indent != step_indent + 4 or ":" not in line: + raise ScannerContractError("scanner with mapping contains unsupported nested syntax") + key, raw = line.strip().split(":", 1) + if key in result: + raise ScannerContractError(f"scanner with mapping contains duplicate input {key!r}") + value = _scalar(raw) + if not value: + raise ScannerContractError(f"scanner input {key!r} is empty or dynamic") + if "${{" in value: + raise ScannerContractError(f"scanner input {key!r} is dynamic") + result[key] = value + return result + + +def _find_action(lines: list[str], repository: str) -> tuple[str, dict[str, str]]: + prefix = f"{repository}@" + matches: list[tuple[int, str]] = [] + for index, line in enumerate(lines): + stripped = line.strip() + if not stripped.startswith("uses:"): + continue + value = _scalar(stripped.split(":", 1)[1]) + if value.startswith(prefix): + matches.append((index, value)) + if len(matches) != 1: + raise ScannerContractError( + f"expected exactly one {repository} scanner step, found {len(matches)}" + ) + index, uses = matches[0] + revision = uses.rsplit("@", 1)[1] + if FULL_SHA.fullmatch(revision) is None: + raise ScannerContractError(f"{repository} scanner is not pinned to a full commit SHA") + return uses, _step_inputs(lines, index) + + +def _require_exact_shell_line(lines: list[str], command: str, label: str) -> None: + count = sum(1 for line in lines if line.strip() == command) + if count != 1: + raise ScannerContractError(f"{label} exact command must appear once, found {count}") + + +def validate_workflow(path: Path) -> dict[str, Any]: + try: + lines = path.read_text(encoding="utf-8").splitlines() + except (OSError, UnicodeError) as error: + raise ScannerContractError(f"cannot read assurance workflow: {error}") from error + + cargo_deny_uses, cargo_deny_inputs = _find_action(lines, "EmbarkStudios/cargo-deny-action") + zizmor_uses, zizmor_inputs = _find_action(lines, "zizmorcore/zizmor-action") + + if cargo_deny_uses != CARGO_DENY_USES: + raise ScannerContractError("cargo-deny executed action commit does not match assurance identity") + if cargo_deny_inputs != CARGO_DENY_INPUTS: + raise ScannerContractError("cargo-deny executed inputs do not match assurance policy") + if zizmor_uses != ZIZMOR_USES: + raise ScannerContractError("zizmor executed action commit does not match assurance identity") + if zizmor_inputs != ZIZMOR_INPUTS: + raise ScannerContractError("zizmor executed inputs do not match assurance policy") + + _require_exact_shell_line(lines, CARGO_AUDIT_INSTALL, "cargo-audit install") + _require_exact_shell_line(lines, CARGO_AUDIT_RUN, "cargo-audit execution") + + contract = { + "cargo_audit": { + "install": CARGO_AUDIT_INSTALL, + "run": CARGO_AUDIT_RUN, + "version": "0.22.2", + }, + "cargo_deny": { + "inputs": cargo_deny_inputs, + "uses": cargo_deny_uses, + }, + "schema": 1, + "zizmor": { + "inputs": zizmor_inputs, + "uses": zizmor_uses, + }, + } + rendered = json.dumps(contract, sort_keys=True, separators=(",", ":")).encode("utf-8") + contract["sha256"] = hashlib.sha256(rendered).hexdigest() + return contract + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("workflow", type=Path) + args = parser.parse_args() + try: + result = validate_workflow(args.workflow) + except ScannerContractError as error: + print(json.dumps({"error": str(error), "ok": False}, sort_keys=True)) + return 1 + print(json.dumps(result, indent=2, sort_keys=True)) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) From 6a64c6b86e72174c39a2ca70b12db55afd054f83 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 14:16:42 +0300 Subject: [PATCH 49/62] test(af01): reject scanner invocation drift --- .../test_validate_af01_scanner_invocations.py | 109 ++++++++++++++++++ 1 file changed, 109 insertions(+) create mode 100644 .github/scripts/test_validate_af01_scanner_invocations.py diff --git a/.github/scripts/test_validate_af01_scanner_invocations.py b/.github/scripts/test_validate_af01_scanner_invocations.py new file mode 100644 index 00000000..bf01b9e5 --- /dev/null +++ b/.github/scripts/test_validate_af01_scanner_invocations.py @@ -0,0 +1,109 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import importlib.util +import sys +import tempfile +import unittest +from pathlib import Path + +MODULE_PATH = Path(__file__).with_name("validate_af01_scanner_invocations.py") +SPEC = importlib.util.spec_from_file_location("af01_scanner_invocations", MODULE_PATH) +assert SPEC is not None and SPEC.loader is not None +VALIDATE = importlib.util.module_from_spec(SPEC) +sys.modules[SPEC.name] = VALIDATE +SPEC.loader.exec_module(VALIDATE) + + +def workflow_text() -> str: + return f"""name: assurance +jobs: + assurance-proof: + runs-on: ubuntu-24.04 + steps: + - name: cargo deny + uses: {VALIDATE.CARGO_DENY_USES} + with: + command: check + arguments: --all-features + command-arguments: advisories bans licenses sources + log-level: warn + - name: install audit + run: {VALIDATE.CARGO_AUDIT_INSTALL} + - name: audit + run: | + set -euo pipefail + {VALIDATE.CARGO_AUDIT_RUN} + - name: zizmor + uses: {VALIDATE.ZIZMOR_USES} + with: + inputs: . + collect: all + online-audits: false + persona: regular + min-severity: medium + version: 1.29.0 + advanced-security: false + color: false + annotations: false + fail-on-no-inputs: true +""" + + +class ScannerInvocationContractTests(unittest.TestCase): + def validate(self, content: str) -> dict[str, object]: + with tempfile.TemporaryDirectory() as directory: + path = Path(directory) / "workflow.yml" + path.write_text(content, encoding="utf-8") + return VALIDATE.validate_workflow(path) + + def test_exact_scanner_invocations_are_canonically_bound(self) -> None: + result = self.validate(workflow_text()) + self.assertEqual(result["schema"], 1) + self.assertEqual(result["cargo_deny"]["uses"], VALIDATE.CARGO_DENY_USES) + self.assertEqual(result["cargo_deny"]["inputs"], VALIDATE.CARGO_DENY_INPUTS) + self.assertEqual(result["zizmor"]["uses"], VALIDATE.ZIZMOR_USES) + self.assertEqual(result["zizmor"]["inputs"], VALIDATE.ZIZMOR_INPUTS) + self.assertEqual(result["cargo_audit"]["version"], "0.22.2") + self.assertRegex(result["sha256"], r"^[0-9a-f]{64}$") + + def test_changed_cargo_deny_commit_fails_closed(self) -> None: + changed = workflow_text().replace( + VALIDATE.CARGO_DENY_USES, + "EmbarkStudios/cargo-deny-action@" + "0" * 40, + ) + with self.assertRaisesRegex( + VALIDATE.ScannerContractError, "cargo-deny executed action commit" + ): + self.validate(changed) + + def test_changed_cargo_deny_arguments_fail_closed(self) -> None: + changed = workflow_text().replace( + "command-arguments: advisories bans licenses sources", + "command-arguments: advisories", + ) + with self.assertRaisesRegex( + VALIDATE.ScannerContractError, "cargo-deny executed inputs" + ): + self.validate(changed) + + def test_changed_zizmor_version_fails_closed(self) -> None: + changed = workflow_text().replace("version: 1.29.0", "version: 9.9.9") + with self.assertRaisesRegex( + VALIDATE.ScannerContractError, "zizmor executed inputs" + ): + self.validate(changed) + + def test_changed_cargo_audit_execution_fails_closed(self) -> None: + changed = workflow_text().replace( + VALIDATE.CARGO_AUDIT_RUN, + "cargo audit --json", + ) + with self.assertRaisesRegex( + VALIDATE.ScannerContractError, "cargo-audit execution exact command" + ): + self.validate(changed) + + +if __name__ == "__main__": + unittest.main() From a9e3b2c1e1e24426dc61500530f966ea1dc6ca38 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 14:16:51 +0300 Subject: [PATCH 50/62] test(af01): expose scanner contract tests to trust suite --- ...st_audit_workflow_trust_scanner_invocations.py | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 .github/scripts/test_audit_workflow_trust_scanner_invocations.py diff --git a/.github/scripts/test_audit_workflow_trust_scanner_invocations.py b/.github/scripts/test_audit_workflow_trust_scanner_invocations.py new file mode 100644 index 00000000..4896a8e3 --- /dev/null +++ b/.github/scripts/test_audit_workflow_trust_scanner_invocations.py @@ -0,0 +1,15 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import importlib.util +import sys +from pathlib import Path + +TARGET = Path(__file__).with_name("test_validate_af01_scanner_invocations.py") +SPEC = importlib.util.spec_from_file_location("scanner_invocation_contract_tests", TARGET) +assert SPEC is not None and SPEC.loader is not None +MODULE = importlib.util.module_from_spec(SPEC) +sys.modules[SPEC.name] = MODULE +SPEC.loader.exec_module(MODULE) + +ScannerInvocationContractTests = MODULE.ScannerInvocationContractTests From 9b6c6425433b2b775c5a3e60fa81bbc00d086a10 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 14:17:37 +0300 Subject: [PATCH 51/62] fix(af01): bind executed scanner contract into final summary --- .../build_af01_assurance_summary_verified.py | 62 ++++++++++++++++++- 1 file changed, 60 insertions(+), 2 deletions(-) diff --git a/.github/scripts/build_af01_assurance_summary_verified.py b/.github/scripts/build_af01_assurance_summary_verified.py index 9496a5f3..04aa9a9f 100644 --- a/.github/scripts/build_af01_assurance_summary_verified.py +++ b/.github/scripts/build_af01_assurance_summary_verified.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -"""Build the final AF-01 assurance summary with independent fetched-crate checksum binding.""" +"""Build final AF-01 assurance with fetched-crate and executed-scanner binding.""" from __future__ import annotations @@ -14,6 +14,7 @@ SCRIPT_DIR = Path(__file__).resolve().parent FETCH_COMMAND = ["cargo", "fetch", "--locked"] CHECKSUM_EVIDENCE = "crate-checksums.json" +ASSURANCE_WORKFLOW = Path(".github/workflows/af01-assurance-proof.yml") def _load_module(name: str, filename: str): @@ -28,6 +29,7 @@ def _load_module(name: str, filename: str): BASE = _load_module("af01_assurance_summary_core", "build_af01_assurance_summary.py") VERIFY = _load_module("af01_crate_checksum_verifier", "verify_crate_checksums.py") +SCANNERS = _load_module("af01_scanner_contract", "validate_af01_scanner_invocations.py") def validate_checksum_binding( @@ -84,6 +86,54 @@ def validate_checksum_binding( return evidence_sha, package_count +def validate_scanner_binding(root: Path, evidence_dir: Path) -> dict[str, Any]: + root = root.resolve() + evidence_dir = evidence_dir.resolve() + try: + contract = SCANNERS.validate_workflow(root / ASSURANCE_WORKFLOW) + except SCANNERS.ScannerContractError as error: + raise BASE.AssuranceError(f"executed scanner contract validation failed: {error}") from error + + cargo_deny = BASE.read_json(evidence_dir / BASE.EVIDENCE_FILES["cargo_deny"]) + cargo_audit = BASE.read_json(evidence_dir / BASE.EVIDENCE_FILES["cargo_audit"]) + zizmor = BASE.read_json(evidence_dir / BASE.EVIDENCE_FILES["zizmor"]) + + deny_contract = contract.get("cargo_deny") + audit_contract = contract.get("cargo_audit") + zizmor_contract = contract.get("zizmor") + if not all(isinstance(value, dict) for value in (deny_contract, audit_contract, zizmor_contract)): + raise BASE.AssuranceError("executed scanner contract has an invalid schema") + + deny_inputs = deny_contract["inputs"] + zizmor_inputs = zizmor_contract["inputs"] + if not isinstance(deny_inputs, dict) or not isinstance(zizmor_inputs, dict): + raise BASE.AssuranceError("executed scanner inputs have an invalid schema") + + deny_revision = str(deny_contract["uses"]).rsplit("@", 1)[-1] + zizmor_revision = str(zizmor_contract["uses"]).rsplit("@", 1)[-1] + deny_checks = str(deny_inputs.get("command-arguments", "")).split() + + if ( + cargo_deny.get("action_commit") != deny_revision + or cargo_deny.get("checks") != deny_checks + ): + raise BASE.AssuranceError("cargo-deny proof does not match the executed action invocation") + + if cargo_audit.get("cargo_audit_version") != audit_contract.get("version"): + raise BASE.AssuranceError("cargo-audit proof does not match the executed install/run contract") + + if ( + zizmor.get("action_commit") != zizmor_revision + or zizmor.get("zizmor_version") != zizmor_inputs.get("version") + or zizmor.get("min_severity") != zizmor_inputs.get("min-severity") + or zizmor.get("online_audits") is not (zizmor_inputs.get("online-audits") == "true") + or zizmor.get("advanced_security") is not (zizmor_inputs.get("advanced-security") == "true") + ): + raise BASE.AssuranceError("zizmor proof does not match the executed action invocation") + + return contract + + def build_verified_summary( root: Path, evidence_dir: Path, @@ -93,16 +143,24 @@ def build_verified_summary( ) -> dict[str, Any]: summary = BASE.build_summary(root, evidence_dir, source_sha, tree_sha) checksum_sha, package_count = validate_checksum_binding(root, evidence_dir, cache_root) + scanner_contract = validate_scanner_binding(root, evidence_dir) dependency_graph = summary.get("dependency_graph") evidence_sha256 = summary.get("evidence_sha256") - if not isinstance(dependency_graph, dict) or not isinstance(evidence_sha256, dict): + execution_identities = summary.get("execution_identities") + if ( + not isinstance(dependency_graph, dict) + or not isinstance(evidence_sha256, dict) + or not isinstance(execution_identities, dict) + ): raise BASE.AssuranceError("core assurance summary has an invalid dependency/evidence schema") dependency_graph["crate_checksum_package_count"] = package_count dependency_graph["crate_checksums_sha256"] = checksum_sha dependency_graph["fetch_command"] = FETCH_COMMAND evidence_sha256["crate_checksums"] = checksum_sha + execution_identities["scanner_contract"] = scanner_contract + execution_identities["scanner_invocations_sha256"] = scanner_contract["sha256"] return summary From df6bd2760084fd61e899b6876a1ad19da7d27dd9 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 14:18:20 +0300 Subject: [PATCH 52/62] test(af01): reject scanner proof and execution divergence --- .../test_validate_af01_scanner_invocations.py | 100 ++++++++++++++++++ 1 file changed, 100 insertions(+) diff --git a/.github/scripts/test_validate_af01_scanner_invocations.py b/.github/scripts/test_validate_af01_scanner_invocations.py index bf01b9e5..80f42777 100644 --- a/.github/scripts/test_validate_af01_scanner_invocations.py +++ b/.github/scripts/test_validate_af01_scanner_invocations.py @@ -2,6 +2,7 @@ from __future__ import annotations import importlib.util +import json import sys import tempfile import unittest @@ -14,6 +15,18 @@ sys.modules[SPEC.name] = VALIDATE SPEC.loader.exec_module(VALIDATE) +VERIFIED_PATH = Path(__file__).with_name("build_af01_assurance_summary_verified.py") +VERIFIED_SPEC = importlib.util.spec_from_file_location("af01_verified_scanner_binding", VERIFIED_PATH) +assert VERIFIED_SPEC is not None and VERIFIED_SPEC.loader is not None +VERIFIED = importlib.util.module_from_spec(VERIFIED_SPEC) +sys.modules[VERIFIED_SPEC.name] = VERIFIED +VERIFIED_SPEC.loader.exec_module(VERIFIED) + + +def write_json(path: Path, value: object) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps(value, indent=2, sort_keys=True) + "\n", encoding="utf-8") + def workflow_text() -> str: return f"""name: assurance @@ -57,6 +70,38 @@ def validate(self, content: str) -> dict[str, object]: path.write_text(content, encoding="utf-8") return VALIDATE.validate_workflow(path) + def proof_fixture(self) -> tuple[tempfile.TemporaryDirectory[str], Path, Path]: + temp = tempfile.TemporaryDirectory() + base = Path(temp.name) + root = base / "repo" + evidence = base / "evidence" + workflow = root / VERIFIED.ASSURANCE_WORKFLOW + workflow.parent.mkdir(parents=True, exist_ok=True) + workflow.write_text(workflow_text(), encoding="utf-8") + evidence.mkdir() + write_json( + evidence / VERIFIED.BASE.EVIDENCE_FILES["cargo_deny"], + { + "action_commit": VALIDATE.CARGO_DENY_USES.rsplit("@", 1)[1], + "checks": ["advisories", "bans", "licenses", "sources"], + }, + ) + write_json( + evidence / VERIFIED.BASE.EVIDENCE_FILES["cargo_audit"], + {"cargo_audit_version": "0.22.2"}, + ) + write_json( + evidence / VERIFIED.BASE.EVIDENCE_FILES["zizmor"], + { + "action_commit": VALIDATE.ZIZMOR_USES.rsplit("@", 1)[1], + "zizmor_version": "1.29.0", + "min_severity": "medium", + "online_audits": False, + "advanced_security": False, + }, + ) + return temp, root, evidence + def test_exact_scanner_invocations_are_canonically_bound(self) -> None: result = self.validate(workflow_text()) self.assertEqual(result["schema"], 1) @@ -104,6 +149,61 @@ def test_changed_cargo_audit_execution_fails_closed(self) -> None: ): self.validate(changed) + def test_cargo_deny_proof_must_match_executed_invocation(self) -> None: + temp, root, evidence = self.proof_fixture() + try: + path = evidence / VERIFIED.BASE.EVIDENCE_FILES["cargo_deny"] + proof = json.loads(path.read_text(encoding="utf-8")) + proof["checks"] = ["advisories"] + write_json(path, proof) + with self.assertRaisesRegex( + VERIFIED.BASE.AssuranceError, + "cargo-deny proof does not match the executed action invocation", + ): + VERIFIED.validate_scanner_binding(root, evidence) + finally: + temp.cleanup() + + def test_zizmor_proof_must_match_executed_inputs(self) -> None: + temp, root, evidence = self.proof_fixture() + try: + path = evidence / VERIFIED.BASE.EVIDENCE_FILES["zizmor"] + proof = json.loads(path.read_text(encoding="utf-8")) + proof["min_severity"] = "low" + write_json(path, proof) + with self.assertRaisesRegex( + VERIFIED.BASE.AssuranceError, + "zizmor proof does not match the executed action invocation", + ): + VERIFIED.validate_scanner_binding(root, evidence) + finally: + temp.cleanup() + + def test_cargo_audit_proof_must_match_executed_version(self) -> None: + temp, root, evidence = self.proof_fixture() + try: + path = evidence / VERIFIED.BASE.EVIDENCE_FILES["cargo_audit"] + proof = json.loads(path.read_text(encoding="utf-8")) + proof["cargo_audit_version"] = "9.9.9" + write_json(path, proof) + with self.assertRaisesRegex( + VERIFIED.BASE.AssuranceError, + "cargo-audit proof does not match the executed install/run contract", + ): + VERIFIED.validate_scanner_binding(root, evidence) + finally: + temp.cleanup() + + def test_scanner_proofs_match_exact_executed_contract(self) -> None: + temp, root, evidence = self.proof_fixture() + try: + contract = VERIFIED.validate_scanner_binding(root, evidence) + self.assertEqual(contract["cargo_deny"]["uses"], VALIDATE.CARGO_DENY_USES) + self.assertEqual(contract["zizmor"]["uses"], VALIDATE.ZIZMOR_USES) + self.assertEqual(contract["cargo_audit"]["version"], "0.22.2") + finally: + temp.cleanup() + if __name__ == "__main__": unittest.main() From 66b27d68703a6dea165a704b88869a8de21482c8 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 14:18:56 +0300 Subject: [PATCH 53/62] fix(af01): recognize exact inline scanner commands --- .github/scripts/validate_af01_scanner_invocations.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.github/scripts/validate_af01_scanner_invocations.py b/.github/scripts/validate_af01_scanner_invocations.py index 2d7a3909..548bda61 100644 --- a/.github/scripts/validate_af01_scanner_invocations.py +++ b/.github/scripts/validate_af01_scanner_invocations.py @@ -132,7 +132,8 @@ def _find_action(lines: list[str], repository: str) -> tuple[str, dict[str, str] def _require_exact_shell_line(lines: list[str], command: str, label: str) -> None: - count = sum(1 for line in lines if line.strip() == command) + accepted = {command, f"run: {command}"} + count = sum(1 for line in lines if line.strip() in accepted) if count != 1: raise ScannerContractError(f"{label} exact command must appear once, found {count}") From 78e25fb71928e247e0661f4517f6dfae2a1a40cf Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 14:40:36 +0300 Subject: [PATCH 54/62] test(af01): reject skipped scanner steps --- .../validate_af01_scanner_invocations.py | 215 ++++++++++++++---- 1 file changed, 177 insertions(+), 38 deletions(-) diff --git a/.github/scripts/validate_af01_scanner_invocations.py b/.github/scripts/validate_af01_scanner_invocations.py index 548bda61..19e53ca5 100644 --- a/.github/scripts/validate_af01_scanner_invocations.py +++ b/.github/scripts/validate_af01_scanner_invocations.py @@ -34,6 +34,16 @@ CARGO_AUDIT_INSTALL = "cargo install cargo-audit --version 0.22.2 --locked" CARGO_AUDIT_RUN = 'cargo audit --file Cargo.lock --json > "$AF01_EVIDENCE_DIR/cargo-audit.json"' FULL_SHA = re.compile(r"^[0-9a-f]{40}$") +STATIC_KEY_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_-]*$") +AUDIT_SHELL_SEQUENCE = ( + "set -euo pipefail", + "set +e", + CARGO_AUDIT_RUN, + "audit_status=$?", + "set -e", + 'python3 - "$audit_status" <<\'PY\'', + 'test "$audit_status" -eq 0', +) class ScannerContractError(ValueError): @@ -53,42 +63,92 @@ def _scalar(value: str) -> str: return value -def _step_bounds(lines: list[str], uses_index: int) -> tuple[int, int, int]: - uses_indent = _indent(lines[uses_index]) - step_indent = uses_indent - 2 - start = uses_index - for index in range(uses_index, -1, -1): +def _step_bounds(lines: list[str], anchor_index: int) -> tuple[int, int, int]: + anchor_indent = _indent(lines[anchor_index]) + start = -1 + step_indent = -1 + for index in range(anchor_index, -1, -1): line = lines[index] - if _indent(line) == step_indent and line.lstrip().startswith("- "): + indent = _indent(line) + if line.strip() and indent < anchor_indent and line.lstrip().startswith("- "): + start = index + step_indent = indent + break + if index == anchor_index and line.lstrip().startswith("- "): start = index + step_indent = indent break - else: - raise ScannerContractError("scanner uses entry is not inside a static workflow step") + if start < 0: + raise ScannerContractError("scanner command is not inside a static workflow step") end = len(lines) for index in range(start + 1, len(lines)): line = lines[index] - if line.strip() and _indent(line) < step_indent: + if not line.strip(): + continue + indent = _indent(line) + if indent < step_indent: end = index break - if _indent(line) == step_indent and line.lstrip().startswith("- "): + if indent == step_indent and line.lstrip().startswith("- "): end = index break return start, end, step_indent -def _step_inputs(lines: list[str], uses_index: int) -> dict[str, str]: - start, end, step_indent = _step_bounds(lines, uses_index) - with_indexes = [ - index - for index in range(start + 1, end) - if _indent(lines[index]) == step_indent + 2 and lines[index].strip() == "with:" - ] - if len(with_indexes) != 1: - raise ScannerContractError("scanner step must contain exactly one static with mapping") +def _step_fields( + lines: list[str], start: int, end: int, step_indent: int, label: str +) -> dict[str, tuple[int, str]]: + fields: dict[str, tuple[int, str]] = {} + for index in range(start, end): + line = lines[index] + if not line.strip() or line.lstrip().startswith("#"): + continue + indent = _indent(line) + if index == start: + raw = line.strip() + if not raw.startswith("- "): + raise ScannerContractError(f"{label} step does not start with a static list item") + raw = raw[2:] + elif indent == step_indent + 2: + raw = line.strip() + else: + continue + if ":" not in raw: + raise ScannerContractError(f"{label} step contains unsupported top-level syntax: {raw!r}") + key, value = raw.split(":", 1) + if not STATIC_KEY_RE.fullmatch(key): + raise ScannerContractError( + f"{label} step contains quoted or unsupported top-level key {key!r}" + ) + if key in fields: + raise ScannerContractError(f"{label} step contains duplicate top-level key {key!r}") + fields[key] = (index, _scalar(value)) + return fields + +def _require_step_shape( + lines: list[str], + anchor_index: int, + *, + label: str, + required_fields: set[str], +) -> tuple[int, int, int, dict[str, tuple[int, str]]]: + start, end, step_indent = _step_bounds(lines, anchor_index) + fields = _step_fields(lines, start, end, step_indent, label) + if set(fields) != required_fields: + extra = sorted(set(fields) - required_fields) + missing = sorted(required_fields - set(fields)) + raise ScannerContractError( + f"{label} step shape mismatch: missing={missing} unsupported={extra}" + ) + return start, end, step_indent, fields + + +def _step_inputs( + lines: list[str], start: int, end: int, step_indent: int, with_index: int, label: str +) -> dict[str, str]: result: dict[str, str] = {} - with_index = with_indexes[0] for index in range(with_index + 1, end): line = lines[index] if not line.strip() or line.lstrip().startswith("#"): @@ -97,20 +157,20 @@ def _step_inputs(lines: list[str], uses_index: int) -> dict[str, str]: if indent <= step_indent + 2: break if indent != step_indent + 4 or ":" not in line: - raise ScannerContractError("scanner with mapping contains unsupported nested syntax") + raise ScannerContractError(f"{label} with mapping contains unsupported nested syntax") key, raw = line.strip().split(":", 1) + if not STATIC_KEY_RE.fullmatch(key): + raise ScannerContractError(f"{label} input key is quoted or unsupported: {key!r}") if key in result: - raise ScannerContractError(f"scanner with mapping contains duplicate input {key!r}") + raise ScannerContractError(f"{label} with mapping contains duplicate input {key!r}") value = _scalar(raw) - if not value: - raise ScannerContractError(f"scanner input {key!r} is empty or dynamic") - if "${{" in value: - raise ScannerContractError(f"scanner input {key!r} is dynamic") + if not value or "${{" in value: + raise ScannerContractError(f"{label} input {key!r} is empty or dynamic") result[key] = value return result -def _find_action(lines: list[str], repository: str) -> tuple[str, dict[str, str]]: +def _find_action(lines: list[str], repository: str, label: str) -> tuple[str, dict[str, str]]: prefix = f"{repository}@" matches: list[tuple[int, str]] = [] for index, line in enumerate(lines): @@ -124,18 +184,85 @@ def _find_action(lines: list[str], repository: str) -> tuple[str, dict[str, str] raise ScannerContractError( f"expected exactly one {repository} scanner step, found {len(matches)}" ) - index, uses = matches[0] + uses_index, uses = matches[0] revision = uses.rsplit("@", 1)[1] if FULL_SHA.fullmatch(revision) is None: raise ScannerContractError(f"{repository} scanner is not pinned to a full commit SHA") - return uses, _step_inputs(lines, index) + + start, end, step_indent, fields = _require_step_shape( + lines, + uses_index, + label=label, + required_fields={"name", "uses", "with"}, + ) + if fields["uses"][0] != uses_index or fields["uses"][1] != uses: + raise ScannerContractError(f"{label} uses entry is not the exact top-level step field") + with_index, with_value = fields["with"] + if with_value: + raise ScannerContractError(f"{label} with field must be a block mapping") + return uses, _step_inputs(lines, start, end, step_indent, with_index, label) -def _require_exact_shell_line(lines: list[str], command: str, label: str) -> None: - accepted = {command, f"run: {command}"} - count = sum(1 for line in lines if line.strip() in accepted) - if count != 1: - raise ScannerContractError(f"{label} exact command must appear once, found {count}") +def _without_heredoc_bodies(lines: list[str]) -> list[str]: + kept: list[str] = [] + delimiter: str | None = None + for line in lines: + stripped = line.strip() + if delimiter is not None: + if stripped == delimiter: + delimiter = None + continue + kept.append(stripped) + matched = re.search(r"<<['\"]?([A-Za-z_][A-Za-z0-9_]*)['\"]?", stripped) + if matched is not None: + delimiter = matched.group(1) + if delimiter is not None: + raise ScannerContractError("cargo-audit run step contains an unterminated heredoc") + return [line for line in kept if line] + + +def _find_run_step( + lines: list[str], + command: str, + *, + label: str, + block: bool, +) -> tuple[int, int, int, dict[str, tuple[int, str]]]: + anchors = [ + index + for index, line in enumerate(lines) + if line.strip() in {command, f"run: {command}"} + ] + if len(anchors) != 1: + raise ScannerContractError(f"{label} exact command must appear once, found {len(anchors)}") + anchor = anchors[0] + start, end, step_indent, fields = _require_step_shape( + lines, + anchor, + label=label, + required_fields={"name", "run"}, + ) + run_index, run_value = fields["run"] + if block: + if run_value not in {"|", "|-", "|+", ">", ">-", ">+"}: + raise ScannerContractError(f"{label} must use a static block run script") + body: list[str] = [] + for index in range(run_index + 1, end): + line = lines[index] + if not line.strip(): + body.append("") + continue + if _indent(line) <= step_indent + 2: + break + body.append(line[step_indent + 4 :]) + shell_lines = tuple(_without_heredoc_bodies(body)) + if shell_lines != AUDIT_SHELL_SEQUENCE: + raise ScannerContractError( + f"{label} run script does not match the exact fail-closed audit sequence" + ) + elif run_value != command: + raise ScannerContractError(f"{label} must be the exact inline run command") + return start, end, step_indent, fields def validate_workflow(path: Path) -> dict[str, Any]: @@ -144,8 +271,10 @@ def validate_workflow(path: Path) -> dict[str, Any]: except (OSError, UnicodeError) as error: raise ScannerContractError(f"cannot read assurance workflow: {error}") from error - cargo_deny_uses, cargo_deny_inputs = _find_action(lines, "EmbarkStudios/cargo-deny-action") - zizmor_uses, zizmor_inputs = _find_action(lines, "zizmorcore/zizmor-action") + cargo_deny_uses, cargo_deny_inputs = _find_action( + lines, "EmbarkStudios/cargo-deny-action", "cargo-deny" + ) + zizmor_uses, zizmor_inputs = _find_action(lines, "zizmorcore/zizmor-action", "zizmor") if cargo_deny_uses != CARGO_DENY_USES: raise ScannerContractError("cargo-deny executed action commit does not match assurance identity") @@ -156,8 +285,18 @@ def validate_workflow(path: Path) -> dict[str, Any]: if zizmor_inputs != ZIZMOR_INPUTS: raise ScannerContractError("zizmor executed inputs do not match assurance policy") - _require_exact_shell_line(lines, CARGO_AUDIT_INSTALL, "cargo-audit install") - _require_exact_shell_line(lines, CARGO_AUDIT_RUN, "cargo-audit execution") + _find_run_step( + lines, + CARGO_AUDIT_INSTALL, + label="cargo-audit install", + block=False, + ) + _find_run_step( + lines, + CARGO_AUDIT_RUN, + label="cargo-audit execution", + block=True, + ) contract = { "cargo_audit": { From e42bfd16848c12f5cead2b93f0899b548a682de1 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 14:41:08 +0300 Subject: [PATCH 55/62] test(af01): cover scanner skip bypasses --- .../test_validate_af01_scanner_invocations.py | 81 +++++++++++++++++++ 1 file changed, 81 insertions(+) diff --git a/.github/scripts/test_validate_af01_scanner_invocations.py b/.github/scripts/test_validate_af01_scanner_invocations.py index 80f42777..6baf5845 100644 --- a/.github/scripts/test_validate_af01_scanner_invocations.py +++ b/.github/scripts/test_validate_af01_scanner_invocations.py @@ -46,7 +46,14 @@ def workflow_text() -> str: - name: audit run: | set -euo pipefail + set +e {VALIDATE.CARGO_AUDIT_RUN} + audit_status=$? + set -e + python3 - "$audit_status" <<'PY' + print("fixture") + PY + test "$audit_status" -eq 0 - name: zizmor uses: {VALIDATE.ZIZMOR_USES} with: @@ -149,6 +156,80 @@ def test_changed_cargo_audit_execution_fails_closed(self) -> None: ): self.validate(changed) + def test_required_scanner_step_condition_is_rejected(self) -> None: + cases = { + "cargo-deny": ( + " - name: cargo deny\n", + " - name: cargo deny\n if: false\n", + ), + "cargo-audit-install": ( + " - name: install audit\n", + " - name: install audit\n if: false\n", + ), + "cargo-audit-execution": ( + " - name: audit\n", + " - name: audit\n if: false\n", + ), + "zizmor": ( + " - name: zizmor\n", + " - name: zizmor\n if: false\n", + ), + } + for label, (old, new) in cases.items(): + with self.subTest(label=label): + with self.assertRaisesRegex( + VALIDATE.ScannerContractError, "step shape mismatch" + ): + self.validate(workflow_text().replace(old, new, 1)) + + def test_required_scanner_continue_on_error_is_rejected(self) -> None: + cases = { + "cargo-deny": ( + " - name: cargo deny\n", + " - name: cargo deny\n continue-on-error: true\n", + ), + "cargo-audit-install": ( + " - name: install audit\n", + " - name: install audit\n continue-on-error: true\n", + ), + "cargo-audit-execution": ( + " - name: audit\n", + " - name: audit\n continue-on-error: true\n", + ), + "zizmor": ( + " - name: zizmor\n", + " - name: zizmor\n continue-on-error: true\n", + ), + } + for label, (old, new) in cases.items(): + with self.subTest(label=label): + with self.assertRaisesRegex( + VALIDATE.ScannerContractError, "step shape mismatch" + ): + self.validate(workflow_text().replace(old, new, 1)) + + def test_cargo_audit_shell_control_drift_is_rejected(self) -> None: + changed = workflow_text().replace( + " set +e\n", + " set +e\n if false; then\n", + 1, + ) + with self.assertRaisesRegex( + VALIDATE.ScannerContractError, "exact fail-closed audit sequence" + ): + self.validate(changed) + + def test_quoted_required_scanner_step_key_is_rejected(self) -> None: + changed = workflow_text().replace( + " uses: " + VALIDATE.CARGO_DENY_USES, + ' "uses": ' + VALIDATE.CARGO_DENY_USES, + 1, + ) + with self.assertRaisesRegex( + VALIDATE.ScannerContractError, "expected exactly one" + ): + self.validate(changed) + def test_cargo_deny_proof_must_match_executed_invocation(self) -> None: temp, root, evidence = self.proof_fixture() try: From 0491b964a6a8844bb9b356fac9a24914c7cf16a0 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 14:41:47 +0300 Subject: [PATCH 56/62] test(af01): reject quoted required-check bypasses --- .../scripts/test_required_check_topology.py | 172 +++++++++++++++--- 1 file changed, 145 insertions(+), 27 deletions(-) diff --git a/.github/scripts/test_required_check_topology.py b/.github/scripts/test_required_check_topology.py index 4d88e84d..08681adf 100644 --- a/.github/scripts/test_required_check_topology.py +++ b/.github/scripts/test_required_check_topology.py @@ -2,6 +2,7 @@ from __future__ import annotations import json +import re import tempfile import unittest from pathlib import Path @@ -10,6 +11,10 @@ CONFIG = ROOT / ".github" / "required-checks.json" WORKFLOWS = ROOT / ".github" / "workflows" GITHUB_ACTIONS_INTEGRATION_ID = 15368 +JOB_ID_RE = re.compile(r"^ (?P[A-Za-z_][A-Za-z0-9_-]*):(?:\s*#.*)?$") +JOB_FIELD_RE = re.compile( + r"^ (?P[A-Za-z_][A-Za-z0-9_-]*):(?P.*)$" +) def display_path(path: Path) -> str: @@ -19,11 +24,22 @@ def display_path(path: Path) -> str: return str(path) +def _scalar(value: str) -> str: + value = value.strip() + if " #" in value: + value = value.split(" #", 1)[0].rstrip() + if len(value) >= 2 and value[0] == value[-1] and value[0] in {"'", '"'}: + value = value[1:-1] + return value + + def pull_request_children(lines: list[str]) -> list[str]: try: start = lines.index(" pull_request:") except ValueError as error: - raise AssertionError("required-check workflow must use a mapping-form pull_request trigger") from error + raise AssertionError( + "required-check workflow must use a mapping-form pull_request trigger" + ) from error children: list[str] = [] for line in lines[start + 1 :]: if line and not line.startswith(" "): @@ -35,11 +51,35 @@ def pull_request_children(lines: list[str]) -> list[str]: return children -def job_range(lines: list[str], job: str) -> tuple[int, int]: +def _jobs_index(lines: list[str]) -> int: try: - jobs_index = lines.index("jobs:") + return lines.index("jobs:") except ValueError as error: raise AssertionError("required-check workflow has no jobs mapping") from error + + +def static_job_ids(lines: list[str]) -> list[str]: + jobs_index = _jobs_index(lines) + job_ids: list[str] = [] + for line in lines[jobs_index + 1 :]: + if line and not line.startswith(" "): + break + if not line.strip() or line.lstrip().startswith("#"): + continue + if line.startswith(" ") and not line.startswith(" "): + matched = JOB_ID_RE.fullmatch(line) + if matched is None: + raise AssertionError( + "workflow jobs mapping contains quoted, dynamic, or unsupported job key syntax" + ) + job_ids.append(matched.group("key")) + if len(set(job_ids)) != len(job_ids): + raise AssertionError("workflow jobs mapping contains duplicate static job ids") + return job_ids + + +def job_range(lines: list[str], job: str) -> tuple[int, int]: + jobs_index = _jobs_index(lines) target = f" {job}:" try: start = lines.index(target, jobs_index + 1) @@ -57,6 +97,29 @@ def job_range(lines: list[str], job: str) -> tuple[int, int]: return start, end +def job_top_level_fields( + lines: list[str], start: int, end: int, *, path: Path | None = None, job: str = "" +) -> dict[str, str]: + fields: dict[str, str] = {} + for line in lines[start + 1 : end]: + if not line.strip() or line.lstrip().startswith("#"): + continue + if line.startswith(" ") and not line.startswith(" "): + matched = JOB_FIELD_RE.fullmatch(line) + if matched is None: + location = ( + f"{display_path(path)} job {job!r}" if path is not None else "workflow job" + ) + raise AssertionError( + f"{location} contains quoted, dynamic, or unsupported top-level job field syntax" + ) + key = matched.group("key") + if key in fields: + raise AssertionError(f"workflow job has duplicate top-level field {key!r}") + fields[key] = matched.group("value").strip() + return fields + + def assert_universal_required_job(path: Path, job: str) -> None: lines = path.read_text(encoding="utf-8").splitlines() children = pull_request_children(lines) @@ -64,12 +127,21 @@ def assert_universal_required_job(path: Path, job: str) -> None: raise AssertionError( f"{display_path(path)} pull_request trigger is filtered or narrowed: {children!r}" ) + static_job_ids(lines) start, end = job_range(lines, job) - for line in lines[start + 1 : end]: - if line.startswith(" if:"): - raise AssertionError( - f"{display_path(path)} job {job!r} has a job-level conditional and is not universally terminal" - ) + fields = job_top_level_fields(lines, start, end, path=path, job=job) + if "if" in fields: + raise AssertionError( + f"{display_path(path)} job {job!r} has a job-level conditional and is not universally terminal" + ) + if "needs" in fields: + raise AssertionError( + f"{display_path(path)} job {job!r} has job dependencies and is not independently universally terminal" + ) + if "continue-on-error" in fields: + raise AssertionError( + f"{display_path(path)} job {job!r} tolerates failure and is not a valid required check" + ) def workflow_paths(root: Path = WORKFLOWS) -> list[Path]: @@ -80,15 +152,14 @@ def workflow_paths(root: Path = WORKFLOWS) -> list[Path]: ) -def literal_job_name(lines: list[str], start: int, end: int) -> str | None: - names = [line[9:].strip() for line in lines[start + 1 : end] if line.startswith(" name:")] - if len(names) > 1: - raise AssertionError("workflow job has multiple top-level name fields") - if not names: +def literal_job_name( + lines: list[str], start: int, end: int, *, path: Path | None = None, job: str = "" +) -> str | None: + fields = job_top_level_fields(lines, start, end, path=path, job=job) + raw_value = fields.get("name") + if raw_value is None: return None - value = names[0] - if len(value) >= 2 and value[0] == value[-1] and value[0] in {"'", '"'}: - value = value[1:-1] + value = _scalar(raw_value) if not value: raise AssertionError("workflow job has an empty name") if "${{" in value: @@ -100,20 +171,13 @@ def literal_job_name(lines: list[str], start: int, end: int) -> str | None: def job_check_contexts(path: Path) -> list[tuple[str, str]]: lines = path.read_text(encoding="utf-8").splitlines() - try: - jobs_index = lines.index("jobs:") - except ValueError: + if "jobs:" not in lines: return [] - job_ids: list[str] = [] - for line in lines[jobs_index + 1 :]: - if line and not line.startswith(" "): - break - if line.startswith(" ") and not line.startswith(" ") and line.rstrip().endswith(":"): - job_ids.append(line.strip()[:-1]) + job_ids = static_job_ids(lines) result: list[tuple[str, str]] = [] for job_id in job_ids: start, end = job_range(lines, job_id) - explicit_name = literal_job_name(lines, start, end) + explicit_name = literal_job_name(lines, start, end, path=path, job=job_id) result.append((job_id, explicit_name or job_id)) return result @@ -204,6 +268,18 @@ def test_counterexample_job_level_if_is_rejected(self) -> None: with self.assertRaisesRegex(AssertionError, "job-level conditional"): assert_universal_required_job(path, "gate") + def test_counterexample_quoted_job_level_if_is_rejected(self) -> None: + with tempfile.TemporaryDirectory() as directory: + path = Path(directory) / "workflow.yml" + path.write_text( + 'name: example\non:\n pull_request:\njobs:\n gate:\n "if": false\n runs-on: ubuntu-24.04\n', + encoding="utf-8", + ) + with self.assertRaisesRegex( + AssertionError, "quoted, dynamic, or unsupported top-level job field syntax" + ): + assert_universal_required_job(path, "gate") + def test_counterexample_duplicate_or_named_spoof_context_is_detected(self) -> None: with tempfile.TemporaryDirectory() as directory: root = Path(directory) @@ -225,6 +301,20 @@ def test_counterexample_duplicate_or_named_spoof_context_is_detected(self) -> No ], ) + def test_counterexample_named_spoof_with_yaml_comment_is_detected(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + (root / "authoritative.yml").write_text( + "name: authoritative\njobs:\n rust:\n runs-on: ubuntu-24.04\n", + encoding="utf-8", + ) + (root / "spoof.yml").write_text( + "name: spoof\njobs:\n harmless-id:\n name: rust # emitted context is still rust\n runs-on: ubuntu-24.04\n", + encoding="utf-8", + ) + producers = required_context_producers(root, {"rust"}) + self.assertEqual(len(producers["rust"]), 2) + def test_counterexample_dynamic_job_name_is_rejected(self) -> None: with tempfile.TemporaryDirectory() as directory: root = Path(directory) @@ -233,7 +323,35 @@ def test_counterexample_dynamic_job_name_is_rejected(self) -> None: "name: dynamic\njobs:\n harmless-id:\n name: ${{ github.event.pull_request.title }}\n runs-on: ubuntu-24.04\n", encoding="utf-8", ) - with self.assertRaisesRegex(AssertionError, "dynamic workflow job names are forbidden"): + with self.assertRaisesRegex( + AssertionError, "dynamic workflow job names are forbidden" + ): + job_check_contexts(path) + + def test_counterexample_quoted_dynamic_job_name_is_rejected(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + path = root / "quoted-name.yml" + path.write_text( + 'name: dynamic\njobs:\n harmless-id:\n "name": ${{ \'rust\' }}\n runs-on: ubuntu-24.04\n', + encoding="utf-8", + ) + with self.assertRaisesRegex( + AssertionError, "quoted, dynamic, or unsupported top-level job field syntax" + ): + job_check_contexts(path) + + def test_counterexample_quoted_job_id_is_rejected(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + path = root / "quoted-job.yml" + path.write_text( + 'name: dynamic\njobs:\n "rust":\n runs-on: ubuntu-24.04\n', + encoding="utf-8", + ) + with self.assertRaisesRegex( + AssertionError, "quoted, dynamic, or unsupported job key syntax" + ): job_check_contexts(path) From 53f874236b3a68962f4ad342d601b38be46b394f Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 14:50:56 +0300 Subject: [PATCH 57/62] security(af01): establish workflow code ownership --- .github/CODEOWNERS | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 .github/CODEOWNERS diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS new file mode 100644 index 00000000..c0829756 --- /dev/null +++ b/.github/CODEOWNERS @@ -0,0 +1,4 @@ +# AF-01 trusted workflow/governance ownership boundary. +# GitHub evaluates CODEOWNERS from the pull request base branch, so a PR cannot +# weaken these ownership requirements by editing this file in the same change. +/.github/ @TheHalfMoon From b107c85656fc5b8930577ccc1872f54b3b7ce25b Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 14:51:12 +0300 Subject: [PATCH 58/62] security(af01): require code owner review --- .github/main-ruleset.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/main-ruleset.json b/.github/main-ruleset.json index 16cdd235..f2606b09 100644 --- a/.github/main-ruleset.json +++ b/.github/main-ruleset.json @@ -25,7 +25,7 @@ "merge" ], "dismiss_stale_reviews_on_push": true, - "require_code_owner_review": false, + "require_code_owner_review": true, "require_last_push_approval": true, "required_approving_review_count": 1, "required_review_thread_resolution": true From e3990431aa19817443df833945fcbb1dc58a51d8 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 14:51:39 +0300 Subject: [PATCH 59/62] test(af01): require code owner governance --- .github/scripts/test_main_ruleset_contract.py | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/.github/scripts/test_main_ruleset_contract.py b/.github/scripts/test_main_ruleset_contract.py index dae313d3..2077f14e 100644 --- a/.github/scripts/test_main_ruleset_contract.py +++ b/.github/scripts/test_main_ruleset_contract.py @@ -28,7 +28,10 @@ def test_ruleset_matches_required_check_topology_and_governance(self) -> None: rules = ruleset.get("rules") self.assertIsInstance(rules, list) by_type = {rule.get("type"): rule for rule in rules if isinstance(rule, dict)} - self.assertEqual(set(by_type), {"deletion", "non_fast_forward", "pull_request", "required_status_checks"}) + self.assertEqual( + set(by_type), + {"deletion", "non_fast_forward", "pull_request", "required_status_checks"}, + ) self.assertEqual(by_type["deletion"], {"type": "deletion"}) self.assertEqual(by_type["non_fast_forward"], {"type": "non_fast_forward"}) @@ -38,7 +41,7 @@ def test_ruleset_matches_required_check_topology_and_governance(self) -> None: { "allowed_merge_methods": ["merge"], "dismiss_stale_reviews_on_push": True, - "require_code_owner_review": False, + "require_code_owner_review": True, "require_last_push_approval": True, "required_approving_review_count": 1, "required_review_thread_resolution": True, @@ -80,6 +83,17 @@ def test_every_required_check_is_bound_to_github_actions(self) -> None: self.assertEqual(set(check), {"context", "integration_id"}) self.assertEqual(check["integration_id"], GITHUB_ACTIONS_INTEGRATION_ID) + def test_workflow_changes_require_code_owner_review(self) -> None: + ruleset = json.loads(RULESET.read_text(encoding="utf-8")) + pull_request_rule = next( + rule for rule in ruleset["rules"] if rule["type"] == "pull_request" + ) + parameters = pull_request_rule["parameters"] + self.assertIs(parameters.get("require_code_owner_review"), True) + self.assertIs(parameters.get("dismiss_stale_reviews_on_push"), True) + self.assertIs(parameters.get("require_last_push_approval"), True) + self.assertGreaterEqual(parameters.get("required_approving_review_count", 0), 1) + if __name__ == "__main__": unittest.main() From f8ba95316601831a4954e772786142b3349b5070 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 14:51:53 +0300 Subject: [PATCH 60/62] test(af01): prove codeowner trust boundary --- .../scripts/test_codeowners_trust_boundary.py | 54 +++++++++++++++++++ 1 file changed, 54 insertions(+) create mode 100644 .github/scripts/test_codeowners_trust_boundary.py diff --git a/.github/scripts/test_codeowners_trust_boundary.py b/.github/scripts/test_codeowners_trust_boundary.py new file mode 100644 index 00000000..2c653667 --- /dev/null +++ b/.github/scripts/test_codeowners_trust_boundary.py @@ -0,0 +1,54 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import json +import unittest +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[2] +CODEOWNERS = ROOT / ".github" / "CODEOWNERS" +RULESET = ROOT / ".github" / "main-ruleset.json" +EXPECTED_OWNER = "@TheHalfMoon" +EXPECTED_PATTERN = "/.github/" + + +def ownership_entries() -> list[tuple[str, tuple[str, ...]]]: + entries: list[tuple[str, tuple[str, ...]]] = [] + for raw in CODEOWNERS.read_text(encoding="utf-8").splitlines(): + line = raw.strip() + if not line or line.startswith("#"): + continue + fields = line.split() + if len(fields) < 2: + raise AssertionError(f"malformed CODEOWNERS entry: {line!r}") + entries.append((fields[0], tuple(fields[1:]))) + return entries + + +class CodeownersTrustBoundaryTests(unittest.TestCase): + def test_github_security_surface_has_one_unambiguous_owner_boundary(self) -> None: + self.assertTrue(CODEOWNERS.is_file()) + self.assertEqual( + ownership_entries(), + [(EXPECTED_PATTERN, (EXPECTED_OWNER,))], + "the .github trust boundary must not contain narrower override patterns", + ) + + def test_ruleset_requires_code_owner_review_for_owned_workflow_changes(self) -> None: + ruleset = json.loads(RULESET.read_text(encoding="utf-8")) + pull_request = next(rule for rule in ruleset["rules"] if rule["type"] == "pull_request") + parameters = pull_request["parameters"] + self.assertIs(parameters.get("require_code_owner_review"), True) + self.assertIs(parameters.get("dismiss_stale_reviews_on_push"), True) + self.assertIs(parameters.get("require_last_push_approval"), True) + self.assertGreaterEqual(parameters.get("required_approving_review_count", 0), 1) + + def test_codeowners_file_is_inside_the_owned_boundary(self) -> None: + self.assertEqual(CODEOWNERS.relative_to(ROOT).as_posix(), ".github/CODEOWNERS") + pattern, owners = ownership_entries()[0] + self.assertEqual(pattern, EXPECTED_PATTERN) + self.assertEqual(owners, (EXPECTED_OWNER,)) + + +if __name__ == "__main__": + unittest.main() From 8955425270ee7931f98e9b3777d2e6ac8d4a341a Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 14:52:03 +0300 Subject: [PATCH 61/62] test(af01): include codeowner boundary in trust suite --- .../test_audit_workflow_trust_codeowners_boundary.py | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 .github/scripts/test_audit_workflow_trust_codeowners_boundary.py diff --git a/.github/scripts/test_audit_workflow_trust_codeowners_boundary.py b/.github/scripts/test_audit_workflow_trust_codeowners_boundary.py new file mode 100644 index 00000000..9fe71369 --- /dev/null +++ b/.github/scripts/test_audit_workflow_trust_codeowners_boundary.py @@ -0,0 +1,8 @@ +#!/usr/bin/env python3 +"""Expose CODEOWNERS trust-boundary regressions through the universal AF-01 trust suite.""" + +from __future__ import annotations + +from test_codeowners_trust_boundary import CodeownersTrustBoundaryTests + +__all__ = ["CodeownersTrustBoundaryTests"] From c82ef6e6f137805074cc5e0c453d47e0d2799839 Mon Sep 17 00:00:00 2001 From: "Abdulaziz M. Shehri" Date: Thu, 27 Aug 2026 15:33:30 +0300 Subject: [PATCH 62/62] fix(af01): separate review bypass from assurance rules --- .github/main-review-ruleset.json | 35 +++++++ .github/main-ruleset.json | 13 --- .../scripts/test_codeowners_trust_boundary.py | 23 ++++- .github/scripts/test_main_ruleset_contract.py | 92 ++++++++++-------- .../stack-c-governance-layering.md | 93 +++++++++++++++++++ 5 files changed, 200 insertions(+), 56 deletions(-) create mode 100644 .github/main-review-ruleset.json create mode 100644 specs/015-af-01-trusted-development-baseline/stack-c-governance-layering.md diff --git a/.github/main-review-ruleset.json b/.github/main-review-ruleset.json new file mode 100644 index 00000000..41a085ed --- /dev/null +++ b/.github/main-review-ruleset.json @@ -0,0 +1,35 @@ +{ + "name": "commandF main review governance", + "target": "branch", + "enforcement": "active", + "bypass_actors": [ + { + "actor_id": 5, + "actor_type": "RepositoryRole", + "bypass_mode": "pull_request" + } + ], + "conditions": { + "ref_name": { + "include": [ + "refs/heads/main" + ], + "exclude": [] + } + }, + "rules": [ + { + "type": "pull_request", + "parameters": { + "allowed_merge_methods": [ + "merge" + ], + "dismiss_stale_reviews_on_push": true, + "require_code_owner_review": true, + "require_last_push_approval": true, + "required_approving_review_count": 1, + "required_review_thread_resolution": true + } + } + ] +} diff --git a/.github/main-ruleset.json b/.github/main-ruleset.json index f2606b09..5a18137c 100644 --- a/.github/main-ruleset.json +++ b/.github/main-ruleset.json @@ -18,19 +18,6 @@ { "type": "non_fast_forward" }, - { - "type": "pull_request", - "parameters": { - "allowed_merge_methods": [ - "merge" - ], - "dismiss_stale_reviews_on_push": true, - "require_code_owner_review": true, - "require_last_push_approval": true, - "required_approving_review_count": 1, - "required_review_thread_resolution": true - } - }, { "type": "required_status_checks", "parameters": { diff --git a/.github/scripts/test_codeowners_trust_boundary.py b/.github/scripts/test_codeowners_trust_boundary.py index 2c653667..0e6dc2be 100644 --- a/.github/scripts/test_codeowners_trust_boundary.py +++ b/.github/scripts/test_codeowners_trust_boundary.py @@ -7,9 +7,11 @@ ROOT = Path(__file__).resolve().parents[2] CODEOWNERS = ROOT / ".github" / "CODEOWNERS" -RULESET = ROOT / ".github" / "main-ruleset.json" +ASSURANCE_RULESET = ROOT / ".github" / "main-ruleset.json" +REVIEW_RULESET = ROOT / ".github" / "main-review-ruleset.json" EXPECTED_OWNER = "@TheHalfMoon" EXPECTED_PATTERN = "/.github/" +ADMIN_REPOSITORY_ROLE_ID = 5 def ownership_entries() -> list[tuple[str, tuple[str, ...]]]: @@ -34,8 +36,8 @@ def test_github_security_surface_has_one_unambiguous_owner_boundary(self) -> Non "the .github trust boundary must not contain narrower override patterns", ) - def test_ruleset_requires_code_owner_review_for_owned_workflow_changes(self) -> None: - ruleset = json.loads(RULESET.read_text(encoding="utf-8")) + def test_review_ruleset_requires_code_owner_review(self) -> None: + ruleset = json.loads(REVIEW_RULESET.read_text(encoding="utf-8")) pull_request = next(rule for rule in ruleset["rules"] if rule["type"] == "pull_request") parameters = pull_request["parameters"] self.assertIs(parameters.get("require_code_owner_review"), True) @@ -43,6 +45,21 @@ def test_ruleset_requires_code_owner_review_for_owned_workflow_changes(self) -> self.assertIs(parameters.get("require_last_push_approval"), True) self.assertGreaterEqual(parameters.get("required_approving_review_count", 0), 1) + def test_admin_escape_hatch_is_pr_only_and_cannot_bypass_assurance(self) -> None: + assurance = json.loads(ASSURANCE_RULESET.read_text(encoding="utf-8")) + review = json.loads(REVIEW_RULESET.read_text(encoding="utf-8")) + self.assertEqual(assurance.get("bypass_actors"), []) + self.assertEqual( + review.get("bypass_actors"), + [ + { + "actor_id": ADMIN_REPOSITORY_ROLE_ID, + "actor_type": "RepositoryRole", + "bypass_mode": "pull_request", + } + ], + ) + def test_codeowners_file_is_inside_the_owned_boundary(self) -> None: self.assertEqual(CODEOWNERS.relative_to(ROOT).as_posix(), ".github/CODEOWNERS") pattern, owners = ownership_entries()[0] diff --git a/.github/scripts/test_main_ruleset_contract.py b/.github/scripts/test_main_ruleset_contract.py index 2077f14e..0a6f47c6 100644 --- a/.github/scripts/test_main_ruleset_contract.py +++ b/.github/scripts/test_main_ruleset_contract.py @@ -7,57 +7,38 @@ ROOT = Path(__file__).resolve().parents[2] TOPOLOGY = ROOT / ".github" / "required-checks.json" -RULESET = ROOT / ".github" / "main-ruleset.json" +ASSURANCE_RULESET = ROOT / ".github" / "main-ruleset.json" +REVIEW_RULESET = ROOT / ".github" / "main-review-ruleset.json" GITHUB_ACTIONS_INTEGRATION_ID = 15368 +ADMIN_REPOSITORY_ROLE_ID = 5 +MAIN_CONDITIONS = {"ref_name": {"include": ["refs/heads/main"], "exclude": []}} class MainRulesetContractTests(unittest.TestCase): - def test_ruleset_matches_required_check_topology_and_governance(self) -> None: + def test_assurance_ruleset_is_unbypassable_and_matches_required_checks(self) -> None: topology = json.loads(TOPOLOGY.read_text(encoding="utf-8")) - ruleset = json.loads(RULESET.read_text(encoding="utf-8")) + ruleset = json.loads(ASSURANCE_RULESET.read_text(encoding="utf-8")) self.assertEqual(ruleset.get("name"), "commandF main assurance") self.assertEqual(ruleset.get("target"), "branch") self.assertEqual(ruleset.get("enforcement"), "active") self.assertEqual(ruleset.get("bypass_actors"), []) - self.assertEqual( - ruleset.get("conditions"), - {"ref_name": {"include": ["refs/heads/main"], "exclude": []}}, - ) + self.assertEqual(ruleset.get("conditions"), MAIN_CONDITIONS) rules = ruleset.get("rules") self.assertIsInstance(rules, list) by_type = {rule.get("type"): rule for rule in rules if isinstance(rule, dict)} - self.assertEqual( - set(by_type), - {"deletion", "non_fast_forward", "pull_request", "required_status_checks"}, - ) + self.assertEqual(set(by_type), {"deletion", "non_fast_forward", "required_status_checks"}) self.assertEqual(by_type["deletion"], {"type": "deletion"}) self.assertEqual(by_type["non_fast_forward"], {"type": "non_fast_forward"}) - pull_request = by_type["pull_request"].get("parameters") - self.assertEqual( - pull_request, - { - "allowed_merge_methods": ["merge"], - "dismiss_stale_reviews_on_push": True, - "require_code_owner_review": True, - "require_last_push_approval": True, - "required_approving_review_count": 1, - "required_review_thread_resolution": True, - }, - ) - required = by_type["required_status_checks"].get("parameters") self.assertIsInstance(required, dict) self.assertFalse(required.get("do_not_enforce_on_create")) self.assertTrue(required.get("strict_required_status_checks_policy")) actual = required.get("required_status_checks", []) expected = [ - { - "context": item["context"], - "integration_id": item["integration_id"], - } + {"context": item["context"], "integration_id": item["integration_id"]} for item in topology["checks"] ] self.assertEqual(actual, expected) @@ -70,12 +51,54 @@ def test_ruleset_matches_required_check_topology_and_governance(self) -> None: ], ) + def test_review_ruleset_requires_review_with_admin_pr_only_escape_hatch(self) -> None: + ruleset = json.loads(REVIEW_RULESET.read_text(encoding="utf-8")) + self.assertEqual(ruleset.get("name"), "commandF main review governance") + self.assertEqual(ruleset.get("target"), "branch") + self.assertEqual(ruleset.get("enforcement"), "active") + self.assertEqual(ruleset.get("conditions"), MAIN_CONDITIONS) + self.assertEqual( + ruleset.get("bypass_actors"), + [ + { + "actor_id": ADMIN_REPOSITORY_ROLE_ID, + "actor_type": "RepositoryRole", + "bypass_mode": "pull_request", + } + ], + ) + + rules = ruleset.get("rules") + self.assertEqual(len(rules), 1) + self.assertEqual(rules[0].get("type"), "pull_request") + self.assertEqual( + rules[0].get("parameters"), + { + "allowed_merge_methods": ["merge"], + "dismiss_stale_reviews_on_push": True, + "require_code_owner_review": True, + "require_last_push_approval": True, + "required_approving_review_count": 1, + "required_review_thread_resolution": True, + }, + ) + + def test_admin_review_bypass_cannot_bypass_assurance_rules(self) -> None: + assurance = json.loads(ASSURANCE_RULESET.read_text(encoding="utf-8")) + review = json.loads(REVIEW_RULESET.read_text(encoding="utf-8")) + self.assertEqual(assurance["bypass_actors"], []) + self.assertNotIn("pull_request", {rule["type"] for rule in assurance["rules"]}) + self.assertEqual({rule["type"] for rule in review["rules"]}, {"pull_request"}) + self.assertNotIn("required_status_checks", {rule["type"] for rule in review["rules"]}) + self.assertNotIn("deletion", {rule["type"] for rule in review["rules"]}) + self.assertNotIn("non_fast_forward", {rule["type"] for rule in review["rules"]}) + def test_every_required_check_is_bound_to_github_actions(self) -> None: topology = json.loads(TOPOLOGY.read_text(encoding="utf-8")) for check in topology["checks"]: self.assertEqual(check.get("integration_id"), GITHUB_ACTIONS_INTEGRATION_ID) - ruleset = json.loads(RULESET.read_text(encoding="utf-8")) + ruleset = json.loads(ASSURANCE_RULESET.read_text(encoding="utf-8")) required_rule = next( rule for rule in ruleset["rules"] if rule["type"] == "required_status_checks" ) @@ -83,17 +106,6 @@ def test_every_required_check_is_bound_to_github_actions(self) -> None: self.assertEqual(set(check), {"context", "integration_id"}) self.assertEqual(check["integration_id"], GITHUB_ACTIONS_INTEGRATION_ID) - def test_workflow_changes_require_code_owner_review(self) -> None: - ruleset = json.loads(RULESET.read_text(encoding="utf-8")) - pull_request_rule = next( - rule for rule in ruleset["rules"] if rule["type"] == "pull_request" - ) - parameters = pull_request_rule["parameters"] - self.assertIs(parameters.get("require_code_owner_review"), True) - self.assertIs(parameters.get("dismiss_stale_reviews_on_push"), True) - self.assertIs(parameters.get("require_last_push_approval"), True) - self.assertGreaterEqual(parameters.get("required_approving_review_count", 0), 1) - if __name__ == "__main__": unittest.main() diff --git a/specs/015-af-01-trusted-development-baseline/stack-c-governance-layering.md b/specs/015-af-01-trusted-development-baseline/stack-c-governance-layering.md new file mode 100644 index 00000000..d65376d3 --- /dev/null +++ b/specs/015-af-01-trusted-development-baseline/stack-c-governance-layering.md @@ -0,0 +1,93 @@ +# AF-01 Stack C Governance Layering + +Status: REVIEW_CANDIDATE + +## Problem + +The AF-01 workflow trust boundary requires `.github/` changes to receive Code Owner review so an untrusted pull request cannot weaken a required-check workflow while preserving the same GitHub Actions check name and integration identity. + +A single personal-repository administrator is also the only valid Code Owner currently available. Requiring that same account to approve its own pull request creates a maintenance deadlock because pull request authors cannot approve their own changes. + +Adding an invented or unverified second Code Owner is not acceptable. Removing Code Owner review would reopen the workflow-self-modification risk. + +## Layered ruleset design + +Stack C therefore separates branch governance into two independently active repository rulesets targeting `refs/heads/main`. + +### `commandF main assurance` + +Source: `.github/main-ruleset.json` + +This ruleset has **no bypass actors** and contains only controls that must remain unbypassable for every actor, including repository administrators: + +- branch deletion blocked; +- non-fast-forward updates blocked; +- strict required status checks; +- exact integration-bound required checks `rust`, `assurance-proof`, and `scorecard`. + +### `commandF main review governance` + +Source: `.github/main-review-ruleset.json` + +This ruleset contains only the pull-request review policy: + +- merge commits only; +- one approval; +- Code Owner review for `.github/` through `.github/CODEOWNERS`; +- stale approvals dismissed on push; +- latest-push approval required; +- review threads resolved. + +The sole escape hatch is: + +```json +{ + "actor_id": 5, + "actor_type": "RepositoryRole", + "bypass_mode": "pull_request" +} +``` + +GitHub repository-role actor ID `5` is the repository administrator role. `pull_request` bypass mode still requires the administrator to use a pull request; it does not authorize a direct push. + +Because the administrator bypass exists only in the review ruleset, it cannot bypass the separate assurance ruleset. Required checks, deletion protection, and non-fast-forward protection remain subject to a ruleset whose `bypass_actors` list is empty. + +## Trust model + +For an untrusted contributor changing `.github/`, the base-branch `CODEOWNERS` rule requires `@TheHalfMoon` review. The contributor cannot make its head version of `CODEOWNERS` govern that same pull request. + +For a pull request authored by the repository administrator, the administrator is the explicit human governance trust root for this user-owned repository. The PR-only review bypass avoids an impossible self-approval while preserving the audit trail. The administrator still cannot bypass the independent required-check/deletion/non-fast-forward ruleset. + +This design does not protect against a fully compromised repository administrator account. Repository administration itself is an out-of-band authority capable of editing repository rules, so claiming protection against total administrator compromise would be false assurance. + +## GitHub semantics relied upon + +Primary GitHub documentation states that: + +- multiple rulesets can target the same branch and their applicable rules are aggregated; +- repository administrators can be granted ruleset bypass; +- `For pull requests only` / `bypass_mode: pull_request` requires the actor to open a pull request instead of pushing directly; +- repository role actor ID `5` represents the administrator role in the Rulesets REST model. + +References: + +- https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/about-rulesets#about-rule-layering +- https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/creating-rulesets-for-a-repository#granting-bypass-permissions-for-your-branch-or-tag-ruleset +- https://docs.github.com/en/rest/repos/rules + +## Regression requirements + +The repository tests must fail closed unless: + +1. the assurance ruleset has no bypass actors; +2. the assurance ruleset contains deletion, non-fast-forward, and exact required-check rules only; +3. the review ruleset contains the pull-request rule only; +4. the review bypass is exactly repository administrator role `5` in `pull_request` mode; +5. the review ruleset keeps Code Owner review, one approval, stale-review dismissal, latest-push approval, and thread resolution; +6. no required status check, deletion rule, or non-fast-forward rule moves into the bypassable review ruleset. + +## Bootstrap and live-enforcement boundary + +These checked-in files remain configuration intent until T038 applies both rulesets through an authorized GitHub administrator path. T039 must read both live rulesets back and prove their target, enforcement state, rules, required-check integrations, and bypass separation. T040 must then verify the negative governance properties. + +No live enforcement, T038/T039/T040 completion, Stack C merge, or `AF-01=CLOSED_CANONICAL` may be inferred from this document or from the checked-in JSON alone.