不要只相信 Agent 说“测试通过”；证明它修复了正确的问题。
Do not merely trust an agent saying “tests passed.” Prove it fixed the right problem.
面向 Coding Agent 平台、评测与 CI/安全工程师的证据级补丁验证器：绑定准确 Diff 与仓库快照，执行补丁前后测试、完整回归和外置隐藏测试，再生成可重验回执。
An evidence-grade patch validator for coding-agent platforms, evaluations, and CI/security engineers: bind the exact diff to a repository snapshot, run fail-before/pass-after checks, full regression, and external hidden tests, then emit a re-verifiable receipt.
图中四阶段结果来自仓库内真实夹具和
make demo-local；proof_grade=true由断网、只读、非 root 的 Docker CI 作业验证。The four phase results come from the repository's real fixture and
make demo-local;proof_grade=trueis established by the network-disabled, read-only, non-root Docker CI job.
**真实第三方证据 / Real third-party evidence：**PatchProof 现在固定并验证 Bottle 的真实 HTTP 条件请求修复：修复前失败、修复后通过、359 项上游回归通过、独立边界测试通过，目标 checkout 保持不变。查看 Bottle 集成案例。
PatchProof now pins and validates a real Bottle HTTP-precondition fix: fail before, pass after, 359 upstream regressions passing, a distinct edge case passing, and an unchanged target checkout. See the Bottle integration case.
| 中文 | English | |
|---|---|---|
| 是什么 / What | 位于 Coding Agent 与合并操作之间的确定性验证层 | A deterministic validation layer between a coding agent and merge |
| 给谁 / Who | Agent 平台、AI 评测、CI 与安全工程师 | Agent-platform, AI-evaluation, CI, and security engineers |
| 解决什么 / Why | 防止“测试绿了，但没复现原问题、改了评分器或执行的不是审批过的补丁” | Prevent green checks that never reproduced the bug, changed the grader, or ran a patch different from the approved one |
PatchProof 验证外部候选补丁，本身不调用模型、不生成补丁，也不替代人工代码审查。
PatchProof validates externally supplied candidate patches. It does not call a model, generate patches, or replace human code review.
普通单次 CI 可以证明“当前工作树通过了某些命令”，却不一定证明“候选补丁修复了声明的问题”。PatchProof 把容易被忽略的条件变成强制协议。
A typical one-pass CI job can show that a current worktree passed commands, but it may not prove that the candidate patch fixed the claimed problem. PatchProof makes the missing conditions explicit and mandatory.
| 需要回答的问题 / Question | 普通单次 CI / Typical one-pass CI | PatchProof |
|---|---|---|
| 补丁前是否真实复现？ / Did the bug reproduce before the patch? | 通常未知 / Usually unknown | 基线复现必须失败 / Baseline reproduction must fail |
| 执行对象是否就是审批对象？ / Was the approved object executed? | 分支和文件名可变 / Branches and filenames can move | Diff + 仓库快照 + 完整哈希绑定 / Diff, snapshot, and full-hash binding |
| 候选补丁能否修改评分器？ / Can the candidate modify its grader? | 仓库内测试可能同时变化 / In-repo tests may change | Reproduction 与 Hidden Tests 外置复制 / Reproduction and hidden tests are copied externally |
| 真实仓库是否被副作用修改？ / Did validation mutate the real repository? | 取决于 Job 实现 / Depends on the job | 一次性副本执行并复查快照 / Disposable copies plus a post-run snapshot check |
| 结果能否离线重验？ / Can the result be rechecked offline? | 常见产物是日志 / Usually logs | 内容寻址 JSON 回执 / Content-addressed JSON receipt |
这一方向不是单纯追逐热点：SWE-bench Verified 使用 FAIL_TO_PASS 测试筛选可信案例，Codex 官方方案也把隔离容器与执行期断网作为重要边界。PatchProof 将这些原则缩小为可嵌入其他 Agent 系统的本地协议。
This direction is more than trend-chasing: SWE-bench Verified uses FAIL_TO_PASS tests to filter credible cases, while the official Codex design treats isolated containers and disabled runtime internet as important boundaries. PatchProof narrows those principles into a local protocol that other agent systems can embed.
- SWE-bench Verified：FAIL_TO_PASS / SWE-bench Verified: FAIL_TO_PASS
- Codex：隔离容器与执行期断网 / Codex: isolated container and disabled runtime internet
本地可信 Demo 只需要 Python 3.11+，不需要 Docker，也不会修改真实夹具仓库。
The trusted-local demo needs only Python 3.11+. It does not require Docker or modify the real fixture repository.
git clone https://github.com/eatdrop/patchproof.git
cd patchproof
make demo-local没有 make 时可直接运行脚本：
If make is unavailable, run the script directly:
# macOS / Linux
PYTHONPATH=src python3 scripts/run_demo.py
# Windows PowerShell
$env:PYTHONPATH="src"; python scripts/run_demo.py你会看到真实执行的四阶段结果：
You will see four phases that actually executed:
[PASS] fail before patch (1 test)
[PASS] pass after patch (1 test)
[PASS] full regression (1 test)
[PASS] external hidden tests (2 tests)
validation: PASSED
repository unchanged: YES
receipt integrity: VERIFIED
proof grade: NO (trusted-local; Docker required)
本地 Demo 故意显示 proof grade: NO：它证明判定逻辑可运行，但没有借用 Docker 隔离路径的安全声明。需要隔离证据时使用下方 Docker 流程，或查看持续通过的 Shadow Validation CI。
The local demo deliberately reports proof grade: NO: it proves the verdict logic runs without borrowing the Docker path's isolation claim. Use the Docker workflow below—or inspect the continuously passing shadow-validation CI—when isolated evidence is required.
适合：
Good fit:
- 构建 Coding Agent、自动修复或 Agent PR 流水线的工程师。 Engineers building coding agents, automated repair, or agent-to-PR pipelines.
- 需要 fail-to-pass、pass-to-pass 与独立隐藏评分的评测作者。 Evaluation authors who need fail-to-pass, pass-to-pass, and independent hidden grading.
- 想在合并前把候选 Diff、审批和执行证据绑定起来的 CI/安全团队。 CI and security teams that need candidate diffs, approvals, and execution evidence bound before merge.
- 研究 Agent 权限分离、正确拒答和可审计执行的学习者。 Practitioners studying agent authority separation, correct abstention, and auditable execution.
暂不适合：
Not yet a fit:
- 用作通用测试框架或普通 CI 的替代品。 Replacing general-purpose test frameworks or normal CI.
- 直接运行任意 shell、安装任意依赖或验证所有语言生态。 Running arbitrary shell commands, installing arbitrary dependencies, or covering every language ecosystem.
- 在强对抗、多租户生产环境中执行恶意代码。 Executing hostile code in a strongly adversarial multi-tenant production environment.
Read-only repository / 只读仓库
│
├── snapshot SHA-256 / 仓库快照
▼
Unified Diff + snapshot → Proposal Hash
补丁 + 快照 → 提案哈希
│
▼
Exact hash approval / 精确哈希审批
│
▼
Disposable baseline + patched copies / 一次性基线与补丁副本
│
├── baseline reproduction must fail / 补丁前复现必须失败
├── patched reproduction must pass / 补丁后复现必须通过
├── full regression must pass / 完整回归必须通过
└── external hidden tests must pass / 外置隐藏测试必须通过
│
▼
Content-addressed receipt + bilingual report
内容寻址回执 + 双语报告
任一阶段出现零测试、ImportError、超时、输出截断或基础设施失败，整体结果都会关闭失败。
Zero tests, ImportError, timeout, output truncation, or infrastructure failure in any phase fails closed.
展开生产形态命令 / Expand production-shaped commands
python -m pip install .
patchproof --helpPatchProof 运行时只使用 Python 标准库。隔离验证需要 Docker CLI 与可用守护进程。
PatchProof uses only the Python standard library at runtime. Isolated validation requires a Docker CLI and daemon.
输出产物必须放在目标仓库之外。
Output artifacts must remain outside the target repository.
mkdir -p /tmp/patchproof-demo
patchproof propose \
--repo fixtures/calculator \
--diff fixtures/division-by-zero.diff \
--out /tmp/patchproof-demo/proposal.jsonPROPOSAL_HASH="$(python -c 'import json; print(json.load(open("/tmp/patchproof-demo/proposal.json"))["proposal_hash"])')"
patchproof approve \
--proposal /tmp/patchproof-demo/proposal.json \
--proposal-hash "$PROPOSAL_HASH" \
--run-id demo \
--approved-by local-reviewer \
--out /tmp/patchproof-demo/approval.jsonapproved_by 是声明式审计元数据，不是认证身份。
approved_by is declared audit metadata, not authenticated identity.
镜像必须包含准确 SHA-256 摘要；PatchProof 不会在验证时自动拉取镜像。
The image must include an exact SHA-256 digest. PatchProof never pulls it during validation.
IMAGE='python:3.12.10-slim@sha256:fd95fa221297a88e1cf49c55ec1828edd7c5a428187e67b5d1805692d11588db'
docker pull "$IMAGE"
patchproof validate \
--repo fixtures/calculator \
--proposal /tmp/patchproof-demo/proposal.json \
--approval /tmp/patchproof-demo/approval.json \
--reproduction-tests fixtures/reproduction \
--hidden-tests fixtures/hidden \
--audit-dir /tmp/patchproof-demo/audit \
--docker-image "$IMAGE"Docker Runner 使用固定摘要镜像、--pull=never、--network=none、只读根与仓库挂载、非 root 用户、移除 Capabilities、no-new-privileges、PID/内存/CPU/输出限制和超时清理。
The Docker runner uses a digest-pinned image, --pull=never, --network=none, read-only root and repository mount, a non-root user, dropped capabilities, no-new-privileges, PID/memory/CPU/output limits, and timeout cleanup.
RECEIPT="$(find /tmp/patchproof-demo/audit -name 'validation-*.json' -print -quit)"
patchproof verify-receipt --receipt "$RECEIPT"该命令重新验证严格结构、阶段语义和内容哈希，并输出 integrity_valid、validation_success 与 proof_grade。它证明内容内部一致且未被静默修改，不认证是谁执行或签署了验证。
The command revalidates strict structure, phase semantics, and the content hash, then reports integrity_valid, validation_success, and proof_grade. It proves internal consistency and detects silent modification; it does not authenticate who executed or signed the validation.
- 80 项自动化测试 / 80 automated tests：覆盖 Diff、路径、符号链接、快照、审批、补丁应用、Runner、CLI、Demo、回执、真实集成 Manifest 与 Skill 脚本。 They cover diffs, paths, symlinks, snapshots, approvals, patch application, runners, CLI, demo behavior, receipts, the real-integration manifest, and Skill scripts.
- Python 3.11/3.12 CI：每次 PR 与
main推送均运行。 It runs on every pull request andmainpush. - 真实 Docker Shadow Validation / Real Docker shadow validation：在 GitHub Actions 中执行完整四阶段闭环并上传审计产物。 It executes the full four-phase loop in GitHub Actions and uploads audit artifacts.
- 第三方开源集成 / Third-party open-source integration：Bottle HTTP 条件请求案例绑定真实修复前提交与上游源码修复 Hunk，并运行 359 项项目原生回归测试。 The Bottle HTTP-precondition case binds a real pre-fix commit and the exact upstream source-fix hunk, then runs 359 project-native regression tests.
- 只读前向案例 / Read-only forward case：IssueLens 复用记录扫描 63 个真实项目文件并验证前后快照一致。 The IssueLens reuse record scans 63 real project files and verifies identical before/after snapshots.
- 可安装 Release / Installable release：GitHub Releases包含经过构建与隔离安装验证的 wheel。 The GitHub Releases include a wheel verified through build and isolated installation.
| 已实现并测试 / Implemented and tested | 明确不声称 / Explicitly not claimed |
|---|---|
| 有界 UTF-8 文本与 Unified Diff / Bounded UTF-8 text and unified diffs | 任意二进制、重命名或所有 Diff 方言 / Arbitrary binaries, renames, or every diff dialect |
固定 Python unittest discover，支持包式 discovery top-level / Fixed Python unittest discover with package top-level support |
pytest/tox/nox、依赖安装或任意命令 / pytest/tox/nox, dependency installation, or arbitrary commands |
| 单租户可信宿主上的 Docker 加固 / Docker hardening on a trusted single-tenant host | 生产级恶意多租户沙箱 / Production hostile multi-tenant sandbox |
| 哈希完整性与内容寻址 / Hash integrity and content addressing | 数字签名、可信时间戳或身份认证 / Digital signatures, trusted timestamps, or identity authentication |
| 独立协议、测试与真实前向审计 / Independent protocol, tests, and real forward audit | 已被大量团队采用或证明适用于所有 Agent / Broad adoption or proof for every agent system |
v0.1 的优先级是把边界做窄、做真、做可拒绝。第二个真实系统集成已经由 Bottle 案例补齐；下一阶段重点是外部用户反馈、更多独立案例，以及在不扩大任意命令权限的前提下评估 Runner 适配器。
The v0.1 priority is to keep the boundary narrow, truthful, and rejectable. The Bottle case now supplies the second real-system integration; the next priorities are external user feedback, more independent cases, and runner adapters that do not open arbitrary-command authority.
不会。它是模型无关的候选补丁验证层，可以接在 Codex、Claude Code、Copilot、自研 Agent 或人工补丁之后。
No. It is a model-agnostic candidate-patch validation layer that can follow Codex, Claude Code, Copilot, an in-house agent, or a human-authored patch.
如果测试在补丁前已经通过，补丁后通过不能证明它修复了目标问题。失败原因仍必须由独立测试准确描述；ImportError 和零测试不会被当成有效复现。
If a test already passed before the patch, passing afterward proves no repair. The independent test must still describe the intended failure; ImportError and zero-test runs are not accepted as reproduction.
本地模式执行可信夹具但不隔离。只有声明 isolated=true 且四阶段全部通过时，proof_grade 才为真。
Local mode runs trusted fixtures without isolation. proof_grade becomes true only when a runner declares isolated=true and all four phases pass.
PatchProof 使用 --pull=never 防止验证时镜像漂移。先手动 docker pull README 中的完整摘要镜像，再运行验证。
PatchProof uses --pull=never to prevent image drift during validation. Pull the full digest-pinned image shown above before running validation.
任意命令会显著扩大命令注入、依赖下载和宿主副作用面。v0.1 选择固定 unittest 协议；扩展 Runner 必须先定义同等严格的权限与证据边界。
Arbitrary commands greatly expand command-injection, dependency-download, and host-side-effect surfaces. v0.1 intentionally fixes the unittest protocol; future runner adapters must first define equally strict authority and evidence boundaries.
不等于。哈希能发现内容变化，但不能认证审批人或验证执行者。需要真实性或不可抵赖性时，应在外层加入签名、身份提供方和可信时间戳。
No. A hash detects content changes but does not authenticate the approver or validation runner. Add signatures, an identity provider, and trusted timestamps when authenticity or non-repudiation is required.
工具本身不调用云端 API；Docker Runner 默认断网并只读挂载一次性副本。但宿主、Docker 守护进程、固定镜像和外置测试仍属于信任边界，请先阅读 SECURITY.md。
The tool itself calls no cloud API; the Docker runner disables networking and mounts disposable copies read-only. The host, Docker daemon, pinned image, and external tests remain trust boundaries, so read SECURITY.md first.
from pathlib import Path
from patchproof import DockerRunner, PatchApproval, PatchProposal
from patchproof.repository import ReadOnlyRepository
from patchproof.validator import validate_patch
repository = ReadOnlyRepository("target-repo")
proposal = PatchProposal.create(
unified_diff=Path("candidate.diff").read_text(),
base_snapshot=repository.snapshot().digest,
)
approval = PatchApproval.create(
run_id="review-001",
proposal=proposal,
approved_by="reviewer",
supplied_proposal_hash=proposal.proposal_hash,
)
receipt = validate_patch(
repository=repository,
proposal=proposal,
approval=approval,
reproduction_tests=Path("external/reproduction"),
hidden_tests=Path("external/hidden"),
runner=DockerRunner(image="python:3.12-slim@sha256:<digest>"),
)
assert receipt.proof_grade仓库还包含两个经过官方结构校验、可独立使用的方法型 Skill：
The repository also includes two independently usable, structurally validated workflow Skills:
verifiable-agent-audit：审计 Agent 权限、信任边界、审批、隔离、恢复、预算、评测与副作用。 Audits agent authority, trust boundaries, approvals, isolation, recovery, budgets, evaluation, and side effects.agent-eval-builder：构建固定 Manifest、独立 Grader、正确拒答、有效分母、失败分类与 CI。 Builds pinned manifests, independent graders, correct-abstention cases, valid denominators, failure taxonomies, and CI gates.
- 架构与信任边界 / Architecture and trust boundaries
- 安全模型与限制 / Security model and limitations
- 工程文章：把不确定补丁放进确定性边界 / Engineering article: putting uncertain patches inside deterministic boundaries
- Bottle 真实第三方集成 / Real third-party Bottle integration
- IssueLens 真实复用案例 / Real IssueLens reuse case
- 版本记录 / Changelog
PatchProof 从 IssueLens 的验证协议中抽取，但拥有独立命名空间、哈希域、公共 API、CLI、测试、CI、文档和 Release 生命周期。它没有复制 AppExpo 或任何实习公司的内部代码、数据与业务实现。
PatchProof was extracted from the IssueLens validation protocol but owns an independent namespace, hash domains, public API, CLI, tests, CI, documentation, and release lifecycle. It contains no copied AppExpo or internship-company internal code, data, or business implementation.