Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 75 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
name: ci

on:
pull_request:
push:
branches: [main]

permissions:
contents: read

jobs:
test:
strategy:
fail-fast: false
matrix:
python-version: ["3.11", "3.12"]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: ${{ matrix.python-version }}
- name: Compile package and scripts
run: python -m compileall -q src tests skills
- name: Run tests
run: make test
- name: Install build backend
if: matrix.python-version == '3.12'
run: python -m pip install "setuptools>=68"
- name: Build wheel
if: matrix.python-version == '3.12'
run: python -m pip wheel . --no-deps --no-build-isolation --wheel-dir dist

shadow-validation:
runs-on: ubuntu-latest
env:
PATCHPROOF_IMAGE: python:3.12.10-slim@sha256:fd95fa221297a88e1cf49c55ec1828edd7c5a428187e67b5d1805692d11588db
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: "3.12"
- name: Pull pinned validation image
run: docker pull "$PATCHPROOF_IMAGE"
- name: Run the real four-phase proof loop
run: |
run_dir="$(mktemp -d)"
PYTHONPATH=src python -m patchproof propose \
--repo fixtures/calculator \
--diff fixtures/division-by-zero.diff \
--out "$run_dir/proposal.json"
proposal_hash="$(python -c 'import json,sys; print(json.load(open(sys.argv[1]))["proposal_hash"])' "$run_dir/proposal.json")"
PYTHONPATH=src python -m patchproof approve \
--proposal "$run_dir/proposal.json" \
--proposal-hash "$proposal_hash" \
--run-id ci-shadow \
--approved-by github-actions \
--out "$run_dir/approval.json"
PYTHONPATH=src python -m patchproof validate \
--repo fixtures/calculator \
--proposal "$run_dir/proposal.json" \
--approval "$run_dir/approval.json" \
--reproduction-tests fixtures/reproduction \
--hidden-tests fixtures/hidden \
--audit-dir /tmp/patchproof-audit \
--docker-image "$PATCHPROOF_IMAGE" \
> "$run_dir/result.json"
python -c 'import json,sys; result=json.load(open(sys.argv[1])); assert result["proof_grade"] is True' "$run_dir/result.json"
- name: Upload proof artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: patchproof-shadow-validation
path: /tmp/patchproof-audit/
if-no-files-found: error
retention-days: 14
24 changes: 24 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# 版本记录 / Changelog

所有重要变更记录于此。`1.0.0` 之前保持 Alpha,次版本可能调整实验性接口。

All notable changes are recorded here. The project remains alpha before `1.0.0`, and minor releases may change experimental interfaces.

## 0.1.0 — 2026-08-03

### 新增 / Added

- 独立的 `patchproof` Python 包与 CLI,无第三方运行时依赖。
Independent `patchproof` Python package and CLI with no third-party runtime dependency.
- 仓库快照绑定的 Unified Diff Proposal 与精确哈希审批。
Repository-snapshot-bound unified-diff proposals and exact-hash approvals.
- 一次性副本中的纯 Python 补丁应用与文件级前后哈希。
Pure-Python patch application in disposable copies with per-file before/after hashes.
- Docker 与显式不安全本地 Runner 协议。
Docker and explicitly unsafe local runner protocols.
- 补丁前复现、补丁后复现、完整回归和隐藏测试四阶段验证。
Four-phase fail-before, pass-after, regression, and hidden-test validation.
- 内容寻址 JSON 回执与中英双语 Markdown 报告。
Content-addressed JSON receipts and bilingual Markdown reports.
- `verifiable-agent-audit` 和 `agent-eval-builder` Codex Skills。
`verifiable-agent-audit` and `agent-eval-builder` Codex Skills.
31 changes: 31 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
PYTHON ?= python3
RUN_DIR ?= /tmp/patchproof-demo
IMAGE ?= python:3.12.10-slim@sha256:fd95fa221297a88e1cf49c55ec1828edd7c5a428187e67b5d1805692d11588db

.PHONY: test demo clean

test:
PYTHONPATH=src $(PYTHON) -m unittest discover -s tests -v

demo:
PYTHONPATH=src $(PYTHON) -m patchproof propose \
--repo fixtures/calculator \
--diff fixtures/division-by-zero.diff \
--out $(RUN_DIR)/proposal.json
PYTHONPATH=src $(PYTHON) -m patchproof approve \
--proposal $(RUN_DIR)/proposal.json \
--proposal-hash "$$(PYTHONPATH=src $(PYTHON) -c 'import json; print(json.load(open("$(RUN_DIR)/proposal.json"))["proposal_hash"])')" \
--run-id demo \
--approved-by local-demo \
--out $(RUN_DIR)/approval.json
PYTHONPATH=src $(PYTHON) -m patchproof validate \
--repo fixtures/calculator \
--proposal $(RUN_DIR)/proposal.json \
--approval $(RUN_DIR)/approval.json \
--reproduction-tests fixtures/reproduction \
--hidden-tests fixtures/hidden \
--audit-dir $(RUN_DIR)/audit \
--docker-image $(IMAGE)

clean:
$(PYTHON) -c 'import shutil; shutil.rmtree("build", ignore_errors=True); shutil.rmtree("dist", ignore_errors=True)'
215 changes: 215 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,215 @@
# PatchProof

[![CI](https://github.com/eatdrop/patchproof/actions/workflows/ci.yml/badge.svg)](https://github.com/eatdrop/patchproof/actions/workflows/ci.yml)
[![Python](https://img.shields.io/badge/Python-3.11%20%7C%203.12-3776AB)](https://www.python.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
[![Status: Alpha](https://img.shields.io/badge/status-alpha-orange.svg)](CHANGELOG.md)

PatchProof 是面向 AI 生成补丁的证据级验证工具。它把“补丁看起来合理”转换为可审计协议:仓库快照绑定、精确哈希审批、补丁前失败复现、补丁后复现、完整回归、外置隐藏测试和内容寻址回执。

PatchProof is an evidence-grade validator for AI-generated patches. It turns “the patch looks plausible” into an auditable protocol: repository-snapshot binding, exact-hash approval, fail-before reproduction, pass-after reproduction, full regression, external hidden tests, and a content-addressed receipt.

> PatchProof 验证外部候选补丁;它本身不是补丁生成 Agent。
>
> PatchProof validates externally supplied candidate patches; it is not a patch-generating Agent.

## 为什么需要 / Why It Exists

普通测试结果无法回答这些问题:测试是否真的执行?补丁前是否能复现故障?审批对象是否就是后来执行的补丁?测试是否被候选补丁修改?真实仓库是否发生了副作用?

A normal test result does not answer whether tests actually ran, the bug reproduced before the patch, the approved object is the executed patch, the candidate altered its grader, or the real repository changed as a side effect.

PatchProof 的闭环如下:

PatchProof uses this proof loop:

```text
Read-only repository / 只读仓库
├── snapshot SHA-256 / 仓库快照
Unified Diff + snapshot → Proposal Hash
补丁 + 快照 → 提案哈希
Exact hash approval / 精确哈希审批
Disposable baseline + patched copies / 一次性基线与补丁副本
├── baseline reproduction must fail / 补丁前复现必须失败
├── patched reproduction must pass / 补丁后复现必须通过
├── full regression must pass / 完整回归必须通过
└── external hidden tests must pass / 外置隐藏测试必须通过
Content-addressed receipt + bilingual report
内容寻址回执 + 双语报告
```

## 安装 / Installation

当前版本为 Alpha,推荐从固定 Release 或源码安装。

The current release is alpha. Install from a pinned release or source revision.

```bash
python -m pip install .
patchproof --help
```

PatchProof 运行时仅使用 Python 标准库。默认隔离验证需要本机 Docker CLI 与可用守护进程。

PatchProof has no third-party runtime dependency. Default isolated validation requires a local Docker CLI and daemon.

## 快速开始 / Quick Start

### 1. 绑定候选补丁 / Bind a Candidate Patch

输出产物必须位于目标仓库外部。

Output artifacts must remain outside the target repository.

```bash
PYTHONPATH=src python -m patchproof propose \
--repo fixtures/calculator \
--diff fixtures/division-by-zero.diff \
--out /tmp/patchproof-demo/proposal.json
```

### 2. 精确审批 / Approve the Exact Proposal

审批必须重新提供完整 Proposal Hash;前缀、过期快照和其他提案都不会通过。

Approval requires the complete Proposal Hash. Prefixes, stale snapshots, and different proposals are rejected.

```bash
PROPOSAL_HASH="$(python -c 'import json; print(json.load(open("/tmp/patchproof-demo/proposal.json"))["proposal_hash"])')"

PYTHONPATH=src python -m patchproof approve \
--proposal /tmp/patchproof-demo/proposal.json \
--proposal-hash "$PROPOSAL_HASH" \
--run-id demo \
--approved-by local-reviewer \
--out /tmp/patchproof-demo/approval.json
```

`approved_by` 当前是声明式审计元数据,不是经过认证的身份。

`approved_by` is declared audit metadata, not an authenticated identity.

### 3. 在 Docker 中验证 / Validate in Docker

镜像必须包含精确 SHA-256 摘要;PatchProof 不会在验证时拉取镜像。

The image must include an exact SHA-256 digest. PatchProof never pulls an image during validation.

```bash
IMAGE='python:3.12.10-slim@sha256:fd95fa221297a88e1cf49c55ec1828edd7c5a428187e67b5d1805692d11588db'
docker pull "$IMAGE"

PYTHONPATH=src python -m patchproof validate \
--repo fixtures/calculator \
--proposal /tmp/patchproof-demo/proposal.json \
--approval /tmp/patchproof-demo/approval.json \
--reproduction-tests fixtures/reproduction \
--hidden-tests fixtures/hidden \
--audit-dir /tmp/patchproof-demo/audit \
--docker-image "$IMAGE"
```

Docker Runner 默认使用:

The Docker runner uses:

- 固定摘要镜像、`--pull=never` / digest-pinned image and `--pull=never`;
- `--network=none`、只读根和只读仓库挂载 / disabled network, read-only root and repository mount;
- UID/GID `65534`、`--cap-drop=ALL`、`no-new-privileges` / non-root user, dropped capabilities, and no-new-privileges;
- PID、内存、CPU、输出和超时限制 / PID, memory, CPU, output, and timeout limits;
- 超时后按随机容器名强制清理 / forced cleanup by randomized container name after timeout.

## 本地可信模式 / Trusted Local Mode

`--unsafe-local` 仅用于可信夹具和开发调试。即使四阶段测试通过,其回执也会明确标记 `isolated=false`、`proof_grade=false`。

`--unsafe-local` is only for trusted fixtures and development. Even when all four phases pass, the receipt explicitly records `isolated=false` and `proof_grade=false`.

```bash
make test
make demo IMAGE="$IMAGE"
```

## Python API / Python API

```python
from pathlib import Path

from patchproof import DockerRunner, PatchApproval, PatchProposal
from patchproof.repository import ReadOnlyRepository
from patchproof.validator import validate_patch

repository = ReadOnlyRepository("target-repo")
proposal = PatchProposal.create(
unified_diff=Path("candidate.diff").read_text(),
base_snapshot=repository.snapshot().digest,
)
approval = PatchApproval.create(
run_id="review-001",
proposal=proposal,
approved_by="reviewer",
supplied_proposal_hash=proposal.proposal_hash,
)
receipt = validate_patch(
repository=repository,
proposal=proposal,
approval=approval,
reproduction_tests=Path("external/reproduction"),
hidden_tests=Path("external/hidden"),
runner=DockerRunner(image="python:3.12-slim@sha256:<digest>"),
)
assert receipt.proof_grade
```

## 可复用 Codex Skills / Reusable Codex Skills

仓库包含两个经过官方结构校验并可独立使用的 Skill:

The repository includes two independently usable, structurally validated Skills:

- [`verifiable-agent-audit`](skills/verifiable-agent-audit/SKILL.md):审计 Agent 权限、信任边界、审批、隔离、恢复、预算、评测和副作用。
Audits Agent authority, trust boundaries, approvals, isolation, recovery, budgets, evaluations, and side effects.
- [`agent-eval-builder`](skills/agent-eval-builder/SKILL.md):构建固定 Manifest、独立 Grader、正确拒答、失败分类、有效分母和 CI。
Builds pinned manifests, independent graders, correct-abstention cases, failure taxonomies, valid denominators, and CI gates.

## 验证现状 / Verification Status

- 70 项标准库自动化测试覆盖 Diff、路径、符号链接、快照、审批、补丁应用、Runner、回执、CLI 和 Skill 脚本。
70 standard-library tests cover diffs, paths, symlinks, snapshots, approvals, patch application, runners, receipts, CLI behavior, and Skill scripts.
- Python 3.11/3.12 CI。
Python 3.11/3.12 CI.
- 独立 Docker 作业运行真实四阶段闭环并上传审计产物。
An independent Docker job runs the real four-phase loop and uploads its audit artifacts.

## 安全边界 / Security Boundary

PatchProof 不提供生产级恶意多租户沙箱,也不认证审批人身份,不安装目标项目依赖,不支持任意测试命令。v0.1 仅支持有界 UTF-8 文本文件、Unified Diff 和 Python `unittest` 发现模式。

PatchProof is not a production hostile multi-tenant sandbox, does not authenticate approver identity, does not install target dependencies, and does not accept arbitrary test commands. v0.1 supports bounded UTF-8 text files, unified diffs, and Python `unittest` discovery only.

详见 [SECURITY.md](SECURITY.md) 和 [架构文档](docs/architecture.md)。

See [SECURITY.md](SECURITY.md) and the [architecture document](docs/architecture.md).

真实复用记录见 [IssueLens 案例](docs/issuelens-case-study.md)。

See the [IssueLens case study](docs/issuelens-case-study.md) for a real reuse record.

## 来源与独立性 / Origin and Independence

PatchProof 从 IssueLens 项目的验证协议中抽取,但拥有独立命名空间、哈希域、公共 API、CLI、测试、CI、文档和 Release 生命周期。IssueLens 是其设计来源和首个公开参考场景;当前版本不要求 IssueLens 依赖 PatchProof。

PatchProof was extracted from the IssueLens validation protocol but owns an independent namespace, hash domains, public API, CLI, tests, CI, documentation, and release lifecycle. IssueLens is its design origin and first public reference scenario; the current release does not require IssueLens to depend on PatchProof.

## 许可证 / License

[MIT](LICENSE)
Loading
Loading