Skip to content

Add pluggable coding-agent backends - #9

Draft
areshand wants to merge 3 commits into
mainfrom
codex/coding-agent-backends
Draft

Add pluggable coding-agent backends#9
areshand wants to merge 3 commits into
mainfrom
codex/coding-agent-backends

Conversation

@areshand

Copy link
Copy Markdown
Owner

Summary

  • extract Codex, Claude Code, and Qwen Code process construction into a typed Rust backend boundary
  • add a shared headless runner with argv/stdin prompt delivery, immutable attempt traces, final-result capture, timeout/cancellation, and capability preflight
  • route isolated role launches through the Rust runner while retaining launch.sh as the source-checkout compatibility entrypoint
  • remove obsolete provider-specific launch branches and keep Python evaluation code limited to packaging/submission and official scoring
  • add concise product requirements and refactoring design documents

Why

Provider command construction was mixed into workflow, permission, and tmux lifecycle code. That made new coding agents harder to add and made tracing/cancellation behavior provider-dependent. The new boundary keeps workflow authority and filesystem isolation in Rust while treating each coding agent as an existing external runtime.

The SWE image source digest also included Cargo's target/ directory, adding hundreds of megabytes of irrelevant build artifacts. The bake filter now excludes it and tests pin that provenance behavior.

Impact

  • role backends can be selected as codex, claude, or qwen
  • raw and normalized traces can be retained outside evaluation containers
  • Qwen Code is supported in headless mode; its model/provider configuration remains owned by Qwen Code
  • no PTY migration is required; existing interactive compatibility remains
  • launch.sh continues to locate/build and exec the Rust workflow for existing callers

Validation

  • cargo fmt --all -- --check
  • cargo clippy --all-targets -- -D warnings
  • cargo test — 27 passed
  • bash tests/run.sh — all Rust/Shell/Python integration and lifecycle suites passed
  • focused SWE provenance/import tests — 13 passed
  • SWE-Bench Pro first ten rows: 6/10 versus 5/10 baseline; all previously solved rows 1, 2, 3, 4, and 6 remained solved, and row 7 newly passed
  • external per-row traces retained for all ten rows

The opt-in live Qwen read/write smoke test is implemented but was not completed because no Qwen Code authentication is configured in the test environment. Offline fake-backend coverage passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant