bench: five-proxy egress benchmark + upstream connection pooling - #7
Draft
yourbuddyconner wants to merge 21 commits into
Draft
bench: five-proxy egress benchmark + upstream connection pooling#7yourbuddyconner wants to merge 21 commits into
yourbuddyconner wants to merge 21 commits into
Conversation
When the tunnel listener MITMs a TLS connection and the client negotiates HTTP/2 (hematite generated cert advertises h2 via ALPN), req.headers() does not include a Host header entry (carried only in the :authority pseudo-header). The upstream send path uses hyper http1::Builder which requires a Host header in the HeaderMap (the path-form URI has no authority). Without Host, nginx rejects the request with 400 Bad Request. Inject a Host header from summary.host when one is absent, so h2 to h1.1 translation produces a well-formed HTTP/1.1 request.
Adds the final three files for the bench-vs-iron-proxy harness: - run.sh: orchestrator (all|perf|footprint|conformance; QUICK=1; runsuite writes suite-status.txt inline to avoid bash subshell issue; exit non-zero if any suite ERRORED) - render-report.sh: renders results/*.json into results/REPORT.md; null-safe fmt_num guards printf against missing data when a suite errored; all four sections (env, perf, footprint, conformance) with verbatim configs and relative-comparison caveat - README.md: usage, knobs, layout, interpretation notes Hardware signing token unavailable; committed with -c commit.gpgsign=false.
Fixes the sustained-load failure found by tests/bench: dialing a fresh TCP+TLS upstream connection per request exhausts ephemeral ports (EADDRNOTAVAIL -> mass 502s) at a few hundred rps. A new pool module keeps idle HTTP/1.1 senders keyed by (host, port, scheme), capped at 8/key and 100 total with a 30s idle timeout (below common upstream keep-alive timeouts to shrink the stale-reuse window; the spec's Part 07 §4 quality-of-implementation allowance). Invariants preserved: pool::acquire is the only path to an upstream sender and consumes the pipeline AllowProof (INV-2); the guard is re-checked against the pooled connection's peer address on every checkout, and the pool dies with its Runtime on config reload, so a new deny CIDR can never be bypassed by a pooled connection. Upgraded (101) connections and non-forwarded responses are never checked in. No post-send retry in v1: a stale reuse surfaces as a rare 502, bounded by the checkout ready/closed gate and the 30s idle cap. New e2e tests: three sequential proxied requests reach the upstream on one connection; a guard-denied peer is unreachable through the pool. dial::connect_upstream now returns the dialed peer address, and dials report a 'reused' metric label.
Matches Go http.Transport semantics: when a send on a REUSED pooled connection fails, redial and replay the identical request exactly once — but only when the replay is provably safe: the body is fully buffered (under max_request_body_bytes) and the method is GET/HEAD/OPTIONS/TRACE. A failed POST or streamed body is never replayed; the upstream may have executed or consumed it. INV-2 stays type-enforced: a reused PooledSender retains the request's AllowProof (it was not spent on a dial), and pool::redial redeems it for exactly one fresh connection; a fresh sender carries no proof, so a fresh dial can never be retried. Guard denial on the redial audits as a guard rejection, same as the first dial. New e2e test: an upstream that kills each kept-alive connection ~5ms after responding cannot surface a 502 to idempotent clients.
New fourth suite modeling what these proxies actually front, instead of uniform 1KB GETs. tests/bench/agentbench (standalone Rust crate, outside the hematite workspace) is one binary with two modes: 'serve' is a mock LLM API on llm.test (SSE chat streaming at a fixed token cadence + tool endpoint), 'attack' replays agent sessions with open-loop Poisson arrivals — an 8-32KB chat POST carrying the proxy token (secret swap on every call) whose SSE response is consumed chunk by chunk, a burst of 6 parallel tool GETs, and a 10% chance of a denied-host request — through baseline/hematite/iron, steady phase then 10x burst. Reported per phase and class: p50/p99, TTFT, worst inter-chunk stall, denied-request refusal latency, error counts, and phase-windowed proxy CPU/RSS (docker stats samples now timestamped). run.sh gains the 'agent' suite; both proxies' allowlists and secret rules gain llm.test; the shared leaf gains the llm.test SAN (gen-certs regenerates when missing). End-of-stream is detected by chunk count, not EOF, since proxies may keep the client leg alive after a chunked response.
De-singles iron-proxy: the harness now compares five egress proxies. Targets and capability flags live in targets.sh; all four suites loop over the list. New services: squid 6 (ubuntu 24.04 squid-openssl, ssl-bump with the shared CA, allowlist ACL), mitmproxy 11.1.3 (regular mode + published 12-line allowlist addon), smokescreen v0.0.4 (built from source; non-MITM CONNECT class with its own private-range guard — fixture subnet explicitly allowed, link-local stays denied). Conformance scenarios a proxy has no feature for score N/A via the capability map (secret swap / containment: hematite+iron only; header strip: hematite; resolved-IP guard: hematite+iron+smokescreen — an in-fixture IMDS result for squid/mitmproxy would be a network artifact, not policy). Scenario FAILs are now recorded as scorecard findings instead of failing the harness run: the first shakedown immediately caught two real ones (squid ssl-bump cannot proxy a bumped WebSocket upgrade — 502 ERR_INVALID_RESP; mitmproxy buffers SSE by default — TTFT equals the whole stream duration). agentbench: denied-class probe now recognizes bump-style refusal (CONNECT 200 to peek SNI, then inner-request 403) as a refusal; only an inner 2xx counts as reachable. Report renderer generalized to dynamic target columns; footprint handles binaryless targets (mitmproxy).
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
A reproducible benchmark harness (
tests/bench/) comparing hematite against four other egress proxies — iron-proxy 0.49.0, Squid 6 (ssl-bump), mitmproxy 11.1.3, and smokescreen v0.0.4 — plus the two production fixes the benchmark immediately surfaced. Results and tradeoffs are written up intests/bench/COMPARISON.md.Harness (one command:
cd tests/bench && ./run.sh)Four suites over a docker-compose fixture (shared CA, semantically equivalent configs printed verbatim in the report, pinned versions):
tests/bench/agentbench, deliberately outside the workspace): open-loop Poisson arrivals of 8–32KB chat POSTs (secret swap where supported) with SSE responses consumed chunk-by-chunk, 6-way parallel tool-call bursts, 10% denied-host requests; steady phase then 10× burst. Reports TTFT, worst inter-chunk stall, per-class p50/p99, phase-windowed CPU/RSS.Production changes (hematite crates)
hematite-proxy/src/pool.rs, Part 07 §4 QoI): the bench's first sustained-load run exposed ephemeral-port exhaustion (fresh TCP+TLS per request → mass 502s at ~660 rps). The pool keeps idle HTTP/1.1 senders keyed by (host, port, scheme), 8/key, 100 total, 30s idle. INV-2 stays type-enforced:pool::acquireis the only path to an upstream sender and consumes theAllowProof; the guard is re-checked against the pooled peer on every checkout; the pool dies with itsRuntimeon reload. After: 100% success at fixture-ceiling throughput (was ~30% success at 1/8th the rate).pool::redialredeems it for exactly one fresh dial, so a fresh connection is unretryable by type.http.rs): h2-negotiated requests through the CONNECT tunnel carried the host only in:authority, producing Host-less (RFC-invalid) upstream requests that nginx 400s. Found by the bench; invisible to the acceptance suite (nothing there negotiates h2).Headline results (M-series laptop, relative numbers — see COMPARISON.md for caveats)
Tests
e2e_pool.rs): sequential requests reuse one upstream connection; guard-denied peers unreachable through the pool; stale-connection retry survives an upstream that kills each kept-alive connection ~5ms after responding.-D warningsclean, all tests green. agentbench has its own unit tests.QUICK=1 ./run.sh(~10 min) is the harness's self-test; the full run (~35 min) produced the committed COMPARISON.md numbers.Known follow-ups (not in this PR)
tls.rs) — root cause the Host fix works around; consider h1-only or config.commit.gpgsignneeds re-enabling locally.