Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 47 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,11 @@ crates/
phantom-tui/ # Ratatui terminal UI
phantom-agent/ # LD_PRELOAD dylib (Linux only, hooks libc send/recv)
tests/
proxy_node_integration.rs # Integration tests: Node.js proxy capture (HTTP + HTTPS)
proxy_node_integration.rs # Integration tests: Node.js proxy capture (HTTP + HTTPS)
proxy_php_integration.rs # Integration test: PHP curl via proxy backend (HTTP + HTTPS)
proxy_php_ldpreload_integration.rs # Integration test: PHP curl via ldpreload backend (Linux only)
apps/node-app/ # Test Node.js app (client.js, client-alts.js, proxy-preload.js)
apps/php-app/ # Test PHP app (client.php, curl extension, PHP 5.3-compatible)
integration/ # Shell-based integration tests
Cargo.toml # Workspace root + binary crate
plan.md # Japanese-language technical design document
Expand Down Expand Up @@ -66,14 +69,16 @@ make check # fmt + clippy + build + test (full CI locally)

| Flag | Default | Description |
|---|---|---|
| `-b, --backend <BACKEND>` | `proxy` | `proxy` (MITM, cross-platform) or `ldpreload` (Linux only) |
| `-b, --backend <BACKEND>` | `proxy` | `proxy` (MITM, cross-platform) or `ldpreload` (Linux only, HTTP + HTTPS) |
| `-o, --output <OUTPUT>` | `tui` | `tui` (interactive) or `jsonl` (stdout stream, auto-exits with child) |
| `-p, --port <PORT>` | `8080` | Proxy capture port |
| `--insecure` | off | Disable TLS verification for backend servers (self-signed certs) |
| `-d, --data-dir <DIR>` | `~/.local/share/phantom/data` | Storage directory |
| `--agent-lib <PATH>` | — | Path to `libphantom_agent.so` (ldpreload backend) |
| `-- <CMD>` | — | Command to spawn and trace automatically |

For any spawned command other than Node.js, phantom sets `HTTP_PROXY`/`HTTPS_PROXY` (and lowercase variants) and clears `NO_PROXY`/`no_proxy`, so libcurl-based clients (curl, PHP's curl extension, etc.) are proxied for both schemes without an inherited `no_proxy` exclusion list defeating capture. Node.js is excluded from this because its injected `proxy-preload.js` already handles HTTPS itself — setting `HTTPS_PROXY` there would make libraries like axios configure a second, conflicting proxy agent from the env var.

---

## Node.js Transparent Proxy Injection
Expand Down Expand Up @@ -104,6 +109,25 @@ When the command after `--` is `node` or `nodejs`, phantom automatically:

---

## PHP Transparent Proxy Injection (curl)

When the command after `--` is `php` (or a version-suffixed binary like `php8.2`), phantom automatically:

1. Exports the MITM CA certificate (generated fresh per run in `ProxyCaptureBackend`, see `crates/phantom-capture/src/proxy.rs`) to a PID-scoped temp PEM file.
2. Prepends `-d curl.cainfo=<tempfile>` to the PHP arguments.
3. Sets `HTTP_PROXY` / `HTTPS_PROXY` (and lowercase variants), and clears `NO_PROXY` / `no_proxy`.
4. Deletes the temp CA file after the child exits (`TempScript` RAII guard, same mechanism as Node/Java).

Unlike Node.js, PHP requires **no injected script**: libcurl (the library backing PHP's curl extension) already reads `HTTP_PROXY`/`HTTPS_PROXY` env vars natively when the application hasn't called `curl_setopt($ch, CURLOPT_PROXY, ...)` itself, so routing traffic through the proxy needs no code injection. The only gap is TLS trust — PHP has no way to monkey-patch `curl_setopt` calls from userland, so phantom cannot force-disable certificate verification the way it does for Node/Java. Instead it exports its own MITM CA and injects it via the `curl.cainfo` ini directive, which the curl extension uses as the default CA bundle whenever the application hasn't set `CURLOPT_CAINFO`/`CURLOPT_SSL_VERIFYPEER` itself.

**Known limitations:**
- Only the **curl extension** is covered. PHP streams (`file_get_contents`, `fopen` with `http://`/`https://` wrappers) and non-curl HTTP clients are out of scope.
- `curl.cainfo` requires **PHP ≥ 5.3.7** (added in that release). On PHP 5.3.0–5.3.6, HTTP capture still works (via `HTTP_PROXY`/`HTTPS_PROXY`) but HTTPS capture fails TLS verification unless the application itself disables it.
- If the application explicitly sets `CURLOPT_CAINFO` or `CURLOPT_SSL_VERIFYPEER`, phantom's injected `curl.cainfo` default is overridden and HTTPS capture may fail with a TLS error — the same class of limitation as Node's double-proxy guard.
- `--backend ldpreload` (Linux only) requires **no PHP-specific code at all**: it hooks libc `send`/`recv` and OpenSSL `SSL_write`/`SSL_read` at a language-agnostic level, so it captures PHP's curl-based HTTP and HTTPS traffic (no MITM cert involved) exactly like it does for any other dynamically-linked process. See `tests/proxy_php_ldpreload_integration.rs`. This depends on the target's curl/OpenSSL actually calling those exact libc/OpenSSL symbols — some PHP builds bundle or `dlopen(RTLD_DEEPBIND)` their own libcurl/libssl, which shields them from `LD_PRELOAD` interposition; the integration test treats "child succeeded, zero traces captured" as an environment-support gap (skip) rather than a failure, since it's a property of the specific PHP build, not of phantom's PHP support code.

---

## JSONL Output Schema

When `--output jsonl` is used, one JSON object is written per line to stdout. All fields are always present unless marked optional.
Expand Down Expand Up @@ -154,6 +178,13 @@ cargo test --test proxy_node_integration -- --nocapture
# Single test function:
cargo test --test proxy_node_integration test_proxy_captures_node_app_traffic -- --nocapture
cargo test --test proxy_node_integration test_proxy_captures_alternative_http_clients -- --nocapture

# PHP curl proxy integration test (requires php with the curl extension in PATH)
cargo test --test proxy_php_integration -- --nocapture

# PHP curl ldpreload integration test (Linux only; builds phantom-agent on demand)
cargo build -p phantom-agent
cargo test --test proxy_php_ldpreload_integration -- --nocapture
```

### Node.js Integration Tests (`tests/proxy_node_integration.rs`)
Expand All @@ -170,6 +201,15 @@ Both tests:
- Parse JSONL output and assert method, path, status code per trace.
- Identify traces by `x-phantom-client` custom header in `request_headers`.

### PHP Integration Tests

| Test | Description |
|------|-------------|
| `tests/proxy_php_integration.rs` | `proxy` backend: PHP curl HTTP + HTTPS via injected `-d curl.cainfo=` (real CA verification, no bypass) — 4 traces |
| `tests/proxy_php_ldpreload_integration.rs` | `ldpreload` backend (Linux only): PHP curl HTTP + HTTPS via libc/OpenSSL hooks, zero PHP-specific code — 4 traces |

Both auto-skip if `php` (or its curl extension) is not available. `client.php` (shared by both tests, `tests/apps/php-app/client.php`) is written in PHP 5.3-compatible syntax and takes an optional `PHANTOM_TEST_INSECURE` env var to disable curl peer verification for the ldpreload test, where there is no phantom CA to trust against the mock backend's self-signed cert.

### Run a Single Unit Test Function

```sh
Expand Down Expand Up @@ -312,14 +352,17 @@ pub enum StorageError {
| `tests/apps/node-app/client.js` | Test client: `http`/`https` module usage (basic integration test) |
| `tests/apps/node-app/client-alts.js` | Test client: `axios`, `undici`, `globalThis.fetch` (alternative HTTP clients test) |
| `tests/apps/node-app/package.json` | Node app deps: `axios ^1.7`, `undici ^7` |
| `tests/proxy_php_integration.rs` | Integration test: PHP curl HTTP/HTTPS via `proxy` backend (`-d curl.cainfo=` injection) |
| `tests/proxy_php_ldpreload_integration.rs` | Integration test: PHP curl HTTP/HTTPS via `ldpreload` backend (Linux only) |
| `tests/apps/php-app/client.php` | Test client: curl extension, PHP 5.3-compatible syntax; shared by both PHP tests |
| `crates/phantom-core/src/trace.rs` | `HttpTrace`, `TraceId`, `SpanId`, `HttpMethod` |
| `crates/phantom-core/src/storage.rs` | `TraceStore` trait |
| `crates/phantom-core/src/capture.rs` | `CaptureBackend` trait |
| `crates/phantom-core/src/error.rs` | `CaptureError`, `StorageError` |
| `crates/phantom-storage/src/fjall_store.rs` | Storage implementation + all storage tests |
| `crates/phantom-capture/src/proxy.rs` | MITM proxy implementation (cross-platform) |
| `crates/phantom-capture/src/proxy.rs` | MITM proxy implementation (cross-platform); also exposes `ca_cert_pem()` for PHP `curl.cainfo` injection |
| `crates/phantom-capture/src/ldpreload.rs` | LD_PRELOAD capture backend (Linux only) |
| `crates/phantom-agent/src/lib.rs` | LD_PRELOAD dylib: hooks libc `send`/`recv`/`close` |
| `crates/phantom-agent/src/lib.rs` | LD_PRELOAD dylib: hooks libc `send`/`recv`/`close` and OpenSSL `SSL_write`/`SSL_read` (HTTPS) |
| `crates/phantom-tui/src/app.rs` | TUI state (`App`) and all state mutation |
| `crates/phantom-tui/src/ui.rs` | Ratatui rendering functions |
| `crates/phantom-tui/src/lib.rs` | TUI entry point and event loop |
Expand Down
13 changes: 13 additions & 0 deletions crates/phantom-capture/src/proxy.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ pub struct ProxyCaptureBackend {
fault_config: FaultConfig,
shutdown_tx: Option<oneshot::Sender<()>>,
task_handle: Option<tokio::task::JoinHandle<()>>,
ca_cert_pem: Arc<std::sync::Mutex<Option<String>>>,
}

impl ProxyCaptureBackend {
Expand All @@ -35,6 +36,7 @@ impl ProxyCaptureBackend {
fault_config: FaultConfig::default(),
shutdown_tx: None,
task_handle: None,
ca_cert_pem: Arc::new(std::sync::Mutex::new(None)),
}
}

Expand All @@ -43,6 +45,15 @@ impl ProxyCaptureBackend {
self.fault_config = config;
self
}

/// Returns the PEM-encoded MITM CA certificate once the proxy has started.
///
/// `None` until `start()` has generated the CA (which happens before the
/// proxy begins listening, so it is always available once the proxy port
/// is confirmed to be accepting connections).
pub fn ca_cert_pem(&self) -> Option<String> {
self.ca_cert_pem.lock().unwrap().clone()
}
}

impl CaptureBackend for ProxyCaptureBackend {
Expand All @@ -58,9 +69,11 @@ impl CaptureBackend for ProxyCaptureBackend {

let port = self.listen_port;
let insecure = self.insecure;
let ca_cert_pem = Arc::clone(&self.ca_cert_pem);

let task_handle = tokio::spawn(async move {
let (key_pair, ca_cert) = generate_ca();
*ca_cert_pem.lock().unwrap() = Some(ca_cert.pem());
let ca = RcgenAuthority::new(key_pair, ca_cert, 1000);

let addr = SocketAddr::from(([127, 0, 0, 1], port));
Expand Down
86 changes: 77 additions & 9 deletions src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ const JAVA_AGENT_JAR: &[u8] = include_bytes!("../crates/phantom-java-agent/phant
enum Backend {
/// MITM proxy — captures HTTP + HTTPS, cross-platform. Node.js HTTPS injected automatically.
Proxy,
/// LD_PRELOAD agent — plain HTTP only, Linux only. No proxy config needed.
/// LD_PRELOAD agent — captures HTTP + HTTPS, Linux only. No proxy config needed.
#[cfg(target_os = "linux")]
Ldpreload,
}
Expand Down Expand Up @@ -62,18 +62,27 @@ The target application requires NO code changes.\n\
• Node.js (`phantom -- node app.js`)\n\
proxy-preload.js is injected automatically via --require. Both http://\n\
and https:// are captured with zero application changes.\n\
\n\
• PHP (`phantom -- php app.php`)\n\
The MITM CA certificate is injected automatically via -d curl.cainfo=.\n\
curl-based HTTP and HTTPS (incl. Guzzle's default handler) are captured\n\
with zero application changes. Requires PHP >= 5.3.7 for curl.cainfo;\n\
only the curl extension is covered (not PHP streams).\n\
\n\
• Other commands (`phantom -- curl http://api.example.com/v1`)\n\
HTTP_PROXY / http_proxy is set automatically. Plain HTTP is captured.\n\
HTTPS requires the application to honour HTTP_PROXY CONNECT tunnelling.\n\
HTTP_PROXY / HTTPS_PROXY (and lowercase variants) are set automatically.\n\
Plain HTTP is captured; HTTPS is captured if the application honours\n\
these env vars for CONNECT tunnelling (as libcurl does by default).\n\
\n\
• Manual (start phantom alone, then configure your app)\n\
Set HTTP_PROXY=http://127.0.0.1:8080 in the target process yourself.\n\
\n\
ldpreload (Linux only)\n\
Injects libphantom_agent.so via LD_PRELOAD. Hooks send/recv/close at\n\
the libc level. No proxy configuration required. Plain HTTP only\n\
(HTTPS traffic is already encrypted at the socket layer).\n\
the libc level for plain HTTP, and OpenSSL SSL_write/SSL_read for HTTPS\n\
(captured above the TLS layer, before encryption). No proxy config\n\
required and no MITM certificate involved — works for any dynamically\n\
linked process, language-agnostic (e.g. PHP's curl extension).\n\
\n\
━━━ OUTPUT MODES ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n\
\n\
Expand Down Expand Up @@ -109,6 +118,9 @@ The target application requires NO code changes.\n\
\n\
# Allow self-signed TLS certs on backend servers:\n\
phantom --insecure --output jsonl -- node app.js\n\
\n\
# Trace a PHP app — curl-based HTTP + HTTPS captured, zero app changes:\n\
phantom -- php app.php\n\
\n\
# Trace any command (plain HTTP only, HTTPS if app uses HTTP_PROXY CONNECT):\n\
phantom -- curl http://api.example.com/v1/users\n\
Expand All @@ -128,7 +140,7 @@ The target application requires NO code changes.\n\
# Build the agent first:\n\
cargo build -p phantom-agent\n\
\n\
# Trace a process (plain HTTP only):\n\
# Trace a process (HTTP + HTTPS, no MITM certificate needed):\n\
phantom --backend ldpreload \\\n\
--agent-lib ./target/debug/libphantom_agent.so \\\n\
-- curl http://api.example.com/v1/users\n\
Expand All @@ -147,7 +159,7 @@ The target application requires NO code changes.\n\
version
)]
struct Cli {
/// Capture backend: 'proxy' (MITM, cross-platform) or 'ldpreload' (Linux, plain HTTP only).
/// Capture backend: 'proxy' (MITM, cross-platform) or 'ldpreload' (Linux, HTTP + HTTPS).
#[arg(short, long, value_enum, default_value = "proxy")]
backend: Backend,

Expand Down Expand Up @@ -415,20 +427,38 @@ fn is_java_command(exe: &str) -> bool {
base == "java" || base == "javaw"
}

/// Returns `true` if `exe` (path or bare name) resolves to `php` or a
/// version-suffixed PHP binary (e.g. `php7.4`, `php8.2`, `php5.3`).
fn is_php_command(exe: &str) -> bool {
let base = Path::new(exe)
.file_name()
.and_then(|n| n.to_str())
.unwrap_or(exe);
base == "php"
|| (base.starts_with("php")
&& !base[3..].is_empty()
&& base[3..].chars().all(|c| c.is_ascii_digit() || c == '.'))
}

/// Spawns `command` as a child process routed through the phantom proxy.
///
/// * `HTTP_PROXY` / `http_proxy` are set so plain HTTP is captured.
/// * `HTTP_PROXY` / `HTTPS_PROXY` (and lowercase variants) are set so plain
/// HTTP and HTTP-client-honoured HTTPS (e.g. libcurl) are captured.
/// * For Node.js executables the embedded proxy-preload script is written to a
/// temp file and prepended as `--require <path>` so HTTPS is also captured
/// without touching the application source.
/// * For Java executables, the phantom-java-agent.jar is injected via -javaagent
/// to force proxy settings and bypass SSL verification globally.
/// * For PHP executables, the MITM CA certificate is written to a temp PEM
/// file and injected via `-d curl.cainfo=<path>` so the curl extension
/// trusts phantom's HTTPS interception without any application changes.
///
/// Returns `(child, Option<TempScript>)`. The `TempScript` must be kept alive
/// until after the child exits so the file is not deleted prematurely.
fn spawn_proxy_child(
command: &[String],
proxy_port: u16,
ca_cert_pem: Option<&str>,
) -> anyhow::Result<(std::process::Child, Option<TempScript>)> {
let exe = &command[0];
let proxy_url = format!("http://127.0.0.1:{proxy_port}");
Expand All @@ -453,11 +483,48 @@ fn spawn_proxy_child(
actual_args = args;
}

if is_php_command(exe)
&& let Some(pem) = ca_cert_pem
{
// Write the MITM CA certificate to a temp PEM file.
let ca_path = std::env::temp_dir().join(format!("phantom-ca-{}.pem", std::process::id()));
std::fs::write(&ca_path, pem)
.map_err(|e| anyhow::anyhow!("failed to write CA cert: {e}"))?;
temp_script = Some(TempScript(ca_path.clone()));

// Prepend -d curl.cainfo=<path> before the rest of the args, so
// the curl extension trusts phantom's MITM certificate without
// any changes to the application's own curl_setopt() calls.
let mut args = vec![
"-d".to_string(),
format!("curl.cainfo={}", ca_path.display()),
];
args.extend_from_slice(&actual_args);
actual_args = args;
}

let mut cmd = std::process::Command::new(exe);
cmd.args(&actual_args)
.env("HTTP_PROXY", &proxy_url)
.env("http_proxy", &proxy_url);

// Node.js handles HTTPS itself via the injected proxy-preload.js (a
// custom ProxyTunnelAgent / undici ProxyAgent). Setting HTTPS_PROXY here
// would make libraries like axios configure their own competing
// httpsAgent from the env var, conflicting with that injected agent and
// breaking HTTPS capture. So HTTPS_PROXY/NO_PROXY are only set for
// non-Node commands (e.g. curl, PHP's curl extension), which rely on
// libcurl's native env-var proxy detection instead.
if !is_node_command(exe) {
cmd.env("HTTPS_PROXY", &proxy_url)
.env("https_proxy", &proxy_url)
// Clear any inherited no-proxy exclusions (e.g. for `localhost`)
// so libcurl doesn't bypass phantom's proxy for local targets.
// Mirrors the Java branch's explicit `-Dhttp.nonProxyHosts=` below.
.env("NO_PROXY", "")
.env("no_proxy", "");
}

// For Java processes, inject proxy settings and the Java Agent via JAVA_TOOL_OPTIONS.
if is_java_command(exe) {
// Write the embedded Java Agent to a temp file.
Expand Down Expand Up @@ -497,7 +564,8 @@ async fn run_proxy(cli: Cli, store: Arc<FjallTraceStore>) -> anyhow::Result<()>
if !cli.command.is_empty() {
// Wait for the proxy to be ready before spawning the child.
wait_for_proxy(cli.port).await?;
let (child, ts) = spawn_proxy_child(&cli.command, cli.port)?;
let ca_cert_pem = backend.ca_cert_pem();
let (child, ts) = spawn_proxy_child(&cli.command, cli.port, ca_cert_pem.as_deref())?;
eprintln!(
"phantom: spawned PID {} → {}",
child.id(),
Expand Down
Loading
Loading