diff --git a/README.md b/README.md index 75c6df5..f297e1d 100644 --- a/README.md +++ b/README.md @@ -83,6 +83,40 @@ tail -f .vm/mitmdump.log # all proxy traffic cat .vm/blocked.jsonl | jq . # blocked requests ``` +## Host tool bridges + +Some tools can't run inside the sandbox because they only exist on the host — Xcode being the obvious one. `vm.py start --bridge PORT=COMMAND` exposes a host-side stdio program to the guest at `10.0.2.101:PORT`. Every TCP connection the guest opens to that address spawns a fresh copy of `COMMAND` on the host with its stdin/stdout wired to the socket, which is exactly the shape of an [MCP](https://modelcontextprotocol.io/) stdio server. The flag is repeatable, and bridges last only for the lifetime of that `vm.py start`. + +> **A bridge is a hole in the sandbox boundary.** Bridge traffic does **not** pass through mitmproxy and is **not** checked against `allowlist.txt`. It is a raw pipe from the guest to a process running on your host, outside the VM, with your own user's privileges — and a process the agent can restart at will, once per connection. Only bridge tools you are willing to let the agent drive unsupervised, and leave the flag off the rest of the time. + +### Example: driving Xcode from inside the VM + +Xcode exposes its tools to external agents over an MCP stdio server, which `xcrun mcpbridge` launches — so the agent in the guest can drive the Xcode running on your host. Three steps, and the first is the one that gets missed: + +**1. On the macOS host, in Xcode:** Settings → Intelligence → enable **"Allow external agents to use Xcode tools"**. Xcode also has to be running with your project open. Without that toggle the bridge connects fine and hands back an empty tool list, which looks like a networking failure and isn't one. + +**2. Start the VM with the bridge:** + +```bash +./vm.py start --bridge '8110=xcrun mcpbridge' +``` + +The port is arbitrary — pick anything free. `10.0.2.101` is internal to the QEMU process; nothing is opened on your host's real network interfaces. + +**3. Inside the guest, from your project folder:** + +```bash +claude mcp add --transport stdio xcode -- nc 10.0.2.101 8110 +``` + +`vm.py start` prints a ready-to-paste version of this line (named `bridge-8110`) when it brings the bridge up, so you don't have to remember the address. `nc` is preinstalled in the guest. Registration lives in the guest's Claude config, so re-run it after `vm.py reset` — that config is on the ephemeral overlay disk. + +Notes and limits: + +- **`COMMAND` must not contain commas.** QEMU splits its netdev argument on commas; spaces are fine. +- **The host command inherits your host environment**, including anything on your `PATH` and any credentials in it. +- **Any MCP stdio server works.** The bridge doesn't care what the command is, so Xcode is just the obvious example — substitute any host-side tool that speaks MCP over stdio. + ## Credentials and the shared directory The `shared/` directory is mounted read-write inside the guest. Be deliberate about what you place there. diff --git a/TODO.md b/TODO.md index 325c08b..8ab9ade 100644 --- a/TODO.md +++ b/TODO.md @@ -4,3 +4,71 @@ 2. ~~Evaluate alternatives to using the `com.apple/agent-vm` pf anchor on macOS.~~ Evaluated — keeping `com.apple/agent-vm`. Alternatives (custom anchor in `/etc/pf.conf`, `pfctl -f`, LaunchDaemon) are all more invasive or dangerous. Added rationale comment and runtime verification that warns if rules aren't active. 4. ~~Simple is harder than complex. Review all code and propose 3 ways it can be simplified based on what you know now.~~ Done — (a) replaced MIME multipart user-data merge with cloud-init's native `#cloud-config-archive` format, (b) unified EFI preparation into a shared `_prepare_efi()` helper, (c) replaced 3-way ISO tool cascade with platform-explicit `_build_iso()`. 5. ~~Explore going sudoless by using QEMU's `-netdev user` (slirp) with `guestfwd` to connect the guest to the proxy.~~ Done — replaced socket_vmnet (macOS) and TAP/bridge (Linux) with slirp `restrict=on` + `guestfwd=tcp:10.0.2.100:PORT-cmd:nc 127.0.0.1 PORT`. Removed all pf/iptables firewall code. Zero sudo prompts. The `cmd:nc` guestfwd mode (revisited with QEMU 10.x) works correctly — each guest TCP connection spawns a fresh `nc` that connects to the host-side proxy. `restrict=on` blocks all other outbound traffic (TCP, UDP, ICMP) at the slirp layer, providing stronger isolation than the previous firewall-based approach. +6. Make the `--bridge` MCP wiring self-serve. Today `vm.py start --bridge '8110=xcrun mcpbridge'` opens the socket, but the user still has to run `claude mcp add --transport stdio xcode -- nc 10.0.2.101 8110` inside the guest by hand. Options, roughly in order of increasing magic: (a) document it in README.md next to the `--bridge` flag — see the required contents below; (b) have `--bridge` print the exact `claude mcp add` line to paste after SSH, e.g. + + Host-tool bridges (bypass the allowlist — raw pipe to host): + 10.0.2.101:8110 -> xcrun mcpbridge + + To use from within the guest, run this command from within your + project folder: + + claude mcp add --transport stdio bridge-8110 -- nc 10.0.2.101 8110 + + (a) is required regardless of whether we do (b) or (c) — the README needs an end-to-end Xcode walkthrough covering all three steps, because the host-side one is not discoverable: + + 1. **On the macOS host, in Xcode:** Settings → Intelligence → enable **"Allow external agents to use Xcode tools"**. Without this the bridge connects but the tools aren't exposed, which looks like a networking failure and isn't one. + 2. **Start the VM with the bridge:** + + ./vm.py start --bridge '8110=xcrun mcpbridge' + + 3. **Inside the guest, from your project folder:** + + claude mcp add --transport stdio xcode -- nc 10.0.2.101 8110 + + Also state plainly in that section what the flag's `--help` already warns: this channel bypasses the mitmproxy allowlist and is a raw pipe to a host process. Someone reading the README to wire up Xcode should not have to read `--help` to learn they've opened a hole in the sandbox boundary. Verify the `xcrun mcpbridge` invocation against whatever Xcode actually ships before publishing it — it's currently just the example string in the `--bridge` help text. + + (c) add a `--xcode` convenience flag that implies the bridge and writes the MCP server entry into the guest's Claude config via cloud-init or a post-boot SSH command. Note (c) means `vm.py` starts owning guest-side agent config, and the registration has to be idempotent across reboots and survive `vm.py reset`; decide whether that belongs in this tool at all. +7. Fix mode bits being lost on the shared directory, and revisit the 9p driver while we're in there. Symptom: editing a file from inside the guest drops its exec bit on the host — `vm.py` went from `100755` to `100644` in the working tree with no `chmod` anywhere. Suspect the 9p transport and/or the `bindfs --force-user=vm --force-group=vm` layer not round-tripping permissions; worth confirming which of the two is responsible before changing anything. The likely fix is switching the share from 9p to virtiofs, which we want regardless for the performance win — but check that it doesn't reintroduce the UID-mapping problem bindfs exists to solve, and that it doesn't pull us back to the "generic" vs "genericcloud" image constraint for the wrong reason (see CLAUDE.md). Until it's fixed, check `git diff` for spurious mode changes before committing. +8. Update the default guest Claude Code settings (`cloud-init/user-data`, the `/opt/provision/claude-settings.json` block around line 80) to match what we actually run with: + + { + "permissions": { + "defaultMode": "bypassPermissions" + }, + "model": "opus[1m]", + "effortLevel": "high", + "skipDangerousModePermissionPrompt": true, + "theme": "dark", + "tui": "default" + } + + Changes from the current block: `model` moves from the pinned `claude-opus-4-6` to the `opus[1m]` alias (large context window, and it tracks the current Opus instead of going stale), plus new `effortLevel`, `skipDangerousModePermissionPrompt`, and `theme` keys. `skipDangerousModePermissionPrompt` suppresses the startup confirmation that `bypassPermissions` otherwise triggers — appropriate here precisely *because* the VM is the sandbox, but it does mean a fresh guest goes straight to autonomous execution with no interstitial. Worth verifying each key is spelled the way the installed Claude Code version expects; unknown keys are ignored silently, so a typo shows up as "the setting just didn't apply." +9. Also add `"tui": "default"` to the guest settings (part of the same block as item 8). This is the `/tui` setting — `"fullscreen"` is the alternate renderer, `"default"` is the normal one. The fullscreen renderer's OSC-based copy/paste doesn't round-trip through the SSH session `vm.py start` drops you into, so the guest should ship with the normal renderer rather than leaving each user to discover `/tui` themselves. Key name confirmed against a live `~/.claude/settings.json` written by `/tui`, not guessed. +10. **Bug:** changes to `cloud-init/` don't rebuild `seed.iso`. `build_seed_iso()` (`vm.py:379`) returns immediately if `.vm/seed.iso` exists, so after editing `cloud-init/user-data`, `meta-data`, or `network-config` the next `vm.py start` boots the *old* config with no warning — you have to know to run `vm.py reset` first. This is a genuine footgun: the edit looks applied, the VM boots fine, and the change silently isn't there. Detect staleness by hashing the fully assembled user-data (not just the `cloud-init/` file mtimes — that misses injected values like the SSH pubkey, git identity, and CA cert) and storing the digest next to the ISO. + + **Do not auto-reset.** Blowing away the user's disk because a config file changed is far more disruptive than the bug it fixes — there's real work in that VM. The fix is *detect and tell*, not *detect and act*: on a digest mismatch, rebuild the ISO and print a clear warning that the new cloud-init config won't take effect until `vm.py reset`, then boot normally. Leave the decision to the user. + + Related wrinkle worth confirming while implementing: cloud-init keys off the `instance-id` in `meta-data`, so even a correctly-rebuilt seed.iso attached to an existing disk generally won't re-run provisioning. That makes the warning the actual deliverable here — the rebuild alone doesn't fix anything on a VM that's already been provisioned. +11. Expose CPU count and disk size as `vm.py start` options. `-smp` is hardcoded in `build_qemu_args()` (`vm.py:440`) and the qcow2 overlay is created at a hardcoded `20G` in `ensure_disk()` (`vm.py:374`). Add `--cpus` and `--disk-size` alongside the existing `--memory`. Note the disk is only sized at creation time, so `--disk-size` has no effect on an existing `.vm/disk.qcow2` — either say so in the help text or detect the mismatch and tell the user to `vm.py reset`. +12. Persist `~/.claude` across `vm.py reset` by backing it with a host-mapped directory. Today it lives on the ephemeral overlay disk, so a reset takes the whole agent history with it: `projects/`, `sessions/`, `history.jsonl`, `todos`/`tasks`, `file-history`, and any per-project config. That's the main thing that makes reset feel expensive. + + Placement: it has to live outside `.vm/`, which is exactly what `cmd_reset()` deletes — mirror how `.images/` survives. shared/.claude probably works fine if Claude itself can be convinced to look there, or otherwise a dedicated gitignored .claude in this same repo. + + Things to work out: + - **Conflicts with items 8/9.** `runcmd` currently does an unconditional `cp /opt/provision/claude-settings.json /home/vm/.claude/settings.json`, instead, copy that file only if a copy doesn't already exist, so it gets bootstrapped but not clobbered. + - **File locking over 9p.** `~/.claude` has `daemon.lock`, append-heavy `history.jsonl`, and a live `daemon/` — advisory locking and rename semantics are where 9p is weakest. This is a strong argument for doing item 7 (virtiofs) *first* and building on that rather than on 9p+bindfs. + - **`.credentials.json` leaves the sandbox.** This is fine since these credentials come from the host box in the first place. + - Consider persisting a subset (`projects/`, `sessions/`, `history.jsonl`, `settings.json`) via symlinks rather than mapping the whole directory — it sidesteps both the daemon-state and credentials questions. +13. Extend the guest's global `CLAUDE.md` (the `/opt/provision/CLAUDE.md` block in `cloud-init/user-data`, currently network-proxy guidance only) with a section on installing things. Two points: + + - **Passwordless sudo is available.** The `vm` user has `sudo: ALL=(ALL) NOPASSWD:ALL`, but nothing tells the agent that, so it may hedge around installs or ask the user to run commands it could run itself. + - **`apt` is the right default.** Prefer `apt install` over language-specific version managers, source builds, or vendored toolchains. The exception is a project that clearly pins versions — a `.tool-versions`, `.nvmrc`, `.python-version`, `rust-toolchain.toml`, or equivalent standard lockfile — in which case honor the pin. The rule should be evidence-based: a committed version file is the signal, not a guess about what the project "probably" wants. + + - **Python is a hard carve-out: never use Debian-provided Python packages.** No `apt install python3-` — not the interpreter, not libraries, not `python3-pip`/`python3-venv`. Python interpreters and packages come from **uv**, which manages interpreter versions too, and uv itself is installed only from its official binary release (never pip/conda/pipx). Debian's Python packaging is system-owned, externally-managed, and version-frozen to the release; mixing it with uv-managed environments is how you get a broken interpreter that's hard to unpick. State this as an absolute, not a preference — the apt-first default must not read as licensing `apt install python3-requests`. + + Framing: "apt is the default for system packages, **except** Python." Both halves need to be in the same section, or an agent that reads only one will do the wrong thing in exactly the case that matters. + + Since all standard Debian apt repos are allowlisted, `apt install` works without any proxy friction — no need to caveat it. Do mention that a 418 on a *third-party* apt repo means that repo isn't allowlisted rather than the package not existing, since that's the one case where an install fails for a reason the agent can't infer from apt's error. +14. Study `grant.py` and make it a first-class capability that is available by default inside the VM, probably as a skill. (Obviously, a key step of the skill is still asking the user to review the proposed policy file and run a command on their host machine to grant it. The command the user runs needs to be committed outside the shared dir so it can be reviewed and can't be modified by a malicious agent between the user reviewing it and running it in a TOCTOU vuln.) Cover how to use this safely in the README (the agent realizes it needs AWS permissions, proposes IAM policy, notifies the user with the PushNotification tool that they need to review and grant, and then the agent waits for the outputted credentials to appear.) +15. Add some kind of Notification support, even if it's just terminal bell, that works through SSH. +16. Add support for a netrc-style credentials file as a sibling to allowlist.txt that the proxy will automatically add to Authorization headers so the VM/agent never see the actual credentials being used. Include a sample file. \ No newline at end of file diff --git a/cloud-init/user-data b/cloud-init/user-data index 044d6dd..5c1142d 100644 --- a/cloud-init/user-data +++ b/cloud-init/user-data @@ -182,6 +182,9 @@ packages: - bindfs - git - docker.io + # nc: used to bridge a stdio MCP server to a host-side tool exposed via + # `vm.py start --bridge` (e.g. Xcode's `xcrun mcpbridge` on a macOS host). + - netcat-openbsd runcmd: - mkdir -p /mnt/9p /home/vm/shared diff --git a/vm.py b/vm.py index 50d356a..f4490e3 100755 --- a/vm.py +++ b/vm.py @@ -35,6 +35,15 @@ # appears on a real network interface — it's internal to the QEMU process. _GUESTFWD_IP = "10.0.2.100" +# A second internal slirp address used to expose host-side stdio tools to +# the guest (see the --bridge flag). Each guest TCP connection to +# _HOSTTOOL_IP: spawns a fresh copy of the configured command on the +# host with its stdin/stdout piped to the socket — exactly the shape of an +# MCP stdio server. WARNING: traffic on these ports does NOT pass through +# the mitmproxy allowlist; it is a raw pipe to a host process. Only bridge +# tools you trust the guest agent to drive. +_HOSTTOOL_IP = "10.0.2.101" + # --------------------------------------------------------------------------- # Architecture enum @@ -84,6 +93,10 @@ def __init__(self, arch: Arch, proxy_port: int = PROXY_PORT, self.arch = arch self.proxy_port = proxy_port self.ssh_host_port = ssh_host_port + # (guest_port, host_command) pairs exposed via guestfwd on + # _HOSTTOOL_IP. Populated from the --bridge flag; empty by default + # so the sandbox's network boundary is unchanged unless opted in. + self.bridges: list[tuple[str, str]] = [] # -- Network -- @@ -93,13 +106,21 @@ def proxy_ip(self) -> str: return _GUESTFWD_IP def qemu_netdev_arg(self) -> str: - """Slirp netdev with restrict=on, SSH hostfwd, and proxy guestfwd.""" - return ( + """Slirp netdev with restrict=on, SSH hostfwd, and proxy guestfwd. + + Any --bridge entries are appended as extra guestfwds on + _HOSTTOOL_IP. QEMU splits the netdev string on commas, so bridge + commands must not contain commas (spaces are fine). + """ + arg = ( f"user,id=net0,restrict=on" f",hostfwd=tcp:127.0.0.1:{self.ssh_host_port}-:22" f",guestfwd=tcp:{_GUESTFWD_IP}:{self.proxy_port}" f"-cmd:nc 127.0.0.1 {self.proxy_port}" ) + for port, command in self.bridges: + arg += f",guestfwd=tcp:{_HOSTTOOL_IP}:{port}-cmd:{command}" + return arg def network_config_override(self) -> str | None: """Cloud-init network-config for slirp DHCP.""" @@ -565,9 +586,50 @@ def _wait_for_cloud_init(ssh_host_port: int, qemu_proc: subprocess.Popen, # Commands # --------------------------------------------------------------------------- +def _print_bridges(bridges: list[tuple[str, str]]) -> None: + """Show the open bridges plus the command that registers each in the guest. + + Opening the socket is only half the job: nothing inside the guest knows + the bridge exists until an MCP client is pointed at it, so print the + exact `claude mcp add` line rather than leaving the user to reconstruct + it from the IP and port. + + cmd_start calls this twice — once up front, because the allowlist bypass + is a security notice that belongs before the VM boots, and once right + before the SSH handoff, because a minute of boot output scrolls the + first copy off screen and the copy you paste from should be next to the + prompt. + """ + if not bridges: + return + print("Host-tool bridges (bypass the allowlist — raw pipe to host):") + for port, command in bridges: + print(f" {_HOSTTOOL_IP}:{port} -> {command}") + print() + this_command = "this command" if len(bridges) == 1 else "these commands" + print(f"To use from within the guest, run {this_command} from within your") + print("project folder:") + print() + for port, _ in bridges: + print(f" claude mcp add --transport stdio bridge-{port}" + f" -- nc {_HOSTTOOL_IP} {port}") + print() + + def cmd_start(args: argparse.Namespace) -> None: backend = make_backend(proxy_port=args.proxy_port, ssh_host_port=args.ssh_port) + + # Parse --bridge PORT=COMMAND entries into (port, command) pairs. + for spec in getattr(args, "bridge", []) or []: + port, sep, command = spec.partition("=") + if not sep or not port.isdigit() or not command.strip(): + sys.exit(f"--bridge must be PORT=COMMAND, got: {spec!r}") + if "," in command: + sys.exit(f"--bridge COMMAND must not contain commas: {spec!r}") + backend.bridges.append((port, command.strip())) + _print_bridges(backend.bridges) + interactive = sys.stdout.isatty() STATE_DIR.mkdir(parents=True, exist_ok=True) @@ -601,6 +663,7 @@ def cmd_start(args: argparse.Namespace) -> None: try: _wait_for_ssh(backend.ssh_host_port, qemu_proc) _wait_for_cloud_init(backend.ssh_host_port, qemu_proc) + _print_bridges(backend.bridges) print(f" Log out of the SSH session to stop the VM.\n") subprocess.run([*_ssh_args(backend.ssh_host_port)]) except KeyboardInterrupt: @@ -679,6 +742,16 @@ def main() -> None: help=f"Host port forwarded to guest SSH (default: {SSH_HOST_PORT}). " "Change to avoid collisions with another running VM.", ) + start_p.add_argument( + "--bridge", action="append", metavar="PORT=COMMAND", default=[], + help="Expose a host-side stdio command to the guest at " + f"{_HOSTTOOL_IP}:PORT (repeatable). Each guest connection spawns " + "COMMAND on the host with its stdio piped to the socket — e.g. an " + "MCP stdio server: --bridge '8110=xcrun mcpbridge'. WARNING: this " + "channel bypasses the mitmproxy allowlist (raw pipe to a host " + "process). COMMAND must not contain commas. See \"Host tool " + "bridges\" in README.md for an end-to-end Xcode walkthrough.", + ) sub.add_parser("reset", help="Destroy ephemeral VM state (keeps base image and SSH key)") ssh_p = sub.add_parser("ssh", help="SSH into the VM")