Add shared Environment browser with progressive agent tools - #99
Conversation
There was a problem hiding this comment.
Important
Two security-facing points worth resolving before merge: guest sessions silently grant clipboard-read/geolocation to every website with no prompt, and in-workspace preview documents execute with a whole-workspace same-origin grant plus unfettered network egress — without any Ask-mode consent when an agent opens them.
Reviewed changes
- Shared Environment browser: profiles (persistent/Incognito/named), responsive viewport presets, capture/recording, floating/PiP presentation, and an annotation editor that attaches selected elements, drawings, style previews and marked screenshots to the composer — all over main-owned sandboxed
WebContentsViewguests that survive panel switches. - Progressive agent disclosure: an 85-token
browsergateway installs the 14 executable tools at the next turn boundary, with outbound-transform and compaction budgets updated in lockstep and per-invocation revalidation of workspace authority and access. - Managed preview service: loopback capability-token leases for local HTML/PDF, exact-file approval with pinned identities for outside-workspace documents, bounded request drains and revocation on close/owner loss.
- Ask-mode action binding: approvals pin tab, page revision and arguments with effect-time re-assertion; human takeover and navigation invalidate approvals and queued actions.
- Context accounting: bounded 32K-character snapshot projection, failure-priority retention, independent browser/Computer-Use image pools, claim-check and mobile activity-label parity, onboarding bento tile, docs, and deterministic-model E2E infrastructure.
⚠️ In-workspace previews give any script inside the opened document a whole-workspace read plus silent network egress
When a workspace HTML/PDF is previewed, prepare keeps the workspace root as the lease root and serve resolves every requested path against it. Any script in the opened document — including scripts the document itself loads from remote origins, since served responses only carry frame-ancestors — can same-origin fetch() every MIME-whitelisted sibling under the workspace (.html, .js, .css, .pdf, .svg, images, fonts) and POST them anywhere. This is more permissive than the file:// open the PR describes itself as replacing, where sibling reads are blocked cross-origin. The agent path amplifies it: an agent-initiated open of an in-workspace document auto-admits with no approval even in an Ask-mode workspace, and nothing in the UI signals that the document gained workspace-read and network-egress capabilities. A downloaded HTML artifact in a repo is the concrete case: the model opens it as instructed, its script scans the workspace, and the exfiltration is unattended and invisible.
Technical details
# Local-document preview trust boundary
## Affected sites
- main/services/browser/files.ts:284-292 — `root = workspaceRoot` for confined (in-workspace) documents; `details.files` stays undefined.
- main/services/browser/files.ts:315-323 — the lease key is `workspace:<id>` and `requiresApproval: Boolean(files)` is false, so no exact grant or approval ever exists for workspace documents.
- main/services/browser/files.ts:747 — `inspectLeaseFile` without a `files` map serves any whitelisted path under the root.
- main/services/llm-client.ts:2279-2280 — in-workspace preparations are auto-admitted into `browserFileApprovals`; the Ask-mode file-approval gate (llm-client.ts:2334-2343) never engages for them, and `browser_open`/`browser_navigate` are outside `BROWSER_MUTATION_TOOL_NAMES` (main/services/browser-tools.ts:27-29) so no action approval is minted either.
- Tests assert only the *exact*-grant confinement (main/services/browser/files.test.ts:261-303); whole-root workspace reads are unasserted.
## Required outcome
- Define an explicit trust boundary for workspace documents: scope the served grant to the entry document's directory plus declared/derived assets (parity with the exact-grant model), or accept whole-root access only with a visible signal and Ask-mode parity for agent-initiated opens. As shipped, `browser_open({path})` on a repo artifact gives its scripts read access to `bank-statement.pdf`-class files anywhere in the workspace.
## Suggested approach
- Reuse the exact-grant machinery (files map + identity pinning) for workspace documents, deriving the common root from the entry + assets instead of the workspace root.
- Or route agent-initiated workspace previews through the same approval flow as outside-workspace files (ask mode only), with the file list in the summary.
## Open questions for the human
- Is executing untrusted workspace HTML with workspace-wide read + egress an intended "existing workspace authority", or should workspace documents be scoped like exact grants?ℹ️ Ask-mode navigation and hidden-tab reads are ungated by design — worth confirming the intent
browser_navigate, browser_open, browser_snapshot, browser_resize, browser_set_appearance and both recording tools never mint an action approval (BROWSER_MUTATION_TOOL_NAMES is limited to click/type/press/scroll/evaluate). That is deliberate and documented ("approvals bind interaction/evaluation"), and the Ask-mode E2E scenario pins navigate/open/snapshot running unprompted. The residual corner: browser_open({ open: false }) creates a hidden tab, so in an Ask workspace an agent can navigate a background tab to any origin its persistent session can reach and browser_snapshot it — the two stated mitigations (shared visible tab, human interrupt) both fail for that flow, and page content flows to the model provider without consent. If background automation on hidden tabs is intended to be part of Ask-mode workspace authority, this is fine; otherwise consider gating open: false navigation on approval or surfacing a "Aiden is browsing in a hidden tab" affordance.
Technical details
# Hidden-tab read path in Ask mode
## Affected sites
- main/services/browser-tools.ts:27-29 (mutation set), 161 (`open: false` documented as background automation), 226-245 (open/create/select flow)
- main/services/service.ts:1675-1686 (create with `show`), 1801-1807 (agent select emits only a `show` when not floating — hidden tabs stay hidden)
- tests/e2e/browser-agent.spec.ts:120-142 (Ask-mode scenario asserting navigate/open/snapshot run unprompted)
## Notes
- The visible-tab redirect case (navigate on the user's focused tab) is the other side of the same decision; both are defensible, but the hidden-tab variant removes even the visibility rationale.ℹ️ Nitpicks
- Mid-flight revocation has no focused test:
agent_access off/defaultsinterrupt queues and cancel file acquisitions (service.ts:1692-1705), but only next-generation disable is asserted (browser-agent.spec.ts:77-90); an action in flight during the toggle is covered by no unit or E2E test. - The disclosure install lands before the harness's post-hostPrepare cancellation checks (
pi-agent-runtime-harness.ts:1350-1357vs the abort path at 1383+), so a Stop in that window leaves the schemas installed inagent.statefor a generation that never reached the provider. Harmless today (terminal outcomes destroy the generation; new generations reset discovery), but the new "Stop … without installing tools" test only covers the pre-install window — the boundary behavior is worth a comment or one assertion.
DeepSeek Flash | 𝕏
There was a problem hiding this comment.
ℹ️ Delta-only review: no new issues in the incremental commit — the two findings from the prior review stand.
Reviewed changes
Incremental review of commit 71ea5acf ("Wait for workspace hydration before mounting the browser"), which changed only renderer/components/environment-panel.tsx (plus a papercuts note):
- Hydration-safe browser mounting:
BrowserPanelnow mounts only when anactiveworkspace exists (active && <BrowserPanel workspaceId={active.id} …>), instead of fabricating"default"during workspace-data load. This matches the gating pattern the Review and Files surfaces already use, and the existingworkspaceId-keyed effects inBrowserPanelhandle later workspace switches. No interaction with the permission-handler, preview-scope, or takeover-window findings from the prior review — those stand as written.
DeepSeek Flash | 𝕏
There was a problem hiding this comment.
ℹ️ No critical issues — the two open threads from the prior review are addressed by these commits; one minor observation inline.
Reviewed changes
- Harden preview grants to exact per-origin file sets (5be7fe6): every preview now serves only its entry plus explicitly declared
assetPaths, each distinct grant on its own loopback origin — closing the whole-workspace origin for in-workspace documents flagged in the prior review. Grants pin file identities at read time, sibling files return 403/404, and repeated identical opens share an origin while distinct grants never widen one another. - Replace cookie auth with frame-scoped request headers: initial navigation uses the bootstrap token; subsequent declared-file requests are authorized by a main-process-injected
X-Aiden-Preview-Authorizationheader restricted to the owned workspace, exact destination origin, and committed requesting-frame origin; legacyaiden_preview_*cookies are stripped from all outbound session requests. A real-Chromium test (files-browser.test.ts) pins cross-port isolation, redirect handling, and cross-origin iframe denial. - Restrict guest permission grants: new
permission-policy.tsauto-grants onlyclipboard-sanitized-writeon an owned same-origin main frame;clipboard-read,geolocation,notificationsand unknown permissions are denied, with dedicated handler tests. This resolves the prior review thread on silent sensitive permission grants. - Native debugger-source takeover matching: keyboard expectations now require Chromium's
kFromDebuggermarker, so physical keys fail closed and interrupt immediately; mouse matching is bounded to dispatch settlement with a documented collision fallback. New E2E tests pin identical-key collision ordering, mid-action access revocation, and a single cleanup release. This resolves the prior review thread on the 500 ms expected-input window. - Recording readiness: start now waits for encoded video data (bounded 5 s poll) instead of succeeding before the encoder produced a frame; E2E decodes three immediate start/stop cycles as real video.
- Diagnostics scanning and redaction assertion fixes (7829be1): the console-policy scan is now syntax-aware (AST) with regression cases so serialized guest runtime strings no longer trip it, and the E2E capability-redaction assertion no longer depends on CDP key enumeration order.
- Docs, README, and the feature guide updated to the exact-grant/header model; papercuts entries record the follow-ups.
DeepSeek Flash | 𝕏
There was a problem hiding this comment.
ℹ️ No critical issues — the open UI-sidecar thread from the prior review is addressed by this commit; one informational observation inline.
Reviewed changes
Incremental review of commit d1231c8 ("Restore static sidecars for direct user browser previews"), the only delta since the prior pullfrog review at 5be7fe6:
- Bounded static discovery for direct user previews: path-only user-originated
open_filecalls (Files → Open in Browser, chat/terminal links, the renderer link handler) now statically parse the entry HTML plus transitively referenced CSS and JS-module sources — parse5/postcss/acorn, never executing page code — and extend the exact grant with the referenced assets, capped at 64 files, depth 8, 1 MiB per source and 4 MiB total, with failures surfaced as capped, redacted tab diagnostics. - Content-pinned grants: parsed sources are fingerprinted (sha256 + size + mtime) into the per-origin grant key, and serving re-verifies identity, size/mtime and a full-content digest per request, so in-place edits, inode swaps and symlink tricks fail closed with 409 and an edited reopen gets a distinct immutable origin.
- Agent strictness preserved: discovery is gated on
context.source === "user"with noassetPaths/preparedFile, so agentbrowser_opencalls remain entry-only; a new real-Electron E2E test opens one document through the real local-link handler (discovered CSS + module graph load, sibling 403) and through the strict agent tool (blank page, distinct origin). - Race and edge hardening: identity/root/directory re-checks across every await in discovery, a cancellation seam, and tests for symlink-to-document targets, absolute-URL spoofing, same-content mtime rewrites and mid-discovery mutation.
- Docs, papercuts, and pinned dependencies (
acorn,parse5,postcss,postcss-value-parser) updated; 36 focused unit tests re-verified green locally.
ℹ️ Nitpicks
- Module graphs the discovery parser cannot follow are silent failures:
new Worker(new URL("./w.js", import.meta.url)), import-map specifiers and classic-script injection produce no tab diagnostic, so such a page loads visibly broken with only console noise. A bounded "detected but not discoverable module reference" warning forimport.meta.url-basednew URLliterals would make the documented static-only boundary observable in the UI.
DeepSeek Flash | 𝕏

Adds the shared Environment browser requested from the T3 Code reference, with compact browser chrome, profiles, responsive viewports, capture/recording, floating/PiP presentation, and annotations that attach selected text/elements/drawings/styles and marked screenshots to the current Aiden composer. User and agent actions operate on the same main-owned sandboxed tabs.
Agent access is progressively disclosed: ordinary eligible workspace generations initially advertise only
browser(85 estimated static tokens, down from 2,679). Calling it installs fourteen executable browser tools and updates both provider-context and compaction budgets at the next turn boundary. New generations reset discovery; disabled access exposes no browser tool. Active schemas/guidance cost about 3,032 estimated tokens. These are Aiden's characters/4 estimates, not billing counts.Local HTML/PDF previews now use managed loopback leases. Every document and declared asset belongs to an exact pinned grant with its own origin. Direct user HTML opens discover bounded static workspace dependencies from HTML, CSS and JavaScript, preserving exact file grants and reporting skipped dependencies. Agent opens still require explicit assetPaths. Parsed sources are content-pinned, and reopening edited sources creates a fresh grant. Outside-workspace files require explicit approval; identity checks, tab/pending/read references, cancellation, revocation and bounded drain prevent orphaned servers. Original files and user-owned servers are preserved. Browser snapshot text is bounded, old outgoing browser images are pruned independently of Computer Use, and journals retain original evidence. Review fixes cover stale approvals, queued actions after takeover, hover interception, cursor cleanup and capability redaction. Guest permissions deny clipboard reads, geolocation and notifications. Preview requests use native frame/origin-scoped headers, never host-wide cookies. Keyboard takeover uses Chromium's native debugger-source flag; mouse events retain a bounded, documented matching fallback. Recording startup waits for encoded data before it succeeds.
The implemented cleanup plan and index entry were removed as requested. The task-owned Python server and temporary log were removed; the temporary HTML was already absent. The browser guide records the complete tool list, architecture, context behavior, edge-case inventory and remaining capability limits. Two pre-existing UI test assertions were aligned with their already-shipped provider icon/opt-in press feedback so preflight passes.
Validation:
fe4ee548: desktop verification, deterministic Electron E2E, Android, native/iOS checks, release-consumer contract and Pullfrog. CI run: https://github.com/sambitcreate/aiden-agent/actions/runs/34254159982. Pullfrog confirmed the sidecar fix and the guide clarifications, with no remaining actionable findings.npm testandnpm run test:preflightpass.--fail-on-flaky-tests, including real model-request inventory, same-tab agent effects, exact-file approval/assets, listener lifetime, owner reload, identity/revocation races and annotations. Recordings are decoded as video across three immediate start/stop cycles; real Chromium coverage checks cross-port and redirect credential isolation.AidenChatTestpasses. iOS app/test targets pass generic-device build-for-testing; physical-device XCTest execution remains unperformed because no device is attached and this repository forbids simulators.