pixel-agent

Personal AI agent for macOS — chat with Claude/Codex/Gemini side by side, dispatch subagents in parallel, see and control your screen via Set-of-Mark, run your tools as an MCP server, and steer everything from your iPhone.

Native Swift · Multi-LLM · iOS remote dashboard · MCP server (14 tools) · Subagent UI · Computer use with Set-of-Mark · 675 tests · 32 ADRs

📸 More screenshots

Mac (fresh launch)	iPhone home (icon)

Demo GIF script: scripts/record-demo.sh (macOS Screen Recording → ffmpeg/gifski). Drops a recording into docs/assets/demo.gif. A polished demo lands with v0.3.

What pixel-agent does

Most "AI desktop apps" are an Electron wrapper around a single chat. pixel-agent is a native macOS power-user tool built around five capabilities you don't usually get together:

Multi-CLI orchestration — talk to Claude Code, Codex, and Gemini side by side (single or dual chat), each with its own conversation history.
Subagent panel — fan out work to up to 3 parallel subagents with a budget (wall-clock + bytes), cancellable from the UI or via the MCP dispatch_subagent tool.
iOS remote dashboard — pair your iPhone with a QR code; from the phone, change the backend/model, toggle Plan Mode, request a screenshot, watch CPU/RAM in real time, cancel subagents. Works over LAN (Bonjour) or Cloudflare relay, with ed25519-signed envelopes.
Built-in MCP server — pixel-mcp-server exposes 14 tools to any MCP client (Claude Code, Cline, Continue, Cursor): clipboard, time, dock badge, notifications, sound, screenshot, dispatch_subagent, plus 5 ui_* AX-first tools (query, click, type, screenshot, resolve).
Computer use with Set-of-Mark — AX-first hybrid UI control (per ADR-0026). Annotate a screenshot with numbered badges so a vision model can say "click #5" — deterministic ID → element mapping, no coordinate guessing.

Plus the things you'd expect: Plan Mode read-only allowlist (per ADR-0017), JSONL conversation persistence, swappable backends, ToolArbiter resource mutex, ed25519 envelope signing, Bonjour LAN-first transport with relay fallback, and a pixel-art mascot in the corner.

Why pixel-agent vs ...

Feature	pixel-agent	Claude Desktop	Cline	Aider
Native macOS (60 MB)	✅ Swift	❌ Electron (~600 MB)	❌ (VS Code)	❌ (terminal)
Multi-LLM side by side	✅ Dual chat	❌	❌	❌
iPhone remote dashboard	✅ TabView (chat + subagents + Mac panel)	❌	❌	❌
MCP server (expose tools)	✅ 14 tools	❌ client only	❌	❌
Subagent UI (parallel)	✅ cap=3	❌	❌	❌
Computer use	✅ AX-first + Set-of-Mark	❌	❌ via tools	❌
Plan Mode toggle	✅	❌	❌	❌
Open source	✅ MIT	❌	✅ Apache	✅ Apache
Architecture docs	✅ 32 ADRs	—	—	—

Pick pixel-agent if you live on macOS, run multiple CLIs, want an iPhone remote, build MCP tools, or care about portfolio-grade Swift architecture.

Quickstart (5 minutes)

1. Install at least one supported CLI

pixel-agent uses your local CLI binaries — no API keys to configure inside the app. Install whichever you have access to:

Make sure claude, codex, or gemini is on your PATH (or in /opt/homebrew/bin, /usr/local/bin, ~/.local/bin, ~/bin) and that you've logged in to that CLI.

2. Install

Option A — Homebrew (recommended, ~30 seconds)

HOMEBREW_CASK_OPTS="--no-quarantine" brew install --cask ErkutYavuzer/tap/pixel-agent
open /Applications/PixelAgent.app

The HOMEBREW_CASK_OPTS env var bypasses macOS Gatekeeper for this install. pixel-agent is currently ad-hoc signed (Apple Developer ID + notarization is on the roadmap). Without it you'd need to manually xattr -d com.apple.quarantine /Applications/PixelAgent.app or right-click → "Open Anyway" in System Settings → Privacy & Security.

Tap source: ErkutYavuzer/homebrew-tap.

Option B — Build from source

git clone https://github.com/ErkutYavuzer/pixel-agent.git
cd pixel-agent
swift build -c release
swift test                              # 443 passing
./scripts/build-app.sh release && open PixelAgent.app

Requirements: macOS 14+ (Sonoma), Apple Silicon (arm64). Swift 6.0+ only for build-from-source. Intel universal2 build is on the roadmap.

3. (Optional) Pair with iPhone

In the Mac app, open the Pairing view → a QR code appears.
On your iPhone, build ios/PixelAgentRemote via xcodegen (cd ios && xcodegen generate && open PixelAgentRemote.xcodeproj) and run it.
Scan the QR. Connection persists across launches; LAN is preferred, Cloudflare relay is the fallback.

4. (Optional) Expose pixel-agent's tools to other MCP clients

pixel-mcp-server is a standalone executable. Point your MCP client at it:

{
  "mcpServers": {
    "pixel-agent": {
      "command": "/absolute/path/to/.build/release/pixel-mcp-server",
      "args": []
    }
  }
}

Tools that require the Mac app (dock badge, notify, dispatch_subagent, ui_*) only work when PixelAgent.app is running — they talk to it over a Unix socket bridge.

Stdio sanity check:

echo '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | swift run pixel-mcp-server

Feature tour

Multi-backend chat

Single mode picks one CLI; Dual mode runs two side by side so you can A/B answers or hand a task between models. Each backend has its own per-kind conversation history (conversation-claude.jsonl, etc.).

Per-backend model picker is in the toolbar — Anthropic aliases (opus/sonnet/haiku, always current) at the top, dated IDs below for pinning.

Subagent dispatching

A dedicated panel shows up to 3 parallel subagents with elapsed time, partial streaming output, and a cancel button. The same runtime is exposed as the MCP tool dispatch_subagent so other clients (or pixel-agent itself, recursively) can fan out work.

Per ADR-0019 → ADR-0024.

iOS remote dashboard

Three-tab TabView:

Chat — full chat with streaming + exponential backoff reconnect.
Subagents — see and cancel what's running on the Mac.
Mac Panel — change backend, model, plan mode from the phone; request a screenshot (zoomable); watch real CPU + RAM gauges (Mach HOST_CPU_LOAD_INFO).

Per ADR-0032.

Computer use with Set-of-Mark

PixelComputerUse is AX-first (Accessibility tree) with an OCR fallback toggle. The new piece is Set-of-Mark visual annotation — overlay numbered badges on a screenshot so a vision model can say "click #7", and pixel-agent maps the ID to the real element deterministically. No more "pixel coordinates please" workflow.

1. ui_query({ role: "AXButton", bundle_id: "com.app.foo" }) → [10 elements]
2. ui_screenshot({ target: "window_content", bundle_id: "com.app.foo", elements: <ui_query result> })
   → { png_base64, marks: [{ id: "1", element, frame_in_image }, ...] }
3. Vision model reads PNG + marks → "click #7"
4. ui_click({ query: { identifier: marks[6].element.identifier } })

Per ADR-0026 → ADR-0031.

MCP server — 14 tools

Tool	Type	Needs Mac app?
`get_clipboard`, `set_clipboard`	pure data	no
`get_current_time`	pure data	no
`get_active_app`	pure data	no
`get_lan_ip`	pure data	no
`dock_badge_set`	bridge	yes (Unix socket)
`notify`	bridge	yes
`play_sound`	bridge	yes
`dispatch_subagent`	bridge	yes
`ui_query`, `ui_click`, `ui_type`, `ui_screenshot`, `ui_resolve`	bridge (`PixelComputerUse`)	yes

Plan Mode is enforced: when active, only read-only ui_query/ui_screenshot/ui_resolve are allowed.

Architecture

graph TD
    App[PixelMacApp]
    MCP[pixel-mcp-server]
    Core[PixelCore]
    Backends[PixelBackends]
    Tools[PixelTools]
    Memory[PixelMemory]
    Mascot[PixelMascot]
    Remote[PixelRemote]
    LAN[PixelLAN]
    Subagent[PixelSubagent]
    MCPLib[PixelMCPServer]
    CompUse[PixelComputerUse]

    App --> Core
    App --> Backends
    App --> Tools
    App --> Memory
    App --> Mascot
    App --> Remote
    App --> LAN
    App --> Subagent
    App --> MCPLib
    App --> CompUse
    MCP --> MCPLib
    Backends --> Core
    Tools --> Core
    Memory --> Core
    Remote --> Core
    LAN --> Remote
    Subagent --> Core
    CompUse --> Core

10 libraries + 2 executables, each with its own XCTest target. Dependencies flow one-way toward PixelCore; cycles are blocked at SPM compile time. Swift 6 strict concurrency throughout (swiftLanguageModes: [.v6]).

Module	Responsibility
`PixelCore`	`ChatBackend` protocol, `ChatOptions`, `Message`/`StreamDelta`, `AgentContext` TaskLocal, `ToolArbiter`
`PixelBackends`	CLI subprocess wrappers (`claude`/`codex`/`gemini`), `CLIDetector`, `ModelCatalog`, `StreamJSONParser`
`PixelTools`	Native macOS toolkit: `DockBadge`, `SystemNotifications`, `SoundEffect`
`PixelMemory`	`ConversationStore` actor (JSONL append-only, per-backend isolation)
`PixelMascot`	12×12 ASCII sprite, 4 animation states, SwiftUI `Canvas` renderer
`PixelRemote`	`RemoteEnvelope` (Codable + ed25519 sig), `RelayClient`, `RemoteHost`, transport protocol
`PixelLAN`	Bonjour: `LANService`/`LANClient`, transport adapters, `FallbackTransport`, `MergeTransport`
`PixelSubagent`	Single-turn runner: `Budget`, `SubagentResult` enum, `SubagentRunner` actor
`PixelMCPServer`	`JSONValue`, `JSONRPCMessage`, `MCPServer` actor, `ToolRegistry`, bridge protocol
`PixelComputerUse`	AX bridge, pointer control, screenshot capture, Set-of-Mark renderer
`PixelMacApp` (exe)	SwiftUI composition root, `ChatView`, `PairingView`, `ControlSocketServer`, `SystemStats`
`pixel-mcp-server` (exe)	MCP stdio executable (3-line `main.swift`)

Full diagram + sequence flows: docs/architecture.md.

Architectural decisions (ADR)

Every major design decision is written down as an ADR. The full set is 32 documents covering monorepo layout, lifecycle, concurrency, transport, signing, MCP, subagents, computer use, and the iOS dashboard protocol.

Highlights:

ADR-0001 Modular SPM monorepo
ADR-0010 CLI subprocess backend (HTTP API rejected)
ADR-0015 ed25519 envelope signing
ADR-0016 + ADR-0018 MCP server expose + Unix socket bridge
ADR-0021 → ADR-0025 LAN-only mode (4 phases)
ADR-0026 → ADR-0031 Computer use (6 phases incl. Set-of-Mark)
ADR-0032 iOS dashboard remote control protocol

Plus a retrospective: v2 lessons — 14 patterns and 3 anti-patterns extracted from the predecessor codebase that informed v3.

Status

v0.2.30 (2026-05-25) · 675 tests passing · 32 ADRs · 10 libraries + 2 executables · Sprint 5 cross-platform parity shipped.

Recent highlights (full changelog):

Version	Date	Highlight	Tests
`v0.2.30`	25 May	Sprint 5 cross-platform parity — iOS connection-lost pulse, mascot subtle animations (bob/wobble/mouth-cycle/shake), drag-drop file context with halo, iOS conversation history viewer (4 new envelope types)	675
`v0.2.29`	24 May	Sprint 4 polish + persistence — EnvelopeType.unknown forward-compat, "Bu sohbete devam et" archive load, screenshot persistence (~/Library/Application Support/.../screenshots), connection-lost pulse animation, screenshot prompt prefill	631
`v0.2.28`	24 May	Sprint 3 persistent state + iOS parity — conversation history sidebar (NavigationSplitView), ⌘, Settings scene (4 tabs), iOS Ayarlar tab (4th), tool-call envelope events broadcast to iOS Mac Paneli	606
`v0.2.27`	24 May	Sprint 2 power-user touches — persistent connection pill, copy-last + per-message context menu, markdown/JSON export, composer focus halo + haptic, cap-reached banner, inline screenshots with SoM mark overlays	574
`v0.2.26`	24 May	Sprint 1 demo-ready foundation — MCP integration helper, empty state, Plan Mode tool list panel, markdown + code copy, ⌘N/⌘⇧P/⌘⇧M shortcuts, typing indicator, iOS config toast, retry banner, auth login launcher, subagent → chat	529
`v0.2.25`	23 May	iOS dashboard, real CPU metric (Mach `HOST_CPU_LOAD_INFO`), ADR-0032	443
`v0.2.16`	23 May	Set-of-Mark visual annotation (ADR-0031)	401
`v0.2.12`	23 May	PixelComputerUse + ToolArbiter implementation	315
`v0.2.11`	22 May	LAN-first iOS default + Bonjour TXT record	250
`v0.2.10`	22 May	Subagent UI panel, parallel cap=3	244
`v0.2.3`	22 May	ed25519 envelope signing + MCP server expose	162
`v0.1.0`	21 May	First release (6 sprints, iOS pairing, DocC)	91

Roadmap: see CHANGELOG → Unreleased for what's coming next.

Documentation

Architecture — full diagram + sequence flows
ADR index — 32 architectural decisions
v2 lessons — what informed v3
CHANGELOG
SECURITY
API docs (DocC): published to GitHub Pages by .github/workflows/docs.yml

Contributing

🐛 File a bug
✨ Suggest a feature
💬 Discuss an idea
🔒 Security: please follow SECURITY.md for responsible disclosure
PR convention: see pull_request_template.md

This is a personal portfolio project; scope is intentionally bounded (see CHANGELOG → Unreleased for the roadmap). Architectural proposals should reference an existing ADR or propose a new one — keep the trail.

License

MIT — see LICENSE.

Credits

Lessons from the predecessor pixel-agent2 codebase are at the heart of this project — especially the ToolArbiter resource mutex, TaskLocal scoping, and ephemeral subagent isolation patterns. The full extracted set lives in docs/architecture-decisions-from-v2.md.

Built with Swift 6 for macOS. Made in Türkiye 🇹🇷.

Name		Name	Last commit message	Last commit date
.github		.github
Sources		Sources
Tests		Tests
docs		docs
ios		ios
relay		relay
scripts		scripts
.gitignore		.gitignore
.swift-format		.swift-format
.swiftlint.yml		.swiftlint.yml
CHANGELOG.md		CHANGELOG.md
LICENSE		LICENSE
PROJECT_MEMORY.md		PROJECT_MEMORY.md
Package.swift		Package.swift
README.md		README.md
README_TR.md		README_TR.md
SECURITY.md		SECURITY.md

Navigation Menu

Repository files navigation

pixel-agent

What pixel-agent does

Why pixel-agent vs ...

Quickstart (5 minutes)

1. Install at least one supported CLI

2. Install

3. (Optional) Pair with iPhone

4. (Optional) Expose pixel-agent's tools to other MCP clients

Feature tour

Multi-backend chat

Subagent dispatching

iOS remote dashboard

Computer use with Set-of-Mark

MCP server — 14 tools

Architecture

Architectural decisions (ADR)

Status

Documentation

Contributing

License

Credits

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

pixel-agent

What pixel-agent does

Why pixel-agent vs ...

Quickstart (5 minutes)

1. Install at least one supported CLI

2. Install

3. (Optional) Pair with iPhone

4. (Optional) Expose pixel-agent's tools to other MCP clients

Feature tour

Multi-backend chat

Subagent dispatching

iOS remote dashboard

Computer use with Set-of-Mark

MCP server — 14 tools

Architecture

Architectural decisions (ADR)

Status

Documentation

Contributing

License

Credits

About

Topics

Resources

Security policy

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages