Provide accessible IMAGE experiences through MCP

[aidanbruneel]

Outcome

Provide a production-supported remote MCP service through which authenticated agents can request accessible IMAGE interpretations, consume text and audio experiences, and ask follow-up questions using explicit, owned handles, without exposing IMAGE's internal Docker execution plane or weakening the existing /render workflow.

Context and value

The mcp-endpoint feature branch proves that a high-level IMAGE tool, text/audio conversion, audio artifacts, and an accessible MCP App are feasible. It is intentionally still a prototype: MCP is embedded in the Docker-aware orchestrator; protocol support was changed from the 2026 SDK v2 to a v1 SDK after host-compatibility problems; authentication is an optional fixed bearer token; artifact links are bearer capabilities; file download protection is incomplete; follow-up state is not integrated; and there is no production admission, ownership, quota, rollout, or operating model.

IMAGE workloads are expensive, long-running, privacy-sensitive, and primarily serve blind and low-vision users. Moving from a demonstration endpoint to a supported agent interface therefore requires coordinated product-contract, protocol, architecture, security, execution, storage, accessibility, testing, documentation, and operations work. This cannot be completed safely as one route-level Feature.

Scope and expected workstreams

• Establish supported agent journeys, tool/resource contracts, and success measures.
• Establish a tested client/protocol/SDK compatibility policy, including a bounded legacy strategy.
• Decide and document service, trust, state, job, and artifact boundaries.
• Refactor a transport-neutral IMAGE application core while preserving /render behavior.
• Add a separately deployable MCP gateway and private typed orchestration API.
• Implement secure image acquisition, OAuth authorization, user ownership, quotas, and abuse controls.
• Implement durable execution, cancellation, explicit experience handles, artifact lifecycle, and follow-up interaction.
• Provide portable text/structured results and an accessible, versioned audio MCP App.
• Add end-to-end observability, privacy controls, conformance/security/load/accessibility verification, onboarding documentation, and staged deployment/rollback.

Success criteria

• Supported ChatGPT and Claude clients can discover the production endpoint, authenticate through a standards-based flow, call the advertised tools, and consume documented fallback results.
• A user can submit each supported image source and receive schema-valid text and, when produced by IMAGE, accessible audio/segment results with a documented expiry.
• A user can ask a follow-up question about an owned, unexpired experience; another user cannot read or continue it.
• The public gateway has no Docker socket and cannot call individual preprocessors/handlers directly; it reaches execution only through the private orchestration contract.
• Every admitted operation has an authenticated principal, correlation/trace ID, bounded resource use, cancellation/timeout behavior, and auditable lifecycle without logging image contents, tokens, or artifact secrets.
• Overload is rejected or queued predictably; it cannot create unbounded concurrent GPU work.
• Artifact access is authorized, supports required media range behavior, expires/revokes predictably, and is not extended accidentally by the legacy request-retention mechanism.
• Protocol conformance, authorization, SSRF/input security, multi-user isolation, end-to-end, load, and regression suites pass in CI or a documented release environment.
• The MCP App passes automated accessibility checks and documented keyboard, NVDA, VoiceOver, zoom, forced-colors, dark-mode, narrow-layout, playback, segment, cancellation, and expiry acceptance tests.
• Operators have readiness checks, dashboards, alerts, capacity guidance, retention controls, incident/rollback runbooks, and a successful staged production exercise.
• New contributors can run the gateway with stub IMAGE services, understand the architecture and MCP concepts, execute conformance tests, and add a backward-compatible tool/result field from the documentation alone.

Constraints and out of scope

• Preserve the existing public /render, /render/preprocess, and /authenticate behavior unless a separate issue explicitly changes it.
• Keep internal preprocessor/handler names, Docker discovery, and route topology out of the public MCP contract.
• Do not rely on MCP transport sessions for application identity, ownership, job state, or conversation state.
• Do not use an optional shared static token as the production authorization design.
• Do not support arbitrary URL ingestion until the SSRF/redirect/DNS/size policy and tests are complete.
• Do not expose haptic, tactile, SVG-layer, or other rendering types until a target host has a faithful accessible representation and acceptance tests.
• Do not add deprecated MCP roots, sampling, or logging features to new production code.
• A local stdio/DXT distribution, a general-purpose IMAGE agent, billing, and redesign of individual ML models are outside this Epic unless later attached as explicit sub-issues.
• Production rollout may not precede the security, ownership, accessibility, and overload gates even if a prototype works in one host.

Existing and related work

• Prototype branch: https://github.com/Shared-Reality-Lab/IMAGE-server/tree/mcp-endpoint
• Prototype handoff/design note: https://github.com/Shared-Reality-Lab/IMAGE-server/blob/mcp-endpoint/20260811_mcp-endpoint_plan.md
• Branch comparison: main...mcp-endpoint
• Current orchestrator architecture: https://github.com/Shared-Reality-Lab/IMAGE-server/tree/main/orchestrator
• MCP 2026-07-28 specification: https://modelcontextprotocol.io/specification/2026-07-28
• MCP TypeScript SDK v2: https://ts.sdk.modelcontextprotocol.io/v2/
• MCP Apps: https://apps.extensions.modelcontextprotocol.io/

Checks

• This outcome requires multiple independently tracked issues rather than one Feature or Task.
• I will use GitHub sub-issues rather than maintaining a duplicate child-issue checklist in this description.
• I removed secrets, credentials, personal information, and confidential data.