Select the MCP gateway, execution-state, and artifact architecture

[aidanbruneel]

Question to answer

Which service boundaries and persistence backends best isolate the public MCP trust boundary from IMAGE's Docker-aware execution plane while supporting authenticated ownership, bounded long-running work, cancellation, horizontal scaling, artifact range access, and predictable retention?

Decision or next step this enables

Before implementation proceeds, document and approve the service boundaries, security model, deployment and network layout, private gateway-to-orchestrator API, job and experience lifecycle, persistence technologies, and migration plan.

Current understanding and key unknowns

Confirmed:

• The orchestrator mounts the Docker socket and dynamically discovers service containers.
• A pipeline request may fan out to multiple costly CPU/GPU services with timeouts around tens of seconds.
• The prototype stores MCP MP3s under /var/log/IMAGE/<uuid> and relies on a cron that may retain marked directories for 60 days.
• Existing follow-up history is process-local.

Current recommendation:

• Public MCP gateway with no Docker socket.
• Private typed orchestration API.
• Explicit job/experience records bound to authenticated principals.
• A durable queue/state store and a separately governed artifact backend.

Key unknowns:

• Required throughput/SLOs and whether current deployment needs horizontal scaling immediately.
• Whether Redis plus local/object storage is acceptable operationally.
• Whether the Tasks extension is supported widely enough to be primary or must supplement a synchronous fallback.

Proposed approach

1. Inventory data flow, privileges, trust zones, retention, failure modes, and deployment constraints.
2. Threat-model authentication, SSRF, cross-user access, prompt/tool injection, capability URLs, Docker socket exposure, queue abuse, denial of wallet/service, and log leakage.
3. Compare embedded adapter, separate gateway, and fully separate execution API options.
4. Prototype the private call, cancellation propagation, an owned experience record, and authorized MP3 range read.
5. Model capacity/backpressure and define SLOs, maximum work, TTLs, and degraded behaviour.
6. Publish ADRs, diagrams, storage lifecycle, failure-mode table, and migration plan.

Completion criteria

• A reviewed threat model identifies trust boundaries, assets, actors, abuse cases, and mitigations.
• ADRs choose the gateway/orchestrator boundary, private API, queue/state store, artifact backend, and canonical-origin strategy.
• Ownership and retention are defined independently from MCP transport sessions and legacy request logging.
• Cancellation, retry, duplicate request, partial failure, restart, and overload behaviour is defined.
• A cost/capacity estimate and initial SLOs are documented.
• The decision is validated by a thin prototype or clearly cited operational evidence.

Evidence and related work

• Orchestrator Docker access: https://github.com/Shared-Reality-Lab/IMAGE-server/blob/main/orchestrator/README.md
• MCP security guidance: https://modelcontextprotocol.io/docs/tutorials/security/security_best_practices

Checks

• I kept this Investigation focused on one bounded question with a clear stopping condition.
• I will track resulting production implementation as separate Bugs, Features, or Tasks.
• I removed secrets, credentials, personal information, and confidential data.