feat(threads): run an isolated application in a dedicated worker thread

[kriszyp]

Summary

First piece of tier 2 of #642: an application whose root-config entry carries isolated: true runs in a worker thread of its own that loads no other application, is restarted on its own, and is reachable only through its own application-named UDS mirror.

The invariant this enforces: an isolated application is loaded by exactly one thread, and that thread loads nothing else. Process globals, process.env mutation and a restart are therefore scoped to it, which a shared thread cannot offer. Branched databases (#2352, #2426, #2517, #2523) supply the data half; this is the start of the runtime half, per the design note on the isolated-app-threads worktree.

What changes

• Declaration. isolated: true on an application's root-config entry, alongside host, urlPath and branchedDatabases; validated as a boolean in assertApplicationConfig, accepted by deploy_component and carried through a package redeploy's rewrite of the entry (so a redeploy cannot silently move the application onto the pool).
• Placement, before import. Both loading passes decide per application whether this thread loads it, before any of its modules are imported: directory applications under componentsRoot, and root-config package: applications loaded as sub-components of the root. A dedicated worker loads only its own application (core plugins still load everywhere; root-level package: entries count as applications, see decisions); pool workers and the main thread load only the non-isolated ones. With threads.count: 0 there is no thread to give it, so it fails closed as a load failure rather than silently sharing the only thread.
• Dedicated workers. startHTTPThreads starts one http worker per admitted isolated application, numbered past the pool so no pool-only duty (worker 0's startup log and last-will replay, the last worker's cleanup) lands on it, and sized against the total worker count. Per-application singletons that used to key on worker 0 (the scheduler's job activation, the data loader) now key on isApplicationPrimaryWorker(applicationName): pool worker 0 for shared applications and for root-level plugins, the dedicated worker only for its own application, so nothing runs twice. A caching table's sourcedFrom subscription runs where the defining application's code runs (pool worker 0, or the dedicated worker). Per-store maintenance is owned per store: TTL scans, storage reclamation and audit cleanup (ownsStoreMaintenance) and expiration eviction (ownsStoreExpiration) stay with the last, respectively first, pool worker for the shared stores every thread opens, while a dedicated worker owns them only for its own branch stores, registered by openBranchDatabase before the open so the table load already sees the ownership. One exception: a TTL that application code configured at runtime (sourcedFrom options, setTTLExpiration from the app) is scanned by the dedicated worker even on a shared store, since no other thread has that configuration. system_information threads carry application for the dedicated ones.
• Admission fails closed, and tells the caller. An isolated application gets a worker only if that worker could be reached: http.securePort set and tls.unixDomainSockets on, and its mirror path within the platform's socket-path limit. threads.maxIsolated (default 8, validated as an integer) caps the count; applications already running keep their place, so adding an entry at the cap refuses the newcomer rather than evicting a running one. A refused application is recorded as a failed component and loaded nowhere, never downgraded to the pool, and deploy_component with isolated: true checks the same admission up front and answers 409 instead of "successfully deployed". isolated is accepted only on package deployments (a payload deploy has no root-config entry to carry it), and a package redeploy that omits it keeps the existing value rather than silently moving the application onto the pool.
• No public ports. A dedicated worker binds none of the shared ports on either the Node or the Bun path: with SO_REUSEPORT the kernel would hand it connections for every other application. It binds only its own per-thread UDS mirrors (never a global domain socket such as the operations API's), named app-<name>-<port> with every UTF-8 byte outside [A-Za-z0-9._-] as a fixed-width %XX (injective; cannot alias a pool socket), whose metadata publishes application and applicationHosts separately from certificate coverage, for the proxy to route by (a later PR on host-manager/symphony consumes it).
• Scoped restarts. restartWorkers takes an application scope: undefined restarts the shared pool only, a name restarts that application's dedicated worker, '*' restarts all. deploy_component/drop_component of an isolated application restart only its worker; of a shared application, only the pool; a deploy restarts only the pool whenever the application has no dedicated worker running (a flip in either direction, or an earlier flip persisted without a restart), since the reconcile starts or stops the moving application's own worker; the scope comes from the running topology, not the config alone. The scope crosses the worker-to-main ITC hop and the rolling-restart job in one wire form (empty string = pool, a name, absent = all, decoded exactly once in restartService), so a shared drop or redeploy executed on a worker still leaves dedicated workers alone, and restart_service still restarts every http worker. To application code a dedicated worker is its application's only worker: server.workerIndex is 0 and server.workerCount is 1 there, while the raw thread getters keep node-wide duties on the pool. Per-thread listeners an isolated application opens with server.socket() publish their secure mirror under the app- name too. After every root-component reload the main thread reconciles dedicated workers with the config and the component directories (both componentsRoot and the <rootPath>/components install location): starts one for a newly isolated application (and does not restart it again in the same call; a start failure is recorded as a failed component), and stops every worker carrying an application that is dropped or no longer isolated, a crashed one's booting replacement included. The stop is the same graceful path the rolling restart uses: shutdown, the worker's drain extension honoured, then the platform-safe force (FORCE_EXIT on Bun, terminate() on Node), and it is awaited, so a following branch removal never races a worker that still holds the store. A dedicated worker that exhausts its crash restarts frees its slot so the next deploy starts a fresh one. The deploy path refreshes the main thread's cached config before reading the previous isolated value and after rewriting the entry, so back-to-back deploys without a restart see each other; a rolling redeploy carries the same scope through restart_service. To application code a dedicated worker is its application's worker 0: source.subscribeOnThisThread receives index 0 there, so caching tables subscribe to their sources.

Verification

• Unit (unitTests/server/threads/isolatedApplications.test.js, unitTests/components/componentLoader.test.js): placement decision table, socket-name injectivity including the non-ASCII collisions the review found, config validation, route metadata, the unreachable-worker refusal, the loader skipping an isolated application on a non-owner thread, and failing it closed when the main thread is the only worker. Mutation checks (each test fails with its fix removed): the loader skip, the fail-closed branch, the socket-name encoding, the route metadata, and the owner rule.
• Integration (integrationTests/components/isolated-application.test.ts, two fixtures, TLS + UDS mirrors on): with a two-worker pool and one isolated application, exactly one dedicated worker appears; the shared port serves the shared application and never the isolated one; the isolated application answers 200 through its own app-isolated-app-<port>.sock mirror whose metadata names it and its host; dropping the shared application replaces every pool worker and leaves the dedicated thread id unchanged; dropping the isolated application stops its worker and does not restart the pool, and a second isolated application's dedicated thread survives that drop.
• The integration suite is skipped on Windows (no UDS mirrors, so admission refuses a dedicated worker there by design) and under the Bun runtime (listener path not yet exercised; a first Bun run surfaced a readonly property throw in the mirror's PROXY-protocol handler worth a separate look).
• Lint by exit code; tsc --noEmit clean. Two unrelated thread unit files (processGroupReclaim, resolvePreload) fail identically on main in this environment.

Decisions taken (reviewers flagged these as open)

• Root-config package: entries are applications. A dedicated worker loads none but its own, which also excludes root-level extension packages installed that way (a global auth middleware, say), because config cannot tell an extension entry from an application entry. An isolated application declares the extensions it needs in its own config.yaml, which is the documented route. If a root-level "load everywhere" marker is wanted, that is a small follow-up.
• A pre-started replacement of a dedicated worker that dies before ready leaves a dead socket path. The incumbent keeps running but its mirror path was rebound by the failed replacement; pool sockets have siblings, a dedicated one does not. Generation fencing (design note) is the fix.
• A flip from isolated to shared briefly serves nothing. The dedicated worker is stopped before the pool's overlapping roll loads the application; closing that gap needs generation fencing (design note).
• Admission is checked per deploy, not atomically across concurrent deploys. Two simultaneous deploys of new isolated apps can both pass the threads.maxIsolated check; the loser is refused and recorded as failed at the next reconcile.
• A newly admitted worker that fails to start is a failed component, not a failed deploy. deploy_component has answered by then; get_component_status shows the failure.

Not in this PR (tracked in the design note)

Proxy route composition from applicationHosts (host-manager/symphony); operations-API host scoping (decided: main executes app-scoped operations, custom operations rejected on the ops API); log read isolation; a topology-wide memory budget beyond the admission cap; generation fencing across a dedicated worker's replacement; an integration case for a package:-deployed isolated application (the placement gate for that pass is the same function the directory pass uses).

🤖 Generated with Claude Code

_{Review-Coverage: authored=claude; ran=codex,cursor-grok; blocked=gemini(permission-denied),domain(exit-1); declined=cursor-composer; rounds=17 @ 39d28d6}

_{Human-Review-Need: 4 @ 39d28d6}

[gemini-code-assist]

Code Review

This pull request implements isolated applications, allowing configured applications to run in dedicated worker threads that are reachable only via Unix Domain Socket (UDS) mirrors. The changes span worker management, socket routing, and deployment validation. The review feedback identifies a style guide violation regarding the node: prefix for the path import, and a bug in server/threads/socketRouter.ts where heapShareCount is calculated inconsistently inside a loop during worker reconciliation.

[gemini-code-assist]

The import of the path module does not use the node: prefix, which violates the repository style guide.

Suggested change

	import { join, basename } from 'path';
	import { join, basename } from 'node:path';

References

1. Node builtins must use the 'node:' prefix. ^(link)

[gemini-code-assist]

The heapShareCount parameter is calculated as poolSize + isolatedSlots.size + 1 inside the loop. Since isolatedSlots.set(application, slot) is called immediately after, isolatedSlots.size increases with each iteration. This causes different workers started in the same reconciliation pass to receive inconsistent heapShareCount values. It should consistently be poolSize + wanted.size for all newly started workers.

Suggested change

	const slot = startHTTPWorker(nextIsolatedIndex++, poolSize, application, poolSize + isolatedSlots.size + 1);
	const slot = startHTTPWorker(nextIsolatedIndex++, poolSize, application, poolSize + wanted.size);