0017. Simplified managed stack architecture

Status: superseded by the Stack package architecture Date: 2026-08-17

This decision record describes an earlier managed-stack architecture. The current package architecture and lifecycle contract are maintained in packages/stack/ARCHITECTURE.md.

Decision

Managed local-stack coordination has one implementation and one durable state document. The stack package owns identity, sticky ports, launch selection, lifecycle transitions, and state writes. A detached Supervisor holds the exclusive ownership lease only while the stack is running or an admitted operation is in flight, while stop-time cleanup remains unproven, or while an unproven destroy leaves durable intent destroying. Public handles communicate with it through same-release Effect RPC and a small release-stable maintenance protocol for probe and stop. The package runs independently of the CLI and accepts normalized StackConfig values without reading config.toml. CLI integration belongs to M5 in separate PRs, including Functions command wiring. Future CLI handlers will call this facade without maintaining a second metadata or PID-based liveness path.

The public API has two deliberate entrypoints: Effect-native operations are available from @supabase/stack/effect, while the package root exposes the Promise handle and root Promise functions (createStack, openStack, findStack, listStacks, and inspectStack). Test callers use the public createTestStack helper from @supabase/stack/testing; it owns a unique temporary project root and exact-identity cleanup through await using, while using the same managed state root as ordinary package callers. All default callers therefore coordinate automatic ports through one registry; helper project roots and identities remain isolated. Temporary test stacks are excluded from listings scoped to another project root because their project roots differ, but remain visible to unfiltered package listStacks(). A failed destroy retains its exact project root and managed state for recovery.

The managed document is private, unreleased state under the user-level managed root. It is not a repository contract, service registry, compatibility facade, or independently versioned database schema. Platform entrypoints only provide filesystem, path, process, HTTP, and control-transport services.

Stack identity is the SHA-256 digest of a length-delimited tuple containing the canonical project root, the current Git branch context, and the stack name. The branch context is the full symbolic ref or detached; projects outside Git use ordinary-workspace. This gives separate identities to worktrees, branches, monorepo project roots, and named stacks while keeping identity resolution read-only. Project relocation is not supported: a moved project resolves to a new identity.

The managed directory supplies the stack ID; state.json stores the identity tuple and validates its digest against that directory instead of duplicating the ID. control.json stores the owner session, lease port, and RPC release. The control endpoint is derived from the directory ID, lease port, and runtime environment. The ownership lock remains separate: it protects acquisition before the owner publishes its ready control metadata.

Port assignments describe usable routes and workload bindings. Disabled capabilities do not reserve public listener ports, and the Functions inspector reserves a private port only when debugging is enabled. Service settings belong in the materialized definition only when the local runtime implements them; hosted-only database network restrictions, SSL enforcement, vault, REST auto-exposure, and Storage Analytics settings are excluded. API TLS is locally meaningful but unsupported by the stack gateway, so it is excluded too. Analytics' Vector port is assigned by the runtime rather than stored as a service setting.

Disabling a capability releases its automatic port assignment; re-enabling it may select a new port.

Stack handles are lightweight identity-scoped clients. Creating or opening one does not launch a Supervisor. Successful stop stops services in dependency order, removes every ephemeral runtime resource, persists stopped state, delivers its response, then closes control and releases ownership. The caller waits for both response and lease release. A stopped stack therefore consumes no live process, container, network, socket, listener, or gateway. Status and retained logs are read from durable files only when owner metadata and the ownership lock are both absent. The same handle lazily launches a fresh Supervisor on its next start.

A Supervisor launched for a mutation also exits when that mutation cannot leave running intent behind. In particular, a failed start before the running state is committed releases its control endpoint and ownership lease after reporting the failure. A successful start keeps the Supervisor alive; a sequential idempotent start returns the current status without resetting lazy activation or publishing a synthetic starting transition.

If exact runtime cleanup cannot be proven, status remains stopping and the owner stays available for a retryable stop(); a cleaned stopped stack never retains that owner.

There is no durable generation counter and runtime resources are never adopted across owner sessions. The exclusive stack lease and ownerSessionId identify the only writer and current ephemeral resources. Explicit start, stop, or destroy first removes exact stack-owned remnants and creates fresh resources; failure to complete or validate cleanup fails closed. Persistent data, sticky ports, secrets, definitions, logs, and artifacts remain identity-scoped.

An acquire operation requires persisted running state and prevalidates every exact public claim and retained public or private assignment still requested. Exact values may be shared only with other stopped exact owners; a desired-running owner blocks regardless of observed process liveness. Conflicts never move retained claims. One registry transaction binds public and private claims before one state commit; successful public sockets remain held and are adopted directly. Temporary private TCP listeners remain held until commit and then close, so a private workload gap remains possible. Fresh automatic claims draw from 20000..32767 with a start derived from the stack's project root, identifier, and listener key, and stride 257, making up to 64 bounded EADDRINUSE/EACCES attempts per newly selected binding while skipping durable sibling claims. Sticky values are stable: they do not migrate on their own, but a stopped stack's own owner startup re-plans an endpoint whose saved port differs from the current configuration while it alone holds the stack's lease: it saves the requested endpoint intents, releases the changed endpoints' registry rows, and claims the new ports through its normal endpoint binding. A taken fixed port fails the start with the usual port conflict; the saved intent stays as requested and nothing is rolled back. The requested creations reach the owner through a private, short-lived file the owner deletes after reading, never through argv. A concurrent start attaches to whichever owner wins the lease instead of re-planning again. Failed acquisition preserves the previous successful arrays, including claims removed or reconfigured by the new definition, and releases attempted sockets; those arrays change only after the next successful acquisition. Candidate retries while holding the registry lock use no sleep, and no workload startup runs under the lock. This preserves the fail-closed registry lock and does not introduce a process-liveness lock redesign.

Before a native cold start creates anything, it verifies every retained public and private port is bindable. PostgreSQL lock evidence containing a live or unclassifiable owner PID fails closed; a lock naming a process that no longer exists is left for PostgreSQL's own stale-lock recovery. A live stop/start composition preflights configuration without trying to bind ports that the current Supervisor intentionally owns. Cold recovery never adopts those resources. An actual bound wildcard may cover the internal API and avoid a duplicate listener or gateway adoption; IPv6 wildcards explicitly use dualstack. A remaining separate internal bridge bind can still fail once its topology is discovered. The reservation policy does not guarantee exclusion from custom dynamic or excluded ranges, or from unrelated processes that occupy sticky numbers while the stack is stopped. Focused port coverage runs on macOS, Linux, and Windows in CI.

Every managed document records one concrete runtime selection. Native and container runtimes never mix. For a new stack, an omitted runtime checks that each container engine in turn, Docker then Podman, resolves the same engine target the stack owner pins and probes it with a 10 s timeout; it selects the first reachable engine and native otherwise, where native artifacts are supported. When Docker is skipped, the created Effect handle carries a notice that the selected runtime (Podman or native) is persisted for that stack, so a later switch to Docker requires destroying the stack or choosing a new stack name; callers such as stack start decide whether to print it. Native as root runs only PostgreSQL as an unprivileged user. Existing state is reused without probing. Callers may explicitly select native, Docker, or Podman without fallback. See ADR-0025 for the daemon probe and its interaction with ephemeral Postgres. Persisted state records the resolved exact engine. Capability releases and workload artifacts are persisted as exact version pins (including their concrete native release and container image) rather than ranges or floating tags. Services with derived low-memory image profiles inherit those same service-level runtime.env defaults in native artifacts, and explicit process environment values take precedence. Smoke-only environment files are excluded from packaged profiles. The artifact release is the boundary for service startup defaults; the stack package owns runtime selection and lifecycle around the selected artifact.

Read-only discovery never acquires ownership. Owner metadata points to a Unix domain socket on POSIX or a named pipe on Windows, and the ownership lock is the single-writer authority. Offline status and retained logs are available only when both metadata and lock are absent. Mutations use the live owner or acquire the lease and launch a fresh Supervisor. Ambiguous ownership fails closed.

The stable maintenance protocol contains only probe and stop. Same-release callers use Effect RPC for status, logs, activation, start, and destroy; explicit artifact preparation runs directly in the caller scope and never uses the Supervisor. An incompatible owner can always be stopped, after which the caller may launch the current Supervisor and start again. The first admitted lifecycle operation runs to completion; concurrent lifecycle mutations fail immediately with a conflict rather than joining or queueing.

Capability modules own the new stack's defaults. All capabilities are enabled by default, including Pooler, with image transformation available through Storage's imgproxy workload and Vector included with Analytics. The CLI preserves explicit enablement overrides while leaving omitted values to the package; the shared CLI configuration's defaults remain unchanged for other commands.

PostgreSQL is the only eager capability by default. Start prepares and launches only the eager dependency closure, so the PostgreSQL readiness barrier remains the default startup path. Capabilities configured for lazy activation start when traffic first reaches their stable gateway route. Lazy artifact preparation is controlled by the top-level StackConfig.preparation setting, which defaults to "background". After a successful start, background mode prepares all enabled lazy artifacts with bounded concurrency without launching their processes. That work is one Supervisor-owned session task: stop and destroy cancel and await it, foreground activation joins shared preparations, and completed cache entries remain available without automatic pruning. "on-demand" skips the background task for callers that want full lazy preparation. Eager capability activation remains independent of this setting. That runtime session owns the shared preparation fibers.

Lazy traffic capabilities may retire independently after their configured idle timeout (see the package traffic stopping documentation). The traffic policy is per capability: supported lazy capabilities default to 60 seconds, false disables retirement, eager capabilities never retire, and unsupported capabilities remain opted out. Activity covers in-flight HTTP requests and streams plus open WebSocket or TCP connections; idle HTTP keep-alive sockets do not count. The Supervisor serializes workload retirement with activation and lifecycle operations, and fences new requests for the retiring capability. Unrelated ingress continues; dependencies stay running while their dependants are active, and Studio with pg-meta retires together. Retained data and listeners let the next request wake the capability and restart its workloads. An unproven cleanup marks the participating capabilities as failed and fences new activation across the stack. This is an operation-level result; the workload ledger tracks resources still requiring removal. Unrelated healthy capabilities retain their observations. An explicit stop retries the retained ledger. A committed destructive intent uses a separate destroy-required recovery state, projects destroying, and accepts only a destroy retry until durable state removal has completed. Public status exposes the recovery operation and reason separately from capability observations. CLI text includes those diagnostics and targets recovery commands to the selected stack. A shared cleanup failure describes the operation's incomplete cleanup, even when workload removal succeeded.

In both modes, lazy activation prepares the requested dependency closure with bounded concurrency before starting its workloads. Explicit stack.prepare(...) remains available as a cache-only warmup for callers that want to prepare selected artifacts while stopped or running. The preparation setting is persisted in the stack definition and survives opening and restart; changing it for a running stack follows the existing stop-before-change configuration rule. Preparation never creates runtime resources; ArtifactStore publishes cache entries in the caller's scope, and runtime-driver cleanup closes transfer resources. Caller-owned explicit preparation cancellation only stops that caller's unfinished transfers while completed cache entries remain.

Preparation uses bounded Effect concurrency to overlap transfers with native decompression and archive subprocesses. An Effect RPC worker pool would add coordination without improving this existing native and subprocess work: decompression already uses Bun's native worker pool, while archive listing and extraction run in subprocesses. Revisit this only if measurement identifies a CPU-bound bottleneck.

Packaged first boot preserves the artifact's ordered, same-role PostgreSQL script groups and initializes the database password before migrations; Realtime's combined configuration evaluation remains owned by its artifact. The stack package batches its own PostgreSQL bootstrap round trips and releases the one-shot bundler helper after each build.

Live status exposes the current session's artifact preparation through its artifacts array. Entries identify the workload and capability and move through queued, preparing, downloading, ready, and failed; the latter carries an actionable error for a later retry. preparing includes validation, transfer verification, and extraction around the download. Artifact status is observational and does not imply that the capability's process is running. During stopping or destroying, the owner may continue to expose active preparation until teardown clears it. Offline status supplies no live artifact states and does not scan the cache to recreate a progress snapshot.

Explicit stack.prepare(...) accepts a synchronous onProgress callback for caller-owned preparation. It receives the same phase values, including ready when an artifact is available while its capability remains dormant. The callback's transfer is local to that invocation and is not reconstructed by a separate status request.

Within a lifecycle operation, workloads start as soon as their dependencies complete, with at most four active starts. Dependency waits do not consume a start permit. Selected artifact preparation runs concurrently with this launch; each workload joins its own shared preparation before resolution. Container setup serializes only exact shared-network and volume identity establishment; image pulls, startup migrations, container creation, readiness checks, and long-lived followers run outside that setup boundary.

Native workloads use a two-minute readiness budget because a cold process can spend more than 30 seconds loading shared libraries before serving requests. Container workloads retain the 30-second budget, while PostgreSQL uses its configured health_timeout; every probe remains bounded and returns as soon as its endpoint is healthy.

Runtime input materialization belongs to the workload startup operation. The Supervisor serializes lifecycle operations with cleanup, so the input owner caches only completed values and leaves unfinished resolution in the startup operation's scope. One shared-input gate serializes materialization among concurrent workload starts. Interrupting native startup therefore interrupts OIDC fetches and file materialization; successful Pooler and Vector files remain owned until the runtime cleanup boundary removes them.

Artifact cache trust begins only after a candidate has passed checksum and runtime-path validation, had its metadata written, and been atomically published. Exact-version artifact metadata is authoritative on a cache hit: the store revalidates its format, key, SHA-256 shape, ordered required paths and kinds, containment, and executable shape without fetching an upstream digest. On a cache miss, the source resolves the trusted digest once; downloaded bytes are verified before decoding or extraction, then structurally validated and atomically published. The cache is user-owned and immutable by contract; manual post-publication changes may execute successfully or fail later at workload start. A partially published or malformed entry is never trusted merely because its directory exists.

Effect failures remain operation-specific tagged errors (for example, configuration, preparation, gateway, port, and lifecycle failures), preserving typed recovery and actionable context at the Effect boundary. The Promise facade translates those failures into native rejections only at its outer edge.

Why this replaces ADR-0015

ADR-0015 proposed exported contract fixtures, a repository boundary, and parallel persistent adapters without shipped producers or consumers. Keeping them would preserve a second architecture. Its identity and lifecycle intent survive only where implemented by the stack package and Supervisor.

Testing decision

Tests follow consumed boundaries:

• stack integration covers identity, sticky ports, durable lifecycle, ownership, stale-owner recovery, and interrupted cleanup;
• supervisor integration covers detached ownership, RPC, stop, destroy, retirement, and wake-up; and
• one shared stack-package E2E journey runs in native, Docker, and Podman modes, starts with PostgreSQL alone, activates every other service through realistic traffic, verifies cross-service behavior, then exercises stop/start and retained offline observability.

Future CLI handler integration will cover only argument translation, output, telemetry, and calls into the stack facade. Private implementation tests are retained only where public scenarios cannot make a branch observable.

Consequences

The architecture has one source of truth for lifecycle state and one owner for ephemeral resources. Refactors update the package facade and its real consumers together. The private document format may change with the current build, while destructive cleanup and ownership remain explicit safeguards.

Supervisor state transitions return {snapshot, notifications, reconcile}. The Supervisor commits the snapshot, performs any requested reconciliation, and then delivers notifications while holding admission. Every transition, including intermediate workload readiness, rollback, and cleanup, follows this commit → reconcile → notify contract, so observers cannot receive completion before state and retirement timer decisions are settled.