Architecture decision: scale-to-zero worker in Yandex Serverless Containers

For the sparse and irregular traffic expected at the start, the worker should be a scale-to-zero execution slot, not a permanently running per-user daemon. This makes Yandex Serverless Containers a good first runtime for the Codex-first vertical slice.

flowchart LR
    TG[Telegram webhook] --> CP[Go control plane]
    CP --> YDB[(YDB: runs, leases, quotas, outbox)]
    CP --> MQ[Yandex Message Queue: run_id]
    MQ --> TR[YMQ trigger]
    TR --> W[Serverless worker<br/>concurrency = 1]
    W --> S3[(Object Storage:<br/>inputs, workspace snapshots, artifacts, skills)]
    W --> YDB
    YDB --> OUT[Telegram outbox sender]
    OUT --> TG

Invocation contract

One invocation processes exactly one run_id for exactly one tenant (telegram_user_id / chat_id) and one subscription connection.

1. The Telegram handler persists the message and creates a run in YDB, then puts only the opaque run_id into Yandex Message Queue. The webhook returns immediately.
2. A YMQ trigger invokes the worker container.
3. The worker atomically claims a YDB lease/idempotency key and reserves the user's quota before starting AI work.
4. It fetches the chat-derived context manifest, encrypted subscription credentials, attachments, selected skills and workspace snapshot.
5. It materializes a unique ephemeral working directory and starts one bounded harness run inside the invocation: initially Codex (codex exec or a local Codex App Server process); later the same worker contract can have separate OpenCode/Hermes/Claude implementations.
6. Progress, usage snapshots and checkpoints are written outside the container. Final artifacts go to Object Storage; the result and Telegram outbox entry go to YDB.
7. Before returning, the worker terminates child processes and removes tenant credentials and workspace data. A warm container may be reused by the platform, so process-local cleanup is required even though durable state never depends on reuse.

sequenceDiagram
    participant T as Telegram
    participant C as Go control plane
    participant D as YDB
    participant Q as Message Queue
    participant W as Serverless worker
    participant O as Object Storage

    T->>C: update
    C->>D: persist update + create run
    C->>Q: enqueue run_id
    C-->>T: accepted / queued
    Q->>W: invoke (at-least-once)
    W->>D: claim lease + reserve quota
    W->>O: fetch inputs / workspace / skills
    W->>W: run harness, concurrency=1
    loop after each turn/tool boundary
        W->>D: checkpoint progress + usage
    end
    W->>O: upload artifacts / next workspace snapshot
    W->>D: commit result + quota + Telegram outbox
    W-->>Q: 2xx only after durable commit
    C->>T: progress/final result from outbox

Why this fits the cost model

Yandex charges Serverless Containers for invocations that make the application run; active execution is metered in 100 ms units. With no provisioned instances, the worker can scale to zero and cold starts become a latency trade-off rather than a fixed monthly compute bill. The current free tier also includes 1 million invocations, 10 GB-hours of RAM and 5 vCPU-hours per month, which is useful for an early low-traffic deployment, but should be treated as a current pricing condition rather than an architectural guarantee.

Do not configure provisioned instances for the MVP. Measure cold-start time separately from harness/model latency before deciding whether a small warm floor is economically justified.

Platform boundaries that shape the design

Current Yandex limits make this workable, but only with an externalized state machine:

• maximum request processing time, including cold start: 1 hour;
• requests over 10 minutes are treated as long-lived; Yandex warns they may still be terminated early and recommends asynchronous execution;
• writable root/temp filesystem is RAM-backed and is not preserved after the instance stops;
• one ephemeral ext4 disk of up to 10 GB can be attached, but it is lifecycle-scoped and must not be the source of truth;
• maximum RAM is 8 GB, maximum CPU is 4 vCPU at that memory size;
• request and response bodies are limited to 3.5 MB, so queue/webhook payloads must contain identifiers, not prompts, repositories or attachments;
• environment variables total only 4 KB, so a full Codex/Claude auth cache should be fetched/decrypted inside the worker rather than injected as a large environment blob;
• CPU is allocated while a request is being processed. A suspended warm instance is an optimization only: network connections may be terminated and correctness cannot depend on a persistent App Server process.

Recommended MVP timeout is 15–30 minutes per run, not the platform maximum. Runs expected to exceed 10 minutes must checkpoint at every harness turn/tool boundary and handle SIGTERM. Workloads that truly require more than one hour, a large repository, or a durable always-mounted filesystem should later spill over to a persistent/sandbox runtime such as Daytona or Modal rather than stretching this execution contract.

Queue, retries and isolation

Use a standard Yandex Message Queue with a Serverless Containers trigger and a dead-letter queue. The trigger is at-least-once: it deletes a message only after a successful invocation and makes it visible again after failure. Therefore:

• YDB lease + idempotent durable commit are mandatory;
• queue visibility timeout must exceed worker timeout plus cleanup/checkpoint margin;
• worker concurrency is 1; never batch different tenants into one harness process;
• duplicate delivery must either resume the same checkpoint or observe an already-completed run and return 2xx;
• credentials are scoped to (user_id, provider_account_id) and materialized only inside that invocation;
• each run receives a unique working directory and explicit Object Storage key prefix; no tenant path is accepted from user-controlled input.

Workspace and skills

Object Storage should not be treated as a writable POSIX workspace even if mounted. Use it as immutable/blob storage:

• download the exact workspace manifest/snapshot to the invocation's ephemeral disk;
• execute locally;
• upload changed files and artifacts under content-addressed keys;
• atomically advance the workspace manifest in YDB;
• package platform skills in the worker image (versioned with it); store user/team skills as versioned Object Storage bundles referenced from YDB and materialize only the allowed set for the run.

This also resolves the SQLite concern: local SQLite may be used only as disposable harness-internal state during one invocation. Product/session/run/quota state belongs in YDB. If a harness insists on a local database, restore it from a versioned encrypted snapshot and upload a new snapshot after a successful checkpoint; never share one SQLite file between concurrent workers.

Subscription quota accounting

Serverless billing and provider-subscription limits are different ledgers. For every run, persist:

• provider_account_id, harness and model;
• reservation time and released/consumed status;
• turns, wall-clock execution, retry count and tool calls;
• provider-reported token counts where exposed;
• remaining-limit/reset metadata where the CLI exposes it;
• normalized confidence: provider_reported, locally_measured, or unknown.

The scheduler must enforce both per-user fairness and per-subscription-account serialization/concurrency limits before enqueue/claim. If Codex/OpenCode/Claude does not expose an authoritative remaining subscription quota, we should not fabricate one: use local consumption estimates plus observed throttle/reset events, and surface that uncertainty in the admin view.

MVP decision

Proceed with one worker-codex Serverless Container, concurrency 1, zero provisioned instances, YMQ trigger, YDB leases/checkpoints/quota ledger, Object Storage snapshots, and an outbox-based Telegram response path. Add separate images (worker-opencode, worker-hermes, etc.) only after the execution contract is stable; this keeps the first image and cold start under control and avoids coupling the control plane to Python-heavy Hermes.

Official references:

• Serverless Containers limits
• Runtime environment and filesystem
• Mounting Object Storage and ephemeral disks
• Long-lived containers
• Termination notifications
• Yandex Message Queue trigger
• Pricing