Research note: YDB changefeeds as a future wake source
Decision for RUNTIME-01: keep the explicit post-commit YMQ wake envelopes. Do not make YDB CDC a dependency of this implementation.
YDB CDC is technically attractive for outbox wake-up generation:
- a changefeed is written only after the table transaction commits;
- records for the same primary key preserve order;
KEYS_ONLYmode can expose only the durable outbox key;- the changefeed is backed by a YDB topic and can use topic auto-partitioning.
Sources:
- YDB Change Data Capture concepts
- YDB changefeed DDL and
KEYS_ONLY/ topic auto-partitioning options - YDB CDC consumer workflow
The current Yandex Serverless Containers trigger catalog, however, has triggers for Yandex Message Queue, Data Streams, Object Storage, Cloud Logging, and several other services, but does not document a trigger for a YDB topic/changefeed:
Therefore CDC would still require one of these bridges:
- an always-running YDB topic consumer that invokes the container or republishes to YMQ;
- a custom scheduled relay, reintroducing polling;
- a verified YDB-topic → Data Streams/YMQ integration that is not currently documented as a native trigger path.
Options 1 and 2 conflict with the sparse serverless cost goal, while option 3 cannot be treated as available without a working provider/API spike. CDC also adds write/storage overhead and an additional retention/consumer-offset failure mode.
Revisit criteria
Re-open this design only when at least one of the following is true:
- Yandex Serverless Containers exposes a native YDB topic/changefeed trigger;
- YDB Serverless provides a documented push subscription;
- a measured relay can scale to zero and costs less than explicit YMQ publication.
A future spike should compare end-to-end latency, idle cost, duplicate/lost-event behavior, IAM, topic partition growth, retention expiry, and operational recovery against the current YDB outbox + deterministic YMQ hint design.
Implementation status — event-driven outbox wake-ups
Implementation is ready in MR !24, head commit 718ff23.
Implemented
- Added payload-free, deterministic
wake.dispatchandwake.telegramenvelopes. They contain only the tenant and durable outbox identifier. - YDB outboxes remain the source of truth. Consumers resolve wakes through tenant-scoped primary-key point reads.
- Missing, duplicate, and already-terminal wakes are acknowledged as successful no-ops.
- Dispatch domain blocks use bounded exponential retries and are then parked; transport/system errors remain subject to YMQ retry/redrive and DLQ behavior.
- Duplicate ingress and duplicate worker deliveries republish the deterministic wake, repairing a post-commit publication outage.
- A failed wake publication after a successful YDB commit is logged for recovery and does not produce a false Telegram webhook failure.
- Local consumers long-poll YMQ. Cloud-dev uses YMQ triggers on
/wake. - Bucket scans are retained only for local startup recovery and six-hour cloud recovery timers.
- Added the scheduler wake queue/DLQ, trigger wiring, least-privilege publisher credentials, Compose wiring, runbooks, and recovery-oriented E2E coverage.
Idle-cost model
For two consumers scanning 24 buckets:
- scheduled YDB bucket reads:
69,120/day -> 192/day; - timer-triggered container invocations:
2,880/day -> 8/day; - reduction in both scheduled idle-work classes: 99.72%.
Message-driven work remains proportional to real traffic.
Verification
Green locally:
make cimake e2e-local- clean-room, twice-migrated
make ydb-integration docker compose config --quiet- Terraform format plus provider-aware bootstrap/cloud-dev validation
GitHub mirror CI is fully green for 718ff23: run #72, including Go, YDB, local multi-service E2E/restart, Terraform, and runtime image jobs.
The first mirrored run exposed a hidden YDB integration-test assumption: a global recovery bucket scan was asserted to contain only the test tenant. Deterministic delivery IDs legitimately placed another tenant in the same bucket. Commit 718ff23 changed the assertion to verify exactly one matching tenant/delivery tuple without rejecting valid co-located rows; the full clean-room suite and mirrored CI then passed.
Remaining acceptance gate
Keep #39 open through merge and cloud-dev deployment. Close it only after monitoring confirms:
- the former approximately
0.033 requests/secondminute-timer baseline is gone; - idle reconciler and Telegram sender activity is limited to four recovery invocations each per day;
- no periodic YMQ requests remain while idle;
- one synthetic dispatch and one Telegram delivery are processed through the YMQ-triggered path.
Completion and cloud handoff
MR !24 is merged into main as commit 2f0337f. The exact pre-merge head 718ff23 passed all mirrored GitHub Actions jobs: Go verification, clean YDB integration, full local multi-service E2E with restart, Terraform validation, and runtime image builds.
The repository implementation requested by #39 is complete:
- normal scheduler and Telegram delivery work is YMQ-triggered;
- consumers use tenant-scoped outbox point reads;
- duplicate/missing/terminal wakes are idempotent;
- post-commit wake loss is recoverable;
- cloud recovery defaults are every six hours rather than every minute;
- the expected scheduled idle-work reduction is 99.72%.
The deployment and live cost/effectiveness evidence is now tracked as RUNTIME-02 #40. That issue owns the saved-plan deployment, real queue-triggered E2E, duplicate and missed-wake checks, a measured idle window, Lockbox/KMS/YDB/YMQ/container evidence, and rollback proof. It also feeds the broader production-readiness gate #14.
Closing #39 as the implementation task; no cloud success is claimed until #40 passes.
Parent and architecture
Problem
The cloud-dev hot path is polling-driven even when the system is idle:
A one-hour cloud-dev sample contained 60 reconciler and 60 sender invocations. The reconciler billed 109.2 seconds and the sender 45.4 seconds. Minute polling is therefore both an architectural and cost defect for a sparse serverless workload.
Outcome
Make dispatch admission and Telegram delivery event-driven on their hot paths while retaining durable YDB outboxes and a low-frequency recovery sweep for lost wake-up events and time-based recovery.
Design
flowchart LR TX["YDB transaction + durable outbox"] --> H["opaque wake-up envelope"] H --> Q["YMQ wake/delivery queue"] Q -->|"YMQ trigger"| C["targeted relay or sender"] C --> O["dispatch queue, worker, or Telegram"] TX -->|"rare bounded recovery sweep"| R["recovery pass"] R --> QWake-up messages are hints, not canonical state. They contain only tenant-scoped opaque identifiers. Consumers point-read the durable outbox and remain idempotent under duplicate delivery. Publishing occurs only after the YDB transaction succeeds. A failed post-commit publish is recoverable from the durable ready index.
Scope
dispatch.runanddeliver.telegram.next_attempt_athas elapsed.Correctness constraints
next_attempt_at.Verification
Acceptance criteria
Non-goals
Estimate