已关闭
[MVP-03A] Validate YDB primary-key distribution and partitioning policy #16
urandon创建于  7月29日关闭于  7月29日
urandon
urandon成员
7月29日 创建

Parent and architecture

Outcome

Turn the current correctness-oriented YDB schema into an explicit, measurable physical partitioning contract before Telegram ingress and scheduler/reconciler traffic make primary-key changes expensive.

Estimate

  • 5 SP / 3 engineering days
  • Risk: medium-high

Why this is a separate task

YDB row tables are range-partitioned by the lexicographically ordered composite primary key. Size-based automatic partitioning is enabled by default, but load-based partitioning is disabled; a small hot table can therefore remain one partition with one CPU core of update capacity.

The current 22-table schema is correct and tenant-isolated, but it does not yet prove write distribution or scale behavior. This work must remain separate from scheduler business logic and must complete before cloud-dev contains durable application data.

Scope

  • Inventory every current row table and document:
    • primary-key component order and cardinality;
    • point, prefix-range and time-range access patterns;
    • expected write/read concurrency;
    • monotonic key components and moving-tail risk;
    • bounded/unbounded query fan-out;
    • transaction paths that may cross partitions.
  • Define the physical ID contract for tenant_id, run_id, attempt, lease, outbox, delivery, manifest and observation identifiers:
    • identifiers used in hot insert paths must be high-entropy and lexicographically distributed;
    • timestamps and human ordering remain separate columns;
    • explicitly reject unmodified time-sortable IDs when they create a hot edge.
  • Classify current tables into:
    • tenant-scoped entity/lookup tables;
    • per-run ordered tables;
    • append-heavy audit/usage tables;
    • globally consumed ready/expiry access tables.
  • Decide and implement the physical layout for dispatch_ready, telegram_delivery_ready, lease_expiry, quota_expiry, and any audit/usage path that requires global consumption:
    • retain tenant-first keys when measured access remains per-tenant and within one partition's capacity; or
    • add a versioned, bounded shard_bucket Uint32 leading key with deterministic derivation and bounded parallel fan-out.
  • Add explicit per-table YDB settings where required:
    • AUTO_PARTITIONING_BY_LOAD;
    • minimum and maximum partition counts;
    • partition size threshold;
    • initial split points or uniform partitions when compatible with the leading key.
  • Add an operator/developer inspection command that reports actual table partition settings and counts through supported YDB schema inspection.
  • Add deterministic synthetic workload fixtures for:
    • many tenants with even traffic;
    • one elephant tenant;
    • bursty monotonically increasing updates/checkpoints/ready/expiry/audit writes.
  • Measure logical key distribution, partition targeting, query fan-out and retry-safe split behavior in YDB Local. Clearly label this as structural validation, not a cloud throughput claim.
  • Define the cloud measurement contract consumed by #12 and #14: per-shard CPU, overloaded shards, partition count/headroom, split/merge activity, latency, RU, throttling/retries and cross-partition transactions.
  • Document expand/backfill/cutover/rollback for every primary-key change. Do not model a primary-key rewrite as an in-place ALTER TABLE.

Design constraints

  • Every operational row and authorization path retains tenant_id; a physical bucket never replaces tenant validation.
  • No unbounded full-table scan or unbounded bucket fan-out on a hot path.
  • Bucket derivation is stable, versioned and shared by writers/readers.
  • Table settings are application migrations, not Terraform-managed application schema.
  • Migrations remain forward-only, repeatable and compatible with the previous active control-plane revision.
  • Secondary indexes are reviewed as independently partitioned physical structures.
  • Local YDB cannot be used to claim production throughput or serverless RU capacity.

Verification

  • Apply the revised schema from zero and repeatedly to an existing YDB Local database.
  • Run key-distribution tests for all generated operational IDs.
  • Prove no hot-path query requires an arbitrary tenant/table scan.
  • Prove bounded-bucket writers and readers use the same versioned derivation.
  • Prove duplicate/retry execution does not duplicate ingress, usage, quota, dispatch or delivery effects.
  • Produce machine-readable partition-setting output for the cloud-dev gate.
  • Run the existing YDB transaction/isolation suite after every migration.

Acceptance criteria

  • Every YDB table has a documented physical key/access-pattern decision.
  • All expected high-write tables either have explicit load-aware partition settings/bucketing or a measured rationale for retaining the tenant-first layout.
  • No expected high-write path uses an unbucketed monotonically increasing leading key.
  • Tenant/run/outbox identifier generation has distribution tests and a stable public contract.
  • Global ready/expiry consumption is bounded and does not depend on a full-table scan.
  • Primary-key changes have executable expand/backfill/cutover procedures.
  • #8 can generate identities and ingest updates without reopening the physical-key decision.
  • #9 can implement scheduling/reconciliation against a stable, measured ready/expiry topology.
  • #12 and #14 have explicit cloud metrics, workloads and pass/fail evidence requirements.
likedislike
urandon
urandon成员
7月29日 评论:

Implementation started

Work has started on branch ai/mvp-03a-ydb-partitioning, based on merged main commit b0a087b.

The first schema/access-pattern audit confirmed the planned distinction:

  • tenant entity and per-run ordered tables retain tenant-first keys, with a non-time-sortable random ID contract and explicit load-aware settings on expected high-write tables;
  • global ready/expiry traversal moves to a versioned 16-bucket physical layout so the reconciler has bounded fan-out and one elephant tenant can be distributed;
  • legacy ready/expiry tables remain available during an expand/backfill/cutover compatibility window.

Implementation includes live JSON schema inspection, deterministic distribution tests, dual writes, an idempotent backfill command, YDB Local contract tests, and the cloud metric gate required by #12/#14.

likedislike
urandonurandon成员
7月29日 关联了pull request:MVP-03A: define the YDB physical partitioning contract
urandon
urandon成员
7月29日 评论:

Implementation update — physical partitioning contract

Implementation is published in MR !7, branch ai/mvp-03a-ydb-partitioning, commit 4c4075c.

Implemented

  • Production opaque IDs use 128 random bits and non-time-sortable Base32 encoding.
  • bucket_v1 is a golden-vector-tested SHA-256 modulo-16 protocol shared by writers/readers.
  • Four bucketed v2 ready/expiry tables use (bucket, time, tenant, object) keys and 16 explicit initial ranges.
  • Expected high-write tenant tables have explicit load/size partition settings.
  • State-store mutations dual-write legacy and v2 ready/expiry rows during the compatibility window.
  • The future reconciler has a bounded per-bucket lease-expiry traversal.
  • schema-inspect emits live primary keys, settings, partition counts and drift as JSON.
  • schema-backfill idempotently copies legacy ready/expiry rows into v2.
  • Public documentation contains the full 22-table inventory, ID contract, expand/backfill/cutover/contract procedure, and cloud metric gate.
  • Tests cover monotonic elephant traffic, many-tenant bursts, bounded fan-out, migration ordering, dual writes, quota/delivery retry, backfill and live partition inspection.

Verified locally

make ci
go test -run '^$' -tags=ydbintegration ./test/ydbintegration
git diff --check

Docker is unavailable on the development workstation, so no YDB Local execution claim is made. The GitHub mirror has not yet received branch ai/mvp-03a-ydb-partitioning; therefore MR !7 remains open and #16 remains in progress until mirror sync triggers the mandatory YDB and image jobs.

likedislike
urandonurandon成员
7月29日 关联了pull request:[MVP-03A] Resolve absolute YDB schema inspection paths
urandon
urandon成员
7月29日 评论:

Post-merge verification update

MR !7 merged as fb31bf8. The exact mirrored SHA reached GitHub Actions run #9.

The run exposed one integration failure, so this issue remains open:

  • all 38 YDB migrations applied successfully twice;
  • Go verification passed;
  • TestPartitioningContractMatchesYDBLocal failed before reading table metadata because native YDB DescribeTable received the relative path tenants instead of the required absolute database-qualified path /local/tenants.

The correction is published in MR !8, commit 275f2f3:

  • the YDB client now exposes its native absolute database path;
  • the inspector resolves every physical table below that path and rejects relative database roots;
  • CLI/integration callers were updated;
  • regression tests prove /local/tenants resolution and fail-closed relative-path handling.

Local make ci, tagged-package compilation, and git diff --check pass. Issue #16 will close only after MR !8 reaches the GitHub mirror and the YDB integration job is green.

likedislike
urandon
urandon成员
7月29日 评论:

Completion report — YDB partitioning contract

The partitioning contract from this issue is now implemented and verified.

Delivered

  • MR !7 added the physical YDB partitioning contract, the repository-owned ydb-partition-check inspector, schema documentation, CI coverage, and the required table/index corrections.
  • The initial post-merge run exposed a real inspection-path defect: native YDB table descriptions require database-qualified absolute paths.
  • MR !8, commit 275f2f3, fixed path resolution and added regression coverage for /local/<table> inspection plus fail-closed handling of relative database paths.

Verification

GitHub Actions run 30461247750 completed successfully for 275f2f3:

  • Go verification — passed;
  • Local multi-service stand — passed;
  • YDB schema and state store — passed, including migrations and physical partition inspection;
  • Runtime images — passed.

MR !8 was merged into main as fbb9f4e. The tested hotfix commit and the merge commit have the same Git tree, d30ad8094c0aa8d700f48019ca7edc1acc723c02, so the green run verifies the exact source tree now present on main.

All acceptance criteria are satisfied.

likedislike
urandonurandon成员
7月29日 关闭了 issue
urandonurandon成员
7月30日 关联了pull request:YDB: decouple logical buckets from physical partition counts
urandon
urandon成员
7月30日 评论:

Follow-up correction — let YDB own physical partition counts

MR !12 corrects an over-constrained part of the MVP-03A executable contract.

The stable output of this issue remains the primary-key layout, high-entropy identifier policy, logical ready/expiry bucket function, and required load-based auto-partitioning. The exact minimum, maximum, target size, and current number of YDB partitions are operational telemetry and capacity tuning, not application schema invariants.

The 16 ready/expiry buckets bound reader fan-out but do not require 16 physical DataShards. Forward migrations 00041–00044 lower the original minimum floor so YDB can merge ranges under low load and split them again by load or size. CI will continue to validate keys and enabled auto-partitioning while accepting partition-count changes made by YDB or evidence-backed environment tuning.

likedislike
urandon
urandon成员
7月30日 评论:

Pre-deployment baseline cleanup

MR !13 supersedes the provisional forward-migration portion of !12.

Because Sessionless has no persistent YDB deployment or application data, migrations 00023–00040 can still be corrected as a baseline. The MR removes 00041–00044, all manual split boundaries, and all numeric partition-count/size tuning. Clean local and CI databases will apply the resulting 40-file baseline from scratch.

The migration set becomes immutable only when the first persistent cloud-dev environment records it as deployed. This does not change the durable primary-key, logical bucket, or auto-partitioning decisions delivered by MVP-03A.

likedislike
urandonurandon成员
7月30日 关联了pull request:YDB: rebase the undeployed migration baseline
urandonurandon成员
8月26日 添加了label:mvpydb
urandonurandon成员
8月26日 关联了里程碑:MVP — Core platform (#6)