已开启
[MVP-09d] Add cloud-dev operational alerts and prove notification delivery #19
urandon创建于  8月3日
urandon
urandon成员
8月3日 创建

Parent and architecture

  • Implementation epic: #6
  • Architecture: #1
  • Cloud bootstrap: #12
  • Depends on: #17, live resources from #12

Outcome

Make the low-traffic cloud-dev environment observable and prove that actionable failures notify the configured operator.

Estimate

  • 3 SP / 2 engineering days
  • Risk: medium

Scope

  • Define alert thresholds and evaluation windows suitable for sparse, bursty development traffic.
  • Create alerts for:
    • Cloudflare Worker exceptions/5xx and Yandex handoff failures;
    • Yandex Workflow failed executions;
    • API Gateway/control API 5xx and latency;
    • trigger errors, retry growth and YMQ DLQ messages;
    • Serverless Container errors, timeouts and concurrency saturation;
    • YDB throttling, RU/storage and transaction errors;
    • Object Storage capacity;
    • terminal Telegram delivery failures.
  • Use the Yandex Monitoring API/console where the Terraform provider has no alert resource.
  • Keep contact channels and private recipient data outside the public repository.
  • Add a redacted alert inventory/runbook to the repository.
  • Execute one controlled test alert and record delivery evidence privately.

Non-goals

  • Changing business-flow retry semantics.
  • Full production SLOs or paging policy.
  • Payload logging or message-content dashboards.
  • Automated cost shutdown; the existing folder budget remains notification-only.

Verification

  • Every alert has an owner, metric/query, threshold, window and recovery condition.
  • Alert IDs are recorded in the private deployment record.
  • A controlled injected failure reaches the configured notification channel.
  • Recovery clears the alert.
  • Normal low-volume idle periods do not page.

Acceptance criteria

  • Terraform apply is no longer treated as proof of monitoring readiness.
  • The cloud-dev operator can distinguish ingress, workflow, control, queue, worker, database and delivery failures.
  • Public documentation contains procedures and redacted identifiers only.
  • No email address, token, Telegram identity or payload is committed or posted publicly.
likedislike
urandonurandon成员
24 天前 添加了label:cloudmvptelegram
urandonurandon成员
24 天前 关联了里程碑:MVP — Core platform (#6)