已关闭
Add lifecycle-aware retention and TTL for daemon job history #102
urandon创建于  8 天前关闭于  7 天前
urandon
urandon成员
8 天前 创建

Why

Daemon job state is persisted in the service runtime jobs.json snapshot. The current policy keeps every active job, the latest 128 terminal jobs across all statuses, and the latest 256 progress events per job. There is no age-based TTL or configuration, and succeeded bounded maintenance slices compete for the same retention budget as diagnostically valuable failed/interrupted jobs.

Initial head/backfill maintenance can produce a sequence of short terminal jobs for one logical work stream, making count-only retention both noisy in the Admin UI and poorly aligned with diagnostic value.

Goal

Introduce deterministic, lifecycle-aware retention for daemon job history so routine successful slices expire quickly while failures remain available long enough for diagnosis, with an absolute storage bound and clear Admin UI visibility.

Scope

  • Define age-based retention by terminal status, with documented defaults for succeeded/superseded versus failed/interrupted/cancelled jobs.
  • Keep queued/running jobs unconditionally until they reach a terminal state; preserve restart conversion to interrupted.
  • Preserve at least the most recent diagnostically significant failure for each maintenance registration/work stream even when its ordinary TTL expires, subject to a separate hard bound.
  • Retain an absolute maximum job count and per-job progress-event cap as final safety limits.
  • Apply pruning deterministically on load, job updates/completion, and periodic daemon reconciliation so an idle service eventually removes expired history.
  • Make retention configurable through bounded, validated non-secret service configuration; reject negative, zero-risk, or unbounded values.
  • Keep the persisted format backward compatible and safely compact existing jobs.json snapshots after upgrade.
  • Expose effective retention policy, oldest retained job time, retained counts by state, and truncation/expiry evidence through the observation API and Admin UI.
  • Ensure job deep links and filters render an explicit expired/not-retained state instead of a generic not-found failure.
  • Document that pruning operational job history never deletes cached GitCode records, sync frontiers, maintenance policy, RAG indexes, or required audit receipts.

Suggested defaults

  • Active: no TTL.
  • Succeeded/superseded: 24–72 hours.
  • Failed/interrupted/cancelled: 7–14 days.
  • Preserve the latest failure per registration/work stream within a separate bounded diagnostic cohort.
  • Keep a global absolute cap in addition to TTL.

Exact defaults should be selected with tests and measured snapshot size rather than treated as API constants.

Acceptance criteria

  1. Short successful maintenance slices no longer remain indefinitely merely because fewer than 128 later jobs exist.
  2. Recent and last-significant failures remain available for diagnosis longer than routine successes.
  3. Active jobs are never pruned; jobs interrupted by restart remain observable under terminal retention rules.
  4. Pruning is deterministic under an injected clock and covered across load, completion, idle reconciliation, and status-specific TTL boundaries.
  5. jobs.json remains bounded, mode 0600, public-safe, and backward compatible.
  6. The Admin UI explains retained versus expired history and shows the effective retention policy without exposing filesystem paths.
  7. Pruning job history does not alter cache contents, maintenance frontiers/stages, or audit evidence required by other controls.
  8. go test ./..., frontend tests, and git diff --check pass.

Non-goals

  • Deleting cached repository content or maintenance registrations.
  • Keeping an unbounded forensic archive.
  • Requiring an external database or telemetry backend.
  • Browser-controlled arbitrary job-file deletion.
likedislike
urandonurandon成员
8 天前 添加了label:admin-uidiagnosticsenhancementservice
urandonurandon成员
7 天前 关联了pull request:Add lifecycle-aware daemon job retention
urandon
urandon成员
7 天前 评论:

Implementation is ready in MR !105 at commit bcdb337.

Delivered:

  • 48h success / 14d diagnostic TTL defaults with bounded global service configuration;
  • latest significant failure preservation per registration/work stream with a separate hard cohort;
  • deterministic pruning on load, updates/completion, and idle reconciliation;
  • backward-compatible mode-0600 jobs.json compaction and per-job progress caps;
  • observation API/Admin UI policy, state counts, oldest retained time, and expiry/truncation evidence;
  • explicit expired/not-retained deep-link semantics and unaffected cache/frontier/RAG/audit boundary.

Verification: full Go suite, Vitest, Svelte check, Playwright (11 passed, 1 intentionally skipped), and git diff --check all pass.

likedislike
urandon
urandon成员
7 天前 评论:

Released in v0.2.2 from merge commit 09718a3. MR !105 and the merged-main CI runs completed successfully; release workflow 32916885259 also succeeded. The patch is now in both GitHub and native GitCode release publication. Closing as delivered.

likedislike
urandonurandon成员
7 天前 关闭了 issue