| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
feat(hooks): add pre_auth hook framework + minimal demo 新增可选的 pre_auth hook 框架,让定制逻辑(加前缀/查表转换/拒审等) 以 cdylib 插件形式接入,gateway 不需要为每家定制逻辑重编。 架构: - boom-hooks-sdk (叶子 crate,0 个 boom-* 依赖): 定义 wire types (PreAuthRequest/Action/Response) + pre_auth_entry/hook_init_entry helper,封装 catch_unwind + JSON 序列化 + buffer 写入 + return code - boom-main/src/hooks/mod.rs: HookRegistry 用 libloading 加载 .so, Symbol<'static> transmute(lib Arc 在同 struct 保活),failure_mode 在 registry 层消化(allow 降级走原生认证,deny 返回 500) - boom-config: HooksConfig/PreAuthHookConfig,默认全关闭 - extractor: 认证入口插入 pre_auth 调用点,5 种 outcome (NoHook/Continue/Replace/Reject/Deny) 安全边界: - C ABI + JSON wire protocol(跨 .so 不传 Rust 类型) - catch_unwind 隔离 plugin panic - allowed_headers 白名单(默认全屏蔽,避免敏感 header 泄漏) - hook 可选: 不配置时零开销走原路径;加载失败启动报错; 运行期 panic 按 failure_mode 兜底 仓库根目录 hook/ 是一个最简 pre_auth demo,独立 workspace: - mask_key(): 前 3 字符 + 中间全 * (len-9) + 末 6,长度 < 9 退化为全脱敏 + (len=N) 避免泄漏 - pre_auth 符号打印 masked key + allowed_headers 到 stderr, 返回 Continue 让 gateway 用原 raw_key 走原生认证 - 5 个单元测试覆盖 13/15/9 字符边界、过短、Unicode 场景 Signed-off-by: liqiang <liqiang64@huawei.com> | 14 天前 | |
feat(prompt-log): otlp field tips as ? hover, endpoint connectivity light Two UX fixes for the OTLP config card the user pushed back on: 1. Field descriptions moved from inline paragraphs to ? hover badges. Previously each OTLP field rendered a <p class="form-field-tip"> under the input, which stretched the card to ~3× its original height and buried the actual form controls under prose. Now the helpers render ${tip(opts.tip)} inside the <label> — reusing the existing viewport-aware field-tip + #vtip tooltip plumbing that every other form (model deployment, plan, health check) already uses. The .form-field-tip style is removed. 2. Connectivity indicator above the OTLP Endpoint input. A colored dot + status text that polls POST /admin/prompt-log/otlp-ping every 5s: - green = collector responded to an empty ExportLogsServiceRequest within the configured timeout (text shows "Reachable · 47 ms") - red = connection refused / timed out / non-2xx (text shows error) - grey pending pulse = mid-probe - grey static = endpoint empty / not yet tested The probe goes through AdminCommand::PingOtlpEndpoint so the dashboard crate doesn't grow a reqwest dependency — boom-main constructs a one-shot OtlpConfig from the request body and calls boom_promptlog::ping_endpoint, a single-attempt no-retry free function added to otlp.rs. The probe uses the operator-edited endpoint value (read from the form input on blur), not the committed YAML — so you can type a new collector address and test it before saving. Verified: cargo build --release clean, 17/17 promptlog + 11/11 dashboard tests still pass, app.js + i18n.js parse. Signed-off-by: liqiang <liqiang@atomgit.com> Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Signed-off-by: liqiang <liqiang64@huawei.com> | 2 天前 | |
fix(auth,log): resolve all-team-models double-expansion + dedup ModelNotAllowed logs - boom-auth: resolve_team_models() collapses litellm special names in team.models (all-team-models/all-proxy-models → empty), so a key with all-team-models whose team also has all-team-models no longer falls through to ModelNotAllowed. - boom-core: GatewayError::should_dedup_log() — strict superset of !should_log_to_db, adds ModelNotFound/ModelNotAllowed/KeyExpired/ KeyBlocked to the dedup-eligible set. - boom-main/request_log: dedup gate uses should_dedup_log; new log_auth_error() handles extractor-phase failures (no identity yet). - boom-main/extractor: pre_auth Reject / Deny / authenticate-failure paths now log via log_auth_error. AuthError(401) is full-volume; KeyExpired/KeyBlocked/BudgetExceeded dedup per (key_hash,"<auth>",60s). - boom-main/routes: tracing::warn → debug in check_model_access (log_error_with_usage path handles console warn under dedup). extract_client_ip made pub(crate) for extractor reuse. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Signed-off-by: liqiang <liqiang64@huawei.com> | 7 天前 | |
fix: health probe only counts connection failure and 5xx as offline Previously any non-2xx response (including 404) was treated as a node failure, which could incorrectly auto-disable a healthy node whose health check path simply doesn't exist. Now 4xx responses are treated as "node alive", only connection failures and 5xx count as offline. Signed-off-by: LYK918 <lyk918@gmail.com> Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Signed-off-by: LYK918 <532160762@qq.com> | 1 个月前 | |
fix(kvc): 固定 tools 前缀移至 messages 之前，修复带 tools 长对话 Trie-Hit 偏低 compute_prefix_bytes 之前拼成 JSON(messages)+JSON(tools)，tools 在后。多轮对话里 messages 每轮增长，连续 trie 匹配在 messages 增长点（分歧点）即 break，分歧点之后 的内容（含整段固定的 tools）即使 record 过也永远走不到，全部计为 miss。这导致带 tools 的长对话 Trie-Hit 稳定卡在 ~70%，而 vLLM 侧 Prefix Hit 仍 98%（差值 ≈ tools 段 占比）。 改为 tools 在前、messages 在后：固定段落在分歧点之前，每轮稳定命中；顺序也与 GLM/Qwen3/MiniMax 的 chat_template 渲染顺序（tools 在 system 段、messages 之后）一致。 query 与 record 共用 step2 算出的同一个 prefix_bytes，无需改动其他路径；发给 vLLM 的原始 OpenAI body 不经过此函数，请求内容不受影响。 Signed-off-by: Lei Gong <gonglei25@huawei.com> Co-Authored-By: Claude <noreply@anthropic.com> | 22 天前 | |
fix(stressmon): show real process-level CPU, drop misleading worker display Reverts the worker_busy_pct experiment from 8a2db7d. That commit switched to tokio's worker_total_busy_duration so CPU peaked at exactly 100%, on the theory that >100% was a bug. It wasn't — the operator pointed out that tokio's blocking pool (prompt-log gzip, DB migrations, file I/O) runs threads *outside* the worker pool, so those CPU cycles legitimately show up at the process level. Displaying "Workers: 8" next to a CPU chart implied the two were coupled, but raising server.workers won't help if the blocking pool is the one cooking. The two numbers were communicating different things in the same UI — confusing. Going back to process-level CPU (utime+stime from /proc/self/stat, diffed against the previous 1Hz sample). Y axis is auto-scaled, so a 16-core box under heavy load displays as 1300% — same definition as top. The red warning band still sits at workers × 80% (the absolute value where the worker pool itself is 80% saturated, before blocking pool contribution); persistent spikes there mean raising workers would actually help, while a flat counter + high CPU means the blocking pool is doing the work. Field rename back: StressmonSample.worker_busy_pct → cpu_pct. record_sample's threshold becomes cpu_pct > num_workers * 80.0. Also dropped the Workers chip from the info bar — its presence was the root of the confusion. The glossary tooltip's blocking_pool entry now explicitly explains why CPU can show 1300% on a 16-core box with 8 workers configured; cpu_over_80 entry explains the new threshold semantics (counter moves when worker pool is the bottleneck). Restored read_proc_cpu_rss() returning (cpu_jiffies, rss_bytes) so /proc is read once per sample for both metrics. PREV_BUSY Vec (one slot per worker, with worker_total_busy_duration diffing) replaced with a single PREV (prev_jiffies, prev_instant) pair. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Signed-off-by: liqiang <liqiang64@huawei.com> | 1 天前 | |
fix(auth,log): resolve all-team-models double-expansion + dedup ModelNotAllowed logs - boom-auth: resolve_team_models() collapses litellm special names in team.models (all-team-models/all-proxy-models → empty), so a key with all-team-models whose team also has all-team-models no longer falls through to ModelNotAllowed. - boom-core: GatewayError::should_dedup_log() — strict superset of !should_log_to_db, adds ModelNotFound/ModelNotAllowed/KeyExpired/ KeyBlocked to the dedup-eligible set. - boom-main/request_log: dedup gate uses should_dedup_log; new log_auth_error() handles extractor-phase failures (no identity yet). - boom-main/extractor: pre_auth Reject / Deny / authenticate-failure paths now log via log_auth_error. AuthError(401) is full-volume; KeyExpired/KeyBlocked/BudgetExceeded dedup per (key_hash,"<auth>",60s). - boom-main/routes: tracing::warn → debug in check_model_access (log_error_with_usage path handles console warn under dedup). extract_client_ip made pub(crate) for extractor reuse. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Signed-off-by: liqiang <liqiang64@huawei.com> | 7 天前 | |
feat(rewrite): strip Claude Code attribution block from /v1/messages Claude Code v2.1.36+ injects a standalone text block beginning with x-anthropic-billing-header: at the head of every system prompt. The per-request cch= field defeats byte-exact KV-cache prefix matching, causing 100% prefix-cache miss on non-Anthropic backends (cache write at 1.25x input vs cache hit at 0.10x, plus full prefill TTFT penalty). Add an optional handler-level rewrite that drops the entire block before forwarding upstream. Algorithm aligned with vLLM PR #36829: block-level starts_with("x-anthropic-billing-header") match, no secondary feature checks. Covers top-level system blocks plus role=system messages nested in messages; string-form system is left untouched (Claude Code uses the blocks form for injection). Gated by router_settings.strip_claude_code_attribution (default false) because the gateway may forward to the official Anthropic API, where stripping could trip anti-piracy defenses. Only /v1/messages is affected; /v1/chat/completions is untouched (Claude Code does not inject this header on the OpenAI protocol). Strip happens before anthropic_request_to_openai so prompt-log capture records the cleaned body. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Signed-off-by: liqiang <liqiang64@huawei.com> | 1 个月前 | |
!74 feat(routing): add MlServiceClient classification strategy From: @leningchen_admin Reviewed-by: @luanjianhai | 2 天前 | |
feat(stressmon): real-time system pressure monitor (CPU, RSS, tokio queues) New boom-stressmon leaf crate — 1Hz ring buffer (3600 slots, 60min × 1Hz, ~140KB fixed) holding CPU%, RSS, worker-queue-depth, blocking-queue-depth, and inflight samples. Sampling logic lives in boom-main (keeps boom-stressmon a pure leaf dep — no boom-routing needed for InFlightTracker). Data sources: - /proc/self/stat fields 14/15 (utime+stime, jiffies, 100Hz) - /proc/self/status VmRSS - tokio runtime metrics: global_queue_depth + Σ worker_local_queue_depth + blocking_queue_depth Dashboard: - New admin stats top panel: <canvas> with 5 overlaid series - 1.5s poll, range selector 5m/15m/30m/60m - Auto-scaling axes (CPU 0..N×100, others adapt to peak) - Stops polling when admin leaves stats section Tokio metrics gate is cfg(tokio_unstable) — a rustc cfg, not a Cargo feature. Enabled via workspace .cargo/config.toml build.rustflags, applied to every crate (ignored by crates that don't touch the gated API). Traited via boom_core::StressmonApi so boom-dashboard stays leaf-of-boom-core (no new dep on boom-stressmon). Signed-off-by: liqiang <liqiang@atomgit.com> Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Signed-off-by: liqiang <liqiang64@huawei.com> | 2 天前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 14 天前 | ||
| 2 天前 | ||
| 7 天前 | ||
| 1 个月前 | ||
| 22 天前 | ||
| 1 天前 | ||
| 7 天前 | ||
| 1 个月前 | ||
| 2 天前 | ||
| 2 天前 |