| chore: clean up LOBE-XXX code annotations (#15135)
* chore: clean up LOBE-XXX annotations from codebase comments
- Remove 【LOBE-XXX】 bracket markers
- Remove LOBE-XXXX references from inline comments
- Clean up test descriptions containing LOBE identifiers
- Preserve linear.app URLs and code-level regex patterns
- Generated: 2026-05-23 02:30:09
* 🐛 fix(tests): restore () in arrow callbacks broken by annotation cleanup
The LOBE-XXX annotation cleanup script over-matched `(LOBE-XXXX', () =>`
and stripped the callback `()`, leaving invalid syntax like
`describe(..., => {` and `it(..., async => {` across 24 test files.
This caused parse failures in Test Packages, Test Desktop App, Test
Database lint, and Test App shard runs. Restoring `()` / `async ()`
unblocks the suites while keeping the ticket-text cleanup intact.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* 🐛 fix(hintFormat-test): restore label + ellipsis in stripMarkdownLinks fixture
The annotation cleanup stripped `LOBE-8516` from a markdown-link's
*label* (`[LOBE-8516](/task/T-1)` → `[](/task/T-1)`), which then survived
`stripMarkdownLinks` because the pattern requires non-empty link text —
the test expected the link to disappear and asserted equality on a
LOBE-free output. The same line also lost a `.` from the trailing
`...` indicator in both input and expected strings.
Substitute a neutral Chinese label (`发布计划`) so the link continues
to exercise the multi-link substitution path, and restore the full
`...` ellipsis.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Arvin Xu <arvinxx@lobehub.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> | 20 小时前 |
| 🐛 fix(context-engine): account for `tool_calls` + reasoning + tool defs in compression budget (#14813)
🐛 fix(context-engine): account for tool_calls + reasoning + tool defs in compression budget
The pre-compression token check (`shouldCompress`) only counted `msg.content`,
which under-counted typical agent conversations by ~58% — tool_calls (~33%
of payload), reasoning traces (~17%), and top-level tool definitions (~2%)
were all silently ignored. As a result, conversations that the provider
tokenizer measured at ~656K passed the harness's 524K threshold without
firing compression, and were rejected upstream as ExceededContextWindow.
Verified empirically against 2 op snapshots in the same topic that hit
the failure mode (LOBE-8964): harness counted 267K, deepseek measured
649K — a 380K (58.8%) gap. ~92% of that gap is fixable by accounting
for the missing fields; the remaining ~8% is `tokenx` vs provider
tokenizer drift, compensated by a 1.25× multiplier on the trigger path.
Changes:
- New `@lobechat/context-engine/tokenAccounting` module exporting
`countContextTokens({messages, tools, options})`. Returns structured
per-source + per-message + per-tool breakdown — usable both by the
compression trigger and by UI panels showing "context by type".
- `shouldCompress` in agent-runtime delegates to `countContextTokens`,
applies the 1.25× drift multiplier on `adjustedTotal` for the trigger
decision, exposes raw count via `currentTokenCount`. Signature now
takes `UIChatMessage[]` directly.
- Removed deprecated `calculateMessageTokens` / `estimateTokens` /
`TokenCountMessage` from agent-runtime — the new module supersedes
them. `createAgentExecutors.ts` updated to call `countContextTokens`
directly for post-compression telemetry.
- Added `raw-md` plugin to agent-runtime vitest config (needed once
context-engine is imported transitively, since the import graph pulls
in `@lobechat/agent-templates` `.md` files).
What's intentionally NOT counted (DB-only fields not sent to provider):
`plugin`, `pluginState`, `chunksList`, `extra`, `fileList`, etc.
Counting these would over-estimate and trigger compression too early.
Tests:
- 19 new unit tests for `countContextTokens` covering content / tool_calls
/ reasoning / tool_call_id / tool definitions / fast-path / aggregation
/ DB-only field exclusion.
- `tokenCounter.test.ts` updated for new drift semantics + UIChatMessage
signature; one boundary case now triggers compression (intentional —
the drift multiplier kicks in at the threshold).
Refs: LOBE-8964 (ECW edge boundary), LOBE-8972 (ECW umbrella),
LOBE-8973 (openrouter `:free` ctx), LOBE-8976 (compression diagnostics).
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> | 9 天前 |
| ♻️ refactor: implement SkillResolver, BaseSystemRoleProvider, and agent document injection pipeline (#13315)
* ♻️ refactor: implement SkillResolver to replace ad-hoc skill assembly
Introduces a two-layer skill resolution architecture mirroring ToolsEngine + ToolResolver:
- SkillEngine (assembly layer): accepts raw skills + enableChecker, outputs OperationSkillSet
- SkillResolver (resolution layer): merges operation + step delta + accumulated activations
Key changes:
- Add SkillResolver, OperationSkillSet, StepSkillDelta, ActivatedStepSkill types
- Enhance SkillEngine with enableChecker and generate() method
- Wire SkillResolver into RuntimeExecutors call_llm
- Replace manual skillMetas assembly in aiAgent with SkillEngine.generate()
- Update client-side skillEngineering to use SkillEngine + enableChecker
- Add activatedStepSkills to AgentState for step-level skill accumulation
Fixes: agent-browser content injected into non-desktop scenarios (Discord bot)
due to missing filterBuiltinSkills call in aiAgent
LOBE-6410
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* ♻️ refactor: extract agent-templates to standalone package and inject documents server-side
- Create @lobechat/agent-templates package with types, templates, and registry
- Move DocumentLoadPosition, DocumentLoadFormat, DocumentLoadRule, etc. to new package
- Move claw templates (AGENTS, BOOTSTRAP, IDENTITY, SOUL) with .md file imports
- Add BOOTSTRAP.md as new onboarding template (priority 1, system-append)
- Fix template positions: AGENTS→before-system, IDENTITY/SOUL→system-append
- Update database package to re-export from @lobechat/agent-templates
- Migrate all consumers to import directly from @lobechat/agent-templates
- Add agent documents injection in server-side RuntimeExecutors (was missing)
- Support -p CLI flag in devStartupSequence for port configuration
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* 🐛 fix: correct import statement for non-type exports from agent-templates
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* 📦 build: add @lobechat/agent-templates to root dependencies
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* ♻️ refactor: remove template proxy files from database package
Stop re-exporting template/templates from database — consumers import
directly from @lobechat/agent-templates. Keep types.ts re-exports for
internal database code only.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* ♻️ refactor: create BaseSystemRoleProvider to unify system message append pattern
All providers that append to the system message now inherit from
BaseSystemRoleProvider and only implement buildSystemRoleContent().
The base class handles find-or-create and join logic.
Migrated providers:
- EvalContextSystemInjector
- BotPlatformContextInjector
- SystemDateProvider
- ToolSystemRoleProvider
- HistorySummaryProvider
- SkillContextProvider
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* 🐛 fix: restore metadata tracking in BaseSystemRoleProvider via onInjected hook
Add onInjected() callback to BaseSystemRoleProvider so subclasses can
update pipeline metadata after successful injection. Also add raw-md
plugin to context-engine vitest config for .md imports.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* ✨ feat: add enabled field to AgentDocumentInjector config
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* ♻️ refactor: add enabled field to all providers, remove spread conditionals in MessagesEngine
All providers now accept an `enabled` config field. MessagesEngine
pipeline is a flat array with no spread conditionals — each provider
is always instantiated and uses `enabled` to skip internally.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* 💄 style: clean up MessagesEngine pipeline comments
Remove numbered prefixes, keep descriptive comments for each provider.
Only phase headers use separator blocks.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* ♻️ refactor: reorganize MessagesEngine pipeline phases by injection target
Phase 1: History Truncation
Phase 2: System Message Assembly (all BaseSystemRoleProvider)
Phase 3: Context Injection (before first user message, BaseFirstUserContentProvider)
Phase 4: User Message Augmentation (last user message injections)
Phase 5: Message Transformation (flatten, template, variables)
Phase 6: Content Processing & Cleanup (multimodal, tool calls, cleanup)
Moved SkillContext, ToolSystemRole, HistorySummary from Phase 3 to
Phase 2 since they append to system message, not user context.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* 💄 style: split Phase 6 into Content Processing (6) and Cleanup (7)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* ♻️ refactor: split AgentDocumentInjector into three position-based injectors
- AgentDocumentSystemInjector (Phase 2): before-system, system-append, system-replace
- AgentDocumentContextInjector (Phase 3): before-first-user
- AgentDocumentMessageInjector (Phase 4): after-first-user, context-end
Shared utilities (filterByRules, formatDocument, sortByPriority) extracted
to AgentDocumentInjector/shared.ts. Old monolithic injector removed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* ♻️ refactor: split AgentDocumentSystemInjector into three separate injectors
- AgentDocumentBeforeSystemInjector: prepends as separate system message (before-system)
- AgentDocumentSystemAppendInjector: appends to system message (system-append)
- AgentDocumentSystemReplaceInjector: replaces entire system message (system-replace)
Each has distinct semantics and correct pipeline placement:
- BeforeSystem → before SystemRoleInjector
- SystemAppend → after HistorySummary (end of Phase 2)
- SystemReplace → last in Phase 2 (destructive)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* ✨ feat: auto-enable agent-documents tool when agent has documents
- Add AgentDocumentsManifest to defaultToolIds
- Add hasAgentDocuments rule in server createServerAgentToolsEngine
- Query agent documents in AiAgentService.execAgent to determine flag
- Pattern matches KnowledgeBase auto-enable via enableChecker rules
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* 🔨 chore: add agent documents status to execAgent operation log
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* update content
* fix tests
* 🐛 fix: add raw-md plugin to database vitest configs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> | 1 个月前 |