草稿
[WIP]feat: expose retrieval config and structured observability #1
DoraA_Mengjie创建于 24 天前
[WIP]feat: expose retrieval config and structured observability #1
草稿
共 114 个文件变更+16618-1389
| @@ -8,7 +8,7 @@ benchmark runner, and reproducible evaluation scripts. | |||
| 8 | 8 | ||
| 9 | - `memory-core`: core long-term memory API and storage implementations. | 9 | - `memory-core`: core long-term memory API and storage implementations. |
| 10 | - `memory-bench`: CLI runner for ingesting benchmark records and searching top-k memories. | 10 | - `memory-bench`: CLI runner for ingesting benchmark records and searching top-k memories. |
| 11 | -- `memory-cases`: local case knowledge service for storing, indexing, and retrieving case documents. | 11 | +- `memory-cases`: embeddable case-library engine for storing, indexing, and retrieving case documents. |
| 12 | - Default local storage: SQLite. | 12 | - Default local storage: SQLite. |
| 13 | - Default retrieval mode: hybrid dense embedding retrieval plus BM25 text retrieval. | 13 | - Default retrieval mode: hybrid dense embedding retrieval plus BM25 text retrieval. |
| 14 | - Embedding providers: OpenAI-compatible embeddings, including self-hosted `/v1/embeddings` | 14 | - Embedding providers: OpenAI-compatible embeddings, including self-hosted `/v1/embeddings` |
| @@ -20,7 +20,7 @@ benchmark runner, and reproducible evaluation scripts. | |||
| 20 | crates/ | 20 | crates/ |
| 21 | memory-core/ # core memory API, record model, stores, retrieval | 21 | memory-core/ # core memory API, record model, stores, retrieval |
| 22 | memory-bench/ # benchmark CLI | 22 | memory-bench/ # benchmark CLI |
| 23 | - memory-cases/ # case API, ingestor, chunk store, and search index | 23 | + memory-cases/ # embeddable case API, ingestion worker, chunk store, and search index |
| 24 | 24 | ||
| 25 | evaluation/ | 25 | evaluation/ |
| 26 | common/ # shared evaluation helpers, metrics, reports, backends | 26 | common/ # shared evaluation helpers, metrics, reports, backends |
| @@ -103,16 +103,16 @@ pub struct SearchMemoryRequest { | |||
| 103 | See [docs/design/memory-cases-storage-split.md](docs/design/memory-cases-storage-split.md) for | 103 | See [docs/design/memory-cases-storage-split.md](docs/design/memory-cases-storage-split.md) for |
| 104 | table ownership and index semantics. | 104 | table ownership and index semantics. |
| 105 | 105 | ||
| 106 | -`memory-cases` defaults to local hash embeddings for offline and demo use. Use | 106 | +`memory-cases` defaults to local hash embeddings for offline and demo use. Configure |
| 107 | -`--embedding-provider openai_compatible` with `--embedding-base-url`, `--embedding-model`, | 107 | +`case_library.embedding_provider` as `openai_compatible` together with its base URL, model, |
| 108 | -`--embedding-api-key-env`, and `--embedding-dimensions` when a real embedding service is | 108 | +API key environment name, and dimensions when a real embedding service is available. The |
| 109 | -available. The endpoint must be OpenAI-compatible and expose `/v1/embeddings`; this includes | 109 | +endpoint must be OpenAI-compatible and expose `/v1/embeddings`; this includes self-hosted |
| 110 | -self-hosted embedding services. | 110 | +embedding services. |
| 111 | 111 | ||
| 112 | Keep the case-library search index in a separate SQLite file from RAM-A personal long-term | 112 | Keep the case-library search index in a separate SQLite file from RAM-A personal long-term |
| 113 | memory for demos and deployments. Sharing one SQLite index is only suitable for narrow smoke | 113 | memory for demos and deployments. Sharing one SQLite index is only suitable for narrow smoke |
| 114 | -tests because two services can contend on SQLite writer locks and case reindex/reset flows | 114 | +tests because independent storage operations can contend on SQLite writer locks and case |
| 115 | -should not touch user memories. If a shared `memory-core` SQLite index is intentionally used, | 115 | +reindex/reset flows should not touch user memories. If a shared `memory-core` SQLite index is intentionally used, |
| 116 | use one embedding profile per search scope. RAM-A stores embedding profile metadata on new | 116 | use one embedding profile per search scope. RAM-A stores embedding profile metadata on new |
| 117 | records and rejects same-scope profile mismatches, because two providers can have the same | 117 | records and rejects same-scope profile mismatches, because two providers can have the same |
| 118 | vector dimensions but incompatible semantic spaces. | 118 | vector dimensions but incompatible semantic spaces. |
| @@ -162,6 +162,49 @@ cargo run -p memory-bench -- ` | |||
| 162 | --output outputs/bge_m3_top10.json | 162 | --output outputs/bge_m3_top10.json |
| 163 | ``` | 163 | ``` |
| 164 | 164 | ||
| 165 | +Graph memory benchmark mode is opt-in. `--graph-build` builds the graph during `add`; | ||
| 166 | +`--graph` enables the graph retrieval channel during `search`. Graph extraction uses an | ||
| 167 | +OpenAI-compatible chat-completions endpoint; by default it reads the same | ||
| 168 | +`OPENROUTER_API_KEY` environment variable and uses OpenRouter. | ||
| 169 | + | ||
| 170 | +```powershell | ||
| 171 | +$env:OPENROUTER_API_KEY="your_openrouter_key" | ||
| 172 | + | ||
| 173 | +cargo run -p memory-bench -- ` | ||
| 174 | + --store data/locomo_graph.sqlite ` | ||
| 175 | + --embedding openrouter ` | ||
| 176 | + --model baai/bge-m3 ` | ||
| 177 | + --dimensions 1024 ` | ||
| 178 | + --graph-build ` | ||
| 179 | + --graph-llm-model openai/gpt-4o-mini ` | ||
| 180 | + add ` | ||
| 181 | + --dataset data/locomo/locomo10.json ` | ||
| 182 | + --text-fields text,content,message,memory | ||
| 183 | + | ||
| 184 | +cargo run -p memory-bench -- ` | ||
| 185 | + --store data/locomo_graph.sqlite ` | ||
| 186 | + --embedding openrouter ` | ||
| 187 | + --model baai/bge-m3 ` | ||
| 188 | + --dimensions 1024 ` | ||
| 189 | + --graph ` | ||
| 190 | + --graph-weight 0.2 ` | ||
| 191 | + search ` | ||
| 192 | + --dataset data/locomo/locomo10.json ` | ||
| 193 | + --query-fields question,query ` | ||
| 194 | + --top-k 10 ` | ||
| 195 | + --output outputs/locomo_graph_top10.json | ||
| 196 | +``` | ||
| 197 | + | ||
| 198 | +In graph `auto` memory-space mode, prepared-schema datasets use `scope_id`, while raw | ||
| 199 | +top-level-array datasets use the top-level JSON path such as `path:$[0]`. Keep graph and | ||
| 200 | +baseline runs in separate SQLite files when comparing scores. | ||
| 201 | + | ||
| 202 | +For ad-hoc graph search with `--query`, pass the memory space through `--filter`, for example | ||
| 203 | +`--filter '{"scope_id":"scope-a"}'`; otherwise no graph memory space can be inferred from the | ||
| 204 | +single query path. With `--resume --graph-build`, existing MemoryRecords are still checked for | ||
| 205 | +graph build: completed graph runs are skipped, missing graph runs are built, and failed/running | ||
| 206 | +graph runs fail explicitly instead of being treated as successful. | ||
| 207 | + | ||
| 165 | Do not commit API keys, local stores, downloaded datasets, or generated reports. | 208 | Do not commit API keys, local stores, downloaded datasets, or generated reports. |
| 166 | 209 | ||
| 167 | ## RAM-A memory MCP service | 210 | ## RAM-A memory MCP service |
| @@ -218,6 +261,9 @@ field changes listed below. | |||
| 218 | }, | 261 | }, |
| 219 | "case_library": { | 262 | "case_library": { |
| 220 | "enabled": true | 263 | "enabled": true |
| 264 | + }, | ||
| 265 | + "graph_memory": { | ||
| 266 | + "enabled": false | ||
| 221 | } | 267 | } |
| 222 | }, | 268 | }, |
| 223 | "http": { | 269 | "http": { |
| @@ -255,6 +301,8 @@ field changes listed below. | |||
| 255 | "rag_store": "data/memory-cases.sqlite", | 301 | "rag_store": "data/memory-cases.sqlite", |
| 256 | "index_store": "data/memory-cases-index.sqlite", | 302 | "index_store": "data/memory-cases-index.sqlite", |
| 257 | "source_dir": "crates/memory-cases/test/accuracy_docs", | 303 | "source_dir": "crates/memory-cases/test/accuracy_docs", |
| 304 | + "api_token_env": "RAM_A_CASES_ADMIN_TOKEN", | ||
| 305 | + "ingestion_poll_ms": 1000, | ||
| 258 | "embedding_provider": "hash", | 306 | "embedding_provider": "hash", |
| 259 | "embedding_model": "hash", | 307 | "embedding_model": "hash", |
| 260 | "embedding_dimensions": 1024, | 308 | "embedding_dimensions": 1024, |
| @@ -267,6 +315,22 @@ field changes listed below. | |||
| 267 | "tenant_ids": ["tenant-local"] | 315 | "tenant_ids": ["tenant-local"] |
| 268 | } | 316 | } |
| 269 | ] | 317 | ] |
| 318 | + }, | ||
| 319 | + "graph_memory": { | ||
| 320 | + "llm_api_key_env": "GRAPH_LLM_API_KEY", | ||
| 321 | + "llm_base_url": "http://127.0.0.1:8000/v1", | ||
| 322 | + "llm_model": "GLM-5.2", | ||
| 323 | + "llm_timeout_ms": 60000, | ||
| 324 | + "build_concurrency": 1, | ||
| 325 | + "retrieval": { | ||
| 326 | + "weight": 0.2, | ||
| 327 | + "rerank_with_graph": false, | ||
| 328 | + "allow_graph_only": false, | ||
| 329 | + "max_graph_only_results": null, | ||
| 330 | + "seed_limit": null, | ||
| 331 | + "max_evidence_records_per_fact": null, | ||
| 332 | + "fail_open": false | ||
| 333 | + } | ||
| 270 | } | 334 | } |
| 271 | } | 335 | } |
| 272 | ``` | 336 | ``` |
| @@ -276,10 +340,15 @@ Change these fields before deployment: | |||
| 276 | - `RAM_A_XIAOO_TOKEN`: environment variable name that contains the RAM-A bearer token. | 340 | - `RAM_A_XIAOO_TOKEN`: environment variable name that contains the RAM-A bearer token. |
| 277 | - `tenant_id`, `user_id`, `agent_id`: identity scope for memory isolation. | 341 | - `tenant_id`, `user_id`, `agent_id`: identity scope for memory isolation. |
| 278 | - `permissions`: include `memory:read`, `memory:write`, and optionally `cases:read`. | 342 | - `permissions`: include `memory:read`, `memory:write`, and optionally `cases:read`. |
| 343 | + Grant `cases:write` only to trusted MCP principals allowed to prepare and confirm case writes. | ||
| 279 | - `features.memory.enabled`: expose or hide RAM-A personal long-term memory tools | 344 | - `features.memory.enabled`: expose or hide RAM-A personal long-term memory tools |
| 280 | (`memory_search`, `memory_ingest`). | 345 | (`memory_search`, `memory_ingest`). |
| 281 | -- `features.case_library.enabled`: expose or hide the operational case-library tool | 346 | +- `features.case_library.enabled`: expose or hide the case-library tools |
| 282 | - (`memory_case_search`). If this is `true`, `case_library` must also be configured. | 347 | + (`memory_case_search`, all `memory_case_prepare_*` tools, and the upload, update, and delete |
| 348 | + confirmation tools). If this is `true`, `case_library` must also be configured. | ||
| 349 | +- `features.graph_memory.enabled`: augment `memory_ingest` and `memory_search` with graph | ||
| 350 | + construction and retrieval. Keep this `false` unless the top-level `graph_memory` settings | ||
| 351 | + and `GRAPH_LLM_API_KEY` are configured. | ||
| 283 | - `allowed_hosts`: host and port used by clients to reach RAM-A. | 352 | - `allowed_hosts`: host and port used by clients to reach RAM-A. |
| 284 | - `storage.database_path`: RAM-A personal long-term memory SQLite path. | 353 | - `storage.database_path`: RAM-A personal long-term memory SQLite path. |
| 285 | - `providers.api_key_env`, `providers.base_url`, `extractor_model`, `verifier_model`: | 354 | - `providers.api_key_env`, `providers.base_url`, `extractor_model`, `verifier_model`: |
| @@ -292,6 +361,10 @@ Change these fields before deployment: | |||
| 292 | from `storage.database_path`. | 361 | from `storage.database_path`. |
| 293 | - `case_library.source_dir`: optional local directory of `.md`/`.txt` documents. When set, | 362 | - `case_library.source_dir`: optional local directory of `.md`/`.txt` documents. When set, |
| 294 | `ram-a-mem` imports new files into the default case-library dataset on startup. | 363 | `ram-a-mem` imports new files into the default case-library dataset on startup. |
| 364 | +- `case_library.api_token_env`: optional environment variable containing the dedicated | ||
| 365 | + administrator token for the case-management REST API. Omit it to keep that API disabled. | ||
| 366 | +- `case_library.ingestion_poll_ms`: polling interval for the ingestion worker embedded in | ||
| 367 | + `ram-a-mem`. The worker continuously processes document create/update tasks. | ||
| 295 | - `case_library.embedding_provider`: case-library retrieval embedding provider. It can be | 368 | - `case_library.embedding_provider`: case-library retrieval embedding provider. It can be |
| 296 | `hash` for demos or `openai_compatible` for a real/self-hosted embedding service. | 369 | `hash` for demos or `openai_compatible` for a real/self-hosted embedding service. |
| 297 | 370 | ||
| @@ -312,7 +385,10 @@ Start RAM-A memory service: | |||
| 312 | 385 | ||
| 313 | ```bash | 386 | ```bash |
| 314 | export RAM_A_XIAOO_TOKEN='replace-with-long-random-token' | 387 | export RAM_A_XIAOO_TOKEN='replace-with-long-random-token' |
| 388 | +export RAM_A_CASES_ADMIN_TOKEN='replace-with-a-separate-admin-token' | ||
| 315 | export LLM_API_KEY='replace-with-llm-provider-key' | 389 | export LLM_API_KEY='replace-with-llm-provider-key' |
| 390 | +# Required only when features.graph_memory.enabled is true. | ||
| 391 | +export GRAPH_LLM_API_KEY='replace-with-graph-provider-key' | ||
| 316 | 392 | ||
| 317 | cargo run -p memory-mcp --bin ram-a-mem | 393 | cargo run -p memory-mcp --bin ram-a-mem |
| 318 | ``` | 394 | ``` |
| @@ -327,6 +403,11 @@ Do not point `case_library.index_store` at `storage.database_path`; case index r | |||
| 327 | personal long-term memories must remain isolated even though both capabilities are served | 403 | personal long-term memories must remain isolated even though both capabilities are served |
| 328 | from the same `ram-a-mem` process and HTTP port. | 404 | from the same `ram-a-mem` process and HTTP port. |
| 329 | 405 | ||
| 406 | +When `case_library.api_token_env` is configured, `ram-a-mem` also serves the authenticated | ||
| 407 | +case-management API under `/api/v1`. Creating or updating a document enqueues an ingestion | ||
| 408 | +task; the background worker in the same process parses, chunks, and indexes it. Do not start | ||
| 409 | +a separate `memory-cases --api` or `memory-cases --ingestor` process. | ||
| 410 | + | ||
| 330 | ### MCP client config | 411 | ### MCP client config |
| 331 | 412 | ||
| 332 | For xiaoO, create `.mcp.json`; see | 413 | For xiaoO, create `.mcp.json`; see |
| @@ -374,6 +455,52 @@ are: | |||
| 374 | "case library" or name the tool. | 455 | "case library" or name the tool. |
| 375 | It searches an authorized case library when `case_library` is configured and the token has | 456 | It searches an authorized case library when `case_library` is configured and the token has |
| 376 | `cases:read`. | 457 | `cases:read`. |
| 458 | +- `memory_case_prepare_upload`: after diagnosis, stage a proposed UTF-8 Markdown/text case | ||
| 459 | + without writing it, and return a proposal plus a one-time confirmation token. | ||
| 460 | +- `memory_case_prepare_update`: stage a proposed replacement for an existing case without | ||
| 461 | + writing it, and return the same confirmation information. | ||
| 462 | +- `memory_case_prepare_delete`: stage the proposed deletion of an existing case, including a | ||
| 463 | + required deletion reason, without removing anything. | ||
| 464 | +- `memory_case_upload`: consume a prepared upload token after explicit user confirmation and | ||
| 465 | + return a generated ingestion task ID. | ||
| 466 | +- `memory_case_update`: consume a prepared update token after explicit user confirmation; | ||
| 467 | + the replacement is indexed asynchronously by the in-process ingestion worker. | ||
| 468 | +- `memory_case_delete`: consume a prepared delete token after explicit user confirmation and | ||
| 469 | + immediately remove the document, source file, tasks, chunks, and search records. | ||
| 470 | + | ||
| 471 | +All six mutation tools require `cases:write`. Preparation accepts `library` rather than | ||
| 472 | +`dataset_id`, so the server keeps dataset selection and tenant authorization under configuration | ||
| 473 | +control. After diagnosis, prepare an upload with arguments such as: | ||
| 474 | + | ||
| 475 | +```json | ||
| 476 | +{ | ||
| 477 | + "library": "ops", | ||
| 478 | + "document_id": "dns-case-001", | ||
| 479 | + "file_name": "dns-failure.md", | ||
| 480 | + "name": "DNS failure recovery", | ||
| 481 | + "diagnosis_summary": "The local resolver held a stale record; flushing it restored DNS.", | ||
| 482 | + "content": "# DNS failure\n\nFlush the resolver cache and verify the upstream DNS server." | ||
| 483 | +} | ||
| 484 | +``` | ||
| 485 | + | ||
| 486 | +Prepare a deletion with the exact document ID and a reason that xiaoO can show to the user: | ||
| 487 | + | ||
| 488 | +```json | ||
| 489 | +{ | ||
| 490 | + "library": "ops", | ||
| 491 | + "document_id": "dns-case-001", | ||
| 492 | + "deletion_reason": "This case is obsolete and its remediation is no longer safe." | ||
| 493 | +} | ||
| 494 | +``` | ||
| 495 | + | ||
| 496 | +The preparation call does not modify the case library. The client must display its proposal and | ||
| 497 | +ask the user, then end the turn. Only after a later explicit confirmation may it call the matching | ||
| 498 | +final tool with `{"confirmation_token":"...","user_confirmed":true}`. Tokens expire after ten | ||
| 499 | +minutes, are bound to tenant/user/agent and operation, are single-use, and are lost on service | ||
| 500 | +restart. Upload and update return `ingestion_status: "pending"`, and the background worker in | ||
| 501 | +`ram-a-mem` processes those tasks automatically. Delete completes synchronously and returns | ||
| 502 | +`deleted: true`. `case_library.api_token_env` is not required for these MCP tools—it only | ||
| 503 | +controls the separate REST management API. | ||
| 377 | 504 | ||
| 378 | If the client model does not reliably choose the case-library tool by itself, add the prompt | 505 | If the client model does not reliably choose the case-library tool by itself, add the prompt |
| 379 | snippet in [`plugins/mcp/case-tool-instruction.md`](plugins/mcp/case-tool-instruction.md) to | 506 | snippet in [`plugins/mcp/case-tool-instruction.md`](plugins/mcp/case-tool-instruction.md) to |
| @@ -11,3 +11,7 @@ memory-core = { path = "../memory-core" } | |||
| 11 | serde.workspace = true | 11 | serde.workspace = true |
| 12 | serde_json.workspace = true | 12 | serde_json.workspace = true |
| 13 | tokio.workspace = true | 13 | tokio.workspace = true |
| 14 | + | ||
| 15 | +[dev-dependencies] | ||
| 16 | +async-trait.workspace = true | ||
| 17 | +tempfile = "3.9" | ||