文件最后提交记录最后更新时间
Revert "feat(firecrawl): add integration tag for Hermes usage in browser and web providers" (#28862) This reverts commit 273ff5c4a47af4499bbe5e3b1139efd313995554.17 天前
feat(cross-platform): psutil for PID/process management + Windows footgun checker ## Why Hermes supports Linux, macOS, and native Windows, but the codebase grew up POSIX-first and has accumulated patterns that silently break (or worse, silently kill!) on Windows: - os.kill(pid, 0) as a liveness probe — on Windows this maps to CTRL_C_EVENT and broadcasts Ctrl+C to the target's entire console process group (bpo-14484, open since 2012). - os.killpg — doesn't exist on Windows at all (AttributeError). - os.setsid / os.getuid / os.geteuid — same. - signal.SIGKILL / signal.SIGHUP / signal.SIGUSR1 — module-attr errors at runtime on Windows. - open(path) / open(path, "r") without explicit encoding= — inherits the platform default, which is cp1252/mbcs on Windows (UTF-8 on POSIX), causing mojibake round-tripping between hosts. - wmic — removed from Windows 10 21H1+. This commit does three things: 1. Makes psutil a core dependency and migrates critical callsites to it. 2. Adds a grep-based CI gate (scripts/check-windows-footguns.py) that blocks new instances of any of the above patterns. 3. Fixes every existing instance in the codebase so the baseline is clean. ## What changed ### 1. psutil as a core dependency (pyproject.toml) Added psutil>=5.9.0,<8 to core deps. psutil is the canonical cross-platform answer for "is this PID alive" and "kill this process tree" — its pid_exists() uses OpenProcess + GetExitCodeProcess on Windows (NOT a signal call), and its Process.children(recursive=True) + .kill() combo replaces os.killpg() portably. ### 2. gateway/status.py::_pid_exists Rewrote to call psutil.pid_exists() first, falling back to the hand-rolled ctypes OpenProcess + WaitForSingleObject dance on Windows (and os.kill(pid, 0) on POSIX) only if psutil is somehow missing — e.g. during the scaffold phase of a fresh install before pip finishes. ### 3. os.killpg migration to psutil (7 callsites, 5 files) - tools/code_execution_tool.py - tools/process_registry.py - tools/tts_tool.py - tools/environments/local.py (3 sites kept as-is, suppressed with # windows-footgun: ok — the pgid semantics psutil can't replicate, and the calls are already Windows-guarded at the outer branch) - gateway/platforms/whatsapp.py ### 4. scripts/check-windows-footguns.py (NEW, 500 lines) Grep-based checker with 11 rules covering every Windows cross-platform footgun we've hit so far: 1. os.kill(pid, 0) — the silent killer 2. os.setsid without guard 3. os.killpg (recommends psutil) 4. os.getuid / os.geteuid / os.getgid 5. os.fork 6. signal.SIGKILL 7. signal.SIGHUP/SIGUSR1/SIGUSR2/SIGALRM/SIGCHLD/SIGPIPE/SIGQUIT 8. subprocess shebang script invocation 9. wmic without shutil.which guard 10. Hardcoded ~/Desktop (OneDrive trap) 11. asyncio.add_signal_handler without try/except 12. open() without encoding= on text mode Features: - Triple-quoted-docstring aware (won't flag prose inside docstrings) - Trailing-comment aware (won't flag mentions in # os.kill(pid, 0) comments) - Guard-hint aware (skips lines with hasattr(os, ...), shutil.which(...), if platform.system() != 'Windows', etc.) - Inline suppression with # windows-footgun: ok — <reason> - --list to print all rules with fixes - --all / --diff <ref> / staged-files (default) modes - Scans 380 files in under 2 seconds ### 5. CI integration A GitHub Actions workflow that runs the checker on every PR and push is staged at /tmp/hermes-stash/windows-footguns.yml — not included in this commit because the GH token on the push machine lacks workflow scope. A maintainer with workflow permissions should add it as .github/workflows/windows-footguns.yml in a follow-up. Content: ```yaml name: Windows footgun check on: push: branches: [main] pull_request: branches: [main] jobs: check: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v5 with: {python-version: "3.11"} - run: python scripts/check-windows-footguns.py --all ``` ### 6. CONTRIBUTING.md — "Cross-Platform Compatibility" expansion Expanded from 5 to 16 rules, each with message, example, and fix. Recommends psutil as the preferred API for PID / process-tree operations. ### 7. Baseline cleanup (91 → 0 findings) - 14 open() sites → added encoding='utf-8' (internal logs/caches) or encoding='utf-8-sig' (user-editable files that Notepad may BOM) - 23 POSIX-only callsites in systemd helpers, pty_bridge, and plugin tool subprocess management → annotated with # windows-footgun: ok — <reason> - 7 os.killpg sites → migrated to psutil (see §3 above) ## Verification ``` $ python scripts/check-windows-footguns.py --all ✓ No Windows footguns found (380 file(s) scanned). $ python -c "from gateway.status import _pid_exists; import os > print('self:', _pid_exists(os.getpid())); print('bogus:', _pid_exists(999999))" self: True bogus: False ``` Proof-of-repro that os.kill(pid, 0) was actually killing processes before this fix — see commit 1cbe39914 and bpo-14484. This commit removes the last hand-rolled ctypes path from the hot liveness-check path and defers to the best-maintained cross-platform answer. 27 天前
chore: ruff auto-fix PLR6201 resweep — tuple → set in membership tests (#27355) Six days after #23937 (608 fixes) the codebase had accumulated 241 new PLR6201 violations. Same mechanical x in (...)x in {...} fix, same zero-risk profile: set lookup is O(1) vs O(n) for tuple and the two are semantically equivalent for hashable scalar membership tests. All 241 instances fixed via `ruff check --select PLR6201 --fix --unsafe-fixes`, zero remaining. Every changed value is a hashable scalar (str/int/None/enum/signal); no risk of unhashable runtime errors. No behavior change. Test plan: - 119 files changed, +244/-244 (net zero) — exactly one-line edits - ruff check clean afterward - Compile checks pass on the largest touched files (cli.py, run_agent.py, gateway/run.py, gateway/platforms/discord.py, model_tools.py) - Subset broad test run on tests/gateway/ tests/hermes_cli/ tests/agent/ tests/tools/: 18187 passed, 59 pre-existing failures (verified against origin/main with the same shape — identical failure count, identical category — all xdist test-order flakes unrelated to this change) Follows the same template as PR #23937 ([tracker: #23972](https://github.com/NousResearch/hermes-agent/issues/23972)).19 天前
fix(dashboard): UI polish — modals, layout, consistency, test fixes Dashboard UX polish pass — consolidates create forms into modals triggered from the page header, fixes layout inconsistencies, adds scroll-to navigation for the Keys page, and aligns the TokenBar with the design system. Changes: - App.tsx: add padding to sidebar header - resolve-page-title.ts: add missing routes, better fallback title - en.ts: fix nav labels (Profiles was 'profiles : multi agents') - ModelsPage: two-col layout, auxiliary tasks modal, TokenBar redesign - ProfilesPage: create button in header, form in modal, Checkbox component - CronPage: create button in header, form in modal - EnvPage: scroll-to sub-nav in header, fix text overflow Modal and dialog standardization: - Replace all native confirm()/window.confirm() with ConfirmDialog (OAuthProvidersCard, PluginsPage, ModelsPage, ConfigPage) - Add useModalBehavior hook (Escape-to-close, scroll lock, focus restore) - Apply hook to ProfilesPage, CronPage, AuxiliaryTasksModal Component fixes (from PR review): - Checkbox: fix controlled/uncontrolled mismatch, add focus-visible ring - TokenBar: add rounded-full to legend dots, remove dead code CI/test fixes: - Fix TS unused imports (noUnusedLocals), type-narrow PickerTarget union - Add windows-footgun suppression on platform-guarded os.killpg - Fix 19 stale unit tests + 9 e2e tests broken by recent main changes - Restore minimal example-dashboard plugin for plugin auth test 24 天前
chore: ruff auto-fix PLR6201 resweep — tuple → set in membership tests (#27355) Six days after #23937 (608 fixes) the codebase had accumulated 241 new PLR6201 violations. Same mechanical x in (...)x in {...} fix, same zero-risk profile: set lookup is O(1) vs O(n) for tuple and the two are semantically equivalent for hashable scalar membership tests. All 241 instances fixed via `ruff check --select PLR6201 --fix --unsafe-fixes`, zero remaining. Every changed value is a hashable scalar (str/int/None/enum/signal); no risk of unhashable runtime errors. No behavior change. Test plan: - 119 files changed, +244/-244 (net zero) — exactly one-line edits - ruff check clean afterward - Compile checks pass on the largest touched files (cli.py, run_agent.py, gateway/run.py, gateway/platforms/discord.py, model_tools.py) - Subset broad test run on tests/gateway/ tests/hermes_cli/ tests/agent/ tests/tools/: 18187 passed, 59 pre-existing failures (verified against origin/main with the same shape — identical failure count, identical category — all xdist test-order flakes unrelated to this change) Follows the same template as PR #23937 ([tracker: #23972](https://github.com/NousResearch/hermes-agent/issues/23972)).19 天前
fix(achievements): use canonical X-Hermes-Session-Token header Follow-up to TreyDong's fix: switch the auth header to X-Hermes-Session-Token (the canonical pattern used by the rest of the dashboard SPA — see web/src/lib/api.ts fetchJSON()). The server still accepts both schemes, so the original `Authorization: Bearer` form would also work; we standardize on X-header to match every other dashboard fetch and only set the header when a token is actually present. Also add scripts/release.py AUTHOR_MAP entry for treydong.zh@gmail.com. 25 天前
feat(xai-oauth): add xAI Grok OAuth (SuperGrok Subscription) provider Adds a new authentication provider that lets SuperGrok subscribers sign in to Hermes with their xAI account via the standard OAuth 2.0 PKCE loopback flow, instead of pasting a raw API key from console.x.ai. Highlights ---------- * OAuth 2.0 PKCE loopback login against accounts.x.ai with discovery, state/nonce, and a strict CORS-origin allowlist on the callback. * Authorize URL carries plan=generic (required for non-allowlisted loopback clients) and referrer=hermes-agent for best-effort attribution in xAI's OAuth server logs. * Token storage in auth.json with file-locked atomic writes; JWT exp-based expiry detection with skew; refresh-token rotation synced both ways between the singleton store and the credential pool so multi-process / multi-profile setups don't tear each other's refresh tokens. * Reactive 401 retry: on a 401 from the xAI Responses API, the agent refreshes the token, swaps it back into self.api_key, and retries the call once. Guarded against silent account swaps when the active key was sourced from a different (manual) pool entry. * Auxiliary tasks (curator, vision, embeddings, etc.) route through a dedicated xAI Responses-mode auxiliary client instead of falling back to OpenRouter billing. * Direct HTTP tools (tools/xai_http.py, transcription, TTS, image-gen plugin) resolve credentials through a unified runtime → singleton → env-var fallback chain so xai-oauth users get them for free. * hermes auth add xai-oauth and hermes auth remove xai-oauth N are wired through the standard auth-commands surface; remove cleans up the singleton loopback_pkce entry so it doesn't silently reinstate. * hermes model provider picker shows "xAI Grok OAuth (SuperGrok Subscription)" and the model-flow falls back to pool credentials when the singleton is missing. Hardening --------- * Discovery and refresh responses validate the returned token_endpoint host against the same *.x.ai allowlist as the authorization endpoint, blocking MITM persistence of a hostile endpoint. * Discovery / refresh / token-exchange response.json() calls are wrapped to raise typed AuthError on malformed bodies (captive portals, proxy error pages) instead of leaking JSONDecodeError tracebacks. * prompt_cache_key is routed through extra_body on the codex transport (sending it as a top-level kwarg trips xAI's SDK with a TypeError). * Credential-pool sync-back preserves active_provider so refreshing an OAuth entry doesn't silently flip the active provider out from under the running agent. Testing ------- * New tests/hermes_cli/test_auth_xai_oauth_provider.py (~63 tests) covers JWT expiry, OAuth URL params (plan + referrer), CORS origins, redirect URI validation, singleton↔pool sync, concurrency races, refresh error paths, runtime resolution, and malformed-JSON guards. * Extended test_credential_pool.py, test_codex_transport.py, and test_run_agent_codex_responses.py cover the pool sync-back, extra_body routing, and 401 reactive refresh paths. * 165 tests passing on this branch via scripts/run_tests.sh. 21 天前
feat(web): migrate dashboard checkboxes to @nous-research/ui + DS polish (#28814) * feat(web): migrate dashboard checkboxes to @nous-research/ui + DS polish Replaces the hand-rolled shadcn-style Checkbox in web/src/components/ui/ with the Nous DS Checkbox (Radix-backed) from @nous-research/ui, bumps the DS to 0.14.2, and picks up two regressions surfaced by the bump. Checkbox migration - bump @nous-research/ui 0.14.0 → ^0.14.2 and remove web/src/components/ui/checkbox.tsx - migrate ProfilesPage and ModelPickerDialog to the DS Checkbox API (onCheckedChange, paired <Label htmlFor>) - expose Checkbox on the dashboard plugin SDK (web/src/plugins/registry.ts) so plugin bundles can use the same DS component - migrate the kanban dashboard plugin's 7 native <input type="checkbox"> call sites to the SDK Checkbox, with a native-input fallback shim so the bundle still renders against older hosts that predate the SDK export Fix: missing font registrations after the 0.14.x split - import @nous-research/ui/styles/fonts.css before globals.css in web/src/index.css. As of 0.14.x, globals.css only declares the --font-* variables (Collapse, Mondwest, Rules Compressed/Expanded); the @font-face registrations now live in a separate fonts.css, so without this import the DS components silently fall back to a system font stack and look unstyled. Fix: right-align page header toolbars on sm+ viewports - The mobile dashboard polish in #28127 flipped four pages' setEnd(...) wrappers from justify-end to w-full ... justify-start so toolbars stack below the title and align left on small screens. But the outer end slot in PageHeaderProvider already has sm:justify-end, and that has no effect when its only child is w-full — once a flex child fills the row, the parent's justify-* can't move it. The toolbar pinned to the *left* of the right-side sm:max-w-md (~448px) slot, making the buttons appear to float a couple-hundred pixels off the right edge on Analytics, Models, Logs, and Plugins. - Re-add sm:justify-end on the inner wrapper of each affected page, preserving the mobile stacked layout. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(nix): update web npmDeps hash for package-lock bump Co-authored-by: Cursor <cursoragent@cursor.com> * fix(nix): refresh npm lockfile hashes * chore(ci): re-trigger checks after nix lockfile hash fix Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>16 天前
chore: ruff auto-fix PLR6201 resweep — tuple → set in membership tests (#27355) Six days after #23937 (608 fixes) the codebase had accumulated 241 new PLR6201 violations. Same mechanical x in (...)x in {...} fix, same zero-risk profile: set lookup is O(1) vs O(n) for tuple and the two are semantically equivalent for hashable scalar membership tests. All 241 instances fixed via `ruff check --select PLR6201 --fix --unsafe-fixes`, zero remaining. Every changed value is a hashable scalar (str/int/None/enum/signal); no risk of unhashable runtime errors. No behavior change. Test plan: - 119 files changed, +244/-244 (net zero) — exactly one-line edits - ruff check clean afterward - Compile checks pass on the largest touched files (cli.py, run_agent.py, gateway/run.py, gateway/platforms/discord.py, model_tools.py) - Subset broad test run on tests/gateway/ tests/hermes_cli/ tests/agent/ tests/tools/: 18187 passed, 59 pre-existing failures (verified against origin/main with the same shape — identical failure count, identical category — all xdist test-order flakes unrelated to this change) Follows the same template as PR #23937 ([tracker: #23972](https://github.com/NousResearch/hermes-agent/issues/23972)).19 天前
feat(azure-foundry): add Microsoft Entra ID auth Use azure-identity DefaultAzureCredential for keyless Foundry auth. Preserve refreshable callable credentials through OpenAI and Anthropic client paths. Add setup, doctor, auth status, docs, and tests for Entra auth. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> 18 天前
fix(plugins): remove unreachable hermes tools → Langfuse path The langfuse plugin is hooks-only (no toolsets), so it never appears in hermes tools — that menu iterates _get_effective_configurable_toolsets() (= CONFIGURABLE_TOOLSETS + plugin-registered toolsets), and "langfuse" is in neither. The TOOL_CATEGORIES["langfuse"] setup wizard (with its post_setup: "langfuse" hook that pip-installs the SDK and writes plugins.enabled) was reachable only when a toolset key "langfuse" got enabled, which can't happen — so it's been dead code, and the docs that promised "Setup (interactive): hermes tools → Langfuse Observability" were silently broken. Right home for that wizard is hermes plugins (e.g. auto-running a plugin's post-setup hook on enable), which is a generic plugin-setup mechanism worth designing properly rather than shoehorning langfuse back into hermes tools. Until that exists, point users at the working manual flow. Code: - Delete TOOL_CATEGORIES["langfuse"] (24 lines) — unreachable. - Delete the post_setup_key == "langfuse" branch in _run_post_setup (29 lines) — only caller was the deleted TOOL_CATEGORIES entry. Docs / comments (point at the manual flow + interactive hermes plugins): - plugins/observability/langfuse/README.md: collapse the two-option setup section to the single working flow. - plugins/observability/langfuse/plugin.yaml: update description. - plugins/observability/langfuse/__init__.py: update module docstring. - hermes_cli/config.py: update inline comment above the LANGFUSE_* env-var allow-list. - website/docs/user-guide/features/built-in-plugins.md: collapse "Setup (interactive)" + "Setup (manual)" into one accurate block. - website/docs/reference/environment-variables.md: update the cross-reference in the Langfuse env-vars section. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> 19 天前
fix(skills): add timeout to Google OAuth urlopen calls 17 天前
refactor(spotify): convert to built-in bundled plugin under plugins/spotify (#15174) Moves the Spotify integration from tools/ into plugins/spotify/, matching the existing pattern established by plugins/image_gen/ for third-party service integrations. Why: - tools/ should be reserved for foundational capabilities (terminal, read_file, web_search, etc.). tools/providers/ was a one-off directory created solely for spotify_client.py. - plugins/ is already the home for image_gen backends, memory providers, context engines, and standalone hook-based plugins. Spotify is a third-party service integration and belongs alongside those, not in tools/. - Future service integrations (eventually: Deezer, Apple Music, etc.) now have a pattern to copy. Changes: - tools/spotify_tool.py → plugins/spotify/tools.py (handlers + schemas) - tools/providers/spotify_client.py → plugins/spotify/client.py - tools/providers/ removed (was only used for Spotify) - New plugins/spotify/__init__.py with register(ctx) calling ctx.register_tool() × 7. The handler/check_fn wiring is unchanged. - New plugins/spotify/plugin.yaml (kind: backend, bundled, auto-load). - tests/tools/test_spotify_client.py: import paths updated. tools_config fix — _DEFAULT_OFF_TOOLSETS now wins over plugin auto-enable: - _get_platform_tools() previously auto-enabled unknown plugin toolsets for new platforms. That was fine for image_gen (which has no toolset of its own) but bad for Spotify, which explicitly requires opt-in (don't ship 7 tool schemas to users who don't use it). Added a check: if a plugin toolset is in _DEFAULT_OFF_TOOLSETS, it stays off until the user picks it in 'hermes tools'. Pre-existing test bug fix: - tests/hermes_cli/test_plugins.py::test_list_returns_sorted asserted names were sorted, but list_plugins() sorts by key (path-derived, e.g. image_gen/openai). With only image_gen plugins bundled, name and key order happened to agree. Adding plugins/spotify broke that coincidence (spotify sorts between openai-codex and xai by name but after xai by key). Updated test to assert key order, which is what the code actually documents. Validation: - scripts/run_tests.sh tests/hermes_cli/test_plugins.py \ tests/hermes_cli/test_tools_config.py \ tests/hermes_cli/test_spotify_auth.py \ tests/tools/test_spotify_client.py \ tests/tools/test_registry.py → 143 passed - E2E plugin load: 'spotify' appears in loaded plugins, all 7 tools register into the spotify toolset, check_fn gating intact.1 个月前
chore: ruff auto-fix PLR6201 resweep — tuple → set in membership tests (#27355) Six days after #23937 (608 fixes) the codebase had accumulated 241 new PLR6201 violations. Same mechanical x in (...)x in {...} fix, same zero-risk profile: set lookup is O(1) vs O(n) for tuple and the two are semantically equivalent for hashable scalar membership tests. All 241 instances fixed via `ruff check --select PLR6201 --fix --unsafe-fixes`, zero remaining. Every changed value is a hashable scalar (str/int/None/enum/signal); no risk of unhashable runtime errors. No behavior change. Test plan: - 119 files changed, +244/-244 (net zero) — exactly one-line edits - ruff check clean afterward - Compile checks pass on the largest touched files (cli.py, run_agent.py, gateway/run.py, gateway/platforms/discord.py, model_tools.py) - Subset broad test run on tests/gateway/ tests/hermes_cli/ tests/agent/ tests/tools/: 18187 passed, 59 pre-existing failures (verified against origin/main with the same shape — identical failure count, identical category — all xdist test-order flakes unrelated to this change) Follows the same template as PR #23937 ([tracker: #23972](https://github.com/NousResearch/hermes-agent/issues/23972)).19 天前
feat(xai-oauth): add xAI Grok OAuth (SuperGrok Subscription) provider Adds a new authentication provider that lets SuperGrok subscribers sign in to Hermes with their xAI account via the standard OAuth 2.0 PKCE loopback flow, instead of pasting a raw API key from console.x.ai. Highlights ---------- * OAuth 2.0 PKCE loopback login against accounts.x.ai with discovery, state/nonce, and a strict CORS-origin allowlist on the callback. * Authorize URL carries plan=generic (required for non-allowlisted loopback clients) and referrer=hermes-agent for best-effort attribution in xAI's OAuth server logs. * Token storage in auth.json with file-locked atomic writes; JWT exp-based expiry detection with skew; refresh-token rotation synced both ways between the singleton store and the credential pool so multi-process / multi-profile setups don't tear each other's refresh tokens. * Reactive 401 retry: on a 401 from the xAI Responses API, the agent refreshes the token, swaps it back into self.api_key, and retries the call once. Guarded against silent account swaps when the active key was sourced from a different (manual) pool entry. * Auxiliary tasks (curator, vision, embeddings, etc.) route through a dedicated xAI Responses-mode auxiliary client instead of falling back to OpenRouter billing. * Direct HTTP tools (tools/xai_http.py, transcription, TTS, image-gen plugin) resolve credentials through a unified runtime → singleton → env-var fallback chain so xai-oauth users get them for free. * hermes auth add xai-oauth and hermes auth remove xai-oauth N are wired through the standard auth-commands surface; remove cleans up the singleton loopback_pkce entry so it doesn't silently reinstate. * hermes model provider picker shows "xAI Grok OAuth (SuperGrok Subscription)" and the model-flow falls back to pool credentials when the singleton is missing. Hardening --------- * Discovery and refresh responses validate the returned token_endpoint host against the same *.x.ai allowlist as the authorization endpoint, blocking MITM persistence of a hostile endpoint. * Discovery / refresh / token-exchange response.json() calls are wrapped to raise typed AuthError on malformed bodies (captive portals, proxy error pages) instead of leaking JSONDecodeError tracebacks. * prompt_cache_key is routed through extra_body on the codex transport (sending it as a top-level kwarg trips xAI's SDK with a TypeError). * Credential-pool sync-back preserves active_provider so refreshing an OAuth entry doesn't silently flip the active provider out from under the running agent. Testing ------- * New tests/hermes_cli/test_auth_xai_oauth_provider.py (~63 tests) covers JWT expiry, OAuth URL params (plan + referrer), CORS origins, redirect URI validation, singleton↔pool sync, concurrency races, refresh error paths, runtime resolution, and malformed-JSON guards. * Extended test_credential_pool.py, test_codex_transport.py, and test_run_agent_codex_responses.py cover the pool sync-back, extra_body routing, and 401 reactive refresh paths. * 165 tests passing on this branch via scripts/run_tests.sh. 21 天前
feat(web): add xAI Web Search provider plugin Adds a new bundled web search provider plugin backed by xAI's agentic Web Search tool (server-side web_search on the Responses API). Slots in alongside the existing Firecrawl / Tavily / Exa / Brave / SearXNG / DDGS providers; opt in via web.backend: xai (or auto-selected by the registry's single-provider shortcut when it's the only available web provider, matching every other backend's behavior). Reuses the existing xAI HTTP credential plumbing (tools/xai_http.py) so it works with both hermes auth login xai-oauth (SuperGrok OAuth) and XAI_API_KEY — no new credential paths, no new env vars, no new setup-wizard prompts. The existing xai_grok post_setup hook handles credential collection. Reference: https://docs.x.ai/developers/tools/web-search Provider behavior ----------------- - Sends a structured prompt to Grok with tools=[{"type": "web_search"}] enabled and include=["no_inline_citations"], then parses results from a {"results": [...]} JSON block (primary), falling back to url_citation annotations (secondary) and the top-level citations list (last-ditch). Annotation fallback falls through to citations when no rows are extractable, so future annotation types xAI may add don't silently mask real data. - HTTP 200 + {"error": {...}} envelopes (model-overload, refusal) are surfaced as failures rather than masked as success-with-empty- results. - HTTP 401 on the OAuth path triggers a single force_refresh=True retry — closes two gaps the resolver's proactive JWT-exp shortcut doesn't cover: opaque (non-JWT) access tokens and mid-window revocation. Env-var (XAI_API_KEY) credentials never retry; they can't be refreshed and an immediate retry would just burn quota. - is_available() is a cheap probe (env var OR auth.json read), never invokes the OAuth resolver — required by the ABC contract because it runs on every hermes tools repaint and at tool-registration time. - Class docstring documents the LLM-in-a-trench-coat trust model so callers piping untrusted input into web_search know returned URLs are model-generated and should be validated before fetching. Config (config.yaml): web: backend: xai xai: model: grok-4.3 # optional, defaults to grok-4.3 allowed_domains: # optional, max 5 — mutex with excluded_domains - arxiv.org excluded_domains: # optional, max 5 - example-spam.com timeout: 90 # optional, seconds Files ----- - plugins/web/xai/plugin.yaml (new) plugin manifest - plugins/web/xai/__init__.py (new) register(ctx) hook - plugins/web/xai/provider.py (new) XAIWebSearchProvider impl - tools/xai_http.py (+47) has_xai_credentials() cheap-probe helper + keyword-only force_refresh arg on resolve_xai_http_ credentials() (backwards compatible; all 9 other call sites unaffected) - tools/web_tools.py (+11) "xai" added to configured- backend set + branch in _is_backend_available() - tests/tools/test_web_providers_xai.py (new, 39 tests) covers identity, cheap-probe semantics, JSON / annotation / citations parse paths, request payload shape, error envelopes, OAuth force-refresh-on-401 retry, env-var-no-retry guard, 500-not- retried guard, refresh-returns- same-token guard, OAuth runtime resolution, and backend wiring. Tests ----- - 39 xai-suite passes - 79 sibling web-provider tests (brave-free, ddgs, searxng, base) pass - 119 cross-suite tests for other xai_http callers (transcription, x_search, tts) pass — verifies the new keyword-only arg is BC - scripts/check-windows-footguns.py: clean on all 5 modified files No edits to run_agent.py, cli.py, gateway/, toolsets, config schema, plugin core, or auth core. 16 天前
feat(memory): pluggable memory provider interface with profile isolation, review fixes, and honcho CLI restoration (#4623) * feat(memory): add pluggable memory provider interface with profile isolation Introduces a pluggable MemoryProvider ABC so external memory backends can integrate with Hermes without modifying core files. Each backend becomes a plugin implementing a standard interface, orchestrated by MemoryManager. Key architecture: - agent/memory_provider.py — ABC with core + optional lifecycle hooks - agent/memory_manager.py — single integration point in the agent loop - agent/builtin_memory_provider.py — wraps existing MEMORY.md/USER.md Profile isolation fixes applied to all 6 shipped plugins: - Cognitive Memory: use get_hermes_home() instead of raw env var - Hindsight Memory: check $HERMES_HOME/hindsight/config.json first, fall back to legacy ~/.hindsight/ for backward compat - Hermes Memory Store: replace hardcoded ~/.hermes paths with get_hermes_home() for config loading and DB path defaults - Mem0 Memory: use get_hermes_home() instead of raw env var - RetainDB Memory: auto-derive profile-scoped project name from hermes_home path (hermes-<profile>), explicit env var overrides - OpenViking Memory: read-only, no local state, isolation via .env MemoryManager.initialize_all() now injects hermes_home into kwargs so every provider can resolve profile-scoped storage without importing get_hermes_home() themselves. Plugin system: adds register_memory_provider() to PluginContext and get_plugin_memory_providers() accessor. Based on PR #3825. 46 tests (37 unit + 5 E2E + 4 plugin registration). * refactor(memory): drop cognitive plugin, rewrite OpenViking as full provider Remove cognitive-memory plugin (#727) — core mechanics are broken: decay runs 24x too fast (hourly not daily), prefetch uses row ID as timestamp, search limited by importance not similarity. Rewrite openviking-memory plugin from a read-only search wrapper into a full bidirectional memory provider using the complete OpenViking session lifecycle API: - sync_turn: records user/assistant messages to OpenViking session (threaded, non-blocking) - on_session_end: commits session to trigger automatic memory extraction into 6 categories (profile, preferences, entities, events, cases, patterns) - prefetch: background semantic search via find() endpoint - on_memory_write: mirrors built-in memory writes to the session - is_available: checks env var only, no network calls (ABC compliance) Tools expanded from 3 to 5: - viking_search: semantic search with mode/scope/limit - viking_read: tiered content (abstract ~100tok / overview ~2k / full) - viking_browse: filesystem-style navigation (list/tree/stat) - viking_remember: explicit memory storage via session - viking_add_resource: ingest URLs/docs into knowledge base Uses direct HTTP via httpx (no openviking SDK dependency needed). Response truncation on viking_read to prevent context flooding. * fix(memory): harden Mem0 plugin — thread safety, non-blocking sync, circuit breaker - Remove redundant mem0_context tool (identical to mem0_search with rerank=true, top_k=5 — wastes a tool slot and confuses the model) - Thread sync_turn so it's non-blocking — Mem0's server-side LLM extraction can take 5-10s, was stalling the agent after every turn - Add threading.Lock around _get_client() for thread-safe lazy init (prefetch and sync threads could race on first client creation) - Add circuit breaker: after 5 consecutive API failures, pause calls for 120s instead of hammering a down server every turn. Auto-resets after cooldown. Logs a warning when tripped. - Track success/failure in prefetch, sync_turn, and all tool calls - Wait for previous sync to finish before starting a new one (prevents unbounded thread accumulation on rapid turns) - Clean up shutdown to join both prefetch and sync threads * fix(memory): enforce single external memory provider limit MemoryManager now rejects a second non-builtin provider with a warning. Built-in memory (MEMORY.md/USER.md) is always accepted. Only ONE external plugin provider is allowed at a time. This prevents tool schema bloat (some providers add 3-5 tools each) and conflicting memory backends. The warning message directs users to configure memory.provider in config.yaml to select which provider to activate. Updated all 47 tests to use builtin + one external pattern instead of multiple externals. Added test_second_external_rejected to verify the enforcement. * feat(memory): add ByteRover memory provider plugin Implements the ByteRover integration (from PR #3499 by hieuntg81) as a MemoryProvider plugin instead of direct run_agent.py modifications. ByteRover provides persistent memory via the brv CLI — a hierarchical knowledge tree with tiered retrieval (fuzzy text then LLM-driven search). Local-first with optional cloud sync. Plugin capabilities: - prefetch: background brv query for relevant context - sync_turn: curate conversation turns (threaded, non-blocking) - on_memory_write: mirror built-in memory writes to brv - on_pre_compress: extract insights before context compression Tools (3): - brv_query: search the knowledge tree - brv_curate: store facts/decisions/patterns - brv_status: check CLI version and context tree state Profile isolation: working directory at $HERMES_HOME/byterover/ (scoped per profile). Binary resolution cached with thread-safe double-checked locking. All write operations threaded to avoid blocking the agent (curate can take 120s with LLM processing). * fix(memory): thread remaining sync_turns, fix holographic, add config key Plugin fixes: - Hindsight: thread sync_turn (was blocking up to 30s via _run_in_thread) - RetainDB: thread sync_turn (was blocking on HTTP POST) - Both: shutdown now joins sync threads alongside prefetch threads Holographic retrieval fixes: - reason(): removed dead intersection_key computation (bundled but never used in scoring). Now reuses pre-computed entity_residuals directly, moved role_content encoding outside the inner loop. - contradict(): added _MAX_CONTRADICT_FACTS=500 scaling guard. Above 500 facts, only checks the most recently updated ones to avoid O(n^2) explosion (~125K comparisons at 500 is acceptable). Config: - Added memory.provider key to DEFAULT_CONFIG ("" = builtin only). No version bump needed (deep_merge handles new keys automatically). * feat(memory): extract Honcho as a MemoryProvider plugin Creates plugins/honcho-memory/ as a thin adapter over the existing honcho_integration/ package. All 4 Honcho tools (profile, search, context, conclude) move from the normal tool registry to the MemoryProvider interface. The plugin delegates all work to HonchoSessionManager — no Honcho logic is reimplemented. It uses the existing config chain: $HERMES_HOME/honcho.json -> ~/.honcho/config.json -> env vars. Lifecycle hooks: - initialize: creates HonchoSessionManager via existing client factory - prefetch: background dialectic query - sync_turn: records messages + flushes to API (threaded) - on_memory_write: mirrors user profile writes as conclusions - on_session_end: flushes all pending messages This is a prerequisite for the MemoryManager wiring in run_agent.py. Once wired, Honcho goes through the same provider interface as all other memory plugins, and the scattered Honcho code in run_agent.py can be consolidated into the single MemoryManager integration point. * feat(memory): wire MemoryManager into run_agent.py Adds 8 integration points for the external memory provider plugin, all purely additive (zero existing code modified): 1. Init (~L1130): Create MemoryManager, find matching plugin provider from memory.provider config, initialize with session context 2. Tool injection (~L1160): Append provider tool schemas to self.tools and self.valid_tool_names after memory_manager init 3. System prompt (~L2705): Add external provider's system_prompt_block alongside existing MEMORY.md/USER.md blocks 4. Tool routing (~L5362): Route provider tool calls through memory_manager.handle_tool_call() before the catchall handler 5. Memory write bridge (~L5353): Notify external provider via on_memory_write() when the built-in memory tool writes 6. Pre-compress (~L5233): Call on_pre_compress() before context compression discards messages 7. Prefetch (~L6421): Inject provider prefetch results into the current-turn user message (same pattern as Honcho turn context) 8. Turn sync + session end (~L8161, ~L8172): sync_all() after each completed turn, queue_prefetch_all() for next turn, on_session_end() + shutdown_all() at conversation end All hooks are wrapped in try/except — a failing provider never breaks the agent. The existing memory system, Honcho integration, and all other code paths are completely untouched. Full suite: 7222 passed, 4 pre-existing failures. * refactor(memory): remove legacy Honcho integration from core Extracts all Honcho-specific code from run_agent.py, model_tools.py, toolsets.py, and gateway/run.py. Honcho is now exclusively available as a memory provider plugin (plugins/honcho-memory/). Removed from run_agent.py (-457 lines): - Honcho init block (session manager creation, activation, config) - 8 Honcho methods: _honcho_should_activate, _strip_honcho_tools, _activate_honcho, _register_honcho_exit_hook, _queue_honcho_prefetch, _honcho_prefetch, _honcho_save_user_observation, _honcho_sync - _inject_honcho_turn_context module-level function - Honcho system prompt block (tool descriptions, CLI commands) - Honcho context injection in api_messages building - Honcho params from __init__ (honcho_session_key, honcho_manager, honcho_config) - HONCHO_TOOL_NAMES constant - All honcho-specific tool dispatch forwarding Removed from other files: - model_tools.py: honcho_tools import, honcho params from handle_function_call - toolsets.py: honcho toolset definition, honcho tools from core tools list - gateway/run.py: honcho params from AIAgent constructor calls Removed tests (-339 lines): - 9 Honcho-specific test methods from test_run_agent.py - TestHonchoAtexitFlush class from test_exit_cleanup_interrupt.py Restored two regex constants (_SURROGATE_RE, _BUDGET_WARNING_RE) that were accidentally removed during the honcho function extraction. The honcho_integration/ package is kept intact — the plugin delegates to it. tools/honcho_tools.py registry entries are now dead code (import commented out in model_tools.py) but the file is preserved for reference. Full suite: 7207 passed, 4 pre-existing failures. Zero regressions. * refactor(memory): restructure plugins, add CLI, clean gateway, migration notice Plugin restructure: - Move all memory plugins from plugins/<name>-memory/ to plugins/memory/<name>/ (byterover, hindsight, holographic, honcho, mem0, openviking, retaindb) - New plugins/memory/__init__.py discovery module that scans the directory directly, loading providers by name without the general plugin system - run_agent.py uses load_memory_provider() instead of get_plugin_memory_providers() CLI wiring: - hermes memory setup — interactive curses picker + config wizard - hermes memory status — show active provider, config, availability - hermes memory off — disable external provider (built-in only) - hermes honcho — now shows migration notice pointing to hermes memory setup Gateway cleanup: - Remove _get_or_create_gateway_honcho (already removed in prev commit) - Remove _shutdown_gateway_honcho and _shutdown_all_gateway_honcho methods - Remove all calls to shutdown methods (4 call sites) - Remove _honcho_managers/_honcho_configs dict references Dead code removal: - Delete tools/honcho_tools.py (279 lines, import was already commented out) - Delete tests/gateway/test_honcho_lifecycle.py (131 lines, tested removed methods) - Remove if False placeholder from run_agent.py Migration: - Honcho migration notice on startup: detects existing honcho.json or ~/.honcho/config.json, prints guidance to run hermes memory setup. Only fires when memory.provider is not set and not in quiet mode. Full suite: 7203 passed, 4 pre-existing failures. Zero regressions. * feat(memory): standardize plugin config + add per-plugin documentation Config architecture: - Add save_config(values, hermes_home) to MemoryProvider ABC - Honcho: writes to $HERMES_HOME/honcho.json (SDK native) - Mem0: writes to $HERMES_HOME/mem0.json - Hindsight: writes to $HERMES_HOME/hindsight/config.json - Holographic: writes to config.yaml under plugins.hermes-memory-store - OpenViking/RetainDB/ByteRover: env-var only (default no-op) Setup wizard (hermes memory setup): - Now calls provider.save_config() for non-secret config - Secrets still go to .env via env vars - Only memory.provider activation key goes to config.yaml Documentation: - README.md for each of the 7 providers in plugins/memory/<name>/ - Requirements, setup (wizard + manual), config reference, tools table - Consistent format across all providers The contract for new memory plugins: - get_config_schema() declares all fields (REQUIRED) - save_config() writes native config (REQUIRED if not env-var-only) - Secrets use env_var field in schema, written to .env by wizard - README.md in the plugin directory * docs: add memory providers user guide + developer guide New pages: - user-guide/features/memory-providers.md — comprehensive guide covering all 7 shipped providers (Honcho, OpenViking, Mem0, Hindsight, Holographic, RetainDB, ByteRover). Each with setup, config, tools, cost, and unique features. Includes comparison table and profile isolation notes. - developer-guide/memory-provider-plugin.md — how to build a new memory provider plugin. Covers ABC, required methods, config schema, save_config, threading contract, profile isolation, testing. Updated pages: - user-guide/features/memory.md — replaced Honcho section with link to new Memory Providers page - user-guide/features/honcho.md — replaced with migration redirect to the new Memory Providers page - sidebars.ts — added both new pages to navigation * fix(memory): auto-migrate Honcho users to memory provider plugin When honcho.json or ~/.honcho/config.json exists but memory.provider is not set, automatically set memory.provider: honcho in config.yaml and activate the plugin. The plugin reads the same config files, so all data and credentials are preserved. Zero user action needed. Persists the migration to config.yaml so it only fires once. Prints a one-line confirmation in non-quiet mode. * fix(memory): only auto-migrate Honcho when enabled + credentialed Check HonchoClientConfig.enabled AND (api_key OR base_url) before auto-migrating — not just file existence. Prevents false activation for users who disabled Honcho, stopped using it (config lingers), or have ~/.honcho/ from a different tool. * feat(memory): auto-install pip dependencies during hermes memory setup Reads pip_dependencies from plugin.yaml, checks which are missing, installs them via pip before config walkthrough. Also shows install guidance for external_dependencies (e.g. brv CLI for ByteRover). Updated all 7 plugin.yaml files with pip_dependencies: - honcho: honcho-ai - mem0: mem0ai - openviking: httpx - hindsight: hindsight-client - holographic: (none) - retaindb: requests - byterover: (external_dependencies for brv CLI) * fix: remove remaining Honcho crash risks from cli.py and gateway cli.py: removed Honcho session re-mapping block (would crash importing deleted tools/honcho_tools.py), Honcho flush on compress, Honcho session display on startup, Honcho shutdown on exit, honcho_session_key AIAgent param. gateway/run.py: removed honcho_session_key params from helper methods, sync_honcho param, _honcho.shutdown() block. tests: fixed test_cron_session_with_honcho_key_skipped (was passing removed honcho_key param to _flush_memories_for_session). * fix: include plugins/ in pyproject.toml package list Without this, plugins/memory/ wouldn't be included in non-editable installs. Hermes always runs from the repo checkout so this is belt- and-suspenders, but prevents breakage if the install method changes. * fix(memory): correct pip-to-import name mapping for dep checks The heuristic dep.replace('-', '_') fails for packages where the pip name differs from the import name: honcho-ai→honcho, mem0ai→mem0, hindsight-client→hindsight_client. Added explicit mapping table so hermes memory setup doesn't try to reinstall already-installed packages. * chore: remove dead code from old plugin memory registration path - hermes_cli/plugins.py: removed register_memory_provider(), _memory_providers list, get_plugin_memory_providers() — memory providers now use plugins/memory/ discovery, not the general plugin system - hermes_cli/main.py: stripped 74 lines of dead honcho argparse subparsers (setup, status, sessions, map, peer, mode, tokens, identity, migrate) — kept only the migration redirect - agent/memory_provider.py: updated docstring to reflect new registration path - tests: replaced TestPluginMemoryProviderRegistration with TestPluginMemoryDiscovery that tests the actual plugins/memory/ discovery system. Added 3 new tests (discover, load, nonexistent). * chore: delete dead honcho_integration/cli.py and its tests cli.py (794 lines) was the old 'hermes honcho' command handler — nobody calls it since cmd_honcho was replaced with a migration redirect. Deleted tests that imported from removed code: - tests/honcho_integration/test_cli.py (tested _resolve_api_key) - tests/honcho_integration/test_config_isolation.py (tested CLI config paths) - tests/tools/test_honcho_tools.py (tested the deleted tools/honcho_tools.py) Remaining honcho_integration/ files (actively used by the plugin): - client.py (445 lines) — config loading, SDK client creation - session.py (991 lines) — session management, queries, flush * refactor: move honcho_integration/ into the honcho plugin Moves client.py (445 lines) and session.py (991 lines) from the top-level honcho_integration/ package into plugins/memory/honcho/. No Honcho code remains in the main codebase. - plugins/memory/honcho/client.py — config loading, SDK client creation - plugins/memory/honcho/session.py — session management, queries, flush - Updated all imports: run_agent.py (auto-migration), hermes_cli/doctor.py, plugin __init__.py, session.py cross-import, all tests - Removed honcho_integration/ package and pyproject.toml entry - Renamed tests/honcho_integration/ → tests/honcho_plugin/ * docs: update architecture + gateway-internals for memory provider system - architecture.md: replaced honcho_integration/ with plugins/memory/ - gateway-internals.md: replaced Honcho-specific session routing and flush lifecycle docs with generic memory provider interface docs * fix: update stale mock path for resolve_active_host after honcho plugin migration * fix(memory): address review feedback — P0 lifecycle, ABC contract, honcho CLI restore Review feedback from Honcho devs (erosika): P0 — Provider lifecycle: - Remove on_session_end() + shutdown_all() from run_conversation() tail (was killing providers after every turn in multi-turn sessions) - Add shutdown_memory_provider() method on AIAgent for callers - Wire shutdown into CLI atexit, reset_conversation, gateway stop/expiry Bug fixes: - Remove sync_honcho=False kwarg from /btw callsites (TypeError crash) - Fix doctor.py references to dead 'hermes honcho setup' command - Cache prefetch_all() before tool loop (was re-calling every iteration) ABC contract hardening (all backwards-compatible): - Add session_id kwarg to prefetch/sync_turn/queue_prefetch - Make on_pre_compress() return str (provider insights in compression) - Add **kwargs to on_turn_start() for runtime context - Add on_delegation() hook for parent-side subagent observation - Document agent_context/agent_identity/agent_workspace kwargs on initialize() (prevents cron corruption, enables profile scoping) - Fix docstring: single external provider, not multiple Honcho CLI restoration: - Add plugins/memory/honcho/cli.py (from main's honcho_integration/cli.py with imports adapted to plugin path) - Restore full hermes honcho command with all subcommands (status, peer, mode, tokens, identity, enable/disable, sync, peers, --target-profile) - Restore auto-clone on profile creation + sync on hermes update - hermes honcho setup now redirects to hermes memory setup * fix(memory): wire on_delegation, skip_memory for cron/flush, fix ByteRover return type - Wire on_delegation() in delegate_tool.py — parent's memory provider is notified with task+result after each subagent completes - Add skip_memory=True to cron scheduler (prevents cron system prompts from corrupting user representations — closes #4052) - Add skip_memory=True to gateway flush agent (throwaway agent shouldn't activate memory provider) - Fix ByteRover on_pre_compress() return type: None -> str * fix(honcho): port profile isolation fixes from PR #4632 Ports 5 bug fixes found during profile testing (erosika's PR #4632): 1. 3-tier config resolution — resolve_config_path() now checks $HERMES_HOME/honcho.json → ~/.hermes/honcho.json → ~/.honcho/config.json (non-default profiles couldn't find shared host blocks) 2. Thread host=_host_key() through from_global_config() in cmd_setup, cmd_status, cmd_identity (--target-profile was being ignored) 3. Use bare profile name as aiPeer (not host key with dots) — Honcho's peer ID pattern is ^[a-zA-Z0-9_-]+$, dots are invalid 4. Wrap add_peers() in try/except — was fatal on new AI peers, killed all message uploads for the session 5. Gate Honcho clone behind --clone/--clone-all on profile create (bare create should be blank-slate) Also: sanitize assistant_peer_id via _sanitize_id() * fix(tests): add module cleanup fixture to test_cli_provider_resolution test_cli_provider_resolution._import_cli() wipes tools.*, cli, and run_agent from sys.modules to force fresh imports, but had no cleanup. This poisoned all subsequent tests on the same xdist worker — mocks targeting tools.file_tools, tools.send_message_tool, etc. patched the NEW module object while already-imported functions still referenced the OLD one. Caused ~25 cascade failures: send_message KeyError, process_registry FileNotFoundError, file_read_guards timeouts, read_loop_detection file-not-found, mcp_oauth None port, and provider_parity/codex_execution stale tool lists. Fix: autouse fixture saves all affected modules before each test and restores them after, matching the pattern in test_managed_browserbase_and_modal.py.2 个月前