| docs: simplify the README and repo docs (#481) * docs: simplify the README and repo docs The README opened with vague paragraphs and internal detail, and the FAQ used feature tables with marketing claims. Rewrite them to match the docs site: say what the tool does, show the commands, link out for the rest. - README: computer-use tagline, short install and quick start, the architecture diagram, links to agent-browser.dev/desktop. - FAQ, install, CONTRIBUTING and the FFI, benchmark and test READMEs: cut filler and fix claims the code contradicts (Windows DLL name, ad_init is optional, FFI threading, Homebrew/Scoop status, TMPDIR). - Skills: small wording fixes only; no rule changed. * docs: use --allow-scripts for global installs in install guide npm install-scripts approve only edits a project's package.json and fails for global installs, so it can't fix the global install shown above. Both npm 11 and 12 accept --allow-scripts on npm install -g. Keep the approve command for project installs only. * fix: point blocked-install users at commands that work The launcher notice and the docs site told npm 12 users to run npm install-scripts approve, which fails for global installs, and labeled the --allow-scripts reinstall as npm 11 only. Both npm 11 and 12 accept --allow-scripts on npm install -g, and re-running it over a blocked install links the native binary. In a project, approving alone does nothing until the script runs, and a plain npm install does not re-run it. npm rebuild agent-desktop does, so the project step now says that. Tested on npm 11.20 and 12.2 with throwaway prefixes and projects. * Fix: The macOS installation guide now claims that commands require a private TMPDIR for a background command host, although the merged CLI no longer runs that host, directing users to troubleshoot a nonexistent installation requirement. This commit fixes the issue reported at docs/install.md:56 The new merge base removed `src/command_host`; `src/main.rs` now stops legacy hosts and runs commands directly. `src/tests/command_host_absent.rs` also asserts that commands never connect to or create host socket files. This paragraph reintroduces the removed architecture and its former `INVALID_ARGS` condition into the installation requirements, misleading users with custom temporary directories. The suggested patch removes only the obsolete host/TMPDIR sentences. Verified against the current entry point and absence tests; `git diff --check` passes. Native macOS tests were not run. Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com> Co-authored-by: lahfir <nmhlahfir2@gmail.com> --------- Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com> Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com> | 6 days ago |
| feat!: support Windows 10 1809+ and Windows Server 2019+ on x64 and ARM64 agent-desktop now runs on Windows through UI Automation, with the same commands, JSON envelope, ref system, sessions and tracing as macOS. The Windows build ships as CLI and FFI release assets for x86_64 and aarch64, and through npm. What Windows supports: snapshot, find, get, is and screenshot; list-surfaces with window, focused, sheet and menu surfaces; the semantic ref actions in headless and --headed mode, except right-click, which is a physical click, and type, which synthesizes keyboard input, both under --headed; double-click, triple-click and focus under --headed; press, hover, drag and the mouse commands, with cursor-moving input under --headed; app and window lifecycle; list-displays; text and image clipboard; every wait predicate; open-system-surface for start-menu, taskbar, system-tray, system-tray-overflow and action-center; Action Center notifications; the cursor overlay, including per-agent cursors; status, permissions, version, batch, skills, session and trace. What Windows does not support: Session 0, Server Core, the secure desktop, a locked desktop and another user's session. quick-settings returns PLATFORM_NOT_SUPPORTED. key-down, key-up, mouse-down and mouse-up fail closed on every platform until a daemon owns held-input lifetime. A minimized window is not drivable until it is restored. Input into an elevated window from a non-elevated process is refused (UIPI). Known limits, measured or not measurable on the available rig: - Not verified live: RDP, locked-desktop and Session 0 behaviour (A10-2); per-monitor scale and mixed-DPI bounds, hit-testing and overlay mapping (A10-3, A18-9); second-display input and capture (A10-6, A20-5, A22-8); modern Windows.Graphics.Capture pixel success, where the legacy capture fallback ships (A10-5); WinUI3/MSIX tree shape and menus (A10-7, A23-4); Chromium/Electron menu detection (A23-3); a settled-Chromium ElementFromPoint read (A18-3); the live ERROR_ELEVATION_REQUIRED launch effect (A21-2). - The live suite legs headed-double-click and interaction-scroll-to-visibility fail on an attended desktop, the same way at the merge base, because the operator's console window occludes the target. - A menu item's advertised affordances can differ under concurrent UIA load, so a ref can appear or vanish between snapshots of an unchanged menu item. - The multi-agent listing race budget stays at five attempts, and a window whose owning process cannot be read at Medium integrity stays out of the signal inventory (A24-14, A24-15). - The cursor overlay's label card can cover a menu that just opened, sits against the screen edge at large sizes, and separates from a white window only by a hairline border, on both platforms. On macOS, list-apps can answer TIMEOUT after 5 s when it starts while another app is quitting; this predates the release (#272). Linux desktop automation is not supported in this release; commands that need a desktop answer PLATFORM_NOT_SUPPORTED there, while platform-independent commands such as version and skills still run. macOS behaviour changes that ship with this release: - snapshot --app for an app that is not running answers APP_NOT_FOUND instead of WINDOW_NOT_FOUND. - With several candidate windows, the window picker prefers a visible window, then the single focused one, before an accessible one, so a hidden background window no longer wins over the visible focused window. - A ref resolution attempt may take up to 4 s of the command deadline instead of 750 ms, so a slow app gets fewer, longer attempts. - Surface names in status.supported_surfaces and in unsupported-surface error details are kebab-case (notification-center, menu-bar-extras). Input still accepts both spellings. - wait --event app-terminated, window-closed or surface-dismissed for an app that is not running answers found: true with target_unresolved: true instead of APP_NOT_FOUND. - launch --cdp with a path whose app is already running fails early with cdp_requires_fresh_launch. - get --property bounds adds a live field. list-apps may report complete: false with the skipped records. snapshot (including --root), find, list-apps and list-windows take --timeout-ms; snapshot's default stays 3000 ms. snapshot also takes --force-electron-a11y, which only the Windows adapter acts on. - A permanent actionability refusal now outranks a transient gap when the report picks its error code. A failed scroll-into-view recovery keeps the original actionability error and records the attempt in details.scroll_into_view_attempted. - window-opened and surface-appeared waits can start before the named app exists. Disappearance waits also track entities first observed after the wait began. - trace export --out refuses a path containing .. or a user-owned symlink directory component, and creates missing parent directories as 0700. - macOS observes a sheet, popover or alert whose owning window has no AX element by matching its window number, and resolves refs of an app whose AXTitle is blank by its CoreGraphics owner names, where both used to fail. - cursor-overlay enable takes PNG arrow and pointer images with hotspots and travel, bow, overshoot, tremor, dwell and motion-seed controls; a session-level enable response carries rendered. - App names match with or without a .exe suffix on every platform, so on macOS a request for Foo can also match an app named Foo.exe. - The exact-window FFI calls refuse an empty window id with AD_RESULT_ERR_INVALID_ARGS before dispatch. - The macOS reference moved out of the core skill into the agent-desktop-macos skill; skills get platform serves the one for the running OS. - Traces redact the subrole, dom, classes, body and actions keys. - A blocked key combo now matches any superset of its modifiers, so cmd+shift+alt+q is refused like cmd+q; --force still overrides. BREAKING CHANGE: the release carries these contract changes. - The C ABI major version is 5. The legacy entrypoints that each have an *_exact replacement are removed: ad_list_surfaces, ad_surface_list_count, ad_surface_list_get, ad_surface_list_free, ad_list_windows, ad_window_list_count, ad_window_list_get, ad_window_list_free, ad_launch_app, ad_release_window_fields, ad_get_tree, ad_find, ad_is, ad_focus_window, ad_window_op, ad_resolve_element and ad_execute_ref_action_with_policy, with the AdSurfaceList and AdWindowList handles. AdScreenshotKind loses WINDOW (FULL_SCREEN keeps 2 and 1 is rejected); AdScreenshotTarget loses pid; AdExactSurfaceInfo is version 2, 48 bytes, with a trailing unclassified field; the snapshot_id argument of ad_execute_by_ref and ad_execute_by_ref_timeout and the AdWaitPredicate.snapshot_id field are removed, so AdWaitPredicate is 40 bytes and AdWaitArgs 104. Consumers must rebuild against the regenerated header. - Bare @eN ref input is removed from every ref-taking CLI command, wait --element, snapshot --root, find --root, drag --from/--to, batch entries and the FFI. Only the snapshot-qualified @<snapshot_id>:eN is accepted, and a bare ref fails with INVALID_ARGS. The --snapshot flag that only qualified a bare ref is removed from every command, and batch entries carrying a snapshot or snapshot_id field are rejected as unknown fields. - get --property text no longer returns the same value as --property value. It prefers the value for textfield, combobox, listbox, datefield and timefield and falls back to the accessible name; every other role prefers the accessible name and falls back to the value. Blank strings count as absent. Ask for --property value explicitly to keep the previous answer. - An io error whose kind is PermissionDenied answers PERM_DENIED instead of INTERNAL, in the CLI envelope and in the FFI envelope, return code and last error alike. - A ref action whose element is gone no longer ends in TIMEOUT. A non-retryable resolution failure answers at once; a retryable one keeps polling, and if no later resolution succeeds the action answers the recorded STALE_REF or AMBIGUOUS_TARGET at the deadline. | 23 days ago |