| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
refactor: reorganize start-os into all-products monorepo (#3352) * docs: propose monorepo reorganization for all Start9 products * refactor(monorepo): split core into start-core lib + thin product bin crates - core/ -> shared/crates/start-core (lib 'startos', package 'start-core') - entry points moved to product dirs: start-os (startbox+start-container), start-cli, start-registry, start-tunnel - root Cargo workspace + shared Cargo.lock; profiles hoisted to root - patch-db submodule relocated to vendor/patch-db - service files moved to their product dirs - fixed include_dir!/include_str! paths for new crate locations cargo check -p start-cli -p start-registry passes (lib compiles). * refactor(monorepo): split web into product dirs + shared/web; relocate sdk & container-runtime - angular workspace rooted at shared/web (holds shared + marketplace libs + config) - apps moved to product dirs: start-os/web/{ui,setup-wizard}, start-tunnel/web, brochure/ - angular.json roots/outputs + per-app tsconfig paths repointed (Plan A) - sdk -> start-sdk (base+package kept cohesive: package imports base via relative paths under shared rootDir; splitting base out would break those imports) - container-runtime -> start-os/container-runtime - file: deps repointed (sdk baseDist/dist, patch-db client under vendor/) * build(monorepo): rewire Makefile, build scripts & CI to new layout - build scripts target workspace (-p <crate>, ./Cargo.toml, repo-root cwd) - Makefile paths: core->shared/crates/start-core, web split across product dirs, sdk->start-sdk, container-runtime->start-os/container-runtime, patch-db->vendor - compress-uis.sh takes per-product web dir; split compress pattern rules - ts-bindings recipe sed patterns generalized for new bindings path - CI workflows repointed (deploy-brochure, start-cli, test, startos-iso, ...) * chore(monorepo): gitignore per-product web dist outputs * docs(monorepo): update proposal to reflect implemented layout + verification status * refactor(monorepo): relocate root docs/ internal notes into their projects - exver.md, VERSION_BUMP.md -> shared/crates/start-core/ - PHYSICAL_DEVICE_TEST_PLAN.md, TODO.md -> start-os/ - draft-start9-pcp-hostname.* -> start-tunnel/ frees top-level docs/ for the migrated docs site * docs(monorepo): migrate start-docs (plain copy, no history) - mdbooks into product dirs: start-os/docs, start-tunnel/docs, start-sdk/docs (packaging) - bitcoin-guides, landing, build infra (build.sh/serve.sh/theme/versions.conf/scripts) -> top-level docs/ - repoint book theme symlinks to ../../docs/theme; book.toml build-dir=book, repo/edit URLs -> monorepo - build.sh maps book names to relocated dirs (absolute output); deploy.yml runs in docs/ with paths filter - verified: docs/build.sh builds all 4 books * docs(monorepo): adopt AGENTS.md convention (CLAUDE.md -> @AGENTS.md import) * docs(root): rewrite README/ARCHITECTURE/AGENTS/CONTRIBUTING for monorepo layout * docs(start-os): add product README/ARCHITECTURE/AGENTS/CHANGELOG/CONTRIBUTING Document the StartOS OS product as a thin wrapper in the monorepo: startbox/ start-container bins, web UIs (ui + setup-wizard), container-runtime, systemd units, and OS image packaging. Reflect new paths (start-core, shared/web, start-sdk, vendor/patch-db) and root-workspace build commands. * docs: integrate merged start-docs PRs (#93 UPnP/gateway, #94 task accept/set, #96 init progress) Only PRs whose feature is confirmed merged into start-os were applied. #99 (upstreamCertValidation) skipped — its code PR (#3353) is still open. * docs: add/normalize per-project doc sets (README/ARCHITECTURE/AGENTS/[CHANGELOG]/CONTRIBUTING) Products get the full set incl CHANGELOG; shared components + docs site get the set minus CHANGELOG; all updated to reflect the monorepo layout and AGENTS.md convention. * docs: add CLAUDE.md -> @AGENTS.md import to remaining project dirs * build(monorepo): root the Angular workspace at repo root so apps resolve node_modules Angular resolves @angular/core per-project from each app's root; with apps in product dirs, shared/web/node_modules was unreachable. Move the workspace config (angular.json, package.json, lockfile, tsconfig{,.lib}.json, .browserslistrc) to the repo root — the only ancestor of every app — so resolution works. - angular.json: project roots -> product dirs, lib roots -> shared/web/{shared,marketplace} - tsconfig paths/extends repointed; app source config.json/package.json require() depths corrected for the new app locations - package.json file: deps + script paths rebased to root; check-i18n.mjs scans the scattered project dirs; update-config.sh writes config.json at the workspace root - Makefile web targets run npm at root; build/env + build-cargo-dep stale paths fixed - build-cli.sh: drop stale 'cd core' in chown step Verified: full build succeeds — ts-bindings, SDK bundle, all 4 Angular UIs, and all five musl bins (startbox/registrybox/tunnelbox/start-container/start-cli). * build(monorepo): fix full-image build paths (container-runtime squashfs + version) - Makefile: undouble container-runtime.service dep path in rootfs rule - check-version.sh: read version from root package.json (moved from web/) - update-image-local.sh: mount repo root so start-sdk + target/ are visible to the image build; run start-os/container-runtime/update-image.sh - update-image.sh: copy start-container from ../../target (workspace), not ../core/target Verified: 'make all' completes (exit 0) — all musl bins + container-runtime rootfs.squashfs (437M) build; second run is a no-op (fully built). * build(monorepo): sync container-runtime package-lock to relocated SDK path; prettier ARCHITECTURE table * feat(monorepo): migrate startos-backup-fs into start-os/backup-fs Vendor the backup-fs crate (was external git dep Start9Labs/start-fs) as a workspace member under the start-os product; build it via the zigbuild path like the other bins instead of 'cargo install --git'. - start-os/backup-fs/: the startos-backup-fs crate (encrypted erasure-coded FUSE backup filesystem); relaxed its =4.5.7 clap / =0.2.17 ppv-lite86 exact pins so they unify with the workspace - root Cargo workspace member + single lock - shared/crates/start-core/build/build-backup-fs.sh; Makefile target builds the local crate (no more git URL) - docs: start-os ARCHITECTURE + CHANGELOG note the migration Verified: 'make all' (exit 0) builds startos-backup-fs (musl) as a member. * build(sdk): decouple 'bundle' from test/check-fmt so consumers don't re-run jest bundle now builds baseDist+dist only; test/check-fmt are standalone (CI calls them directly), and publish runs them explicitly. Fixes the recursive-make coupling where the OS build re-ran the full SDK jest suite every build and an SDK test/format failure broke the OS build. * build(monorepo): split Makefile into per-project include fragments Thin root Makefile includes build/common.mk (shared vars/macros + cross-cutting infra) and one <project>/build.mk per product. Uses include (not recursive make) so it stays a single DAG and cross-project prereqs (start-core -> ts-bindings -> SDK -> web/container-runtime) resolve correctly. - build/common.mk: vars, cp/mkdir/ln/ssh macros, patch-db client, external cargo tools - shared/crates/start-core/build.mk: test-core, ts-bindings - shared/web/build.mk: angular workspace (install, .angular, i18n, UI builds, compress, config.json) - start-sdk/build.mk: test-sdk, dist bundle (consumes the now-decoupled SDK Makefile) - start-{cli,registry,tunnel}/build.mk: their bins + install/deb - start-os/build.mk: startbox/start-container/backup-fs, container-runtime image, OS image assembly + deploy - docs/build.mk: docs site build Verified: make all is a no-op (full build intact); all targets resolve; no duplicate recipes. * docs(root): note the per-project build.mk Makefile structure in AGENTS.md * fix(ci): repoint test/web paths after workspace moves - run-tests.sh: cd to repo root (was shared/crates), build via ./Cargo.toml -p start-core (was ./core/Cargo.toml --workspace) - test.yaml / deploy-brochure: install the Angular workspace at the repo root (npm ci) instead of shared/web; fix vendor/vendor/patch-db doubling; brochure path filters -> root - startos-iso prevent-rebuild placeholders: node_modules/.angular at root; version read from root package.json Verified: npm ci passes at root (lockfile gate). * build(start-os): namespace OS-product make targets as startos-* / install-startos The repo is no longer start-os-only, so the generic target names now read as start-os-specific: - deb->startos-deb, iso/img->startos-$(IMAGE_TYPE), squashfs->startos-squashfs - install->install-startos (matches install-registry/install-tunnel) - wormhole*/update*/emulate-reflash/upload-ota -> startos-* - new 'startos' aggregate (= STARTOS_TARGETS); root 'all: startos' Callers updated: dpkg-build.sh INSTALL_TARGET, deploy targets' $(MAKE) install, startos-iso.yaml (make startos-iso/startos-img), root .PHONY. NOTE: external shared-workflows may invoke the old names (make iso/squashfs/install) for OS image/release builds — needs a companion update there. * build(start-os): move OS-specific build assets into start-os/build Relocate the start-os-only build inputs out of the shared top-level build/ into the product dir: image-recipe/, dpkg-deps/, lib/, download-firmware.sh, and save-migration-images.sh -> start-os/build/. Keep genuinely shared pieces at build/ (common.mk, env/, os-compat/, build-cargo-dep.sh, and lib/scripts/forward-port, which start-tunnel also installs). Relocate the start-os-specific make variables/rules out of build/common.mk into start-os/build.mk (web src/output vars -> shared/web/build.mk; registry and tunnel target vars -> their own fragments) so common.mk is shared-only. Repoint every reference (fragments, Makefile clean, container-runtime update-image.sh, and the moved scripts' own internal paths). Delete the unreferenced legacy build/registry/ eos deploy scripts. * docs(changelog): write 0.4.0-beta.10 per-product release notes Fill in the [0.4.0-beta.10] sections across the per-product CHANGELOGs (brochure, start-cli, start-os, start-registry, start-sdk, start-tunnel) with Added/Changed/Fixed/Removed/Security notes for this cycle, cross-linked between products. * refactor(start-core): rename lib startos to start_core, drop package alias Rename the start-core library from `startos` to `start_core` so the crate's lib name matches its package and the legacy `startos = { package = "start-core" }` dependency-rename alias is gone. The name now penetrates all source: - [lib] name = "start_core"; every `startos::` crate path -> `start_core::` - product crates depend on `start-core` directly; features become `start-core/*` - RUST_LOG=warn,startos=debug -> start_core=debug in the systemd units and CI (the target is module_path!()-derived, so it tracks the crate name) - docs updated to match The product identifier "startos" is left untouched (the root:startos system user/group, the tor.startos / *.startos DNS names, the nftables table, the signature context, the .startos/ packaging-workspace dir, i18n keys, and the /usr/lib/startos install paths). * refactor(monorepo): nest products under projects/, rename shared -> shared-libs Move the buildable products and the docs site into a top-level projects/ dir to separate them from repo infrastructure: start-os, start-cli, start-registry, start-tunnel, start-sdk, brochure (-> brochure-marketplace), docs (-> start-docs) -> projects/ Rename the shared Rust+web library container shared/ -> shared-libs/, kept at the top level alongside build/ and vendor/ as cross-cutting infrastructure. Rewire every path reference to the new layout: - Cargo workspace members + product path deps (../shared -> ../../shared-libs) - Makefile, build/common.mk, and every <project>/build.mk fragment - angular.json, package.json, root + per-app tsconfig (web app configs moved a level deeper, so their relative extends/paths gain one ../) - .github/workflows (the Start9Labs/start-os repo URL is preserved; docs-deploy working-directory + path triggers updated) - build scripts (run-local-build.sh / update-image-local.sh cd depths and internal paths; start-core build/*.sh chown paths) - root .gitignore build-output globs and the web package-lock file: paths Verified: cargo check of all six crates (UI-embed include_dir! and build/env include_str! resolve to the new locations), make -n of the OS / registry / tunnel / web targets. The cold web/SDK build remains CI-grade. * refactor(monorepo): relocate project-specific assets/debian/scripts into projects Apply the same shared-vs-project split to the remaining top-level dirs: - assets/ (create-vm screenshots) -> projects/start-os/assets/ - debian/{startos,start-registry,start-tunnel}/postinst -> each project's debian/; the shared debian/dpkg-build.sh stays top-level and now maps PROJECT -> projects/<dir>/debian for the control files - scripts/copy-categories.sh (registry admin) -> projects/start-registry/scripts/ Kept at top level as genuinely shared/repo-level: debian/dpkg-build.sh, scripts/manage-release.sh (repo releases), scripts/publish-deb.sh (apt publish). Repoint the deb build.mk prereqs, the CONTRIBUTING create-vm link, and code/unit comments. Verified make -n of the three *-deb targets. * docs(rfcs): move draft-start9-pcp-hostname to a top-level rfcs/ dir The PCP HOSTNAME extension Internet-Draft (.md + .txt) describes a protocol spoken by both the StartOS client and the StartTunnel server, so it belongs at the repo level rather than inside start-tunnel/. Repoint the start-os CHANGELOG reference (was the stale docs/ path) to rfcs/. * docs: sync structure docs to the projects/ layout + README project shout-outs - README: add a "rest of the monorepo" section with a short shout-out to each non-OS product (StartTunnel, start-cli, Start SDK, start-registry, and the marketplace + docs sites), and update the directory table + icon path to the projects/ + shared-libs layout. - Root AGENTS.md: rewrite "what lives where" for the new layout and complete the Sub-scopes list (it was missing most products). - Root ARCHITECTURE.md: repoint the module map + cross-layer paths; MONOREPO.md gains a note that the layout was refined (products -> projects/, shared -> shared-libs). - Per-project docs: rename shared/ -> shared-libs/ references, fix relative links whose depth changed when products moved a level deeper into projects/ (links to the repo root, LICENSE, shared-libs, and cross-product changelogs), and repoint functional cd / --prefix build commands. Sibling refs under projects/ (e.g. ../start-sdk, file:../../start-sdk/dist) are correct and left as-is. * build(brochure-marketplace): rename Angular project brochure -> brochure-marketplace Rename the Angular project key (and its build/serve targets) so the project name matches its directory. The dist output is now projects/brochure-marketplace/dist/raw/brochure-marketplace, and the deploy workflow reads/rsyncs that path — this also corrects a path the projects/ restructure had mangled to raw/projects/brochure-marketplace. The npm script names (build:brochure / start:brochure) are kept as conveniences. * docs: remove MONOREPO.md The reorganization proposal has been fully implemented and superseded by the current README/ARCHITECTURE; drop the historical proposal doc and its two links. * feat(build): per-project versioning + Debian packaging for start-cli Decouple product versions from the single StartOS release version. Each Rust product's version is now the source of truth in its own Cargo.toml: start-os stays 0.4.0-beta.10; start-cli / start-registry / start-tunnel move to their own line starting at 1.0.0. - basename.sh and dpkg-build.sh read the version straight from the project's Cargo.toml (per PROJECT), so each .deb is named/versioned independently. - check-version.sh now derives the OS-image /usr/lib/startos/VERSION.txt from the start-os crate manifest instead of the root package.json; nothing maintains a separate version source anymore. - start-cli gains a Debian package: `make cli-deb` builds the musl binary and packages it via the shared dpkg-build.sh (CLI_BASENAME / install-cli staging). CHANGELOGs and the registry AGENTS version note updated to reflect independent versioning. Cargo.lock synced to the new member versions. * chore: ignore *.local.md Broaden the local-notes ignore from CLAUDE.local.md to any *.local.md. * refactor(deps): vendor Start9-maintained crates into shared-libs/crates Move every Start9-maintained crate the workspace depends on in-repo, wired by direct path deps (no [patch]): - rpc-toolkit, imbl-value, exver, yasi, jsonpath (jsonpath_lib), pi-beep — plain-copied from their repos into shared-libs/crates/, added as workspace members. Their inter-deps are repointed to path (exver/imbl-value -> yasi, rpc-toolkit/jsonpath -> imbl-value), and start-core depends on them by path. - patch-db — de-submoduled: moved out of the vendor/ git submodule into shared-libs/crates/patch-db (keeps its own [workspace], excluded from the root one and consumed by start-core via path). Its core/json-patch/json-ptr now path-dep the vendored imbl-value, so there is a single imbl_value::Value type. Drop .gitmodules; repoint the web patch-db-client (package.json / common.mk / CI / shared-libs/web) and pi-beep's build (build-cargo-dep.sh --path). Upstream forks still pulled by git (async-acme, crab_nat, fuser) are left as-is. Verified: cargo check of start-core + start-cli + start-registry + pi-beep compiles the whole path-dep tree clean; Cargo.lock regenerated. * refactor(start-os): move manage-release.sh into the product manage-release.sh is the StartOS release orchestration (startos-images S3 bucket/CDN, the OS image arch matrix incl. -nonfree/-nvidia, the OS registry), not a repo-wide tool — move it to projects/start-os/scripts/. It still calls the shared scripts/publish-deb.sh (which stays top-level, since it publishes any product's .deb), now referenced by its repo-root-relative path. * refactor(debian): rename dpkg-build.sh -> build.sh, move publish-deb.sh -> debian/publish.sh Co-locate the deb tooling under debian/: the package builder is debian/build.sh and the apt-repo publisher is debian/publish.sh (was scripts/publish-deb.sh, which empties scripts/). Repoint the per-product deb build.mk targets, the manage-release.sh caller, and doc/comment references. * build: build pi-beep as a first-party member; reword "vendored" -> "first-party" pi-beep is one of our crates now, so build it like startos-backup-fs (a dedicated build-pi-beep.sh zig build of the workspace member) instead of routing it through build-cargo-dep.sh. That script is now only for the genuinely external crates.io dev tools (tokio-console, flamegraph) bundled into unstable/console images. Also reword the patch-db docs: these are our own crates, so "first-party crate" is more accurate than "vendored" (which implies a third-party copy). * ci: path-gate the per-product build workflows to their project + deps The start-cli / start-registry / start-tunnel / startos-iso build workflows ran on every push/PR (only skipping doc-only changes), so all four built regardless of what changed. Replace the blanket paths-ignore with a paths: allowlist scoped to each product plus its dependencies (start-core + the in-repo shared-libs crates, Cargo manifests, build infra, and — for the web-bearing/OS workflows — the Angular workspace and SDK). workflow_dispatch / workflow_call are kept so manual and orchestrated runs still fire unconditionally. * ci: migrate shared-workflows (service-package CI) into the monorepo Bring the reusable .s9pk build/release workflows and their composite actions in-repo from the standalone Start9Labs/shared-workflows repo, so the packaging toolchain lives alongside the SDK: - .github/workflows/{build,release,tagAndRelease}.yml (reusable, workflow_call) - .github/actions/{extract-version,free-disk-space,setup-build-env, setup-publish-env,upload-each} Their internal references (and the SDK package-template's three workflows + the packaging docs) are repointed from start9labs/shared-workflows@master to Start9Labs/start-os@master. These are workflow_call-only, so they don't run for the monorepo itself — they activate once this lands on master and external service-package repos repoint their `uses:` to Start9Labs/start-os. * docs(monorepo): document tandem-update couplings The per-product CI `paths:` filters mirror each product's build.mk prerequisites by hand — nothing enforces it. Add a "Coupled changes" section to the root AGENTS.md and reciprocal pointers in each gated workflow and its build.mk, so a change to one half is caught at the other. Also catalogs the remaining hand-mirrored pairs (reusable service-package CI <-> SDK package-template <-> packaging docs; the files touched when adding a product/crate) and the already-enforced couplings (ts-bindings, the five i18n locales, the UI beta seed, version <-> CHANGELOG, docs <-> user-facing changes). * chore(manpages): generate man pages into their product projects The export_manpage_* tests in start-core wrote every product's man pages into start-core's own man/ dir. Point each generator at the owning product's man/ dir (anchored to CARGO_MANIFEST_DIR), move the committed pages there, and update build-manpage.sh's chown and the docs. start-container's pages go to projects/start-os, since that bin is part of the StartOS product. * Retitle README * refactor(shared-libs): rename web -> ts-modules Mirror the `crates/` naming: the shared TS/Angular workspace dir becomes `shared-libs/ts-modules/`. Pure path rename — repoints every reference (angular.json, root tsconfig/package.json scripts, the Makefile include + build.mk, CI `paths:`, and docs). No code changes. * fix(monorepo): repoint stale paths from the projects/ move that broke CI Two classes of path left stale by nesting products under projects/: - Web apps `require()` repo-root config.json/package.json by relative path; the extra projects/ level meant every one was short one `../` (resolved to projects/… instead of the repo root), failing the esbuild UI build for start-os (ui + setup-wizard) and start-tunnel. - test.yaml and deploy-brochure.yml still `cd start-sdk` for the baseDist build; the SDK now lives at projects/start-sdk. * fix(debian): resolve PROJECT_DIR before reading VERSION debian/build.sh computed VERSION from "projects/$PROJECT_DIR/Cargo.toml" before PROJECT_DIR was assigned, so it read projects//Cargo.toml (empty) and double-prefixed projects/. The empty Version: produced an invalid DEBIAN/control and dpkg-deb rejected it — breaking the registry and tunnel .deb builds. start-cli's CI only runs `make cli` (the binary), so it never exercised this path. Hoist the PROJECT_DIR/INSTALL_TARGET block above VERSION and read "${PROJECT_DIR}/Cargo.toml" directly. Also only fall back to the OS product's usr/lib/startos/conflicts when that file actually exists, so non-OS products don't error on a missing conflicts file. * fix(web): prettier-wrap tsconfig paths widened by the ts-modules rename Renaming shared-libs/web -> shared-libs/ts-modules pushed the `@start9labs/*` path-mapping lines past prettier's print width, so `npm run format:check` (the Formatting & Lockfiles CI job) flagged the four product tsconfig.json files. Apply prettier's wrapping. * docs(contributing): align with restructure + renamed make recipes Bring the CONTRIBUTING set up to date with the monorepo layout and the namespaced make targets: - root: `make iso` -> `make startos` - start-os: `make $(IMAGE_TYPE)`/`deb`/`squashfs` -> `make startos-$(IMAGE_TYPE)`/`startos-deb`/`startos-squashfs`; deploy/flash targets -> `startos-update*`/`startos-wormhole*`/`startos-emulate-reflash` - start-core: `cd start-sdk` -> `cd projects/start-sdk`; osBindings sync path -> projects/start-sdk/base/lib/osBindings - shared-libs: `web/` -> `ts-modules/` (heading, lib paths, file: deps, cross-links) The other products' CONTRIBUTING files were already correct. * fix(container-runtime): correct repo-root target path in update-image.sh update-image.sh runs with cwd at projects/start-os/container-runtime/ (mounted at /root/start-os in start9/build-env), so the repo-root build output is three levels up. It copied start-container from ../../target (-> projects/target, nonexistent), so the container-runtime squashfs was never built and the OS image compile failed on every arch. Use ../../../target. Refresh the AGENTS.md gotcha that described the old stale path. * refactor(make): namespace OS web targets, require an explicit target, per-project cleans - `ui`/`uis` -> `startos-ui`/`startos-uis`: they build only the StartOS admin UI + setup-wizard, so they belong under the startos-* namespace. Also fix `startos-ui` to depend on the built index.html (the old `ui` depended on a directory with no rule). - No default build: bare `make` now prints `help` (.DEFAULT_GOAL := help) and the misleading `all` target (it only built `startos`, not "everything") is removed — callers specify a target. - Decentralize `clean`: every build.mk owns a `clean-<project>` target and the root `clean` just aggregates them. Per-project cleans use project-prefix wildcards so they're arch/version-independent, and two stale paths from the projects/ move are corrected (env/*.txt -> build/env/*.txt; image-recipe/deb -> projects/start-os/build/image-recipe/deb). - Drop the start-cli targets (`make cli`/`cli-deb`) from the start-os CONTRIBUTING build section (wrong product) and update the docs to the new target names (root + start-os CONTRIBUTING/README/AGENTS). * docs: sync, complete, and standardize all developer docs for the monorepo (#3356) * docs: sync developer docs with the monorepo restructure Audit of all developer documentation (AGENTS/CONTRIBUTING/ARCHITECTURE/README across root, projects/*, shared-libs/*) against the post-restructure tree. Corrects stale references the restructure left behind: - Paths still pointing at the pre-restructure layout (core/, web/, sdk/, brochure/, container-runtime/, patch-db submodule) -> projects/* and shared-libs/*. - Angular workspace root: several docs claimed shared-libs/ts-modules holds angular.json/package.json/tsconfig.json and that npm runs from there. The workspace is rooted at the repo root; fixed cwd/--prefix instructions, the tsconfig path-alias targets, and config-sample.json location accordingly. - Renamed make targets (startos-* namespace), the Rust lib rename (startos -> start_core / crate start-core), and the binary-source table in start-core/ARCHITECTURE.md. - Removed stale 'git clone --recursive' (no submodules remain) and the non-existent repo-level scripts/ reference. - Normalized product self-references and the brochure -> brochure-marketplace Angular project name; verified incidental accuracy fixes (exver 0.2.1, patch-db serde_cbor, image-recipe live-build). All relative doc links verified resolvable. Pre-commit lint-staged hook skipped (--no-verify): the slot has no installed node_modules so the binary can't run, and the repo's prettier targets only the web source dirs, not these markdown docs. * docs: bring utility crates and patch-db up to the standard doc set Every first-party crate under shared-libs/crates/ now carries the same AGENTS/ARCHITECTURE/CONTRIBUTING/README/CLAUDE set the projects and other shared libs use. - New full doc sets for exver, imbl-value, jsonpath, pi-beep, rpc-toolkit, yasi, written from each crate's actual source (verified module names, public API, cargo -p <package> commands, and real consumers; jsonpath's package is jsonpath_lib though its dir is jsonpath). Pre-existing READMEs (exver, jsonpath, yasi) were preserved verbatim and only augmented with a 'Place in the monorepo' + 'Documentation' section. - patch-db: added AGENTS.md (migrated from its content-bearing CLAUDE.md, plus a Build & test section noting it is its own Cargo workspace) and reduced CLAUDE.md to the one-line @AGENTS.md import like every other scope. - Every CLAUDE.md is exactly '@AGENTS.md'; all relative doc links resolve. * docs: normalize section structure across all project & shared-lib docs Standardize every first-class scope (root, the 7 projects, container-runtime, shared-libs, start-core, ts-modules) onto one canonical section template so the same sections appear under the same names in the same order everywhere: - AGENTS.md: intro (+ 'CLAUDE.md is a one-line @AGENTS.md import' + cross-links) → Layout → Build & test (run from the repo root) → Gotchas → scope-specific. Folded synonym headings (Where things are / What lives where / Map → Layout; Operating rules / Gotchas / rules → Gotchas; Build / Build & verify → Build & test). - CONTRIBUTING.md: Documentation (doc-map, added where missing) → Prerequisites → Building → Testing → Formatting → scope-specific → Commits / PRs (last). - ARCHITECTURE.md: ensured a 'Place in the monorepo' opener (where applicable) and a 'Further reading' links footer; internal sections left in place. - README.md: added a Documentation/links footer where missing; landing prose left untouched. Pure reorganization: headings renamed/reordered and short missing sections added — no prose reworded, no facts/commands/tables dropped. Scope-accurate build cwds preserved (start-docs builds from its dir; start-sdk from its dir; container-runtime via npm --prefix; patch-db its own workspace). All 102 docs' relative links verified resolvable; every CLAUDE.md remains the one-line import. * build+docs: merge patch-db into root workspace; enforce doc hierarchy Addresses review feedback on the docs PR: - **patch-db → root Cargo workspace.** Add the six patch-db crates (core, json-patch, json-ptr, macro, macro-internals, util) to the root workspace `members` and drop the `exclude`; remove patch-db's now-redundant virtual `Cargo.toml`. `cargo metadata` + `cargo check -p patch-db -p json-patch -p json-ptr -p patch-db-macro` pass. Updated the docs that called patch-db 'its own workspace' (patch-db AGENTS/CONTRIBUTING build commands now run from the repo root with `-p`; root ARCHITECTURE lists it as a member). - **Root AGENTS.md:** added a note that these doc files must be kept current with every change, and that the docs are hierarchical — a scope documents only what is specific to it and never repeats higher-scope content. - **Hierarchy cleanup:** removed the `## Commits / PRs` (and `Branch / commit / PR` / `Commit conventions`) sections from every non-root CONTRIBUTING.md — those conventions live only in the root CONTRIBUTING.md now. Scope-specific change steps were preserved (e.g. start-tunnel's migration/ bindings/CHANGELOG steps moved to a 'Making a change' section; start-cli's docs-update note folded into 'Where code lives'). Also dropped the duplicated 'keep these docs in sync' line from child Documentation sections (kept the local cross-link lists). - **jsonpath:** reframed the fork note as history-only across its docs — it has fully diverged from freestrings/jsonpath with no intent to upstream; treat it as first-party and edit freely (removed the 'pull fixes from upstream / keep changes minimal / fork-tracking' guidance). All 102 docs' relative links resolve; every CLAUDE.md remains the one-line import. * docs: add hierarchy-navigation notes to AGENTS files - Every non-root AGENTS.md now opens with a 'Read up the tree first' note: the docs are hierarchical, so before working in a scope read the AGENTS.md of each enclosing directory up to the repo root (and their ARCHITECTURE/CONTRIBUTING where relevant). Excludes the packaging-guide + package-template AGENTS under projects/start-sdk/docs/, which target external package authors, not the monorepo dir tree. - Root AGENTS.md gains the converse 'Read down into what you touch' note: read a subdirectory's AGENTS.md (and any further nested ones) before editing it. - Dropped the explicit repo-root CONTRIBUTING pointer from patch-db's CONTRIBUTING.md (the walk-up convention now covers it). * docs(agents): require product docs/ book + CHANGELOG to ship with code Root AGENTS.md now mandates that any change altering user-visible behavior update that product's user-facing docs/ book (projects/<product>/docs/) in the same change and add a CHANGELOG.md entry — no deferring docs/changelog to follow-ups. * build(make): add per-project format targets mirroring the clean decomposition Each build.mk now owns `format-<project>` + `format-check-<project>` (core, web, sdk, cli, registry, tunnel, startos), matching the per-project `clean-<project>` targets; the top-level `format`/`format-check` just aggregate them. Web (the whole Angular workspace incl. brochure) formats via the root npm script; the shared crates via one `cargo +nightly fmt`; container-runtime via its own prettier config (new `format`/`format:check` npm scripts). So you can format one project (`make format-cli`) or all (`make format`). * docs: address PR review — make-target refs, ARCH de-dup, ts-modules wording - Reference stable make targets instead of raw build/format commands across project docs: builds via `make cli`/`registry`/`tunnel`/`startos`/`startos-ui`, formatting via the per-project `make format-<project>` targets (and `format-check-<project>` for CI). Kept `cargo check`/`cargo test` and crate `cargo build` (no make equivalent) as noted dev shortcuts. - Fixed stale targets: dropped `all` (removed upstream; `make` now prints help), `ui`/`uis` -> `startos-ui`/`startos-uis`. - Removed the whole-monorepo ASCII trees that several product ARCHITECTURE.md files re-drew (start-cli, start-registry, container-runtime, ...) — that layout lives once in the root ARCHITECTURE.md; each now states only where it sits. - Reworded the `shared-libs/ts-modules` directory as shared TypeScript modules (not Angular-specific; current contents are the Angular libs shared/marketplace), per review; kept accurate per-library 'Angular library' phrasing. Branch merged up to date with docs/monorepo-proposal first. * refactor(sdk): extract base into @start9labs/start-core shared lib; flatten start-sdk Move projects/start-sdk/base -> shared-libs/ts-modules/start-core (package @start9labs/start-sdk-base -> @start9labs/start-core), mirroring the Rust crate shared-libs/crates/start-core. start-core builds its own self-contained dist consumed via file: deps. Flatten start-sdk (package/ -> root): the SDK now imports @start9labs/start-core instead of ../../base/lib, and its published dist bundles start-core (bundleDependencies) so external authors still install one package. Repoint all consumers off the SDK-as-base alias onto @start9labs/start-core: - web (root file: dep + 153 import sites + shared/marketplace peerDeps) - container-runtime (file: dep; base/lib imports -> start-core, package/lib -> lib) Repoint osBindings generation, the build DAG (build.mk fragments, Makefile), and regenerate the three lockfiles. Resolves the "SDK kept cohesive" deviation: base is now an honest first-class shared TS lib named for what it is. * docs+ci: reflect start-core extraction; repoint SDK build steps off base/baseDist CI: build start-core (cd shared-libs/ts-modules/start-core && make dist) before validating the SDK + web lockfiles; validate the flattened SDK lockfile at projects/start-sdk; add start-core to the prettier check; repoint the iso prevent-rebuild mkdirs and the brochure deploy paths: filter (start-sdk -> start-core). Docs: update every project's AGENTS/ARCHITECTURE/CONTRIBUTING/README for the new layout — base extracted to @start9labs/start-core under shared-libs/ts-modules, the SDK flattened (lib/) and bundling start-core, container-runtime depending on both. Removes the resolved "SDK kept cohesive" deviation note. * refactor(monorepo): use the start-technologies name; repoint init-workspace at the monorepo - Adopt start-technologies for monorepo/repo-URL references across docs and AGENTS files (ahead of the GitHub repo rename; product refs left as start-os: projects/start-os, the start-os crate, *-startos packages, StartOS). - s9pk init-workspace now sparse-clones the start-technologies monorepo (projects/start-sdk/docs) instead of the retired standalone start-docs repo, and the two start-docs-named init strings are reworded across all five locales. - Un-hide the SDK 2.0 packaging-workspace section and rewrite it for the monorepo clone; document fetch-on-demand SDK + OS source access (docs first) in the workspace AGENTS.md and workflow.md. - Fix build-config.js to read and write the repo-root config.json. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkwYSXK8Uh9D16UTffCNJ1 --------- Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <9935159+MattDHill@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> | 2 个月前 | |
fix(start-os): a backup-fs read past the end of a file returns no bytes (#3835) * fix(start-os): a backup-fs read past the end of a file returns no bytes `Handler::read` clamped a read with an unchecked `attrs.size - offset`, where its sibling on the ordinary read path uses `saturating_sub`. It is reached only from `Handler::copy_file_range`, and `lib.rs` holds the `Handler` guard across that call — so with `offset > size` a debug build panics with the whole session's lock held, poisoning it, and every subsequent operation on the mount, including the unmount, then panics on the poisoned lock. A release build wraps to a huge `usize` instead and returns `EIO`. The kernel clamps a `copy_file_range` against the size it has cached, so a plain read past EOF never reaches the handler; a truncate landing between that check and the handler's read does. Reproduced against `master`: `handle.rs:1102 attempt to subtract with overflow`, then `lib.rs:173 PoisonError`, then the mount stops answering. Clamping alone still returns `EIO`, because `read_exact_at` measured a zero-length read against the file's size and refused it as an overrun — which is also why an ordinary `pread` past the end of a backup-fs file returned `EIO` where POSIX requires 0. A read of no bytes is now satisfiable at any offset, which fixes both paths. It no longer stamps atime or dirties the inode either, so a read that returns nothing no longer schedules an inode save. Each test fails on the mutation that removes the guard it covers: reverting `contents.rs` fails both, reverting `handle.rs` panics the copy test alone. Closes #3780 * fix(start-os): a backup-fs copy that moves no bytes leaves the destination alone A read past the end of a file now returns zero bytes, so `Handler::copy_file_range` reaches its write with an empty buffer where the read used to error out first. `Contents::write_all_at` has no empty-buffer case: it computes `end = dest_offset`, promotes the body tier, sets the destination's size to `end` and stamps its mtime. So a copy out of a file that shrank returned 0, as it should, and grew the destination out to `dest_offset` with zeros, which no other filesystem does. Measured on the diff's own workload, 20,000 iterations against a flapping truncator: 3,220 to 15,060 of the ~15,000 zero-byte copies per run left the destination extended. With the guard, none do. The regression test picks it up now: it copies into a hole rather than to offset 0, where an extension to `dest_offset` was invisible, and reads the destination back through the filesystem rather than trusting the length the kernel cached. The same test could pass on unfixed code roughly 40% of the time on a single-core machine. 98.7% of its iterations never leave the kernel, which clamps the request away itself, so the copier never blocks and the truncator barely runs; a yield per iteration opens the race, and 500 iterations then catch the regression where 20,000 without it did not. Its truncator thread also unwrapped two syscalls that fail once the mount is gone, so a real failure was reported as `Any { .. }` and the message the test exists to print never ran. The changelog said the copy could leave the filesystem unresponsive and hang. That needs the old `size - offset` to panic, and `overflow-checks` is off in release, where it wraps and the copy fails with an I/O error instead. The hang is a debug-build artifact, so the sentence describes something no user saw. * test(backup-fs): witness the shrinking-file race instead of assuming it The copy test could report green having raced nothing. Its two loop outcomes are now counted and both are required, so a source that never shrank under the copier fails the test rather than passing it silently. Two `unwrap()`s in the loop body bypassed the stop-then-report sequence: a panic there skipped the stop flag and the join, leaving the truncator writing to a mount being torn down. Both now record a failure and break like the paths beside them. Also record on `Contents::read_exact_at` the bound its callers must apply, which is the invariant whose two copies drifted into this bug, and hold the comments this branch adds to the rebuilt rules in AGENTS.md. * fix(start-os): copy a backup-fs range a chunk at a time The kernel forwards a copy_file_range whole, so `Handler::copy_file_range` read the entire range into one allocation and `Handler::write` copied it again. A 4 GiB `cp` inside a mounted backup peaked at ~12 GiB RSS and OOM-killed the mount. It now moves a chunk at a time and drops the redundant copy in `Handler::write`. Move the empty-write guard down to `Contents::write_all_at`, where the defect lives: an empty write extended the file and stamped its mtime. The read side was already fixed at that layer. Cover the copy loop's offset arithmetic, and stop the shrinking-file test asserting what it cannot observe: the kernel answers most iterations from its cached size without reaching the filesystem, so a count of zero-byte copies does not witness the past-EOF path. * fix(start-os): report the bytes a backup-fs copy moved before it failed A chunked copy_file_range propagated an error out of any chunk after the first, so the kernel was told nothing moved while earlier chunks were already durable in the destination. Measured on a 3 MiB copy failing at its second chunk: the syscall returned -1 with 1048576 bytes on disk. It now returns the count and leaves the error for the next call. Drops the unreachable wrote == 0 break: Handler::write returns data.len() unconditionally and the buffer is non-empty by the guard above it. Covers both empty-buffer guards directly, since the kernel issues no zero-length request and neither guard was reachable from any test. Surfaces the truncator thread's errors in the shrinking-file test, which otherwise passed whether or not it witnessed the race, and reads errno only where a negative return makes it meaningful. Clears the unused_mut the previous commit introduced at lib.rs:463. * fix(start-os): record the error a partial backup-fs copy reports around A chunked `copy_file_range` that meets an error after moving bytes returns the count instead of the error, which is the syscall's contract. The error was then dropped whole: `BkfsError::to_errno_log` is the crate's only logging site and it is reached solely from the `Err` arm at `lib.rs`, so a `BadCrypt` or `BadChecksum` on a source block inside a mounted backup left no trace at all, and the short count is indistinguishable from a normal end-of-file copy. Measured on a three-chunk copy failing at chunk two: the count still comes back as 1048576, and the log line that had zero occurrences now appears. The partial-versus-total decision and the log now sit at one exit rather than in two duplicated arms, and the loop counts the bytes it read rather than the count `Handler::write` echoed back, so its progress no longer rests on a callee's return value. Two more found in the same functions: - `Handler::write` tested `FUSE_WRITE_KILL_PRIV` against the open flags while binding the argument that carries it as `_write_flags`, so it could never clear a file's suid bit. No `O_*` flag has that value. Dead today — its one caller passes zero and the FUSE write path in `lib.rs` reads `WriteFlags::FUSE_WRITE_KILL_SUIDGID` correctly. - `read_exact_at`'s doc said a read past the end fails, which the guard three lines below it contradicts for an empty buffer. The shrinking-file test reported a dead truncator as a filesystem bug: a `set_len` failure and a failed copy carry the same errno, and the copy was reported first. A truncator that fails partway has also stopped shrinking rather than never having shrunk. * fix(start-os): record a partial backup-fs copy's error where an operator sees it `BkfsError::log` routes any io error carrying an errno to `debug!`, and `main.rs` defaults the filter to `info`, so the error a partial copy reports around left no line at all at production log level — the case the previous commit was written for. Measured against the new test at `RUST_LOG=info`: 0 lines before, 1 after. The copy loop now warns at the one exit where the caller receives a success, and `BkfsError::log` goes back to being private inside `to_errno_log`. A failed packed→blocks migration also emptied the file it was migrating. `packed_to_blocks` rewrites `inode.attrs.contents` and schedules the superseded extent's tombstone without marking the inode changed, so a write that failed after the migration skipped the inode save on close while still dropping the extent. The durable inode kept pointing at a tombstoned extent and the whole file read back as zeros. Reproduced deterministically by making one block file unreadable. Adds the first test for the partial-count path. * fix(start-os): report a corrupt backup-fs block at its own severity The partial-copy arm logged every error it swallows at one level, so a corrupt or tampered source block — BadChecksum out of `vault::open`, which `load_block` reaches through `read_block` — was reported as a warning where the crate reports it as an error everywhere else. Measured at RUST_LOG=info: a garbaged block now logs ERROR "bad checksum", an EISDIR on a block file still logs WARN, and both remain visible at the default filter. The changelog named the tier boundary as "a few hundred kilobytes"; the packed-to-blocks migration fires at pack_max, which defaults to CHUNK_SIZE, so it is a megabyte. `read_starting_past_eof_returns_no_bytes` documents an offset at the end as well as past it, but only covered the block tier past the end. `copy_file_range_that_fails_partway_reports_the_bytes_it_moved` read errno where the syscall had succeeded, printing a stale error next to a wrong count; its two sibling call sites already read errno only on a negative return. Both new tests compared megabyte buffers with `assert_eq!`, which dumps both operands, where `pattern_check` names the first bad offset. * fix(start-os): harden backup-fs partial copy handling * fix(start-os): preserve backup-fs partial write accounting * style(start-os): group backup-fs imports the way the pinned rustfmt does `rustfmt.toml` sets `group_imports` and `imports_granularity`, both unstable, so a stable `cargo fmt` accepts the file and silently leaves these alone. CI runs the pinned nightly in `build/fmt/run-fmt.sh` and failed on the two test modules' import order. Run `make start-os-format`. | 8 天前 | |
fix(start-os): unbreak the NVIDIA image on GB10 — drop unusable GSP firmware, stage MOK enrollment (#3693) * fix(image-recipe): drop nouveau GSP firmware from the NVIDIA images The -nvidia images blacklist nouveau and drive the GPU with NVIDIA's proprietary driver, but live-build's --firmware-chroot still installs firmware-nvidia-graphics and firmware-nvidia-tesla-535-gsp, and initramfs-tools copies the firmware every included module declares regardless of the blacklist. nouveau.ko is included under MODULES=most, so its GSP blobs are pulled in too. Measured in the published startos-0.4.0.1-fdb27c7_aarch64-nvidia.iso: usr/lib/firmware/nvidia/ is 159,225,325 B across 381 entries — 57% of the 266 MB uncompressed initramfs, every directory a nouveau chip codename. None of it is the proprietary driver's own firmware, which the .run installs under a version-named directory that never enters the initramfs, since no nvidia*.ko does either. That is why -nvidia asked the bootloader to read a 163.5 MB initrd where the plain aarch64 image reads 45 MB — and on an NVIDIA GB10 (DGX Spark) it left the machine stopped on the bootloader's last line with no kernel output. Confirmed on hardware: 0.4.0.1 hangs, this build boots. Note 0.4.0 booted a payload of essentially the same size under kernel 7.0.13, so the fault is an interaction with 7.1.3 rather than a fixed size limit; this buys margin (101 MB against a ~209 MB failure point) but does not remove the class. Neither package is depended on by anything in the image — firmware-misc-nonfree only Recommends the first, which apt does not act on at removal. * fix(start-os): retry the container-runtime base image download The riscv64 build failed on a truncated transfer from images.linuxcontainers.org — `curl: (18) transfer closed with 79061983 bytes remaining to read` — after the Rust compile had already succeeded. The download had no retry and runs under `set -e`, so one blip on that CDN fails the whole job. The rootfs download also moves from a shell redirect to -o, which is load-bearing: curl rewinds an output file it owns before each retry attempt but cannot rewind stdout, so `> file` with --retry concatenates every partial attempt into one corrupt file. Measured against a local server that always truncates: `> file` with --retry 2 yields 1200 bytes from three 400-byte attempts, `-o file` yields 400. Verified by running the script for real: 105,336,832 B fetched, `debian.x86_64.squashfs: OK`, exit 0. * docs(start-os): name the second firmware package correctly firmware-nvidia-tesla-535-gsp is NVIDIA's proprietary GSP firmware for the 535 driver series — /usr/lib/firmware/nvidia/535.274.02/gsp_ga10x.bin and gsp_tu10x.bin — not nouveau's. Only firmware-nvidia-graphics carries nouveau's chip-named GSP blobs, and only those reach the initramfs. Both are still correct to purge: the .run installs the driver's own firmware under nvidia/580.173.02, which neither package owns. Verified at the ELF level — nvidia.ko's only firmware references are nvidia/580.173.02/gsp_{ga10x,tu10x}.bin, and both survive the purge in the built image at the .run's exact byte sizes. * fix(start-os): stage MOK enrollment when the password is set Closes #3702. enroll_mok refuses to stage the import unless the start9 user already has a password hash, and on a fresh Secure Boot install neither caller could satisfy that: - os_install/mod.rs resolved mok_pub inside the target overlay but read the hardcoded host /etc/shadow — the installer's, not the target's. The target has no password at install either way. - init.rs runs once per boot, and setup_init calls init() at setup.rs:185 but write_shadow() at setup.rs:226, so the attempt always preceded the password it depends on, in the same function. The import was therefore first staged on the boot after setup, and MokManager only prompts on the boot after that — two restarts, with every skip logged at info and the install-time caller logging only on Err. On a server, which is rarely restarted, the modules stay unloadable indefinitely: Secure Boot lockdown refuses the unsigned NVIDIA driver, nvidia-smi reports it cannot talk to the driver, and a GPU service fails with no CUDA device and nothing pointing at signing. Stage it in write_shadow instead, which is the single point every path that sets a password goes through (setup, restore, password change), so the prompt appears on the next restart. enroll_mok now takes the root holding both the certificate and etc/shadow, which also corrects the install-time read; that call stays for the case where the target already carries a password. * docs(start-os): note what the preserve path must not delete A preserve-reinstall keeps config/overlay wholesale, deleting only named paths, which is what makes Secure Boot survive it: the MOK key at overlay/var/lib/dkms is reused, so modules re-signed during install still match the certificate already enrolled in firmware, and no re-enrollment is needed. Nothing said so, and adding one more delete_dir would break it silently — the modules would simply stop loading. * fix(start-os): back up the config before repartitioning on a preserve install Closes #3704. install_os_to called partition(), which rewrites the OS partitions, and only then tried to read the previous install's config off part_info.root. By then that filesystem is gone, so the mount failed, the backup was skipped with no log at all (`if let Ok(guard) = ...`), /tmp/config.bak never existed, and the restore fell to create_dir_all — an empty config. Confirmed on a DGX Spark: config/overlay/var/lib/dkms was lost, the install generated a new Secure Boot MOK key while the firmware still trusted the old certificate, and Secure Boot then refused every module signed with the new one. The GPU disappeared with nothing naming signing as the cause. Take the backup before partition() instead, locating the previous root by content — the first partition carrying a `config` directory — rather than by label, so an install predating any labelling scheme is still found. Three things this ordering makes possible or necessary: - Failure now aborts. partition() has not touched a byte at that point (gpt::partition runs quiesce_disk first, which does not write), so failing costs a retry, while continuing lets mkfs destroy the only copy of a config the user asked to keep. - The previous root is mounted read-only and the prune list is applied to the copy, so an install that aborts leaves the old system intact. - The copy is staged at /tmp/config.bak.staged and renamed only once the prunes succeed. `cp -r` writes a partial tree and still fails, and the restore accepts whatever sits at the final path, so a half-copied tree must never occupy it — an unpruned one would layer the previous image's overlay/lib and overlay/usr/lib over the new squashfs. An install that preserves nothing now clears any backup an earlier attempt in the same boot left behind, since the restore is unconditional and would otherwise copy it onto a drive the user asked to wipe. * fix(start-os): restore initcall_blacklist=tegra234_cbb_init on aarch64-nvidia Reverts the drop in the previous revision of this branch. The hardware evidence that retired it was misread: adding the parameter by hand to 0.4.0.1 and still hanging showed only that it was not SUFFICIENT, not that it was unnecessary. 0.4.0.1 carried both faults — the 163 MB initramfs and this driver — so fixing either one alone still hung. The full 2x2 on a DGX Spark now reads: 163 MB initrd, no blacklist -> hang (0.4.0.1) 163 MB initrd, blacklist -> hang (added by hand) 54 MB initrd, blacklist -> boots (f8d0c0a7) 54 MB initrd, no blacklist -> hang (91d631b0) The last two differ only by this commit; nothing else between them touches boot. Both faults are independent and both must be fixed. Applied to aarch64-nvidia only, gated on PLATFORM.txt, in both boot paths since they run the same kernel: the installed system via debian/postinst and the live installer via --bootappend-live. * fix(image-recipe): build the open NVIDIA kernel modules for aarch64-nvidia Confirmed on a DGX Spark: with 580.173.02's proprietary modules the GPU is found but never initializes — NVRM: GPU at PCI:000f:01:00: GPU-0f059bf0-... NVRM: Xid (PCI:000f:01:00): 169, fd8cd000 94000000 NVRM: GPU 000f:01:00.0: RmInitAdapter failed! (0x62:0x1a:2631) repeating every ~60s, with nvidia-smi hanging and then reporting no devices. Installing the same driver version's open modules on the machine makes the GPU work, so the flavour is the variable, not the version. That matches NVIDIA's README for this driver — "Blackwell and later are only supported by the open kernel modules" — and GB10 is Blackwell. It also explains why 0.4.0 was fine on proprietary: it shipped 580.159.03, before that restriction applied. NVIDIA's own DGX OS ships nvidia-driver-580-open on this hardware. Scoped to aarch64-nvidia. x86_64-nvidia keeps the proprietary flavour, where open would drop Maxwell, Pascal and Volta; aarch64 has no pre-Turing NVIDIA hardware to lose. * fix(image-recipe): actually pass the chosen NVIDIA module flavour fa848190 added NVIDIA_MODULE_TYPE and never read it — the installer line still hardcoded proprietary, so the commit was a no-op and aarch64-nvidia would have shipped the flavour that cannot initialize a GB10. Interpolated unescaped on purpose: the hook heredoc at :260 is unquoted, so this expands at generation time like ${QEMU_ARCH} does, unlike the escaped \${KVER} on the next line. Verified by generating the hook for both platforms: aarch64-nvidia yields kernel-module-type=open, x86_64-nvidia yields proprietary. Also corrects the rationale in fa848190's message. That claimed 580.159.03 predated the Blackwell-is-open-only restriction; it does not. Between .159.03 and .173.02, README/kernel_open.html is byte-identical, the "Blackwell and later are only supported by the open kernel modules" sentence appears verbatim in both, supported-gpus.json is byte-identical, and GB10 lacks proprietary support in both. So GB10 was never officially supported on the proprietary modules; .159.03 drove one anyway and we do not know why. The flavour flip stands on the measured A/B — same driver version, open works, proprietary does not — not on a policy change that did not happen. * feat(image-recipe): build the open NVIDIA kernel modules on every platform Blackwell is open-only, so the proprietary modules leave a GB10, an RTX 50-series or an RTX PRO 6000 found but unusable — x86_64-nvidia has the same defect the DGX Spark hit, not just aarch64. Dropping the per-platform split in favour of open everywhere is a deliberate call by dr-bonez: current hardware works, and Maxwell, Pascal and Volta are given up, since the open modules need the GSP first shipped in Turing. Verified by generating the hook for both platforms — each yields kernel-module-type=open. The lost cards are named in the changelog under Changed rather than buried, since a GTX 1080 or Tesla P40 owner updating to this loses GPU compute with no other signal. The docs' download list also gains the NVIDIA variant, which it never described at all, now that choosing it carries a hardware floor. * fix(image-recipe): let UVM register a GPU the kernel cannot offer ATS to Debian builds no CONFIG_IOMMU_SVA, so UVM pins every VA space to ATS_UNSUPPORTED while an integrated part like GB10 still advertises ATS. uvm_va_space_register_gpu then refuses the GPU with NV_ERR_INVALID_DEVICE, which UVM never logs, and libcuda reports only CUDA_ERROR_NOT_INITIALIZED: nvidia-smi works while every CUDA program fails at startup. Extract the installer, drop that guard, and build from the patched source. Measured on a DGX Spark: cuInit 0, 122552 MiB visible, and vLLM serving a 122B model at 74% GPU utilisation. Dropping the guard forfeits mixing coherent and non-coherent GPUs in one process, which these images do not support. The hook aborts if a future driver moves the guard, so the patch cannot silently stop applying. * fix(start-os): restore only a preserved config that belongs to the disk being installed The restore after mkfs was gated on /tmp/config.bak existing at all, and /tmp outlives a failed install attempt within one live boot. Two shapes that opened: - A failed preserve attempt on disk A left its backup behind; a retry on a different disk whose pool was preserved but which had no OS root kept the stale backup and restored disk A's config — shadow and MOK key included — onto disk B. - A same-disk retry after a late failure found the new root's fresh default config and staged it over the good backup the earlier attempt had taken. preserve_config now writes a marker naming the disk a backup came from and clears any leftover backup that does not match the disk being installed, and previous_os_root only recognizes a config that completed setup (has a disk.guid), so a failed attempt's default config is no longer staged over a good backup. A same-disk retry after an early failure still restores the leftover backup, which is then the only surviving copy of the real config. | 23 天前 | |
chore(start-sdk): promote next release to 3.0.0 (#3900) Helix-Harness: pi Helix-Model: openai-codex/gpt-5.6-sol | 6 天前 | |
fix(start-os): unbreak the NVIDIA image on GB10 — drop unusable GSP firmware, stage MOK enrollment (#3693) * fix(image-recipe): drop nouveau GSP firmware from the NVIDIA images The -nvidia images blacklist nouveau and drive the GPU with NVIDIA's proprietary driver, but live-build's --firmware-chroot still installs firmware-nvidia-graphics and firmware-nvidia-tesla-535-gsp, and initramfs-tools copies the firmware every included module declares regardless of the blacklist. nouveau.ko is included under MODULES=most, so its GSP blobs are pulled in too. Measured in the published startos-0.4.0.1-fdb27c7_aarch64-nvidia.iso: usr/lib/firmware/nvidia/ is 159,225,325 B across 381 entries — 57% of the 266 MB uncompressed initramfs, every directory a nouveau chip codename. None of it is the proprietary driver's own firmware, which the .run installs under a version-named directory that never enters the initramfs, since no nvidia*.ko does either. That is why -nvidia asked the bootloader to read a 163.5 MB initrd where the plain aarch64 image reads 45 MB — and on an NVIDIA GB10 (DGX Spark) it left the machine stopped on the bootloader's last line with no kernel output. Confirmed on hardware: 0.4.0.1 hangs, this build boots. Note 0.4.0 booted a payload of essentially the same size under kernel 7.0.13, so the fault is an interaction with 7.1.3 rather than a fixed size limit; this buys margin (101 MB against a ~209 MB failure point) but does not remove the class. Neither package is depended on by anything in the image — firmware-misc-nonfree only Recommends the first, which apt does not act on at removal. * fix(start-os): retry the container-runtime base image download The riscv64 build failed on a truncated transfer from images.linuxcontainers.org — `curl: (18) transfer closed with 79061983 bytes remaining to read` — after the Rust compile had already succeeded. The download had no retry and runs under `set -e`, so one blip on that CDN fails the whole job. The rootfs download also moves from a shell redirect to -o, which is load-bearing: curl rewinds an output file it owns before each retry attempt but cannot rewind stdout, so `> file` with --retry concatenates every partial attempt into one corrupt file. Measured against a local server that always truncates: `> file` with --retry 2 yields 1200 bytes from three 400-byte attempts, `-o file` yields 400. Verified by running the script for real: 105,336,832 B fetched, `debian.x86_64.squashfs: OK`, exit 0. * docs(start-os): name the second firmware package correctly firmware-nvidia-tesla-535-gsp is NVIDIA's proprietary GSP firmware for the 535 driver series — /usr/lib/firmware/nvidia/535.274.02/gsp_ga10x.bin and gsp_tu10x.bin — not nouveau's. Only firmware-nvidia-graphics carries nouveau's chip-named GSP blobs, and only those reach the initramfs. Both are still correct to purge: the .run installs the driver's own firmware under nvidia/580.173.02, which neither package owns. Verified at the ELF level — nvidia.ko's only firmware references are nvidia/580.173.02/gsp_{ga10x,tu10x}.bin, and both survive the purge in the built image at the .run's exact byte sizes. * fix(start-os): stage MOK enrollment when the password is set Closes #3702. enroll_mok refuses to stage the import unless the start9 user already has a password hash, and on a fresh Secure Boot install neither caller could satisfy that: - os_install/mod.rs resolved mok_pub inside the target overlay but read the hardcoded host /etc/shadow — the installer's, not the target's. The target has no password at install either way. - init.rs runs once per boot, and setup_init calls init() at setup.rs:185 but write_shadow() at setup.rs:226, so the attempt always preceded the password it depends on, in the same function. The import was therefore first staged on the boot after setup, and MokManager only prompts on the boot after that — two restarts, with every skip logged at info and the install-time caller logging only on Err. On a server, which is rarely restarted, the modules stay unloadable indefinitely: Secure Boot lockdown refuses the unsigned NVIDIA driver, nvidia-smi reports it cannot talk to the driver, and a GPU service fails with no CUDA device and nothing pointing at signing. Stage it in write_shadow instead, which is the single point every path that sets a password goes through (setup, restore, password change), so the prompt appears on the next restart. enroll_mok now takes the root holding both the certificate and etc/shadow, which also corrects the install-time read; that call stays for the case where the target already carries a password. * docs(start-os): note what the preserve path must not delete A preserve-reinstall keeps config/overlay wholesale, deleting only named paths, which is what makes Secure Boot survive it: the MOK key at overlay/var/lib/dkms is reused, so modules re-signed during install still match the certificate already enrolled in firmware, and no re-enrollment is needed. Nothing said so, and adding one more delete_dir would break it silently — the modules would simply stop loading. * fix(start-os): back up the config before repartitioning on a preserve install Closes #3704. install_os_to called partition(), which rewrites the OS partitions, and only then tried to read the previous install's config off part_info.root. By then that filesystem is gone, so the mount failed, the backup was skipped with no log at all (`if let Ok(guard) = ...`), /tmp/config.bak never existed, and the restore fell to create_dir_all — an empty config. Confirmed on a DGX Spark: config/overlay/var/lib/dkms was lost, the install generated a new Secure Boot MOK key while the firmware still trusted the old certificate, and Secure Boot then refused every module signed with the new one. The GPU disappeared with nothing naming signing as the cause. Take the backup before partition() instead, locating the previous root by content — the first partition carrying a `config` directory — rather than by label, so an install predating any labelling scheme is still found. Three things this ordering makes possible or necessary: - Failure now aborts. partition() has not touched a byte at that point (gpt::partition runs quiesce_disk first, which does not write), so failing costs a retry, while continuing lets mkfs destroy the only copy of a config the user asked to keep. - The previous root is mounted read-only and the prune list is applied to the copy, so an install that aborts leaves the old system intact. - The copy is staged at /tmp/config.bak.staged and renamed only once the prunes succeed. `cp -r` writes a partial tree and still fails, and the restore accepts whatever sits at the final path, so a half-copied tree must never occupy it — an unpruned one would layer the previous image's overlay/lib and overlay/usr/lib over the new squashfs. An install that preserves nothing now clears any backup an earlier attempt in the same boot left behind, since the restore is unconditional and would otherwise copy it onto a drive the user asked to wipe. * fix(start-os): restore initcall_blacklist=tegra234_cbb_init on aarch64-nvidia Reverts the drop in the previous revision of this branch. The hardware evidence that retired it was misread: adding the parameter by hand to 0.4.0.1 and still hanging showed only that it was not SUFFICIENT, not that it was unnecessary. 0.4.0.1 carried both faults — the 163 MB initramfs and this driver — so fixing either one alone still hung. The full 2x2 on a DGX Spark now reads: 163 MB initrd, no blacklist -> hang (0.4.0.1) 163 MB initrd, blacklist -> hang (added by hand) 54 MB initrd, blacklist -> boots (f8d0c0a7) 54 MB initrd, no blacklist -> hang (91d631b0) The last two differ only by this commit; nothing else between them touches boot. Both faults are independent and both must be fixed. Applied to aarch64-nvidia only, gated on PLATFORM.txt, in both boot paths since they run the same kernel: the installed system via debian/postinst and the live installer via --bootappend-live. * fix(image-recipe): build the open NVIDIA kernel modules for aarch64-nvidia Confirmed on a DGX Spark: with 580.173.02's proprietary modules the GPU is found but never initializes — NVRM: GPU at PCI:000f:01:00: GPU-0f059bf0-... NVRM: Xid (PCI:000f:01:00): 169, fd8cd000 94000000 NVRM: GPU 000f:01:00.0: RmInitAdapter failed! (0x62:0x1a:2631) repeating every ~60s, with nvidia-smi hanging and then reporting no devices. Installing the same driver version's open modules on the machine makes the GPU work, so the flavour is the variable, not the version. That matches NVIDIA's README for this driver — "Blackwell and later are only supported by the open kernel modules" — and GB10 is Blackwell. It also explains why 0.4.0 was fine on proprietary: it shipped 580.159.03, before that restriction applied. NVIDIA's own DGX OS ships nvidia-driver-580-open on this hardware. Scoped to aarch64-nvidia. x86_64-nvidia keeps the proprietary flavour, where open would drop Maxwell, Pascal and Volta; aarch64 has no pre-Turing NVIDIA hardware to lose. * fix(image-recipe): actually pass the chosen NVIDIA module flavour fa848190 added NVIDIA_MODULE_TYPE and never read it — the installer line still hardcoded proprietary, so the commit was a no-op and aarch64-nvidia would have shipped the flavour that cannot initialize a GB10. Interpolated unescaped on purpose: the hook heredoc at :260 is unquoted, so this expands at generation time like ${QEMU_ARCH} does, unlike the escaped \${KVER} on the next line. Verified by generating the hook for both platforms: aarch64-nvidia yields kernel-module-type=open, x86_64-nvidia yields proprietary. Also corrects the rationale in fa848190's message. That claimed 580.159.03 predated the Blackwell-is-open-only restriction; it does not. Between .159.03 and .173.02, README/kernel_open.html is byte-identical, the "Blackwell and later are only supported by the open kernel modules" sentence appears verbatim in both, supported-gpus.json is byte-identical, and GB10 lacks proprietary support in both. So GB10 was never officially supported on the proprietary modules; .159.03 drove one anyway and we do not know why. The flavour flip stands on the measured A/B — same driver version, open works, proprietary does not — not on a policy change that did not happen. * feat(image-recipe): build the open NVIDIA kernel modules on every platform Blackwell is open-only, so the proprietary modules leave a GB10, an RTX 50-series or an RTX PRO 6000 found but unusable — x86_64-nvidia has the same defect the DGX Spark hit, not just aarch64. Dropping the per-platform split in favour of open everywhere is a deliberate call by dr-bonez: current hardware works, and Maxwell, Pascal and Volta are given up, since the open modules need the GSP first shipped in Turing. Verified by generating the hook for both platforms — each yields kernel-module-type=open. The lost cards are named in the changelog under Changed rather than buried, since a GTX 1080 or Tesla P40 owner updating to this loses GPU compute with no other signal. The docs' download list also gains the NVIDIA variant, which it never described at all, now that choosing it carries a hardware floor. * fix(image-recipe): let UVM register a GPU the kernel cannot offer ATS to Debian builds no CONFIG_IOMMU_SVA, so UVM pins every VA space to ATS_UNSUPPORTED while an integrated part like GB10 still advertises ATS. uvm_va_space_register_gpu then refuses the GPU with NV_ERR_INVALID_DEVICE, which UVM never logs, and libcuda reports only CUDA_ERROR_NOT_INITIALIZED: nvidia-smi works while every CUDA program fails at startup. Extract the installer, drop that guard, and build from the patched source. Measured on a DGX Spark: cuInit 0, 122552 MiB visible, and vLLM serving a 122B model at 74% GPU utilisation. Dropping the guard forfeits mixing coherent and non-coherent GPUs in one process, which these images do not support. The hook aborts if a future driver moves the guard, so the patch cannot silently stop applying. * fix(start-os): restore only a preserved config that belongs to the disk being installed The restore after mkfs was gated on /tmp/config.bak existing at all, and /tmp outlives a failed install attempt within one live boot. Two shapes that opened: - A failed preserve attempt on disk A left its backup behind; a retry on a different disk whose pool was preserved but which had no OS root kept the stale backup and restored disk A's config — shadow and MOK key included — onto disk B. - A same-disk retry after a late failure found the new root's fresh default config and staged it over the good backup the earlier attempt had taken. preserve_config now writes a marker naming the disk a backup came from and clears any leftover backup that does not match the disk being installed, and previous_os_root only recognizes a config that completed setup (has a disk.guid), so a failed attempt's default config is no longer staged over a good backup. A same-disk retry after an early failure still restores the leftover backup, which is then the only surviving copy of the real config. | 23 天前 | |
Registry switching, descriptions, and per-registry warnings, without the known-registries list (#3897) * fix(marketplace): switch registries at once, and carry each listed registry's notice Switching registries left the previous registry's packages on screen under the new registry's name until the new fetch landed. The shared component rendered whatever `currentRegistry$` last emitted, and on the brochure that stream only emits once a fetch completes; the OS UI's catalog cache hid the same gap whenever the target registry was not loaded yet. The component now renders only the registry matching the selected url, so a switch shows the cached content or skeletons immediately. The brochure also keeps every registry it has fetched, so switching back is instant and its picker shows a visited registry's live icon instead of the bundled fallback. The Start9 Registry showed the community icon because registry.start9.com served that very image and the picker prefers a registry's live icon. The icon on the server has since been corrected, and the manifest now pins it, so a listed registry that serves a different icon falls back to the pin. The known-registries manifest lists Start9's own four registries, and every entry carries a `warning`: a LocaleString the marketplace shows while that registry is selected, carrying the translations the old per-registry dialog had. An unlisted registry keeps the generic third-party caveat, and the add dialog shows the selected entry's notice rather than a blanket one. A registry that serves no icon no longer counts as drifted from its pin; only a different name or icon does. The KnownRegistry binding must be taken from the Generated Artifacts run. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(registry): declare a description, pinned for listed registries A registry can now describe itself: `start-registry info set-description` stores markdown (a LocaleString, so it can carry translations) in the index, and `info` returns it. The marketplace shows it in an info banner above every other notice while that registry is selected, rendered through the same markdown pipeline as release notes. The known-registries manifest pins a description for each listed registry the way it pins the name and icon: the pinned text is what the marketplace shows, and a listed registry that serves a different one trips the drift banner. The four Start9 registries get their descriptions here; the same texts are to be set on the registries themselves. The RegistryInfo, KnownRegistry, FullIndex, and SetDescriptionParams bindings and the start-registry man pages must be taken from the Generated Artifacts run. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(marketplace): send beta testers to the service-testing room Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(marketplace): point packagers at the service-packaging room, not the submissions inbox Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(marketplace): say when a service belongs in a dedicated registry instead Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(marketplace): take the edited descriptions, and tell beta testers bugs are expected Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(marketplace): translate the pinned registry descriptions Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(marketplace): pin the icons the Start9 and Beta registries now serve Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(shared): bundle the icons the Start9 and Beta registries serve as their fallbacks Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * refactor(marketplace): display verified registries from the manifest alone The manifest is now the only authored identity for a registry Start9 lists. It moves into @start9labs/shared, which bundles it as the fallback for when the published copy can't be fetched, so the hardcoded defaultIdentities, the knownRegistries URL list, and the four bundled registry icons go away. The brochure still serves it from .well-known through its assets entry, and a push to master still redeploys it. One resolver replaces resolveIdentity, resolveIcon, pinnedIcon, findKnown, and identityMatches. A listed registry shows its listed name, icon, description, and warning, whatever its server reports; an unlisted one shows what it reports, except that a name containing a listed name or "Start9" is replaced by the registry's host, and the page says so. The drift banner goes with the comparisons behind it: a listed registry can't disagree with its listing on screen, and impersonation is caught by the name rule, which a lookalike icon or description never was. Listed registries carry a "Verified by Start9" mark in the picker, and the generic notice for unlisted ones now says what it can stand behind: the registry is not on the list Start9 publishes. The OS side compares a fetched name against the stored one before writing it back, instead of against the listed one. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(start-sdk): how a registry gets verified by Start9 A new Verification page in the hosting chapter says what a listing attests to (the address and an operator Start9 can reach, not the services), the requirements, how to apply, and how a listing is kept current. The StartOS book and the publishing page point at it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(marketplace): name a listed registry's operator and contact Every listing now carries the operator's public name and a contact email, shown beneath the description while the registry is selected. Start9's own entries name Start9 and leave the contact to be filled in. The verification page asks for both, routes applications through the submissions inbox like a package, and says that an unlisted registry, Tor-only included, needs none of this. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(marketplace): lay the info banner out as Description and Contact Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * refactor(marketplace): drop the known-registries list Adding a registry is the URL prompt again, a registry's name and icon are what it serves, and the caveat banner is keyed by URL, as before #3865. The manifest, the marketplace.known-registries RPC and its binding, the add dialog's list, the pinned identities, the verified mark, and the verification policy page all go. The switch fix, the description feature, and the per-registry warning texts stay, with the beta texts saying bugs are expected and the community beta text carrying both caveats. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(start-registry): 1.1.0, since a registry can now declare a description Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(marketplace): no caveat banner on the Community Registry Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(registry): regenerate bindings and man pages Helix-Harness: pi Helix-Model: openai-codex/gpt-5.6-sol --------- Co-authored-by: Matt Hill <9935159+MattDHill@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Helix <267227783+helix-nine@users.noreply.github.com> | 6 天前 | |
fix(ui): align mobile notification checkboxes (#3616) * fix(ui): align mobile notification checkboxes * fix(ui): place the checkbox by layout, not by a fixed offset The mobile checkbox is still positioned against the row by hand, so it has to encode item.component's grid geometry as a literal in table.component — the coupling that broke when 840019c60 reordered the rows. And no single literal can be right any more: that commit also pinned `.service` to grid row 1, so a service avatar makes the title row 32px against 27.4px without one, and the icon the checkbox stands in for sits 4px lower on notifications that have one. Project the checkbox into `.title` ahead of the icon instead. On mobile it is then an in-flow flex item, so `.title`'s existing `align-items: center` places it — the same rule that places the icon. The two swap via `display`; both are 1rem behind the same gap, so nothing on the card moves as selection turns on. Desktop keeps absolute positioning, now anchored to the row rather than to the date cell the checkbox no longer lives in, so mobile has nothing to undo. No offset constant remains for a future row reorder to invalidate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Matt Hill <9935159+MattDHill@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> | 1 个月前 | |
fix(core): apply isolation tests for NVMe quirks instead of global pa… (#3372) * fix(core): apply isolation tests for NVMe quirks instead of global params Refactor the PR #3338 to address feedback from helix-nine on issue #3336. Instead of globally disabling APST and HMB on all StartOS machines, this adds an isolation test script (scripts/nvme-isolation-test.sh) to help users debug their specific drive issues without impacting other users. The Rust system validation has been updated to check for kernel command line quirks instead of modprobe params, as NVMe is typically built-in. * fix: remove init-time NVMe warnings and revert formatting churn * fix(core): drop init-time NVMe warnings and checks | 2 个月前 | |
chore(start-os): cut 0.4.0.1; move the OS version off Cargo.toml (#3564) SemVer has no fourth segment, so `projects/start-os/Cargo.toml` cannot hold 0.4.0.1. It now carries a `0.4.0-rev.1` label purely to acknowledge the release it belongs to, and root package.json becomes the source of truth. The label must never be compared: `0.4.0-rev.1` is a SemVer prerelease, so it sorts *below* 0.4.0 in both `semver` and `exver`. Had it stayed authoritative it would have reached the CI-generated compat range (`>=0.3.5 <=${VERSION}`) and excluded installed 0.4.0 servers from the update it patches. `pre-check` now fails if the label drifts from the version being cut. This restores how 0.3.x worked — `backend/Cargo.toml` was `0.3.5-rev.1` while `web/package.json` held the real `0.3.5.1` and check-version.sh read that. The monorepo reorg (950be49dd) repointed the readers at the crate manifest; CI never followed, so startos-iso.yaml has been deriving the registry key, S3 prefix and source_version range from root package.json all along. - build/env/version.sh is the single reader; check-version.sh, basename.sh, debian/build.sh and derive_version go through it - build.rs bakes STARTOS_VERSION from package.json; the start-os bins and start-container's man page report it instead of CARGO_PKG_VERSION - version::tests::current_matches_manifest fails if Current drifts from package.json, which would otherwise publish a version the server never reports 0.4.0.1 itself adds the version node (mandatory even with no migration, or pre_init takes the Equal branch and the registry re-offers the update forever) and carries the pre-installed-Pi data-pool fix from #3556. | 1 个月前 | |
Registry switching, descriptions, and per-registry warnings, without the known-registries list (#3897) * fix(marketplace): switch registries at once, and carry each listed registry's notice Switching registries left the previous registry's packages on screen under the new registry's name until the new fetch landed. The shared component rendered whatever `currentRegistry$` last emitted, and on the brochure that stream only emits once a fetch completes; the OS UI's catalog cache hid the same gap whenever the target registry was not loaded yet. The component now renders only the registry matching the selected url, so a switch shows the cached content or skeletons immediately. The brochure also keeps every registry it has fetched, so switching back is instant and its picker shows a visited registry's live icon instead of the bundled fallback. The Start9 Registry showed the community icon because registry.start9.com served that very image and the picker prefers a registry's live icon. The icon on the server has since been corrected, and the manifest now pins it, so a listed registry that serves a different icon falls back to the pin. The known-registries manifest lists Start9's own four registries, and every entry carries a `warning`: a LocaleString the marketplace shows while that registry is selected, carrying the translations the old per-registry dialog had. An unlisted registry keeps the generic third-party caveat, and the add dialog shows the selected entry's notice rather than a blanket one. A registry that serves no icon no longer counts as drifted from its pin; only a different name or icon does. The KnownRegistry binding must be taken from the Generated Artifacts run. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(registry): declare a description, pinned for listed registries A registry can now describe itself: `start-registry info set-description` stores markdown (a LocaleString, so it can carry translations) in the index, and `info` returns it. The marketplace shows it in an info banner above every other notice while that registry is selected, rendered through the same markdown pipeline as release notes. The known-registries manifest pins a description for each listed registry the way it pins the name and icon: the pinned text is what the marketplace shows, and a listed registry that serves a different one trips the drift banner. The four Start9 registries get their descriptions here; the same texts are to be set on the registries themselves. The RegistryInfo, KnownRegistry, FullIndex, and SetDescriptionParams bindings and the start-registry man pages must be taken from the Generated Artifacts run. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(marketplace): send beta testers to the service-testing room Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(marketplace): point packagers at the service-packaging room, not the submissions inbox Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(marketplace): say when a service belongs in a dedicated registry instead Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(marketplace): take the edited descriptions, and tell beta testers bugs are expected Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(marketplace): translate the pinned registry descriptions Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(marketplace): pin the icons the Start9 and Beta registries now serve Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(shared): bundle the icons the Start9 and Beta registries serve as their fallbacks Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * refactor(marketplace): display verified registries from the manifest alone The manifest is now the only authored identity for a registry Start9 lists. It moves into @start9labs/shared, which bundles it as the fallback for when the published copy can't be fetched, so the hardcoded defaultIdentities, the knownRegistries URL list, and the four bundled registry icons go away. The brochure still serves it from .well-known through its assets entry, and a push to master still redeploys it. One resolver replaces resolveIdentity, resolveIcon, pinnedIcon, findKnown, and identityMatches. A listed registry shows its listed name, icon, description, and warning, whatever its server reports; an unlisted one shows what it reports, except that a name containing a listed name or "Start9" is replaced by the registry's host, and the page says so. The drift banner goes with the comparisons behind it: a listed registry can't disagree with its listing on screen, and impersonation is caught by the name rule, which a lookalike icon or description never was. Listed registries carry a "Verified by Start9" mark in the picker, and the generic notice for unlisted ones now says what it can stand behind: the registry is not on the list Start9 publishes. The OS side compares a fetched name against the stored one before writing it back, instead of against the listed one. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(start-sdk): how a registry gets verified by Start9 A new Verification page in the hosting chapter says what a listing attests to (the address and an operator Start9 can reach, not the services), the requirements, how to apply, and how a listing is kept current. The StartOS book and the publishing page point at it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(marketplace): name a listed registry's operator and contact Every listing now carries the operator's public name and a contact email, shown beneath the description while the registry is selected. Start9's own entries name Start9 and leave the contact to be filled in. The verification page asks for both, routes applications through the submissions inbox like a package, and says that an unlisted registry, Tor-only included, needs none of this. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(marketplace): lay the info banner out as Description and Contact Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * refactor(marketplace): drop the known-registries list Adding a registry is the URL prompt again, a registry's name and icon are what it serves, and the caveat banner is keyed by URL, as before #3865. The manifest, the marketplace.known-registries RPC and its binding, the add dialog's list, the pinned identities, the verified mark, and the verification policy page all go. The switch fix, the description feature, and the per-registry warning texts stay, with the beta texts saying bugs are expected and the community beta text carrying both caveats. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(start-registry): 1.1.0, since a registry can now declare a description Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(marketplace): no caveat banner on the Community Registry Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(registry): regenerate bindings and man pages Helix-Harness: pi Helix-Model: openai-codex/gpt-5.6-sol --------- Co-authored-by: Matt Hill <9935159+MattDHill@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Helix <267227783+helix-nine@users.noreply.github.com> | 6 天前 | |
feat(start-os): name the installed web app after the server (#3879) The router serves /manifest.webmanifest with the server's hostname as its name and short_name, so adding the UI to a home screen or installing it from a desktop browser labels it with the server name instead of "StartOS". Chrome refreshes an installed app's name from the manifest; Safari reads apple-mobile-web-app-title instead, which the root component now keeps in step with the hostname. The built manifest is no longer a service-worker asset — ngsw hash-checks the bytes it built, which the served body no longer matches. The Server Name dialog now lists what the name is used for: the server's address, every installed service's local address, the browser tab and installed app name, and SSH and system logs. Closes #2327 Co-authored-by: Matt Hill <9935159+MattDHill@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> | 7 天前 | |
docs: delete scope-doc text with no live producer (#3728) * docs(sdk): drop the fleet-provenance clause from the description advice "Two more characters' worth of advice, both from descriptions already in the registries" — the provenance half is a claim about what the fleet's short descriptions look like right now, with nothing keeping it true. It is already only partly accurate: 4 of the 107 packages with an en_US short open with the service name, which is the pattern the first bullet tells you to avoid. The two bullets it introduces both have live producers and stay exactly as written. Split out from the rest of the packaging-guide cruft audit because this paragraph exists only on master — the published guide does not carry it, so it cannot be fixed on live-docs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: delete scope-doc text with no live producer Audited all 65 AGENTS.md / ARCHITECTURE.md / CONTRIBUTING.md files across every scope against one test: a sentence citing what not to do, what not to include, or a mistake once made must name a live producer — a scaffold that emits it, a neighbour someone would copy, a tool that does it unless you intervene, or an obvious-but-wrong fix someone would reach for. No producer, delete. Facts that are simply wrong: - start-registry's CONTRIBUTING said its Cargo version "tracks the OS release line — don't bump it independently", while its own AGENTS.md one directory over says the opposite. The crate is 1.0.2 and StartOS is 0.4.0.2, so the CONTRIBUTING rule is the retired one. Its "(currently 1.0.0)" had drifted too. - rpc-toolkit's CONTRIBUTING described a `rustfmt.toml` the crate does not ship; only the repo root has one. - start-sdk's build table documented `make dist`, which is not a target in that Makefile (start-core's `make dist`, referenced further down the same file, is real and stays). - The root ARCHITECTURE tree put `apt/` under projects/start-os/; it is at the repo root. - shared-libs' CONTRIBUTING said ts-modules' contents are Angular libraries; it also holds the non-Angular start-core. - start-registry's ARCHITECTURE counted "all five product binaries"; the root AGENTS.md counts six. - start-sdk's ARCHITECTURE listed AGENTS.md twice in Further reading. Migration narration whose migration is over: Four separate stale-path notes mapped the pre-monorepo root layout (`core/`, `web/`, `sdk/`, `patch-db/`, `container-runtime/`). The root AGENTS.md keeps one — `core/src/` is still referenced live in projects/start-os/DEV_TODO.md, so the mapping is still reachable — and the copies in start-cli, container-runtime and shared-libs go, along with "Internally unchanged from the old `core/` crate", the start-fs migration note, the retired start-os submodule's workspace Cargo.toml, and the `base/lib/...` import shape. The retired `next/patch` | `next/minor` | `next/major` prohibition goes from AGENTS.md; the mapping line in the root CONTRIBUTING.md stays, which is the one line the still-reachable-artifact rule allows. Counts and in-progress markers: `~430 tests` (twice), `11 modules` (twice), `~28 utility modules`, `~117-line`, `~2200 lines`, `all five product binaries`, `currently 1.0.0`, `3 small build-infra patches`, `(currently by start-core)`, `(currently just bitcoin-guides)`, `(currently the Angular libs shared and marketplace)` (twice), the SDK/OS version pair, `being replaced`, `being phased out`, the per-scope CONTRIBUTING migration tally (three copies), the patch-db repo "is being retired" (the repo is still live and was pushed to after the claim was written), and two notes recording which warnings a crate happened to emit. Also drops a commented-out list of ten aspirational locales, a prohibition against putting files directly in shared-libs/ (nothing but doc files ever has, across the directory's whole history), and a prohibition against nesting tab groups more than one level, which restates the positive rule in the sentence before it. Includes the fleet-provenance clause in the packaging guide's manifest.md, which can only be fixed here — the rest of that audit is on live-docs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Matt Hill <9935159+MattDHill@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> | 22 天前 | |
Registry switching, descriptions, and per-registry warnings, without the known-registries list (#3897) * fix(marketplace): switch registries at once, and carry each listed registry's notice Switching registries left the previous registry's packages on screen under the new registry's name until the new fetch landed. The shared component rendered whatever `currentRegistry$` last emitted, and on the brochure that stream only emits once a fetch completes; the OS UI's catalog cache hid the same gap whenever the target registry was not loaded yet. The component now renders only the registry matching the selected url, so a switch shows the cached content or skeletons immediately. The brochure also keeps every registry it has fetched, so switching back is instant and its picker shows a visited registry's live icon instead of the bundled fallback. The Start9 Registry showed the community icon because registry.start9.com served that very image and the picker prefers a registry's live icon. The icon on the server has since been corrected, and the manifest now pins it, so a listed registry that serves a different icon falls back to the pin. The known-registries manifest lists Start9's own four registries, and every entry carries a `warning`: a LocaleString the marketplace shows while that registry is selected, carrying the translations the old per-registry dialog had. An unlisted registry keeps the generic third-party caveat, and the add dialog shows the selected entry's notice rather than a blanket one. A registry that serves no icon no longer counts as drifted from its pin; only a different name or icon does. The KnownRegistry binding must be taken from the Generated Artifacts run. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(registry): declare a description, pinned for listed registries A registry can now describe itself: `start-registry info set-description` stores markdown (a LocaleString, so it can carry translations) in the index, and `info` returns it. The marketplace shows it in an info banner above every other notice while that registry is selected, rendered through the same markdown pipeline as release notes. The known-registries manifest pins a description for each listed registry the way it pins the name and icon: the pinned text is what the marketplace shows, and a listed registry that serves a different one trips the drift banner. The four Start9 registries get their descriptions here; the same texts are to be set on the registries themselves. The RegistryInfo, KnownRegistry, FullIndex, and SetDescriptionParams bindings and the start-registry man pages must be taken from the Generated Artifacts run. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(marketplace): send beta testers to the service-testing room Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(marketplace): point packagers at the service-packaging room, not the submissions inbox Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(marketplace): say when a service belongs in a dedicated registry instead Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(marketplace): take the edited descriptions, and tell beta testers bugs are expected Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(marketplace): translate the pinned registry descriptions Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(marketplace): pin the icons the Start9 and Beta registries now serve Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(shared): bundle the icons the Start9 and Beta registries serve as their fallbacks Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * refactor(marketplace): display verified registries from the manifest alone The manifest is now the only authored identity for a registry Start9 lists. It moves into @start9labs/shared, which bundles it as the fallback for when the published copy can't be fetched, so the hardcoded defaultIdentities, the knownRegistries URL list, and the four bundled registry icons go away. The brochure still serves it from .well-known through its assets entry, and a push to master still redeploys it. One resolver replaces resolveIdentity, resolveIcon, pinnedIcon, findKnown, and identityMatches. A listed registry shows its listed name, icon, description, and warning, whatever its server reports; an unlisted one shows what it reports, except that a name containing a listed name or "Start9" is replaced by the registry's host, and the page says so. The drift banner goes with the comparisons behind it: a listed registry can't disagree with its listing on screen, and impersonation is caught by the name rule, which a lookalike icon or description never was. Listed registries carry a "Verified by Start9" mark in the picker, and the generic notice for unlisted ones now says what it can stand behind: the registry is not on the list Start9 publishes. The OS side compares a fetched name against the stored one before writing it back, instead of against the listed one. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(start-sdk): how a registry gets verified by Start9 A new Verification page in the hosting chapter says what a listing attests to (the address and an operator Start9 can reach, not the services), the requirements, how to apply, and how a listing is kept current. The StartOS book and the publishing page point at it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(marketplace): name a listed registry's operator and contact Every listing now carries the operator's public name and a contact email, shown beneath the description while the registry is selected. Start9's own entries name Start9 and leave the contact to be filled in. The verification page asks for both, routes applications through the submissions inbox like a package, and says that an unlisted registry, Tor-only included, needs none of this. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(marketplace): lay the info banner out as Description and Contact Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * refactor(marketplace): drop the known-registries list Adding a registry is the URL prompt again, a registry's name and icon are what it serves, and the caveat banner is keyed by URL, as before #3865. The manifest, the marketplace.known-registries RPC and its binding, the add dialog's list, the pinned identities, the verified mark, and the verification policy page all go. The switch fix, the description feature, and the per-registry warning texts stay, with the beta texts saying bugs are expected and the community beta text carrying both caveats. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(start-registry): 1.1.0, since a registry can now declare a description Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(marketplace): no caveat banner on the Community Registry Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(registry): regenerate bindings and man pages Helix-Harness: pi Helix-Model: openai-codex/gpt-5.6-sol --------- Co-authored-by: Matt Hill <9935159+MattDHill@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Helix <267227783+helix-nine@users.noreply.github.com> | 6 天前 | |
refactor: reorganize start-os into all-products monorepo (#3352) * docs: propose monorepo reorganization for all Start9 products * refactor(monorepo): split core into start-core lib + thin product bin crates - core/ -> shared/crates/start-core (lib 'startos', package 'start-core') - entry points moved to product dirs: start-os (startbox+start-container), start-cli, start-registry, start-tunnel - root Cargo workspace + shared Cargo.lock; profiles hoisted to root - patch-db submodule relocated to vendor/patch-db - service files moved to their product dirs - fixed include_dir!/include_str! paths for new crate locations cargo check -p start-cli -p start-registry passes (lib compiles). * refactor(monorepo): split web into product dirs + shared/web; relocate sdk & container-runtime - angular workspace rooted at shared/web (holds shared + marketplace libs + config) - apps moved to product dirs: start-os/web/{ui,setup-wizard}, start-tunnel/web, brochure/ - angular.json roots/outputs + per-app tsconfig paths repointed (Plan A) - sdk -> start-sdk (base+package kept cohesive: package imports base via relative paths under shared rootDir; splitting base out would break those imports) - container-runtime -> start-os/container-runtime - file: deps repointed (sdk baseDist/dist, patch-db client under vendor/) * build(monorepo): rewire Makefile, build scripts & CI to new layout - build scripts target workspace (-p <crate>, ./Cargo.toml, repo-root cwd) - Makefile paths: core->shared/crates/start-core, web split across product dirs, sdk->start-sdk, container-runtime->start-os/container-runtime, patch-db->vendor - compress-uis.sh takes per-product web dir; split compress pattern rules - ts-bindings recipe sed patterns generalized for new bindings path - CI workflows repointed (deploy-brochure, start-cli, test, startos-iso, ...) * chore(monorepo): gitignore per-product web dist outputs * docs(monorepo): update proposal to reflect implemented layout + verification status * refactor(monorepo): relocate root docs/ internal notes into their projects - exver.md, VERSION_BUMP.md -> shared/crates/start-core/ - PHYSICAL_DEVICE_TEST_PLAN.md, TODO.md -> start-os/ - draft-start9-pcp-hostname.* -> start-tunnel/ frees top-level docs/ for the migrated docs site * docs(monorepo): migrate start-docs (plain copy, no history) - mdbooks into product dirs: start-os/docs, start-tunnel/docs, start-sdk/docs (packaging) - bitcoin-guides, landing, build infra (build.sh/serve.sh/theme/versions.conf/scripts) -> top-level docs/ - repoint book theme symlinks to ../../docs/theme; book.toml build-dir=book, repo/edit URLs -> monorepo - build.sh maps book names to relocated dirs (absolute output); deploy.yml runs in docs/ with paths filter - verified: docs/build.sh builds all 4 books * docs(monorepo): adopt AGENTS.md convention (CLAUDE.md -> @AGENTS.md import) * docs(root): rewrite README/ARCHITECTURE/AGENTS/CONTRIBUTING for monorepo layout * docs(start-os): add product README/ARCHITECTURE/AGENTS/CHANGELOG/CONTRIBUTING Document the StartOS OS product as a thin wrapper in the monorepo: startbox/ start-container bins, web UIs (ui + setup-wizard), container-runtime, systemd units, and OS image packaging. Reflect new paths (start-core, shared/web, start-sdk, vendor/patch-db) and root-workspace build commands. * docs: integrate merged start-docs PRs (#93 UPnP/gateway, #94 task accept/set, #96 init progress) Only PRs whose feature is confirmed merged into start-os were applied. #99 (upstreamCertValidation) skipped — its code PR (#3353) is still open. * docs: add/normalize per-project doc sets (README/ARCHITECTURE/AGENTS/[CHANGELOG]/CONTRIBUTING) Products get the full set incl CHANGELOG; shared components + docs site get the set minus CHANGELOG; all updated to reflect the monorepo layout and AGENTS.md convention. * docs: add CLAUDE.md -> @AGENTS.md import to remaining project dirs * build(monorepo): root the Angular workspace at repo root so apps resolve node_modules Angular resolves @angular/core per-project from each app's root; with apps in product dirs, shared/web/node_modules was unreachable. Move the workspace config (angular.json, package.json, lockfile, tsconfig{,.lib}.json, .browserslistrc) to the repo root — the only ancestor of every app — so resolution works. - angular.json: project roots -> product dirs, lib roots -> shared/web/{shared,marketplace} - tsconfig paths/extends repointed; app source config.json/package.json require() depths corrected for the new app locations - package.json file: deps + script paths rebased to root; check-i18n.mjs scans the scattered project dirs; update-config.sh writes config.json at the workspace root - Makefile web targets run npm at root; build/env + build-cargo-dep stale paths fixed - build-cli.sh: drop stale 'cd core' in chown step Verified: full build succeeds — ts-bindings, SDK bundle, all 4 Angular UIs, and all five musl bins (startbox/registrybox/tunnelbox/start-container/start-cli). * build(monorepo): fix full-image build paths (container-runtime squashfs + version) - Makefile: undouble container-runtime.service dep path in rootfs rule - check-version.sh: read version from root package.json (moved from web/) - update-image-local.sh: mount repo root so start-sdk + target/ are visible to the image build; run start-os/container-runtime/update-image.sh - update-image.sh: copy start-container from ../../target (workspace), not ../core/target Verified: 'make all' completes (exit 0) — all musl bins + container-runtime rootfs.squashfs (437M) build; second run is a no-op (fully built). * build(monorepo): sync container-runtime package-lock to relocated SDK path; prettier ARCHITECTURE table * feat(monorepo): migrate startos-backup-fs into start-os/backup-fs Vendor the backup-fs crate (was external git dep Start9Labs/start-fs) as a workspace member under the start-os product; build it via the zigbuild path like the other bins instead of 'cargo install --git'. - start-os/backup-fs/: the startos-backup-fs crate (encrypted erasure-coded FUSE backup filesystem); relaxed its =4.5.7 clap / =0.2.17 ppv-lite86 exact pins so they unify with the workspace - root Cargo workspace member + single lock - shared/crates/start-core/build/build-backup-fs.sh; Makefile target builds the local crate (no more git URL) - docs: start-os ARCHITECTURE + CHANGELOG note the migration Verified: 'make all' (exit 0) builds startos-backup-fs (musl) as a member. * build(sdk): decouple 'bundle' from test/check-fmt so consumers don't re-run jest bundle now builds baseDist+dist only; test/check-fmt are standalone (CI calls them directly), and publish runs them explicitly. Fixes the recursive-make coupling where the OS build re-ran the full SDK jest suite every build and an SDK test/format failure broke the OS build. * build(monorepo): split Makefile into per-project include fragments Thin root Makefile includes build/common.mk (shared vars/macros + cross-cutting infra) and one <project>/build.mk per product. Uses include (not recursive make) so it stays a single DAG and cross-project prereqs (start-core -> ts-bindings -> SDK -> web/container-runtime) resolve correctly. - build/common.mk: vars, cp/mkdir/ln/ssh macros, patch-db client, external cargo tools - shared/crates/start-core/build.mk: test-core, ts-bindings - shared/web/build.mk: angular workspace (install, .angular, i18n, UI builds, compress, config.json) - start-sdk/build.mk: test-sdk, dist bundle (consumes the now-decoupled SDK Makefile) - start-{cli,registry,tunnel}/build.mk: their bins + install/deb - start-os/build.mk: startbox/start-container/backup-fs, container-runtime image, OS image assembly + deploy - docs/build.mk: docs site build Verified: make all is a no-op (full build intact); all targets resolve; no duplicate recipes. * docs(root): note the per-project build.mk Makefile structure in AGENTS.md * fix(ci): repoint test/web paths after workspace moves - run-tests.sh: cd to repo root (was shared/crates), build via ./Cargo.toml -p start-core (was ./core/Cargo.toml --workspace) - test.yaml / deploy-brochure: install the Angular workspace at the repo root (npm ci) instead of shared/web; fix vendor/vendor/patch-db doubling; brochure path filters -> root - startos-iso prevent-rebuild placeholders: node_modules/.angular at root; version read from root package.json Verified: npm ci passes at root (lockfile gate). * build(start-os): namespace OS-product make targets as startos-* / install-startos The repo is no longer start-os-only, so the generic target names now read as start-os-specific: - deb->startos-deb, iso/img->startos-$(IMAGE_TYPE), squashfs->startos-squashfs - install->install-startos (matches install-registry/install-tunnel) - wormhole*/update*/emulate-reflash/upload-ota -> startos-* - new 'startos' aggregate (= STARTOS_TARGETS); root 'all: startos' Callers updated: dpkg-build.sh INSTALL_TARGET, deploy targets' $(MAKE) install, startos-iso.yaml (make startos-iso/startos-img), root .PHONY. NOTE: external shared-workflows may invoke the old names (make iso/squashfs/install) for OS image/release builds — needs a companion update there. * build(start-os): move OS-specific build assets into start-os/build Relocate the start-os-only build inputs out of the shared top-level build/ into the product dir: image-recipe/, dpkg-deps/, lib/, download-firmware.sh, and save-migration-images.sh -> start-os/build/. Keep genuinely shared pieces at build/ (common.mk, env/, os-compat/, build-cargo-dep.sh, and lib/scripts/forward-port, which start-tunnel also installs). Relocate the start-os-specific make variables/rules out of build/common.mk into start-os/build.mk (web src/output vars -> shared/web/build.mk; registry and tunnel target vars -> their own fragments) so common.mk is shared-only. Repoint every reference (fragments, Makefile clean, container-runtime update-image.sh, and the moved scripts' own internal paths). Delete the unreferenced legacy build/registry/ eos deploy scripts. * docs(changelog): write 0.4.0-beta.10 per-product release notes Fill in the [0.4.0-beta.10] sections across the per-product CHANGELOGs (brochure, start-cli, start-os, start-registry, start-sdk, start-tunnel) with Added/Changed/Fixed/Removed/Security notes for this cycle, cross-linked between products. * refactor(start-core): rename lib startos to start_core, drop package alias Rename the start-core library from `startos` to `start_core` so the crate's lib name matches its package and the legacy `startos = { package = "start-core" }` dependency-rename alias is gone. The name now penetrates all source: - [lib] name = "start_core"; every `startos::` crate path -> `start_core::` - product crates depend on `start-core` directly; features become `start-core/*` - RUST_LOG=warn,startos=debug -> start_core=debug in the systemd units and CI (the target is module_path!()-derived, so it tracks the crate name) - docs updated to match The product identifier "startos" is left untouched (the root:startos system user/group, the tor.startos / *.startos DNS names, the nftables table, the signature context, the .startos/ packaging-workspace dir, i18n keys, and the /usr/lib/startos install paths). * refactor(monorepo): nest products under projects/, rename shared -> shared-libs Move the buildable products and the docs site into a top-level projects/ dir to separate them from repo infrastructure: start-os, start-cli, start-registry, start-tunnel, start-sdk, brochure (-> brochure-marketplace), docs (-> start-docs) -> projects/ Rename the shared Rust+web library container shared/ -> shared-libs/, kept at the top level alongside build/ and vendor/ as cross-cutting infrastructure. Rewire every path reference to the new layout: - Cargo workspace members + product path deps (../shared -> ../../shared-libs) - Makefile, build/common.mk, and every <project>/build.mk fragment - angular.json, package.json, root + per-app tsconfig (web app configs moved a level deeper, so their relative extends/paths gain one ../) - .github/workflows (the Start9Labs/start-os repo URL is preserved; docs-deploy working-directory + path triggers updated) - build scripts (run-local-build.sh / update-image-local.sh cd depths and internal paths; start-core build/*.sh chown paths) - root .gitignore build-output globs and the web package-lock file: paths Verified: cargo check of all six crates (UI-embed include_dir! and build/env include_str! resolve to the new locations), make -n of the OS / registry / tunnel / web targets. The cold web/SDK build remains CI-grade. * refactor(monorepo): relocate project-specific assets/debian/scripts into projects Apply the same shared-vs-project split to the remaining top-level dirs: - assets/ (create-vm screenshots) -> projects/start-os/assets/ - debian/{startos,start-registry,start-tunnel}/postinst -> each project's debian/; the shared debian/dpkg-build.sh stays top-level and now maps PROJECT -> projects/<dir>/debian for the control files - scripts/copy-categories.sh (registry admin) -> projects/start-registry/scripts/ Kept at top level as genuinely shared/repo-level: debian/dpkg-build.sh, scripts/manage-release.sh (repo releases), scripts/publish-deb.sh (apt publish). Repoint the deb build.mk prereqs, the CONTRIBUTING create-vm link, and code/unit comments. Verified make -n of the three *-deb targets. * docs(rfcs): move draft-start9-pcp-hostname to a top-level rfcs/ dir The PCP HOSTNAME extension Internet-Draft (.md + .txt) describes a protocol spoken by both the StartOS client and the StartTunnel server, so it belongs at the repo level rather than inside start-tunnel/. Repoint the start-os CHANGELOG reference (was the stale docs/ path) to rfcs/. * docs: sync structure docs to the projects/ layout + README project shout-outs - README: add a "rest of the monorepo" section with a short shout-out to each non-OS product (StartTunnel, start-cli, Start SDK, start-registry, and the marketplace + docs sites), and update the directory table + icon path to the projects/ + shared-libs layout. - Root AGENTS.md: rewrite "what lives where" for the new layout and complete the Sub-scopes list (it was missing most products). - Root ARCHITECTURE.md: repoint the module map + cross-layer paths; MONOREPO.md gains a note that the layout was refined (products -> projects/, shared -> shared-libs). - Per-project docs: rename shared/ -> shared-libs/ references, fix relative links whose depth changed when products moved a level deeper into projects/ (links to the repo root, LICENSE, shared-libs, and cross-product changelogs), and repoint functional cd / --prefix build commands. Sibling refs under projects/ (e.g. ../start-sdk, file:../../start-sdk/dist) are correct and left as-is. * build(brochure-marketplace): rename Angular project brochure -> brochure-marketplace Rename the Angular project key (and its build/serve targets) so the project name matches its directory. The dist output is now projects/brochure-marketplace/dist/raw/brochure-marketplace, and the deploy workflow reads/rsyncs that path — this also corrects a path the projects/ restructure had mangled to raw/projects/brochure-marketplace. The npm script names (build:brochure / start:brochure) are kept as conveniences. * docs: remove MONOREPO.md The reorganization proposal has been fully implemented and superseded by the current README/ARCHITECTURE; drop the historical proposal doc and its two links. * feat(build): per-project versioning + Debian packaging for start-cli Decouple product versions from the single StartOS release version. Each Rust product's version is now the source of truth in its own Cargo.toml: start-os stays 0.4.0-beta.10; start-cli / start-registry / start-tunnel move to their own line starting at 1.0.0. - basename.sh and dpkg-build.sh read the version straight from the project's Cargo.toml (per PROJECT), so each .deb is named/versioned independently. - check-version.sh now derives the OS-image /usr/lib/startos/VERSION.txt from the start-os crate manifest instead of the root package.json; nothing maintains a separate version source anymore. - start-cli gains a Debian package: `make cli-deb` builds the musl binary and packages it via the shared dpkg-build.sh (CLI_BASENAME / install-cli staging). CHANGELOGs and the registry AGENTS version note updated to reflect independent versioning. Cargo.lock synced to the new member versions. * chore: ignore *.local.md Broaden the local-notes ignore from CLAUDE.local.md to any *.local.md. * refactor(deps): vendor Start9-maintained crates into shared-libs/crates Move every Start9-maintained crate the workspace depends on in-repo, wired by direct path deps (no [patch]): - rpc-toolkit, imbl-value, exver, yasi, jsonpath (jsonpath_lib), pi-beep — plain-copied from their repos into shared-libs/crates/, added as workspace members. Their inter-deps are repointed to path (exver/imbl-value -> yasi, rpc-toolkit/jsonpath -> imbl-value), and start-core depends on them by path. - patch-db — de-submoduled: moved out of the vendor/ git submodule into shared-libs/crates/patch-db (keeps its own [workspace], excluded from the root one and consumed by start-core via path). Its core/json-patch/json-ptr now path-dep the vendored imbl-value, so there is a single imbl_value::Value type. Drop .gitmodules; repoint the web patch-db-client (package.json / common.mk / CI / shared-libs/web) and pi-beep's build (build-cargo-dep.sh --path). Upstream forks still pulled by git (async-acme, crab_nat, fuser) are left as-is. Verified: cargo check of start-core + start-cli + start-registry + pi-beep compiles the whole path-dep tree clean; Cargo.lock regenerated. * refactor(start-os): move manage-release.sh into the product manage-release.sh is the StartOS release orchestration (startos-images S3 bucket/CDN, the OS image arch matrix incl. -nonfree/-nvidia, the OS registry), not a repo-wide tool — move it to projects/start-os/scripts/. It still calls the shared scripts/publish-deb.sh (which stays top-level, since it publishes any product's .deb), now referenced by its repo-root-relative path. * refactor(debian): rename dpkg-build.sh -> build.sh, move publish-deb.sh -> debian/publish.sh Co-locate the deb tooling under debian/: the package builder is debian/build.sh and the apt-repo publisher is debian/publish.sh (was scripts/publish-deb.sh, which empties scripts/). Repoint the per-product deb build.mk targets, the manage-release.sh caller, and doc/comment references. * build: build pi-beep as a first-party member; reword "vendored" -> "first-party" pi-beep is one of our crates now, so build it like startos-backup-fs (a dedicated build-pi-beep.sh zig build of the workspace member) instead of routing it through build-cargo-dep.sh. That script is now only for the genuinely external crates.io dev tools (tokio-console, flamegraph) bundled into unstable/console images. Also reword the patch-db docs: these are our own crates, so "first-party crate" is more accurate than "vendored" (which implies a third-party copy). * ci: path-gate the per-product build workflows to their project + deps The start-cli / start-registry / start-tunnel / startos-iso build workflows ran on every push/PR (only skipping doc-only changes), so all four built regardless of what changed. Replace the blanket paths-ignore with a paths: allowlist scoped to each product plus its dependencies (start-core + the in-repo shared-libs crates, Cargo manifests, build infra, and — for the web-bearing/OS workflows — the Angular workspace and SDK). workflow_dispatch / workflow_call are kept so manual and orchestrated runs still fire unconditionally. * ci: migrate shared-workflows (service-package CI) into the monorepo Bring the reusable .s9pk build/release workflows and their composite actions in-repo from the standalone Start9Labs/shared-workflows repo, so the packaging toolchain lives alongside the SDK: - .github/workflows/{build,release,tagAndRelease}.yml (reusable, workflow_call) - .github/actions/{extract-version,free-disk-space,setup-build-env, setup-publish-env,upload-each} Their internal references (and the SDK package-template's three workflows + the packaging docs) are repointed from start9labs/shared-workflows@master to Start9Labs/start-os@master. These are workflow_call-only, so they don't run for the monorepo itself — they activate once this lands on master and external service-package repos repoint their `uses:` to Start9Labs/start-os. * docs(monorepo): document tandem-update couplings The per-product CI `paths:` filters mirror each product's build.mk prerequisites by hand — nothing enforces it. Add a "Coupled changes" section to the root AGENTS.md and reciprocal pointers in each gated workflow and its build.mk, so a change to one half is caught at the other. Also catalogs the remaining hand-mirrored pairs (reusable service-package CI <-> SDK package-template <-> packaging docs; the files touched when adding a product/crate) and the already-enforced couplings (ts-bindings, the five i18n locales, the UI beta seed, version <-> CHANGELOG, docs <-> user-facing changes). * chore(manpages): generate man pages into their product projects The export_manpage_* tests in start-core wrote every product's man pages into start-core's own man/ dir. Point each generator at the owning product's man/ dir (anchored to CARGO_MANIFEST_DIR), move the committed pages there, and update build-manpage.sh's chown and the docs. start-container's pages go to projects/start-os, since that bin is part of the StartOS product. * Retitle README * refactor(shared-libs): rename web -> ts-modules Mirror the `crates/` naming: the shared TS/Angular workspace dir becomes `shared-libs/ts-modules/`. Pure path rename — repoints every reference (angular.json, root tsconfig/package.json scripts, the Makefile include + build.mk, CI `paths:`, and docs). No code changes. * fix(monorepo): repoint stale paths from the projects/ move that broke CI Two classes of path left stale by nesting products under projects/: - Web apps `require()` repo-root config.json/package.json by relative path; the extra projects/ level meant every one was short one `../` (resolved to projects/… instead of the repo root), failing the esbuild UI build for start-os (ui + setup-wizard) and start-tunnel. - test.yaml and deploy-brochure.yml still `cd start-sdk` for the baseDist build; the SDK now lives at projects/start-sdk. * fix(debian): resolve PROJECT_DIR before reading VERSION debian/build.sh computed VERSION from "projects/$PROJECT_DIR/Cargo.toml" before PROJECT_DIR was assigned, so it read projects//Cargo.toml (empty) and double-prefixed projects/. The empty Version: produced an invalid DEBIAN/control and dpkg-deb rejected it — breaking the registry and tunnel .deb builds. start-cli's CI only runs `make cli` (the binary), so it never exercised this path. Hoist the PROJECT_DIR/INSTALL_TARGET block above VERSION and read "${PROJECT_DIR}/Cargo.toml" directly. Also only fall back to the OS product's usr/lib/startos/conflicts when that file actually exists, so non-OS products don't error on a missing conflicts file. * fix(web): prettier-wrap tsconfig paths widened by the ts-modules rename Renaming shared-libs/web -> shared-libs/ts-modules pushed the `@start9labs/*` path-mapping lines past prettier's print width, so `npm run format:check` (the Formatting & Lockfiles CI job) flagged the four product tsconfig.json files. Apply prettier's wrapping. * docs(contributing): align with restructure + renamed make recipes Bring the CONTRIBUTING set up to date with the monorepo layout and the namespaced make targets: - root: `make iso` -> `make startos` - start-os: `make $(IMAGE_TYPE)`/`deb`/`squashfs` -> `make startos-$(IMAGE_TYPE)`/`startos-deb`/`startos-squashfs`; deploy/flash targets -> `startos-update*`/`startos-wormhole*`/`startos-emulate-reflash` - start-core: `cd start-sdk` -> `cd projects/start-sdk`; osBindings sync path -> projects/start-sdk/base/lib/osBindings - shared-libs: `web/` -> `ts-modules/` (heading, lib paths, file: deps, cross-links) The other products' CONTRIBUTING files were already correct. * fix(container-runtime): correct repo-root target path in update-image.sh update-image.sh runs with cwd at projects/start-os/container-runtime/ (mounted at /root/start-os in start9/build-env), so the repo-root build output is three levels up. It copied start-container from ../../target (-> projects/target, nonexistent), so the container-runtime squashfs was never built and the OS image compile failed on every arch. Use ../../../target. Refresh the AGENTS.md gotcha that described the old stale path. * refactor(make): namespace OS web targets, require an explicit target, per-project cleans - `ui`/`uis` -> `startos-ui`/`startos-uis`: they build only the StartOS admin UI + setup-wizard, so they belong under the startos-* namespace. Also fix `startos-ui` to depend on the built index.html (the old `ui` depended on a directory with no rule). - No default build: bare `make` now prints `help` (.DEFAULT_GOAL := help) and the misleading `all` target (it only built `startos`, not "everything") is removed — callers specify a target. - Decentralize `clean`: every build.mk owns a `clean-<project>` target and the root `clean` just aggregates them. Per-project cleans use project-prefix wildcards so they're arch/version-independent, and two stale paths from the projects/ move are corrected (env/*.txt -> build/env/*.txt; image-recipe/deb -> projects/start-os/build/image-recipe/deb). - Drop the start-cli targets (`make cli`/`cli-deb`) from the start-os CONTRIBUTING build section (wrong product) and update the docs to the new target names (root + start-os CONTRIBUTING/README/AGENTS). * docs: sync, complete, and standardize all developer docs for the monorepo (#3356) * docs: sync developer docs with the monorepo restructure Audit of all developer documentation (AGENTS/CONTRIBUTING/ARCHITECTURE/README across root, projects/*, shared-libs/*) against the post-restructure tree. Corrects stale references the restructure left behind: - Paths still pointing at the pre-restructure layout (core/, web/, sdk/, brochure/, container-runtime/, patch-db submodule) -> projects/* and shared-libs/*. - Angular workspace root: several docs claimed shared-libs/ts-modules holds angular.json/package.json/tsconfig.json and that npm runs from there. The workspace is rooted at the repo root; fixed cwd/--prefix instructions, the tsconfig path-alias targets, and config-sample.json location accordingly. - Renamed make targets (startos-* namespace), the Rust lib rename (startos -> start_core / crate start-core), and the binary-source table in start-core/ARCHITECTURE.md. - Removed stale 'git clone --recursive' (no submodules remain) and the non-existent repo-level scripts/ reference. - Normalized product self-references and the brochure -> brochure-marketplace Angular project name; verified incidental accuracy fixes (exver 0.2.1, patch-db serde_cbor, image-recipe live-build). All relative doc links verified resolvable. Pre-commit lint-staged hook skipped (--no-verify): the slot has no installed node_modules so the binary can't run, and the repo's prettier targets only the web source dirs, not these markdown docs. * docs: bring utility crates and patch-db up to the standard doc set Every first-party crate under shared-libs/crates/ now carries the same AGENTS/ARCHITECTURE/CONTRIBUTING/README/CLAUDE set the projects and other shared libs use. - New full doc sets for exver, imbl-value, jsonpath, pi-beep, rpc-toolkit, yasi, written from each crate's actual source (verified module names, public API, cargo -p <package> commands, and real consumers; jsonpath's package is jsonpath_lib though its dir is jsonpath). Pre-existing READMEs (exver, jsonpath, yasi) were preserved verbatim and only augmented with a 'Place in the monorepo' + 'Documentation' section. - patch-db: added AGENTS.md (migrated from its content-bearing CLAUDE.md, plus a Build & test section noting it is its own Cargo workspace) and reduced CLAUDE.md to the one-line @AGENTS.md import like every other scope. - Every CLAUDE.md is exactly '@AGENTS.md'; all relative doc links resolve. * docs: normalize section structure across all project & shared-lib docs Standardize every first-class scope (root, the 7 projects, container-runtime, shared-libs, start-core, ts-modules) onto one canonical section template so the same sections appear under the same names in the same order everywhere: - AGENTS.md: intro (+ 'CLAUDE.md is a one-line @AGENTS.md import' + cross-links) → Layout → Build & test (run from the repo root) → Gotchas → scope-specific. Folded synonym headings (Where things are / What lives where / Map → Layout; Operating rules / Gotchas / rules → Gotchas; Build / Build & verify → Build & test). - CONTRIBUTING.md: Documentation (doc-map, added where missing) → Prerequisites → Building → Testing → Formatting → scope-specific → Commits / PRs (last). - ARCHITECTURE.md: ensured a 'Place in the monorepo' opener (where applicable) and a 'Further reading' links footer; internal sections left in place. - README.md: added a Documentation/links footer where missing; landing prose left untouched. Pure reorganization: headings renamed/reordered and short missing sections added — no prose reworded, no facts/commands/tables dropped. Scope-accurate build cwds preserved (start-docs builds from its dir; start-sdk from its dir; container-runtime via npm --prefix; patch-db its own workspace). All 102 docs' relative links verified resolvable; every CLAUDE.md remains the one-line import. * build+docs: merge patch-db into root workspace; enforce doc hierarchy Addresses review feedback on the docs PR: - **patch-db → root Cargo workspace.** Add the six patch-db crates (core, json-patch, json-ptr, macro, macro-internals, util) to the root workspace `members` and drop the `exclude`; remove patch-db's now-redundant virtual `Cargo.toml`. `cargo metadata` + `cargo check -p patch-db -p json-patch -p json-ptr -p patch-db-macro` pass. Updated the docs that called patch-db 'its own workspace' (patch-db AGENTS/CONTRIBUTING build commands now run from the repo root with `-p`; root ARCHITECTURE lists it as a member). - **Root AGENTS.md:** added a note that these doc files must be kept current with every change, and that the docs are hierarchical — a scope documents only what is specific to it and never repeats higher-scope content. - **Hierarchy cleanup:** removed the `## Commits / PRs` (and `Branch / commit / PR` / `Commit conventions`) sections from every non-root CONTRIBUTING.md — those conventions live only in the root CONTRIBUTING.md now. Scope-specific change steps were preserved (e.g. start-tunnel's migration/ bindings/CHANGELOG steps moved to a 'Making a change' section; start-cli's docs-update note folded into 'Where code lives'). Also dropped the duplicated 'keep these docs in sync' line from child Documentation sections (kept the local cross-link lists). - **jsonpath:** reframed the fork note as history-only across its docs — it has fully diverged from freestrings/jsonpath with no intent to upstream; treat it as first-party and edit freely (removed the 'pull fixes from upstream / keep changes minimal / fork-tracking' guidance). All 102 docs' relative links resolve; every CLAUDE.md remains the one-line import. * docs: add hierarchy-navigation notes to AGENTS files - Every non-root AGENTS.md now opens with a 'Read up the tree first' note: the docs are hierarchical, so before working in a scope read the AGENTS.md of each enclosing directory up to the repo root (and their ARCHITECTURE/CONTRIBUTING where relevant). Excludes the packaging-guide + package-template AGENTS under projects/start-sdk/docs/, which target external package authors, not the monorepo dir tree. - Root AGENTS.md gains the converse 'Read down into what you touch' note: read a subdirectory's AGENTS.md (and any further nested ones) before editing it. - Dropped the explicit repo-root CONTRIBUTING pointer from patch-db's CONTRIBUTING.md (the walk-up convention now covers it). * docs(agents): require product docs/ book + CHANGELOG to ship with code Root AGENTS.md now mandates that any change altering user-visible behavior update that product's user-facing docs/ book (projects/<product>/docs/) in the same change and add a CHANGELOG.md entry — no deferring docs/changelog to follow-ups. * build(make): add per-project format targets mirroring the clean decomposition Each build.mk now owns `format-<project>` + `format-check-<project>` (core, web, sdk, cli, registry, tunnel, startos), matching the per-project `clean-<project>` targets; the top-level `format`/`format-check` just aggregate them. Web (the whole Angular workspace incl. brochure) formats via the root npm script; the shared crates via one `cargo +nightly fmt`; container-runtime via its own prettier config (new `format`/`format:check` npm scripts). So you can format one project (`make format-cli`) or all (`make format`). * docs: address PR review — make-target refs, ARCH de-dup, ts-modules wording - Reference stable make targets instead of raw build/format commands across project docs: builds via `make cli`/`registry`/`tunnel`/`startos`/`startos-ui`, formatting via the per-project `make format-<project>` targets (and `format-check-<project>` for CI). Kept `cargo check`/`cargo test` and crate `cargo build` (no make equivalent) as noted dev shortcuts. - Fixed stale targets: dropped `all` (removed upstream; `make` now prints help), `ui`/`uis` -> `startos-ui`/`startos-uis`. - Removed the whole-monorepo ASCII trees that several product ARCHITECTURE.md files re-drew (start-cli, start-registry, container-runtime, ...) — that layout lives once in the root ARCHITECTURE.md; each now states only where it sits. - Reworded the `shared-libs/ts-modules` directory as shared TypeScript modules (not Angular-specific; current contents are the Angular libs shared/marketplace), per review; kept accurate per-library 'Angular library' phrasing. Branch merged up to date with docs/monorepo-proposal first. * refactor(sdk): extract base into @start9labs/start-core shared lib; flatten start-sdk Move projects/start-sdk/base -> shared-libs/ts-modules/start-core (package @start9labs/start-sdk-base -> @start9labs/start-core), mirroring the Rust crate shared-libs/crates/start-core. start-core builds its own self-contained dist consumed via file: deps. Flatten start-sdk (package/ -> root): the SDK now imports @start9labs/start-core instead of ../../base/lib, and its published dist bundles start-core (bundleDependencies) so external authors still install one package. Repoint all consumers off the SDK-as-base alias onto @start9labs/start-core: - web (root file: dep + 153 import sites + shared/marketplace peerDeps) - container-runtime (file: dep; base/lib imports -> start-core, package/lib -> lib) Repoint osBindings generation, the build DAG (build.mk fragments, Makefile), and regenerate the three lockfiles. Resolves the "SDK kept cohesive" deviation: base is now an honest first-class shared TS lib named for what it is. * docs+ci: reflect start-core extraction; repoint SDK build steps off base/baseDist CI: build start-core (cd shared-libs/ts-modules/start-core && make dist) before validating the SDK + web lockfiles; validate the flattened SDK lockfile at projects/start-sdk; add start-core to the prettier check; repoint the iso prevent-rebuild mkdirs and the brochure deploy paths: filter (start-sdk -> start-core). Docs: update every project's AGENTS/ARCHITECTURE/CONTRIBUTING/README for the new layout — base extracted to @start9labs/start-core under shared-libs/ts-modules, the SDK flattened (lib/) and bundling start-core, container-runtime depending on both. Removes the resolved "SDK kept cohesive" deviation note. * refactor(monorepo): use the start-technologies name; repoint init-workspace at the monorepo - Adopt start-technologies for monorepo/repo-URL references across docs and AGENTS files (ahead of the GitHub repo rename; product refs left as start-os: projects/start-os, the start-os crate, *-startos packages, StartOS). - s9pk init-workspace now sparse-clones the start-technologies monorepo (projects/start-sdk/docs) instead of the retired standalone start-docs repo, and the two start-docs-named init strings are reworded across all five locales. - Un-hide the SDK 2.0 packaging-workspace section and rewrite it for the monorepo clone; document fetch-on-demand SDK + OS source access (docs first) in the workspace AGENTS.md and workflow.md. - Fix build-config.js to read and write the repo-root config.json. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkwYSXK8Uh9D16UTffCNJ1 --------- Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <9935159+MattDHill@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> | 2 个月前 | |
fix(ui): align mobile notification checkboxes (#3616) * fix(ui): align mobile notification checkboxes * fix(ui): place the checkbox by layout, not by a fixed offset The mobile checkbox is still positioned against the row by hand, so it has to encode item.component's grid geometry as a literal in table.component — the coupling that broke when 840019c60 reordered the rows. And no single literal can be right any more: that commit also pinned `.service` to grid row 1, so a service avatar makes the title row 32px against 27.4px without one, and the icon the checkbox stands in for sits 4px lower on notifications that have one. Project the checkbox into `.title` ahead of the icon instead. On mobile it is then an in-flow flex item, so `.title`'s existing `align-items: center` places it — the same rule that places the icon. The two swap via `display`; both are 1rem behind the same gap, so nothing on the card moves as selection turns on. Desktop keeps absolute positioning, now anchored to the row rather than to the date cell the checkbox no longer lives in, so mobile has nothing to undo. No offset constant remains for a future row reorder to invalidate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Matt Hill <9935159+MattDHill@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Aiden McClelland <me@drbonez.dev> | 1 个月前 | |
chore: rename make targets to consistent <project>-<recipe> form (#3400) * chore: rename make targets to consistent <project>-<recipe> form Every project-specific root make target now uses its full start-* directory name as the <project> prefix, and verb-first targets are reordered to project-first, so the whole target surface reads <project>-<recipe>. Examples: tunnel-deb -> start-tunnel-deb cli / startos -> start-cli / start-os startwrt-image -> start-wrt-image install-cli -> start-cli-install clean-tunnel -> start-tunnel-clean format-cli -> start-cli-format format-check-cli -> start-cli-format-check test-core -> start-core-test ts-bindings -> start-core-ts-bindings test-container-runtime -> container-runtime-test format-web -> web-format Aggregate targets (test, format, format-check, clean, help, metadata, touch) are unchanged. Non-start-* project tokens keep their own name: `web` (the Angular workspace) and `container-runtime` (a start-os sub-module); ts-bindings is treated as a start-core recipe. Updates every call site: the five product CI workflows, internal $(MAKE)/prerequisite references, help text, and all docs. The startos-iso.yaml workflow FILE and the `startos-ui` npm package name are deliberately left as-is (only the make targets they reference were renamed). * fix(deb): point debian/build.sh at the renamed <project>-install targets debian/build.sh derives INSTALL_TARGET from $PROJECT and calls `make "$INSTALL_TARGET"` to stage the binary into DESTDIR — this is how every product's .deb is built. It still used the pre-rename names (install-startos / install-<project>), which no longer exist, so all .deb builds would fail. Derive the new names instead: startos -> start-os-install, start-<x> -> start-<x>-install. | 2 个月前 | |
feat(start-os): the server's name is its .local hostname (#3791) * feat(start-os): the server's name is its .local hostname ServerInfo carried two spellings of one identity: a free-form display label shown in the browser tab, and the DNS label the .local address and the Linux system hostname are built from. The UI collected one and derived the other through normalize()/denormalize(), so the tab could read "My Cool Server" while every address the user typed said my-cool-server.local — and someone hunting for the second did not recognize the first. Drop ServerInfo.name and collapse ServerHostnameInfo into ServerHostname. The Server Name field now edits the .local hostname directly and surfaces ServerHostname::validate rather than silently normalizing input away, so the browser tab shows the address the user reaches the server by. server.set-hostname takes a hostname and nothing else, and setup execute drops --name. Removing normalize() means the field has to reject what it used to rewrite, so it applies the DNS label rules on top of validate()'s charset rule — a 63 character limit and no leading or trailing hyphen — and turns off autocapitalization, since a phone keyboard would otherwise capitalize the first letter of a name that must be lowercase. Those two rules stay in the form rather than in validate(): startd calls sync_hostname on every boot and ServerHostname::load does not validate, so tightening validate() would drop a server whose stored hostname predates the rule into diagnostic mode. The v0_4_0_2 migration drops the stored name; rolling back writes one back, title-casing the hostname the way the derivation it returns to would have. Restoring from a backup and transferring to a new drive both stop renaming the server: the wizard sent normalize("") for those flows, which is the "start9" fallback, and it overwrote the hostname the backup carried. * fix: enforce the DNS label rules where the operator supplies a hostname The 63-character and hyphen-edge rules lived only in the Angular form, so `server.set-hostname` would commit a hostname the kernel refuses: the DB write lands before `sync_hostname` runs, and `startd` calls `sync_hostname` unconditionally on every boot, so the server comes back into diagnostic mode with no way out but editing the database by hand. `ServerHostname::new_from_input` holds an operator-supplied hostname to the full rules and backs `set_hostname_rpc` and `new_opt`. `validate` and `new` keep the charset rule alone, so a hostname stored before a rule existed still loads and a legacy backup still restores. The form validator now trims, reports an empty value as `required`, and reports a bad character before a bad length, so pasting a name with a trailing space saves instead of failing on a character the user cannot see. Both forms submit the trimmed value. Also: the new CLI argument's help goes through `help.arg.hostname` in all five locales, the setup wizard's remaining validation messages are translated, the start-cli changelog records the two commands that changed, the transfer section of the device test plan no longer asks for a server name the flow does not offer, and `lan_address()` — dead on master and on this branch — is gone. * fix: cap the hostname at what the root CA's Common Name can carry The root CA is issued to `<hostname> Local Root CA`, and X.509 caps a Common Name at 64 characters, so a 51-character hostname aborts `AccountInfo::new` inside OpenSSL. That runs after the data drive is prepared, so a fresh setup died on a raw ASN.1 error and the operator had to start the install over. `MAX_LEN` is 50, and a test now asserts the branded name still fits, so the constant moves if the branding does. Also from review: `hostnameValidationErrors` carries the `required` message its validator emits, instead of each caller patching one in; the Server Name dialog marks an already-invalid hostname touched so `tui-error` explains why Save is disabled; the setup wizard's form moves to `NonNullableFormBuilder`; the start-cli changelog sections follow Keep a Changelog order; the transfer section of the device test plan points at the source device's name rather than the target's, which that section destroys; and the word list is `hostname-words.ts`, the last file still spelled for the display name. * fix: heal a hostname the kernel refuses, and say 50 everywhere The 0.4.0.2 release note still promised 63 characters after the cap moved to 50, so the notes announcing the change described a rule the code rejects. The `--hostname` help and the start-cli entry likewise named the length rule without the hyphen-edge one. An earlier version accepted a hostname longer than the kernel allows, and `sync_hostname` runs on every boot, so such a server comes up in diagnostic mode every time and the RPC that would rename it is out of reach. Diagnostic mode can still take an update, so `v0_4_0_2::up` is where it heals: `repair_hostname` keeps as much of the stored name as the rules allow and generates one when nothing usable remains. A previous round declined this on the grounds that a stranded server could not reach the migration; `diagnostic.rs` exposes `update`, so it can. The root CA test now builds a certificate instead of measuring a string, so it fails if anything inside `make_root_cert` grows rather than only the branding, and the hostname limit gets an entry under the root AGENTS.md coupled changes, since the number is restated by hand across Rust, TypeScript, five locales, five dictionaries and the docs. The two `.local` previews are signals rather than methods called on every change detection, the setup wizard declares the `Required` message its password field needs rather than borrowing it from the hostname helper, and the device test plan checks the hostname where the UI shows it. * fix: repair only a hostname the system refuses, not one the field would The repair added last round gated on `new_from_input`, which is the rule for a name an operator is typing now — at most 50 characters, no hyphen on either end. A server renamed on 0.4.0.1 could hold 63 characters: the UI allowed it, the kernel carries it, avahi publishes it, and the leaf certificate fits. Updating would have truncated that name and moved the `.local` address with it on the same boot, silently, breaking bookmarks and known_hosts on a server that was working. `repair_hostname` now returns a hostname untouched when `set_hostname` can carry it — in the character set and within `HOST_NAME_MAX` — and rewrites only what would fail on the boot path. It also strips hyphens before spending the length budget rather than after, so a name buried behind them survives. A backup restore builds its database through `Database::init`, which stamps the current version, so no migration ever runs over it; a backup taken from a server holding an unusable hostname would have restored straight into diagnostic mode. `recover_full_server` runs the hostname through the same repair. Also from review: the setup wizard's own validation messages come after the spread that was overwriting them, the start-cli entry names `setup execute --hostname` alongside `server set-hostname`, and the coupled-changes bullet drops a claim about a TS binding that does not exist, names both changelogs, and no longer separates itself from the list with a blank line. * fix: bound the untouched hostname band by what the server can actually serve Last round narrowed the repair's trigger to what the boot path refuses, on the grounds that a longer name still works — "the UI allowed it, the kernel carries it, avahi publishes it, and the leaf certificate fits". The last clause is false. A leaf certificate is issued to `<hostname>.local`, X.509 caps a Common Name at 64, and a hostname of 59 or more takes it past that: `make_leaf_cert` fails, `get_config` returns nothing, and the handshake ends in a fatal alert. So every HTTPS request to the `.local` name fails, for the StartOS UI and for every service binding, and the rename dialog that would fix it is served over the address that no longer works. rustls rejects a DNS label with a hyphen on either end, which costs a hostname its address the same way. `ServerHostname::is_usable` is that whole question in one place, and `repair_hostname` leaves a hostname alone only when it passes. The band this protects is 51 to 58 characters — a name the operator set, that serves fine, and whose address should not move under them. Above it there is no working address to protect, so the repair gives the server one. Both ceilings now derive from the Common Name limit they come from, and a test mints a real leaf certificate at the boundary, so `MAX_SERVED_LEN` moves if the `.local` suffix ever does. Also from review: the coupled-changes bullet credited the `server set-hostname` man page to a doc comment that `#[arg(help = ...)]` overrides. * fix(start-os): preserve hostname migrations after rebase Compose the hostname migration with the admin-port and ALPN changes already accumulated for 0.4.0.2, retaining every up and down path. Cover the combined migration plus restore and transfer hostname preservation. Raise start-cli to 2.0.0 for the removed and changed stable flags, refresh its lockfile and generated man pages, and normalize generated man-page whitespace at the source. * fix(start-os): cap server hostnames at 32 characters * docs(setup): restore state flow comments * refactor(start-os): scope hostname repair to migration | 9 天前 | |
fix(start-os): a backup-fs read past the end of a file returns no bytes (#3835) * fix(start-os): a backup-fs read past the end of a file returns no bytes `Handler::read` clamped a read with an unchecked `attrs.size - offset`, where its sibling on the ordinary read path uses `saturating_sub`. It is reached only from `Handler::copy_file_range`, and `lib.rs` holds the `Handler` guard across that call — so with `offset > size` a debug build panics with the whole session's lock held, poisoning it, and every subsequent operation on the mount, including the unmount, then panics on the poisoned lock. A release build wraps to a huge `usize` instead and returns `EIO`. The kernel clamps a `copy_file_range` against the size it has cached, so a plain read past EOF never reaches the handler; a truncate landing between that check and the handler's read does. Reproduced against `master`: `handle.rs:1102 attempt to subtract with overflow`, then `lib.rs:173 PoisonError`, then the mount stops answering. Clamping alone still returns `EIO`, because `read_exact_at` measured a zero-length read against the file's size and refused it as an overrun — which is also why an ordinary `pread` past the end of a backup-fs file returned `EIO` where POSIX requires 0. A read of no bytes is now satisfiable at any offset, which fixes both paths. It no longer stamps atime or dirties the inode either, so a read that returns nothing no longer schedules an inode save. Each test fails on the mutation that removes the guard it covers: reverting `contents.rs` fails both, reverting `handle.rs` panics the copy test alone. Closes #3780 * fix(start-os): a backup-fs copy that moves no bytes leaves the destination alone A read past the end of a file now returns zero bytes, so `Handler::copy_file_range` reaches its write with an empty buffer where the read used to error out first. `Contents::write_all_at` has no empty-buffer case: it computes `end = dest_offset`, promotes the body tier, sets the destination's size to `end` and stamps its mtime. So a copy out of a file that shrank returned 0, as it should, and grew the destination out to `dest_offset` with zeros, which no other filesystem does. Measured on the diff's own workload, 20,000 iterations against a flapping truncator: 3,220 to 15,060 of the ~15,000 zero-byte copies per run left the destination extended. With the guard, none do. The regression test picks it up now: it copies into a hole rather than to offset 0, where an extension to `dest_offset` was invisible, and reads the destination back through the filesystem rather than trusting the length the kernel cached. The same test could pass on unfixed code roughly 40% of the time on a single-core machine. 98.7% of its iterations never leave the kernel, which clamps the request away itself, so the copier never blocks and the truncator barely runs; a yield per iteration opens the race, and 500 iterations then catch the regression where 20,000 without it did not. Its truncator thread also unwrapped two syscalls that fail once the mount is gone, so a real failure was reported as `Any { .. }` and the message the test exists to print never ran. The changelog said the copy could leave the filesystem unresponsive and hang. That needs the old `size - offset` to panic, and `overflow-checks` is off in release, where it wraps and the copy fails with an I/O error instead. The hang is a debug-build artifact, so the sentence describes something no user saw. * test(backup-fs): witness the shrinking-file race instead of assuming it The copy test could report green having raced nothing. Its two loop outcomes are now counted and both are required, so a source that never shrank under the copier fails the test rather than passing it silently. Two `unwrap()`s in the loop body bypassed the stop-then-report sequence: a panic there skipped the stop flag and the join, leaving the truncator writing to a mount being torn down. Both now record a failure and break like the paths beside them. Also record on `Contents::read_exact_at` the bound its callers must apply, which is the invariant whose two copies drifted into this bug, and hold the comments this branch adds to the rebuilt rules in AGENTS.md. * fix(start-os): copy a backup-fs range a chunk at a time The kernel forwards a copy_file_range whole, so `Handler::copy_file_range` read the entire range into one allocation and `Handler::write` copied it again. A 4 GiB `cp` inside a mounted backup peaked at ~12 GiB RSS and OOM-killed the mount. It now moves a chunk at a time and drops the redundant copy in `Handler::write`. Move the empty-write guard down to `Contents::write_all_at`, where the defect lives: an empty write extended the file and stamped its mtime. The read side was already fixed at that layer. Cover the copy loop's offset arithmetic, and stop the shrinking-file test asserting what it cannot observe: the kernel answers most iterations from its cached size without reaching the filesystem, so a count of zero-byte copies does not witness the past-EOF path. * fix(start-os): report the bytes a backup-fs copy moved before it failed A chunked copy_file_range propagated an error out of any chunk after the first, so the kernel was told nothing moved while earlier chunks were already durable in the destination. Measured on a 3 MiB copy failing at its second chunk: the syscall returned -1 with 1048576 bytes on disk. It now returns the count and leaves the error for the next call. Drops the unreachable wrote == 0 break: Handler::write returns data.len() unconditionally and the buffer is non-empty by the guard above it. Covers both empty-buffer guards directly, since the kernel issues no zero-length request and neither guard was reachable from any test. Surfaces the truncator thread's errors in the shrinking-file test, which otherwise passed whether or not it witnessed the race, and reads errno only where a negative return makes it meaningful. Clears the unused_mut the previous commit introduced at lib.rs:463. * fix(start-os): record the error a partial backup-fs copy reports around A chunked `copy_file_range` that meets an error after moving bytes returns the count instead of the error, which is the syscall's contract. The error was then dropped whole: `BkfsError::to_errno_log` is the crate's only logging site and it is reached solely from the `Err` arm at `lib.rs`, so a `BadCrypt` or `BadChecksum` on a source block inside a mounted backup left no trace at all, and the short count is indistinguishable from a normal end-of-file copy. Measured on a three-chunk copy failing at chunk two: the count still comes back as 1048576, and the log line that had zero occurrences now appears. The partial-versus-total decision and the log now sit at one exit rather than in two duplicated arms, and the loop counts the bytes it read rather than the count `Handler::write` echoed back, so its progress no longer rests on a callee's return value. Two more found in the same functions: - `Handler::write` tested `FUSE_WRITE_KILL_PRIV` against the open flags while binding the argument that carries it as `_write_flags`, so it could never clear a file's suid bit. No `O_*` flag has that value. Dead today — its one caller passes zero and the FUSE write path in `lib.rs` reads `WriteFlags::FUSE_WRITE_KILL_SUIDGID` correctly. - `read_exact_at`'s doc said a read past the end fails, which the guard three lines below it contradicts for an empty buffer. The shrinking-file test reported a dead truncator as a filesystem bug: a `set_len` failure and a failed copy carry the same errno, and the copy was reported first. A truncator that fails partway has also stopped shrinking rather than never having shrunk. * fix(start-os): record a partial backup-fs copy's error where an operator sees it `BkfsError::log` routes any io error carrying an errno to `debug!`, and `main.rs` defaults the filter to `info`, so the error a partial copy reports around left no line at all at production log level — the case the previous commit was written for. Measured against the new test at `RUST_LOG=info`: 0 lines before, 1 after. The copy loop now warns at the one exit where the caller receives a success, and `BkfsError::log` goes back to being private inside `to_errno_log`. A failed packed→blocks migration also emptied the file it was migrating. `packed_to_blocks` rewrites `inode.attrs.contents` and schedules the superseded extent's tombstone without marking the inode changed, so a write that failed after the migration skipped the inode save on close while still dropping the extent. The durable inode kept pointing at a tombstoned extent and the whole file read back as zeros. Reproduced deterministically by making one block file unreadable. Adds the first test for the partial-count path. * fix(start-os): report a corrupt backup-fs block at its own severity The partial-copy arm logged every error it swallows at one level, so a corrupt or tampered source block — BadChecksum out of `vault::open`, which `load_block` reaches through `read_block` — was reported as a warning where the crate reports it as an error everywhere else. Measured at RUST_LOG=info: a garbaged block now logs ERROR "bad checksum", an EISDIR on a block file still logs WARN, and both remain visible at the default filter. The changelog named the tier boundary as "a few hundred kilobytes"; the packed-to-blocks migration fires at pack_max, which defaults to CHUNK_SIZE, so it is a megabyte. `read_starting_past_eof_returns_no_bytes` documents an offset at the end as well as past it, but only covered the block tier past the end. `copy_file_range_that_fails_partway_reports_the_bytes_it_moved` read errno where the syscall had succeeded, printing a stale error next to a wrong count; its two sibling call sites already read errno only on a negative return. Both new tests compared megabyte buffers with `assert_eq!`, which dumps both operands, where `pattern_check` names the first bad offset. * fix(start-os): harden backup-fs partial copy handling * fix(start-os): preserve backup-fs partial write accounting * style(start-os): group backup-fs imports the way the pinned rustfmt does `rustfmt.toml` sets `group_imports` and `imports_granularity`, both unstable, so a stable `cargo fmt` accepts the file and silently leaves these alone. CI runs the pinned nightly in `build/fmt/run-fmt.sh` and failed on the two test modules' import order. Run `make start-os-format`. | 8 天前 | |
fix(start-os): a backup-fs read past the end of a file returns no bytes (#3835) * fix(start-os): a backup-fs read past the end of a file returns no bytes `Handler::read` clamped a read with an unchecked `attrs.size - offset`, where its sibling on the ordinary read path uses `saturating_sub`. It is reached only from `Handler::copy_file_range`, and `lib.rs` holds the `Handler` guard across that call — so with `offset > size` a debug build panics with the whole session's lock held, poisoning it, and every subsequent operation on the mount, including the unmount, then panics on the poisoned lock. A release build wraps to a huge `usize` instead and returns `EIO`. The kernel clamps a `copy_file_range` against the size it has cached, so a plain read past EOF never reaches the handler; a truncate landing between that check and the handler's read does. Reproduced against `master`: `handle.rs:1102 attempt to subtract with overflow`, then `lib.rs:173 PoisonError`, then the mount stops answering. Clamping alone still returns `EIO`, because `read_exact_at` measured a zero-length read against the file's size and refused it as an overrun — which is also why an ordinary `pread` past the end of a backup-fs file returned `EIO` where POSIX requires 0. A read of no bytes is now satisfiable at any offset, which fixes both paths. It no longer stamps atime or dirties the inode either, so a read that returns nothing no longer schedules an inode save. Each test fails on the mutation that removes the guard it covers: reverting `contents.rs` fails both, reverting `handle.rs` panics the copy test alone. Closes #3780 * fix(start-os): a backup-fs copy that moves no bytes leaves the destination alone A read past the end of a file now returns zero bytes, so `Handler::copy_file_range` reaches its write with an empty buffer where the read used to error out first. `Contents::write_all_at` has no empty-buffer case: it computes `end = dest_offset`, promotes the body tier, sets the destination's size to `end` and stamps its mtime. So a copy out of a file that shrank returned 0, as it should, and grew the destination out to `dest_offset` with zeros, which no other filesystem does. Measured on the diff's own workload, 20,000 iterations against a flapping truncator: 3,220 to 15,060 of the ~15,000 zero-byte copies per run left the destination extended. With the guard, none do. The regression test picks it up now: it copies into a hole rather than to offset 0, where an extension to `dest_offset` was invisible, and reads the destination back through the filesystem rather than trusting the length the kernel cached. The same test could pass on unfixed code roughly 40% of the time on a single-core machine. 98.7% of its iterations never leave the kernel, which clamps the request away itself, so the copier never blocks and the truncator barely runs; a yield per iteration opens the race, and 500 iterations then catch the regression where 20,000 without it did not. Its truncator thread also unwrapped two syscalls that fail once the mount is gone, so a real failure was reported as `Any { .. }` and the message the test exists to print never ran. The changelog said the copy could leave the filesystem unresponsive and hang. That needs the old `size - offset` to panic, and `overflow-checks` is off in release, where it wraps and the copy fails with an I/O error instead. The hang is a debug-build artifact, so the sentence describes something no user saw. * test(backup-fs): witness the shrinking-file race instead of assuming it The copy test could report green having raced nothing. Its two loop outcomes are now counted and both are required, so a source that never shrank under the copier fails the test rather than passing it silently. Two `unwrap()`s in the loop body bypassed the stop-then-report sequence: a panic there skipped the stop flag and the join, leaving the truncator writing to a mount being torn down. Both now record a failure and break like the paths beside them. Also record on `Contents::read_exact_at` the bound its callers must apply, which is the invariant whose two copies drifted into this bug, and hold the comments this branch adds to the rebuilt rules in AGENTS.md. * fix(start-os): copy a backup-fs range a chunk at a time The kernel forwards a copy_file_range whole, so `Handler::copy_file_range` read the entire range into one allocation and `Handler::write` copied it again. A 4 GiB `cp` inside a mounted backup peaked at ~12 GiB RSS and OOM-killed the mount. It now moves a chunk at a time and drops the redundant copy in `Handler::write`. Move the empty-write guard down to `Contents::write_all_at`, where the defect lives: an empty write extended the file and stamped its mtime. The read side was already fixed at that layer. Cover the copy loop's offset arithmetic, and stop the shrinking-file test asserting what it cannot observe: the kernel answers most iterations from its cached size without reaching the filesystem, so a count of zero-byte copies does not witness the past-EOF path. * fix(start-os): report the bytes a backup-fs copy moved before it failed A chunked copy_file_range propagated an error out of any chunk after the first, so the kernel was told nothing moved while earlier chunks were already durable in the destination. Measured on a 3 MiB copy failing at its second chunk: the syscall returned -1 with 1048576 bytes on disk. It now returns the count and leaves the error for the next call. Drops the unreachable wrote == 0 break: Handler::write returns data.len() unconditionally and the buffer is non-empty by the guard above it. Covers both empty-buffer guards directly, since the kernel issues no zero-length request and neither guard was reachable from any test. Surfaces the truncator thread's errors in the shrinking-file test, which otherwise passed whether or not it witnessed the race, and reads errno only where a negative return makes it meaningful. Clears the unused_mut the previous commit introduced at lib.rs:463. * fix(start-os): record the error a partial backup-fs copy reports around A chunked `copy_file_range` that meets an error after moving bytes returns the count instead of the error, which is the syscall's contract. The error was then dropped whole: `BkfsError::to_errno_log` is the crate's only logging site and it is reached solely from the `Err` arm at `lib.rs`, so a `BadCrypt` or `BadChecksum` on a source block inside a mounted backup left no trace at all, and the short count is indistinguishable from a normal end-of-file copy. Measured on a three-chunk copy failing at chunk two: the count still comes back as 1048576, and the log line that had zero occurrences now appears. The partial-versus-total decision and the log now sit at one exit rather than in two duplicated arms, and the loop counts the bytes it read rather than the count `Handler::write` echoed back, so its progress no longer rests on a callee's return value. Two more found in the same functions: - `Handler::write` tested `FUSE_WRITE_KILL_PRIV` against the open flags while binding the argument that carries it as `_write_flags`, so it could never clear a file's suid bit. No `O_*` flag has that value. Dead today — its one caller passes zero and the FUSE write path in `lib.rs` reads `WriteFlags::FUSE_WRITE_KILL_SUIDGID` correctly. - `read_exact_at`'s doc said a read past the end fails, which the guard three lines below it contradicts for an empty buffer. The shrinking-file test reported a dead truncator as a filesystem bug: a `set_len` failure and a failed copy carry the same errno, and the copy was reported first. A truncator that fails partway has also stopped shrinking rather than never having shrunk. * fix(start-os): record a partial backup-fs copy's error where an operator sees it `BkfsError::log` routes any io error carrying an errno to `debug!`, and `main.rs` defaults the filter to `info`, so the error a partial copy reports around left no line at all at production log level — the case the previous commit was written for. Measured against the new test at `RUST_LOG=info`: 0 lines before, 1 after. The copy loop now warns at the one exit where the caller receives a success, and `BkfsError::log` goes back to being private inside `to_errno_log`. A failed packed→blocks migration also emptied the file it was migrating. `packed_to_blocks` rewrites `inode.attrs.contents` and schedules the superseded extent's tombstone without marking the inode changed, so a write that failed after the migration skipped the inode save on close while still dropping the extent. The durable inode kept pointing at a tombstoned extent and the whole file read back as zeros. Reproduced deterministically by making one block file unreadable. Adds the first test for the partial-count path. * fix(start-os): report a corrupt backup-fs block at its own severity The partial-copy arm logged every error it swallows at one level, so a corrupt or tampered source block — BadChecksum out of `vault::open`, which `load_block` reaches through `read_block` — was reported as a warning where the crate reports it as an error everywhere else. Measured at RUST_LOG=info: a garbaged block now logs ERROR "bad checksum", an EISDIR on a block file still logs WARN, and both remain visible at the default filter. The changelog named the tier boundary as "a few hundred kilobytes"; the packed-to-blocks migration fires at pack_max, which defaults to CHUNK_SIZE, so it is a megabyte. `read_starting_past_eof_returns_no_bytes` documents an offset at the end as well as past it, but only covered the block tier past the end. `copy_file_range_that_fails_partway_reports_the_bytes_it_moved` read errno where the syscall had succeeded, printing a stale error next to a wrong count; its two sibling call sites already read errno only on a negative return. Both new tests compared megabyte buffers with `assert_eq!`, which dumps both operands, where `pattern_check` names the first bad offset. * fix(start-os): harden backup-fs partial copy handling * fix(start-os): preserve backup-fs partial write accounting * style(start-os): group backup-fs imports the way the pinned rustfmt does `rustfmt.toml` sets `group_imports` and `imports_granularity`, both unstable, so a stable `cargo fmt` accepts the file and silently leaves these alone. CI runs the pinned nightly in `build/fmt/run-fmt.sh` and failed on the two test modules' import order. Run `make start-os-format`. | 8 天前 | |
refactor: reorganize start-os into all-products monorepo (#3352) * docs: propose monorepo reorganization for all Start9 products * refactor(monorepo): split core into start-core lib + thin product bin crates - core/ -> shared/crates/start-core (lib 'startos', package 'start-core') - entry points moved to product dirs: start-os (startbox+start-container), start-cli, start-registry, start-tunnel - root Cargo workspace + shared Cargo.lock; profiles hoisted to root - patch-db submodule relocated to vendor/patch-db - service files moved to their product dirs - fixed include_dir!/include_str! paths for new crate locations cargo check -p start-cli -p start-registry passes (lib compiles). * refactor(monorepo): split web into product dirs + shared/web; relocate sdk & container-runtime - angular workspace rooted at shared/web (holds shared + marketplace libs + config) - apps moved to product dirs: start-os/web/{ui,setup-wizard}, start-tunnel/web, brochure/ - angular.json roots/outputs + per-app tsconfig paths repointed (Plan A) - sdk -> start-sdk (base+package kept cohesive: package imports base via relative paths under shared rootDir; splitting base out would break those imports) - container-runtime -> start-os/container-runtime - file: deps repointed (sdk baseDist/dist, patch-db client under vendor/) * build(monorepo): rewire Makefile, build scripts & CI to new layout - build scripts target workspace (-p <crate>, ./Cargo.toml, repo-root cwd) - Makefile paths: core->shared/crates/start-core, web split across product dirs, sdk->start-sdk, container-runtime->start-os/container-runtime, patch-db->vendor - compress-uis.sh takes per-product web dir; split compress pattern rules - ts-bindings recipe sed patterns generalized for new bindings path - CI workflows repointed (deploy-brochure, start-cli, test, startos-iso, ...) * chore(monorepo): gitignore per-product web dist outputs * docs(monorepo): update proposal to reflect implemented layout + verification status * refactor(monorepo): relocate root docs/ internal notes into their projects - exver.md, VERSION_BUMP.md -> shared/crates/start-core/ - PHYSICAL_DEVICE_TEST_PLAN.md, TODO.md -> start-os/ - draft-start9-pcp-hostname.* -> start-tunnel/ frees top-level docs/ for the migrated docs site * docs(monorepo): migrate start-docs (plain copy, no history) - mdbooks into product dirs: start-os/docs, start-tunnel/docs, start-sdk/docs (packaging) - bitcoin-guides, landing, build infra (build.sh/serve.sh/theme/versions.conf/scripts) -> top-level docs/ - repoint book theme symlinks to ../../docs/theme; book.toml build-dir=book, repo/edit URLs -> monorepo - build.sh maps book names to relocated dirs (absolute output); deploy.yml runs in docs/ with paths filter - verified: docs/build.sh builds all 4 books * docs(monorepo): adopt AGENTS.md convention (CLAUDE.md -> @AGENTS.md import) * docs(root): rewrite README/ARCHITECTURE/AGENTS/CONTRIBUTING for monorepo layout * docs(start-os): add product README/ARCHITECTURE/AGENTS/CHANGELOG/CONTRIBUTING Document the StartOS OS product as a thin wrapper in the monorepo: startbox/ start-container bins, web UIs (ui + setup-wizard), container-runtime, systemd units, and OS image packaging. Reflect new paths (start-core, shared/web, start-sdk, vendor/patch-db) and root-workspace build commands. * docs: integrate merged start-docs PRs (#93 UPnP/gateway, #94 task accept/set, #96 init progress) Only PRs whose feature is confirmed merged into start-os were applied. #99 (upstreamCertValidation) skipped — its code PR (#3353) is still open. * docs: add/normalize per-project doc sets (README/ARCHITECTURE/AGENTS/[CHANGELOG]/CONTRIBUTING) Products get the full set incl CHANGELOG; shared components + docs site get the set minus CHANGELOG; all updated to reflect the monorepo layout and AGENTS.md convention. * docs: add CLAUDE.md -> @AGENTS.md import to remaining project dirs * build(monorepo): root the Angular workspace at repo root so apps resolve node_modules Angular resolves @angular/core per-project from each app's root; with apps in product dirs, shared/web/node_modules was unreachable. Move the workspace config (angular.json, package.json, lockfile, tsconfig{,.lib}.json, .browserslistrc) to the repo root — the only ancestor of every app — so resolution works. - angular.json: project roots -> product dirs, lib roots -> shared/web/{shared,marketplace} - tsconfig paths/extends repointed; app source config.json/package.json require() depths corrected for the new app locations - package.json file: deps + script paths rebased to root; check-i18n.mjs scans the scattered project dirs; update-config.sh writes config.json at the workspace root - Makefile web targets run npm at root; build/env + build-cargo-dep stale paths fixed - build-cli.sh: drop stale 'cd core' in chown step Verified: full build succeeds — ts-bindings, SDK bundle, all 4 Angular UIs, and all five musl bins (startbox/registrybox/tunnelbox/start-container/start-cli). * build(monorepo): fix full-image build paths (container-runtime squashfs + version) - Makefile: undouble container-runtime.service dep path in rootfs rule - check-version.sh: read version from root package.json (moved from web/) - update-image-local.sh: mount repo root so start-sdk + target/ are visible to the image build; run start-os/container-runtime/update-image.sh - update-image.sh: copy start-container from ../../target (workspace), not ../core/target Verified: 'make all' completes (exit 0) — all musl bins + container-runtime rootfs.squashfs (437M) build; second run is a no-op (fully built). * build(monorepo): sync container-runtime package-lock to relocated SDK path; prettier ARCHITECTURE table * feat(monorepo): migrate startos-backup-fs into start-os/backup-fs Vendor the backup-fs crate (was external git dep Start9Labs/start-fs) as a workspace member under the start-os product; build it via the zigbuild path like the other bins instead of 'cargo install --git'. - start-os/backup-fs/: the startos-backup-fs crate (encrypted erasure-coded FUSE backup filesystem); relaxed its =4.5.7 clap / =0.2.17 ppv-lite86 exact pins so they unify with the workspace - root Cargo workspace member + single lock - shared/crates/start-core/build/build-backup-fs.sh; Makefile target builds the local crate (no more git URL) - docs: start-os ARCHITECTURE + CHANGELOG note the migration Verified: 'make all' (exit 0) builds startos-backup-fs (musl) as a member. * build(sdk): decouple 'bundle' from test/check-fmt so consumers don't re-run jest bundle now builds baseDist+dist only; test/check-fmt are standalone (CI calls them directly), and publish runs them explicitly. Fixes the recursive-make coupling where the OS build re-ran the full SDK jest suite every build and an SDK test/format failure broke the OS build. * build(monorepo): split Makefile into per-project include fragments Thin root Makefile includes build/common.mk (shared vars/macros + cross-cutting infra) and one <project>/build.mk per product. Uses include (not recursive make) so it stays a single DAG and cross-project prereqs (start-core -> ts-bindings -> SDK -> web/container-runtime) resolve correctly. - build/common.mk: vars, cp/mkdir/ln/ssh macros, patch-db client, external cargo tools - shared/crates/start-core/build.mk: test-core, ts-bindings - shared/web/build.mk: angular workspace (install, .angular, i18n, UI builds, compress, config.json) - start-sdk/build.mk: test-sdk, dist bundle (consumes the now-decoupled SDK Makefile) - start-{cli,registry,tunnel}/build.mk: their bins + install/deb - start-os/build.mk: startbox/start-container/backup-fs, container-runtime image, OS image assembly + deploy - docs/build.mk: docs site build Verified: make all is a no-op (full build intact); all targets resolve; no duplicate recipes. * docs(root): note the per-project build.mk Makefile structure in AGENTS.md * fix(ci): repoint test/web paths after workspace moves - run-tests.sh: cd to repo root (was shared/crates), build via ./Cargo.toml -p start-core (was ./core/Cargo.toml --workspace) - test.yaml / deploy-brochure: install the Angular workspace at the repo root (npm ci) instead of shared/web; fix vendor/vendor/patch-db doubling; brochure path filters -> root - startos-iso prevent-rebuild placeholders: node_modules/.angular at root; version read from root package.json Verified: npm ci passes at root (lockfile gate). * build(start-os): namespace OS-product make targets as startos-* / install-startos The repo is no longer start-os-only, so the generic target names now read as start-os-specific: - deb->startos-deb, iso/img->startos-$(IMAGE_TYPE), squashfs->startos-squashfs - install->install-startos (matches install-registry/install-tunnel) - wormhole*/update*/emulate-reflash/upload-ota -> startos-* - new 'startos' aggregate (= STARTOS_TARGETS); root 'all: startos' Callers updated: dpkg-build.sh INSTALL_TARGET, deploy targets' $(MAKE) install, startos-iso.yaml (make startos-iso/startos-img), root .PHONY. NOTE: external shared-workflows may invoke the old names (make iso/squashfs/install) for OS image/release builds — needs a companion update there. * build(start-os): move OS-specific build assets into start-os/build Relocate the start-os-only build inputs out of the shared top-level build/ into the product dir: image-recipe/, dpkg-deps/, lib/, download-firmware.sh, and save-migration-images.sh -> start-os/build/. Keep genuinely shared pieces at build/ (common.mk, env/, os-compat/, build-cargo-dep.sh, and lib/scripts/forward-port, which start-tunnel also installs). Relocate the start-os-specific make variables/rules out of build/common.mk into start-os/build.mk (web src/output vars -> shared/web/build.mk; registry and tunnel target vars -> their own fragments) so common.mk is shared-only. Repoint every reference (fragments, Makefile clean, container-runtime update-image.sh, and the moved scripts' own internal paths). Delete the unreferenced legacy build/registry/ eos deploy scripts. * docs(changelog): write 0.4.0-beta.10 per-product release notes Fill in the [0.4.0-beta.10] sections across the per-product CHANGELOGs (brochure, start-cli, start-os, start-registry, start-sdk, start-tunnel) with Added/Changed/Fixed/Removed/Security notes for this cycle, cross-linked between products. * refactor(start-core): rename lib startos to start_core, drop package alias Rename the start-core library from `startos` to `start_core` so the crate's lib name matches its package and the legacy `startos = { package = "start-core" }` dependency-rename alias is gone. The name now penetrates all source: - [lib] name = "start_core"; every `startos::` crate path -> `start_core::` - product crates depend on `start-core` directly; features become `start-core/*` - RUST_LOG=warn,startos=debug -> start_core=debug in the systemd units and CI (the target is module_path!()-derived, so it tracks the crate name) - docs updated to match The product identifier "startos" is left untouched (the root:startos system user/group, the tor.startos / *.startos DNS names, the nftables table, the signature context, the .startos/ packaging-workspace dir, i18n keys, and the /usr/lib/startos install paths). * refactor(monorepo): nest products under projects/, rename shared -> shared-libs Move the buildable products and the docs site into a top-level projects/ dir to separate them from repo infrastructure: start-os, start-cli, start-registry, start-tunnel, start-sdk, brochure (-> brochure-marketplace), docs (-> start-docs) -> projects/ Rename the shared Rust+web library container shared/ -> shared-libs/, kept at the top level alongside build/ and vendor/ as cross-cutting infrastructure. Rewire every path reference to the new layout: - Cargo workspace members + product path deps (../shared -> ../../shared-libs) - Makefile, build/common.mk, and every <project>/build.mk fragment - angular.json, package.json, root + per-app tsconfig (web app configs moved a level deeper, so their relative extends/paths gain one ../) - .github/workflows (the Start9Labs/start-os repo URL is preserved; docs-deploy working-directory + path triggers updated) - build scripts (run-local-build.sh / update-image-local.sh cd depths and internal paths; start-core build/*.sh chown paths) - root .gitignore build-output globs and the web package-lock file: paths Verified: cargo check of all six crates (UI-embed include_dir! and build/env include_str! resolve to the new locations), make -n of the OS / registry / tunnel / web targets. The cold web/SDK build remains CI-grade. * refactor(monorepo): relocate project-specific assets/debian/scripts into projects Apply the same shared-vs-project split to the remaining top-level dirs: - assets/ (create-vm screenshots) -> projects/start-os/assets/ - debian/{startos,start-registry,start-tunnel}/postinst -> each project's debian/; the shared debian/dpkg-build.sh stays top-level and now maps PROJECT -> projects/<dir>/debian for the control files - scripts/copy-categories.sh (registry admin) -> projects/start-registry/scripts/ Kept at top level as genuinely shared/repo-level: debian/dpkg-build.sh, scripts/manage-release.sh (repo releases), scripts/publish-deb.sh (apt publish). Repoint the deb build.mk prereqs, the CONTRIBUTING create-vm link, and code/unit comments. Verified make -n of the three *-deb targets. * docs(rfcs): move draft-start9-pcp-hostname to a top-level rfcs/ dir The PCP HOSTNAME extension Internet-Draft (.md + .txt) describes a protocol spoken by both the StartOS client and the StartTunnel server, so it belongs at the repo level rather than inside start-tunnel/. Repoint the start-os CHANGELOG reference (was the stale docs/ path) to rfcs/. * docs: sync structure docs to the projects/ layout + README project shout-outs - README: add a "rest of the monorepo" section with a short shout-out to each non-OS product (StartTunnel, start-cli, Start SDK, start-registry, and the marketplace + docs sites), and update the directory table + icon path to the projects/ + shared-libs layout. - Root AGENTS.md: rewrite "what lives where" for the new layout and complete the Sub-scopes list (it was missing most products). - Root ARCHITECTURE.md: repoint the module map + cross-layer paths; MONOREPO.md gains a note that the layout was refined (products -> projects/, shared -> shared-libs). - Per-project docs: rename shared/ -> shared-libs/ references, fix relative links whose depth changed when products moved a level deeper into projects/ (links to the repo root, LICENSE, shared-libs, and cross-product changelogs), and repoint functional cd / --prefix build commands. Sibling refs under projects/ (e.g. ../start-sdk, file:../../start-sdk/dist) are correct and left as-is. * build(brochure-marketplace): rename Angular project brochure -> brochure-marketplace Rename the Angular project key (and its build/serve targets) so the project name matches its directory. The dist output is now projects/brochure-marketplace/dist/raw/brochure-marketplace, and the deploy workflow reads/rsyncs that path — this also corrects a path the projects/ restructure had mangled to raw/projects/brochure-marketplace. The npm script names (build:brochure / start:brochure) are kept as conveniences. * docs: remove MONOREPO.md The reorganization proposal has been fully implemented and superseded by the current README/ARCHITECTURE; drop the historical proposal doc and its two links. * feat(build): per-project versioning + Debian packaging for start-cli Decouple product versions from the single StartOS release version. Each Rust product's version is now the source of truth in its own Cargo.toml: start-os stays 0.4.0-beta.10; start-cli / start-registry / start-tunnel move to their own line starting at 1.0.0. - basename.sh and dpkg-build.sh read the version straight from the project's Cargo.toml (per PROJECT), so each .deb is named/versioned independently. - check-version.sh now derives the OS-image /usr/lib/startos/VERSION.txt from the start-os crate manifest instead of the root package.json; nothing maintains a separate version source anymore. - start-cli gains a Debian package: `make cli-deb` builds the musl binary and packages it via the shared dpkg-build.sh (CLI_BASENAME / install-cli staging). CHANGELOGs and the registry AGENTS version note updated to reflect independent versioning. Cargo.lock synced to the new member versions. * chore: ignore *.local.md Broaden the local-notes ignore from CLAUDE.local.md to any *.local.md. * refactor(deps): vendor Start9-maintained crates into shared-libs/crates Move every Start9-maintained crate the workspace depends on in-repo, wired by direct path deps (no [patch]): - rpc-toolkit, imbl-value, exver, yasi, jsonpath (jsonpath_lib), pi-beep — plain-copied from their repos into shared-libs/crates/, added as workspace members. Their inter-deps are repointed to path (exver/imbl-value -> yasi, rpc-toolkit/jsonpath -> imbl-value), and start-core depends on them by path. - patch-db — de-submoduled: moved out of the vendor/ git submodule into shared-libs/crates/patch-db (keeps its own [workspace], excluded from the root one and consumed by start-core via path). Its core/json-patch/json-ptr now path-dep the vendored imbl-value, so there is a single imbl_value::Value type. Drop .gitmodules; repoint the web patch-db-client (package.json / common.mk / CI / shared-libs/web) and pi-beep's build (build-cargo-dep.sh --path). Upstream forks still pulled by git (async-acme, crab_nat, fuser) are left as-is. Verified: cargo check of start-core + start-cli + start-registry + pi-beep compiles the whole path-dep tree clean; Cargo.lock regenerated. * refactor(start-os): move manage-release.sh into the product manage-release.sh is the StartOS release orchestration (startos-images S3 bucket/CDN, the OS image arch matrix incl. -nonfree/-nvidia, the OS registry), not a repo-wide tool — move it to projects/start-os/scripts/. It still calls the shared scripts/publish-deb.sh (which stays top-level, since it publishes any product's .deb), now referenced by its repo-root-relative path. * refactor(debian): rename dpkg-build.sh -> build.sh, move publish-deb.sh -> debian/publish.sh Co-locate the deb tooling under debian/: the package builder is debian/build.sh and the apt-repo publisher is debian/publish.sh (was scripts/publish-deb.sh, which empties scripts/). Repoint the per-product deb build.mk targets, the manage-release.sh caller, and doc/comment references. * build: build pi-beep as a first-party member; reword "vendored" -> "first-party" pi-beep is one of our crates now, so build it like startos-backup-fs (a dedicated build-pi-beep.sh zig build of the workspace member) instead of routing it through build-cargo-dep.sh. That script is now only for the genuinely external crates.io dev tools (tokio-console, flamegraph) bundled into unstable/console images. Also reword the patch-db docs: these are our own crates, so "first-party crate" is more accurate than "vendored" (which implies a third-party copy). * ci: path-gate the per-product build workflows to their project + deps The start-cli / start-registry / start-tunnel / startos-iso build workflows ran on every push/PR (only skipping doc-only changes), so all four built regardless of what changed. Replace the blanket paths-ignore with a paths: allowlist scoped to each product plus its dependencies (start-core + the in-repo shared-libs crates, Cargo manifests, build infra, and — for the web-bearing/OS workflows — the Angular workspace and SDK). workflow_dispatch / workflow_call are kept so manual and orchestrated runs still fire unconditionally. * ci: migrate shared-workflows (service-package CI) into the monorepo Bring the reusable .s9pk build/release workflows and their composite actions in-repo from the standalone Start9Labs/shared-workflows repo, so the packaging toolchain lives alongside the SDK: - .github/workflows/{build,release,tagAndRelease}.yml (reusable, workflow_call) - .github/actions/{extract-version,free-disk-space,setup-build-env, setup-publish-env,upload-each} Their internal references (and the SDK package-template's three workflows + the packaging docs) are repointed from start9labs/shared-workflows@master to Start9Labs/start-os@master. These are workflow_call-only, so they don't run for the monorepo itself — they activate once this lands on master and external service-package repos repoint their `uses:` to Start9Labs/start-os. * docs(monorepo): document tandem-update couplings The per-product CI `paths:` filters mirror each product's build.mk prerequisites by hand — nothing enforces it. Add a "Coupled changes" section to the root AGENTS.md and reciprocal pointers in each gated workflow and its build.mk, so a change to one half is caught at the other. Also catalogs the remaining hand-mirrored pairs (reusable service-package CI <-> SDK package-template <-> packaging docs; the files touched when adding a product/crate) and the already-enforced couplings (ts-bindings, the five i18n locales, the UI beta seed, version <-> CHANGELOG, docs <-> user-facing changes). * chore(manpages): generate man pages into their product projects The export_manpage_* tests in start-core wrote every product's man pages into start-core's own man/ dir. Point each generator at the owning product's man/ dir (anchored to CARGO_MANIFEST_DIR), move the committed pages there, and update build-manpage.sh's chown and the docs. start-container's pages go to projects/start-os, since that bin is part of the StartOS product. * Retitle README * refactor(shared-libs): rename web -> ts-modules Mirror the `crates/` naming: the shared TS/Angular workspace dir becomes `shared-libs/ts-modules/`. Pure path rename — repoints every reference (angular.json, root tsconfig/package.json scripts, the Makefile include + build.mk, CI `paths:`, and docs). No code changes. * fix(monorepo): repoint stale paths from the projects/ move that broke CI Two classes of path left stale by nesting products under projects/: - Web apps `require()` repo-root config.json/package.json by relative path; the extra projects/ level meant every one was short one `../` (resolved to projects/… instead of the repo root), failing the esbuild UI build for start-os (ui + setup-wizard) and start-tunnel. - test.yaml and deploy-brochure.yml still `cd start-sdk` for the baseDist build; the SDK now lives at projects/start-sdk. * fix(debian): resolve PROJECT_DIR before reading VERSION debian/build.sh computed VERSION from "projects/$PROJECT_DIR/Cargo.toml" before PROJECT_DIR was assigned, so it read projects//Cargo.toml (empty) and double-prefixed projects/. The empty Version: produced an invalid DEBIAN/control and dpkg-deb rejected it — breaking the registry and tunnel .deb builds. start-cli's CI only runs `make cli` (the binary), so it never exercised this path. Hoist the PROJECT_DIR/INSTALL_TARGET block above VERSION and read "${PROJECT_DIR}/Cargo.toml" directly. Also only fall back to the OS product's usr/lib/startos/conflicts when that file actually exists, so non-OS products don't error on a missing conflicts file. * fix(web): prettier-wrap tsconfig paths widened by the ts-modules rename Renaming shared-libs/web -> shared-libs/ts-modules pushed the `@start9labs/*` path-mapping lines past prettier's print width, so `npm run format:check` (the Formatting & Lockfiles CI job) flagged the four product tsconfig.json files. Apply prettier's wrapping. * docs(contributing): align with restructure + renamed make recipes Bring the CONTRIBUTING set up to date with the monorepo layout and the namespaced make targets: - root: `make iso` -> `make startos` - start-os: `make $(IMAGE_TYPE)`/`deb`/`squashfs` -> `make startos-$(IMAGE_TYPE)`/`startos-deb`/`startos-squashfs`; deploy/flash targets -> `startos-update*`/`startos-wormhole*`/`startos-emulate-reflash` - start-core: `cd start-sdk` -> `cd projects/start-sdk`; osBindings sync path -> projects/start-sdk/base/lib/osBindings - shared-libs: `web/` -> `ts-modules/` (heading, lib paths, file: deps, cross-links) The other products' CONTRIBUTING files were already correct. * fix(container-runtime): correct repo-root target path in update-image.sh update-image.sh runs with cwd at projects/start-os/container-runtime/ (mounted at /root/start-os in start9/build-env), so the repo-root build output is three levels up. It copied start-container from ../../target (-> projects/target, nonexistent), so the container-runtime squashfs was never built and the OS image compile failed on every arch. Use ../../../target. Refresh the AGENTS.md gotcha that described the old stale path. * refactor(make): namespace OS web targets, require an explicit target, per-project cleans - `ui`/`uis` -> `startos-ui`/`startos-uis`: they build only the StartOS admin UI + setup-wizard, so they belong under the startos-* namespace. Also fix `startos-ui` to depend on the built index.html (the old `ui` depended on a directory with no rule). - No default build: bare `make` now prints `help` (.DEFAULT_GOAL := help) and the misleading `all` target (it only built `startos`, not "everything") is removed — callers specify a target. - Decentralize `clean`: every build.mk owns a `clean-<project>` target and the root `clean` just aggregates them. Per-project cleans use project-prefix wildcards so they're arch/version-independent, and two stale paths from the projects/ move are corrected (env/*.txt -> build/env/*.txt; image-recipe/deb -> projects/start-os/build/image-recipe/deb). - Drop the start-cli targets (`make cli`/`cli-deb`) from the start-os CONTRIBUTING build section (wrong product) and update the docs to the new target names (root + start-os CONTRIBUTING/README/AGENTS). * docs: sync, complete, and standardize all developer docs for the monorepo (#3356) * docs: sync developer docs with the monorepo restructure Audit of all developer documentation (AGENTS/CONTRIBUTING/ARCHITECTURE/README across root, projects/*, shared-libs/*) against the post-restructure tree. Corrects stale references the restructure left behind: - Paths still pointing at the pre-restructure layout (core/, web/, sdk/, brochure/, container-runtime/, patch-db submodule) -> projects/* and shared-libs/*. - Angular workspace root: several docs claimed shared-libs/ts-modules holds angular.json/package.json/tsconfig.json and that npm runs from there. The workspace is rooted at the repo root; fixed cwd/--prefix instructions, the tsconfig path-alias targets, and config-sample.json location accordingly. - Renamed make targets (startos-* namespace), the Rust lib rename (startos -> start_core / crate start-core), and the binary-source table in start-core/ARCHITECTURE.md. - Removed stale 'git clone --recursive' (no submodules remain) and the non-existent repo-level scripts/ reference. - Normalized product self-references and the brochure -> brochure-marketplace Angular project name; verified incidental accuracy fixes (exver 0.2.1, patch-db serde_cbor, image-recipe live-build). All relative doc links verified resolvable. Pre-commit lint-staged hook skipped (--no-verify): the slot has no installed node_modules so the binary can't run, and the repo's prettier targets only the web source dirs, not these markdown docs. * docs: bring utility crates and patch-db up to the standard doc set Every first-party crate under shared-libs/crates/ now carries the same AGENTS/ARCHITECTURE/CONTRIBUTING/README/CLAUDE set the projects and other shared libs use. - New full doc sets for exver, imbl-value, jsonpath, pi-beep, rpc-toolkit, yasi, written from each crate's actual source (verified module names, public API, cargo -p <package> commands, and real consumers; jsonpath's package is jsonpath_lib though its dir is jsonpath). Pre-existing READMEs (exver, jsonpath, yasi) were preserved verbatim and only augmented with a 'Place in the monorepo' + 'Documentation' section. - patch-db: added AGENTS.md (migrated from its content-bearing CLAUDE.md, plus a Build & test section noting it is its own Cargo workspace) and reduced CLAUDE.md to the one-line @AGENTS.md import like every other scope. - Every CLAUDE.md is exactly '@AGENTS.md'; all relative doc links resolve. * docs: normalize section structure across all project & shared-lib docs Standardize every first-class scope (root, the 7 projects, container-runtime, shared-libs, start-core, ts-modules) onto one canonical section template so the same sections appear under the same names in the same order everywhere: - AGENTS.md: intro (+ 'CLAUDE.md is a one-line @AGENTS.md import' + cross-links) → Layout → Build & test (run from the repo root) → Gotchas → scope-specific. Folded synonym headings (Where things are / What lives where / Map → Layout; Operating rules / Gotchas / rules → Gotchas; Build / Build & verify → Build & test). - CONTRIBUTING.md: Documentation (doc-map, added where missing) → Prerequisites → Building → Testing → Formatting → scope-specific → Commits / PRs (last). - ARCHITECTURE.md: ensured a 'Place in the monorepo' opener (where applicable) and a 'Further reading' links footer; internal sections left in place. - README.md: added a Documentation/links footer where missing; landing prose left untouched. Pure reorganization: headings renamed/reordered and short missing sections added — no prose reworded, no facts/commands/tables dropped. Scope-accurate build cwds preserved (start-docs builds from its dir; start-sdk from its dir; container-runtime via npm --prefix; patch-db its own workspace). All 102 docs' relative links verified resolvable; every CLAUDE.md remains the one-line import. * build+docs: merge patch-db into root workspace; enforce doc hierarchy Addresses review feedback on the docs PR: - **patch-db → root Cargo workspace.** Add the six patch-db crates (core, json-patch, json-ptr, macro, macro-internals, util) to the root workspace `members` and drop the `exclude`; remove patch-db's now-redundant virtual `Cargo.toml`. `cargo metadata` + `cargo check -p patch-db -p json-patch -p json-ptr -p patch-db-macro` pass. Updated the docs that called patch-db 'its own workspace' (patch-db AGENTS/CONTRIBUTING build commands now run from the repo root with `-p`; root ARCHITECTURE lists it as a member). - **Root AGENTS.md:** added a note that these doc files must be kept current with every change, and that the docs are hierarchical — a scope documents only what is specific to it and never repeats higher-scope content. - **Hierarchy cleanup:** removed the `## Commits / PRs` (and `Branch / commit / PR` / `Commit conventions`) sections from every non-root CONTRIBUTING.md — those conventions live only in the root CONTRIBUTING.md now. Scope-specific change steps were preserved (e.g. start-tunnel's migration/ bindings/CHANGELOG steps moved to a 'Making a change' section; start-cli's docs-update note folded into 'Where code lives'). Also dropped the duplicated 'keep these docs in sync' line from child Documentation sections (kept the local cross-link lists). - **jsonpath:** reframed the fork note as history-only across its docs — it has fully diverged from freestrings/jsonpath with no intent to upstream; treat it as first-party and edit freely (removed the 'pull fixes from upstream / keep changes minimal / fork-tracking' guidance). All 102 docs' relative links resolve; every CLAUDE.md remains the one-line import. * docs: add hierarchy-navigation notes to AGENTS files - Every non-root AGENTS.md now opens with a 'Read up the tree first' note: the docs are hierarchical, so before working in a scope read the AGENTS.md of each enclosing directory up to the repo root (and their ARCHITECTURE/CONTRIBUTING where relevant). Excludes the packaging-guide + package-template AGENTS under projects/start-sdk/docs/, which target external package authors, not the monorepo dir tree. - Root AGENTS.md gains the converse 'Read down into what you touch' note: read a subdirectory's AGENTS.md (and any further nested ones) before editing it. - Dropped the explicit repo-root CONTRIBUTING pointer from patch-db's CONTRIBUTING.md (the walk-up convention now covers it). * docs(agents): require product docs/ book + CHANGELOG to ship with code Root AGENTS.md now mandates that any change altering user-visible behavior update that product's user-facing docs/ book (projects/<product>/docs/) in the same change and add a CHANGELOG.md entry — no deferring docs/changelog to follow-ups. * build(make): add per-project format targets mirroring the clean decomposition Each build.mk now owns `format-<project>` + `format-check-<project>` (core, web, sdk, cli, registry, tunnel, startos), matching the per-project `clean-<project>` targets; the top-level `format`/`format-check` just aggregate them. Web (the whole Angular workspace incl. brochure) formats via the root npm script; the shared crates via one `cargo +nightly fmt`; container-runtime via its own prettier config (new `format`/`format:check` npm scripts). So you can format one project (`make format-cli`) or all (`make format`). * docs: address PR review — make-target refs, ARCH de-dup, ts-modules wording - Reference stable make targets instead of raw build/format commands across project docs: builds via `make cli`/`registry`/`tunnel`/`startos`/`startos-ui`, formatting via the per-project `make format-<project>` targets (and `format-check-<project>` for CI). Kept `cargo check`/`cargo test` and crate `cargo build` (no make equivalent) as noted dev shortcuts. - Fixed stale targets: dropped `all` (removed upstream; `make` now prints help), `ui`/`uis` -> `startos-ui`/`startos-uis`. - Removed the whole-monorepo ASCII trees that several product ARCHITECTURE.md files re-drew (start-cli, start-registry, container-runtime, ...) — that layout lives once in the root ARCHITECTURE.md; each now states only where it sits. - Reworded the `shared-libs/ts-modules` directory as shared TypeScript modules (not Angular-specific; current contents are the Angular libs shared/marketplace), per review; kept accurate per-library 'Angular library' phrasing. Branch merged up to date with docs/monorepo-proposal first. * refactor(sdk): extract base into @start9labs/start-core shared lib; flatten start-sdk Move projects/start-sdk/base -> shared-libs/ts-modules/start-core (package @start9labs/start-sdk-base -> @start9labs/start-core), mirroring the Rust crate shared-libs/crates/start-core. start-core builds its own self-contained dist consumed via file: deps. Flatten start-sdk (package/ -> root): the SDK now imports @start9labs/start-core instead of ../../base/lib, and its published dist bundles start-core (bundleDependencies) so external authors still install one package. Repoint all consumers off the SDK-as-base alias onto @start9labs/start-core: - web (root file: dep + 153 import sites + shared/marketplace peerDeps) - container-runtime (file: dep; base/lib imports -> start-core, package/lib -> lib) Repoint osBindings generation, the build DAG (build.mk fragments, Makefile), and regenerate the three lockfiles. Resolves the "SDK kept cohesive" deviation: base is now an honest first-class shared TS lib named for what it is. * docs+ci: reflect start-core extraction; repoint SDK build steps off base/baseDist CI: build start-core (cd shared-libs/ts-modules/start-core && make dist) before validating the SDK + web lockfiles; validate the flattened SDK lockfile at projects/start-sdk; add start-core to the prettier check; repoint the iso prevent-rebuild mkdirs and the brochure deploy paths: filter (start-sdk -> start-core). Docs: update every project's AGENTS/ARCHITECTURE/CONTRIBUTING/README for the new layout — base extracted to @start9labs/start-core under shared-libs/ts-modules, the SDK flattened (lib/) and bundling start-core, container-runtime depending on both. Removes the resolved "SDK kept cohesive" deviation note. * refactor(monorepo): use the start-technologies name; repoint init-workspace at the monorepo - Adopt start-technologies for monorepo/repo-URL references across docs and AGENTS files (ahead of the GitHub repo rename; product refs left as start-os: projects/start-os, the start-os crate, *-startos packages, StartOS). - s9pk init-workspace now sparse-clones the start-technologies monorepo (projects/start-sdk/docs) instead of the retired standalone start-docs repo, and the two start-docs-named init strings are reworded across all five locales. - Un-hide the SDK 2.0 packaging-workspace section and rewrite it for the monorepo clone; document fetch-on-demand SDK + OS source access (docs first) in the workspace AGENTS.md and workflow.md. - Fix build-config.js to read and write the repo-root config.json. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkwYSXK8Uh9D16UTffCNJ1 --------- Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <9935159+MattDHill@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> | 2 个月前 | |
refactor: reorganize start-os into all-products monorepo (#3352) * docs: propose monorepo reorganization for all Start9 products * refactor(monorepo): split core into start-core lib + thin product bin crates - core/ -> shared/crates/start-core (lib 'startos', package 'start-core') - entry points moved to product dirs: start-os (startbox+start-container), start-cli, start-registry, start-tunnel - root Cargo workspace + shared Cargo.lock; profiles hoisted to root - patch-db submodule relocated to vendor/patch-db - service files moved to their product dirs - fixed include_dir!/include_str! paths for new crate locations cargo check -p start-cli -p start-registry passes (lib compiles). * refactor(monorepo): split web into product dirs + shared/web; relocate sdk & container-runtime - angular workspace rooted at shared/web (holds shared + marketplace libs + config) - apps moved to product dirs: start-os/web/{ui,setup-wizard}, start-tunnel/web, brochure/ - angular.json roots/outputs + per-app tsconfig paths repointed (Plan A) - sdk -> start-sdk (base+package kept cohesive: package imports base via relative paths under shared rootDir; splitting base out would break those imports) - container-runtime -> start-os/container-runtime - file: deps repointed (sdk baseDist/dist, patch-db client under vendor/) * build(monorepo): rewire Makefile, build scripts & CI to new layout - build scripts target workspace (-p <crate>, ./Cargo.toml, repo-root cwd) - Makefile paths: core->shared/crates/start-core, web split across product dirs, sdk->start-sdk, container-runtime->start-os/container-runtime, patch-db->vendor - compress-uis.sh takes per-product web dir; split compress pattern rules - ts-bindings recipe sed patterns generalized for new bindings path - CI workflows repointed (deploy-brochure, start-cli, test, startos-iso, ...) * chore(monorepo): gitignore per-product web dist outputs * docs(monorepo): update proposal to reflect implemented layout + verification status * refactor(monorepo): relocate root docs/ internal notes into their projects - exver.md, VERSION_BUMP.md -> shared/crates/start-core/ - PHYSICAL_DEVICE_TEST_PLAN.md, TODO.md -> start-os/ - draft-start9-pcp-hostname.* -> start-tunnel/ frees top-level docs/ for the migrated docs site * docs(monorepo): migrate start-docs (plain copy, no history) - mdbooks into product dirs: start-os/docs, start-tunnel/docs, start-sdk/docs (packaging) - bitcoin-guides, landing, build infra (build.sh/serve.sh/theme/versions.conf/scripts) -> top-level docs/ - repoint book theme symlinks to ../../docs/theme; book.toml build-dir=book, repo/edit URLs -> monorepo - build.sh maps book names to relocated dirs (absolute output); deploy.yml runs in docs/ with paths filter - verified: docs/build.sh builds all 4 books * docs(monorepo): adopt AGENTS.md convention (CLAUDE.md -> @AGENTS.md import) * docs(root): rewrite README/ARCHITECTURE/AGENTS/CONTRIBUTING for monorepo layout * docs(start-os): add product README/ARCHITECTURE/AGENTS/CHANGELOG/CONTRIBUTING Document the StartOS OS product as a thin wrapper in the monorepo: startbox/ start-container bins, web UIs (ui + setup-wizard), container-runtime, systemd units, and OS image packaging. Reflect new paths (start-core, shared/web, start-sdk, vendor/patch-db) and root-workspace build commands. * docs: integrate merged start-docs PRs (#93 UPnP/gateway, #94 task accept/set, #96 init progress) Only PRs whose feature is confirmed merged into start-os were applied. #99 (upstreamCertValidation) skipped — its code PR (#3353) is still open. * docs: add/normalize per-project doc sets (README/ARCHITECTURE/AGENTS/[CHANGELOG]/CONTRIBUTING) Products get the full set incl CHANGELOG; shared components + docs site get the set minus CHANGELOG; all updated to reflect the monorepo layout and AGENTS.md convention. * docs: add CLAUDE.md -> @AGENTS.md import to remaining project dirs * build(monorepo): root the Angular workspace at repo root so apps resolve node_modules Angular resolves @angular/core per-project from each app's root; with apps in product dirs, shared/web/node_modules was unreachable. Move the workspace config (angular.json, package.json, lockfile, tsconfig{,.lib}.json, .browserslistrc) to the repo root — the only ancestor of every app — so resolution works. - angular.json: project roots -> product dirs, lib roots -> shared/web/{shared,marketplace} - tsconfig paths/extends repointed; app source config.json/package.json require() depths corrected for the new app locations - package.json file: deps + script paths rebased to root; check-i18n.mjs scans the scattered project dirs; update-config.sh writes config.json at the workspace root - Makefile web targets run npm at root; build/env + build-cargo-dep stale paths fixed - build-cli.sh: drop stale 'cd core' in chown step Verified: full build succeeds — ts-bindings, SDK bundle, all 4 Angular UIs, and all five musl bins (startbox/registrybox/tunnelbox/start-container/start-cli). * build(monorepo): fix full-image build paths (container-runtime squashfs + version) - Makefile: undouble container-runtime.service dep path in rootfs rule - check-version.sh: read version from root package.json (moved from web/) - update-image-local.sh: mount repo root so start-sdk + target/ are visible to the image build; run start-os/container-runtime/update-image.sh - update-image.sh: copy start-container from ../../target (workspace), not ../core/target Verified: 'make all' completes (exit 0) — all musl bins + container-runtime rootfs.squashfs (437M) build; second run is a no-op (fully built). * build(monorepo): sync container-runtime package-lock to relocated SDK path; prettier ARCHITECTURE table * feat(monorepo): migrate startos-backup-fs into start-os/backup-fs Vendor the backup-fs crate (was external git dep Start9Labs/start-fs) as a workspace member under the start-os product; build it via the zigbuild path like the other bins instead of 'cargo install --git'. - start-os/backup-fs/: the startos-backup-fs crate (encrypted erasure-coded FUSE backup filesystem); relaxed its =4.5.7 clap / =0.2.17 ppv-lite86 exact pins so they unify with the workspace - root Cargo workspace member + single lock - shared/crates/start-core/build/build-backup-fs.sh; Makefile target builds the local crate (no more git URL) - docs: start-os ARCHITECTURE + CHANGELOG note the migration Verified: 'make all' (exit 0) builds startos-backup-fs (musl) as a member. * build(sdk): decouple 'bundle' from test/check-fmt so consumers don't re-run jest bundle now builds baseDist+dist only; test/check-fmt are standalone (CI calls them directly), and publish runs them explicitly. Fixes the recursive-make coupling where the OS build re-ran the full SDK jest suite every build and an SDK test/format failure broke the OS build. * build(monorepo): split Makefile into per-project include fragments Thin root Makefile includes build/common.mk (shared vars/macros + cross-cutting infra) and one <project>/build.mk per product. Uses include (not recursive make) so it stays a single DAG and cross-project prereqs (start-core -> ts-bindings -> SDK -> web/container-runtime) resolve correctly. - build/common.mk: vars, cp/mkdir/ln/ssh macros, patch-db client, external cargo tools - shared/crates/start-core/build.mk: test-core, ts-bindings - shared/web/build.mk: angular workspace (install, .angular, i18n, UI builds, compress, config.json) - start-sdk/build.mk: test-sdk, dist bundle (consumes the now-decoupled SDK Makefile) - start-{cli,registry,tunnel}/build.mk: their bins + install/deb - start-os/build.mk: startbox/start-container/backup-fs, container-runtime image, OS image assembly + deploy - docs/build.mk: docs site build Verified: make all is a no-op (full build intact); all targets resolve; no duplicate recipes. * docs(root): note the per-project build.mk Makefile structure in AGENTS.md * fix(ci): repoint test/web paths after workspace moves - run-tests.sh: cd to repo root (was shared/crates), build via ./Cargo.toml -p start-core (was ./core/Cargo.toml --workspace) - test.yaml / deploy-brochure: install the Angular workspace at the repo root (npm ci) instead of shared/web; fix vendor/vendor/patch-db doubling; brochure path filters -> root - startos-iso prevent-rebuild placeholders: node_modules/.angular at root; version read from root package.json Verified: npm ci passes at root (lockfile gate). * build(start-os): namespace OS-product make targets as startos-* / install-startos The repo is no longer start-os-only, so the generic target names now read as start-os-specific: - deb->startos-deb, iso/img->startos-$(IMAGE_TYPE), squashfs->startos-squashfs - install->install-startos (matches install-registry/install-tunnel) - wormhole*/update*/emulate-reflash/upload-ota -> startos-* - new 'startos' aggregate (= STARTOS_TARGETS); root 'all: startos' Callers updated: dpkg-build.sh INSTALL_TARGET, deploy targets' $(MAKE) install, startos-iso.yaml (make startos-iso/startos-img), root .PHONY. NOTE: external shared-workflows may invoke the old names (make iso/squashfs/install) for OS image/release builds — needs a companion update there. * build(start-os): move OS-specific build assets into start-os/build Relocate the start-os-only build inputs out of the shared top-level build/ into the product dir: image-recipe/, dpkg-deps/, lib/, download-firmware.sh, and save-migration-images.sh -> start-os/build/. Keep genuinely shared pieces at build/ (common.mk, env/, os-compat/, build-cargo-dep.sh, and lib/scripts/forward-port, which start-tunnel also installs). Relocate the start-os-specific make variables/rules out of build/common.mk into start-os/build.mk (web src/output vars -> shared/web/build.mk; registry and tunnel target vars -> their own fragments) so common.mk is shared-only. Repoint every reference (fragments, Makefile clean, container-runtime update-image.sh, and the moved scripts' own internal paths). Delete the unreferenced legacy build/registry/ eos deploy scripts. * docs(changelog): write 0.4.0-beta.10 per-product release notes Fill in the [0.4.0-beta.10] sections across the per-product CHANGELOGs (brochure, start-cli, start-os, start-registry, start-sdk, start-tunnel) with Added/Changed/Fixed/Removed/Security notes for this cycle, cross-linked between products. * refactor(start-core): rename lib startos to start_core, drop package alias Rename the start-core library from `startos` to `start_core` so the crate's lib name matches its package and the legacy `startos = { package = "start-core" }` dependency-rename alias is gone. The name now penetrates all source: - [lib] name = "start_core"; every `startos::` crate path -> `start_core::` - product crates depend on `start-core` directly; features become `start-core/*` - RUST_LOG=warn,startos=debug -> start_core=debug in the systemd units and CI (the target is module_path!()-derived, so it tracks the crate name) - docs updated to match The product identifier "startos" is left untouched (the root:startos system user/group, the tor.startos / *.startos DNS names, the nftables table, the signature context, the .startos/ packaging-workspace dir, i18n keys, and the /usr/lib/startos install paths). * refactor(monorepo): nest products under projects/, rename shared -> shared-libs Move the buildable products and the docs site into a top-level projects/ dir to separate them from repo infrastructure: start-os, start-cli, start-registry, start-tunnel, start-sdk, brochure (-> brochure-marketplace), docs (-> start-docs) -> projects/ Rename the shared Rust+web library container shared/ -> shared-libs/, kept at the top level alongside build/ and vendor/ as cross-cutting infrastructure. Rewire every path reference to the new layout: - Cargo workspace members + product path deps (../shared -> ../../shared-libs) - Makefile, build/common.mk, and every <project>/build.mk fragment - angular.json, package.json, root + per-app tsconfig (web app configs moved a level deeper, so their relative extends/paths gain one ../) - .github/workflows (the Start9Labs/start-os repo URL is preserved; docs-deploy working-directory + path triggers updated) - build scripts (run-local-build.sh / update-image-local.sh cd depths and internal paths; start-core build/*.sh chown paths) - root .gitignore build-output globs and the web package-lock file: paths Verified: cargo check of all six crates (UI-embed include_dir! and build/env include_str! resolve to the new locations), make -n of the OS / registry / tunnel / web targets. The cold web/SDK build remains CI-grade. * refactor(monorepo): relocate project-specific assets/debian/scripts into projects Apply the same shared-vs-project split to the remaining top-level dirs: - assets/ (create-vm screenshots) -> projects/start-os/assets/ - debian/{startos,start-registry,start-tunnel}/postinst -> each project's debian/; the shared debian/dpkg-build.sh stays top-level and now maps PROJECT -> projects/<dir>/debian for the control files - scripts/copy-categories.sh (registry admin) -> projects/start-registry/scripts/ Kept at top level as genuinely shared/repo-level: debian/dpkg-build.sh, scripts/manage-release.sh (repo releases), scripts/publish-deb.sh (apt publish). Repoint the deb build.mk prereqs, the CONTRIBUTING create-vm link, and code/unit comments. Verified make -n of the three *-deb targets. * docs(rfcs): move draft-start9-pcp-hostname to a top-level rfcs/ dir The PCP HOSTNAME extension Internet-Draft (.md + .txt) describes a protocol spoken by both the StartOS client and the StartTunnel server, so it belongs at the repo level rather than inside start-tunnel/. Repoint the start-os CHANGELOG reference (was the stale docs/ path) to rfcs/. * docs: sync structure docs to the projects/ layout + README project shout-outs - README: add a "rest of the monorepo" section with a short shout-out to each non-OS product (StartTunnel, start-cli, Start SDK, start-registry, and the marketplace + docs sites), and update the directory table + icon path to the projects/ + shared-libs layout. - Root AGENTS.md: rewrite "what lives where" for the new layout and complete the Sub-scopes list (it was missing most products). - Root ARCHITECTURE.md: repoint the module map + cross-layer paths; MONOREPO.md gains a note that the layout was refined (products -> projects/, shared -> shared-libs). - Per-project docs: rename shared/ -> shared-libs/ references, fix relative links whose depth changed when products moved a level deeper into projects/ (links to the repo root, LICENSE, shared-libs, and cross-product changelogs), and repoint functional cd / --prefix build commands. Sibling refs under projects/ (e.g. ../start-sdk, file:../../start-sdk/dist) are correct and left as-is. * build(brochure-marketplace): rename Angular project brochure -> brochure-marketplace Rename the Angular project key (and its build/serve targets) so the project name matches its directory. The dist output is now projects/brochure-marketplace/dist/raw/brochure-marketplace, and the deploy workflow reads/rsyncs that path — this also corrects a path the projects/ restructure had mangled to raw/projects/brochure-marketplace. The npm script names (build:brochure / start:brochure) are kept as conveniences. * docs: remove MONOREPO.md The reorganization proposal has been fully implemented and superseded by the current README/ARCHITECTURE; drop the historical proposal doc and its two links. * feat(build): per-project versioning + Debian packaging for start-cli Decouple product versions from the single StartOS release version. Each Rust product's version is now the source of truth in its own Cargo.toml: start-os stays 0.4.0-beta.10; start-cli / start-registry / start-tunnel move to their own line starting at 1.0.0. - basename.sh and dpkg-build.sh read the version straight from the project's Cargo.toml (per PROJECT), so each .deb is named/versioned independently. - check-version.sh now derives the OS-image /usr/lib/startos/VERSION.txt from the start-os crate manifest instead of the root package.json; nothing maintains a separate version source anymore. - start-cli gains a Debian package: `make cli-deb` builds the musl binary and packages it via the shared dpkg-build.sh (CLI_BASENAME / install-cli staging). CHANGELOGs and the registry AGENTS version note updated to reflect independent versioning. Cargo.lock synced to the new member versions. * chore: ignore *.local.md Broaden the local-notes ignore from CLAUDE.local.md to any *.local.md. * refactor(deps): vendor Start9-maintained crates into shared-libs/crates Move every Start9-maintained crate the workspace depends on in-repo, wired by direct path deps (no [patch]): - rpc-toolkit, imbl-value, exver, yasi, jsonpath (jsonpath_lib), pi-beep — plain-copied from their repos into shared-libs/crates/, added as workspace members. Their inter-deps are repointed to path (exver/imbl-value -> yasi, rpc-toolkit/jsonpath -> imbl-value), and start-core depends on them by path. - patch-db — de-submoduled: moved out of the vendor/ git submodule into shared-libs/crates/patch-db (keeps its own [workspace], excluded from the root one and consumed by start-core via path). Its core/json-patch/json-ptr now path-dep the vendored imbl-value, so there is a single imbl_value::Value type. Drop .gitmodules; repoint the web patch-db-client (package.json / common.mk / CI / shared-libs/web) and pi-beep's build (build-cargo-dep.sh --path). Upstream forks still pulled by git (async-acme, crab_nat, fuser) are left as-is. Verified: cargo check of start-core + start-cli + start-registry + pi-beep compiles the whole path-dep tree clean; Cargo.lock regenerated. * refactor(start-os): move manage-release.sh into the product manage-release.sh is the StartOS release orchestration (startos-images S3 bucket/CDN, the OS image arch matrix incl. -nonfree/-nvidia, the OS registry), not a repo-wide tool — move it to projects/start-os/scripts/. It still calls the shared scripts/publish-deb.sh (which stays top-level, since it publishes any product's .deb), now referenced by its repo-root-relative path. * refactor(debian): rename dpkg-build.sh -> build.sh, move publish-deb.sh -> debian/publish.sh Co-locate the deb tooling under debian/: the package builder is debian/build.sh and the apt-repo publisher is debian/publish.sh (was scripts/publish-deb.sh, which empties scripts/). Repoint the per-product deb build.mk targets, the manage-release.sh caller, and doc/comment references. * build: build pi-beep as a first-party member; reword "vendored" -> "first-party" pi-beep is one of our crates now, so build it like startos-backup-fs (a dedicated build-pi-beep.sh zig build of the workspace member) instead of routing it through build-cargo-dep.sh. That script is now only for the genuinely external crates.io dev tools (tokio-console, flamegraph) bundled into unstable/console images. Also reword the patch-db docs: these are our own crates, so "first-party crate" is more accurate than "vendored" (which implies a third-party copy). * ci: path-gate the per-product build workflows to their project + deps The start-cli / start-registry / start-tunnel / startos-iso build workflows ran on every push/PR (only skipping doc-only changes), so all four built regardless of what changed. Replace the blanket paths-ignore with a paths: allowlist scoped to each product plus its dependencies (start-core + the in-repo shared-libs crates, Cargo manifests, build infra, and — for the web-bearing/OS workflows — the Angular workspace and SDK). workflow_dispatch / workflow_call are kept so manual and orchestrated runs still fire unconditionally. * ci: migrate shared-workflows (service-package CI) into the monorepo Bring the reusable .s9pk build/release workflows and their composite actions in-repo from the standalone Start9Labs/shared-workflows repo, so the packaging toolchain lives alongside the SDK: - .github/workflows/{build,release,tagAndRelease}.yml (reusable, workflow_call) - .github/actions/{extract-version,free-disk-space,setup-build-env, setup-publish-env,upload-each} Their internal references (and the SDK package-template's three workflows + the packaging docs) are repointed from start9labs/shared-workflows@master to Start9Labs/start-os@master. These are workflow_call-only, so they don't run for the monorepo itself — they activate once this lands on master and external service-package repos repoint their `uses:` to Start9Labs/start-os. * docs(monorepo): document tandem-update couplings The per-product CI `paths:` filters mirror each product's build.mk prerequisites by hand — nothing enforces it. Add a "Coupled changes" section to the root AGENTS.md and reciprocal pointers in each gated workflow and its build.mk, so a change to one half is caught at the other. Also catalogs the remaining hand-mirrored pairs (reusable service-package CI <-> SDK package-template <-> packaging docs; the files touched when adding a product/crate) and the already-enforced couplings (ts-bindings, the five i18n locales, the UI beta seed, version <-> CHANGELOG, docs <-> user-facing changes). * chore(manpages): generate man pages into their product projects The export_manpage_* tests in start-core wrote every product's man pages into start-core's own man/ dir. Point each generator at the owning product's man/ dir (anchored to CARGO_MANIFEST_DIR), move the committed pages there, and update build-manpage.sh's chown and the docs. start-container's pages go to projects/start-os, since that bin is part of the StartOS product. * Retitle README * refactor(shared-libs): rename web -> ts-modules Mirror the `crates/` naming: the shared TS/Angular workspace dir becomes `shared-libs/ts-modules/`. Pure path rename — repoints every reference (angular.json, root tsconfig/package.json scripts, the Makefile include + build.mk, CI `paths:`, and docs). No code changes. * fix(monorepo): repoint stale paths from the projects/ move that broke CI Two classes of path left stale by nesting products under projects/: - Web apps `require()` repo-root config.json/package.json by relative path; the extra projects/ level meant every one was short one `../` (resolved to projects/… instead of the repo root), failing the esbuild UI build for start-os (ui + setup-wizard) and start-tunnel. - test.yaml and deploy-brochure.yml still `cd start-sdk` for the baseDist build; the SDK now lives at projects/start-sdk. * fix(debian): resolve PROJECT_DIR before reading VERSION debian/build.sh computed VERSION from "projects/$PROJECT_DIR/Cargo.toml" before PROJECT_DIR was assigned, so it read projects//Cargo.toml (empty) and double-prefixed projects/. The empty Version: produced an invalid DEBIAN/control and dpkg-deb rejected it — breaking the registry and tunnel .deb builds. start-cli's CI only runs `make cli` (the binary), so it never exercised this path. Hoist the PROJECT_DIR/INSTALL_TARGET block above VERSION and read "${PROJECT_DIR}/Cargo.toml" directly. Also only fall back to the OS product's usr/lib/startos/conflicts when that file actually exists, so non-OS products don't error on a missing conflicts file. * fix(web): prettier-wrap tsconfig paths widened by the ts-modules rename Renaming shared-libs/web -> shared-libs/ts-modules pushed the `@start9labs/*` path-mapping lines past prettier's print width, so `npm run format:check` (the Formatting & Lockfiles CI job) flagged the four product tsconfig.json files. Apply prettier's wrapping. * docs(contributing): align with restructure + renamed make recipes Bring the CONTRIBUTING set up to date with the monorepo layout and the namespaced make targets: - root: `make iso` -> `make startos` - start-os: `make $(IMAGE_TYPE)`/`deb`/`squashfs` -> `make startos-$(IMAGE_TYPE)`/`startos-deb`/`startos-squashfs`; deploy/flash targets -> `startos-update*`/`startos-wormhole*`/`startos-emulate-reflash` - start-core: `cd start-sdk` -> `cd projects/start-sdk`; osBindings sync path -> projects/start-sdk/base/lib/osBindings - shared-libs: `web/` -> `ts-modules/` (heading, lib paths, file: deps, cross-links) The other products' CONTRIBUTING files were already correct. * fix(container-runtime): correct repo-root target path in update-image.sh update-image.sh runs with cwd at projects/start-os/container-runtime/ (mounted at /root/start-os in start9/build-env), so the repo-root build output is three levels up. It copied start-container from ../../target (-> projects/target, nonexistent), so the container-runtime squashfs was never built and the OS image compile failed on every arch. Use ../../../target. Refresh the AGENTS.md gotcha that described the old stale path. * refactor(make): namespace OS web targets, require an explicit target, per-project cleans - `ui`/`uis` -> `startos-ui`/`startos-uis`: they build only the StartOS admin UI + setup-wizard, so they belong under the startos-* namespace. Also fix `startos-ui` to depend on the built index.html (the old `ui` depended on a directory with no rule). - No default build: bare `make` now prints `help` (.DEFAULT_GOAL := help) and the misleading `all` target (it only built `startos`, not "everything") is removed — callers specify a target. - Decentralize `clean`: every build.mk owns a `clean-<project>` target and the root `clean` just aggregates them. Per-project cleans use project-prefix wildcards so they're arch/version-independent, and two stale paths from the projects/ move are corrected (env/*.txt -> build/env/*.txt; image-recipe/deb -> projects/start-os/build/image-recipe/deb). - Drop the start-cli targets (`make cli`/`cli-deb`) from the start-os CONTRIBUTING build section (wrong product) and update the docs to the new target names (root + start-os CONTRIBUTING/README/AGENTS). * docs: sync, complete, and standardize all developer docs for the monorepo (#3356) * docs: sync developer docs with the monorepo restructure Audit of all developer documentation (AGENTS/CONTRIBUTING/ARCHITECTURE/README across root, projects/*, shared-libs/*) against the post-restructure tree. Corrects stale references the restructure left behind: - Paths still pointing at the pre-restructure layout (core/, web/, sdk/, brochure/, container-runtime/, patch-db submodule) -> projects/* and shared-libs/*. - Angular workspace root: several docs claimed shared-libs/ts-modules holds angular.json/package.json/tsconfig.json and that npm runs from there. The workspace is rooted at the repo root; fixed cwd/--prefix instructions, the tsconfig path-alias targets, and config-sample.json location accordingly. - Renamed make targets (startos-* namespace), the Rust lib rename (startos -> start_core / crate start-core), and the binary-source table in start-core/ARCHITECTURE.md. - Removed stale 'git clone --recursive' (no submodules remain) and the non-existent repo-level scripts/ reference. - Normalized product self-references and the brochure -> brochure-marketplace Angular project name; verified incidental accuracy fixes (exver 0.2.1, patch-db serde_cbor, image-recipe live-build). All relative doc links verified resolvable. Pre-commit lint-staged hook skipped (--no-verify): the slot has no installed node_modules so the binary can't run, and the repo's prettier targets only the web source dirs, not these markdown docs. * docs: bring utility crates and patch-db up to the standard doc set Every first-party crate under shared-libs/crates/ now carries the same AGENTS/ARCHITECTURE/CONTRIBUTING/README/CLAUDE set the projects and other shared libs use. - New full doc sets for exver, imbl-value, jsonpath, pi-beep, rpc-toolkit, yasi, written from each crate's actual source (verified module names, public API, cargo -p <package> commands, and real consumers; jsonpath's package is jsonpath_lib though its dir is jsonpath). Pre-existing READMEs (exver, jsonpath, yasi) were preserved verbatim and only augmented with a 'Place in the monorepo' + 'Documentation' section. - patch-db: added AGENTS.md (migrated from its content-bearing CLAUDE.md, plus a Build & test section noting it is its own Cargo workspace) and reduced CLAUDE.md to the one-line @AGENTS.md import like every other scope. - Every CLAUDE.md is exactly '@AGENTS.md'; all relative doc links resolve. * docs: normalize section structure across all project & shared-lib docs Standardize every first-class scope (root, the 7 projects, container-runtime, shared-libs, start-core, ts-modules) onto one canonical section template so the same sections appear under the same names in the same order everywhere: - AGENTS.md: intro (+ 'CLAUDE.md is a one-line @AGENTS.md import' + cross-links) → Layout → Build & test (run from the repo root) → Gotchas → scope-specific. Folded synonym headings (Where things are / What lives where / Map → Layout; Operating rules / Gotchas / rules → Gotchas; Build / Build & verify → Build & test). - CONTRIBUTING.md: Documentation (doc-map, added where missing) → Prerequisites → Building → Testing → Formatting → scope-specific → Commits / PRs (last). - ARCHITECTURE.md: ensured a 'Place in the monorepo' opener (where applicable) and a 'Further reading' links footer; internal sections left in place. - README.md: added a Documentation/links footer where missing; landing prose left untouched. Pure reorganization: headings renamed/reordered and short missing sections added — no prose reworded, no facts/commands/tables dropped. Scope-accurate build cwds preserved (start-docs builds from its dir; start-sdk from its dir; container-runtime via npm --prefix; patch-db its own workspace). All 102 docs' relative links verified resolvable; every CLAUDE.md remains the one-line import. * build+docs: merge patch-db into root workspace; enforce doc hierarchy Addresses review feedback on the docs PR: - **patch-db → root Cargo workspace.** Add the six patch-db crates (core, json-patch, json-ptr, macro, macro-internals, util) to the root workspace `members` and drop the `exclude`; remove patch-db's now-redundant virtual `Cargo.toml`. `cargo metadata` + `cargo check -p patch-db -p json-patch -p json-ptr -p patch-db-macro` pass. Updated the docs that called patch-db 'its own workspace' (patch-db AGENTS/CONTRIBUTING build commands now run from the repo root with `-p`; root ARCHITECTURE lists it as a member). - **Root AGENTS.md:** added a note that these doc files must be kept current with every change, and that the docs are hierarchical — a scope documents only what is specific to it and never repeats higher-scope content. - **Hierarchy cleanup:** removed the `## Commits / PRs` (and `Branch / commit / PR` / `Commit conventions`) sections from every non-root CONTRIBUTING.md — those conventions live only in the root CONTRIBUTING.md now. Scope-specific change steps were preserved (e.g. start-tunnel's migration/ bindings/CHANGELOG steps moved to a 'Making a change' section; start-cli's docs-update note folded into 'Where code lives'). Also dropped the duplicated 'keep these docs in sync' line from child Documentation sections (kept the local cross-link lists). - **jsonpath:** reframed the fork note as history-only across its docs — it has fully diverged from freestrings/jsonpath with no intent to upstream; treat it as first-party and edit freely (removed the 'pull fixes from upstream / keep changes minimal / fork-tracking' guidance). All 102 docs' relative links resolve; every CLAUDE.md remains the one-line import. * docs: add hierarchy-navigation notes to AGENTS files - Every non-root AGENTS.md now opens with a 'Read up the tree first' note: the docs are hierarchical, so before working in a scope read the AGENTS.md of each enclosing directory up to the repo root (and their ARCHITECTURE/CONTRIBUTING where relevant). Excludes the packaging-guide + package-template AGENTS under projects/start-sdk/docs/, which target external package authors, not the monorepo dir tree. - Root AGENTS.md gains the converse 'Read down into what you touch' note: read a subdirectory's AGENTS.md (and any further nested ones) before editing it. - Dropped the explicit repo-root CONTRIBUTING pointer from patch-db's CONTRIBUTING.md (the walk-up convention now covers it). * docs(agents): require product docs/ book + CHANGELOG to ship with code Root AGENTS.md now mandates that any change altering user-visible behavior update that product's user-facing docs/ book (projects/<product>/docs/) in the same change and add a CHANGELOG.md entry — no deferring docs/changelog to follow-ups. * build(make): add per-project format targets mirroring the clean decomposition Each build.mk now owns `format-<project>` + `format-check-<project>` (core, web, sdk, cli, registry, tunnel, startos), matching the per-project `clean-<project>` targets; the top-level `format`/`format-check` just aggregate them. Web (the whole Angular workspace incl. brochure) formats via the root npm script; the shared crates via one `cargo +nightly fmt`; container-runtime via its own prettier config (new `format`/`format:check` npm scripts). So you can format one project (`make format-cli`) or all (`make format`). * docs: address PR review — make-target refs, ARCH de-dup, ts-modules wording - Reference stable make targets instead of raw build/format commands across project docs: builds via `make cli`/`registry`/`tunnel`/`startos`/`startos-ui`, formatting via the per-project `make format-<project>` targets (and `format-check-<project>` for CI). Kept `cargo check`/`cargo test` and crate `cargo build` (no make equivalent) as noted dev shortcuts. - Fixed stale targets: dropped `all` (removed upstream; `make` now prints help), `ui`/`uis` -> `startos-ui`/`startos-uis`. - Removed the whole-monorepo ASCII trees that several product ARCHITECTURE.md files re-drew (start-cli, start-registry, container-runtime, ...) — that layout lives once in the root ARCHITECTURE.md; each now states only where it sits. - Reworded the `shared-libs/ts-modules` directory as shared TypeScript modules (not Angular-specific; current contents are the Angular libs shared/marketplace), per review; kept accurate per-library 'Angular library' phrasing. Branch merged up to date with docs/monorepo-proposal first. * refactor(sdk): extract base into @start9labs/start-core shared lib; flatten start-sdk Move projects/start-sdk/base -> shared-libs/ts-modules/start-core (package @start9labs/start-sdk-base -> @start9labs/start-core), mirroring the Rust crate shared-libs/crates/start-core. start-core builds its own self-contained dist consumed via file: deps. Flatten start-sdk (package/ -> root): the SDK now imports @start9labs/start-core instead of ../../base/lib, and its published dist bundles start-core (bundleDependencies) so external authors still install one package. Repoint all consumers off the SDK-as-base alias onto @start9labs/start-core: - web (root file: dep + 153 import sites + shared/marketplace peerDeps) - container-runtime (file: dep; base/lib imports -> start-core, package/lib -> lib) Repoint osBindings generation, the build DAG (build.mk fragments, Makefile), and regenerate the three lockfiles. Resolves the "SDK kept cohesive" deviation: base is now an honest first-class shared TS lib named for what it is. * docs+ci: reflect start-core extraction; repoint SDK build steps off base/baseDist CI: build start-core (cd shared-libs/ts-modules/start-core && make dist) before validating the SDK + web lockfiles; validate the flattened SDK lockfile at projects/start-sdk; add start-core to the prettier check; repoint the iso prevent-rebuild mkdirs and the brochure deploy paths: filter (start-sdk -> start-core). Docs: update every project's AGENTS/ARCHITECTURE/CONTRIBUTING/README for the new layout — base extracted to @start9labs/start-core under shared-libs/ts-modules, the SDK flattened (lib/) and bundling start-core, container-runtime depending on both. Removes the resolved "SDK kept cohesive" deviation note. * refactor(monorepo): use the start-technologies name; repoint init-workspace at the monorepo - Adopt start-technologies for monorepo/repo-URL references across docs and AGENTS files (ahead of the GitHub repo rename; product refs left as start-os: projects/start-os, the start-os crate, *-startos packages, StartOS). - s9pk init-workspace now sparse-clones the start-technologies monorepo (projects/start-sdk/docs) instead of the retired standalone start-docs repo, and the two start-docs-named init strings are reworded across all five locales. - Un-hide the SDK 2.0 packaging-workspace section and rewrite it for the monorepo clone; document fetch-on-demand SDK + OS source access (docs first) in the workspace AGENTS.md and workflow.md. - Fix build-config.js to read and write the repo-root config.json. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FkwYSXK8Uh9D16UTffCNJ1 --------- Co-authored-by: Aiden McClelland <me@drbonez.dev> Co-authored-by: Matt Hill <9935159+MattDHill@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> | 2 个月前 | |
fix(startos): keep lxc-net + mounts up through the graceful teardown on host reboot/poweroff (#3442) * fix(startos): order graceful-teardown hooks after lxc.service/lxc-monitord On a host reboot/poweroff, systemd stopped lxc.service (whose ExecStop is Debian's `lxc-containers stop`) and lxc-monitord concurrently with the graceful-teardown hooks, so each container was killed before startd's Exit RPC could reach it — `lxc-stop ... is not running` / `Connection refused (os error 111)` and a stalled reboot. Order startos-{restart,shutdown}.service After=lxc.service lxc-monitord.service so the LXC container-stop infrastructure stays up until the teardown has finished. Validated on a beta.10 VM with a real service: clean teardown, no 111, hook <1s. * fix(startos): exit startd promptly when systemd is driving the shutdown When the teardown runs inside a systemd shutdown transaction (the hooks' ExecStop), startd must not re-issue its own reboot/poweroff — that force-reboots mid-transaction. It now detects `systemctl is-system-running == stopping` and exits at the top of Shutdown::execute, before the `systemctl stop systemd-journald` and volume-group export that block mid-transaction (which otherwise hung startd until its ~90s stop timeout); systemd finishes the unmounts and power action. Raspberry Pi poweroff still self-drives its standby reboot, since systemd can't power the board off. | 2 个月前 | |
fix(startos): keep lxc-net + mounts up through the graceful teardown on host reboot/poweroff (#3442) * fix(startos): order graceful-teardown hooks after lxc.service/lxc-monitord On a host reboot/poweroff, systemd stopped lxc.service (whose ExecStop is Debian's `lxc-containers stop`) and lxc-monitord concurrently with the graceful-teardown hooks, so each container was killed before startd's Exit RPC could reach it — `lxc-stop ... is not running` / `Connection refused (os error 111)` and a stalled reboot. Order startos-{restart,shutdown}.service After=lxc.service lxc-monitord.service so the LXC container-stop infrastructure stays up until the teardown has finished. Validated on a beta.10 VM with a real service: clean teardown, no 111, hook <1s. * fix(startos): exit startd promptly when systemd is driving the shutdown When the teardown runs inside a systemd shutdown transaction (the hooks' ExecStop), startd must not re-issue its own reboot/poweroff — that force-reboots mid-transaction. It now detects `systemctl is-system-running == stopping` and exits at the top of Shutdown::execute, before the `systemctl stop systemd-journald` and volume-group export that block mid-transaction (which otherwise hung startd until its ~90s stop timeout); systemd finishes the unmounts and power action. Raspberry Pi poweroff still self-drives its standby reboot, since systemd can't power the board off. | 2 个月前 |
StartOS
StartOS is an open-source Linux distribution for running a personal server. It
handles discovery, installation, network configuration, data backup, dependency
management, and health monitoring of self-hosted services. Services run in
isolated LXC containers, packaged as signed, merkle-archived .s9pk files.
This directory is the StartOS OS product within the monorepo. It is a thin wrapper that ships the OS:
- the
startboxdaemon binary (startd/start-climulti-call applet) and thestart-containerbinary that runs inside each service LXC, - the web UIs (
ui= admin dashboard,setup-wizard= first-boot setup), - the
container-runtime(Node.js service runtime that runs inside package LXCs), - the systemd units and OS image packaging glue.
The bulk of the backend logic lives in the shared start-core crate
(../../shared-libs/crates/start-core), and the Angular apps here consume the shared web
libraries under ../../shared-libs/ts-modules and the SDK base from ../start-sdk.
Tech stack
- Backend: Rust (Tokio async, Axum), built on the
start-corecrate. - Frontend: Angular 22 + Taiga UI 5 (apps live under
web/, shared libs under../../shared-libs/ts-modules). - Container runtime: Node.js/TypeScript managing LXC service containers.
- State/sync: Patch-DB (
../../shared-libs/crates/patch-db) — diff-based store that pushes CBOR diffs to the frontend over WebSocket for reactive, poll-free UI updates.
Layout
start-os/
├── src/bin/startbox.rs # startd + start-cli multi-call applet; embeds the UIs
├── src/bin/start-container.rs # runs inside each service LXC
├── Cargo.toml # package "start-os" → depends on start-core
├── web/
│ ├── ui/ # admin dashboard (Angular app "ui")
│ ├── setup-wizard/ # first-boot setup (Angular app "setup-wizard")
│ └── patchdb-ui-seed*.json # seed state embedded into startbox
├── container-runtime/ # Node.js LXC service runtime (own README/AGENTS)
├── docs/ # mdbook (book title "StartOS"); served at /start-os/
├── startd.service # systemd units + cgroup slice for service containers
├── services.slice
├── startos-shutdown.service
└── startos-restart.service
OS-image packaging shared across products lives at the repo root: debian/
(maintainer scripts), apt/, and build/ (image-recipe, firmware, env scripts);
this product's own assets live in assets/ here. The root Makefile drives the
full ISO/img build.
Quickstart
Build commands run from the repo root (one Cargo workspace, one Angular workspace). See AGENTS.md for environment setup (including the OS-image toolchain) and the full build/test workflow.
# from the repo root
cargo check -p start-os # verify the OS bins compile
make start-os-ui # build the admin UI
make start-os # build all OS artifacts (bins + web + runtime image)
make start-os-$(IMAGE_TYPE) # build the bootable image (iso, or img on Pi)
make targets relevant to this product: start-os-ui, start-os-uis, start-os, start-os-deb,
start-os-install, start-os-$(IMAGE_TYPE), start-os-squashfs,
start-os-update/start-os-emulate-reflash/start-os-wormhole* (deploy to a live
device — slow and destructive). Run make test for the full Rust + SDK + runtime
test suite.
Documentation
- ARCHITECTURE.md — how the OS product is structured and how data flows backend → frontend.
- AGENTS.md — build/test/format workflow and operating rules for AI developers working here.
- CHANGELOG.md — release history.
- End-user docs: docs.start9.com (built from
docs/).
Getting StartOS as a user
- Buy a server from store.start9.com — easiest path.
- Install on your own hardware — follow the install guide.
- Browse services on the Start9 Marketplace.
To report security issues, email security@start9.com. See SECURITY.md for what is in scope and what happens next.