| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
fix(key): restrict overwritten key exports to owner-only permissions (#11428) * fix(key): restrict overwritten key exports to owner-only permissions Signed-off-by: questfever <questfever@outlook.com> * fix(atomicfile): temp file leak and name limit Both problems surface through `ipfs key export`, which now writes through this helper, but they affect every caller: config writes, repo migrations and `ipfs update`. - remove the temporary file when the rename fails, so one holding private key material is not left next to the target - keep the ".tmp-" prefix and the random suffix within the 255 byte file name limit, so a target with a long name can still be written * fix(key): route key export by target type Choosing the write path with os.Lstat treated /dev/stdout, /dev/stderr and /dev/fd/N as plain paths, because they are symlinks into /proc/self/fd, so the atomic write failed on targets that worked before. Decide by what the path resolves to, and state the whole contract in the command help. - regular file or nothing yet: written to a temporary file and renamed over the target, following symlinks, including one whose target does not exist yet - character device or pipe: streamed in place, confirmed on the open descriptor and without O_TRUNC, so a path swapped for a regular file can neither receive the key nor be emptied - anything else: refused, naming the path - errors name the file the user asked for, and the temporary file is flushed before the rename * fix(key): stop export landing on the wrong file An export could replace a file the target symlink does not point at. resolveSymlink joined a relative link target onto the path as typed, so ".." collapsed lexically. Where a parent component was itself a symlink, the join named a file outside the directory the link resolves to: the key was renamed over that file, and the intended target was never written. The link's parent is now resolved with filepath.EvalSymlinks before the join. * test(key): drop umask dependency in export test os.WriteFile applies the umask, so under umask 077 the fixture was created 0600 and the check that a failed export leaves the file at 0644 failed for reasons unrelated to the code under test. The CLI test already chmods for the same reason. --------- Signed-off-by: questfever <questfever@outlook.com> Co-authored-by: Marcin Rataj <lidel@lidel.org> | 22 天前 | |
chore: bump go-libp2p-kad-dht to v0.42 (#11398) * chore: bump go-libp2p-kad-dht to v0.42.0 * bump kad-dht * fix(init): stop publishing an empty-directory IPNS record * bump kad-dht to v0.42.1 * bump boxo * chore: bump boxo, clarify IPNS storage changelog Bumps boxo, bringing the offline router retention change (records are served until their EOL, no receive-age cap by default; ipfs/boxo#1189) and a gateway 304 cache-freshness fix (ipfs/boxo#1188). The v0.43 "Unified IPNS record storage" highlight now spells out how retention works: with the offline router and the DHT sharing the /dht datastore prefix, online nodes drop value records 48h after storage (the DHT's value-store GC) while offline nodes keep them until EOL. * fix: sync DHT purge once after deletion * bump boxo * depend on boxo@master * chore: bump boxo to main, align module pins Pick up boxo main HEAD (4794174d), which includes the merged offline router value-store change (ipfs/boxo#1189). This brings the root module, the kubo-as-a-library example, and test/dependencies onto one boxo pin; they previously pointed at three different commits. - go-libp2p moves to ec408fcc as a transitive floor required by boxo - changelog: note the one-time full-datastore key scan on first start after upgrading, and update the pinned boxo and go-libp2p hashes * chore: require make mod_tidy before commit/push Spell out that the repo has three go.mod files that must stay on the same dependency versions, and that a bare `go mod tidy` only tidies one module and lets pins drift between them. --------- Co-authored-by: Marcin Rataj <lidel@lidel.org> | 1 个月前 | |
fix: Ipfs-Uri gateway header (IPIP-548) (#11437) * feat: Ipfs-Uri gateway header (IPIP-548) Bump boxo to the IPIP-548 implementation (ipfs/boxo#1209): gateway responses carry a canonical percent-encoded Ipfs-Uri header and stop sending the deprecated X-Ipfs-Path, which cannot represent every UnixFS file name. - sharness: CORS expects Ipfs-Uri exposed, X-Ipfs-Path gone - gateway-conformance CI pinned to the IPIP-548 test suite (ipfs/gateway-conformance#301) until a release ships - reverse-proxy doc and v0.44 changelog updated Refs ipfs/specs#548 * feat: opt-in Gateway.DeprecatedXIpfsPath Expose boxo's opt-in for the legacy X-Ipfs-Path response header as a kubo config flag, default off. Unsafe: the legacy value cannot represent every UnixFS file name, so it must only be used to facilitate migration to Ipfs-Uri, and even when enabled the header is still skipped when the value would include non-ASCII byte sequences. Refs ipfs/specs#548 * ci: bump gateway-conformance pin * ci: gateway-conformance v0.14 * chore: boxo with IPIP-548 from boxo/main * docs: Ipfs-Uri changelog in v0.43.1 * docs: assemble v0.43.1 changelog Move the v0.44 highlights and dependency lines into a new v0.43.1 section, and add the missing entry for owner-only key exports (#11428). v0.44.md returns to an empty skeleton. * docs: note boxo v0.42.2 fixes in v0.43.1 | 9 天前 | |
fix(key): restore secp256k1 keygen and add PEM PKCS8 import/export (#11387) * fix(key): restore secp256k1 keygen and add PEM PKCS8 import/export * fix(key): validate --size for fixed-size key types ed25519 and secp256k1 keys have a single valid size, so a --size (--bits for init) that does not match it is now an error instead of being accepted and ignored. core/coreapi Generate and config.CreateIdentity share a new options.CheckKeySize helper, so key gen, key rotate, and init accept --size only when it equals the fixed 256 bits. RSA keeps its variable size. * test(cli): cover key lifecycle for all key types Adds end-to-end CLI coverage of the key commands (gen, list, export, import, rename, rm, rotate) for rsa, ed25519, and secp256k1. This mirrors the sharness keystore and rotate suites and extends them to secp256k1, which they never exercised. - OpenSSL fixtures under testdata/ pin byte-identical PKCS#8 export and import in both directions - rotate checks the previous identity survives, usable, under the backup name - reserved-name ('self') and restricted-type imports assert the specific refusal, not just a non-zero exit * chore(deps): note secp256k1 version alignment go-libp2p/core/crypto's key types alias this package, so the direct and transitive pins must stay on one version to avoid two copies in the build. The go.mod comment flags that for future dependency bumps. --------- Co-authored-by: Marcin Rataj <lidel@lidel.org> | 1 个月前 | |
fix(mfs): stop repo gc from freezing files ops (#11386) * fix(mfs): stop repo gc from freezing files ops Running `ipfs repo gc` alongside `ipfs files` writes could leave MFS permanently hung: GC deleted directory-node blocks a write had written to the blockstore but not yet linked into the persisted MFS root, and the next path lookup blocked forever fetching the missing block while holding the MFS directory lock, so every later files command piled up behind it. MFS mutations now take the pin lock, the same lock `ipfs add` uses, and GC computes the MFS root only after it holds the GC lock. A write's blocks are therefore either fully linked into the root before GC runs, or the write waits for GC to finish, so GC never collects data a live write still needs. - core/commands/files.go: hold the pin lock across write, cp, mkdir, mv, rm, flush, chcid, chmod, and touch - gc/gc.go: take the best-effort MFS root snapshot after acquiring the GC lock, closing the snapshot-before-lock window - core/corerepo/gc.go: pass the root as a callback evaluated under the lock - core/node/core.go: document why MFS keeps its online DAG service so lazy `ipfs files cp /ipfs/<cid>` pointers still resolve - gc/gc_test.go: assert the root snapshot is taken under the GC lock Closes #10842 * fix(mfs): extend gc pin lock to add and fuse The pin lock that stops garbage collection from collecting live MFS blocks now covers every path that mutates MFS, not just `ipfs files`, and every live MFS root is part of the GC live set. - core/commands/add.go: hold the pin lock while `ipfs add --to-files` links the added content into the MFS root - fuse/writable, fuse/mfs: hold the pin lock across FUSE `/mfs` structural writes and file flush, fsync, and release - fuse/ipns: same for the per-key `/ipns` mounts, and register their roots so GC keeps their blocks live - core/core.go: track mounted MFS roots for the GC live set - core/corerepo/gc.go: build the live set from the files root plus every registered root, skipping a root that errors instead of aborting the whole GC Closes #6113 Closes #7008 Closes #9553 * fix(mfs): fail fast on a missing block A directory-node block that is missing locally and unreachable would block an MFS operation forever while it held the directory lock, wedging the whole MFS and a clean shutdown until the process was killed. This is what remained after a repo was damaged by an older Kubo, a manual `ipfs block rm`, or a crash (the lockup in #7844). MFS now bounds those network reads: an unreachable block fails the operation with a timeout and releases the lock, while lazily-referenced content (`ipfs files cp /ipfs/<cid>`) still loads as before. - config: add DefaultMFSFetchTimeout and pass it to the MFS root via mfs.WithFetchTimeout in Import.MFSRootOptions - go.mod: bump boxo to pull in mfs.WithFetchTimeout * feat(mfs): honor --timeout in files read and write Adopt boxo's mfs.File.Open(ctx) so MFS file operations carry a context and a read or write stuck on a missing block can be cancelled instead of blocking forever. - go.mod: bump boxo to pick up mfs.File.Open(ctx) - core/commands/files.go: pass the request context to files read and write, so a client --timeout ends a stuck read or write - fuse/writable: bind long-lived FUSE descriptors (Create, Open) to the mount context via the new Config.MountCtx; the transient Setattr truncate uses the per-operation context - fuse/mfs, fuse/ipns: set MountCtx to the node/mount context - test/cli: cover files read and write honoring --timeout on unreachable content * chore(deps): bump boxo to merged mfs fix Switch from the pre-merge branch pseudo-version to the merged commit of ipfs/boxo#1185, the boxo side of the MFS under-lock work this branch depends on. * refactor(fuse): pass request ctx to pin lock Flush, Release, and Fsync passed context.Background() to the pin lock while their own ctx argument was in scope. Thread the passed ctx through instead, matching the other handlers. boxo's default GCLocker discards the context, so this is a consistency change, not a behavior change. https://github.com/ipfs/kubo/pull/11386#discussion_r3534131824 https://github.com/ipfs/kubo/pull/11386#discussion_r3534138912 https://github.com/ipfs/kubo/pull/11386#discussion_r3534140861 Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> * refactor(gc): const for buffer, simplify roots Name the GC result channel buffer size as a const, drop the temporary in the best-effort root snapshot, and use t.Context() in the snapshot test. The buffer size predates this branch; it only shifted in the diff. https://github.com/ipfs/kubo/pull/11386#discussion_r3534154606 https://github.com/ipfs/kubo/pull/11386#discussion_r3534077507 https://github.com/ipfs/kubo/pull/11386#discussion_r3534085418 https://github.com/ipfs/kubo/pull/11386#discussion_r3534106766 Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> * refactor(add): collapse pin lock defer Fold the pin lock's unlock into the defer at both --to-files call sites, matching the one-line style used elsewhere. https://github.com/ipfs/kubo/pull/11386#discussion_r3534006489 https://github.com/ipfs/kubo/pull/11386#discussion_r3534013057 Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> | 1 个月前 | |
test: fix flakes that force CI re-runs (#11413) * fix(routing): keep peers found before the timeout The DHT returns the closest peers it reached together with the context error when a lookup runs past its deadline. We dropped both, so any lookup slower than the routing server's per-request timeout came back as HTTP 500 with nothing in it, indistinguishable from a lookup that found no peers at all. Return what we have, and only error when the set is empty. * test: use local dht swarm for routing v1 test GetClosestPeers joined the public Amino DHT with real bootstrap peers, so the assertions depended on a CI runner reaching bootstrap.libp2p.io from a cold repo. When it could not, the test retried for five minutes and failed; ten such failures since v0.42.0, every one green on re-run. Bootstrap from the harness's in-process DHT peers instead, which the provider tests already use and this one predates. The window drops from five minutes to sixty seconds because there is no longer anything slow to wait for, and passing runs go from tens of seconds to under one. * test: stop handing out ports the kernel reuses NewRandPort binds port zero, notes the number, closes the socket and hands the number to the caller, which leaves a window for anything else on the machine to take it. The number also came from the ephemeral range, the same pool every outgoing connection draws from, and the CLI suite opens a lot of those. Both TestP2PForeground tunnel subtests died on "bind: address already in use" for a server the test binds itself. - NewTCPListener hands back the bound listener, closing that window for callers that listen in-process - ports for daemons we spawn now come from below the ephemeral range, so an outgoing connection cannot land on one * test: sync gc tests to the adder, not the clock TestAddGCLive asserted that gc had not started yet, but the only thing it waited for was the first file's output event. Between that event and the adder reaching the next file there is a gap, and the adder hands the pin lock to a waiting gc at exactly that boundary, so on a loaded runner gc really had started and the assertion was right to fail. Wrap the pipe so the test learns when the adder is inside the hanging file, and poll GCRequested instead of sleeping 100ms to know gc is queued. TestAddMultipleGCLive gets the same treatment for its two sleeps: too short there means gc never gets the lock and the test waits out its five second timeout instead. * test: move watched file in atomically os.WriteFile creates the file and fills it in two steps, and ipfswatch adds whatever is on disk when the create event wakes it. Catch it between the two and it adds an empty file, so the CID the test pulls out of the log reads back as nothing. Stage the file outside the watched directory and rename it in, which the watcher sees as one event for a file that is already complete. * test(sharness): poll the daemon request log The test backgrounded "ipfs log tail", slept 100ms and expected the daemon to be listing the request. The daemon only sees it once the client has started up and connected, which on a loaded runner takes longer than that, and then both the active and the inactive assertion fail together because the entry never appears at all. Poll for each state instead. The extra requests that polling makes push the daemon closer to the point where it drops finished entries from the log, so keep them with "diag cmds set-time" first. * test(sharness): drop stale peer count check The connect case opened by re-asserting that the previous case had left zero peers connected. Disconnecting is not permanent: the DHT keeps the other node in its routing table and re-dials it on any refresh, so that count is only true for as long as nothing else runs. What this case is named for, connecting with a bare /p2p/ address, is still covered by the connect itself and the peer count after it. * test(fuse): mount one node at a time Every parallel subtest does identical setup before mounting, so they all reach the mount together and around twenty setuid fusermount helpers open /dev/fuse inside the same instant. One occasionally comes back with a bare exit status 1. Take a lock for the mount call itself, which the subtests only hold for tens of milliseconds. Also report the failure instead of panicking: a panic failed all 37 tests in the package and left daemons behind, and the daemon's stderr, where fusermount says what actually went wrong, was captured and then thrown away. * test: compare cat output byte for byte The payload is 100 random bytes and the comparison ran through Trimmed(), which strips one trailing newline. Roughly one run in 256 ends in 0x0a and loses it. * test: wait for the fast-provide log line The daemon writes the line before it answers the RPC, but the test reads a buffer that a goroutine fills by copying the daemon's stderr, and that copy can still be behind when the command returns. Wait for the line rather than assuming it has landed. * test: allow for ipns republish mid-test A minute after the daemon starts, the republisher re-signs every key and publishes it again, giving the same value a new signature and expiry. The test captured one PUT body and compared it byte for byte with what routing returned, so a run slow enough to straddle that minute compared the first record against the second. Keep every record the mock is sent and require that routing's answer is one of them, which is what the assertion was reaching for. * fix(examples): turn off mdns in library example The example connects its two nodes by address, but left mDNS on, so local discovery could connect them first. A connection opened while a node is still being built is invisible to that node's bitswap, which only learns about connections made after it registers its notifier, and with no routing configured there is nothing to fall back on. The final fetch then waited forever and the test died on its two minute timeout with no clue why. Turning mDNS off makes the explicit dial the only way the two can meet, and keeps the example off the reader's LAN. Alongside that: - connectToPeers returns dial errors instead of logging and continuing into a fetch that cannot succeed - the example's own deadline now fits inside the test budget, so a stall names the step that hung - CommandContext so a hung child does not outlive the test * ci: make helia-interop job resilient Seven failures since v0.42.0 came from this job's setup rather than from any incompatibility. It installs whatever @helia/interop published last, and upstream shipped three packages in a row whose test config does not work from inside node_modules; a GitHub blip took out the rest. - find the compiled specs and pass them to aegir, instead of patching the config upstream ships into node_modules and grepping its text - pin node to a major: setup-node resolves an lts/ alias through a GitHub manifest with no retry and no fallback, and newer node rejects a flag aegir sets unconditionally - retry the registry lookup and fail loudly, since the old one-liner could not fail and left an empty cache key behind - install the exact version the cache key names, and only save the cache once the install is known good - drop the playwright apt packages, unused since this job stopped running browser targets | 1 个月前 | |
fix: deadlock on retrieving WebTransport addresses (#9857) Co-authored-by: Marco Polo <git@marcopolo.io> | 3 年前 | |
fix(http-routing): keep browser transports in provider records (#11394) * chore: bump go-libp2p for sorted confirmed addrs Pin the head commit of libp2p/go-libp2p#3526: AutoNAT V2's ConfirmedAddrs returned unsorted buckets, and removeNotInSource silently dropped webrtc-direct from the confirmed set. Switch to a master pseudo-version once the PR merges. * fix: keep browser transports in provider records Provider records sent to HTTP routers were narrowed to the addresses AutoNAT V2 confirmed reachable, which silently dropped the only two transports a browser can dial: the AutoTLS /tls/ws address and webrtc-direct. A publicly reachable node was invisible to browser and Helia clients that found it through a delegated router, even though ipfs id and the DHT both advertised those addresses. AutoNAT only ever sees listen addresses, so the AutoTLS address, which the AddrsFactory synthesizes afterwards, can never reach the confirmed set. webrtc-direct does get confirmed, but go-libp2p loses it again in getConfirmedAddrs, which feeds an unsorted slice to a scan that assumes sorted input; that one is fixed upstream in libp2p/go-libp2p#3526. Announce host.Addrs() instead, the same set identify sends to peers and the DHT already publishes, narrowed to globally routable addresses so loopback and LAN entries stay out of a public index. Nodes with no public address keep announcing what they have, so LAN-only setups pointing at a local router are unaffected. - core/node/libp2p/routingopt.go: drop the ConfirmedAddrs branch from httpRouterAddrFunc, filter host.Addrs() with manet.IsPublicAddr; AppendAnnounce is emitted exactly once and does not count toward the public-addr check - core/commands/swarm_addrs_autonat.go: take over the BasicHost compile-time assertion, now the only ConfirmedAddrs consumer Fixes #11369 * docs: move highlight to v0.43 and scope it The fix ships in v0.43, so the entry moves out of v0.44.md and in next to the other browser-retrieval highlights. - names the config it applies to: Routing.Type=custom with a provide method on an HTTP router. Default auto provides over the DHT alone and is unaffected, since constructDefaultHTTPRouters leaves ProvideRouter as a noop. - cites bitsocial.net, which runs libp2p in the browser and uses delegated routers to find peers, as the app the gap broke | 1 个月前 | |
feat: bound graceful shutdown, add diag healthy (#11329) * feat: bound graceful shutdown, add diag healthy Replace unbounded app.Stop(context.Background()) with a deadline-bounded context driven by a new Internal.ShutdownTimeout config (default 12h, 0 disables). Add an os.Exit(1) watchdog at the same deadline so an FX OnStop hook that never returns can no longer hang the daemon. Add ipfs diag healthy: fails when shutdown has been initiated or when the DAG pipeline cannot resolve the well-known empty-directory CID. Dockerfile HEALTHCHECK now uses it so orchestrators recycle half- shutdown daemons. - core/shutdown: new pkg; atomic startedAt + CloseWithCtx helper - core/builder.go: app.Stop bounded by ShutdownTimeout - cmd/ipfs/kubo/daemon.go: watchdog + MarkStarted on signal - core/commands/diag.go: new healthy subcommand - core/node/{bitswap,libp2p/host,libp2p/routing}.go: OnStop hooks wrapped - config/internal.go: ShutdownTimeout + DefaultShutdownTimeout=12h - Dockerfile: HEALTHCHECK uses "ipfs diag healthy" - docs/{config,changelogs/v0.42}.md: documented - test/cli: enabled + disabled path tests * feat: bound provider stats and ADD_PROVIDER sends bumps go-libp2p-kad-dht past v0.39.2 to b73e1e8 to pick up two related provider bug fixes. - ipfs provide stat now honors client cancellation and deadlines instead of blocking indefinitely behind a slow keystore lookup - adds Provide.DHT.SendProviderRecordTimeout capping each ADD_PROVIDER RPC so unresponsive peers cannot pin a provide worker and stall reprovide cycles - internal reprovide-alert poller bounds its Stats call so a hung keystore.Size cannot delay shutdown * test(shutdown): use synctest for timeout test, document sleep CloseWithCtx_timesOut now runs in a synctest bubble so the deadline assertion is exact (no wall-clock slack), and the simulated close uses a release channel to drain the bubble cleanly after the leak point. The two happy-path tests stay unchanged because their close funcs return immediately and gain nothing from a fake clock. Comment the 2ms sleep in TestMarkStartedPreservesFirstTimestamp so its role (forcing time.Now() to advance between the two MarkStarted calls so a CAS to Store regression is detectable) is not lost. Addresses ipfs/kubo#11329 (review). * fix(pinner): bound pinner Close with shutdown deadline The boxo Pinner.Close contract notes that an in-flight op ignoring its ctx (a downstream bug) can block Close, so the host must bound it at the call site. Wrapping the OnStop hook with CloseWithCtx honors Internal.ShutdownTimeout and surfaces an actionable "subsystem 'pinner' failed to close" log on hang instead of leaving only the watchdog os.Exit(1) trace. * fix(shutdown): bound remaining I/O-touching OnStop hooks Wrap the OnStop hooks whose Close can plausibly block on disk or network: repo (datastore flush + lock release), mfs-root (datastore writes via DAGService), peering (waits on libp2p peer goroutines), legacy-provider (in-flight reprovide RPCs), and the dht-provider plus keystore pair under SweepingProvider. In-memory closes (blockservice, peerstore, resource-manager) are left as-is since they cannot realistically hang. For the dht-provider/keystore pair, provider closes first so nothing can access the keystore afterwards. If the shutdown ctx fires mid-provider-drain, the keystore close sees an expired ctx and returns immediately; the watchdog os.Exit(1) is the ultimate backstop, and keystore writes are fsync'd on put so missing the explicit close is recoverable on next boot. * fix(shutdown): bound remaining in-memory OnStop hooks Wrap blockservice, peerstore, and resource-manager Close hooks with CloseWithCtx for uniformity. These are pure in-memory operations unlikely to hang in practice, but wrapping costs nothing and makes the shutdown audit trail uniform: every OnStop hook now honors the deadline and surfaces a named subsystem on timeout. * fix(shutdown): bound autoRelayFeeder OnStop on ctx OnStop waited on the feeder goroutine via <-done without honoring the shutdown ctx. The goroutine itself selects on ctx in every loop case, so cancel() normally suffices, but a stuck downstream dht.WAN.GetClosestPeers that ignored its ctx could block fx.Stop indefinitely. Adding the ctx.Done() select case mirrors the reprovideAlert pattern in provider.go and lets the shutdown deadline reclaim control even with a misbehaving DHT. * docs(changelog): merge shutdown entries into one user-facing section Combine the pinner-on-shutdown paragraph with the bounded-shutdown section under a single "Reliable shutdown and container health checks" heading. Lead with the visible symptoms (half-shutdown daemons, healthy-but-dead container reports, manual docker restart) instead of fx OnStop jargon. Frame Internal.ShutdownTimeout as a belt-and-suspenders ceiling, with the 12-hour default sized against the 22-hour DHT provider record expiration. | 3 个月前 | |
ignore testdb | 11 年前 | |
chore(deps): bump go-libp2p and go-libp2p-pubsub (#11389) * chore(deps): bump go-libp2p and go-libp2p-pubsub Both are pinned to master pseudo-versions ahead of tagged releases, with a TODO in go.mod to switch once the tags ship. go-libp2p adds webrtc-direct v2 handshake support, keeps /certhash stable across restarts, and stops stale addresses from accumulating in the peerstore and in published signed peer records. It also fixes a data race that could take the daemon down mid-response during ipfs routing findprovs, findpeer, and dht query. go-libp2p-pubsub frees topic state once the last peer leaves a topic, closing an unbounded memory growth path (libp2p/go-libp2p-pubsub#705, the Go counterpart of CVE-2026-46679). Only nodes that opt into Pubsub.Enabled or Ipns.UsePubsub run the pubsub stack and are affected. * feat(config): Internal.NonPublicAddrPublishing Exposes go-libp2p's NonPublicAddrPublishing as an Internal flag, so a node can be told to stop publishing addresses the wider internet cannot reach: private, CGNAT, link-local, loopback, ULA, reserved IPv6, and special-use DNS names such as .local. Left unset, kubo passes no option and go-libp2p's default decides. The default has shifted before (libp2p/go-libp2p#3460), so the flag gives operators a way to pin the behavior, and makes it easy to see exactly what a node publishes while chasing a routing problem. Only the peerstore self-entry and the signed peer record are filtered. Listening and dialing are untouched, so `ipfs id` keeps reporting the full set. | 1 个月前 | |
feat: bound graceful shutdown, add diag healthy (#11329) * feat: bound graceful shutdown, add diag healthy Replace unbounded app.Stop(context.Background()) with a deadline-bounded context driven by a new Internal.ShutdownTimeout config (default 12h, 0 disables). Add an os.Exit(1) watchdog at the same deadline so an FX OnStop hook that never returns can no longer hang the daemon. Add ipfs diag healthy: fails when shutdown has been initiated or when the DAG pipeline cannot resolve the well-known empty-directory CID. Dockerfile HEALTHCHECK now uses it so orchestrators recycle half- shutdown daemons. - core/shutdown: new pkg; atomic startedAt + CloseWithCtx helper - core/builder.go: app.Stop bounded by ShutdownTimeout - cmd/ipfs/kubo/daemon.go: watchdog + MarkStarted on signal - core/commands/diag.go: new healthy subcommand - core/node/{bitswap,libp2p/host,libp2p/routing}.go: OnStop hooks wrapped - config/internal.go: ShutdownTimeout + DefaultShutdownTimeout=12h - Dockerfile: HEALTHCHECK uses "ipfs diag healthy" - docs/{config,changelogs/v0.42}.md: documented - test/cli: enabled + disabled path tests * feat: bound provider stats and ADD_PROVIDER sends bumps go-libp2p-kad-dht past v0.39.2 to b73e1e8 to pick up two related provider bug fixes. - ipfs provide stat now honors client cancellation and deadlines instead of blocking indefinitely behind a slow keystore lookup - adds Provide.DHT.SendProviderRecordTimeout capping each ADD_PROVIDER RPC so unresponsive peers cannot pin a provide worker and stall reprovide cycles - internal reprovide-alert poller bounds its Stats call so a hung keystore.Size cannot delay shutdown * test(shutdown): use synctest for timeout test, document sleep CloseWithCtx_timesOut now runs in a synctest bubble so the deadline assertion is exact (no wall-clock slack), and the simulated close uses a release channel to drain the bubble cleanly after the leak point. The two happy-path tests stay unchanged because their close funcs return immediately and gain nothing from a fake clock. Comment the 2ms sleep in TestMarkStartedPreservesFirstTimestamp so its role (forcing time.Now() to advance between the two MarkStarted calls so a CAS to Store regression is detectable) is not lost. Addresses ipfs/kubo#11329 (review). * fix(pinner): bound pinner Close with shutdown deadline The boxo Pinner.Close contract notes that an in-flight op ignoring its ctx (a downstream bug) can block Close, so the host must bound it at the call site. Wrapping the OnStop hook with CloseWithCtx honors Internal.ShutdownTimeout and surfaces an actionable "subsystem 'pinner' failed to close" log on hang instead of leaving only the watchdog os.Exit(1) trace. * fix(shutdown): bound remaining I/O-touching OnStop hooks Wrap the OnStop hooks whose Close can plausibly block on disk or network: repo (datastore flush + lock release), mfs-root (datastore writes via DAGService), peering (waits on libp2p peer goroutines), legacy-provider (in-flight reprovide RPCs), and the dht-provider plus keystore pair under SweepingProvider. In-memory closes (blockservice, peerstore, resource-manager) are left as-is since they cannot realistically hang. For the dht-provider/keystore pair, provider closes first so nothing can access the keystore afterwards. If the shutdown ctx fires mid-provider-drain, the keystore close sees an expired ctx and returns immediately; the watchdog os.Exit(1) is the ultimate backstop, and keystore writes are fsync'd on put so missing the explicit close is recoverable on next boot. * fix(shutdown): bound remaining in-memory OnStop hooks Wrap blockservice, peerstore, and resource-manager Close hooks with CloseWithCtx for uniformity. These are pure in-memory operations unlikely to hang in practice, but wrapping costs nothing and makes the shutdown audit trail uniform: every OnStop hook now honors the deadline and surfaces a named subsystem on timeout. * fix(shutdown): bound autoRelayFeeder OnStop on ctx OnStop waited on the feeder goroutine via <-done without honoring the shutdown ctx. The goroutine itself selects on ctx in every loop case, so cancel() normally suffices, but a stuck downstream dht.WAN.GetClosestPeers that ignored its ctx could block fx.Stop indefinitely. Adding the ctx.Done() select case mirrors the reprovideAlert pattern in provider.go and lets the shutdown deadline reclaim control even with a misbehaving DHT. * docs(changelog): merge shutdown entries into one user-facing section Combine the pinner-on-shutdown paragraph with the bounded-shutdown section under a single "Reliable shutdown and container health checks" heading. Lead with the visible symptoms (half-shutdown daemons, healthy-but-dead container reports, manual docker restart) instead of fx OnStop jargon. Frame Internal.ShutdownTimeout as a belt-and-suspenders ceiling, with the 12-hour default sized against the 22-hour DHT provider record expiration. | 3 个月前 | |
fix(mfs): stop repo gc from freezing files ops (#11386) * fix(mfs): stop repo gc from freezing files ops Running `ipfs repo gc` alongside `ipfs files` writes could leave MFS permanently hung: GC deleted directory-node blocks a write had written to the blockstore but not yet linked into the persisted MFS root, and the next path lookup blocked forever fetching the missing block while holding the MFS directory lock, so every later files command piled up behind it. MFS mutations now take the pin lock, the same lock `ipfs add` uses, and GC computes the MFS root only after it holds the GC lock. A write's blocks are therefore either fully linked into the root before GC runs, or the write waits for GC to finish, so GC never collects data a live write still needs. - core/commands/files.go: hold the pin lock across write, cp, mkdir, mv, rm, flush, chcid, chmod, and touch - gc/gc.go: take the best-effort MFS root snapshot after acquiring the GC lock, closing the snapshot-before-lock window - core/corerepo/gc.go: pass the root as a callback evaluated under the lock - core/node/core.go: document why MFS keeps its online DAG service so lazy `ipfs files cp /ipfs/<cid>` pointers still resolve - gc/gc_test.go: assert the root snapshot is taken under the GC lock Closes #10842 * fix(mfs): extend gc pin lock to add and fuse The pin lock that stops garbage collection from collecting live MFS blocks now covers every path that mutates MFS, not just `ipfs files`, and every live MFS root is part of the GC live set. - core/commands/add.go: hold the pin lock while `ipfs add --to-files` links the added content into the MFS root - fuse/writable, fuse/mfs: hold the pin lock across FUSE `/mfs` structural writes and file flush, fsync, and release - fuse/ipns: same for the per-key `/ipns` mounts, and register their roots so GC keeps their blocks live - core/core.go: track mounted MFS roots for the GC live set - core/corerepo/gc.go: build the live set from the files root plus every registered root, skipping a root that errors instead of aborting the whole GC Closes #6113 Closes #7008 Closes #9553 * fix(mfs): fail fast on a missing block A directory-node block that is missing locally and unreachable would block an MFS operation forever while it held the directory lock, wedging the whole MFS and a clean shutdown until the process was killed. This is what remained after a repo was damaged by an older Kubo, a manual `ipfs block rm`, or a crash (the lockup in #7844). MFS now bounds those network reads: an unreachable block fails the operation with a timeout and releases the lock, while lazily-referenced content (`ipfs files cp /ipfs/<cid>`) still loads as before. - config: add DefaultMFSFetchTimeout and pass it to the MFS root via mfs.WithFetchTimeout in Import.MFSRootOptions - go.mod: bump boxo to pull in mfs.WithFetchTimeout * feat(mfs): honor --timeout in files read and write Adopt boxo's mfs.File.Open(ctx) so MFS file operations carry a context and a read or write stuck on a missing block can be cancelled instead of blocking forever. - go.mod: bump boxo to pick up mfs.File.Open(ctx) - core/commands/files.go: pass the request context to files read and write, so a client --timeout ends a stuck read or write - fuse/writable: bind long-lived FUSE descriptors (Create, Open) to the mount context via the new Config.MountCtx; the transient Setattr truncate uses the per-operation context - fuse/mfs, fuse/ipns: set MountCtx to the node/mount context - test/cli: cover files read and write honoring --timeout on unreachable content * chore(deps): bump boxo to merged mfs fix Switch from the pre-merge branch pseudo-version to the merged commit of ipfs/boxo#1185, the boxo side of the MFS under-lock work this branch depends on. * refactor(fuse): pass request ctx to pin lock Flush, Release, and Fsync passed context.Background() to the pin lock while their own ctx argument was in scope. Thread the passed ctx through instead, matching the other handlers. boxo's default GCLocker discards the context, so this is a consistency change, not a behavior change. https://github.com/ipfs/kubo/pull/11386#discussion_r3534131824 https://github.com/ipfs/kubo/pull/11386#discussion_r3534138912 https://github.com/ipfs/kubo/pull/11386#discussion_r3534140861 Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> * refactor(gc): const for buffer, simplify roots Name the GC result channel buffer size as a const, drop the temporary in the best-effort root snapshot, and use t.Context() in the snapshot test. The buffer size predates this branch; it only shifted in the diff. https://github.com/ipfs/kubo/pull/11386#discussion_r3534154606 https://github.com/ipfs/kubo/pull/11386#discussion_r3534077507 https://github.com/ipfs/kubo/pull/11386#discussion_r3534085418 https://github.com/ipfs/kubo/pull/11386#discussion_r3534106766 Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> * refactor(add): collapse pin lock defer Fold the pin lock's unlock into the defer at both --to-files call sites, matching the one-line style used elsewhere. https://github.com/ipfs/kubo/pull/11386#discussion_r3534006489 https://github.com/ipfs/kubo/pull/11386#discussion_r3534013057 Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> | 1 个月前 | |
feat(cli/rpc/add): fast provide of root CID (#11046) * feat: fast provide * Check error from provideRoot * do not provide if nil router * fix(commands): prevent panic from typed nil DHTClient interface Fixes panic when ipfsNode.DHTClient is a non-nil interface containing a nil pointer value (typed nil). This happened when Routing.Type=delegated or when using HTTP-only routing without DHT. The panic occurred because: - Go interfaces can be non-nil while containing nil pointer values - Simple `if DHTClient == nil` checks pass, but calling methods panics - Example: `(*ddht.DHT)(nil)` stored in interface passes nil check Solution: - Add HasActiveDHTClient() method to check both interface and concrete value - Update all 7 call sites to use proper check before DHT operations - Rename provideRoot → provideCIDSync for clarity - Add structured logging with "fast-provide" prefix for easier filtering - Add tests covering nil cases and valid DHT configurations Fixes: https://github.com/ipfs/kubo/pull/11046#issuecomment-3525313349 * feat(add): split fast-provide into two flags for async/sync control Renames --fast-provide to --fast-provide-root and adds --fast-provide-wait to give users control over synchronous vs asynchronous providing behavior. Changes: - --fast-provide-root (default: true): enables immediate root CID providing - --fast-provide-wait (default: false): controls whether to block until complete - Default behavior: async provide (fast, non-blocking) - Opt-in: --fast-provide-wait for guaranteed discoverability (slower, blocking) - Can disable with --fast-provide-root=false to rely on background reproviding Implementation: - Async mode: launches goroutine with detached context for fire-and-forget - Added 10 second timeout to prevent hanging on network issues - Timeout aligns with other kubo operations (ping, DNS resolve, p2p) - Sufficient for DHT with sweep provider or accelerated client - Sync mode: blocks on provideCIDSync until completion (uses req.Context) - Improved structured logging with "fast-provide-root:" prefix - Removed redundant "root CID" from messages (already in prefix) - Clear async/sync distinction in log messages - Added FAST PROVIDE OPTIMIZATION section to ipfs add --help explaining: - The problem: background queue takes time, content not immediately discoverable - The solution: extra immediate announcement of just the root CID - The benefit: peers can find content right away while queue handles rest - Usage: async by default, --fast-provide-wait for guaranteed completion Changelog: - Added highlight section for fast root CID providing feature - Updated TOC and overview - Included usage examples with clear comments explaining each mode - Emphasized this is extra announcement independent of background queue The feature works best with sweep provider and accelerated DHT client where provide operations are significantly faster. * fix(add): respect Provide config in fast-provide-root fast-provide-root should honor the same config settings as the regular provide system: - skip when Provide.Enabled is false - skip when Provide.DHT.Interval is 0 - respect Provide.Strategy (all/pinned/roots/mfs/combinations) This ensures fast-provide only runs when appropriate based on user configuration and the nature of the content being added (pinned vs unpinned, added to MFS or not). * Update core/commands/add.go --------- Co-authored-by: gammazero <11790789+gammazero@users.noreply.github.com> Co-authored-by: Marcin Rataj <lidel@lidel.org> | 9 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 22 天前 | ||
| 1 个月前 | ||
| 9 天前 | ||
| 1 个月前 | ||
| 1 个月前 | ||
| 1 个月前 | ||
| 3 年前 | ||
| 1 个月前 | ||
| 3 个月前 | ||
| 11 年前 | ||
| 1 个月前 | ||
| 3 个月前 | ||
| 1 个月前 | ||
| 9 个月前 |