IPFS implementation in Go: a daemon that stores and serves content-addressed data, with a CLI, HTTP Gateway, and RPC API
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
fix: Ipfs-Uri gateway header (IPIP-548) (#11437) * feat: Ipfs-Uri gateway header (IPIP-548) Bump boxo to the IPIP-548 implementation (ipfs/boxo#1209): gateway responses carry a canonical percent-encoded Ipfs-Uri header and stop sending the deprecated X-Ipfs-Path, which cannot represent every UnixFS file name. - sharness: CORS expects Ipfs-Uri exposed, X-Ipfs-Path gone - gateway-conformance CI pinned to the IPIP-548 test suite (ipfs/gateway-conformance#301) until a release ships - reverse-proxy doc and v0.44 changelog updated Refs ipfs/specs#548 * feat: opt-in Gateway.DeprecatedXIpfsPath Expose boxo's opt-in for the legacy X-Ipfs-Path response header as a kubo config flag, default off. Unsafe: the legacy value cannot represent every UnixFS file name, so it must only be used to facilitate migration to Ipfs-Uri, and even when enabled the header is still skipped when the value would include non-ASCII byte sequences. Refs ipfs/specs#548 * ci: bump gateway-conformance pin * ci: gateway-conformance v0.14 * chore: boxo with IPIP-548 from boxo/main * docs: Ipfs-Uri changelog in v0.43.1 * docs: assemble v0.43.1 changelog Move the v0.44 highlights and dependency lines into a new v0.43.1 section, and add the missing entry for owner-only key exports (#11428). v0.44.md returns to an empty skeleton. * docs: note boxo v0.42.2 fixes in v0.43.1 | 3 天前 | |
docs: drop expired pgp key from security notes (#11422) The key expired in 2018 and the pgp.mit.edu lookup it pointed at returns 503, so anyone following it hits a dead end twice. No successor key is published for security@ipfs.io on any keyserver, so the notes point at the repository security policy instead. Seeding the init docs changes their directory CID, so the constants the tests assert against move with it. | 23 天前 | |
chore(mkreleaselog): exclude bot accounts Bot accounts like dependabot[bot] are not human contributors, so drop entries whose GitHub handle or author name ends in the [bot] suffix from the release contributor table. | 2 个月前 | |
refactor: apply go fix modernizers from Go 1.26 (#11190) * chore: apply go fix modernizers from Go 1.26 automated refactoring: interface{} to any, slices.Contains, and other idiomatic updates. * feat(ci): add `go fix` check to Go analysis workflow ensures Go 1.26 modernizers are applied, fails CI if `go fix ./...` produces any changes (similar to existing `go fmt` enforcement) | 6 个月前 | |
fix(rpc): validate UnixFS in `object patch` (#11248) * fix(object): validate UnixFS type in patch add-link Reject adding named links to non-directory nodes in `object patch add-link`, which previously produced invalid DAGs silently. - reject UnixFS File/Symlink/etc nodes (only Directory and HAMTShard support named links per the UnixFS spec) - reject non-UnixFS dag-pb nodes (no UnixFS metadata to validate) - add `--allow-non-unixfs` flag to bypass both checks - pass `allow-non-unixfs` in client/rpc when SkipUnixFSValidation is set - test all three node types: bare dag-pb, UnixFS File, UnixFS Directory - reproduce the exact data-loss scenario from #7190 Fixes: https://github.com/ipfs/kubo/issues/7190 * fix(object): reject HAMTShard in patch add-link dagutils.Editor operates at the dag-pb level and does not update HAMT bitfields, so mutating HAMTShard nodes produces corrupt DAGs. - reject HAMTShard in add-link (was incorrectly allowed) - update help text to note dag-pb limitations and suggest ipfs files - add HAMT test cases to sharness and API tests - expect full error strings in all validation tests - update changelog to cover all rejected node types * fix(object): validate UnixFS type in patch rm-link Same issue as add-link: dagutils.Editor operates at the dag-pb level and cannot update UnixFS metadata, so mutating non-Directory nodes produces corrupt DAGs. - add UnixFS validation to rm-link (Directory allowed, all else rejected) - add --allow-non-unixfs flag to rm-link command - add ObjectRmLinkSettings/ObjectRmLinkOption types - update ObjectAPI.RmLink interface to accept options - pass allow-non-unixfs in client/rpc - update rm-link help text to note dag-pb limitations - add rm-link validation tests for all four node types | 5 个月前 | |
chore: bump go-libp2p-kad-dht to v0.42 (#11398) * chore: bump go-libp2p-kad-dht to v0.42.0 * bump kad-dht * fix(init): stop publishing an empty-directory IPNS record * bump kad-dht to v0.42.1 * bump boxo * chore: bump boxo, clarify IPNS storage changelog Bumps boxo, bringing the offline router retention change (records are served until their EOL, no receive-age cap by default; ipfs/boxo#1189) and a gateway 304 cache-freshness fix (ipfs/boxo#1188). The v0.43 "Unified IPNS record storage" highlight now spells out how retention works: with the offline router and the DHT sharing the /dht datastore prefix, online nodes drop value records 48h after storage (the DHT's value-store GC) while offline nodes keep them until EOL. * fix: sync DHT purge once after deletion * bump boxo * depend on boxo@master * chore: bump boxo to main, align module pins Pick up boxo main HEAD (4794174d), which includes the merged offline router value-store change (ipfs/boxo#1189). This brings the root module, the kubo-as-a-library example, and test/dependencies onto one boxo pin; they previously pointed at three different commits. - go-libp2p moves to ec408fcc as a transitive floor required by boxo - changelog: note the one-time full-datastore key scan on first start after upgrading, and update the pinned boxo and go-libp2p hashes * chore: require make mod_tidy before commit/push Spell out that the repo has three go.mod files that must stay on the same dependency versions, and that a bare `go mod tidy` only tidies one module and lets pins drift between them. --------- Co-authored-by: Marcin Rataj <lidel@lidel.org> | 1 个月前 | |
refactor: apply go fix modernizers from Go 1.26 (#11190) * chore: apply go fix modernizers from Go 1.26 automated refactoring: interface{} to any, slices.Contains, and other idiomatic updates. * feat(ci): add `go fix` check to Go analysis workflow ensures Go 1.26 modernizers are applied, fails CI if `go fix ./...` produces any changes (similar to existing `go fmt` enforcement) | 6 个月前 | |
fix: Ipfs-Uri gateway header (IPIP-548) (#11437) * feat: Ipfs-Uri gateway header (IPIP-548) Bump boxo to the IPIP-548 implementation (ipfs/boxo#1209): gateway responses carry a canonical percent-encoded Ipfs-Uri header and stop sending the deprecated X-Ipfs-Path, which cannot represent every UnixFS file name. - sharness: CORS expects Ipfs-Uri exposed, X-Ipfs-Path gone - gateway-conformance CI pinned to the IPIP-548 test suite (ipfs/gateway-conformance#301) until a release ships - reverse-proxy doc and v0.44 changelog updated Refs ipfs/specs#548 * feat: opt-in Gateway.DeprecatedXIpfsPath Expose boxo's opt-in for the legacy X-Ipfs-Path response header as a kubo config flag, default off. Unsafe: the legacy value cannot represent every UnixFS file name, so it must only be used to facilitate migration to Ipfs-Uri, and even when enabled the header is still skipped when the value would include non-ASCII byte sequences. Refs ipfs/specs#548 * ci: bump gateway-conformance pin * ci: gateway-conformance v0.14 * chore: boxo with IPIP-548 from boxo/main * docs: Ipfs-Uri changelog in v0.43.1 * docs: assemble v0.43.1 changelog Move the v0.44 highlights and dependency lines into a new v0.43.1 section, and add the missing entry for owner-only key exports (#11428). v0.44.md returns to an empty skeleton. * docs: note boxo v0.42.2 fixes in v0.43.1 | 3 天前 | |
fix: Ipfs-Uri gateway header (IPIP-548) (#11437) * feat: Ipfs-Uri gateway header (IPIP-548) Bump boxo to the IPIP-548 implementation (ipfs/boxo#1209): gateway responses carry a canonical percent-encoded Ipfs-Uri header and stop sending the deprecated X-Ipfs-Path, which cannot represent every UnixFS file name. - sharness: CORS expects Ipfs-Uri exposed, X-Ipfs-Path gone - gateway-conformance CI pinned to the IPIP-548 test suite (ipfs/gateway-conformance#301) until a release ships - reverse-proxy doc and v0.44 changelog updated Refs ipfs/specs#548 * feat: opt-in Gateway.DeprecatedXIpfsPath Expose boxo's opt-in for the legacy X-Ipfs-Path response header as a kubo config flag, default off. Unsafe: the legacy value cannot represent every UnixFS file name, so it must only be used to facilitate migration to Ipfs-Uri, and even when enabled the header is still skipped when the value would include non-ASCII byte sequences. Refs ipfs/specs#548 * ci: bump gateway-conformance pin * ci: gateway-conformance v0.14 * chore: boxo with IPIP-548 from boxo/main * docs: Ipfs-Uri changelog in v0.43.1 * docs: assemble v0.43.1 changelog Move the v0.44 highlights and dependency lines into a new v0.43.1 section, and add the missing entry for owner-only key exports (#11428). v0.44.md returns to an empty skeleton. * docs: note boxo v0.42.2 fixes in v0.43.1 | 3 天前 | |
fix(fuse): switch to hanwen/go-fuse (#11272) * test(fuse): consolidate FUSE tests into test/cli/fuse Move FUSE integration tests from sharness shell scripts (t0030, t0031, t0032) and test/cli/fuse_test.go into a dedicated test/cli/fuse/ Go sub-package, ensuring all FUSE test cases run in CI. - git mv test/cli/fuse_test.go to test/cli/fuse/ (package fuse) - convert all sharness FUSE tests to Go subtests under TestFUSE: mount failure, IPNS symlink, IPNS NS map resolution, MFS file/dir creation, xattr (Linux), files write, add --to-files, file removal, nested dirs, publish-while-mounted block, sharded directory reads - add xattr helpers with build tags (linux/other) using unix.Getxattr - split make test_fuse into test_fuse_unit (./fuse/...) and test_fuse_cli (./test/cli/fuse/...) sub-targets - set TEST_FUSE=0 in test_cli so FUSE tests skip in cli-tests CI job - increase fuse-tests CI timeout from 5m to 10m for CLI tests - delete sharness t0030, t0031, t0032 (were always skipped in CI) * docs: document FUSE test split between unit and e2e Add cross-reference comments between the unit tests in fuse/readonly/, fuse/ipns/, fuse/mfs/ and the end-to-end CLI tests in test/cli/fuse/. Also fix AGENTS.md to use a temp dir for fusermount symlink instead of sudo. * ci: prevent stale FUSE mounts from failing fuse-tests On shared self-hosted runners, leftover mount points from previous runs can exhaust the kernel FUSE mount limit. - add job-level concurrency group so only one fuse-tests runs at a time - lazy-unmount stale /tmp/fusetest* mounts before running tests * ci: only symlink fusermount3 when fusermount is missing * fix(fuse): remove goroutine leak in IPNS Flush handler The Flush handler wrapped fi.fi.Flush() in a goroutine so it could return early when the FUSE context was canceled. But the goroutine kept running in the background, and when Release arrived it called Close on the same file descriptor concurrently. The two paths both entered DagModifier.Sync, racing on its internal write buffer and causing a nil pointer panic. The fix is to call Flush directly without a goroutine. The MFS flush cannot be safely canceled mid-operation anyway, so the goroutine only added the illusion of cancellation while leaking work and masking the real error. Also bumps boxo to pick up the matching defense-in-depth fix that serializes FileDescriptor.Flush and Close with a mutex. * fix(fuse): add mutex to IPNS file handle operations bazil/fuse dispatches each FUSE request in its own goroutine. The IPNS File handle had no synchronization, so concurrent Read/Write/Flush/Release calls could overlap on the underlying DagModifier which is not safe for concurrent use. Add sync.Mutex to File, matching the pattern already used by the MFS FileHandler. * refactor(fuse): remove dead File.Forget method bazil/fuse only dispatches Forget to nodes via the NodeForgetter interface. File is a handle, not a node, so this method was never called. The /mfs mount has no equivalent. * fix(fuse): flush IPNS directory after Remove and Rename The /mfs mount flushes the directory after Unlink and Rename so changes propagate to the MFS root immediately. The /ipns mount did not, leaving mutations pending until an unrelated flush. Also add an empty-directory check before removing directories, matching the /mfs mount's safety check. * fix(fuse): inherit CID builder and flush on IPNS Create New files created via the /ipns FUSE mount now inherit the CID builder from their parent directory, preventing CIDv0 nodes from appearing inside a CIDv1 tree. The directory is also flushed after AddChild so the new entry propagates to the MFS root immediately, matching the /mfs mount. * test(fuse): add IPNS Remove and non-empty rmdir tests Cover the file removal path and the empty-directory safety check added in the previous commit. TestRemoveFile verifies a created file can be removed and is gone afterwards. TestRemoveNonEmptyDirectory verifies that rmdir on a directory with children fails, and succeeds once the children are removed first. * feat(fuse): read UnixFS mode/mtime, add StoreMtime/StoreMode config All three FUSE mounts now read mode and mtime from UnixFS metadata when present, falling back to POSIX defaults when absent. Most IPFS data does not include this optional metadata. Writing mode and mtime is opt-in via two new config flags: - Mounts.StoreMtime: persist mtime on file create and open-for-write - Mounts.StoreMode: persist mode on chmod Other changes in this commit: - align default file/dir modes across /ipns and /mfs to 0644/0755 - share mode constants via fuse/mount/mode.go - convert Mounts.FuseAllowOther from bool to Flag for consistency - add Setattr to /ipns FileNode and /mfs File for chmod and touch - move dead File.Setattr from IPNS handle to FileNode (node) - bump boxo for Directory.Mode() and Directory.ModTime() getters * feat(fuse): add ipfs.cid xattr to all mounts All three FUSE mounts now expose the node's CID via the ipfs.cid extended attribute on both files and directories. The /mfs mount also accepts the old ipfs_cid name for backward compatibility. The /ipfs mount previously had a stub that returned nil for all xattrs; it now returns the correct CID. The xattr name follows the convention used by CephFS (ceph.*), Btrfs (btrfs.*), and GlusterFS (glusterfs.*). * feat(fuse): switch from bazil.org/fuse to hanwen/go-fuse v2 Replace the unmaintained bazil.org/fuse (last commit 2020) with hanwen/go-fuse v2.9.0, fixing two architectural issues that could not be solved with the old library. ftruncate now works: hanwen/go-fuse passes the open file handle to NodeSetattrer, so Setattr can truncate through the existing write descriptor instead of trying to open a second one (which deadlocks on MFS's single-writer lock). fsync now works: FileFsyncer runs on the handle directly, flushing the write buffer through the open descriptor. Previously a no-op because bazil dispatched Fsync to the inode only. mount package: - NewMount takes (InodeEmbedder, mountpoint, *fs.Options) instead of (fs.FS, mountpoint, allowOther) - mount/unmount collapses to a single fs.Mount call - fusermount3 tried before fusermount in ForceUnmount all three mounts: - structs embed fs.Inode (hanwen's InodeEmbedder pattern) - Remove split into Unlink + Rmdir (separate FUSE interfaces) - ReadDirAll replaced with Readdir returning DirStream - fillAttr helper shared between Getattr and Lookup responses - kernel cache invalidation via NotifyContent after Flush - 1s entry/attr timeout for writable mounts (matches go-fuse default, gocryptfs, rclone) - O_APPEND tracked on file handle, writes seek to end - build tags standardized to (linux || darwin || freebsd) && !nofuse tests: - replaced bazil fstestutil.MountedT with shared fusetest.TestMount - fixed TestConcurrentRW: channel drain mismatch and missing sync between write Close and read start - added TestFsync, TestFtruncate, TestReadlink, TestSeekRead, TestLargeFile, TestRmdir, TestCrossDirRename, TestUnknownXattr - added StoreMtime disabled/enabled subtests * fix(fuse): close fd on error in Open to prevent leak MFS enforces a single-writer lock, so a leaked write descriptor blocks all subsequent opens of that file until GC. * fix(fuse): detect external unmount via server.Wait Without this, IsActive stays true after `fusermount -u` and Unmount returns nil instead of ErrNotMounted. * fix(fuse): return actual error from Unlink/Rmdir, not ENOENT After confirming the child exists, an Unlink failure could be an IO error. Returning ENOENT would hide the real cause. * fix(fuse): reuse DagReader per open, pass ctx to all reads Readonly Open now returns a file handle holding a DagReader instead of recreating one per Read call. Sequential reads no longer re-traverse the DAG from the root on each kernel request. All three mounts now use CtxReadFull with the kernel's per-request context so killing a process mid-read cancels in-flight block fetches instead of letting them complete uselessly. * chore(fuse): cleanup dead code, add var comments - remove dead `_ = mntDir` in TestXattrCID - comment why immutableAttrCacheTime and mutableCacheTime are var - add TODO for using IPNS record TTL as cache timeout * chore(fuse): replace OSXFUSE 2.x check with macFUSE detection The old check tried to verify OSXFUSE >= 2.7.2 to avoid a kernel panic from 2015. It used sysctl, tried to `go install` a third-party tool at runtime, and referenced paths that no longer exist. Replace with a simple check for the macFUSE mount helper, matching the same paths go-fuse looks for. If neither macFUSE nor OSXFUSE is found, point the user to the install page. Also standardize build tags to (linux || darwin || freebsd) && !nofuse and use strings.ReplaceAll. * fix(fuse): include mountpoint path in mount errors go-fuse's fusermount errors don't include the path, so tools that check error messages for the mountpoint name couldn't tell which mount failed. * chore(ci): remove bazil fusermount workaround go-fuse finds fusermount3 natively, no symlink needed. The stale mount cleanup was for bazil's fstestutil which we no longer use. * docs: update v0.41 changelog for FUSE rewrite * chore(deps): bump boxo for full FileDescriptor serialization boxo@64be0815 extends the mutex from Flush/Close to all FileDescriptor operations (Read, Write, Seek, Truncate, Size), preventing data races on the underlying DagModifier. * chore(deps): bump boxo to merged ipfs/boxo#1133 Picks up full FileDescriptor serialization: the mutex now covers all operations (Read, Write, Seek, Truncate, Size), not just Flush and Close. * feat(fuse): CAP_ATOMIC_O_TRUNC, new integration tests Advertise CAP_ATOMIC_O_TRUNC so the kernel sends O_TRUNC inside Open instead of doing a separate SETATTR(size=0) first. Without this, the kernel's SETATTR needs to open a write descriptor inside Setattr, which deadlocks on MFS's single-writer lock. Move kernel cache invalidation from Flush to Release because mfsFD.Close (in Release) is where the final DAG node is committed. Upgrade go-fuse to latest for ExtraCapabilities support. New tests for both MFS and IPNS: - TestOpenTrunc, TestSeekAndWrite, TestOverwriteExisting - TestTempFileRename, TestVimSavePattern, TestRsyncPattern (skipped pending rename-over-existing and cache fixes) * fix(fuse): rename-over-existing, bump boxo for flushUp race fix IPNS Rename now unlinks the target before AddChild, matching MFS. Without this, renaming onto an existing name returned "directory already has entry". Bump boxo to pick up the flushUp unlinked-entry fix (ipfs/boxo@8ae46d5): when a file descriptor outlives its directory entry (FUSE RELEASE racing with RENAME), flushUp no longer re-adds the stale name. Unskip TestTempFileRename and TestRsyncPattern on both mounts. * fix(fuse): unskip VimSavePattern, bump boxo for setNodeData fix boxo@552d8e7 fixes File.setNodeData dropping content links when updating metadata (mode, mtime). chmod or touch after write no longer makes the file appear empty. Unskip TestVimSavePattern on both mounts. Remove debug logging and temporary test functions added during investigation. * fix(fuse): build tags for cross-compilation go-fuse does not compile on windows/openbsd/netbsd/plan9. Move WritableMountCapabilities (which imports go-fuse) from mode.go (no build tag) to caps.go (platform-gated). Align build tags on fusetest and core/commands/mount stubs so unsupported platforms don't pull in go-fuse transitively. * fix(test): use fusermount3 in CLI FUSE tests The doUnmount helper hardcoded fusermount, but systems with only fuse3 installed have fusermount3. Try fusermount3 first, matching what go-fuse and our ForceUnmount already do. * feat(fuse): symlink support on writable mounts Add NodeSymlinker to MFS and IPNS directories. Symlinks are stored as UnixFS TSymlink nodes in the DAG, the same format used by `ipfs add` for directories containing symlinks. The readonly /ipfs mount already rendered existing symlinks; now /mfs and /ipns can create them too. The target string is cached at Lookup time to avoid re-parsing the DAG node on every Readlink call. Symlink permissions are always 0777 per POSIX convention (access control uses the target's mode). * fix(fuse): checked type assertion in MFS Rename The direct type assertion on newParent could panic if the kernel passed a non-directory inode. Use a checked assertion with EINVAL fallback, matching the type-switch pattern in the IPNS mount. * fix(test): add missing continue in stress test Missing continue after error sends let execution fall through to nil type assertions (read.(files.File)) that would panic on error. Also cancel the context before continuing to avoid leaking it. * fix(fuse): return error from Readdir when DAG.Get fails Abort the directory listing instead of silently omitting the unretrievable entry. Callers get EIO, which is more honest than a partial listing that hides missing blocks. * docs: remove duplicate fsync bullet in changelog * ci: clean up stale FUSE mounts in fuse-tests job On shared self-hosted runners, leftover mounts from crashed runs can exhaust the kernel mount_max limit. Lazy-unmount kubo-test and harness temp mounts before and after tests. * chore(deps): bump boxo to merged ipfs/boxo#1134 Picks up flushUp unlinked-entry guard and setNodeData content link preservation. * docs: add build tag comments, normalize tag style Add a one-line comment above every //go:build directive explaining why the constraint exists. Normalize tag style: positive platform constraints first, then feature flags/negations. Simplify redundant expressions. * fix(fuse): add Setattr to directories for chmod and mtime Tools like tar and rsync call utimensat on directories after extraction. Without Setattr on Dir, this returned ENOTSUP. Add Setattr to Dir (MFS) and Directory (IPNS) that handles mode and mtime the same way as the file-level Setattr. When StoreMtime or StoreMode is disabled the call succeeds silently, matching the file-level behavior. * docs: clarify directory support and spec link for StoreMtime/StoreMode - mention that touch and chmod work on both files and directories - note tar and rsync as practical use cases - link to UnixFS spec for optional metadata storage * fix(fuse): use proper mode conversion, document 9-bit limit Use files.UnixPermsToModePerms and files.ModePermsToUnixPerms for converting between FUSE kernel mode (unix 12-bit layout) and Go's os.FileMode (different bit positions for setuid/setgid/sticky). The UnixFS spec supports all 12 permission bits, but boxo's MFS layer (File.Mode, Directory.Mode) exposes only the lower 9. FUSE mounts are always nosuid so the upper 3 bits would have no effect. Add TestSetuidBitsStripped to both mounts confirming the behavior. * feat(fuse): symlink Setattr with mtime persistence Wire the backing mfs.File into the FUSE Symlink struct so Setattr can call SetModTime when StoreMtime is enabled. boxo's File methods (SetModTime, ModTime) already work on TSymlink nodes since they operate on the FSNode protobuf without checking the type. Without Setattr, rsync -a fails with "failed to set times" on symlinks. Every major FUSE filesystem (gocryptfs, rclone, sshfs, s3fs) implements Setattr on symlinks for this reason. Mode is always 0777 per POSIX convention, so chmod requests are silently accepted but not stored. * fix(fuse): return EIO instead of panicking on unknown node type Replace panic with log.Errorf + syscall.EIO in IPNS Directory.Lookup for unexpected MFS node types. Also remove duplicate comment block on File.Flush. * docs: update FUSE docs for go-fuse migration - fuse.md: replace stale OSXFUSE section with macFUSE, remove obsolete go-fuse-version tool, fix broken FreeBSD sudo echo, update xattr example to ipfs.cid with CIDv1, add mode/mtime section, add unixfs-v1-2025 tip, add debug logging section, add TOC, link to hanwen/go-fuse - changelog: refine bullet wording, link to fuse.md - config.md: fix double space, update fuse.md link text - experimental-features.md: fix double space, soften wording - README.md: add FUSE to features list and docs table * refactor(fuse): extract shared writable types and test suite Extract duplicated code from fuse/mfs and fuse/ipns into a shared fuse/writable package, and consolidate duplicated tests into a reusable suite in fuse/fusetest. - fuse/writable: Dir, FileInode, FileHandle, Symlink types with all FUSE interface methods, shared by both mounts - fuse/fusetest: RunWritableSuite with helpers, exercised by both mfs and ipns via mount-specific factories - fix cache invalidation race: NotifyContent in Flush (synchronous) in addition to Release (async), so stat after close sees new size - drop deprecated ipfs_cid xattr, log error guiding users to ipfs.cid - mfs_unix.go: 632 -> 19 lines (thin wrapper over writable.Dir) - ipns_unix.go: 795 -> 170 lines (Root + key resolution only) - mfs_test.go: 1183 -> 95 lines (factory + persistence test) - ipns_test.go: 1309 -> 162 lines (factory + IPNS-specific tests) - tests that were only in one mount now run on both * feat(fuse): add macOS-specific mount options Set volname, noapplexattr, and noappledouble on macOS via PlatformMountOpts, applied in NewMount so all three mounts benefit automatically. - volname: shows mount name in Finder instead of "macfuse Volume 0" - noapplexattr: suppresses Finder's com.apple.* xattr probes - noappledouble: prevents ._ resource fork sidecar files * fix(fuse): detect symlinks in readdir, fix stale refs Readdir on writable mounts now checks the underlying DAG node type for TFile entries, reporting S_IFLNK for symlinks instead of regular file. This makes ls -l and find -type l work correctly. - writable: Readdir checks SymlinkTarget for TFile entries - writablesuite: add SymlinkReaddir regression test - readonly: add TestReaddirSymlink regression test - test/cli/fuse: fix stale bazil.org/fuse reference in doc comment * fix(fuse): normalize deprecated ipfs_cid xattr to ipfs.cid Getxattr for the old "ipfs_cid" name now returns the CID instead of ENOATTR, keeping existing tooling working during the deprecation period. A log error is emitted on each access to nudge migration. * fix(fuse): serialize concurrent reads on readonly file handles The go-fuse server dispatches each FUSE request in its own goroutine. On files larger than 128 KB the kernel issues concurrent readahead Read requests on the same file handle, racing on the shared DagReader's Seek+CtxReadFull sequence and corrupting its internal state. Add sync.Mutex to roFileHandle (matching the existing pattern in writable.FileHandle) and lock in Read and Release. - fuse/readonly/readonly_unix.go: add mu sync.Mutex to roFileHandle - fuse/readonly/ipfs_test.go: add TestConcurrentLargeFileRead - fuse/fusetest/writablesuite.go: add LargeFileConcurrentRead to shared writable suite (exercised by both /mfs and /ipns tests) * fix(fuse): bypass MFS locking for read-only opens MFS uses an RWMutex (desclock) that holds RLock for the lifetime of a read descriptor and requires exclusive Lock for writes. Tools like rsync --inplace open the same file for reading and writing from separate processes, deadlocking on this mutex. For O_RDONLY opens, create a DagReader directly from the current DAG node instead of going through MFS. The reader gets a point-in-time snapshot and never touches desclock, so writers proceed independently. - fuse/writable/writable.go: add roFileHandle with DagReader for read-only opens, add DAG field to Config - fuse/mfs/mfs_unix.go: pass ipfs.DAG to writable Config - fuse/ipns/ipns_unix.go: pass ipfs.Dag() to writable Config - fuse/fusetest/writablesuite.go: add ConcurrentReadWrite test exercising simultaneous read and write on the same file * fix(fuse): support truncate(path, size) without open fd Open a temporary write descriptor in Setattr when the kernel sends a size change without a file handle (the truncate(2) syscall, as opposed to ftruncate(fd) which passes the handle). Previously this returned ENOTSUP. - fuse/writable: open, truncate, flush, close in Setattr else branch - fuse/fusetest: add TruncatePath to the shared writable suite - test/cli/fuse: add end-to-end truncation test covering ftruncate(fd), syscall.Truncate(path), and open(O_TRUNC) through a real daemon * ci(fuse): get stack traces on test hangs The fuse-tests job was being silently cancelled by GitHub at 10min because Go's per-test timeout (5m) was the same order as the job timeout, and GOTRACEBACK=single hid the hung goroutines anyway. - shrink TEST_FUSE_TIMEOUT to 4m so Go's panic fires first - shrink job timeout-minutes to 6 (normal run is ~3min) - set GOTRACEBACK=all so the panic dumps every goroutine, not just the timer * fix(fuse): fill attrs in FileInode.Setattr response Without this, the kernel could cache zero attrs after a chmod, touch, or ftruncate until AttrTimeout (1s) expired. Dir.Setattr and Symlink.Setattr already fill out.Attr; FileInode.Setattr now matches. * docs(config): clarify Mounts.IPNS writability scope Only directories backed by keys the node holds are writable. All other names resolve via IPNS to read-only symlinks into the /ipfs mount. * fuse: review cleanup for go-fuse migration Final pass on #11272 addressing review feedback. - writable: panic in NewDir if Config.DAG is nil. Both call sites already supply it, but a nil value silently fell back to the MFS path in FileInode.Open, re-introducing the rsync --inplace deadlock the read-only fast path was added to fix. - writable: document Dir.Rename non-atomicity. Source unlink happens before destination add, so any failure between the two loses the source. An atomic fix requires changes in boxo/mfs. - writable: add unit test locking in that Symlink.Setattr accepts a mode-only request without erroring and does not store the requested mode (POSIX symlinks have no meaningful permission bits). - docs/config: correct StoreMode default modes; the previous text listed 0666 for files, which the code never uses. * docs(config): list StoreMtime and StoreMode in Mounts TOC * fix(fuse): fill EntryOut attrs in Dir.Create and Dir.Mkdir Without this, fstat on the file handle returned by Create reports mode 0 and size 0 for up to AttrTimeout (1s), because the kernel caches the empty attrs from the Create response. Path-based stat goes through Lookup which already fills attrs, so the bug only shows up via fstat. Mirrors the same fix already applied to FileInode.Setattr. Dir.Mkdir gets the same fillAttr treatment for consistency, plus a TODO noting that boxo's mfs.Directory.Mkdir accepts no mode arg so the caller's mode is dropped on creation. Adds CreateAttrsImmediate and MkdirAttrsImmediate to the shared writable suite to guard both paths against future regressions. * fix(fuse): map context cancellation to EINTR in read paths When a userspace process is killed mid-read (Ctrl-C, SIGKILL on a stuck cat) the kernel sends FUSE_INTERRUPT and go-fuse cancels the per-request context. fs.ToErrno does not recognise context.Canceled and falls through to "function not implemented", which the kernel cannot act on. Map context.Canceled and DeadlineExceeded to EINTR so the syscall is correctly aborted. - mount/errno.go: new ReadErrno helper used by all context-aware read paths in both readonly and writable mounts - readonly: applied to Node.Open, Node.Readdir, roFileHandle.Read - writable: applied to FileInode.Open, FileHandle.Read, roFileHandle.Read - readonly/ipfs_test.go: TestReadCancellationUnblocks guards the contract via a blocking DagReader fake; without ReadErrno the test reports "function not implemented" instead of EINTR * test(fuse): add OExcl, DirRename, SparseWrite, FsyncCrossHandle Coverage gaps in the shared writable suite: - OExcl: lock files and atomic-create patterns rely on the second open with O_CREATE|O_EXCL failing with EEXIST - DirRename: previously only file rename and cross-dir file rename were tested; this exercises Rename on a directory inode - SparseWrite: WriteAt past the end of an empty file must report the correct size and return zeros for the gap - FsyncCrossHandle: a reader on a fresh fd must see data flushed by fsync on the writer fd, not just after close * test(fuse): cover external unmount on /ipns and /mfs Previously TestExternalUnmount only exercised /ipfs, leaving the goroutine that watches fuse.Server.Wait() untested for the other two mounts. Refactor into a table-driven test that runs the same fusermount/umount-then-IsActive flow against all three mounts. Switch to coremock.NewMockNode so the node is online: doMount only attaches the /ipns mount when node.IsOnline is true, and the table needs all three populated. * fix(commands): align 'ipfs mount' output columns MountCmd's LongDescription has "MFS mounted at:" with two spaces so the column lines up with the 4-char "IPFS" and "IPNS" rows above, but the runtime encoder and the daemon's startup print used a single space and produced misaligned output. Bring both runtime sites in line with the help text, and update the two existing test fixtures (test/cli/fuse and the sharness test-lib helper that t0040-add-and-cat.sh still uses) to expect the aligned form. * fix(fuse): invalidate kernel cache on Fsync FileHandle.Fsync only flushed the MFS file descriptor and left the kernel's cached attrs and content for the inode untouched. A fresh reader on the same path then saw the size cached from the original Create response (zero), reading zero bytes regardless of how much the writer had synced. Mirror the cache invalidation already done in Flush via inode.NotifyContent(0, 0) so a writer that fsyncs while another process opens the file (vim then a follow-up cat, IDE then a language server) sees consistent state. Sharpen the FsyncCrossHandle assertion to report the size delta on failure; the bug surfaced as got=0/want=500 only after switching from bytes.Equal to require.Equal. * chore(gitignore): ignore test_fuse_unit and test_fuse_cli json output The new test_fuse_unit and test_fuse_cli make targets emit test/fuse/fuse-unit-tests.json and test/fuse/fuse-cli-tests.json respectively, the same gotestsum --jsonfile pattern that test_unit and test_cli already use. Add them to the same .gitignore section so a local test run does not leave the working tree dirty. * test(fuse): end-to-end coverage with real POSIX tools Adds TestFUSERealWorld in test/cli/fuse/realworld_test.go: a single shared-daemon test with 18 subtests that exercise the writable /mfs mount through the actual binaries users invoke (sh, cat, seq, wc, ls, stat, cp, mv, rm, ln, readlink, find, dd, sha256sum, tar, rsync, vim). Each subtest verifies the result both via the FUSE filesystem and via 'ipfs files read|stat|ls' so both views agree. Synthetic payloads default to 1 MiB + 1 byte so multi-chunk read/write paths are exercised, not just single-chunk fast paths. External tools are required, not optional: a missing binary fails the test loudly so a CI image change cannot silently turn the suite green. The whole-suite TEST_FUSE gate is the only place a developer is allowed to skip. runCmd forces LC_ALL=C so locale-sensitive tool output (date formats in 'ls -l', decimal separators in 'wc', localized error messages, find/ls collation) is deterministic regardless of the runner's locale settings. One shared daemon across all 18 subtests keeps total runtime under two seconds; isolation comes from per-subtest subdirectories under the mount. | 4 个月前 | |
fix: Ipfs-Uri gateway header (IPIP-548) (#11437) * feat: Ipfs-Uri gateway header (IPIP-548) Bump boxo to the IPIP-548 implementation (ipfs/boxo#1209): gateway responses carry a canonical percent-encoded Ipfs-Uri header and stop sending the deprecated X-Ipfs-Path, which cannot represent every UnixFS file name. - sharness: CORS expects Ipfs-Uri exposed, X-Ipfs-Path gone - gateway-conformance CI pinned to the IPIP-548 test suite (ipfs/gateway-conformance#301) until a release ships - reverse-proxy doc and v0.44 changelog updated Refs ipfs/specs#548 * feat: opt-in Gateway.DeprecatedXIpfsPath Expose boxo's opt-in for the legacy X-Ipfs-Path response header as a kubo config flag, default off. Unsafe: the legacy value cannot represent every UnixFS file name, so it must only be used to facilitate migration to Ipfs-Uri, and even when enabled the header is still skipped when the value would include non-ASCII byte sequences. Refs ipfs/specs#548 * ci: bump gateway-conformance pin * ci: gateway-conformance v0.14 * chore: boxo with IPIP-548 from boxo/main * docs: Ipfs-Uri changelog in v0.43.1 * docs: assemble v0.43.1 changelog Move the v0.44 highlights and dependency lines into a new v0.43.1 section, and add the missing entry for owner-only key exports (#11428). v0.44.md returns to an empty skeleton. * docs: note boxo v0.42.2 fixes in v0.43.1 | 3 天前 | |
fix(fuse): keep what you write after a rename (#11430) * fix(fuse): keep a rename's writes go-fuse hands the kernel's existing node to the new name once Dir.Rename returns, and that node still held the MFS handle the rename had unlinked. MFS treats such a handle as gone: a write through it was accepted and then dropped, so `mv a b` followed by a write to `b` read back the old contents a second later, once the entry cache expired. A directory was worse. Creating a file in one that had just been renamed flushed through the dead handle, which carried the name the rename had moved away from, so the new file was lost and the old directory came back for good. Each node now reaches its MFS handle through an atomic, and a rename points the moved node at the entry that exists afterwards. Entries the kernel had already looked up underneath a renamed directory hang off the handle it was reached through, so the walk follows them down; it covers what the kernel is holding, not the whole tree. Invalidating the entry instead was tried and does not work: the kernel processes FUSE_NOTIFY_INVAL_ENTRY while holding the parent inode lock, so notifying from inside Rename deadlocks, and notifying asynchronously still loses most of the writes it races. Left unfixed: a write through a file descriptor held open across the rename still goes to the descriptor opened from the old handle. * fix(fuse): refuse to replace a non-empty directory A rename may only overwrite a directory that is empty. MFS removes a directory and everything under it without complaint, so `mv -T src dst` took dst's contents with it and reported success. Rmdir already had the check; Rename now makes it too, before it unlinks anything. The check for an absent destination also goes through errors.Is now. It compares against a sentinel that boxo returns bare today, and the cost of that changing is the source file, which by then has been unlinked. * fix(fuse): read /ipfs blocks we cannot decode A UnixFS directory can link to a block of any codec. stat reports one the mount cannot decode as a file the size of the block, but every read of it failed, because the read path went looking for a UnixFS DAG that is not there. A size stat promises has to be a size reads deliver, so serve the block itself. * fix(fuse): list a directory with a missing block One child whose block is not held locally failed the whole listing, and with an errno the caller could make nothing of: ipld.ErrNotFound has no mapping, so it arrived as ENOSYS. The readable entries are worth having, so report the one that is missing with no type and let a stat of it say what is wrong. * fix(fuse): report the CID the path used The ipfs.cid xattr answered with a CID the caller had never seen. A lookup rebuilds the node by decoding the block, which drops the version and codec of the path it came from, so a v1 dag-pb path reported its v0 form. Keep the CID the entry resolved to and report that. The changelog entry also covers the rename check from the commit before it, which landed without one. * test(fuse): make the rename tests catch their bugs TestRenameOntoNamespaceRoot read the file back through the mount, which answers from the entry the kernel still has cached and so succeeds whether or not the rename took the file away. It passed against the bug it was written for. Ask MFS instead. The dirent helper also loops on a record length it never checks, which would spin rather than fail if the kernel ever sent zero. * test(ipns): settle the repo path before mounting TestStatfs assigned Root.RepoPath once the server was already serving, and Statfs reads it from a FUSE handler goroutine, so `go test -race ./fuse/...` reported a data race on every run. The mfs and readonly tests already settle it before their mount; do the same here. | 13 天前 | |
fix(mfs): stop repo gc from freezing files ops (#11386) * fix(mfs): stop repo gc from freezing files ops Running `ipfs repo gc` alongside `ipfs files` writes could leave MFS permanently hung: GC deleted directory-node blocks a write had written to the blockstore but not yet linked into the persisted MFS root, and the next path lookup blocked forever fetching the missing block while holding the MFS directory lock, so every later files command piled up behind it. MFS mutations now take the pin lock, the same lock `ipfs add` uses, and GC computes the MFS root only after it holds the GC lock. A write's blocks are therefore either fully linked into the root before GC runs, or the write waits for GC to finish, so GC never collects data a live write still needs. - core/commands/files.go: hold the pin lock across write, cp, mkdir, mv, rm, flush, chcid, chmod, and touch - gc/gc.go: take the best-effort MFS root snapshot after acquiring the GC lock, closing the snapshot-before-lock window - core/corerepo/gc.go: pass the root as a callback evaluated under the lock - core/node/core.go: document why MFS keeps its online DAG service so lazy `ipfs files cp /ipfs/<cid>` pointers still resolve - gc/gc_test.go: assert the root snapshot is taken under the GC lock Closes #10842 * fix(mfs): extend gc pin lock to add and fuse The pin lock that stops garbage collection from collecting live MFS blocks now covers every path that mutates MFS, not just `ipfs files`, and every live MFS root is part of the GC live set. - core/commands/add.go: hold the pin lock while `ipfs add --to-files` links the added content into the MFS root - fuse/writable, fuse/mfs: hold the pin lock across FUSE `/mfs` structural writes and file flush, fsync, and release - fuse/ipns: same for the per-key `/ipns` mounts, and register their roots so GC keeps their blocks live - core/core.go: track mounted MFS roots for the GC live set - core/corerepo/gc.go: build the live set from the files root plus every registered root, skipping a root that errors instead of aborting the whole GC Closes #6113 Closes #7008 Closes #9553 * fix(mfs): fail fast on a missing block A directory-node block that is missing locally and unreachable would block an MFS operation forever while it held the directory lock, wedging the whole MFS and a clean shutdown until the process was killed. This is what remained after a repo was damaged by an older Kubo, a manual `ipfs block rm`, or a crash (the lockup in #7844). MFS now bounds those network reads: an unreachable block fails the operation with a timeout and releases the lock, while lazily-referenced content (`ipfs files cp /ipfs/<cid>`) still loads as before. - config: add DefaultMFSFetchTimeout and pass it to the MFS root via mfs.WithFetchTimeout in Import.MFSRootOptions - go.mod: bump boxo to pull in mfs.WithFetchTimeout * feat(mfs): honor --timeout in files read and write Adopt boxo's mfs.File.Open(ctx) so MFS file operations carry a context and a read or write stuck on a missing block can be cancelled instead of blocking forever. - go.mod: bump boxo to pick up mfs.File.Open(ctx) - core/commands/files.go: pass the request context to files read and write, so a client --timeout ends a stuck read or write - fuse/writable: bind long-lived FUSE descriptors (Create, Open) to the mount context via the new Config.MountCtx; the transient Setattr truncate uses the per-operation context - fuse/mfs, fuse/ipns: set MountCtx to the node/mount context - test/cli: cover files read and write honoring --timeout on unreachable content * chore(deps): bump boxo to merged mfs fix Switch from the pre-merge branch pseudo-version to the merged commit of ipfs/boxo#1185, the boxo side of the MFS under-lock work this branch depends on. * refactor(fuse): pass request ctx to pin lock Flush, Release, and Fsync passed context.Background() to the pin lock while their own ctx argument was in scope. Thread the passed ctx through instead, matching the other handlers. boxo's default GCLocker discards the context, so this is a consistency change, not a behavior change. https://github.com/ipfs/kubo/pull/11386#discussion_r3534131824 https://github.com/ipfs/kubo/pull/11386#discussion_r3534138912 https://github.com/ipfs/kubo/pull/11386#discussion_r3534140861 Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> * refactor(gc): const for buffer, simplify roots Name the GC result channel buffer size as a const, drop the temporary in the best-effort root snapshot, and use t.Context() in the snapshot test. The buffer size predates this branch; it only shifted in the diff. https://github.com/ipfs/kubo/pull/11386#discussion_r3534154606 https://github.com/ipfs/kubo/pull/11386#discussion_r3534077507 https://github.com/ipfs/kubo/pull/11386#discussion_r3534085418 https://github.com/ipfs/kubo/pull/11386#discussion_r3534106766 Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> * refactor(add): collapse pin lock defer Fold the pin lock's unlock into the defer at both --to-files call sites, matching the one-line style used elsewhere. https://github.com/ipfs/kubo/pull/11386#discussion_r3534006489 https://github.com/ipfs/kubo/pull/11386#discussion_r3534013057 Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> | 1 个月前 | |
fix(fuse): IPNS writes actually publish (#11271) * fix(fuse): persist IPNS writes across restarts The IPNS FUSE mount's MFS republisher calls Name.Publish to persist changes, but checkPublishAllowed blocks all publishes while the mount is active. This means writes through the FUSE mount are silently dropped and lost on daemon restart. Add a context key so the mount's internal publishes bypass the guard while manual `ipfs name publish` from CLI/RPC remains blocked. - core/coreiface/name.go: context key and helpers for mount publish - core/coreapi: checkPublishAllowed checks context before blocking - fuse/ipns: ipnsPubFunc marks its context as mount-internal - fuse/ipns: tests set node.Mounts.Ipns to exercise the guard Fixes #2168 * docs: add IPNS FUSE persistence fix to v0.41 changelog * refactor(fuse): move publish bypass to internal package Move the FUSE mount publish context key from the public coreiface package to internal/fusemount, preventing external consumers from bypassing the publish guard. - internal/fusemount/context.go: new internal package with context helpers - core/coreiface/name.go: remove exported ContextWithMountPublish / IsMountPublish - core/coreapi/coreapi.go: use fusemount.IsPublish for the guard check - fuse/ipns/ipns_unix.go: use fusemount.ContextWithPublish to tag context | 4 个月前 | |
docs: guide for running on low-memory devices (#11427) * docs: guide for running on low-memory devices Add docs/production/low-memory.md with tuning for 8 GiB devices like Raspberry Pi: GOMEMLIMIT, systemd MemoryHigh/MemoryMax, and Provide.DHT.MaxWorkers, plus how to verify with cgroup memory pressure. - README.md, docs/README.md: link the guide - docs/config.md: link from lowpower profile and ResourceMgr notes, caution against setting Swarm.ResourceMgr.MaxMemory too low - misc/systemd: commented example limits pointing at the guide * docs: announcement sizing and dht client advice Extend the low-memory guide with measured announcement rates, the 48h record expiry floor, and a worked 10M-CID example (MaxWorkers=6, Interval=32h). Advise client-only routing (autoclient), disabling the AutoNAT and relay services, and warn against enabling the accelerated DHT client on constrained hardware. - docs/environment-variables.md: document GOMEMLIMIT - docs/config.md: broaden lowpower profile pointer to the guide | 17 天前 | |
feat: derive AgentSuffix from build origin (#11341) Fork builds previously announced as plain `kubo/<ver>/<commit>`, indistinguishable from upstream in ecosystem dashboards. When `Version.AgentSuffix` and `--agent-version-suffix` are both unset, kubo now derives a default from the build origin so fork traffic self-identifies in the swarm. - mk/git.mk, cmd/ipfs/Rules.mk: normalize `git remote get-url origin` to `host/org/repo` and inject as `buildOrigin` ldflag - version.go: ImplicitAgentSuffix prefers buildOrigin, falls back to debug.ReadBuildInfo Main.Path; suffixFromForkPath strips known forges (github, gitlab, codeberg, bitbucket) and trailing `/kubo` - cmd/ipfs/kubo/daemon.go: use as fallback when explicit values empty - AGENTS.md: state builds must use `make build` so ldflags are set - docs/config.md: document the implicit-suffix behavior Co-authored-by: Guillaume Michel <guillaumemichel@users.noreply.github.com> | 2 个月前 | |
feat(p2p): add --foreground flag to listen and forward commands (#11099) * feat(p2p): add --foreground flag to listen and forward commands adds `-f/--foreground` option that keeps the command running until interrupted (SIGTERM/Ctrl+C) or closed via `ipfs p2p close`. the listener/forwarder is automatically removed when the command exits. useful for systemd services and scripts that need cleanup on exit. * docs: add p2p-tunnels.md with systemd examples - add dedicated docs/p2p-tunnels.md covering: - why p2p tunnels (NAT traversal, no public IP needed) - quick start with netcat - background and foreground modes - systemd integration with path-based activation - security considerations and troubleshooting - document Experimental.Libp2pStreamMounting in docs/config.md - simplify docs/experimental-features.md, link to new doc - add "Learn more" links to ipfs p2p listen/forward --help - update changelog entry with doc link - add cross-reference in misc/README.md * chore: reference kubo#5460 for p2p config Ref. https://github.com/ipfs/kubo/issues/5460 * fix(daemon): write api/gateway files only after HTTP server is ready fixes race condition where $IPFS_PATH/api and $IPFS_PATH/gateway files were written before the HTTP servers were ready to accept connections. this caused issues for tools like systemd path units that immediately try to connect when these files appear. changes: - add corehttp.ServeWithReady() that signals when server is ready - wait for ready signal before writing address files - use sync.WaitGroup.Go() (Go 1.25) for cleaner goroutine management - add TestAddressFileReady to verify both api and gateway files * fix(daemon): buffer errc channel and wait for all listeners - buffer error channel with len(listeners) to prevent deadlock when multiple servers write errors simultaneously - wait for ALL listeners to be ready before writing api/gateway file, not just the first one Feedback-from: https://github.com/ipfs/kubo/pull/11099#pullrequestreview-3593885839 * docs(changelog): improve p2p tunnel section clarity reframe to lead with user benefit and add example output * docs(p2p): remove obsolete race condition caveat the "First launch fails but restarts work" troubleshooting section described a race where the api file was written before the daemon was ready. this was fixed in 80b703a which ensures api/gateway files are only written after HTTP servers are ready to accept connections. --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> | 7 个月前 | |
chore: restore default telemetry for now (#11415) Telemetry reports to https://telemetry.ipshipyard.dev by default again, as it did through v0.42. This is a stopgap: it holds while the devgrant support window is active. Every way to turn telemetry off now lives in one place, so ending it later is a config change or a one-line diff rather than a rewrite. - endpoint is a linker-settable var: building with -ldflags "-X ...telemetry.defaultEndpoint=" yields a binary with no destination, which collects nothing and writes no identifier - DO_NOT_TRACK is honored, ranking between IPFS_TELEMETRY and the config Mode, so one variable opts a machine out of every tool - a collector answering 410 Gone retires itself: the node drops its identifier and never sends there again, on this run or a later one, which stops reporting across deployed nodes without a release - first-run notice names DO_NOT_TRACK next to the Kubo switches - docs/telemetry.md leads with how to disable, including at build time - AGENTS.md: telemetry opt-outs are a rule, not a courtesy - changelog: drop the opt-in highlight, v0.43 ships no telemetry change | 1 个月前 | |
chore: use go-log/v2 (#10801) * chore: update to go-log/v2 go-log v2 has been out for quite a while now and it is time to deprecate v1. Replace all use of go-log with go-log/v2 Makes /api/v0/log/tail useful over HTTP Updates dependencies that have moved to go-lov/v2 Removes support for ContextWithLoggable as this is not needed for tracing-like functionality - Replaces: PR #8765 - Closes issue #8753 - Closes issue #9245 - Closes issue #10809 Other fixes: * update go-ipfs-cmds * update http logs test * fix test * Read/send one line of log data at a time * Update -log-level docs | 1 年前 | |
fix(key): restrict overwritten key exports to owner-only permissions (#11428) * fix(key): restrict overwritten key exports to owner-only permissions Signed-off-by: questfever <questfever@outlook.com> * fix(atomicfile): temp file leak and name limit Both problems surface through `ipfs key export`, which now writes through this helper, but they affect every caller: config writes, repo migrations and `ipfs update`. - remove the temporary file when the rename fails, so one holding private key material is not left next to the target - keep the ".tmp-" prefix and the random suffix within the 255 byte file name limit, so a target with a long name can still be written * fix(key): route key export by target type Choosing the write path with os.Lstat treated /dev/stdout, /dev/stderr and /dev/fd/N as plain paths, because they are symlinks into /proc/self/fd, so the atomic write failed on targets that worked before. Decide by what the path resolves to, and state the whole contract in the command help. - regular file or nothing yet: written to a temporary file and renamed over the target, following symlinks, including one whose target does not exist yet - character device or pipe: streamed in place, confirmed on the open descriptor and without O_TRUNC, so a path swapped for a regular file can neither receive the key nor be emptied - anything else: refused, naming the path - errors name the file the user asked for, and the temporary file is flushed before the rename * fix(key): stop export landing on the wrong file An export could replace a file the target symlink does not point at. resolveSymlink joined a relative link target onto the path as typed, so ".." collapsed lexically. Where a parent component was itself a symlink, the join named a file outside the directory the link resolves to: the key was renamed over that file, and the intended target was never written. The link's parent is now resolved with filepath.EvalSymlinks before the join. * test(key): drop umask dependency in export test os.WriteFile applies the umask, so under umask 077 the fixture was created 0600 and the check that a failed export leaves the file at 0644 failed for reasons unrelated to the code under test. The CLI test already chmods for the same reason. --------- Signed-off-by: questfever <questfever@outlook.com> Co-authored-by: Marcin Rataj <lidel@lidel.org> | 16 天前 | |
chore: bump go-libp2p-kad-dht to v0.42 (#11398) * chore: bump go-libp2p-kad-dht to v0.42.0 * bump kad-dht * fix(init): stop publishing an empty-directory IPNS record * bump kad-dht to v0.42.1 * bump boxo * chore: bump boxo, clarify IPNS storage changelog Bumps boxo, bringing the offline router retention change (records are served until their EOL, no receive-age cap by default; ipfs/boxo#1189) and a gateway 304 cache-freshness fix (ipfs/boxo#1188). The v0.43 "Unified IPNS record storage" highlight now spells out how retention works: with the offline router and the DHT sharing the /dht datastore prefix, online nodes drop value records 48h after storage (the DHT's value-store GC) while offline nodes keep them until EOL. * fix: sync DHT purge once after deletion * bump boxo * depend on boxo@master * chore: bump boxo to main, align module pins Pick up boxo main HEAD (4794174d), which includes the merged offline router value-store change (ipfs/boxo#1189). This brings the root module, the kubo-as-a-library example, and test/dependencies onto one boxo pin; they previously pointed at three different commits. - go-libp2p moves to ec408fcc as a transitive floor required by boxo - changelog: note the one-time full-datastore key scan on first start after upgrading, and update the pinned boxo and go-libp2p hashes * chore: require make mod_tidy before commit/push Spell out that the repo has three go.mod files that must stay on the same dependency versions, and that a bare `go mod tidy` only tidies one module and lets pins drift between them. --------- Co-authored-by: Marcin Rataj <lidel@lidel.org> | 1 个月前 | |
fix: Ipfs-Uri gateway header (IPIP-548) (#11437) * feat: Ipfs-Uri gateway header (IPIP-548) Bump boxo to the IPIP-548 implementation (ipfs/boxo#1209): gateway responses carry a canonical percent-encoded Ipfs-Uri header and stop sending the deprecated X-Ipfs-Path, which cannot represent every UnixFS file name. - sharness: CORS expects Ipfs-Uri exposed, X-Ipfs-Path gone - gateway-conformance CI pinned to the IPIP-548 test suite (ipfs/gateway-conformance#301) until a release ships - reverse-proxy doc and v0.44 changelog updated Refs ipfs/specs#548 * feat: opt-in Gateway.DeprecatedXIpfsPath Expose boxo's opt-in for the legacy X-Ipfs-Path response header as a kubo config flag, default off. Unsafe: the legacy value cannot represent every UnixFS file name, so it must only be used to facilitate migration to Ipfs-Uri, and even when enabled the header is still skipped when the value would include non-ASCII byte sequences. Refs ipfs/specs#548 * ci: bump gateway-conformance pin * ci: gateway-conformance v0.14 * chore: boxo with IPIP-548 from boxo/main * docs: Ipfs-Uri changelog in v0.43.1 * docs: assemble v0.43.1 changelog Move the v0.44 highlights and dependency lines into a new v0.43.1 section, and add the missing entry for owner-only key exports (#11428). v0.44.md returns to an empty skeleton. * docs: note boxo v0.42.2 fixes in v0.43.1 | 3 天前 | |
remove unneeded thirdparty packages (#10871) * remove unneeded thirdparty packages Remove unnecessary packages from `thirdparty` in repo. - Remove `thirdparty/assert` (replaced by `github.com/stretchr/testify/require`) - Remove `thirdparty/dir` (replacd by `misc/fsutil`) - Remove `thirdparty/notifier` (unused) | 1 年前 | |
chore: fix broken url in comment (#10606) Signed-off-by: hishope <csqiye@126.com> | 1 年前 | |
disable codeclimate 'maintainability' checks License: MIT Signed-off-by: Jeromy <jeromyj@gmail.com> | 8 年前 | |
feat: telemetry plugin (#10866) * Initial pass at Telemetry plugin Currently, IP Shipyard, with the help of Probelab, monitor and extract Amino/IPFS public network metrics with the use of DHT crawlers and bootstrappers (via peerlog plugin). For example, we log all peer IDs seen and their AgentVersion/Addresses obtained from the `identify` protocol, which provides insights into protocol usage, total number of peers etc. We would like to increase the ability to obtain more insights from the network by collecting some more information in the future, but also to give users more control over this collection (i.e. opt-out). The information collected will not allow unique identification of anyone and is only used for aggregation. Now, this PR explores a way of moving in this direction: * A new "telemetry" fx plugin is in charge of dealing with telemetry * The FX plugin allows to plug and make decisions / take actions during the setup phase: * We can inspect whether we are using Private Networks before the libp2p.Host has been initialized. * We can send telemetry after the libp2p Host is initialized. * Everything is self-contained. Custom builds can remove the plugin altogether without needing to surgically edit the code. As for behaviour: * The user can opt-in/out via EnvVar, file in the repo path or plugin configuration. * Users on private networks or with custom bootstrappers are detected, offered a wall of text explaining why we need telemetry and invited to opt-in. Opt-out happens otherwise on a timeout (with no input). Their preferences are stored. * Users on standard settings are opted-in by default. This is the status quo in Kubo already, except they don't get a chance to opt out. The telemetry libp2p protocol is yet to be defined, but expect something similar to identify, with a protobuf being pushed to bootstrappers or to a specific telemetry node that we define. In the case of pnets, this will be done with a temporary peer. * checkpoint * telemetry plugin: second pass * On first run it generates a UUID and shows a message to the user. * UUID is persistend to "telemetry_uuid" * Sends telemetry 1 minute after boot and every 24h * LogEvent is the thing containing all the telemetry that is sent * Opt-out possible via env-var or plugin configuration * Telemetry: add changelog and environment variable documentation * docs: improved daemon message making it more obvious nothing was sent yet and that user had 15m to out-out plus some debug logs that confirm opt-out * refactor: rename IPFS_TELEMETRY_MODE to IPFS_TELEMETRY * fix: add User-Agent header to telemetry requests --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> Co-authored-by: Marcin Rataj <lidel@lidel.org> | 1 年前 | |
feat: re-enable docker sharness tests (#8808) The Docker sharness tests were disabled years ago when go-ipfs moved from Travis to CircleCI. This makes the tweaks necessary to re-enable them. The Docker image has since moved to be based on BusyBox which doesn't have the requisite wget version for the existing tests to work, so this adds some functionality to the pollEndpoint program to support polling HTTP endpoints as well. | 4 年前 | |
feat(config): AutoConf with "auto" placeholders (#10883) https://github.com/ipfs/kubo/pull/10883 https://github.com/ipshipyard/config.ipfs-mainnet.org/issues/3 --------- Co-authored-by: gammazero <gammazero@users.noreply.github.com> | 1 年前 | |
fix(fuse): switch to hanwen/go-fuse (#11272) * test(fuse): consolidate FUSE tests into test/cli/fuse Move FUSE integration tests from sharness shell scripts (t0030, t0031, t0032) and test/cli/fuse_test.go into a dedicated test/cli/fuse/ Go sub-package, ensuring all FUSE test cases run in CI. - git mv test/cli/fuse_test.go to test/cli/fuse/ (package fuse) - convert all sharness FUSE tests to Go subtests under TestFUSE: mount failure, IPNS symlink, IPNS NS map resolution, MFS file/dir creation, xattr (Linux), files write, add --to-files, file removal, nested dirs, publish-while-mounted block, sharded directory reads - add xattr helpers with build tags (linux/other) using unix.Getxattr - split make test_fuse into test_fuse_unit (./fuse/...) and test_fuse_cli (./test/cli/fuse/...) sub-targets - set TEST_FUSE=0 in test_cli so FUSE tests skip in cli-tests CI job - increase fuse-tests CI timeout from 5m to 10m for CLI tests - delete sharness t0030, t0031, t0032 (were always skipped in CI) * docs: document FUSE test split between unit and e2e Add cross-reference comments between the unit tests in fuse/readonly/, fuse/ipns/, fuse/mfs/ and the end-to-end CLI tests in test/cli/fuse/. Also fix AGENTS.md to use a temp dir for fusermount symlink instead of sudo. * ci: prevent stale FUSE mounts from failing fuse-tests On shared self-hosted runners, leftover mount points from previous runs can exhaust the kernel FUSE mount limit. - add job-level concurrency group so only one fuse-tests runs at a time - lazy-unmount stale /tmp/fusetest* mounts before running tests * ci: only symlink fusermount3 when fusermount is missing * fix(fuse): remove goroutine leak in IPNS Flush handler The Flush handler wrapped fi.fi.Flush() in a goroutine so it could return early when the FUSE context was canceled. But the goroutine kept running in the background, and when Release arrived it called Close on the same file descriptor concurrently. The two paths both entered DagModifier.Sync, racing on its internal write buffer and causing a nil pointer panic. The fix is to call Flush directly without a goroutine. The MFS flush cannot be safely canceled mid-operation anyway, so the goroutine only added the illusion of cancellation while leaking work and masking the real error. Also bumps boxo to pick up the matching defense-in-depth fix that serializes FileDescriptor.Flush and Close with a mutex. * fix(fuse): add mutex to IPNS file handle operations bazil/fuse dispatches each FUSE request in its own goroutine. The IPNS File handle had no synchronization, so concurrent Read/Write/Flush/Release calls could overlap on the underlying DagModifier which is not safe for concurrent use. Add sync.Mutex to File, matching the pattern already used by the MFS FileHandler. * refactor(fuse): remove dead File.Forget method bazil/fuse only dispatches Forget to nodes via the NodeForgetter interface. File is a handle, not a node, so this method was never called. The /mfs mount has no equivalent. * fix(fuse): flush IPNS directory after Remove and Rename The /mfs mount flushes the directory after Unlink and Rename so changes propagate to the MFS root immediately. The /ipns mount did not, leaving mutations pending until an unrelated flush. Also add an empty-directory check before removing directories, matching the /mfs mount's safety check. * fix(fuse): inherit CID builder and flush on IPNS Create New files created via the /ipns FUSE mount now inherit the CID builder from their parent directory, preventing CIDv0 nodes from appearing inside a CIDv1 tree. The directory is also flushed after AddChild so the new entry propagates to the MFS root immediately, matching the /mfs mount. * test(fuse): add IPNS Remove and non-empty rmdir tests Cover the file removal path and the empty-directory safety check added in the previous commit. TestRemoveFile verifies a created file can be removed and is gone afterwards. TestRemoveNonEmptyDirectory verifies that rmdir on a directory with children fails, and succeeds once the children are removed first. * feat(fuse): read UnixFS mode/mtime, add StoreMtime/StoreMode config All three FUSE mounts now read mode and mtime from UnixFS metadata when present, falling back to POSIX defaults when absent. Most IPFS data does not include this optional metadata. Writing mode and mtime is opt-in via two new config flags: - Mounts.StoreMtime: persist mtime on file create and open-for-write - Mounts.StoreMode: persist mode on chmod Other changes in this commit: - align default file/dir modes across /ipns and /mfs to 0644/0755 - share mode constants via fuse/mount/mode.go - convert Mounts.FuseAllowOther from bool to Flag for consistency - add Setattr to /ipns FileNode and /mfs File for chmod and touch - move dead File.Setattr from IPNS handle to FileNode (node) - bump boxo for Directory.Mode() and Directory.ModTime() getters * feat(fuse): add ipfs.cid xattr to all mounts All three FUSE mounts now expose the node's CID via the ipfs.cid extended attribute on both files and directories. The /mfs mount also accepts the old ipfs_cid name for backward compatibility. The /ipfs mount previously had a stub that returned nil for all xattrs; it now returns the correct CID. The xattr name follows the convention used by CephFS (ceph.*), Btrfs (btrfs.*), and GlusterFS (glusterfs.*). * feat(fuse): switch from bazil.org/fuse to hanwen/go-fuse v2 Replace the unmaintained bazil.org/fuse (last commit 2020) with hanwen/go-fuse v2.9.0, fixing two architectural issues that could not be solved with the old library. ftruncate now works: hanwen/go-fuse passes the open file handle to NodeSetattrer, so Setattr can truncate through the existing write descriptor instead of trying to open a second one (which deadlocks on MFS's single-writer lock). fsync now works: FileFsyncer runs on the handle directly, flushing the write buffer through the open descriptor. Previously a no-op because bazil dispatched Fsync to the inode only. mount package: - NewMount takes (InodeEmbedder, mountpoint, *fs.Options) instead of (fs.FS, mountpoint, allowOther) - mount/unmount collapses to a single fs.Mount call - fusermount3 tried before fusermount in ForceUnmount all three mounts: - structs embed fs.Inode (hanwen's InodeEmbedder pattern) - Remove split into Unlink + Rmdir (separate FUSE interfaces) - ReadDirAll replaced with Readdir returning DirStream - fillAttr helper shared between Getattr and Lookup responses - kernel cache invalidation via NotifyContent after Flush - 1s entry/attr timeout for writable mounts (matches go-fuse default, gocryptfs, rclone) - O_APPEND tracked on file handle, writes seek to end - build tags standardized to (linux || darwin || freebsd) && !nofuse tests: - replaced bazil fstestutil.MountedT with shared fusetest.TestMount - fixed TestConcurrentRW: channel drain mismatch and missing sync between write Close and read start - added TestFsync, TestFtruncate, TestReadlink, TestSeekRead, TestLargeFile, TestRmdir, TestCrossDirRename, TestUnknownXattr - added StoreMtime disabled/enabled subtests * fix(fuse): close fd on error in Open to prevent leak MFS enforces a single-writer lock, so a leaked write descriptor blocks all subsequent opens of that file until GC. * fix(fuse): detect external unmount via server.Wait Without this, IsActive stays true after `fusermount -u` and Unmount returns nil instead of ErrNotMounted. * fix(fuse): return actual error from Unlink/Rmdir, not ENOENT After confirming the child exists, an Unlink failure could be an IO error. Returning ENOENT would hide the real cause. * fix(fuse): reuse DagReader per open, pass ctx to all reads Readonly Open now returns a file handle holding a DagReader instead of recreating one per Read call. Sequential reads no longer re-traverse the DAG from the root on each kernel request. All three mounts now use CtxReadFull with the kernel's per-request context so killing a process mid-read cancels in-flight block fetches instead of letting them complete uselessly. * chore(fuse): cleanup dead code, add var comments - remove dead `_ = mntDir` in TestXattrCID - comment why immutableAttrCacheTime and mutableCacheTime are var - add TODO for using IPNS record TTL as cache timeout * chore(fuse): replace OSXFUSE 2.x check with macFUSE detection The old check tried to verify OSXFUSE >= 2.7.2 to avoid a kernel panic from 2015. It used sysctl, tried to `go install` a third-party tool at runtime, and referenced paths that no longer exist. Replace with a simple check for the macFUSE mount helper, matching the same paths go-fuse looks for. If neither macFUSE nor OSXFUSE is found, point the user to the install page. Also standardize build tags to (linux || darwin || freebsd) && !nofuse and use strings.ReplaceAll. * fix(fuse): include mountpoint path in mount errors go-fuse's fusermount errors don't include the path, so tools that check error messages for the mountpoint name couldn't tell which mount failed. * chore(ci): remove bazil fusermount workaround go-fuse finds fusermount3 natively, no symlink needed. The stale mount cleanup was for bazil's fstestutil which we no longer use. * docs: update v0.41 changelog for FUSE rewrite * chore(deps): bump boxo for full FileDescriptor serialization boxo@64be0815 extends the mutex from Flush/Close to all FileDescriptor operations (Read, Write, Seek, Truncate, Size), preventing data races on the underlying DagModifier. * chore(deps): bump boxo to merged ipfs/boxo#1133 Picks up full FileDescriptor serialization: the mutex now covers all operations (Read, Write, Seek, Truncate, Size), not just Flush and Close. * feat(fuse): CAP_ATOMIC_O_TRUNC, new integration tests Advertise CAP_ATOMIC_O_TRUNC so the kernel sends O_TRUNC inside Open instead of doing a separate SETATTR(size=0) first. Without this, the kernel's SETATTR needs to open a write descriptor inside Setattr, which deadlocks on MFS's single-writer lock. Move kernel cache invalidation from Flush to Release because mfsFD.Close (in Release) is where the final DAG node is committed. Upgrade go-fuse to latest for ExtraCapabilities support. New tests for both MFS and IPNS: - TestOpenTrunc, TestSeekAndWrite, TestOverwriteExisting - TestTempFileRename, TestVimSavePattern, TestRsyncPattern (skipped pending rename-over-existing and cache fixes) * fix(fuse): rename-over-existing, bump boxo for flushUp race fix IPNS Rename now unlinks the target before AddChild, matching MFS. Without this, renaming onto an existing name returned "directory already has entry". Bump boxo to pick up the flushUp unlinked-entry fix (ipfs/boxo@8ae46d5): when a file descriptor outlives its directory entry (FUSE RELEASE racing with RENAME), flushUp no longer re-adds the stale name. Unskip TestTempFileRename and TestRsyncPattern on both mounts. * fix(fuse): unskip VimSavePattern, bump boxo for setNodeData fix boxo@552d8e7 fixes File.setNodeData dropping content links when updating metadata (mode, mtime). chmod or touch after write no longer makes the file appear empty. Unskip TestVimSavePattern on both mounts. Remove debug logging and temporary test functions added during investigation. * fix(fuse): build tags for cross-compilation go-fuse does not compile on windows/openbsd/netbsd/plan9. Move WritableMountCapabilities (which imports go-fuse) from mode.go (no build tag) to caps.go (platform-gated). Align build tags on fusetest and core/commands/mount stubs so unsupported platforms don't pull in go-fuse transitively. * fix(test): use fusermount3 in CLI FUSE tests The doUnmount helper hardcoded fusermount, but systems with only fuse3 installed have fusermount3. Try fusermount3 first, matching what go-fuse and our ForceUnmount already do. * feat(fuse): symlink support on writable mounts Add NodeSymlinker to MFS and IPNS directories. Symlinks are stored as UnixFS TSymlink nodes in the DAG, the same format used by `ipfs add` for directories containing symlinks. The readonly /ipfs mount already rendered existing symlinks; now /mfs and /ipns can create them too. The target string is cached at Lookup time to avoid re-parsing the DAG node on every Readlink call. Symlink permissions are always 0777 per POSIX convention (access control uses the target's mode). * fix(fuse): checked type assertion in MFS Rename The direct type assertion on newParent could panic if the kernel passed a non-directory inode. Use a checked assertion with EINVAL fallback, matching the type-switch pattern in the IPNS mount. * fix(test): add missing continue in stress test Missing continue after error sends let execution fall through to nil type assertions (read.(files.File)) that would panic on error. Also cancel the context before continuing to avoid leaking it. * fix(fuse): return error from Readdir when DAG.Get fails Abort the directory listing instead of silently omitting the unretrievable entry. Callers get EIO, which is more honest than a partial listing that hides missing blocks. * docs: remove duplicate fsync bullet in changelog * ci: clean up stale FUSE mounts in fuse-tests job On shared self-hosted runners, leftover mounts from crashed runs can exhaust the kernel mount_max limit. Lazy-unmount kubo-test and harness temp mounts before and after tests. * chore(deps): bump boxo to merged ipfs/boxo#1134 Picks up flushUp unlinked-entry guard and setNodeData content link preservation. * docs: add build tag comments, normalize tag style Add a one-line comment above every //go:build directive explaining why the constraint exists. Normalize tag style: positive platform constraints first, then feature flags/negations. Simplify redundant expressions. * fix(fuse): add Setattr to directories for chmod and mtime Tools like tar and rsync call utimensat on directories after extraction. Without Setattr on Dir, this returned ENOTSUP. Add Setattr to Dir (MFS) and Directory (IPNS) that handles mode and mtime the same way as the file-level Setattr. When StoreMtime or StoreMode is disabled the call succeeds silently, matching the file-level behavior. * docs: clarify directory support and spec link for StoreMtime/StoreMode - mention that touch and chmod work on both files and directories - note tar and rsync as practical use cases - link to UnixFS spec for optional metadata storage * fix(fuse): use proper mode conversion, document 9-bit limit Use files.UnixPermsToModePerms and files.ModePermsToUnixPerms for converting between FUSE kernel mode (unix 12-bit layout) and Go's os.FileMode (different bit positions for setuid/setgid/sticky). The UnixFS spec supports all 12 permission bits, but boxo's MFS layer (File.Mode, Directory.Mode) exposes only the lower 9. FUSE mounts are always nosuid so the upper 3 bits would have no effect. Add TestSetuidBitsStripped to both mounts confirming the behavior. * feat(fuse): symlink Setattr with mtime persistence Wire the backing mfs.File into the FUSE Symlink struct so Setattr can call SetModTime when StoreMtime is enabled. boxo's File methods (SetModTime, ModTime) already work on TSymlink nodes since they operate on the FSNode protobuf without checking the type. Without Setattr, rsync -a fails with "failed to set times" on symlinks. Every major FUSE filesystem (gocryptfs, rclone, sshfs, s3fs) implements Setattr on symlinks for this reason. Mode is always 0777 per POSIX convention, so chmod requests are silently accepted but not stored. * fix(fuse): return EIO instead of panicking on unknown node type Replace panic with log.Errorf + syscall.EIO in IPNS Directory.Lookup for unexpected MFS node types. Also remove duplicate comment block on File.Flush. * docs: update FUSE docs for go-fuse migration - fuse.md: replace stale OSXFUSE section with macFUSE, remove obsolete go-fuse-version tool, fix broken FreeBSD sudo echo, update xattr example to ipfs.cid with CIDv1, add mode/mtime section, add unixfs-v1-2025 tip, add debug logging section, add TOC, link to hanwen/go-fuse - changelog: refine bullet wording, link to fuse.md - config.md: fix double space, update fuse.md link text - experimental-features.md: fix double space, soften wording - README.md: add FUSE to features list and docs table * refactor(fuse): extract shared writable types and test suite Extract duplicated code from fuse/mfs and fuse/ipns into a shared fuse/writable package, and consolidate duplicated tests into a reusable suite in fuse/fusetest. - fuse/writable: Dir, FileInode, FileHandle, Symlink types with all FUSE interface methods, shared by both mounts - fuse/fusetest: RunWritableSuite with helpers, exercised by both mfs and ipns via mount-specific factories - fix cache invalidation race: NotifyContent in Flush (synchronous) in addition to Release (async), so stat after close sees new size - drop deprecated ipfs_cid xattr, log error guiding users to ipfs.cid - mfs_unix.go: 632 -> 19 lines (thin wrapper over writable.Dir) - ipns_unix.go: 795 -> 170 lines (Root + key resolution only) - mfs_test.go: 1183 -> 95 lines (factory + persistence test) - ipns_test.go: 1309 -> 162 lines (factory + IPNS-specific tests) - tests that were only in one mount now run on both * feat(fuse): add macOS-specific mount options Set volname, noapplexattr, and noappledouble on macOS via PlatformMountOpts, applied in NewMount so all three mounts benefit automatically. - volname: shows mount name in Finder instead of "macfuse Volume 0" - noapplexattr: suppresses Finder's com.apple.* xattr probes - noappledouble: prevents ._ resource fork sidecar files * fix(fuse): detect symlinks in readdir, fix stale refs Readdir on writable mounts now checks the underlying DAG node type for TFile entries, reporting S_IFLNK for symlinks instead of regular file. This makes ls -l and find -type l work correctly. - writable: Readdir checks SymlinkTarget for TFile entries - writablesuite: add SymlinkReaddir regression test - readonly: add TestReaddirSymlink regression test - test/cli/fuse: fix stale bazil.org/fuse reference in doc comment * fix(fuse): normalize deprecated ipfs_cid xattr to ipfs.cid Getxattr for the old "ipfs_cid" name now returns the CID instead of ENOATTR, keeping existing tooling working during the deprecation period. A log error is emitted on each access to nudge migration. * fix(fuse): serialize concurrent reads on readonly file handles The go-fuse server dispatches each FUSE request in its own goroutine. On files larger than 128 KB the kernel issues concurrent readahead Read requests on the same file handle, racing on the shared DagReader's Seek+CtxReadFull sequence and corrupting its internal state. Add sync.Mutex to roFileHandle (matching the existing pattern in writable.FileHandle) and lock in Read and Release. - fuse/readonly/readonly_unix.go: add mu sync.Mutex to roFileHandle - fuse/readonly/ipfs_test.go: add TestConcurrentLargeFileRead - fuse/fusetest/writablesuite.go: add LargeFileConcurrentRead to shared writable suite (exercised by both /mfs and /ipns tests) * fix(fuse): bypass MFS locking for read-only opens MFS uses an RWMutex (desclock) that holds RLock for the lifetime of a read descriptor and requires exclusive Lock for writes. Tools like rsync --inplace open the same file for reading and writing from separate processes, deadlocking on this mutex. For O_RDONLY opens, create a DagReader directly from the current DAG node instead of going through MFS. The reader gets a point-in-time snapshot and never touches desclock, so writers proceed independently. - fuse/writable/writable.go: add roFileHandle with DagReader for read-only opens, add DAG field to Config - fuse/mfs/mfs_unix.go: pass ipfs.DAG to writable Config - fuse/ipns/ipns_unix.go: pass ipfs.Dag() to writable Config - fuse/fusetest/writablesuite.go: add ConcurrentReadWrite test exercising simultaneous read and write on the same file * fix(fuse): support truncate(path, size) without open fd Open a temporary write descriptor in Setattr when the kernel sends a size change without a file handle (the truncate(2) syscall, as opposed to ftruncate(fd) which passes the handle). Previously this returned ENOTSUP. - fuse/writable: open, truncate, flush, close in Setattr else branch - fuse/fusetest: add TruncatePath to the shared writable suite - test/cli/fuse: add end-to-end truncation test covering ftruncate(fd), syscall.Truncate(path), and open(O_TRUNC) through a real daemon * ci(fuse): get stack traces on test hangs The fuse-tests job was being silently cancelled by GitHub at 10min because Go's per-test timeout (5m) was the same order as the job timeout, and GOTRACEBACK=single hid the hung goroutines anyway. - shrink TEST_FUSE_TIMEOUT to 4m so Go's panic fires first - shrink job timeout-minutes to 6 (normal run is ~3min) - set GOTRACEBACK=all so the panic dumps every goroutine, not just the timer * fix(fuse): fill attrs in FileInode.Setattr response Without this, the kernel could cache zero attrs after a chmod, touch, or ftruncate until AttrTimeout (1s) expired. Dir.Setattr and Symlink.Setattr already fill out.Attr; FileInode.Setattr now matches. * docs(config): clarify Mounts.IPNS writability scope Only directories backed by keys the node holds are writable. All other names resolve via IPNS to read-only symlinks into the /ipfs mount. * fuse: review cleanup for go-fuse migration Final pass on #11272 addressing review feedback. - writable: panic in NewDir if Config.DAG is nil. Both call sites already supply it, but a nil value silently fell back to the MFS path in FileInode.Open, re-introducing the rsync --inplace deadlock the read-only fast path was added to fix. - writable: document Dir.Rename non-atomicity. Source unlink happens before destination add, so any failure between the two loses the source. An atomic fix requires changes in boxo/mfs. - writable: add unit test locking in that Symlink.Setattr accepts a mode-only request without erroring and does not store the requested mode (POSIX symlinks have no meaningful permission bits). - docs/config: correct StoreMode default modes; the previous text listed 0666 for files, which the code never uses. * docs(config): list StoreMtime and StoreMode in Mounts TOC * fix(fuse): fill EntryOut attrs in Dir.Create and Dir.Mkdir Without this, fstat on the file handle returned by Create reports mode 0 and size 0 for up to AttrTimeout (1s), because the kernel caches the empty attrs from the Create response. Path-based stat goes through Lookup which already fills attrs, so the bug only shows up via fstat. Mirrors the same fix already applied to FileInode.Setattr. Dir.Mkdir gets the same fillAttr treatment for consistency, plus a TODO noting that boxo's mfs.Directory.Mkdir accepts no mode arg so the caller's mode is dropped on creation. Adds CreateAttrsImmediate and MkdirAttrsImmediate to the shared writable suite to guard both paths against future regressions. * fix(fuse): map context cancellation to EINTR in read paths When a userspace process is killed mid-read (Ctrl-C, SIGKILL on a stuck cat) the kernel sends FUSE_INTERRUPT and go-fuse cancels the per-request context. fs.ToErrno does not recognise context.Canceled and falls through to "function not implemented", which the kernel cannot act on. Map context.Canceled and DeadlineExceeded to EINTR so the syscall is correctly aborted. - mount/errno.go: new ReadErrno helper used by all context-aware read paths in both readonly and writable mounts - readonly: applied to Node.Open, Node.Readdir, roFileHandle.Read - writable: applied to FileInode.Open, FileHandle.Read, roFileHandle.Read - readonly/ipfs_test.go: TestReadCancellationUnblocks guards the contract via a blocking DagReader fake; without ReadErrno the test reports "function not implemented" instead of EINTR * test(fuse): add OExcl, DirRename, SparseWrite, FsyncCrossHandle Coverage gaps in the shared writable suite: - OExcl: lock files and atomic-create patterns rely on the second open with O_CREATE|O_EXCL failing with EEXIST - DirRename: previously only file rename and cross-dir file rename were tested; this exercises Rename on a directory inode - SparseWrite: WriteAt past the end of an empty file must report the correct size and return zeros for the gap - FsyncCrossHandle: a reader on a fresh fd must see data flushed by fsync on the writer fd, not just after close * test(fuse): cover external unmount on /ipns and /mfs Previously TestExternalUnmount only exercised /ipfs, leaving the goroutine that watches fuse.Server.Wait() untested for the other two mounts. Refactor into a table-driven test that runs the same fusermount/umount-then-IsActive flow against all three mounts. Switch to coremock.NewMockNode so the node is online: doMount only attaches the /ipns mount when node.IsOnline is true, and the table needs all three populated. * fix(commands): align 'ipfs mount' output columns MountCmd's LongDescription has "MFS mounted at:" with two spaces so the column lines up with the 4-char "IPFS" and "IPNS" rows above, but the runtime encoder and the daemon's startup print used a single space and produced misaligned output. Bring both runtime sites in line with the help text, and update the two existing test fixtures (test/cli/fuse and the sharness test-lib helper that t0040-add-and-cat.sh still uses) to expect the aligned form. * fix(fuse): invalidate kernel cache on Fsync FileHandle.Fsync only flushed the MFS file descriptor and left the kernel's cached attrs and content for the inode untouched. A fresh reader on the same path then saw the size cached from the original Create response (zero), reading zero bytes regardless of how much the writer had synced. Mirror the cache invalidation already done in Flush via inode.NotifyContent(0, 0) so a writer that fsyncs while another process opens the file (vim then a follow-up cat, IDE then a language server) sees consistent state. Sharpen the FsyncCrossHandle assertion to report the size delta on failure; the bug surfaced as got=0/want=500 only after switching from bytes.Equal to require.Equal. * chore(gitignore): ignore test_fuse_unit and test_fuse_cli json output The new test_fuse_unit and test_fuse_cli make targets emit test/fuse/fuse-unit-tests.json and test/fuse/fuse-cli-tests.json respectively, the same gotestsum --jsonfile pattern that test_unit and test_cli already use. Add them to the same .gitignore section so a local test run does not leave the working tree dirty. * test(fuse): end-to-end coverage with real POSIX tools Adds TestFUSERealWorld in test/cli/fuse/realworld_test.go: a single shared-daemon test with 18 subtests that exercise the writable /mfs mount through the actual binaries users invoke (sh, cat, seq, wc, ls, stat, cp, mv, rm, ln, readlink, find, dd, sha256sum, tar, rsync, vim). Each subtest verifies the result both via the FUSE filesystem and via 'ipfs files read|stat|ls' so both views agree. Synthetic payloads default to 1 MiB + 1 byte so multi-chunk read/write paths are exercised, not just single-chunk fast paths. External tools are required, not optional: a missing binary fails the test loudly so a CI image change cannot silently turn the suite green. The whole-suite TEST_FUSE gate is the only place a developer is allowed to skip. runCmd forces LC_ALL=C so locale-sensitive tool output (date formats in 'ls -l', decimal separators in 'wc', localized error messages, find/ls collation) is deterministic regardless of the runner's locale settings. One shared daemon across all 18 subtests keeps total runtime under two seconds; isolation comes from per-subtest subdirectories under the mount. | 4 个月前 | |
client/rpc: migrate go-ipfs-http-client | 3 年前 | |
feat(ci): docker linting (#10927) * feat(ci): docker linting adds hadolint to validate dockerfile best practices configures project-specific rules in .hadolint.yaml * fix(ci): enable hadolint console output adds verbose and tty format to see linting results in CI logs * test: trigger hadolint warning remove --no-install-recommends to test CI output * fix(ci): fail hadolint on warnings stricter linting to catch all best practice violations * fix: add --no-install-recommends to apt-get reduces image size by avoiding unnecessary packages * refactor: use WORKDIR instead of cd in dockerfile replaces cd commands with WORKDIR for cleaner dockerfile removes unnecessary hadolint ignore rules DL3003 and DL3009 * chore: simplify hadolint config removes unnecessary override rules for cleaner config | 11 个月前 | |
chore(mailcap): document rational Co-authored-by: Peter Rabbitson <ribasushi@protocol.ai> | 6 年前 | |
docs: fuse test cleanup and /tmp symlink loops Both failure modes showed up while running the FUSE suite locally, and both outlive the test process that caused them. A mount left behind by a killed or deadlocked test blocks every later stat() of it in uninterruptible D state, which hangs find /tmp, the tmpwatch cron, and backups. Releasing those processes needs a connection abort; the unmount alone does not do it. A symlink loop left in /tmp sends fuser into an unbounded readlink() walk, so the daily tmpwatch --fuser cron pins a core and /tmp stops being cleaned. | 8 天前 | |
chore: start v0.44.0 dev cycle Opens development for the next release after v0.43. - version.go -> 0.44.0-dev - add docs/changelogs/v0.44.md stub and link it in CHANGELOG.md | 1 个月前 | |
docs: add contribution and stability guardrails Give a new maintainer the guardrails that keep contributions safe as people come and go. AGENTS.md carries the hard rules a change must not cross; CONTRIBUTING.md carries the why and who the project is for. - AGENTS.md: stability rules for the /api/v0 RPC, the HTTP Gateway, default CIDs, and protocol changes via IPIP, each with a way to verify it; boxo-vs-kubo boundary; configurable-endpoint rule for user agency; note that CLI/RPC reference docs are generated from help text; PRs need a description and tests, test/cli over sharness; release tooling is off-limits outside a release - CONTRIBUTING.md: what Kubo optimizes for (the self-hoster first), and a pointer to AGENTS.md for humans working with an LLM assistant | 1 个月前 | |
chore: go 1.26.5 and deps (#11400) * chore: bump go to 1.26.5 latest 1.26.x patch; keeps go.mod, sub-module go.mods, and the Dockerfile GO_VERSION default in sync. * chore(deps): bump go-ipld-cbor, go-ds-measure part of the ipfs-ecosystem dependabot group (#11395); go-test in that group is skipped since master is already on v0.4.1. - go-ipld-cbor v0.2.1 -> v0.3.0 - go-ds-measure v0.2.2 -> v0.2.3 | 1 个月前 | |
Create FUNDING.json (#10734) Add FUNDING.json to participate in Optimism rPGF Co-authored-by: Adin Schmahmann <adin.schmahmann@gmail.com> | 1 年前 | |
fix: require gmake IPFS doesn't build correctly under non-GNU make implementations. | 6 年前 | |
chore(license): fix broken link to apache-2.0 Fixes #9130 | 4 年前 | |
Create LICENSE-APACHE | 7 年前 | |
License update to dual MIT and Apache 2 This series of commits aims to update go-ipfs to follow a dual-licensing best practice based on research into open-source licensing by @ianjdarrow. He recommends a dual MIT and Apache 2.0 license - > This has two major benefits: > - There are concerns in the open source community about whether the MIT license leaves users vulnerable to patent infringement claims. We think the pure legal risk is small, but the way the open source community interacts with our project is really important. It makes sense to pick the license that makes the largest number of people comfortable. - There's now no reason to adopt a separate DCO, since the Apache-2 license grant addresses the same issue. > Why use a dual license, instead of just Apache-2? The Apache-2 license is incompatible with the GPLv2 license, which includes things like the Linux kernel. With a dual license, GPLv2 projects can just use the MIT license instead. Our goal is to make our software available to as many projects as possible, so we'd rather adopt a licensing scheme that doesn't exclude anyone. In addition to these commits, we also need to get an explicit OK from current and past contributors to give their consent to relicensing - which will happen in an issue thread. | 7 年前 | |
feat: add default makefile to call gnumake * gmake will prefer GNUmakefile over Makefile. * Other make implementations will read Makefile and then call gmake. | 6 年前 | |
docs: guide for running on low-memory devices (#11427) * docs: guide for running on low-memory devices Add docs/production/low-memory.md with tuning for 8 GiB devices like Raspberry Pi: GOMEMLIMIT, systemd MemoryHigh/MemoryMax, and Provide.DHT.MaxWorkers, plus how to verify with cgroup memory pressure. - README.md, docs/README.md: link the guide - docs/config.md: link from lowpower profile and ResourceMgr notes, caution against setting Swarm.ResourceMgr.MaxMemory too low - misc/systemd: commented example limits pointing at the guide * docs: announcement sizing and dht client advice Extend the low-memory guide with measured announcement rates, the 48h record expiry floor, and a worked 10M-CID example (MaxWorkers=6, Interval=32h). Advise client-only routing (autoclient), disabling the AutoNAT and relay services, and warn against enabling the accelerated DHT client on constrained hardware. - docs/environment-variables.md: document GOMEMLIMIT - docs/config.md: broaden lowpower profile pointer to the guide | 17 天前 | |
fix(mfs): fix fsync deadlock, set attrs, disable default caching (#11255) * fix(MFS): fix deadlock, attrs, caching * unmount ipns and mfs in mount tests; allow offline * set attrs Uid, Gid, and Valid for readonly and /ipns * doc: update changelog * fix(fuse): maximize kernel cache for immutable /ipfs paths /ipfs content is addressed by CID and never changes, so kernel attribute caching is safe and avoids unnecessary FUSE round-trips. Also sets uid/gid on Root.Attr for consistency. * docs: move FUSE changelog to v0.41 highlights * fix(fuse): make IPNS fsync a no-op Calling fsync on a file opened through /ipns deadlocks and eventually panics, taking down the entire IPNS mount. The Fsync handler called mfs.File.Flush(), which tries to open a second write descriptor on the same file. Only one write descriptor can exist at a time (desclock is exclusive), and the first one from Open is still held. The new one blocks forever waiting for the lock. After the FUSE timeout, Release tries to close the original descriptor and hits a nil pointer panic in DagModifier.Sync. Make Fsync a no-op, matching the MFS mount. Data gets flushed when the file is closed. Also improve the MFS Fsync comment to explain the same constraint. * fix(fuse): set uid/gid on IPNS symlinks The "local" symlink in /ipns showed uid=0 gid=0 (root) while directories and files showed the daemon's uid/gid. Set uid/gid and disable attr caching to match other mutable IPNS nodes. Also add TODO comments across all three FUSE mounts for using Mode and Mtime from UnixFS records when present, and for wiring IPNS record TTL into attr cache duration. * fix(fuse): return empty listing for empty directories IPNS Directory.ReadDirAll and readonly Node.ReadDirAll returned ENOENT when a directory had no children. An empty directory still exists, it just has nothing in it. Return an empty slice instead. MFS already handles this correctly. The readonly Root.ReadDirAll correctly returns EPERM (you can't list all of /ipfs). The IPNS Root.ReadDirAll always has entries (peer keys), so it was never affected. This matters for /ipfs because empty directories are valid content-addressed objects (e.g. QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn is a well-known CID of an empty UnixFS directory). * fix(fuse): always sync MFS writes to root on close The Sync flag was computed as `req.Flags|fuse.OpenSync > 0` (bitwise OR), which is always true because fuse.OpenSync is non-zero. Replace with an explicit `true` to match the IPNS mount and make the intent clear: FUSE writes must always propagate to the MFS root when the file is closed, regardless of whether the caller set O_SYNC. * docs: update FUSE changelog for new fixes * test(fuse): add empty directory listing tests Verify that listing an empty directory returns an empty result instead of an error, for all three FUSE mounts: - /mfs: empty root + empty subdirectory - /ipns: empty peer directory + empty subdirectory - /ipfs: empty UnixFS directory added to the DAG * test(fuse): add append and byte-at-a-time write tests for MFS IPNS had TestAppendFile and TestMultiWrite but MFS did not. Add matching tests to cover appending to an existing file and writing one byte at a time. * ci(fuse): add dedicated FUSE test job with auto-detection Add a fuse-tests CI job that installs fuse3, sets TEST_FUSE=1, and runs FUSE unit tests. Previously these tests were compiled out by the nofuse build tag (set when TEST_FUSE=0 in the unit-tests job). Introduce fuse/fusetest package with shared test helpers: - SkipUnlessFUSE: respects TEST_FUSE env var (0=skip, 1=run) with auto-detection fallback that checks for fusermount in PATH - MountError: fatals when TEST_FUSE=1 (CI expects FUSE to work), skips when auto-detecting (local dev without FUSE) Replace the old ci.NoFuse() (checked TEST_NO_FUSE, a dead env var nobody set) and per-file maybeSkipFuseTests wrappers. On Linux, bazil.org/fuse hardcodes "fusermount" but modern distros only ship "fusermount3". The CI job creates a symlink; the auto-detect gives a helpful skip message when only fusermount3 is found locally. * fix(fuse): handle EINTR on close in IPNS concurrent write test TestConcurrentWrites was flaky because Go's goroutine preemption signal (SIGURG) can interrupt the FUSE FLUSH inside close(), returning EINTR. The write itself already succeeded and the kernel will still send RELEASE to the daemon, so the data is safe. Replace os.WriteFile with explicit open/write/close so we can ignore EINTR on close while still catching real errors. * fix(fuse): resolve bare file CIDs on /ipfs mount Accessing a file by its CID at the /ipfs FUSE mount root returned ENOENT because ProtoNodeConverter cannot handle UnixFS file ADLs. Decode dag-pb blocks directly from bytes instead. Closes https://github.com/ipfs/kubo/issues/9044 * fix(fuse): fix same-directory rename on /mfs Renaming a file within the same MFS directory left the source behind. The directory's entry cache was re-synced before the old name was removed. Unlink the source before AddChild to match the working IPNS pattern. * test(fuse): add mixed dag-pb/raw directory test Covers the scenario from https://github.com/ipfs/kubo/issues/9044: a directory with both dag-pb and raw-leaf children read through the /ipfs FUSE mount. * test(fuse): remove redundant testing.Short() checks SkipUnlessFUSE(t) already handles skipping via TEST_FUSE. The testing.Short() guard was a second skip gate that served no purpose since FUSE tests only run under make test_fuse. * fix(fuse): get DAG node before unlinking source in rename Move GetNode() before Unlink() in both mfs and ipns Rename so that a GetNode() failure does not leave the source entry already removed. Also add FUSE test instructions to AGENTS.md. * ci: skip fuse3 install when fusermount exists Self-hosted runners persist state, so after the first run fuse3 and the symlink are already in place. Skip apt-get update and install entirely when fusermount is in PATH. --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com> Co-authored-by: Marcin Rataj <lidel@lidel.org> | 4 个月前 | |
chore: refresh issue templates and docs links Rewrites the security policy around a single reporting route and links it from the issue template chooser. Reports go to security@ipfs.io by email. - SECURITY.md: drops the stale go-ipfs issue link and the "heavy development" framing, adds what to do if nobody replies - ISSUE_TEMPLATE: security contact link, listed first - ipfs.tech/help returned 404, replaced with docs.ipfs.tech/community | 24 天前 | |
chore: clean up after circleci removal | 3 年前 | |
doc: remove out of date documentation | 7 年前 | |
fix: listen on loopback for API and gateway ports in docker-compose.yaml (#8773) | 4 年前 | |
fix: Ipfs-Uri gateway header (IPIP-548) (#11437) * feat: Ipfs-Uri gateway header (IPIP-548) Bump boxo to the IPIP-548 implementation (ipfs/boxo#1209): gateway responses carry a canonical percent-encoded Ipfs-Uri header and stop sending the deprecated X-Ipfs-Path, which cannot represent every UnixFS file name. - sharness: CORS expects Ipfs-Uri exposed, X-Ipfs-Path gone - gateway-conformance CI pinned to the IPIP-548 test suite (ipfs/gateway-conformance#301) until a release ships - reverse-proxy doc and v0.44 changelog updated Refs ipfs/specs#548 * feat: opt-in Gateway.DeprecatedXIpfsPath Expose boxo's opt-in for the legacy X-Ipfs-Path response header as a kubo config flag, default off. Unsafe: the legacy value cannot represent every UnixFS file name, so it must only be used to facilitate migration to Ipfs-Uri, and even when enabled the header is still skipped when the value would include non-ASCII byte sequences. Refs ipfs/specs#548 * ci: bump gateway-conformance pin * ci: gateway-conformance v0.14 * chore: boxo with IPIP-548 from boxo/main * docs: Ipfs-Uri changelog in v0.43.1 * docs: assemble v0.43.1 changelog Move the v0.44 highlights and dependency lines into a new v0.43.1 section, and add the missing entry for owner-only key exports (#11428). v0.44.md returns to an empty skeleton. * docs: note boxo v0.42.2 fixes in v0.43.1 | 3 天前 | |
fix: Ipfs-Uri gateway header (IPIP-548) (#11437) * feat: Ipfs-Uri gateway header (IPIP-548) Bump boxo to the IPIP-548 implementation (ipfs/boxo#1209): gateway responses carry a canonical percent-encoded Ipfs-Uri header and stop sending the deprecated X-Ipfs-Path, which cannot represent every UnixFS file name. - sharness: CORS expects Ipfs-Uri exposed, X-Ipfs-Path gone - gateway-conformance CI pinned to the IPIP-548 test suite (ipfs/gateway-conformance#301) until a release ships - reverse-proxy doc and v0.44 changelog updated Refs ipfs/specs#548 * feat: opt-in Gateway.DeprecatedXIpfsPath Expose boxo's opt-in for the legacy X-Ipfs-Path response header as a kubo config flag, default off. Unsafe: the legacy value cannot represent every UnixFS file name, so it must only be used to facilitate migration to Ipfs-Uri, and even when enabled the header is still skipped when the value would include non-ASCII byte sequences. Refs ipfs/specs#548 * ci: bump gateway-conformance pin * ci: gateway-conformance v0.14 * chore: boxo with IPIP-548 from boxo/main * docs: Ipfs-Uri changelog in v0.43.1 * docs: assemble v0.43.1 changelog Move the v0.44 highlights and dependency lines into a new v0.43.1 section, and add the missing entry for owner-only key exports (#11428). v0.44.md returns to an empty skeleton. * docs: note boxo v0.42.2 fixes in v0.43.1 | 3 天前 | |
chore: start v0.44.0 dev cycle Opens development for the next release after v0.43. - version.go -> 0.44.0-dev - add docs/changelogs/v0.44.md stub and link it in CHANGELOG.md | 1 个月前 | |
feat: derive AgentSuffix from build origin (#11341) Fork builds previously announced as plain `kubo/<ver>/<commit>`, indistinguishable from upstream in ecosystem dashboards. When `Version.AgentSuffix` and `--agent-version-suffix` are both unset, kubo now derives a default from the build origin so fork traffic self-identifies in the swarm. - mk/git.mk, cmd/ipfs/Rules.mk: normalize `git remote get-url origin` to `host/org/repo` and inject as `buildOrigin` ldflag - version.go: ImplicitAgentSuffix prefers buildOrigin, falls back to debug.ReadBuildInfo Main.Path; suffixFromForkPath strips known forges (github, gitlab, codeberg, bitbucket) and trailing `/kubo` - cmd/ipfs/kubo/daemon.go: use as fallback when explicit values empty - AGENTS.md: state builds must use `make build` so ldflags are set - docs/config.md: document the implicit-suffix behavior Co-authored-by: Guillaume Michel <guillaumemichel@users.noreply.github.com> | 2 个月前 |
Kubo: IPFS Implementation in Go
The first implementation of IPFS.
What is Kubo? | Quick Taste | Install | Documentation | Development | Getting Help
What is Kubo?
Kubo was the first IPFS implementation and is the most widely used one today. It takes an opinionated approach to content-addressing (CIDs, DAGs) that maximizes interoperability: UnixFS for files and directories, HTTP Gateways for web browsers, Bitswap and HTTP for verifiable data transfer.
Features:
- Runs an IPFS node as a network service (LAN mDNS and WAN Amino DHT)
- Command-line interface (
ipfs --help) - WebUI for node management
- HTTP Gateway for trusted and trustless content retrieval
- HTTP RPC API to control the daemon
- HTTP Routing V1 client and server for delegated routing
- FUSE mounts for mounting
/ipfs,/ipns, and/mfsas local filesystems (experimental) - Content blocking for public node operators
Other IPFS implementations: Helia (JavaScript), more...
Quick Taste
After installing Kubo, verify it works:
$ ipfs init
generating ED25519 keypair...done
peer identity: 12D3KooWGcSLQdLDBi2BvoP8WnpdHvhWPbxpGcqkf93rL2XMZK7R
$ ipfs daemon &
Daemon is ready
$ echo "hello IPFS" | ipfs add -q --cid-version 1
bafkreicouv3sksjuzxb3rbb6rziy6duakk2aikegsmtqtz5rsuppjorxsa
$ ipfs cat bafkreicouv3sksjuzxb3rbb6rziy6duakk2aikegsmtqtz5rsuppjorxsa
hello IPFS
Verify this CID is provided by your node to the IPFS network: https://check.ipfs.network/?cid=bafkreicouv3sksjuzxb3rbb6rziy6duakk2aikegsmtqtz5rsuppjorxsa
See ipfs add --help for all import options. Ready for more? Follow the command-line quick start.
Install
Follow the official installation guide, or choose: prebuilt binary | Docker | package manager | from source.
Prefer a GUI? Try IPFS Desktop and/or IPFS Companion.
Minimal System Requirements
Kubo runs on most Linux, macOS, and Windows systems. For optimal performance, we recommend at least 6 GB of RAM and 2 CPU cores (more is ideal, as Kubo is highly parallel).
Important
Larger pinsets require additional memory, with an estimated ~1 GiB of RAM per 20 million items for reproviding to the Amino DHT.
Caution
Systems with less than the recommended memory may experience instability, frequent OOM errors or restarts, and missing data announcement (reprovider window), which can make data fully or partially inaccessible to other peers. Running Kubo on underprovisioned hardware is at your own risk.
For running Kubo on constrained hardware such as a Raspberry Pi, see Kubo on low-memory devices.
Official Prebuilt Binaries
Download from https://dist.ipfs.tech#kubo or GitHub Releases.
Docker
Official images are published at https://hub.docker.com/r/ipfs/kubo/:
🟢 Release Images
Use these for production deployments.
latestandreleasealways point at the latest stable releasevN.N.Npoints at a specific release tag
$ docker pull ipfs/kubo:latest
$ docker run --rm -it --net=host ipfs/kubo:latest
To customize your node, pass config via -e or mount scripts in /container-init.d.
🟠 Developer Preview Images
For internal testing, not intended for production.
master-latestpoints atHEADofmastermaster-YYYY-DD-MM-GITSHApoints at a specific commit
🔴 Internal Staging Images
For testing arbitrary commits and experimental patches (force push to staging branch).
staging-latestpoints atHEADofstagingstaging-YYYY-DD-MM-GITSHApoints at a specific commit
Build from Source
git clone https://github.com/ipfs/kubo.git
cd kubo
make build # creates cmd/ipfs/ipfs
make install # installs to $GOPATH/bin/ipfs
See the Developer Guide for details, Windows instructions, and troubleshooting.
Package Managers
Kubo is available in community-maintained packages across many operating systems, Linux distributions, and package managers. See Repology for the full list: 
Warning
These packages are maintained by third-party volunteers. The IPFS Project and Kubo maintainers are not responsible for their contents or supply chain security. For increased security, build from source.
Linux
| Distribution | Install | Version |
|---|---|---|
| Ubuntu | PPA: sudo apt install ipfs-kubo |
 |
| Arch | pacman -S kubo |
 |
| Fedora | COPR: dnf install kubo |
 |
| Nix | nix-env -i kubo |
 |
| Gentoo | emerge -a net-p2p/kubo |
 |
| openSUSE | zypper install kubo |
 |
| Solus | sudo eopkg install kubo |
 |
| Guix | guix install kubo |
 |
| other | See Repology for the full list |
Snap no longer supported (#8688)
macOS
| Manager | Install | Version |
|---|---|---|
| Homebrew | brew install ipfs |
 |
| MacPorts | sudo port install ipfs |
 |
| Nix | nix-env -i kubo |
|
| other | See Repology for the full list |
Windows
| Manager | Install | Version |
|---|---|---|
| Scoop | scoop install kubo |
 |
| other | See Repology for the full list |
Chocolatey no longer supported (#9341)
Documentation
| Topic | Description |
|---|---|
| Configuration | All config options reference |
| Environment variables | Runtime settings via env vars |
| Experimental features | Opt-in features in development |
| HTTP Gateway | Path, subdomain, and trustless gateway setup |
| HTTP RPC clients | Client libraries for Go, JS |
| Delegated routing | Multi-router and HTTP routing |
| Metrics & monitoring | Prometheus metrics |
| FUSE mounts | Mount /ipfs, /ipns, /mfs as local filesystems |
| Content blocking | Denylist for public nodes |
| Customizing | Unsure if use Plugins, Boxo, or fork? |
| Debug guide | CPU profiles, memory analysis, tracing |
| Changelogs | Release notes for each version |
| All documentation | Full list of docs |
Development
See the Developer Guide for build instructions, testing, and contribution workflow. AI coding agents should follow AGENTS.md.
Getting Help
- IPFS Forum - community support, questions, and discussion
- Community - chat, events, and working groups
- GitHub Issues - bug reports for Kubo specifically
- IPFS Docs Issues - documentation issues
Security Issues
See SECURITY.md.
Contributing
We welcome contributions. See CONTRIBUTING.md and the Developer Guide.
This repository follows the IPFS Code of Conduct.
Maintainer Info
License
Dual-licensed under Apache 2.0 and MIT: