| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
fix(frontend): use webv1 for legacy launch and tooling Update legacy fallback, shared static assets, package data, build loops, CI filters, and developer commands. Keep generated contracts and frontend storage identities unchanged; align guidance and launch fixtures. | 1 个月前 | |
fix(pid): make PiD 1.5 usable for FLUX.1 and FLUX.2 Add pinned PiD 1.5 starters for FLUX.1 and FLUX.2 (bf16 and int8) and pin the Qwen-Image ones to the same commit. Calibrate the decode working-memory estimate to the device-level peak, per allocator, so the cache evicts enough. Show the PiD Decode mode in Generate for every PiD base, not only SDXL. | 27 天前 | |
fix(loaders): refuse quantization payloads a plain scale fold cannot decode An MXFP8 grid is E8M0 exponent bytes, so folding it as a multiplier made every weight ~127x too large at the right shape and dtype, silently, in three folds. The shared ComfyUI fold also took nvfp4 — whole and missing its global scale — multiplying packed codes by the block grid and logging a dequantized weight. PiD checked foreign scales only when int8 markers were present, so a purely scaled-fp8 checkpoint had its fp8 codes copied into float32 parameters. | 1 个月前 | |
feat(flux3): add FLUX.3 Image Klein and Klein Base (#9883) Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com> | 6 天前 | |
fix(model loaders): ignore unexpected checkpoint keys, report them at debug only (#9581) * fix(model loaders): ignore unexpected checkpoint keys, report them at debug only Single-file loaders disagreed, loader by loader, about what an unexpected key from `load_state_dict(strict=False)` means: some raised, one warned, most ignored it silently, and the rest used `strict=True` and let torch raise. The ones that hard-failed turned any harmless extra tensor an exporter happened to serialize into a user-facing crash that needed a code change and a release — Anima went through this twice (#9201, #9402) for tensors the model does not need and the official checkpoint does not contain. Per the team decision on #9437, extra keys are now reported at DEBUG and otherwise ignored everywhere. New `invokeai/backend/util/state_dict_loading.py` holds the single policy: - `log_unexpected_keys()` — DEBUG only, never raises. - `load_state_dict_ignoring_extras()` — a drop-in for `strict=True` that keeps the strictness that matters (every required parameter must be filled, shape mismatches still raise) and drops the strictness that only produces whack-a-mole. - `reject_incomplete_load()` — the meta-device completeness sweep, generalized out of krea2. Stronger than `missing_keys` for models built under `init_empty_weights()`: immune to non-persistent buffers and tied weights. Every previously existing missing-key guard is preserved exactly; only the unexpected-key policy changed. `flux.py`'s bare `assert len(unexpected_keys) == 0` — which carried no message and was stripped entirely under `python -O` — is gone with it. Two consequences worth calling out: - `configs/pid_decoder.py` rejected unexpected keys at *identification* time, deliberately mirroring the loader ("both are fatal there"). Left alone, the PiD relaxation would have been unreachable and the installer would refuse a file that now loads fine. It keeps refusing non-string keys, which `load_state_dict` genuinely cannot survive. - Anima's unexpected-key `RuntimeError` was its only hard load-time guard, so it is replaced with the meta-device sweep rather than dropped — otherwise an incomplete checkpoint would fail mid-inference with "Cannot copy out of meta tensor" instead of at load time. `wan.py::_raise_for_incompatible_keys` deliberately keeps raising: Wan derivatives (Animate, S2V, Fun-Camera) are supersets whose extra branches are the feature the checkpoint exists for, not exporter noise, and it strips the benign extras before that check. Closes #9437 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V4nLTdWBQaaDUFVLx9TyZH * docs(model loaders): fix two stale unexpected-key comments Review follow-up for #9581. The ideogram4 fp8 encoder branch and the z_image SDNQ Qwen3 loader still described unexpected checkpoint keys as fatal, but both now go through helpers that log them at DEBUG and ignore them (`load_fp8_state_dict`, `raise_on_incomplete_sdnq_load`). Update the comments to match the code; missing-key strictness is unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NxYxw5RUtuRTBRWTn79M9d --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Jonathan <34005131+JPPhoto@users.noreply.github.com> | 1 个月前 | |
feat(pid): load NVIDIA PiD 1.5 decoders - PiD decoders are identified and built by generation, read from the LQ projection width: v1.5 adds the per-token gate, replicate padding, PiT injection, rope_ref 2048 and FLUX.2's latent unpatchify. - Undistilled v1.5 teacher files are rejected by name; Comfy-Org's size-named presets are recognised. - Adds PiD 1.5 Decoder Qwen-Image as a starter model and documents the second generation. | 1 个月前 | |
Add opt-in PiD memory optimizations (#9460) * perf(pid): chunk pixel activations * perf(pid): gate memory optimizations * chore: openapi schema * refactor(pid): rename memory optimization setting * Fix generated JSON settings file * fix(pid): make the memory optimization actually free VRAM, and say what it costs Addresses every point from the review of #9460. The setting freed activation memory that the cache then withheld anyway. `estimate_pid_decode_working_memory` was not flag-aware, so a decode that peaks at 1.5 GiB reserved the unoptimized 3.9 GiB; the cache takes max(working_mem_bytes, device_working_mem_gb) and subtracts that from the weight budget, so the saving never became weight residency - it only avoided a hard OOM, on precisely the low-VRAM machines this feature exists for. The estimate now takes the flag, and each node reads the setting once and feeds both the estimate and the decode from it, so the two cannot drift apart. Calibrated against measured peaks (RTX 4090, fp32 PidNet, bf16 autocast, 4 steps, B=1): 1024px 509 MiB 1536px 934 MiB 2048px 1533 MiB which is 85.3 * U + 167 MiB, not a pure multiple of the output area - chunking bounds the per-block activations to a fixed working set, so a single scaling constant would under-reserve at small sizes or over-reserve at large ones. The constants carry ~15% headroom. Below the chunk size the pixel blocks run unchunked, so the estimate clamps to the unoptimized one rather than charging for a working set that is never allocated. The documented cost was the wrong cost. "at the cost of slower decoding" is not observable - 2.78 s either way at 2048px, median of 3 with warmup - while the cost that is real went unmentioned: the option changes the decoded image. Both parts are non-bit-exact and the few-step sampler amplifies them (~43 dB PSNR end-to-end, visually indistinguishable, not reproducible against an unoptimized decode). The setting description and both docs pages now state that, with the measured VRAM numbers. The equivalence test could not fail. It asserted `assert_close` at pixel_hidden_size=4 / BL=8 / chunk=3 on the CPU, while the shipped path is CUDA under bf16 autocast with BL in the thousands. Measured at production dimensions: CPU fp32 is bit-identical (including B=2, where chunks straddle images), CUDA fp32 differs by 9.5e-07, CUDA bf16 by 1.57e-02 - systematic, not noise, both paths being internally deterministic. So chunking is exact as *mathematics*; the divergence is cuBLAS picking different kernels for 1024-row slices. `test_pid_chunked_equivalence.py` now pins both halves: exactness on the CPU, and an absolute tolerance contract on CUDA with 3x headroom (relative tolerances are meaningless here - activations cross zero, so max|rel| reaches 1e3 on elements whose absolute error is one bf16 ULP). Two review points did not survive measurement, and are documented rather than "fixed": - The fp32 `_velocity_to_x0` branch is kept. Fusing the multiply-subtract in fp64 is bit-identical to the default expression but frees nothing (288 MiB either way at 2048px), so the 192 MiB is bought entirely with precision - max|diff| 4.8e-07 per call, 8.6% of the flag's total saving. That is a fair trade for someone who opted into trading quality for VRAM, but it is now written down, in the function, in the setting description and in the docs. - The doubled `adaLN_modulation` per chunk stays. Global attention sits between the two halves, so reusing the slices means holding them for every chunk - the full-resolution tensor the path exists to avoid (536 MiB in bf16 at 2048px). Projecting only the needed slices is no better: they are interleaved per pixel position, so selecting them gathers rows of a 1536x24576 weight, ~50 MiB per call. The recompute is ~9.9 TFLOP per 2048px decode, about 4% wall clock - and the honest source of the "slower decoding" the setting advertises. Observability: a decode with the flag on now logs the resolution, the patch-token count and whether chunking actually engaged. The setting is server-wide and never enters image metadata, so this is the only record that a given decode ran optimized, and the only feedback that a yaml-only, restart-required knob took effect at all. Tests: AST sweeps assert that every module building a `PiDDecodeConfig` forwards the flag, estimates working memory for the same mode, and reads the setting exactly once - so an eighth PiD node is covered the day it lands. Plus estimate behaviour (shrinks when enabled, keeps the fixed term, never exceeds the unoptimized estimate, still returns 0 for unsupported backbones), batch > 1 across chunk boundaries, and the two paths the flag must not reach: the discriminator feature extraction returns before the pixel blocks, and context parallelism is unreachable in this codebase (its only caller lives in a vendored class InvokeAI never instantiates) - both pinned so a refactor cannot quietly put them under the flag. All new tests mutation-verified: breaking the chunked assembly, making the estimate ignore the flag, dropping the fixed term, and omitting the flag from a node's decode or estimate each fail at least one test. tests/app + tests/backend/pid: 2202 passed. The 9 failures are the pre-existing network-dependent ones in test_model_install / test_load_api / test_download_queue. * test(pid): fix a chunking test that was both unportable and partly vacuous CI caught this on macos-default py3.11; every other job in the matrix was cancelled by fail-fast. Test-only change, no production code touched. Two separate mistakes, both mine: 1. The CPU comparison asserted `torch.equal`. That held on x86-64 with MKL and failed on macOS/Accelerate. Splitting a GEMM along its row dimension can select a micro-kernel with different K-blocking, so bit-exactness there is a property of the BLAS, not of the chunking. Only reassociation-closeness is portable. 2. Worse, and only found while investigating the first: the `batch_size=1` parametrization never entered the chunked path at all. The dispatch guard is `BL > chunk_size`, and 512px with B=1 puts BL at exactly 1024 - so it compared the unchunked path against itself and passed for the wrong reason. Verified by spying on `_forward_chunked`: zero calls. Both cases now demonstrably chunk - 768px/B=1 (BL 2304, boundaries inside one image) and 512px/B=2 (BL 2048, boundaries straddling images) - and a context manager fails the test if `_forward_chunked` is not entered, so the comparison cannot silently empty out again. Bit-equality is replaced by a signal-relative bound, calibrated rather than guessed. At these dimensions the signal is ~5.7, so one fp32 ULP is ~6.8e-07: correct code, x86-64/MKL max|diff| = 0 attention contribution off by 1e-6 max|diff| = 7.2e-07 (1.3e-07 relative, sub-ULP) attention contribution off by 1e-4 max|diff| = 1.0e-05 (1.9e-06 relative, ~15 ULP) 1e-5 relative is ~84 ULP: above any BLAS reassociation, four orders of magnitude below a structural break. The docstring states what that gives up - a uniform scaling error below ~2e-06 relative is indistinguishable from legitimate reassociation and no portable test can claim it - and what it still guards, which is the bug class that matters. Mutation-verified against realistic breakage: an off-by-one on the last chunk, a wrong `s_cond` slice, and skipping the chunked path each fail 5 of the 7 tests. 98 passed locally, ruff clean. * fix(pid): account for batch size in memory estimate * test(pid): avoid unsupported Windows CPU bf16 * fix(pid): account for autocast cache in memory estimate * docs(pid): explain working memory floor * fix(pid): correct working memory calibration * test(pid): guard against cache-term regression --------- Co-authored-by: Alexander Eichhorn <alex@eichhorn.dev> | 2 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 1 个月前 | ||
| 27 天前 | ||
| 1 个月前 | ||
| 6 天前 | ||
| 1 个月前 | ||
| 1 个月前 | ||
| 2 个月前 |