| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
Add __init__.py file to scripts dir for pytest | 1 年前 | |
scripts: add allocate_vram script Allocates the specified amount of VRAM, or allocates enough VRAM such that you have the specified amount of VRAM free. Useful to simulate an environment with a specific amount of VRAM. | 1 年前 | |
docs: add comments to classifiers stuff | 1 年前 | |
fix(qwen): estimate Qwen Image VAE working memory so the cache frees room before decode/encode (#9305) * fix(qwen): estimate VAE working memory so the cache frees room before decode/encode The Qwen Image l2i/i2l invocations called `model_on_device()` without a `working_mem_bytes` estimate, unlike the SD/SDXL path. The model cache therefore only reserved the default `device_working_mem_gb` and never evicted the resident transformer/text encoder before the VAE decode. On a near-full card (e.g. Qwen Image Edit Q8_0 with transformer + text encoder resident) the decode then OOMs trying to allocate its working set into the fragmented remainder. Add `estimate_vae_working_memory_qwen_image()` and pass it into both the decode and encode paths so the cache makes room (evicting other models when needed) before the operation runs. The constant is calibrated against a measured decode on an AMD W7900: at 1248x832 the decode grew CUDA reserved memory by ~10.06 GiB (implied constant ~5082), rounded up to 5500 for headroom. It tracks peak *reserved* (not just allocated) memory so that whenever the cache declines to free room (free >= estimate) the decode is still guaranteed to fit. Encode uses ~half, matching the other estimators (not independently measured). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(qwen): cover VAE working-memory estimate is passed to cache Address review feedback from @Pfannkuchensack on #9305: - Add test_qwen_image_working_memory.py mirroring the z-image pattern, asserting both decode and encode paths call model_on_device with the estimated working_mem_bytes (regression guard for the OOM fix). - Clarify the qwen estimator comment: the encode constant is not independently measured (half of decode, matching siblings' ratio) and should be recalibrated against a measured encode. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(qwen): recalibrate VAE working-memory constants from a measured grid Add scripts/calibrate_qwen_vae_working_memory.py, a backend-portable (CUDA/ROCm) harness that measures peak reserved-memory growth for VAE decode/encode across a resolution grid, one fresh subprocess per point. Calibrating on an AMD W7900 (fp16) showed the encode constant was wrong: the previous 2750 ("half of decode") under-estimated by ~2x at every measured resolution, the exact OOM mode Qwen Image Edit (which encodes a real image) would hit. Raise encode 2750 -> 6300. Decode 5500 is confirmed safe across the full 512^2..2048^2 range and left unchanged. The grid also showed memory is super-linear in area above ~1792^2 (an attention term) and non-monotonic (likely an SDPA-backend crossover on ROCm); both documented in the estimator. Constants are the conservative ROCm side and will be max-merged with a pending NVIDIA/CUDA run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(qwen): branch VAE working-memory constants by backend (ROCm vs CUDA) Calibrating the same fp16 grid on an NVIDIA card showed CUDA reserves ~2x (decode) to ~4x (encode) less than ROCm: the Qwen VAE is attention- heavy, and CUDA's Flash/efficient attention is O(area) and flat while the ROCm math-attention fallback is O(area^2). The backends diverge far more than any headroom, so a single constant either under-estimates on ROCm (OOM) or massively over-budgets CUDA (needless eviction). Select constants via torch.version.hip: decode: ROCm 5500 / CUDA 2900 encode: ROCm 6300 / CUDA 1600 Each verified to cover its measured grid (19 points/backend) with ~8% headroom. The CUDA run also confirms the linear model holds with Flash attention (the ROCm super-linear/non-monotonic behavior is a math- attention artifact), and that "encode is half of decode" is CUDA-only. Add parametrized tests asserting the constant selected for each (operation, backend) so a refactor can't silently swap them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(backend): ruff * calibrate: support single-file Qwen Image VAE checkpoints The calibration script only loaded the Qwen VAE from a diffusers directory via from_pretrained, so passing a single .safetensors file failed. Add _load_vae, which loads a directory as before and handles a single-file checkpoint by loading the state dict directly: a strict load for the diffusers layout, falling back to convert_wan_vae_to_diffusers for the original Qwen-Image/Wan release layout (downsamples/residual/ time_conv keys) before retrying. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Alexander Eichhorn <alex@eichhorn.dev> | 2 个月前 | |
Add opt-in low-VRAM mode for Wan generation (#9462) * feat(video): optimize Wan memory usage * chore: openapi schema * Added additional optimizations * fix(wan): address video memory optimization review * test(cache): keep VRAM budget test CPU-safe * fix(wan): address memory optimization review * fix(wan): calibrate VAE from checkpoint files * feat(wan): add tiled VAE calibration mode * fix(wan): align VAE calibration with estimator | 20 天前 | |
ci: check aarch64 dependency resolution in uv-lock-checks (#9357) * ci: assert uv.lock keeps torch installable on linux/aarch64 Nothing in CI covered aarch64, so a dependency bump could silently re-break the PyPI fallback that linux/aarch64 relies on for torch and torchvision. That already happened once, when a ROCm bump narrowed tool.uv.environments. Check the lockfile rather than re-resolving: tool.uv.environments must still admit aarch64, every torch extra must resolve torch and torchvision there, and the pinned versions must ship linux/aarch64 wheels for each supported Python. Narrowing tool.uv.environments or dropping the extras' aarch64 fallback pins both leave uv lock --locked green, so neither was caught before. * fix(ci): survive percent-encoded wheel URLs in aarch64 lock check The PyTorch WHL indexes percent-encode the `+` of local versions in wheel URLs, so a marker regression that pins aarch64 torch to a WHL index crashed `parse_wheel_filename` with a raw traceback instead of a verdict -- in exactly the still-locks-cleanly state the check exists for. Unquote the filename, and turn any remaining unparseable name into a one-line failure instead of a traceback. Also from review: look up version-less dependency entries (uv omits version/source when a package resolves to a single version across the lockfile), check every dependency entry matching the aarch64 environment rather than the first, and cap `--with packaging` below 26 so a parsing-strictness bump can't change the verdict on its own. --------- Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com> | 1 个月前 | |
docs: add comments to classifiers stuff | 1 年前 | |
feat: add native Intel XPU (torch.xpu) device support (#9401) * feat(backend): add Intel XPU (torch.xpu) device support Additive xpu branches only: device selection and normalization, float16 default, VRAM queries with a passthrough-VM fallback (missing SYCL free-memory aspect), fp8 layerwise casting via a runtime probe, VAE auto-tiling, partial loading, stats/OOM handling, multi-GPU parallel session execution (device enumeration, config/API validation, worker pinning, and the generation-device options endpoint), and the auxiliary image utilities (depth/SAM/DINO pipelines accept xpu instead of falling back to CPU; cache clearing is device-agnostic). CUDA (incl. ROCm), MPS, and CPU behavior unchanged. Verified end to end on Arc Pro B70 hardware, including dual-GPU worker startup. * test(backend): add XPU coverage for TorchDevice Mock-based, mirroring the CUDA/MPS suites: device choice, dtype, normalize, the xpu_mem_get_info fallback branches, and multi-GPU generation_devices resolution/validation/labeling on XPU. Also makes the auto-without-CUDA generation-devices test hermetic on XPU machines. * build: add [xpu] extra torch 2.7.1+xpu / torchvision 0.22.1+xpu / pytorch-triton-xpu 3.3.1 from the torch-xpu index, gated to linux-x86_64 and win_amd64; uv.lock regenerated. * feat(backend): extend idle-GPU text encoder offload to XPU The idle-device arbiter and the session processor's borrow path both gated on `device.type == "cuda"`, so on a multi-XPU system no device ever registered and `offload_text_encoders_to_idle_gpus` (enabled by default) silently did nothing: encoders kept churning the denoise model in and out of VRAM. Register and lend XPU devices alongside CUDA. MPS is deliberately excluded -- it is always a single shared device, so there is never another GPU to borrow. Verified on a dual Intel Arc Pro B70 host: a text encoder node now runs on the idle GPU while the session denoises on the other ("Running compel on idle device xpu:0 (session device xpu:1)"). * feat(ui): show the executing GPU for XPU sessions Queue items already persist the executing device generically (e.g. "xpu:1"), but both readers dropped it: the session event only forwarded devices starting with "cuda", and the frontend index parser only matched /^cuda:(\d+)$/. On a multi-XPU system the progress circle and queue-item badges were therefore always blank. Accept indexed XPU devices in both places, and correct the queue-item field description, which claimed the device is set only on CUDA. * fix(mm): gate Krea 2 fp8 encoder casting on fp8 storage support The Qwen3-VL encoder kept its fp8 storage only on CUDA, so elsewhere an fp8 checkpoint was loaded as full bf16 (~8.9GB instead of ~4.4GB) and thrashed partial loading when sharing a GPU with a large transformer. Reuse the existing cached `_device_supports_fp8_storage` probe, which already backs the layerwise-casting path. It returns True unconditionally on CUDA, so CUDA behaviour is unchanged. * chore: label XPU devices by index in load logs and fp8 help text Model load lines printed the device index only for CUDA, so every model on a multi-XPU host logged as a bare "xpu device", making it impossible to tell the GPUs apart. The FP8 Storage tooltip likewise claimed CUDA-only support. * test: cover XPU config validation, progress device and fp8 probe Three paths changed by this branch had no coverage: - The `device` field pattern was untested. `test_device_choice_xpu` looks like it covers it, but the config model does not enable `validate_assignment`, so assigning `config.device` skips validation entirely; only constructing the model exercises the pattern. Added constructor-based valid/invalid cases. - `generation_devices` validation was parametrized for cuda/cpu/mps only. - The progress event's device field, which now reports XPU sessions. Also cover `_device_supports_fp8_storage`, which gates FP8 storage in both the generic layerwise-casting path and the Krea 2 encoder: CUDA answers True without probing, CPU is rejected, and a failing XPU probe returns False instead of raising. Each new test was verified to fail when the corresponding fix is reverted. * fix(nodes): recognise XPU out-of-memory errors in the Anima VAE retry The Anima VAE decode catches OOM and retries once with tiling, which caps peak allocation. Detection matched `torch.cuda.OutOfMemoryError` or the words "out of memory" in the message, so it missed XPU entirely: torch's XPU backend does not raise a recoverable `torch.OutOfMemoryError` on exhaustion, it surfaces the Level Zero/UR result code as a plain RuntimeError -- and `UR_RESULT_ERROR_OUT_OF_DEVICE_MEMORY` contains no spaces, so the existing substring never matched. The decode therefore failed outright instead of retrying tiled. Match the `*_OUT_OF_DEVICE_MEMORY` / `*_OUT_OF_HOST_MEMORY` spellings (both UR and ZE prefixes) alongside the existing conditions, and fold the cuDNN/cuBLAS checks into the same case-insensitive comparison. Extends the existing parametrized retry test with the three XPU spellings; each was verified to fail before this change. Note the driver behaviour itself is not reproducible on the hardware used here -- this stack overcommits into host RAM and hangs rather than raising -- so the tests pin the classifier, not the driver. * style: wrap long vram_usage_gb ternary for ruff * fix: drop CUDA-only wording from progress device description Matches the committed openapi/schema artifacts, which already say "on a GPU". * docs: regenerate settings data for xpu device values * fix: stop xpu VRAM probe from reporting an unknown total as zero (0, 0) made the cache's available-VRAM arithmetic collapse to a constant -working_mem budget for the life of the process. Also widen the except: the failure type moves between torch releases (RuntimeError for the missing SYCL aspect, AssertionError from _lazy_init), and warn once when the blind estimate is in use. * fix: probe fp8 support on the target device, per device, without caching failures The probe allocated via an index-less "xpu", which resolves through the thread's current XPU device rather than the device being loaded onto -- so during idle-GPU encoder offload it measured the busy denoise GPU. It was also keyed on device type, letting one device decide for another, and memoised transient failures (it runs during a load, when the device may be momentarily full) with no way back but a restart. Also probe the bf16 upcast, which is the runtime path for Krea-2/FLUX. * fix: pin torch current device when borrowing an idle GPU Worker startup set both the session device and torch's per-thread current device; the offload borrow set only the former, leaving index-less allocations on the worker's own GPU. Extracted the shared helper and guarded it on backend availability. * fix: keep idle-GPU borrows within one device type generation_devices accepts a mixed list, so a cuda session could be handed an xpu device for its text encoder. * feat: detect Intel integrated GPUs via Level Zero torch exposes no is-integrated flag, but Level Zero does (ZE_DEVICE_PROPERTY_FLAG_INTEGRATED), and its loader already ships with the torch+xpu runtime -- so no new dependency and no compiled extension. Use it to keep iGPUs out of `generation_devices: auto` when a discrete GPU exists, and to stop budgeting them as dedicated VRAM (they share system RAM, like MPS). An unknown answer keeps the previous behaviour, an iGPU-only machine keeps its device, and an explicit device list can still opt one in. * feat: add xpu torch index to pins.json Gives the launcher an Intel install option instead of requiring a manual pip install of the extra. * fix: report VRAM diagnostics for the device in use All three sites dispatched on torch.cuda.is_available() first, so a mixed NVIDIA + Arc box running on xpu reported a constant 0.0 GB and logged "CUDA Memory Allocated" -- which would make XPU bug reports unactionable. * docs: record why xpu takes the CUDA VAE constants and keeps the broad OOM needle XPU SDPA was measured on Arc Pro B70 / torch 2.13+xpu: peak memory doubles when sequence length doubles (2.00x across 2048-16384; 2.0 MB at seq=16384 vs 512 MB for a materialised score matrix). So XPU is in CUDA's O(area) regime, not ROCm's math-attention regime, and the existing constants are correct rather than accidental. * fix: derive rand_device metadata from the backend's devices Was hardcoded to 'cuda' for any non-CPU noise, which is wrong on Arc. Falls back to 'cuda' when the device query has not resolved, so Nvidia metadata is unchanged. * docs: add Intel Arc install, driver and VRAM-reporting notes * fix: probe fp8 device support only when a model requests it The probe was the first statement in _should_use_fp8, so it allocated on the GPU during the first load of any model at all -- tokenizer, VAE, scheduler -- and on API/install threads it forced XPU lazy SYCL init on a thread that never generates. Moved below the exclusions. * fix: query Level Zero Sysman for driver-global free VRAM on xpu The blind estimate (total minus this process's reserved bytes) is what made _get_vram_available over-commit on a shared GPU: it feeds a formula that assumes a driver-global figure. Sysman's zesMemoryGetState reports that figure and is often available when the SYCL ext_intel_free_memory aspect is not, so try it before estimating. Measured on Arc Pro B70 with 16 GiB held by another process: Sysman reported 15.553 GiB free, the estimate 31.725 -- a 16.172 GiB error, exactly the foreign allocation. Sysman is not a guaranteed substitute (torch's query bottoms out in the same layer), so the estimate remains as a last resort. * fix: make the fp8 probe mirror the runtime cast path The storage cast happens on CPU while params are still CPU-resident, then the fp8 tensor is copied to the device and the pre-hook upcasts there. Probing all three steps on the device would pass on a build where the host->device fp8 copy or one upcast target fails, and break at forward time instead. Verified on Arc Pro B70 / torch 2.13+xpu: the full sequence works on both cards. * fix: degrade gracefully when a backend cannot name a device torch.xpu.get_device_name goes through _lazy_init, which raises AssertionError on a build without XPU. Naming is used only for labels and logs, so fall back to the device string rather than propagating. * fix: resolve an index-less device in the Sysman VRAM query Returning None for a device with no index would skip the driver-global query and fall through to the blind estimate with no visible symptom. Callers currently always pass a concrete device, so this is a latent hazard rather than a live bug. * fix: declare ctypes prototypes for the Level Zero calls Handles come back from (c_void_p * n)() as plain Python ints, and ctypes converts an undeclared int argument to a C int -- 32 bits. Any handle above 2**31 was being silently truncated; a direct test of that path segfaults. It happened to work on the B70 because the handles fit. Also: release the idle-GPU borrow if re-pinning raises (the setup was outside the try, so a failure there stranded the lock for the life of the process), and report a failing fp8 probe once per device instead of on every model load. * fix: drop the ZES_ENABLE_SYSMAN mutation from the Sysman probe Setting a process-wide environment variable from a read-only query leaks into child processes. It also bought nothing: the variable only gates Sysman on runtimes predating zesInit and must be set before Level Zero initialises, which torch has already done by then. Verified on Arc Pro B70 that zesInit succeeds with the variable unset. * refactor: tidy up the xpu additions after a cleanup review level_zero: cache the loader so it is opened and its prototypes configured once rather than twice, share the driver/device enumeration and its ordering guard between the two probes, and collapse the Sysman pair of globals into one nullable tuple. Also: fp8 support cache is a set (it only ever stored True), the pbr_maps empty_cache is routed through TorchDevice like the PR's other conversions, the shared-memory VRAM branch stops re-testing the device type it matched on, `_auto_generation_devices` partitions in one pass, and rand_device only answers when every generation device is the same accelerator. Merges three duplicate mem_get_info tests into one parametrized case and drops two fp8 probe tests fully subsumed by the cast-sequence test. * build: pin the xpu extra to torch 2.13.0 Intel's XPU backend matured considerably after 2.7.1: torch.xpu.mem_get_info() works on driver/kernel combinations where it previously raised, and the oneAPI user-space runtime ships with the wheel, so upgrading torch upgrades it too. Follows the rocm extra, which already pins ahead of cpu/cuda. pytorch-triton-xpu was renamed triton-xpu upstream. The darwin/aarch64 fallbacks stay on 2.7.1 to match the other extras and the project's torch<2.8.0 constraint on darwin. cpu/cuda/rocm exports are unchanged package-for-package (196/211/197); the only delta is a dropped "via pytorch-triton-xpu" comment annotation from the rename. * test: stub Sysman in the unknown-total xpu probe test Without it the test only passes where Level Zero cannot answer -- never on the Intel hardware the probe exists for, where Sysman returns before the tier under test is reached. * build: teach the pins check about the xpu index Its per-platform allowlist rejects anything unlisted, so pins.json's xpu entry fails it. PyTorch publishes XPU wheels for win32 and linux x86_64, matching the extra's markers. * fix: defer the xpu device pin like cuda's torch.xpu.set_device() brings up a SYCL context that holds VRAM in an otherwise idle process, the same reason the CUDA pin waits for the first claimed queue item. * docs: regenerate settings data on linux Regenerating on Windows flips two path defaults to backslashes, which the docs check rejects. * fix(mm): handle shared memory on integrated GPUs Their VRAM is system RAM, so a RAM copy doubles each model's footprint against the same pool. Drop it, letting a full load move weights rather than copy them. Keep partial loading on -- it is the only path that respects vram_available -- and raise a clean error when a full-load-only model cannot fit, instead of walking into an uncatchable OOM-kill. Warn once when a setting is overridden. Scoped to integrated XPU; CPU and MPS are unchanged. * docs: note intel device selection and integrated-GPU memory `auto` prefers CUDA on a mixed Nvidia/Arc box, and keep_ram_copy_of_weights is ignored on an integrated GPU. * chore(ui): typegen for the xpu device values * fix(mm): compare the integrated-GPU full-load guard against bytes still to move A resident model's weights occupy the same DRAM that vram_available is read from, so its total can exceed "available" precisely because it is loaded. lock() runs on every use and full_load_to_vram() is a no-op when resident; comparing the total refused the re-lock and evicted a healthy model on every other generation. Compare what full_load_to_vram() will actually move instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: run the integrated-GPU cache tests on CPU-only torch ModelCache.__init__ sizes the RAM cache from the device's total VRAM, which on an xpu execution device reads torch.xpu.get_device_properties() -- an AssertionError on the CPU-only builds CI runs, failing 9 of these tests before they reached their subject. Stub a fixed total during construction, and add a regression test for the resident-model re-lock guard. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: LexiconCode <aaronwalker@protonmail.com> Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> | 21 天前 | |
refactor: model manager v3 (#8607) * feat(mm): add UnknownModelConfig * refactor(ui): move model categorisation-ish logic to central location, simplify model manager models list * refactor(ui)refactor(ui): more cleanup of model categories * refactor(ui): remove unused excludeSubmodels I can't remember what this was for and don't see any reference to it. Maybe it's just remnants from a previous implementation? * feat(nodes): add unknown as model base * chore(ui): typegen * feat(ui): add unknown model base support in ui * feat(ui): allow changing model type in MM, fix up base and variant selects * feat(mm): omit model description instead of making it "base type filename model" * feat(app): add setting to allow unknown models * feat(ui): allow changing model format in MM * feat(app): add the installed model config to install complete events * chore(ui): typegen * feat(ui): toast warning when installed model is unidentified * docs: update config docstrings * chore(ui): typegen * tests(mm): fix test for MM, leave the UnknownModelConfig class in the list of configs * tidy(ui): prefer types from zod schemas for model attrs * chore(ui): lint * fix(ui): wrong translation string * feat(mm): normalized model storage Store models in a flat directory structure. Each model is in a dir named its unique key (a UUID). Inside that dir is either the model file or the model dir. * feat(mm): add migration to flat model storage * fix(mm): normalized multi-file/diffusers model installation no worky now worky * refactor: port MM probes to new api - Add concept of match certainty to new probe - Port CLIP Embed models to new API - Fiddle with stuff * feat(mm): port TIs to new API * tidy(mm): remove unused probes * feat(mm): port spandrel to new API * fix(mm): parsing for spandrel * fix(mm): loader for clip embed * fix(mm): tis use existing weight_files method * feat(mm): port vae to new API * fix(mm): vae class inheritance and config_path * tidy(mm): patcher types and import paths * feat(mm): better errors when invalid model config found in db * feat(mm): port t5 to new API * feat(mm): make config_path optional * refactor(mm): simplify model classification process Previously, we had a multi-phase strategy to identify models from their files on disk: 1. Run each model config classes' `matches()` method on the files. It checks if the model could possibly be an identified as the candidate model type. This was intended to be a quick check. Break on the first match. 2. If we have a match, run the config class's `parse()` method. It derive some additional model config attrs from the model files. This was intended to encapsulate heavier operations that may require loading the model into memory. 3. Derive the common model config attrs, like name, description, calculate the hash, etc. Some of these are also heavier operations. This strategy has some issues: - It is not clear how the pieces fit together. There is some back-and-forth between different methods and the config base class. It is hard to trace the flow of logic until you fully wrap your head around the system and therefore difficult to add a model architecture to the probe. - The assumption that we could do quick, lightweight checks before heavier checks is incorrect. We often _must_ load the model state dict in the `matches()` method. So there is no practical perf benefit to splitting up the responsibility of `matches()` and `parse()`. - Sometimes we need to do the same checks in `matches()` and `parse()`. In these cases, splitting the logic is has a negative perf impact because we are doing the same work twice. - As we introduce the concept of an "unknown" model config (i.e. a model that we cannot identify, but still record in the db; see #8582), we will _always_ run _all_ the checks for every model. Therefore we need not try to defer heavier checks or resource-intensive ops like hashing. We are going to do them anyways. - There are situations where a model may match multiple configs. One known case are SD pipeline models with merged LoRAs. In the old probe API, we relied on the implicit order of checks to know that if a model matched for pipeline _and_ LoRA, we prefer the pipeline match. But, in the new API, we do not have this implicit ordering of checks. To resolve this in a resilient way, we need to get all matches up front, then use tie-breaker logic to figure out which should win (or add "differential diagnosis" logic to the matchers). - Field overrides weren't handled well by this strategy. They were only applied at the very end, if a model matched successfully. This means we cannot tell the system "Hey, this model is type X with base Y. Trust me bro.". We cannot override the match logic. As we move towards letting users correct mis-identified models (see #8582), this is a requirement. We can simplify the process significantly and better support "unknown" models. Firstly, model config classes now have a single `from_model_on_disk()` method that attempts to construct an instance of the class from the model files. This replaces the `matches()` and `parse()` methods. If we fail to create the config instance, a special exception is raised that indicates why we think the files cannot be identified as the given model config class. Next, the flow for model identification is a bit simpler: - Derive all the common fields up-front (name, desc, hash, etc). - Merge in overrides. - Call `from_model_on_disk()` for every config class, passing in the fields. Overrides are handled in this method. - Record the results for each config class and choose the best one. The identification logic is a bit more verbose, with the special exceptions and handling of overrides, but it is very clear what is happening. The one downside I can think of for this strategy is we do need to check every model type, instead of stopping at the first match. It's a bit less efficient. In practice, however, this isn't a hot code path, and the improved clarity is worth far more than perf optimizations that the end user will likely never notice. * refactor(mm): remove unused methods in config.py * refactor(mm): add model config parsing utils * fix(mm): abstractmethod bork * tidy(mm): clarify that model id utils are private * fix(mm): fall back to UnknownModelConfig correctly * feat(mm): port CLIPVisionDiffusersConfig to new api * feat(mm): port SigLIPDiffusersConfig to new api * feat(mm): make match helpers more succint * feat(mm): port flux redux to new api * feat(mm): port ip adapter to new api * tidy(mm): skip optimistic override handling for now * refactor(mm): continue iterating on config * feat(mm): port flux "control lora" and t2i adapter to new api * tidy(ui): use Extract to get model config types * fix(mm): t2i base determination * feat(mm): port cnet to new api * refactor(mm): add config validation utils, make it all consistent and clean * feat(mm): wip port of main models to new api * feat(mm): wip port of main models to new api * feat(mm): wip port of main models to new api * docs(mm): add todos * tidy(mm): removed unused model merge class * feat(mm): wip port main models to new api * tidy(mm): clean up model heuristic utils * tidy(mm): clean up ModelOnDisk caching * tidy(mm): flux lora format util * refactor(mm): make config classes narrow Simpler logic to identify, less complexity to add new model, fewer useless attrs that do not relate to the model arch, etc * refactor(mm): diffusers loras w * feat(mm): consistent naming for all model config classes * fix(mm): tag generation & scattered probe fixes * tidy(mm): consistent class names * refactor(mm): split configs into separate files * docs(mm): add comments for identification utils * chore(ui): typegen * refactor(mm): remove legacy probe, new configs dir structure, update imports * fix(mm): inverted condition * docs(mm): update docsstrings in factory.py * docs(mm): document flux variant attr * feat(mm): add helper method for legacy configs * feat(mm): satisfy type checker in flux denoise * docs(mm): remove extraneous comment * fix(mm): ensure unknown model configs get unknown attrs * fix(mm): t5 identification * fix(mm): sdxl ip adapter identification * feat(mm): more flexible config matching utils * fix(mm): clip vision identification * feat(mm): add sanity checks before probing paths * docs(mm): add reminder for self for field migrations * feat(mm): clearer naming for main config class hierarchy * feat(mm): fix clip vision starter model bases, add ref to actual models * feat(mm): add model config schema migration logic * fix(mm): duplicate import * refactor(mm): split big migration into 3 Split the big migration that did all of these things into 3: - Migration 22: Remove unique contraint on base/name/type in models table - Migration 23: Migrate configs to v6.8.0 schemas - Migration 24: Normalize file storage * fix(mm): pop base/type/format when creating unknown model config * fix(db): migration 22 insert only real cols * fix(db): migration 23 fall back to unknown model when config change fails * feat(db): run migrations 23 and 24 * fix(mm): false negative on flux lora * fix(mm): vae checkpoint probe checking for dir instead of file * fix(mm): ModelOnDisk skips dirs when looking for weights Previously a path w/ any of the known weights suffixes would be seen as a weights file, even if it was a directory. We now check to ensure the candidate path is actually a file before adding it to the list of weights. * feat(mm): add method to get main model defaults from a base * feat(mm): do not log when multiple non-unknown model matches * refactor(mm): continued iteration on model identifcation * tests(mm): refactor model identification tests Overhaul of model identification (probing) tests. Previously we didn't test the correctness of probing except in a few narrow cases - now we do. See tests/model_identification/README.md for a detailed overview of the new test setup. It includes instructions for adding a new test case. In brief: - Download the model you want to add as a test case - Run a script against it to generate the test model files - Fill in the expected model type/format/base/etc in the generated test metadata JSON file Included test cases: - All starter models - A handful of other models that I had installed - Models present in the previous test cases as smoke tests, now also tested for correctness * fix(mm): omit type/format/base when creating unknown config instance * feat(mm): use ValueError for model id sanity checks * feat(mm): add flag for updating models to allow class changes * tests(mm): fix remaining MM tests * feat: allow users to edit models freely * feat(ui): add warning for model settings edit * tests(mm): flux state dict tests * tidy: remove unused file * fix(mm): lora state dict loading in model id * feat(ui): use translation string for model edit warning * docs(db): update version numbers in migration comments * chore: bump version to v6.9.0a1 * docs: update model id readme * tests(mm): attempt to fix windows model id tests * fix(mm): issue with deleting single file models * feat(mm): just delete the dir w/ rmtree when deleting model * tests(mm): windows CI issue * fix(ui): typegen schema sync * fix(mm): fixes for migration 23 - Handle CLIP Embed and Main SD models missing variant field - Handle errors when calling the discriminator function, previously only handled ValidationError but it could be a ValueError or something else - Better logging for config migration * chore: bump version to v6.9.0a2 * chore: bump version to v6.9.0a3 | 10 个月前 | |
Add scripts/extract_sd_keys_and_shapes.py | 1 年前 | |
Feature(backend): Add a command-line utility for running gallery maintenance (#8827) * (bugfix) Add a command-line utility for running gallery maintenance * chore(backend): ruff | 6 个月前 | |
Revert "Revert "New Documentation Fixes (#9061)" (#9065)" (#9066) This reverts commit b513a3d3c6ee61915ff899ef28212bcfa0d6db73. | 4 个月前 | |
feat: Video generation (#9163) * feat(model): add Wan 2.2 image generation support (Phases 0-2) Foundation + TI2V-5B MVP + A14B dual-expert MoE for Wan 2.2 image generation. Wan was trained on video but is competitive with leading open-source image models when run at num_frames=1; this commit wires that path into InvokeAI. Phase 0 — Foundation: - BaseModelType.Wan + WanVariantType {T2V_A14B, TI2V_5B} - SubModelType.Transformer2 for the dual-expert MoE - MainModelDefaultSettings per variant - step_callback Wan branch (16-channel preview; 48-channel TI2V-5B falls back to slicing first 16 channels until proper factors land) - Frontend enums + node colour Phase 1 — TI2V-5B Diffusers MVP: - Main_Diffusers_Wan_Config probe (variant from transformer_2/ + vae/config.json::z_dim, with filename heuristic fallback) - WanDiffusersModel loader (subclasses GenericDiffusersLoader) - WanT5EncoderField, WanTransformerField (with dual-expert slots), WanConditioningField, WanConditioningInfo - New invocations: wan_model_loader, wan_text_encoder, wan_denoise, wan_image_to_latents, wan_latents_to_image - FlowMatchEulerDiscreteScheduler integration with on-disk config load - RectifiedFlowInpaintExtension reused for inpaint - 5D <-> 4D shape juggling: latents stay 4D in InvokeAI's pipeline, re-add T=1 only inside the transformer call / VAE encode-decode Phase 2 — A14B dual-expert MoE: - Probe reads boundary_ratio from model_index.json - Loader emits both transformer (high-noise) and transformer_low_noise (low-noise expert at transformer_2/) for A14B - _ExpertSwapper in wan_denoise drives GPU residency between experts: high-noise for t >= boundary_ratio * num_train_timesteps, low-noise below. Only one expert locked at a time so the cache can evict the other - relies on existing CachedModelWithPartialLoad to handle oversized models on lower-VRAM GPUs. - guidance_scale_low_noise field for separate low-noise CFG override Tests: - 24 passing tests covering probe variant detection, default settings, noise sampling, end-to-end denoise on a synthetic transformer (CPU), dual-expert boundary swap, CFG branch - 1 heavy-test placeholder gated by INVOKEAI_HEAVY_TESTS=1 for the real-weights smoke test Phase 3+ deferred: standalone VAE/encoder configs, GGUF, LoRA, ControlNet, ref image, inpaint UI, frontend wiring, starter models. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(model): Wan 2.2 Phase 3 + tokenizer-load fix Phase 3 adds standalone VAE and UMT5-XXL encoder configs so users can run GGUF-quantized Wan transformers (Phase 4) without installing the full ~30 GB Diffusers pipeline. VAE configs: - VAE_Checkpoint_Wan_Config + VAE_Diffusers_Wan_Config (16-channel A14B vs 48-channel TI2V-5B, distinguished by decoder.conv_in z_dim). - 16-channel files share the AutoencoderKLWan architecture with Qwen Image; disambiguated via filename heuristic ("wan" in name -> Wan, otherwise -> Qwen Image). Mirror exclusion in QwenImage's probe. - VAELoader gets a Wan branch that builds AutoencoderKLWan(z_dim=...) via init_empty_weights, mirroring the QwenImage single-file pattern. - Existing standard VAE probe excludes both QwenImage- and Wan-style state dicts. UMT5-XXL encoder: - New ModelType.WanT5Encoder + ModelFormat.WanT5Encoder. - WanT5Encoder_WanT5Encoder_Config probes the diffusers folder layout (text_encoder/config.json with model_type=umt5, or flat layout with config.json at root). Refuses full Wan pipelines. - WanT5EncoderLoader handles both layouts and loads UMT5EncoderModel + AutoTokenizer. Component-source plumbing: - WanModelLoaderInvocation now exposes wan_t5_encoder_model and component_source pickers (mirrors QwenImage pattern). Resolution order: standalone > main (if Diffusers) > component_source. Required when the main model is a single-file format in Phase 4. Bug fix in wan_text_encoder: - Tokenizer was loading via AutoTokenizer.from_pretrained(<root>) directly, which fails for nested layouts where files live in <root>/tokenizer/. Now routed through the model cache so the registered loaders handle layout differences correctly. Frontend: - New type guards (isWanVAEModelConfig, isWanT5EncoderModelConfig, isWanMainModelConfig, isWanDiffusersMainModelConfig) and hooks/ selectors (useWanVAEModels, useWanT5EncoderModels, useWanDiffusersModels). New zSubModelType / zModelType / zModelFormat enum entries for transformer_2 and wan_t5_encoder. Tests: - 16 new tests covering z_dim detection, VAE checkpoint/diffusers probes, the bidirectional Qwen-vs-Wan filename deferral, and the UMT5 encoder probe (nested + flat + T5 + full-pipeline rejection). - Total Wan test count: 41 passing, 1 heavy-test placeholder skipped. - Full config test suite (63 tests) still passes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): unbreak frontend lint after Wan additions Five issues turned up running `make frontend-lint`: 1. wan_denoise.py used `from __future__ import annotations`, which made the `invoke()` return annotation a string ('LatentsOutput'). The InvocationRegistry's `get_output_annotation()` returns the raw annotation, so OpenAPI generation crashed with `'str' object has no attribute '__name__'`. Removed the future-import and added `Any` to the typing imports. 2. ModelRecordChanges.variant didn't list WanVariantType, so the generated schema's install/update endpoints rejected `t2v_a14b` and `ti2v_5b`. Added it. 3. Regenerated frontend/web/src/services/api/schema.ts from the live backend so it now includes BaseModelType.wan, ModelType.wan_t5_encoder, SubModelType.transformer_2, ModelFormat.wan_t5_encoder, the Wan variants, all Wan invocation types and their conditioning/transformer field types. 4. modelManagerV2/models.ts: added `wan_t5_encoder` to the category map, `wan` to the base color/long-name/short-name maps, the two Wan variants to the variant-name map, and `wan_t5_encoder` to the format-name map. 5. ModelManagerPanel/ModelFormatBadge.tsx: added `wan_t5_encoder` to FORMAT_NAME_MAP and FORMAT_COLOR_MAP. `make frontend-lint` now passes cleanly (tsc, dpdm, eslint, prettier). All 41 Wan Python tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> chore(wan): drop unused FE exports flagged by knip These were forward-compatibility wiring for Phase 9 (the FE graph builder) that has no consumers yet; knip rightly flagged them. Removed or de-exported. They'll come back when the graph builder lands and needs them. - common.ts: zWanVariantType drops `export` (still used internally by zAnyModelVariant). - types.ts: drop isWanMainModelConfig, isWanDiffusersMainModelConfig, isWanVAEModelConfig (no callers). The remaining isWanT5EncoderModelConfig is used by models.ts. WanT5EncoderModelConfig type drops `export` (still used as the type guard's narrowing target). - modelsByType.ts: drop the six unused useWan*/selectWan* hooks + selectors and their type-guard imports. `make frontend-lint` (tsc + dpdm + eslint + prettier + knip) now green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> docs(wan): use *-Diffusers HF repo names in plan The Wan-AI org publishes two flavours of each release: * Wan-AI/Wan2.2-{TI2V-5B,T2V-A14B,I2V-A14B} ← upstream native * Wan-AI/Wan2.2-{TI2V-5B,T2V-A14B,I2V-A14B}-Diffusers ← convertible The native release has _class_name=WanModel in config.json and ships weights flat at the repo root with no transformer/, vae/, text_encoder/ subdirs. It is not loadable by Diffusers' WanPipeline.from_pretrained. Update plan doc to reference the -Diffusers repos throughout (probe notes, starter-model entries) so the plumbing path matches what the Diffusers loader actually expects. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): accept 0 as 'unset' sentinel for guidance_scale_low_noise The frontend renders Optional[float] inputs with default 0 in the numeric input rather than passing null/unset. Combined with ge=1.0, this caused every wan_denoise invocation to fail Pydantic validation with "Input should be greater than or equal to 1" until the user manually entered a value (or knew to leave the field disconnected). The validation error was rejected before invocation logging, so it never showed up in the server log either - making the failure hard to diagnose. Relaxing the constraint to ge=0.0 and treating values below 1.0 as the "fall back to primary Guidance Scale" sentinel. The user's natural FE default (0) now works as expected. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): correct preview dimensions and colors for TI2V-5B Two bugs in the Wan branch of the diffusion step callback: 1. Wrong dimensions. The reported preview size hardcoded `* 8` for the spatial downscale ratio, but TI2V-5B's Wan2.2-VAE uses 16x. A 1024x1024 target was being announced to the FE as 512x512. 2. Wrong colors. The previous fallback for 48-channel TI2V-5B latents sliced the first 16 channels and applied the standard 16-channel Wan-VAE projection. Those channel layouts are unrelated, so the projection produced meaningless colors. Adding the proper Wan2.2-VAE 48-channel RGB projection matrix (and bias) from ComfyUI's Wan22 latent format, and selecting the right matrix + spatial-scale by latent channel count: 16 → A14B (Wan VAE, 8x), 48 → TI2V-5B (Wan2.2-VAE, 16x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): honor model's _class_name when building scheduler TI2V-5B's scheduler_config.json declares _class_name=UniPCMultistepScheduler with flow_shift=5.0. The previous code hardcoded FlowMatchEulerDiscreteScheduler.from_pretrained(...), which silently constructed a default-config FlowMatch instead of the UniPC the model expects. The mismatched noise schedule manifests as soft / under-denoised faces and global graininess in the final images. Now: read scheduler_config.json, look up the named class on the diffusers module, and instantiate that class via from_pretrained. UniPC and FlowMatch share the same step()/set_timesteps()/sigmas/num_train_timesteps interfaces, so the denoise loop works transparently for either. A14B continues to use FlowMatchEulerDiscreteScheduler when its scheduler config says so (its reference is FlowMatchEuler with shift=8.0). Falls back to FlowMatchEulerDiscreteScheduler defaults when no on-disk config is available. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): match diffusers WanPipeline tokenizer length and latent dtype Two divergences from the Diffusers reference that were hurting image quality (soft / grainy / distorted faces at default settings): 1. Tokenizer max_sequence_length was 226 in wan_text_encoder, but the model was trained with 512-token sequences. The upstream native config.json has text_len: 512, and Diffusers' WanPipeline.__call__ default is 512 (overriding _get_t5_prompt_embeds's stale 226 default). Wan's cross-attention sees padded zeros past the prompt's actual length but expects to be looking at a 512-position context window. 2. Latents were stored in bf16 throughout the denoise loop. Diffusers' WanPipeline.prepare_latents explicitly uses dtype=torch.float32 and only casts to the transformer's dtype right at the forward call: latent_model_input = latents.to(transformer_dtype) Storing in bf16 between steps accumulates ~40 steps of bf16 quantization on the scheduler's small per-step deltas. Now latent_dtype = torch.float32 throughout, with a per-step cast for the transformer forward pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> chore(wan): add diffusers reference comparison script scripts/wan_diffusers_reference.py runs a Diffusers-format Wan 2.2 checkpoint directly via WanPipeline.from_pretrained, with the same arguments InvokeAI's wan_denoise uses. Use to A/B against InvokeAI output when image quality is questionable. Defaults to enable_model_cpu_offload so the script fits on 16 GB cards where the full pipeline (transformer + UMT5-XXL + VAE) would otherwise OOM. --offload {model,sequential,none} controls the strategy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(model): Wan 2.2 Phase 4 - GGUF transformer support Adds single-file GGUF support for Wan 2.2 transformers, the path that makes A14B usable on consumer GPUs (~7 GB/expert at Q4_K_M instead of ~28 GB at bf16). Probe (configs/main.py): - New helpers: _has_wan_keys (Wan vs Qwen/FLUX/Z-Image fingerprint via condition_embedder.text_embedder.linear_1 + patch_embedding); _detect_wan_gguf_variant (16ch -> A14B, 48ch -> TI2V-5B from patch_embedding.weight.shape[1]); _detect_wan_gguf_expert (filename heuristic for high_noise / low_noise / none). - Main_GGUF_Wan_Config(base=Wan, format=GGUFQuantized, variant, expert). Tolerates the ComfyUI 'model.diffusion_model.' / 'diffusion_model.' prefixes via _has_wan_keys' multi-prefix scan. - Registered in factory.py. Loader (model_loaders/wan.py): - WanGGUFCheckpointModel mirrors the QwenImage GGUF pattern: gguf_sd_loader -> strip ComfyUI prefix -> auto-detect arch from state dict shapes (num_layers, inner_dim, ffn_dim, text_dim, in_channels, num_heads = inner_dim/128) -> init_empty_weights + load_state_dict(strict=False, assign=True). Loader invocation (wan_model_loader.py): - New 'Transformer (Low Noise)' picker: optional second GGUF for the A14B dual-expert MoE. Auto-swaps if the user wired the experts in the wrong order. Warns when an A14B GGUF is loaded without a paired low-noise expert (single-expert run, degraded quality). - GGUF mains require either a standalone VAE+encoder or a Diffusers Component Source (which can also supply boundary_ratio). - Diffusers main path unchanged (still pulls both experts from transformer/ + transformer_2/). Tests (tests/.../test_wan_gguf_config.py): - 14 tests across key fingerprint, variant detection, expert filename heuristic, and the full probe (A14B high/low, TI2V-5B, GGUF rejection, unrecognised state-dict rejection, explicit override). Total Wan tests: 55 passing (no regressions). FE lint clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): support QuantStack-style GGUFs and standalone Diffusers VAE The city96 Wan 2.2 GGUF repos have been removed from Hugging Face, leaving QuantStack as the surviving distributor. QuantStack ships the native upstream Wan key layout (text_embedding.0/2, self_attn/cross_attn, ffn.0/2, head.head, head.modulation, ...) rather than the diffusers naming city96 used; biases are stored as F16 rather than BF16; and the standalone Wan VAE installs as a flat AutoencoderKLWan folder which the generic loader rejects. Three fixes: 1. Probe now recognises both diffusers and native key layouts via a new _is_native_wan_layout helper; _has_wan_keys accepts either text-proj fingerprint. 2. GGUF loader converts native -> diffusers keys (mirroring diffusers' convert_wan_transformer_to_diffusers) and unwraps non-quantized GGMLTensors to plain tensors at compute_dtype. The unwrap is needed because conv3d isn't in GGMLTensor's dispatch table, so the F16 patch_embedding bias would otherwise hit conv3d against bf16 latents. 3. VAELoader gains a VAE_Diffusers_Wan_Config branch that loads AutoencoderKLWan directly; the generic path can't handle a flat single-class folder when a submodel_type is provided. Adds 12 tests covering the native layout (probe + converter + unwrap). Verified end-to-end against Wan2.2-T2V-A14B-Q4_K_M from QuantStack: 1095 tensors round-trip key-for-key against WanTransformer3DModel. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(model): Wan 2.2 Phase 5 - LoRA support Probe + config (LoRA_LyCORIS_Wan_Config): - Detects Wan LoRAs in three layouts: diffusers PEFT, native upstream PEFT (ComfyUI), and Kohya (both naming variants). - Anti-pattern guards prevent collisions with Anima (Cosmos DiT q_proj convention), QwenImage (transformer_blocks), Flux (double/single blocks), and Z-Image (diffusion_model.layers). - Optional ``expert: "high" | "low" | None`` field; auto-detected from filename (high_noise / low_noise / hyphenated / concatenated variants). Key conversion (wan_lora_conversion_utils): - Native upstream keys (self_attn/cross_attn, ffn.0/2) -> diffusers (attn1/attn2, ffn.net.0.proj / ffn.net.2). - Strips ``transformer.``, ``diffusion_model.``, ``base_model.model.transformer.`` prefixes from PEFT-style keys. - Kohya layer names mapped through an explicit longest-match table. - Output paths use diffusers naming so the LayerPatcher can resolve them against WanTransformer3DModel parameter paths. Loader integration: - Adds BaseModelType.Wan branch to LoRALoader._load_model. Invocation nodes (wan_lora_loader.py): - WanLoRALoaderInvocation: single LoRA with auto/both/high/low target field. - WanLoRACollectionLoader: list of LoRAs, auto-routed by each LoRA's recorded expert tag. - Output WanLoRALoaderOutput carries the WanTransformerField with updated ``loras`` / ``loras_low_noise`` lists. Denoise integration: - _ExpertSwapper now manages both the model_on_device context and the LayerPatcher.apply_smart_model_patches context per expert. LoRA patches are entered after device load and exited before device release, with fresh iterators per swap. - GGUF (quantized) experts request sidecar patching so GGMLTensor weights aren't touched directly. - Low-noise expert falls back to the primary loras list when ``loras_low_noise`` is empty (matches WanTransformerField semantics). Tests: 81 new tests covering probe accept/reject across formats, anti-pattern guards on competing architectures, converter round-trips for all three layouts, invocation target resolution + routing + duplicate guards, and the _ExpertSwapper lifecycle (lora context opens/closes in the right order around the device swap, quantized flag forwards, no-LoRA path skips the patch context, re-entering the same label is a no-op). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): probe Wan LoRA before Anima in the config union Native-PEFT Wan LoRAs (lightx2v's Lightning, most ComfyUI-trained Wan LoRAs) carry keys like ``diffusion_model.blocks.X.cross_attn.k.lora_A.weight``. Anima's probe matches on the bare ``cross_attn``/``self_attn`` substring — it does not require the Anima-specific ``_proj`` suffix nor any of the ``mlp``/``adaln_modulation`` Cosmos DiT markers — so these Wan LoRAs were classified as ``BaseModelType.Anima`` because Anima happened to run first. Reorder the LyCORIS section of ``AnyModelConfig`` so Wan probes first. Wan's probe is strictly more restrictive (it rejects Anima's ``_proj`` attention suffix via the anti-pattern guard added in the previous commit), so Anima LoRAs are still correctly classified after this reorder. Existing users with mis-tagged installs need to delete the affected LoRA records and reinstall. Adds two regression tests: a union-ordering assertion, and a sanity check that demonstrates Anima's probe *would* match Wan native keys if asked directly — pinning the constraint that motivates the ordering. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> chore(i18n): add Wan2.2 T5 Encoder model-manager label The frontend source already references ``modelManager.wanT5Encoder``; the locale key was added with a casing typo (``want5Encoder``). Fix the key so the Wan T5 Encoder model type renders its display name correctly in the model manager UI. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(model): Wan 2.2 Phase 7 - reference-image (I2V) conditioning Re-implementation after the first attempt — which used CLIP-vision conditioning — was reverted. Wan 2.2 I2V-A14B does NOT use a CLIP-vision encoder (the Diffusers repo ships ``image_encoder: [null, null]`` in ``model_index.json``); instead it conditions on a reference image by VAE-encoding it and concatenating the resulting latents (plus a first-frame mask) to the noise latents along the channel dim. The I2V transformer therefore has ``in_channels=36`` (16 noise + 16 ref-image latents + 4 mask) vs ``in_channels=16`` for T2V. Taxonomy: - Re-adds ``WanVariantType.I2V_A14B``. Probes: - Diffusers: ``_detect_wan_variant`` reads ``transformer/config.json::in_channels``; 36 → I2V_A14B, 16 → T2V_A14B (both share the dual-expert layout). - GGUF: ``_detect_wan_gguf_variant`` recognises ``in_channels=36`` from the patch_embedding tensor shape and emits I2V_A14B. Backend extension (``backend/wan/extensions/wan_ref_image_extension.py``): - ``preprocess_reference_image`` resizes + normalises to a 5D pixel tensor. - ``encode_reference_image_to_condition`` VAE-encodes the image and stacks a 4-channel first-frame mask on top, producing the ``[1, 20, 1, H/8, W/8]`` condition tensor the denoise loop consumes. - Mirrors diffusers ``WanImageToVideoPipeline.prepare_latents`` with ``num_frames=1`` and ``expand_timesteps=False``. Invocation node (``wan_ref_image_encoder.py``): - "Reference Image - Wan 2.2": image + VAE + width/height pickers. - Output ``WanRefImageConditioningField`` carries the condition tensor name plus the dimensions used (so the denoise step can validate dim parity). Denoise integration: - ``WanDenoiseInvocation`` gains an optional ``ref_image`` field. - Variant gate: rejects ref_image on T2V_A14B and TI2V-5B with a clear error before doing any work. - Dimension gate: rejects ref-image width/height mismatch vs denoise. - At every transformer call, concatenates the 20-channel condition tensor to the 16-channel noise latents along the channel dim before passing to the transformer (giving the 36-channel input I2V expects). Tests: 14 new across the probe, the extension, and the denoise loop. The synthetic ``_ZeroTransformer`` test stand-in now mirrors the real I2V transformer's ``in_channels=36, out_channels=16`` asymmetry by slicing its zero output back to 16 channels when the input is 36-wide. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): derive GGUF out_channels from proj_out shape (I2V support) The GGUF loader was setting ``out_channels = in_channels`` which is wrong for Wan 2.2 I2V-A14B: that variant has ``in_channels=36`` (16 noise + 16 ref-image latents + 4 first-frame mask, concatenated by the denoise loop) but ``out_channels=16`` since the transformer only predicts the noise component back. Loading an I2V GGUF would build a transformer with the wrong proj_out shape and crash: RuntimeError: Error(s) in loading state_dict for WanTransformer3DModel: size mismatch for proj_out.weight: copying a param with shape torch.Size([64, 5120]) from checkpoint, the shape in current model is torch.Size([144, 5120]). (144 = 36 * 4, 64 = 16 * 4 — patch_size=(1, 2, 2) → prod=4) Read out_channels directly from the ``proj_out.weight`` shape in the state dict. This is correct for all three Wan 2.2 variants without needing to know the variant in advance. Also tighten the num_layers fallback: T2V_A14B and I2V_A14B share 40 layers; only TI2V-5B has 30. The fallback is rarely hit in practice (the per-block count comes from the state dict scan), but the previous code would have defaulted I2V_A14B to 30 layers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(model): make Anima LoRA probe mutually exclusive with Wan InvokeAI's ``Config_Base.CONFIG_CLASSES`` is a Python ``set``, so iteration order during model probing is non-deterministic across process restarts. First-match-wins ordering in ``AnyModelConfig`` is documentation only — it has no effect on which config is iterated first. Anima's previous probe accepted any state dict containing the substring ``cross_attn`` or ``self_attn``, which collides with Wan's native LoRA key layout (``diffusion_model.blocks.X.cross_attn.q.lora_down.weight``). Both probes accepted Wan native LoRAs (including lightx2v's Lightning T2V and I2V distillations), and the ``matches.sort_key`` tiebreaker only disambiguates by ModelType, not within LoRA configs. So which config "won" depended on dict hash order — sometimes Wan, sometimes Anima. The previous mitigation reordered the AnyModelConfig union to put Wan before Anima. That worked by luck and was inherently fragile. Tighten Anima's probe to require Cosmos-DiT-exclusive subcomponents: ``mlp``, ``adaln_modulation``, or ``_proj``-suffixed attention names (``q_proj``/``k_proj``/``v_proj``/``output_proj``) — none of which appear in any Wan LoRA. Wan native uses bare ``.q``/``.k``/``.v``/``.o`` on ``self_attn``/``cross_attn``, and ``ffn.N``/``ffn.net.N`` instead of ``mlp``. The new strict detectors live alongside the original loose ones so the Anima conversion utility (which runs after probing) still works. Regression tests in ``test_wan_lora_probe_independence.py`` cover: - I2V Lightning V1 (the bug-triggering LoRA), T2V Lightning V2, Wan Kohya and Wan diffusers PEFT layouts — Wan probe accepts, Anima probe rejects. - Anima PEFT and Kohya layouts — Anima accepts, Wan rejects. - A meta-test that runs every LoRA config in CONFIG_CLASSES against the Lightning state dicts and asserts exactly one accepts — this catches ANY future probe collision, not just Wan vs Anima. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): defer expert model loading in _ExpertSwapper to avoid cache thrash The swapper used to take pre-loaded ``LoadedModel`` handles at construction: high_info = context.models.load(self.transformer.transformer) low_info = context.models.load(self.transformer.transformer_low_noise) swapper = _ExpertSwapper(high_info=high_info, low_info=low_info, ...) With dual ~9 GB A14B GGUF experts plus the ~10 GB UMT5-XXL encoder competing for the same RAM cache, the LRU policy frequently dropped one expert by the time the denoise loop swapped into it. The model manager then emitted [MODEL CACHE] Locking model cache entry ... but it has already been dropped from the RAM cache. This is a sign that the model loading order is non-optimal in the invocation code (See ... #7513). and reloaded the weights from disk (~1.2s extra per swap). Refactor the swapper to take the ``ModelIdentifierField`` plus the ``InvocationContext`` and call ``context.models.load(model_id)`` lazily inside ``get()``. Each swap obtains a fresh handle, the LRU window is small, and the warning goes away. Config metadata (used to compute ``is_quantized``) is read upfront via ``context.models.get_config()`` — that's metadata, not weights, so it doesn't put pressure on the cache. Tests: existing swapper lifecycle tests refactored to use a fake context whose ``models.load`` is logged. A new ``test_lazy_load_per_swap_not_upfront`` pins the regression — it asserts ``models.load`` is NOT called at swapper construction, only at first get() per expert. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(wan): add Phase 8 inpaint regression tests The denoise_mask wiring + RectifiedFlowInpaintExtension integration in wan_denoise.py was put in place during Phase 2/3 alongside the rest of the denoise loop. Phase 8 of the plan was about ensuring this path worked and is locked in by tests. Three new tests under TestWanDenoiseInpaint: 1. test_preserved_region_matches_init_exactly: builds a half/half mask (left = preserve, right = regenerate in user-side convention), runs full denoise with the synthetic zero-output transformer, and asserts the preserved half of the final latents equals the init exactly while the regenerated half does not. Pins the mask-inversion + per-step merge behavior. 2. test_inpaint_requires_init_latents: a mask without init latents must raise a clear ValueError — the merge has nothing to weld back to. 3. test_no_mask_path_is_unchanged: regression that adding the inpaint extension didn't perturb the non-inpaint codepath (with init latents + denoising_start=0.5 but no mask, the loop just runs img2img). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> feat(frontend): add I2V_A14B to Wan variant zod enum + manager label Phase 7 added the I2V_A14B backend variant. The frontend's zod enum (features/nodes/types/common.ts:zWanVariantType) and the model manager's variant-label map (features/modelManagerV2/models.ts) were still on the two-variant list, so: - ModelIdentifierField inputs with ui_model_variant filters on Wan couldn't list I2V models. - The model manager UI showed a raw 'i2v_a14b' string instead of the human label. Phase 9 (full linear-view wiring — type guards, hooks, params slice, graph builder, tab UI) is in progress on a follow-up commit; this lands the two small enum fixes first so the I2V probe / install paths work correctly end-to-end with the existing FE. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): Phase 9 piece #1 - linear-view T2V txt2img graph builder Adds the minimum frontend wiring needed to generate Wan 2.2 images from the linear view: - buildWanGraph.ts (new): text-to-image graph (model_loader → text_encoder × 2 → denoise → l2i). Diffusers main model only — transformer, VAE and UMT5 encoder all resolve from the same repo, so no Wan-specific params slice fields are required yet. CFG-skip branch when guidance_scale ≤ 1.0. - useEnqueueGenerate / useEnqueueCanvas dispatchers: route base === 'wan' to buildWanGraph. - graph/types.ts: add wan_l2i / wan_i2l / wan_denoise / wan_model_loader to the relevant node-type unions. - addTextToImage / addImageToImage: include wan_denoise / wan_l2i so width/height are wired correctly and the txt2img helper accepts the Wan l2i node. - isMainModelWithoutUnet: include wan_model_loader (Wan has no UNet, same as the other modern bases). - metadata.py: add wan_txt2img / wan_img2img / wan_inpaint to the generation_mode enum (img2img / inpaint pieces land next). - schema.ts: regenerated to pick up the metadata enum + new Wan invocations. Pieces left in Phase 9: params slice (standalone VAE / T5 / GGUF low-noise / LoRA / ref-image fields + selectors), img2img + I2V + inpaint branches in the graph builder, and Wan-specific UI components. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> feat(wan): Phase 9 piece #2 - GGUF support and CFG-Low control in linear view Adds the three Wan-specific params + UI controls that gate GGUF workflows plus a separate low-noise CFG slider for A14B users. Params slice: - wanTransformerLowNoise (the second-expert GGUF for A14B) - wanComponentSource (Diffusers Wan model providing VAE + UMT5-XXL when the main is a GGUF) - wanGuidanceScaleLowNoise (optional separate CFG for the low-noise expert; null = fall back to the primary CFG) Plus a `selectIsWan` selector for accordion gating. UI components: - ParamWanModelSelects.tsx (Advanced accordion): two model pickers — Transformer (Low Noise) filtered to Wan GGUF mains, and VAE/Encoder Source filtered to Wan Diffusers mains. Mirrors the ParamQwenImageComponentSourceSelect structure. - ParamWanGuidanceScaleLowNoise.tsx (Generation accordion): slider + number input with an "auto" indicator when cleared. Default 3.5 matches the diffusers reference 4.0 / 3.0 split. Wiring: - Generation accordion: ParamWanGuidanceScaleLowNoise shown when base is wan, scheduler excluded for wan (same pattern as Anima/Qwen). - Advanced accordion: ParamWanModelSelects shown when base is wan, and Wan excluded from the SD-family VAE/CFG-rescale blocks. - buildWanGraph.ts: forwards the three new params to the model loader and denoise nodes (transformer_low_noise_model, component_source, guidance_scale_low_noise) and adds them to the graph metadata. Hooks/types: - useWanDiffusersModels + useWanGGUFModels in modelsByType.ts. - isWanDiffusersMainModelConfig + isWanGGUFMainModelConfig type guards. - Three new locale strings (wanComponentSource, wanTransformerLowNoise, wanGuidanceScaleLowNoise[Auto]). GGUF workflow now works end-to-end in the linear view: pick a Wan GGUF main, set Transformer (Low Noise) to the paired second-expert GGUF, set VAE/Encoder Source to any Diffusers Wan repo (TI2V-5B is convenient at ~12 GB) — generate produces an image. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): UX polish on the Wan linear-view controls Bundles four small fixes applied during a usability review of the Wan linear-view section (piece #2): 1. **Filter Main vs Transformer (Low Noise) dropdowns by expert tag.** The Wan GGUF probe records each file's ``expert`` field (``"high"`` / ``"low"`` / ``"none"``) via filename heuristic. - ``MainModelPicker``: hides ``expert === 'low'`` Wan GGUFs so users can't accidentally wire a low-noise expert as the primary main. - Transformer (Low Noise) picker (``useWanGGUFLowNoiseModels``): shows ``expert === 'low'`` Wan GGUFs only. Diffusers Wan mains and TI2V-5B aren't affected — they don't carry the ``expert`` field on their config schema. The backend's auto-swap safety net stays in place. 2. **Match the primary CFG slider's range.** The Wan low-noise CFG slider was constrained to 1–10 while the primary CFG ranges 1–20. With the diffusers reference 4/3 split, the low-noise slider thumb sat noticeably further right than the primary — visually misleading. Both sliders now share the 1–20 range with marks at [1, 10, 20]. 3. **Label fits the form column.** "CFG (Low Noise)" → "CFG (Low)" so the slider fits cleanly next to its label instead of overlapping. 4. **Indicator state for the low-noise CFG slider.** Replaced the inline "(auto)" / "(same as cfg)" text — which kept overlapping the slider regardless of how short the label got — with an X-only reset button that's only visible when the user has set an explicit value. Absence of the X conveys auto/fallback state without any text overhang. 5. **Friendlier Transformer (Low Noise) placeholder.** "Second-expert GGUF for A14B (pair with the high-noise main)" → "Add for full detail" — concise nudge for users who haven't paired the second expert yet. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> feat(wan): Phase 9 piece #3 - linear-view img2img branch Adds Wan 2.2 image-to-image to the linear view, mirroring the Qwen Image pattern. The mode switches on the canvas state — pure-prompt runs go through addTextToImage as before; canvas runs with an init image go through addImageToImage which wires a fresh wan_i2l (Image to Latents - Wan 2.2) node between the init image and the denoise's `latents` input, honoring the existing denoise_start slider. buildWanGraph: - Drops the txt2img-only guard, branches on generationMode. - img2img: spins up a wan_i2l node and hands it to addImageToImage alongside the existing denoise / l2i / modelLoader (as vaeSource). - inpaint / outpaint still fail loudly — pieces #4-#6. graphBuilderUtils.getDenoisingStartAndEnd: - Adds 'wan' to the simple-linear case (denoising_start = 1 - denoisingStrength). Note: Wan's flow-matching schedule is "sticky" on the init compared to SDXL — users will likely need denoisingStrength ≥ 0.7 to see substantial change, matching the user-found 0.15-0.3 denoising_start sweet spot from earlier img2img testing. We may revisit this with an exponent rescale (like FLUX uses) if the response curve feels off. addImageToImage: - Adds 'wan_i2l' to the i2l-node-type union so the Wan i2l can be threaded through the shared helper. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): add wan_denoise to addImageToImage/addInpaint/addOutpaint type checks Three sibling graph-helper utilities had the same modern-base list as addTextToImage did, and the buildWanGraph img2img branch tripped one of them at canvas-Generate time: error [generation]: Failed to build graph {name: 'Error', message: 'Wrong assertion encountered'} The else-branch in each helper assumes 'denoise_latents' (the SD1.5/SDXL legacy path) and asserts that — failing for any modern base not listed above the branch. addTextToImage was already updated in Phase 9 piece #1; this catches the parallel cases that the img2img/inpaint/outpaint flows go through. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> feat(wan): Phase 9 piece #4 - linear-view inpaint and outpaint branches Wires Wan 2.2 inpaint and outpaint through the existing addInpaint / addOutpaint helpers. The backend's RectifiedFlowInpaintExtension was plumbed into wan_denoise.py back in Phase 8 (commit ab54617173); this just connects the FE. buildWanGraph: - generationMode === 'inpaint' → spin up a wan_i2l, call addInpaint with denoise + l2i + modelLoader (used as both vaeSource and modelLoader since the Wan model loader carries the VAE). - generationMode === 'outpaint' → parallel branch with addOutpaint. addInpaint: - i2l-node-type union now includes 'wan_i2l' (the addImageToImage and addOutpaint type unions already do — different union shapes). metadata.py: - generation_mode literal adds "wan_outpaint" alongside the existing wan_txt2img / wan_img2img / wan_inpaint entries. isMainModelWithoutUnet already includes wan_model_loader (Phase 9 piece create_gradient_mask when Wan is the main. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> feat(wan): Phase 9 piece #5 - linear-view I2V branch (raster as reference image) Wan 2.2 I2V-A14B models condition on a reference image whose VAE-encoded latents are concatenated to the noise along the channel dim each step (in_channels=36 on the I2V transformer). In the linear view this maps cleanly onto the existing canvas raster layer: pick an I2V model, drag an image to raster, generate. buildWanGraph: - Fetch the modelConfig early so the variant gate (i2v_a14b vs the rest) can drive the branch shape instead of being a post-hoc check. - I2V + txt2img: fail loudly ("Switch to the canvas tab and drag an image to the raster layer"). I2V models won't produce useful output without a reference, and the backend would crash trying to concatenate a missing condition tensor. - I2V + img2img: pull the raster image via the canvas compositor, wire it through a wan_ref_image_encoder (which VAE-encodes it and builds the 4-mask + 16-latent condition tensor backend-side), then feed the result into denoise.ref_image. Denoise runs from fresh noise (denoising_start=0, no init_latents) — the ref image is cross-attention/concat conditioning, not a noise-trajectory anchor. - I2V + inpaint/outpaint: fail clearly. Combining ref-image conditioning with a denoise mask is conceptually possible but the backend interaction hasn't been validated end-to-end. metadata.py: - Adds "wan_i2v" to the generation_mode literal so the metadata field on I2V renders correctly. T2V flows (txt2img / img2img / inpaint / outpaint) are unchanged for non-I2V Wan variants (T2V-A14B and TI2V-5B). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): enforce multiple-of-16 dimensions to match transformer patch grid Wan 2.2's transformer has ``patch_size=(1, 2, 2)``: it patch-embeds with stride 2 then un-patches by 2. Combined with the VAE's 8x spatial scale, canvas H/W must be a multiple of ``8 * 2 = 16`` — not just 8 — for the patch round-trip to land exactly. Otherwise the latents and noise prediction disagree by one in the spatial dim and the scheduler step fails: RuntimeError: The size of tensor a (147) must match the size of tensor b (146) at non-singleton dimension 3 (here latent_w=147 → patch_w=73 → un-patched_w=146 ≠ 147) This was silent for T2V at 1024x1024 (already a multiple of 16) but fired for I2V at non-multiple-of-16 canvas sizes. Fixes: - ``optimalDimension.getGridSize``: Wan moves from the default 8 case to the multiple-of-16 case (alongside flux / sd-3 / qwen-image / z-image which have the same patch arithmetic). The canvas bbox UI now snaps Wan dimensions to multiples of 16. - ``wan_denoise.py`` and ``wan_ref_image_encoder.py``: bump width/height ``multiple_of`` from 8 to 16. Defense-in-depth — workflow-editor users won't be able to send a non-16-aligned dim either. Existing backend tests (23 passing) still hold — 1024 is divisible by 16 so the test fixtures didn't exercise the off-by-one path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): show negative prompt box in Wan linear-view Wan was missing from SUPPORTS_NEGATIVE_PROMPT_BASE_MODELS, so the linear-view negative-prompt input was hidden even though the Wan denoise node already wires negative conditioning when CFG > 1 (buildWanGraph.ts:67-75). Adds 'wan' to the list. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> feat(wan): Phase 9 piece #6 - Wan LoRA collection in linear view Adds Wan LoRA wiring to buildWanGraph, mirroring the Qwen Image pattern. The shared LoRASelect / LoRAList UI in the linear view already filters LoRAs by the selected main model's base, so Wan LoRAs surface automatically when a Wan main is picked — no UI changes needed. addWanLoRAs (new): - Filters state.loras.loras to enabled Wan LoRAs. - For each LoRA: spawns a ``lora_selector`` node and threads it through a single ``collect`` collector. - Routes the collector into a ``wan_lora_collection_loader`` which sits between modelLoader and denoise — modelLoader.transformer → loader, then loader.transformer → denoise (rerouting the original modelLoader → denoise edge). - Emits per-LoRA metadata so PNG metadata + workflow restore work. The dual-expert routing (high-noise vs low-noise vs untagged) is handled entirely on the backend by ``WanLoRACollectionLoader`` based on each LoRA's recorded ``expert`` tag (set by the probe from the filename heuristic in piece #5 of Phase 5). The FE just hands over the bag of LoRAs; no per-list FE plumbing needed. buildWanGraph: - Calls addWanLoRAs(state, g, denoise, modelLoader) after the base transformer edge is in place. The helper is a no-op when no Wan LoRAs are enabled, so it's safe to call unconditionally. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): detect LoRA variant and filter by main model Wan 2.2 A14B (inner_dim=5120) and TI2V-5B (inner_dim=3072) LoRAs are not interchangeable — applying one against the wrong main model crashes the layer patcher with a tensor-shape error (e.g. A14B Lightning on TI2V-5B mains produced ``shape '[3072, 3072]' is invalid for input of size 26214400``). Probe Wan LoRAs' inner-dim at install time and record the family on a new ``variant`` field (``a14b`` / ``5b`` / null). The LoRA picker in the linear view hides incompatible variants when the user selects a main, and the graph builder filters any still-enabled mismatches at submit time with a warning. Untagged LoRAs (probe couldn't identify) pass through so they aren't silently hidden. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> feat(wan): ref-image panel, GGUF readiness, and auto-default sources Wan 2.2 I2V now uses the global Reference Images panel (same UX as Qwen Image Edit and FLUX.2 Klein) instead of pulling the conditioning image from a canvas raster layer. Adds: - WanReferenceImageConfig zod type + isWanReferenceImageConfig guard; integrated into the ref-image discriminated union, settings panel, layer hooks, and validators. - 'wan' added to SUPPORTS_REF_IMAGES_BASE_MODELS, but the panel only shows for the i2v_a14b variant (T2V and TI2V-5B don't consume ref images, so the panel is hidden for them). - buildWanGraph I2V branch reads the first enabled wan_reference_image from refImagesSlice; the canvas-raster-as-ref path is removed. I2V now only supports txt2img mode (canvas img2img/inpaint/outpaint assert with a clear message). GGUF Wan readiness check: GGUF mains carry only the transformer, so the loader needs a Diffusers Component Source (or standalone VAE + UMT5-XXL encoder) to resolve the VAE and text encoder. Without one, enqueue is now blocked with a clear reason. The low-noise A14B partner expert remains optional (loader falls back to the high-noise expert when it's missing). Adds standalone Wan VAE and Wan T5 Encoder selectors to the Advanced accordion (Qwen pattern). Wires them as vae_model / wan_t5_encoder_model on the wan_model_loader node — backend priority is standalone > diffusers main > component source. Auto-default on Wan selection (so GGUF users don't have to fiddle with Advanced): when the new main is a Wan GGUF, fill the Component Source, standalone VAE, and standalone T5 encoder with first available matches if not already set. Component Source is matched by variant family (A14B GGUF prefers an A14B Diffusers; TI2V-5B prefers a TI2V-5B Diffusers) since the two families use different VAE channel counts (16 vs 48); within A14B, T2V and I2V share VAE/encoder so they're interchangeable as a source. Runs on every Wan selection (including Diffusers -> GGUF switches), only fills empty slots. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): add Wan 2.2 starter models and bundle Wan 2.2 starter pack (selected when the user picks the Wan 2.2 bundle) brings up the minimal-cost path to running A14B T2V end-to-end: - Standalone UMT5-XXL encoder and A14B VAE (so GGUF mains don't need a full Diffusers download for their VAE/encoder sources). - T2V A14B Q4_K_M and Q8_0 GGUF expert pairs (high + low noise). - T2V Lightning V1.1 Seko rank-64 LoRA pair (4-step inference). Additional Wan 2.2 starter models browseable from the model manager: - Full Diffusers T2V A14B, I2V A14B, and TI2V-5B. - I2V A14B Q4_K_M and Q8_0 GGUF expert pairs + Lightning V1 LoRA pair. - TI2V-5B Q4_K_M and Q8_0 GGUFs + the 48-channel TI2V-5B VAE. Each "high noise" GGUF lists its low-noise partner plus the shared VAE and UMT5-XXL encoder as dependencies, so installing one of them pulls in everything the loader needs. QuantStack's HighNoise/LowNoise file naming and lightx2v's high_noise_model/low_noise_model.safetensors are both picked up by the existing filename heuristic in the GGUF probe. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> docs(wan): add Wan 2.2 hardware requirements Adds Wan 2.2 A14B (T2V/I2V) and TI2V-5B rows to the hardware requirements table with rough VRAM/RAM guidance per quantization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(wan): recall low-noise transformer, component source, and standalone VAE/T5 Wan-specific metadata fields embedded by the graph builder (wan_transformer_low_noise, wan_component_source, wan_vae_model, wan_t5_encoder_model, wan_guidance_scale_low_noise) had no recall handlers in features/metadata/parsing.tsx, so recalling an image's parameters would leave these fields empty. Adds a handler for each that dispatches the matching paramsSlice action and renders a row in the metadata viewer. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): add default Wan 2.2 T2V and I2V workflows Ships two default workflows in the library, tagged so they appear in "Browse Workflows" under the wan2.2 / text to image / image to image tags: - Text to Image - Wan 2.2: full T2V/TI2V-5B graph (model loader, positive + negative encoders, denoise, l2i). Exposes the five model slots, prompts, steps, dual CFG, and dimensions. - Image to Image - Wan 2.2: I2V A14B graph that adds a wan_ref_image_encoder. Exposes the reference image input plus the standard fields. Both follow default-workflow rules: IDs prefixed with default_, meta.category = "default", and no references to user-installed resources. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(videos): Phase 1 - backend video storage, records, REST API Adds a parallel video pipeline alongside the existing image pipeline so the gallery can host MP4 alongside PNGs. Implements: - New service modules (parallel to image equivalents): video_records/ record store + sqlite impl video_files/ disk file store (mp4 + first-frame webp thumb) videos/ orchestrating service board_video_records/ board <-> video association - migration_32 creates `videos` and `board_videos` tables - /api/v1/videos/ router: upload, list, get DTO, /full (with HTTP Range so HTML5 <video> seek/scrub works), /thumbnail, /metadata, star/unstar, delete, batch delete, board add/remove - LocalUrlService.get_video_url and SimpleNameService.create_video_name - imageio[ffmpeg] dep for video encode (used in later phases) - Wires all four new services into InvocationServices, dependencies.py, api_app.py, and three test fixtures Verified end-to-end against an in-memory db + tmp output dir: upload, probe, save (file + thumbnail + record), DTO build, list, delete. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(videos): Phase 2 - polymorphic gallery list endpoint Adds /api/v1/gallery/items/ and /api/v1/gallery/items/names returning a unified time-sorted stream of images + videos so the frontend can render them interleaved with a single virtualized query. - gallery_common: GalleryItem discriminated union (kind + name + shared fields + nullable video duration/fps), GalleryItemRef, names result - gallery_default: SqliteGalleryService implements UNION ALL across the images and videos tables, applying identical filters (origin/category/ is_intermediate/board_id/search) to each half; pagination via outer ORDER BY + LIMIT/OFFSET; counts are summed across the two halves - URLs are resolved at row -> DTO conversion time so each item routes to the correct /api/v1/images or /api/v1/videos endpoint - Wired into InvocationServices, dependencies.py, api_app.py, and the three test fixtures Existing /api/v1/images endpoints are unchanged so any non-gallery consumers (queue, recall, metadata workflows) continue to work as-is. Verified e2e: 2 images + 2 videos inserted in alternating order, both list_items and list_item_names return the correct interleaved order; category filter narrows to a single kind; starring an item bumps it to the top when starred_first=True. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(videos): Phase 3 - frontend RTK endpoints + MP4 upload routing Adds the typed API surface and upload integration so videos can be uploaded through the same gallery upload button that handles images. Schema: re-ran pnpm typegen against the running backend to pick up VideoDTO, VideoRecordChanges, GalleryItem, GalleryItemKind, GalleryItemRef, GalleryItemNamesResult and the two new paginated result types. RTK Query (services/api/endpoints/videos.ts) - parallel to images.ts: listVideos, getVideoDTO, getVideoMetadata, getVideoNames, uploadVideo, deleteVideo / deleteVideos, changeVideoIsIntermediate, starVideos / unstarVideos, addVideoToBoard / removeVideoFromBoard. Imperative helpers (getVideoDTO, getVideoDTOSafe, uploadVideo, uploadVideos) and the useVideoDTO convenience hook ride alongside, mirroring the image side. Tag types and invalidation: added Video / VideoList / VideoMetadata / VideoNameList / BoardVideosTotal / GalleryItemList / GalleryItemNameList to the api root. Board-affecting mutations now invalidate the polymorphic gallery list/name caches so videos and images stay coherent once the gallery wiring lands in Phase 4. Added a sibling getTagsToInvalidateForVideoMutation helper. Upload UX: useImageUploadButton.tsx's dropzone now accepts video/mp4, video/webm, video/quicktime alongside the existing image MIMEs. The drop handler splits files into image/video sets and routes each through its own mutation; a new onUploadVideo callback parallels the existing onUpload. Existing image-only callers pass through unchanged. Polymorphic gallery query endpoints + the useGalleryItemDTO hook will land with Phase 4 where they have actual consumers; the schema types they'll need are already in place under @knipignore tags. Verified: pnpm lint (knip, dpdm, eslint, prettier, tsc) all green; pnpm test 1103/1103 pass; live curl against the running dev server uploads an MP4 and serves both the webp thumbnail and the MP4 with a working HTTP Range response (206 + Content-Range). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(videos): Phase 4 - mixed gallery grid with video play badge Videos now appear in the same gallery grid as images, interleaved by created_at. Video thumbnails get a centered play-button badge so they read as videos at a glance; everything else (selection, virtualization, search, paged/virtual gallery views, keyboard nav) is unchanged. Approach: selection state stays `string[]` of names. The kind is recovered from the filename extension (.mp4 = video, anything else = image), which is reliable because the backend's SimpleNameService always emits `<uuid>.png` for images and `<uuid>.mp4` for videos. This sidesteps a 32-file cross-cut from changing the selection shape to a discriminated union, and selection is persist-denylisted so no migration is needed. Frontend: - new isVideoName helper in features/gallery/store/types - new endpoints/gallery.ts (deferred from Phase 3): useGetGalleryItemNamesQuery - new ImageGrid/GalleryItemPlayBadge: centered triangular badge over thumbnail - new ImageGrid/GalleryItemVideoStarIconButton: video-typed star toggle - new ImageGrid/GalleryVideoItem: counterpart to GalleryImage; reuses galleryItemContainerSX, GalleryItemSizeBadge (width/height-only stand-in), selection handling (single/shift/ctrl/cmd); alt-click falls through to a normal select since comparison is image-only - use-gallery-image-names now calls the polymorphic gallery names endpoint and exposes a mixed flat name list (existing callers - paged grid, search, navigation hotkeys - get the same shape) - useRangeBasedImageFetching partitions visible names by extension; images bulk-fetch via the existing getImageDTOsByNames mutation, videos dispatch individual getVideoDTO queries (no batch endpoint yet) - GalleryImageGrid's ImageAtPosition dispatches on isVideoName to render GalleryImage or GalleryVideoItem; star hotkey dispatches to the right star/unstar mutation based on kind - pruned the now-unused useGetImageNamesQuery / isImageName exports Verified: pnpm lint (knip, dpdm, eslint, prettier, tsc) all green; pnpm test 1103/1103 pass; live curl of /api/v1/gallery/items returns 57 polymorphic items with video duration populated and image duration null, /api/v1/gallery/items/names returns matching {kind, name} refs. The useGalleryItemDTO hook is intentionally deferred to Phase 5 where the polymorphic viewer is its first real consumer. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(videos): Phase 5 - inline video player in the image viewer Selecting a video now renders a polymorphic preview inside the existing viewer panel: thumbnail with a centered play button by default; clicking play swaps in an HTML5 <video controls autoplay>. Switching to a different item drops the video element back to idle (auto-pauses) and selecting an image again returns to the normal image preview. New components (features/gallery/components/ImageViewer/): - VideoPlayButtonOverlay: large centered play button with hover/shadow, used over the thumbnail in the idle state. - CurrentVideoPreview: idle/playing state machine. Resets on video_name change. The <video> src points at /api/v1/videos/i/.../full which supports HTTP Range, so seek/scrub work natively in the browser. New hook: - common/hooks/useGalleryItemDTO: polymorphic DTO resolver that dispatches between useImageDTO and useVideoDTO based on filename extension (isVideoName). Centralizes the kind-dispatch the viewer and toolbar both need. Wiring: - ImageViewer dispatches on galleryItem.kind to render CurrentImagePreview or CurrentVideoPreview. The compare-image DnD drop target is hidden when a video is selected (comparison is image-only). - ImageViewerToolbar hides the image-specific action row (CurrentImageButtons - load workflow, recall metadata, edit, etc.) and the metadata viewer toggle when a video is selected. The general-purpose ToggleProgressButton stays. Out of scope (per the plan): video deletion from the viewer (use gallery hover icons), video-specific metadata viewer, comparison-mode support for videos. Verified: pnpm lint (knip, dpdm, eslint, prettier, tsc) all green; pnpm test 1103/1103 pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(videos): accept MP4 (and other video) drops on the fullscreen dropzone The gallery-wide drag-and-drop target lives in FullscreenDropzone, not in useImageUploadButton (which only powers the upload button). It had its own hardcoded image-only zod allowlist that rejected MP4 files with "File type / extension is not supported". - Broaden the zod refines to accept video/mp4, video/webm, video/quicktime, video/x-matroska and the matching extensions - Add isVideoFile helper, split dropped files into image/video sets, and route each set through its own uploader (uploadImages / uploadVideos). Both update their respective RTK caches and invalidate the polymorphic gallery list/names. - Skip the canvas-paste fast-path for single-video drops — the canvas doesn't host videos as layers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(videos): right-click context menu on video items Adds a three-item context menu (delete, change board, download) on right-click / long-press of any gallery video item. Mirrors the image context menu's singleton-portal architecture so re-renders stay cheap. New files: - features/gallery/contexts/VideoDTOContext: small React context that scopes the active video DTO to the menu items (parallels ImageDTOContext). - features/gallery/components/ContextMenu/MenuItems/ ContextMenuItemDeleteVideo: window.confirm + deleteVideo mutation. Videos can't be referenced from canvas/nodes/refs, so the image modal's usage analysis is unnecessary; a one-step confirm matches the "minimal" scope. ContextMenuItemDownloadVideo: reuses the existing useDownloadItem hook against videoDTO.video_url / video_name. ContextMenuItemChangeBoardVideo: dispatches videosToChangeSelected and opens the (now polymorphic) ChangeBoardModal. - features/gallery/components/ContextMenu/VideoContextMenu: singleton pattern lifted from ImageContextMenu — registers gallery video elements via a Map; right-click looks up the target node and opens the menu at the cursor. Extended files: - features/changeBoardModal/store/slice: added video_names alongside image_names plus a videosToChangeSelected action. The two arrays are mutually exclusive — setting one clears the other. - features/changeBoardModal/components/ChangeBoardModal: now dispatches the matching video board mutations (add/removeVideoToBoard, plural endpoints don't exist yet so videos move one at a time — the menu acts on a single selection so this is a one-iteration loop). - features/gallery/components/ImageGrid/GalleryVideoItem: registers itself with useVideoContextMenu. - app/components/GlobalModalIsolator: mounts the singleton. Verified: pnpm lint (knip, dpdm, eslint, prettier, tsc) all green; pnpm test 1103/1103 pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): Phase 6 - Wan 2.2 T2V/I2V workflow nodes Adds two new invocation nodes that produce MP4 videos from a Wan 2.2 A14B transformer + VAE, plus the supporting plumbing. New invocations: - WanVideoDenoise (wan_video_denoise) — multi-frame counterpart to WanDenoise. Same per-step logic (CFG, MoE expert swap at the boundary timestep, LoRA patching, scheduler dispatch) — reuses _ExpertSwapper, _resolve_variant, and the scheduler/LoRA helpers from wan_denoise. Difference: the noise tensor has a real temporal dim built from num_frames, and the I2V condition is built across all latent frames (frame 0 conditioned, rest zero). Defaults match the Wan 2.2 reference: 832x480 / 81 frames / 40 steps / CFG 5.0 (high) + 4.0 (low). Inpaint / img2img are out of scope for this first cut. TI2V-5B is rejected; T2V/I2V A14B only. - WanLatentsToVideo (wan_l2v) — VAE-decodes 5D latents to RGB frames via AutoencoderKLWan (T_pixel = (T_lat - 1) * 4 + 1), then encodes an MP4 with imageio[ffmpeg] (libx264, yuv420p for browser compatibility). The temp file is moved into outputs/videos/ via context.videos.save(). Backend shared pieces: - make_noise gains num_latent_frames (default 1, backward compatible). - Added num_latent_frames_for(num_frames, scale=4) helper. - New encode_reference_image_to_video_condition mirrors diffusers' WanImageToVideoPipeline.prepare_latents with last_image=None and expand_timesteps=False: pads the reference image with zero pixel-frames, VAE-encodes the full pseudo-video, normalises, and builds the 4-channel temporal-rearranged first-frame mask. Verified numerically: 21 latent frames for num_frames=81, first latent frame's 4 mask channels = 1, rest = 0. - The existing single-frame encoder is left untouched. Schema / context: - New VideoField primitive (parallel to ImageField) and VideoOutput invocation output (width/height/num_frames/fps/duration/video). - New VideosInterface on InvocationContext with .save(source_path, width, height, duration, fps, ...) returning VideoDTO. Mirrors ImagesInterface — falls back to WithBoard / WithMetadata mixins and embeds the queue item's workflow/graph as a JSON sidecar. - WanRefImageConditioningField now carries num_frames so the denoise nodes can sanity-check the I2V condition. WanRefImageEncoder bumps to v1.1.0 and gains num_frames=1 input (use 81+ for video I2V; the encoder dispatches between the single- and multi-frame helpers). - Image WanDenoise now rejects multi-frame conditions with a clear message pointing at WanVideoDenoise. Verified: pnpm lint (5/5) green; pnpm tests (multiuser auth 122/122 + broader suite via prior runs); numerical shape checks for noise and ref-image condition; end-to-end smoke via VideoService.create. A restart of the InvokeAI server is required to pick up the new invocations in the workflow editor. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): add Wan 2.2 T2V and I2V starter video workflows Two new default workflows for the workflow editor 'Browse' modal: - 'Text to Video - Wan 2.2' — model loader -> two text encoders -> wan_video_denoise -> wan_l2v. Exposes prompt, model picks, CFG (high + low), dimensions, frames, fps, and steps. - 'Image to Video - Wan 2.2' — same shape plus a wan_ref_image_encoder feeding the denoise node's ref_image input. Exposes the reference image and the frames field on the ref-image node (must match the denoise node's frames — there is a clear validation error if they diverge, but the starter has them in sync at 81). Both default to the Wan 2.2 reference settings: 832x480, 81 frames @ 16 FPS (~5 s), 40 steps, CFG 5.0 (high expert) + 4.0 (low expert), seeded by a rand_int. Pass the existing _sync_default_workflows validator (id starts with default_, meta.category=default). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(wan): startup crash from stringified VideoOutput annotation run_app.py validates every invocation's return-type annotation against the output-class registry. wan_latents_to_video.py had a stray 'from __future__ import annotations' which made the `invoke()` return annotation a string ('VideoOutput') at runtime. The registry mismatch triggered the unregistered-output warning path, which itself crashed on output_annotation.__name__ because the annotation was a str: AttributeError: 'str' object has no attribute '__name__' The other Wan invocations don't use future annotations — drop the import to match. Verified post-fix: api_app import populates 95 output classes, wan_l2v annotation resolves to the real VideoOutput class and is in the registry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): add Wan 2.2 Lightning T2V starter workflow Same graph as 'Text to Video - Wan 2.2' but with two Apply LoRA - Wan 2.2 nodes chained between the model loader and the denoise node, and defaults retuned for the Lightning distillation: 4 steps and CFG 1.0 on both experts (CFG=1 skips the negative-conditioning forward pass entirely, ~20x faster than the 40-step / CFG-5.0 baseline at similar quality). Adapted from a user-saved workflow; cleaned for distribution by stripping the install-specific model/LoRA key bindings (defaults should not bake in local UUIDs), bumping to a fresh default_-prefixed id with meta.category=default, exposing the two LoRA fields (lora + weight) so users can swap LoRAs without diving into the canvas, and flagging the negative-prompt node as unused at CFG=1. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): add Wan 2.2 Lightning T2V and I2V starter workflows Two new default workflows that wire the Lightning LoRA pair into the T2V and I2V video pipelines for a ~20x speedup: - 'Text to Video - Wan 2.2 Lightning' — model loader -> apply LoRA (high) -> apply LoRA (low) -> text encoders -> wan_video_denoise -> wan_l2v. Defaults to 4 steps and CFG 1.0 (no negative branch). Cleaned-up version of Lincoln's saved Lightning workflow: stripped per-install model/LoRA keys, switched meta.category to 'default' with a default_ id, and exposed both LoRA loaders' lora/weight/ target fields so users can swap LoRAs without diving into the canvas. - 'Image to Video - Wan 2.2 Lightning' — same chain plus a wan_ref_image_encoder (v1.1.0 with num_frames) feeding the denoise ref_image input. Defaults match the non-Lightning I2V starter (832x480, 81 frames @ 16 FPS) but with 4 steps / CFG 1.0. LoRA target defaults to 'auto' so properly-tagged Lightning LoRAs route themselves; both workflow descriptions tell users to set explicit 'high'/'low' targets if their LoRAs are untagged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(wan): use FFMPEG plugin (not pyav) for MP4 encode wan_latents_to_video was passing plugin='pyav' to iio.imwrite, but the runtime only has imageio-ffmpeg installed (no PyAV). The encode step at the very end of generation crashed with: ImportError: The `pyav` plugin is not installed. Use `pip install imageio[pyav]` to install it Switch to plugin='FFMPEG' — backed by the bundled imageio-ffmpeg binary that pyproject already requires via imageio[ffmpeg]. libx264 yuv420p is the FFMPEG plugin's default for .mp4, so the explicit pixel_format is dropped (specifying it just produced a "Multiple -pix_fmt options" warning). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): log VAE decode and MP4 encode milestones in wan_l2v The video VAE decode + MP4 encode tail can take 30-90s on top of the denoise loop, and the toast-style signal_progress() messages don't land in the server log. Add context.logger.info() at: - VAE decode start: latent frame count -> pixel frame count + resolution - MP4 encode start: frames, fps, duration, dimensions - MP4 encode complete: encoded file size - Video saved: final video_name Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(wan): switch video thumbnail/probe to imageio[ffmpeg] backend After wan_l2v wrote a successful libx264 MP4 to disk, the invocation would hang in DiskVideoFileStorage.save() during the cv2.VideoCapture thumbnail-extraction step. cv2 wheels on this build can't reliably decode our libx264/yuv420p output (most often the wheel was compiled without an h264 decoder, but the failure mode is silent hang rather than a clear error). The net effect: the MP4 ends up in outputs/videos but the queue item never completes, so the frontend spinner spins forever and the gallery doesn't pick up the new entry. Fix: rewrite extract_video_frame and probe_video to try imageio's FFMPEG plugin first (same backend that did the encoding — so reading our own output is guaranteed to work), with cv2 retained only as a fallback for uploaded videos in formats imageio can't decode. Also add fine-grained log lines + exception guards inside DiskVideoFileStorage.save() so a future thumbnail failure can no longer hang the whole save — it now logs a warning and continues, leaving the video record in place even if the thumbnail step errored. With logging at each step (video written, thumbnail written, sidecar written) any future hang will be obvious from the last log line. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(videos): handle VideoField outputs in invocation_complete After wan_l2v wrote its MP4 successfully, the gallery and viewer were never updated: the new video didn't appear and the viewer stayed stuck on the previous "Saving video" progress spinner indefinitely. Root cause: onInvocationComplete.tsx only inspected results for isImageField / isImageFieldCollection. VideoField outputs were silently dropped, so the polymorphic gallery list never invalidated and no auto-switch happened. The viewer therefore kept rendering CurrentImagePreview, whose ImageViewerContext-local $progressEvent / $progressImage atoms intentionally aren't cleared on queue completion when autoSwitch is on — they rely on the new image's DndImage onLoad to clear them, which never fires for a video. Fix: add isVideoField (mirrors isImageField against {video_name}) and plumb video outputs through onInvocationComplete: - getResultVideoDTOs pulls VideoDTOs via getVideoDTOSafe - addVideosToGallery invalidates GalleryItemNameList / GalleryItemList so the polymorphic gallery refetches and the new video shows up - auto-switch dispatches the video name into selection (selection is a polymorphic string[]; useGalleryItemDTO already discriminates by filename extension) The selection change swaps CurrentImagePreview for CurrentVideoPreview, which unmounts the stale progress overlay along with it — so the stuck spinner clears as a side-effect of the auto-switch. Also drops the now-stale @knipignore on getVideoDTOSafe, which has a real consumer now. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(video): add 'Frame from Video' invocation Extracts a single frame from a VideoField input and saves it as a regular ImageDTO via context.images.save, so it appears in the gallery like any other generated image. Primary use case is I2V "shot extension": take the last frame of a Wan-generated clip (default frame_index=-1) and feed it back as the reference image for the next clip, then stitch the MP4s to get videos longer than the model's single-shot frame budget at a given VRAM. Negative frame_index is resolved against the actual decoded frame count via probe_video() rather than passed through to imageio — not all imageio plugins handle index=-1 uniformly, and being explicit lets us emit a precise out-of-range error. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(video): add 'Concatenate Videos' invocation Joins two or more videos into a single MP4 with one of three transition modes between consecutive clips: - cut: hard splice, no blending. Total length = sum of inputs. - crossfade: linear A→B dissolve over transition_frames. Each boundary consumes N frames from both surrounding clips, shrinking total length by N per boundary. - fade_through_black: A fades to black, then B fades in. Each boundary consumes N/2 from each side and emits N output frames — total length is preserved. Implementation decodes via imageio's FFMPEG plugin (matching wan_l2v on the encode side) and runs the blends in numpy. All decoded frames are kept in memory at once; fine for the few-hundred-frame I2V chains that motivated this, would want streaming if anyone ever feeds in hour-long uploads. Up-front validation enforces matching dimensions across inputs and checks that each clip has enough frames to spare from its head and tail for the requested transitions — saves a wasted decode pass when the transition window is too wide for one of the clips. Pairs with 'Frame from Video' for I2V shot extension: generate N clips chained via last-frame-as-ref-image, then glue them with a crossfade. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(videos): show full-resolution first frame in viewer The viewer used a chakra <Image src={thumbnail_url}> in the idle (not- playing) state, so once a clip auto-selected after generation the preview snapped from the full-resolution denoise progress image to the small WebP gallery thumbnail upscaled to fit — visibly soft compared to what the user was watching seconds earlier. Switch to a single <video> element that spans both states: - idle: muted, no controls, preload="metadata". With no `poster` attr the browser decodes and shows the video's actual first frame at full resolution (this is the documented HTMLVideoElement default). - playing: same DOM node with controls+audio toggled on, kicked off via ref.play(). No reload between states — the decoded buffer carries over. `key={videoName}` swaps the element cleanly when the user moves to a different clip, dropping any in-progress playback state. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(nodes): show 'Save in gallery' on video-output nodes The footer checkbox was gated on useNodeHasImageOutput, which only matched ImageField outputs. wan_l2v and video_concat produce VideoField and so had no toggle — users had no UI path to flip is_intermediate on them, even though VideoOutput goes through context.videos.save and lands in the gallery the same way ImageOutput does. Rename the hook to useNodeHasGalleryOutput and extend it to match VideoField as well. Update the three call sites (the hook itself, the checkbox, and the footer wrapper) so the toggle and the footer render whenever a node produces something destined for the gallery. The image primitive ('image' type) is still excluded since it doesn't save a new image; no equivalent video primitive exists yet, so no analogous exclusion is needed for VideoField. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: remove unwanted planning documents * chore: fix ruff I001 import-order violations Auto-fix from `ruff check --select I001 --fix`. Touches 10 files across the Wan and videos changes where added imports landed out of order. No behavior change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(videos): restrict uploads to MP4 only The upload allowlist previously included .mov/.webm/.mkv, but the names service (create_video_name) unconditionally emits {uuid}.mp4 and we don't transcode on upload. The result: non-MP4 containers were stored under a .mp4 name and served with the .mp4 MIME type, which silently broke playback in browsers when the container didn't match. Drop the non-MP4 extensions from ACCEPTED_VIDEO_EXTENSIONS and tighten the accepted MIME prefix to "video/mp4". Wan-generated output is MP4 anyway, so this matches current reality. If we want to support more containers later, the right move is to extend the names service to preserve the source extension, then re-add the formats here. Also drops the now-dead suffix-detection block in upload_video and the os import it required. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(videos): clean up stale @knipignore on consumed hooks useDeleteVideoMutation, useAddVideoToBoardMutation, and useRemoveVideoFromBoardMutation are now consumed by Phase 4 components (context menu, change-board modal) but were still annotated with the multi-phase @knipignore tag — that generated false-positive knip warnings and misrepresented the implementation status. Move those three into the unconditional export block. The remaining five hooks (useListVideosQuery, useGetVideoMetadataQuery, useGetVideoNamesQuery, useDeleteVideosMutation, useChangeVideoIsIntermediateMutation) are still unused in the current codebase and stay under a narrower @knipignore. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(videos): document invalidate/select race in addVideosToGallery The video gallery path uses tag invalidation rather than an optimistic insert (the image path's `insertImageIntoNamesResult` doesn't have a polymorphic equivalent yet). Because invalidation kicks off an async refetch, the `imageSelected` dispatch below it fires before the new video name is in `imageNames`, so the gallery grid's `useKeepSelectedImageInView` no-ops on its first pass. The scroll self-corrects on the next pass when the refetch lands and the `imageNames` dep updates. The user-visible effect is just a small lag on gallery scroll-to- selection — the viewer selection applies immediately — so this is a documented limitation rather than a bug. Worth a follow-up if the lag becomes noticeable in practice. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(video): add Video Primitive invocation and VideoField input UI Mirrors the Image Primitive flow end-to-end for videos. Users can now drag a video from the gallery onto a "Video Primitive" node and feed its output into downstream nodes like Frame from Video or Concatenate Videos — exactly the way Image Primitive feeds the rest of the image pipeline. Backend (invokeai/app/invocations/primitives.py): - New VideoInvocation, declared *after* VideoOutput so the return annotation is a real class (not a forward-ref string). Stringified output annotations crashed startup before — see cac366229a — so the ordering matters. Frontend: - Register VideoField as a stateful field type in types/field.ts: zVideoFieldType, zVideoFieldValue, zVideoFieldInputInstance/Template, output template + type guards, plus entries in the four stateful unions (FieldType, FieldValue, InputInstance, InputTemplate). - buildFieldInputTemplate / buildFieldInputInstance gain VideoField branches so OpenAPI-derived templates resolve correctly. - nodesSlice: fieldVideoValueChanged reducer + export. - imageActions/actions.ts: setNodeVideoFieldVideo helper. - dnd.ts: singleVideoDndSource + setNodeVideoFieldVideoDndTarget, wired into the dndTargets array. - GalleryVideoItem: register itself as a drag source so videos in the gallery actually drag (previously they were click-only). - VideoFieldInputComponent: parallel to ImageFieldInputComponent — shows the WebP thumbnail with a dimensions badge, accepts video DnD, drops stale references on reconnect if the underlying video was deleted. - InputFieldRenderer: dispatch VideoField templates to the new component (placed right after the ImageField branch). - useNodeHasGalleryOutput: also exclude the new `video` primitive type so the "Save in gallery" toggle does not render on the pass-through node (same treatment the `image` primitive already gets). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(video): allow video drops to reach DnD target handlers useDndMonitor is the global drop monitor that actually invokes each target's handler() — DndDropTarget only does enter/leave bookkeeping. Its canMonitor gate explicitly allowlists source types and only listed singleImageDndSource + multipleImageDndSource. So when a video was dragged from the gallery onto a VideoField input, the drop was visible to the DOM but the monitor silently filtered it out, the handler never ran, and fieldVideoValueChanged was never dispatched. Add singleVideoDndSource to the allowlist. Dropping a video onto a Video Primitive (or any other VideoField input) now wires the asset into the field as intended. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(backend): ruff * chore(frontend): typegen * feat(wan): split Wan 2.2 starter bundle into T2V and I2V Replaces the single ~63 GB Wan 2.2 bundle with two smaller bundles so users only pay for the capability they need. T2V (~36 GB) covers text-to-video plus a low-VRAM image-to-video option via TI2V-5B; I2V (~32 GB) adds the heavier I2V-A14B path. Drops the Q8 T2V pair from the default bundle — both Q8 variants and full Diffusers builds remain available as a-la-carte starters. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(videos): tighten multiuser isolation in list and board-move endpoints Three related fixes flagged in code review (PR #9163, JPPhoto): 1. Video and gallery list/name SQL paths only filtered by user_id when board_id was the literal "none" sentinel. When the URL parameter was omitted entirely, no user filter applied and non-admin callers could enumerate every user's videos / mixed gallery items. Added an explicit per-user isolation branch for the omitted case. 2. /v1/videos/ and /v1/videos/names accepted explicit board IDs with no read-access check; the route now mirrors the images and gallery routers and calls _assert_board_read_access for non-"none" values. 3. add_video_to_board and remove_video_from_board only validated video ownership, not destination/source board write access — a caller could move their video into someone else's private board. Added _assert_board_write_access and a strict _assert_video_direct_owner helper (no board-owner / public-board fallback) for board-move ops, mirroring _assert_image_direct_owner. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(boards): cascade video deletion when deleting a board with media Previously delete_board only handled images. With include_images=true the backend would delete images on the board but the videos would silently cascade out of board_videos and survive as uncategorized records — almost certainly not what the caller intended. Without include_images, the same mismatch meant the response could not report affected videos. Now: - include_images=true also calls videos.delete_videos_on_board - include_images=false collects the soon-to-be-uncategorized video names - DeleteBoardResult gains deleted_board_videos and deleted_videos fields (default empty so existing clients are unaffected) Frontend deleteBoard / deleteBoardAndImages mutations gain the matching VideoList / VideoNameList / GalleryItem* tag invalidations so the polymorphic gallery and video list views refresh. Reported in code review (PR #9163, JPPhoto). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(videos): return affected_boards from board move/remove endpoints removeVideoFromBoard previously returned VideoDTO; the frontend then read result.board_id (null after removal) and only invalidated the 'none' board cache — the previous board's list stayed stale until refetch. addVideoToBoard had the same problem (the route never knew the source board, so the old-board cache was never invalidated). Mirror the image equivalents (board_images.py): the routes now return AddVideosToBoardResult / RemoveVideosFromBoardResult with the moved video name(s) and the full set of affected board IDs. Both old and new boards get invalidated atomically. Frontend mutations updated to consume the new shape via getTagsToInvalidateForBoardAffectingMutation on result.affected_boards. The auto-generated schema.ts will need a typegen pass after the dev server restart to pick up the new response types. Reported in code review (PR #9163, JPPhoto). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(videos): stream uploads, FileResponse for full video + thumbnail Three perf items from the code review (PR #9163, JPPhoto): - upload_video read the entire UploadFile into a Python bytes object before writing to the temp file. Multi-GB videos allocated multi-GB buffers. Now chunk-stream into the temp file with a 1 GB per-upload cap (HTTP 413 on overflow). Cap is intentionally generous — the goal is RAM-exhaustion protection, not content policy. - get_video_full read the whole MP4 into RAM when no Range header was present. Browsers usually send Range, but curl / direct downloads / CDN edge fetches do not, and a multi-GB load per such request is a trivial DoS vector. Replaced with FileResponse (sendfile). - get_video_thumbnail similarly buffered the WebP. Thumbnails are tiny so this was minor, but FileResponse is idiomatic and shaves the unnecessary copy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(videos): localize video UI strings - Add gallery.deleteVideo_one / deleteVideo_other, deleteVideoConfirmation, and playVideo to en.json - ContextMenuItemDeleteVideo: drop the inline English defaultValue (the translation key now exists) and use gallery.deleteVideo for aria/tooltip (was reusing gallery.deleteImage so it rendered "Delete Image") - VideoPlayButtonOverlay: replace the hardcoded "Play video" aria with t('gallery.playVideo') Reported in code review (PR #9163, JPPhoto). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(video-invocations): exact frame counts and odd-tf transitions video_frame_extract: resolving frame_index=-1 previously computed n_frames as round(duration * fps). For VFR uploads or containers with approximate metadata that can overshoot the actual decoded frame count, making the last-frame extraction fail. Use iio.improps(plugin='FFMPEG') for the exact decoder count when available; fall back to the duration * fps estimate only if the props query fails. video_concat fade_through_black: with an odd transition_frames the symmetric half = tf // 2 split emitted tf - 1 frames per boundary, violating the documented "emits transition_frames" contract. Split asymmetrically (tail_half = tf // 2, head_half = tf - tail_half) so the emitted count equals tf exactly for both even and odd values. Validation and docstring updated to match. Verified with manual cases: tf=1, tf=4, tf=5 all emit the documented total length for two 10-frame inputs. Reported in code review (PR #9163, JPPhoto). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(gallery): describe video gallery items, upload, and deletion Update the Gallery Panel docs to reflect the polymorphic gallery added in the Wan 2.2 video feature branch: - Gallery intro now mentions images + videos coexist on boards. - Board deletion warning clarified to cover both kinds of media. - New "Videos in the Gallery" section covering: how video items appear (first-frame thumbnail + play badge), MP4-only upload constraint with the typical re-encode command, the video context menu, and that videos count toward board totals. Reported in code review (PR #9163, JPPhoto). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(videos): regression coverage for PR #9163 review fixes Adds tests that pin the behaviour fixed in the JPPhoto review and would catch a recurrence: - tests/app/services/video_records/test_video_records_sqlite.py get_many / get_video_names: non-admin callers only see their own videos when board_id is omitted; admins see all; the "none" branch still filters by user. - tests/app/services/gallery/test_gallery_default.py Same multiuser isolation guarantee through the polymorphic gallery union for both images and videos. - tests/app/routers/test_videos_multiuser.py /v1/videos/ and /v1/videos/names: 403 when a non-owner passes an explicit private board_id; 200 for owners, admins, "none", and omitted board_id (the auth-required smoke tests pin the 401 paths too). - tests/app/routers/test_boards_multiuser.py Adds two delete-board cases proving the video cascade: include_images invokes delete_videos_on_board and reports deleted_videos; the no-include path reports deleted_board_videos without calling the destructive service. Existing fixture extended to stub the video services that the new route logic now touches. - tests/app/invocations/test_video_concat.py Parametric coverage that fade_through_black emits exactly tf frames for both even and odd tf, plus three-clip chains, plus the crossfade and cut/zero-tf cases as guards. - tests/app/invocations/test_video_frame_extract.py _decoder_frame_count returns the exact count via the cv2 fallback for several clip lengths and gracefully returns None for missing / non-video inputs (caller falls back to duration * fps). Bug found during test authoring: _decoder_frame_count over-flowed int() on iio's "inf" nframes for libx264 streams, and improps never returns a real count for that codec anyway. Helper now ignores non-finite shapes and falls back to cv2's CAP_PROP_FRAME_COUNT, which gives the exact value for libx264. schema.ts regenerated to pick up the AddVideosToBoardResult / RemoveVideosFromBoardResult / extended DeleteBoardResult types added in earlier commits in this series. All 70 tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): add 'Wan 2.2 I2V Ideal Dimensions' invocation Computes Wan I2V-compatible (width, height) for a source W×H at a target short-side resolution (e.g. 720 for "720p"), snapping each output to a multiple of 16 (Wan's transformer patch_size × VAE 8x pixel-grid constraint enforced by wan_ref_image_encoder). Replaces the 6-node math chain (Float Math × 4 + Float To Integer × 2) that was otherwise required to compute these dimensions from an arbitrary input image. Wire the Image Primitive's width/height outputs into this node, and feed its (width, height) outputs into both wan_ref_image_encoder and wan_denoise (they must match). Three rounding modes: - nearest (default): minimizes aspect-ratio drift - floor: guaranteed not to exceed unsnapped target (safer for VRAM) - ceiling: rounds up Output schema reuses IdealSizeOutput so it slots into existing pipes that already consume Ideal Size — SD1.5, SDXL. Includes regression tests covering the documented common-case table, all three rounding modes, postcondition invariants (multiple of 16, aspect ratio within 1.2%, never zero), and input validation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): swap target_short_side int for 480p/720p/1080p preset dropdown Wan 2.2 was trained at 480p and 720p; a free integer encouraged users to pick noncanonical short sides that the model handles poorly. Replace the int field with a Literal dropdown of "480p" / "720p" / "1080p" (via ui_choice_labels) so the UI surfaces the canonical choices. 1080p is included with a label noting it's extrapolated from training (not a Wan native size) — useful for users with VRAM headroom but shouldn't be the default. Version bumped to 1.1.0 since the field schema changed (the node was only committed locally; no published workflow needs migrating). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(wan): empty_cache around the I2V reference-image VAE encode Two-sided fix to avoid VRAM allocator fragmentation that was causing the subsequent denoise-transformer partial load to OOM: - Before vae.encode(): clears blocks left over from earlier nodes (the denoise expert swap especially leaves the cache fragmented). - After the condition tensor is on CPU: returns the VAE encode's intermediates so the next partial_load_to_vram sees a real free contiguous range. Mirrors the same pattern in wan_latents_to_image.py and wan_latents_to_video.py — those are the existing precedent. The cost is a handful of microseconds per encoder invocation and only the cache state is touched; model weights stay resident. Observed-by symptom from a workflow review: at encoder=480x720 and a source image of 880x1184, the encoder ran fine but the I2V high-noise expert failed to partial-load with a cryptic CUDA OOM at _load_state_dict_with_fast_device_conversion. Pre-resizing the source to 80% incidentally cleared the allocator state and let the run succeed; this fix removes the incidental dependency on source size. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): support TI2V-5B in the video denoise node (T2V mode) The video denoise node previously hard-errored on TI2V-5B with "not supported." Most of the surrounding machinery (variant-aware spatial scale, variant-aware scheduler, single-expert ExpertSwapper path) was already in place — the gate just needed lifting and the hard-coded A14B latent channel count needed to follow the variant. Changes: - Drop the upfront "TI2V-5B is not supported" raise. - Use get_default_latent_channels(variant) so latents are 48-channel for TI2V-5B and 16-channel for the A14B family (matches the image denoise node's existing logic). - For TI2V-5B with a Reference Image input, raise a sharper, accurate error that explains TI2V-5B's I2V uses diffusers' expand_timesteps path (first-frame-mask blend + per-position timestep gating) which this node does not implement yet — pointing the user at the working T2V path or the I2V-A14B model. - Update the transformer field description to reflect what's now supported. Image-to-video with TI2V-5B remains a follow-up; the conditioning math is genuinely different from A14B (no 36-channel concat) and warrants a separate code path rather than parameterising this one. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(wan): instantiate TI2V-5B VAE with the right architectural config The single-file Wan VAE loader was always calling ``AutoencoderKLWan(z_dim=config.latent_channels)`` and relying on diffusers' constructor defaults for every other parameter — but those defaults match the Wan 2.1 / A14B VAE (base_dim=96, in/out=3, 8x spatial, no patchify). For TI2V-5B's Wan 2.2-VAE the architecture is materially different: - base_dim=160, decoder_base_dim=256 - in_channels=12, out_channels=12 (3 RGB x 2x2 patch) - patch_size=2 - scale_factor_spatial=16 - is_residual=True - 48-vector latents_mean / latents_std (required for the model's encode/decode normalisation to produce non-garbage outputs) Loading the TI2V-5B VAE state_dict into the default-constructed model failed with shape mismatches throughout the encoder + decoder, surfaced in wan_l2v as "Error(s) in loading state_dict for AutoencoderKLWan." This commit routes z_dim=48 to a verbatim copy of the TI2V-5B VAE config (from vae/config.json in Wan-AI/Wan2.2-TI2V-5B-Diffusers); z_dim=16 keeps the previous A14B / Wan 2.1 default behaviour. Verified end-to-end: both kwargs construct cleanly and produce the expected layer shapes (decoder.conv_out emits 12 channels for TI2V-5B, 3 channels for A14B). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(wan): variant-aware default scheduler for standalone installs When the main model has no on-disk ``scheduler/`` directory (every standalone GGUF / single-file install), ``_build_scheduler`` previously fell back to ``FlowMatchEulerDiscreteScheduler()`` for every variant. That's correct for the A14B family but wrong for TI2V-5B, which ships ``UniPCMultistepScheduler`` with ``flow_shift=5.0`` + ``prediction_type="flow_prediction"`` + ``use_flow_sigmas=True``. The mismatch produces drifty samples on TI2V-5B. Add a ``_default_scheduler_for_variant`` helper that reconstructs the right scheduler from the variant tag (values verbatim from each variant's ``scheduler/scheduler_config.json`` in the matching Wan-AI/Wan2.2-*-Diffusers repo). The on-disk-config-present path is unchanged — if the model ships a scheduler dir, that wins. Full scheduler-selection UI is deferred to a future PR per discussion; this special-case keeps the standalone TI2V-5B path producing the right sampler without surfacing a new field. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): TI2V-5B image-to-video support TI2V-5B I2V uses a fundamentally different conditioning scheme from A14B I2V. Implement diffusers' ``expand_timesteps`` path so the same ``Reference Image - Wan 2.2`` node and ``Denoise Video - Wan 2.2`` node work for both variants, dispatched by VAE z_dim / transformer variant. Encoder side (wan_ref_image_extension.py / wan_ref_image_encoder.py) - Add ``encode_reference_image_to_ti2v_condition`` that VAE-encodes a single image frame to ``[1, 48, 1, H/16, W/16]`` with the Wan2.2-VAE normalisation, no mask channels. - ``WanRefImageEncoderInvocation`` dispatches on ``vae.config.z_dim``: z_dim=48 → TI2V-5B path, z_dim=16 → existing A14B path. - Enforce ``multiple_of=32`` for width/height in the TI2V-5B case (16x VAE * 2 transformer patch = pixel dims must divide by 32) with a clear error message pointing at the constraint. Denoise side (wan_video_denoise.py) - Replace the "TI2V-5B I2V not supported" raise with a variant-aware dispatch on ``ref_condition.shape`` and ``variant``. - For TI2V-5B I2V build a ``first_frame_mask`` once (0 at frame 0, 1 elsewhere). At each step: latent_model_input = (1 - mask) * condition + mask * latents temp_ts = (mask[0,0,:,::2,::2] * t).flatten() timestep = temp_ts.unsqueeze(0).expand(B, -1) Per-token timesteps gate the model: frame 0 sees t=0 (locked to condition), other frames see t (normal denoise). - After the denoise loop, re-clamp frame 0 to the clean condition so the locked first frame doesn't show scheduler drift in the final VAE decode. Mirrors WanImageToVideoPipeline:813-814. - Skip the encoder-num_frames-must-match check for TI2V-5B (its condition is always single-frame regardless of output length). Tests - Three new tests on encode_reference_image_to_ti2v_condition covering output shape at small and Wan-realistic dims plus the no-mask-channels invariant. Full video-denoise integration tests would need a new fixture stack (none exist for wan_video_denoise yet) — deferred. A14B I2V is unchanged. TI2V-5B T2V (added in the previous commit) is unchanged. Verified at the import + encoder-shape level; end-to-end verification requires a TI2V-5B I2V workflow. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(videos): show denoise progress overlay over the video viewer CurrentVideoPreview rendered only the <video> element, so when the last-selected gallery item was a video, a freshly-started render's denoise preview images had nowhere to display — the user saw the static first-frame still of the previously-loaded video until the new render's final video swapped in. Mirror CurrentImagePreview's progress-overlay pattern: subscribe to $progressImage / $progressEvent, gate on selectShouldShowProgressInViewer, and render a ProgressImage stack on top of the video when a render is in progress. Hide the play-button overlay while progress is showing so it doesn't sit on top of the preview. Reported by Lincoln during TI2V-5B testing: previews started working after restarting the server only because there was no video loaded at that point; once a video was selected, the previews silently dropped. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(lint): apply ruff format + isort across recent Wan video work ruff check found one I001 (import order) in ``invokeai/backend/model_manager/load/model_loaders/vae.py`` and ruff format flagged five files. All cosmetic; no behaviour changes. - vae.py: import reorder - video_concat.py: minor reflow - test_wan_ideal_dimensions.py / test_boards_multiuser.py / test_videos_multiuser.py: prettier-style wrapping Verified: full ruff check + ruff format --check clean, 141 backend tests pass, and ``pnpm lint`` (knip + dpdm + eslint + prettier + tsc) all green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(features): add user guide for Wan 2.2 video generation Comprehensive guide covering: - The three Wan 2.2 variants (T2V-A14B, I2V-A14B, TI2V-5B), their conditioning differences, and the dual-expert MoE explanation - Lightning LoRA distillation for 4-step A14B inference - Starter bundles (Text-to-Video and Image-to-Video splits) - Workflow setup for T2V and I2V with the constraint matrix: * frame count: (num_frames - 1) % 4 == 0 * pixel dims: multiple of 16 for A14B, 32 for TI2V-5B * encoder + denoise must agree on width/height - The chain-and-concat trick for making longer videos, with the bridge-frame degradation mitigations - Troubleshooting: OOM, late-frame artifacts, dim mismatches, VAE load errors, scheduler issues, preview-not-appearing, MP4 glitches Lands under Features → Video Generation (experimental). Astro auto-generates the sidebar from features/ so no nav config change needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(nodes): skip image-DTO fetch for videos in Current Image node CurrentImageNode unconditionally called useImageDTO(lastSelectedItem) even when the selected gallery item was a video, firing GET /api/v1/images/i/<uuid>.mp4 on every video thumbnail click. The endpoint 404s and the backend logged "Image record not found" each time — benign but noisy. Apply the same null-skip pattern useGalleryItemDTO uses: pass the name only when it's not a video, so RTK Query skips the request for video selections. Current Image is image-only by design, so videos rendering the empty fallback matches existing behaviour. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(videos): clear stale progress overlay + force first-frame paint Two viewer bugs after auto-switching to a freshly-rendered video: - The denoise progress overlay never cleared. CurrentImagePreview clears the ImageViewerContext $progressImage/$progressEvent atoms via DndImage's onLoad callback; the video viewer had no analog, so the last progress still sat on top of the new video forever — clicking other video thumbnails did nothing visible, and only selecting an image (which fires onLoadImage via DndImage) cleared it. - Even with the overlay gone, the <video> element rendered its black background instead of the first frame. preload="metadata" loads dimensions/duration but doesn't guarantee a decoded first frame on all browsers; an explicit seek is needed to force a paint. Wire onLoadedMetadata to (1) call onLoadImage() — mirroring DndImage's onLoad — and (2) nudge currentTime to 0.0001 so the decoder paints the first frame without measurably advancing playback. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(hotkeys): skip image-DTO fetch for videos in GlobalImageHotkeys Companion to a3bdc3304e (CurrentImageNode). GlobalImageHotkeys is a mounted-everywhere singleton that wires recall hotkeys (seed, prompts, remix, etc.) to whatever item is currently selected. It was passing the raw selection name through to useImageDTO unconditionally, so every video thumbnail click fired GET /api/v1/images/i/<uuid>.mp4 → 404 and the "Image record not found" log line. Gate on isVideoName(), mirroring the polymorphic null-skip pattern in useGalleryItemDTO. Recall hotkeys don't apply to videos anyway, so this just suppresses the noise. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(wan): empty CUDA cache between A14B expert swaps The dual-expert swapper releases the active expert via its context manager exit, but PyTorch's caching allocator retains the freed blocks as reserved-not-yet-claimable space until empty_cache runs. The next partial_load_to_vram for the incoming expert then sees a fragmented free pool and offloads layers it could otherwise have kept on device. Users running A14B observed the low-noise expert ending up far more CPU-resident than the high-noise one on otherwise identical settings — that was the leftover reservation from the high-noise expert masking real free VRAM. Call TorchDevice.empty_cache() between the release and the next load. Same pattern as the VAE-encode fix earlier in this branch. Regression test in test_wan_expert_swapper.py mocks empty_cache and asserts it fires on every actual swap but not on a same-label re-get. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(videos): allow drag-and-drop to change a video's board Dropping a video thumbnail onto a board in the boards list was a no-op (the dnd target only accepted image sources). Extend addImageToBoardDndTarget and removeImageFromBoardDndTarget to also accept SingleVideoDndSourceData and dispatch the corresponding video mutations. Permission UX mirrors the image path: - Same canMoveFromSourceBoard gate (owner / public source board) - Same "do nothing if dropping on the current board" early-out Backend enforcement on /api/v1/videos/board already mirrors the image endpoints — _assert_board_write_access on the destination plus _assert_video_direct_owner on the video. The frontend gate intentionally mirrors only the source-board part of that, leaving the direct-owner check to surface as a 403 on attempt (same compromise as images, where the client doesn't have per-item owner info to gate cleanly). Multi-video drag is not supported yet (the gallery only registers a single-video draggable per item, no multi-select bundle), so this only wires the SingleVideoDndSourceData path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(wan): force outgoing A14B expert off GPU on swap The previous empty_cache() fix (53b2f4d4c7) was insufficient. unlock() only decrements the cache record's lock counter — the weights stay on GPU until the cache's automatic offload decides to free them on the next lock(). That heuristic uses ``torch.cuda.memory_allocated() - working_mem`` to estimate free space, which under-frees when the previous denoise step's workspace activations are still allocated alongside the just-unlocked expert. The user-visible symptom was a log line like Loaded model '...:transformer' onto cuda device in 0.37s. Total model size: 9203.13MB, VRAM: 2381.18MB (25.9%) for the incoming low-noise expert, while the high-noise expert continued to hold ~9 GB of VRAM. The swapper now stashes the LoadedModel info handle and, on each swap, explicitly invokes ``cached_model.full_unload_from_vram()`` on the outgoing expert before locking the incoming one. This sidesteps the heuristic and guarantees the previous expert's weights leave GPU before partial_load_to_vram measures available room. The access path ``info._cache_record.cached_model`` reaches into a private attribute — there is no public LoadedModel API for "unload from VRAM but keep in RAM" today, and a broader backend refactor felt out of scope. The call is wrapped in getattr/try-except and pinned by a regression test so a future refactor breaks the test, not the swap. Tests: - Updated existing dual-expert lifecycle test to expect the new full-unload step in the swap log sequence. - New test_outgoing_expert_force_unloaded_from_vram covers the per-swap behavior (outgoing only, no initial unload). - New test_force_unload_failure_does_not_break_swap pins the defensive fallback so swap reliability survives a future LoadedModel refactor. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(gallery): restore shift/ctrl-click range selection in image grid GalleryImage's modifier-key click handler was reading the legacy imagesApi getImageNames cache to compute range-selection indices, but the gallery grid was switched to the polymorphic galleryApi getGalleryItemNames endpoint (the only source that includes videos). The legacy cache is no longer populated for the grid, so the ordered-name list came back empty and the handler fell into its "no names cached" early-return: if (imageNames.length === 0) { if (!shiftKey && !ctrlKey && !metaKey && !altKey) { dispatch(selectionChanged([imageName])); } return; } making shift- and ctrl-click no-ops. GalleryVideoItem already had the correct reader inlined as a private helper. Hoist it to a shared module (features/gallery/store/selectCachedGalleryItemNames) so both grids use the polymorphic cache, and update GalleryImage to call it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(typegen): regenerate schema.ts Refresh of the OpenAPI-derived TypeScript bindings against the current backend. No hand edits — this is the output of the typegen step re-run against the Wan video routes and recent backend changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(startup): silence HF tokenizers fork-after-parallelism warning Set TOKENIZERS_PARALLELISM=false at startup (via os.environ.setdefault so users can override) before any HF library is imported. The Rust ``tokenizers`` library warms a thread pool the first time a tokenizer runs — for us that's UMT5 / T5 text encoding during Wan / FLUX / SD3 conditioning. Every subsequent fork() then logs huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks... In video generation we fork on every MP4 encode (imageio's FFMPEG plugin uses subprocess.Popen → fork+exec), so this warning lands once per generation in the server log. The advisory is benign — the child correctly falls back to single-threaded tokenization before exec(), and the parent's thread pool is unaffected — but the noise obscures real warnings. Setting the env var before any HF import prevents the thread pool from warming up at all, so the fork detector stays quiet without sacrificing anything: tokenization happens once per generation and isn't a hot path for us. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(startup): hoist TOKENIZERS_PARALLELISM=false to module level Follow-up to 2106f10ec4 — the previous attempt set the env var inside ``run_app()``, which races against any transitive HF import triggered by the console-script's from invokeai.app.run_app import run_app If ``tokenizers`` is imported anywhere in that import chain (directly or via diffusers/transformers re-exports), the library's fork detector registers before our setdefault runs and the warning still fires. Move the setdefault to module level so it executes the instant ``run_app.py`` is loaded — i.e. before the function defs are even parsed, and well before any HF library has a chance to import. Note for testing: jurigged hot-reload only re-runs function bodies, so picking up this fix requires a full server restart, not just a file save under ``--dev-reload``. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(videos): replace window.confirm with ConfirmationAlertDialog Use the in-app delete-confirmation dialog (the same Chakra ConfirmationAlertDialog the image flow uses) instead of the browser's window.confirm() prompt. Matches the visual + interaction language of the rest of the gallery and picks up the shared ``shouldConfirmOnDelete`` system preference — flipping the "Don't ask me again" toggle now silences the prompt for both images and videos. Implementation mirrors features/deleteImageModal/ but trimmed: the image dialog computes "usage" (canvas layers, node fields, reference images, upscale source) so the user knows what they'll break. Videos have no analogous attachment points, so the video state machine is a straight confirm-then-delete with no usage analysis. - features/deleteVideoModal/store/state.ts — nanostores atom + an awaitable ``deleteVideosWithDialog`` that opens the dialog and resolves/rejects on confirm/cancel. Skips the dialog entirely when shouldConfirmOnDelete is off. - features/deleteVideoModal/components/DeleteVideoModal.tsx — ConfirmationAlertDialog with the new deleteVideoPermanent message and the shared "Don't ask me again" switch. - GlobalModalIsolator.tsx — mount the new modal alongside DeleteImageModal. - ContextMenuItemDeleteVideo.tsx — call useDeleteVideoModalApi().delete instead of window.confirm + useDeleteVideoMutation. - en.json — added gallery.deleteVideoPermanent, dropped the now-unused gallery.deleteVideoConfirmation. - videos.ts — useDeleteVideoMutation moves into the @knipignore export group since the only call site now uses videosApi.endpoints.deleteVideo.initiate via the modal. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(gallery): refetch polymorphic gallery cache on image completion The gallery grid subscribes to the polymorphic ``getGalleryItemNames`` RTK Query endpoint (so images and videos interleave by created_at). But ``onInvocationComplete``'s image path only did an optimistic insert into the image-only ``getImageNames`` cache, leaving the polymorphic cache stale — a freshly-generated image landed correctly in board totals and the per-DTO cache, but never showed up in the grid until the user reloaded the page. Mirror the videos path (which has invalidated these tags since the polymorphic endpoint was introduced) and dispatch ``galleryApi.util.invalidateTags(['GalleryItemNameList', 'GalleryItemList'])`` after image outputs are processed. The cost is one extra HTTP round-trip per generation; a future optimization could optimistically splice the new entry into the polymorphic shape, but that requires a different ``insertImageIntoNamesResult`` for the ``GetGalleryItemNamesResult`` shape and is a bigger change. Regression test in onInvocationComplete.test.ts pins the behavior: verifies the invalidation fires on a fake image complete event, and verifies it does NOT fire for denylisted passthrough node types (load_image, image). Confirmed test correctly fails when the fix is reverted via git stash. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: address 2nd-pass code review findings Self-review pass before re-pinging external reviewers. Five fixes; the three medium ones have user-visible consequences, the two low ones are guard + docstring. 1. videos.py: delete_video no longer swallows service errors into a misleading HTTP 200. Missing DTO -> 404, delete failure -> 500. The prior shape returned 200 with an empty deleted_videos list, which the frontend treated as success, dropped from cache, and left the video on disk — silent data-consistency failure visible only on next page reload. 2. videos.ts: starVideos / unstarVideos invalidate the LIST_TAG-scoped { type: 'VideoList' } entry alongside the per-video and board-affecting tags. Without this, starred_first=true gallery queries kept the just-starred video in its old position until the next list-affecting mutation. Mirrors the delete + upload pattern. 3. wan_denoise.py: _ExpertSwapper.get() stashes _active_device_ctx right after device_ctx.__enter__() succeeds, before attempting the LoRA patcher's __enter__. If the LoRA enter raises, _release() can now actually find the device context and exit it — previously the ctx was unreachable and 8-9 GB of GGUF expert weights stayed pinned to GPU until the model cache LRU evicted them. 4. wan_ideal_dimensions.py: reject sources whose longer side is below the 16-px Wan grid. The downstream max(w, 16) clamp would otherwise silently disconnect the output from the requested aspect ratio (returning 16×16 regardless of the source's actual shape). 6. wan_video_denoise.py: docstring now explains the deliberate absence of denoising_start / denoising_end / initial-latents inputs (video i2v uses reference-frame conditioning, not noise injection; the image denoise node still handles still-image img2img). Tests: - test_device_context_released_when_lora_enter_raises pins #3. - test_input_smaller_than_pixel_grid_rejected pins #4. - test_output_dims_never_zero renamed to test_smallest_valid_input_still_snaps_to_16_grid (now exercises 16×16 rather than 8×8 since the latter is now correctly rejected). All 58 affected backend tests pass, frontend lint clean. Audit note for the PR description (NOT a fix): delete_video's _assert_video_owner permits write access on public boards (mirroring the image router's _assert_image_owner — intentional symmetry). The stricter _assert_video_direct_owner is reserved for board-move ops. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(frontend): typegen * fix(gallery): multi-select context menu actions for videos The video gallery context menu only operated on the single right-clicked item, so selecting multiple videos and hitting the trash icon deleted just the first one. Adds a video-side multi-selection menu mirroring the image one for star/unstar/download/change-board/delete, switched in on selectionCount > 1. Each menu now filters the polymorphic selection to its own kind and labels the action with an explicit count + kind (e.g. "Delete 3 Videos", "Move 2 Images to Board"). The destructive items disable when the kind-filtered subset is empty, so a video-only selection greys out the image menu and vice versa. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(multiuser): address Pfannkuchensack PR #9163 review findings Finding 1 (Medium): delete_board cascade ignored per-video / per-image ownership, letting a board owner destroy other users' contributions to a public/shared board just by deleting the board with include_images=true. Adds user_id filtering through get_all_board_*_names_for_board and delete_*_on_board (base + sqlite + image wrapper). Non-admin requests pass the requester's id so the SQL WHERE clause narrows the cascade to that user's rows; admins still pass None for the unrestricted path. Other users' content cascades to "uncategorized" via the existing FK on board_videos / board_images. Finding 2 (Low, i18n): GalleryItemStarIconButton and GalleryItemVideoStarIconButton shipped raw English "Star"/"Unstar" tooltips. Both now use the gallery.starImage / starVideo translation keys. Finding 3 (Low): delete_videos_from_list and delete_images_from_list re-raised HTTPException mid-loop, throwing away the response payload for items already deleted before the foreign name was hit. The frontend cache never learned about those partial successes, so deleted records reappeared in the UI until the next manual refresh. Both routes now skip auth-failed items in-loop and return 200 with the partial-success list. Residual: adds a test that an upload with an .mp4 extension but non-decodable bytes (a) reaches probe_video, (b) surfaces 415, (c) unlinks the streamed-to-disk temp file so the server doesn't leak storage on garbage uploads. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(openapi): regenerate openapi.json The committed schema was stale relative to the current server (missing the utilities/expand-prompt and utilities/image-to-prompt endpoints, the ModelRecordOrderBy / SQLiteDirection list params, and the Wan / QwenImage / QwenVLEncoder config variants this branch adds). Regenerated via the same command the new openapi-checks workflow uses so the diff CI is empty. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(workflows): add "Image to Video - Add Frames" starter workflow Extends an existing video by extracting its penultimate frame, running it through Wan 2.2 I2V A14B + the Lightning LoRA pair to generate a new clip, and concatenating the result onto the source with a short crossfade. Cleaned per the default-workflows README: stripped value references on the four model loader fields and both Lightning LoRA fields so the workflow ships without keys/hashes for user-installed resources, gave the LoRA nodes "Apply LoRA (High)" / "(Low)" labels matching the existing Lightning default, remapped six stale exposedFields entries that pointed to template LoRA IDs no longer present in the graph, and synced the wan_video_denoise num_frames default to the value driven by the connected integer node. Tagged with both Text to Video and Image to Video so it surfaces under either filter in the Workflow Library. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(gallery): video viewer polish and selection regressions - Restore auto-select-on-startup and on-board-switch: the polymorphic getGalleryItemNames endpoint replaced getImageNames as the grid's source of truth, so appStarted and boardIdSelected now wait on / read that cache instead of timing out forever. - Delete-then-select: video delete used to clear selection to null; image delete read a cache that's no longer warmed. Both now snapshot the gallery list before deletion and advance to the adjacent surviving item (prev > next > null) via a shared pickSelectionAfterDelete helper. - Video Viewer: right-aligned action bar with Open in new tab, Copy frame, Download, Delete, and a labelled Close video player button that only appears while playback is active. Copy uses canvas + ClipboardItem since video MIME types aren't supported cross-browser. - Next/prev arrows + galleryNav hotkeys now work when a video is in the Viewer (previously image-only). - Video context menu uses full-width text MenuItems instead of the cramped icon group, and gains an Open in new tab entry. * fix(gallery): bulk video drag-to-board and shift-click range selection - Bulk video drag: introduced multipleVideoDndSource so a multi-selection dragged from a video thumbnail moves every selected video, not just the first. The whitelist in useDndMonitor.ts also needed updating — without it the monitor's canMonitor gate silently dropped the new source type. - Mixed selections: both the multi-image and multi-video drag payloads now carry image_names + video_names side-by-side, so dragging from either kind in a mixed selection dispatches addImagesToBoard + addVideosToBoard together. Previously the image side leaked video names into image_names and the image router 404'd on each one. - Bulk video helpers: added addVideosToBoard / removeVideosFromBoard that fan out over the existing singular video router endpoint (no batch endpoint exists yet) — mirrors the change-board modal's existing loop. - Shift-click range selection: selectCachedGalleryItemNames now looks up the cache entry matching the gallery's current query args instead of taking the first entry from selectInvalidatedBy. RTK Query keeps unused entries warm for 60s after a board switch, and the old "first wins" behavior frequently landed on a stale board's name list, making shift-click silently no-op until a delete/move forced a refetch. * fix(scripts): force generate_openapi_schema.py to resolve invokeai from the repo root When the script was invoked as ``python scripts/generate_openapi_schema.py``, Python placed the script's directory at ``sys.path[0]`` rather than the repo root. ``import invokeai`` then resolved via the venv's site-packages, which on multi-worktree editable installs ends up importing ``invokeai`` as a PEP 420 namespace package that aggregates every worktree's ``invokeai/`` directory. Side-effect imports driven by submodule discovery silently miss whichever worktree isn't first on the namespace path, so the registry came up short by the invocations declared only in this worktree (the wan/video set, 15 classes). Running the same imports via ``python -c`` worked because ``sys.path[0]`` defaulted to the cwd and ``invokeai/__init__.py`` resolved cleanly to the worktree. Prepend the resolved repo root to ``sys.path`` before importing ``invokeai.*`` so the script always picks up the local sources regardless of how it was launched. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(video): add Frame Range from Video invocation with scrubbable preview New ``extract_video_range`` node trims a source video to a contiguous frame range and re-encodes it as MP4, slotting in naturally between a generated clip and Concatenate Videos for I2V chain shaping. Bounds are inclusive and support negative indices (``end_frame=-1`` keeps the final frame), matching Frame from Video. Output fps inherits from the input unless overridden. The node renders a per-type preview inside the workflow editor: two ``<video>`` tiles side by side, each driven by a CompositeSlider that scrubs the corresponding integer field. The tile uses ``currentTime = frame / fps`` so browsers display the seeked frame natively without a canvas roundtrip. Negative-index entries in the standard integer input are resolved against the source frame count for display only; the underlying field value is preserved verbatim. The custom UI is wired in via a ``CustomNodeBody`` dispatcher in ``InvocationNode.tsx`` rather than a registry — small enough to be explicit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(video): emit resolved frame indices and move preview to per-field renderer Three changes to the ``extract_video_range`` invocation: 1. New ``ExtractVideoRangeOutput`` mirrors ``VideoOutput`` and additionally emits the resolved (positive, 0-based) ``start_frame`` and ``end_frame`` indices. Chained workflows can feed those back into a downstream Frame from Video to extract the same boundary frame the trim landed on. 2. ``fps`` is now a plain ``int`` defaulting to 16 (was ``Optional[int]`` with an "inherit from input" fallback). Matches the default used by wan_l2v and the other Wan video producers, so chained workflows agree on framerate without each node guessing. 3. The frame preview is now a per-field widget driven by a new ``UIComponent.VideoFrameIndex`` hint. ``start_frame`` and ``end_frame`` are tagged with it; the new ``VideoFrameIndexFieldInput`` renders a number input plus a live <video> thumbnail and a scrubber slider, all writing to the same Redux field. Negative indices entered in the number input are still resolved against the source frame count for display only — the backend re-resolves at invoke time. The widget reads its companion ``VideoField`` (by convention, the sibling field named ``video`` on the same node) via direct Redux selectors, so it works wherever ``InputFieldRenderer`` is used — the workflow editor's node body AND the Form Builder's view/edit modes. The previous node-body ``ExtractVideoRangePreview`` and its ``CustomNodeBody`` dispatcher in ``InvocationNode.tsx`` are removed; the per-field widget supersedes both. In the workflow editor, side-by-side framing is lost in exchange for Form Builder support; users wanting the side-by-side layout in a form can group the two frame fields in a row container. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: ruff * fix(video): address PR #9163 review follow-ups - delete_board: include_images query description and OpenAPI schema now mention videos alongside images - get_video_thumbnail: check path existence before returning FileResponse so a missing thumbnail produces the documented 404 instead of an after-route error - delete_videos_on_board: stop deleting records for videos whose files failed to delete, so a transient FS error no longer orphans the file with no record pointing at it - DeleteBoardModal: destructive button and warning copy now mention videos * fix(video): address PR #9163 May-22 review and failing CI - remove_video_from_board now accepts either the direct video owner or a board write-access holder, so videos uploaded to a board that later flipped Public -> Shared/Private aren't stranded. - VideoService.create rolls back the DB record and board association if the underlying file save fails, preventing ghost records whose file endpoints 404. - delete_videos_on_board returns the actually-deleted names; delete_board uses that list so the response can't claim a video was destroyed when its record was preserved due to a file-delete failure. - Local test_videos_multiuser fixture now patches invokeai.app.api.routers._access so list/names route 403 checks work. - Regenerate schema.ts to pick up the CacheStats description. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(frontend): rebuild openapi * fix(video): register Viewer <video> as drag source Drag-and-drop from the Viewer pane now produces the same singleVideoDndSource (and multipleVideoDndSource for active multi-selection) as the gallery thumbnail, so a video can be dropped onto a Video Primitive's "Starting Video" field directly from the Viewer. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(video): add frame preview to Frame from Video node Tag frame_index with ui_component=VideoFrameIndex so the node renders the same live frame thumbnail + scrubber as Frame Range from Video. The widget keys off the sibling 'video' field, which this node already has, so no frontend changes are needed. Bump node version 1.0.0 -> 1.1.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(video): derive Frame Range fps from source video by default Make the fps field optional (default None). When unset, the output frame rate is inherited from the probed source video so a trimmed clip plays back at the same speed as its source, falling back to 16 fps when the source rate can't be probed. An explicit fps still overrides. Bump node version 1.0.0 -> 1.1.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video): use fps=0 sentinel for source-derived Frame Range rate The previous Optional[int]/None design had no natural way to express 'unset' in the node's number input, and the ge=1 constraint rejected the intuitive fps=0 with a validation error. Make fps a plain int defaulting to 0, allow ge=0, and treat 0 as 'match the source video's frame rate'. Keeps in-progress workflows (already saved with fps=0) working without a version bump. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(wan): add Wan 2.2 TI2V Ideal Dimensions node TI2V-5B uses the 16x Wan 2.2-VAE plus a 2x transformer patch, so pixel dims must be multiples of 32 (the existing I2V node snaps to 16, which the TI2V-5B patchify step rejects). Add a wan_ti2v_ideal_dimensions node that snaps to 32. Factor the shared scale-and-snap math into _scale_and_snap(multiple=...) so both nodes derive from one implementation; the I2V node is unchanged behaviorally (its existing tests still pass). Add a mirrored TI2V test suite. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(wan): add A14B/5B model hints to ideal-dimensions node titles Suffix the node titles with the target model family (A14B / 5B) so they're distinguishable in the add-node search and node header, and rewrite both docstrings to lead with which Wan 2.2 model they're for and cross-reference the other node. Purely UI metadata — no behavior or schema change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(workflows): replace bundled Wan 2.2 video workflows with curated set Remove the 6 previously-bundled Wan 2.2 *video* default workflows (Text to Video, Text to Video Lightning x2, Image to Video, Image to Video Lightning, Image to Video - Add Frames) and replace them with the 8 curated starter workflows: Text/Image to Video Lightning (+ Concept LoRA variants), Extend Video Lightning (+ Concept LoRA variant), and the TI2V-5B text/image-to-video low-quality variants. Each is assigned a stable default_ id and meta.category=default. Model fields are intentionally blanked (per-install keys don't resolve cross-instance) with the required models listed in each workflow's Notes. The two Wan 2.2 *image* workflows (Image to Image, Text to Image) are retained. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(video): add beginner Video Workflows guide for the 8 starter workflows New Features-section page (sibling to Video Generation) describing the eight bundled Wan 2.2 video workflows in plain language: how to choose between the Text/Image/Extend families and their Lightning / Concept-LoRA / TI2V-5B variants, how to select models from each workflow's Notes, how to run one, and a quick per-GPU guide. Cross-linked both ways with the Video Generation technical reference. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(video): fix Concept LoRA slot guidance (slots are required) The w/ Concept LoRAs workflows wire required lora fields (lora_selector / wan_lora_loader, no default) into the graph, so an empty slot blocks invocation. Correct the earlier claim that empty concept slots behave like the base workflow: every LoRA slot must be filled, and users without concept LoRAs should use the plain variant. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(video): drop '(experimental)' from Video Generation title Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: refresh uv.lock Routine lock refresh (transitive dev deps: docutils, idna, platformdirs, python_discovery, tornado). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(wan): add first-last-frame interpolation (FLF2V) to I2V-A14B The Reference Image - Wan 2.2 node gains an optional End Image input: when set, encode_reference_image_to_video_condition places the end image in the final temporal slot and anchors the mask at both the first and last latent frames, so I2V-A14B interpolates a clip from the start image to the end image. Mirrors diffusers WanImageToVideoPipeline.prepare_latents with last_image set. The denoise loop is unchanged - for A14B it just concatenates the 20-channel condition, which is agnostic to one vs two anchors. FLF2V is A14B video only (num_frames > 1); the encoder raises a clear error for TI2V-5B or single-frame. Bump wan_ref_image_encoder to 1.2.0; add mask-anchoring unit tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(workflows): add 'Interpolate 2 Images to Video' starter + FLF2V docs Ship a default workflow that wires the new FLF2V End Image input end to end (I2V-A14B + Lightning, two image inputs interpolated). Model fields blanked with the required models listed in Notes, default_ id + category=default. Document FLF2V in the Video Generation reference and add the workflow to the Video Workflows guide. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(workflows): add Text/Image/Video to Video library filters + fix video tags Add 'Text to Video', 'Image to Video', and 'Video to Video' to the Common Tasks filter list in the Workflow Library browser. Fix the tags on the nine bundled Wan 2.2 video workflows, which were all copy-pasted as 'text to video': - Text to Video: the three T2V workflows - Image to Video: the I2V workflows + Interpolate (two-image) - Video to Video: the two Extend Video workflows The TI2V-5B variants also drop the spurious lightning/lora tags (they have no LoRAs). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(workflows): add 'Extend Video to Image' FLF2V starter + docs Ship a default workflow that extends a video toward a user-provided target image: the new segment interpolates (FLF2V) from the source video's last frame to the destination image, then concatenates onto the original with a cross-fade. Model fields blanked, default_ id + category=default, tagged 'video to video'. Document it (card + usage instructions) in the Video Workflows guide. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(video): reorganize Video Workflows guide sections Group the Interpolate section and the Concept-LoRA / TI2V-5B asides with the image workflows, keep the Extend family (including Extend Video to Image) at the end, and retitle the section to 'Bundled video workflows'. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(frontend): regenerate openapi and typegen * chore(backend): ruff * fix(future): make the WAN LoRA loader compatible with LoRA picker node PR #9259 * chore(frontend): remove unused selectT5EncoderModels import * chore(frontend): remove unused export * fix(gallery): count videos and pick video covers for board tiles Gallery boards previously joined only the `images` table for their headline count and cover thumbnail, so a board containing nothing but videos rendered as empty with no preview. BoardDTO now exposes `video_count` and an optional `cover_video_name`; the boards service picks the best cover across both tables using the same (starred DESC, created_at DESC) tie-break the image path already used, and the gallery list renders `image_count + video_count` everywhere it previously rendered just images (real boards, no-board pseudo-board, and the tooltip). Adds `getBoardVideosTotal` to round out the no-board counts (the BoardVideosTotal tag was already wired into invalidation). * test(boards): wire video record storage into multiuser test fixtures After the board cover/count fix started reading from `video_records` and `board_video_records`, the multiuser test fixtures that still set both to `None` started erroring out — the boards router's catch-all turned the AttributeError into a 404, cascading through every test that PATCHes or GETs a board (auth, workflows, data-isolation suites). Swap the `None` placeholders for real SqliteVideoRecordStorage / SqliteBoardVideoRecordStorage instances (paralleling the existing image storage setup), and pin sane defaults on the MagicMocks in `test_videos_multiuser.py` so the get_dto cover/count lookups don't trip Pydantic validation. * fix(ui): widen useVideoContextMenu ref type to allow null The ref param was typed RefObject<HTMLElement>, but useRef produces RefObject<HTMLElement | null>, breaking lint:tsc in GalleryVideoItem. Match the sibling useImageContextMenu signature. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(backend): ruff * fix(tests): pass video/gallery services to InvocationServices in workflow-call router tests The workflow-call router tests from main construct InvocationServices directly and predate the video/gallery services added on this branch, so every test in the file errored with missing positional arguments. Mirror tests/conftest.py: real sqlite stores for video_records and board_video_records, None for the services the tests never touch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): refresh board caches when a generated video completes Video completion previously invalidated only the polymorphic gallery list tags, so the new video appeared in the grid while the board's video_count, cover thumbnail (Board tag / listAllBoards), and BoardVideosTotal stayed stale until an unrelated mutation refetched them. Use the shared getTagsToInvalidateForBoardAffectingMutation helper over the affected boards, matching the video mutation endpoints. Reported by @JPPhoto in PR #9163 review. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: include videos in date-based virtual boards Date virtual boards were image-only even though the gallery grid is now polymorphic: video-only dates never appeared, and mixed dates omitted videos from counts/contents/covers. - SqliteGalleryService owns virtual-board dates now: get_dates() unions images+videos per date (video_count added to VirtualSubBoardDTO, cover is the newest item of either kind via cover_image_name/cover_video_name), and list_item_names() gained a created_date filter. - New GET /api/v1/virtual_boards/by_date/{date}/item_names returns the same polymorphic (kind, name) refs as the gallery names endpoint; the legacy image_names route is kept for API compatibility. - Frontend virtual-board selection consumes the new endpoint, so videos show up in virtual date boards; VirtualBoardItem shows video counts (localized tooltip) and falls back to the video thumbnail for video covers. - tests/conftest.py wires a real SqliteGalleryService so router tests exercise the filter SQL; service + router tests cover video-only dates, mixed dates, cover selection, and per-user isolation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): don't advance gallery selection for videos whose delete failed handleDeletions treated every requested video as deleted when picking the post-delete selection, so a 403/500 on deleteVideo could jump the Viewer away from a video that still exists, and a surviving neighbour was skipped as a replacement candidate. Only successfully deleted names now count: a failed displayed video keeps its selection, and failed neighbours remain valid replacements. Covered by state.test.ts with rejected deleteVideo dispatches. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): count uncategorized videos when deciding the gallery has content useHasImages only looked at boards and the uncategorized *image* total, so a gallery whose only content was an uncategorized video rendered the new-user/get-started view instead of the normal no-selection state. The hook now also reads the uncategorized video total (getBoardVideosTotal('none')); the decision logic is extracted as getHasGalleryContent and unit-tested. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): stop labeling board video counts as images in tooltips Board tooltips folded video_count into image_count and rendered boards.imagesWithCount, so a video-only board read e.g. '1 image, 0 assets'. Tooltips now show split image/video/asset counts using the new boards.videosWithCount translation; the compact unlabeled headline count in the boards list stays combined so video-only boards don't read as empty. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): restrict client-side video upload acceptance to MP4 only The dropzone accept map advertised .webm/.mov and isVideoFile treated .webm/.mov/.mkv as videos, but the upload router accepts MP4 only, so those files were accepted client-side and then rejected with 415 after the bytes were uploaded. Consolidate the accepted-media lists into common/util/uploadMediaAccept.ts (single source of truth shared by useImageUploadButton and FullscreenDropzone) and pin them to the backend contract with a regression test. Also split the accept map: image-only upload fields (board covers, style presets, model images, workflow thumbnails) no longer advertise video/mp4, which they had inherited when video entries were added to the shared map. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): bulk video star/unstar returns partial successes instead of 403 mid-batch star_videos_in_list and unstar_videos_in_list re-raised the ownership HTTPException mid-loop, so a batch containing one foreign (or stale) name mutated the earlier owned videos and then returned 403 with no payload — the client never invalidated caches for the videos that did change. Skip unauthorized names and return 200 with the actually starred/unstarred videos, mirroring delete_videos_from_list. Router tests cover the mixed-ownership batch for both routes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): localize virtual board section header and toggle The 'By Date' header and the Collapse/Expand aria-label in VirtualBoardSection were hardcoded English. Add boards.byDate and common.collapse/common.expand translation keys and use them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(app): failed video saves no longer orphan files on disk DiskVideoFileStorage.save() moves the source MP4 into permanent storage before writing the thumbnail and sidecar, so a failure in either later step used to leave the moved MP4 (and partial artifacts) on disk with no DB record through which they could be managed. save() now removes its destination files before raising, and VideoService.create()'s rollback also deletes files to cover failures after a successful file save (e.g. building the DTO). Also documents why board attachment during create is best-effort (mirroring ImageService.create: a board deleted mid-generation must not destroy the render) and pins the explicit fallback — DTO reports the actual missing board association and a warning is logged — with a service test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(app): document videos.user_id lifecycle and pin user-deletion behavior videos.user_id deliberately has no FK to users, matching images/boards/ workflows (migration_27 adds those user_id columns index-only): deleting a user leaves their media in place for admin review/cleanup rather than cascading a row delete that would strand files on disk. A migration comment now states the parallel, and a migration-backed test creates a user and a video, deletes the user, and asserts the record survives, stays attributed to the deleted owner, and remains visible only to admins. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(app): support VideoField values in session queue batch data Batch.data previously allowed ImageField but not VideoField, so submitting multiple VideoField values through the generic batching capability failed Pydantic validation before enqueueing. VideoField now joins the BatchScalarDataType union; a test asserts a VideoField batch validates and expands into separate sessions. schema.ts/openapi.json regenerated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): video uploads are opt-in per consumer; upload validation fixes - useImageUploadButton gains an allowVideos opt-in (default off). Only the gallery uploader accepts videos; image-only consumers (ref images, board covers, launchpad buttons, image-to-prompt, etc.) no longer let a selected MP4 upload into the gallery while the requested image action goes nowhere. Videos are excluded from their accept map and rejected at runtime if the file dialog bypasses it, with tests via partitionUploadFiles. - The hook's loading state now covers both the image and video mutations, so an in-flight MP4 upload shows a loading button and blocks resubmission. - The fullscreen drag-drop/paste validator accepts a file when either its MIME type or its extension is recognized — a clip.mp4 with an empty File.type used to be rejected even though the backend accepts it. The validator moved to a pure module with tests. - Failed video uploads no longer toast "Image Upload Failed": video-only batches use a new toast.videoUploadFailed key, mixed batches the neutral toast.uploadFailed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(app): bound untrusted video decoding with a killable subprocess timeout probe_video / extract_video_frame / decoder_frame_count now run in a short-lived child process (video_decode_worker.py) killed after a hard timeout. Previously a crafted MP4 that failed the imageio probe and then hung inside cv2.VideoCapture()/read() would pin the FastAPI request worker that called it forever; repeated uploads could exhaust the pool. The worker is run by file path (not -m) and imports only imageio/PIL/cv2 so it starts without pulling in the invokeai package or torch. Tests substitute a never-returning worker command and assert the helpers fail within a bounded interval, plus happy-path tests against a real synthetic MP4 to validate the subprocess plumbing end to end. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(app): stream frames through the video concat/trim nodes video_concat and extract_video_range fully decoded their inputs into lists of uncompressed frames before encoding; with the 1 GB upload cap a long 1080p source can expand to tens of gigabytes of RAM, so any user able to enqueue these nodes could exhaust server memory. Frames now stream from the decoder straight into an incremental FFMPEG writer. The concat node buffers only the transition windows (bounded by transition_frames), and the range node holds one frame at a time and stops decoding at the end of the requested range. Tests use lazy frame iterators to pin that encoding begins before the inputs are exhausted and that look-ahead stays bounded. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): video upload feedback parity with images - Add uploadVideo matchFulfilled/matchRejected listeners mirroring the image upload listeners: success toasts name the destination board and navigate the gallery on the first upload of a batch; failure toasts name the failed file, which is what makes partially failed Promise.allSettled batches attributable (uploadVideos and the fullscreen dropzone aggregate without rethrowing, same as images). - GalleryUploadButton now uses the hook's combined isUploading, so an in-flight MP4 shows a spinner and blocks resubmission. - Media-neutral labels on the two video-enabled surfaces: gallery uploader aria/tooltip says Upload Media, the fullscreen overlay says uploaded items (not images) will be added, and its invalid-file toast mentions MP4. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: include videos in the user-deletion data-loss note The admin guide's user-deletion warning enumerated boards, images, workflows, queue items, and style presets but not videos. State that video records survive with the deleted user_id, that files remain under outputs/videos, and that administrators keep gallery visibility of the orphaned media for review/cleanup — matching the behavior pinned by the video_records user-deletion lifecycle test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix video processing resource bounds * fix video lifecycle edge cases * stabilize decoder inactivity test * chore(deps): declare psutil as a direct dependency video_thumbnails.py now imports psutil for decode-worker process-tree termination, but it was only present transitively (via accelerate and friends). Declare it so the import can't silently break when an upstream package drops it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix video security and lifecycle regressions * chore: regenerate OpenAPI schema * fix: preserve exact frame dimensions in video encoders imageio's FFMPEG writer defaults to macro_block_size=16, which makes ffmpeg silently rescale frames to the next multiple of 16 — a 1920x1080 upload trimmed by Frame Range from Video came back as 1920x1088 while the DTO recorded 1080, so concatenating the trim with its own source failed the same-dimensions check. - New invokeai/app/util/video_encoding.make_mp4_writer single-sources the encoder settings (libx264, macro_block_size=1) for wan_latents_to_video, video_concat, and video_frame_extract_range. - yuv420p requires even dimensions, so concat and extract-range now reject odd-dimension sources up front with a clear error instead of an opaque ffmpeg failure mid-encode. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: correct A14B fallback scheduler and stop LoRA leakage to low-noise expert Two silent-wrong-output bugs on the GGUF A14B path: - The no-scheduler-dir fallback returned FlowMatchEulerDiscreteScheduler for A14B, but both Wan-AI/Wan2.2-{T2V,I2V}-A14B-Diffusers repos ship UniPCMultistepScheduler with flow_shift=3.0 / flow_prediction / use_flow_sigmas (verified against the upstream scheduler_config.json). Every A14B GGUF render ran an unshifted first-order Euler schedule, degrading output and skewing how many steps land above the MoE boundary. An unreadable on-disk config now also falls back to the variant default instead of bare FlowMatchEuler. - low_loras fell back to the primary list when loras_low_noise was empty, but the Wan LoRA loader deliberately routes expert-tagged LoRAs to exactly one list — so a high-noise-only LoRA (e.g. a Lightning high-noise distill) was silently applied to the low-noise expert too, and high-only targeting was impossible. An empty low list now means no LoRAs on the low expert. Also (here and in the previous commit): the Wan VAE decode nodes now raise a clear latent-channel mismatch error (16-channel A14B latents vs 48-channel TI2V-5B VAE and vice versa) instead of an opaque tensor-size RuntimeError when the wrong VAE is selected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: keep viewer selection on surviving item after image deletion The image-side handleDeletions cleared the gallery selection (imageSelected(null)) whenever the deletion intersected the multi-selection but the displayed item was not among the deleted names — e.g. a video displayed while only images were deleted from a mixed selection, or a hover-delete of a non-displayed selected image. It also treated every requested name as deleted, ignoring the server's deleted_images response, so a partial failure could jump the selection away from an image that still exists. Port the deleteVideoModal logic: only server-confirmed deletions count, a surviving displayed item stays selected, and the usage-reset sweep (nodes/canvas/ref-image layers) runs only for actually-deleted images. Regression tests mirror deleteVideoModal/store/state.test.ts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * perf: ~48x faster Wan VAE decode on ROCm via conv2d decomposition MIOpen has no implicit-GEMM 3D-convolution kernels for the Wan VAE's shapes on RDNA3 and falls back to Im3d2Col (61% of decode GPU time in a torch profile). An 81-frame 832x480 decode took 730s on a W7900 vs 13s on an RTX 5060; dtype changes and cudnn.benchmark kernel search were all within +/-7%. A stride-1 kTxkHxkW conv3d is exactly the sum of kT conv2d taps over shifted temporal slices, and MIOpen's conv2d kernels are well optimized. This rebinds WanCausalConv3d.forward (class-level, idempotent, ROCm builds only) to that decomposition: - same 3-latent-frame decode: 81.6s -> 1.71s (~48x), extrapolating to ~12s for the 81-frame workload — matching NVIDIA wall-clock - numerically equivalent up to summation order: ~1e-6 max error vs F.conv3d in fp32; full bf16 decode differs by <=3/255 in pixel space (0.1% of pixels by more than 1/255) - strided encoder downsample convs keep the stock F.conv3d path (temporal taps couple under stride) - applied from every AutoencoderKLWan load site (Wan checkpoint/diffusers VAE loaders, Wan main-model VAE submodel, Anima VAE), so decode, encode, and ref-image conditioning all benefit; CUDA builds are untouched Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: self-heal the media cookie for sessions that predate or outlive it Video playback authenticates via an HttpOnly cookie (media elements can't send Authorization headers) that was only issued at login. A session restored from localStorage can hold a valid JWT without the cookie — the session may predate the cookie's introduction, or the cookie may have been cleared while the JWT survived. Every API call works, but each <video> request 401s and the player silently renders black with 0:00 duration (hit during PR #9163 functional testing). - New POST /api/v1/auth/media-cookie re-issues the cookie from a valid Bearer token: same live-user check as get_current_user, cookie lifetime clamped to the token's remaining validity, successful no-op in single-user mode. Cookie attributes are shared with login via _set_media_cookie so they can't drift. - Frontend calls it once per app load when an authenticated session exists (useMediaCookieRefresh in GlobalHookIsolator); failures are left to the existing global 401 handling. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: use Apply LoRA Collection node in Wan concept-LoRA workflow templates Replace the per-slot concept-LoRA plumbing in the three 'w/ Concept LoRAs' video templates (Text to Video, Image to Video, Extend Video) with the single wan_lora_collection_loader node: users now add any number of concept LoRAs through one multi-LoRA form field instead of two fixed slots (T2V/Extend) or the lora_selector + collect chain (I2V). Chain in all three: model loader -> Lightning high-noise LoRA -> Lightning low-noise LoRA -> Apply LoRA Collection (concept LoRAs, ships empty) -> denoise. Also prunes exposedFields entries that referenced nodes deleted in an earlier revision of these templates (pre-existing; the frontend ignored them, but they were dead weight). Validated: backend WorkflowValidator + default-sync asserts, node versions current, every edge/form/exposedFields reference resolves, frontend parseAndMigrateWorkflow accepts all three, no machine-specific model identifiers ship. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: enforce multiuser image authorization * fix: address remaining Wan video review findings * test: give closed-stream decoder test headroom for slow Windows spawn The 0.2s decode timeout raced against Python subprocess startup on the Windows CI runner: the inactivity deadline fired before the worker could close its stdout, raising the generic decode timeout instead of the expected 'decoder worker' one. A generous timeout makes the EOF path deterministic; proc.wait still bounds the test at ~1s. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address adversarial Wan video review findings * fix: resolve remaining Wan video review issues * feat(ui): rename gallery/board strings from Images to Images/Videos The gallery grid, selections, board operations, and related settings now operate on both images and videos, so the user-facing strings that describe them say so. Image-only surfaces (compare, reference images, progress previews, image storage maintenance, upload-format errors) are unchanged, as are unused legacy keys. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): harden video/media API per PR review - Scope the media cookie (set + delete) and the sliding-token middleware's auth-route exclusions to the reverse-proxy root_path, so media auth works behind sub-path proxies and a proxied logout can't mint a replacement token. - Video upload: run filesystem writes, MP4 validation, ffmpeg probing, and create() in the thread pool; add VideoUploadLimitASGIMiddleware to bound request size before multipart spooling and cap concurrent uploads. - Add GET /videos/i/{name}/workflow (mirrors the image route) so persisted video workflows/graphs are retrievable, with read-access checks. - Add DELETE /videos/uncategorized so the "Delete All Uncategorized Images/Videos" action can cover both media kinds. - Make polymorphic gallery ordering deterministic on created_at ties with kind+name tie-breakers, and pick virtual-board covers via ROW_NUMBER instead of a bare-column MAX() aggregate. - Add cpu_only to WanT5Encoder_WanT5Encoder_Config (parity with the other standalone text-encoder configs; the loader already honors the field). - Regenerate schema.ts/openapi.json. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(backend): Wan VAE effective device + generalized working-memory estimation - Move VAE inputs to get_effective_device(vae) instead of the globally selected device — a cpu_only Wan VAE previously crashed every Wan VAE invocation on GPU hosts. - Add estimate_vae_working_memory_wan (per-frame conv working set + resident RGB clip, config-driven spatial scale for TI2V's 16x compression) and reserve working memory in all four Wan VAE paths, replacing the Flux estimator / missing reservations. - Fall back to spatial tiling for video decodes whose full-frame working set exceeds the execution device's VRAM, and move the decoded clip to the CPU before MP4 encoding so VRAM isn't held for the encode's duration. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): video gallery/deletion/workflow fixes per PR review - Video deletion: use the batch endpoint (one request per invocation), clear workflow-node VideoField inputs only for server-confirmed deletions, and invalidate per-video DTO/metadata/workflow caches on delete (including board-cascade deletions in the board mutations). - VideoFieldInputComponent resets its value only on a confirmed 404, not on transient auth/server/network errors. - Global Delete hotkey partitions the polymorphic selection and routes videos through the video delete flow. - "Delete All Uncategorized Images/Videos" now deletes both media kinds; "Download Board" relabeled "Download Board Images" (image-only endpoint). - Translation splits: image-only multi-select actions revert to "Images"; polymorphic gallery search + star hotkey become media-neutral; the multi- drag preview counts the whole mixed selection. - Expose video metadata + workflow in the viewer: new video details overlay (metadata/workflow/graph tabs), a Load Workflow toolbar action for videos, and a 'video' source for the load-workflow dialog. - Model Manager: wan_t5_encoder gets the encoder settings panel (Run on CPU). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: make gallery docs video-aware; fix video workflow count Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address remaining video review findings * fix(backend): Wan inference fixes from full-PR self-review - Force bfloat16 in the standalone Wan VAE checkpoint loader: `precision: auto` resolves to fp16 on CUDA, and fp16 is unstable on the Wan VAE (the diffusers folder path already forced bf16). Both starter VAEs route through this loader. - Count the decoded RGB clip twice in the Wan working-memory estimator: diffusers' frame accumulation transiently holds ~2x the clip at peak, which the tiled-decode fallback previously undercounted by up to ~2 GB. - Ignore a wired 'Transformer (Low Noise)' for TI2V-5B (warn instead of raising a misleading A14B error), matching the field's documented behavior. - Release the expert swapper's device context even when LoRA weight-restore raises, so a failed unwind can't pin an 8-9 GB expert in VRAM. - Validate LoRA variant (A14B vs 5B) against the wired transformer in both Wan LoRA loaders — a mismatch previously crashed mid-denoise with an opaque layer-patcher shape error. - Fix the WanDiffusersModel exception ladder: the old-diffusers torch_dtype retry now also gets the missing-variant OSError fallback, with the matching dtype kwarg. - Mark both Wan ideal-dimensions nodes Prototype like every other Wan node; correct the text-encoder docstring (seq_len 512, not 226). - Add CPU tests for the multi-frame WanVideoDenoise loop (T_lat>1 shapes, zero-velocity invariant, A14B I2V 36-channel concat across frames, TI2V-5B expand-timesteps mask blend incl. per-token timesteps and frame-0 restore). Node version bumps: wan_model_loader, wan_lora_loader, wan_lora_collection_loader -> 1.0.1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): video service fixes from full-PR self-review - Purge cached invocation outputs on video deletion: the memory invocation cache registered images/tensors/conditioning on_deleted hooks but not videos, so re-running an identical graph after deleting its output "succeeded" with a cached VideoOutput naming a 404 video. - Add the single-user early-return to VideosInterface's read-access and board-save checks, matching ImagesInterface — after a multiuser->single-user switch, video workflows no longer fail with PermissionError where identical image operations succeed. - Restructure staged-delete recovery to match the image side: video_records .get() raises rather than returning None, so the explicit commit branch was unreachable and recovery semantics lived in the exception handler by luck. - Return 416 (not 206 with "bytes 0--1/0") for any Range request against a zero-length video file; add tests for the whole Range-parser matrix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): video UX fixes from full-PR self-review - Add 'video' to the invocation-complete passthrough denylist: a Video Primitive completing mid-run invalidated gallery caches and auto-switched the user's selection/board to the node's *input* video. - Show the multi-selection context menu only when the clicked item is part of the selection (both image and video menus): right-clicking a video with 2+ images selected previously produced a menu with every action disabled. - Clear workflow VideoField references when videos are cascade-deleted via board deletion or delete-uncategorized, matching the direct-delete flow. - Toast on total video-delete-batch failure (the untracked mutation was otherwise silent) and on failed logout (the button previously did nothing when the server was unreachable). - Check resp.ok in useDownloadItem so an expired media cookie can't save error bodies as .mp4/.png files. - Provide the LIST_TAG-scoped VideoList tag from listVideos so the star/board invalidations that reference it actually match; fix the misleading comment; dedupe the doubled BoardVideosTotal tag type. - Validate VideoField access on workflow load (checkVideoAccess), resetting stale refs with a warning like image fields. - Wire middle-click-open-in-new-tab for gallery videos (the setting label already promised it). - Show the effective fallback (primary CFG) in the low-noise guidance slider when unset, instead of a constant the run never uses. - "Moving 1 image/video to board:" singular form for mixed-media moves. - Regenerate schema.ts/openapi.json (node version bumps, classification, docstring fixes). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: workflows/docs/deps fixes from full-PR self-review - Update all 12 bundled Wan workflows to current node versions (wan_model_loader / wan_lora_loader / wan_lora_collection_loader 1.0.1, wan_ref_image_encoder 1.2.0, backfilling the optional end_image/num_frames inputs) so fresh installs don't open with "node needs update" badges; add a registry-consistency test over the bundled Wan/video workflows so stale embeds can't recur. - Docs: the A14B auto scheduler is UniPC (not FlowMatchEuler); note that the bundled TI2V-5B workflows ship 20 steps as a speed compromise vs the 40-50 quality recommendation. - Pin imageio[ffmpeg]>=2.37 and psutil>=6 (imageio encode behavior is version-sensitive enough that we carry a regression test for it); relock. - De-flake the thumbnail worker descendant-kill test (0.5s was the only tight ceiling in the file; a loaded runner could kill the worker before the child pid file existed). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ci: raise python-tests job timeout to 30 minutes Main already runs 9-11 min per platform and this PR pushed py3.11 windows-cpu past the 15-minute cap (cancelled mid-pytest at 15m10s on the last run). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: resolve CI failures from main's route-auth audit and knip The route-authorization audit merged from main (#9367) only recognized the two Bearer-token dependencies, so it flagged the video media routes (which authenticate via get_current_media_user_or_default) and the media-cookie endpoint (which validated its Bearer token inline). Teach the audit about the media dependency, drop the image media routes from PUBLIC_ROUTES (they now carry cookie auth on this branch), and give refresh_media_cookie a CurrentUserOrDefault dependency in place of its duplicated inline validation. The media-cookie tests now patch auth_dependencies' ApiDependencies like every other auth-dependent route test. knip: getDeletedVideosFromDeleteBoardAction was exported but only used in-module; cover it in the listener unit tests like its image twin. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(backend): Wan invocation fixes from JPPhoto's 2026-07-21 review - Both Wan LoRA loaders now validate the *resolved* config (type=LoRA, base=Wan) instead of trusting the client-supplied identifier fields; a mislabeled Flux/SDXL/main key is rejected up front instead of reaching the layer patcher. - The collection loader rejects LoRAs already applied upstream on either expert list (same invariant as the single loader) instead of silently doubling their effective weight. - A LoRA routed only to the low-noise list of a TI2V-5B main now logs a warning — the single-transformer path never consumes that list, so the routing was a silent no-op. - _ExpertSwapper._release clears its slots in a nested finally, so a device-context exit failure can no longer leave stale contexts that a later close() would double-exit. - WanLatentsToImage rejects multi-frame (T>1) video latents with a clear error pointing at wan_l2v, before the VAE is even loaded — previously it ran the full multi-frame decode and died in an opaque einops rank error. - wan_ref_image_encoder docstrings now describe both the 20-channel A14B and 48-channel TI2V-5B condition paths (they claimed A14B-only and told users to omit the node for TI2V-5B, contradicting the implementation). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): per-user upload slots, probe validation, stream decode fallback - VideoUploadLimitASGIMiddleware now accounts upload slots per user (cap 2) on top of the global cap, so one tenant's slow chunked uploads can no longer hold all four slots and starve other users into 429s. Single-user mode keeps the whole global capacity (no per-user quota). - probe_video validates decoder-reported metadata: non-positive or over-limit dimensions (> 64 MP) and non-finite/negative durations are rejected before the upload path persists them; garbage fps degrades to None (unknown). The decode worker refuses to decode frames from files whose probed dimensions exceed the bound — a small crafted container claiming 100k x 100k would otherwise trigger a ~30 GB allocation. - The worker's stream command falls back to cv2 like probe/frame/count do, so an MP4 accepted at upload via the cv2 path now also works in the frame-range and concat nodes. The fallback only engages before the first emitted frame; a mid-stream decoder death still surfaces as an error. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): media resilience fixes from JPPhoto's 2026-07-21 review - Partial board deletion: the boardAndImagesDeleted listener now invalidates the per-item Image*/Video* tags for the confirmed-deleted names it parses out of the 500 detail — the rejected mutation runs invalidatesTags with no result, so those caches previously stayed readable. - Media-cookie refresh retries transient failures on a bounded backoff (2s, 10s) instead of latching before the request and giving up forever; 401 still bails (session genuinely expired). - Thumbnail 404s degrade gracefully: BoardTooltip, GalleryBoard, VirtualBoardItem, and VideoFieldInputComponent show an icon fallback via fallbackStrategy="onError" (thumbnail generation is best-effort server- side), and GalleryVideoThumbnail's <video> fallback does the near-zero seek on loadedmetadata so browsers that don't auto-paint the first frame no longer show a black tile. - CurrentVideoPreview handles play() rejection (rolls isPlaying back) and media element errors (drops back to the play overlay) instead of hiding the overlay over a dead element with an unhandled promise rejection. - Hardening from the disputed items: changeVideoIsIntermediate also invalidates the VideoList LIST_TAG (covers a future omitted-board_id list); logout clears gallery.selection and the logout mutation documents that resetApiState in store.ts is what actually clears cross-user caches. - Typegen regenerated for the wan_lora_loader / wan_ref_image_encoder docstring updates. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address final video review findings * fix: harden video workflows and auth refresh * chore: regenerate OpenAPI schema * fix: close video workflow review gaps * fix: address self-review findings on video workflows and auth refresh Fixes the confirmed findings from the 2026-07-22 self-review round (github.com/invoke-ai/InvokeAI/pull/9163#issuecomment-5051225515): Frontend: - DeleteVideoModal: detach the dialog promise callbacks before the async deletion so the accept path's synchronous onClose (bound to cancel) no longer rejects a confirmed deletion as "User canceled"; dismissal still rejects. Adds behavioral tests for both paths. - Sliding-window refresh: bound the media-cookie sync fetch with a 10s AbortSignal timeout so a stalled request can't hold the exclusive cross-tab media-auth lock (shared with login/logout) forever; commit the refreshed token even when the cookie sync fails with a 5xx/network error (only a 401/403 rejection of the token blocks the commit); throttle acceptance to once per minute so bulk mutations don't pay a serialized cookie round trip per request. - Fallback media-auth lock: waiters renew their ticket lease while queueing so a >30s wait no longer lets a later ticket enter concurrently. - useMediaCookieRefresh: a pause() abort now resumes the same attempt instead of consuming a retry slot (and no longer permanently disables self-heal when the final attempt was paused); effect cleanup aborts in-flight refreshes so every logout path (sessionExpiredLogout, direct logout) stops a pending refresh from re-minting the cookie post-logout. - CurrentVideoPreview: a benign AbortError from play() rolls back silently, and load errors during the pending media-cookie self-heal window no longer raise a spurious "Unable to Play Video" toast. Backend: - Decode-worker memory bounds resized for legal near-cap frames: worker RLIMIT_AS headroom 1->4 GiB, parent RSS kill threshold 1->3 GiB (with keep-in-sync cross-references), monitor poll 50->250 ms. - Upload probe: a decode-worker timeout is now inconclusive (upload proceeds) instead of a 415; the probe's decoded first frame is reused as the thumbnail source, dropping one worker subprocess per upload. - delete_images_on_board / delete_videos_on_board return (deleted, failed) and delete_board reports the services' ground truth instead of a racy router-side listing diff (which also doubled the DB work). - Video list/uncategorized delete endpoints skip HTTPException (ownership skips, 404 races) silently instead of reporting them as failures, matching the image endpoints; delete_images_from_list now populates failed_images for genuine failures, matching the video path. - video_concat: an unknown probed fps mixed with agreeing known rates uses the known rate again instead of hard-erroring; disagreeing known rates still require an explicit Output FPS. - SlidingWindowTokenMiddleware runs its synchronous SQLite user lookup via run_in_threadpool so a contended DB lock can't stall the event loop. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(tests): deflake drip-feed upload timeout test on Windows The test gave the middleware a 20 ms absolute upload deadline and delivered a chunk every 5 ms — but Windows event-loop timers have ~15.6 ms granularity, so the deadline could expire before the first chunk was ever delivered. The request then ended at receive_calls == 1 and the `receive_calls > 1` assertion failed (py3.12 windows-cpu CI). Widen the margins so the scenario the test describes actually occurs on coarse timers: 250 ms absolute deadline (many chunks flow first on every platform) with a 1 s idle timeout that never fires between 5 ms chunks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(tests): sort imports in test_video_upload_limits (ruff I001) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address in-PR items from JPPhoto's non-merge-blocker list Fixes the subset of JPPhoto's 2026-07-22 "Still Open, Non-Merge Blockers" that are small, self-contained, and scoped to surfaces this PR introduced; the rest are deferred to a follow-on PR (triage rationale posted on the PR). - video_thumbnails._run_worker: an unexpected exception from communicate() (e.g. OSError) now terminates the worker process tree unconditionally — previously the finally stopped the RSS-monitor backstop while the except path left the worker and its ffmpeg child running forever. Adds the injected-OSError test JPPhoto asked for. - reduxRemember driver: client-state persistence POSTs now commit X-Refreshed-Token via the same acceptance flow as dynamicBaseQuery (extracted as acceptRefreshedToken, sharing the cross-tab lock, cookie sync, throttle, and generation guards), so persistence-only sessions no longer hard-expire mid-activity. - delete_videos_from_list / delete_images_from_list: dedup request names — a repeated name was processed twice and landed in both deleted_* and failed_* under the admin ownership bypass, toasting a spurious partial failure. Regression test added. - gallery + videos list endpoints: bound offset (ge=0) and limit (ge=0, le=MAX_PAGE_SIZE=1000) — these flowed verbatim into SQL, where a negative LIMIT means unlimited in SQLite, so one request could materialize the entire gallery. openapi.json regenerated (schema.ts is unchanged — constraints don't alter the generated types). - get_video_full: open the file once and serve HEAD/range/full from the fd (full downloads now stream chunked from the handle instead of FileResponse's lazy path-based open), eliminating the delete-race 500; deletion's atomic rename can no longer invalidate a path between check and open. - upload_video: close the multipart spool immediately after the body copy, shrinking the double-temp-disk window (2 x 1 GiB x 4 concurrent worst case) to the copy loop itself. - docs/gallery.mdx: document shared-board deletion semantics (only your own media is permanently deleted; admins delete everything) and the kept-on-failure -> Uncategorized behavior with its UI warning. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix video gallery review findings * fix Windows video thumbnail handling * test: cover remaining video review findings * test: cover adversarial video and Wan findings * fix: address remaining video and Wan review findings * test: call now-sync star/unstar routes directly 11b38696bf converted the video batch routes from async def to sync def (so FastAPI offloads them to its threadpool), but the star/unstar dedupe test still drove them through asyncio.run(), which requires a coroutine. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: cover Wan conditioning and video link regressions * fix: validate Wan conditions and video links --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Jonathan <34005131+JPPhoto@users.noreply.github.com> Co-authored-by: JPPhoto <jpollack@jpollackphoto.com> | 1 个月前 | |
feat: add profiler util (#5601) * feat(config): add profiling config settings - `profile_graphs` enables graph profiling with cProfile - `profiles_dir` sets the output for profiles * feat(nodes): add Profiler util Simple wrapper around cProfile. * feat(nodes): use Profiler in invocation processor * scripts: add generate_profile_graphs.sh script Helper to generate graphs for profiles. * pkg: add snakeviz and gprof2dot to dev deps These are useful for profiling. * tests: add tests for profiler util * fix(profiler): handle previous profile not stopped cleanly * feat(profiler): add profile_prefix config setting The prefix is used when writing profile output files. Useful to organise profiles into sessions. * tidy(profiler): add `_` to private API * feat(profiler): simplify API * feat(profiler): use child logger for profiler logs * chore(profiler): update docstrings * feat(profiler): stop() returns output path * chore(profiler): fix docstring * tests(profiler): update tests * chore: ruff | 2 年前 | |
Fix lint error. | 1 年前 | |
chore: fix some comments Signed-off-by: jiangmencity <jiangmen@52it.net> | 1 年前 | |
feat: single app entrypoint with CLI arg parsing We have two problems with how argparse is being utilized: - We parse CLI args as the `api_app.py` file is read. This causes a problem pytest, which has an incompatible set of CLI args. Some tests import the FastAPI app, which triggers the config to parse CLI args, which receives the pytest args and fails. - We've repeatedly had problems when something that uses the config is imported before the CLI args are parsed. When this happens, the root dir may not be set correctly, so we attempt to operate on incorrect paths. To resolve these issues, we need to lift CLI arg parsing outside of the application code, but still let the application access the CLI args. We can create a external app entrypoint to do this. - `InvokeAIArgs` is a simple helper class that parses CLI args and stores the result. - `run_app()` is the new entrypoint. It first parses CLI args, then runs `invoke_api` to start the app. The `invokeai-web` project script and `invokeai-web.py` dev script now call `run_app()` instead of `invoke_api()`. The first time `get_config()` is called to get the singleton config object, it retrieves the args from `InvokeAIArgs`, sets the root dir if provided, then merges settings in from `invokeai.yaml`. CLI arg parsing is now safely insulated from application code, but still accessible. And we don't need to worry about import order having an impact on anything, because by the time the app is running, we have already parsed CLI args. Whew! | 2 年前 | |
feat: multi-GPU parallel session execution (#9263) * feat(app): parallel multi-GPU session execution Run one generation session per configured GPU concurrently, with a tiled progress preview. Multi-user isolation is unchanged. Backed by five seams: - Per-thread device context (TorchDevice.set/get/clear_session_device); choose_torch_device() consults it first, so all device-selecting call sites resolve to the calling worker's GPU with no per-node changes. - Per-device model caches: build_model_manager builds one ModelCache per generation device; ModelLoadService.ram_cache resolves by current thread device; ram_caches fans out clear/drop/shutdown. - Atomic concurrent dequeue: a dequeue lock makes select+claim atomic so concurrent workers never claim the same item (works on FIFO; round-robin from #9086 slots in later). - Worker pool: one _SessionWorker per device, each pinning torch.cuda.set_device and its session device, with its own runner and cancel event; cancellation routes via an {item_id -> worker} lookup. Single-device installs keep the exact legacy single-worker behavior. Profiling disabled when >1 worker. - New config `generation_devices`; unset = legacy single-worker mode. Frontend: the canvas staging area already tiles per queue item; the main ImageViewer now tracks progress per session and renders a tile grid (ProgressImageTiles) when more than one session is active. Also adds a lock to ObjectSerializerForwardCache for concurrent access. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(tests): restore global device after multi-GPU cache routing test test_model_load_device_routing mutated the process-wide get_config() singleton (device = "cuda:0") to exercise the per-thread cache routing, but never restored it. The leaked CUDA device was then picked up by a later test (test_model_load::test_loading) via choose_torch_device(), which crashed with "Torch not compiled with CUDA enabled" on the CUDA-less CI runner. Add an autouse fixture to save/restore device and clear any pinned session device. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(ui): regenerate openapi schema and frontend types for generation_devices Regenerate openapi.json (make frontend-openapi) and the frontend schema.ts types (make frontend-typegen) so they include the new generation_devices config field, fixing the openapi-checks and typegen-checks CI jobs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(ui): regenerate openapi.json with uv to match CI generator `make frontend-openapi` used a bare `python` from a different environment that emitted the CacheStats @dataclass docstring as a schema description. CI generates the schema via `uv run`, which does not, so openapi-checks failed on the diff. Regenerate with the uv-locked environment to drop the stray description while keeping the generation_devices field. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(model-manager): serialize model construction against VRAM moves to prevent meta-device corruption Parallel multi-GPU session workers could intermittently crash with "unrecognized device meta" (denoise) or "Cannot copy out of meta tensor; no data!" (l2i), because model loading relies on process-global, non-thread-safe monkey-patches. accelerate.init_empty_weights() (used directly by the loaders and implicitly by diffusers' default low_cpu_mem_usage=True in from_pretrained) swaps torch.nn.Module.register_parameter globally for the duration of a load, routing every newly-registered parameter to the meta device. The model cache's VRAM load/unload runs nn.Module.load_state_dict(assign=True), whose assign path does setattr -> __setattr__ -> register_parameter. When one worker's VRAM move overlapped another worker's from_pretrained, the move's real weights got hijacked onto meta and blew up on the next .to(device). Introduce MODEL_LOAD_LOCK, a write-preferring readers-writer lock: - write lock = model construction (_load_and_cache, load_model_from_path), exclusive. - read lock = VRAM load/unload (ModelCache.lock(), repair_required_tensors_on_device). VRAM transfers across GPUs still overlap each other; they only block while a construction holds the write lock. The lock is always acquired before any per-cache lock to keep a consistent order and avoid an AB-BA deadlock with the writer's make_room/put. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(backend): fix outpainting crash caused by model download collisions * fix(backend): make DiskImageFileStorage thread-safe for parallel sessions Image.open() is lazy: it reads the header but defers pixel decoding (and holds the file handle open) until the first .load()/.copy()/.convert(). The opened object was cached and the same object handed to every caller, so in multi-GPU parallel mode two session-processor worker threads could call .copy() on it concurrently and race on the shared file handle and decoder state. This surfaced as "broken data stream when reading image file" and "AssertionError: self.png is not None" during inpainting with batch >1. Force the decode (image.load()) before the object enters the cache so the cached object is safe for concurrent reads, and guard the cache structures (__cache / __cache_ids) with a lock since they are now mutated from multiple threads. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(ui): stack per-session progress bars during parallel generation The generation progress bars (under the Invoke button and the Viewer tab) both read a single global $lastProgressEvent atom, which every session overwrites. With parallel multi-GPU sessions this made the bar jump back and forth between sessions. Track progress per queue item id and render one bar per in-flight session, stacked vertically, each removed as its session reaches a terminal state. - stores.ts: add $progressEvents (map keyed by item_id), $activeProgressEvents (sorted), and set/clear helpers. - setEventListeners.tsx: populate per-item progress on invocation_progress; clear per item on terminal status; clear all on connect/disconnect/queue cleared. - ProgressBar.tsx: render a vertical stack of bars (one per active session) with a single-bar fallback for the idle / model-loading window; add containerProps so dockview tabs can position the stack. - Dockview tab call sites: move positioning into containerProps. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(ui): make $progressEvents module-local to satisfy knip $progressEvents is only referenced within stores.ts (via the $activeProgressEvents computed and the set/clear helpers), so exporting it tripped knip's unused-exports check. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(ui): cap stacked tab progress bars to fit below the tab label With 4 GPUs the stacked per-session progress bars grew past the bottom strip of the dockview tab and overlapped the "Viewer" label. Add a fitHeightPx prop: in fit mode the stack is capped to the available strip (10px below the ~40px tab's centered label) and the bars flex to share it, shrinking below their natural height only once they no longer fit. With 1-2 sessions the bars keep their familiar thin height; with 3+ they scale down to stay within the strip. The sidebar bar is unaffected and continues to stack at natural height (it has the vertical room). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(config): support "auto" generation_devices to use all GPUs by default generation_devices now accepts "auto" (the new default), which expands to every visible CUDA device — so multi-GPU parallel generation works out of the box without manually listing devices. On GPU-less systems "auto" resolves to the single cpu/mps device, preserving serial behavior. - config_default.py: type is now Union[Literal["auto"], list[str]], default "auto"; validator accepts "auto" or a list of device strings. - devices.py: add TorchDevice.get_generation_devices(), the single resolver that expands "auto", normalizes, and deduplicates. - session_processor / model_manager: both consumers use the resolver instead of iterating the raw config value (which would have iterated the characters of the "auto" string). - Regenerated docs/src/generated/settings.json. - Tests for the resolver (auto-with/without-CUDA, dedup, empty). An explicit single-device list (e.g. [cuda:0]) or an empty list opts out of parallelism. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(frontend): typegen+openapi * docs(multi-gpu): add configuration information * chore(frontend): typegen + openapi again * feat(settings): add Generation Devices selector to Settings dialog Add a badges UI in the Generation section of the Settings dialog for choosing which devices `generation_devices` should use, modeled on the Log Namespaces toggle UI. Backend: - New `GET /api/v1/app/generation_device_options` endpoint listing the selectable devices (cuda:N with GPU names, or the sole mps/cpu fallback). - Add `generation_devices` to the runtime-config update allowlist with validation rejecting invalid device strings and explicit nulls. Frontend: - New SettingsGenerationDevices component with active/inactive badges. "Auto (all GPUs)" is exclusive; removing the last explicit device reverts to auto. Admin/multiuser gated; notes restart requirement. - Wire into the Generation section; regenerate schema; add en strings. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(settings): boldface the restart notice on Generation Devices Split the restart sentence into its own string and render it bold so users notice that device changes require restarting InvokeAI. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(settings): show GPU name in Generation Devices badges Render device badges as "cuda:0 (RTX 3090 #1)" so identical cards can be told apart. Strips the "NVIDIA GeForce" vendor prefix and adds a 1-based "#N" suffix only when multiple cards share a name. The full device name remains available as the badge tooltip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(frontend): openapi * feat(multi-gpu): surface per-session GPU number in logs and UI Help users track which CUDA device is processing each session: - Model-load log: "Loaded model ... onto cuda device #N in ..s" - Denoise progress bars: "Denoising (#N)" across all architectures (SD1.5/SDXL, FLUX, FLUX2, Z-Image, Anima, SD3, CogView4) - Progress preview circle: GPU number centered in the ring, via a new `device` field on InvocationProgressEvent (resolved from the worker's thread-local session device) - Session Queue: new "GPU #" column between STATUS and TIME, backed by a `device` column on session_queue (migration_32) recorded when a worker claims an item Adds TorchDevice.get_session_device_label()/get_session_device_index() helpers and a frontend getCudaDeviceIndex() parser (with tests). Shows the number on CUDA only; CPU/MPS show nothing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(multi-gpu): show per-device names in startup log and progress circles - Startup log lists each generation device with its GPU number and id, e.g. "Using torch device: [AMD Radeon PRO W7900 #1 (cuda:0), ...]". Single-device setups keep the bare device name. - Canvas progress circles now show the CUDA device index in the center, matching the viewer panel. - Progress-circle tooltips show the device name and number on hover. - Both are hidden when only a single GPU is available. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(model-cache): share one CPU copy of model weights across per-GPU caches In multi-GPU mode the model manager builds one ModelCache per generation device, each with storage_device="cpu" and its own RAM-resident copy of every model. A model loaded on N GPUs therefore occupied N copies in RAM, and each cache sized itself against max_cache_ram_gb independently, so RAM use during the text/reference-image encoding phases skyrocketed and the system swapped — worst when two images rendered at once. This deduplicates the CPU-resident weights and makes RAM accounting global. - SharedCpuWeightsStore: process-/manager-global, refcounted store of one canonical CPU state_dict per model key. The first device to load a key registers its weights; subsequent devices adopt the canonical tensors and re-point their module's params at them (load_state_dict(assign=True)), freeing the duplicate. Weights live once in RAM regardless of GPU count; freed only when the last device releases. Per-device modules are kept (params are device-shuffled in place, so two GPUs need two modules), but their CPU-resident params alias the shared tensors. - RamBudget: single system-wide RAM authority. Splits RAM into shared (counted once via the store) and non-shared (per-instance). ModelCache eviction now runs against the global, deduplicated total and re-checks availability each iteration, since evicting a model another device still holds frees no RAM. build_model_manager wires one store + one budget into all device caches; the cap is max_cache_ram_gb as a true system-wide limit, else the sum of per-cache heuristics. Passing ram_budget=None preserves the prior local accounting. - LoRA/patch safety: direct LoRA patching did an in-place copy_ on the weight, which would corrupt the now-shared canonical tensor (and taint keep_ram_copy even with one GPU) when patching a CPU-resident weight. Switched to an out-of-place add (memory- equivalent) so the canonical tensor is never mutated; fixed the FluxControlLoRA expansion path to target the module's live parameter. Sidecar patching and FreeU/Seamless (which patch forward methods) were already safe. Validated on 2x AMD W7900 / ROCm: correct inference on both GPUs from one shared copy (full + partial load + Q8_0 GGUF quantized), concurrent load/unload without corruption, and LoRA isolation across devices. ~40 new tests; existing suites unchanged. Adds scripts/multigpu_ram_driver.py to drive concurrent dual-GPU generations via the queue API and measure peak RSS / leak drift. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(session-queue): cancel all in-progress items in bulk-cancel APIs (multi-GPU) With one session-processor worker per device, multiple queue items can be in_progress at once. cancel_by_batch_ids(), cancel_by_destination() and cancel_by_queue_id() excluded in_progress rows from their bulk UPDATE and then canceled only the single get_current() item (LIMIT 1), so on multi-GPU the other running items kept consuming a GPU and could still produce output after the user requested cancellation. Each running item must be canceled via _set_queue_item_status(), which emits the QueueItemStatusChangedEvent that the processor maps to the worker running that item_id and uses to set its cancel event. Add _cancel_in_progress_matching() to cancel every in-progress item matching the same filter (with user-id scoping preserved) and call it from all three bulk-cancel methods. The returned `canceled` count now includes canceled in-progress items. Adds regression tests that dequeue two items onto separate devices and assert every bulk cancel API moves all matching in_progress items to canceled and emits a cancel event for each (and that user-scoped cancel leaves another user's in-progress item running). Reported by JPPhoto in review of #9263. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(multi-gpu): address review findings (cancel race, bulk delete, device guards, refcount leak) Fixes from the code review of PR #9263: - Cancellation could be silently lost around dequeue: the per-iteration worker.cancel_event.clear() ran AFTER dequeue + gc.collect() + logging, so a cancel arriving in that window was set by the status handler and then wiped. Move the clear to before dequeue, and after claiming an item re-check (cancel_event + a fresh DB status read via _is_queue_item_terminal) and skip running if it is already terminal, closing both race windows. The runner's stale queue_item.status check could not catch this. - delete_by_destination only stopped one in-progress item (get_current) before deleting all matching rows, leaving other GPU workers running (and then failing to update a deleted row). Cancel every matching in-progress item via _cancel_in_progress_matching first. - generation_devices validation: a bare non-"auto" string (e.g. "cuda:0") was iterated character-by-character; an empty list silently fell back to one device. Reject both with a clear message. - get_generation_devices now fails fast on a CUDA device that does not exist (index past device_count, or CUDA unavailable) instead of starting a worker that errors cryptically at first allocation. - Shared-weights wrappers: if the canonical re-point (load_state_dict assign=True) threw after acquire(), the reference was leaked (the wrapper never entered the cache). Compute size metadata first, make acquire the last step, and release on failure. Adds tests for each: post-dequeue terminal guard, delete_by_destination cancellation, generation_devices validation, absent-device rejection, and acquire-released-on-repoint-failure. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): ruff format + make CPU-incompatible device test mock CUDA - Apply ruff 0.11.2 formatting to the files flagged by `ruff format --check`. - The new fail-fast guard in get_generation_devices() (reject a CUDA device that doesn't exist) made the pre-existing test_get_generation_devices_explicit_list_is_deduplicated fail on CPU-only CI runners, since it passes a cuda list with no CUDA present. Mock torch.cuda.is_available/device_count in that test (matching the existing pattern in this file) so it validates dedup on any runner. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(multi-gpu): stop RAM blowup/swapping during concurrent generations Three RAM fixes for multi-GPU (and one that helps single-GPU too), addressing transient spikes to ~100% RAM and swapping during text-encode/transformer loads: 1. Cap the global RAM-cache budget at a safe fraction of system RAM. When max_cache_ram_gb is unset, the budget was the *sum* of the per-device cache heuristics, so N GPUs each claiming ~50% of RAM summed to ~N*50% and starved the OS. Now clamp the sum to ModelCache.calc_system_ram_headroom_bytes() (50% of RAM - 2GB baseline, floored at 4GB). Promote the sizing magic numbers to named constants shared by the per-device heuristic and the global cap. 2. Adopt already-resident CPU weights across devices at load time. When a second device loads a model another device already holds, deep-copy a registered meta-weight structural clone and assign the shared canonical weights, instead of re-reading the model from disk and materializing a full transient second copy. Loader-agnostic (one mechanism in ModelLoader, no per-loader code): works for diffusers, single-file checkpoint, GGUF and transformers models, and preserves registered hooks (e.g. fp8 layerwise-cast). Best-effort with a meta-tensor self-check and fallback to a normal disk load on any failure. Skipped on single-device installs. 3. Dequantize FLUX.2 FP8 checkpoints straight to bf16. _dequantize_fp8_weights materialized the whole model in float32 (~36GB for 9B) before a later cast to bf16; now the multiply is done in float32 but stored bf16 per-weight, so the model is never held in float32. Numerically identical; halves the cold-load transient (helps single-GPU too). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(qwen-image): reserve VAE working memory so decode/encode don't OOM The Qwen Image VAE encode/decode invocations called model_on_device() without a working-memory estimate, unlike every other VAE family (SD/SDXL/SD3/CogView4/FLUX). So the model cache reserved only its small default working memory, never offloaded a large resident transformer (the VAE weights themselves are tiny), and the VAE's forward-pass activations then OOM'd VRAM — e.g. a ~40GB Qwen Image Edit transformer left ~1GB free while decode needed ~5GB. Reproduces single-GPU; unrelated to the multi-GPU RAM work. Add estimate_vae_working_memory_qwen_image() (same per-output-pixel scaling as the other estimators, handling the 5D Qwen latents) and pass it from both the i2l (encode, used for reference images in Image Edit) and l2i (decode) nodes, so the cache offloads the transformer before the VAE runs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(flux2): tile reference-image VAE encode to avoid VRAM OOM The FLUX.2 VAE encoder's mid-block self-attention scales quadratically with the input's spatial size, and on ROCm scaled_dot_product_attention falls back to a materialized attention matrix. Encoding a reference image (kontext) at full size therefore allocated ~15GB in a single attention call at 1024px — and hundreds of GB at the 2024px reference cap — OOMing VRAM regardless of how much other model memory was freed. Tile the reference-image encode to bound per-tile attention. The VAE's default tile size equals its sample_size (1024), whose per-tile attention still OOMs, so force a 512px tile (with a matching latent tile size derived from the config). Save/restore the VAE's tiling config since it is a shared, cached instance, so the final image decode does not inherit these settings. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(multi-gpu): query execution device for VRAM-in-use accounting ModelCache._get_vram_in_use() called torch.cuda.memory_allocated() with no device argument, while _get_vram_available() reads memory_allocated(execution_device). The formula relies on those two canceling. In multi-GPU mode each worker calls torch.cuda.set_device for its own GPU, so the process-current device flips between workers; the no-argument call can then read a different (e.g. idle) GPU's allocation, breaking the cancellation and inflating "available" VRAM toward the card total. The cache then believes there is room and never offloads, so VRAM offloading effectively ignores device_working_mem_gb in multi-GPU. Single-GPU was unaffected (current device always equals the execution device). Query self._execution_device in both _get_vram_in_use() and the cache-state debug log. Add a regression test asserting the per-cache execution device is used. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(qwen-image): calibrate VAE working-memory estimate to the 3D-conv decode peak The Qwen Image VAE is a 3D-conv (video) VAE whose decode allocates large conv3d feature maps. A ~1MP decode was measured to peak at ~17 GiB of VRAM — far above what the generic 2200/1100 SD/FLUX constants reserved (~4.6 GiB), so the cache concluded the decode "fit" alongside the resident 20GB transformer + 15GB text encoder, never offloaded them, and OOMed. The offload only frees ~(working_mem - free) bytes, so the reservation must both cover the real peak and be large enough to trigger the offload of models the decode doesn't need. Raise the Qwen decode/encode constants (13000/6500) to match the measured peak. It's linear in output pixels, so it over-reserves past ~1.5MP (where the decode can exceed the card even after offloading) — that case is covered by force_tiled_decode. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(qwen-image): honor force_tiled_decode in the l2i node The Qwen Image latents-to-image node hardcoded vae.disable_tiling(), ignoring the global force_tiled_decode setting that the SD/SDXL l2i node honors. Wire it up the same way so users can opt into tiled VAE decode for very large outputs that exceed VRAM even after the transformer/text encoder are offloaded. Off by default, so normal-size decodes are unchanged (full-frame, no tile blending). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(ui): stop progress disk flashing during indeterminate phases The preview-panel progress circle re-renders on every InvocationProgressEvent. The parent passes a fresh progressEvent object each event, so the CircularProgress re-rendered constantly; during the indeterminate phases (everything except denoising) that restarted its CSS spin animation each time, which looked like the disk flashing. (Determinate denoising was unaffected because the value genuinely changes per step.) Split the circle into a memoized, ref-forwarding subcomponent keyed on its visual props (isIndeterminate, value, device label) so message-only updates no longer re-render it and the spin animation stays continuous. The Tooltip still anchors to it via the forwarded ref. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(multi-gpu): offload text encoders to idle GPUs Adds `offload_text_encoders_to_idle_gpus` (default on): when more than one generation device is configured and a GPU is idle, a session's text/prompt encoder runs on the idle GPU instead of the one running its denoise pipeline. This avoids evicting the denoise model from VRAM to make room for the encoder, and lets a cached encoder be reused across generations. Under full load (no idle GPU) behavior is unchanged. Mechanism: - New GENERATION_DEVICE_POOL arbiter (backend/util/device_pool.py) with a per-device exclusive-use lock. A native session blocking-acquires its own device's lock for the whole run; an encoder node try-borrows an idle device's lock for the duration of the node. This makes a borrowed encoder and a native session mutually exclusive on a GPU -- preventing the shared-encoder corruption that produced garbled images -- and is deadlock-free (borrows are non-blocking; a session only ever blocks on its own device). - DefaultSessionRunner re-pins the worker thread to the borrowed device for the whole encoder node; conditioning is stored on the CPU and the denoiser picks it up on its own GPU afterward. - Nodes opt in via @invocation(idle_gpu_offloadable=True), mirroring the existing `bottleneck` ClassVar marker. Applied to the text/prompt encoder nodes (compel + sdxl/refiner, flux, sd3, qwen-image, anima, cogview4, flux2 klein, z-image, flux_redux). Inspired by #9310; supersedes it. Tests: device-pool lock semantics, two concurrency regression tests asserting a session and a borrow never use a GPU at the same time, the runner offload context-manager behavior, and a marker-wiring check. Docs: invokeai-yaml.mdx (config setting) and creating-nodes.mdx (how to support the feature in a node). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(multi-gpu): adopt GGUF weights across devices to stop RAM spikes _build_meta_shell built meta placeholders with torch.empty_like, which GGMLTensor.__torch_dispatch__ rejects (NotImplemented for aten.empty_like). It threw on the first parameter, hit the silent except, and returned None — so GGUF models (e.g. a Q8_0 transformer) never registered a shell and the second GPU re-loaded the full model from disk, stacking a ~20GB transient on the retained copy and spiking RAM to ~70%. Fall back to a plain meta placeholder (logical shape/dtype) when empty_like isn't implemented by a tensor subclass; verified the adopted GGMLTensor shares the quantized storage, so it's one RAM copy across devices. Peak drops ~66→~46GB. Log shell-build failures at debug so a future un-adoptable family is diagnosable instead of silently double-loading. Also restore log_memory_usage's per-cold-load RAM logging (the capture method had no callers), slimmed to baseline→transient-peak process RAM. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(multi-gpu): tie device #N label to cuda index, not filtered position The backend device summary computed the disambiguating #N suffix by enumerating the filtered generation_devices list, so disabling a device (e.g. cuda:1) renumbered the survivors. The frontend labels over the full device set, so the two disagreed. Compute the suffix over all available devices instead, keeping the label stable and consistent with the frontend. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(multi-gpu): flash restart reminder when generation devices change Reword the Generation Devices caption to "Restart InvokeAI for changes to take effect." and flash that same warning as a toast on every successful change, so the restart requirement is hard to miss. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(queue): device-affinity dequeue to reduce model reload thrash on multi-GPU When a GPU worker dequeues, prefer — among the fairness-chosen user's equal-priority pending items — one whose models are already resident in that device's cache. Cross-device model reloads cost tens of seconds for large models; picking a warm item instead cuts thrash when a user queues a mix of models. Guardrails (from adversarial review): - Round-robin user choice and priority tiers are never overridden; the swap pool is limited to the candidate's user and priority. - The swap window is capped at AFFINITY_MAX_LOOKAHEAD past the candidate's item_id, bounding both cold-item deferral and per-dequeue scan cost. - Explicitly configured session_queue_mode=FIFO opts out of reordering. - Resident keys are snapshotted before the dequeue lock, and ModelCache.cached_model_keys() acquires its lock non-blockingly, so a long-running VRAM transfer can never stall other workers' dequeues. - Path-keyed cache entries (load_model_from_path) are excluded so a Windows drive letter can't poison substring scoring. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(qwen): restore legacy key remapping for single-file VL encoders under transformers 5.x The single-file Qwen2.5-VL encoder loader relied on Qwen2_5_VLForConditionalGeneration._checkpoint_conversion_mapping to translate ComfyUI's legacy key layout (visual.*, model.layers.*) to the modern one (model.visual.*, model.language_model.*). transformers 5.x ships that mapping empty — the conversion moved into from_pretrained's weight-converter machinery, which our manual load_state_dict path bypasses — so the vision tower was left on the meta device and loading failed with "Meta tensors remain". Fall back to the equivalent hardcoded mapping when the class attribute is empty or absent. Verified against qwen_2.5_vl_7b_fp8_scaled.safetensors: loads all 8.29B params with no meta tensors remaining. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address multi-GPU review findings from PR #9263 review - Shared CPU weights: drop_model() now invalidates the model's canonical entries in SharedCpuWeightsStore, so a rebuild on another device can never adopt pre-settings-change weights still aliased by a locked (stale-marked) entry. release() is identity-checked so a stale holder's eviction cannot decrement a newly registered canonical. update_model_record holds MODEL_LOAD_LOCK.write_lock() (off the event loop) across the multi-cache drop to exclude in-flight loads. - Runtime config API: generation_devices is now fully validated at the route boundary — empty lists and unavailable devices (e.g. cuda:99) return 422 without mutating or persisting config, using the same TorchDevice resolution as startup. - Cache stats: /v2/models/stats aggregates per-device caches instead of reporting only the API thread's default cache. - Config/docs contract: session_queue_mode description now documents device-affinity reordering in single-user multi-GPU mode (and that explicit FIFO disables it), and that user rotation outranks priority across users in round_robin mode. Multi-GPU docs no longer claim generation_devices: [] is valid, and describe shared-RAM weight deduplication instead of per-GPU duplication. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address multi-GPU review findings (RAM accounting, stats aggregation, MPS validation) - SharedCpuWeightsStore.invalidate() now retires still-referenced entries instead of dropping them from accounting, so RamBudget keeps counting retired weights until the last locked holder releases them. Prevents admitting models past max_cache_ram_gb while a replacement and a stale copy are both resident. - /models/stats aggregation takes max of cache_size and high_watermark across per-device caches (they share one global RamBudget, so summing over-reported an N-GPU system ~N times); event counters are still summed. - TorchDevice.get_generation_devices() rejects 'mps' when MPS is unavailable, so the runtime_config API 422s instead of persisting a device that fails at first tensor op. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(frontend): lint:prettier * fix: address JPPhoto's 2026-07-21 review (12 items) Backend: - layer_patcher: hold MODEL_LOAD_LOCK.read_lock() across patch application so FLUX Control LoRA shape expansion (register_parameter) cannot overlap a concurrent model construction's process-global init_empty_weights patch - flux_redux/flux_denoise: store Redux conditioning on CPU (it may be produced on a borrowed idle GPU) and assign the .to() result when consuming it - model_cache/ram_budget: coordinate eviction across device caches — when a cache's own stack is exhausted and the global budget is still short, peers evict their unlocked entries (non-blocking lock, deadlock-free), so max_cache_ram_gb holds even when RAM is retained only by an idle device - session_queue: 'except current' operations protect the workflow-call chain of EVERY in-progress item, not one arbitrary get_current() row - session_queue: _cancel_in_progress_matching tolerates rows deleted by a concurrent clear between its id SELECT and the per-item cancel - session_processor: the post-dequeue cancel guard cancels the freshly claimed item when skipping it (a stale cancel_event must not abandon it in_progress) - session_processor: _clone_session_runner refuses to downgrade DefaultSessionRunner subclasses or share custom runners across workers - session_processor: an offloaded encoder's cache activity is attributed to the running session's CacheStats (borrowed cache's stale stats pointer swapped for the borrow duration) - events: progress events report the queue item's persisted device, not the thread-local (temporarily borrowed) one - devices/config docs: generation_devices 'auto' defers to an explicitly pinned legacy 'device:' setting so upgrades don't start workers on every GPU Frontend: - ImageViewer context: a terminal status only clears the shared progress event/image globals when that item owns them (multi-GPU: canceling item A no longer blanks item B's live preview) - SettingsGenerationDevices: device tags are keyboard-operable (tabIndex + Enter/Space activation) Each fix has an exposure test per the review's suggestions. * chore: regenerate openapi.json (auth on get_generation_device_options) * fix(backend): avoid MODEL_LOAD_LOCK self-deadlock when patching a LoRA on a cold cache apply_smart_model_patches() held MODEL_LOAD_LOCK.read_lock() across its patch loop, but callers pass a lazy generator (e.g. flux_text_encoder._t5_lora_iterator) that constructs each LoRA via context.models.load() on demand. A cold-cache load takes MODEL_LOAD_LOCK.write_lock(); since the lock is non-reentrant and write-preferring, acquiring the write lock while this same thread already holds the read lock deadlocks (write waits for readers==0, but the consuming thread is that reader). The generation hung silently right after the encoder/tokenizer load, whenever a LoRA was applied and not already cached. Materialize the patch iterable before taking the read lock so every LoRA construction takes (and releases) the write lock first; the read lock then covers patch application only, which is its actual purpose (FLUX Control LoRA shape expansion calls register_parameter and must exclude concurrent construction). Compatible with wan_denoise's per-call iterator factory, and unrelated to the SD UNet path, which loads the LoRA before calling the singular patcher (no lock held). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: address JPPhoto's four merge blockers from the 2026-07-22 review 1. model_cache: a peer whose lock is contended during cross-cache eviction no longer leaves the shared RAM budget exceeded indefinitely. evict_unlocked_for_peer returns None on contention; the requester records a reconcile request on each skipped peer, and the synchronized-decorator hook honors it as soon as the peer's current operation releases the lock (outermost frame only — the RLock may be held reentrantly). The pending flag stays set until the budget is actually satisfied, so overshoot held by locked entries reconciles when their unlock releases the lock. 2. session_processor: a stale cancellation event from the previous item no longer cancels the freshly claimed, unrelated item. The post-dequeue guard now treats the DB status as the authority: a terminal row is skipped; a set cancel_event with a non-terminal row is a stale signal (a genuine cancel writes the row terminal BEFORE emitting) and is cleared, with a post-clear terminal re-check closing the clear's own race window. A shutdown-raced claim is still canceled so it isn't abandoned in_progress. 3. flux2_klein_text_encoder: conditioning is detached and moved to CPU before context.conditioning.save(), matching flux_text_encoder and flux_redux — the node is idle_gpu_offloadable, and GPU-resident embeddings would pin VRAM on a borrowed device after its pool lock is released. 4. session_queue clear: user-scoped clearing no longer assumes one current item. clear() cancels every in-progress item in scope via _cancel_in_progress_matching (each item's own status-changed event signals the worker running exactly that item) before deleting rows — same pattern as delete_by_destination; the router's arbitrary get_current() check (which could 403 the owner or cancel another user's item) is removed; and _on_queue_cleared honors the event's user_id so a scoped clear cannot stop other users' workers and abandon their rows. Each fix carries the regression test JPPhoto specified: contended-peer budget reconcile, stale-event-runs-item (plus the mid-clear race and shutdown cases), CPU-backed Klein conditioning, and Alice/Bob concurrent clear isolation at both the service and the event-handler layer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: regenerate OpenAPI schema for the clear endpoint docstring The merge-blocker fix 68edb02127 reworded the clear route's docstring, which is the OpenAPI operation description — openapi.json and schema.ts must follow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(model cache): close lost-wakeup race in deferred RAM-budget reconcile The deferred reconcile request was recorded pre-admission and honored only by the peer's next lock release. Two interleavings could strand the shared RAM budget above its cap indefinitely: - Lost wakeup: the busy peer releases its lock (running its reconcile hook while the flag is still unset) before request_budget_reconcile() sets the flag; if the peer then stays idle, no future release honors the request. - Pre-admission clearing: a peer's reconcile could run between the request and the new model being counted, see the budget as satisfied, and clear the flag before the admission pushed usage over the cap. Fix both by (1) moving the reconcile request to the end of put(), after the new model is counted, so peers always evaluate the true budget state, and (2) having request_budget_reconcile() attempt the reconcile inline with a non-blocking lock acquire: either the peer's lock is free now and the reconcile runs immediately, or it is still held and the eventual release hook — which runs strictly after the flag is set — performs it. The prior regression test masked the race by touching cache_b.stats after the request; it now emulates the production release hook in the holder thread and asserts reconciliation with no subsequent cache access, and a new test forces the lost-wakeup interleaving by delaying the request until the peer's operation has fully finished. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(model cache): close remaining RAM-budget reconcile gaps Addresses the three lingering issues from review of the deferred budget-reconcile mechanism: 1. Manual lock releases bypass the reconcile hook. cached_model_keys() and evict_unlocked_for_peer() acquire/release _lock without the synchronized decorator, so a reconcile request whose inline attempt failed on their held lock was stranded when they released. Both now run the same reconcile hook after their manual release (non-blocking, preserving cached_model_keys' no-stall guarantee and avoiding the hold-A-block-on-B deadlock shape in evict_unlocked_for_peer). 2. clear() can wipe a concurrent request. A reconciler observing a satisfied budget could clear the pending flag just after a peer's admission (already counted, budget negative) set it, and the peer's inline attempt then saw the flag unset and returned — leaving the budget exceeded with no pending request. The reconcile now runs as a loop with a single guarded clear site: because admissions are counted before the flag is set, a negative budget re-check immediately after the clear proves a request may have been wiped; the flag is restored and reconciliation continues. This covers both former clear sites (satisfied early-out and post-eviction). 3. No reconcile trigger when the admitting cache itself holds the overshoot. put() requests reconciles from peers only, so when the exceeded budget was held by the admitting cache's own locked entry, no pending request existed anywhere and the eventual unlock ran its hook with the flag unset. unlock() now records a reconcile request on its own cache whenever it completes with the shared budget exceeded, so the entry that just became evictable triggers the reconcile. Supporting change: put() admitting a model while a peer's reconcile request is already pending must not let its own release hook evict the just-admitted entry before the loader's immediately-following get() (that would break the in-flight load with an IndexError). CacheRecord gains an awaiting_first_use grace flag, set on admission and cleared on first get()/lock(), which the asynchronous eviction paths (budget reconcile, peer-requested eviction) skip. The local make_room path ignores it: cold loads are serialized under MODEL_LOAD_LOCK, so it can never see another loader's entry inside the put()->get() window, and this bounds the flag's lifetime if a load errors out in between. Each new regression test was verified to fail against the previous implementation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(model cache): bound the admission grace and keep cached_model_keys stall-free Addresses the three issues from JPPhoto's 2026-07-27 review: 1. Prefetched submodels can no longer shield the budget forever. The SD single-file loader's proactive submodel put()s are now admitted with prefetch=True (no post-admission grace), since nothing ever get()s or lock()s them. As a backstop, put() sweeps stale grace flags from prior loads — cold loads are serialized under MODEL_LOAD_LOCK, so any flag still standing at the next admission belongs to a dead load (errored before get(), or LoadedModel dropped before lock()) and is cleared. 2. The grace now survives get() and ends at lock(). get() is synchronized, so clearing the flag inside it let get()'s own release hook run a pending reconcile and evict the very record it had just selected — detaching a live model from the cache and its RAM accounting before the caller could lock it. load_default also retrieves immediately after put() so no failure in between can orphan a graced record. 3. cached_model_keys()'s manual-release hook hands a pending reconcile to a short-lived background thread instead of running it inline: reconciliation evicts models and calls gc.collect(), which would break the method's no-stall contract and pause session dequeue. Each new regression test verified to fail with its mechanism reverted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(model cache): release abandoned admission grace * fix(model cache): keep grace release off the collecting thread release_first_use_grace() is invoked from a weakref.finalize callback, so it runs at an arbitrary decref/garbage-collection point in an arbitrary thread. Making it @synchronized therefore made ModelCache._lock — and, through the decorator's release hook, a full budget reconcile — reachable from anywhere. That inverts the lock order RamBudget documents as impossible. The hook's _reconcile_budget_if_pending reads RamBudget.available() -> SharedCpuWeightsStore.total_bytes_in_use(), both plain non-reentrant locks. A thread inside SharedCpuWeightsStore.acquire() holds the store lock while summing tensor sizes, an allocation loop that trips generational GC; if that collection reclaims an abandoned wrapper belonging to another device's cache, the release hook re-enters the store lock the thread is already holding and the thread deadlocks against itself, still holding it. Every other cache then blocks on its next _delete_cache_entry -> release_shared_weights(). Reproduced on a two-cache budget: the collecting thread wedges in total_bytes_in_use() and never returns. The same hook also ran evictions, gc.collect() and empty_cache() inline in whatever unrelated thread happened to drop the reference — including the API event loop — undoing the no-stall contract cached_model_keys() was just given. Do no locking work in the callback: hand the release to a short-lived background thread, exactly as cached_model_keys() does with its own pending reconcile. The thread may wait on the cache lock and do the slow work; the collecting thread returns immediately. The existing abandoned-wrapper test now polls for the (asynchronous) release. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(model cache): harden the deferred grace release Follow-ups from an adversarial review of 55a6fc496d: - Thread.start() can raise RuntimeError under thread/process limits. A weakref.finalize callback gets no retry (weakref retires it before invoking it) and its exceptions go to sys.unraisablehook, so the release was silently lost and the record kept shielding an idle cache. Fall back to clearing the flag inline under a non-blocking acquire, which takes no store or budget lock and so still cannot deadlock the collecting thread. No reconcile on that path by design: a pending request stays set for the next cache operation. - The regression test's outcome was a pure function of the ambient allocation count: nothing pinned the cycle between its creation and the collector thread, so an automatic gen-0 pass landing in the setup reclaimed it on the main thread and the test passed vacuously (or tripped its own setup assertions). Under an allocation-shifting plugin it failed at 6 of 12 offsets. Disable automatic gc across the setup so only the explicit collect reclaims the cycle; the same sweep is now 12 of 12 passing, and the test still fails against 7ffee4db04 with the expected re-entrancy report. - Correct the docstring: Thread.start() waits for the child to bootstrap, so the guarantee is "no lock waits", not "returns immediately". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(model cache): queue deferred cache work * fix(model cache): stop the deferred worker pinning records and dying silently Follow-ups from an adversarial review of c384ea187d, which replaced the per-release background threads with one long-lived worker per cache. - The worker's `work` local stays bound while it blocks in the next get(), so the last-processed CacheRecord — and transitively its model's CPU weights — was pinned until some unrelated item happened to be queued behind it. That is worse than an ordinary leak: _release_first_use_grace's release hook can evict that very record, removing it from the cache AND subtracting its bytes from the RamBudget, so the budget under-reported a model that was still resident and the next admission over-committed. Reproduced on a two-cache budget: after eviction plus an explicit gc.collect(), both the record and its module were still alive; queueing one more item freed them. The per-call threads this replaced did not have the bug — Thread._bootstrap_inner deletes _args on exit. Clear the reference in a finally before looping back. - `if self._deferred_work_thread.ident is None` is a "was it ever started" check, not a liveness check: ident is never cleared and a Thread cannot be restarted. A worker lost to an unexpected error (a logging handler that raises, os.fork(), or shutdown() before the first put(), which leaves its _DEFERRED_STOP queued for the thread that put() then starts) was gone for the life of the process, silently disabling every later grace release and budget reconcile — the failure this mechanism exists to prevent. Create a fresh thread whenever the previous one has exited, and never after shutdown(). - Only the worker drains the queue, but both dispatch sites enqueued unconditionally. cached_model_keys() runs on every dequeue (session_queue_sqlite._get_device_resident_model_keys), so an idle-device cache that never admitted a model — and so has no worker — accumulated one queued reconcile per dequeue forever; post-shutdown the same held for both sites, stranding CacheRecords in a queue nothing would drain. Route both through _dispatch_deferred, which drops the item when no worker is running. Dropping loses nothing: such a cache has nothing to evict, and put() re-runs a pending reconcile through the synchronized release hook when it admits one. - shutdown()'s early return meant a keep-alive timer re-armed by a post-shutdown put() was never cancelled by a later shutdown(). Don't arm timers on a shut-down cache. Tests: the two *_thread_start_failure_* tests installed their monkeypatch after put() had already started the worker, and c384ea187d removed the only Thread.start() from those paths — the patched raise was unreachable, so both passed without exercising their premise. Retargeted to what they actually verify (the finalizer and the lookup must not block, and the reconcile still happens). Four regression tests added; each fails against c384ea187d. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(model cache): stop the deferred worker outliving and pinning its cache Second round of adversarial-review follow-ups on the deferred-work thread. - The worker held a bound method, so a running thread — reachable from threading._active — kept its ModelCache alive, and with it every CacheRecord and every model's CPU weights. A cache released without shutdown() was therefore immortal, which is the opposite of what RamBudget's weakref registry is built for. Measured against the parent commit 7ffee4db04: five caches dropped without shutdown() left 5/5 caches and 5/5 models resident and five worker threads running, where the parent left 0/5 and no threads. The previous `ident is None` guard had accidentally bounded this (a cache whose worker died could never re-acquire a pinning thread); reviving the worker removed that bound, so the fix has to remove the strong reference itself. The worker is now a module-level function taking a weakref, and a weakref.finalize pushes _DEFERRED_STOP when the cache is collected so the parked thread exits instead of leaking one thread per abandoned cache. - Thread.start() is called from put(), which runs under both the cache lock and MODEL_LOAD_LOCK's write lock while completing a load. Under thread/pid exhaustion (RLIMIT_NPROC, a container's pids.max) its RuntimeError escaped and failed a generation whose model had already been fully constructed — to lose an optimization that _dispatch_deferred is explicitly designed to survive the absence of. Log and carry on; the next admission retries. - _dispatch_deferred justified dropping work with "a cache without a worker has never admitted a model". That was false: put() after shutdown() is reachable in production, because Invoker.stop() stops model_manager before session_processor, so an in-flight generation can admit a model after every cache has been shut down — and thread exhaustion reaches the same state without a teardown to bound it. Such a record was admitted with the first-use grace, then permanently shielded from both asynchronous eviction paths with its bytes still charged to the shared budget. Make the claim true instead of rewording it: put() grants the grace only when a worker is running to release it. lock() still clears it on the normal path, so nothing changes when the worker is healthy. Tests: the post-shutdown half of the drop test never set the pending flag, so cached_model_keys() short-circuited before reaching the dispatch guard and the assertion held unconditionally. Poll the budget in the pin test rather than reading it the instant the key disappears (_delete_cache_entry pops before it releases the weights). Two regression tests added; all seven of this series' new tests fail against c384ea187d. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(model cache): release abandoned RAM budget * fix(model cache): pin LoRA patches during use * fix(model cache): harden the LoRA pin path and close its review gaps Review fixes for the LoRA-pinning commit (1142430f87): - Fix the CI failure it shipped: test_krea2_text_encoder's fake LoadedModel lacked model_in_ram(), and the encoder's LoRA iterator now calls it. The fake now models the pin (with depth tracking), and the test asserts the patch spec carries a working pin. - Stop a keep-alive Timer.start() failure from leaking a permanent pin. @record_activity runs after lock_in_ram() has incremented the lock count but before model_in_ram()'s unlock-pairing try block is entered, so a RuntimeError under thread/pid exhaustion would pin the record (and its shared-budget bytes) for the life of the process. The timer is an optimization: log and continue instead. - Give lock_in_ram() the same already-dropped-record diagnostic as lock()/unlock(), so a pin on a detached record produces a matching lock-side message (issue 7513). - Pin the LoRA cache record in LoRAExt.patch_unet (the modular-denoise path) while its tensors are read during direct patching. This was the one remaining producer that dropped its LoadedModel handle at load time, leaving the record evictable by a peer cache mid-patch. - Close test vacuities: the pin-retention test in test_layer_patcher could not detect a dropped cache_pins.close() (the ExitStack would be collected silently), and the cache-side pin test only covered a warm record, leaving lock_in_ram's grace-clearing dead code under test. Added pin-release assertions for the normal, body-raise, restore-raise, and mid-materialization-raise paths (all verified to fail with close() neutered), a cold-record grace/finalizer test, and a Timer-failure regression test (verified to fail pre-fix). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(model cache): close the remaining raise-after-lock leak and widen the LoRAExt pin Self-review fixes for d63c1f37e0: - The synchronized-decorator's post-release reconcile hook runs inside the caller's frame after the method body, so a raise there (e.g. TorchDevice.empty_cache on a sick CUDA context after an eviction) escaped lock_in_ram()/lock() after the lock count was incremented — the same permanent-pin leak as the Timer.start() case, via a different path. The reconcile is deferrable housekeeping: swallow and log; the pending flag is only cleared once the budget is satisfied, so the next lock release retries. Regression test verified to fail pre-fix. - LoRAExt.patch_unet's pin now spans the yielded scope, not just the patch application: despite force_direct_patching=True, fp8-storage modules are routed to sidecar patching (float8 weights cannot be patched in place), which stores a live reference to the cached patch's layers inside the UNet for the whole denoise. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Jonathan <34005131+JPPhoto@users.noreply.github.com> Co-authored-by: JPPhoto <jpollack@jpollackphoto.com> | 1 个月前 | |
feat(model_manager): Add scan and delete of orphaned models (#8826) * Add script and UI to remove orphaned model files - This commit adds command-line and Web GUI functionality for identifying and optionally removing models in the models directory that are not referenced in the database. Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Add backend service and API routes for orphaned models sync Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Add expandable file list to orphaned models dialog Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Fix cache invalidation after deleting orphaned models Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * (bugfix) improve status messages * docs(backend): add info on the orphaned model detection/removal feature * Update docs/features/orphaned_model_removal.md --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Co-authored-by: dunkeroni <dunkeroni@gmail.com> | 6 个月前 | |
build: remove installer & convert installer build script to only build the wheel | 1 年前 | |
feat: Video generation (#9163) * feat(model): add Wan 2.2 image generation support (Phases 0-2) Foundation + TI2V-5B MVP + A14B dual-expert MoE for Wan 2.2 image generation. Wan was trained on video but is competitive with leading open-source image models when run at num_frames=1; this commit wires that path into InvokeAI. Phase 0 — Foundation: - BaseModelType.Wan + WanVariantType {T2V_A14B, TI2V_5B} - SubModelType.Transformer2 for the dual-expert MoE - MainModelDefaultSettings per variant - step_callback Wan branch (16-channel preview; 48-channel TI2V-5B falls back to slicing first 16 channels until proper factors land) - Frontend enums + node colour Phase 1 — TI2V-5B Diffusers MVP: - Main_Diffusers_Wan_Config probe (variant from transformer_2/ + vae/config.json::z_dim, with filename heuristic fallback) - WanDiffusersModel loader (subclasses GenericDiffusersLoader) - WanT5EncoderField, WanTransformerField (with dual-expert slots), WanConditioningField, WanConditioningInfo - New invocations: wan_model_loader, wan_text_encoder, wan_denoise, wan_image_to_latents, wan_latents_to_image - FlowMatchEulerDiscreteScheduler integration with on-disk config load - RectifiedFlowInpaintExtension reused for inpaint - 5D <-> 4D shape juggling: latents stay 4D in InvokeAI's pipeline, re-add T=1 only inside the transformer call / VAE encode-decode Phase 2 — A14B dual-expert MoE: - Probe reads boundary_ratio from model_index.json - Loader emits both transformer (high-noise) and transformer_low_noise (low-noise expert at transformer_2/) for A14B - _ExpertSwapper in wan_denoise drives GPU residency between experts: high-noise for t >= boundary_ratio * num_train_timesteps, low-noise below. Only one expert locked at a time so the cache can evict the other - relies on existing CachedModelWithPartialLoad to handle oversized models on lower-VRAM GPUs. - guidance_scale_low_noise field for separate low-noise CFG override Tests: - 24 passing tests covering probe variant detection, default settings, noise sampling, end-to-end denoise on a synthetic transformer (CPU), dual-expert boundary swap, CFG branch - 1 heavy-test placeholder gated by INVOKEAI_HEAVY_TESTS=1 for the real-weights smoke test Phase 3+ deferred: standalone VAE/encoder configs, GGUF, LoRA, ControlNet, ref image, inpaint UI, frontend wiring, starter models. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(model): Wan 2.2 Phase 3 + tokenizer-load fix Phase 3 adds standalone VAE and UMT5-XXL encoder configs so users can run GGUF-quantized Wan transformers (Phase 4) without installing the full ~30 GB Diffusers pipeline. VAE configs: - VAE_Checkpoint_Wan_Config + VAE_Diffusers_Wan_Config (16-channel A14B vs 48-channel TI2V-5B, distinguished by decoder.conv_in z_dim). - 16-channel files share the AutoencoderKLWan architecture with Qwen Image; disambiguated via filename heuristic ("wan" in name -> Wan, otherwise -> Qwen Image). Mirror exclusion in QwenImage's probe. - VAELoader gets a Wan branch that builds AutoencoderKLWan(z_dim=...) via init_empty_weights, mirroring the QwenImage single-file pattern. - Existing standard VAE probe excludes both QwenImage- and Wan-style state dicts. UMT5-XXL encoder: - New ModelType.WanT5Encoder + ModelFormat.WanT5Encoder. - WanT5Encoder_WanT5Encoder_Config probes the diffusers folder layout (text_encoder/config.json with model_type=umt5, or flat layout with config.json at root). Refuses full Wan pipelines. - WanT5EncoderLoader handles both layouts and loads UMT5EncoderModel + AutoTokenizer. Component-source plumbing: - WanModelLoaderInvocation now exposes wan_t5_encoder_model and component_source pickers (mirrors QwenImage pattern). Resolution order: standalone > main (if Diffusers) > component_source. Required when the main model is a single-file format in Phase 4. Bug fix in wan_text_encoder: - Tokenizer was loading via AutoTokenizer.from_pretrained(<root>) directly, which fails for nested layouts where files live in <root>/tokenizer/. Now routed through the model cache so the registered loaders handle layout differences correctly. Frontend: - New type guards (isWanVAEModelConfig, isWanT5EncoderModelConfig, isWanMainModelConfig, isWanDiffusersMainModelConfig) and hooks/ selectors (useWanVAEModels, useWanT5EncoderModels, useWanDiffusersModels). New zSubModelType / zModelType / zModelFormat enum entries for transformer_2 and wan_t5_encoder. Tests: - 16 new tests covering z_dim detection, VAE checkpoint/diffusers probes, the bidirectional Qwen-vs-Wan filename deferral, and the UMT5 encoder probe (nested + flat + T5 + full-pipeline rejection). - Total Wan test count: 41 passing, 1 heavy-test placeholder skipped. - Full config test suite (63 tests) still passes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): unbreak frontend lint after Wan additions Five issues turned up running `make frontend-lint`: 1. wan_denoise.py used `from __future__ import annotations`, which made the `invoke()` return annotation a string ('LatentsOutput'). The InvocationRegistry's `get_output_annotation()` returns the raw annotation, so OpenAPI generation crashed with `'str' object has no attribute '__name__'`. Removed the future-import and added `Any` to the typing imports. 2. ModelRecordChanges.variant didn't list WanVariantType, so the generated schema's install/update endpoints rejected `t2v_a14b` and `ti2v_5b`. Added it. 3. Regenerated frontend/web/src/services/api/schema.ts from the live backend so it now includes BaseModelType.wan, ModelType.wan_t5_encoder, SubModelType.transformer_2, ModelFormat.wan_t5_encoder, the Wan variants, all Wan invocation types and their conditioning/transformer field types. 4. modelManagerV2/models.ts: added `wan_t5_encoder` to the category map, `wan` to the base color/long-name/short-name maps, the two Wan variants to the variant-name map, and `wan_t5_encoder` to the format-name map. 5. ModelManagerPanel/ModelFormatBadge.tsx: added `wan_t5_encoder` to FORMAT_NAME_MAP and FORMAT_COLOR_MAP. `make frontend-lint` now passes cleanly (tsc, dpdm, eslint, prettier). All 41 Wan Python tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> chore(wan): drop unused FE exports flagged by knip These were forward-compatibility wiring for Phase 9 (the FE graph builder) that has no consumers yet; knip rightly flagged them. Removed or de-exported. They'll come back when the graph builder lands and needs them. - common.ts: zWanVariantType drops `export` (still used internally by zAnyModelVariant). - types.ts: drop isWanMainModelConfig, isWanDiffusersMainModelConfig, isWanVAEModelConfig (no callers). The remaining isWanT5EncoderModelConfig is used by models.ts. WanT5EncoderModelConfig type drops `export` (still used as the type guard's narrowing target). - modelsByType.ts: drop the six unused useWan*/selectWan* hooks + selectors and their type-guard imports. `make frontend-lint` (tsc + dpdm + eslint + prettier + knip) now green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> docs(wan): use *-Diffusers HF repo names in plan The Wan-AI org publishes two flavours of each release: * Wan-AI/Wan2.2-{TI2V-5B,T2V-A14B,I2V-A14B} ← upstream native * Wan-AI/Wan2.2-{TI2V-5B,T2V-A14B,I2V-A14B}-Diffusers ← convertible The native release has _class_name=WanModel in config.json and ships weights flat at the repo root with no transformer/, vae/, text_encoder/ subdirs. It is not loadable by Diffusers' WanPipeline.from_pretrained. Update plan doc to reference the -Diffusers repos throughout (probe notes, starter-model entries) so the plumbing path matches what the Diffusers loader actually expects. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): accept 0 as 'unset' sentinel for guidance_scale_low_noise The frontend renders Optional[float] inputs with default 0 in the numeric input rather than passing null/unset. Combined with ge=1.0, this caused every wan_denoise invocation to fail Pydantic validation with "Input should be greater than or equal to 1" until the user manually entered a value (or knew to leave the field disconnected). The validation error was rejected before invocation logging, so it never showed up in the server log either - making the failure hard to diagnose. Relaxing the constraint to ge=0.0 and treating values below 1.0 as the "fall back to primary Guidance Scale" sentinel. The user's natural FE default (0) now works as expected. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): correct preview dimensions and colors for TI2V-5B Two bugs in the Wan branch of the diffusion step callback: 1. Wrong dimensions. The reported preview size hardcoded `* 8` for the spatial downscale ratio, but TI2V-5B's Wan2.2-VAE uses 16x. A 1024x1024 target was being announced to the FE as 512x512. 2. Wrong colors. The previous fallback for 48-channel TI2V-5B latents sliced the first 16 channels and applied the standard 16-channel Wan-VAE projection. Those channel layouts are unrelated, so the projection produced meaningless colors. Adding the proper Wan2.2-VAE 48-channel RGB projection matrix (and bias) from ComfyUI's Wan22 latent format, and selecting the right matrix + spatial-scale by latent channel count: 16 → A14B (Wan VAE, 8x), 48 → TI2V-5B (Wan2.2-VAE, 16x). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): honor model's _class_name when building scheduler TI2V-5B's scheduler_config.json declares _class_name=UniPCMultistepScheduler with flow_shift=5.0. The previous code hardcoded FlowMatchEulerDiscreteScheduler.from_pretrained(...), which silently constructed a default-config FlowMatch instead of the UniPC the model expects. The mismatched noise schedule manifests as soft / under-denoised faces and global graininess in the final images. Now: read scheduler_config.json, look up the named class on the diffusers module, and instantiate that class via from_pretrained. UniPC and FlowMatch share the same step()/set_timesteps()/sigmas/num_train_timesteps interfaces, so the denoise loop works transparently for either. A14B continues to use FlowMatchEulerDiscreteScheduler when its scheduler config says so (its reference is FlowMatchEuler with shift=8.0). Falls back to FlowMatchEulerDiscreteScheduler defaults when no on-disk config is available. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): match diffusers WanPipeline tokenizer length and latent dtype Two divergences from the Diffusers reference that were hurting image quality (soft / grainy / distorted faces at default settings): 1. Tokenizer max_sequence_length was 226 in wan_text_encoder, but the model was trained with 512-token sequences. The upstream native config.json has text_len: 512, and Diffusers' WanPipeline.__call__ default is 512 (overriding _get_t5_prompt_embeds's stale 226 default). Wan's cross-attention sees padded zeros past the prompt's actual length but expects to be looking at a 512-position context window. 2. Latents were stored in bf16 throughout the denoise loop. Diffusers' WanPipeline.prepare_latents explicitly uses dtype=torch.float32 and only casts to the transformer's dtype right at the forward call: latent_model_input = latents.to(transformer_dtype) Storing in bf16 between steps accumulates ~40 steps of bf16 quantization on the scheduler's small per-step deltas. Now latent_dtype = torch.float32 throughout, with a per-step cast for the transformer forward pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> chore(wan): add diffusers reference comparison script scripts/wan_diffusers_reference.py runs a Diffusers-format Wan 2.2 checkpoint directly via WanPipeline.from_pretrained, with the same arguments InvokeAI's wan_denoise uses. Use to A/B against InvokeAI output when image quality is questionable. Defaults to enable_model_cpu_offload so the script fits on 16 GB cards where the full pipeline (transformer + UMT5-XXL + VAE) would otherwise OOM. --offload {model,sequential,none} controls the strategy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(model): Wan 2.2 Phase 4 - GGUF transformer support Adds single-file GGUF support for Wan 2.2 transformers, the path that makes A14B usable on consumer GPUs (~7 GB/expert at Q4_K_M instead of ~28 GB at bf16). Probe (configs/main.py): - New helpers: _has_wan_keys (Wan vs Qwen/FLUX/Z-Image fingerprint via condition_embedder.text_embedder.linear_1 + patch_embedding); _detect_wan_gguf_variant (16ch -> A14B, 48ch -> TI2V-5B from patch_embedding.weight.shape[1]); _detect_wan_gguf_expert (filename heuristic for high_noise / low_noise / none). - Main_GGUF_Wan_Config(base=Wan, format=GGUFQuantized, variant, expert). Tolerates the ComfyUI 'model.diffusion_model.' / 'diffusion_model.' prefixes via _has_wan_keys' multi-prefix scan. - Registered in factory.py. Loader (model_loaders/wan.py): - WanGGUFCheckpointModel mirrors the QwenImage GGUF pattern: gguf_sd_loader -> strip ComfyUI prefix -> auto-detect arch from state dict shapes (num_layers, inner_dim, ffn_dim, text_dim, in_channels, num_heads = inner_dim/128) -> init_empty_weights + load_state_dict(strict=False, assign=True). Loader invocation (wan_model_loader.py): - New 'Transformer (Low Noise)' picker: optional second GGUF for the A14B dual-expert MoE. Auto-swaps if the user wired the experts in the wrong order. Warns when an A14B GGUF is loaded without a paired low-noise expert (single-expert run, degraded quality). - GGUF mains require either a standalone VAE+encoder or a Diffusers Component Source (which can also supply boundary_ratio). - Diffusers main path unchanged (still pulls both experts from transformer/ + transformer_2/). Tests (tests/.../test_wan_gguf_config.py): - 14 tests across key fingerprint, variant detection, expert filename heuristic, and the full probe (A14B high/low, TI2V-5B, GGUF rejection, unrecognised state-dict rejection, explicit override). Total Wan tests: 55 passing (no regressions). FE lint clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): support QuantStack-style GGUFs and standalone Diffusers VAE The city96 Wan 2.2 GGUF repos have been removed from Hugging Face, leaving QuantStack as the surviving distributor. QuantStack ships the native upstream Wan key layout (text_embedding.0/2, self_attn/cross_attn, ffn.0/2, head.head, head.modulation, ...) rather than the diffusers naming city96 used; biases are stored as F16 rather than BF16; and the standalone Wan VAE installs as a flat AutoencoderKLWan folder which the generic loader rejects. Three fixes: 1. Probe now recognises both diffusers and native key layouts via a new _is_native_wan_layout helper; _has_wan_keys accepts either text-proj fingerprint. 2. GGUF loader converts native -> diffusers keys (mirroring diffusers' convert_wan_transformer_to_diffusers) and unwraps non-quantized GGMLTensors to plain tensors at compute_dtype. The unwrap is needed because conv3d isn't in GGMLTensor's dispatch table, so the F16 patch_embedding bias would otherwise hit conv3d against bf16 latents. 3. VAELoader gains a VAE_Diffusers_Wan_Config branch that loads AutoencoderKLWan directly; the generic path can't handle a flat single-class folder when a submodel_type is provided. Adds 12 tests covering the native layout (probe + converter + unwrap). Verified end-to-end against Wan2.2-T2V-A14B-Q4_K_M from QuantStack: 1095 tensors round-trip key-for-key against WanTransformer3DModel. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(model): Wan 2.2 Phase 5 - LoRA support Probe + config (LoRA_LyCORIS_Wan_Config): - Detects Wan LoRAs in three layouts: diffusers PEFT, native upstream PEFT (ComfyUI), and Kohya (both naming variants). - Anti-pattern guards prevent collisions with Anima (Cosmos DiT q_proj convention), QwenImage (transformer_blocks), Flux (double/single blocks), and Z-Image (diffusion_model.layers). - Optional ``expert: "high" | "low" | None`` field; auto-detected from filename (high_noise / low_noise / hyphenated / concatenated variants). Key conversion (wan_lora_conversion_utils): - Native upstream keys (self_attn/cross_attn, ffn.0/2) -> diffusers (attn1/attn2, ffn.net.0.proj / ffn.net.2). - Strips ``transformer.``, ``diffusion_model.``, ``base_model.model.transformer.`` prefixes from PEFT-style keys. - Kohya layer names mapped through an explicit longest-match table. - Output paths use diffusers naming so the LayerPatcher can resolve them against WanTransformer3DModel parameter paths. Loader integration: - Adds BaseModelType.Wan branch to LoRALoader._load_model. Invocation nodes (wan_lora_loader.py): - WanLoRALoaderInvocation: single LoRA with auto/both/high/low target field. - WanLoRACollectionLoader: list of LoRAs, auto-routed by each LoRA's recorded expert tag. - Output WanLoRALoaderOutput carries the WanTransformerField with updated ``loras`` / ``loras_low_noise`` lists. Denoise integration: - _ExpertSwapper now manages both the model_on_device context and the LayerPatcher.apply_smart_model_patches context per expert. LoRA patches are entered after device load and exited before device release, with fresh iterators per swap. - GGUF (quantized) experts request sidecar patching so GGMLTensor weights aren't touched directly. - Low-noise expert falls back to the primary loras list when ``loras_low_noise`` is empty (matches WanTransformerField semantics). Tests: 81 new tests covering probe accept/reject across formats, anti-pattern guards on competing architectures, converter round-trips for all three layouts, invocation target resolution + routing + duplicate guards, and the _ExpertSwapper lifecycle (lora context opens/closes in the right order around the device swap, quantized flag forwards, no-LoRA path skips the patch context, re-entering the same label is a no-op). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): probe Wan LoRA before Anima in the config union Native-PEFT Wan LoRAs (lightx2v's Lightning, most ComfyUI-trained Wan LoRAs) carry keys like ``diffusion_model.blocks.X.cross_attn.k.lora_A.weight``. Anima's probe matches on the bare ``cross_attn``/``self_attn`` substring — it does not require the Anima-specific ``_proj`` suffix nor any of the ``mlp``/``adaln_modulation`` Cosmos DiT markers — so these Wan LoRAs were classified as ``BaseModelType.Anima`` because Anima happened to run first. Reorder the LyCORIS section of ``AnyModelConfig`` so Wan probes first. Wan's probe is strictly more restrictive (it rejects Anima's ``_proj`` attention suffix via the anti-pattern guard added in the previous commit), so Anima LoRAs are still correctly classified after this reorder. Existing users with mis-tagged installs need to delete the affected LoRA records and reinstall. Adds two regression tests: a union-ordering assertion, and a sanity check that demonstrates Anima's probe *would* match Wan native keys if asked directly — pinning the constraint that motivates the ordering. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> chore(i18n): add Wan2.2 T5 Encoder model-manager label The frontend source already references ``modelManager.wanT5Encoder``; the locale key was added with a casing typo (``want5Encoder``). Fix the key so the Wan T5 Encoder model type renders its display name correctly in the model manager UI. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(model): Wan 2.2 Phase 7 - reference-image (I2V) conditioning Re-implementation after the first attempt — which used CLIP-vision conditioning — was reverted. Wan 2.2 I2V-A14B does NOT use a CLIP-vision encoder (the Diffusers repo ships ``image_encoder: [null, null]`` in ``model_index.json``); instead it conditions on a reference image by VAE-encoding it and concatenating the resulting latents (plus a first-frame mask) to the noise latents along the channel dim. The I2V transformer therefore has ``in_channels=36`` (16 noise + 16 ref-image latents + 4 mask) vs ``in_channels=16`` for T2V. Taxonomy: - Re-adds ``WanVariantType.I2V_A14B``. Probes: - Diffusers: ``_detect_wan_variant`` reads ``transformer/config.json::in_channels``; 36 → I2V_A14B, 16 → T2V_A14B (both share the dual-expert layout). - GGUF: ``_detect_wan_gguf_variant`` recognises ``in_channels=36`` from the patch_embedding tensor shape and emits I2V_A14B. Backend extension (``backend/wan/extensions/wan_ref_image_extension.py``): - ``preprocess_reference_image`` resizes + normalises to a 5D pixel tensor. - ``encode_reference_image_to_condition`` VAE-encodes the image and stacks a 4-channel first-frame mask on top, producing the ``[1, 20, 1, H/8, W/8]`` condition tensor the denoise loop consumes. - Mirrors diffusers ``WanImageToVideoPipeline.prepare_latents`` with ``num_frames=1`` and ``expand_timesteps=False``. Invocation node (``wan_ref_image_encoder.py``): - "Reference Image - Wan 2.2": image + VAE + width/height pickers. - Output ``WanRefImageConditioningField`` carries the condition tensor name plus the dimensions used (so the denoise step can validate dim parity). Denoise integration: - ``WanDenoiseInvocation`` gains an optional ``ref_image`` field. - Variant gate: rejects ref_image on T2V_A14B and TI2V-5B with a clear error before doing any work. - Dimension gate: rejects ref-image width/height mismatch vs denoise. - At every transformer call, concatenates the 20-channel condition tensor to the 16-channel noise latents along the channel dim before passing to the transformer (giving the 36-channel input I2V expects). Tests: 14 new across the probe, the extension, and the denoise loop. The synthetic ``_ZeroTransformer`` test stand-in now mirrors the real I2V transformer's ``in_channels=36, out_channels=16`` asymmetry by slicing its zero output back to 16 channels when the input is 36-wide. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): derive GGUF out_channels from proj_out shape (I2V support) The GGUF loader was setting ``out_channels = in_channels`` which is wrong for Wan 2.2 I2V-A14B: that variant has ``in_channels=36`` (16 noise + 16 ref-image latents + 4 first-frame mask, concatenated by the denoise loop) but ``out_channels=16`` since the transformer only predicts the noise component back. Loading an I2V GGUF would build a transformer with the wrong proj_out shape and crash: RuntimeError: Error(s) in loading state_dict for WanTransformer3DModel: size mismatch for proj_out.weight: copying a param with shape torch.Size([64, 5120]) from checkpoint, the shape in current model is torch.Size([144, 5120]). (144 = 36 * 4, 64 = 16 * 4 — patch_size=(1, 2, 2) → prod=4) Read out_channels directly from the ``proj_out.weight`` shape in the state dict. This is correct for all three Wan 2.2 variants without needing to know the variant in advance. Also tighten the num_layers fallback: T2V_A14B and I2V_A14B share 40 layers; only TI2V-5B has 30. The fallback is rarely hit in practice (the per-block count comes from the state dict scan), but the previous code would have defaulted I2V_A14B to 30 layers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(model): make Anima LoRA probe mutually exclusive with Wan InvokeAI's ``Config_Base.CONFIG_CLASSES`` is a Python ``set``, so iteration order during model probing is non-deterministic across process restarts. First-match-wins ordering in ``AnyModelConfig`` is documentation only — it has no effect on which config is iterated first. Anima's previous probe accepted any state dict containing the substring ``cross_attn`` or ``self_attn``, which collides with Wan's native LoRA key layout (``diffusion_model.blocks.X.cross_attn.q.lora_down.weight``). Both probes accepted Wan native LoRAs (including lightx2v's Lightning T2V and I2V distillations), and the ``matches.sort_key`` tiebreaker only disambiguates by ModelType, not within LoRA configs. So which config "won" depended on dict hash order — sometimes Wan, sometimes Anima. The previous mitigation reordered the AnyModelConfig union to put Wan before Anima. That worked by luck and was inherently fragile. Tighten Anima's probe to require Cosmos-DiT-exclusive subcomponents: ``mlp``, ``adaln_modulation``, or ``_proj``-suffixed attention names (``q_proj``/``k_proj``/``v_proj``/``output_proj``) — none of which appear in any Wan LoRA. Wan native uses bare ``.q``/``.k``/``.v``/``.o`` on ``self_attn``/``cross_attn``, and ``ffn.N``/``ffn.net.N`` instead of ``mlp``. The new strict detectors live alongside the original loose ones so the Anima conversion utility (which runs after probing) still works. Regression tests in ``test_wan_lora_probe_independence.py`` cover: - I2V Lightning V1 (the bug-triggering LoRA), T2V Lightning V2, Wan Kohya and Wan diffusers PEFT layouts — Wan probe accepts, Anima probe rejects. - Anima PEFT and Kohya layouts — Anima accepts, Wan rejects. - A meta-test that runs every LoRA config in CONFIG_CLASSES against the Lightning state dicts and asserts exactly one accepts — this catches ANY future probe collision, not just Wan vs Anima. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): defer expert model loading in _ExpertSwapper to avoid cache thrash The swapper used to take pre-loaded ``LoadedModel`` handles at construction: high_info = context.models.load(self.transformer.transformer) low_info = context.models.load(self.transformer.transformer_low_noise) swapper = _ExpertSwapper(high_info=high_info, low_info=low_info, ...) With dual ~9 GB A14B GGUF experts plus the ~10 GB UMT5-XXL encoder competing for the same RAM cache, the LRU policy frequently dropped one expert by the time the denoise loop swapped into it. The model manager then emitted [MODEL CACHE] Locking model cache entry ... but it has already been dropped from the RAM cache. This is a sign that the model loading order is non-optimal in the invocation code (See ... #7513). and reloaded the weights from disk (~1.2s extra per swap). Refactor the swapper to take the ``ModelIdentifierField`` plus the ``InvocationContext`` and call ``context.models.load(model_id)`` lazily inside ``get()``. Each swap obtains a fresh handle, the LRU window is small, and the warning goes away. Config metadata (used to compute ``is_quantized``) is read upfront via ``context.models.get_config()`` — that's metadata, not weights, so it doesn't put pressure on the cache. Tests: existing swapper lifecycle tests refactored to use a fake context whose ``models.load`` is logged. A new ``test_lazy_load_per_swap_not_upfront`` pins the regression — it asserts ``models.load`` is NOT called at swapper construction, only at first get() per expert. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(wan): add Phase 8 inpaint regression tests The denoise_mask wiring + RectifiedFlowInpaintExtension integration in wan_denoise.py was put in place during Phase 2/3 alongside the rest of the denoise loop. Phase 8 of the plan was about ensuring this path worked and is locked in by tests. Three new tests under TestWanDenoiseInpaint: 1. test_preserved_region_matches_init_exactly: builds a half/half mask (left = preserve, right = regenerate in user-side convention), runs full denoise with the synthetic zero-output transformer, and asserts the preserved half of the final latents equals the init exactly while the regenerated half does not. Pins the mask-inversion + per-step merge behavior. 2. test_inpaint_requires_init_latents: a mask without init latents must raise a clear ValueError — the merge has nothing to weld back to. 3. test_no_mask_path_is_unchanged: regression that adding the inpaint extension didn't perturb the non-inpaint codepath (with init latents + denoising_start=0.5 but no mask, the loop just runs img2img). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> feat(frontend): add I2V_A14B to Wan variant zod enum + manager label Phase 7 added the I2V_A14B backend variant. The frontend's zod enum (features/nodes/types/common.ts:zWanVariantType) and the model manager's variant-label map (features/modelManagerV2/models.ts) were still on the two-variant list, so: - ModelIdentifierField inputs with ui_model_variant filters on Wan couldn't list I2V models. - The model manager UI showed a raw 'i2v_a14b' string instead of the human label. Phase 9 (full linear-view wiring — type guards, hooks, params slice, graph builder, tab UI) is in progress on a follow-up commit; this lands the two small enum fixes first so the I2V probe / install paths work correctly end-to-end with the existing FE. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): Phase 9 piece #1 - linear-view T2V txt2img graph builder Adds the minimum frontend wiring needed to generate Wan 2.2 images from the linear view: - buildWanGraph.ts (new): text-to-image graph (model_loader → text_encoder × 2 → denoise → l2i). Diffusers main model only — transformer, VAE and UMT5 encoder all resolve from the same repo, so no Wan-specific params slice fields are required yet. CFG-skip branch when guidance_scale ≤ 1.0. - useEnqueueGenerate / useEnqueueCanvas dispatchers: route base === 'wan' to buildWanGraph. - graph/types.ts: add wan_l2i / wan_i2l / wan_denoise / wan_model_loader to the relevant node-type unions. - addTextToImage / addImageToImage: include wan_denoise / wan_l2i so width/height are wired correctly and the txt2img helper accepts the Wan l2i node. - isMainModelWithoutUnet: include wan_model_loader (Wan has no UNet, same as the other modern bases). - metadata.py: add wan_txt2img / wan_img2img / wan_inpaint to the generation_mode enum (img2img / inpaint pieces land next). - schema.ts: regenerated to pick up the metadata enum + new Wan invocations. Pieces left in Phase 9: params slice (standalone VAE / T5 / GGUF low-noise / LoRA / ref-image fields + selectors), img2img + I2V + inpaint branches in the graph builder, and Wan-specific UI components. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> feat(wan): Phase 9 piece #2 - GGUF support and CFG-Low control in linear view Adds the three Wan-specific params + UI controls that gate GGUF workflows plus a separate low-noise CFG slider for A14B users. Params slice: - wanTransformerLowNoise (the second-expert GGUF for A14B) - wanComponentSource (Diffusers Wan model providing VAE + UMT5-XXL when the main is a GGUF) - wanGuidanceScaleLowNoise (optional separate CFG for the low-noise expert; null = fall back to the primary CFG) Plus a `selectIsWan` selector for accordion gating. UI components: - ParamWanModelSelects.tsx (Advanced accordion): two model pickers — Transformer (Low Noise) filtered to Wan GGUF mains, and VAE/Encoder Source filtered to Wan Diffusers mains. Mirrors the ParamQwenImageComponentSourceSelect structure. - ParamWanGuidanceScaleLowNoise.tsx (Generation accordion): slider + number input with an "auto" indicator when cleared. Default 3.5 matches the diffusers reference 4.0 / 3.0 split. Wiring: - Generation accordion: ParamWanGuidanceScaleLowNoise shown when base is wan, scheduler excluded for wan (same pattern as Anima/Qwen). - Advanced accordion: ParamWanModelSelects shown when base is wan, and Wan excluded from the SD-family VAE/CFG-rescale blocks. - buildWanGraph.ts: forwards the three new params to the model loader and denoise nodes (transformer_low_noise_model, component_source, guidance_scale_low_noise) and adds them to the graph metadata. Hooks/types: - useWanDiffusersModels + useWanGGUFModels in modelsByType.ts. - isWanDiffusersMainModelConfig + isWanGGUFMainModelConfig type guards. - Three new locale strings (wanComponentSource, wanTransformerLowNoise, wanGuidanceScaleLowNoise[Auto]). GGUF workflow now works end-to-end in the linear view: pick a Wan GGUF main, set Transformer (Low Noise) to the paired second-expert GGUF, set VAE/Encoder Source to any Diffusers Wan repo (TI2V-5B is convenient at ~12 GB) — generate produces an image. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): UX polish on the Wan linear-view controls Bundles four small fixes applied during a usability review of the Wan linear-view section (piece #2): 1. **Filter Main vs Transformer (Low Noise) dropdowns by expert tag.** The Wan GGUF probe records each file's ``expert`` field (``"high"`` / ``"low"`` / ``"none"``) via filename heuristic. - ``MainModelPicker``: hides ``expert === 'low'`` Wan GGUFs so users can't accidentally wire a low-noise expert as the primary main. - Transformer (Low Noise) picker (``useWanGGUFLowNoiseModels``): shows ``expert === 'low'`` Wan GGUFs only. Diffusers Wan mains and TI2V-5B aren't affected — they don't carry the ``expert`` field on their config schema. The backend's auto-swap safety net stays in place. 2. **Match the primary CFG slider's range.** The Wan low-noise CFG slider was constrained to 1–10 while the primary CFG ranges 1–20. With the diffusers reference 4/3 split, the low-noise slider thumb sat noticeably further right than the primary — visually misleading. Both sliders now share the 1–20 range with marks at [1, 10, 20]. 3. **Label fits the form column.** "CFG (Low Noise)" → "CFG (Low)" so the slider fits cleanly next to its label instead of overlapping. 4. **Indicator state for the low-noise CFG slider.** Replaced the inline "(auto)" / "(same as cfg)" text — which kept overlapping the slider regardless of how short the label got — with an X-only reset button that's only visible when the user has set an explicit value. Absence of the X conveys auto/fallback state without any text overhang. 5. **Friendlier Transformer (Low Noise) placeholder.** "Second-expert GGUF for A14B (pair with the high-noise main)" → "Add for full detail" — concise nudge for users who haven't paired the second expert yet. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> feat(wan): Phase 9 piece #3 - linear-view img2img branch Adds Wan 2.2 image-to-image to the linear view, mirroring the Qwen Image pattern. The mode switches on the canvas state — pure-prompt runs go through addTextToImage as before; canvas runs with an init image go through addImageToImage which wires a fresh wan_i2l (Image to Latents - Wan 2.2) node between the init image and the denoise's `latents` input, honoring the existing denoise_start slider. buildWanGraph: - Drops the txt2img-only guard, branches on generationMode. - img2img: spins up a wan_i2l node and hands it to addImageToImage alongside the existing denoise / l2i / modelLoader (as vaeSource). - inpaint / outpaint still fail loudly — pieces #4-#6. graphBuilderUtils.getDenoisingStartAndEnd: - Adds 'wan' to the simple-linear case (denoising_start = 1 - denoisingStrength). Note: Wan's flow-matching schedule is "sticky" on the init compared to SDXL — users will likely need denoisingStrength ≥ 0.7 to see substantial change, matching the user-found 0.15-0.3 denoising_start sweet spot from earlier img2img testing. We may revisit this with an exponent rescale (like FLUX uses) if the response curve feels off. addImageToImage: - Adds 'wan_i2l' to the i2l-node-type union so the Wan i2l can be threaded through the shared helper. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): add wan_denoise to addImageToImage/addInpaint/addOutpaint type checks Three sibling graph-helper utilities had the same modern-base list as addTextToImage did, and the buildWanGraph img2img branch tripped one of them at canvas-Generate time: error [generation]: Failed to build graph {name: 'Error', message: 'Wrong assertion encountered'} The else-branch in each helper assumes 'denoise_latents' (the SD1.5/SDXL legacy path) and asserts that — failing for any modern base not listed above the branch. addTextToImage was already updated in Phase 9 piece #1; this catches the parallel cases that the img2img/inpaint/outpaint flows go through. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> feat(wan): Phase 9 piece #4 - linear-view inpaint and outpaint branches Wires Wan 2.2 inpaint and outpaint through the existing addInpaint / addOutpaint helpers. The backend's RectifiedFlowInpaintExtension was plumbed into wan_denoise.py back in Phase 8 (commit ab54617173); this just connects the FE. buildWanGraph: - generationMode === 'inpaint' → spin up a wan_i2l, call addInpaint with denoise + l2i + modelLoader (used as both vaeSource and modelLoader since the Wan model loader carries the VAE). - generationMode === 'outpaint' → parallel branch with addOutpaint. addInpaint: - i2l-node-type union now includes 'wan_i2l' (the addImageToImage and addOutpaint type unions already do — different union shapes). metadata.py: - generation_mode literal adds "wan_outpaint" alongside the existing wan_txt2img / wan_img2img / wan_inpaint entries. isMainModelWithoutUnet already includes wan_model_loader (Phase 9 piece create_gradient_mask when Wan is the main. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> feat(wan): Phase 9 piece #5 - linear-view I2V branch (raster as reference image) Wan 2.2 I2V-A14B models condition on a reference image whose VAE-encoded latents are concatenated to the noise along the channel dim each step (in_channels=36 on the I2V transformer). In the linear view this maps cleanly onto the existing canvas raster layer: pick an I2V model, drag an image to raster, generate. buildWanGraph: - Fetch the modelConfig early so the variant gate (i2v_a14b vs the rest) can drive the branch shape instead of being a post-hoc check. - I2V + txt2img: fail loudly ("Switch to the canvas tab and drag an image to the raster layer"). I2V models won't produce useful output without a reference, and the backend would crash trying to concatenate a missing condition tensor. - I2V + img2img: pull the raster image via the canvas compositor, wire it through a wan_ref_image_encoder (which VAE-encodes it and builds the 4-mask + 16-latent condition tensor backend-side), then feed the result into denoise.ref_image. Denoise runs from fresh noise (denoising_start=0, no init_latents) — the ref image is cross-attention/concat conditioning, not a noise-trajectory anchor. - I2V + inpaint/outpaint: fail clearly. Combining ref-image conditioning with a denoise mask is conceptually possible but the backend interaction hasn't been validated end-to-end. metadata.py: - Adds "wan_i2v" to the generation_mode literal so the metadata field on I2V renders correctly. T2V flows (txt2img / img2img / inpaint / outpaint) are unchanged for non-I2V Wan variants (T2V-A14B and TI2V-5B). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): enforce multiple-of-16 dimensions to match transformer patch grid Wan 2.2's transformer has ``patch_size=(1, 2, 2)``: it patch-embeds with stride 2 then un-patches by 2. Combined with the VAE's 8x spatial scale, canvas H/W must be a multiple of ``8 * 2 = 16`` — not just 8 — for the patch round-trip to land exactly. Otherwise the latents and noise prediction disagree by one in the spatial dim and the scheduler step fails: RuntimeError: The size of tensor a (147) must match the size of tensor b (146) at non-singleton dimension 3 (here latent_w=147 → patch_w=73 → un-patched_w=146 ≠ 147) This was silent for T2V at 1024x1024 (already a multiple of 16) but fired for I2V at non-multiple-of-16 canvas sizes. Fixes: - ``optimalDimension.getGridSize``: Wan moves from the default 8 case to the multiple-of-16 case (alongside flux / sd-3 / qwen-image / z-image which have the same patch arithmetic). The canvas bbox UI now snaps Wan dimensions to multiples of 16. - ``wan_denoise.py`` and ``wan_ref_image_encoder.py``: bump width/height ``multiple_of`` from 8 to 16. Defense-in-depth — workflow-editor users won't be able to send a non-16-aligned dim either. Existing backend tests (23 passing) still hold — 1024 is divisible by 16 so the test fixtures didn't exercise the off-by-one path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): show negative prompt box in Wan linear-view Wan was missing from SUPPORTS_NEGATIVE_PROMPT_BASE_MODELS, so the linear-view negative-prompt input was hidden even though the Wan denoise node already wires negative conditioning when CFG > 1 (buildWanGraph.ts:67-75). Adds 'wan' to the list. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> feat(wan): Phase 9 piece #6 - Wan LoRA collection in linear view Adds Wan LoRA wiring to buildWanGraph, mirroring the Qwen Image pattern. The shared LoRASelect / LoRAList UI in the linear view already filters LoRAs by the selected main model's base, so Wan LoRAs surface automatically when a Wan main is picked — no UI changes needed. addWanLoRAs (new): - Filters state.loras.loras to enabled Wan LoRAs. - For each LoRA: spawns a ``lora_selector`` node and threads it through a single ``collect`` collector. - Routes the collector into a ``wan_lora_collection_loader`` which sits between modelLoader and denoise — modelLoader.transformer → loader, then loader.transformer → denoise (rerouting the original modelLoader → denoise edge). - Emits per-LoRA metadata so PNG metadata + workflow restore work. The dual-expert routing (high-noise vs low-noise vs untagged) is handled entirely on the backend by ``WanLoRACollectionLoader`` based on each LoRA's recorded ``expert`` tag (set by the probe from the filename heuristic in piece #5 of Phase 5). The FE just hands over the bag of LoRAs; no per-list FE plumbing needed. buildWanGraph: - Calls addWanLoRAs(state, g, denoise, modelLoader) after the base transformer edge is in place. The helper is a no-op when no Wan LoRAs are enabled, so it's safe to call unconditionally. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(wan): detect LoRA variant and filter by main model Wan 2.2 A14B (inner_dim=5120) and TI2V-5B (inner_dim=3072) LoRAs are not interchangeable — applying one against the wrong main model crashes the layer patcher with a tensor-shape error (e.g. A14B Lightning on TI2V-5B mains produced ``shape '[3072, 3072]' is invalid for input of size 26214400``). Probe Wan LoRAs' inner-dim at install time and record the family on a new ``variant`` field (``a14b`` / ``5b`` / null). The LoRA picker in the linear view hides incompatible variants when the user selects a main, and the graph builder filters any still-enabled mismatches at submit time with a warning. Untagged LoRAs (probe couldn't identify) pass through so they aren't silently hidden. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> feat(wan): ref-image panel, GGUF readiness, and auto-default sources Wan 2.2 I2V now uses the global Reference Images panel (same UX as Qwen Image Edit and FLUX.2 Klein) instead of pulling the conditioning image from a canvas raster layer. Adds: - WanReferenceImageConfig zod type + isWanReferenceImageConfig guard; integrated into the ref-image discriminated union, settings panel, layer hooks, and validators. - 'wan' added to SUPPORTS_REF_IMAGES_BASE_MODELS, but the panel only shows for the i2v_a14b variant (T2V and TI2V-5B don't consume ref images, so the panel is hidden for them). - buildWanGraph I2V branch reads the first enabled wan_reference_image from refImagesSlice; the canvas-raster-as-ref path is removed. I2V now only supports txt2img mode (canvas img2img/inpaint/outpaint assert with a clear message). GGUF Wan readiness check: GGUF mains carry only the transformer, so the loader needs a Diffusers Component Source (or standalone VAE + UMT5-XXL encoder) to resolve the VAE and text encoder. Without one, enqueue is now blocked with a clear reason. The low-noise A14B partner expert remains optional (loader falls back to the high-noise expert when it's missing). Adds standalone Wan VAE and Wan T5 Encoder selectors to the Advanced accordion (Qwen pattern). Wires them as vae_model / wan_t5_encoder_model on the wan_model_loader node — backend priority is standalone > diffusers main > component source. Auto-default on Wan selection (so GGUF users don't have to fiddle with Advanced): when the new main is a Wan GGUF, fill the Component Source, standalone VAE, and standalone T5 encoder with first available matches if not already set. Component Source is matched by variant family (A14B GGUF prefers an A14B Diffusers; TI2V-5B prefers a TI2V-5B Diffusers) since the two families use different VAE channel counts (16 vs 48); within A14B, T2V and I2V share VAE/encoder so they're interchangeable as a source. Runs on every Wan selection (including Diffusers -> GGUF switches), only fills empty slots. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): add Wan 2.2 starter models and bundle Wan 2.2 starter pack (selected when the user picks the Wan 2.2 bundle) brings up the minimal-cost path to running A14B T2V end-to-end: - Standalone UMT5-XXL encoder and A14B VAE (so GGUF mains don't need a full Diffusers download for their VAE/encoder sources). - T2V A14B Q4_K_M and Q8_0 GGUF expert pairs (high + low noise). - T2V Lightning V1.1 Seko rank-64 LoRA pair (4-step inference). Additional Wan 2.2 starter models browseable from the model manager: - Full Diffusers T2V A14B, I2V A14B, and TI2V-5B. - I2V A14B Q4_K_M and Q8_0 GGUF expert pairs + Lightning V1 LoRA pair. - TI2V-5B Q4_K_M and Q8_0 GGUFs + the 48-channel TI2V-5B VAE. Each "high noise" GGUF lists its low-noise partner plus the shared VAE and UMT5-XXL encoder as dependencies, so installing one of them pulls in everything the loader needs. QuantStack's HighNoise/LowNoise file naming and lightx2v's high_noise_model/low_noise_model.safetensors are both picked up by the existing filename heuristic in the GGUF probe. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> docs(wan): add Wan 2.2 hardware requirements Adds Wan 2.2 A14B (T2V/I2V) and TI2V-5B rows to the hardware requirements table with rough VRAM/RAM guidance per quantization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(wan): recall low-noise transformer, component source, and standalone VAE/T5 Wan-specific metadata fields embedded by the graph builder (wan_transformer_low_noise, wan_component_source, wan_vae_model, wan_t5_encoder_model, wan_guidance_scale_low_noise) had no recall handlers in features/metadata/parsing.tsx, so recalling an image's parameters would leave these fields empty. Adds a handler for each that dispatches the matching paramsSlice action and renders a row in the metadata viewer. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): add default Wan 2.2 T2V and I2V workflows Ships two default workflows in the library, tagged so they appear in "Browse Workflows" under the wan2.2 / text to image / image to image tags: - Text to Image - Wan 2.2: full T2V/TI2V-5B graph (model loader, positive + negative encoders, denoise, l2i). Exposes the five model slots, prompts, steps, dual CFG, and dimensions. - Image to Image - Wan 2.2: I2V A14B graph that adds a wan_ref_image_encoder. Exposes the reference image input plus the standard fields. Both follow default-workflow rules: IDs prefixed with default_, meta.category = "default", and no references to user-installed resources. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(videos): Phase 1 - backend video storage, records, REST API Adds a parallel video pipeline alongside the existing image pipeline so the gallery can host MP4 alongside PNGs. Implements: - New service modules (parallel to image equivalents): video_records/ record store + sqlite impl video_files/ disk file store (mp4 + first-frame webp thumb) videos/ orchestrating service board_video_records/ board <-> video association - migration_32 creates `videos` and `board_videos` tables - /api/v1/videos/ router: upload, list, get DTO, /full (with HTTP Range so HTML5 <video> seek/scrub works), /thumbnail, /metadata, star/unstar, delete, batch delete, board add/remove - LocalUrlService.get_video_url and SimpleNameService.create_video_name - imageio[ffmpeg] dep for video encode (used in later phases) - Wires all four new services into InvocationServices, dependencies.py, api_app.py, and three test fixtures Verified end-to-end against an in-memory db + tmp output dir: upload, probe, save (file + thumbnail + record), DTO build, list, delete. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(videos): Phase 2 - polymorphic gallery list endpoint Adds /api/v1/gallery/items/ and /api/v1/gallery/items/names returning a unified time-sorted stream of images + videos so the frontend can render them interleaved with a single virtualized query. - gallery_common: GalleryItem discriminated union (kind + name + shared fields + nullable video duration/fps), GalleryItemRef, names result - gallery_default: SqliteGalleryService implements UNION ALL across the images and videos tables, applying identical filters (origin/category/ is_intermediate/board_id/search) to each half; pagination via outer ORDER BY + LIMIT/OFFSET; counts are summed across the two halves - URLs are resolved at row -> DTO conversion time so each item routes to the correct /api/v1/images or /api/v1/videos endpoint - Wired into InvocationServices, dependencies.py, api_app.py, and the three test fixtures Existing /api/v1/images endpoints are unchanged so any non-gallery consumers (queue, recall, metadata workflows) continue to work as-is. Verified e2e: 2 images + 2 videos inserted in alternating order, both list_items and list_item_names return the correct interleaved order; category filter narrows to a single kind; starring an item bumps it to the top when starred_first=True. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(videos): Phase 3 - frontend RTK endpoints + MP4 upload routing Adds the typed API surface and upload integration so videos can be uploaded through the same gallery upload button that handles images. Schema: re-ran pnpm typegen against the running backend to pick up VideoDTO, VideoRecordChanges, GalleryItem, GalleryItemKind, GalleryItemRef, GalleryItemNamesResult and the two new paginated result types. RTK Query (services/api/endpoints/videos.ts) - parallel to images.ts: listVideos, getVideoDTO, getVideoMetadata, getVideoNames, uploadVideo, deleteVideo / deleteVideos, changeVideoIsIntermediate, starVideos / unstarVideos, addVideoToBoard / removeVideoFromBoard. Imperative helpers (getVideoDTO, getVideoDTOSafe, uploadVideo, uploadVideos) and the useVideoDTO convenience hook ride alongside, mirroring the image side. Tag types and invalidation: added Video / VideoList / VideoMetadata / VideoNameList / BoardVideosTotal / GalleryItemList / GalleryItemNameList to the api root. Board-affecting mutations now invalidate the polymorphic gallery list/name caches so videos and images stay coherent once the gallery wiring lands in Phase 4. Added a sibling getTagsToInvalidateForVideoMutation helper. Upload UX: useImageUploadButton.tsx's dropzone now accepts video/mp4, video/webm, video/quicktime alongside the existing image MIMEs. The drop handler splits files into image/video sets and routes each through its own mutation; a new onUploadVideo callback parallels the existing onUpload. Existing image-only callers pass through unchanged. Polymorphic gallery query endpoints + the useGalleryItemDTO hook will land with Phase 4 where they have actual consumers; the schema types they'll need are already in place under @knipignore tags. Verified: pnpm lint (knip, dpdm, eslint, prettier, tsc) all green; pnpm test 1103/1103 pass; live curl against the running dev server uploads an MP4 and serves both the webp thumbnail and the MP4 with a working HTTP Range response (206 + Content-Range). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(videos): Phase 4 - mixed gallery grid with video play badge Videos now appear in the same gallery grid as images, interleaved by created_at. Video thumbnails get a centered play-button badge so they read as videos at a glance; everything else (selection, virtualization, search, paged/virtual gallery views, keyboard nav) is unchanged. Approach: selection state stays `string[]` of names. The kind is recovered from the filename extension (.mp4 = video, anything else = image), which is reliable because the backend's SimpleNameService always emits `<uuid>.png` for images and `<uuid>.mp4` for videos. This sidesteps a 32-file cross-cut from changing the selection shape to a discriminated union, and selection is persist-denylisted so no migration is needed. Frontend: - new isVideoName helper in features/gallery/store/types - new endpoints/gallery.ts (deferred from Phase 3): useGetGalleryItemNamesQuery - new ImageGrid/GalleryItemPlayBadge: centered triangular badge over thumbnail - new ImageGrid/GalleryItemVideoStarIconButton: video-typed star toggle - new ImageGrid/GalleryVideoItem: counterpart to GalleryImage; reuses galleryItemContainerSX, GalleryItemSizeBadge (width/height-only stand-in), selection handling (single/shift/ctrl/cmd); alt-click falls through to a normal select since comparison is image-only - use-gallery-image-names now calls the polymorphic gallery names endpoint and exposes a mixed flat name list (existing callers - paged grid, search, navigation hotkeys - get the same shape) - useRangeBasedImageFetching partitions visible names by extension; images bulk-fetch via the existing getImageDTOsByNames mutation, videos dispatch individual getVideoDTO queries (no batch endpoint yet) - GalleryImageGrid's ImageAtPosition dispatches on isVideoName to render GalleryImage or GalleryVideoItem; star hotkey dispatches to the right star/unstar mutation based on kind - pruned the now-unused useGetImageNamesQuery / isImageName exports Verified: pnpm lint (knip, dpdm, eslint, prettier, tsc) all green; pnpm test 1103/1103 pass; live curl of /api/v1/gallery/items returns 57 polymorphic items with video duration populated and image duration null, /api/v1/gallery/items/names returns matching {kind, name} refs. The useGalleryItemDTO hook is intentionally deferred to Phase 5 where the polymorphic viewer is its first real consumer. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(videos): Phase 5 - inline video player in the image viewer Selecting a video now renders a polymorphic preview inside the existing viewer panel: thumbnail with a centered play button by default; clicking play swaps in an HTML5 <video controls autoplay>. Switching to a different item drops the video element back to idle (auto-pauses) and selecting an image again returns to the normal image preview. New components (features/gallery/components/ImageViewer/): - VideoPlayButtonOverlay: large centered play button with hover/shadow, used over the thumbnail in the idle state. - CurrentVideoPreview: idle/playing state machine. Resets on video_name change. The <video> src points at /api/v1/videos/i/.../full which supports HTTP Range, so seek/scrub work natively in the browser. New hook: - common/hooks/useGalleryItemDTO: polymorphic DTO resolver that dispatches between useImageDTO and useVideoDTO based on filename extension (isVideoName). Centralizes the kind-dispatch the viewer and toolbar both need. Wiring: - ImageViewer dispatches on galleryItem.kind to render CurrentImagePreview or CurrentVideoPreview. The compare-image DnD drop target is hidden when a video is selected (comparison is image-only). - ImageViewerToolbar hides the image-specific action row (CurrentImageButtons - load workflow, recall metadata, edit, etc.) and the metadata viewer toggle when a video is selected. The general-purpose ToggleProgressButton stays. Out of scope (per the plan): video deletion from the viewer (use gallery hover icons), video-specific metadata viewer, comparison-mode support for videos. Verified: pnpm lint (knip, dpdm, eslint, prettier, tsc) all green; pnpm test 1103/1103 pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(videos): accept MP4 (and other video) drops on the fullscreen dropzone The gallery-wide drag-and-drop target lives in FullscreenDropzone, not in useImageUploadButton (which only powers the upload button). It had its own hardcoded image-only zod allowlist that rejected MP4 files with "File type / extension is not supported". - Broaden the zod refines to accept video/mp4, video/webm, video/quicktime, video/x-matroska and the matching extensions - Add isVideoFile helper, split dropped files into image/video sets, and route each set through its own uploader (uploadImages / uploadVideos). Both update their respective RTK caches and invalidate the polymorphic gallery list/names. - Skip the canvas-paste fast-path for single-video drops — the canvas doesn't host videos as layers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(videos): right-click context menu on video items Adds a three-item context menu (delete, change board, download) on right-click / long-press of any gallery video item. Mirrors the image context menu's singleton-portal architecture so re-renders stay cheap. New files: - features/gallery/contexts/VideoDTOContext: small React context that scopes the active video DTO to the menu items (parallels ImageDTOContext). - features/gallery/components/ContextMenu/MenuItems/ ContextMenuItemDeleteVideo: window.confirm + deleteVideo mutation. Videos can't be referenced from canvas/nodes/refs, so the image modal's usage analysis is unnecessary; a one-step confirm matches the "minimal" scope. ContextMenuItemDownloadVideo: reuses the existing useDownloadItem hook against videoDTO.video_url / video_name. ContextMenuItemChangeBoardVideo: dispatches videosToChangeSelected and opens the (now polymorphic) ChangeBoardModal. - features/gallery/components/ContextMenu/VideoContextMenu: singleton pattern lifted from ImageContextMenu — registers gallery video elements via a Map; right-click looks up the target node and opens the menu at the cursor. Extended files: - features/changeBoardModal/store/slice: added video_names alongside image_names plus a videosToChangeSelected action. The two arrays are mutually exclusive — setting one clears the other. - features/changeBoardModal/components/ChangeBoardModal: now dispatches the matching video board mutations (add/removeVideoToBoard, plural endpoints don't exist yet so videos move one at a time — the menu acts on a single selection so this is a one-iteration loop). - features/gallery/components/ImageGrid/GalleryVideoItem: registers itself with useVideoContextMenu. - app/components/GlobalModalIsolator: mounts the singleton. Verified: pnpm lint (knip, dpdm, eslint, prettier, tsc) all green; pnpm test 1103/1103 pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): Phase 6 - Wan 2.2 T2V/I2V workflow nodes Adds two new invocation nodes that produce MP4 videos from a Wan 2.2 A14B transformer + VAE, plus the supporting plumbing. New invocations: - WanVideoDenoise (wan_video_denoise) — multi-frame counterpart to WanDenoise. Same per-step logic (CFG, MoE expert swap at the boundary timestep, LoRA patching, scheduler dispatch) — reuses _ExpertSwapper, _resolve_variant, and the scheduler/LoRA helpers from wan_denoise. Difference: the noise tensor has a real temporal dim built from num_frames, and the I2V condition is built across all latent frames (frame 0 conditioned, rest zero). Defaults match the Wan 2.2 reference: 832x480 / 81 frames / 40 steps / CFG 5.0 (high) + 4.0 (low). Inpaint / img2img are out of scope for this first cut. TI2V-5B is rejected; T2V/I2V A14B only. - WanLatentsToVideo (wan_l2v) — VAE-decodes 5D latents to RGB frames via AutoencoderKLWan (T_pixel = (T_lat - 1) * 4 + 1), then encodes an MP4 with imageio[ffmpeg] (libx264, yuv420p for browser compatibility). The temp file is moved into outputs/videos/ via context.videos.save(). Backend shared pieces: - make_noise gains num_latent_frames (default 1, backward compatible). - Added num_latent_frames_for(num_frames, scale=4) helper. - New encode_reference_image_to_video_condition mirrors diffusers' WanImageToVideoPipeline.prepare_latents with last_image=None and expand_timesteps=False: pads the reference image with zero pixel-frames, VAE-encodes the full pseudo-video, normalises, and builds the 4-channel temporal-rearranged first-frame mask. Verified numerically: 21 latent frames for num_frames=81, first latent frame's 4 mask channels = 1, rest = 0. - The existing single-frame encoder is left untouched. Schema / context: - New VideoField primitive (parallel to ImageField) and VideoOutput invocation output (width/height/num_frames/fps/duration/video). - New VideosInterface on InvocationContext with .save(source_path, width, height, duration, fps, ...) returning VideoDTO. Mirrors ImagesInterface — falls back to WithBoard / WithMetadata mixins and embeds the queue item's workflow/graph as a JSON sidecar. - WanRefImageConditioningField now carries num_frames so the denoise nodes can sanity-check the I2V condition. WanRefImageEncoder bumps to v1.1.0 and gains num_frames=1 input (use 81+ for video I2V; the encoder dispatches between the single- and multi-frame helpers). - Image WanDenoise now rejects multi-frame conditions with a clear message pointing at WanVideoDenoise. Verified: pnpm lint (5/5) green; pnpm tests (multiuser auth 122/122 + broader suite via prior runs); numerical shape checks for noise and ref-image condition; end-to-end smoke via VideoService.create. A restart of the InvokeAI server is required to pick up the new invocations in the workflow editor. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): add Wan 2.2 T2V and I2V starter video workflows Two new default workflows for the workflow editor 'Browse' modal: - 'Text to Video - Wan 2.2' — model loader -> two text encoders -> wan_video_denoise -> wan_l2v. Exposes prompt, model picks, CFG (high + low), dimensions, frames, fps, and steps. - 'Image to Video - Wan 2.2' — same shape plus a wan_ref_image_encoder feeding the denoise node's ref_image input. Exposes the reference image and the frames field on the ref-image node (must match the denoise node's frames — there is a clear validation error if they diverge, but the starter has them in sync at 81). Both default to the Wan 2.2 reference settings: 832x480, 81 frames @ 16 FPS (~5 s), 40 steps, CFG 5.0 (high expert) + 4.0 (low expert), seeded by a rand_int. Pass the existing _sync_default_workflows validator (id starts with default_, meta.category=default). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(wan): startup crash from stringified VideoOutput annotation run_app.py validates every invocation's return-type annotation against the output-class registry. wan_latents_to_video.py had a stray 'from __future__ import annotations' which made the `invoke()` return annotation a string ('VideoOutput') at runtime. The registry mismatch triggered the unregistered-output warning path, which itself crashed on output_annotation.__name__ because the annotation was a str: AttributeError: 'str' object has no attribute '__name__' The other Wan invocations don't use future annotations — drop the import to match. Verified post-fix: api_app import populates 95 output classes, wan_l2v annotation resolves to the real VideoOutput class and is in the registry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): add Wan 2.2 Lightning T2V starter workflow Same graph as 'Text to Video - Wan 2.2' but with two Apply LoRA - Wan 2.2 nodes chained between the model loader and the denoise node, and defaults retuned for the Lightning distillation: 4 steps and CFG 1.0 on both experts (CFG=1 skips the negative-conditioning forward pass entirely, ~20x faster than the 40-step / CFG-5.0 baseline at similar quality). Adapted from a user-saved workflow; cleaned for distribution by stripping the install-specific model/LoRA key bindings (defaults should not bake in local UUIDs), bumping to a fresh default_-prefixed id with meta.category=default, exposing the two LoRA fields (lora + weight) so users can swap LoRAs without diving into the canvas, and flagging the negative-prompt node as unused at CFG=1. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): add Wan 2.2 Lightning T2V and I2V starter workflows Two new default workflows that wire the Lightning LoRA pair into the T2V and I2V video pipelines for a ~20x speedup: - 'Text to Video - Wan 2.2 Lightning' — model loader -> apply LoRA (high) -> apply LoRA (low) -> text encoders -> wan_video_denoise -> wan_l2v. Defaults to 4 steps and CFG 1.0 (no negative branch). Cleaned-up version of Lincoln's saved Lightning workflow: stripped per-install model/LoRA keys, switched meta.category to 'default' with a default_ id, and exposed both LoRA loaders' lora/weight/ target fields so users can swap LoRAs without diving into the canvas. - 'Image to Video - Wan 2.2 Lightning' — same chain plus a wan_ref_image_encoder (v1.1.0 with num_frames) feeding the denoise ref_image input. Defaults match the non-Lightning I2V starter (832x480, 81 frames @ 16 FPS) but with 4 steps / CFG 1.0. LoRA target defaults to 'auto' so properly-tagged Lightning LoRAs route themselves; both workflow descriptions tell users to set explicit 'high'/'low' targets if their LoRAs are untagged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(wan): use FFMPEG plugin (not pyav) for MP4 encode wan_latents_to_video was passing plugin='pyav' to iio.imwrite, but the runtime only has imageio-ffmpeg installed (no PyAV). The encode step at the very end of generation crashed with: ImportError: The `pyav` plugin is not installed. Use `pip install imageio[pyav]` to install it Switch to plugin='FFMPEG' — backed by the bundled imageio-ffmpeg binary that pyproject already requires via imageio[ffmpeg]. libx264 yuv420p is the FFMPEG plugin's default for .mp4, so the explicit pixel_format is dropped (specifying it just produced a "Multiple -pix_fmt options" warning). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): log VAE decode and MP4 encode milestones in wan_l2v The video VAE decode + MP4 encode tail can take 30-90s on top of the denoise loop, and the toast-style signal_progress() messages don't land in the server log. Add context.logger.info() at: - VAE decode start: latent frame count -> pixel frame count + resolution - MP4 encode start: frames, fps, duration, dimensions - MP4 encode complete: encoded file size - Video saved: final video_name Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(wan): switch video thumbnail/probe to imageio[ffmpeg] backend After wan_l2v wrote a successful libx264 MP4 to disk, the invocation would hang in DiskVideoFileStorage.save() during the cv2.VideoCapture thumbnail-extraction step. cv2 wheels on this build can't reliably decode our libx264/yuv420p output (most often the wheel was compiled without an h264 decoder, but the failure mode is silent hang rather than a clear error). The net effect: the MP4 ends up in outputs/videos but the queue item never completes, so the frontend spinner spins forever and the gallery doesn't pick up the new entry. Fix: rewrite extract_video_frame and probe_video to try imageio's FFMPEG plugin first (same backend that did the encoding — so reading our own output is guaranteed to work), with cv2 retained only as a fallback for uploaded videos in formats imageio can't decode. Also add fine-grained log lines + exception guards inside DiskVideoFileStorage.save() so a future thumbnail failure can no longer hang the whole save — it now logs a warning and continues, leaving the video record in place even if the thumbnail step errored. With logging at each step (video written, thumbnail written, sidecar written) any future hang will be obvious from the last log line. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(videos): handle VideoField outputs in invocation_complete After wan_l2v wrote its MP4 successfully, the gallery and viewer were never updated: the new video didn't appear and the viewer stayed stuck on the previous "Saving video" progress spinner indefinitely. Root cause: onInvocationComplete.tsx only inspected results for isImageField / isImageFieldCollection. VideoField outputs were silently dropped, so the polymorphic gallery list never invalidated and no auto-switch happened. The viewer therefore kept rendering CurrentImagePreview, whose ImageViewerContext-local $progressEvent / $progressImage atoms intentionally aren't cleared on queue completion when autoSwitch is on — they rely on the new image's DndImage onLoad to clear them, which never fires for a video. Fix: add isVideoField (mirrors isImageField against {video_name}) and plumb video outputs through onInvocationComplete: - getResultVideoDTOs pulls VideoDTOs via getVideoDTOSafe - addVideosToGallery invalidates GalleryItemNameList / GalleryItemList so the polymorphic gallery refetches and the new video shows up - auto-switch dispatches the video name into selection (selection is a polymorphic string[]; useGalleryItemDTO already discriminates by filename extension) The selection change swaps CurrentImagePreview for CurrentVideoPreview, which unmounts the stale progress overlay along with it — so the stuck spinner clears as a side-effect of the auto-switch. Also drops the now-stale @knipignore on getVideoDTOSafe, which has a real consumer now. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(video): add 'Frame from Video' invocation Extracts a single frame from a VideoField input and saves it as a regular ImageDTO via context.images.save, so it appears in the gallery like any other generated image. Primary use case is I2V "shot extension": take the last frame of a Wan-generated clip (default frame_index=-1) and feed it back as the reference image for the next clip, then stitch the MP4s to get videos longer than the model's single-shot frame budget at a given VRAM. Negative frame_index is resolved against the actual decoded frame count via probe_video() rather than passed through to imageio — not all imageio plugins handle index=-1 uniformly, and being explicit lets us emit a precise out-of-range error. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(video): add 'Concatenate Videos' invocation Joins two or more videos into a single MP4 with one of three transition modes between consecutive clips: - cut: hard splice, no blending. Total length = sum of inputs. - crossfade: linear A→B dissolve over transition_frames. Each boundary consumes N frames from both surrounding clips, shrinking total length by N per boundary. - fade_through_black: A fades to black, then B fades in. Each boundary consumes N/2 from each side and emits N output frames — total length is preserved. Implementation decodes via imageio's FFMPEG plugin (matching wan_l2v on the encode side) and runs the blends in numpy. All decoded frames are kept in memory at once; fine for the few-hundred-frame I2V chains that motivated this, would want streaming if anyone ever feeds in hour-long uploads. Up-front validation enforces matching dimensions across inputs and checks that each clip has enough frames to spare from its head and tail for the requested transitions — saves a wasted decode pass when the transition window is too wide for one of the clips. Pairs with 'Frame from Video' for I2V shot extension: generate N clips chained via last-frame-as-ref-image, then glue them with a crossfade. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(videos): show full-resolution first frame in viewer The viewer used a chakra <Image src={thumbnail_url}> in the idle (not- playing) state, so once a clip auto-selected after generation the preview snapped from the full-resolution denoise progress image to the small WebP gallery thumbnail upscaled to fit — visibly soft compared to what the user was watching seconds earlier. Switch to a single <video> element that spans both states: - idle: muted, no controls, preload="metadata". With no `poster` attr the browser decodes and shows the video's actual first frame at full resolution (this is the documented HTMLVideoElement default). - playing: same DOM node with controls+audio toggled on, kicked off via ref.play(). No reload between states — the decoded buffer carries over. `key={videoName}` swaps the element cleanly when the user moves to a different clip, dropping any in-progress playback state. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(nodes): show 'Save in gallery' on video-output nodes The footer checkbox was gated on useNodeHasImageOutput, which only matched ImageField outputs. wan_l2v and video_concat produce VideoField and so had no toggle — users had no UI path to flip is_intermediate on them, even though VideoOutput goes through context.videos.save and lands in the gallery the same way ImageOutput does. Rename the hook to useNodeHasGalleryOutput and extend it to match VideoField as well. Update the three call sites (the hook itself, the checkbox, and the footer wrapper) so the toggle and the footer render whenever a node produces something destined for the gallery. The image primitive ('image' type) is still excluded since it doesn't save a new image; no equivalent video primitive exists yet, so no analogous exclusion is needed for VideoField. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: remove unwanted planning documents * chore: fix ruff I001 import-order violations Auto-fix from `ruff check --select I001 --fix`. Touches 10 files across the Wan and videos changes where added imports landed out of order. No behavior change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(videos): restrict uploads to MP4 only The upload allowlist previously included .mov/.webm/.mkv, but the names service (create_video_name) unconditionally emits {uuid}.mp4 and we don't transcode on upload. The result: non-MP4 containers were stored under a .mp4 name and served with the .mp4 MIME type, which silently broke playback in browsers when the container didn't match. Drop the non-MP4 extensions from ACCEPTED_VIDEO_EXTENSIONS and tighten the accepted MIME prefix to "video/mp4". Wan-generated output is MP4 anyway, so this matches current reality. If we want to support more containers later, the right move is to extend the names service to preserve the source extension, then re-add the formats here. Also drops the now-dead suffix-detection block in upload_video and the os import it required. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(videos): clean up stale @knipignore on consumed hooks useDeleteVideoMutation, useAddVideoToBoardMutation, and useRemoveVideoFromBoardMutation are now consumed by Phase 4 components (context menu, change-board modal) but were still annotated with the multi-phase @knipignore tag — that generated false-positive knip warnings and misrepresented the implementation status. Move those three into the unconditional export block. The remaining five hooks (useListVideosQuery, useGetVideoMetadataQuery, useGetVideoNamesQuery, useDeleteVideosMutation, useChangeVideoIsIntermediateMutation) are still unused in the current codebase and stay under a narrower @knipignore. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(videos): document invalidate/select race in addVideosToGallery The video gallery path uses tag invalidation rather than an optimistic insert (the image path's `insertImageIntoNamesResult` doesn't have a polymorphic equivalent yet). Because invalidation kicks off an async refetch, the `imageSelected` dispatch below it fires before the new video name is in `imageNames`, so the gallery grid's `useKeepSelectedImageInView` no-ops on its first pass. The scroll self-corrects on the next pass when the refetch lands and the `imageNames` dep updates. The user-visible effect is just a small lag on gallery scroll-to- selection — the viewer selection applies immediately — so this is a documented limitation rather than a bug. Worth a follow-up if the lag becomes noticeable in practice. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(video): add Video Primitive invocation and VideoField input UI Mirrors the Image Primitive flow end-to-end for videos. Users can now drag a video from the gallery onto a "Video Primitive" node and feed its output into downstream nodes like Frame from Video or Concatenate Videos — exactly the way Image Primitive feeds the rest of the image pipeline. Backend (invokeai/app/invocations/primitives.py): - New VideoInvocation, declared *after* VideoOutput so the return annotation is a real class (not a forward-ref string). Stringified output annotations crashed startup before — see cac366229a — so the ordering matters. Frontend: - Register VideoField as a stateful field type in types/field.ts: zVideoFieldType, zVideoFieldValue, zVideoFieldInputInstance/Template, output template + type guards, plus entries in the four stateful unions (FieldType, FieldValue, InputInstance, InputTemplate). - buildFieldInputTemplate / buildFieldInputInstance gain VideoField branches so OpenAPI-derived templates resolve correctly. - nodesSlice: fieldVideoValueChanged reducer + export. - imageActions/actions.ts: setNodeVideoFieldVideo helper. - dnd.ts: singleVideoDndSource + setNodeVideoFieldVideoDndTarget, wired into the dndTargets array. - GalleryVideoItem: register itself as a drag source so videos in the gallery actually drag (previously they were click-only). - VideoFieldInputComponent: parallel to ImageFieldInputComponent — shows the WebP thumbnail with a dimensions badge, accepts video DnD, drops stale references on reconnect if the underlying video was deleted. - InputFieldRenderer: dispatch VideoField templates to the new component (placed right after the ImageField branch). - useNodeHasGalleryOutput: also exclude the new `video` primitive type so the "Save in gallery" toggle does not render on the pass-through node (same treatment the `image` primitive already gets). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> fix(video): allow video drops to reach DnD target handlers useDndMonitor is the global drop monitor that actually invokes each target's handler() — DndDropTarget only does enter/leave bookkeeping. Its canMonitor gate explicitly allowlists source types and only listed singleImageDndSource + multipleImageDndSource. So when a video was dragged from the gallery onto a VideoField input, the drop was visible to the DOM but the monitor silently filtered it out, the handler never ran, and fieldVideoValueChanged was never dispatched. Add singleVideoDndSource to the allowlist. Dropping a video onto a Video Primitive (or any other VideoField input) now wires the asset into the field as intended. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(backend): ruff * chore(frontend): typegen * feat(wan): split Wan 2.2 starter bundle into T2V and I2V Replaces the single ~63 GB Wan 2.2 bundle with two smaller bundles so users only pay for the capability they need. T2V (~36 GB) covers text-to-video plus a low-VRAM image-to-video option via TI2V-5B; I2V (~32 GB) adds the heavier I2V-A14B path. Drops the Q8 T2V pair from the default bundle — both Q8 variants and full Diffusers builds remain available as a-la-carte starters. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(videos): tighten multiuser isolation in list and board-move endpoints Three related fixes flagged in code review (PR #9163, JPPhoto): 1. Video and gallery list/name SQL paths only filtered by user_id when board_id was the literal "none" sentinel. When the URL parameter was omitted entirely, no user filter applied and non-admin callers could enumerate every user's videos / mixed gallery items. Added an explicit per-user isolation branch for the omitted case. 2. /v1/videos/ and /v1/videos/names accepted explicit board IDs with no read-access check; the route now mirrors the images and gallery routers and calls _assert_board_read_access for non-"none" values. 3. add_video_to_board and remove_video_from_board only validated video ownership, not destination/source board write access — a caller could move their video into someone else's private board. Added _assert_board_write_access and a strict _assert_video_direct_owner helper (no board-owner / public-board fallback) for board-move ops, mirroring _assert_image_direct_owner. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(boards): cascade video deletion when deleting a board with media Previously delete_board only handled images. With include_images=true the backend would delete images on the board but the videos would silently cascade out of board_videos and survive as uncategorized records — almost certainly not what the caller intended. Without include_images, the same mismatch meant the response could not report affected videos. Now: - include_images=true also calls videos.delete_videos_on_board - include_images=false collects the soon-to-be-uncategorized video names - DeleteBoardResult gains deleted_board_videos and deleted_videos fields (default empty so existing clients are unaffected) Frontend deleteBoard / deleteBoardAndImages mutations gain the matching VideoList / VideoNameList / GalleryItem* tag invalidations so the polymorphic gallery and video list views refresh. Reported in code review (PR #9163, JPPhoto). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(videos): return affected_boards from board move/remove endpoints removeVideoFromBoard previously returned VideoDTO; the frontend then read result.board_id (null after removal) and only invalidated the 'none' board cache — the previous board's list stayed stale until refetch. addVideoToBoard had the same problem (the route never knew the source board, so the old-board cache was never invalidated). Mirror the image equivalents (board_images.py): the routes now return AddVideosToBoardResult / RemoveVideosFromBoardResult with the moved video name(s) and the full set of affected board IDs. Both old and new boards get invalidated atomically. Frontend mutations updated to consume the new shape via getTagsToInvalidateForBoardAffectingMutation on result.affected_boards. The auto-generated schema.ts will need a typegen pass after the dev server restart to pick up the new response types. Reported in code review (PR #9163, JPPhoto). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(videos): stream uploads, FileResponse for full video + thumbnail Three perf items from the code review (PR #9163, JPPhoto): - upload_video read the entire UploadFile into a Python bytes object before writing to the temp file. Multi-GB videos allocated multi-GB buffers. Now chunk-stream into the temp file with a 1 GB per-upload cap (HTTP 413 on overflow). Cap is intentionally generous — the goal is RAM-exhaustion protection, not content policy. - get_video_full read the whole MP4 into RAM when no Range header was present. Browsers usually send Range, but curl / direct downloads / CDN edge fetches do not, and a multi-GB load per such request is a trivial DoS vector. Replaced with FileResponse (sendfile). - get_video_thumbnail similarly buffered the WebP. Thumbnails are tiny so this was minor, but FileResponse is idiomatic and shaves the unnecessary copy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(videos): localize video UI strings - Add gallery.deleteVideo_one / deleteVideo_other, deleteVideoConfirmation, and playVideo to en.json - ContextMenuItemDeleteVideo: drop the inline English defaultValue (the translation key now exists) and use gallery.deleteVideo for aria/tooltip (was reusing gallery.deleteImage so it rendered "Delete Image") - VideoPlayButtonOverlay: replace the hardcoded "Play video" aria with t('gallery.playVideo') Reported in code review (PR #9163, JPPhoto). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(video-invocations): exact frame counts and odd-tf transitions video_frame_extract: resolving frame_index=-1 previously computed n_frames as round(duration * fps). For VFR uploads or containers with approximate metadata that can overshoot the actual decoded frame count, making the last-frame extraction fail. Use iio.improps(plugin='FFMPEG') for the exact decoder count when available; fall back to the duration * fps estimate only if the props query fails. video_concat fade_through_black: with an odd transition_frames the symmetric half = tf // 2 split emitted tf - 1 frames per boundary, violating the documented "emits transition_frames" contract. Split asymmetrically (tail_half = tf // 2, head_half = tf - tail_half) so the emitted count equals tf exactly for both even and odd values. Validation and docstring updated to match. Verified with manual cases: tf=1, tf=4, tf=5 all emit the documented total length for two 10-frame inputs. Reported in code review (PR #9163, JPPhoto). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(gallery): describe video gallery items, upload, and deletion Update the Gallery Panel docs to reflect the polymorphic gallery added in the Wan 2.2 video feature branch: - Gallery intro now mentions images + videos coexist on boards. - Board deletion warning clarified to cover both kinds of media. - New "Videos in the Gallery" section covering: how video items appear (first-frame thumbnail + play badge), MP4-only upload constraint with the typical re-encode command, the video context menu, and that videos count toward board totals. Reported in code review (PR #9163, JPPhoto). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(videos): regression coverage for PR #9163 review fixes Adds tests that pin the behaviour fixed in the JPPhoto review and would catch a recurrence: - tests/app/services/video_records/test_video_records_sqlite.py get_many / get_video_names: non-admin callers only see their own videos when board_id is omitted; admins see all; the "none" branch still filters by user. - tests/app/services/gallery/test_gallery_default.py Same multiuser isolation guarantee through the polymorphic gallery union for both images and videos. - tests/app/routers/test_videos_multiuser.py /v1/videos/ and /v1/videos/names: 403 when a non-owner passes an explicit private board_id; 200 for owners, admins, "none", and omitted board_id (the auth-required smoke tests pin the 401 paths too). - tests/app/routers/test_boards_multiuser.py Adds two delete-board cases proving the video cascade: include_images invokes delete_videos_on_board and reports deleted_videos; the no-include path reports deleted_board_videos without calling the destructive service. Existing fixture extended to stub the video services that the new route logic now touches. - tests/app/invocations/test_video_concat.py Parametric coverage that fade_through_black emits exactly tf frames for both even and odd tf, plus three-clip chains, plus the crossfade and cut/zero-tf cases as guards. - tests/app/invocations/test_video_frame_extract.py _decoder_frame_count returns the exact count via the cv2 fallback for several clip lengths and gracefully returns None for missing / non-video inputs (caller falls back to duration * fps). Bug found during test authoring: _decoder_frame_count over-flowed int() on iio's "inf" nframes for libx264 streams, and improps never returns a real count for that codec anyway. Helper now ignores non-finite shapes and falls back to cv2's CAP_PROP_FRAME_COUNT, which gives the exact value for libx264. schema.ts regenerated to pick up the AddVideosToBoardResult / RemoveVideosFromBoardResult / extended DeleteBoardResult types added in earlier commits in this series. All 70 tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): add 'Wan 2.2 I2V Ideal Dimensions' invocation Computes Wan I2V-compatible (width, height) for a source W×H at a target short-side resolution (e.g. 720 for "720p"), snapping each output to a multiple of 16 (Wan's transformer patch_size × VAE 8x pixel-grid constraint enforced by wan_ref_image_encoder). Replaces the 6-node math chain (Float Math × 4 + Float To Integer × 2) that was otherwise required to compute these dimensions from an arbitrary input image. Wire the Image Primitive's width/height outputs into this node, and feed its (width, height) outputs into both wan_ref_image_encoder and wan_denoise (they must match). Three rounding modes: - nearest (default): minimizes aspect-ratio drift - floor: guaranteed not to exceed unsnapped target (safer for VRAM) - ceiling: rounds up Output schema reuses IdealSizeOutput so it slots into existing pipes that already consume Ideal Size — SD1.5, SDXL. Includes regression tests covering the documented common-case table, all three rounding modes, postcondition invariants (multiple of 16, aspect ratio within 1.2%, never zero), and input validation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): swap target_short_side int for 480p/720p/1080p preset dropdown Wan 2.2 was trained at 480p and 720p; a free integer encouraged users to pick noncanonical short sides that the model handles poorly. Replace the int field with a Literal dropdown of "480p" / "720p" / "1080p" (via ui_choice_labels) so the UI surfaces the canonical choices. 1080p is included with a label noting it's extrapolated from training (not a Wan native size) — useful for users with VRAM headroom but shouldn't be the default. Version bumped to 1.1.0 since the field schema changed (the node was only committed locally; no published workflow needs migrating). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(wan): empty_cache around the I2V reference-image VAE encode Two-sided fix to avoid VRAM allocator fragmentation that was causing the subsequent denoise-transformer partial load to OOM: - Before vae.encode(): clears blocks left over from earlier nodes (the denoise expert swap especially leaves the cache fragmented). - After the condition tensor is on CPU: returns the VAE encode's intermediates so the next partial_load_to_vram sees a real free contiguous range. Mirrors the same pattern in wan_latents_to_image.py and wan_latents_to_video.py — those are the existing precedent. The cost is a handful of microseconds per encoder invocation and only the cache state is touched; model weights stay resident. Observed-by symptom from a workflow review: at encoder=480x720 and a source image of 880x1184, the encoder ran fine but the I2V high-noise expert failed to partial-load with a cryptic CUDA OOM at _load_state_dict_with_fast_device_conversion. Pre-resizing the source to 80% incidentally cleared the allocator state and let the run succeed; this fix removes the incidental dependency on source size. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): support TI2V-5B in the video denoise node (T2V mode) The video denoise node previously hard-errored on TI2V-5B with "not supported." Most of the surrounding machinery (variant-aware spatial scale, variant-aware scheduler, single-expert ExpertSwapper path) was already in place — the gate just needed lifting and the hard-coded A14B latent channel count needed to follow the variant. Changes: - Drop the upfront "TI2V-5B is not supported" raise. - Use get_default_latent_channels(variant) so latents are 48-channel for TI2V-5B and 16-channel for the A14B family (matches the image denoise node's existing logic). - For TI2V-5B with a Reference Image input, raise a sharper, accurate error that explains TI2V-5B's I2V uses diffusers' expand_timesteps path (first-frame-mask blend + per-position timestep gating) which this node does not implement yet — pointing the user at the working T2V path or the I2V-A14B model. - Update the transformer field description to reflect what's now supported. Image-to-video with TI2V-5B remains a follow-up; the conditioning math is genuinely different from A14B (no 36-channel concat) and warrants a separate code path rather than parameterising this one. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(wan): instantiate TI2V-5B VAE with the right architectural config The single-file Wan VAE loader was always calling ``AutoencoderKLWan(z_dim=config.latent_channels)`` and relying on diffusers' constructor defaults for every other parameter — but those defaults match the Wan 2.1 / A14B VAE (base_dim=96, in/out=3, 8x spatial, no patchify). For TI2V-5B's Wan 2.2-VAE the architecture is materially different: - base_dim=160, decoder_base_dim=256 - in_channels=12, out_channels=12 (3 RGB x 2x2 patch) - patch_size=2 - scale_factor_spatial=16 - is_residual=True - 48-vector latents_mean / latents_std (required for the model's encode/decode normalisation to produce non-garbage outputs) Loading the TI2V-5B VAE state_dict into the default-constructed model failed with shape mismatches throughout the encoder + decoder, surfaced in wan_l2v as "Error(s) in loading state_dict for AutoencoderKLWan." This commit routes z_dim=48 to a verbatim copy of the TI2V-5B VAE config (from vae/config.json in Wan-AI/Wan2.2-TI2V-5B-Diffusers); z_dim=16 keeps the previous A14B / Wan 2.1 default behaviour. Verified end-to-end: both kwargs construct cleanly and produce the expected layer shapes (decoder.conv_out emits 12 channels for TI2V-5B, 3 channels for A14B). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(wan): variant-aware default scheduler for standalone installs When the main model has no on-disk ``scheduler/`` directory (every standalone GGUF / single-file install), ``_build_scheduler`` previously fell back to ``FlowMatchEulerDiscreteScheduler()`` for every variant. That's correct for the A14B family but wrong for TI2V-5B, which ships ``UniPCMultistepScheduler`` with ``flow_shift=5.0`` + ``prediction_type="flow_prediction"`` + ``use_flow_sigmas=True``. The mismatch produces drifty samples on TI2V-5B. Add a ``_default_scheduler_for_variant`` helper that reconstructs the right scheduler from the variant tag (values verbatim from each variant's ``scheduler/scheduler_config.json`` in the matching Wan-AI/Wan2.2-*-Diffusers repo). The on-disk-config-present path is unchanged — if the model ships a scheduler dir, that wins. Full scheduler-selection UI is deferred to a future PR per discussion; this special-case keeps the standalone TI2V-5B path producing the right sampler without surfacing a new field. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(wan): TI2V-5B image-to-video support TI2V-5B I2V uses a fundamentally different conditioning scheme from A14B I2V. Implement diffusers' ``expand_timesteps`` path so the same ``Reference Image - Wan 2.2`` node and ``Denoise Video - Wan 2.2`` node work for both variants, dispatched by VAE z_dim / transformer variant. Encoder side (wan_ref_image_extension.py / wan_ref_image_encoder.py) - Add ``encode_reference_image_to_ti2v_condition`` that VAE-encodes a single image frame to ``[1, 48, 1, H/16, W/16]`` with the Wan2.2-VAE normalisation, no mask channels. - ``WanRefImageEncoderInvocation`` dispatches on ``vae.config.z_dim``: z_dim=48 → TI2V-5B path, z_dim=16 → existing A14B path. - Enforce ``multiple_of=32`` for width/height in the TI2V-5B case (16x VAE * 2 transformer patch = pixel dims must divide by 32) with a clear error message pointing at the constraint. Denoise side (wan_video_denoise.py) - Replace the "TI2V-5B I2V not supported" raise with a variant-aware dispatch on ``ref_condition.shape`` and ``variant``. - For TI2V-5B I2V build a ``first_frame_mask`` once (0 at frame 0, 1 elsewhere). At each step: latent_model_input = (1 - mask) * condition + mask * latents temp_ts = (mask[0,0,:,::2,::2] * t).flatten() timestep = temp_ts.unsqueeze(0).expand(B, -1) Per-token timesteps gate the model: frame 0 sees t=0 (locked to condition), other frames see t (normal denoise). - After the denoise loop, re-clamp frame 0 to the clean condition so the locked first frame doesn't show scheduler drift in the final VAE decode. Mirrors WanImageToVideoPipeline:813-814. - Skip the encoder-num_frames-must-match check for TI2V-5B (its condition is always single-frame regardless of output length). Tests - Three new tests on encode_reference_image_to_ti2v_condition covering output shape at small and Wan-realistic dims plus the no-mask-channels invariant. Full video-denoise integration tests would need a new fixture stack (none exist for wan_video_denoise yet) — deferred. A14B I2V is unchanged. TI2V-5B T2V (added in the previous commit) is unchanged. Verified at the import + encoder-shape level; end-to-end verification requires a TI2V-5B I2V workflow. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(videos): show denoise progress overlay over the video viewer CurrentVideoPreview rendered only the <video> element, so when the last-selected gallery item was a video, a freshly-started render's denoise preview images had nowhere to display — the user saw the static first-frame still of the previously-loaded video until the new render's final video swapped in. Mirror CurrentImagePreview's progress-overlay pattern: subscribe to $progressImage / $progressEvent, gate on selectShouldShowProgressInViewer, and render a ProgressImage stack on top of the video when a render is in progress. Hide the play-button overlay while progress is showing so it doesn't sit on top of the preview. Reported by Lincoln during TI2V-5B testing: previews started working after restarting the server only because there was no video loaded at that point; once a video was selected, the previews silently dropped. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(lint): apply ruff format + isort across recent Wan video work ruff check found one I001 (import order) in ``invokeai/backend/model_manager/load/model_loaders/vae.py`` and ruff format flagged five files. All cosmetic; no behaviour changes. - vae.py: import reorder - video_concat.py: minor reflow - test_wan_ideal_dimensions.py / test_boards_multiuser.py / test_videos_multiuser.py: prettier-style wrapping Verified: full ruff check + ruff format --check clean, 141 backend tests pass, and ``pnpm lint`` (knip + dpdm + eslint + prettier + tsc) all green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(features): add user guide for Wan 2.2 video generation Comprehensive guide covering: - The three Wan 2.2 variants (T2V-A14B, I2V-A14B, TI2V-5B), their conditioning differences, and the dual-expert MoE explanation - Lightning LoRA distillation for 4-step A14B inference - Starter bundles (Text-to-Video and Image-to-Video splits) - Workflow setup for T2V and I2V with the constraint matrix: * frame count: (num_frames - 1) % 4 == 0 * pixel dims: multiple of 16 for A14B, 32 for TI2V-5B * encoder + denoise must agree on width/height - The chain-and-concat trick for making longer videos, with the bridge-frame degradation mitigations - Troubleshooting: OOM, late-frame artifacts, dim mismatches, VAE load errors, scheduler issues, preview-not-appearing, MP4 glitches Lands under Features → Video Generation (experimental). Astro auto-generates the sidebar from features/ so no nav config change needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(nodes): skip image-DTO fetch for videos in Current Image node CurrentImageNode unconditionally called useImageDTO(lastSelectedItem) even when the selected gallery item was a video, firing GET /api/v1/images/i/<uuid>.mp4 on every video thumbnail click. The endpoint 404s and the backend logged "Image record not found" each time — benign but noisy. Apply the same null-skip pattern useGalleryItemDTO uses: pass the name only when it's not a video, so RTK Query skips the request for video selections. Current Image is image-only by design, so videos rendering the empty fallback matches existing behaviour. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(videos): clear stale progress overlay + force first-frame paint Two viewer bugs after auto-switching to a freshly-rendered video: - The denoise progress overlay never cleared. CurrentImagePreview clears the ImageViewerContext $progressImage/$progressEvent atoms via DndImage's onLoad callback; the video viewer had no analog, so the last progress still sat on top of the new video forever — clicking other video thumbnails did nothing visible, and only selecting an image (which fires onLoadImage via DndImage) cleared it. - Even with the overlay gone, the <video> element rendered its black background instead of the first frame. preload="metadata" loads dimensions/duration but doesn't guarantee a decoded first frame on all browsers; an explicit seek is needed to force a paint. Wire onLoadedMetadata to (1) call onLoadImage() — mirroring DndImage's onLoad — and (2) nudge currentTime to 0.0001 so the decoder paints the first frame without measurably advancing playback. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(hotkeys): skip image-DTO fetch for videos in GlobalImageHotkeys Companion to a3bdc3304e (CurrentImageNode). GlobalImageHotkeys is a mounted-everywhere singleton that wires recall hotkeys (seed, prompts, remix, etc.) to whatever item is currently selected. It was passing the raw selection name through to useImageDTO unconditionally, so every video thumbnail click fired GET /api/v1/images/i/<uuid>.mp4 → 404 and the "Image record not found" log line. Gate on isVideoName(), mirroring the polymorphic null-skip pattern in useGalleryItemDTO. Recall hotkeys don't apply to videos anyway, so this just suppresses the noise. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(wan): empty CUDA cache between A14B expert swaps The dual-expert swapper releases the active expert via its context manager exit, but PyTorch's caching allocator retains the freed blocks as reserved-not-yet-claimable space until empty_cache runs. The next partial_load_to_vram for the incoming expert then sees a fragmented free pool and offloads layers it could otherwise have kept on device. Users running A14B observed the low-noise expert ending up far more CPU-resident than the high-noise one on otherwise identical settings — that was the leftover reservation from the high-noise expert masking real free VRAM. Call TorchDevice.empty_cache() between the release and the next load. Same pattern as the VAE-encode fix earlier in this branch. Regression test in test_wan_expert_swapper.py mocks empty_cache and asserts it fires on every actual swap but not on a same-label re-get. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(videos): allow drag-and-drop to change a video's board Dropping a video thumbnail onto a board in the boards list was a no-op (the dnd target only accepted image sources). Extend addImageToBoardDndTarget and removeImageFromBoardDndTarget to also accept SingleVideoDndSourceData and dispatch the corresponding video mutations. Permission UX mirrors the image path: - Same canMoveFromSourceBoard gate (owner / public source board) - Same "do nothing if dropping on the current board" early-out Backend enforcement on /api/v1/videos/board already mirrors the image endpoints — _assert_board_write_access on the destination plus _assert_video_direct_owner on the video. The frontend gate intentionally mirrors only the source-board part of that, leaving the direct-owner check to surface as a 403 on attempt (same compromise as images, where the client doesn't have per-item owner info to gate cleanly). Multi-video drag is not supported yet (the gallery only registers a single-video draggable per item, no multi-select bundle), so this only wires the SingleVideoDndSourceData path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(wan): force outgoing A14B expert off GPU on swap The previous empty_cache() fix (53b2f4d4c7) was insufficient. unlock() only decrements the cache record's lock counter — the weights stay on GPU until the cache's automatic offload decides to free them on the next lock(). That heuristic uses ``torch.cuda.memory_allocated() - working_mem`` to estimate free space, which under-frees when the previous denoise step's workspace activations are still allocated alongside the just-unlocked expert. The user-visible symptom was a log line like Loaded model '...:transformer' onto cuda device in 0.37s. Total model size: 9203.13MB, VRAM: 2381.18MB (25.9%) for the incoming low-noise expert, while the high-noise expert continued to hold ~9 GB of VRAM. The swapper now stashes the LoadedModel info handle and, on each swap, explicitly invokes ``cached_model.full_unload_from_vram()`` on the outgoing expert before locking the incoming one. This sidesteps the heuristic and guarantees the previous expert's weights leave GPU before partial_load_to_vram measures available room. The access path ``info._cache_record.cached_model`` reaches into a private attribute — there is no public LoadedModel API for "unload from VRAM but keep in RAM" today, and a broader backend refactor felt out of scope. The call is wrapped in getattr/try-except and pinned by a regression test so a future refactor breaks the test, not the swap. Tests: - Updated existing dual-expert lifecycle test to expect the new full-unload step in the swap log sequence. - New test_outgoing_expert_force_unloaded_from_vram covers the per-swap behavior (outgoing only, no initial unload). - New test_force_unload_failure_does_not_break_swap pins the defensive fallback so swap reliability survives a future LoadedModel refactor. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(gallery): restore shift/ctrl-click range selection in image grid GalleryImage's modifier-key click handler was reading the legacy imagesApi getImageNames cache to compute range-selection indices, but the gallery grid was switched to the polymorphic galleryApi getGalleryItemNames endpoint (the only source that includes videos). The legacy cache is no longer populated for the grid, so the ordered-name list came back empty and the handler fell into its "no names cached" early-return: if (imageNames.length === 0) { if (!shiftKey && !ctrlKey && !metaKey && !altKey) { dispatch(selectionChanged([imageName])); } return; } making shift- and ctrl-click no-ops. GalleryVideoItem already had the correct reader inlined as a private helper. Hoist it to a shared module (features/gallery/store/selectCachedGalleryItemNames) so both grids use the polymorphic cache, and update GalleryImage to call it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(typegen): regenerate schema.ts Refresh of the OpenAPI-derived TypeScript bindings against the current backend. No hand edits — this is the output of the typegen step re-run against the Wan video routes and recent backend changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(startup): silence HF tokenizers fork-after-parallelism warning Set TOKENIZERS_PARALLELISM=false at startup (via os.environ.setdefault so users can override) before any HF library is imported. The Rust ``tokenizers`` library warms a thread pool the first time a tokenizer runs — for us that's UMT5 / T5 text encoding during Wan / FLUX / SD3 conditioning. Every subsequent fork() then logs huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks... In video generation we fork on every MP4 encode (imageio's FFMPEG plugin uses subprocess.Popen → fork+exec), so this warning lands once per generation in the server log. The advisory is benign — the child correctly falls back to single-threaded tokenization before exec(), and the parent's thread pool is unaffected — but the noise obscures real warnings. Setting the env var before any HF import prevents the thread pool from warming up at all, so the fork detector stays quiet without sacrificing anything: tokenization happens once per generation and isn't a hot path for us. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(startup): hoist TOKENIZERS_PARALLELISM=false to module level Follow-up to 2106f10ec4 — the previous attempt set the env var inside ``run_app()``, which races against any transitive HF import triggered by the console-script's from invokeai.app.run_app import run_app If ``tokenizers`` is imported anywhere in that import chain (directly or via diffusers/transformers re-exports), the library's fork detector registers before our setdefault runs and the warning still fires. Move the setdefault to module level so it executes the instant ``run_app.py`` is loaded — i.e. before the function defs are even parsed, and well before any HF library has a chance to import. Note for testing: jurigged hot-reload only re-runs function bodies, so picking up this fix requires a full server restart, not just a file save under ``--dev-reload``. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(videos): replace window.confirm with ConfirmationAlertDialog Use the in-app delete-confirmation dialog (the same Chakra ConfirmationAlertDialog the image flow uses) instead of the browser's window.confirm() prompt. Matches the visual + interaction language of the rest of the gallery and picks up the shared ``shouldConfirmOnDelete`` system preference — flipping the "Don't ask me again" toggle now silences the prompt for both images and videos. Implementation mirrors features/deleteImageModal/ but trimmed: the image dialog computes "usage" (canvas layers, node fields, reference images, upscale source) so the user knows what they'll break. Videos have no analogous attachment points, so the video state machine is a straight confirm-then-delete with no usage analysis. - features/deleteVideoModal/store/state.ts — nanostores atom + an awaitable ``deleteVideosWithDialog`` that opens the dialog and resolves/rejects on confirm/cancel. Skips the dialog entirely when shouldConfirmOnDelete is off. - features/deleteVideoModal/components/DeleteVideoModal.tsx — ConfirmationAlertDialog with the new deleteVideoPermanent message and the shared "Don't ask me again" switch. - GlobalModalIsolator.tsx — mount the new modal alongside DeleteImageModal. - ContextMenuItemDeleteVideo.tsx — call useDeleteVideoModalApi().delete instead of window.confirm + useDeleteVideoMutation. - en.json — added gallery.deleteVideoPermanent, dropped the now-unused gallery.deleteVideoConfirmation. - videos.ts — useDeleteVideoMutation moves into the @knipignore export group since the only call site now uses videosApi.endpoints.deleteVideo.initiate via the modal. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(gallery): refetch polymorphic gallery cache on image completion The gallery grid subscribes to the polymorphic ``getGalleryItemNames`` RTK Query endpoint (so images and videos interleave by created_at). But ``onInvocationComplete``'s image path only did an optimistic insert into the image-only ``getImageNames`` cache, leaving the polymorphic cache stale — a freshly-generated image landed correctly in board totals and the per-DTO cache, but never showed up in the grid until the user reloaded the page. Mirror the videos path (which has invalidated these tags since the polymorphic endpoint was introduced) and dispatch ``galleryApi.util.invalidateTags(['GalleryItemNameList', 'GalleryItemList'])`` after image outputs are processed. The cost is one extra HTTP round-trip per generation; a future optimization could optimistically splice the new entry into the polymorphic shape, but that requires a different ``insertImageIntoNamesResult`` for the ``GetGalleryItemNamesResult`` shape and is a bigger change. Regression test in onInvocationComplete.test.ts pins the behavior: verifies the invalidation fires on a fake image complete event, and verifies it does NOT fire for denylisted passthrough node types (load_image, image). Confirmed test correctly fails when the fix is reverted via git stash. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: address 2nd-pass code review findings Self-review pass before re-pinging external reviewers. Five fixes; the three medium ones have user-visible consequences, the two low ones are guard + docstring. 1. videos.py: delete_video no longer swallows service errors into a misleading HTTP 200. Missing DTO -> 404, delete failure -> 500. The prior shape returned 200 with an empty deleted_videos list, which the frontend treated as success, dropped from cache, and left the video on disk — silent data-consistency failure visible only on next page reload. 2. videos.ts: starVideos / unstarVideos invalidate the LIST_TAG-scoped { type: 'VideoList' } entry alongside the per-video and board-affecting tags. Without this, starred_first=true gallery queries kept the just-starred video in its old position until the next list-affecting mutation. Mirrors the delete + upload pattern. 3. wan_denoise.py: _ExpertSwapper.get() stashes _active_device_ctx right after device_ctx.__enter__() succeeds, before attempting the LoRA patcher's __enter__. If the LoRA enter raises, _release() can now actually find the device context and exit it — previously the ctx was unreachable and 8-9 GB of GGUF expert weights stayed pinned to GPU until the model cache LRU evicted them. 4. wan_ideal_dimensions.py: reject sources whose longer side is below the 16-px Wan grid. The downstream max(w, 16) clamp would otherwise silently disconnect the output from the requested aspect ratio (returning 16×16 regardless of the source's actual shape). 6. wan_video_denoise.py: docstring now explains the deliberate absence of denoising_start / denoising_end / initial-latents inputs (video i2v uses reference-frame conditioning, not noise injection; the image denoise node still handles still-image img2img). Tests: - test_device_context_released_when_lora_enter_raises pins #3. - test_input_smaller_than_pixel_grid_rejected pins #4. - test_output_dims_never_zero renamed to test_smallest_valid_input_still_snaps_to_16_grid (now exercises 16×16 rather than 8×8 since the latter is now correctly rejected). All 58 affected backend tests pass, frontend lint clean. Audit note for the PR description (NOT a fix): delete_video's _assert_video_owner permits write access on public boards (mirroring the image router's _assert_image_owner — intentional symmetry). The stricter _assert_video_direct_owner is reserved for board-move ops. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(frontend): typegen * fix(gallery): multi-select context menu actions for videos The video gallery context menu only operated on the single right-clicked item, so selecting multiple videos and hitting the trash icon deleted just the first one. Adds a video-side multi-selection menu mirroring the image one for star/unstar/download/change-board/delete, switched in on selectionCount > 1. Each menu now filters the polymorphic selection to its own kind and labels the action with an explicit count + kind (e.g. "Delete 3 Videos", "Move 2 Images to Board"). The destructive items disable when the kind-filtered subset is empty, so a video-only selection greys out the image menu and vice versa. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(multiuser): address Pfannkuchensack PR #9163 review findings Finding 1 (Medium): delete_board cascade ignored per-video / per-image ownership, letting a board owner destroy other users' contributions to a public/shared board just by deleting the board with include_images=true. Adds user_id filtering through get_all_board_*_names_for_board and delete_*_on_board (base + sqlite + image wrapper). Non-admin requests pass the requester's id so the SQL WHERE clause narrows the cascade to that user's rows; admins still pass None for the unrestricted path. Other users' content cascades to "uncategorized" via the existing FK on board_videos / board_images. Finding 2 (Low, i18n): GalleryItemStarIconButton and GalleryItemVideoStarIconButton shipped raw English "Star"/"Unstar" tooltips. Both now use the gallery.starImage / starVideo translation keys. Finding 3 (Low): delete_videos_from_list and delete_images_from_list re-raised HTTPException mid-loop, throwing away the response payload for items already deleted before the foreign name was hit. The frontend cache never learned about those partial successes, so deleted records reappeared in the UI until the next manual refresh. Both routes now skip auth-failed items in-loop and return 200 with the partial-success list. Residual: adds a test that an upload with an .mp4 extension but non-decodable bytes (a) reaches probe_video, (b) surfaces 415, (c) unlinks the streamed-to-disk temp file so the server doesn't leak storage on garbage uploads. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(openapi): regenerate openapi.json The committed schema was stale relative to the current server (missing the utilities/expand-prompt and utilities/image-to-prompt endpoints, the ModelRecordOrderBy / SQLiteDirection list params, and the Wan / QwenImage / QwenVLEncoder config variants this branch adds). Regenerated via the same command the new openapi-checks workflow uses so the diff CI is empty. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(workflows): add "Image to Video - Add Frames" starter workflow Extends an existing video by extracting its penultimate frame, running it through Wan 2.2 I2V A14B + the Lightning LoRA pair to generate a new clip, and concatenating the result onto the source with a short crossfade. Cleaned per the default-workflows README: stripped value references on the four model loader fields and both Lightning LoRA fields so the workflow ships without keys/hashes for user-installed resources, gave the LoRA nodes "Apply LoRA (High)" / "(Low)" labels matching the existing Lightning default, remapped six stale exposedFields entries that pointed to template LoRA IDs no longer present in the graph, and synced the wan_video_denoise num_frames default to the value driven by the connected integer node. Tagged with both Text to Video and Image to Video so it surfaces under either filter in the Workflow Library. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(gallery): video viewer polish and selection regressions - Restore auto-select-on-startup and on-board-switch: the polymorphic getGalleryItemNames endpoint replaced getImageNames as the grid's source of truth, so appStarted and boardIdSelected now wait on / read that cache instead of timing out forever. - Delete-then-select: video delete used to clear selection to null; image delete read a cache that's no longer warmed. Both now snapshot the gallery list before deletion and advance to the adjacent surviving item (prev > next > null) via a shared pickSelectionAfterDelete helper. - Video Viewer: right-aligned action bar with Open in new tab, Copy frame, Download, Delete, and a labelled Close video player button that only appears while playback is active. Copy uses canvas + ClipboardItem since video MIME types aren't supported cross-browser. - Next/prev arrows + galleryNav hotkeys now work when a video is in the Viewer (previously image-only). - Video context menu uses full-width text MenuItems instead of the cramped icon group, and gains an Open in new tab entry. * fix(gallery): bulk video drag-to-board and shift-click range selection - Bulk video drag: introduced multipleVideoDndSource so a multi-selection dragged from a video thumbnail moves every selected video, not just the first. The whitelist in useDndMonitor.ts also needed updating — without it the monitor's canMonitor gate silently dropped the new source type. - Mixed selections: both the multi-image and multi-video drag payloads now carry image_names + video_names side-by-side, so dragging from either kind in a mixed selection dispatches addImagesToBoard + addVideosToBoard together. Previously the image side leaked video names into image_names and the image router 404'd on each one. - Bulk video helpers: added addVideosToBoard / removeVideosFromBoard that fan out over the existing singular video router endpoint (no batch endpoint exists yet) — mirrors the change-board modal's existing loop. - Shift-click range selection: selectCachedGalleryItemNames now looks up the cache entry matching the gallery's current query args instead of taking the first entry from selectInvalidatedBy. RTK Query keeps unused entries warm for 60s after a board switch, and the old "first wins" behavior frequently landed on a stale board's name list, making shift-click silently no-op until a delete/move forced a refetch. * fix(scripts): force generate_openapi_schema.py to resolve invokeai from the repo root When the script was invoked as ``python scripts/generate_openapi_schema.py``, Python placed the script's directory at ``sys.path[0]`` rather than the repo root. ``import invokeai`` then resolved via the venv's site-packages, which on multi-worktree editable installs ends up importing ``invokeai`` as a PEP 420 namespace package that aggregates every worktree's ``invokeai/`` directory. Side-effect imports driven by submodule discovery silently miss whichever worktree isn't first on the namespace path, so the registry came up short by the invocations declared only in this worktree (the wan/video set, 15 classes). Running the same imports via ``python -c`` worked because ``sys.path[0]`` defaulted to the cwd and ``invokeai/__init__.py`` resolved cleanly to the worktree. Prepend the resolved repo root to ``sys.path`` before importing ``invokeai.*`` so the script always picks up the local sources regardless of how it was launched. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(video): add Frame Range from Video invocation with scrubbable preview New ``extract_video_range`` node trims a source video to a contiguous frame range and re-encodes it as MP4, slotting in naturally between a generated clip and Concatenate Videos for I2V chain shaping. Bounds are inclusive and support negative indices (``end_frame=-1`` keeps the final frame), matching Frame from Video. Output fps inherits from the input unless overridden. The node renders a per-type preview inside the workflow editor: two ``<video>`` tiles side by side, each driven by a CompositeSlider that scrubs the corresponding integer field. The tile uses ``currentTime = frame / fps`` so browsers display the seeked frame natively without a canvas roundtrip. Negative-index entries in the standard integer input are resolved against the source frame count for display only; the underlying field value is preserved verbatim. The custom UI is wired in via a ``CustomNodeBody`` dispatcher in ``InvocationNode.tsx`` rather than a registry — small enough to be explicit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(video): emit resolved frame indices and move preview to per-field renderer Three changes to the ``extract_video_range`` invocation: 1. New ``ExtractVideoRangeOutput`` mirrors ``VideoOutput`` and additionally emits the resolved (positive, 0-based) ``start_frame`` and ``end_frame`` indices. Chained workflows can feed those back into a downstream Frame from Video to extract the same boundary frame the trim landed on. 2. ``fps`` is now a plain ``int`` defaulting to 16 (was ``Optional[int]`` with an "inherit from input" fallback). Matches the default used by wan_l2v and the other Wan video producers, so chained workflows agree on framerate without each node guessing. 3. The frame preview is now a per-field widget driven by a new ``UIComponent.VideoFrameIndex`` hint. ``start_frame`` and ``end_frame`` are tagged with it; the new ``VideoFrameIndexFieldInput`` renders a number input plus a live <video> thumbnail and a scrubber slider, all writing to the same Redux field. Negative indices entered in the number input are still resolved against the source frame count for display only — the backend re-resolves at invoke time. The widget reads its companion ``VideoField`` (by convention, the sibling field named ``video`` on the same node) via direct Redux selectors, so it works wherever ``InputFieldRenderer`` is used — the workflow editor's node body AND the Form Builder's view/edit modes. The previous node-body ``ExtractVideoRangePreview`` and its ``CustomNodeBody`` dispatcher in ``InvocationNode.tsx`` are removed; the per-field widget supersedes both. In the workflow editor, side-by-side framing is lost in exchange for Form Builder support; users wanting the side-by-side layout in a form can group the two frame fields in a row container. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: ruff * fix(video): address PR #9163 review follow-ups - delete_board: include_images query description and OpenAPI schema now mention videos alongside images - get_video_thumbnail: check path existence before returning FileResponse so a missing thumbnail produces the documented 404 instead of an after-route error - delete_videos_on_board: stop deleting records for videos whose files failed to delete, so a transient FS error no longer orphans the file with no record pointing at it - DeleteBoardModal: destructive button and warning copy now mention videos * fix(video): address PR #9163 May-22 review and failing CI - remove_video_from_board now accepts either the direct video owner or a board write-access holder, so videos uploaded to a board that later flipped Public -> Shared/Private aren't stranded. - VideoService.create rolls back the DB record and board association if the underlying file save fails, preventing ghost records whose file endpoints 404. - delete_videos_on_board returns the actually-deleted names; delete_board uses that list so the response can't claim a video was destroyed when its record was preserved due to a file-delete failure. - Local test_videos_multiuser fixture now patches invokeai.app.api.routers._access so list/names route 403 checks work. - Regenerate schema.ts to pick up the CacheStats description. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(frontend): rebuild openapi * fix(video): register Viewer <video> as drag source Drag-and-drop from the Viewer pane now produces the same singleVideoDndSource (and multipleVideoDndSource for active multi-selection) as the gallery thumbnail, so a video can be dropped onto a Video Primitive's "Starting Video" field directly from the Viewer. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(video): add frame preview to Frame from Video node Tag frame_index with ui_component=VideoFrameIndex so the node renders the same live frame thumbnail + scrubber as Frame Range from Video. The widget keys off the sibling 'video' field, which this node already has, so no frontend changes are needed. Bump node version 1.0.0 -> 1.1.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(video): derive Frame Range fps from source video by default Make the fps field optional (default None). When unset, the output frame rate is inherited from the probed source video so a trimmed clip plays back at the same speed as its source, falling back to 16 fps when the source rate can't be probed. An explicit fps still overrides. Bump node version 1.0.0 -> 1.1.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(video): use fps=0 sentinel for source-derived Frame Range rate The previous Optional[int]/None design had no natural way to express 'unset' in the node's number input, and the ge=1 constraint rejected the intuitive fps=0 with a validation error. Make fps a plain int defaulting to 0, allow ge=0, and treat 0 as 'match the source video's frame rate'. Keeps in-progress workflows (already saved with fps=0) working without a version bump. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(wan): add Wan 2.2 TI2V Ideal Dimensions node TI2V-5B uses the 16x Wan 2.2-VAE plus a 2x transformer patch, so pixel dims must be multiples of 32 (the existing I2V node snaps to 16, which the TI2V-5B patchify step rejects). Add a wan_ti2v_ideal_dimensions node that snaps to 32. Factor the shared scale-and-snap math into _scale_and_snap(multiple=...) so both nodes derive from one implementation; the I2V node is unchanged behaviorally (its existing tests still pass). Add a mirrored TI2V test suite. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(wan): add A14B/5B model hints to ideal-dimensions node titles Suffix the node titles with the target model family (A14B / 5B) so they're distinguishable in the add-node search and node header, and rewrite both docstrings to lead with which Wan 2.2 model they're for and cross-reference the other node. Purely UI metadata — no behavior or schema change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(workflows): replace bundled Wan 2.2 video workflows with curated set Remove the 6 previously-bundled Wan 2.2 *video* default workflows (Text to Video, Text to Video Lightning x2, Image to Video, Image to Video Lightning, Image to Video - Add Frames) and replace them with the 8 curated starter workflows: Text/Image to Video Lightning (+ Concept LoRA variants), Extend Video Lightning (+ Concept LoRA variant), and the TI2V-5B text/image-to-video low-quality variants. Each is assigned a stable default_ id and meta.category=default. Model fields are intentionally blanked (per-install keys don't resolve cross-instance) with the required models listed in each workflow's Notes. The two Wan 2.2 *image* workflows (Image to Image, Text to Image) are retained. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(video): add beginner Video Workflows guide for the 8 starter workflows New Features-section page (sibling to Video Generation) describing the eight bundled Wan 2.2 video workflows in plain language: how to choose between the Text/Image/Extend families and their Lightning / Concept-LoRA / TI2V-5B variants, how to select models from each workflow's Notes, how to run one, and a quick per-GPU guide. Cross-linked both ways with the Video Generation technical reference. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(video): fix Concept LoRA slot guidance (slots are required) The w/ Concept LoRAs workflows wire required lora fields (lora_selector / wan_lora_loader, no default) into the graph, so an empty slot blocks invocation. Correct the earlier claim that empty concept slots behave like the base workflow: every LoRA slot must be filled, and users without concept LoRAs should use the plain variant. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(video): drop '(experimental)' from Video Generation title Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: refresh uv.lock Routine lock refresh (transitive dev deps: docutils, idna, platformdirs, python_discovery, tornado). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(wan): add first-last-frame interpolation (FLF2V) to I2V-A14B The Reference Image - Wan 2.2 node gains an optional End Image input: when set, encode_reference_image_to_video_condition places the end image in the final temporal slot and anchors the mask at both the first and last latent frames, so I2V-A14B interpolates a clip from the start image to the end image. Mirrors diffusers WanImageToVideoPipeline.prepare_latents with last_image set. The denoise loop is unchanged - for A14B it just concatenates the 20-channel condition, which is agnostic to one vs two anchors. FLF2V is A14B video only (num_frames > 1); the encoder raises a clear error for TI2V-5B or single-frame. Bump wan_ref_image_encoder to 1.2.0; add mask-anchoring unit tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(workflows): add 'Interpolate 2 Images to Video' starter + FLF2V docs Ship a default workflow that wires the new FLF2V End Image input end to end (I2V-A14B + Lightning, two image inputs interpolated). Model fields blanked with the required models listed in Notes, default_ id + category=default. Document FLF2V in the Video Generation reference and add the workflow to the Video Workflows guide. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(workflows): add Text/Image/Video to Video library filters + fix video tags Add 'Text to Video', 'Image to Video', and 'Video to Video' to the Common Tasks filter list in the Workflow Library browser. Fix the tags on the nine bundled Wan 2.2 video workflows, which were all copy-pasted as 'text to video': - Text to Video: the three T2V workflows - Image to Video: the I2V workflows + Interpolate (two-image) - Video to Video: the two Extend Video workflows The TI2V-5B variants also drop the spurious lightning/lora tags (they have no LoRAs). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(workflows): add 'Extend Video to Image' FLF2V starter + docs Ship a default workflow that extends a video toward a user-provided target image: the new segment interpolates (FLF2V) from the source video's last frame to the destination image, then concatenates onto the original with a cross-fade. Model fields blanked, default_ id + category=default, tagged 'video to video'. Document it (card + usage instructions) in the Video Workflows guide. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(video): reorganize Video Workflows guide sections Group the Interpolate section and the Concept-LoRA / TI2V-5B asides with the image workflows, keep the Extend family (including Extend Video to Image) at the end, and retitle the section to 'Bundled video workflows'. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(frontend): regenerate openapi and typegen * chore(backend): ruff * fix(future): make the WAN LoRA loader compatible with LoRA picker node PR #9259 * chore(frontend): remove unused selectT5EncoderModels import * chore(frontend): remove unused export * fix(gallery): count videos and pick video covers for board tiles Gallery boards previously joined only the `images` table for their headline count and cover thumbnail, so a board containing nothing but videos rendered as empty with no preview. BoardDTO now exposes `video_count` and an optional `cover_video_name`; the boards service picks the best cover across both tables using the same (starred DESC, created_at DESC) tie-break the image path already used, and the gallery list renders `image_count + video_count` everywhere it previously rendered just images (real boards, no-board pseudo-board, and the tooltip). Adds `getBoardVideosTotal` to round out the no-board counts (the BoardVideosTotal tag was already wired into invalidation). * test(boards): wire video record storage into multiuser test fixtures After the board cover/count fix started reading from `video_records` and `board_video_records`, the multiuser test fixtures that still set both to `None` started erroring out — the boards router's catch-all turned the AttributeError into a 404, cascading through every test that PATCHes or GETs a board (auth, workflows, data-isolation suites). Swap the `None` placeholders for real SqliteVideoRecordStorage / SqliteBoardVideoRecordStorage instances (paralleling the existing image storage setup), and pin sane defaults on the MagicMocks in `test_videos_multiuser.py` so the get_dto cover/count lookups don't trip Pydantic validation. * fix(ui): widen useVideoContextMenu ref type to allow null The ref param was typed RefObject<HTMLElement>, but useRef produces RefObject<HTMLElement | null>, breaking lint:tsc in GalleryVideoItem. Match the sibling useImageContextMenu signature. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(backend): ruff * fix(tests): pass video/gallery services to InvocationServices in workflow-call router tests The workflow-call router tests from main construct InvocationServices directly and predate the video/gallery services added on this branch, so every test in the file errored with missing positional arguments. Mirror tests/conftest.py: real sqlite stores for video_records and board_video_records, None for the services the tests never touch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): refresh board caches when a generated video completes Video completion previously invalidated only the polymorphic gallery list tags, so the new video appeared in the grid while the board's video_count, cover thumbnail (Board tag / listAllBoards), and BoardVideosTotal stayed stale until an unrelated mutation refetched them. Use the shared getTagsToInvalidateForBoardAffectingMutation helper over the affected boards, matching the video mutation endpoints. Reported by @JPPhoto in PR #9163 review. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: include videos in date-based virtual boards Date virtual boards were image-only even though the gallery grid is now polymorphic: video-only dates never appeared, and mixed dates omitted videos from counts/contents/covers. - SqliteGalleryService owns virtual-board dates now: get_dates() unions images+videos per date (video_count added to VirtualSubBoardDTO, cover is the newest item of either kind via cover_image_name/cover_video_name), and list_item_names() gained a created_date filter. - New GET /api/v1/virtual_boards/by_date/{date}/item_names returns the same polymorphic (kind, name) refs as the gallery names endpoint; the legacy image_names route is kept for API compatibility. - Frontend virtual-board selection consumes the new endpoint, so videos show up in virtual date boards; VirtualBoardItem shows video counts (localized tooltip) and falls back to the video thumbnail for video covers. - tests/conftest.py wires a real SqliteGalleryService so router tests exercise the filter SQL; service + router tests cover video-only dates, mixed dates, cover selection, and per-user isolation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): don't advance gallery selection for videos whose delete failed handleDeletions treated every requested video as deleted when picking the post-delete selection, so a 403/500 on deleteVideo could jump the Viewer away from a video that still exists, and a surviving neighbour was skipped as a replacement candidate. Only successfully deleted names now count: a failed displayed video keeps its selection, and failed neighbours remain valid replacements. Covered by state.test.ts with rejected deleteVideo dispatches. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): count uncategorized videos when deciding the gallery has content useHasImages only looked at boards and the uncategorized *image* total, so a gallery whose only content was an uncategorized video rendered the new-user/get-started view instead of the normal no-selection state. The hook now also reads the uncategorized video total (getBoardVideosTotal('none')); the decision logic is extracted as getHasGalleryContent and unit-tested. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): stop labeling board video counts as images in tooltips Board tooltips folded video_count into image_count and rendered boards.imagesWithCount, so a video-only board read e.g. '1 image, 0 assets'. Tooltips now show split image/video/asset counts using the new boards.videosWithCount translation; the compact unlabeled headline count in the boards list stays combined so video-only boards don't read as empty. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): restrict client-side video upload acceptance to MP4 only The dropzone accept map advertised .webm/.mov and isVideoFile treated .webm/.mov/.mkv as videos, but the upload router accepts MP4 only, so those files were accepted client-side and then rejected with 415 after the bytes were uploaded. Consolidate the accepted-media lists into common/util/uploadMediaAccept.ts (single source of truth shared by useImageUploadButton and FullscreenDropzone) and pin them to the backend contract with a regression test. Also split the accept map: image-only upload fields (board covers, style presets, model images, workflow thumbnails) no longer advertise video/mp4, which they had inherited when video entries were added to the shared map. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): bulk video star/unstar returns partial successes instead of 403 mid-batch star_videos_in_list and unstar_videos_in_list re-raised the ownership HTTPException mid-loop, so a batch containing one foreign (or stale) name mutated the earlier owned videos and then returned 403 with no payload — the client never invalidated caches for the videos that did change. Skip unauthorized names and return 200 with the actually starred/unstarred videos, mirroring delete_videos_from_list. Router tests cover the mixed-ownership batch for both routes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): localize virtual board section header and toggle The 'By Date' header and the Collapse/Expand aria-label in VirtualBoardSection were hardcoded English. Add boards.byDate and common.collapse/common.expand translation keys and use them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(app): failed video saves no longer orphan files on disk DiskVideoFileStorage.save() moves the source MP4 into permanent storage before writing the thumbnail and sidecar, so a failure in either later step used to leave the moved MP4 (and partial artifacts) on disk with no DB record through which they could be managed. save() now removes its destination files before raising, and VideoService.create()'s rollback also deletes files to cover failures after a successful file save (e.g. building the DTO). Also documents why board attachment during create is best-effort (mirroring ImageService.create: a board deleted mid-generation must not destroy the render) and pins the explicit fallback — DTO reports the actual missing board association and a warning is logged — with a service test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(app): document videos.user_id lifecycle and pin user-deletion behavior videos.user_id deliberately has no FK to users, matching images/boards/ workflows (migration_27 adds those user_id columns index-only): deleting a user leaves their media in place for admin review/cleanup rather than cascading a row delete that would strand files on disk. A migration comment now states the parallel, and a migration-backed test creates a user and a video, deletes the user, and asserts the record survives, stays attributed to the deleted owner, and remains visible only to admins. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(app): support VideoField values in session queue batch data Batch.data previously allowed ImageField but not VideoField, so submitting multiple VideoField values through the generic batching capability failed Pydantic validation before enqueueing. VideoField now joins the BatchScalarDataType union; a test asserts a VideoField batch validates and expands into separate sessions. schema.ts/openapi.json regenerated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): video uploads are opt-in per consumer; upload validation fixes - useImageUploadButton gains an allowVideos opt-in (default off). Only the gallery uploader accepts videos; image-only consumers (ref images, board covers, launchpad buttons, image-to-prompt, etc.) no longer let a selected MP4 upload into the gallery while the requested image action goes nowhere. Videos are excluded from their accept map and rejected at runtime if the file dialog bypasses it, with tests via partitionUploadFiles. - The hook's loading state now covers both the image and video mutations, so an in-flight MP4 upload shows a loading button and blocks resubmission. - The fullscreen drag-drop/paste validator accepts a file when either its MIME type or its extension is recognized — a clip.mp4 with an empty File.type used to be rejected even though the backend accepts it. The validator moved to a pure module with tests. - Failed video uploads no longer toast "Image Upload Failed": video-only batches use a new toast.videoUploadFailed key, mixed batches the neutral toast.uploadFailed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(app): bound untrusted video decoding with a killable subprocess timeout probe_video / extract_video_frame / decoder_frame_count now run in a short-lived child process (video_decode_worker.py) killed after a hard timeout. Previously a crafted MP4 that failed the imageio probe and then hung inside cv2.VideoCapture()/read() would pin the FastAPI request worker that called it forever; repeated uploads could exhaust the pool. The worker is run by file path (not -m) and imports only imageio/PIL/cv2 so it starts without pulling in the invokeai package or torch. Tests substitute a never-returning worker command and assert the helpers fail within a bounded interval, plus happy-path tests against a real synthetic MP4 to validate the subprocess plumbing end to end. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(app): stream frames through the video concat/trim nodes video_concat and extract_video_range fully decoded their inputs into lists of uncompressed frames before encoding; with the 1 GB upload cap a long 1080p source can expand to tens of gigabytes of RAM, so any user able to enqueue these nodes could exhaust server memory. Frames now stream from the decoder straight into an incremental FFMPEG writer. The concat node buffers only the transition windows (bounded by transition_frames), and the range node holds one frame at a time and stops decoding at the end of the requested range. Tests use lazy frame iterators to pin that encoding begins before the inputs are exhausted and that look-ahead stays bounded. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): video upload feedback parity with images - Add uploadVideo matchFulfilled/matchRejected listeners mirroring the image upload listeners: success toasts name the destination board and navigate the gallery on the first upload of a batch; failure toasts name the failed file, which is what makes partially failed Promise.allSettled batches attributable (uploadVideos and the fullscreen dropzone aggregate without rethrowing, same as images). - GalleryUploadButton now uses the hook's combined isUploading, so an in-flight MP4 shows a spinner and blocks resubmission. - Media-neutral labels on the two video-enabled surfaces: gallery uploader aria/tooltip says Upload Media, the fullscreen overlay says uploaded items (not images) will be added, and its invalid-file toast mentions MP4. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: include videos in the user-deletion data-loss note The admin guide's user-deletion warning enumerated boards, images, workflows, queue items, and style presets but not videos. State that video records survive with the deleted user_id, that files remain under outputs/videos, and that administrators keep gallery visibility of the orphaned media for review/cleanup — matching the behavior pinned by the video_records user-deletion lifecycle test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix video processing resource bounds * fix video lifecycle edge cases * stabilize decoder inactivity test * chore(deps): declare psutil as a direct dependency video_thumbnails.py now imports psutil for decode-worker process-tree termination, but it was only present transitively (via accelerate and friends). Declare it so the import can't silently break when an upstream package drops it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix video security and lifecycle regressions * chore: regenerate OpenAPI schema * fix: preserve exact frame dimensions in video encoders imageio's FFMPEG writer defaults to macro_block_size=16, which makes ffmpeg silently rescale frames to the next multiple of 16 — a 1920x1080 upload trimmed by Frame Range from Video came back as 1920x1088 while the DTO recorded 1080, so concatenating the trim with its own source failed the same-dimensions check. - New invokeai/app/util/video_encoding.make_mp4_writer single-sources the encoder settings (libx264, macro_block_size=1) for wan_latents_to_video, video_concat, and video_frame_extract_range. - yuv420p requires even dimensions, so concat and extract-range now reject odd-dimension sources up front with a clear error instead of an opaque ffmpeg failure mid-encode. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: correct A14B fallback scheduler and stop LoRA leakage to low-noise expert Two silent-wrong-output bugs on the GGUF A14B path: - The no-scheduler-dir fallback returned FlowMatchEulerDiscreteScheduler for A14B, but both Wan-AI/Wan2.2-{T2V,I2V}-A14B-Diffusers repos ship UniPCMultistepScheduler with flow_shift=3.0 / flow_prediction / use_flow_sigmas (verified against the upstream scheduler_config.json). Every A14B GGUF render ran an unshifted first-order Euler schedule, degrading output and skewing how many steps land above the MoE boundary. An unreadable on-disk config now also falls back to the variant default instead of bare FlowMatchEuler. - low_loras fell back to the primary list when loras_low_noise was empty, but the Wan LoRA loader deliberately routes expert-tagged LoRAs to exactly one list — so a high-noise-only LoRA (e.g. a Lightning high-noise distill) was silently applied to the low-noise expert too, and high-only targeting was impossible. An empty low list now means no LoRAs on the low expert. Also (here and in the previous commit): the Wan VAE decode nodes now raise a clear latent-channel mismatch error (16-channel A14B latents vs 48-channel TI2V-5B VAE and vice versa) instead of an opaque tensor-size RuntimeError when the wrong VAE is selected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: keep viewer selection on surviving item after image deletion The image-side handleDeletions cleared the gallery selection (imageSelected(null)) whenever the deletion intersected the multi-selection but the displayed item was not among the deleted names — e.g. a video displayed while only images were deleted from a mixed selection, or a hover-delete of a non-displayed selected image. It also treated every requested name as deleted, ignoring the server's deleted_images response, so a partial failure could jump the selection away from an image that still exists. Port the deleteVideoModal logic: only server-confirmed deletions count, a surviving displayed item stays selected, and the usage-reset sweep (nodes/canvas/ref-image layers) runs only for actually-deleted images. Regression tests mirror deleteVideoModal/store/state.test.ts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * perf: ~48x faster Wan VAE decode on ROCm via conv2d decomposition MIOpen has no implicit-GEMM 3D-convolution kernels for the Wan VAE's shapes on RDNA3 and falls back to Im3d2Col (61% of decode GPU time in a torch profile). An 81-frame 832x480 decode took 730s on a W7900 vs 13s on an RTX 5060; dtype changes and cudnn.benchmark kernel search were all within +/-7%. A stride-1 kTxkHxkW conv3d is exactly the sum of kT conv2d taps over shifted temporal slices, and MIOpen's conv2d kernels are well optimized. This rebinds WanCausalConv3d.forward (class-level, idempotent, ROCm builds only) to that decomposition: - same 3-latent-frame decode: 81.6s -> 1.71s (~48x), extrapolating to ~12s for the 81-frame workload — matching NVIDIA wall-clock - numerically equivalent up to summation order: ~1e-6 max error vs F.conv3d in fp32; full bf16 decode differs by <=3/255 in pixel space (0.1% of pixels by more than 1/255) - strided encoder downsample convs keep the stock F.conv3d path (temporal taps couple under stride) - applied from every AutoencoderKLWan load site (Wan checkpoint/diffusers VAE loaders, Wan main-model VAE submodel, Anima VAE), so decode, encode, and ref-image conditioning all benefit; CUDA builds are untouched Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: self-heal the media cookie for sessions that predate or outlive it Video playback authenticates via an HttpOnly cookie (media elements can't send Authorization headers) that was only issued at login. A session restored from localStorage can hold a valid JWT without the cookie — the session may predate the cookie's introduction, or the cookie may have been cleared while the JWT survived. Every API call works, but each <video> request 401s and the player silently renders black with 0:00 duration (hit during PR #9163 functional testing). - New POST /api/v1/auth/media-cookie re-issues the cookie from a valid Bearer token: same live-user check as get_current_user, cookie lifetime clamped to the token's remaining validity, successful no-op in single-user mode. Cookie attributes are shared with login via _set_media_cookie so they can't drift. - Frontend calls it once per app load when an authenticated session exists (useMediaCookieRefresh in GlobalHookIsolator); failures are left to the existing global 401 handling. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: use Apply LoRA Collection node in Wan concept-LoRA workflow templates Replace the per-slot concept-LoRA plumbing in the three 'w/ Concept LoRAs' video templates (Text to Video, Image to Video, Extend Video) with the single wan_lora_collection_loader node: users now add any number of concept LoRAs through one multi-LoRA form field instead of two fixed slots (T2V/Extend) or the lora_selector + collect chain (I2V). Chain in all three: model loader -> Lightning high-noise LoRA -> Lightning low-noise LoRA -> Apply LoRA Collection (concept LoRAs, ships empty) -> denoise. Also prunes exposedFields entries that referenced nodes deleted in an earlier revision of these templates (pre-existing; the frontend ignored them, but they were dead weight). Validated: backend WorkflowValidator + default-sync asserts, node versions current, every edge/form/exposedFields reference resolves, frontend parseAndMigrateWorkflow accepts all three, no machine-specific model identifiers ship. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: enforce multiuser image authorization * fix: address remaining Wan video review findings * test: give closed-stream decoder test headroom for slow Windows spawn The 0.2s decode timeout raced against Python subprocess startup on the Windows CI runner: the inactivity deadline fired before the worker could close its stdout, raising the generic decode timeout instead of the expected 'decoder worker' one. A generous timeout makes the EOF path deterministic; proc.wait still bounds the test at ~1s. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address adversarial Wan video review findings * fix: resolve remaining Wan video review issues * feat(ui): rename gallery/board strings from Images to Images/Videos The gallery grid, selections, board operations, and related settings now operate on both images and videos, so the user-facing strings that describe them say so. Image-only surfaces (compare, reference images, progress previews, image storage maintenance, upload-format errors) are unchanged, as are unused legacy keys. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): harden video/media API per PR review - Scope the media cookie (set + delete) and the sliding-token middleware's auth-route exclusions to the reverse-proxy root_path, so media auth works behind sub-path proxies and a proxied logout can't mint a replacement token. - Video upload: run filesystem writes, MP4 validation, ffmpeg probing, and create() in the thread pool; add VideoUploadLimitASGIMiddleware to bound request size before multipart spooling and cap concurrent uploads. - Add GET /videos/i/{name}/workflow (mirrors the image route) so persisted video workflows/graphs are retrievable, with read-access checks. - Add DELETE /videos/uncategorized so the "Delete All Uncategorized Images/Videos" action can cover both media kinds. - Make polymorphic gallery ordering deterministic on created_at ties with kind+name tie-breakers, and pick virtual-board covers via ROW_NUMBER instead of a bare-column MAX() aggregate. - Add cpu_only to WanT5Encoder_WanT5Encoder_Config (parity with the other standalone text-encoder configs; the loader already honors the field). - Regenerate schema.ts/openapi.json. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(backend): Wan VAE effective device + generalized working-memory estimation - Move VAE inputs to get_effective_device(vae) instead of the globally selected device — a cpu_only Wan VAE previously crashed every Wan VAE invocation on GPU hosts. - Add estimate_vae_working_memory_wan (per-frame conv working set + resident RGB clip, config-driven spatial scale for TI2V's 16x compression) and reserve working memory in all four Wan VAE paths, replacing the Flux estimator / missing reservations. - Fall back to spatial tiling for video decodes whose full-frame working set exceeds the execution device's VRAM, and move the decoded clip to the CPU before MP4 encoding so VRAM isn't held for the encode's duration. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): video gallery/deletion/workflow fixes per PR review - Video deletion: use the batch endpoint (one request per invocation), clear workflow-node VideoField inputs only for server-confirmed deletions, and invalidate per-video DTO/metadata/workflow caches on delete (including board-cascade deletions in the board mutations). - VideoFieldInputComponent resets its value only on a confirmed 404, not on transient auth/server/network errors. - Global Delete hotkey partitions the polymorphic selection and routes videos through the video delete flow. - "Delete All Uncategorized Images/Videos" now deletes both media kinds; "Download Board" relabeled "Download Board Images" (image-only endpoint). - Translation splits: image-only multi-select actions revert to "Images"; polymorphic gallery search + star hotkey become media-neutral; the multi- drag preview counts the whole mixed selection. - Expose video metadata + workflow in the viewer: new video details overlay (metadata/workflow/graph tabs), a Load Workflow toolbar action for videos, and a 'video' source for the load-workflow dialog. - Model Manager: wan_t5_encoder gets the encoder settings panel (Run on CPU). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: make gallery docs video-aware; fix video workflow count Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address remaining video review findings * fix(backend): Wan inference fixes from full-PR self-review - Force bfloat16 in the standalone Wan VAE checkpoint loader: `precision: auto` resolves to fp16 on CUDA, and fp16 is unstable on the Wan VAE (the diffusers folder path already forced bf16). Both starter VAEs route through this loader. - Count the decoded RGB clip twice in the Wan working-memory estimator: diffusers' frame accumulation transiently holds ~2x the clip at peak, which the tiled-decode fallback previously undercounted by up to ~2 GB. - Ignore a wired 'Transformer (Low Noise)' for TI2V-5B (warn instead of raising a misleading A14B error), matching the field's documented behavior. - Release the expert swapper's device context even when LoRA weight-restore raises, so a failed unwind can't pin an 8-9 GB expert in VRAM. - Validate LoRA variant (A14B vs 5B) against the wired transformer in both Wan LoRA loaders — a mismatch previously crashed mid-denoise with an opaque layer-patcher shape error. - Fix the WanDiffusersModel exception ladder: the old-diffusers torch_dtype retry now also gets the missing-variant OSError fallback, with the matching dtype kwarg. - Mark both Wan ideal-dimensions nodes Prototype like every other Wan node; correct the text-encoder docstring (seq_len 512, not 226). - Add CPU tests for the multi-frame WanVideoDenoise loop (T_lat>1 shapes, zero-velocity invariant, A14B I2V 36-channel concat across frames, TI2V-5B expand-timesteps mask blend incl. per-token timesteps and frame-0 restore). Node version bumps: wan_model_loader, wan_lora_loader, wan_lora_collection_loader -> 1.0.1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): video service fixes from full-PR self-review - Purge cached invocation outputs on video deletion: the memory invocation cache registered images/tensors/conditioning on_deleted hooks but not videos, so re-running an identical graph after deleting its output "succeeded" with a cached VideoOutput naming a 404 video. - Add the single-user early-return to VideosInterface's read-access and board-save checks, matching ImagesInterface — after a multiuser->single-user switch, video workflows no longer fail with PermissionError where identical image operations succeed. - Restructure staged-delete recovery to match the image side: video_records .get() raises rather than returning None, so the explicit commit branch was unreachable and recovery semantics lived in the exception handler by luck. - Return 416 (not 206 with "bytes 0--1/0") for any Range request against a zero-length video file; add tests for the whole Range-parser matrix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): video UX fixes from full-PR self-review - Add 'video' to the invocation-complete passthrough denylist: a Video Primitive completing mid-run invalidated gallery caches and auto-switched the user's selection/board to the node's *input* video. - Show the multi-selection context menu only when the clicked item is part of the selection (both image and video menus): right-clicking a video with 2+ images selected previously produced a menu with every action disabled. - Clear workflow VideoField references when videos are cascade-deleted via board deletion or delete-uncategorized, matching the direct-delete flow. - Toast on total video-delete-batch failure (the untracked mutation was otherwise silent) and on failed logout (the button previously did nothing when the server was unreachable). - Check resp.ok in useDownloadItem so an expired media cookie can't save error bodies as .mp4/.png files. - Provide the LIST_TAG-scoped VideoList tag from listVideos so the star/board invalidations that reference it actually match; fix the misleading comment; dedupe the doubled BoardVideosTotal tag type. - Validate VideoField access on workflow load (checkVideoAccess), resetting stale refs with a warning like image fields. - Wire middle-click-open-in-new-tab for gallery videos (the setting label already promised it). - Show the effective fallback (primary CFG) in the low-noise guidance slider when unset, instead of a constant the run never uses. - "Moving 1 image/video to board:" singular form for mixed-media moves. - Regenerate schema.ts/openapi.json (node version bumps, classification, docstring fixes). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: workflows/docs/deps fixes from full-PR self-review - Update all 12 bundled Wan workflows to current node versions (wan_model_loader / wan_lora_loader / wan_lora_collection_loader 1.0.1, wan_ref_image_encoder 1.2.0, backfilling the optional end_image/num_frames inputs) so fresh installs don't open with "node needs update" badges; add a registry-consistency test over the bundled Wan/video workflows so stale embeds can't recur. - Docs: the A14B auto scheduler is UniPC (not FlowMatchEuler); note that the bundled TI2V-5B workflows ship 20 steps as a speed compromise vs the 40-50 quality recommendation. - Pin imageio[ffmpeg]>=2.37 and psutil>=6 (imageio encode behavior is version-sensitive enough that we carry a regression test for it); relock. - De-flake the thumbnail worker descendant-kill test (0.5s was the only tight ceiling in the file; a loaded runner could kill the worker before the child pid file existed). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * ci: raise python-tests job timeout to 30 minutes Main already runs 9-11 min per platform and this PR pushed py3.11 windows-cpu past the 15-minute cap (cancelled mid-pytest at 15m10s on the last run). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: resolve CI failures from main's route-auth audit and knip The route-authorization audit merged from main (#9367) only recognized the two Bearer-token dependencies, so it flagged the video media routes (which authenticate via get_current_media_user_or_default) and the media-cookie endpoint (which validated its Bearer token inline). Teach the audit about the media dependency, drop the image media routes from PUBLIC_ROUTES (they now carry cookie auth on this branch), and give refresh_media_cookie a CurrentUserOrDefault dependency in place of its duplicated inline validation. The media-cookie tests now patch auth_dependencies' ApiDependencies like every other auth-dependent route test. knip: getDeletedVideosFromDeleteBoardAction was exported but only used in-module; cover it in the listener unit tests like its image twin. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(backend): Wan invocation fixes from JPPhoto's 2026-07-21 review - Both Wan LoRA loaders now validate the *resolved* config (type=LoRA, base=Wan) instead of trusting the client-supplied identifier fields; a mislabeled Flux/SDXL/main key is rejected up front instead of reaching the layer patcher. - The collection loader rejects LoRAs already applied upstream on either expert list (same invariant as the single loader) instead of silently doubling their effective weight. - A LoRA routed only to the low-noise list of a TI2V-5B main now logs a warning — the single-transformer path never consumes that list, so the routing was a silent no-op. - _ExpertSwapper._release clears its slots in a nested finally, so a device-context exit failure can no longer leave stale contexts that a later close() would double-exit. - WanLatentsToImage rejects multi-frame (T>1) video latents with a clear error pointing at wan_l2v, before the VAE is even loaded — previously it ran the full multi-frame decode and died in an opaque einops rank error. - wan_ref_image_encoder docstrings now describe both the 20-channel A14B and 48-channel TI2V-5B condition paths (they claimed A14B-only and told users to omit the node for TI2V-5B, contradicting the implementation). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(api): per-user upload slots, probe validation, stream decode fallback - VideoUploadLimitASGIMiddleware now accounts upload slots per user (cap 2) on top of the global cap, so one tenant's slow chunked uploads can no longer hold all four slots and starve other users into 429s. Single-user mode keeps the whole global capacity (no per-user quota). - probe_video validates decoder-reported metadata: non-positive or over-limit dimensions (> 64 MP) and non-finite/negative durations are rejected before the upload path persists them; garbage fps degrades to None (unknown). The decode worker refuses to decode frames from files whose probed dimensions exceed the bound — a small crafted container claiming 100k x 100k would otherwise trigger a ~30 GB allocation. - The worker's stream command falls back to cv2 like probe/frame/count do, so an MP4 accepted at upload via the cv2 path now also works in the frame-range and concat nodes. The fallback only engages before the first emitted frame; a mid-stream decoder death still surfaces as an error. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ui): media resilience fixes from JPPhoto's 2026-07-21 review - Partial board deletion: the boardAndImagesDeleted listener now invalidates the per-item Image*/Video* tags for the confirmed-deleted names it parses out of the 500 detail — the rejected mutation runs invalidatesTags with no result, so those caches previously stayed readable. - Media-cookie refresh retries transient failures on a bounded backoff (2s, 10s) instead of latching before the request and giving up forever; 401 still bails (session genuinely expired). - Thumbnail 404s degrade gracefully: BoardTooltip, GalleryBoard, VirtualBoardItem, and VideoFieldInputComponent show an icon fallback via fallbackStrategy="onError" (thumbnail generation is best-effort server- side), and GalleryVideoThumbnail's <video> fallback does the near-zero seek on loadedmetadata so browsers that don't auto-paint the first frame no longer show a black tile. - CurrentVideoPreview handles play() rejection (rolls isPlaying back) and media element errors (drops back to the play overlay) instead of hiding the overlay over a dead element with an unhandled promise rejection. - Hardening from the disputed items: changeVideoIsIntermediate also invalidates the VideoList LIST_TAG (covers a future omitted-board_id list); logout clears gallery.selection and the logout mutation documents that resetApiState in store.ts is what actually clears cross-user caches. - Typegen regenerated for the wan_lora_loader / wan_ref_image_encoder docstring updates. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address final video review findings * fix: harden video workflows and auth refresh * chore: regenerate OpenAPI schema * fix: close video workflow review gaps * fix: address self-review findings on video workflows and auth refresh Fixes the confirmed findings from the 2026-07-22 self-review round (github.com/invoke-ai/InvokeAI/pull/9163#issuecomment-5051225515): Frontend: - DeleteVideoModal: detach the dialog promise callbacks before the async deletion so the accept path's synchronous onClose (bound to cancel) no longer rejects a confirmed deletion as "User canceled"; dismissal still rejects. Adds behavioral tests for both paths. - Sliding-window refresh: bound the media-cookie sync fetch with a 10s AbortSignal timeout so a stalled request can't hold the exclusive cross-tab media-auth lock (shared with login/logout) forever; commit the refreshed token even when the cookie sync fails with a 5xx/network error (only a 401/403 rejection of the token blocks the commit); throttle acceptance to once per minute so bulk mutations don't pay a serialized cookie round trip per request. - Fallback media-auth lock: waiters renew their ticket lease while queueing so a >30s wait no longer lets a later ticket enter concurrently. - useMediaCookieRefresh: a pause() abort now resumes the same attempt instead of consuming a retry slot (and no longer permanently disables self-heal when the final attempt was paused); effect cleanup aborts in-flight refreshes so every logout path (sessionExpiredLogout, direct logout) stops a pending refresh from re-minting the cookie post-logout. - CurrentVideoPreview: a benign AbortError from play() rolls back silently, and load errors during the pending media-cookie self-heal window no longer raise a spurious "Unable to Play Video" toast. Backend: - Decode-worker memory bounds resized for legal near-cap frames: worker RLIMIT_AS headroom 1->4 GiB, parent RSS kill threshold 1->3 GiB (with keep-in-sync cross-references), monitor poll 50->250 ms. - Upload probe: a decode-worker timeout is now inconclusive (upload proceeds) instead of a 415; the probe's decoded first frame is reused as the thumbnail source, dropping one worker subprocess per upload. - delete_images_on_board / delete_videos_on_board return (deleted, failed) and delete_board reports the services' ground truth instead of a racy router-side listing diff (which also doubled the DB work). - Video list/uncategorized delete endpoints skip HTTPException (ownership skips, 404 races) silently instead of reporting them as failures, matching the image endpoints; delete_images_from_list now populates failed_images for genuine failures, matching the video path. - video_concat: an unknown probed fps mixed with agreeing known rates uses the known rate again instead of hard-erroring; disagreeing known rates still require an explicit Output FPS. - SlidingWindowTokenMiddleware runs its synchronous SQLite user lookup via run_in_threadpool so a contended DB lock can't stall the event loop. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(tests): deflake drip-feed upload timeout test on Windows The test gave the middleware a 20 ms absolute upload deadline and delivered a chunk every 5 ms — but Windows event-loop timers have ~15.6 ms granularity, so the deadline could expire before the first chunk was ever delivered. The request then ended at receive_calls == 1 and the `receive_calls > 1` assertion failed (py3.12 windows-cpu CI). Widen the margins so the scenario the test describes actually occurs on coarse timers: 250 ms absolute deadline (many chunks flow first on every platform) with a 1 s idle timeout that never fires between 5 ms chunks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(tests): sort imports in test_video_upload_limits (ruff I001) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address in-PR items from JPPhoto's non-merge-blocker list Fixes the subset of JPPhoto's 2026-07-22 "Still Open, Non-Merge Blockers" that are small, self-contained, and scoped to surfaces this PR introduced; the rest are deferred to a follow-on PR (triage rationale posted on the PR). - video_thumbnails._run_worker: an unexpected exception from communicate() (e.g. OSError) now terminates the worker process tree unconditionally — previously the finally stopped the RSS-monitor backstop while the except path left the worker and its ffmpeg child running forever. Adds the injected-OSError test JPPhoto asked for. - reduxRemember driver: client-state persistence POSTs now commit X-Refreshed-Token via the same acceptance flow as dynamicBaseQuery (extracted as acceptRefreshedToken, sharing the cross-tab lock, cookie sync, throttle, and generation guards), so persistence-only sessions no longer hard-expire mid-activity. - delete_videos_from_list / delete_images_from_list: dedup request names — a repeated name was processed twice and landed in both deleted_* and failed_* under the admin ownership bypass, toasting a spurious partial failure. Regression test added. - gallery + videos list endpoints: bound offset (ge=0) and limit (ge=0, le=MAX_PAGE_SIZE=1000) — these flowed verbatim into SQL, where a negative LIMIT means unlimited in SQLite, so one request could materialize the entire gallery. openapi.json regenerated (schema.ts is unchanged — constraints don't alter the generated types). - get_video_full: open the file once and serve HEAD/range/full from the fd (full downloads now stream chunked from the handle instead of FileResponse's lazy path-based open), eliminating the delete-race 500; deletion's atomic rename can no longer invalidate a path between check and open. - upload_video: close the multipart spool immediately after the body copy, shrinking the double-temp-disk window (2 x 1 GiB x 4 concurrent worst case) to the copy loop itself. - docs/gallery.mdx: document shared-board deletion semantics (only your own media is permanently deleted; admins delete everything) and the kept-on-failure -> Uncategorized behavior with its UI warning. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix video gallery review findings * fix Windows video thumbnail handling * test: cover remaining video review findings * test: cover adversarial video and Wan findings * fix: address remaining video and Wan review findings * test: call now-sync star/unstar routes directly 11b38696bf converted the video batch routes from async def to sync def (so FastAPI offloads them to its threadpool), but the star/unstar dedupe test still drove them through asyncio.run(), which requires a coroutine. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: cover Wan conditioning and video link regressions * fix: validate Wan conditions and video links --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Jonathan <34005131+JPPhoto@users.noreply.github.com> Co-authored-by: JPPhoto <jpollack@jpollackphoto.com> | 1 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 2 个月前 | ||
| 20 天前 | ||
| 1 个月前 | ||
| 1 年前 | ||
| 21 天前 | ||
| 10 个月前 | ||
| 1 年前 | ||
| 6 个月前 | ||
| 4 个月前 | ||
| 1 个月前 | ||
| 2 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 2 年前 | ||
| 1 个月前 | ||
| 6 个月前 | ||
| 1 年前 | ||
| 1 个月前 |