main

GGitHubfix(model loaders): ignore unexpected checkpoint keys, report them at debug only (#9581)

6515d906创建于 11 天前历史提交

文件	最后提交记录	最后更新时间
gguf	fix(gguf): honor ComfyUI's comfy.gguf.orig_shape metadata (#9564) * fix(gguf): honor ComfyUI's comfy.gguf.orig_shape metadata ComfyUI's GGUF converter can only quantize 2-D tensors, so it reshapes any tensor the quantizer rejects and records the native shape under a `comfy.gguf.orig_shape.<tensor name>` KV entry. `gguf_sd_loader` ignored those entries and used the stored shape, so such a checkpoint failed at load with a size mismatch. Concretely, Krea-2's `first.weight` is (6144, 64) but is stored as (1536, 256), which produced: size mismatch for img_in.weight: copying a param with shape torch.Size([1536, 256]) from checkpoint, the shape in current model is torch.Size([6144, 64]) The loader now reads the metadata and uses the declared native shape, rejecting an entry whose element count doesn't match the stored tensor and warning on a malformed one. This is architecture-agnostic, not a Krea-2 special case. Verified end-to-end: the affected checkpoint from the issue installs, loads and generates a coherent image. Closes #9537 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(gguf): reshape numpy dequant fallback and harden orig_shape parsing Follow-up to the comfy.gguf.orig_shape support, addressing review feedback. Qtypes without a torch dequantize kernel fall back to gguf's numpy implementation, which infers the output shape from the stored data. That matched the logical shape only as long as both were identical -- with a ComfyUI-reshaped tensor the fallback returned the stored shape, so loading succeeded and inference then failed. Reshape the fallback output to tensor_shape, as the torch path already does via oshape. Dimension values from comfy.gguf.orig_shape.* were passed straight to int(), which truncates non-integral values (int(2.5) == 2) and raises an uncaught OverflowError on inf. Both contradict the documented warn-and-ignore behaviour. Reject anything that is not a finite, integral, positive number, and verify that the metadata is an array at all. Tests cover the fallback qtype shape and the malformed-metadata cases. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Jonathan <34005131+JPPhoto@users.noreply.github.com>	11 天前
sdnq	fix(model loaders): ignore unexpected checkpoint keys, report them at debug only (#9581) * fix(model loaders): ignore unexpected checkpoint keys, report them at debug only Single-file loaders disagreed, loader by loader, about what an unexpected key from `load_state_dict(strict=False)` means: some raised, one warned, most ignored it silently, and the rest used `strict=True` and let torch raise. The ones that hard-failed turned any harmless extra tensor an exporter happened to serialize into a user-facing crash that needed a code change and a release — Anima went through this twice (#9201, #9402) for tensors the model does not need and the official checkpoint does not contain. Per the team decision on #9437, extra keys are now reported at DEBUG and otherwise ignored everywhere. New `invokeai/backend/util/state_dict_loading.py` holds the single policy: - `log_unexpected_keys()` — DEBUG only, never raises. - `load_state_dict_ignoring_extras()` — a drop-in for `strict=True` that keeps the strictness that matters (every required parameter must be filled, shape mismatches still raise) and drops the strictness that only produces whack-a-mole. - `reject_incomplete_load()` — the meta-device completeness sweep, generalized out of krea2. Stronger than `missing_keys` for models built under `init_empty_weights()`: immune to non-persistent buffers and tied weights. Every previously existing missing-key guard is preserved exactly; only the unexpected-key policy changed. `flux.py`'s bare `assert len(unexpected_keys) == 0` — which carried no message and was stripped entirely under `python -O` — is gone with it. Two consequences worth calling out: - `configs/pid_decoder.py` rejected unexpected keys at *identification* time, deliberately mirroring the loader ("both are fatal there"). Left alone, the PiD relaxation would have been unreachable and the installer would refuse a file that now loads fine. It keeps refusing non-string keys, which `load_state_dict` genuinely cannot survive. - Anima's unexpected-key `RuntimeError` was its only hard load-time guard, so it is replaced with the meta-device sweep rather than dropped — otherwise an incomplete checkpoint would fail mid-inference with "Cannot copy out of meta tensor" instead of at load time. `wan.py::_raise_for_incompatible_keys` deliberately keeps raising: Wan derivatives (Animate, S2V, Fun-Camera) are supersets whose extra branches are the feature the checkpoint exists for, not exporter noise, and it strips the benign extras before that check. Closes #9437 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V4nLTdWBQaaDUFVLx9TyZH * docs(model loaders): fix two stale unexpected-key comments Review follow-up for #9581. The ideogram4 fp8 encoder branch and the z_image SDNQ Qwen3 loader still described unexpected checkpoint keys as fatal, but both now go through helpers that log them at DEBUG and ignore them (`load_fp8_state_dict`, `raise_on_incomplete_sdnq_load`). Update the comments to match the code; missing-key strictness is unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NxYxw5RUtuRTBRWTn79M9d --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Jonathan <34005131+JPPhoto@users.noreply.github.com>	11 天前
test_bnb_llm_int8.py	Reduce peak memory used for unit tests.	1 年前
test_sdnq_detection.py	feat: Adding Support for SD.Next Quantization Engine (SDNQ) (Flux1&Flux2klein4B/9B&Z-Image) (#9228) * feat(quantization): add SDNQ (SD.Next Quantization) support Add support for loading SDNQ-quantized models with on-the-fly CPU dequantization, similar to existing GGUF support. New features: - SDNQTensor class with __torch_dispatch__ for automatic dequantization - Support for symmetric/asymmetric int8/uint8/fp8 quantization - Optional SVD correction (low-rank approximation) - Model loaders for Flux and Z-Image SDNQ models - Automatic format detection via weight+scale key pairs New files: - invokeai/backend/quantization/sdnq/ (core module) - tests/backend/quantization/sdnq/ (unit tests) Modified files: - taxonomy.py: Add ModelFormat.SDNQQuantized - configs/main.py: Add Main_SDNQ_FLUX_Config, Main_SDNQ_ZImage_Config - configs/factory.py: Register SDNQ configs - model_loaders/flux.py: Add FluxSDNQCheckpointModel - model_loaders/z_image.py: Add ZImageSDNQCheckpointModel * fix(sdnq): improve uint4 dequantization and add diffusers format support - Add uint4 per-group quantization with packed weight unpacking - Handle 1D flattened weights (reshape to 2D before unpacking) - Support SDNQ diffusers format for FLUX transformer and T5 - Add SDNQ VAE loading with AutoencoderKL - Add diagnostic logging for debugging dequantization - Fix bit order in uint4 unpacking (lower, upper) * test(sdnq): align asymmetric dequant expectation with upstream convention The test was checking `(weight - zero_point) * scale`, but SDNQ (Disty0/sdnq) defines asymmetric dequantization as `zero_point + weight * scale` (via torch.addcmul), where zero_point is a post-scale bias rather than a pre-scale integer offset. The implementation already follows this convention; only the test expectation was wrong. * test(sdnq): align asymmetric dequant expectation with upstream convention The test was checking `(weight - zero_point) * scale`, but SDNQ (Disty0/sdnq) defines asymmetric dequantization as `zero_point + weight * scale` (via torch.addcmul), where zero_point is a post-scale bias rather than a pre-scale integer offset. The implementation already follows this convention; only the test expectation was wrong. feat(sdnq): support sidecar LoRA application on SDNQ-quantized layers Bring SDNQ to feature parity with GGUF in the sidecar patching path so LoRA, LoKr, DoRA, FullLayer, and FluxControl patches apply correctly to SDNQ-quantized Linear and Conv2d modules. Without this, the sidecar aggregate replaced the SDNQTensor weight with a meta tensor and patches silently produced wrong results. - Add SDNQTensor branch in CustomModuleMixin._aggregate_patch_parameters mirroring the GGMLTensor branch. - Extend the (GGMLTensor) dtype-cast exclusion to also cover SDNQTensor in CustomLinear, CustomConv2d, CustomInvokeLinearNF4, and CustomInvokeLinear8bitLt. - Add `linear_with_sdnq_quantized_tensor` and `linear_sdnq_quantized` fixtures so the existing custom-module test matrix exercises SDNQ alongside GGUF, BnB-8bit, and NF4. * feat(sdnq): support SDNQ-quantized T5 encoders Add T5Encoder_SDNQ_Config for diffusers-style T5 bundles whose text_encoder_2/ folder holds SDNQ-quantized safetensors (detected via quantization_config.json's quant_method or via the SDNQ-style weight+scale key pairs). Add T5EncoderSDNQLoader that materializes the T5EncoderModel on meta, then loads the SDNQ state dict, and re-shares the embed_tokens/shared weight per HuggingFace's tied- weight convention. * feat(sdnq): support SDNQ-quantized FLUX.2 transformers Add Main_SDNQ_Flux2_Config covering Klein 4B/9B and their Base variants (detected via _get_flux2_variant on the dequantized SDNQTensor shapes plus the existing filename heuristic), and Flux2SDNQCheckpointModel that loads diffusers-layout SDNQ FLUX.2 checkpoints straight into Flux2Transformer2DModel. Architecture (num_layers, hidden_size, attention head count, guidance presence) is detected from state-dict shapes the same way the fp16 loader does, since SDNQTensor.shape reports the dequantized shape. BFL-layout SDNQ FLUX.2 checkpoints are not supported here — that would require an SDNQTensor-aware port of the _convert_flux2_bfl_to_diffusers fuse logic. * feat(sdnq): support full ZImagePipeline diffusers folders Add Main_SDNQ_Diffusers_ZImage_Config so a complete SDNQ ZImagePipeline folder (model_index.json + transformer/ + text_encoder/ + tokenizer/ + vae/) is recognised on install and its submodels are wired up. Extend ZImageSDNQCheckpointModel to load the transformer from the subfolder using ZImageTransformer2DModel.from_config() so non-default architecture parameters (e.g. axes_lens [1536,512,512] in newer Z-Image Turbo SDNQ exports) are honoured instead of the single-file path's hardcoded [1024,512,512]. Verified end-to-end against Tongyi-MAI/Z-Image-Turbo-SDNQ-uint4-svd-r32: 269 quantized + 252 regular tensors load into a 6.15B-param model with 0 missing / 0 unexpected keys. * fix(sdnq): match T5 SDNQ submodel layout in FluxPipeline bundles T5Encoder_SDNQ_Config originally only looked for text_encoder_2/ as a subfolder of mod.path, which works for standalone T5 bundles but misses the case where a parent FluxPipeline / similar config registers its T5 submodel with path_or_prefix pointing straight at the text_encoder_2 folder. Allow both layouts in both the config's detection logic and T5EncoderSDNQLoader's te_dir resolution. Verified end-to-end with Disty0/FLUX.1-schnell-SDNQ-uint4-svd-r32. * fix(sdnq): swap scale/shift halves in FLUX BFL converter's norm_out The diffusers→BFL state-dict converter renamed norm_out.linear.{weight,bias} to final_layer.adaLN_modulation.1.{weight,bias} but did not swap the two halves along dim 0. diffusers' AdaLayerNormContinuous packs the linear output as (scale, shift); BFL's LastLayer packs as (shift, scale). Without the swap, the final adaLN modulation runs with scale and shift permuted, which produces structured-but-very-noisy output for every pixel. Reuse the same pattern the FLUX.2 converter applies for the analogous adaLN_modulation key. * feat(sdnq): dispatch all ZImagePipeline submodels via SDNQ loader ZImageSDNQCheckpointModel only handled the Transformer submodel, so attempts to use an SDNQ ZImagePipeline as the "Qwen3 & VAE source model" (which triggers loads for TextEncoder / Tokenizer / VAE) crashed with "Only Transformer submodels are currently supported". Add per-submodel handlers that load text_encoder/ via sdnq_sd_loader into an empty Qwen3ForCausalLM (re-sharing lm_head with embed_tokens when tied), tokenizer/ via AutoTokenizer, and vae/ via AutoencoderKL.from_pretrained. The single-file SDNQ checkpoint path keeps its transformer-only behaviour but now raises a clearer error when asked for a different submodel. * feat(sdnq): support FLUX.2 Klein dynamic mixed-precision pipelines Add support for SDNQ-quantized Flux2KleinPipeline folders, which mix uint4 and int5 dtypes across layers (chosen dynamically by SDNQ during quantization to stay under a per-group loss budget). Core changes: - Add INT5_ASYM quantization type + unpack_uint5 + dequantize_int5_per_group. Sign-extension matches Disty0/sdnq's unpack_int convention (raw 0..31 - 16). zero_point is optional (dynamic-mixed sometimes emits scale-only int5 tensors). - _infer_quantization_type now takes a per_tensor_dtype override; the loader builds an inverted map from quantization_config.json's modules_dtype_dict. - _get_original_shape uses the packed weight size as the authoritative source for in_features, fixing a bug where Klein 4B's group_size=64 layers were misread as group_size=128 (the previous fallback). Pipeline integration: - Add Main_SDNQ_Diffusers_Flux2_Config matching Flux2Pipeline / Flux2KleinPipeline folders with quantized transformer. - Flux2SDNQCheckpointModel now dispatches all pipeline submodels: transformer (Flux2Transformer2DModel.from_config + sdnq state dict), text_encoder (Qwen3ForCausalLM SDNQ + lm_head/embed_tokens tie), tokenizer (AutoTokenizer), vae (AutoencoderKLFlux2 / AutoencoderKL). - Extend flux2_klein_model_loader._validate_diffusers_format and the isFlux2DiffusersMainModelConfig FE filter to also accept SDNQ pipeline configs (when submodels is populated). Verified against Disty0/FLUX.2-klein-4B-SDNQ-4bit-dynamic: 98 uint4 + 2 int5 tensors load into a 3.88B-param Flux2Transformer2DModel with 0 missing / 0 unexpected keys; both dequant paths produce reasonable zero-centred weight distributions. * - Reject SDNQ-quantized folders in Main_Diffusers_FLUX_Config and Main_Diffusers_Flux2_Config so identification routes them to the SDNQ configs instead. Without this both configs accept the folder and the plain diffusers loader wins, then crashes when reading packed uint8 weights as bf16. * - Merge multi-shard safetensors in sdnq_sd_loader so Klein 9B's diffusion_pytorch_model-{00001,00002}-of-00002.safetensors and FLUX.2 dev's sharded transformer both load. Detect cross-shard key collisions as a corruption signal. * - Treat SDNQ ZImagePipeline / Flux2KleinPipeline folders as "main_is_diffusers" in z_image_model_loader and flux2_klein_model_loader so the auto-extract-submodels branch handles them. Without this the loader demanded a separate VAE/Qwen3 source even though the SDNQ pipeline carries those submodels itself. - Drop the ui_model_format=Diffusers hint on Klein's qwen3_source_model field so the FE combobox can also show SDNQ pipeline configs (the FE filter already accepts them). * fix(sdnq): unblock FLUX.2 Klein SDNQ pipelines in the UI Loading the Klein 4B SDNQ pipeline as the main model errored with "No Qwen3 Encoder selected" in the UI even though the pipeline carries its own Qwen3 + VAE submodels, and the Model Manager showed no format badge at all on SDNQ models. - flux2_klein_model_loader now treats SDNQ-with-submodels as main_is_diffusers, so the auto-extract-submodels branch handles SDNQ pipelines exactly like plain diffusers. Drop the ui_model_format=Diffusers hint on qwen3_source_model so the combobox can also show SDNQ pipeline configs. - readiness.ts no longer demands a standalone VAE/Qwen3 for FLUX.2 Klein when the main model is itself a pipeline (diffusers or SDNQ-with-submodels). Without this the Invoke button stayed disabled with "Non-diffusers FLUX.2 Klein models require a standalone Qwen3 Encoder" even when the SDNQ pipeline could self-source everything. - Register sdnq_quantized in zModelFormat, the manually-edited OpenAPI schema, ModelFormatBadge, and MODEL_FORMAT_TO_LONG_NAME so SDNQ models render an "sdnq" badge instead of an empty placeholder. * feat(sdnq): add starter models and user-facing docs - 4 new starter models covering all SDNQ pipelines verified end-to-end in this branch: FLUX.1 schnell, Z-Image Turbo, FLUX.2 Klein 4B (dynamic mixed), FLUX.2 Klein 9B (dynamic mixed + SVD). Each entry is self-contained (no separate encoder/VAE dependencies because the SDNQ pipeline folder bundles them). - New /configuration/sdnq-quantization/ page: support matrix, VRAM footprints, install steps (Starter Models + HF + Folder), LoRA compatibility notes, SDNQ-vs-SVDQuant/Nunchaku disambiguation, comparison with GGUF/NF4/FP8, troubleshooting. - Cross-link from fp8-storage.mdx's "no-op on quantized" caution. * Chore Fix Path * fix(sdnq): add missing variant/cpu_only fields to SDNQ configs Z-Image and Qwen3 SDNQ configs were missing `variant` (and `cpu_only` on Qwen3) fields that exist on the other variants of the same union, breaking TypeScript narrowing on the FE. - Main_SDNQ_ZImage_Config: add variant (default Turbo) - Main_SDNQ_Diffusers_ZImage_Config: add variant, detect from scheduler_config.json shift value - Qwen3Encoder_SDNQ_Config: add cpu_only + variant, detect from embed_tokens shape - Qwen3Encoder_SDNQ_Folder_Config: add cpu_only + variant, detect from config.json hidden_size - Regenerate FE schema.ts Discriminator tags are unchanged since variant has no default. * Fix openapi schema.ts * Fix Path * feat(pid): vendor PiD decoder backend (phase A of integration) Adds a vendored subset of NVIDIA's PiD (Pixel Diffusion Decoder) at invokeai/backend/pid/ as the foundation for upcoming FLUX / FLUX.2 / SD3 / Z-Image PiD decode nodes plus a future PiD-based 4x upscale node. Upstream: https://github.com/nv-tlabs/PiD (Apache 2.0). Vendor scope: * _src/{networks,models,modules}: PidNet, PixDiT_T2I, LQProjection2D, PidModel, PidDistillModel, PixelDiTModel, GeneralConditioner. * _ext/imaginaire: minimal Imaginaire framework subset (lazy_config, model, utils/{log,misc,distributed,device,count_params}). * configs/, tokenizers/, checkpointer/, trainer.py, visualize/, _demo_*, from_*, easy_io/, S3/wandb training helpers were intentionally excluded. Dependency stripping (no new hard deps introduced): * loguru, termcolor -> stdlib logging shim * iopath PathManager -> stdlib pathlib stub * fvcore Registry -> minimal stdlib Registry * lazy_config/lazy.py: yaml/dill/cloudpickle/detectron2 save/load paths replaced with a minimal LazyCall stub * lazy_config/instantiate.py: omegaconf DictConfig/ListConfig branches removed; configs are plain dict / LazyCall mappings * megatron, pynvml, boto3/wandb imports are try/except-guarded or local to functions and stay inert in our inference path All pid.* imports rewritten to invokeai.backend.pid.*; SPDX-Apache-2.0 headers retained on vendored files; attribution and detailed list of local modifications added in LICENSE-PiD.txt. The pre-trained PiD checkpoints distributed by NVIDIA remain under NSCLv1 (non-commercial); this commit only vendors code. Smoke test: PidNet, PidModel, PidDistillModel, GeneralConditioner import cleanly; LazyCall -> instantiate round-trip resolves to the expected nn.Module. ruff check passes. * feat(pid): wire PiD + Gemma-2 into model-manager and add decode nodes Adds the model-manager plumbing and workflow nodes needed to use the vendored PiD decoder (phase A) end-to-end with FLUX, SD3 and Z-Image. Model manager (Phase B + B.5): * taxonomy: ModelType.PiDDecoder, PiDDecoderVariantType (Res2k_Sr4x / Res2kTo4k_Sr4x), ModelType.Gemma2Encoder + ModelFormat.Gemma2Encoder, both added to AnyVariant + variant_type_adapter. * configs/pid_decoder.py: per-backbone PiD configs (FLUX / FLUX.2 / SD3) with state-dict probing on 'lq_proj' substring and backbone/variant detection from the official NVIDIA filenames. * configs/gemma2_encoder.py: Gemma-2 directory probing on Gemma2ForCausalLM architecture + tokenizer files. * AnyModelConfig union updated. * model_loaders/pid_decoder.py: loads .pth / .safetensors, strips the upstream 'net.' prefix, supports torch.load(weights_only=True). * model_loaders/gemma2_encoder.py: SubModelType.{Tokenizer, TextEncoder} dispatch; returns the causal LM's inner Gemma2Model (transformers 4.56's get_decoder() returns None for Gemma2). Decode pipeline (Phase C): * backend/pid/decode.py: build_pid_net + load_pid_decoder (per-backbone PixDiT_T2I hyperparams derived from PiD's pid_sr4x base + per-experiment overrides), encode_caption_for_pid (chi-prompt + Gemma encoding, mirrors PixelDiTModel._encode_text_raw), and a PiDDecoder wrapper with a reimplemented few-step distill sampler (no autocast / no distributed / no PixelDiTModel init paths from upstream). Invocations (Phase 6.x): * Gemma2EncoderField + PiDDecoderField in invocations/model.py. * gemma2_encoder_loader / pid_decoder_loader: thin ModelIdentifierField pickers that emit the corresponding fields. * z_image_pid_decode (pilot), flux_pid_decode, sd3_pid_decode: caption encode -> Gemma offload -> PiD state dict load -> PidNet construct -> decode. Per-backbone latent denormalisation (FLUX1 ae_params, SD3 hardcoded 1.5305/0.0609, Z-Image piggybacks on FLUX VAE). End-to-end validated with the released PiD_res2k_sr4x_official_flux_distill_4step.pth checkpoint and gemma-2-2b-it: PidNet rebuilds at exactly 456 keys / 1.36B params, sampler runs at ~5 GB VRAM peak (Gemma dominates), output shape and range match. FLUX.2 PiD decode is deliberately deferred: it needs BN-based latent denormalisation and 32->128 channel packing, and we have no FLUX.2 checkpoint to validate against yet. * feat(pid): end-to-end PiD pixel-diffusion decoder integration Adds the NVIDIA PiD decoder as a 4x super-resolution alternative to the regular VAE/RAE decode path. Includes model-manager configs and loaders for both the PiD checkpoints and the Gemma-2 caption encoder they require, plus four invocations: latent-in decode for FLUX / SD3 / Z-Image and an image-in pid_upscale node. - Decode pipeline keeps PidNet params in fp32 and uses bf16 autocast only for matmuls; caption embeddings have outliers that overflow bf16 RMSNorm. - encode_caption_for_pid forces tokenizer padding_side="right" (Gemma defaults to left, PiD trained with right) and returns the attention mask as bool so it stays compatible with SDPA. - Z-Image reuses the FLUX-trained checkpoint and reads scale/shift from the VAE config at runtime (PiD upstream notes they are checkpoint-specific). - TextLLM config now excludes Gemma2ForCausalLM so it falls through to the dedicated Gemma2 encoder config instead of being misclassified. - Frontend: new model_type / model_format / variant enums, type guards and category metadata; schema.ts regenerated via pnpm typegen. * Fix closing Step Tag * Chore Ruff * Chore Typegen * Chore Knip * fix(pid): remove unused vendored models/utils.py (broken easy_io import) * feat(pid): identify decoder backbone from weight shapes, not filename Read latent channel count from lq_proj.latent_proj.0.weight (FLUX.2=128, FLUX.1/SD3=16) as the primary discriminator; fall back to filename/dir name only to disambiguate the architecturally identical FLUX.1/SD3 pair. Fixes FLUX.2 checkpoints (model_ema_bf16.pth) not being recognised, and correctly rejects unsupported backbones (RAE/dinov2, 768ch). Fix Flux2 docstring 32->128. * feat(ui): PiD decode (Fit mode) for FLUX text-to-image Add a "PiD Decode" mode select (Off / Fit / Native) to the FLUX advanced settings with PiD decoder + Gemma-2 encoder pickers. In Fit mode the FLUX graph swaps the VAE decode for a PiD 4x super-resolution decode and downscales back to the requested size. Adds params state (pidMode, decoder, encoder, steps) with a v3->v4 migration, model hooks, readiness checks, and graph guards for the not-yet-wired Native and non-txt2img paths. * feat(ui): PiD Native 4x mode for FLUX text-to-image Make the generation dimension helpers PiD-aware via an optional pidScale: in Native mode the user-facing dimensions are the 4x target (grid 64, optimal 2048), generation runs at target/4, and PiD's 4x output is used directly with no downscale. Thread pidScale through the params dimension reducers and the optimal-dimension/grid-size selectors, resync dimensions when toggling Native, and wire the Native path in the FLUX graph builder. Add working_mem_bytes for PiD Decode * feat(ui): PiD Fit decode for FLUX image-to-image Extract the PiD decode chain into buildPidDecodeChain (loaders + decode + fit-downscale, no denoise setup) so it can substitute for the VAE decode across generation modes. Widen addImageToImage's l2i param to ImageOutputNodes (it only consumes .image) and wire the PiD chain into the img2img branch in Fit mode. Native stays txt2img-only (a 4x result can't composite onto the bbox); inpaint/outpaint remain gated off for now. * feat(ui): PiD Native 4x decode for FLUX image-to-image Add addPidImageToImageNative: the canvas bbox is the 4x target, so the init image is downscaled to bbox/4, denoised at that resolution, and PiD decodes straight back up to the full bbox with no post-decode downscale - preserving all PiD detail while still compositing cleanly onto the region. Wire it into the img2img branch of buildFLUXGraph (native vs fit vs off) and drop the native-txt2img-only guard. Make the canvas FLUX grid check PiD-aware so a native bbox must be a multiple of 64 (16 * 4) for bbox/4 to land on the grid. * feat(ui): add informational popover to PiD Decode setting Explain PiD usage on hover, mirroring the DyPE popover: what the decoder is (NVIDIA Pixel Diffusion Decoder, 4x SR, needs a PiD decoder + Gemma-2 encoder), Fit vs Native modes, the 2K / 2K-to-4K target resolutions, that Steps can be lowered, and that Scale Before Processing must be off. Links to nv-tlabs/PiD. * feat(models): add PiD decoder + Gemma-2 encoder to starter models Register NVIDIA's PiD FLUX decoders (2K and 2K-to-4K presets, from nvidia/PiD) and the Efficient-Large-Model/gemma-2-2b-it caption encoder as starter models so they can be installed from the Model Manager. The Gemma-2 encoder is wired as a dependency of each decoder (and offered standalone). * feat(pid): add FLUX.2 Klein PiD 4x-SR decode support Add a flux2_pid_decode node that packs the stored FLUX.2 latent (32ch @ H/8) into PiD's 128ch @ H/16 layout before decoding; FLUX.2's BatchNorm denormalization is already applied in flux2_denoise, so no scalar denorm is needed (optional vae input reads identity constants). Generalize the frontend PiD decode chain (decodeNodeType, optional vaeSource) and wire the isFlux2 graph path for txt2img/img2img (Fit & Native). Base-aware PiD gating/decoder-filter, FLUX.2 readiness checks, and two nvidia/PiD FLUX.2 starter decoders (2K, 2Kto4K). Standard FLUX PiD path unchanged. * feat(pid): add SD3 PiD 4x-SR decode support Wire the existing sd3_pid_decode node into the SD3 graph builder (txt2img and img2img, Fit & Native) with a PiD guard, base-aware gating/decoder-filter (sd-3), and SD3 readiness checks. Add two nvidia/PiD SD3 starter decoders (2K, 2Kto4K). Harden the PiD config probe against the 16-channel FLUX.1/SD3 ambiguity: when the checkpoint's directory name is silent (the HF single-file download renames it), trust an explicit base override so SD3 checkpoints are not misidentified as FLUX.1. Also benefits Qwen. FLUX / FLUX.2 identification is unchanged. * feat(pid): add SDXL PiD 4x-SR decode support Build the full SDXL PiD backend stack: _PER_BACKBONE[SDXL] (4ch/down8), PiDDecoder_Checkpoint_SDXL_Config with a 4-channel latent-map entry, factory union + loader registration, and a new sdxl_pid_decode node (reads the VAE's scaling_factor/shift at runtime; SDXL fallbacks 0.13025/0.0). 4-channel latents are unambiguous, so no directory-name disambiguation is needed. Generalize the shared PiD decode chain to support SD-family denoise: denoise_latents has no width/height, so thread an optional noise node for sizing and round to the model's native grid (8 for SDXL, 16 for FLUX). Wire buildSDXLGraph (txt2img + img2img, Fit & Native) with the VAE as the decode's scaling source, base-aware gating/readiness, and a starter decoder (SDXL 2Kto4K only). PiD + SDXL refiner is blocked for now via a graph guard and a readiness reason. FLUX/FLUX.2/SD3 paths are unchanged. * feat(pid): add Z-Image PiD 4x-SR decode support Wire the existing z_image_pid_decode node into the Z-Image graph builder (txt2img and img2img, Fit & Native) with a PiD guard and readiness checks. Z-Image shares FLUX.1's 16-channel VAE and has no PiD checkpoints of its own, so it reuses the FLUX decoder: the decoder filter maps z-image -> flux, showing FLUX PiD decoders when a Z-Image model is active. The Z-Image VAE is passed to the decode node so it reads the real scaling_factor / shift instead of the fallback constants. No backend, schema, or starter-model changes. FLUX/FLUX.2/SD3/SDXL paths are unchanged. * feat(pid): add Qwen-Image PiD 4x-SR decode support Build the full Qwen-Image PiD backend stack: _PER_BACKBONE[QwenImage] (16ch/down8), PiDDecoder_Checkpoint_QwenImage_Config (added to the 16-channel latent map + filename heuristic), factory union + loader registration, and a new qwen_image_pid_decode node. Unlike the scalar-scaling bases, the Qwen-Image VAE normalizes per channel (latents_mean / latents_std) and stores a 5D video-style latent, so the node denormalizes per-channel (z * std + mean, read from the VAE config) and drops the singleton temporal frame before decoding - matching qwen_image_l2i. Wire buildQwenImageGraph (txt2img + img2img, Fit & Native) with the Qwen-Image VAE as the decode's normalization source, base-aware gating/readiness, and a starter decoder (Qwen-Image 2Kto4K only). The 16-channel FLUX/SD3/Qwen ambiguity is handled by the existing trusted- base-override probe hardening. FLUX/FLUX.2/SD3/SDXL/Z-Image paths are unchanged. * Chore Ruff * Add Docs * fix(pid): green up frontend tests and knip for the PiD branch - graph-builder tests: set pidMode 'off' in the FLUX / Qwen-Image / SDXL+SD3 param fixtures so the PiD guard doesn't fire on an undefined pidMode and call the (unmocked) size helpers - paramsSlice migration test: expect _version 4 (v3→v4 adds the PiD fields) - remove the unused setPidSteps action and selectPidSteps selector flagged by knip; the pidSteps state field stays at its default of 4 * Chore openapi + typegen * Merge remote-tracking branch 'origin/main' into feature/svd-quantization Resolve conflicts in qwen3_encoder.py (keep both SDNQ tensor/key detection and the Qwen-VL visual-tower guard) and regenerate openapi.json + schema.ts from the merged backend. Fix T5TokenizerFast under transformers 5.x: it is now an alias of T5Tokenizer. Use T5Tokenizer in flux.py (was used unimported -> F821) and also match the new "T5Tokenizer" class name in the SDNQ FLUX model_index.json submodel probe. * Fix SDNQ mixed-group dequant, Z-Image self-contained pipelines, and FLUX.2 single-file identification Address three SDNQ review findings (PR #9228): - loaders.py: derive per-tensor group_size from the scale tensor's group dimension instead of trusting quantization_config.json. Dynamic-mixed models (e.g. FLUX.2 Klein 4B) quantize some layers with a group size that differs from the nominal config value; trusting the config produced a num_groups that disagreed with scale.shape[1] and broke the reshape/ broadcast in dequantize_uint4/int5_per_group. - z_image_model_loader: fall back to the main model for the VAE/Qwen3 submodels when it is a self-contained SDNQ pipeline (format=sdnq_quantized with submodels). Mirror the same self-contained handling in the frontend readiness checks and buildZImageGraph, so a freshly installed SDNQ Z-Image pipeline generates without manually selecting a component source. - FLUX.2 single-file identification: list SDNQ FLUX.2 before FLUX.1 in the config union, reject FLUX.2 state dicts in Main_SDNQ_FLUX_Config so the two stay mutually exclusive regardless of iteration order, and accept the bare diffusers transformer layout (transformer_blocks./context_embedder.) in Main_SDNQ_Flux2_Config so the real single-file SDNQ FLUX.2 checkpoint is identified instead of falling through to FLUX.1 or unknown. Add regression tests for all three: mixed-group uint4 dequant, the Z-Image loader/readiness self-contained fallback, and SDNQ FLUX.2 identification. * Chore typegen + openapi * Fix SDNQ mixed-group dequant, Z-Image self-contained pipelines, and FLUX.2/Z-Image identification Address five SDNQ review findings (PR #9228): - loaders.py: derive per-tensor group_size from the scale tensor's group dimension instead of trusting quantization_config.json. Dynamic-mixed models (e.g. FLUX.2 Klein 4B) quantize some layers with a group size that differs from the nominal config value; trusting the config produced a num_groups that disagreed with scale.shape[1] and broke the reshape/ broadcast in dequantize_uint4/int5_per_group. - z_image_model_loader: fall back to the main model for the VAE/Qwen3 submodels when it is a self-contained SDNQ pipeline (format=sdnq_quantized with submodels). Mirror the same self-contained handling in the frontend readiness checks and buildZImageGraph, so a freshly installed SDNQ Z-Image pipeline generates without manually selecting a component source. - FLUX.2 single-file identification: list SDNQ FLUX.2 before FLUX.1 in the config union, reject FLUX.2 state dicts in Main_SDNQ_FLUX_Config so the two stay mutually exclusive regardless of iteration order, and accept the bare diffusers transformer layout (transformer_blocks./context_embedder.) in Main_SDNQ_Flux2_Config so the real single-file SDNQ FLUX.2 checkpoint is identified instead of falling through to FLUX.1 or unknown. - Main_Diffusers_ZImage_Config: reject SDNQ-quantized ZImagePipeline folders so Main_SDNQ_Diffusers_ZImage_Config matches them instead (mirrors the FLUX and FLUX.2 guards). Without this a full SDNQ Z-Image pipeline could be classified as plain diffusers, mis-reading packed weights and breaking the self-contained path. - z_image loader: check missing/unexpected keys after load_state_dict in the SDNQ transformer folder path and fail fast with the offending key list, instead of silently returning a model with required params on meta tensors that fails later during device movement or inference. Add regression tests for all five: mixed-group uint4 dequant, the Z-Image loader/readiness self-contained fallback, SDNQ FLUX.2 identification, and SDNQ ZImagePipeline-folder identification. * Harden SDNQ model identification, loading, and Z-Image pipeline support Address the SDNQ review findings on the feature branch: Dequantization / loading correctness - sdnq/loaders.py: derive per-tensor group_size from the scale tensor's group dimension instead of trusting quantization_config.json. Dynamic-mixed models (e.g. FLUX.2 Klein 4B) quantize some layers with a group size that differs from the nominal config value; trusting the config produced a num_groups that disagreed with scale.shape[1] and broke the reshape/broadcast in dequantize_uint4/int5_per_group. - Add shared raise_on_incomplete_sdnq_load() and apply it to every SDNQ folder loader that used load_state_dict(strict=False) without checking the result: FLUX.2 pipeline transformer, SDNQ CLIP/T5/VAE (flux.py), standalone SDNQ VAE (vae.py), and both Z-Image Qwen3 text-encoder paths (z_image.py). A partial or mismatched export now fails fast with the offending key list instead of returning a model with parameters left on the meta device. Tied/re-shared weights (T5 encoder.embed_tokens, Qwen3 lm_head, CLIP position_ids) are allow-listed. Model identification - Reject FLUX.2 in Main_SDNQ_FLUX_Config and accept the bare diffusers transformer layout (transformer_blocks./context_embedder.) in Main_SDNQ_Flux2_Config, and list FLUX.2 before FLUX.1 in the config union, so a single-file SDNQ FLUX.2 checkpoint identifies as FLUX.2 instead of falling through to FLUX.1 or unknown. - Main_Diffusers_ZImage_Config: reject SDNQ-quantized ZImagePipeline folders so Main_SDNQ_Diffusers_ZImage_Config matches them instead (mirrors the FLUX and FLUX.2 guards). - Qwen3Encoder_SDNQ_Folder_Config: verify the folder is actually a Qwen3 encoder (Qwen3 config class name or Qwen3 state-dict keys) before claiming it, so an SDNQ transformer/VAE folder is not stored as type=qwen3_encoder. - Qwen3Encoder_Qwen3Encoder_Config: reject SDNQ folders so it no longer competes with the SDNQ folder config (both share the Qwen3Encoder type, making the factory tiebreak non-deterministic). Z-Image self-contained SDNQ pipelines - z_image_model_loader: fall back to the main model for the VAE/Qwen3 submodels when it is a self-contained SDNQ pipeline (format=sdnq_quantized with submodels), and drop the ui_model_format=diffusers hint from qwen3_source_model so SDNQ pipelines are not filtered out of the node/workflow model pickers. - Mirror the self-contained handling in the frontend readiness checks and buildZImageGraph so a freshly installed SDNQ Z-Image pipeline generates without a manually selected component source. Add regression tests for every fix: mixed-group uint4 dequant, the load-guard helper, SDNQ FLUX.2 identification, SDNQ ZImagePipeline-folder identification, Qwen3 encoder folder identification, the Z-Image self-contained loader/readiness path, and the qwen3_source_model field template. * Chore Typegen + OpenApi * Harden SDNQ model identification, loading, and Z-Image pipeline support Address the SDNQ review findings on the feature branch: Dequantization / loading correctness - sdnq/loaders.py: derive per-tensor group_size from the scale tensor's group dimension instead of trusting quantization_config.json. Dynamic-mixed models (e.g. FLUX.2 Klein 4B) quantize some layers with a group size that differs from the nominal config value; trusting the config produced a num_groups that disagreed with scale.shape[1] and broke the reshape/broadcast in dequantize_uint4/int5_per_group. - Add shared raise_on_incomplete_sdnq_load() and use it in every SDNQ folder loader that called load_state_dict(strict=False) without checking the result: FLUX.2 pipeline transformer, SDNQ CLIP/T5/VAE (flux.py), standalone SDNQ VAE (vae.py), and both Z-Image Qwen3 text-encoder paths (z_image.py). A partial or mismatched export now fails fast with the offending key list instead of returning a model with parameters left on the meta device. Tied/re-shared weights (T5 encoder.embed_tokens, Qwen3 lm_head, CLIP position_ids) are allow-listed. Replace the T5 SDNQ assert checks (stripped by python -O) and add the missing unexpected-key check to the standalone Qwen3 SDNQ loader. Model identification - Reject FLUX.2 in Main_SDNQ_FLUX_Config and accept the bare diffusers transformer layout (transformer_blocks./context_embedder.) in Main_SDNQ_Flux2_Config, and list FLUX.2 before FLUX.1 in the config union, so a single-file SDNQ FLUX.2 checkpoint identifies as FLUX.2 instead of falling through to FLUX.1 or unknown. - Main_Diffusers_ZImage_Config: reject SDNQ-quantized ZImagePipeline folders so Main_SDNQ_Diffusers_ZImage_Config matches them instead (mirrors the FLUX/FLUX.2 guards). - Qwen3Encoder_SDNQ_Folder_Config: verify the folder is actually a Qwen3 encoder (Qwen3 config class or Qwen3 state-dict keys) and reject complete causal LMs (root config.json + tokenizer files) so an SDNQ transformer/VAE/TextLLM folder is not stored as type=qwen3_encoder. - Qwen3Encoder_Qwen3Encoder_Config: reject SDNQ folders so it no longer competes with the SDNQ folder config (both share the Qwen3Encoder type, making the factory tiebreak non-deterministic). Z-Image self-contained SDNQ pipelines - z_image_model_loader: fall back to the main model for the VAE/Qwen3 submodels when it is a self-contained SDNQ pipeline, but only when the pipeline actually exposes the required vae + text_encoder + tokenizer submodels (a truthy submodels dict is not enough — a partial pipeline would fail later on missing folders). Drop the ui_model_format=diffusers hint from qwen3_source_model so SDNQ pipelines are not filtered out of the node/workflow model pickers. - Mirror both the self-contained handling and the specific-submodels requirement in the frontend readiness checks and buildZImageGraph, so a freshly installed SDNQ Z-Image pipeline generates without a manually selected component source. Add regression tests for every fix: mixed-group uint4 dequant, the load-guard helper, SDNQ FLUX.2 identification, SDNQ ZImagePipeline-folder identification, Qwen3 encoder folder identification (SDNQ rejection, causal-LM rejection), the Z-Image self-contained loader/readiness path incl. partial-pipeline handling, and the qwen3_source_model field template. * Chore Ruff * Harden SDNQ model identification, loading, and Z-Image / FLUX.2 pipeline support Address the SDNQ review findings on the feature branch: Dequantization / loading correctness - sdnq/loaders.py: derive per-tensor group_size from the scale tensor's group dimension instead of trusting quantization_config.json. Dynamic-mixed models (e.g. FLUX.2 Klein 4B) quantize some layers with a group size that differs from the nominal config value; trusting the config produced a num_groups that disagreed with scale.shape[1] and broke the reshape/broadcast in dequantize_uint4/int5_per_group. - Add shared raise_on_incomplete_sdnq_load() and use it in every SDNQ folder loader that called load_state_dict(strict=False) without checking the result: FLUX.2 pipeline transformer, SDNQ CLIP/T5/VAE (flux.py), standalone SDNQ VAE (vae.py), and both Z-Image Qwen3 text-encoder paths (z_image.py). A partial or mismatched export now fails fast with the offending key list instead of returning a model with parameters left on the meta device. Tied/re-shared weights (T5 encoder.embed_tokens, Qwen3 lm_head, CLIP position_ids) are allow-listed. Replace the T5 SDNQ assert checks (stripped by python -O) and add the missing unexpected-key check to the standalone Qwen3 SDNQ loader. Model identification - Reject FLUX.2 in Main_SDNQ_FLUX_Config and accept the bare diffusers transformer layout (transformer_blocks./context_embedder.) in Main_SDNQ_Flux2_Config, and list FLUX.2 before FLUX.1 in the config union, so a single-file SDNQ FLUX.2 checkpoint identifies as FLUX.2 instead of falling through to FLUX.1 or unknown. - Main_Diffusers_ZImage_Config: reject SDNQ-quantized ZImagePipeline folders so Main_SDNQ_Diffusers_ZImage_Config matches them instead (mirrors the FLUX/FLUX.2 guards). - Qwen3Encoder_SDNQ_Folder_Config: verify the folder is actually a Qwen3 encoder, reject complete causal LMs (root config.json + tokenizer files), accept the same compatible Qwen architectures as the unquantized config (Qwen2VLForConditionalGeneration / Qwen2ForCausalLM / Qwen3ForCausalLM via a shared constant), and make the state-dict fallback resilient to sharded folders, so an SDNQ transformer/VAE/TextLLM folder is not stored as type=qwen3_encoder and a compatible sharded encoder is not wrongly rejected. - Qwen3Encoder_Qwen3Encoder_Config: reject SDNQ folders so it no longer competes with the SDNQ folder config (both share the Qwen3Encoder type, making the factory tiebreak non-deterministic). Z-Image / FLUX.2 self-contained SDNQ pipelines - Add a shared is_self_contained_sdnq_pipeline() helper requiring the specific VAE + Qwen3 (text_encoder + tokenizer) submodels a pipeline install must ship. A truthy submodels dict is not enough: Main_SDNQ_Diffusers_* configs record whichever submodels they recognize, so a partial pipeline can expose only the transformer and would fail at runtime on missing fixed subfolders. - z_image_model_loader and flux2_klein_model_loader: use the helper for both the main-model self-contained fallback and the explicit qwen3_source validator, so a partial pipeline (as main model or selected source) requires an explicit VAE/Qwen3 source instead of emitting requests against missing folders. - Drop the ui_model_format=diffusers hint from z_image qwen3_source_model so SDNQ pipelines are not filtered out of the node/workflow model pickers. - Mirror the same specific-submodels requirement in the frontend readiness checks (Z-Image + FLUX.2, generate and canvas tabs) and buildZImageGraph, and bring the FLUX.2 canvas readiness block to parity with the generate tab (it previously used format !== 'diffusers' and never recognized SDNQ pipelines). Add regression tests for every fix: mixed-group uint4 dequant, the load-guard helper, SDNQ FLUX.2 identification, SDNQ ZImagePipeline-folder identification, Qwen3 encoder folder identification (SDNQ rejection, causal-LM rejection, compatible Qwen2 class names, sharded folders), the Z-Image and FLUX.2 self-contained loader/readiness paths incl. partial-pipeline handling, and the qwen3_source_model field template. * Harden SDNQ model identification, loading, and Z-Image / FLUX.2 pipeline support Address the SDNQ review findings on the feature branch: Dequantization / loading correctness - sdnq/loaders.py: derive per-tensor group_size from the scale tensor's group dimension instead of trusting quantization_config.json. Dynamic-mixed models (e.g. FLUX.2 Klein 4B) quantize some layers with a group size that differs from the nominal config value; trusting the config produced a num_groups that disagreed with scale.shape[1] and broke the reshape/broadcast in dequantize_uint4/int5_per_group. - Add shared raise_on_incomplete_sdnq_load() and use it in every SDNQ folder loader that called load_state_dict(strict=False) without checking the result: FLUX.2 pipeline transformer, SDNQ CLIP/T5/VAE (flux.py), standalone SDNQ VAE (vae.py), and both Z-Image Qwen3 text-encoder paths (z_image.py). A partial or mismatched export now fails fast with the offending key list instead of returning a model with parameters left on the meta device. Tied/re-shared weights (T5 encoder.embed_tokens, Qwen3 lm_head, CLIP position_ids) are allow-listed. Replace the T5 SDNQ assert checks (stripped by python -O) and add the missing unexpected-key check to the standalone Qwen3 SDNQ loader. Model identification - Reject FLUX.2 in Main_SDNQ_FLUX_Config and accept the bare diffusers transformer layout (transformer_blocks./context_embedder.) in Main_SDNQ_Flux2_Config, and list FLUX.2 before FLUX.1 in the config union, so a single-file SDNQ FLUX.2 checkpoint identifies as FLUX.2 instead of falling through to FLUX.1 or unknown. - Main_Diffusers_ZImage_Config: reject SDNQ-quantized ZImagePipeline folders so Main_SDNQ_Diffusers_ZImage_Config matches them instead (mirrors the FLUX/FLUX.2 guards). - Qwen3Encoder_SDNQ_Folder_Config: verify the folder is actually a Qwen3 encoder, reject complete causal LMs (root config.json + tokenizer files), accept the same compatible Qwen architectures as the unquantized config (Qwen2VLForConditionalGeneration / Qwen2ForCausalLM / Qwen3ForCausalLM via a shared _QWEN3_ENCODER_ARCHITECTURES constant), and make the state-dict fallback resilient to sharded folders — so an SDNQ transformer/VAE/TextLLM folder is not stored as type=qwen3_encoder and a compatible sharded encoder is not wrongly rejected. - Qwen3Encoder_Qwen3Encoder_Config: reject SDNQ folders so it no longer competes with the SDNQ folder config (both share the Qwen3Encoder type, making the factory tiebreak non-deterministic). SDNQ pipeline submodel discovery - Main_SDNQ_Diffusers_Flux2_Config / Main_SDNQ_Diffusers_ZImage_Config _get_submodels(): record the TextEncoder for any compatible encoder class (_QWEN3_ENCODER_ARCHITECTURES) and the Tokenizer for the slow and fast Qwen2 tokenizer classes, not just Qwen3ForCausalLM / Qwen2Tokenizer. Otherwise a valid pipeline advertising e.g. Qwen2ForCausalLM or Qwen2TokenizerFast was recorded as partial and forced to use separate VAE/Qwen3 sources. Z-Image / FLUX.2 self-contained SDNQ pipelines - Add a shared is_self_contained_sdnq_pipeline() helper requiring the specific VAE + Qwen3 (text_encoder + tokenizer) submodels a pipeline install must ship. A truthy submodels dict is not enough: a partial (or partially recognized) pipeline can expose only the transformer and would fail at runtime on missing fixed subfolders. - z_image_model_loader and flux2_klein_model_loader: use the helper for both the main-model self-contained fallback and the explicit qwen3_source validator, so a partial pipeline (as main model or selected source) requires an explicit VAE/Qwen3 source instead of emitting requests against missing folders. - Drop the ui_model_format=diffusers hint from z_image qwen3_source_model so SDNQ pipelines are not filtered out of the node/workflow model pickers. - Mirror the same specific-submodels requirement in the frontend readiness checks (Z-Image + FLUX.2, generate and canvas tabs) and buildZImageGraph, and bring the FLUX.2 canvas readiness block to parity with the generate tab (it previously used format !== 'diffusers' and never recognized SDNQ pipelines). Add regression tests for every fix: mixed-group uint4 dequant, the load-guard helper, SDNQ FLUX.2 identification, SDNQ ZImagePipeline-folder identification, Qwen3 encoder folder identification (SDNQ rejection, causal-LM rejection, compatible Qwen2 class names, sharded folders), SDNQ pipeline submodel discovery across compatible encoder/tokenizer classes, the Z-Image and FLUX.2 self-contained loader/readiness paths incl. partial-pipeline handling, and the qwen3_source_model field template. * Exclude Qwen2-VL encoder from SDNQ pipeline self-contained discovery The SDNQ FLUX.2 / Z-Image pipeline loaders instantiate a text-only Qwen3ForCausalLM for the discovered text_encoder/ folder, so they cannot load a Qwen2VLForConditionalGeneration model (multimodal, visual tower). Recording it as a self-contained TextEncoder marked the pipeline complete even though the loader would fail on the visual-tower weights, causing readiness/invocation to select the main model as the Qwen source. Narrow _get_submodels()'s accepted TextEncoder classes to the text-only Qwen causal-LM classes, excluding the Qwen-VL class. * Harden SDNQ model identification, loading, dequantization, and cache accounting Address the SDNQ review findings on the feature branch: Dequantization / tensor correctness - sdnq/loaders.py: derive per-tensor group_size from the scale tensor's group dimension instead of trusting quantization_config.json, so dynamic-mixed models (e.g. FLUX.2 Klein 4B) whose per-layer group size differs from the config value no longer break the reshape/broadcast in dequantize_uint4/int5_per_group. - sdnq/utils.py: normalize 2D [out_features, num_groups] scale/zero_point with a trailing singleton before per-group arithmetic; a 2D param previously right-aligned against the 3D grouped weight and failed broadcasting. - sdnq/sdnq_tensor.py: when an SDNQTensor is moved with .to(device) (aten _to_copy), move its scale/zero_point/svd payloads too. Previously only quantized_data moved, leaving auxiliary tensors in system RAM and forcing a host->device copy of all of them on every dequantization of every layer, every step. - Add SDNQTensor.sdnq_storage_nbytes() and make calc_tensor_size() SDNQ-aware (duck-typed) so cache eviction / partial-load budgets use real storage (packed data + scale + zero_point + svd) instead of the wrapper's dequantized shape × uint8, which over-counted packed weights and omitted the auxiliary tensors. Loader robustness - Add shared raise_on_incomplete_sdnq_load() and use it in every SDNQ folder loader that called load_state_dict(strict=False) without checking the result (FLUX.2 pipeline transformer, SDNQ CLIP/T5/VAE, standalone SDNQ VAE, both Z-Image Qwen3 text-encoder paths). Partial/mismatched exports now fail fast with the offending keys. Tied/re-shared weights are allow-listed; replace the T5 assert checks (stripped by python -O) and add the missing unexpected-key check to the standalone Qwen3 SDNQ loader. - vae.py: SDNQ VAE detection now inspects all safetensors shards (union of keys) and the quantization_config.json marker, and _load_sdnq_vae hands sharded directories to sdnq_sd_loader — a VAE split across standard shard files is detected and loaded instead of falling through to the generic diffusers loader. Model identification - Single-file SDNQ FLUX.2: accept only the bare diffusers layout (transformer_blocks./context_embedder.) that the loader supports and reject BFL / ComfyUI (model.diffusion_model.double_blocks.*) checkpoints, which the loader cannot convert. Reject FLUX.2 in Main_SDNQ_FLUX_Config and list FLUX.2 before FLUX.1 in the config union so an ambiguous checkpoint never reaches the FLUX.1 loader. - Main_Diffusers_ZImage_Config: reject SDNQ-quantized ZImagePipeline folders so the SDNQ config matches them instead (mirrors the FLUX/FLUX.2 guards). - Qwen3Encoder_Qwen3Encoder_Config: reject SDNQ folders so it no longer competes (non-deterministically) with the SDNQ folder config. - Qwen3Encoder_SDNQ_Folder_Config: verify the folder is a Qwen3 encoder, reject complete causal LMs (root config.json + tokenizer files), and accept only the architecture the SDNQ loader can build (Qwen3ForCausalLM) — rejecting a declared Qwen2ForCausalLM / Qwen2VLForConditionalGeneration folder rather than accepting one the loader's strict guard would fail on. SDNQ pipeline submodel discovery - Main_SDNQ_Diffusers_Flux2_Config / Main_SDNQ_Diffusers_ZImage_Config _get_submodels(): record the Tokenizer for the slow/fast Qwen2 tokenizer classes, and the TextEncoder only for the loadable Qwen3ForCausalLM class. A pipeline advertising Qwen2ForCausalLM or the multimodal Qwen2VLForConditionalGeneration is therefore not marked self-contained, since the pipeline loaders build a text-only Qwen3ForCausalLM. Self-contained SDNQ pipelines (Z-Image / FLUX.2) - Add shared is_self_contained_sdnq_pipeline() requiring the specific VAE + Qwen3 (text_encoder + tokenizer) submodels; a truthy submodels dict is not enough. - z_image_model_loader and flux2_klein_model_loader use it for both the main-model fallback and the explicit qwen3_source validator; drop the ui_model_format=diffusers hint from z_image qwen3_source_model so SDNQ pipelines aren't filtered out of the pickers. - Mirror the specific-submodels requirement in the frontend readiness checks (Z-Image + FLUX.2, generate and canvas tabs) and buildZImageGraph, and bring the FLUX.2 canvas readiness block to parity with the generate tab. Docs - sdnq-quantization.mdx: stop recommending the unsupported FLUX.2 dev SDNQ export; recommend a supported Klein 9B export and note FLUX.2 dev is not implemented. Add regression tests for every fix: mixed-group and 2D-scale uint4/int5 dequant, device movement of all payloads, SDNQ-aware size accounting, the load-guard helper, sharded SDNQ VAE detection, SDNQ FLUX.2 identification (bare accepted / BFL rejected), SDNQ ZImagePipeline-folder identification, Qwen3 encoder folder identification (SDNQ/causal-LM rejection, Qwen3-only architecture), SDNQ pipeline submodel discovery (loadable vs unloadable encoder classes, fast tokenizer), the self-contained loader/readiness paths incl. partial-pipeline handling, and the qwen3_source_model field template. * Chore Ruff * docs: regenerate invocation-context data for offload_from_vram * Fix: address SDNQ review merge blockers (partial pipelines, Qwen/T5 identification, cat/stack) - SDNQ FLUX.2 & Z-Image submodel discovery now records a component only when its subfolder exists on disk, not merely because model_index.json advertises it. A partial download that keeps the full index no longer classifies as a self-contained pipeline and then fails in the loader on missing fixed vae/, text_encoder/, tokenizer/ subfolders. - Single-file SDNQ Qwen encoder identification now requires Qwen3-only QK-norm (q_norm/k_norm) weights and rejects a bundled Qwen-VL visual tower, so a Qwen2 causal LM or a Qwen-VL checkpoint no longer resolves to Qwen3Encoder_SDNQ_Config (the loader builds a text-only Qwen3ForCausalLM and would fail on either). - SDNQ T5 encoder: share resolve_text_encoder_dir/resolve_tokenizer_dir so the tokenizer is located per layout (child in the standalone bundle, sibling in the inline text_encoder_2 layout) instead of always <path>/tokenizer_2, and reject an install with no resolvable tokenizer_2 at identification time. - SDNQTensor.dequantize_and_run recurses into list/tuple args so torch.cat / torch.stack dequantize their operands instead of redispatching onto the SDNQTensors until RecursionError. Adds identification tests for partial pipelines, single-file Qwen2/Qwen-VL rejection, inline/standalone T5 tokenizer resolution, and torch.cat/torch.stack. * fix(pid): resolve PiD decoder review findings (state, readiness, steps, race) Address all reviewer findings on the PiD decoder feature: - Guard offload_model_from_vram with @synchronized, matching its sibling drop_model, to prevent a race when the VRAM working set is mutated concurrently during model swaps. - On base change (modelChanged), clear a PiD decoder that is incompatible with the new main model's decoder base at the root, respecting the Z-Image -> FLUX decoder reuse so a still-valid decoder is kept. - On switching to a base without PiD support, reset pidMode to 'off' and refit the dimensions so no hidden 4x native grid survives. - Extend the scaled-grid bbox readiness validation to SD3, SDXL and Z-Image, mirroring the existing FLUX.2 native-grid check. - Add a PiD Steps control (slider + number input, 1-8, default 4) with a pidStepsChanged action and selectPidSteps selector, so the documented step count is actually configurable and flows into the graph. Add readiness and paramsSlice tests covering the scaled-grid blocking and the base-change decoder/pidMode behavior * fix(pid): address review — cap steps at 4, honor encoder device, decoder/base guards Merge blockers: - PiD steps 5-8 produced duplicate timesteps: the student schedule has only 4 transitions (a 5-point list), so sub-sampling to >4 steps rounded distinct indices onto the same point and wasted network forwards on repeated timesteps. Cap the public range at 4 across the backend fields (le=4), the UI slider/input (max=4), and the pidSteps zod schema (int, 1-4), and harden _get_t_list with a strictly-decreasing assertion as a safety net. - CPU-only Gemma encoders crashed on CUDA hosts: each PiD invocation passed the global compute device to caption encoding, pushing the tokenizer output to CUDA while a cpu_only encoder stayed on the CPU. Encode on the encoder's actual device instead (next(encoder.parameters()).device), honoring model_on_device(). - Gemma 2 could no longer be configured as a generic TextLLM: the specialised- architecture exclusion rejected Gemma2ForCausalLM unconditionally. Only defer to the encoder config during automatic classification; keep an explicit type=text_llm request valid (the generic causal-LM loader supports it). Follow-ups folded in: - Reject incompatible Gemma 2 sizes before execution: PiD's caption projection is fixed at Gemma-2-2b's 2304-dim hidden state, so the encoder config now rejects 9B (3584) / 27B (4608) up front instead of failing deep in inference. - Validate the PiD decoder's base against each base-specific decode node: the base-agnostic loader let the Nodes editor wire any decoder into any node. Add assert_pid_decoder_matches_base and call it in all seven decode nodes, preserving the Z-Image-reuses-FLUX-decoder case (its node backbone is FLUX). Add regression tests: the distill schedule (strictly decreasing 1-4, safety net trips at 5) and decoder/base validation; the Gemma2 hidden-size gate; and TextLLM classification (auto-defers, explicit type still matches, plain causal LMs match). The expand-prompt pipeline always sent a dedicated "system" role message, which some chat templates (notably Gemma) reject with "System role not supported", 500-ing prompt expansion for those models. When applying the chat template fails with a system-role error, fold the system prompt into the first user turn and retry instead of failing. Adds regression tests for both the fallback and the normal (system-supported) path. * Chore fix * fix(pid): keep large Gemma2 as TextLLM, raise (not assert) schedule guard, compute_device, narrow pid_upscale VAE Address the latest review on the PiD PR: - Merge blocker: automatic classification sent Gemma 2 9B/27B to Unknown. The PiD Gemma2 encoder config rejects their non-2304 hidden size, and TextLLM deferred *every* Gemma2ForCausalLM, so neither matched. TextLLM now defers only the size the encoder config accepts (2304 = Gemma-2-2b); larger variants stay TextLLM. - Schedule safety net used assert, which is stripped under `python -O`, leaving _get_t_list(num_steps=5) returning a duplicate schedule. Raise ValueError instead so the guard holds in optimized runtimes; the regression test now asserts ValueError and passes under `python -O`. - All seven PiD caption paths derived the Gemma device from the first parameter, which is wrong under partial loading (first param on CPU, later modules on CUDA). Use the cache contract's LoadedModel.compute_device instead. - pid_upscale advertised Z-Image / 16-channel VAEs but delegates to the FLUX-only vae_encode. Narrow the field description and validate the VAE is a FLUX AutoEncoder up front (a diffusers AutoencoderKL now fails with a clear error instead of a stripped-assert failure inside vae_encode). Update the TextLLM/Gemma2 tests (per-size config-level + a factory-level check that 2304 -> Gemma2Encoder and 3584/4608 -> TextLLM) and the schedule test (ValueError, green under python -O). * chore: regenerate OpenAPI schema and frontend types * feat(pid): accept single-file GGUF Gemma-2-2b as the PiD caption encoder The PiD Gemma encoder was directory + HuggingFace only, so a llama.cpp GGUF (e.g. gemma-2-2b-it-Q4_K_M.gguf) could not be used. Add GGUF support: - Gemma2Encoder_GGUF_Config: identifies a single .gguf file, reads the GGUF metadata and requires general.architecture == "gemma2" and <arch>.embedding_length == 2304 (Gemma-2-2b), rejecting 9B/27B as the directory config does. - Gemma2EncoderGGUFLoader (format gguf_quantized): loads via transformers from_pretrained(<dir>, gguf_file=<name>) — transformers dequantizes gemma2 GGUFs and reads the tokenizer from the GGUF metadata — then exposes the Gemma2Model decoder, matching the directory loader. PiD encodes the caption once and offloads the encoder, so dequantizing at load is acceptable. - Register the config in the AnyModelConfig union. No frontend change: the PiD encoder picker filters by type=gemma2_encoder, so the GGUF variant appears automatically. Verified end-to-end against a real q4_k_m file: it classifies as Gemma2Encoder_GGUF_Config and loads to a Gemma2Model producing 2304-dim hidden states. Adds config identification tests (match, 9B/27B rejected, non-gemma2 rejected, non-.gguf rejected). * Chore Ruff * fix(models): stop Qwen3 GGUF config from matching Gemma-2 GGUF encoders A Gemma-2 GGUF satisfies the generic Qwen3 GGUF key heuristic (token_embd.weight + blk.* keys), so it matched both Qwen3Encoder_GGUF_Config and the intended Gemma2Encoder_GGUF_Config. On a fresh install the Gemma config happened to win, but re-identification could pick Qwen3, mis-classifying the model. Add _has_gemma2_keys (Gemma uses blk.*.post_attention_norm / post_ffw_norm, which a Qwen3 encoder never has — Qwen3 has attn_q_norm/attn_k_norm instead) and reject such state dicts in both Qwen3 encoder configs' _validate_looks_like_qwen3_model (GGUF and checkpoint), mirroring the existing T5 / Qwen-VL exclusions. The Gemma config already rejects Qwen3 GGUFs via the general.architecture metadata, so the two are now mutually exclusive and identification is deterministic. Add regression tests: _has_gemma2_keys detection and that the Qwen3 GGUF config rejects a Gemma-keyed state dict. * test(models): assert re-identified Gemma GGUF drops the stale Qwen3 variant The Gemma2 GGUF encoder config has no `variant` field, so re-identifying a model previously mis-detected as a Qwen3 GGUF (which carries a variant) drops it — the serialized record has no variant key and replace_model overwrites it away. Assert this explicitly in the Gemma GGUF identification test. * fix(pid): point 2K-to-4K starter decoders at NVIDIA's v1.5 replacements NVIDIA deprecated the FLUX / FLUX.2 / Qwen-Image `res2kto4k_sr4x` PiD decoders and moved them to `checkpoints_deprecated/`, replacing them with the recommended `v1pt5_res2kto4k_sr4x` checkpoints. Our starter models still pointed at the old `checkpoints/` paths, which now 404 on install. Repoint the three affected 2K-to-4K starters (FLUX, FLUX.2, Qwen-Image) to the v1.5 successors and note the upgrade in their descriptions. The 2K (`res2k_sr4x`) decoders and the SD3 / SDXL 2K-to-4K decoders are not deprecated and are unchanged. Base and variant are still sent as explicit overrides, so config identification is unaffected by the new directory name (res2kto4k -> Res2kTo4k_Sr4x). * feat(pid): load Gemma-2 GGUF encoders natively (keep weights quantized) The GGUF Gemma encoder used transformers' from_pretrained(gguf_file=...), which dequantizes every weight at load — so a quantized Gemma cost the same VRAM as the unquantized model. Load it via InvokeAI's GGMLTensor path instead: read the config from GGUF metadata, map llama.cpp tensor names to Gemma2Model, and keep the 2D projection weights as GGMLTensor (dequantized on demand by the model cache). Materialize only the embedding and the RMSNorm weights, subtracting 1 from the norms (llama.cpp folds +1 in; Gemma2RMSNorm re-adds it), and assert nothing is left on meta. Verified: hidden states match the fully-dequantized loader within quantization tolerance. Adds key-mapping tests and a local load/compare test. NVIDIA's v1.5 decoders use a different network (lq_hidden_dim=1024, PiT injection) that build_pid_net (512-dim legacy) cannot load, causing a size-mismatch crash. - Point the FLUX/FLUX.2/Qwen 2K-to-4K starters back at the legacy checkpoints (moved to checkpoints_deprecated/) that the current network loads. - Reject a checkpoint whose lq_proj hidden dim is not the supported 512 at identification time, instead of accepting it and failing inside the decode. - Enumerate all supported backbones (add SDXL, Qwen-Image) in the loader title and correct the variant enum docs (not every backbone ships both presets). Full v1.5 architecture support is planned as a follow-up. Adds PiD decoder identification tests (legacy accepted, 1024-dim v1.5 rejected). * Chore openapi + typegen * Docs update * Chore Ruff format * Chore openapi * fix(sdnq): harden self-contained pipeline detection (review 9228) Addresses the four merge blockers from the PR #9228 review and checks in the reviewer's failing tests. - _get_submodels (FLUX.2 + Z-Image) only checked is_dir(), so the empty component folders an interrupted download leaves behind were recorded as present. Require the files the component's loader actually needs: config + weights for transformer/text_encoder/vae, a vocab/config file for the weightless tokenizer folder. - is_self_contained_sdnq_pipeline() did not require the Transformer submodel, so a malformed model_index.json advertising only VAE/encoder/ tokenizer passed the check even though every loader requests the transformer. - Qwen3Encoder_SDNQ_Folder_Config's config-less fallback matched the generic Qwen keys (model.layers.* / model.embed_tokens.weight), which Qwen2 and Qwen2-VL folders carry too. Mirror the single-file path: reject a bundled visual tower and require the Qwen3-only q_norm/k_norm weights. - isZImage/isFlux2DiffusersMainModelConfig returned true for any nonempty SDNQ submodels map, offering a transformer-only pipeline in the source pickers. Added isSelfContainedSDNQPipeline() mirroring the backend's required-submodel set and reused it in readiness.ts and buildZImageGraph.ts so both sides agree on what "complete" means. Test fixtures now write real component content, since discovery no longer accepts bare directories; the empty-dir case is requested explicitly. * fix(sdnq): address the two merge blockers from the 9228 review flux_vae_decode raised TypeError on a VAE without a shift factor. `shift_factor` is optional on diffusers' AutoencoderKL: the FLUX VAE sets one, a plain SD-style config leaves it None, and `tensor + None` raises. Guard it the way every sibling decode path already does (getattr + explicit None check) — a sweep found this was the only unguarded one. Self-contained pipeline detection trusted the advertised class name. model_index.json records what a pipeline claims each component is, and that claim selects the loader, but nothing forces it to be true. A folder can advertise Qwen3ForCausalLM for text_encoder/ and ship a Qwen2 or multimodal Qwen2-VL model: the class check passes, the file-presence check passes, the pipeline is recorded as self-contained, and the mismatch only surfaces when the loader builds a Qwen3ForCausalLM against a state dict that cannot satisfy it. Components must now also be what the index says they are, read from the component's own config (architectures / _class_name / tokenizer_class). Strict on contradiction, lenient on silence: a component declaring no class at all is still accepted, so repos omitting those keys keep working — the same trade-off as Qwen3Encoder_SDNQ_Config._validate_is_qwen3_encoder. The FLUX.1 _get_submodels is deliberately untouched; the review lists it under follow-on candidates, not blockers. Also carries two fixups from the main merge: the Klein loader's error message now names the accepted shape ("Diffusers-style FLUX.2 pipeline") since it also accepts SDNQ pipelines, and the SDNQ-added readiness test builders gained main's new required hasFlux2DevDiffusersSource field. * fix(sdnq): reject pipeline components that do not identify themselves Follow-up to the previous blocker fix, which only went half way: it rejected a component whose config *contradicted* the class model_index advertises, but still accepted one that declared nothing at all. "Some config.json plus some weight file" is exactly the shape an unrelated model has, so that left the hole open for every mismatched component whose config happens to omit the key — which is what the review means by "any component directory containing a config file and weight file". Reproduced before fixing: a text_encoder/ holding a Qwen2-VL state dict (visual.* keys) with a config declaring no architecture, under an index advertising Qwen3ForCausalLM, was recorded as a complete self-contained pipeline. Weight-bearing components (transformer / text_encoder / vae) must now name their own class. save_pretrained writes _class_name (diffusers) or architectures (transformers) for anything it saves, so one that names nothing is not a normally-produced component. The tokenizer stays lenient: it carries no weights, so nothing can be mis-instantiated against it, and tokenizer_class is written less consistently. Rejecting is not fatal to the install — the component is left unrecorded, the pipeline is simply not self-contained, and the user wires it up explicitly, which is the right outcome for a folder we cannot confirm. The fixtures wrote empty component configs and would have failed under the stricter rule; they now carry the class a real save_pretrained writes. The old leniency test is replaced by two: rejection for text_encoder/vae, and the remaining leniency for the tokenizer. * fix(sdnq): work through the follow-on findings from the 9228 review Discovery (main.py) - FLUX.1 recorded components on the index's word alone. It now runs the same two guards as the FLUX.2 / Z-Image pipelines: the folder must hold the files its loader needs, and must declare a class consistent with what the index advertises. Generalising this surfaced a latent bug — _sdnq_component_dir_is_populated only exempted Tokenizer, so FLUX.1's second tokenizer slot would have been rejected for carrying no weights. Hence _WEIGHTLESS_SUBMODEL_TYPES. - Submodels are no longer taken from a serialized config. A persisted map records what existed at install time, so a component deleted since stayed "present" and the pipeline stayed self-contained until a loader opened a folder that was gone. Loading (flux.py, vae.py) - The SDNQ loader reconstructed model_path / submodel_type.value, which assumes each index key equals its slot name. A pipeline that names its CLIP encoder something else was discovered fine and then loaded from a path that does not exist. It now follows the discovered path. - The SDNQ loaders build modules with init_empty_weights + load_state_dict rather than from_pretrained, so nothing called .eval() and anything dropout- or norm-sensitive stayed in training mode. Identification (t5_encoder.py) - The SDNQ weight/scale pair is matched across the whole directory, not within each shard. Sharding splits by tensor order, so the pair routinely straddles files; per-file matching reported such a checkpoint as unquantized whenever it carried no quantization marker. Diagnostics (sdnq/utils.py, sdnq/loaders.py, sdnq/sdnq_tensor.py) - The uint4 diagnostic ran full-tensor reductions and a unique() sort on the first dequantization of every model, before anything checked whether the output would be read. Now gated on the log level and bounded to a fixed sample. Three stdout prints removed; two were verbatim duplicates of the logger call beneath them. flux_model_loader - A complete SDNQ pipeline ships its own T5, CLIP and VAE, which is what the SDNQ docs promise, but the node required separate identifiers anyway and users had to install duplicates. The three inputs are now optional and fall back to the pipeline. FLUX.1 needs six submodels (the T5 pair on top), so this uses its own completeness check rather than the single-encoder one. Models that cannot supply the parts are still rejected, naming every missing part at once. Node version 1.1.0. * fix(sdnq): close the review-4891383742 blockers and generalise the two loader fixes Merge blockers - The frontend still demanded standalone T5 / CLIP / VAE for every FLUX.1 model, in readiness (both tabs) and as an assert in buildFLUXGraph, so the backend's self-contained path was unreachable through the UI. A complete SDNQ pipeline now satisfies readiness on its own and the graph omits the inputs, letting the node resolve them from the main model. FLUX.1 needs six submodels (the T5 pair on top), so the frontend check mirrors is_self_contained_sdnq_flux1_pipeline rather than the single-encoder one. Documented in sdnq-quantization.mdx. - The component population check accepted .bin/.pt/.pth/.ckpt/.gguf, but sdnq_sd_loader globs *.safetensors and raises when it finds none, so a correctly declared component holding an unsupported format was recorded as self-contained and failed at load time. Made loader-specific rather than narrowing everything: a *quantized* component (quantization_config .json present) must ship safetensors, an unquantized one — typically the VAE, which SDNQ exports leave in bfloat16 — still goes through from_pretrained and may ship what diffusers reads. Generalising the two follow-ons, per review and beyond it A survey of every loader that hand-builds a module found 43 of 52 returning without .eval(), across FLUX, FLUX.2, Z-Image, Krea-2, Qwen-Image, Anima, Wan, Mistral and Ideogram — not the handful named. Rather than 43 edits, put_in_eval_mode now runs in load_default._load_and_cache, the single construction choke point every loader passes through, so it also covers loaders that do not exist yet. It is idempotent for the nine that already did it, and passes tokenizers, schedulers and pipelines through untouched. The local _in_eval_mode helper added to flux.py earlier is gone; one mechanism, not two. Likewise the path fix: resolve_submodel_path replaces the per-site Path(config.path) / "<slot>" reconstruction in the FLUX.1 dispatch, the FLUX.2 transformer/encoder/tokenizer/VAE loaders and the Z-Image equivalents — the loaders whose configs carry an index-key-derived submodels map. The plain-diffusers loaders (Stable Diffusion, ONNX, CogView4, Wan, Qwen-Image, Krea-2) keep the conventional layout on purpose: they have no such map. Verified against real installs: Z-Image-Turbo-SDNQ-uint4-svd-r32 and FLUX.2-klein-9B-SDNQ-4bit-dynamic-svd-r32 identify unchanged, with all four submodels and correct variants. Both declare a class on every weight-bearing component and ship only safetensors, which is what makes the stricter discovery safe. * fix(sdnq): decide component weight formats by loader, and size what we load Blocker — the quantization marker was the wrong criterion Whether `sdnq_sd_loader` reads a component is a property of the loader, not of the folder, but the population check asked `_is_sdnq_folder`, which only looks for quantization_config.json. A component carrying SDNQ weight/scale keys without that marker read as unquantized, so a .bin or .gguf passed discovery and then failed in a loader that was always going to want safetensors. Modelled explicitly instead, because it genuinely differs per pipeline: FLUX.2 and Z-Image build their transformer *and* Qwen3 text encoder with sdnq_sd_loader unconditionally, while FLUX.1 does so only for the transformer — its CLIP, T5 and VAE branch on the folder marker and fall back to from_pretrained. Requiring safetensors across the board would have rejected a legitimate unquantized CLIP encoder shipping .bin. Finding — cache sizing had drifted from the load path Loading follows the discovered component path now, but get_size_fs still appended submodel_type.value. For an index key like `clip_encoder` that measures a `text_encoder/` that does not exist, returns 0, and make_room reserves nothing before a multi-GB component is read. It now goes through resolve_submodel_path, on a branch that only triggers when the discovered path differs from the conventional one — the normal case is byte-identical, which matters because calc_model_size_by_fs has a standalone-install fallback keyed on the subfolder name. Docs The reviewer asked for documentation of the weight-format rule. It is a sentence on the existing "no safetensors files found" troubleshooting entry rather than a section in the how-to flow: published SDNQ repos ship safetensors throughout, so a user installing one can never hit it, and the symptom already had a home. Both mutation-checked: ignoring the per-loader set fails four tests, reverting get_size_fs fails one. Z-Image-Turbo-SDNQ-uint4-svd-r32 and FLUX.2-klein-9B-SDNQ-4bit-dynamic-svd-r32 identify unchanged. * fix(ui): keep the FLUX.1 readiness test off MainModelConfig.submodels The fixture is cast to MainModelConfig, which has no `submodels` field — the generated schema doesn't carry the SDNQ variants — so reading it back through the cast failed tsc in CI. Keep the map as its own const. vitest's typecheck only covers *.test-d.ts, so a full `tsc --noEmit` is what catches this; running the suite is not enough after touching a test file. Also formats readiness.ts, which prettier would have failed on next. * fix(sdnq): one SDNQ detector, and bump the two nodes whose contract changed Detection had four near-identical implementations — configs/main.py, the FLUX and Z-Image loaders, and vae.py — and only vae.py looked past the quantization_config.json marker. That divergence is a defect, not untidiness: identification and loading inspect the same folder and must reach the same verdict. On a markerless export they didn't, so identification handed the folder to a plain-diffusers config and the loader then ran from_pretrained() over packed SDNQ weights. They now all delegate to quantization/sdnq/detection.py, which checks the marker first and falls back to a weight/scale key pair resolved across the union of every shard (sharding routinely separates a weight from its scale). t5_encoder's key-only check is the same code. The tests drive every detector over the same cases so they cannot drift apart again. Node versions: flux2_klein_model_loader 1.0.0 -> 1.1.0 and z_image_model_loader 3.0.0 -> 3.1.0. Both dropped their ui_model_format=Diffusers pin and gained a self-contained-pipeline fallback in this PR, so a node serialized against the old version still carries the old field contract. Also reads the FLUX.2 variant through flux2_variant.py instead of comparing the transformer geometry by hand. The hand-rolled comparison had no [dev] branch, so [dev]'s 15360 / 6144 fell into the else and a [dev] SDNQ pipeline was silently identified as Klein4B. This is not [dev] support — that is separate work — only a correct label instead of a wrong one; the Klein path is covered by its own test. Verified against real installs: Z-Image-Turbo-SDNQ-uint4-svd-r32 and FLUX.2-klein-9B-SDNQ-4bit-dynamic-svd-r32 identify unchanged. * fix(sdnq): reject a transformer whose weights the loader cannot read `_validate_has_sdnq_transformer` only checked that the folder looked SDNQ. Discovery, which uses a stricter predicate, then left Transformer out of `submodels` — but the config was created anyway, so the model installed cleanly and only failed when a loader opened the transformer path at generation time. Identification now requires `_sdnq_component_dir_is_populated` for the transformer: the same predicate discovery uses, so the two cannot disagree about the slot every SDNQ loader needs. Applied in all three pipeline configs. Docs: the detection section still named `quant_method: "sdnq"` as the criterion. Markerless exports are recognized by their weight/scale key pairs now, across shards, and the transformer's weights have to be readable safetensors — both documented. --------- Co-authored-by: Jonathan <34005131+JPPhoto@users.noreply.github.com> Co-authored-by: JPPhoto <jpollack@jpollackphoto.com>	1 个月前
test_sdnq_diagnostics_and_eval_mode.py	feat: Adding Support for SD.Next Quantization Engine (SDNQ) (Flux1&Flux2klein4B/9B&Z-Image) (#9228) * feat(quantization): add SDNQ (SD.Next Quantization) support Add support for loading SDNQ-quantized models with on-the-fly CPU dequantization, similar to existing GGUF support. New features: - SDNQTensor class with __torch_dispatch__ for automatic dequantization - Support for symmetric/asymmetric int8/uint8/fp8 quantization - Optional SVD correction (low-rank approximation) - Model loaders for Flux and Z-Image SDNQ models - Automatic format detection via weight+scale key pairs New files: - invokeai/backend/quantization/sdnq/ (core module) - tests/backend/quantization/sdnq/ (unit tests) Modified files: - taxonomy.py: Add ModelFormat.SDNQQuantized - configs/main.py: Add Main_SDNQ_FLUX_Config, Main_SDNQ_ZImage_Config - configs/factory.py: Register SDNQ configs - model_loaders/flux.py: Add FluxSDNQCheckpointModel - model_loaders/z_image.py: Add ZImageSDNQCheckpointModel * fix(sdnq): improve uint4 dequantization and add diffusers format support - Add uint4 per-group quantization with packed weight unpacking - Handle 1D flattened weights (reshape to 2D before unpacking) - Support SDNQ diffusers format for FLUX transformer and T5 - Add SDNQ VAE loading with AutoencoderKL - Add diagnostic logging for debugging dequantization - Fix bit order in uint4 unpacking (lower, upper) * test(sdnq): align asymmetric dequant expectation with upstream convention The test was checking `(weight - zero_point) * scale`, but SDNQ (Disty0/sdnq) defines asymmetric dequantization as `zero_point + weight * scale` (via torch.addcmul), where zero_point is a post-scale bias rather than a pre-scale integer offset. The implementation already follows this convention; only the test expectation was wrong. * test(sdnq): align asymmetric dequant expectation with upstream convention The test was checking `(weight - zero_point) * scale`, but SDNQ (Disty0/sdnq) defines asymmetric dequantization as `zero_point + weight * scale` (via torch.addcmul), where zero_point is a post-scale bias rather than a pre-scale integer offset. The implementation already follows this convention; only the test expectation was wrong. feat(sdnq): support sidecar LoRA application on SDNQ-quantized layers Bring SDNQ to feature parity with GGUF in the sidecar patching path so LoRA, LoKr, DoRA, FullLayer, and FluxControl patches apply correctly to SDNQ-quantized Linear and Conv2d modules. Without this, the sidecar aggregate replaced the SDNQTensor weight with a meta tensor and patches silently produced wrong results. - Add SDNQTensor branch in CustomModuleMixin._aggregate_patch_parameters mirroring the GGMLTensor branch. - Extend the (GGMLTensor) dtype-cast exclusion to also cover SDNQTensor in CustomLinear, CustomConv2d, CustomInvokeLinearNF4, and CustomInvokeLinear8bitLt. - Add `linear_with_sdnq_quantized_tensor` and `linear_sdnq_quantized` fixtures so the existing custom-module test matrix exercises SDNQ alongside GGUF, BnB-8bit, and NF4. * feat(sdnq): support SDNQ-quantized T5 encoders Add T5Encoder_SDNQ_Config for diffusers-style T5 bundles whose text_encoder_2/ folder holds SDNQ-quantized safetensors (detected via quantization_config.json's quant_method or via the SDNQ-style weight+scale key pairs). Add T5EncoderSDNQLoader that materializes the T5EncoderModel on meta, then loads the SDNQ state dict, and re-shares the embed_tokens/shared weight per HuggingFace's tied- weight convention. * feat(sdnq): support SDNQ-quantized FLUX.2 transformers Add Main_SDNQ_Flux2_Config covering Klein 4B/9B and their Base variants (detected via _get_flux2_variant on the dequantized SDNQTensor shapes plus the existing filename heuristic), and Flux2SDNQCheckpointModel that loads diffusers-layout SDNQ FLUX.2 checkpoints straight into Flux2Transformer2DModel. Architecture (num_layers, hidden_size, attention head count, guidance presence) is detected from state-dict shapes the same way the fp16 loader does, since SDNQTensor.shape reports the dequantized shape. BFL-layout SDNQ FLUX.2 checkpoints are not supported here — that would require an SDNQTensor-aware port of the _convert_flux2_bfl_to_diffusers fuse logic. * feat(sdnq): support full ZImagePipeline diffusers folders Add Main_SDNQ_Diffusers_ZImage_Config so a complete SDNQ ZImagePipeline folder (model_index.json + transformer/ + text_encoder/ + tokenizer/ + vae/) is recognised on install and its submodels are wired up. Extend ZImageSDNQCheckpointModel to load the transformer from the subfolder using ZImageTransformer2DModel.from_config() so non-default architecture parameters (e.g. axes_lens [1536,512,512] in newer Z-Image Turbo SDNQ exports) are honoured instead of the single-file path's hardcoded [1024,512,512]. Verified end-to-end against Tongyi-MAI/Z-Image-Turbo-SDNQ-uint4-svd-r32: 269 quantized + 252 regular tensors load into a 6.15B-param model with 0 missing / 0 unexpected keys. * fix(sdnq): match T5 SDNQ submodel layout in FluxPipeline bundles T5Encoder_SDNQ_Config originally only looked for text_encoder_2/ as a subfolder of mod.path, which works for standalone T5 bundles but misses the case where a parent FluxPipeline / similar config registers its T5 submodel with path_or_prefix pointing straight at the text_encoder_2 folder. Allow both layouts in both the config's detection logic and T5EncoderSDNQLoader's te_dir resolution. Verified end-to-end with Disty0/FLUX.1-schnell-SDNQ-uint4-svd-r32. * fix(sdnq): swap scale/shift halves in FLUX BFL converter's norm_out The diffusers→BFL state-dict converter renamed norm_out.linear.{weight,bias} to final_layer.adaLN_modulation.1.{weight,bias} but did not swap the two halves along dim 0. diffusers' AdaLayerNormContinuous packs the linear output as (scale, shift); BFL's LastLayer packs as (shift, scale). Without the swap, the final adaLN modulation runs with scale and shift permuted, which produces structured-but-very-noisy output for every pixel. Reuse the same pattern the FLUX.2 converter applies for the analogous adaLN_modulation key. * feat(sdnq): dispatch all ZImagePipeline submodels via SDNQ loader ZImageSDNQCheckpointModel only handled the Transformer submodel, so attempts to use an SDNQ ZImagePipeline as the "Qwen3 & VAE source model" (which triggers loads for TextEncoder / Tokenizer / VAE) crashed with "Only Transformer submodels are currently supported". Add per-submodel handlers that load text_encoder/ via sdnq_sd_loader into an empty Qwen3ForCausalLM (re-sharing lm_head with embed_tokens when tied), tokenizer/ via AutoTokenizer, and vae/ via AutoencoderKL.from_pretrained. The single-file SDNQ checkpoint path keeps its transformer-only behaviour but now raises a clearer error when asked for a different submodel. * feat(sdnq): support FLUX.2 Klein dynamic mixed-precision pipelines Add support for SDNQ-quantized Flux2KleinPipeline folders, which mix uint4 and int5 dtypes across layers (chosen dynamically by SDNQ during quantization to stay under a per-group loss budget). Core changes: - Add INT5_ASYM quantization type + unpack_uint5 + dequantize_int5_per_group. Sign-extension matches Disty0/sdnq's unpack_int convention (raw 0..31 - 16). zero_point is optional (dynamic-mixed sometimes emits scale-only int5 tensors). - _infer_quantization_type now takes a per_tensor_dtype override; the loader builds an inverted map from quantization_config.json's modules_dtype_dict. - _get_original_shape uses the packed weight size as the authoritative source for in_features, fixing a bug where Klein 4B's group_size=64 layers were misread as group_size=128 (the previous fallback). Pipeline integration: - Add Main_SDNQ_Diffusers_Flux2_Config matching Flux2Pipeline / Flux2KleinPipeline folders with quantized transformer. - Flux2SDNQCheckpointModel now dispatches all pipeline submodels: transformer (Flux2Transformer2DModel.from_config + sdnq state dict), text_encoder (Qwen3ForCausalLM SDNQ + lm_head/embed_tokens tie), tokenizer (AutoTokenizer), vae (AutoencoderKLFlux2 / AutoencoderKL). - Extend flux2_klein_model_loader._validate_diffusers_format and the isFlux2DiffusersMainModelConfig FE filter to also accept SDNQ pipeline configs (when submodels is populated). Verified against Disty0/FLUX.2-klein-4B-SDNQ-4bit-dynamic: 98 uint4 + 2 int5 tensors load into a 3.88B-param Flux2Transformer2DModel with 0 missing / 0 unexpected keys; both dequant paths produce reasonable zero-centred weight distributions. * - Reject SDNQ-quantized folders in Main_Diffusers_FLUX_Config and Main_Diffusers_Flux2_Config so identification routes them to the SDNQ configs instead. Without this both configs accept the folder and the plain diffusers loader wins, then crashes when reading packed uint8 weights as bf16. * - Merge multi-shard safetensors in sdnq_sd_loader so Klein 9B's diffusion_pytorch_model-{00001,00002}-of-00002.safetensors and FLUX.2 dev's sharded transformer both load. Detect cross-shard key collisions as a corruption signal. * - Treat SDNQ ZImagePipeline / Flux2KleinPipeline folders as "main_is_diffusers" in z_image_model_loader and flux2_klein_model_loader so the auto-extract-submodels branch handles them. Without this the loader demanded a separate VAE/Qwen3 source even though the SDNQ pipeline carries those submodels itself. - Drop the ui_model_format=Diffusers hint on Klein's qwen3_source_model field so the FE combobox can also show SDNQ pipeline configs (the FE filter already accepts them). * fix(sdnq): unblock FLUX.2 Klein SDNQ pipelines in the UI Loading the Klein 4B SDNQ pipeline as the main model errored with "No Qwen3 Encoder selected" in the UI even though the pipeline carries its own Qwen3 + VAE submodels, and the Model Manager showed no format badge at all on SDNQ models. - flux2_klein_model_loader now treats SDNQ-with-submodels as main_is_diffusers, so the auto-extract-submodels branch handles SDNQ pipelines exactly like plain diffusers. Drop the ui_model_format=Diffusers hint on qwen3_source_model so the combobox can also show SDNQ pipeline configs. - readiness.ts no longer demands a standalone VAE/Qwen3 for FLUX.2 Klein when the main model is itself a pipeline (diffusers or SDNQ-with-submodels). Without this the Invoke button stayed disabled with "Non-diffusers FLUX.2 Klein models require a standalone Qwen3 Encoder" even when the SDNQ pipeline could self-source everything. - Register sdnq_quantized in zModelFormat, the manually-edited OpenAPI schema, ModelFormatBadge, and MODEL_FORMAT_TO_LONG_NAME so SDNQ models render an "sdnq" badge instead of an empty placeholder. * feat(sdnq): add starter models and user-facing docs - 4 new starter models covering all SDNQ pipelines verified end-to-end in this branch: FLUX.1 schnell, Z-Image Turbo, FLUX.2 Klein 4B (dynamic mixed), FLUX.2 Klein 9B (dynamic mixed + SVD). Each entry is self-contained (no separate encoder/VAE dependencies because the SDNQ pipeline folder bundles them). - New /configuration/sdnq-quantization/ page: support matrix, VRAM footprints, install steps (Starter Models + HF + Folder), LoRA compatibility notes, SDNQ-vs-SVDQuant/Nunchaku disambiguation, comparison with GGUF/NF4/FP8, troubleshooting. - Cross-link from fp8-storage.mdx's "no-op on quantized" caution. * Chore Fix Path * fix(sdnq): add missing variant/cpu_only fields to SDNQ configs Z-Image and Qwen3 SDNQ configs were missing `variant` (and `cpu_only` on Qwen3) fields that exist on the other variants of the same union, breaking TypeScript narrowing on the FE. - Main_SDNQ_ZImage_Config: add variant (default Turbo) - Main_SDNQ_Diffusers_ZImage_Config: add variant, detect from scheduler_config.json shift value - Qwen3Encoder_SDNQ_Config: add cpu_only + variant, detect from embed_tokens shape - Qwen3Encoder_SDNQ_Folder_Config: add cpu_only + variant, detect from config.json hidden_size - Regenerate FE schema.ts Discriminator tags are unchanged since variant has no default. * Fix openapi schema.ts * Fix Path * feat(pid): vendor PiD decoder backend (phase A of integration) Adds a vendored subset of NVIDIA's PiD (Pixel Diffusion Decoder) at invokeai/backend/pid/ as the foundation for upcoming FLUX / FLUX.2 / SD3 / Z-Image PiD decode nodes plus a future PiD-based 4x upscale node. Upstream: https://github.com/nv-tlabs/PiD (Apache 2.0). Vendor scope: * _src/{networks,models,modules}: PidNet, PixDiT_T2I, LQProjection2D, PidModel, PidDistillModel, PixelDiTModel, GeneralConditioner. * _ext/imaginaire: minimal Imaginaire framework subset (lazy_config, model, utils/{log,misc,distributed,device,count_params}). * configs/, tokenizers/, checkpointer/, trainer.py, visualize/, _demo_*, from_*, easy_io/, S3/wandb training helpers were intentionally excluded. Dependency stripping (no new hard deps introduced): * loguru, termcolor -> stdlib logging shim * iopath PathManager -> stdlib pathlib stub * fvcore Registry -> minimal stdlib Registry * lazy_config/lazy.py: yaml/dill/cloudpickle/detectron2 save/load paths replaced with a minimal LazyCall stub * lazy_config/instantiate.py: omegaconf DictConfig/ListConfig branches removed; configs are plain dict / LazyCall mappings * megatron, pynvml, boto3/wandb imports are try/except-guarded or local to functions and stay inert in our inference path All pid.* imports rewritten to invokeai.backend.pid.*; SPDX-Apache-2.0 headers retained on vendored files; attribution and detailed list of local modifications added in LICENSE-PiD.txt. The pre-trained PiD checkpoints distributed by NVIDIA remain under NSCLv1 (non-commercial); this commit only vendors code. Smoke test: PidNet, PidModel, PidDistillModel, GeneralConditioner import cleanly; LazyCall -> instantiate round-trip resolves to the expected nn.Module. ruff check passes. * feat(pid): wire PiD + Gemma-2 into model-manager and add decode nodes Adds the model-manager plumbing and workflow nodes needed to use the vendored PiD decoder (phase A) end-to-end with FLUX, SD3 and Z-Image. Model manager (Phase B + B.5): * taxonomy: ModelType.PiDDecoder, PiDDecoderVariantType (Res2k_Sr4x / Res2kTo4k_Sr4x), ModelType.Gemma2Encoder + ModelFormat.Gemma2Encoder, both added to AnyVariant + variant_type_adapter. * configs/pid_decoder.py: per-backbone PiD configs (FLUX / FLUX.2 / SD3) with state-dict probing on 'lq_proj' substring and backbone/variant detection from the official NVIDIA filenames. * configs/gemma2_encoder.py: Gemma-2 directory probing on Gemma2ForCausalLM architecture + tokenizer files. * AnyModelConfig union updated. * model_loaders/pid_decoder.py: loads .pth / .safetensors, strips the upstream 'net.' prefix, supports torch.load(weights_only=True). * model_loaders/gemma2_encoder.py: SubModelType.{Tokenizer, TextEncoder} dispatch; returns the causal LM's inner Gemma2Model (transformers 4.56's get_decoder() returns None for Gemma2). Decode pipeline (Phase C): * backend/pid/decode.py: build_pid_net + load_pid_decoder (per-backbone PixDiT_T2I hyperparams derived from PiD's pid_sr4x base + per-experiment overrides), encode_caption_for_pid (chi-prompt + Gemma encoding, mirrors PixelDiTModel._encode_text_raw), and a PiDDecoder wrapper with a reimplemented few-step distill sampler (no autocast / no distributed / no PixelDiTModel init paths from upstream). Invocations (Phase 6.x): * Gemma2EncoderField + PiDDecoderField in invocations/model.py. * gemma2_encoder_loader / pid_decoder_loader: thin ModelIdentifierField pickers that emit the corresponding fields. * z_image_pid_decode (pilot), flux_pid_decode, sd3_pid_decode: caption encode -> Gemma offload -> PiD state dict load -> PidNet construct -> decode. Per-backbone latent denormalisation (FLUX1 ae_params, SD3 hardcoded 1.5305/0.0609, Z-Image piggybacks on FLUX VAE). End-to-end validated with the released PiD_res2k_sr4x_official_flux_distill_4step.pth checkpoint and gemma-2-2b-it: PidNet rebuilds at exactly 456 keys / 1.36B params, sampler runs at ~5 GB VRAM peak (Gemma dominates), output shape and range match. FLUX.2 PiD decode is deliberately deferred: it needs BN-based latent denormalisation and 32->128 channel packing, and we have no FLUX.2 checkpoint to validate against yet. * feat(pid): end-to-end PiD pixel-diffusion decoder integration Adds the NVIDIA PiD decoder as a 4x super-resolution alternative to the regular VAE/RAE decode path. Includes model-manager configs and loaders for both the PiD checkpoints and the Gemma-2 caption encoder they require, plus four invocations: latent-in decode for FLUX / SD3 / Z-Image and an image-in pid_upscale node. - Decode pipeline keeps PidNet params in fp32 and uses bf16 autocast only for matmuls; caption embeddings have outliers that overflow bf16 RMSNorm. - encode_caption_for_pid forces tokenizer padding_side="right" (Gemma defaults to left, PiD trained with right) and returns the attention mask as bool so it stays compatible with SDPA. - Z-Image reuses the FLUX-trained checkpoint and reads scale/shift from the VAE config at runtime (PiD upstream notes they are checkpoint-specific). - TextLLM config now excludes Gemma2ForCausalLM so it falls through to the dedicated Gemma2 encoder config instead of being misclassified. - Frontend: new model_type / model_format / variant enums, type guards and category metadata; schema.ts regenerated via pnpm typegen. * Fix closing Step Tag * Chore Ruff * Chore Typegen * Chore Knip * fix(pid): remove unused vendored models/utils.py (broken easy_io import) * feat(pid): identify decoder backbone from weight shapes, not filename Read latent channel count from lq_proj.latent_proj.0.weight (FLUX.2=128, FLUX.1/SD3=16) as the primary discriminator; fall back to filename/dir name only to disambiguate the architecturally identical FLUX.1/SD3 pair. Fixes FLUX.2 checkpoints (model_ema_bf16.pth) not being recognised, and correctly rejects unsupported backbones (RAE/dinov2, 768ch). Fix Flux2 docstring 32->128. * feat(ui): PiD decode (Fit mode) for FLUX text-to-image Add a "PiD Decode" mode select (Off / Fit / Native) to the FLUX advanced settings with PiD decoder + Gemma-2 encoder pickers. In Fit mode the FLUX graph swaps the VAE decode for a PiD 4x super-resolution decode and downscales back to the requested size. Adds params state (pidMode, decoder, encoder, steps) with a v3->v4 migration, model hooks, readiness checks, and graph guards for the not-yet-wired Native and non-txt2img paths. * feat(ui): PiD Native 4x mode for FLUX text-to-image Make the generation dimension helpers PiD-aware via an optional pidScale: in Native mode the user-facing dimensions are the 4x target (grid 64, optimal 2048), generation runs at target/4, and PiD's 4x output is used directly with no downscale. Thread pidScale through the params dimension reducers and the optimal-dimension/grid-size selectors, resync dimensions when toggling Native, and wire the Native path in the FLUX graph builder. Add working_mem_bytes for PiD Decode * feat(ui): PiD Fit decode for FLUX image-to-image Extract the PiD decode chain into buildPidDecodeChain (loaders + decode + fit-downscale, no denoise setup) so it can substitute for the VAE decode across generation modes. Widen addImageToImage's l2i param to ImageOutputNodes (it only consumes .image) and wire the PiD chain into the img2img branch in Fit mode. Native stays txt2img-only (a 4x result can't composite onto the bbox); inpaint/outpaint remain gated off for now. * feat(ui): PiD Native 4x decode for FLUX image-to-image Add addPidImageToImageNative: the canvas bbox is the 4x target, so the init image is downscaled to bbox/4, denoised at that resolution, and PiD decodes straight back up to the full bbox with no post-decode downscale - preserving all PiD detail while still compositing cleanly onto the region. Wire it into the img2img branch of buildFLUXGraph (native vs fit vs off) and drop the native-txt2img-only guard. Make the canvas FLUX grid check PiD-aware so a native bbox must be a multiple of 64 (16 * 4) for bbox/4 to land on the grid. * feat(ui): add informational popover to PiD Decode setting Explain PiD usage on hover, mirroring the DyPE popover: what the decoder is (NVIDIA Pixel Diffusion Decoder, 4x SR, needs a PiD decoder + Gemma-2 encoder), Fit vs Native modes, the 2K / 2K-to-4K target resolutions, that Steps can be lowered, and that Scale Before Processing must be off. Links to nv-tlabs/PiD. * feat(models): add PiD decoder + Gemma-2 encoder to starter models Register NVIDIA's PiD FLUX decoders (2K and 2K-to-4K presets, from nvidia/PiD) and the Efficient-Large-Model/gemma-2-2b-it caption encoder as starter models so they can be installed from the Model Manager. The Gemma-2 encoder is wired as a dependency of each decoder (and offered standalone). * feat(pid): add FLUX.2 Klein PiD 4x-SR decode support Add a flux2_pid_decode node that packs the stored FLUX.2 latent (32ch @ H/8) into PiD's 128ch @ H/16 layout before decoding; FLUX.2's BatchNorm denormalization is already applied in flux2_denoise, so no scalar denorm is needed (optional vae input reads identity constants). Generalize the frontend PiD decode chain (decodeNodeType, optional vaeSource) and wire the isFlux2 graph path for txt2img/img2img (Fit & Native). Base-aware PiD gating/decoder-filter, FLUX.2 readiness checks, and two nvidia/PiD FLUX.2 starter decoders (2K, 2Kto4K). Standard FLUX PiD path unchanged. * feat(pid): add SD3 PiD 4x-SR decode support Wire the existing sd3_pid_decode node into the SD3 graph builder (txt2img and img2img, Fit & Native) with a PiD guard, base-aware gating/decoder-filter (sd-3), and SD3 readiness checks. Add two nvidia/PiD SD3 starter decoders (2K, 2Kto4K). Harden the PiD config probe against the 16-channel FLUX.1/SD3 ambiguity: when the checkpoint's directory name is silent (the HF single-file download renames it), trust an explicit base override so SD3 checkpoints are not misidentified as FLUX.1. Also benefits Qwen. FLUX / FLUX.2 identification is unchanged. * feat(pid): add SDXL PiD 4x-SR decode support Build the full SDXL PiD backend stack: _PER_BACKBONE[SDXL] (4ch/down8), PiDDecoder_Checkpoint_SDXL_Config with a 4-channel latent-map entry, factory union + loader registration, and a new sdxl_pid_decode node (reads the VAE's scaling_factor/shift at runtime; SDXL fallbacks 0.13025/0.0). 4-channel latents are unambiguous, so no directory-name disambiguation is needed. Generalize the shared PiD decode chain to support SD-family denoise: denoise_latents has no width/height, so thread an optional noise node for sizing and round to the model's native grid (8 for SDXL, 16 for FLUX). Wire buildSDXLGraph (txt2img + img2img, Fit & Native) with the VAE as the decode's scaling source, base-aware gating/readiness, and a starter decoder (SDXL 2Kto4K only). PiD + SDXL refiner is blocked for now via a graph guard and a readiness reason. FLUX/FLUX.2/SD3 paths are unchanged. * feat(pid): add Z-Image PiD 4x-SR decode support Wire the existing z_image_pid_decode node into the Z-Image graph builder (txt2img and img2img, Fit & Native) with a PiD guard and readiness checks. Z-Image shares FLUX.1's 16-channel VAE and has no PiD checkpoints of its own, so it reuses the FLUX decoder: the decoder filter maps z-image -> flux, showing FLUX PiD decoders when a Z-Image model is active. The Z-Image VAE is passed to the decode node so it reads the real scaling_factor / shift instead of the fallback constants. No backend, schema, or starter-model changes. FLUX/FLUX.2/SD3/SDXL paths are unchanged. * feat(pid): add Qwen-Image PiD 4x-SR decode support Build the full Qwen-Image PiD backend stack: _PER_BACKBONE[QwenImage] (16ch/down8), PiDDecoder_Checkpoint_QwenImage_Config (added to the 16-channel latent map + filename heuristic), factory union + loader registration, and a new qwen_image_pid_decode node. Unlike the scalar-scaling bases, the Qwen-Image VAE normalizes per channel (latents_mean / latents_std) and stores a 5D video-style latent, so the node denormalizes per-channel (z * std + mean, read from the VAE config) and drops the singleton temporal frame before decoding - matching qwen_image_l2i. Wire buildQwenImageGraph (txt2img + img2img, Fit & Native) with the Qwen-Image VAE as the decode's normalization source, base-aware gating/readiness, and a starter decoder (Qwen-Image 2Kto4K only). The 16-channel FLUX/SD3/Qwen ambiguity is handled by the existing trusted- base-override probe hardening. FLUX/FLUX.2/SD3/SDXL/Z-Image paths are unchanged. * Chore Ruff * Add Docs * fix(pid): green up frontend tests and knip for the PiD branch - graph-builder tests: set pidMode 'off' in the FLUX / Qwen-Image / SDXL+SD3 param fixtures so the PiD guard doesn't fire on an undefined pidMode and call the (unmocked) size helpers - paramsSlice migration test: expect _version 4 (v3→v4 adds the PiD fields) - remove the unused setPidSteps action and selectPidSteps selector flagged by knip; the pidSteps state field stays at its default of 4 * Chore openapi + typegen * Merge remote-tracking branch 'origin/main' into feature/svd-quantization Resolve conflicts in qwen3_encoder.py (keep both SDNQ tensor/key detection and the Qwen-VL visual-tower guard) and regenerate openapi.json + schema.ts from the merged backend. Fix T5TokenizerFast under transformers 5.x: it is now an alias of T5Tokenizer. Use T5Tokenizer in flux.py (was used unimported -> F821) and also match the new "T5Tokenizer" class name in the SDNQ FLUX model_index.json submodel probe. * Fix SDNQ mixed-group dequant, Z-Image self-contained pipelines, and FLUX.2 single-file identification Address three SDNQ review findings (PR #9228): - loaders.py: derive per-tensor group_size from the scale tensor's group dimension instead of trusting quantization_config.json. Dynamic-mixed models (e.g. FLUX.2 Klein 4B) quantize some layers with a group size that differs from the nominal config value; trusting the config produced a num_groups that disagreed with scale.shape[1] and broke the reshape/ broadcast in dequantize_uint4/int5_per_group. - z_image_model_loader: fall back to the main model for the VAE/Qwen3 submodels when it is a self-contained SDNQ pipeline (format=sdnq_quantized with submodels). Mirror the same self-contained handling in the frontend readiness checks and buildZImageGraph, so a freshly installed SDNQ Z-Image pipeline generates without manually selecting a component source. - FLUX.2 single-file identification: list SDNQ FLUX.2 before FLUX.1 in the config union, reject FLUX.2 state dicts in Main_SDNQ_FLUX_Config so the two stay mutually exclusive regardless of iteration order, and accept the bare diffusers transformer layout (transformer_blocks./context_embedder.) in Main_SDNQ_Flux2_Config so the real single-file SDNQ FLUX.2 checkpoint is identified instead of falling through to FLUX.1 or unknown. Add regression tests for all three: mixed-group uint4 dequant, the Z-Image loader/readiness self-contained fallback, and SDNQ FLUX.2 identification. * Chore typegen + openapi * Fix SDNQ mixed-group dequant, Z-Image self-contained pipelines, and FLUX.2/Z-Image identification Address five SDNQ review findings (PR #9228): - loaders.py: derive per-tensor group_size from the scale tensor's group dimension instead of trusting quantization_config.json. Dynamic-mixed models (e.g. FLUX.2 Klein 4B) quantize some layers with a group size that differs from the nominal config value; trusting the config produced a num_groups that disagreed with scale.shape[1] and broke the reshape/ broadcast in dequantize_uint4/int5_per_group. - z_image_model_loader: fall back to the main model for the VAE/Qwen3 submodels when it is a self-contained SDNQ pipeline (format=sdnq_quantized with submodels). Mirror the same self-contained handling in the frontend readiness checks and buildZImageGraph, so a freshly installed SDNQ Z-Image pipeline generates without manually selecting a component source. - FLUX.2 single-file identification: list SDNQ FLUX.2 before FLUX.1 in the config union, reject FLUX.2 state dicts in Main_SDNQ_FLUX_Config so the two stay mutually exclusive regardless of iteration order, and accept the bare diffusers transformer layout (transformer_blocks./context_embedder.) in Main_SDNQ_Flux2_Config so the real single-file SDNQ FLUX.2 checkpoint is identified instead of falling through to FLUX.1 or unknown. - Main_Diffusers_ZImage_Config: reject SDNQ-quantized ZImagePipeline folders so Main_SDNQ_Diffusers_ZImage_Config matches them instead (mirrors the FLUX and FLUX.2 guards). Without this a full SDNQ Z-Image pipeline could be classified as plain diffusers, mis-reading packed weights and breaking the self-contained path. - z_image loader: check missing/unexpected keys after load_state_dict in the SDNQ transformer folder path and fail fast with the offending key list, instead of silently returning a model with required params on meta tensors that fails later during device movement or inference. Add regression tests for all five: mixed-group uint4 dequant, the Z-Image loader/readiness self-contained fallback, SDNQ FLUX.2 identification, and SDNQ ZImagePipeline-folder identification. * Harden SDNQ model identification, loading, and Z-Image pipeline support Address the SDNQ review findings on the feature branch: Dequantization / loading correctness - sdnq/loaders.py: derive per-tensor group_size from the scale tensor's group dimension instead of trusting quantization_config.json. Dynamic-mixed models (e.g. FLUX.2 Klein 4B) quantize some layers with a group size that differs from the nominal config value; trusting the config produced a num_groups that disagreed with scale.shape[1] and broke the reshape/broadcast in dequantize_uint4/int5_per_group. - Add shared raise_on_incomplete_sdnq_load() and apply it to every SDNQ folder loader that used load_state_dict(strict=False) without checking the result: FLUX.2 pipeline transformer, SDNQ CLIP/T5/VAE (flux.py), standalone SDNQ VAE (vae.py), and both Z-Image Qwen3 text-encoder paths (z_image.py). A partial or mismatched export now fails fast with the offending key list instead of returning a model with parameters left on the meta device. Tied/re-shared weights (T5 encoder.embed_tokens, Qwen3 lm_head, CLIP position_ids) are allow-listed. Model identification - Reject FLUX.2 in Main_SDNQ_FLUX_Config and accept the bare diffusers transformer layout (transformer_blocks./context_embedder.) in Main_SDNQ_Flux2_Config, and list FLUX.2 before FLUX.1 in the config union, so a single-file SDNQ FLUX.2 checkpoint identifies as FLUX.2 instead of falling through to FLUX.1 or unknown. - Main_Diffusers_ZImage_Config: reject SDNQ-quantized ZImagePipeline folders so Main_SDNQ_Diffusers_ZImage_Config matches them instead (mirrors the FLUX and FLUX.2 guards). - Qwen3Encoder_SDNQ_Folder_Config: verify the folder is actually a Qwen3 encoder (Qwen3 config class name or Qwen3 state-dict keys) before claiming it, so an SDNQ transformer/VAE folder is not stored as type=qwen3_encoder. - Qwen3Encoder_Qwen3Encoder_Config: reject SDNQ folders so it no longer competes with the SDNQ folder config (both share the Qwen3Encoder type, making the factory tiebreak non-deterministic). Z-Image self-contained SDNQ pipelines - z_image_model_loader: fall back to the main model for the VAE/Qwen3 submodels when it is a self-contained SDNQ pipeline (format=sdnq_quantized with submodels), and drop the ui_model_format=diffusers hint from qwen3_source_model so SDNQ pipelines are not filtered out of the node/workflow model pickers. - Mirror the self-contained handling in the frontend readiness checks and buildZImageGraph so a freshly installed SDNQ Z-Image pipeline generates without a manually selected component source. Add regression tests for every fix: mixed-group uint4 dequant, the load-guard helper, SDNQ FLUX.2 identification, SDNQ ZImagePipeline-folder identification, Qwen3 encoder folder identification, the Z-Image self-contained loader/readiness path, and the qwen3_source_model field template. * Chore Typegen + OpenApi * Harden SDNQ model identification, loading, and Z-Image pipeline support Address the SDNQ review findings on the feature branch: Dequantization / loading correctness - sdnq/loaders.py: derive per-tensor group_size from the scale tensor's group dimension instead of trusting quantization_config.json. Dynamic-mixed models (e.g. FLUX.2 Klein 4B) quantize some layers with a group size that differs from the nominal config value; trusting the config produced a num_groups that disagreed with scale.shape[1] and broke the reshape/broadcast in dequantize_uint4/int5_per_group. - Add shared raise_on_incomplete_sdnq_load() and use it in every SDNQ folder loader that called load_state_dict(strict=False) without checking the result: FLUX.2 pipeline transformer, SDNQ CLIP/T5/VAE (flux.py), standalone SDNQ VAE (vae.py), and both Z-Image Qwen3 text-encoder paths (z_image.py). A partial or mismatched export now fails fast with the offending key list instead of returning a model with parameters left on the meta device. Tied/re-shared weights (T5 encoder.embed_tokens, Qwen3 lm_head, CLIP position_ids) are allow-listed. Replace the T5 SDNQ assert checks (stripped by python -O) and add the missing unexpected-key check to the standalone Qwen3 SDNQ loader. Model identification - Reject FLUX.2 in Main_SDNQ_FLUX_Config and accept the bare diffusers transformer layout (transformer_blocks./context_embedder.) in Main_SDNQ_Flux2_Config, and list FLUX.2 before FLUX.1 in the config union, so a single-file SDNQ FLUX.2 checkpoint identifies as FLUX.2 instead of falling through to FLUX.1 or unknown. - Main_Diffusers_ZImage_Config: reject SDNQ-quantized ZImagePipeline folders so Main_SDNQ_Diffusers_ZImage_Config matches them instead (mirrors the FLUX/FLUX.2 guards). - Qwen3Encoder_SDNQ_Folder_Config: verify the folder is actually a Qwen3 encoder (Qwen3 config class or Qwen3 state-dict keys) and reject complete causal LMs (root config.json + tokenizer files) so an SDNQ transformer/VAE/TextLLM folder is not stored as type=qwen3_encoder. - Qwen3Encoder_Qwen3Encoder_Config: reject SDNQ folders so it no longer competes with the SDNQ folder config (both share the Qwen3Encoder type, making the factory tiebreak non-deterministic). Z-Image self-contained SDNQ pipelines - z_image_model_loader: fall back to the main model for the VAE/Qwen3 submodels when it is a self-contained SDNQ pipeline, but only when the pipeline actually exposes the required vae + text_encoder + tokenizer submodels (a truthy submodels dict is not enough — a partial pipeline would fail later on missing folders). Drop the ui_model_format=diffusers hint from qwen3_source_model so SDNQ pipelines are not filtered out of the node/workflow model pickers. - Mirror both the self-contained handling and the specific-submodels requirement in the frontend readiness checks and buildZImageGraph, so a freshly installed SDNQ Z-Image pipeline generates without a manually selected component source. Add regression tests for every fix: mixed-group uint4 dequant, the load-guard helper, SDNQ FLUX.2 identification, SDNQ ZImagePipeline-folder identification, Qwen3 encoder folder identification (SDNQ rejection, causal-LM rejection), the Z-Image self-contained loader/readiness path incl. partial-pipeline handling, and the qwen3_source_model field template. * Chore Ruff * Harden SDNQ model identification, loading, and Z-Image / FLUX.2 pipeline support Address the SDNQ review findings on the feature branch: Dequantization / loading correctness - sdnq/loaders.py: derive per-tensor group_size from the scale tensor's group dimension instead of trusting quantization_config.json. Dynamic-mixed models (e.g. FLUX.2 Klein 4B) quantize some layers with a group size that differs from the nominal config value; trusting the config produced a num_groups that disagreed with scale.shape[1] and broke the reshape/broadcast in dequantize_uint4/int5_per_group. - Add shared raise_on_incomplete_sdnq_load() and use it in every SDNQ folder loader that called load_state_dict(strict=False) without checking the result: FLUX.2 pipeline transformer, SDNQ CLIP/T5/VAE (flux.py), standalone SDNQ VAE (vae.py), and both Z-Image Qwen3 text-encoder paths (z_image.py). A partial or mismatched export now fails fast with the offending key list instead of returning a model with parameters left on the meta device. Tied/re-shared weights (T5 encoder.embed_tokens, Qwen3 lm_head, CLIP position_ids) are allow-listed. Replace the T5 SDNQ assert checks (stripped by python -O) and add the missing unexpected-key check to the standalone Qwen3 SDNQ loader. Model identification - Reject FLUX.2 in Main_SDNQ_FLUX_Config and accept the bare diffusers transformer layout (transformer_blocks./context_embedder.) in Main_SDNQ_Flux2_Config, and list FLUX.2 before FLUX.1 in the config union, so a single-file SDNQ FLUX.2 checkpoint identifies as FLUX.2 instead of falling through to FLUX.1 or unknown. - Main_Diffusers_ZImage_Config: reject SDNQ-quantized ZImagePipeline folders so Main_SDNQ_Diffusers_ZImage_Config matches them instead (mirrors the FLUX/FLUX.2 guards). - Qwen3Encoder_SDNQ_Folder_Config: verify the folder is actually a Qwen3 encoder, reject complete causal LMs (root config.json + tokenizer files), accept the same compatible Qwen architectures as the unquantized config (Qwen2VLForConditionalGeneration / Qwen2ForCausalLM / Qwen3ForCausalLM via a shared constant), and make the state-dict fallback resilient to sharded folders, so an SDNQ transformer/VAE/TextLLM folder is not stored as type=qwen3_encoder and a compatible sharded encoder is not wrongly rejected. - Qwen3Encoder_Qwen3Encoder_Config: reject SDNQ folders so it no longer competes with the SDNQ folder config (both share the Qwen3Encoder type, making the factory tiebreak non-deterministic). Z-Image / FLUX.2 self-contained SDNQ pipelines - Add a shared is_self_contained_sdnq_pipeline() helper requiring the specific VAE + Qwen3 (text_encoder + tokenizer) submodels a pipeline install must ship. A truthy submodels dict is not enough: Main_SDNQ_Diffusers_* configs record whichever submodels they recognize, so a partial pipeline can expose only the transformer and would fail at runtime on missing fixed subfolders. - z_image_model_loader and flux2_klein_model_loader: use the helper for both the main-model self-contained fallback and the explicit qwen3_source validator, so a partial pipeline (as main model or selected source) requires an explicit VAE/Qwen3 source instead of emitting requests against missing folders. - Drop the ui_model_format=diffusers hint from z_image qwen3_source_model so SDNQ pipelines are not filtered out of the node/workflow model pickers. - Mirror the same specific-submodels requirement in the frontend readiness checks (Z-Image + FLUX.2, generate and canvas tabs) and buildZImageGraph, and bring the FLUX.2 canvas readiness block to parity with the generate tab (it previously used format !== 'diffusers' and never recognized SDNQ pipelines). Add regression tests for every fix: mixed-group uint4 dequant, the load-guard helper, SDNQ FLUX.2 identification, SDNQ ZImagePipeline-folder identification, Qwen3 encoder folder identification (SDNQ rejection, causal-LM rejection, compatible Qwen2 class names, sharded folders), the Z-Image and FLUX.2 self-contained loader/readiness paths incl. partial-pipeline handling, and the qwen3_source_model field template. * Harden SDNQ model identification, loading, and Z-Image / FLUX.2 pipeline support Address the SDNQ review findings on the feature branch: Dequantization / loading correctness - sdnq/loaders.py: derive per-tensor group_size from the scale tensor's group dimension instead of trusting quantization_config.json. Dynamic-mixed models (e.g. FLUX.2 Klein 4B) quantize some layers with a group size that differs from the nominal config value; trusting the config produced a num_groups that disagreed with scale.shape[1] and broke the reshape/broadcast in dequantize_uint4/int5_per_group. - Add shared raise_on_incomplete_sdnq_load() and use it in every SDNQ folder loader that called load_state_dict(strict=False) without checking the result: FLUX.2 pipeline transformer, SDNQ CLIP/T5/VAE (flux.py), standalone SDNQ VAE (vae.py), and both Z-Image Qwen3 text-encoder paths (z_image.py). A partial or mismatched export now fails fast with the offending key list instead of returning a model with parameters left on the meta device. Tied/re-shared weights (T5 encoder.embed_tokens, Qwen3 lm_head, CLIP position_ids) are allow-listed. Replace the T5 SDNQ assert checks (stripped by python -O) and add the missing unexpected-key check to the standalone Qwen3 SDNQ loader. Model identification - Reject FLUX.2 in Main_SDNQ_FLUX_Config and accept the bare diffusers transformer layout (transformer_blocks./context_embedder.) in Main_SDNQ_Flux2_Config, and list FLUX.2 before FLUX.1 in the config union, so a single-file SDNQ FLUX.2 checkpoint identifies as FLUX.2 instead of falling through to FLUX.1 or unknown. - Main_Diffusers_ZImage_Config: reject SDNQ-quantized ZImagePipeline folders so Main_SDNQ_Diffusers_ZImage_Config matches them instead (mirrors the FLUX/FLUX.2 guards). - Qwen3Encoder_SDNQ_Folder_Config: verify the folder is actually a Qwen3 encoder, reject complete causal LMs (root config.json + tokenizer files), accept the same compatible Qwen architectures as the unquantized config (Qwen2VLForConditionalGeneration / Qwen2ForCausalLM / Qwen3ForCausalLM via a shared _QWEN3_ENCODER_ARCHITECTURES constant), and make the state-dict fallback resilient to sharded folders — so an SDNQ transformer/VAE/TextLLM folder is not stored as type=qwen3_encoder and a compatible sharded encoder is not wrongly rejected. - Qwen3Encoder_Qwen3Encoder_Config: reject SDNQ folders so it no longer competes with the SDNQ folder config (both share the Qwen3Encoder type, making the factory tiebreak non-deterministic). SDNQ pipeline submodel discovery - Main_SDNQ_Diffusers_Flux2_Config / Main_SDNQ_Diffusers_ZImage_Config _get_submodels(): record the TextEncoder for any compatible encoder class (_QWEN3_ENCODER_ARCHITECTURES) and the Tokenizer for the slow and fast Qwen2 tokenizer classes, not just Qwen3ForCausalLM / Qwen2Tokenizer. Otherwise a valid pipeline advertising e.g. Qwen2ForCausalLM or Qwen2TokenizerFast was recorded as partial and forced to use separate VAE/Qwen3 sources. Z-Image / FLUX.2 self-contained SDNQ pipelines - Add a shared is_self_contained_sdnq_pipeline() helper requiring the specific VAE + Qwen3 (text_encoder + tokenizer) submodels a pipeline install must ship. A truthy submodels dict is not enough: a partial (or partially recognized) pipeline can expose only the transformer and would fail at runtime on missing fixed subfolders. - z_image_model_loader and flux2_klein_model_loader: use the helper for both the main-model self-contained fallback and the explicit qwen3_source validator, so a partial pipeline (as main model or selected source) requires an explicit VAE/Qwen3 source instead of emitting requests against missing folders. - Drop the ui_model_format=diffusers hint from z_image qwen3_source_model so SDNQ pipelines are not filtered out of the node/workflow model pickers. - Mirror the same specific-submodels requirement in the frontend readiness checks (Z-Image + FLUX.2, generate and canvas tabs) and buildZImageGraph, and bring the FLUX.2 canvas readiness block to parity with the generate tab (it previously used format !== 'diffusers' and never recognized SDNQ pipelines). Add regression tests for every fix: mixed-group uint4 dequant, the load-guard helper, SDNQ FLUX.2 identification, SDNQ ZImagePipeline-folder identification, Qwen3 encoder folder identification (SDNQ rejection, causal-LM rejection, compatible Qwen2 class names, sharded folders), SDNQ pipeline submodel discovery across compatible encoder/tokenizer classes, the Z-Image and FLUX.2 self-contained loader/readiness paths incl. partial-pipeline handling, and the qwen3_source_model field template. * Exclude Qwen2-VL encoder from SDNQ pipeline self-contained discovery The SDNQ FLUX.2 / Z-Image pipeline loaders instantiate a text-only Qwen3ForCausalLM for the discovered text_encoder/ folder, so they cannot load a Qwen2VLForConditionalGeneration model (multimodal, visual tower). Recording it as a self-contained TextEncoder marked the pipeline complete even though the loader would fail on the visual-tower weights, causing readiness/invocation to select the main model as the Qwen source. Narrow _get_submodels()'s accepted TextEncoder classes to the text-only Qwen causal-LM classes, excluding the Qwen-VL class. * Harden SDNQ model identification, loading, dequantization, and cache accounting Address the SDNQ review findings on the feature branch: Dequantization / tensor correctness - sdnq/loaders.py: derive per-tensor group_size from the scale tensor's group dimension instead of trusting quantization_config.json, so dynamic-mixed models (e.g. FLUX.2 Klein 4B) whose per-layer group size differs from the config value no longer break the reshape/broadcast in dequantize_uint4/int5_per_group. - sdnq/utils.py: normalize 2D [out_features, num_groups] scale/zero_point with a trailing singleton before per-group arithmetic; a 2D param previously right-aligned against the 3D grouped weight and failed broadcasting. - sdnq/sdnq_tensor.py: when an SDNQTensor is moved with .to(device) (aten _to_copy), move its scale/zero_point/svd payloads too. Previously only quantized_data moved, leaving auxiliary tensors in system RAM and forcing a host->device copy of all of them on every dequantization of every layer, every step. - Add SDNQTensor.sdnq_storage_nbytes() and make calc_tensor_size() SDNQ-aware (duck-typed) so cache eviction / partial-load budgets use real storage (packed data + scale + zero_point + svd) instead of the wrapper's dequantized shape × uint8, which over-counted packed weights and omitted the auxiliary tensors. Loader robustness - Add shared raise_on_incomplete_sdnq_load() and use it in every SDNQ folder loader that called load_state_dict(strict=False) without checking the result (FLUX.2 pipeline transformer, SDNQ CLIP/T5/VAE, standalone SDNQ VAE, both Z-Image Qwen3 text-encoder paths). Partial/mismatched exports now fail fast with the offending keys. Tied/re-shared weights are allow-listed; replace the T5 assert checks (stripped by python -O) and add the missing unexpected-key check to the standalone Qwen3 SDNQ loader. - vae.py: SDNQ VAE detection now inspects all safetensors shards (union of keys) and the quantization_config.json marker, and _load_sdnq_vae hands sharded directories to sdnq_sd_loader — a VAE split across standard shard files is detected and loaded instead of falling through to the generic diffusers loader. Model identification - Single-file SDNQ FLUX.2: accept only the bare diffusers layout (transformer_blocks./context_embedder.) that the loader supports and reject BFL / ComfyUI (model.diffusion_model.double_blocks.*) checkpoints, which the loader cannot convert. Reject FLUX.2 in Main_SDNQ_FLUX_Config and list FLUX.2 before FLUX.1 in the config union so an ambiguous checkpoint never reaches the FLUX.1 loader. - Main_Diffusers_ZImage_Config: reject SDNQ-quantized ZImagePipeline folders so the SDNQ config matches them instead (mirrors the FLUX/FLUX.2 guards). - Qwen3Encoder_Qwen3Encoder_Config: reject SDNQ folders so it no longer competes (non-deterministically) with the SDNQ folder config. - Qwen3Encoder_SDNQ_Folder_Config: verify the folder is a Qwen3 encoder, reject complete causal LMs (root config.json + tokenizer files), and accept only the architecture the SDNQ loader can build (Qwen3ForCausalLM) — rejecting a declared Qwen2ForCausalLM / Qwen2VLForConditionalGeneration folder rather than accepting one the loader's strict guard would fail on. SDNQ pipeline submodel discovery - Main_SDNQ_Diffusers_Flux2_Config / Main_SDNQ_Diffusers_ZImage_Config _get_submodels(): record the Tokenizer for the slow/fast Qwen2 tokenizer classes, and the TextEncoder only for the loadable Qwen3ForCausalLM class. A pipeline advertising Qwen2ForCausalLM or the multimodal Qwen2VLForConditionalGeneration is therefore not marked self-contained, since the pipeline loaders build a text-only Qwen3ForCausalLM. Self-contained SDNQ pipelines (Z-Image / FLUX.2) - Add shared is_self_contained_sdnq_pipeline() requiring the specific VAE + Qwen3 (text_encoder + tokenizer) submodels; a truthy submodels dict is not enough. - z_image_model_loader and flux2_klein_model_loader use it for both the main-model fallback and the explicit qwen3_source validator; drop the ui_model_format=diffusers hint from z_image qwen3_source_model so SDNQ pipelines aren't filtered out of the pickers. - Mirror the specific-submodels requirement in the frontend readiness checks (Z-Image + FLUX.2, generate and canvas tabs) and buildZImageGraph, and bring the FLUX.2 canvas readiness block to parity with the generate tab. Docs - sdnq-quantization.mdx: stop recommending the unsupported FLUX.2 dev SDNQ export; recommend a supported Klein 9B export and note FLUX.2 dev is not implemented. Add regression tests for every fix: mixed-group and 2D-scale uint4/int5 dequant, device movement of all payloads, SDNQ-aware size accounting, the load-guard helper, sharded SDNQ VAE detection, SDNQ FLUX.2 identification (bare accepted / BFL rejected), SDNQ ZImagePipeline-folder identification, Qwen3 encoder folder identification (SDNQ/causal-LM rejection, Qwen3-only architecture), SDNQ pipeline submodel discovery (loadable vs unloadable encoder classes, fast tokenizer), the self-contained loader/readiness paths incl. partial-pipeline handling, and the qwen3_source_model field template. * Chore Ruff * docs: regenerate invocation-context data for offload_from_vram * Fix: address SDNQ review merge blockers (partial pipelines, Qwen/T5 identification, cat/stack) - SDNQ FLUX.2 & Z-Image submodel discovery now records a component only when its subfolder exists on disk, not merely because model_index.json advertises it. A partial download that keeps the full index no longer classifies as a self-contained pipeline and then fails in the loader on missing fixed vae/, text_encoder/, tokenizer/ subfolders. - Single-file SDNQ Qwen encoder identification now requires Qwen3-only QK-norm (q_norm/k_norm) weights and rejects a bundled Qwen-VL visual tower, so a Qwen2 causal LM or a Qwen-VL checkpoint no longer resolves to Qwen3Encoder_SDNQ_Config (the loader builds a text-only Qwen3ForCausalLM and would fail on either). - SDNQ T5 encoder: share resolve_text_encoder_dir/resolve_tokenizer_dir so the tokenizer is located per layout (child in the standalone bundle, sibling in the inline text_encoder_2 layout) instead of always <path>/tokenizer_2, and reject an install with no resolvable tokenizer_2 at identification time. - SDNQTensor.dequantize_and_run recurses into list/tuple args so torch.cat / torch.stack dequantize their operands instead of redispatching onto the SDNQTensors until RecursionError. Adds identification tests for partial pipelines, single-file Qwen2/Qwen-VL rejection, inline/standalone T5 tokenizer resolution, and torch.cat/torch.stack. * fix(pid): resolve PiD decoder review findings (state, readiness, steps, race) Address all reviewer findings on the PiD decoder feature: - Guard offload_model_from_vram with @synchronized, matching its sibling drop_model, to prevent a race when the VRAM working set is mutated concurrently during model swaps. - On base change (modelChanged), clear a PiD decoder that is incompatible with the new main model's decoder base at the root, respecting the Z-Image -> FLUX decoder reuse so a still-valid decoder is kept. - On switching to a base without PiD support, reset pidMode to 'off' and refit the dimensions so no hidden 4x native grid survives. - Extend the scaled-grid bbox readiness validation to SD3, SDXL and Z-Image, mirroring the existing FLUX.2 native-grid check. - Add a PiD Steps control (slider + number input, 1-8, default 4) with a pidStepsChanged action and selectPidSteps selector, so the documented step count is actually configurable and flows into the graph. Add readiness and paramsSlice tests covering the scaled-grid blocking and the base-change decoder/pidMode behavior * fix(pid): address review — cap steps at 4, honor encoder device, decoder/base guards Merge blockers: - PiD steps 5-8 produced duplicate timesteps: the student schedule has only 4 transitions (a 5-point list), so sub-sampling to >4 steps rounded distinct indices onto the same point and wasted network forwards on repeated timesteps. Cap the public range at 4 across the backend fields (le=4), the UI slider/input (max=4), and the pidSteps zod schema (int, 1-4), and harden _get_t_list with a strictly-decreasing assertion as a safety net. - CPU-only Gemma encoders crashed on CUDA hosts: each PiD invocation passed the global compute device to caption encoding, pushing the tokenizer output to CUDA while a cpu_only encoder stayed on the CPU. Encode on the encoder's actual device instead (next(encoder.parameters()).device), honoring model_on_device(). - Gemma 2 could no longer be configured as a generic TextLLM: the specialised- architecture exclusion rejected Gemma2ForCausalLM unconditionally. Only defer to the encoder config during automatic classification; keep an explicit type=text_llm request valid (the generic causal-LM loader supports it). Follow-ups folded in: - Reject incompatible Gemma 2 sizes before execution: PiD's caption projection is fixed at Gemma-2-2b's 2304-dim hidden state, so the encoder config now rejects 9B (3584) / 27B (4608) up front instead of failing deep in inference. - Validate the PiD decoder's base against each base-specific decode node: the base-agnostic loader let the Nodes editor wire any decoder into any node. Add assert_pid_decoder_matches_base and call it in all seven decode nodes, preserving the Z-Image-reuses-FLUX-decoder case (its node backbone is FLUX). Add regression tests: the distill schedule (strictly decreasing 1-4, safety net trips at 5) and decoder/base validation; the Gemma2 hidden-size gate; and TextLLM classification (auto-defers, explicit type still matches, plain causal LMs match). The expand-prompt pipeline always sent a dedicated "system" role message, which some chat templates (notably Gemma) reject with "System role not supported", 500-ing prompt expansion for those models. When applying the chat template fails with a system-role error, fold the system prompt into the first user turn and retry instead of failing. Adds regression tests for both the fallback and the normal (system-supported) path. * Chore fix * fix(pid): keep large Gemma2 as TextLLM, raise (not assert) schedule guard, compute_device, narrow pid_upscale VAE Address the latest review on the PiD PR: - Merge blocker: automatic classification sent Gemma 2 9B/27B to Unknown. The PiD Gemma2 encoder config rejects their non-2304 hidden size, and TextLLM deferred *every* Gemma2ForCausalLM, so neither matched. TextLLM now defers only the size the encoder config accepts (2304 = Gemma-2-2b); larger variants stay TextLLM. - Schedule safety net used assert, which is stripped under `python -O`, leaving _get_t_list(num_steps=5) returning a duplicate schedule. Raise ValueError instead so the guard holds in optimized runtimes; the regression test now asserts ValueError and passes under `python -O`. - All seven PiD caption paths derived the Gemma device from the first parameter, which is wrong under partial loading (first param on CPU, later modules on CUDA). Use the cache contract's LoadedModel.compute_device instead. - pid_upscale advertised Z-Image / 16-channel VAEs but delegates to the FLUX-only vae_encode. Narrow the field description and validate the VAE is a FLUX AutoEncoder up front (a diffusers AutoencoderKL now fails with a clear error instead of a stripped-assert failure inside vae_encode). Update the TextLLM/Gemma2 tests (per-size config-level + a factory-level check that 2304 -> Gemma2Encoder and 3584/4608 -> TextLLM) and the schedule test (ValueError, green under python -O). * chore: regenerate OpenAPI schema and frontend types * feat(pid): accept single-file GGUF Gemma-2-2b as the PiD caption encoder The PiD Gemma encoder was directory + HuggingFace only, so a llama.cpp GGUF (e.g. gemma-2-2b-it-Q4_K_M.gguf) could not be used. Add GGUF support: - Gemma2Encoder_GGUF_Config: identifies a single .gguf file, reads the GGUF metadata and requires general.architecture == "gemma2" and <arch>.embedding_length == 2304 (Gemma-2-2b), rejecting 9B/27B as the directory config does. - Gemma2EncoderGGUFLoader (format gguf_quantized): loads via transformers from_pretrained(<dir>, gguf_file=<name>) — transformers dequantizes gemma2 GGUFs and reads the tokenizer from the GGUF metadata — then exposes the Gemma2Model decoder, matching the directory loader. PiD encodes the caption once and offloads the encoder, so dequantizing at load is acceptable. - Register the config in the AnyModelConfig union. No frontend change: the PiD encoder picker filters by type=gemma2_encoder, so the GGUF variant appears automatically. Verified end-to-end against a real q4_k_m file: it classifies as Gemma2Encoder_GGUF_Config and loads to a Gemma2Model producing 2304-dim hidden states. Adds config identification tests (match, 9B/27B rejected, non-gemma2 rejected, non-.gguf rejected). * Chore Ruff * fix(models): stop Qwen3 GGUF config from matching Gemma-2 GGUF encoders A Gemma-2 GGUF satisfies the generic Qwen3 GGUF key heuristic (token_embd.weight + blk.* keys), so it matched both Qwen3Encoder_GGUF_Config and the intended Gemma2Encoder_GGUF_Config. On a fresh install the Gemma config happened to win, but re-identification could pick Qwen3, mis-classifying the model. Add _has_gemma2_keys (Gemma uses blk.*.post_attention_norm / post_ffw_norm, which a Qwen3 encoder never has — Qwen3 has attn_q_norm/attn_k_norm instead) and reject such state dicts in both Qwen3 encoder configs' _validate_looks_like_qwen3_model (GGUF and checkpoint), mirroring the existing T5 / Qwen-VL exclusions. The Gemma config already rejects Qwen3 GGUFs via the general.architecture metadata, so the two are now mutually exclusive and identification is deterministic. Add regression tests: _has_gemma2_keys detection and that the Qwen3 GGUF config rejects a Gemma-keyed state dict. * test(models): assert re-identified Gemma GGUF drops the stale Qwen3 variant The Gemma2 GGUF encoder config has no `variant` field, so re-identifying a model previously mis-detected as a Qwen3 GGUF (which carries a variant) drops it — the serialized record has no variant key and replace_model overwrites it away. Assert this explicitly in the Gemma GGUF identification test. * fix(pid): point 2K-to-4K starter decoders at NVIDIA's v1.5 replacements NVIDIA deprecated the FLUX / FLUX.2 / Qwen-Image `res2kto4k_sr4x` PiD decoders and moved them to `checkpoints_deprecated/`, replacing them with the recommended `v1pt5_res2kto4k_sr4x` checkpoints. Our starter models still pointed at the old `checkpoints/` paths, which now 404 on install. Repoint the three affected 2K-to-4K starters (FLUX, FLUX.2, Qwen-Image) to the v1.5 successors and note the upgrade in their descriptions. The 2K (`res2k_sr4x`) decoders and the SD3 / SDXL 2K-to-4K decoders are not deprecated and are unchanged. Base and variant are still sent as explicit overrides, so config identification is unaffected by the new directory name (res2kto4k -> Res2kTo4k_Sr4x). * feat(pid): load Gemma-2 GGUF encoders natively (keep weights quantized) The GGUF Gemma encoder used transformers' from_pretrained(gguf_file=...), which dequantizes every weight at load — so a quantized Gemma cost the same VRAM as the unquantized model. Load it via InvokeAI's GGMLTensor path instead: read the config from GGUF metadata, map llama.cpp tensor names to Gemma2Model, and keep the 2D projection weights as GGMLTensor (dequantized on demand by the model cache). Materialize only the embedding and the RMSNorm weights, subtracting 1 from the norms (llama.cpp folds +1 in; Gemma2RMSNorm re-adds it), and assert nothing is left on meta. Verified: hidden states match the fully-dequantized loader within quantization tolerance. Adds key-mapping tests and a local load/compare test. NVIDIA's v1.5 decoders use a different network (lq_hidden_dim=1024, PiT injection) that build_pid_net (512-dim legacy) cannot load, causing a size-mismatch crash. - Point the FLUX/FLUX.2/Qwen 2K-to-4K starters back at the legacy checkpoints (moved to checkpoints_deprecated/) that the current network loads. - Reject a checkpoint whose lq_proj hidden dim is not the supported 512 at identification time, instead of accepting it and failing inside the decode. - Enumerate all supported backbones (add SDXL, Qwen-Image) in the loader title and correct the variant enum docs (not every backbone ships both presets). Full v1.5 architecture support is planned as a follow-up. Adds PiD decoder identification tests (legacy accepted, 1024-dim v1.5 rejected). * Chore openapi + typegen * Docs update * Chore Ruff format * Chore openapi * fix(sdnq): harden self-contained pipeline detection (review 9228) Addresses the four merge blockers from the PR #9228 review and checks in the reviewer's failing tests. - _get_submodels (FLUX.2 + Z-Image) only checked is_dir(), so the empty component folders an interrupted download leaves behind were recorded as present. Require the files the component's loader actually needs: config + weights for transformer/text_encoder/vae, a vocab/config file for the weightless tokenizer folder. - is_self_contained_sdnq_pipeline() did not require the Transformer submodel, so a malformed model_index.json advertising only VAE/encoder/ tokenizer passed the check even though every loader requests the transformer. - Qwen3Encoder_SDNQ_Folder_Config's config-less fallback matched the generic Qwen keys (model.layers.* / model.embed_tokens.weight), which Qwen2 and Qwen2-VL folders carry too. Mirror the single-file path: reject a bundled visual tower and require the Qwen3-only q_norm/k_norm weights. - isZImage/isFlux2DiffusersMainModelConfig returned true for any nonempty SDNQ submodels map, offering a transformer-only pipeline in the source pickers. Added isSelfContainedSDNQPipeline() mirroring the backend's required-submodel set and reused it in readiness.ts and buildZImageGraph.ts so both sides agree on what "complete" means. Test fixtures now write real component content, since discovery no longer accepts bare directories; the empty-dir case is requested explicitly. * fix(sdnq): address the two merge blockers from the 9228 review flux_vae_decode raised TypeError on a VAE without a shift factor. `shift_factor` is optional on diffusers' AutoencoderKL: the FLUX VAE sets one, a plain SD-style config leaves it None, and `tensor + None` raises. Guard it the way every sibling decode path already does (getattr + explicit None check) — a sweep found this was the only unguarded one. Self-contained pipeline detection trusted the advertised class name. model_index.json records what a pipeline claims each component is, and that claim selects the loader, but nothing forces it to be true. A folder can advertise Qwen3ForCausalLM for text_encoder/ and ship a Qwen2 or multimodal Qwen2-VL model: the class check passes, the file-presence check passes, the pipeline is recorded as self-contained, and the mismatch only surfaces when the loader builds a Qwen3ForCausalLM against a state dict that cannot satisfy it. Components must now also be what the index says they are, read from the component's own config (architectures / _class_name / tokenizer_class). Strict on contradiction, lenient on silence: a component declaring no class at all is still accepted, so repos omitting those keys keep working — the same trade-off as Qwen3Encoder_SDNQ_Config._validate_is_qwen3_encoder. The FLUX.1 _get_submodels is deliberately untouched; the review lists it under follow-on candidates, not blockers. Also carries two fixups from the main merge: the Klein loader's error message now names the accepted shape ("Diffusers-style FLUX.2 pipeline") since it also accepts SDNQ pipelines, and the SDNQ-added readiness test builders gained main's new required hasFlux2DevDiffusersSource field. * fix(sdnq): reject pipeline components that do not identify themselves Follow-up to the previous blocker fix, which only went half way: it rejected a component whose config *contradicted* the class model_index advertises, but still accepted one that declared nothing at all. "Some config.json plus some weight file" is exactly the shape an unrelated model has, so that left the hole open for every mismatched component whose config happens to omit the key — which is what the review means by "any component directory containing a config file and weight file". Reproduced before fixing: a text_encoder/ holding a Qwen2-VL state dict (visual.* keys) with a config declaring no architecture, under an index advertising Qwen3ForCausalLM, was recorded as a complete self-contained pipeline. Weight-bearing components (transformer / text_encoder / vae) must now name their own class. save_pretrained writes _class_name (diffusers) or architectures (transformers) for anything it saves, so one that names nothing is not a normally-produced component. The tokenizer stays lenient: it carries no weights, so nothing can be mis-instantiated against it, and tokenizer_class is written less consistently. Rejecting is not fatal to the install — the component is left unrecorded, the pipeline is simply not self-contained, and the user wires it up explicitly, which is the right outcome for a folder we cannot confirm. The fixtures wrote empty component configs and would have failed under the stricter rule; they now carry the class a real save_pretrained writes. The old leniency test is replaced by two: rejection for text_encoder/vae, and the remaining leniency for the tokenizer. * fix(sdnq): work through the follow-on findings from the 9228 review Discovery (main.py) - FLUX.1 recorded components on the index's word alone. It now runs the same two guards as the FLUX.2 / Z-Image pipelines: the folder must hold the files its loader needs, and must declare a class consistent with what the index advertises. Generalising this surfaced a latent bug — _sdnq_component_dir_is_populated only exempted Tokenizer, so FLUX.1's second tokenizer slot would have been rejected for carrying no weights. Hence _WEIGHTLESS_SUBMODEL_TYPES. - Submodels are no longer taken from a serialized config. A persisted map records what existed at install time, so a component deleted since stayed "present" and the pipeline stayed self-contained until a loader opened a folder that was gone. Loading (flux.py, vae.py) - The SDNQ loader reconstructed model_path / submodel_type.value, which assumes each index key equals its slot name. A pipeline that names its CLIP encoder something else was discovered fine and then loaded from a path that does not exist. It now follows the discovered path. - The SDNQ loaders build modules with init_empty_weights + load_state_dict rather than from_pretrained, so nothing called .eval() and anything dropout- or norm-sensitive stayed in training mode. Identification (t5_encoder.py) - The SDNQ weight/scale pair is matched across the whole directory, not within each shard. Sharding splits by tensor order, so the pair routinely straddles files; per-file matching reported such a checkpoint as unquantized whenever it carried no quantization marker. Diagnostics (sdnq/utils.py, sdnq/loaders.py, sdnq/sdnq_tensor.py) - The uint4 diagnostic ran full-tensor reductions and a unique() sort on the first dequantization of every model, before anything checked whether the output would be read. Now gated on the log level and bounded to a fixed sample. Three stdout prints removed; two were verbatim duplicates of the logger call beneath them. flux_model_loader - A complete SDNQ pipeline ships its own T5, CLIP and VAE, which is what the SDNQ docs promise, but the node required separate identifiers anyway and users had to install duplicates. The three inputs are now optional and fall back to the pipeline. FLUX.1 needs six submodels (the T5 pair on top), so this uses its own completeness check rather than the single-encoder one. Models that cannot supply the parts are still rejected, naming every missing part at once. Node version 1.1.0. * fix(sdnq): close the review-4891383742 blockers and generalise the two loader fixes Merge blockers - The frontend still demanded standalone T5 / CLIP / VAE for every FLUX.1 model, in readiness (both tabs) and as an assert in buildFLUXGraph, so the backend's self-contained path was unreachable through the UI. A complete SDNQ pipeline now satisfies readiness on its own and the graph omits the inputs, letting the node resolve them from the main model. FLUX.1 needs six submodels (the T5 pair on top), so the frontend check mirrors is_self_contained_sdnq_flux1_pipeline rather than the single-encoder one. Documented in sdnq-quantization.mdx. - The component population check accepted .bin/.pt/.pth/.ckpt/.gguf, but sdnq_sd_loader globs *.safetensors and raises when it finds none, so a correctly declared component holding an unsupported format was recorded as self-contained and failed at load time. Made loader-specific rather than narrowing everything: a *quantized* component (quantization_config .json present) must ship safetensors, an unquantized one — typically the VAE, which SDNQ exports leave in bfloat16 — still goes through from_pretrained and may ship what diffusers reads. Generalising the two follow-ons, per review and beyond it A survey of every loader that hand-builds a module found 43 of 52 returning without .eval(), across FLUX, FLUX.2, Z-Image, Krea-2, Qwen-Image, Anima, Wan, Mistral and Ideogram — not the handful named. Rather than 43 edits, put_in_eval_mode now runs in load_default._load_and_cache, the single construction choke point every loader passes through, so it also covers loaders that do not exist yet. It is idempotent for the nine that already did it, and passes tokenizers, schedulers and pipelines through untouched. The local _in_eval_mode helper added to flux.py earlier is gone; one mechanism, not two. Likewise the path fix: resolve_submodel_path replaces the per-site Path(config.path) / "<slot>" reconstruction in the FLUX.1 dispatch, the FLUX.2 transformer/encoder/tokenizer/VAE loaders and the Z-Image equivalents — the loaders whose configs carry an index-key-derived submodels map. The plain-diffusers loaders (Stable Diffusion, ONNX, CogView4, Wan, Qwen-Image, Krea-2) keep the conventional layout on purpose: they have no such map. Verified against real installs: Z-Image-Turbo-SDNQ-uint4-svd-r32 and FLUX.2-klein-9B-SDNQ-4bit-dynamic-svd-r32 identify unchanged, with all four submodels and correct variants. Both declare a class on every weight-bearing component and ship only safetensors, which is what makes the stricter discovery safe. * fix(sdnq): decide component weight formats by loader, and size what we load Blocker — the quantization marker was the wrong criterion Whether `sdnq_sd_loader` reads a component is a property of the loader, not of the folder, but the population check asked `_is_sdnq_folder`, which only looks for quantization_config.json. A component carrying SDNQ weight/scale keys without that marker read as unquantized, so a .bin or .gguf passed discovery and then failed in a loader that was always going to want safetensors. Modelled explicitly instead, because it genuinely differs per pipeline: FLUX.2 and Z-Image build their transformer *and* Qwen3 text encoder with sdnq_sd_loader unconditionally, while FLUX.1 does so only for the transformer — its CLIP, T5 and VAE branch on the folder marker and fall back to from_pretrained. Requiring safetensors across the board would have rejected a legitimate unquantized CLIP encoder shipping .bin. Finding — cache sizing had drifted from the load path Loading follows the discovered component path now, but get_size_fs still appended submodel_type.value. For an index key like `clip_encoder` that measures a `text_encoder/` that does not exist, returns 0, and make_room reserves nothing before a multi-GB component is read. It now goes through resolve_submodel_path, on a branch that only triggers when the discovered path differs from the conventional one — the normal case is byte-identical, which matters because calc_model_size_by_fs has a standalone-install fallback keyed on the subfolder name. Docs The reviewer asked for documentation of the weight-format rule. It is a sentence on the existing "no safetensors files found" troubleshooting entry rather than a section in the how-to flow: published SDNQ repos ship safetensors throughout, so a user installing one can never hit it, and the symptom already had a home. Both mutation-checked: ignoring the per-loader set fails four tests, reverting get_size_fs fails one. Z-Image-Turbo-SDNQ-uint4-svd-r32 and FLUX.2-klein-9B-SDNQ-4bit-dynamic-svd-r32 identify unchanged. * fix(ui): keep the FLUX.1 readiness test off MainModelConfig.submodels The fixture is cast to MainModelConfig, which has no `submodels` field — the generated schema doesn't carry the SDNQ variants — so reading it back through the cast failed tsc in CI. Keep the map as its own const. vitest's typecheck only covers *.test-d.ts, so a full `tsc --noEmit` is what catches this; running the suite is not enough after touching a test file. Also formats readiness.ts, which prettier would have failed on next. * fix(sdnq): one SDNQ detector, and bump the two nodes whose contract changed Detection had four near-identical implementations — configs/main.py, the FLUX and Z-Image loaders, and vae.py — and only vae.py looked past the quantization_config.json marker. That divergence is a defect, not untidiness: identification and loading inspect the same folder and must reach the same verdict. On a markerless export they didn't, so identification handed the folder to a plain-diffusers config and the loader then ran from_pretrained() over packed SDNQ weights. They now all delegate to quantization/sdnq/detection.py, which checks the marker first and falls back to a weight/scale key pair resolved across the union of every shard (sharding routinely separates a weight from its scale). t5_encoder's key-only check is the same code. The tests drive every detector over the same cases so they cannot drift apart again. Node versions: flux2_klein_model_loader 1.0.0 -> 1.1.0 and z_image_model_loader 3.0.0 -> 3.1.0. Both dropped their ui_model_format=Diffusers pin and gained a self-contained-pipeline fallback in this PR, so a node serialized against the old version still carries the old field contract. Also reads the FLUX.2 variant through flux2_variant.py instead of comparing the transformer geometry by hand. The hand-rolled comparison had no [dev] branch, so [dev]'s 15360 / 6144 fell into the else and a [dev] SDNQ pipeline was silently identified as Klein4B. This is not [dev] support — that is separate work — only a correct label instead of a wrong one; the Klein path is covered by its own test. Verified against real installs: Z-Image-Turbo-SDNQ-uint4-svd-r32 and FLUX.2-klein-9B-SDNQ-4bit-dynamic-svd-r32 identify unchanged. * fix(sdnq): reject a transformer whose weights the loader cannot read `_validate_has_sdnq_transformer` only checked that the folder looked SDNQ. Discovery, which uses a stricter predicate, then left Transformer out of `submodels` — but the config was created anyway, so the model installed cleanly and only failed when a loader opened the transformer path at generation time. Identification now requires `_sdnq_component_dir_is_populated` for the transformer: the same predicate discovery uses, so the two cannot disagree about the slot every SDNQ loader needs. Applied in all three pipeline configs. Docs: the detection section still named `quant_method: "sdnq"` as the criterion. Markerless exports are recognized by their weight/scale key pairs now, across shards, and the transformer's weights have to be readable safetensors — both documented. --------- Co-authored-by: Jonathan <34005131+JPPhoto@users.noreply.github.com> Co-authored-by: JPPhoto <jpollack@jpollackphoto.com>	1 个月前