JJPPhotoRound 2 copyright removal
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
Store CIELab_to_UPLab.icc within the repo | 1 年前 | |
Apply ruff rule to disallow all relative imports. | 2 年前 | |
feat: add native Intel XPU (torch.xpu) device support (#9401) * feat(backend): add Intel XPU (torch.xpu) device support Additive xpu branches only: device selection and normalization, float16 default, VRAM queries with a passthrough-VM fallback (missing SYCL free-memory aspect), fp8 layerwise casting via a runtime probe, VAE auto-tiling, partial loading, stats/OOM handling, multi-GPU parallel session execution (device enumeration, config/API validation, worker pinning, and the generation-device options endpoint), and the auxiliary image utilities (depth/SAM/DINO pipelines accept xpu instead of falling back to CPU; cache clearing is device-agnostic). CUDA (incl. ROCm), MPS, and CPU behavior unchanged. Verified end to end on Arc Pro B70 hardware, including dual-GPU worker startup. * test(backend): add XPU coverage for TorchDevice Mock-based, mirroring the CUDA/MPS suites: device choice, dtype, normalize, the xpu_mem_get_info fallback branches, and multi-GPU generation_devices resolution/validation/labeling on XPU. Also makes the auto-without-CUDA generation-devices test hermetic on XPU machines. * build: add [xpu] extra torch 2.7.1+xpu / torchvision 0.22.1+xpu / pytorch-triton-xpu 3.3.1 from the torch-xpu index, gated to linux-x86_64 and win_amd64; uv.lock regenerated. * feat(backend): extend idle-GPU text encoder offload to XPU The idle-device arbiter and the session processor's borrow path both gated on `device.type == "cuda"`, so on a multi-XPU system no device ever registered and `offload_text_encoders_to_idle_gpus` (enabled by default) silently did nothing: encoders kept churning the denoise model in and out of VRAM. Register and lend XPU devices alongside CUDA. MPS is deliberately excluded -- it is always a single shared device, so there is never another GPU to borrow. Verified on a dual Intel Arc Pro B70 host: a text encoder node now runs on the idle GPU while the session denoises on the other ("Running compel on idle device xpu:0 (session device xpu:1)"). * feat(ui): show the executing GPU for XPU sessions Queue items already persist the executing device generically (e.g. "xpu:1"), but both readers dropped it: the session event only forwarded devices starting with "cuda", and the frontend index parser only matched /^cuda:(\d+)$/. On a multi-XPU system the progress circle and queue-item badges were therefore always blank. Accept indexed XPU devices in both places, and correct the queue-item field description, which claimed the device is set only on CUDA. * fix(mm): gate Krea 2 fp8 encoder casting on fp8 storage support The Qwen3-VL encoder kept its fp8 storage only on CUDA, so elsewhere an fp8 checkpoint was loaded as full bf16 (~8.9GB instead of ~4.4GB) and thrashed partial loading when sharing a GPU with a large transformer. Reuse the existing cached `_device_supports_fp8_storage` probe, which already backs the layerwise-casting path. It returns True unconditionally on CUDA, so CUDA behaviour is unchanged. * chore: label XPU devices by index in load logs and fp8 help text Model load lines printed the device index only for CUDA, so every model on a multi-XPU host logged as a bare "xpu device", making it impossible to tell the GPUs apart. The FP8 Storage tooltip likewise claimed CUDA-only support. * test: cover XPU config validation, progress device and fp8 probe Three paths changed by this branch had no coverage: - The `device` field pattern was untested. `test_device_choice_xpu` looks like it covers it, but the config model does not enable `validate_assignment`, so assigning `config.device` skips validation entirely; only constructing the model exercises the pattern. Added constructor-based valid/invalid cases. - `generation_devices` validation was parametrized for cuda/cpu/mps only. - The progress event's device field, which now reports XPU sessions. Also cover `_device_supports_fp8_storage`, which gates FP8 storage in both the generic layerwise-casting path and the Krea 2 encoder: CUDA answers True without probing, CPU is rejected, and a failing XPU probe returns False instead of raising. Each new test was verified to fail when the corresponding fix is reverted. * fix(nodes): recognise XPU out-of-memory errors in the Anima VAE retry The Anima VAE decode catches OOM and retries once with tiling, which caps peak allocation. Detection matched `torch.cuda.OutOfMemoryError` or the words "out of memory" in the message, so it missed XPU entirely: torch's XPU backend does not raise a recoverable `torch.OutOfMemoryError` on exhaustion, it surfaces the Level Zero/UR result code as a plain RuntimeError -- and `UR_RESULT_ERROR_OUT_OF_DEVICE_MEMORY` contains no spaces, so the existing substring never matched. The decode therefore failed outright instead of retrying tiled. Match the `*_OUT_OF_DEVICE_MEMORY` / `*_OUT_OF_HOST_MEMORY` spellings (both UR and ZE prefixes) alongside the existing conditions, and fold the cuDNN/cuBLAS checks into the same case-insensitive comparison. Extends the existing parametrized retry test with the three XPU spellings; each was verified to fail before this change. Note the driver behaviour itself is not reproducible on the hardware used here -- this stack overcommits into host RAM and hangs rather than raising -- so the tests pin the classifier, not the driver. * style: wrap long vram_usage_gb ternary for ruff * fix: drop CUDA-only wording from progress device description Matches the committed openapi/schema artifacts, which already say "on a GPU". * docs: regenerate settings data for xpu device values * fix: stop xpu VRAM probe from reporting an unknown total as zero (0, 0) made the cache's available-VRAM arithmetic collapse to a constant -working_mem budget for the life of the process. Also widen the except: the failure type moves between torch releases (RuntimeError for the missing SYCL aspect, AssertionError from _lazy_init), and warn once when the blind estimate is in use. * fix: probe fp8 support on the target device, per device, without caching failures The probe allocated via an index-less "xpu", which resolves through the thread's current XPU device rather than the device being loaded onto -- so during idle-GPU encoder offload it measured the busy denoise GPU. It was also keyed on device type, letting one device decide for another, and memoised transient failures (it runs during a load, when the device may be momentarily full) with no way back but a restart. Also probe the bf16 upcast, which is the runtime path for Krea-2/FLUX. * fix: pin torch current device when borrowing an idle GPU Worker startup set both the session device and torch's per-thread current device; the offload borrow set only the former, leaving index-less allocations on the worker's own GPU. Extracted the shared helper and guarded it on backend availability. * fix: keep idle-GPU borrows within one device type generation_devices accepts a mixed list, so a cuda session could be handed an xpu device for its text encoder. * feat: detect Intel integrated GPUs via Level Zero torch exposes no is-integrated flag, but Level Zero does (ZE_DEVICE_PROPERTY_FLAG_INTEGRATED), and its loader already ships with the torch+xpu runtime -- so no new dependency and no compiled extension. Use it to keep iGPUs out of `generation_devices: auto` when a discrete GPU exists, and to stop budgeting them as dedicated VRAM (they share system RAM, like MPS). An unknown answer keeps the previous behaviour, an iGPU-only machine keeps its device, and an explicit device list can still opt one in. * feat: add xpu torch index to pins.json Gives the launcher an Intel install option instead of requiring a manual pip install of the extra. * fix: report VRAM diagnostics for the device in use All three sites dispatched on torch.cuda.is_available() first, so a mixed NVIDIA + Arc box running on xpu reported a constant 0.0 GB and logged "CUDA Memory Allocated" -- which would make XPU bug reports unactionable. * docs: record why xpu takes the CUDA VAE constants and keeps the broad OOM needle XPU SDPA was measured on Arc Pro B70 / torch 2.13+xpu: peak memory doubles when sequence length doubles (2.00x across 2048-16384; 2.0 MB at seq=16384 vs 512 MB for a materialised score matrix). So XPU is in CUDA's O(area) regime, not ROCm's math-attention regime, and the existing constants are correct rather than accidental. * fix: derive rand_device metadata from the backend's devices Was hardcoded to 'cuda' for any non-CPU noise, which is wrong on Arc. Falls back to 'cuda' when the device query has not resolved, so Nvidia metadata is unchanged. * docs: add Intel Arc install, driver and VRAM-reporting notes * fix: probe fp8 device support only when a model requests it The probe was the first statement in _should_use_fp8, so it allocated on the GPU during the first load of any model at all -- tokenizer, VAE, scheduler -- and on API/install threads it forced XPU lazy SYCL init on a thread that never generates. Moved below the exclusions. * fix: query Level Zero Sysman for driver-global free VRAM on xpu The blind estimate (total minus this process's reserved bytes) is what made _get_vram_available over-commit on a shared GPU: it feeds a formula that assumes a driver-global figure. Sysman's zesMemoryGetState reports that figure and is often available when the SYCL ext_intel_free_memory aspect is not, so try it before estimating. Measured on Arc Pro B70 with 16 GiB held by another process: Sysman reported 15.553 GiB free, the estimate 31.725 -- a 16.172 GiB error, exactly the foreign allocation. Sysman is not a guaranteed substitute (torch's query bottoms out in the same layer), so the estimate remains as a last resort. * fix: make the fp8 probe mirror the runtime cast path The storage cast happens on CPU while params are still CPU-resident, then the fp8 tensor is copied to the device and the pre-hook upcasts there. Probing all three steps on the device would pass on a build where the host->device fp8 copy or one upcast target fails, and break at forward time instead. Verified on Arc Pro B70 / torch 2.13+xpu: the full sequence works on both cards. * fix: degrade gracefully when a backend cannot name a device torch.xpu.get_device_name goes through _lazy_init, which raises AssertionError on a build without XPU. Naming is used only for labels and logs, so fall back to the device string rather than propagating. * fix: resolve an index-less device in the Sysman VRAM query Returning None for a device with no index would skip the driver-global query and fall through to the blind estimate with no visible symptom. Callers currently always pass a concrete device, so this is a latent hazard rather than a live bug. * fix: declare ctypes prototypes for the Level Zero calls Handles come back from (c_void_p * n)() as plain Python ints, and ctypes converts an undeclared int argument to a C int -- 32 bits. Any handle above 2**31 was being silently truncated; a direct test of that path segfaults. It happened to work on the B70 because the handles fit. Also: release the idle-GPU borrow if re-pinning raises (the setup was outside the try, so a failure there stranded the lock for the life of the process), and report a failing fp8 probe once per device instead of on every model load. * fix: drop the ZES_ENABLE_SYSMAN mutation from the Sysman probe Setting a process-wide environment variable from a read-only query leaks into child processes. It also bought nothing: the variable only gates Sysman on runtimes predating zesInit and must be set before Level Zero initialises, which torch has already done by then. Verified on Arc Pro B70 that zesInit succeeds with the variable unset. * refactor: tidy up the xpu additions after a cleanup review level_zero: cache the loader so it is opened and its prototypes configured once rather than twice, share the driver/device enumeration and its ordering guard between the two probes, and collapse the Sysman pair of globals into one nullable tuple. Also: fp8 support cache is a set (it only ever stored True), the pbr_maps empty_cache is routed through TorchDevice like the PR's other conversions, the shared-memory VRAM branch stops re-testing the device type it matched on, `_auto_generation_devices` partitions in one pass, and rand_device only answers when every generation device is the same accelerator. Merges three duplicate mem_get_info tests into one parametrized case and drops two fp8 probe tests fully subsumed by the cast-sequence test. * build: pin the xpu extra to torch 2.13.0 Intel's XPU backend matured considerably after 2.7.1: torch.xpu.mem_get_info() works on driver/kernel combinations where it previously raised, and the oneAPI user-space runtime ships with the wheel, so upgrading torch upgrades it too. Follows the rocm extra, which already pins ahead of cpu/cuda. pytorch-triton-xpu was renamed triton-xpu upstream. The darwin/aarch64 fallbacks stay on 2.7.1 to match the other extras and the project's torch<2.8.0 constraint on darwin. cpu/cuda/rocm exports are unchanged package-for-package (196/211/197); the only delta is a dropped "via pytorch-triton-xpu" comment annotation from the rename. * test: stub Sysman in the unknown-total xpu probe test Without it the test only passes where Level Zero cannot answer -- never on the Intel hardware the probe exists for, where Sysman returns before the tier under test is reached. * build: teach the pins check about the xpu index Its per-platform allowlist rejects anything unlisted, so pins.json's xpu entry fails it. PyTorch publishes XPU wheels for win32 and linux x86_64, matching the extra's markers. * fix: defer the xpu device pin like cuda's torch.xpu.set_device() brings up a SYCL context that holds VRAM in an otherwise idle process, the same reason the CUDA pin waits for the first claimed queue item. * docs: regenerate settings data on linux Regenerating on Windows flips two path defaults to backslashes, which the docs check rejects. * fix(mm): handle shared memory on integrated GPUs Their VRAM is system RAM, so a RAM copy doubles each model's footprint against the same pool. Drop it, letting a full load move weights rather than copy them. Keep partial loading on -- it is the only path that respects vram_available -- and raise a clean error when a full-load-only model cannot fit, instead of walking into an uncatchable OOM-kill. Warn once when a setting is overridden. Scoped to integrated XPU; CPU and MPS are unchanged. * docs: note intel device selection and integrated-GPU memory `auto` prefers CUDA on a mixed Nvidia/Arc box, and keep_ram_copy_of_weights is ignored on an integrated GPU. * chore(ui): typegen for the xpu device values * fix(mm): compare the integrated-GPU full-load guard against bytes still to move A resident model's weights occupy the same DRAM that vram_available is read from, so its total can exceed "available" precisely because it is loaded. lock() runs on every use and full_load_to_vram() is a no-op when resident; comparing the total refused the re-lock and evicted a healthy model on every other generation. Compare what full_load_to_vram() will actually move instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: run the integrated-GPU cache tests on CPU-only torch ModelCache.__init__ sizes the RAM cache from the device's total VRAM, which on an xpu execution device reads torch.xpu.get_device_properties() -- an AssertionError on the CPU-only builds CI runs, failing 9 of these tests before they reached their subject. Stub a fixed total during construction, and add a regression test for the resident-model re-lock guard. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: LexiconCode <aaronwalker@protonmail.com> Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> | 1 个月前 | |
tidy(nodes): remove matplotlib dependency It was only used for a single color conversion function. Replaced with cv2 code, tested functionality to confirm it works the same. | 1 年前 | |
feat: add native Intel XPU (torch.xpu) device support (#9401) * feat(backend): add Intel XPU (torch.xpu) device support Additive xpu branches only: device selection and normalization, float16 default, VRAM queries with a passthrough-VM fallback (missing SYCL free-memory aspect), fp8 layerwise casting via a runtime probe, VAE auto-tiling, partial loading, stats/OOM handling, multi-GPU parallel session execution (device enumeration, config/API validation, worker pinning, and the generation-device options endpoint), and the auxiliary image utilities (depth/SAM/DINO pipelines accept xpu instead of falling back to CPU; cache clearing is device-agnostic). CUDA (incl. ROCm), MPS, and CPU behavior unchanged. Verified end to end on Arc Pro B70 hardware, including dual-GPU worker startup. * test(backend): add XPU coverage for TorchDevice Mock-based, mirroring the CUDA/MPS suites: device choice, dtype, normalize, the xpu_mem_get_info fallback branches, and multi-GPU generation_devices resolution/validation/labeling on XPU. Also makes the auto-without-CUDA generation-devices test hermetic on XPU machines. * build: add [xpu] extra torch 2.7.1+xpu / torchvision 0.22.1+xpu / pytorch-triton-xpu 3.3.1 from the torch-xpu index, gated to linux-x86_64 and win_amd64; uv.lock regenerated. * feat(backend): extend idle-GPU text encoder offload to XPU The idle-device arbiter and the session processor's borrow path both gated on `device.type == "cuda"`, so on a multi-XPU system no device ever registered and `offload_text_encoders_to_idle_gpus` (enabled by default) silently did nothing: encoders kept churning the denoise model in and out of VRAM. Register and lend XPU devices alongside CUDA. MPS is deliberately excluded -- it is always a single shared device, so there is never another GPU to borrow. Verified on a dual Intel Arc Pro B70 host: a text encoder node now runs on the idle GPU while the session denoises on the other ("Running compel on idle device xpu:0 (session device xpu:1)"). * feat(ui): show the executing GPU for XPU sessions Queue items already persist the executing device generically (e.g. "xpu:1"), but both readers dropped it: the session event only forwarded devices starting with "cuda", and the frontend index parser only matched /^cuda:(\d+)$/. On a multi-XPU system the progress circle and queue-item badges were therefore always blank. Accept indexed XPU devices in both places, and correct the queue-item field description, which claimed the device is set only on CUDA. * fix(mm): gate Krea 2 fp8 encoder casting on fp8 storage support The Qwen3-VL encoder kept its fp8 storage only on CUDA, so elsewhere an fp8 checkpoint was loaded as full bf16 (~8.9GB instead of ~4.4GB) and thrashed partial loading when sharing a GPU with a large transformer. Reuse the existing cached `_device_supports_fp8_storage` probe, which already backs the layerwise-casting path. It returns True unconditionally on CUDA, so CUDA behaviour is unchanged. * chore: label XPU devices by index in load logs and fp8 help text Model load lines printed the device index only for CUDA, so every model on a multi-XPU host logged as a bare "xpu device", making it impossible to tell the GPUs apart. The FP8 Storage tooltip likewise claimed CUDA-only support. * test: cover XPU config validation, progress device and fp8 probe Three paths changed by this branch had no coverage: - The `device` field pattern was untested. `test_device_choice_xpu` looks like it covers it, but the config model does not enable `validate_assignment`, so assigning `config.device` skips validation entirely; only constructing the model exercises the pattern. Added constructor-based valid/invalid cases. - `generation_devices` validation was parametrized for cuda/cpu/mps only. - The progress event's device field, which now reports XPU sessions. Also cover `_device_supports_fp8_storage`, which gates FP8 storage in both the generic layerwise-casting path and the Krea 2 encoder: CUDA answers True without probing, CPU is rejected, and a failing XPU probe returns False instead of raising. Each new test was verified to fail when the corresponding fix is reverted. * fix(nodes): recognise XPU out-of-memory errors in the Anima VAE retry The Anima VAE decode catches OOM and retries once with tiling, which caps peak allocation. Detection matched `torch.cuda.OutOfMemoryError` or the words "out of memory" in the message, so it missed XPU entirely: torch's XPU backend does not raise a recoverable `torch.OutOfMemoryError` on exhaustion, it surfaces the Level Zero/UR result code as a plain RuntimeError -- and `UR_RESULT_ERROR_OUT_OF_DEVICE_MEMORY` contains no spaces, so the existing substring never matched. The decode therefore failed outright instead of retrying tiled. Match the `*_OUT_OF_DEVICE_MEMORY` / `*_OUT_OF_HOST_MEMORY` spellings (both UR and ZE prefixes) alongside the existing conditions, and fold the cuDNN/cuBLAS checks into the same case-insensitive comparison. Extends the existing parametrized retry test with the three XPU spellings; each was verified to fail before this change. Note the driver behaviour itself is not reproducible on the hardware used here -- this stack overcommits into host RAM and hangs rather than raising -- so the tests pin the classifier, not the driver. * style: wrap long vram_usage_gb ternary for ruff * fix: drop CUDA-only wording from progress device description Matches the committed openapi/schema artifacts, which already say "on a GPU". * docs: regenerate settings data for xpu device values * fix: stop xpu VRAM probe from reporting an unknown total as zero (0, 0) made the cache's available-VRAM arithmetic collapse to a constant -working_mem budget for the life of the process. Also widen the except: the failure type moves between torch releases (RuntimeError for the missing SYCL aspect, AssertionError from _lazy_init), and warn once when the blind estimate is in use. * fix: probe fp8 support on the target device, per device, without caching failures The probe allocated via an index-less "xpu", which resolves through the thread's current XPU device rather than the device being loaded onto -- so during idle-GPU encoder offload it measured the busy denoise GPU. It was also keyed on device type, letting one device decide for another, and memoised transient failures (it runs during a load, when the device may be momentarily full) with no way back but a restart. Also probe the bf16 upcast, which is the runtime path for Krea-2/FLUX. * fix: pin torch current device when borrowing an idle GPU Worker startup set both the session device and torch's per-thread current device; the offload borrow set only the former, leaving index-less allocations on the worker's own GPU. Extracted the shared helper and guarded it on backend availability. * fix: keep idle-GPU borrows within one device type generation_devices accepts a mixed list, so a cuda session could be handed an xpu device for its text encoder. * feat: detect Intel integrated GPUs via Level Zero torch exposes no is-integrated flag, but Level Zero does (ZE_DEVICE_PROPERTY_FLAG_INTEGRATED), and its loader already ships with the torch+xpu runtime -- so no new dependency and no compiled extension. Use it to keep iGPUs out of `generation_devices: auto` when a discrete GPU exists, and to stop budgeting them as dedicated VRAM (they share system RAM, like MPS). An unknown answer keeps the previous behaviour, an iGPU-only machine keeps its device, and an explicit device list can still opt one in. * feat: add xpu torch index to pins.json Gives the launcher an Intel install option instead of requiring a manual pip install of the extra. * fix: report VRAM diagnostics for the device in use All three sites dispatched on torch.cuda.is_available() first, so a mixed NVIDIA + Arc box running on xpu reported a constant 0.0 GB and logged "CUDA Memory Allocated" -- which would make XPU bug reports unactionable. * docs: record why xpu takes the CUDA VAE constants and keeps the broad OOM needle XPU SDPA was measured on Arc Pro B70 / torch 2.13+xpu: peak memory doubles when sequence length doubles (2.00x across 2048-16384; 2.0 MB at seq=16384 vs 512 MB for a materialised score matrix). So XPU is in CUDA's O(area) regime, not ROCm's math-attention regime, and the existing constants are correct rather than accidental. * fix: derive rand_device metadata from the backend's devices Was hardcoded to 'cuda' for any non-CPU noise, which is wrong on Arc. Falls back to 'cuda' when the device query has not resolved, so Nvidia metadata is unchanged. * docs: add Intel Arc install, driver and VRAM-reporting notes * fix: probe fp8 device support only when a model requests it The probe was the first statement in _should_use_fp8, so it allocated on the GPU during the first load of any model at all -- tokenizer, VAE, scheduler -- and on API/install threads it forced XPU lazy SYCL init on a thread that never generates. Moved below the exclusions. * fix: query Level Zero Sysman for driver-global free VRAM on xpu The blind estimate (total minus this process's reserved bytes) is what made _get_vram_available over-commit on a shared GPU: it feeds a formula that assumes a driver-global figure. Sysman's zesMemoryGetState reports that figure and is often available when the SYCL ext_intel_free_memory aspect is not, so try it before estimating. Measured on Arc Pro B70 with 16 GiB held by another process: Sysman reported 15.553 GiB free, the estimate 31.725 -- a 16.172 GiB error, exactly the foreign allocation. Sysman is not a guaranteed substitute (torch's query bottoms out in the same layer), so the estimate remains as a last resort. * fix: make the fp8 probe mirror the runtime cast path The storage cast happens on CPU while params are still CPU-resident, then the fp8 tensor is copied to the device and the pre-hook upcasts there. Probing all three steps on the device would pass on a build where the host->device fp8 copy or one upcast target fails, and break at forward time instead. Verified on Arc Pro B70 / torch 2.13+xpu: the full sequence works on both cards. * fix: degrade gracefully when a backend cannot name a device torch.xpu.get_device_name goes through _lazy_init, which raises AssertionError on a build without XPU. Naming is used only for labels and logs, so fall back to the device string rather than propagating. * fix: resolve an index-less device in the Sysman VRAM query Returning None for a device with no index would skip the driver-global query and fall through to the blind estimate with no visible symptom. Callers currently always pass a concrete device, so this is a latent hazard rather than a live bug. * fix: declare ctypes prototypes for the Level Zero calls Handles come back from (c_void_p * n)() as plain Python ints, and ctypes converts an undeclared int argument to a C int -- 32 bits. Any handle above 2**31 was being silently truncated; a direct test of that path segfaults. It happened to work on the B70 because the handles fit. Also: release the idle-GPU borrow if re-pinning raises (the setup was outside the try, so a failure there stranded the lock for the life of the process), and report a failing fp8 probe once per device instead of on every model load. * fix: drop the ZES_ENABLE_SYSMAN mutation from the Sysman probe Setting a process-wide environment variable from a read-only query leaks into child processes. It also bought nothing: the variable only gates Sysman on runtimes predating zesInit and must be set before Level Zero initialises, which torch has already done by then. Verified on Arc Pro B70 that zesInit succeeds with the variable unset. * refactor: tidy up the xpu additions after a cleanup review level_zero: cache the loader so it is opened and its prototypes configured once rather than twice, share the driver/device enumeration and its ordering guard between the two probes, and collapse the Sysman pair of globals into one nullable tuple. Also: fp8 support cache is a set (it only ever stored True), the pbr_maps empty_cache is routed through TorchDevice like the PR's other conversions, the shared-memory VRAM branch stops re-testing the device type it matched on, `_auto_generation_devices` partitions in one pass, and rand_device only answers when every generation device is the same accelerator. Merges three duplicate mem_get_info tests into one parametrized case and drops two fp8 probe tests fully subsumed by the cast-sequence test. * build: pin the xpu extra to torch 2.13.0 Intel's XPU backend matured considerably after 2.7.1: torch.xpu.mem_get_info() works on driver/kernel combinations where it previously raised, and the oneAPI user-space runtime ships with the wheel, so upgrading torch upgrades it too. Follows the rocm extra, which already pins ahead of cpu/cuda. pytorch-triton-xpu was renamed triton-xpu upstream. The darwin/aarch64 fallbacks stay on 2.7.1 to match the other extras and the project's torch<2.8.0 constraint on darwin. cpu/cuda/rocm exports are unchanged package-for-package (196/211/197); the only delta is a dropped "via pytorch-triton-xpu" comment annotation from the rename. * test: stub Sysman in the unknown-total xpu probe test Without it the test only passes where Level Zero cannot answer -- never on the Intel hardware the probe exists for, where Sysman returns before the tier under test is reached. * build: teach the pins check about the xpu index Its per-platform allowlist rejects anything unlisted, so pins.json's xpu entry fails it. PyTorch publishes XPU wheels for win32 and linux x86_64, matching the extra's markers. * fix: defer the xpu device pin like cuda's torch.xpu.set_device() brings up a SYCL context that holds VRAM in an otherwise idle process, the same reason the CUDA pin waits for the first claimed queue item. * docs: regenerate settings data on linux Regenerating on Windows flips two path defaults to backslashes, which the docs check rejects. * fix(mm): handle shared memory on integrated GPUs Their VRAM is system RAM, so a RAM copy doubles each model's footprint against the same pool. Drop it, letting a full load move weights rather than copy them. Keep partial loading on -- it is the only path that respects vram_available -- and raise a clean error when a full-load-only model cannot fit, instead of walking into an uncatchable OOM-kill. Warn once when a setting is overridden. Scoped to integrated XPU; CPU and MPS are unchanged. * docs: note intel device selection and integrated-GPU memory `auto` prefers CUDA on a mixed Nvidia/Arc box, and keep_ram_copy_of_weights is ignored on an integrated GPU. * chore(ui): typegen for the xpu device values * fix(mm): compare the integrated-GPU full-load guard against bytes still to move A resident model's weights occupy the same DRAM that vram_available is read from, so its total can exceed "available" precisely because it is loaded. lock() runs on every use and full_load_to_vram() is a no-op when resident; comparing the total refused the re-lock and evicted a healthy model on every other generation. Compare what full_load_to_vram() will actually move instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: run the integrated-GPU cache tests on CPU-only torch ModelCache.__init__ sizes the RAM cache from the device's total VRAM, which on an xpu execution device reads torch.xpu.get_device_properties() -- an AssertionError on the CPU-only builds CI runs, failing 9 of these tests before they reached their subject. Stub a fixed total during construction, and add a regression test for the resident-model re-lock guard. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: LexiconCode <aaronwalker@protonmail.com> Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> | 1 个月前 | |
Fix bare except clauses and mutable default arguments (#8871) * Fix bare except clauses and mutable default arguments Replace bare `except:` with `except Exception:` in sqlite_database.py and mlsd/utils.py to avoid catching KeyboardInterrupt and SystemExit, which can prevent graceful shutdowns and mask critical errors (PEP 8 E722). Replace mutable default arguments (lists) with None in imwatermark/vendor.py to prevent shared state between calls, which is a known Python gotcha that can cause subtle bugs when default mutable objects are modified in place. * add tests for mutable defaults and bare except fixes * Simplify exception propagation tests * Remove unused db initialization in error propagation tests Removed unused database initialization in tests for KeyboardInterrupt and SystemExit. --------- Co-authored-by: Jonathan <34005131+JPPhoto@users.noreply.github.com> Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com> | 7 个月前 | |
Imports | 1 年前 | |
feat(nodes): add `MediaPipeFaceDetectionInvocation` Similar to the existing node, but without any resizing and with a revised model loading API that uses the model manager. All code related to the invocation now lives in the Invoke repo. | 2 年前 | |
Fix bare except clauses and mutable default arguments (#8871) * Fix bare except clauses and mutable default arguments Replace bare `except:` with `except Exception:` in sqlite_database.py and mlsd/utils.py to avoid catching KeyboardInterrupt and SystemExit, which can prevent graceful shutdowns and mask critical errors (PEP 8 E722). Replace mutable default arguments (lists) with None in imwatermark/vendor.py to prevent shared state between calls, which is a known Python gotcha that can cause subtle bugs when default mutable objects are modified in place. * add tests for mutable defaults and bare except fixes * Simplify exception propagation tests * Remove unused db initialization in error propagation tests Removed unused database initialization in tests for KeyboardInterrupt and SystemExit. --------- Co-authored-by: Jonathan <34005131+JPPhoto@users.noreply.github.com> Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com> | 7 个月前 | |
Revert some files | 1 年前 | |
feat: add native Intel XPU (torch.xpu) device support (#9401) * feat(backend): add Intel XPU (torch.xpu) device support Additive xpu branches only: device selection and normalization, float16 default, VRAM queries with a passthrough-VM fallback (missing SYCL free-memory aspect), fp8 layerwise casting via a runtime probe, VAE auto-tiling, partial loading, stats/OOM handling, multi-GPU parallel session execution (device enumeration, config/API validation, worker pinning, and the generation-device options endpoint), and the auxiliary image utilities (depth/SAM/DINO pipelines accept xpu instead of falling back to CPU; cache clearing is device-agnostic). CUDA (incl. ROCm), MPS, and CPU behavior unchanged. Verified end to end on Arc Pro B70 hardware, including dual-GPU worker startup. * test(backend): add XPU coverage for TorchDevice Mock-based, mirroring the CUDA/MPS suites: device choice, dtype, normalize, the xpu_mem_get_info fallback branches, and multi-GPU generation_devices resolution/validation/labeling on XPU. Also makes the auto-without-CUDA generation-devices test hermetic on XPU machines. * build: add [xpu] extra torch 2.7.1+xpu / torchvision 0.22.1+xpu / pytorch-triton-xpu 3.3.1 from the torch-xpu index, gated to linux-x86_64 and win_amd64; uv.lock regenerated. * feat(backend): extend idle-GPU text encoder offload to XPU The idle-device arbiter and the session processor's borrow path both gated on `device.type == "cuda"`, so on a multi-XPU system no device ever registered and `offload_text_encoders_to_idle_gpus` (enabled by default) silently did nothing: encoders kept churning the denoise model in and out of VRAM. Register and lend XPU devices alongside CUDA. MPS is deliberately excluded -- it is always a single shared device, so there is never another GPU to borrow. Verified on a dual Intel Arc Pro B70 host: a text encoder node now runs on the idle GPU while the session denoises on the other ("Running compel on idle device xpu:0 (session device xpu:1)"). * feat(ui): show the executing GPU for XPU sessions Queue items already persist the executing device generically (e.g. "xpu:1"), but both readers dropped it: the session event only forwarded devices starting with "cuda", and the frontend index parser only matched /^cuda:(\d+)$/. On a multi-XPU system the progress circle and queue-item badges were therefore always blank. Accept indexed XPU devices in both places, and correct the queue-item field description, which claimed the device is set only on CUDA. * fix(mm): gate Krea 2 fp8 encoder casting on fp8 storage support The Qwen3-VL encoder kept its fp8 storage only on CUDA, so elsewhere an fp8 checkpoint was loaded as full bf16 (~8.9GB instead of ~4.4GB) and thrashed partial loading when sharing a GPU with a large transformer. Reuse the existing cached `_device_supports_fp8_storage` probe, which already backs the layerwise-casting path. It returns True unconditionally on CUDA, so CUDA behaviour is unchanged. * chore: label XPU devices by index in load logs and fp8 help text Model load lines printed the device index only for CUDA, so every model on a multi-XPU host logged as a bare "xpu device", making it impossible to tell the GPUs apart. The FP8 Storage tooltip likewise claimed CUDA-only support. * test: cover XPU config validation, progress device and fp8 probe Three paths changed by this branch had no coverage: - The `device` field pattern was untested. `test_device_choice_xpu` looks like it covers it, but the config model does not enable `validate_assignment`, so assigning `config.device` skips validation entirely; only constructing the model exercises the pattern. Added constructor-based valid/invalid cases. - `generation_devices` validation was parametrized for cuda/cpu/mps only. - The progress event's device field, which now reports XPU sessions. Also cover `_device_supports_fp8_storage`, which gates FP8 storage in both the generic layerwise-casting path and the Krea 2 encoder: CUDA answers True without probing, CPU is rejected, and a failing XPU probe returns False instead of raising. Each new test was verified to fail when the corresponding fix is reverted. * fix(nodes): recognise XPU out-of-memory errors in the Anima VAE retry The Anima VAE decode catches OOM and retries once with tiling, which caps peak allocation. Detection matched `torch.cuda.OutOfMemoryError` or the words "out of memory" in the message, so it missed XPU entirely: torch's XPU backend does not raise a recoverable `torch.OutOfMemoryError` on exhaustion, it surfaces the Level Zero/UR result code as a plain RuntimeError -- and `UR_RESULT_ERROR_OUT_OF_DEVICE_MEMORY` contains no spaces, so the existing substring never matched. The decode therefore failed outright instead of retrying tiled. Match the `*_OUT_OF_DEVICE_MEMORY` / `*_OUT_OF_HOST_MEMORY` spellings (both UR and ZE prefixes) alongside the existing conditions, and fold the cuDNN/cuBLAS checks into the same case-insensitive comparison. Extends the existing parametrized retry test with the three XPU spellings; each was verified to fail before this change. Note the driver behaviour itself is not reproducible on the hardware used here -- this stack overcommits into host RAM and hangs rather than raising -- so the tests pin the classifier, not the driver. * style: wrap long vram_usage_gb ternary for ruff * fix: drop CUDA-only wording from progress device description Matches the committed openapi/schema artifacts, which already say "on a GPU". * docs: regenerate settings data for xpu device values * fix: stop xpu VRAM probe from reporting an unknown total as zero (0, 0) made the cache's available-VRAM arithmetic collapse to a constant -working_mem budget for the life of the process. Also widen the except: the failure type moves between torch releases (RuntimeError for the missing SYCL aspect, AssertionError from _lazy_init), and warn once when the blind estimate is in use. * fix: probe fp8 support on the target device, per device, without caching failures The probe allocated via an index-less "xpu", which resolves through the thread's current XPU device rather than the device being loaded onto -- so during idle-GPU encoder offload it measured the busy denoise GPU. It was also keyed on device type, letting one device decide for another, and memoised transient failures (it runs during a load, when the device may be momentarily full) with no way back but a restart. Also probe the bf16 upcast, which is the runtime path for Krea-2/FLUX. * fix: pin torch current device when borrowing an idle GPU Worker startup set both the session device and torch's per-thread current device; the offload borrow set only the former, leaving index-less allocations on the worker's own GPU. Extracted the shared helper and guarded it on backend availability. * fix: keep idle-GPU borrows within one device type generation_devices accepts a mixed list, so a cuda session could be handed an xpu device for its text encoder. * feat: detect Intel integrated GPUs via Level Zero torch exposes no is-integrated flag, but Level Zero does (ZE_DEVICE_PROPERTY_FLAG_INTEGRATED), and its loader already ships with the torch+xpu runtime -- so no new dependency and no compiled extension. Use it to keep iGPUs out of `generation_devices: auto` when a discrete GPU exists, and to stop budgeting them as dedicated VRAM (they share system RAM, like MPS). An unknown answer keeps the previous behaviour, an iGPU-only machine keeps its device, and an explicit device list can still opt one in. * feat: add xpu torch index to pins.json Gives the launcher an Intel install option instead of requiring a manual pip install of the extra. * fix: report VRAM diagnostics for the device in use All three sites dispatched on torch.cuda.is_available() first, so a mixed NVIDIA + Arc box running on xpu reported a constant 0.0 GB and logged "CUDA Memory Allocated" -- which would make XPU bug reports unactionable. * docs: record why xpu takes the CUDA VAE constants and keeps the broad OOM needle XPU SDPA was measured on Arc Pro B70 / torch 2.13+xpu: peak memory doubles when sequence length doubles (2.00x across 2048-16384; 2.0 MB at seq=16384 vs 512 MB for a materialised score matrix). So XPU is in CUDA's O(area) regime, not ROCm's math-attention regime, and the existing constants are correct rather than accidental. * fix: derive rand_device metadata from the backend's devices Was hardcoded to 'cuda' for any non-CPU noise, which is wrong on Arc. Falls back to 'cuda' when the device query has not resolved, so Nvidia metadata is unchanged. * docs: add Intel Arc install, driver and VRAM-reporting notes * fix: probe fp8 device support only when a model requests it The probe was the first statement in _should_use_fp8, so it allocated on the GPU during the first load of any model at all -- tokenizer, VAE, scheduler -- and on API/install threads it forced XPU lazy SYCL init on a thread that never generates. Moved below the exclusions. * fix: query Level Zero Sysman for driver-global free VRAM on xpu The blind estimate (total minus this process's reserved bytes) is what made _get_vram_available over-commit on a shared GPU: it feeds a formula that assumes a driver-global figure. Sysman's zesMemoryGetState reports that figure and is often available when the SYCL ext_intel_free_memory aspect is not, so try it before estimating. Measured on Arc Pro B70 with 16 GiB held by another process: Sysman reported 15.553 GiB free, the estimate 31.725 -- a 16.172 GiB error, exactly the foreign allocation. Sysman is not a guaranteed substitute (torch's query bottoms out in the same layer), so the estimate remains as a last resort. * fix: make the fp8 probe mirror the runtime cast path The storage cast happens on CPU while params are still CPU-resident, then the fp8 tensor is copied to the device and the pre-hook upcasts there. Probing all three steps on the device would pass on a build where the host->device fp8 copy or one upcast target fails, and break at forward time instead. Verified on Arc Pro B70 / torch 2.13+xpu: the full sequence works on both cards. * fix: degrade gracefully when a backend cannot name a device torch.xpu.get_device_name goes through _lazy_init, which raises AssertionError on a build without XPU. Naming is used only for labels and logs, so fall back to the device string rather than propagating. * fix: resolve an index-less device in the Sysman VRAM query Returning None for a device with no index would skip the driver-global query and fall through to the blind estimate with no visible symptom. Callers currently always pass a concrete device, so this is a latent hazard rather than a live bug. * fix: declare ctypes prototypes for the Level Zero calls Handles come back from (c_void_p * n)() as plain Python ints, and ctypes converts an undeclared int argument to a C int -- 32 bits. Any handle above 2**31 was being silently truncated; a direct test of that path segfaults. It happened to work on the B70 because the handles fit. Also: release the idle-GPU borrow if re-pinning raises (the setup was outside the try, so a failure there stranded the lock for the life of the process), and report a failing fp8 probe once per device instead of on every model load. * fix: drop the ZES_ENABLE_SYSMAN mutation from the Sysman probe Setting a process-wide environment variable from a read-only query leaks into child processes. It also bought nothing: the variable only gates Sysman on runtimes predating zesInit and must be set before Level Zero initialises, which torch has already done by then. Verified on Arc Pro B70 that zesInit succeeds with the variable unset. * refactor: tidy up the xpu additions after a cleanup review level_zero: cache the loader so it is opened and its prototypes configured once rather than twice, share the driver/device enumeration and its ordering guard between the two probes, and collapse the Sysman pair of globals into one nullable tuple. Also: fp8 support cache is a set (it only ever stored True), the pbr_maps empty_cache is routed through TorchDevice like the PR's other conversions, the shared-memory VRAM branch stops re-testing the device type it matched on, `_auto_generation_devices` partitions in one pass, and rand_device only answers when every generation device is the same accelerator. Merges three duplicate mem_get_info tests into one parametrized case and drops two fp8 probe tests fully subsumed by the cast-sequence test. * build: pin the xpu extra to torch 2.13.0 Intel's XPU backend matured considerably after 2.7.1: torch.xpu.mem_get_info() works on driver/kernel combinations where it previously raised, and the oneAPI user-space runtime ships with the wheel, so upgrading torch upgrades it too. Follows the rocm extra, which already pins ahead of cpu/cuda. pytorch-triton-xpu was renamed triton-xpu upstream. The darwin/aarch64 fallbacks stay on 2.7.1 to match the other extras and the project's torch<2.8.0 constraint on darwin. cpu/cuda/rocm exports are unchanged package-for-package (196/211/197); the only delta is a dropped "via pytorch-triton-xpu" comment annotation from the rename. * test: stub Sysman in the unknown-total xpu probe test Without it the test only passes where Level Zero cannot answer -- never on the Intel hardware the probe exists for, where Sysman returns before the tier under test is reached. * build: teach the pins check about the xpu index Its per-platform allowlist rejects anything unlisted, so pins.json's xpu entry fails it. PyTorch publishes XPU wheels for win32 and linux x86_64, matching the extra's markers. * fix: defer the xpu device pin like cuda's torch.xpu.set_device() brings up a SYCL context that holds VRAM in an otherwise idle process, the same reason the CUDA pin waits for the first claimed queue item. * docs: regenerate settings data on linux Regenerating on Windows flips two path defaults to backslashes, which the docs check rejects. * fix(mm): handle shared memory on integrated GPUs Their VRAM is system RAM, so a RAM copy doubles each model's footprint against the same pool. Drop it, letting a full load move weights rather than copy them. Keep partial loading on -- it is the only path that respects vram_available -- and raise a clean error when a full-load-only model cannot fit, instead of walking into an uncatchable OOM-kill. Warn once when a setting is overridden. Scoped to integrated XPU; CPU and MPS are unchanged. * docs: note intel device selection and integrated-GPU memory `auto` prefers CUDA on a mixed Nvidia/Arc box, and keep_ram_copy_of_weights is ignored on an integrated GPU. * chore(ui): typegen for the xpu device values * fix(mm): compare the integrated-GPU full-load guard against bytes still to move A resident model's weights occupy the same DRAM that vram_available is read from, so its total can exceed "available" precisely because it is loaded. lock() runs on every use and full_load_to_vram() is a no-op when resident; comparing the total refused the re-lock and evicted a healthy model on every other generation. Compare what full_load_to_vram() will actually move instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: run the integrated-GPU cache tests on CPU-only torch ModelCache.__init__ sizes the RAM cache from the device's total VRAM, which on an xpu execution device reads torch.xpu.get_device_properties() -- an AssertionError on the CPU-only builds CI runs, failing 9 of these tests before they reached their subject. Stub a fixed total during construction, and add a regression test for the resident-model re-lock guard. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: LexiconCode <aaronwalker@protonmail.com> Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> | 1 个月前 | |
Add get_effective_device(...) utility to aid in determining the effective device of models that are partially loaded. | 1 年前 | |
Imports | 1 年前 | |
Merge upstream/main into integration/upstream-main-2026-08-16 | 1 个月前 | |
Apply ruff rule to disallow all relative imports. | 2 年前 | |
tidy: "fit_image_to_resolution" -> "resize_image_to_resolution" | 2 年前 | |
Add OKLab and Oklch image utilities and nodes, refactor color space nodes (#8999) * Add OKLab and Oklch image utilities and nodes * refactor: unify oklab color conversions * refactor: unify shared color conversions * chore: typegen * chore: typegen --------- Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com> | 5 个月前 | |
Round 2 copyright removal | 17 天前 | |
feat(nodes): update content shuffle node - Better field names | 2 年前 | |
External models (Gemini Nano Banana & OpenAI GPT Image) (#8633) (#8884) * feat: initial external model support * feat: support reference images for external models * fix: sorting lint error * chore: hide Reidentify button for external models * review: enable auto-install/remove fro external models * feat: show external mode name during install * review: model descriptions * review: implemented review comments * review: added optional seed control for external models * chore: fix linter warning * review: save api keys to a seperate file * docs: updated external model docs * chore: fix linter errors * fix: sync configured external starter models on startup * feat(ui): add provider-specific external generation nodes * feat: expose external panel schemas in model configs * feat(ui): drive external panels from panel schema * docs: sync app config docstring order * feat: add gemini 3.1 flash image preview starter model * feat: update gemini image model limits * fix: resolve TypeScript errors and move external provider config to api_keys.yaml Add 'external', 'external_image_generator', and 'external_api' to Zod enum schemas (zBaseModelType, zModelType, zModelFormat) to match the generated OpenAPI types. Remove redundant union workarounds from component prop types and Record definitions. Fix type errors in ModelEdit (react-hook-form Control invariance), parsing.tsx (model identifier narrowing), buildExternalGraph (edge typing), and ModelSettings import/export buttons. Move external_gemini_base_url and external_openai_base_url into api_keys.yaml alongside the API keys so all external provider config lives in one dedicated file, separate from invokeai.yaml. * feat: add resolution presets and imageConfig support for Gemini 3 models Add combined resolution preset selector for external models that maps aspect ratio + image size to fixed dimensions. Gemini 3 Pro and 3.1 Flash now send imageConfig (aspectRatio + imageSize) via generationConfig instead of text-based aspect ratio hints used by Gemini 2.5 Flash. Backend: ExternalResolutionPreset model, resolution_presets capability field, image_size on ExternalGenerationRequest, and Gemini provider imageConfig logic. Frontend: ExternalSettingsAccordion with combo resolution select, dimension slider disabling for fixed-size models, and panel schema constraint wiring for Steps/Guidance/Seed controls. * Remove unused external model fields and add provider-specific parameters - Remove negative_prompt, steps, guidance, reference_image_weights, reference_image_modes from external model nodes (unused by any provider) - Remove supports_negative_prompt, supports_steps, supports_guidance from ExternalModelCapabilities - Add provider_options dict to ExternalGenerationRequest for provider-specific parameters - Add OpenAI-specific fields: quality, background, input_fidelity - Add Gemini-specific fields: temperature, thinking_level - Add new OpenAI starter models: GPT Image 1.5, GPT Image 1 Mini, DALL-E 3, DALL-E 2 - Fix OpenAI provider to use output_format (GPT Image) vs response_format (DALL-E) and send model ID in requests - Add fixed aspect ratio sizes for OpenAI models (bucketing) - Add ExternalProviderRateLimitError with retry logic for 429 responses - Add provider-specific UI components in ExternalSettingsAccordion - Simplify ParamSteps/ParamGuidance by removing dead external overrides - Update all backend and frontend tests * Chore Ruff check & format * Chore typegen * feat: full canvas workflow integration for external models - Add missing aspect ratios (4:5, 5:4, 8:1, 4:1, 1:4, 1:8) to type system for external model support - Sync canvas bbox when external model resolution preset is selected - Use params preset dimensions in buildExternalGraph to prevent "unsupported aspect ratio" errors - Lock all bbox controls (resize handles, aspect ratio select, width/height sliders, swap/optimal buttons) for external models with fixed dimension presets - Disable denoise strength slider for external models (not applicable) - Sync bbox aspect ratio changes back to paramsSlice for external models - Initialize bbox dimensions when switching to an external model * Chore typegen Linux seperator * feat: full canvas workflow integration for external models - Update buildExternalGraph test to include dimensions in mock params * Merge remote-tracking branch 'upstream/main' into external-models * Chore pnpm fix * add missing parameter * docs: add External Models guide with Gemini and OpenAI provider pages * fix(external-models): address PR review feedback - Gemini recall: write temperature, thinking_level, image_size to image metadata; wire external graph as metadata receiver; add recall handlers. - Canvas: gate regional guidance, inpaint mask, and control layer for external models. - Canvas: throw a clear error on outpainting for external models (was falling back to inpaint and hitting an API-side mask/image size mismatch). - Workflow editor: add ui_model_provider_id filter so OpenAI and Gemini nodes only list their own provider's models. - Workflow editor: silently drop seed when the selected model does not support it instead of raising a capability error. - Remove the legacy external_image_generation invocation and the graph-builder fallback; providers must register a dedicated node. - Regenerate schema.ts. - remove Gemini debug dumps to outputs/external_debug * fix(external-models): resolve TSC errors in metadata parsing and external graph - Export imageSizeChanged from paramsSlice (required by the new ImageSize recall handler). - Emit the external graph's metadata model entry via zModelIdentifierField since ExternalApiModelConfig is not part of the AnyModelConfig union. * chore: prettier format ModelIdentifierFieldInputComponent * fix: remove unsupported thinkingConfig from Gemini image models and restrict GPT Image models to txt2img * chore typegen * chore(docs): regenerate settings.json for external provider fields * fix(external): fix mask handling and mode support for external providers - Remove img2img and inpaint modes from Gemini models (Gemini has no bitmap mask or dedicated edit API; image editing works via reference images in the UI) - Fix DALL-E 2 inpainting: convert grayscale mask to RGBA with alpha channel transparency (OpenAI expects transparent=edit area) and convert init image to RGBA when mask is present * fix(external): update mode support and UI for external providers - Remove DALL-E 2 from starter models (deprecated, shutdown May 12 2026) - Enable img2img for GPT Image 1/1.5/1-mini (supports edits endpoint) - Set Gemini models to txt2img only (no mask/edit API; editing via ref images) - Hide mode/init_image/mask_image fields on Gemini node (not usable) - Hide mask_image field on OpenAI node (no model supports inpaint) * Chore typegen * fix(external): improve OpenAI node UX and disable cache by default - Hide OpenAI node's mode and init_image fields: OpenAI's API has no img2img/inpaint distinction (the edits endpoint is invoked automatically when reference images are provided). init_image is functionally identical to a reference image and was misleading users. - Default use_cache to False for external image generation nodes: external API calls are non-deterministic and incur usage costs. Cache hits returned stale image references that did not produce new gallery entries on repeat invokes. * fix(external): duplicate cached images on cache hit instead of skipping External image generation nodes use the standard invocation cache, but returning the cached output (with stale image_name references) on cache hits resulted in no new gallery entries — the Invoke button would spin indefinitely on repeat invokes with identical parameters. Override invoke_internal so that on cache hit, the cached images are loaded and re-saved as new gallery entries. The expensive API call is still skipped (cost saving), but the user sees a new image as expected. * Chore typegen + ruff * CHore ruff format * fix(external): restore OpenAI advanced settings on Remix recall Remix recall iterates through ImageMetadataHandlers but only Gemini's temperature handler was wired up — OpenAI's quality, background, and input_fidelity were stored in image metadata but never parsed back into the params slice. Add the three missing handlers so Remix restores these settings as expected. --------- Co-authored-by: Alexander Eichhorn <alex@eichhorn.dev> Co-authored-by: Alexander Eichhorn <alex@code-with.us> Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com> | 5 个月前 | |
Add get_effective_device(...) utility to aid in determining the effective device of models that are partially loaded. | 1 年前 | |
Feature (frontend): Add invisible watermark decoder node. (#8967) * Initial plan * Add invisible watermark decoding node and utility method Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * chore(frontend): typegen --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lstein <111189+lstein@users.noreply.github.com> | 6 个月前 | |
fix(controlnet): preserve lineart source dimensions Lineart's stride-2 generator rounds odd input dimensions up to a multiple of four. Resize its float output back to the source dimensions before quantization so filter previews and alpha restoration receive matching images. | 1 个月前 | |
Add get_effective_device(...) utility to aid in determining the effective device of models that are partially loaded. | 1 年前 | |
Formatting | 1 年前 | |
fix: close read-side meta-tensor gap and harden the new lock sites Follow-up to the initial commit on this branch, from an adversarial review of the diff. - wan_denoise: take MODEL_LOAD_LOCK.read_lock() around full_unload_from_vram(). It runs load_state_dict(assign=True) -> register_parameter and was the only VRAM-move caller outside model_cache.py holding no lock, so a construction on another worker could strand the outgoing expert's weights on the meta device and leave the still-cached record poisoned. The comment records what this does NOT cover: it holds no ModelCache._lock, so it can still overlap the cache's own _offload_unlocked_models on the same record (a pre-existing race), and the read lock is held across a multi-GB copy, which a write-preferring lock turns into a queue for other workers' VRAM moves. - safety_checker: serialize the lazy init behind a dedicated _INIT_LOCK with a double-check inside it, matching ModelLoader._load_and_cache and load_remote_model. Concurrent workers previously duplicated the download and construction, ran save_pretrained() into one directory from several threads, and could observe a half-initialized checker. Persist the checker atomically via a staging directory and os.replace(). save_pretrained() creates its own destination, so writing the two objects straight into model_path made that directory exist as soon as the FIRST save landed; any later failure left a weightless directory that the model_path.exists() branch trusts forever. The image processor then loads, the checker doesn't, the error is swallowed, and has_nsfw_concept() reports the missing checker as "not NSFW" -- so images ship unblurred permanently and the download is never retried. The install is now either complete or absent. The in-memory objects are also published before persisting, so failing to cache them on disk no longer costs this run its checker. - upscale: use the read lock, not the write lock, for RRDBNet. RRDBNet is plain torch -- it calls register_parameter but never installs the init_empty_weights patch -- so it needs the reader role. The write lock gave no extra safety and blocked every other worker's VRAM moves. - qwen_image_text_encoder: make MODEL_LOAD_LOCK the outer context manager. warnings.catch_warnings() swaps the process-global filter list, and holding it across an unbounded wait for the lock left this thread's filter installed -- and clobberable by another thread's SilenceWarnings -- for the duration of someone else's model load. - _ModelLoadReadWriteLock.write_lock: decrement _writers_waiting in a finally. It was incremented before the wait loop and decremented after, so a raising _cond.wait() leaked the count -- and because read_lock defers while _writers_waiting > 0, that would block every VRAM move in the process for the rest of its life. A writer that gives up without acquiring now also notifies the readers that were deferring to it, since nothing else would wake them. read_lock needs no equivalent guard: it increments only after its wait loop. Adds regression tests for the safety-checker init race, the failed-download path, the failed-construction path, and the lock's waiting-count bookkeeping; each fails against the revision that preceded it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> | 1 个月前 | |
feat(nodes): add `PiDiNetEdgeDetectionInvocation` Similar to the existing node, but without any resizing and with a revised model loading API that uses the model manager. All code related to the invocation now lives in the Invoke repo. | 2 年前 |