SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer
| Files | Last commit | Last update |
|---|---|---|
ci: allow selecting a single test job for manual runs (#492) | 3 days ago | |
Add SANA-WM initial release (#379) * Add SANA-WM: camera-controlled image-to-video diffusion - 2.6 B bidirectional Sana DiT with hybrid GDN + softmax attention - LTX-2 sink-bidirectional Euler refiner (Stage 2) - Action-string / camera-npy CLI, Pi3X auto-intrinsics, action overlay - All weights resolved from hf://Efficient-Large-Model/SANA-WM_bidirectional * docs: fix SANA-WM logo URL + initial-release note Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: pre-commit fix (trailing whitespace + mdformat) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: use diffusers LTX-2 refiner for Sana-WM * refactor(sana-wm): default refiner to refiner_diffusers/ Drop the legacy --refiner_checkpoint single-file path; the refiner now loads exclusively from refiner_diffusers/{transformer,connectors}/ via diffusers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * update all env requirements into toml file and remove txt one; update markdown; env sh file will also be removed after thourough CI test; * refactor(sana-wm): flatten HF refiner layout Refiner now lives at refiner/{transformer,connectors,text_encoder}/ instead of split refiner_diffusers/ + refiner/text_encoder/. CLI arg renamed from --refiner_diffusers_root to --refiner_root and the RefinerSettings.diffusers_root field to .root. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(sana-wm): drop non-chunkwise GDN kernels Unreachable at inference (fused_bigdn_func short-circuits to chunkwise; use_autograd_kernel paths only fired in training). Net -5,161 lines; smoke test md5 byte-identical to pre-cleanup baseline. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Fix SANA-WM VAE pretrained path * feat(sana-wm): snap num_frames to nearest 8k+1 with warning LTX-2 VAE requires (num_frames - 1) % 8 == 0; otherwise the script silently truncated to floor(8k+1). Now we snap to the nearest valid value (ties round up) and log a warning so the user sees the change. Verified: --num_frames 80 → warns + produces 81-frame mp4. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: update .gitignore and CI workflow, add license headers, and refactor model paths - Added pattern to .gitignore for locally-symlinked Hugging Face models. - Updated CI workflow to include new training job dependency for 'test-training-sol-rl'. - Fixed shebang in add_license_all.sh script. - Added Apache License headers to multiple files across the project. - Updated test scripts to include LongSANA training. - Converted Sana-WM refiner to diffusers components with appropriate licensing. * merge all env together; * docs(sana-wm): commit logo + drop verbose vae_pretrained comment Addresses PR #379 review feedback from lawrence-cj: * Logo: asset/sana-wm-logo.png was caught by the global ``*.png`` .gitignore rule, so the absolute ``raw.githubusercontent.com/.../main/...`` URL 404'd during PR review. Whitelist the file, commit it, and switch the two doc references to relative paths (matching the main README style: ``asset/logo.png``). * Config: drop the 3-line explanatory comment block in configs/sana_wm/sana_wm_1600m_720p.yaml. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: enable fork PR testing + fix autolint label removal (#382) - Switch ci.yaml trigger to pull_request_target so fork PRs can access secrets required by our self-hosted runners. The 'run-tests' label is the maintainer-controlled trust gate (triage+ only). - Each checkout sets ref=PR head sha + persist-credentials:false so we test PR code without leaving a token in .git/config. - bot-autolint.yaml: add top-level permissions (contents / pull-requests / issues = write) so the workflow can push lint fixes and remove the 'lint wanted' label. Previously the label removal returned 403 "Resource not accessible by integration" and the label stayed. - bot-autolint.yaml: add if:always() to the remove-label step so the label is cleared even when an earlier step fails. - Remove orphan tools/scoring/dover/DOVER gitlink: not referenced anywhere in the codebase, missing from .gitmodules, caused "fatal: No url found for submodule path" warnings in every CI run. * add SANA-WM in inference ci; * chore: ignore .claude/ harness directory Local Claude worktrees + state under .claude/ shouldn't show up in git status of the repo. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(sana-wm): drop CP-only and dead utility code Single-GPU inference release: the context-parallel scan and the frame_gdn Triton kernels are dead — cp_enabled() always returns False and the training-only triton_* update_rule_func branches are never selected. Deletes: * diffusion/distributed/ (CP setup, ~1.4k LOC). * diffusion/model/ops/frame_gdn/ (frame-wise GDN scan/recurrent kernels, ~1.1k LOC). * diffusion/model/ops/fused_gdn_cp.py (CP wrappers, 1832 LOC). * diffusion/model/ops/fused_gdn_chunkwise_bwd.py (1356 LOC) and fused_gdn_chunkwise_stateful_bwd.py (948 LOC) — backward kernels consumed only by the now-deleted modules. Prunes the CP-only forward branches and triton_* update_rule_func entries from the 4 live nets that imported the dropped modules (sana_gdn_blocks, sana_gdn_blocks_triton, sana_gdn_camctrl_blocks, sana_multi_scale_video_camctrl). Net: ~7.8k LOC removed. Inference output is byte-identical to the prior cleanup baseline (md5 0343e294253835ff13beef7f3d3a1b04 on the demo_0 / 321-frame / step=60 / flow_shift=8.0 smoke test). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * remove used code; * fix(sana-wm): auto-chunk GLUMBConv spatial conv for long videos At minute-scale video lengths (e.g. 961 frames at default CFG → B*T=242) the GLUMBConvTemp depth_conv input is (242, 8064, 22, 40) ≈ 1.7 B elements, which overflows PyTorch's 32-bit indexing in both cuDNN and the ATEN fallback. Pulls the four-line spatial pipeline (inverted_conv → depth_conv → GLU → point_conv) into ``_apply_spatial`` and wraps it with ``_apply_spatial_autochunked`` that splits along dim 0 only when the per-call input would exceed ``_INT32_SAFE_CONV_ELEMENTS`` (~1 B). Short videos take the original fused path with no chunking (no overhead, no numerical change — 321-frame smoke output md5 is byte-identical to the pre-patch baseline ``0343e294...``). 961-frame inference now runs end to end and produces a 60.06 s mp4. Same helper is shared across GLUMBConv / GLUMBConvTemp / ChunkGLUMBConvTemp / CachedGLUMBConvTemp, replacing four copies of the same spatial block. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ci(sana-wm): bump inference test to 641 frames for chunk regression The previous 9-frame WM CI command exercised the GDN/refiner path but not the GLUMBConv auto-chunk fix landed in the previous commit. Bumps the inference smoke to 641 frames (latent T=81, BT=162, just over the chunk threshold ~151) so any future regression in ``_apply_spatial_autochunked`` will fail CI here. Swaps ``--camera`` / ``--intrinsics`` (321-frame asset files would auto-clamp num_frames back to 321) for ``--action=w-641`` so the test actually exercises the long-video path. Pi3X handles intrinsics from the input image. Step count stays at 4 to keep total CI time bounded. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Owen718 <2647180055@qq.com> Co-authored-by: junsong <cjs1020440147@icloud.com> Co-authored-by: Sunwood-ai-labs <sunwood.ai.labs@gmail.com> Co-authored-by: yujincheng08 <5022927+yujincheng08@users.noreply.github.com> Co-authored-by: HaozheLiu-ST <53887227+HaozheLiu-ST@users.noreply.github.com> Co-authored-by: HeliosZhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: xieenze <20984575+xieenze@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> | 3 months ago | |
Add SANA-WM initial release (#379) * Add SANA-WM: camera-controlled image-to-video diffusion - 2.6 B bidirectional Sana DiT with hybrid GDN + softmax attention - LTX-2 sink-bidirectional Euler refiner (Stage 2) - Action-string / camera-npy CLI, Pi3X auto-intrinsics, action overlay - All weights resolved from hf://Efficient-Large-Model/SANA-WM_bidirectional * docs: fix SANA-WM logo URL + initial-release note Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: pre-commit fix (trailing whitespace + mdformat) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: use diffusers LTX-2 refiner for Sana-WM * refactor(sana-wm): default refiner to refiner_diffusers/ Drop the legacy --refiner_checkpoint single-file path; the refiner now loads exclusively from refiner_diffusers/{transformer,connectors}/ via diffusers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * update all env requirements into toml file and remove txt one; update markdown; env sh file will also be removed after thourough CI test; * refactor(sana-wm): flatten HF refiner layout Refiner now lives at refiner/{transformer,connectors,text_encoder}/ instead of split refiner_diffusers/ + refiner/text_encoder/. CLI arg renamed from --refiner_diffusers_root to --refiner_root and the RefinerSettings.diffusers_root field to .root. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(sana-wm): drop non-chunkwise GDN kernels Unreachable at inference (fused_bigdn_func short-circuits to chunkwise; use_autograd_kernel paths only fired in training). Net -5,161 lines; smoke test md5 byte-identical to pre-cleanup baseline. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Fix SANA-WM VAE pretrained path * feat(sana-wm): snap num_frames to nearest 8k+1 with warning LTX-2 VAE requires (num_frames - 1) % 8 == 0; otherwise the script silently truncated to floor(8k+1). Now we snap to the nearest valid value (ties round up) and log a warning so the user sees the change. Verified: --num_frames 80 → warns + produces 81-frame mp4. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: update .gitignore and CI workflow, add license headers, and refactor model paths - Added pattern to .gitignore for locally-symlinked Hugging Face models. - Updated CI workflow to include new training job dependency for 'test-training-sol-rl'. - Fixed shebang in add_license_all.sh script. - Added Apache License headers to multiple files across the project. - Updated test scripts to include LongSANA training. - Converted Sana-WM refiner to diffusers components with appropriate licensing. * merge all env together; * docs(sana-wm): commit logo + drop verbose vae_pretrained comment Addresses PR #379 review feedback from lawrence-cj: * Logo: asset/sana-wm-logo.png was caught by the global ``*.png`` .gitignore rule, so the absolute ``raw.githubusercontent.com/.../main/...`` URL 404'd during PR review. Whitelist the file, commit it, and switch the two doc references to relative paths (matching the main README style: ``asset/logo.png``). * Config: drop the 3-line explanatory comment block in configs/sana_wm/sana_wm_1600m_720p.yaml. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: enable fork PR testing + fix autolint label removal (#382) - Switch ci.yaml trigger to pull_request_target so fork PRs can access secrets required by our self-hosted runners. The 'run-tests' label is the maintainer-controlled trust gate (triage+ only). - Each checkout sets ref=PR head sha + persist-credentials:false so we test PR code without leaving a token in .git/config. - bot-autolint.yaml: add top-level permissions (contents / pull-requests / issues = write) so the workflow can push lint fixes and remove the 'lint wanted' label. Previously the label removal returned 403 "Resource not accessible by integration" and the label stayed. - bot-autolint.yaml: add if:always() to the remove-label step so the label is cleared even when an earlier step fails. - Remove orphan tools/scoring/dover/DOVER gitlink: not referenced anywhere in the codebase, missing from .gitmodules, caused "fatal: No url found for submodule path" warnings in every CI run. * add SANA-WM in inference ci; * chore: ignore .claude/ harness directory Local Claude worktrees + state under .claude/ shouldn't show up in git status of the repo. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(sana-wm): drop CP-only and dead utility code Single-GPU inference release: the context-parallel scan and the frame_gdn Triton kernels are dead — cp_enabled() always returns False and the training-only triton_* update_rule_func branches are never selected. Deletes: * diffusion/distributed/ (CP setup, ~1.4k LOC). * diffusion/model/ops/frame_gdn/ (frame-wise GDN scan/recurrent kernels, ~1.1k LOC). * diffusion/model/ops/fused_gdn_cp.py (CP wrappers, 1832 LOC). * diffusion/model/ops/fused_gdn_chunkwise_bwd.py (1356 LOC) and fused_gdn_chunkwise_stateful_bwd.py (948 LOC) — backward kernels consumed only by the now-deleted modules. Prunes the CP-only forward branches and triton_* update_rule_func entries from the 4 live nets that imported the dropped modules (sana_gdn_blocks, sana_gdn_blocks_triton, sana_gdn_camctrl_blocks, sana_multi_scale_video_camctrl). Net: ~7.8k LOC removed. Inference output is byte-identical to the prior cleanup baseline (md5 0343e294253835ff13beef7f3d3a1b04 on the demo_0 / 321-frame / step=60 / flow_shift=8.0 smoke test). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * remove used code; * fix(sana-wm): auto-chunk GLUMBConv spatial conv for long videos At minute-scale video lengths (e.g. 961 frames at default CFG → B*T=242) the GLUMBConvTemp depth_conv input is (242, 8064, 22, 40) ≈ 1.7 B elements, which overflows PyTorch's 32-bit indexing in both cuDNN and the ATEN fallback. Pulls the four-line spatial pipeline (inverted_conv → depth_conv → GLU → point_conv) into ``_apply_spatial`` and wraps it with ``_apply_spatial_autochunked`` that splits along dim 0 only when the per-call input would exceed ``_INT32_SAFE_CONV_ELEMENTS`` (~1 B). Short videos take the original fused path with no chunking (no overhead, no numerical change — 321-frame smoke output md5 is byte-identical to the pre-patch baseline ``0343e294...``). 961-frame inference now runs end to end and produces a 60.06 s mp4. Same helper is shared across GLUMBConv / GLUMBConvTemp / ChunkGLUMBConvTemp / CachedGLUMBConvTemp, replacing four copies of the same spatial block. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ci(sana-wm): bump inference test to 641 frames for chunk regression The previous 9-frame WM CI command exercised the GDN/refiner path but not the GLUMBConv auto-chunk fix landed in the previous commit. Bumps the inference smoke to 641 frames (latent T=81, BT=162, just over the chunk threshold ~151) so any future regression in ``_apply_spatial_autochunked`` will fail CI here. Swaps ``--camera`` / ``--intrinsics`` (321-frame asset files would auto-clamp num_frames back to 321) for ``--action=w-641`` so the test actually exercises the long-video path. Pi3X handles intrinsics from the input image. Step count stays at 4 to keep total CI time bounded. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Owen718 <2647180055@qq.com> Co-authored-by: junsong <cjs1020440147@icloud.com> Co-authored-by: Sunwood-ai-labs <sunwood.ai.labs@gmail.com> Co-authored-by: yujincheng08 <5022927+yujincheng08@users.noreply.github.com> Co-authored-by: HaozheLiu-ST <53887227+HaozheLiu-ST@users.noreply.github.com> Co-authored-by: HeliosZhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: xieenze <20984575+xieenze@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> | 3 months ago | |
feat: add SANA-Video 2.0 four-step preview (#471) * feat: add SANA-Video 2.0 four-step preview * docs: add verified four-step preview * docs: align four-step preview controls Co-authored-by: Shuchen Xue <74777491+scxue@users.noreply.github.com> Co-authored-by: Yuyang Zhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: Haozhe Liu <53887227+HaozheLiu-ST@users.noreply.github.com> Co-authored-by: Yitong Li <135998170+lyttttt3333@users.noreply.github.com> Co-authored-by: Enze Xie <20984575+xieenze@users.noreply.github.com> Co-authored-by: Jincheng Yu <19924374+loveSy@users.noreply.github.com> * docs: announce four-step preview * docs: simplify four-step preview news --------- Co-authored-by: Shuchen Xue <74777491+scxue@users.noreply.github.com> Co-authored-by: Yuyang Zhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: Haozhe Liu <53887227+HaozheLiu-ST@users.noreply.github.com> Co-authored-by: Yitong Li <135998170+lyttttt3333@users.noreply.github.com> Co-authored-by: Enze Xie <20984575+xieenze@users.noreply.github.com> Co-authored-by: Jincheng Yu <19924374+loveSy@users.noreply.github.com> | 3 days ago | |
feat: open-source Sana-Video 2.0 (#439) * feat: add Sana-Video 2.0 release code * refactor: reuse shared Sana-Video workflows * docs: release SANA-Video 2.0 5B checkpoint * fix: validate SANA-Video 2.0 release inference * docs: update SANA-Video 2.0 demo seed * fix: align SANA-Video 2.0 linear attention * docs: link SANA-Video 2.0 live demo | 16 days ago | |
feat: add SANA-Video 2.0 four-step preview (#471) * feat: add SANA-Video 2.0 four-step preview * docs: add verified four-step preview * docs: align four-step preview controls Co-authored-by: Shuchen Xue <74777491+scxue@users.noreply.github.com> Co-authored-by: Yuyang Zhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: Haozhe Liu <53887227+HaozheLiu-ST@users.noreply.github.com> Co-authored-by: Yitong Li <135998170+lyttttt3333@users.noreply.github.com> Co-authored-by: Enze Xie <20984575+xieenze@users.noreply.github.com> Co-authored-by: Jincheng Yu <19924374+loveSy@users.noreply.github.com> * docs: announce four-step preview * docs: simplify four-step preview news --------- Co-authored-by: Shuchen Xue <74777491+scxue@users.noreply.github.com> Co-authored-by: Yuyang Zhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: Haozhe Liu <53887227+HaozheLiu-ST@users.noreply.github.com> Co-authored-by: Yitong Li <135998170+lyttttt3333@users.noreply.github.com> Co-authored-by: Enze Xie <20984575+xieenze@users.noreply.github.com> Co-authored-by: Jincheng Yu <19924374+loveSy@users.noreply.github.com> | 3 days ago | |
feat: add SANA-Video 2.0 four-step preview (#471) * feat: add SANA-Video 2.0 four-step preview * docs: add verified four-step preview * docs: align four-step preview controls Co-authored-by: Shuchen Xue <74777491+scxue@users.noreply.github.com> Co-authored-by: Yuyang Zhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: Haozhe Liu <53887227+HaozheLiu-ST@users.noreply.github.com> Co-authored-by: Yitong Li <135998170+lyttttt3333@users.noreply.github.com> Co-authored-by: Enze Xie <20984575+xieenze@users.noreply.github.com> Co-authored-by: Jincheng Yu <19924374+loveSy@users.noreply.github.com> * docs: announce four-step preview * docs: simplify four-step preview news --------- Co-authored-by: Shuchen Xue <74777491+scxue@users.noreply.github.com> Co-authored-by: Yuyang Zhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: Haozhe Liu <53887227+HaozheLiu-ST@users.noreply.github.com> Co-authored-by: Yitong Li <135998170+lyttttt3333@users.noreply.github.com> Co-authored-by: Enze Xie <20984575+xieenze@users.noreply.github.com> Co-authored-by: Jincheng Yu <19924374+loveSy@users.noreply.github.com> | 3 days ago | |
feat: add SANA-Video 2.0 four-step preview (#471) * feat: add SANA-Video 2.0 four-step preview * docs: add verified four-step preview * docs: align four-step preview controls Co-authored-by: Shuchen Xue <74777491+scxue@users.noreply.github.com> Co-authored-by: Yuyang Zhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: Haozhe Liu <53887227+HaozheLiu-ST@users.noreply.github.com> Co-authored-by: Yitong Li <135998170+lyttttt3333@users.noreply.github.com> Co-authored-by: Enze Xie <20984575+xieenze@users.noreply.github.com> Co-authored-by: Jincheng Yu <19924374+loveSy@users.noreply.github.com> * docs: announce four-step preview * docs: simplify four-step preview news --------- Co-authored-by: Shuchen Xue <74777491+scxue@users.noreply.github.com> Co-authored-by: Yuyang Zhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: Haozhe Liu <53887227+HaozheLiu-ST@users.noreply.github.com> Co-authored-by: Yitong Li <135998170+lyttttt3333@users.noreply.github.com> Co-authored-by: Enze Xie <20984575+xieenze@users.noreply.github.com> Co-authored-by: Jincheng Yu <19924374+loveSy@users.noreply.github.com> | 3 days ago | |
fix: inherit HF token in sana-run (#493) | 3 days ago | |
[codex review] fix all chinese comment to english | 2 months ago | |
feat: add SANA-Video 2.0 four-step preview (#471) * feat: add SANA-Video 2.0 four-step preview * docs: add verified four-step preview * docs: align four-step preview controls Co-authored-by: Shuchen Xue <74777491+scxue@users.noreply.github.com> Co-authored-by: Yuyang Zhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: Haozhe Liu <53887227+HaozheLiu-ST@users.noreply.github.com> Co-authored-by: Yitong Li <135998170+lyttttt3333@users.noreply.github.com> Co-authored-by: Enze Xie <20984575+xieenze@users.noreply.github.com> Co-authored-by: Jincheng Yu <19924374+loveSy@users.noreply.github.com> * docs: announce four-step preview * docs: simplify four-step preview news --------- Co-authored-by: Shuchen Xue <74777491+scxue@users.noreply.github.com> Co-authored-by: Yuyang Zhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: Haozhe Liu <53887227+HaozheLiu-ST@users.noreply.github.com> Co-authored-by: Yitong Li <135998170+lyttttt3333@users.noreply.github.com> Co-authored-by: Enze Xie <20984575+xieenze@users.noreply.github.com> Co-authored-by: Jincheng Yu <19924374+loveSy@users.noreply.github.com> | 3 days ago | |
fix(sana-wm): add public benchmark eval scripts (#410) | 2 months ago | |
docs: fix some typos (#383) * docs: fix some typos Signed-off-by: John E <jeis4wpi@outlook.com> * pre-commit; --------- Signed-off-by: John E <jeis4wpi@outlook.com> Co-authored-by: junsong <cjs1020440147@icloud.com> | 3 months ago | |
feat: open-source Sana-Video 2.0 (#439) * feat: add Sana-Video 2.0 release code * refactor: reuse shared Sana-Video workflows * docs: release SANA-Video 2.0 5B checkpoint * fix: validate SANA-Video 2.0 release inference * docs: update SANA-Video 2.0 demo seed * fix: align SANA-Video 2.0 linear attention * docs: link SANA-Video 2.0 live demo | 16 days ago | |
Release SANA-WM Streaming (#401) * feat(sana-wm): add chunk-causal streaming inference Streams 720p video chunk-by-chunk (distilled stage-1 + chunk-causal LTX-2 refiner + causal VAE) into a progressive MP4; weights auto-download from the SANA-WM_streaming HF repo. * style(sana-wm): apply pre-commit formatting to streaming files (black/isort/pyupgrade/autoflake/mdformat) * feat(sana-wm): CUDA chunkwise GDN cam scan (default on; SANA_GDN_CUDA=0 to disable) bf16 CUDA kernels for the cam-branch chunkwise GDN (Phase A + Phase C) in diffusion/model/ops/fused_gdn_chunkwise_cuda.py, with a Triton fp32-state Phase B; on by default for the bidirectional and streaming paths, transparent Triton fallback for unsupported shapes/dtype or when the extension can't build (no nvcc). Auto-detects GPU arch (sm_90/100/120) and the CUDA toolkit. Verified on H100: Phase A bit-exact, cam max_rel 3.9e-3, ~38x less cam transient VRAM (547->14MB). Correctness/VRAM tests in tests/sana_wm_gdn_cuda/. * fix(sana-wm): clean streaming inference path * chore(sana-wm): remove gdn cuda debug scripts * docs: update Sana-WM README links * docs: add Sana-WM demo links * docs: add Sana-WM co-authors Co-authored-by: Yuyang Zhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: Enze Xie <johnny_ez@163.com> Co-authored-by: Jincheng Yu <jinchengy@nvidia.com> Co-authored-by: Haozhe Liu <53887227+HaozheLiu-ST@users.noreply.github.com> Co-authored-by: Tian Ye <57895293+Owen718@users.noreply.github.com> * docs: label Sana-WM reactor demo * docs: add Sana-WM co-author trailers Co-authored-by: Yuyang Zhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: Enze Xie <20984575+xieenze@users.noreply.github.com> Co-authored-by: Jincheng Yu <19924374+loveSy@users.noreply.github.com> Co-authored-by: Haozhe Liu <53887227+HaozheLiu-ST@users.noreply.github.com> Co-authored-by: Tian Ye <57895293+Owen718@users.noreply.github.com> --------- Co-authored-by: junsong <cjs1020440147@icloud.com> Co-authored-by: Tian Ye <57895293+Owen718@users.noreply.github.com> Co-authored-by: Yuyang Zhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: Enze Xie <20984575+xieenze@users.noreply.github.com> Co-authored-by: Jincheng Yu <19924374+loveSy@users.noreply.github.com> Co-authored-by: Haozhe Liu <53887227+HaozheLiu-ST@users.noreply.github.com> | 3 months ago | |
fix minor docs bug; (#349) * fix minor docs bug; * fix minor bug; update readme; Co-authored-by: Enze Xie <johnny_ez@163.com> --------- Co-authored-by: Enze Xie <johnny_ez@163.com> | 8 months ago | |
[SANA-Video] Adding SANA-Video Training & Inference code & Dataloader and so on (#313) * 1. support sana-video training; 2. support wan model; 3. add dataloader for video training; 4. add all we need to train a SANA-Video; * update train scripts code; * 1. update train scripts code; 2. update wan code; * remove used code; * update config files * ci code update; train code update; * add inference code and run success; * remove unused code; * fix path error; * update several readme for sana-video * [CI-Lint] Fix code style issues with pre-commit 7bc87de3966810a7ff9741da009dfe22f1154cb4 * update readme; Co-authored-by: Enze Xie <johnny_ez@163.com> * update version * fix bug; * update env Co-authored-by: xieenze <Johnny_ez@163.com> * env update; * code update; * change env; * env update; * update env; * env update; * update toy data download script; * update code; * skip ci env test; * update env; * update; * fix * pre-commit; * fix; * pre-commit; * fix; * fix; * remove python setup; * fix; * pre-commit * fix bug; * fix bug; * fix bug; * fix bug; * fix bug; * remove flex attn; * fix train bug; * seperate data downloading in ci; * pre-commit * update; * pre-commit * update; * fix * fix no gpu import bug; fix test train video bug; * pre-commit * update env in each ci test; * fix; * code update; * update; * Update Sana_2000M_256px_AdamW_fsdp.yaml * Update Sana_2000M_480px_AdamW_fsdp.yaml * fix * ivjoint is not supported in ci; * fix; * Remove HunyuanVAE and duplicated siglip * Delete diffusion/model/dc_ae/efficientvit/models copy/__init__.py * Remove old ar sampler * Self-forcing and longlive will have a new scheduler * Add ChunkFlowEuler back, in case someone want to train AR * Remove siglip * Delete diffusion/scheduler/ar_flow_euler_sampler.py * remove peft load * remove duplicated wan config * [CI-Lint] Fix code style issues with pre-commit ec5160791010e0f820e900ba6ceeb73ad9d8529d * Remove chunk ar long video scheduler and self-forcing scheduler * Remove task=df * Update builder.py * Update config.py * [CI-Lint] Fix code style issues with pre-commit 0f3d3b6d935499792dae6267917c3f57536b7377 * fix config bug * [CI-Lint] Fix code style issues with pre-commit baa4215169f13bd549038fe0557b104f68014527 --------- Co-authored-by: GitHub Action <action@github.com> Co-authored-by: Enze Xie <johnny_ez@163.com> Co-authored-by: Yuyang Zhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: HeliosZhao <yuyangzhao@outlook.com> | 10 months ago | |
support dockerfile | 1 year ago | |
license update; Signed-off-by: lawrence-cj <cjs1020440147@icloud.com> | 1 year ago | |
feat: add SANA-Video 2.0 four-step preview (#471) * feat: add SANA-Video 2.0 four-step preview * docs: add verified four-step preview * docs: align four-step preview controls Co-authored-by: Shuchen Xue <74777491+scxue@users.noreply.github.com> Co-authored-by: Yuyang Zhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: Haozhe Liu <53887227+HaozheLiu-ST@users.noreply.github.com> Co-authored-by: Yitong Li <135998170+lyttttt3333@users.noreply.github.com> Co-authored-by: Enze Xie <20984575+xieenze@users.noreply.github.com> Co-authored-by: Jincheng Yu <19924374+loveSy@users.noreply.github.com> * docs: announce four-step preview * docs: simplify four-step preview news --------- Co-authored-by: Shuchen Xue <74777491+scxue@users.noreply.github.com> Co-authored-by: Yuyang Zhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: Haozhe Liu <53887227+HaozheLiu-ST@users.noreply.github.com> Co-authored-by: Yitong Li <135998170+lyttttt3333@users.noreply.github.com> Co-authored-by: Enze Xie <20984575+xieenze@users.noreply.github.com> Co-authored-by: Jincheng Yu <19924374+loveSy@users.noreply.github.com> | 3 days ago | |
Support stage-1 CP training, SANA-WM Benchmark & fp8/fp4 quantized streaming inference (#404) * feat(sana-wm): support stage-1 CP training * docs(sana-wm): expose chunk-causal training entry * style: run pre-commit on sana-wm training changes * docs: add SANA-WM benchmark guide * SANA-WM streaming: per-component fp8/fp4 quantized inference Add --stage1_precision/--refiner_precision (bf16/fp8/fp4) with per-block fp8_autocast scoping that preserves action-following, fp8 recipe selector for stage-1 + refiner, default Transformer Engine install, and benchmark docs. * Polish Sana-WM PR docs and formatting * Use end-to-end Sana-WM inference CI * Add SANA-WM stage1 training CI * Run SANA-WM stage1 smoke in video CI * Keep SANA-WM training in dedicated CI job --------- Co-authored-by: junsong <cjs1020440147@icloud.com> | 2 months ago | |
feat: open-source Sana-Video 2.0 (#439) * feat: add Sana-Video 2.0 release code * refactor: reuse shared Sana-Video workflows * docs: release SANA-Video 2.0 5B checkpoint * fix: validate SANA-Video 2.0 release inference * docs: update SANA-Video 2.0 demo seed * fix: align SANA-Video 2.0 linear attention * docs: link SANA-Video 2.0 live demo | 16 days ago | |
Release SANA-WM Streaming (#401) * feat(sana-wm): add chunk-causal streaming inference Streams 720p video chunk-by-chunk (distilled stage-1 + chunk-causal LTX-2 refiner + causal VAE) into a progressive MP4; weights auto-download from the SANA-WM_streaming HF repo. * style(sana-wm): apply pre-commit formatting to streaming files (black/isort/pyupgrade/autoflake/mdformat) * feat(sana-wm): CUDA chunkwise GDN cam scan (default on; SANA_GDN_CUDA=0 to disable) bf16 CUDA kernels for the cam-branch chunkwise GDN (Phase A + Phase C) in diffusion/model/ops/fused_gdn_chunkwise_cuda.py, with a Triton fp32-state Phase B; on by default for the bidirectional and streaming paths, transparent Triton fallback for unsupported shapes/dtype or when the extension can't build (no nvcc). Auto-detects GPU arch (sm_90/100/120) and the CUDA toolkit. Verified on H100: Phase A bit-exact, cam max_rel 3.9e-3, ~38x less cam transient VRAM (547->14MB). Correctness/VRAM tests in tests/sana_wm_gdn_cuda/. * fix(sana-wm): clean streaming inference path * chore(sana-wm): remove gdn cuda debug scripts * docs: update Sana-WM README links * docs: add Sana-WM demo links * docs: add Sana-WM co-authors Co-authored-by: Yuyang Zhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: Enze Xie <johnny_ez@163.com> Co-authored-by: Jincheng Yu <jinchengy@nvidia.com> Co-authored-by: Haozhe Liu <53887227+HaozheLiu-ST@users.noreply.github.com> Co-authored-by: Tian Ye <57895293+Owen718@users.noreply.github.com> * docs: label Sana-WM reactor demo * docs: add Sana-WM co-author trailers Co-authored-by: Yuyang Zhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: Enze Xie <20984575+xieenze@users.noreply.github.com> Co-authored-by: Jincheng Yu <19924374+loveSy@users.noreply.github.com> Co-authored-by: Haozhe Liu <53887227+HaozheLiu-ST@users.noreply.github.com> Co-authored-by: Tian Ye <57895293+Owen718@users.noreply.github.com> --------- Co-authored-by: junsong <cjs1020440147@icloud.com> Co-authored-by: Tian Ye <57895293+Owen718@users.noreply.github.com> Co-authored-by: Yuyang Zhao <43061147+HeliosZhao@users.noreply.github.com> Co-authored-by: Enze Xie <20984575+xieenze@users.noreply.github.com> Co-authored-by: Jincheng Yu <19924374+loveSy@users.noreply.github.com> Co-authored-by: Haozhe Liu <53887227+HaozheLiu-ST@users.noreply.github.com> | 3 months ago |
Project README is empty