| Files | Last commit | Last update |
|---|---|---|
Merge upstream/master into trainer_dev Bring master's content into trainer_dev, resolving conflicts between master's platform-abstraction removal and resolver API naming (normalize_value / annotation / ConfigResolutionError) and trainer_dev's omni data lifecycle, runtime input adapter, DSA attention support, and model integration diagnostics. | 1 month ago | |
Merge pull request #932 from 1372298/bugfix_cp_master fix(cp): avoid expanding GQA K/V before all-gather | 1 month ago | |
feat(expert_parallel): overlapping dispatch, capacity limit and chunked exchange Port the expert-parallel work the Kimi-K2.6 reproduction depends on: the overlapping dispatch/combine split (ep_routed_dispatch + ep_routed_experts_and_combine, exposed through build_ep_compute(overlap_fn=...)), the capacity limit, chunked/fused dispatch, the balanced router and the static-split planner -- together with the unit tests that pin all of it (tests/ut/core/expert_parallel, 179 cases; 263 with the dual-mode EP suite). The capacity limit drops the slots of an over-capacity expert by routing weight (HP_EP_CAPACITY_ORDER=score, the new default) instead of by position in the expert-major order. The number kept per expert -- the invariant the memory bound rests on -- is identical in both policies, so the archived drop-rate / peak / step-time sweeps stay valid while "which slots die" becomes a defined policy (the cheapest contributions). HP_EP_CAPACITY_ORDER=first reproduces the archived behaviour bit-for-bit, and the per-step log line now names the policy it used. Both keys share ONE fp32 sort key (expert index + inverted weight), which keeps the AICore float sort and adds no host synchronisation. Measured on 2SN (GBS 256, ep 128 / dp_shard 128 / edp 2, real router, two rounds each): the score policy costs +0.015 s per step against the position policy (~0.07%; drop rate 70.23% in both, peak 40.09 vs 40.28 GB), so the policy switch is free at this shape. Enabling the capacity limit at all costs +0.51 s per step (~2.5%): one AICore argsort plus the mask bookkeeping per MoE layer, paid whenever HP_EP_CAPACITY_FACTOR is set, whatever the policy. Measured on 2SN (GBS 256, seq 8192, ep 128 / dp_shard 128 / edp 2): 43.7672 GB with the overlap off -> 41.4853 GB with it on. | 1 month ago | |
fix: clean up tp/cp wrapper code warnings | 1 month ago | |
remove_activation_memory_warn | 1 month ago | |
Merge upstream/master into trainer_dev Bring master's content into trainer_dev, resolving conflicts between master's platform-abstraction removal and resolver API naming (normalize_value / annotation / ConfigResolutionError) and trainer_dev's omni data lifecycle, runtime input adapter, DSA attention support, and model integration diagnostics. | 1 month ago | |
refactor!: reorganize auto_models into top-level models/components/distributed/trainer/data Squashed 58 commits of the AutoModels directory restructure (M0-M5, S4a-S8d, top-level split): - characterization tests first (M0): UT collection baseline, public API contract snapshots, build-order golden cases, shared fixtures, training YAML snapshots - distributed split (S4a-S4g): sharding_config/planner/applier/injection split into distributed/{plan,recipe_spec,apply,compile, activation_checkpoint} + private _builder/; CP/EP into context_parallel/ and expert_parallel/; fsdp2 into fsdp_adapter + source_shard - models adapter layer (M1-M5): adapter_spec/registry skeleton, Qwen3-MoE replacements and CP/EP contracts migrated into models/qwen3_moe/adapter, single recipe converged onto models/qwen3_moe/recipes/train.yaml, legacy fusion files deleted; per-family sharding_rules declared via ModelAdapterSpec with directory-convention auto-discovery registration - build pipeline (S5a-S5g): checkpoint finalization into checkpoint_loader, plan DTO boundary normalized, god module split into model_builder, registry split into config_resolver + models/registry - data stack (S6a-S6f): indexed/text/online/batching/parallel/vlm/tools migrated to top-level hyper_parallel/data - trainer (S7a-S7e): config tree/callbacks/runtime/base+text+vlm migrated to top-level hyper_parallel/trainer, utils split eliminated - closeout (S8a-S8d): legacy core/shard API, legacy hyper_parallel/models and components compatibility package deleted; import boundary gates - top-level split: auto_models/ dissolved into models/ (adapter layer + api/build_options/replacement/_transformers, external interface), components/ (modules/functional/quantization/losses/optim/checkpoint) and distributed/; references rewritten across py/yaml/sh and setup.py; boundary gates updated (distributed -> models limited to the build_options leaf DTO plus the planner's function-level registry lookup) Verified: full UT 4562 passed / 39 failed (identical to the pre-refactor environment-failure baseline), characterization suites green. | 2 months ago | |
remove_activation_memory_warn | 1 month ago | |
fix: clean master static analysis warnings | 1 month ago | |
fix: clean Meng107 static analysis warnings | 1 month ago | |
refactor!: reorganize auto_models into top-level models/components/distributed/trainer/data Squashed 58 commits of the AutoModels directory restructure (M0-M5, S4a-S8d, top-level split): - characterization tests first (M0): UT collection baseline, public API contract snapshots, build-order golden cases, shared fixtures, training YAML snapshots - distributed split (S4a-S4g): sharding_config/planner/applier/injection split into distributed/{plan,recipe_spec,apply,compile, activation_checkpoint} + private _builder/; CP/EP into context_parallel/ and expert_parallel/; fsdp2 into fsdp_adapter + source_shard - models adapter layer (M1-M5): adapter_spec/registry skeleton, Qwen3-MoE replacements and CP/EP contracts migrated into models/qwen3_moe/adapter, single recipe converged onto models/qwen3_moe/recipes/train.yaml, legacy fusion files deleted; per-family sharding_rules declared via ModelAdapterSpec with directory-convention auto-discovery registration - build pipeline (S5a-S5g): checkpoint finalization into checkpoint_loader, plan DTO boundary normalized, god module split into model_builder, registry split into config_resolver + models/registry - data stack (S6a-S6f): indexed/text/online/batching/parallel/vlm/tools migrated to top-level hyper_parallel/data - trainer (S7a-S7e): config tree/callbacks/runtime/base+text+vlm migrated to top-level hyper_parallel/trainer, utils split eliminated - closeout (S8a-S8d): legacy core/shard API, legacy hyper_parallel/models and components compatibility package deleted; import boundary gates - top-level split: auto_models/ dissolved into models/ (adapter layer + api/build_options/replacement/_transformers, external interface), components/ (modules/functional/quantization/losses/optim/checkpoint) and distributed/; references rewritten across py/yaml/sh and setup.py; boundary gates updated (distributed -> models limited to the build_options leaf DTO plus the planner's function-level registry lookup) Verified: full UT 4562 passed / 39 failed (identical to the pre-refactor environment-failure baseline), characterization suites green. | 2 months ago | |
feat: support DeepSeek v4.1 model with DSA attention and VLM training pipeline | 1 month ago |
| Files | Last commit | Last update |
|---|---|---|
| 1 month ago | ||
| 1 month ago | ||
| 1 month ago | ||
| 1 month ago | ||
| 1 month ago | ||
| 1 month ago | ||
| 2 months ago | ||
| 1 month ago | ||
| 1 month ago | ||
| 1 month ago | ||
| 2 months ago | ||
| 1 month ago |