| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 6 个月前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 6 个月前 | |
refactor(api): migrate allocation_mode to per-engine backend fields (#1044) * refactor(api): migrate allocation_mode to per-engine backend fields Replace the centralized `allocation_mode` string with explicit `backend` fields on `TrainEngineConfig` and `InferenceEngineConfig`. Each engine now owns its own backend+parallelism spec (e.g. `fsdp:d4`, `sglang:d4t2`), eliminating implicit auto-backend selection and the shared `AllocationMode` object. Key changes: - Add `backend` field to TrainEngineConfig and InferenceEngineConfig - Add `ModelAllocation.from_str()` for single-component parsing - Remove `AllocationMode` public export (replaced by `ModelAllocation`) - Rename internal `AllocationMode` to `_AllocationMode` for SPMD launcher backward compatibility with FutureWarning - Remove auto-backend selection — explicit backend prefix is now required - Controllers (`TrainController`, `RolloutController`) parse `backend` directly instead of receiving `alloc_mode` from trainers - `WeightUpdateMeta.alloc_mode` replaced by `gen_allocation` (single `ModelAllocation`) - Add `RWTrainer` and `ArchonRWEngine` for reward model training - Remove `get_model_update_meta()` helper (logic moved to trainers) - Update all YAML configs, examples, docs (EN+ZH), and tests BREAKING CHANGE: `AllocationMode` is removed from public API. Users must migrate to per-engine `backend` fields. SPMD launchers emit deprecation warnings. * chore(ci): fix backend specifier for vlm sft test * fix: fix bare dims for actor backends * chore(docs): fix reminder for bare allocation dims | 5 个月前 | |
feat: add MTP training with speculative decoding rollout (#1659) * feat: add MTP training with NEXTN speculative decoding rollout Train the built-in MTP head jointly with GRPO/SFT on Megatron and use it during rollout through SGLang NEXTN speculative decoding. Online weight synchronization updates both the target model and the built-in MTP draft runner, ensuring that speculative decoding uses the latest RL-trained MTP weights instead of a stale initialization. MTP is trained through Megatron-Core's auxiliary cross-entropy path with a configurable loss scaling factor of `0.1` by default. AReaL supplies independent MTP label and loss-mask channels while keeping the main forward path logits-based. Shared output weights, backbone hidden states, and embedding inputs are detached from the MTP loss graph, so the backbone receives only the policy/SFT gradient while the MTP-specific parameters learn from future-token supervision. For packed THD training with context parallelism, AReaL applies the same per-sequence zigzag CP split and rank-local repacking to MTP labels and loss masks as it does to input IDs. Megatron-Core's CP-aware rolling then aligns future-token targets across CP ranks without crossing packed sequence boundaries. For online rollout, a focused compatibility bridge for `sglang==0.5.10.post1` receives each distributed weight bucket once and applies it to both the built-in MTP draft runner and the target runner. It supports both SGLang speculative worker layouts, handles SGLang's internal `NEXTN`-to-`EAGLE` normalization, and leaves external EAGLE draft models untouched. Draft-weight CPU backup can be enabled so the server remains available while updated weights arrive online. End-to-end validation was performed with Qwen3.5-2B on Geometry3K GRPO. The training-side MTP weights changed during optimization, and the same updated tensors were loaded into all SGLang draft runners. Key changes: - Add `enable_mtp_training` and `mtp_loss_scaling_factor` to the Megatron engine configuration. MTP training implies retaining the model's MTP layers and is incompatible with `lm_head_loss_chunk_size`. - Feed independent MTP labels and loss masks through Megatron forward passes while preserving the main logits-based loss path. - Patch Megatron-Core `GPTModel` and Megatron-Bridge `Qwen3VLGPTModel` forwarding so Qwen3.5 text and multimodal batches can train MTP. - Align MTP labels and masks with padded or packed execution layouts and prevent targets from crossing sequence or padding boundaries. - Support MTP training with `CP > 1` for wrapper-owned packed THD by applying the same per-sequence zigzag split and rank-local repacking to input IDs, MTP labels, and MTP loss masks. - Reuse Megatron-Core's packed, CP-aware rolling semantics to align future-token supervision across CP ranks. - Isolate MTP gradients from shared output weights, embeddings, and backbone hidden states. - Report the auxiliary `mtp_loss` in training statistics. - Add SGLang speculative-decoding configuration passthrough for `NEXTN`, speculative steps, EAGLE top-k, draft-token count, external draft-model path, and draft-weight CPU backup. - Add an SGLang distributed weight-update bridge that updates both target and built-in MTP draft runners from the same received tensors. - Support both SGLang Spec v1 and Spec v2 draft-runner layouts, while failing fast for unsupported SGLang versions, missing draft runners, unsupported load formats, and inference pipeline parallelism. - Record `rollout/spec_accept_rate` and `rollout/spec_accept_length` from SGLang response metadata. - Add a Qwen3.5-2B Geometry3K GRPO example with MTP training and NEXTN rollout enabled. - Add unit coverage for padded and packed MTP label/mask layouts, CP zigzag alignment, multimodal forwarding, NEXTN/EAGLE routing, Spec v1/v2 compatibility, and draft/target online weight updates. - Regenerate the English and Chinese CLI reference documentation. Current limitations: - MTP training with `CP > 1` is supported only for wrapper-owned packed THD. Padded BSHD, VLM, and model-owned THD execution still require `CP=1`. - The current gradient-isolation implementation supports a single MTP prediction layer. Multi-layer MTP gradient propagation is not supported. - The SGLang distributed MTP update bridge requires inference pipeline parallel size `1`. - The compatibility bridge is intentionally pinned to `sglang==0.5.10.post1` because it relies on version-specific internal weight-update and draft-runner APIs. * test: use HttpGenerationResult in rollout version race test * fix(engine): align MTP masks and detach untied output weights Keep next-token-aligned masks unchanged before MCore's per-layer roll, and detach internal output-layer weights for untied models. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(engine): unify Megatron main and auxiliary loss scaling Let MCore apply the optimizer loss scale to both the main backward path and separately seeded MTP/MoE auxiliary gradients. This prevents FP16 optimizer unscaling from suppressing auxiliary updates. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(models): reject unsupported multilayer MTP training Fail before model construction when MTP training requests more than one prediction layer, whose gradients are not fully supported yet. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: huaqingyuan <huaqingyuan@tencent.com> Co-authored-by: Cursor <cursoragent@cursor.com> | 1 天前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 6 个月前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 6 个月前 | |
refactor: flatten sub-module imports to use parent package re-exports (#996) Add __init__.py with lazy re-exports (__getattr__ + __all__) for areal/api, areal/engine, areal/reward, areal/workflow, and __all__ for areal/dataset, then rewrite all external imports across the codebase to use the shorter parent-package form (e.g. `from areal.api import TrainEngine`). Key changes: - All re-exports are fully lazy via __getattr__ (no eager imports) - Flatten ~100 files across areal/, tests/, examples/ - Preserve cli_args deep imports (dozens of config classes, including SchedulingSpec which stays in cli_args) - Preserve intra-package relative imports to avoid circular deps - Reward submodules use sibling-relative imports (from . import ...) - Preserve non-exported symbols (DeviceRuntimeInfo, HttpRequest, etc.) Co-authored-by: Wentai Zhang <zhangwentai.zwt@antgroup.com> | 5 个月前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 6 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 6 个月前 | ||
| 6 个月前 | ||
| 5 个月前 | ||
| 1 天前 | ||
| 6 个月前 | ||
| 6 个月前 | ||
| 5 个月前 | ||
| 6 个月前 |