| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
feat(archon): add FP8 blockwise training support (#1087) Enable blockwise 128x128 FP8 e4m3fn matmuls for Archon engine via torchao, with BF16 master weights and on-the-fly quantization. Key changes: - Add FP8 linear patching (enable_fp8_linear) and expert patching (enable_fp8_experts) with deepcopy-safe types.MethodType binding - Add FP8 checkpoint detection, preparation, and dequantization with CPU fallback and Shard(0) DTensor support - Add ArchonFP8Config with mode/exclude_modules/include_experts/use_triton - Add post-parallelism shard alignment validation for TP safety - Disable torch.compile when FP8 is active (incompatible with 2D scales) - Comprehensive test suite: forward/backward correctness, checkpoint detect/prepare/dequant, MoE dispatch, distributed sharded dequant | 6 个月前 | |
feat(archon): add FP8 blockwise training support (#1087) Enable blockwise 128x128 FP8 e4m3fn matmuls for Archon engine via torchao, with BF16 master weights and on-the-fly quantization. Key changes: - Add FP8 linear patching (enable_fp8_linear) and expert patching (enable_fp8_experts) with deepcopy-safe types.MethodType binding - Add FP8 checkpoint detection, preparation, and dequantization with CPU fallback and Shard(0) DTensor support - Add ArchonFP8Config with mode/exclude_modules/include_experts/use_triton - Add post-parallelism shard alignment validation for TP safety - Disable torch.compile when FP8 is active (incompatible with 2D scales) - Comprehensive test suite: forward/backward correctness, checkpoint detect/prepare/dequant, MoE dispatch, distributed sharded dequant | 6 个月前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 7 个月前 | |
feat(archon): add FP8 blockwise training support (#1087) Enable blockwise 128x128 FP8 e4m3fn matmuls for Archon engine via torchao, with BF16 master weights and on-the-fly quantization. Key changes: - Add FP8 linear patching (enable_fp8_linear) and expert patching (enable_fp8_experts) with deepcopy-safe types.MethodType binding - Add FP8 checkpoint detection, preparation, and dequantization with CPU fallback and Shard(0) DTensor support - Add ArchonFP8Config with mode/exclude_modules/include_experts/use_triton - Add post-parallelism shard alignment validation for TP safety - Disable torch.compile when FP8 is active (incompatible with 2D scales) - Comprehensive test suite: forward/backward correctness, checkpoint detect/prepare/dequant, MoE dispatch, distributed sharded dequant | 6 个月前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 7 个月前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 7 个月前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 7 个月前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 7 个月前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 7 个月前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 7 个月前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 7 个月前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 7 个月前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 7 个月前 | |
feat(api): add unified RejectionSamplingConfig for async training (#1088) Replace behave_imp_weight_cap/behave_imp_weight_mode with unified RejectionSamplingConfig supporting multiple metrics (ratio, kl_k1, kl_k2, kl_k3), levels (token/sequence), and actions (mask/clamp). Key changes: - Add RejectionSamplingConfig dataclass with comprehensive validation - Implement apply_rejection_sampling for 1D packed and 2D padded formats - Fix loss denominator scaling bug in mask mode (save count before filtering) - Use geometric mean for sequence-level ratio aggregation (matching GSPO) - Broadcast sequence-level geometric mean as uniform behave_imp_weight - Warn when use_decoupled_loss=True but rejection_sampling is None - Update ppo_actor_loss_fn and grpo_loss_fn to use new config - Migrate 40 example configs to new rejection_sampling field - Add 43 unit tests covering all modes, metrics, and edge cases Refs: #1052 | 5 个月前 | |
feat(archon): add Qwen3.5 dense and MoE model support (DP-only) (#1012) Add Archon engine support for the Qwen3.5 model family, a hybrid architecture mixing full_attention (GatedAttention with sigmoid output gate) and linear_attention (GatedDeltaNet with chunk-based delta rule) layers. Both dense and MoE variants are supported. Currently supports FSDP2 data parallelism. TP/CP/EP/PP are not yet implemented for the hybrid attention layers. Key changes: - Qwen3_5Model with GatedDeltaNet + GatedAttention hybrid layers - Two RMSNorm variants: (1+w)*norm(x) and w*norm(x)*silu(gate) - Partial RoPE (25% of head_dim) via rope_parameters config - HF<->Archon state dict adapter for dense and MoE weights - MoE shared_expert_gate: sigmoid(Linear(dim,1)) gating on shared expert output, added to shared MoE infrastructure - FSDP2 parallelization with activation checkpointing + compile - Comprehensive test suite: 29 unit tests, 4 E2E HF parity tests - Refactored existing parity tests to use partial_config pattern | 7 个月前 | |
feat(archon): add Qwen3.5 dense and MoE model support (DP-only) (#1012) Add Archon engine support for the Qwen3.5 model family, a hybrid architecture mixing full_attention (GatedAttention with sigmoid output gate) and linear_attention (GatedDeltaNet with chunk-based delta rule) layers. Both dense and MoE variants are supported. Currently supports FSDP2 data parallelism. TP/CP/EP/PP are not yet implemented for the hybrid attention layers. Key changes: - Qwen3_5Model with GatedDeltaNet + GatedAttention hybrid layers - Two RMSNorm variants: (1+w)*norm(x) and w*norm(x)*silu(gate) - Partial RoPE (25% of head_dim) via rope_parameters config - HF<->Archon state dict adapter for dense and MoE weights - MoE shared_expert_gate: sigmoid(Linear(dim,1)) gating on shared expert output, added to shared MoE infrastructure - FSDP2 parallelization with activation checkpointing + compile - Comprehensive test suite: 29 unit tests, 4 E2E HF parity tests - Refactored existing parity tests to use partial_config pattern | 7 个月前 | |
feat(archon): add Qwen3.5 dense and MoE model support (DP-only) (#1012) Add Archon engine support for the Qwen3.5 model family, a hybrid architecture mixing full_attention (GatedAttention with sigmoid output gate) and linear_attention (GatedDeltaNet with chunk-based delta rule) layers. Both dense and MoE variants are supported. Currently supports FSDP2 data parallelism. TP/CP/EP/PP are not yet implemented for the hybrid attention layers. Key changes: - Qwen3_5Model with GatedDeltaNet + GatedAttention hybrid layers - Two RMSNorm variants: (1+w)*norm(x) and w*norm(x)*silu(gate) - Partial RoPE (25% of head_dim) via rope_parameters config - HF<->Archon state dict adapter for dense and MoE weights - MoE shared_expert_gate: sigmoid(Linear(dim,1)) gating on shared expert output, added to shared MoE infrastructure - FSDP2 parallelization with activation checkpointing + compile - Comprehensive test suite: 29 unit tests, 4 E2E HF parity tests - Refactored existing parity tests to use partial_config pattern | 7 个月前 | |
feat(archon): add Qwen3.5 dense and MoE model support (DP-only) (#1012) Add Archon engine support for the Qwen3.5 model family, a hybrid architecture mixing full_attention (GatedAttention with sigmoid output gate) and linear_attention (GatedDeltaNet with chunk-based delta rule) layers. Both dense and MoE variants are supported. Currently supports FSDP2 data parallelism. TP/CP/EP/PP are not yet implemented for the hybrid attention layers. Key changes: - Qwen3_5Model with GatedDeltaNet + GatedAttention hybrid layers - Two RMSNorm variants: (1+w)*norm(x) and w*norm(x)*silu(gate) - Partial RoPE (25% of head_dim) via rope_parameters config - HF<->Archon state dict adapter for dense and MoE weights - MoE shared_expert_gate: sigmoid(Linear(dim,1)) gating on shared expert output, added to shared MoE infrastructure - FSDP2 parallelization with activation checkpointing + compile - Comprehensive test suite: 29 unit tests, 4 E2E HF parity tests - Refactored existing parity tests to use partial_config pattern | 7 个月前 | |
feat(archon): add Qwen3.5 dense and MoE model support (DP-only) (#1012) Add Archon engine support for the Qwen3.5 model family, a hybrid architecture mixing full_attention (GatedAttention with sigmoid output gate) and linear_attention (GatedDeltaNet with chunk-based delta rule) layers. Both dense and MoE variants are supported. Currently supports FSDP2 data parallelism. TP/CP/EP/PP are not yet implemented for the hybrid attention layers. Key changes: - Qwen3_5Model with GatedDeltaNet + GatedAttention hybrid layers - Two RMSNorm variants: (1+w)*norm(x) and w*norm(x)*silu(gate) - Partial RoPE (25% of head_dim) via rope_parameters config - HF<->Archon state dict adapter for dense and MoE weights - MoE shared_expert_gate: sigmoid(Linear(dim,1)) gating on shared expert output, added to shared MoE infrastructure - FSDP2 parallelization with activation checkpointing + compile - Comprehensive test suite: 29 unit tests, 4 E2E HF parity tests - Refactored existing parity tests to use partial_config pattern | 7 个月前 | |
fix(archon): Wrap router gate in nn.Module for DTensor hook compatibility (#1029) * fix(archon): wrap router gate in nn.Module for DTensor hook compatibility Replace direct nn.Linear gate with RouterGateLinear(nn.Linear) so that ReplicateParallel DTensor hooks fire on module.__call__() rather than bypassing them via a standalone function call. Key changes: - Add RouterGateLinear(nn.Linear) class that delegates forward() to router_gating_linear(), preserving memory-efficient FP32 GEMM - Replace nn.Linear gate in TokenChoiceTopKRouter with RouterGateLinear - Simplify forward() to single self.gate(x) call (no dtype branching) - Add 4 unit tests in TestRouterGateLinear covering state dict compat, dtype handling, and backward pass memory optimization * fix(ci): enlarge sft example CI test time --------- Co-authored-by: 博惟 <bowei.fw@antgroup.com> | 7 个月前 | |
feat(archon): add Qwen3.5 dense and MoE model support (DP-only) (#1012) Add Archon engine support for the Qwen3.5 model family, a hybrid architecture mixing full_attention (GatedAttention with sigmoid output gate) and linear_attention (GatedDeltaNet with chunk-based delta rule) layers. Both dense and MoE variants are supported. Currently supports FSDP2 data parallelism. TP/CP/EP/PP are not yet implemented for the hybrid attention layers. Key changes: - Qwen3_5Model with GatedDeltaNet + GatedAttention hybrid layers - Two RMSNorm variants: (1+w)*norm(x) and w*norm(x)*silu(gate) - Partial RoPE (25% of head_dim) via rope_parameters config - HF<->Archon state dict adapter for dense and MoE weights - MoE shared_expert_gate: sigmoid(Linear(dim,1)) gating on shared expert output, added to shared MoE infrastructure - FSDP2 parallelization with activation checkpointing + compile - Comprehensive test suite: 29 unit tests, 4 E2E HF parity tests - Refactored existing parity tests to use partial_config pattern | 7 个月前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 7 个月前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 7 个月前 | |
chore(deps): upgrade runtime dependencies and CI workflow (#1206) * chore(deps): upgrade runtime dependencies and CI workflow Upgrade megatron-core, sglang, vllm (0.19.1), transformers, and related packages. Pin deepep/deepgemm commits and lock trackio version for reproducibility. Key changes: - Upgrade sglang, vllm, transformers, megatron-core versions - Pin vllm to 0.19.1, fix compatibility across backends - Optimize Dockerfile and slim Docker image - Add uv_sync.sh install script, fix uv install on Linux - Fix Archon Qwen3.5 precision and port range issues - Remove integration tests from PR CI (moved to nightly) - Sync GRPO integration test config Refs: upgrade-deps branch * chore: remove duplicated test | 5 个月前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 7 个月前 | |
refactor(api): migrate allocation_mode to per-engine backend fields (#1044) * refactor(api): migrate allocation_mode to per-engine backend fields Replace the centralized `allocation_mode` string with explicit `backend` fields on `TrainEngineConfig` and `InferenceEngineConfig`. Each engine now owns its own backend+parallelism spec (e.g. `fsdp:d4`, `sglang:d4t2`), eliminating implicit auto-backend selection and the shared `AllocationMode` object. Key changes: - Add `backend` field to TrainEngineConfig and InferenceEngineConfig - Add `ModelAllocation.from_str()` for single-component parsing - Remove `AllocationMode` public export (replaced by `ModelAllocation`) - Rename internal `AllocationMode` to `_AllocationMode` for SPMD launcher backward compatibility with FutureWarning - Remove auto-backend selection — explicit backend prefix is now required - Controllers (`TrainController`, `RolloutController`) parse `backend` directly instead of receiving `alloc_mode` from trainers - `WeightUpdateMeta.alloc_mode` replaced by `gen_allocation` (single `ModelAllocation`) - Add `RWTrainer` and `ArchonRWEngine` for reward model training - Remove `get_model_update_meta()` helper (logic moved to trainers) - Update all YAML configs, examples, docs (EN+ZH), and tests BREAKING CHANGE: `AllocationMode` is removed from public API. Users must migrate to per-engine `backend` fields. SPMD launchers emit deprecation warnings. * chore(ci): fix backend specifier for vlm sft test * fix: fix bare dims for actor backends * chore(docs): fix reminder for bare allocation dims | 6 个月前 | |
feat(archon): add Qwen3.5 dense and MoE model support (DP-only) (#1012) Add Archon engine support for the Qwen3.5 model family, a hybrid architecture mixing full_attention (GatedAttention with sigmoid output gate) and linear_attention (GatedDeltaNet with chunk-based delta rule) layers. Both dense and MoE variants are supported. Currently supports FSDP2 data parallelism. TP/CP/EP/PP are not yet implemented for the hybrid attention layers. Key changes: - Qwen3_5Model with GatedDeltaNet + GatedAttention hybrid layers - Two RMSNorm variants: (1+w)*norm(x) and w*norm(x)*silu(gate) - Partial RoPE (25% of head_dim) via rope_parameters config - HF<->Archon state dict adapter for dense and MoE weights - MoE shared_expert_gate: sigmoid(Linear(dim,1)) gating on shared expert output, added to shared MoE infrastructure - FSDP2 parallelization with activation checkpointing + compile - Comprehensive test suite: 29 unit tests, 4 E2E HF parity tests - Refactored existing parity tests to use partial_config pattern | 7 个月前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 7 个月前 | |
refactor(api): migrate allocation_mode to per-engine backend fields (#1044) * refactor(api): migrate allocation_mode to per-engine backend fields Replace the centralized `allocation_mode` string with explicit `backend` fields on `TrainEngineConfig` and `InferenceEngineConfig`. Each engine now owns its own backend+parallelism spec (e.g. `fsdp:d4`, `sglang:d4t2`), eliminating implicit auto-backend selection and the shared `AllocationMode` object. Key changes: - Add `backend` field to TrainEngineConfig and InferenceEngineConfig - Add `ModelAllocation.from_str()` for single-component parsing - Remove `AllocationMode` public export (replaced by `ModelAllocation`) - Rename internal `AllocationMode` to `_AllocationMode` for SPMD launcher backward compatibility with FutureWarning - Remove auto-backend selection — explicit backend prefix is now required - Controllers (`TrainController`, `RolloutController`) parse `backend` directly instead of receiving `alloc_mode` from trainers - `WeightUpdateMeta.alloc_mode` replaced by `gen_allocation` (single `ModelAllocation`) - Add `RWTrainer` and `ArchonRWEngine` for reward model training - Remove `get_model_update_meta()` helper (logic moved to trainers) - Update all YAML configs, examples, docs (EN+ZH), and tests BREAKING CHANGE: `AllocationMode` is removed from public API. Users must migrate to per-engine `backend` fields. SPMD launchers emit deprecation warnings. * chore(ci): fix backend specifier for vlm sft test * fix: fix bare dims for actor backends * chore(docs): fix reminder for bare allocation dims | 6 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 6 个月前 | ||
| 6 个月前 | ||
| 7 个月前 | ||
| 6 个月前 | ||
| 7 个月前 | ||
| 7 个月前 | ||
| 7 个月前 | ||
| 7 个月前 | ||
| 7 个月前 | ||
| 7 个月前 | ||
| 7 个月前 | ||
| 7 个月前 | ||
| 7 个月前 | ||
| 5 个月前 | ||
| 7 个月前 | ||
| 7 个月前 | ||
| 7 个月前 | ||
| 7 个月前 | ||
| 7 个月前 | ||
| 7 个月前 | ||
| 7 个月前 | ||
| 7 个月前 | ||
| 7 个月前 | ||
| 5 个月前 | ||
| 7 个月前 | ||
| 6 个月前 | ||
| 7 个月前 | ||
| 7 个月前 | ||
| 6 个月前 |