| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 6 个月前 | |
refactor(api): migrate allocation_mode to per-engine backend fields (#1044) * refactor(api): migrate allocation_mode to per-engine backend fields Replace the centralized `allocation_mode` string with explicit `backend` fields on `TrainEngineConfig` and `InferenceEngineConfig`. Each engine now owns its own backend+parallelism spec (e.g. `fsdp:d4`, `sglang:d4t2`), eliminating implicit auto-backend selection and the shared `AllocationMode` object. Key changes: - Add `backend` field to TrainEngineConfig and InferenceEngineConfig - Add `ModelAllocation.from_str()` for single-component parsing - Remove `AllocationMode` public export (replaced by `ModelAllocation`) - Rename internal `AllocationMode` to `_AllocationMode` for SPMD launcher backward compatibility with FutureWarning - Remove auto-backend selection — explicit backend prefix is now required - Controllers (`TrainController`, `RolloutController`) parse `backend` directly instead of receiving `alloc_mode` from trainers - `WeightUpdateMeta.alloc_mode` replaced by `gen_allocation` (single `ModelAllocation`) - Add `RWTrainer` and `ArchonRWEngine` for reward model training - Remove `get_model_update_meta()` helper (logic moved to trainers) - Update all YAML configs, examples, docs (EN+ZH), and tests BREAKING CHANGE: `AllocationMode` is removed from public API. Users must migrate to per-engine `backend` fields. SPMD launchers emit deprecation warnings. * chore(ci): fix backend specifier for vlm sft test * fix: fix bare dims for actor backends * chore(docs): fix reminder for bare allocation dims | 5 个月前 | |
refactor(api): migrate allocation_mode to per-engine backend fields (#1044) * refactor(api): migrate allocation_mode to per-engine backend fields Replace the centralized `allocation_mode` string with explicit `backend` fields on `TrainEngineConfig` and `InferenceEngineConfig`. Each engine now owns its own backend+parallelism spec (e.g. `fsdp:d4`, `sglang:d4t2`), eliminating implicit auto-backend selection and the shared `AllocationMode` object. Key changes: - Add `backend` field to TrainEngineConfig and InferenceEngineConfig - Add `ModelAllocation.from_str()` for single-component parsing - Remove `AllocationMode` public export (replaced by `ModelAllocation`) - Rename internal `AllocationMode` to `_AllocationMode` for SPMD launcher backward compatibility with FutureWarning - Remove auto-backend selection — explicit backend prefix is now required - Controllers (`TrainController`, `RolloutController`) parse `backend` directly instead of receiving `alloc_mode` from trainers - `WeightUpdateMeta.alloc_mode` replaced by `gen_allocation` (single `ModelAllocation`) - Add `RWTrainer` and `ArchonRWEngine` for reward model training - Remove `get_model_update_meta()` helper (logic moved to trainers) - Update all YAML configs, examples, docs (EN+ZH), and tests BREAKING CHANGE: `AllocationMode` is removed from public API. Users must migrate to per-engine `backend` fields. SPMD launchers emit deprecation warnings. * chore(ci): fix backend specifier for vlm sft test * fix: fix bare dims for actor backends * chore(docs): fix reminder for bare allocation dims | 5 个月前 | |
refactor(api): migrate allocation_mode to per-engine backend fields (#1044) * refactor(api): migrate allocation_mode to per-engine backend fields Replace the centralized `allocation_mode` string with explicit `backend` fields on `TrainEngineConfig` and `InferenceEngineConfig`. Each engine now owns its own backend+parallelism spec (e.g. `fsdp:d4`, `sglang:d4t2`), eliminating implicit auto-backend selection and the shared `AllocationMode` object. Key changes: - Add `backend` field to TrainEngineConfig and InferenceEngineConfig - Add `ModelAllocation.from_str()` for single-component parsing - Remove `AllocationMode` public export (replaced by `ModelAllocation`) - Rename internal `AllocationMode` to `_AllocationMode` for SPMD launcher backward compatibility with FutureWarning - Remove auto-backend selection — explicit backend prefix is now required - Controllers (`TrainController`, `RolloutController`) parse `backend` directly instead of receiving `alloc_mode` from trainers - `WeightUpdateMeta.alloc_mode` replaced by `gen_allocation` (single `ModelAllocation`) - Add `RWTrainer` and `ArchonRWEngine` for reward model training - Remove `get_model_update_meta()` helper (logic moved to trainers) - Update all YAML configs, examples, docs (EN+ZH), and tests BREAKING CHANGE: `AllocationMode` is removed from public API. Users must migrate to per-engine `backend` fields. SPMD launchers emit deprecation warnings. * chore(ci): fix backend specifier for vlm sft test * fix: fix bare dims for actor backends * chore(docs): fix reminder for bare allocation dims | 5 个月前 | |
feat(infra): add distributed data loading service (#1120) * feat(infra): add distributed data loading service Introduce a controller/router/gateway/worker data service so single-controller training can offload dataset access to remote workers while preserving existing trainer workflows. Key changes: - add RDataset-based remote dataset registration and fetch path - integrate DataController lifecycle into RL/SFT/RW trainers - add guard reuse, data-service APIs, and full infra test coverage * chore: revert stats_logger to make trackio test happy | 4 个月前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 6 个月前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 6 个月前 | |
refactor(tests): move tests from areal/tests to top-level tests directory (#944) * refactor(tests): move tests from areal/tests to top-level tests directory Move the test directory from areal/tests/ to a top-level tests/ directory to follow Python best practices and improve project structure. Key changes: - Move all test files from areal/tests/ to tests/ - Create __init__.py files for tests package and subdirectories - Create areal/utils/testing_utils.py with shared utilities: - get_model_path, get_dataset_path (model/dataset path resolution) - MODEL_PATHS, DENSE_MODEL_PATHS, MOE_MODEL_PATHS (test model configs) - load_archon_model (Archon model loading for tests) - Update areal/tools/profiling_utils to re-export from testing_utils - Update test utilities to import from areal/utils/testing_utils, keeping test-specific behavior (pytest.skip) as wrappers - Update all imports from 'areal.tests.' to 'tests.' - Update hardcoded torchrun script paths - Update workflow string references for dynamic class loading - Update CI workflow paths in .github/workflows/test-areal.yml - Update documentation references in CLAUDE.md, AGENTS.md, CONTRIBUTING.md, and all .claude/ and .opencode/ skill/agent files * minor fix * remove import side effect * fix(tests): add tests/utils.py module for test imports Re-export get_model_path and get_dataset_path from areal.utils.testing_utils to fix ModuleNotFoundError in tests that import from tests.utils. | 6 个月前 | |
feat(infra): add microservice-based training service (controller v2) (#1169) * feat(infra): add microservice-based training service (controller v2) Add GatewayTrainController that decomposes training into five HTTP microservices: guard (process manager), worker (engine container), data proxy (batch dispatcher), router (service registry), and gateway (API ingress). This enables training orchestration without requiring the scheduler's RPC infrastructure. Key changes: - Add GatewayTrainController with 7-step async initialization - Add guard /set_env endpoint for NCCL env propagation - Add worker, router, gateway, data proxy FastAPI/Flask services - Wire TrainEngineConfig.log_level to suppress HTTP access logs - Add create_process_group stubs to existing engines - Add comprehensive unit and integration tests * fix(infra): stabilize controller v2 training service Consolidate the controller v2 training-service follow-up work into one atomic infra commit. This keeps startup, routing, dispatch, recovery, and health-handling changes together as the post-9c70 stabilization series while restoring a clean branch history. * feat(infra): add microservice-based training service (controller v2) Add GatewayTrainController that decomposes training into five HTTP microservices: guard (process manager), worker (engine container), data proxy (batch dispatcher), router (service registry), and gateway (API ingress). This enables training orchestration without requiring the scheduler's RPC infrastructure. Key changes: - Add GatewayTrainController with 7-step async initialization - Add guard /set_env endpoint for NCCL env propagation - Add worker, router, gateway, data proxy FastAPI/Flask services - Wire TrainEngineConfig.log_level to suppress HTTP access logs - Add create_process_group stubs to existing engines - Add comprehensive unit and integration tests * fix(infra): stabilize controller v2 training service Consolidate the controller v2 training-service follow-up work into one atomic infra commit. This keeps startup, routing, dispatch, recovery, and health-handling changes together as the post-9c70 stabilization series while restoring a clean branch history. * chore: add SPDX headers to training service modules --------- Co-authored-by: Wentai Zhang <zhangwentai.zwt@antgroup.com> | 4 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 6 个月前 | ||
| 5 个月前 | ||
| 5 个月前 | ||
| 5 个月前 | ||
| 4 个月前 | ||
| 6 个月前 | ||
| 6 个月前 | ||
| 6 个月前 | ||
| 4 个月前 |