AReaL/docs/en/best_practices · are/AReaL

GGitHubDocs: Update README and cli docs (#1521)

文件	最后提交记录	最后更新时间
algo_perf.md	feat(api): add unified RejectionSamplingConfig for async training (#1088) Replace behave_imp_weight_cap/behave_imp_weight_mode with unified RejectionSamplingConfig supporting multiple metrics (ratio, kl_k1, kl_k2, kl_k3), levels (token/sequence), and actions (mask/clamp). Key changes: - Add RejectionSamplingConfig dataclass with comprehensive validation - Implement apply_rejection_sampling for 1D packed and 2D padded formats - Fix loss denominator scaling bug in mask mode (save count before filtering) - Use geometric mean for sequence-level ratio aggregation (matching GSPO) - Broadcast sequence-level geometric mean as uniform behave_imp_weight - Warn when use_decoupled_loss=True but rejection_sampling is None - Update ppo_actor_loss_fn and grpo_loss_fn to use new config - Migrate 40 example configs to new rejection_sampling field - Add 43 unit tests covering all modes, metrics, and edge cases Refs: #1052	5 个月前
cli_guide.md	Docs: Update README and cli docs (#1521) * docs(readme): add v2.0.0 release news and update online RL doc link Add a News entry for the v2.0.0 release highlighting the microservice architecture (training/inference/agent/weight-update services) and the two agentic RL training examples (Hermes and SWE agent), with a link to the technical report on arXiv. Also update the Highlights section's online RL training link to the new areal-ai.io docs domain. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(readme): refine v2.0 news wording Drop the "(v2.0.0)" tag from the news date and rephrase the examples line so each example gets a specific, self-describing name: the Hermes online RL loop, and end-to-end SWE RL training examples. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(readme): refresh v2.0 highlights and swap logo - add Hermes and SWE agent examples to Agentic RL table - collapse [2026/04/23] Scaffoldings news under a <details> - mark Qwen2.5-VL / Qwen3-VL as Megatron-supported - link 2026 H2 roadmap issue in Future Roadmap - add CLI Configurations link under Tutorial - replace assets/figures/logo.png with v2.0 logo Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs: add consolidated CLI guide and tidy README sections - add docs/{en,zh}/best_practices/cli_guide.md consolidating the training / inference / agent v2 CLI guides; README Tutorial now points at the EN version - Agentic RL table: drop "(v2.0)" labels, rename SWE Agent RL to Coding Agent RL, note AReaL-SWEAgent/Claude Code Agent support - fold [2026/04/23], [2026/04/18], [2026/03/02] into the Previous Releases <details> - swap Q1 roadmap link for Q2 (#1302) in Future Roadmap Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(cli): migrate v2 per-service guides into consolidated CLI guide Content of areal/v2/cli/{training,inference,agent}/cli_guide.md is now maintained solely in docs/{en,zh}/best_practices/cli_guide.md. Removed the "canonical source" callback that pointed back at the per-service files. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(readme): fix Asynchronous RL Guide link path Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(readme): remove gitcgr badge link Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(assets): resize logo to 972x1250 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * style: apply mdformat to README and zh cli_guide Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>	2 个月前
debugging.md	refactor(api): migrate allocation_mode to per-engine backend fields (#1044) * refactor(api): migrate allocation_mode to per-engine backend fields Replace the centralized `allocation_mode` string with explicit `backend` fields on `TrainEngineConfig` and `InferenceEngineConfig`. Each engine now owns its own backend+parallelism spec (e.g. `fsdp:d4`, `sglang:d4t2`), eliminating implicit auto-backend selection and the shared `AllocationMode` object. Key changes: - Add `backend` field to TrainEngineConfig and InferenceEngineConfig - Add `ModelAllocation.from_str()` for single-component parsing - Remove `AllocationMode` public export (replaced by `ModelAllocation`) - Rename internal `AllocationMode` to `_AllocationMode` for SPMD launcher backward compatibility with FutureWarning - Remove auto-backend selection — explicit backend prefix is now required - Controllers (`TrainController`, `RolloutController`) parse `backend` directly instead of receiving `alloc_mode` from trainers - `WeightUpdateMeta.alloc_mode` replaced by `gen_allocation` (single `ModelAllocation`) - Add `RWTrainer` and `ArchonRWEngine` for reward model training - Remove `get_model_update_meta()` helper (logic moved to trainers) - Update all YAML configs, examples, docs (EN+ZH), and tests BREAKING CHANGE: `AllocationMode` is removed from public API. Users must migrate to per-engine `backend` fields. SPMD launchers emit deprecation warnings. * chore(ci): fix backend specifier for vlm sft test * fix: fix bare dims for actor backends * chore(docs): fix reminder for bare allocation dims	6 个月前
handling_oom.md	fix(fsdp): maintain fp32 master weights for AdamW (#1292) (#1369) Fix issue #1292: torch.optim.AdamW silently inherits bf16 dtype from model.parameters() when actor.dtype=bfloat16, causing late-stage SFT loss to plateau ~3x higher than DS-Z3 / Megatron precision-aware optimizer paths. Decouple parameter storage dtype from forward/backward compute dtype so AdamW sees fp32 params natively (fp32 master weights), while FSDP2 MixedPrecisionPolicy(param_dtype=dtype) keeps forward/backward in bf16. Cast back to compute dtype on HF export and xccl weight sync to keep deployment artefacts and rollout broadcasts unchanged. Changes: - Add TrainEngineConfig.optimizer_dtype (default 'float32') with __post_init__ validation and fp32/bf16 alias canonicalization - FSDPEngine: load model in optimizer_dtype (was: dtype) so AdamW creates fp32 exp_avg/exp_avg_sq when default. FSDP2 MP policy unchanged - Add _cast_to_compute_dtype helper; use in _save_model_to_hf and _update_weights_from_distributed (xccl) to cast fp32 storage back to compute dtype before disk export / SGLang broadcast - Replace misleading adam_bf16 'less stable' warning with dtype-aware logic; explicitly warn when adam + optimizer_dtype=bfloat16 combination triggers #1292 - Regression test (1 / 2 GPU) covering 5 dtype invariants: storage / Adam state / forward / HF export / xccl cast - Document fp32 master tradeoffs and adam_bf16 OOM fallback in en + zh handling_oom.md; regenerate CLI reference Backward compatibility: setting optimizer_dtype='bfloat16' explicitly restores pre-fix behaviour bit-for-bit (storage bf16, Adam state bf16, all casts no-op). The fix is opt-out via config rather than breaking change. Verified on 2x RTX 4090 with Qwen3-0.6B, GSM8K, 466 steps: - First 2 steps bit-identical between BEFORE/AFTER (sanity) - Last-50 mean: BEFORE 0.3904 vs AFTER 0.3461 (gap +0.044, 11%) - All 3 dtype-invariant pytest cases PASSED Fixes #1292.	4 个月前
perf_profiling.md	feat:Add chinese doc (#969) * fix(docs): add language toggle button with proper styling and icon - Add lang-toggle.js and lang-toggle.css to en/zh _static directories - Update _config.yml to reference correct paths - Modify build_all.sh to copy static files and figures - Fix URL redirect logic (remove trailing slash) - Style button with translate icon, gray theme matching nav icons - Dynamically adjust icon size based on existing header icons * fix(docs): add language toggle button with proper styling and icon - Add lang-toggle.js and lang-toggle.css to en/zh _static directories - Update _config.yml to reference correct paths - Modify build_all.sh to copy static files and figures - Fix URL redirect logic (remove trailing slash) - Style button with translate icon, gray theme matching nav icons - Dynamically adjust icon size based on existing header icons * Update docs/build_all.sh Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update docs/_static/js/lang-toggle.js Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>	7 个月前
workflow.md	feat:Add chinese doc (#969) * fix(docs): add language toggle button with proper styling and icon - Add lang-toggle.js and lang-toggle.css to en/zh _static directories - Update _config.yml to reference correct paths - Modify build_all.sh to copy static files and figures - Fix URL redirect logic (remove trailing slash) - Style button with translate icon, gray theme matching nav icons - Dynamically adjust icon size based on existing header icons * fix(docs): add language toggle button with proper styling and icon - Add lang-toggle.js and lang-toggle.css to en/zh _static directories - Update _config.yml to reference correct paths - Modify build_all.sh to copy static files and figures - Fix URL redirect logic (remove trailing slash) - Style button with translate icon, gray theme matching nav icons - Dynamically adjust icon size based on existing header icons * Update docs/build_all.sh Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update docs/_static/js/lang-toggle.js Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>	7 个月前