| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
docs: clarify terminology in the M2PO guides (#1635) Signed-off-by: shicyu <77009832+Wrestle7@users.noreply.github.com> | 12 天前 | |
Docs: Update README and cli docs (#1521) * docs(readme): add v2.0.0 release news and update online RL doc link Add a News entry for the v2.0.0 release highlighting the microservice architecture (training/inference/agent/weight-update services) and the two agentic RL training examples (Hermes and SWE agent), with a link to the technical report on arXiv. Also update the Highlights section's online RL training link to the new areal-ai.io docs domain. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(readme): refine v2.0 news wording Drop the "(v2.0.0)" tag from the news date and rephrase the examples line so each example gets a specific, self-describing name: the Hermes online RL loop, and end-to-end SWE RL training examples. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(readme): refresh v2.0 highlights and swap logo - add Hermes and SWE agent examples to Agentic RL table - collapse [2026/04/23] Scaffoldings news under a <details> - mark Qwen2.5-VL / Qwen3-VL as Megatron-supported - link 2026 H2 roadmap issue in Future Roadmap - add CLI Configurations link under Tutorial - replace assets/figures/logo.png with v2.0 logo Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs: add consolidated CLI guide and tidy README sections - add docs/{en,zh}/best_practices/cli_guide.md consolidating the training / inference / agent v2 CLI guides; README Tutorial now points at the EN version - Agentic RL table: drop "(v2.0)" labels, rename SWE Agent RL to Coding Agent RL, note AReaL-SWEAgent/Claude Code Agent support - fold [2026/04/23], [2026/04/18], [2026/03/02] into the Previous Releases <details> - swap Q1 roadmap link for Q2 (#1302) in Future Roadmap Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(cli): migrate v2 per-service guides into consolidated CLI guide Content of areal/v2/cli/{training,inference,agent}/cli_guide.md is now maintained solely in docs/{en,zh}/best_practices/cli_guide.md. Removed the "canonical source" callback that pointed back at the per-service files. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(readme): fix Asynchronous RL Guide link path Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(readme): remove gitcgr badge link Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(assets): resize logo to 972x1250 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * style: apply mdformat to README and zh cli_guide Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> | 1 个月前 | |
docs: allow ray scheduler in agent workflow compatibility notes (#1567) | 1 个月前 | |
docs: allow ray scheduler in agent workflow compatibility notes (#1567) | 1 个月前 | |
chore: bump version to v2.1.0 (#1639) Prepare repository metadata and published image references for the v2.1.0 release. Key changes: - bump SGLang and vLLM package metadata and lockfiles - update installation and SkyPilot image references - align the manual release workflow input example | 11 天前 | |
chore: migrate repo references from InclusionAI to areal-project (#1325) Update all internal URLs and org references after migrating the repository from InclusionAI/AReaL to areal-project/AReaL. Key changes: - Update GitHub repo URLs (github.com/inclusionAI/AReaL -> areal-project/AReaL) - Update GitHub Pages docs URLs (inclusionai.github.io -> areal-project.github.io) - Update GHCR Docker image refs (ghcr.io/inclusionai/ -> areal-project/) - Update CI workflow usernames and GitHub API org references - Update DeepWiki and gitcgr badge URLs - Update issue references in code comments HuggingFace model/dataset URLs intentionally left unchanged. Vendored directories (sglang-src, Megatron-LM, Megatron-Bridge) skipped. | 3 个月前 | |
feat(trainer): add dpo (#1190) * feat(trainer): add DPO trainer with FSDP backend Add Direct Preference Optimization (Rafailov et al. 2023) as a new trainer. The policy is directly optimized to prefer chosen over rejected responses via a contrastive loss on log-probability ratios against a frozen reference model, removing the need for a separately trained reward model. Reference logprobs are computed online each step by a colocated ref engine, following the PPO/GRPO pattern. FSDP is the supported backend; Megatron and Archon variants raise NotImplementedError as placeholders. Verified on Qwen2.5-7B-Base + Anthropic/hh-rlhf (1 epoch, no SFT): reward_accuracy rises from 0.50 to ~0.70 and reward_margin grows monotonically, matching the original DPO paper's HH-RLHF results. * fix(trainer): fix DPO config forwarding, require ref model, and correct IPO normalization Fixes several issues found during PR review of the DPO trainer: Key changes: - Create DPOEngineConfig(TrainEngineConfig) embedding beta and loss_type, fixing silent parameter drop in single-controller mode (as_controller never forwarded beta/loss_type to workers) - Make ref a required field in DPOConfig (ref_logprobs are required at runtime, so config should enforce this upfront) - Remove zero-ref fallback in compute_dpo_loss; use input_["ref_logprobs"] directly - Add IPO loss with per-token length normalization matching TRL author- confirmed convention (normalize per-sequence logratios by completion length before the squared loss) - Remove all ref-is-None guard branches from DPOTrainer - Update docs, YAML config, and tests for all changes Refs: #1190 --------- Co-authored-by: 博惟 <bowei.fw@antgroup.com> | 4 个月前 | |
feat(v2): support grouped reward normalization (#1619) * feat(v2): support grouped reward normalization Normalize per-prompt rollout groups at the data-proxy export boundary before trajectory tensors are merged and remotized. Preserve raw rewards for filtering and metrics while sharing the normalization implementation with v1. Keep reward metadata local for rollout filters and clear remote shards for rejected or failed trajectories so filtered v2 rollouts do not leak data-proxy storage. * fix(v2): bound rejected trajectory cleanup | 11 天前 | |
feat:Add chinese doc (#969) * fix(docs): add language toggle button with proper styling and icon - Add lang-toggle.js and lang-toggle.css to en/zh _static directories - Update _config.yml to reference correct paths - Modify build_all.sh to copy static files and figures - Fix URL redirect logic (remove trailing slash) - Style button with translate icon, gray theme matching nav icons - Dynamically adjust icon size based on existing header icons * fix(docs): add language toggle button with proper styling and icon - Add lang-toggle.js and lang-toggle.css to en/zh _static directories - Update _config.yml to reference correct paths - Modify build_all.sh to copy static files and figures - Fix URL redirect logic (remove trailing slash) - Style button with translate icon, gray theme matching nav icons - Dynamically adjust icon size based on existing header icons * Update docs/build_all.sh Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update docs/_static/js/lang-toggle.js Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> | 5 个月前 | |
chore: migrate repo references from InclusionAI to areal-project (#1325) Update all internal URLs and org references after migrating the repository from InclusionAI/AReaL to areal-project/AReaL. Key changes: - Update GitHub repo URLs (github.com/inclusionAI/AReaL -> areal-project/AReaL) - Update GitHub Pages docs URLs (inclusionai.github.io -> areal-project.github.io) - Update GHCR Docker image refs (ghcr.io/inclusionai/ -> areal-project/) - Update CI workflow usernames and GitHub API org references - Update DeepWiki and gitcgr badge URLs - Update issue references in code comments HuggingFace model/dataset URLs intentionally left unchanged. Vendored directories (sglang-src, Megatron-LM, Megatron-Bridge) skipped. | 3 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 12 天前 | ||
| 1 个月前 | ||
| 1 个月前 | ||
| 1 个月前 | ||
| 11 天前 | ||
| 3 个月前 | ||
| 4 个月前 | ||
| 11 天前 | ||
| 5 个月前 | ||
| 3 个月前 |