[feat] 新增verl-torchtitan qwen30B-A3B RL训练样例-cann-recipes-train-AtomGit

MREADME.md+5-5

MThird_Party_Open_Source_Software_Notice+2-2

	@@ -1629,7 +1629,7 @@ Official Repository: https://github.com/volcengine/verl
1629	Usage:"	1629	Usage:"
1630	agent_rl\qwen3_tool_agent\run_qwen3_4b_dapo_npu.sh	1630	agent_rl\qwen3_tool_agent\run_qwen3_4b_dapo_npu.sh
1631	agent_rl\qwen3_tool_agent\run_qwen3_4b_sft_npu.sh	1631	agent_rl\qwen3_tool_agent\run_qwen3_4b_sft_npu.sh
1632	-llm_rl\qwen3\patches\verl\utils\hybrid_data_parallel\hdp.py"	1632	+llm_rl\qwen3\verl-mindspeed\patches\verl\utils\hybrid_data_parallel\hdp.py"
1633	Copyright Notice(s):	1633	Copyright Notice(s):
1634	Copyright 2024 Bytedance Ltd.	1634	Copyright 2024 Bytedance Ltd.
1635	Copyright 2025 CollabLLM team	1635	Copyright 2025 CollabLLM team
	@@ -1778,4 +1778,4 @@ Copyright 2025 The vLLM team
1778	Copyright (c) 2024 Roberto Lopez Castro (roberto.lopez.castro@udc.es). All Rights Reserved	1778	Copyright (c) 2024 Roberto Lopez Castro (roberto.lopez.castro@udc.es). All Rights Reserved
1779	Copyright 2021 The EleutherAI and HuggingFace Teams. All rights reserved	1779	Copyright 2021 The EleutherAI and HuggingFace Teams. All rights reserved
1780	License: Apache License V2.0	1780	License: Apache License V2.0
1781	-Please see above	1781	+Please see above

Mci/validate_all_projects.sh+60-20

@@ -34,6 +34,12 @@ SCAN_LIST=(

34 # Other paths that needed check... 34 # Other paths that needed check...

35 ) 35 )

36 36

37 +REQUIRED_FILES=(

38 + "download_frameworks_source_code.sh"

39 + "build_project.sh"

40 + "apply_all_patches.sh"

41 +)

42 +

37 echo -e "${CYAN}=== CI Starts ===${RESET}" 43 echo -e "${CYAN}=== CI Starts ===${RESET}"

38 44

39 set +e 45 set +e

@@ -78,6 +84,12 @@ validate_project() {

78 echo -e "${GREEN}[OK] Project built.${RESET}" 84 echo -e "${GREEN}[OK] Project built.${RESET}"

79 85

80 echo -e "${CYAN}=== Step 4: Apply patches ===${RESET}" 86 echo -e "${CYAN}=== Step 4: Apply patches ===${RESET}"

87 +

88 + if grep -q "apply_all_patches.sh" build_project.sh; then

atomgit-bot6月23日

🟡 Medium Priority

变更行：ci/validate_all_projects.sh 第 88 行新增的 grep -q "apply_all_patches.sh" build_project.sh

受影响行为：CI 的 validate_project() 函数在 Step 4（应用 patch）之前，通过 grep 检查 build_project.sh 是否已包含对 apply_all_patches.sh 的调用。若匹配到，则跳过 patch 应用步骤并直接返回成功。

失败模式：grep -q "apply_all_patches.sh" 是一个纯子串匹配，不区分该字符串出现在可执行语句还是注释中。如果将来某人在 build_project.sh 中添加注释（例如 # apply_all_patches.sh is called separately 或 # TODO: remove apply_all_patches.sh），CI 会错误地跳过 patch 校验并报告成功，而实际上 patch 可能根本没有被应用——这是一个 CI 假阴性（false negative），会导致有问题的 patch 状态被漏过。

当前代码库状态：

llm_rl/qwen3/mindspeed/build_project.sh 不含该字符串，不匹配，正确。

当前代码库下行为正确，但匹配模式对未来的注释/文档性引用不具备抵抗力。

更稳健的方案是：在 build_project.sh 中显式设置一个标记（如 # CI_PATCHES_APPLIED=1），让 CI 检测该标记而非 grep 文件名字符串。

建议：将 grep -q "apply_all_patches.sh" build_project.sh 替换为更精确的模式，如 grep -qE '^\s*bash\s+.*apply_all_patches\.sh' build_project.sh，仅匹配实际执行的行而非注释中的文本引用。

改动建议

88	- if grep -q "apply_all_patches.sh" build_project.sh; then
88	+ if grep -qE '^\sbash\s+.apply_all_patches\.sh' build_project.sh; then

应用建议

不准确？

zhangjianshe

6月25日评论：

atomgit-bot6月23日

🟡 Medium Priority

CI 的 validate_project() 函数在 Step 4（应用 patch）之前，通过第 88 行的 grep -q "apply_all_patches.sh" build_project.sh 检查 build_project.sh 是否已包含 patch 应用逻辑。如果匹配到，则跳过额外的 apply_all_patches.sh 调用。

grep 模式 "apply_all_patches.sh" 过于宽泛——它匹配任何包含该字符串的行，包括：

注释（如 # We don't call apply_all_patches.sh here because...）
多行字符串/heredoc 中的文件路径说明
被注释掉的旧命令（如 # bash apply_all_patches.sh）

如果误匹配到注释或被禁用的调用，CI 会错误地跳过 patch 应用步骤，导致 patch 未实际应用而 CI 却显示成功。上一轮审查已报告，本轮未修复。

建议：将 grep 模式替换为更精确的正则，如 grep -qE '^\s*bash\s+.*apply_all_patches\.sh' build_project.sh，只匹配以 bash ...apply_all_patches.sh 开头的非注释行。

不准确？

zhangjianshe

6月25日评论：

89 + echo -e "${YELLOW}[SKIP] build_project.sh already applies patches.${RESET}"

90 + echo -e "${GREEN}=== Project CI completed successfully ===${RESET}"

91 + return 0

92 + fi

81 93

82 set +e 94 set +e

83 95

@@ -121,36 +133,64 @@ validate_project() {

121 echo -e "${GREEN}=== Project CI completed successfully ===${RESET}" 133 echo -e "${GREEN}=== Project CI completed successfully ===${RESET}"

122 } 134 }

123 135

124 -for PROJECT in "${SCAN_LIST[@]}"; do 136 +has_required_files() {

125 - FULL_PATH="${ROOT_DIR}/${PROJECT}" 137 + local project_dir="$1"

138 +

139 + for f in "${REQUIRED_FILES[@]}"; do

140 + if [ ! -f "${project_dir}/${f}" ]; then

141 + return 1

142 + fi

143 + done

144 +

145 + return 0

146 +}

147 +

148 +for SCAN_PATH in "${SCAN_LIST[@]}"; do

149 + FULL_PATH="${ROOT_DIR}/${SCAN_PATH}"

126 150

127 if [ ! -d "$FULL_PATH" ]; then 151 if [ ! -d "$FULL_PATH" ]; then

128 echo -e "${RED}[ERROR] Project directory not found: ${FULL_PATH}${RESET}" 152 echo -e "${RED}[ERROR] Project directory not found: ${FULL_PATH}${RESET}"

129 exit 1 153 exit 1

130 fi 154 fi

131 155

132 - PROJECT_BASENAME=$(basename "$FULL_PATH") 156 + PROJECT_DIRS=()

157 + if has_required_files "$FULL_PATH"; then

158 + PROJECT_DIRS+=("$FULL_PATH")

159 + else

160 + while IFS= read -r SUBDIR; do

161 + if has_required_files "$SUBDIR"; then

162 + PROJECT_DIRS+=("$SUBDIR")

163 + fi

164 + done < <(find "$FULL_PATH" -mindepth 1 -maxdepth 1 -type d | sort)

165 + fi

133 166

134 - echo -e "${CYAN}--- Running CI for project: ${PROJECT} ---${RESET}" 167 + if [ ${#PROJECT_DIRS[@]} -eq 0 ]; then

135 - 168 + SCAN_BASENAME=$(basename "$FULL_PATH")

136 - for f in download_frameworks_source_code.sh build_project.sh apply_all_patches.sh; do 169 + echo -e "${RED}[ERROR] Missing required project scripts under ${SCAN_BASENAME}${RESET}"

137 - if [ ! -f "${FULL_PATH}/${f}" ]; then 170 + for f in "${REQUIRED_FILES[@]}"; do

138 - echo -e "${RED}[ERROR] Missing ${f} in project ${PROJECT_BASENAME}${RESET}" 171 + echo -e "${RED}[ERROR] Missing ${f} in project ${SCAN_BASENAME}${RESET}"

139 - exit 1 172 + done

140 - fi

141 - done

142 -

143 - echo -e "${CYAN}Validating project ${PROJECT}${RESET}"

144 - pushd "${FULL_PATH}" >/dev/null

145 -

146 - if ! validate_project; then

147 - echo -e "${RED}[ERROR] CI pipeline failed for ${PROJECT_BASENAME}${RESET}"

148 - popd >/dev/null

149 exit 1 173 exit 1

150 fi 174 fi

151 175

152 - echo -e "${GREEN}[OK] Project ${PROJECT_BASENAME} passed CI.${RESET}" 176 + for PROJECT_DIR in "${PROJECT_DIRS[@]}"; do

153 - popd >/dev/null 177 + PROJECT="${PROJECT_DIR#${ROOT_DIR}/}"

178 + PROJECT_BASENAME=$(basename "$PROJECT_DIR")

179 +

180 + echo -e "${CYAN}--- Running CI for project: ${PROJECT} ---${RESET}"

181 +

182 + echo -e "${CYAN}Validating project ${PROJECT}${RESET}"

183 + pushd "${PROJECT_DIR}" >/dev/null

184 +

185 + if ! validate_project; then

186 + echo -e "${RED}[ERROR] CI pipeline failed for ${PROJECT_BASENAME}${RESET}"

187 + popd >/dev/null

188 + exit 1

189 + fi

190 +

191 + echo -e "${GREEN}[OK] Project ${PROJECT_BASENAME} passed CI.${RESET}"

192 + popd >/dev/null

193 + done

154 done 194 done

155 195

156 echo -e "${GREEN}=== All projects passed CI ===${RESET}" 196 echo -e "${GREEN}=== All projects passed CI ===${RESET}"

Mdocs/features/length_aware_resampler.md+4-4

	@@ -14,10 +14,10 @@ RL On-Policy 训练的 Rollout 阶段通常会遇到明显的 response 长尾问
14	本特性以 Git Patch 的形式交付，文件位于：	14	本特性以 Git Patch 的形式交付，文件位于：
15		15
16	```text	16	```text
17	-llm_rl/qwen3/patches/verl/0019-verl-feature-length_aware_resampler.patch	17	+llm_rl/qwen3/verl-mindspeed/patches/verl/0019-verl-feature-length_aware_resampler.patch
18	```	18	```
19		19
20	-在 `llm_rl/qwen3/` 目录下执行以下命令即可应用全部 patch：	20	+在 `llm_rl/qwen3/verl-mindspeed/` 目录下执行以下命令即可应用全部 patch：
21		21
22	```bash	22	```bash
23	bash apply_all_patches.sh	23	bash apply_all_patches.sh
	@@ -59,7 +59,7 @@ cap = min(response_length, max(min_tokens, factor * expected_len))
59	本样例提供了一个最小可复现脚本：	59	本样例提供了一个最小可复现脚本：
60		60
61	```text	61	```text
62	-llm_rl/qwen3/internal/train_grpo_qwen3_resampler_example.sh	62	+llm_rl/qwen3/verl-mindspeed/internal/train_grpo_qwen3_resampler_example.sh
63	```	63	```
64		64
65	该脚本保留了以下内容：	65	该脚本保留了以下内容：
	@@ -90,7 +90,7 @@ export TEST_FILE=/path/to/test.parquet
90	export REWARD_FUNCTION_PATH=/path/to/reward_function.py	90	export REWARD_FUNCTION_PATH=/path/to/reward_function.py
91	```	91	```
92		92
93	-在 `llm_rl/qwen3/` 目录下执行：	93	+在 `llm_rl/qwen3/verl-mindspeed/` 目录下执行：
94		94
95	```bash	95	```bash
96	bash internal/train_grpo_qwen3_resampler_example.sh	96	bash internal/train_grpo_qwen3_resampler_example.sh

Adocs/llm_rl/figures/qwen3_figures/image43.png+0-0

Mdocs/llm_rl/sam_decoding.md+2-2

	@@ -179,7 +179,7 @@ SAM（suffix automaton，后缀自动机）是一个能够高效解决许多字
179		179
180	### 3.1 性能优化	180	### 3.1 性能优化
181		181
182	-投机解码的性能取决于两个因素：第一是接受率，第二是投机算法本身带来的额外耗时（draft token 的生成和验证、拒绝采样）。为了能够最大化 SAM 无损投机在RL训练中的收益，我们需要尽可能地提高接受率，并减少额外耗时。为此，我们做了如下两大优化——关于 batch size 的自适应开关以及拒绝采样加速，相关代码已在[cann\-recipes\-train](https://gitcode.com/cann/cann-recipes-train/blob/master/llm_rl/qwen3) 全部开源。	182	+投机解码的性能取决于两个因素：第一是接受率，第二是投机算法本身带来的额外耗时（draft token 的生成和验证、拒绝采样）。为了能够最大化 SAM 无损投机在RL训练中的收益，我们需要尽可能地提高接受率，并减少额外耗时。为此，我们做了如下两大优化——关于 batch size 的自适应开关以及拒绝采样加速，相关代码已在[cann\-recipes\-train](https://gitcode.com/cann/cann-recipes-train/blob/master/llm_rl/qwen3/verl-mindspeed) 全部开源。
183		183
184	#### 3.1.1 自适应开关	184	#### 3.1.1 自适应开关
185		185
	@@ -203,7 +203,7 @@ SAM（suffix automaton，后缀自动机）是一个能够高效解决许多字
203		203
204	以`sample_recovered_tokens_pytorch`为例，[原生实现](https://github.com/vllm-project/vllm-ascend/blob/v0.11.0rc0/vllm_ascend/sample/rejection_sampler.py#L461)中含有嵌套的for循环操作。这种逐个元素的操作方式无法利用NPU的并行计算能力，并且在循环内部反复创建新张量（如 `.clone()`, `torch.full()`），导致效率极低。	204	以`sample_recovered_tokens_pytorch`为例，[原生实现](https://github.com/vllm-project/vllm-ascend/blob/v0.11.0rc0/vllm_ascend/sample/rejection_sampler.py#L461)中含有嵌套的for循环操作。这种逐个元素的操作方式无法利用NPU的并行计算能力，并且在循环内部反复创建新张量（如 `.clone()`, `torch.full()`），导致效率极低。
205		205
206	-我们优化的核心思想是向量化 $Vectorization$：去掉所有Python循环，使用PyTorch的张量操作来一次性处理所有数据。我们使用张量操作消除了循环内重复的张量创建，并通过高效的索引和广播实现了关键变量的计算。具体实现参考我们[开源的代码](https://gitcode.com/cann/cann-recipes-train/blob/master/llm_rl/qwen3/patches/vllm_ascend/0009-vllm_ascend-feature-rewrote-rejection-sampler.patch#L215)。后续可以使用 Triton 对这个模块做进一步的优化。	206	+我们优化的核心思想是向量化 $Vectorization$：去掉所有Python循环，使用PyTorch的张量操作来一次性处理所有数据。我们使用张量操作消除了循环内重复的张量创建，并通过高效的索引和广播实现了关键变量的计算。具体实现参考我们[开源的代码](https://gitcode.com/cann/cann-recipes-train/blob/master/llm_rl/qwen3/verl-mindspeed/patches/vllm_ascend)。后续可以使用 Triton 对这个模块做进一步的优化。
207		207
208	### 3.2 RL 训练实测收益	208	### 3.2 RL 训练实测收益
209		209

Rllm_rl/qwen3/Dockerfile.vllm_ascend.mindspeed.qwen3→llm_rl/qwen3/verl-mindspeed/Dockerfile.vllm_ascend.mindspeed.qwen3+0-0

文件重命名但无更改。

Rllm_rl/qwen3/README.md→llm_rl/qwen3/verl-mindspeed/README.md+10-10

	@@ -3,9 +3,9 @@
3	## 概述	3	## 概述
4	本样例针对Qwen3-235B-A22B和Qwen3-32B模型，基于[veRL开源框架](https://github.com/volcengine/verl)，以及veRL原生支持的MindSpeed和vLLM-Ascend框架，完成了多项强化学习实践。	4	本样例针对Qwen3-235B-A22B和Qwen3-32B模型，基于[veRL开源框架](https://github.com/volcengine/verl)，以及veRL原生支持的MindSpeed和vLLM-Ascend框架，完成了多项强化学习实践。
5		5
6	-针对Qwen3-235B-A22B RL训练全流程的优化适配，参见[Qwen3-235B 32K长序列RL训练优化实践](../../docs/llm_rl/qwen3_235B_32k_longseq_rl_train_optimization.md)。	6	+针对Qwen3-235B-A22B RL训练全流程的优化适配，参见[Qwen3-235B 32K长序列RL训练优化实践](../../../docs/llm_rl/qwen3_235B_32k_longseq_rl_train_optimization.md)。
7		7
8	-针对Qwen3-32B上的SAM投机推理实践，参见[SAM投机推理：长序列强化学习训练加速利器](../../docs/llm_rl/sam_decoding.md)。	8	+针对Qwen3-32B上的SAM投机推理实践，参见[SAM投机推理：长序列强化学习训练加速利器](../../../docs/llm_rl/sam_decoding.md)。
9		9
10	注：在当前版本，由于使用npugraph_ex替代了GE图模式，235B长序列优化实践文档中部分特性的patch已经失效，若想查看实践中完整的优化实现，请参考[v0.1.0版本](https://gitcode.com/cann/cann-recipes-train/tree/v0.1.0/llm_rl/qwen3)的代码。	10	注：在当前版本，由于使用npugraph_ex替代了GE图模式，235B长序列优化实践文档中部分特性的patch已经失效，若想查看实践中完整的优化实现，请参考[v0.1.0版本](https://gitcode.com/cann/cann-recipes-train/tree/v0.1.0/llm_rl/qwen3)的代码。
11		11
	@@ -101,7 +101,7 @@
101	# 下载本样例所在代码仓，以master分支为例	101	# 下载本样例所在代码仓，以master分支为例
102	git clone https://gitcode.com/cann/cann-recipes-train.git	102	git clone https://gitcode.com/cann/cann-recipes-train.git
103		103
104	- cd ./cann-recipes-train/llm_rl/qwen3/	104	+ cd ./cann-recipes-train/llm_rl/qwen3/verl-mindspeed/
105		105
106	# 添加镜像中已经准备好的依赖文件	106	# 添加镜像中已经准备好的依赖文件
107	bash build_project.sh	107	bash build_project.sh
	@@ -111,13 +111,13 @@
111	```	111	```
112		112
113	3. 使能patch修改：	113	3. 使能patch修改：
114	- 可通过 apply_all_patches.sh 一键执行，在当前目录下（cann-recipes-train/llm_rl/qwen3）运行：	114	+ 可通过 apply_all_patches.sh 一键执行，在当前目录下（cann-recipes-train/llm_rl/qwen3/verl-mindspeed）运行：
115	```bash	115	```bash
116	bash apply_all_patches.sh	116	bash apply_all_patches.sh
117	```	117	```
118		118
119	## 数据集准备	119	## 数据集准备
120	-本样例中GRPO使用的deepscaler数据集准备方法与DeepSeek示例相同，可参考[数据集准备](../deepseek/README.md#数据集准备)，将处理后的训练数据放在 `data/deepscaler` 目录下。	120	+本样例中GRPO使用的deepscaler数据集准备方法与DeepSeek示例相同，可参考[数据集准备](../../deepseek/README.md#数据集准备)，将处理后的训练数据放在 `data/deepscaler` 目录下。
121		121
122	DAPO使用的dapo-math-17k数据集，验证集使用AIME，可参考[DAPO数据准备](https://github.com/volcengine/verl/blob/main/recipe/dapo/prepare_dapo_data.sh)，将训练数据放在 `data/dapo_math` 目录下。	122	DAPO使用的dapo-math-17k数据集，验证集使用AIME，可参考[DAPO数据准备](https://github.com/volcengine/verl/blob/main/recipe/dapo/prepare_dapo_data.sh)，将训练数据放在 `data/dapo_math` 目录下。
123		123
	@@ -217,13 +217,13 @@ export VLLM_ROLLOUT_EARLY_STOP_FACTOR=2.0
217	export VLLM_ROLLOUT_EARLY_STOP_MIN_TOKENS=10000	217	export VLLM_ROLLOUT_EARLY_STOP_MIN_TOKENS=10000
218	```	218	```
219		219
220	-本样例还提供了一个最小可复现脚本 [train_grpo_qwen3_resampler_example.sh](internal/train_grpo_qwen3_resampler_example.sh)。该脚本保留了 16 卡 GRPO、Megatron 并行配置和 resampler 相关参数，但移除了个人路径、profile 和 draft train 相关配置。使用前只需按实际环境设置 `MODEL_PATH`、`DISTCP_PATH`、`TRAIN_FILE`、`TEST_FILE`，然后在 `llm_rl/qwen3/` 目录下执行：	220	+本样例还提供了一个最小可复现脚本 [train_grpo_qwen3_resampler_example.sh](internal/train_grpo_qwen3_resampler_example.sh)。该脚本保留了 16 卡 GRPO、Megatron 并行配置和 resampler 相关参数，但移除了个人路径、profile 和 draft train 相关配置。使用前只需按实际环境设置 `MODEL_PATH`、`DISTCP_PATH`、`TRAIN_FILE`、`TEST_FILE`，然后在 `llm_rl/qwen3/verl-mindspeed/` 目录下执行：
221		221
222	```bash	222	```bash
223	bash internal/train_grpo_qwen3_resampler_example.sh	223	bash internal/train_grpo_qwen3_resampler_example.sh
224	```	224	```
225		225
226	-该特性的设计背景、实现方案和收益说明可参考[Length-Aware Resampler：基于历史 Response 长度的 Rollout 重采样优化](../../docs/features/length_aware_resampler.md)。	226	+该特性的设计背景、实现方案和收益说明可参考[Length-Aware Resampler：基于历史 Response 长度的 Rollout 重采样优化](../../../docs/features/length_aware_resampler.md)。
227		227
228	## 附录	228	## 附录
229		229
	@@ -246,7 +246,7 @@ bash internal/train_grpo_qwen3_resampler_example.sh
246	\|verl\|[0008-verl-bugfix-enable_compile.patch](patches/verl/0008-verl-bugfix-enable_compile.patch)\|NPU上MindSpeed训练框架会无效化torch.compile规避训练侧的compile失败，在推理时开启compile\|	246	\|verl\|[0008-verl-bugfix-enable_compile.patch](patches/verl/0008-verl-bugfix-enable_compile.patch)\|NPU上MindSpeed训练框架会无效化torch.compile规避训练侧的compile失败，在推理时开启compile\|
247	\|verl\|[0009-verl-feature-support_EPLB.patch](patches/verl/0009-verl-feature-support_EPLB.patch)\|`VLLM_ENABLE_EPLB`开启时，使能推理的EPLB\|	247	\|verl\|[0009-verl-feature-support_EPLB.patch](patches/verl/0009-verl-feature-support_EPLB.patch)\|`VLLM_ENABLE_EPLB`开启时，使能推理的EPLB\|
248	\|verl\|[0010-verl-feature-enable_hdp.patch](patches/verl/0010-verl-feature-enable_hdp.patch)\|`USE_HDP`开启时，使能HDP功能\|	248	\|verl\|[0010-verl-feature-enable_hdp.patch](patches/verl/0010-verl-feature-enable_hdp.patch)\|`USE_HDP`开启时，使能HDP功能\|
249	-\|verl\|[0011-verl-feature-enable_rollout_rebalance.patch](patches/verl/0011-verl-feature-enable_rollout_rebalance.patch)\|`ROLLOUT_REBALANCE_ENABLE`开启时，使能Rollout Rebalance功能，详细说明可参考[RL On-Policy 推理场景的序列级均衡调度引擎](../../docs/features/rollout_rebalance.md)\|	249	+\|verl\|[0011-verl-feature-enable_rollout_rebalance.patch](patches/verl/0011-verl-feature-enable_rollout_rebalance.patch)\|`ROLLOUT_REBALANCE_ENABLE`开启时，使能Rollout Rebalance功能，详细说明可参考[RL On-Policy 推理场景的序列级均衡调度引擎](../../../docs/features/rollout_rebalance.md)\|
250	\|verl\|[0012-verl-feature-npugraph_ex_for_spec_decode.patch](patches/verl/0012-verl-feature-npugraph_ex_for_spec_decode.patch)\|允许通过脚本配置项配置投机推理以及npugraph_ex相关参数\|	250	\|verl\|[0012-verl-feature-npugraph_ex_for_spec_decode.patch](patches/verl/0012-verl-feature-npugraph_ex_for_spec_decode.patch)\|允许通过脚本配置项配置投机推理以及npugraph_ex相关参数\|
251	\|verl\|[0013-verl-bugfix-dataProto_concat.patch](patches/verl/0013-verl-bugfix-dataProto_concat.patch)\|合并DataProto数据时，避免因不同节点的`data['timing']['generate_sequences']`存在细微差异导致报错\|	251	\|verl\|[0013-verl-bugfix-dataProto_concat.patch](patches/verl/0013-verl-bugfix-dataProto_concat.patch)\|合并DataProto数据时，避免因不同节点的`data['timing']['generate_sequences']`存在细微差异导致报错\|
252	\|verl\|[0014-verl-feature-dapo_data_rebalance.patch](patches/verl/0014-verl-feature-dapo_data_rebalance.patch)\|`data_rebalance` DAPO算法适配\|	252	\|verl\|[0014-verl-feature-dapo_data_rebalance.patch](patches/verl/0014-verl-feature-dapo_data_rebalance.patch)\|`data_rebalance` DAPO算法适配\|
	@@ -254,7 +254,7 @@ bash internal/train_grpo_qwen3_resampler_example.sh
254	\|verl\|[0016-verl-bugfix-hot_swap_expandable_segments.patch](patches/verl/0016-verl-bugfix-hot_swap_expandable_segments.patch)\|在sleep mode下使能虚拟内存特性热切换\|	254	\|verl\|[0016-verl-bugfix-hot_swap_expandable_segments.patch](patches/verl/0016-verl-bugfix-hot_swap_expandable_segments.patch)\|在sleep mode下使能虚拟内存特性热切换\|
255	\|verl\|[0017-verl-bugfix-adapt_new_vllm_version.patch](patches/verl/0017-verl-bugfix-adapt_new_vllm_version.patch)\|修复切换到vllm>=0.13.0版本引入的import error\|	255	\|verl\|[0017-verl-bugfix-adapt_new_vllm_version.patch](patches/verl/0017-verl-bugfix-adapt_new_vllm_version.patch)\|修复切换到vllm>=0.13.0版本引入的import error\|
256	\|verl\|[0018-verl-bugfix-ignore_redundant_logs.patch](patches/verl/0018-verl-bugfix-ignore_redundant_logs.patch)\|去除多余的警告日志\|	256	\|verl\|[0018-verl-bugfix-ignore_redundant_logs.patch](patches/verl/0018-verl-bugfix-ignore_redundant_logs.patch)\|去除多余的警告日志\|
257	-\|verl\|[0019-verl-feature-length_aware_resampler.patch](patches/verl/0019-verl-feature-length_aware_resampler.patch)\|新增 Length-Aware Resampler：基于历史 response 长度对 epoch 内样本重新排序，并提供可选的 rollout 长尾保护机制，详细说明可参考[Length-Aware Resampler：基于历史 Response 长度的 Rollout 重采样优化](../../docs/features/length_aware_resampler.md)\|	257	+\|verl\|[0019-verl-feature-length_aware_resampler.patch](patches/verl/0019-verl-feature-length_aware_resampler.patch)\|新增 Length-Aware Resampler：基于历史 response 长度对 epoch 内样本重新排序，并提供可选的 rollout 长尾保护机制，详细说明可参考[Length-Aware Resampler：基于历史 Response 长度的 Rollout 重采样优化](../../../docs/features/length_aware_resampler.md)\|
258	\|vllm\|[0001-vllm-feature-disable_gc.patch](patches/vllm/0001-vllm-feature-disable_gc.patch)\|在decode step前关闭gc，避免因内存管理导致host bound影响推理性能\|	258	\|vllm\|[0001-vllm-feature-disable_gc.patch](patches/vllm/0001-vllm-feature-disable_gc.patch)\|在decode step前关闭gc，避免因内存管理导致host bound影响推理性能\|
259	\|vllm\|[0002-vllm-feature-enable_sam_decoding.patch](patches/vllm/0002-vllm-feature-enable_sam_decoding.patch)\|SAM投机推理适配vllm框架：在投机推理的配置中支持`method`为`sam`的选项\|	259	\|vllm\|[0002-vllm-feature-enable_sam_decoding.patch](patches/vllm/0002-vllm-feature-enable_sam_decoding.patch)\|SAM投机推理适配vllm框架：在投机推理的配置中支持`method`为`sam`的选项\|
260	\|vllm\|[0003-vllm-bugfix-rope_registry.patch](patches/vllm/0003-vllm-bugfix-rope_registry.patch)\|修复ROPE注册时import flash_attn的bug\|	260	\|vllm\|[0003-vllm-bugfix-rope_registry.patch](patches/vllm/0003-vllm-bugfix-rope_registry.patch)\|修复ROPE注册时import flash_attn的bug\|
	@@ -303,7 +303,7 @@ bash internal/train_grpo_qwen3_resampler_example.sh
303		303
304	为了让使用者和开发者直观了解我们基于开源代码做的修改，本样例中只包含patch代码，其他框架代码需要拉取。	304	为了让使用者和开发者直观了解我们基于开源代码做的修改，本样例中只包含patch代码，其他框架代码需要拉取。
305		305
306	- 在当前目录（cann-recipes-train/llm_rl/qwen3）执行如下脚本。请注意，确保当前环境能够访问互联网。	306	+ 在当前目录（cann-recipes-train/llm_rl/qwen3/verl-mindspeed）执行如下脚本。请注意，确保当前环境能够访问互联网。
307	```bash	307	```bash
308	bash download_frameworks_source_code.sh	308	bash download_frameworks_source_code.sh
309	```	309	```

Rllm_rl/qwen3/apply_all_patches.sh→llm_rl/qwen3/verl-mindspeed/apply_all_patches.sh+7-4

	@@ -15,14 +15,17 @@
15	#!/bin/bash	15	#!/bin/bash
16	set -o pipefail	16	set -o pipefail
17		17
		18	+SCRIPT_DIR=$(cd "$(dirname "$0")" && pwd)
		19	+REPO_DIR=$(cd "${SCRIPT_DIR}/../../.." && pwd)
		20	+PATCH_ROOT="llm_rl/qwen3/verl-mindspeed/patches"
		21	+
		22	+cd "${REPO_DIR}"
		23	+
18	echo "Applying patches in numerical order..."	24	echo "Applying patches in numerical order..."
19		25
20	-find ./patches -type f -name "*.patch" \| \	26	+find "./${PATCH_ROOT}" -type f -name "*.patch" \| \
21	sort -V \| \	27	sort -V \| \
22	-
23	-## apply patch
24	while IFS= read -r PATCH_FILE; do	28	while IFS= read -r PATCH_FILE; do
25	- # skip empty lines
26	[[ -z "$PATCH_FILE" ]] && continue	29	[[ -z "$PATCH_FILE" ]] && continue
27	PATCH_REL_PATH=$(realpath --relative-to=. "$PATCH_FILE")	30	PATCH_REL_PATH=$(realpath --relative-to=. "$PATCH_FILE")
28		31

Rllm_rl/qwen3/build_project.sh→llm_rl/qwen3/verl-mindspeed/build_project.sh+3-0

	@@ -13,6 +13,9 @@
13	# limitations under the License.	13	# limitations under the License.
14		14
15	#!/bin/bash	15	#!/bin/bash
		16	+SCRIPT_DIR=$(cd "$(dirname "$0")" && pwd)
		17	+cd "${SCRIPT_DIR}"
		18	+
16	cp -r /workspace/verl/verl ./	19	cp -r /workspace/verl/verl ./
17	cp -r /workspace/verl/recipe/r1_ascend ./	20	cp -r /workspace/verl/recipe/r1_ascend ./
18	cp /workspace/verl/scripts/converter_hf_to_mcore.py ./	21	cp /workspace/verl/scripts/converter_hf_to_mcore.py ./

Rllm_rl/qwen3/deepscaler.py→llm_rl/qwen3/verl-mindspeed/deepscaler.py+0-0

文件重命名但无更改。

Rllm_rl/qwen3/download_frameworks_source_code.sh→llm_rl/qwen3/verl-mindspeed/download_frameworks_source_code.sh+0-0

文件重命名但无更改。

Rllm_rl/qwen3/internal/qwen3_235b_env.sh→llm_rl/qwen3/verl-mindspeed/internal/qwen3_235b_env.sh+0-0

文件重命名但无更改。

Rllm_rl/qwen3/internal/qwen3_32b_env.sh→llm_rl/qwen3/verl-mindspeed/internal/qwen3_32b_env.sh+0-0

文件重命名但无更改。

Rllm_rl/qwen3/internal/train_dapo_qwen3_235b_128die_true_weight.sh→llm_rl/qwen3/verl-mindspeed/internal/train_dapo_qwen3_235b_128die_true_weight.sh+0-0

文件重命名但无更改。

Rllm_rl/qwen3/internal/train_dapo_qwen3_32b_32die_true_weight.sh→llm_rl/qwen3/verl-mindspeed/internal/train_dapo_qwen3_32b_32die_true_weight.sh+0-0

文件重命名但无更改。

Rllm_rl/qwen3/internal/train_grpo_qwen3_235b_128die_random_init.sh→llm_rl/qwen3/verl-mindspeed/internal/train_grpo_qwen3_235b_128die_random_init.sh+0-0

文件重命名但无更改。

Rllm_rl/qwen3/internal/train_grpo_qwen3_235b_128die_true_weight.sh→llm_rl/qwen3/verl-mindspeed/internal/train_grpo_qwen3_235b_128die_true_weight.sh+0-0

文件重命名但无更改。

Rllm_rl/qwen3/internal/train_grpo_qwen3_32b_32die_true_weight.sh→llm_rl/qwen3/verl-mindspeed/internal/train_grpo_qwen3_32b_32die_true_weight.sh+0-0

文件重命名但无更改。

Rllm_rl/qwen3/internal/train_grpo_qwen3_resampler_example.sh→llm_rl/qwen3/verl-mindspeed/internal/train_grpo_qwen3_resampler_example.sh+0-0

文件重命名但无更改。

Rllm_rl/qwen3/patches/0001-feature-model_converter.patch→llm_rl/qwen3/verl-mindspeed/patches/0001-feature-model_converter.patch+9-9

	@@ -4,16 +4,16 @@ Date: Thu, 20 Nov 2025 20:00:15 +0800
4	Subject: [PATCH] Modify model converter	4	Subject: [PATCH] Modify model converter
5		5
6	---	6	---
7	- llm_rl/qwen3/converter_hf_to_mcore.py \| 60 +++++++++++++++++++------	7	+ llm_rl/qwen3/verl-mindspeed/converter_hf_to_mcore.py \| 60 +++++++++++++++++++------
8	1 file changed, 46 insertions(+), 14 deletions(-)	8	1 file changed, 46 insertions(+), 14 deletions(-)
9		9
10	-diff --git a/llm_rl/qwen3/converter_hf_to_mcore.py b/llm_rl/qwen3/converter_hf_to_mcore.py	10	+diff --git a/llm_rl/qwen3/verl-mindspeed/converter_hf_to_mcore.py b/llm_rl/qwen3/verl-mindspeed/converter_hf_to_mcore.py
11	index 6e7cdf2..b6c7ed6 100644	11	index 6e7cdf2..b6c7ed6 100644
12	---- a/llm_rl/qwen3/converter_hf_to_mcore.py	12	+--- a/llm_rl/qwen3/verl-mindspeed/converter_hf_to_mcore.py
13	-+++ b/llm_rl/qwen3/converter_hf_to_mcore.py	13	++++ b/llm_rl/qwen3/verl-mindspeed/converter_hf_to_mcore.py
14	@@ -177,19 +177,40 @@ def convert_checkpoint_from_transformers_to_megatron(	14	@@ -177,19 +177,40 @@ def convert_checkpoint_from_transformers_to_megatron(
15	numel += safe_copy(hf_layer.mlp.gate.weight, layer.mlp.router.weight)	15	numel += safe_copy(hf_layer.mlp.gate.weight, layer.mlp.router.weight)
16	-	16	+
17	for idx, hf_expert in enumerate(hf_layer.mlp.experts):	17	for idx, hf_expert in enumerate(hf_layer.mlp.experts):
18	- num_experts = len(hf_layer.mlp.experts)	18	- num_experts = len(hf_layer.mlp.experts)
19	- num_local_experts = num_experts // ep_size	19	- num_local_experts = num_experts // ep_size
	@@ -62,7 +62,7 @@ index 6e7cdf2..b6c7ed6 100644
62	+ numel += safe_copy(	62	+ numel += safe_copy(
63	+ hf_expert.down_proj.weight, layer.mlp.experts.linear_fc2._parameters[f"weight{local_expert_idx}"]	63	+ hf_expert.down_proj.weight, layer.mlp.experts.linear_fc2._parameters[f"weight{local_expert_idx}"]
64	+ )	64	+ )
65	-	65	+
66	if has_share_expert:	66	if has_share_expert:
67	numel += safe_copy(hf_layer.mlp.shared_expert_gate.weight, layer.mlp.shared_experts.gate_weight)	67	numel += safe_copy(hf_layer.mlp.shared_expert_gate.weight, layer.mlp.shared_experts.gate_weight)
68	@@ -482,7 +503,17 @@ def convert_hf_to_mcore(	68	@@ -482,7 +503,17 @@ def convert_hf_to_mcore(
	@@ -81,7 +81,7 @@ index 6e7cdf2..b6c7ed6 100644
81	+ if os.getenv('USE_ALLTOALL_OVERLAP', '0') == '1':	81	+ if os.getenv('USE_ALLTOALL_OVERLAP', '0') == '1':
82	+ config_repatch['moe_alltoall_overlap_comm'] = True	82	+ config_repatch['moe_alltoall_overlap_comm'] = True
83	+ repatch(config_repatch)	83	+ repatch(config_repatch)
84	-	84	+
85	if world_size > 1 and not support_distributed_convert(hf_config):	85	if world_size > 1 and not support_distributed_convert(hf_config):
86	raise NotImplementedError(f"distributed conversion is not supported for {hf_config.architectures} yet.")	86	raise NotImplementedError(f"distributed conversion is not supported for {hf_config.architectures} yet.")
87	@@ -497,6 +528,7 @@ def convert_hf_to_mcore(	87	@@ -497,6 +528,7 @@ def convert_hf_to_mcore(
	@@ -90,8 +90,8 @@ index 6e7cdf2..b6c7ed6 100644
90	tfconfig.use_cpu_initialization = use_cpu_initialization	90	tfconfig.use_cpu_initialization = use_cpu_initialization
91	+ tfconfig.moe_grouped_gemm = True	91	+ tfconfig.moe_grouped_gemm = True
92	tie_word_embeddings = getattr(hf_config, "tie_word_embeddings", False)	92	tie_word_embeddings = getattr(hf_config, "tie_word_embeddings", False)
93	-	93	+
94	# init megatron model	94	# init megatron model
95	---	95	+--
96	2.50.1.windows.1	96	2.50.1.windows.1
97		97

Rllm_rl/qwen3/patches/megatron/0001-megatron-bugfix-state_ten_verification.patch→llm_rl/qwen3/verl-mindspeed/patches/megatron/0001-megatron-bugfix-state_ten_verification.patch+5-5

Rllm_rl/qwen3/patches/megatron/0002-megatron-feature-enable_hdp.patch→llm_rl/qwen3/verl-mindspeed/patches/megatron/0002-megatron-feature-enable_hdp.patch+9-9

Rllm_rl/qwen3/patches/megatron/0003-megatron-bugfix-fix_hf_converter.patch→llm_rl/qwen3/verl-mindspeed/patches/megatron/0003-megatron-bugfix-fix_hf_converter.patch+7-7

	@@ -1,5 +1,5 @@
1	From de3f604816fb7e67352e391dc5da57de32bbda2a Mon Sep 17 00:00:00 2001	1	From de3f604816fb7e67352e391dc5da57de32bbda2a Mon Sep 17 00:00:00 2001
2	-From:	2	+From:
3	Date: Mon, 20 Apr 2026 20:07:55 +0800	3	Date: Mon, 20 Apr 2026 20:07:55 +0800
4	Subject: [PATCH] patch	4	Subject: [PATCH] patch
5		5
	@@ -7,10 +7,10 @@ Subject: [PATCH] patch
7	.../core/dist_checkpointing/strategies/filesystem_async.py \| 5 +++--	7	.../core/dist_checkpointing/strategies/filesystem_async.py \| 5 +++--
8	1 file changed, 3 insertions(+), 2 deletions(-)	8	1 file changed, 3 insertions(+), 2 deletions(-)
9		9
10	-diff --git a/llm_rl/qwen3/megatron/core/dist_checkpointing/strategies/filesystem_async.py b/llm_rl/qwen3/megatron/core/dist_checkpointing/strategies/filesystem_async.py	10	+diff --git a/llm_rl/qwen3/verl-mindspeed/megatron/core/dist_checkpointing/strategies/filesystem_async.py b/llm_rl/qwen3/verl-mindspeed/megatron/core/dist_checkpointing/strategies/filesystem_async.py
11	index 2af90a6..e466b75 100644	11	index 2af90a6..e466b75 100644
12	---- a/llm_rl/qwen3/megatron/core/dist_checkpointing/strategies/filesystem_async.py	12	+--- a/llm_rl/qwen3/verl-mindspeed/megatron/core/dist_checkpointing/strategies/filesystem_async.py
13	-+++ b/llm_rl/qwen3/megatron/core/dist_checkpointing/strategies/filesystem_async.py	13	++++ b/llm_rl/qwen3/verl-mindspeed/megatron/core/dist_checkpointing/strategies/filesystem_async.py
14	@@ -315,15 +315,16 @@ class FileSystemWriterAsync(FileSystemWriter):	14	@@ -315,15 +315,16 @@ class FileSystemWriterAsync(FileSystemWriter):
15	try:	15	try:
16	file_name, storage_key, (bytes_data, tensor_data) = write_bucket	16	file_name, storage_key, (bytes_data, tensor_data) = write_bucket
	@@ -21,15 +21,15 @@ index 2af90a6..e466b75 100644
21	- _write_item(*transform_list, stream, data, write_item, storage_key)	21	- _write_item(*transform_list, stream, data, write_item, storage_key)
22	+ _write_item(*transform_list, stream, data, write_item, storage_key, SerializationFormat.TORCH_SAVE)	22	+ _write_item(*transform_list, stream, data, write_item, storage_key, SerializationFormat.TORCH_SAVE)
23	)	23	)
24	-	24	+
25	for write_item, tensor in tensor_data:	25	for write_item, tensor in tensor_data:
26	assert tensor.is_cpu	26	assert tensor.is_cpu
27	local_results.append(	27	local_results.append(
28	- _write_item(*transform_list, stream, tensor, write_item, storage_key)	28	- _write_item(*transform_list, stream, tensor, write_item, storage_key)
29	+ _write_item(*transform_list, stream, tensor, write_item, storage_key, SerializationFormat.TORCH_SAVE)	29	+ _write_item(*transform_list, stream, tensor, write_item, storage_key, SerializationFormat.TORCH_SAVE)
30	)	30	)
31	-	31	+
32	if use_fsync:	32	if use_fsync:
33	---	33	+--
34	2.50.1	34	2.50.1
35		35

Rllm_rl/qwen3/patches/mindspeed/0001-mindspeed-bugfix-builder.patch→llm_rl/qwen3/verl-mindspeed/patches/mindspeed/0001-mindspeed-bugfix-builder.patch+7-7

	@@ -5,22 +5,22 @@ Subject: [PATCH 1/2] Update mindspeed: builder
5	Fix the header file missing issue in compiler version 24.03.	5	Fix the header file missing issue in compiler version 24.03.
6		6
7	---	7	---
8	- llm_rl/qwen3/mindspeed/op_builder/builder.py \| 2 +-	8	+ llm_rl/qwen3/verl-mindspeed/mindspeed/op_builder/builder.py \| 2 +-
9	1 file changed, 1 insertion(+), 1 deletion(-)	9	1 file changed, 1 insertion(+), 1 deletion(-)
10		10
11	-diff --git a/llm_rl/qwen3/mindspeed/op_builder/builder.py b/llm_rl/qwen3/mindspeed/op_builder/builder.py	11	+diff --git a/llm_rl/qwen3/verl-mindspeed/mindspeed/op_builder/builder.py b/llm_rl/qwen3/verl-mindspeed/mindspeed/op_builder/builder.py
12	index 8cacac2..1b10806 100644	12	index 8cacac2..1b10806 100644
13	---- a/llm_rl/qwen3/mindspeed/op_builder/builder.py	13	+--- a/llm_rl/qwen3/verl-mindspeed/mindspeed/op_builder/builder.py
14	-+++ b/llm_rl/qwen3/mindspeed/op_builder/builder.py	14	++++ b/llm_rl/qwen3/verl-mindspeed/mindspeed/op_builder/builder.py
15	@@ -52,7 +52,7 @@ class MindSpeedOpBuilder(ABC):	15	@@ -52,7 +52,7 @@ class MindSpeedOpBuilder(ABC):
16	-	16	+
17	def cxx_args(self):	17	def cxx_args(self):
18	args = ['-fstack-protector-all', '-Wl,-z,relro,-z,now,-z,noexecstack', '-fPIC', '-pie',	18	args = ['-fstack-protector-all', '-Wl,-z,relro,-z,now,-z,noexecstack', '-fPIC', '-pie',
19	- '-s', '-fvisibility=hidden', '-D_FORTIFY_SOURCE=2', '-O2']	19	- '-s', '-fvisibility=hidden', '-D_FORTIFY_SOURCE=2', '-O2']
20	+ '-s', '-fvisibility=hidden', '-D_FORTIFY_SOURCE=2', '-O2', '-include cstring']	20	+ '-s', '-fvisibility=hidden', '-D_FORTIFY_SOURCE=2', '-O2', '-include cstring']
21	return args	21	return args
22	-	22	+
23	def extra_ldflags(self):	23	def extra_ldflags(self):
24	---	24	+--
25	2.50.1.windows.1	25	2.50.1.windows.1
26		26

Rllm_rl/qwen3/patches/mindspeed/0002-mindspeed-feature-enable_hdp.patch→llm_rl/qwen3/verl-mindspeed/patches/mindspeed/0002-mindspeed-feature-enable_hdp.patch+14-14

Rllm_rl/qwen3/patches/r1_ascend/0001-r1_ascend-bugfix-correct_DP_initialization.patch→llm_rl/qwen3/verl-mindspeed/patches/r1_ascend/0001-r1_ascend-bugfix-correct_DP_initialization.patch+34-34

	@@ -1,34 +1,34 @@
1	-From 2bb17f583f8c75b1af14bc1f30f7a96cf8af19f1 Mon Sep 17 00:00:00 2001	1	+From 2bb17f583f8c75b1af14bc1f30f7a96cf8af19f1 Mon Sep 17 00:00:00 2001
2	-From: huyuanquan1 <huyuanquan1@huawei.com>	2	+From: huyuanquan1 <huyuanquan1@huawei.com>
3	-Date: Fri, 27 Feb 2026 14:46:57 +0800	3	+Date: Fri, 27 Feb 2026 14:46:57 +0800
4	-Subject: [PATCH] bugfix for DP initialization	4	+Subject: [PATCH] bugfix for DP initialization
5	-	5	+
6	----	6	+---
7	- recipe/r1_ascend/vllm_parallel_state.py \| 4 +++-	7	+ recipe/r1_ascend/vllm_parallel_state.py \| 4 +++-
8	- 1 file changed, 3 insertions(+), 1 deletion(-)	8	+ 1 file changed, 3 insertions(+), 1 deletion(-)
9	-	9	+
10	-diff --git a/llm_rl/qwen3/r1_ascend/vllm_parallel_state.py b/llm_rl/qwen3/r1_ascend/vllm_parallel_state.py	10	+diff --git a/llm_rl/qwen3/verl-mindspeed/r1_ascend/vllm_parallel_state.py b/llm_rl/qwen3/verl-mindspeed/r1_ascend/vllm_parallel_state.py
11	-index 82eb4175..9bfe08e7 100644	11	+index 82eb4175..9bfe08e7 100644
12	---- a/llm_rl/qwen3/r1_ascend/vllm_parallel_state.py	12	+--- a/llm_rl/qwen3/verl-mindspeed/r1_ascend/vllm_parallel_state.py
13	-+++ b/llm_rl/qwen3/r1_ascend/vllm_parallel_state.py	13	++++ b/llm_rl/qwen3/verl-mindspeed/r1_ascend/vllm_parallel_state.py
14	-@@ -16,6 +16,7 @@ from vllm.distributed.parallel_state import (	14	+@@ -16,6 +16,7 @@ from vllm.distributed.parallel_state import (
15	- init_distributed_environment,	15	+ init_distributed_environment,
16	- initialize_model_parallel,	16	+ initialize_model_parallel,
17	- )	17	+ )
18	-+from vllm.config import VllmConfig, set_current_vllm_config	18	++from vllm.config import VllmConfig, set_current_vllm_config
19	-	19	+
20	- logger = logging.getLogger(__file__)	20	+ logger = logging.getLogger(__file__)
21	- logger.setLevel(os.getenv("VERL_LOGGING_LEVEL", "WARN"))	21	+ logger.setLevel(os.getenv("VERL_LOGGING_LEVEL", "WARN"))
22	-@@ -88,7 +89,8 @@ def init_parallel_state(tensor_parallel_size):	22	+@@ -88,7 +89,8 @@ def init_parallel_state(tensor_parallel_size):
23	- backend = "hccl"	23	+ backend = "hccl"
24	- init_distributed_environment(world_size, rank, distributed_init_method, local_rank, backend)	24	+ init_distributed_environment(world_size, rank, distributed_init_method, local_rank, backend)
25	-	25	+
26	-- initialize_model_parallel(tensor_parallel_size)	26	+- initialize_model_parallel(tensor_parallel_size)
27	-+ with set_current_vllm_config(VllmConfig()):	27	++ with set_current_vllm_config(VllmConfig()):
28	-+ initialize_model_parallel(tensor_parallel_size)	28	++ initialize_model_parallel(tensor_parallel_size)
29	- logger.info(	29	+ logger.info(
30	- f"[DEBUG]: RANK[{rank}]: TP group: {vllm_ps._TP.ranks}\n"	30	+ f"[DEBUG]: RANK[{rank}]: TP group: {vllm_ps._TP.ranks}\n"
31	- f"[DEBUG]: RANK[{rank}]: PP group: {vllm_ps._PP.ranks}\n"	31	+ f"[DEBUG]: RANK[{rank}]: PP group: {vllm_ps._PP.ranks}\n"
32	---	32	+--
33	-2.45.1.windows.1	33	+2.45.1.windows.1
34	-	34	+

Rllm_rl/qwen3/patches/verl/0001-verl-feature-enable_alltoall_overlap.patch→llm_rl/qwen3/verl-mindspeed/patches/verl/0001-verl-feature-enable_alltoall_overlap.patch+9-9

	@@ -4,17 +4,17 @@ Date: Tue, 11 Nov 2025 10:45:32 +0800
4	Subject: [PATCH] Update verl: handle USE_ALLTOALL_OVERLAP	4	Subject: [PATCH] Update verl: handle USE_ALLTOALL_OVERLAP
5		5
6	Adds special handling logic for mlp.experts.weight in MoE model when USE_ALLTOALL_OVERLAP is enabled.	6	Adds special handling logic for mlp.experts.weight in MoE model when USE_ALLTOALL_OVERLAP is enabled.
7	-After distributed checkpoint loading, it explicitly copies the expert weights loaded back into	7	+After distributed checkpoint loading, it explicitly copies the expert weights loaded back into
8	the model's state_dict and parameter tensors to ensure they take effect properly.	8	the model's state_dict and parameter tensors to ensure they take effect properly.
9		9
10	---	10	---
11	- llm_rl/qwen3/verl/utils/model.py \| 13 +++++++++++--	11	+ llm_rl/qwen3/verl-mindspeed/verl/utils/model.py \| 13 +++++++++++--
12	1 file changed, 11 insertions(+), 2 deletions(-)	12	1 file changed, 11 insertions(+), 2 deletions(-)
13		13
14	-diff --git a/llm_rl/qwen3/verl/utils/model.py b/llm_rl/qwen3/verl/utils/model.py	14	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/utils/model.py b/llm_rl/qwen3/verl-mindspeed/verl/utils/model.py
15	index 15fdecd..009b3de 100644	15	index 15fdecd..009b3de 100644
16	---- a/llm_rl/qwen3/verl/utils/model.py	16	+--- a/llm_rl/qwen3/verl-mindspeed/verl/utils/model.py
17	-+++ b/llm_rl/qwen3/verl/utils/model.py	17	++++ b/llm_rl/qwen3/verl-mindspeed/verl/utils/model.py
18	@@ -556,8 +556,17 @@ def load_mcore_dist_weights(parallel_model, dist_weight_path, is_value_model=Fal	18	@@ -556,8 +556,17 @@ def load_mcore_dist_weights(parallel_model, dist_weight_path, is_value_model=Fal
19	for k in list(ssd.keys()):	19	for k in list(ssd.keys()):
20	if "output_layer" in k:	20	if "output_layer" in k:
	@@ -31,10 +31,10 @@ index 15fdecd..009b3de 100644
31	+ sd[key].copy_(new_ssd[key]) # tensor update	31	+ sd[key].copy_(new_ssd[key]) # tensor update
32	+ else:	32	+ else:
33	+ dist_checkpointing.load(ssd, dist_weight_path, strict=strict)	33	+ dist_checkpointing.load(ssd, dist_weight_path, strict=strict)
34	-+	34	++
35	return	35	return
36	-	36	+
37	-	37	+
38	---	38	+--
39	2.50.1.windows.1	39	2.50.1.windows.1
40		40

Rllm_rl/qwen3/patches/verl/0002-verl-feature-set_use_tqdm_true.patch→llm_rl/qwen3/verl-mindspeed/patches/verl/0002-verl-feature-set_use_tqdm_true.patch+5-5

Rllm_rl/qwen3/patches/verl/0003-verl-feature-recompute_old_log_prob.patch→llm_rl/qwen3/verl-mindspeed/patches/verl/0003-verl-feature-recompute_old_log_prob.patch+22-22

	@@ -3,36 +3,36 @@ From: caojingyi <caojingyi@noreply.gitcode.com>
3	Date: Tue, 11 Nov 2025 10:55:09 +0800	3	Date: Tue, 11 Nov 2025 10:55:09 +0800
4	Subject: [PATCH 03/18] Update verl: recompute_old_log_prob config	4	Subject: [PATCH 03/18] Update verl: recompute_old_log_prob config
5	For GRPO algorithm, add a new configuration item config `recompute_old_log_prob` (default can be True)	5	For GRPO algorithm, add a new configuration item config `recompute_old_log_prob` (default can be True)
6	-and corresponding handling logic: when the configuration is set to False and `ppo_epochs=1`, save one	6	+and corresponding handling logic: when the configuration is set to False and `ppo_epochs=1`, save one
7	forward computation for performance optimization.	7	forward computation for performance optimization.
8		8
9	---	9	---
10	.../verl/trainer/config/actor/actor.yaml \| 3 +++	10	.../verl/trainer/config/actor/actor.yaml \| 3 +++
11	.../qwen3/verl/trainer/ppo/ray_trainer.py \| 21 +++++++++++--------	11	.../qwen3/verl/trainer/ppo/ray_trainer.py \| 21 +++++++++++--------
12	.../verl/workers/actor/megatron_actor.py \| 8 +++++--	12	.../verl/workers/actor/megatron_actor.py \| 8 +++++--
13	- llm_rl/qwen3/verl/workers/config/actor.py \| 3 +++	13	+ llm_rl/qwen3/verl-mindspeed/verl/workers/config/actor.py \| 3 +++
14	4 files changed, 24 insertions(+), 11 deletions(-)	14	4 files changed, 24 insertions(+), 11 deletions(-)
15		15
16	-diff --git a/llm_rl/qwen3/verl/trainer/config/actor/actor.yaml b/llm_rl/qwen3/verl/trainer/config/actor/actor.yaml	16	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/trainer/config/actor/actor.yaml b/llm_rl/qwen3/verl-mindspeed/verl/trainer/config/actor/actor.yaml
17	index 7c55df2..5f89134 100644	17	index 7c55df2..5f89134 100644
18	---- a/llm_rl/qwen3/verl/trainer/config/actor/actor.yaml	18	+--- a/llm_rl/qwen3/verl-mindspeed/verl/trainer/config/actor/actor.yaml
19	-+++ b/llm_rl/qwen3/verl/trainer/config/actor/actor.yaml	19	++++ b/llm_rl/qwen3/verl-mindspeed/verl/trainer/config/actor/actor.yaml
20	@@ -93,6 +93,9 @@ ppo_epochs: 1	20	@@ -93,6 +93,9 @@ ppo_epochs: 1
21	# Shuffle training data across PPO epochs	21	# Shuffle training data across PPO epochs
22	shuffle: false	22	shuffle: false
23	-	23	+
24	+# Recompute old_log_prob	24	+# Recompute old_log_prob
25	+recompute_old_log_prob: True	25	+recompute_old_log_prob: True
26	+	26	+
27	# checkpoint configs	27	# checkpoint configs
28	checkpoint:	28	checkpoint:
29	-	29	+
30	-diff --git a/llm_rl/qwen3/verl/trainer/ppo/ray_trainer.py b/llm_rl/qwen3/verl/trainer/ppo/ray_trainer.py	30	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/trainer/ppo/ray_trainer.py b/llm_rl/qwen3/verl-mindspeed/verl/trainer/ppo/ray_trainer.py
31	index 61b710c..6f50a6e 100644	31	index 61b710c..6f50a6e 100644
32	---- a/llm_rl/qwen3/verl/trainer/ppo/ray_trainer.py	32	+--- a/llm_rl/qwen3/verl-mindspeed/verl/trainer/ppo/ray_trainer.py
33	-+++ b/llm_rl/qwen3/verl/trainer/ppo/ray_trainer.py	33	++++ b/llm_rl/qwen3/verl-mindspeed/verl/trainer/ppo/ray_trainer.py
34	@@ -1101,15 +1101,18 @@ class RayPPOTrainer:	34	@@ -1101,15 +1101,18 @@ class RayPPOTrainer:
35	-	35	+
36	# recompute old_log_probs	36	# recompute old_log_probs
37	with marked_timer("old_log_prob", timing_raw, color="blue"):	37	with marked_timer("old_log_prob", timing_raw, color="blue"):
38	- old_log_prob = self.actor_rollout_wg.compute_log_prob(batch)	38	- old_log_prob = self.actor_rollout_wg.compute_log_prob(batch)
	@@ -56,13 +56,13 @@ index 61b710c..6f50a6e 100644
56	+ batch = batch.union(old_log_prob)	56	+ batch = batch.union(old_log_prob)
57	+ else:	57	+ else:
58	+ batch.meta_info["temperature"] = self.config.actor_rollout_ref.rollout.temperature	58	+ batch.meta_info["temperature"] = self.config.actor_rollout_ref.rollout.temperature
59	-	59	+
60	if "rollout_log_probs" in batch.batch.keys():	60	if "rollout_log_probs" in batch.batch.keys():
61	# TODO: we may want to add diff of probs too.	61	# TODO: we may want to add diff of probs too.
62	-diff --git a/llm_rl/qwen3/verl/workers/actor/megatron_actor.py b/llm_rl/qwen3/verl/workers/actor/megatron_actor.py	62	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/workers/actor/megatron_actor.py b/llm_rl/qwen3/verl-mindspeed/verl/workers/actor/megatron_actor.py
63	index 55c4d60..6f9b5bb 100644	63	index 55c4d60..6f9b5bb 100644
64	---- a/llm_rl/qwen3/verl/workers/actor/megatron_actor.py	64	+--- a/llm_rl/qwen3/verl-mindspeed/verl/workers/actor/megatron_actor.py
65	-+++ b/llm_rl/qwen3/verl/workers/actor/megatron_actor.py	65	++++ b/llm_rl/qwen3/verl-mindspeed/verl/workers/actor/megatron_actor.py
66	@@ -311,9 +311,10 @@ class MegatronPPOActor(BasePPOActor):	66	@@ -311,9 +311,10 @@ class MegatronPPOActor(BasePPOActor):
67	"attention_mask",	67	"attention_mask",
68	"response_mask",	68	"response_mask",
	@@ -85,18 +85,18 @@ index 55c4d60..6f9b5bb 100644
85	+ else:	85	+ else:
86	+ old_log_prob = log_prob.detach()	86	+ old_log_prob = log_prob.detach()
87	advantages = data["advantages"]	87	advantages = data["advantages"]
88	-	88	+
89	entropy_coeff = self.config.entropy_coeff	89	entropy_coeff = self.config.entropy_coeff
90	-diff --git a/llm_rl/qwen3/verl/workers/config/actor.py b/llm_rl/qwen3/verl/workers/config/actor.py	90	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/workers/config/actor.py b/llm_rl/qwen3/verl-mindspeed/verl/workers/config/actor.py
91	index 60ba030..2d22c1a 100644	91	index 60ba030..2d22c1a 100644
92	---- a/llm_rl/qwen3/verl/workers/config/actor.py	92	+--- a/llm_rl/qwen3/verl-mindspeed/verl/workers/config/actor.py
93	-+++ b/llm_rl/qwen3/verl/workers/config/actor.py	93	++++ b/llm_rl/qwen3/verl-mindspeed/verl/workers/config/actor.py
94	@@ -120,6 +120,7 @@ class ActorConfig(BaseConfig):	94	@@ -120,6 +120,7 @@ class ActorConfig(BaseConfig):
95	data_loader_seed = 1	95	data_loader_seed = 1
96	rollout_n: int = 1 # must be override by sampling config	96	rollout_n: int = 1 # must be override by sampling config
97	model_config: HFModelConfig = field(default_factory=BaseConfig)	97	model_config: HFModelConfig = field(default_factory=BaseConfig)
98	+ recompute_old_log_prob: bool = True	98	+ recompute_old_log_prob: bool = True
99	-	99	+
100	def __post_init__(self):	100	def __post_init__(self):
101	"""Validate actor configuration parameters."""	101	"""Validate actor configuration parameters."""
102	@@ -146,6 +147,8 @@ class ActorConfig(BaseConfig):	102	@@ -146,6 +147,8 @@ class ActorConfig(BaseConfig):
	@@ -105,9 +105,9 @@ index 60ba030..2d22c1a 100644
105	raise ValueError(f"Invalid loss_agg_mode: {self.loss_agg_mode}")	105	raise ValueError(f"Invalid loss_agg_mode: {self.loss_agg_mode}")
106	+ if not self.recompute_old_log_prob:	106	+ if not self.recompute_old_log_prob:
107	+ assert self.ppo_epochs == 1	107	+ assert self.ppo_epochs == 1
108	-	108	+
109	def validate(self, n_gpus: int, train_batch_size: int, model_config: dict = None):	109	def validate(self, n_gpus: int, train_batch_size: int, model_config: dict = None):
110	"""Validate actor configuration with runtime parameters."""	110	"""Validate actor configuration with runtime parameters."""
111	---	111	+--
112	2.50.1.windows.1	112	2.50.1.windows.1
113		113

Rllm_rl/qwen3/patches/verl/0004-verl-feature-data_rebalance.patch→llm_rl/qwen3/verl-mindspeed/patches/verl/0004-verl-feature-data_rebalance.patch+11-11

Rllm_rl/qwen3/patches/verl/0005-verl-feature-moe_alltoallv.patch→llm_rl/qwen3/verl-mindspeed/patches/verl/0005-verl-feature-moe_alltoallv.patch+16-16

Rllm_rl/qwen3/patches/verl/0006-verl-feature-weight_converter_alltoall_overlap.patch→llm_rl/qwen3/verl-mindspeed/patches/verl/0006-verl-feature-weight_converter_alltoall_overlap.patch+5-5

	@@ -5,13 +5,13 @@ Subject: [PATCH 14/18] Update verl: weight_converter
5	Improve model parameter name conversion from Mcore to HF format.	5	Improve model parameter name conversion from Mcore to HF format.
6		6
7	---	7	---
8	- llm_rl/qwen3/verl/models/mcore/weight_converter.py \| 9 +++++++++	8	+ llm_rl/qwen3/verl-mindspeed/verl/models/mcore/weight_converter.py \| 9 +++++++++
9	1 file changed, 9 insertions(+)	9	1 file changed, 9 insertions(+)
10		10
11	-diff --git a/llm_rl/qwen3/verl/models/mcore/weight_converter.py b/llm_rl/qwen3/verl/models/mcore/weight_converter.py	11	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/models/mcore/weight_converter.py b/llm_rl/qwen3/verl-mindspeed/verl/models/mcore/weight_converter.py
12	index 791513f..de060be 100644	12	index 791513f..de060be 100644
13	---- a/llm_rl/qwen3/verl/models/mcore/weight_converter.py	13	+--- a/llm_rl/qwen3/verl-mindspeed/verl/models/mcore/weight_converter.py
14	-+++ b/llm_rl/qwen3/verl/models/mcore/weight_converter.py	14	++++ b/llm_rl/qwen3/verl-mindspeed/verl/models/mcore/weight_converter.py
15	@@ -474,6 +474,15 @@ class McoreToHFWeightConverterQwen3Moe(McoreToHFWeightConverterDense):	15	@@ -474,6 +474,15 @@ class McoreToHFWeightConverterQwen3Moe(McoreToHFWeightConverterDense):
16	expert_id = name.split("weight")[-1]	16	expert_id = name.split("weight")[-1]
17	convert_names.append(f"model.layers.{layer_number}.mlp.experts.{expert_id}.down_proj.weight")	17	convert_names.append(f"model.layers.{layer_number}.mlp.experts.{expert_id}.down_proj.weight")
	@@ -28,6 +28,6 @@ index 791513f..de060be 100644
28	else:	28	else:
29	raise NotImplementedError(f"Unsupported parameter name: {name}")	29	raise NotImplementedError(f"Unsupported parameter name: {name}")
30	return convert_names, params	30	return convert_names, params
31	---	31	+--
32	2.50.1.windows.1	32	2.50.1.windows.1
33		33

Rllm_rl/qwen3/patches/verl/0007-verl-bugfix-moe_update_weights.patch→llm_rl/qwen3/verl-mindspeed/patches/verl/0007-verl-bugfix-moe_update_weights.patch+8-8

Rllm_rl/qwen3/patches/verl/0008-verl-bugfix-enable_compile.patch→llm_rl/qwen3/verl-mindspeed/patches/verl/0008-verl-bugfix-enable_compile.patch+12-12

Rllm_rl/qwen3/patches/verl/0009-verl-feature-support_EPLB.patch→llm_rl/qwen3/verl-mindspeed/patches/verl/0009-verl-feature-support_EPLB.patch+13-13

	@@ -4,42 +4,42 @@ Date: Mon, 26 Jan 2026 16:07:12 +0800
4	Subject: [PATCH] Update verl: support EPLB	4	Subject: [PATCH] Update verl: support EPLB
5		5
6	---	6	---
7	- llm_rl/qwen3/verl/workers/megatron_workers.py \| 2 +	7	+ llm_rl/qwen3/verl-mindspeed/verl/workers/megatron_workers.py \| 2 +
8	.../rollout/vllm_rollout/vllm_rollout_spmd.py \| 37 +++++++++++++++++++	8	.../rollout/vllm_rollout/vllm_rollout_spmd.py \| 37 +++++++++++++++++++
9	2 files changed, 39 insertions(+)	9	2 files changed, 39 insertions(+)
10		10
11	-diff --git a/llm_rl/qwen3/verl/workers/megatron_workers.py b/llm_rl/qwen3/verl/workers/megatron_workers.py	11	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/workers/megatron_workers.py b/llm_rl/qwen3/verl-mindspeed/verl/workers/megatron_workers.py
12	index 9d20ca6..6cb8086 100644	12	index 9d20ca6..6cb8086 100644
13	---- a/llm_rl/qwen3/verl/workers/megatron_workers.py	13	+--- a/llm_rl/qwen3/verl-mindspeed/verl/workers/megatron_workers.py
14	-+++ b/llm_rl/qwen3/verl/workers/megatron_workers.py	14	++++ b/llm_rl/qwen3/verl-mindspeed/verl/workers/megatron_workers.py
15	@@ -599,9 +599,11 @@ class ActorRolloutRefWorker(MegatronWorker, DistProfilerExtension):	15	@@ -599,9 +599,11 @@ class ActorRolloutRefWorker(MegatronWorker, DistProfilerExtension):
16	# important: need to manually set the random states of each tp to be identical.	16	# important: need to manually set the random states of each tp to be identical.
17	self.torch_random_states = get_torch_device().get_rng_state()	17	self.torch_random_states = get_torch_device().get_rng_state()
18	get_torch_device().set_rng_state(self.gen_random_states)	18	get_torch_device().set_rng_state(self.gen_random_states)
19	+ self.rollout.eplb_start()	19	+ self.rollout.eplb_start()
20	-	20	+
21	async def trainer_mode(self):	21	async def trainer_mode(self):
22	"""Context switch hybridengine to trainer mode."""	22	"""Context switch hybridengine to trainer mode."""
23	+ self.rollout.eplb_end()	23	+ self.rollout.eplb_end()
24	if self.config.rollout.free_cache_engine:	24	if self.config.rollout.free_cache_engine:
25	log_gpu_memory_usage("Before rollout offload", logger=logger)	25	log_gpu_memory_usage("Before rollout offload", logger=logger)
26	await self.rollout.release()	26	await self.rollout.release()
27	-diff --git a/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py b/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	27	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py b/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
28	index 2bd2814..c3adcaf 100644	28	index 2bd2814..c3adcaf 100644
29	---- a/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	29	+--- a/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
30	-+++ b/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	30	++++ b/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
31	@@ -52,6 +52,7 @@ from torch.distributed.device_mesh import DeviceMesh	31	@@ -52,6 +52,7 @@ from torch.distributed.device_mesh import DeviceMesh
32	from vllm import LLM, SamplingParams	32	from vllm import LLM, SamplingParams
33	from vllm.config import CompilationConfig, CompilationLevel, LoRAConfig	33	from vllm.config import CompilationConfig, CompilationLevel, LoRAConfig
34	from vllm.lora.request import LoRARequest	34	from vllm.lora.request import LoRARequest
35	+from vllm_ascend.ascend_config import get_ascend_config	35	+from vllm_ascend.ascend_config import get_ascend_config
36	-	36	+
37	try:	37	try:
38	from vllm.worker.worker_base import WorkerWrapperBase	38	from vllm.worker.worker_base import WorkerWrapperBase
39	@@ -194,6 +195,7 @@ class vLLMRollout(BaseRollout):	39	@@ -194,6 +195,7 @@ class vLLMRollout(BaseRollout):
40	else:	40	else:
41	logger.warning(f"cudagraph_capture_sizes must be a list, but got {cudagraph_capture_sizes}")	41	logger.warning(f"cudagraph_capture_sizes must be a list, but got {cudagraph_capture_sizes}")
42	-	42	+
43	+ self.dynamic_eplb = int(os.environ.get("VLLM_ENABLE_EPLB", "0")) == 1	43	+ self.dynamic_eplb = int(os.environ.get("VLLM_ENABLE_EPLB", "0")) == 1
44	self.inference_engine = LLM(	44	self.inference_engine = LLM(
45	model=model_path,	45	model=model_path,
	@@ -63,9 +63,9 @@ index 2bd2814..c3adcaf 100644
63	**self.lora_kwargs,	63	**self.lora_kwargs,
64	**engine_kwargs,	64	**engine_kwargs,
65	@@ -237,6 +250,30 @@ class vLLMRollout(BaseRollout):	65	@@ -237,6 +250,30 @@ class vLLMRollout(BaseRollout):
66	-	66	+
67	self.pad_token_id = tokenizer.pad_token_id	67	self.pad_token_id = tokenizer.pad_token_id
68	-	68	+
69	+ self.eplb_end()	69	+ self.eplb_end()
70	+	70	+
71	+ def eplb_start(self):	71	+ def eplb_start(self):
	@@ -93,6 +93,6 @@ index 2bd2814..c3adcaf 100644
93	@contextmanager	93	@contextmanager
94	def update_sampling_params(self, **kwargs):	94	def update_sampling_params(self, **kwargs):
95	# update sampling params	95	# update sampling params
96	---	96	+--
97	2.45.1.windows.1	97	2.45.1.windows.1
98		98

Rllm_rl/qwen3/patches/verl/0010-verl-feature-enable_hdp.patch→llm_rl/qwen3/verl-mindspeed/patches/verl/0010-verl-feature-enable_hdp.patch+16-16

Rllm_rl/qwen3/patches/verl/0011-verl-feature-enable_rollout_rebalance.patch→llm_rl/qwen3/verl-mindspeed/patches/verl/0011-verl-feature-enable_rollout_rebalance.patch+6-6

Rllm_rl/qwen3/patches/verl/0012-verl-feature-npugraph_ex_for_spec_decode.patch→llm_rl/qwen3/verl-mindspeed/patches/verl/0012-verl-feature-npugraph_ex_for_spec_decode.patch+7-7

Rllm_rl/qwen3/patches/verl/0013-verl-bugfix-dataProto_concat.patch→llm_rl/qwen3/verl-mindspeed/patches/verl/0013-verl-bugfix-dataProto_concat.patch+7-7

	@@ -6,17 +6,17 @@ Subject: [PATCH] Fix DataProto concat bug
6	DataProto concat bug when timing generate_sequences is sightly different	6	DataProto concat bug when timing generate_sequences is sightly different
7	https://github.com/volcengine/verl/pull/4030	7	https://github.com/volcengine/verl/pull/4030
8	---	8	---
9	- llm_rl/qwen3/verl/protocol.py \| 25 ++++++++++++++++++++++++-	9	+ llm_rl/qwen3/verl-mindspeed/verl/protocol.py \| 25 ++++++++++++++++++++++++-
10	1 file changed, 24 insertions(+), 1 deletion(-)	10	1 file changed, 24 insertions(+), 1 deletion(-)
11		11
12	-diff --git a/llm_rl/qwen3/verl/protocol.py b/llm_rl/qwen3/verl/protocol.py	12	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/protocol.py b/llm_rl/qwen3/verl-mindspeed/verl/protocol.py
13	index e0b1aff..fda1e7c 100644	13	index e0b1aff..fda1e7c 100644
14	---- a/llm_rl/qwen3/verl/protocol.py	14	+--- a/llm_rl/qwen3/verl-mindspeed/verl/protocol.py
15	-+++ b/llm_rl/qwen3/verl/protocol.py	15	++++ b/llm_rl/qwen3/verl-mindspeed/verl/protocol.py
16	@@ -940,6 +940,24 @@ class DataProto:	16	@@ -940,6 +940,24 @@ class DataProto:
17	for key, val in non_tensor_batch.items():	17	for key, val in non_tensor_batch.items():
18	non_tensor_batch[key] = np.concatenate(val, axis=0)	18	non_tensor_batch[key] = np.concatenate(val, axis=0)
19	-	19	+
20	+ def meta_info_equal(a, b):	20	+ def meta_info_equal(a, b):
21	+ if type(a) is not type(b):	21	+ if type(a) is not type(b):
22	+ return False	22	+ return False
	@@ -51,7 +51,7 @@ index e0b1aff..fda1e7c 100644
51	+ assert merged_meta_info[k] == v, f"Conflicting values for meta_info key '{k}'"	51	+ assert merged_meta_info[k] == v, f"Conflicting values for meta_info key '{k}'"
52	else:	52	else:
53	merged_meta_info[k] = v	53	merged_meta_info[k] = v
54	-	54	+
55	---	55	+--
56	2.50.1.windows.1	56	2.50.1.windows.1
57		57

Rllm_rl/qwen3/patches/verl/0014-verl-feature-dapo_data_rebalance.patch→llm_rl/qwen3/verl-mindspeed/patches/verl/0014-verl-feature-dapo_data_rebalance.patch+8-8

Rllm_rl/qwen3/patches/verl/0015-verl-feature-hdp_binpack_optimization.patch→llm_rl/qwen3/verl-mindspeed/patches/verl/0015-verl-feature-hdp_binpack_optimization.patch+41-41

	@@ -1,23 +1,23 @@
1	From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001	1	From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
2	Subject: [PATCH] verl: hdp binpack optimization	2	Subject: [PATCH] verl: hdp binpack optimization
3		3
4	-diff --git a/llm_rl/qwen3/patches/verl/utils/hybrid_data_parallel/hdp.py b/llm_rl/qwen3/patches/verl/utils/hybrid_data_parallel/hdp.py	4	+diff --git a/llm_rl/qwen3/verl-mindspeed/patches/verl/utils/hybrid_data_parallel/hdp.py b/llm_rl/qwen3/verl-mindspeed/patches/verl/utils/hybrid_data_parallel/hdp.py
5	---- a/llm_rl/qwen3/patches/verl/utils/hybrid_data_parallel/hdp.py	5	+--- a/llm_rl/qwen3/verl-mindspeed/patches/verl/utils/hybrid_data_parallel/hdp.py
6	-+++ b/llm_rl/qwen3/patches/verl/utils/hybrid_data_parallel/hdp.py	6	++++ b/llm_rl/qwen3/verl-mindspeed/patches/verl/utils/hybrid_data_parallel/hdp.py
7	@@ -17,6 +17,9 @@	7	@@ -17,6 +17,9 @@
8	# limitations under the License.	8	# limitations under the License.
9	-	9	+
10	from functools import wraps	10	from functools import wraps
11	+import json	11	+import json
12	+import math	12	+import math
13	+import os	13	+import os
14	import torch	14	import torch
15	from torch import distributed as dist	15	from torch import distributed as dist
16	-	16	+
17	@@ -28,27 +31,77 @@	17	@@ -28,27 +31,77 @@
18	-	18	+
19	from patches.verl.utils.hybrid_data_parallel.utils import set_batch_hdp_group, get_batch_hdp_group	19	from patches.verl.utils.hybrid_data_parallel.utils import set_batch_hdp_group, get_batch_hdp_group
20	-	20	+
21	+_HDP_MAX_TOKEN_LEN = None	21	+_HDP_MAX_TOKEN_LEN = None
22	+	22	+
23	+	23	+
	@@ -77,10 +77,10 @@ diff --git a/llm_rl/qwen3/patches/verl/utils/hybrid_data_parallel/hdp.py b/llm_r
77	+ with open(filename, "a") as f:	77	+ with open(filename, "a") as f:
78	+ f.write("\n".join(lines) + "\n")	78	+ f.write("\n".join(lines) + "\n")
79	+	79	+
80	-	80	+
81	def generate_hdp_group_from_batch(	81	def generate_hdp_group_from_batch(
82	-- atten_mask: torch.Tensor,	82	+- atten_mask: torch.Tensor,
83	-- per_rank_overload_threshold=0.1,	83	+- per_rank_overload_threshold=0.1,
84	- fractional_roundup_threshold=0.9,	84	- fractional_roundup_threshold=0.9,
85	- max_fraction_threshold=0.5	85	- max_fraction_threshold=0.5
86	+ atten_mask: torch.Tensor,	86	+ atten_mask: torch.Tensor,
	@@ -89,26 +89,26 @@ diff --git a/llm_rl/qwen3/patches/verl/utils/hybrid_data_parallel/hdp.py b/llm_r
89	):	89	):
90	"""	90	"""
91	Generate HDP groups for micro-batch to optimize Ring Attention communication in CP.	91	Generate HDP groups for micro-batch to optimize Ring Attention communication in CP.
92	-	92	+
93	Args:	93	Args:
94	atten_mask: Attention mask tensor of shape [micro_batch_size, sequence_length]	94	atten_mask: Attention mask tensor of shape [micro_batch_size, sequence_length]
95	-- per_rank_overload_threshold: Maximum allowed overload per rank. For sequences already assigned	95	+- per_rank_overload_threshold: Maximum allowed overload per rank. For sequences already assigned
96	-- to N ranks, if the fractional part is less than N * this threshold,	96	+- to N ranks, if the fractional part is less than N * this threshold,
97	- the fractional tokens can be absorbed without adding a new rank.	97	- the fractional tokens can be absorbed without adding a new rank.
98	-- Example: sequence needs 2.1 ranks with threshold=0.1 → assigned to 2 ranks.	98	+- Example: sequence needs 2.1 ranks with threshold=0.1 -> assigned to 2 ranks.
99	-- fractional_roundup_threshold: Threshold for rounding up fractional rank requirements. When the	99	+- fractional_roundup_threshold: Threshold for rounding up fractional rank requirements. When the
100	- fractional part exceeds this value, round up to the next integer.	100	- fractional part exceeds this value, round up to the next integer.
101	-- Example: sequence needs 1.9 ranks with threshold=0.9 → assigned to 2 ranks.	101	+- Example: sequence needs 1.9 ranks with threshold=0.9 -> assigned to 2 ranks.
102	- max_fraction_threshold: Maximum fractional overload allowed regardless of per-rank threshold.	102	- max_fraction_threshold: Maximum fractional overload allowed regardless of per-rank threshold.
103	- Prevents excessive overload when sequences are assigned many ranks.	103	- Prevents excessive overload when sequences are assigned many ranks.
104	+ max_token_len: Max token length budget used for micro-batch packing.	104	+ max_token_len: Max token length budget used for micro-batch packing.
105	-	105	+
106	Returns:	106	Returns:
107	List of lists containing rank indices assigned to each sequence	107	List of lists containing rank indices assigned to each sequence
108	@@ -72,54 +125,51 @@	108	@@ -72,54 +125,51 @@
109	processed completely within a single rank.	109	processed completely within a single rank.
110	"""	110	"""
111	-	111	+
112	- if not hasattr(generate_hdp_group_from_batch, 'has_run'):	112	- if not hasattr(generate_hdp_group_from_batch, 'has_run'):
113	+ if not hasattr(generate_hdp_group_from_batch, "has_run"):	113	+ if not hasattr(generate_hdp_group_from_batch, "has_run"):
114	# First run hdp, use cp instead, for initializing the cp group.	114	# First run hdp, use cp instead, for initializing the cp group.
	@@ -139,7 +139,7 @@ diff --git a/llm_rl/qwen3/patches/verl/utils/hybrid_data_parallel/hdp.py b/llm_r
139	+ if total_ranks < cp_size:	139	+ if total_ranks < cp_size:
140	+ max_idx = max(range(len(seq_len_list)), key=lambda i: seq_len_list[i])	140	+ max_idx = max(range(len(seq_len_list)), key=lambda i: seq_len_list[i])
141	+ ranks_per_seq[max_idx] += cp_size - total_ranks	141	+ ranks_per_seq[max_idx] += cp_size - total_ranks
142	-	142	+
143	start_rank = 0	143	start_rank = 0
144	- # Initialize HDP groups for each sequence in the batch	144	- # Initialize HDP groups for each sequence in the batch
145	- batch_hdp_group = [[] for _ in ranks_per_seq]	145	- batch_hdp_group = [[] for _ in ranks_per_seq]
	@@ -150,7 +150,7 @@ diff --git a/llm_rl/qwen3/patches/verl/utils/hybrid_data_parallel/hdp.py b/llm_r
150	- if ranks >= 1:	150	- if ranks >= 1:
151	- batch_hdp_group[i] = list(range(start_rank, start_rank + int(ranks)))	151	- batch_hdp_group[i] = list(range(start_rank, start_rank + int(ranks)))
152	- start_rank += int(ranks)	152	- start_rank += int(ranks)
153	-- if frac < min(per_rank_overload_threshold * int(ranks), max_fraction_threshold):	153	+- if frac < min(per_rank_overload_threshold * int(ranks), max_fraction_threshold):
154	- continue	154	- continue
155	- if frac > fractional_roundup_threshold:	155	- if frac > fractional_roundup_threshold:
156	- batch_hdp_group[i].append(start_rank)	156	- batch_hdp_group[i].append(start_rank)
	@@ -189,25 +189,25 @@ diff --git a/llm_rl/qwen3/patches/verl/utils/hybrid_data_parallel/hdp.py b/llm_r
189	set_batch_hdp_group(batch_hdp_group)	189	set_batch_hdp_group(batch_hdp_group)
190	+ _maybe_dump_hdp_groups(seq_len_effective, batch_hdp_group, cp_size)	190	+ _maybe_dump_hdp_groups(seq_len_effective, batch_hdp_group, cp_size)
191	return	191	return
192	-	192	+
193	-	193	+
194	@@ -214,39 +264,45 @@	194	@@ -214,39 +264,45 @@
195	return batch_hdp_group, start_rank	195	return batch_hdp_group, start_rank
196	-	196	+
197	-	197	+
198	-def pack_sequences_into_buckets(seqlen_list: list[int], max_bucket_length: int, num_buckets: int) -> list[list[int]]:	198	-def pack_sequences_into_buckets(seqlen_list: list[int], max_bucket_length: int, num_buckets: int) -> list[list[int]]:
199	+def pack_sequences_into_buckets(seqlen_list: list[int], max_token_len: int) -> list[list[int]]:	199	+def pack_sequences_into_buckets(seqlen_list: list[int], max_token_len: int) -> list[list[int]]:
200	"""	200	"""
201	- Pack sequences into buckets using greedy algorithm for subsequent HDP grouping.	201	- Pack sequences into buckets using greedy algorithm for subsequent HDP grouping.
202	--	202	+-
203	- Args:	203	- Args:
204	- seqlen_list: List of sequence lengths to be packed	204	- seqlen_list: List of sequence lengths to be packed
205	- max_bucket_length: Maximum allowed total sequence length per bucket	205	- max_bucket_length: Maximum allowed total sequence length per bucket
206	- num_buckets: Number of available buckets for distribution	206	- num_buckets: Number of available buckets for distribution
207	--	207	+-
208	- Returns:	208	- Returns:
209	- List of buckets, where each bucket contains the original indices of sequences assigned to it	209	- List of buckets, where each bucket contains the original indices of sequences assigned to it
210	--	210	+-
211	+ Pack sequences into buckets based on rank capacity (cp_size).	211	+ Pack sequences into buckets based on rank capacity (cp_size).
212	+	212	+
213	+ Each sequence consumes ceil(seqlen / (max_token_len / cp_size)) ranks, and each bucket	213	+ Each sequence consumes ceil(seqlen / (max_token_len / cp_size)) ranks, and each bucket
	@@ -215,7 +215,7 @@ diff --git a/llm_rl/qwen3/patches/verl/utils/hybrid_data_parallel/hdp.py b/llm_r
215	"""	215	"""
216	- # Sort by length in descending order, preserving original indices	216	- # Sort by length in descending order, preserving original indices
217	- indexed_seqlens = sorted(enumerate(seqlen_list), key=lambda x: -x[1])	217	- indexed_seqlens = sorted(enumerate(seqlen_list), key=lambda x: -x[1])
218	--	218	+-
219	- buckets = [[] for _ in range(num_buckets)]	219	- buckets = [[] for _ in range(num_buckets)]
220	- bucket_sums = [0] * num_buckets	220	- bucket_sums = [0] * num_buckets
221	+ cp_size = mpu.get_context_parallel_world_size()	221	+ cp_size = mpu.get_context_parallel_world_size()
	@@ -227,7 +227,7 @@ diff --git a/llm_rl/qwen3/patches/verl/utils/hybrid_data_parallel/hdp.py b/llm_r
227	+	227	+
228	+ buckets = []	228	+ buckets = []
229	+ bucket_loads = []	229	+ bucket_loads = []
230	-	230	+
231	- # Assign each sequence to the first bucket that can accommodate it	231	- # Assign each sequence to the first bucket that can accommodate it
232	- for idx, length in indexed_seqlens:	232	- for idx, length in indexed_seqlens:
233	- placed = False	233	- placed = False
	@@ -264,21 +264,21 @@ diff --git a/llm_rl/qwen3/patches/verl/utils/hybrid_data_parallel/hdp.py b/llm_r
264	+ else:	264	+ else:
265	+ buckets[best_bucket].append(idx)	265	+ buckets[best_bucket].append(idx)
266	+ bucket_loads[best_bucket] += rank_need	266	+ bucket_loads[best_bucket] += rank_need
267	-
268	- return buckets
269	-
270		267
271	-diff --git a/llm_rl/qwen3/verl/utils/seqlen_balancing.py b/llm_rl/qwen3/verl/utils/seqlen_balancing.py	268	+ return buckets
272	---- a/llm_rl/qwen3/verl/utils/seqlen_balancing.py	269	+
273	-+++ b/llm_rl/qwen3/verl/utils/seqlen_balancing.py	270	+
		271	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/utils/seqlen_balancing.py b/llm_rl/qwen3/verl-mindspeed/verl/utils/seqlen_balancing.py
		272	+--- a/llm_rl/qwen3/verl-mindspeed/verl/utils/seqlen_balancing.py
		273	++++ b/llm_rl/qwen3/verl-mindspeed/verl/utils/seqlen_balancing.py
274	@@ -24,7 +24,7 @@	274	@@ -24,7 +24,7 @@
275	from verl.utils import tensordict_utils as tu	275	from verl.utils import tensordict_utils as tu
276	from verl.utils.device import get_device_name	276	from verl.utils.device import get_device_name
277	-	277	+
278	-from patches.verl.utils.hybrid_data_parallel.hdp import pack_sequences_into_buckets	278	-from patches.verl.utils.hybrid_data_parallel.hdp import pack_sequences_into_buckets
279	+from patches.verl.utils.hybrid_data_parallel.hdp import pack_sequences_into_buckets, set_hdp_max_token_len	279	+from patches.verl.utils.hybrid_data_parallel.hdp import pack_sequences_into_buckets, set_hdp_max_token_len
280	-	280	+
281	-	281	+
282	def karmarkar_karp(seqlen_list: list[int], k_partitions: int, equal_size: bool):	282	def karmarkar_karp(seqlen_list: list[int], k_partitions: int, equal_size: bool):
283	@@ -259,6 +259,7 @@	283	@@ -259,6 +259,7 @@
284	same_micro_num_in_dp=True,	284	same_micro_num_in_dp=True,
	@@ -293,7 +293,7 @@ diff --git a/llm_rl/qwen3/verl/utils/seqlen_balancing.py b/llm_rl/qwen3/verl/uti
293	min_num_micro_batch (int, optional): force at least this many splits (pads empty ones).	293	min_num_micro_batch (int, optional): force at least this many splits (pads empty ones).
294	use_dynamic_bsz_balance (bool, optional): balance the computational workload between micro-batches	294	use_dynamic_bsz_balance (bool, optional): balance the computational workload between micro-batches
295	+ rank_overload_scale (float, optional): Safety/margin multiplier applied to max_token_len (default: 1.18).	295	+ rank_overload_scale (float, optional): Safety/margin multiplier applied to max_token_len (default: 1.18).
296	-	296	+
297	Returns:	297	Returns:
298	List[TensorDict]: the micro-batches.	298	List[TensorDict]: the micro-batches.
299	@@ -289,25 +291,44 @@	299	@@ -289,25 +291,44 @@
	@@ -314,7 +314,7 @@ diff --git a/llm_rl/qwen3/verl/utils/seqlen_balancing.py b/llm_rl/qwen3/verl/uti
314	-	314	-
315	seq_len_effective = seq_len_effective.tolist()	315	seq_len_effective = seq_len_effective.tolist()
316	- assert num_micro_batches <= len(seq_len_effective)	316	- assert num_micro_batches <= len(seq_len_effective)
317	-	317	+
318	if os.environ.get("USE_HDP") == "1":	318	if os.environ.get("USE_HDP") == "1":
319	- max_token_len = ceildiv(total_seqlen, num_micro_batches)	319	- max_token_len = ceildiv(total_seqlen, num_micro_batches)
320	- micro_bsz_idx = pack_sequences_into_buckets(seq_len_effective, max_token_len, num_micro_batches)	320	- micro_bsz_idx = pack_sequences_into_buckets(seq_len_effective, max_token_len, num_micro_batches)
	@@ -354,5 +354,5 @@ diff --git a/llm_rl/qwen3/verl/utils/seqlen_balancing.py b/llm_rl/qwen3/verl/uti
354	+	354	+
355	+ assert num_micro_batches <= len(seq_len_effective)	355	+ assert num_micro_batches <= len(seq_len_effective)
356	micro_bsz_idx = get_seqlen_balanced_partitions(seq_len_effective, num_micro_batches, equal_size=False)	356	micro_bsz_idx = get_seqlen_balanced_partitions(seq_len_effective, num_micro_batches, equal_size=False)
357	-	357	+
358	if use_dynamic_bsz_balance:	358	if use_dynamic_bsz_balance:

Rllm_rl/qwen3/patches/verl/0016-verl-bugfix-hot_swap_expandable_segments.patch→llm_rl/qwen3/verl-mindspeed/patches/verl/0016-verl-bugfix-hot_swap_expandable_segments.patch+22-22

	@@ -1,22 +1,22 @@
1	-From 6ba7750e196cdd92001ccc3584f1743e1b950ba8 Mon Sep 17 00:00:00 2001	1	+From 6ba7750e196cdd92001ccc3584f1743e1b950ba8 Mon Sep 17 00:00:00 2001
2	-From: huyuanquan1 <huyuanquan1@huawei.com>	2	+From: huyuanquan1 <huyuanquan1@huawei.com>
3	-Date: Tue, 30 Dec 2025 21:36:23 +0800	3	+Date: Tue, 30 Dec 2025 21:36:23 +0800
4	-Subject: [PATCH] enable hot swap for expandable segments	4	+Subject: [PATCH] enable hot swap for expandable segments
5	-	5	+
6	----	6	+---
7	- verl/utils/device.py \| 2 ++	7	+ verl/utils/device.py \| 2 ++
8	- 1 file changed, 2 insertions(+)	8	+ 1 file changed, 2 insertions(+)
9	-	9	+
10	-diff --git a/llm_rl/qwen3/verl/utils/device.py b/llm_rl/qwen3/verl/utils/device.py	10	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/utils/device.py b/llm_rl/qwen3/verl-mindspeed/verl/utils/device.py
11	-index 28695ddb..04522217 100644	11	+index 28695ddb..04522217 100644
12	---- a/llm_rl/qwen3/verl/utils/device.py	12	+--- a/llm_rl/qwen3/verl-mindspeed/verl/utils/device.py
13	-+++ b/llm_rl/qwen3/verl/utils/device.py	13	++++ b/llm_rl/qwen3/verl-mindspeed/verl/utils/device.py
14	-@@ -93,3 +93,5 @@ def set_expandable_segments(enable: bool) -> None:	14	+@@ -93,3 +93,5 @@ def set_expandable_segments(enable: bool) -> None:
15	- """	15	+ """
16	- if is_cuda_available:	16	+ if is_cuda_available:
17	- torch.cuda.memory._set_allocator_settings(f"expandable_segments:{enable}")	17	+ torch.cuda.memory._set_allocator_settings(f"expandable_segments:{enable}")
18	-+ if is_npu_available:	18	++ if is_npu_available:
19	-+ torch.npu.memory._set_allocator_settings(f"expandable_segments:{enable}")	19	++ torch.npu.memory._set_allocator_settings(f"expandable_segments:{enable}")
20	---	20	+--
21	-2.45.1.windows.1	21	+2.45.1.windows.1
22	-	22	+

Rllm_rl/qwen3/patches/verl/0017-verl-bugfix-adapt_new_vllm_version.patch→llm_rl/qwen3/verl-mindspeed/patches/verl/0017-verl-bugfix-adapt_new_vllm_version.patch+44-44

	@@ -1,44 +1,44 @@
1	-From d85ac51d3414237ab519fd6ada5249be113e0be6 Mon Sep 17 00:00:00 2001	1	+From d85ac51d3414237ab519fd6ada5249be113e0be6 Mon Sep 17 00:00:00 2001
2	-From: huyuanquan1 <huyuanquan1@huawei.com>	2	+From: huyuanquan1 <huyuanquan1@huawei.com>
3	-Date: Tue, 6 Jan 2026 00:21:59 +0800	3	+Date: Tue, 6 Jan 2026 00:21:59 +0800
4	-Subject: [PATCH] adapt vllm 0.13.0	4	+Subject: [PATCH] adapt vllm 0.13.0
5	-	5	+
6	----	6	+---
7	- verl/utils/vllm/utils.py \| 7 ++++++-	7	+ verl/utils/vllm/utils.py \| 7 ++++++-
8	- verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py \| 2 +-	8	+ verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py \| 2 +-
9	- 2 files changed, 7 insertions(+), 2 deletions(-)	9	+ 2 files changed, 7 insertions(+), 2 deletions(-)
10	-	10	+
11	-diff --git a/llm_rl/qwen3/verl/utils/vllm/utils.py b/llm_rl/qwen3/verl/utils/vllm/utils.py	11	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/utils/vllm/utils.py b/llm_rl/qwen3/verl-mindspeed/verl/utils/vllm/utils.py
12	-index acf24398..751791e0 100644	12	+index acf24398..751791e0 100644
13	---- a/llm_rl/qwen3/verl/utils/vllm/utils.py	13	+--- a/llm_rl/qwen3/verl-mindspeed/verl/utils/vllm/utils.py
14	-+++ b/llm_rl/qwen3/verl/utils/vllm/utils.py	14	++++ b/llm_rl/qwen3/verl-mindspeed/verl/utils/vllm/utils.py
15	-@@ -15,7 +15,12 @@	15	+@@ -15,7 +15,12 @@
16	-	16	+
17	- from msgspec import field	17	+ from msgspec import field
18	- from packaging import version as vs	18	+ from packaging import version as vs
19	--from vllm.lora.models import LoRAModel	19	+-from vllm.lora.models import LoRAModel
20	-+	20	++
21	-+try:	21	++try:
22	-+ from vllm.lora.lora_model import LoRAModel	22	++ from vllm.lora.lora_model import LoRAModel
23	-+except ImportError:	23	++except ImportError:
24	-+ from vllm.lora.models import LoRAModel	24	++ from vllm.lora.models import LoRAModel
25	-+	25	++
26	- from vllm.lora.request import LoRARequest	26	+ from vllm.lora.request import LoRARequest
27	- from vllm.lora.utils import get_adapter_absolute_path	27	+ from vllm.lora.utils import get_adapter_absolute_path
28	- from vllm.lora.worker_manager import LRUCacheWorkerLoRAManager	28	+ from vllm.lora.worker_manager import LRUCacheWorkerLoRAManager
29	-diff --git a/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py b/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	29	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py b/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
30	-index b126fdbe..786e2d77 100644	30	+index b126fdbe..786e2d77 100644
31	---- a/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	31	+--- a/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
32	-+++ b/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	32	++++ b/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
33	-@@ -50,7 +50,7 @@ from omegaconf import ListConfig	33	+@@ -50,7 +50,7 @@ from omegaconf import ListConfig
34	- from tensordict import TensorDict	34	+ from tensordict import TensorDict
35	- from torch.distributed.device_mesh import DeviceMesh	35	+ from torch.distributed.device_mesh import DeviceMesh
36	- from vllm import LLM, SamplingParams	36	+ from vllm import LLM, SamplingParams
37	--from vllm.config import CompilationConfig, CompilationLevel, LoRAConfig	37	+-from vllm.config import CompilationConfig, CompilationLevel, LoRAConfig
38	-+from vllm.config.lora import LoRAConfig	38	++from vllm.config.lora import LoRAConfig
39	- from vllm.lora.request import LoRARequest	39	+ from vllm.lora.request import LoRARequest
40	- from vllm_ascend.ascend_config import get_ascend_config	40	+ from vllm_ascend.ascend_config import get_ascend_config
41	-	41	+
42	---	42	+--
43	-2.45.1.windows.1	43	+2.45.1.windows.1
44	-	44	+

Rllm_rl/qwen3/patches/verl/0018-verl-bugfix-ignore_redundant_logs.patch→llm_rl/qwen3/verl-mindspeed/patches/verl/0018-verl-bugfix-ignore_redundant_logs.patch+29-29

	@@ -1,29 +1,29 @@
1	-From 32b2679e2bf8194204ee441badce6f3c401b31b1 Mon Sep 17 00:00:00 2001	1	+From 32b2679e2bf8194204ee441badce6f3c401b31b1 Mon Sep 17 00:00:00 2001
2	-From: huyuanquan1 <huyuanquan1@huawei.com>	2	+From: huyuanquan1 <huyuanquan1@huawei.com>
3	-Date: Fri, 27 Feb 2026 15:15:58 +0800	3	+Date: Fri, 27 Feb 2026 15:15:58 +0800
4	-Subject: [PATCH] ignore redundant logs	4	+Subject: [PATCH] ignore redundant logs
5	-	5	+
6	----	6	+---
7	- verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py \| 6 ++++++	7	+ verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py \| 6 ++++++
8	- 1 file changed, 6 insertions(+)	8	+ 1 file changed, 6 insertions(+)
9	-	9	+
10	-diff --git a/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py b/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	10	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py b/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
11	-index fc60fd1e..c28da03d 100644	11	+index fc60fd1e..c28da03d 100644
12	---- a/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	12	+--- a/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
13	-+++ b/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	13	++++ b/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
14	-@@ -74,6 +74,12 @@ from verl.workers.rollout.base import BaseRollout	14	+@@ -74,6 +74,12 @@ from verl.workers.rollout.base import BaseRollout
15	- logger = logging.getLogger(__file__)	15	+ logger = logging.getLogger(__file__)
16	- logger.setLevel(os.getenv("VERL_LOGGING_LEVEL", "WARN"))	16	+ logger.setLevel(os.getenv("VERL_LOGGING_LEVEL", "WARN"))
17	-	17	+
18	-+# ignore redundant logs	18	++# ignore redundant logs
19	-+import warnings	19	++import warnings
20	-+from numba.core.errors import NumbaPendingDeprecationWarning	20	++from numba.core.errors import NumbaPendingDeprecationWarning
21	-+warnings.filterwarnings("ignore", category=NumbaPendingDeprecationWarning)	21	++warnings.filterwarnings("ignore", category=NumbaPendingDeprecationWarning)
22	-+logging.getLogger("torch._dynamo").setLevel(logging.CRITICAL)	22	++logging.getLogger("torch._dynamo").setLevel(logging.CRITICAL)
23	-+	23	++
24	- # TODO	24	+ # TODO
25	- # 1. support pp in vllm	25	+ # 1. support pp in vllm
26	- # 2. passing tokenizer is not necessary? no encoding/decoding is happending here	26	+ # 2. passing tokenizer is not necessary? no encoding/decoding is happending here
27	---	27	+--
28	-2.45.1.windows.1	28	+2.45.1.windows.1
29	-	29	+

Rllm_rl/qwen3/patches/verl/0019-verl-feature-length_aware_resampler.patch→llm_rl/qwen3/verl-mindspeed/patches/verl/0019-verl-feature-length_aware_resampler.patch+18-18

	@@ -1,7 +1,7 @@
1	-diff --git a/llm_rl/qwen3/verl/experimental/dataset/length_bucket_sampler.py b/llm_rl/qwen3/verl/experimental/dataset/length_bucket_sampler.py	1	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/experimental/dataset/length_bucket_sampler.py b/llm_rl/qwen3/verl-mindspeed/verl/experimental/dataset/length_bucket_sampler.py
2	new file mode 100644	2	new file mode 100644
3	--- /dev/null	3	--- /dev/null
4	-+++ b/llm_rl/qwen3/verl/experimental/dataset/length_bucket_sampler.py	4	++++ b/llm_rl/qwen3/verl-mindspeed/verl/experimental/dataset/length_bucket_sampler.py
5	@@ -0,0 +1,126 @@	5	@@ -0,0 +1,126 @@
6	+# Copyright 2026 Bytedance Ltd. and/or its affiliates	6	+# Copyright 2026 Bytedance Ltd. and/or its affiliates
7	+#	7	+#
	@@ -129,13 +129,13 @@ new file mode 100644
129	+ values = np.asarray(values)	129	+ values = np.asarray(values)
130	+ return values	130	+ return values
131	+ return None	131	+ return None
132	-diff --git a/llm_rl/qwen3/verl/trainer/ppo/ray_trainer.py b/llm_rl/qwen3/verl/trainer/ppo/ray_trainer.py	132	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/trainer/ppo/ray_trainer.py b/llm_rl/qwen3/verl-mindspeed/verl/trainer/ppo/ray_trainer.py
133	---- a/llm_rl/qwen3/verl/trainer/ppo/ray_trainer.py	133	+--- a/llm_rl/qwen3/verl-mindspeed/verl/trainer/ppo/ray_trainer.py
134	-+++ b/llm_rl/qwen3/verl/trainer/ppo/ray_trainer.py	134	++++ b/llm_rl/qwen3/verl-mindspeed/verl/trainer/ppo/ray_trainer.py
135	@@ -341,11 +341,22 @@	135	@@ -341,11 +341,22 @@
136	-	136	+
137	self._create_dataloader(train_dataset, val_dataset, collate_fn, train_sampler)	137	self._create_dataloader(train_dataset, val_dataset, collate_fn, train_sampler)
138	-	138	+
139	+ self.rollout_early_stop_enable = os.getenv("VLLM_ROLLOUT_EARLY_STOP_ENABLE", "1").lower() in (	139	+ self.rollout_early_stop_enable = os.getenv("VLLM_ROLLOUT_EARLY_STOP_ENABLE", "1").lower() in (
140	+ "1",	140	+ "1",
141	+ "true",	141	+ "true",
	@@ -153,7 +153,7 @@ diff --git a/llm_rl/qwen3/verl/trainer/ppo/ray_trainer.py b/llm_rl/qwen3/verl/tr
153	+ from torch.utils.data import Subset	153	+ from torch.utils.data import Subset
154	+	154	+
155	from verl.trainer.main_ppo import create_rl_dataset, create_rl_sampler	155	from verl.trainer.main_ppo import create_rl_dataset, create_rl_sampler
156	-	156	+
157	if train_dataset is None:	157	if train_dataset is None:
158	@@ -356,6 +367,24 @@	158	@@ -356,6 +367,24 @@
159	val_dataset = create_rl_dataset(	159	val_dataset = create_rl_dataset(
	@@ -178,12 +178,12 @@ diff --git a/llm_rl/qwen3/verl/trainer/ppo/ray_trainer.py b/llm_rl/qwen3/verl/tr
178	+ # Reset it so we rebuild a sampler that matches the trimmed dataset.	178	+ # Reset it so we rebuild a sampler that matches the trimmed dataset.
179	+ train_sampler = None	179	+ train_sampler = None
180	self.train_dataset, self.val_dataset = train_dataset, val_dataset	180	self.train_dataset, self.val_dataset = train_dataset, val_dataset
181	-	181	+
182	if train_sampler is None:	182	if train_sampler is None:
183	@@ -516,6 +541,41 @@	183	@@ -516,6 +541,41 @@
184	-	184	+
185	return gen_batch	185	return gen_batch
186	-	186	+
187	+ def _maybe_compute_rollout_response_cap(self, batch: DataProto) -> Optional[int]:	187	+ def _maybe_compute_rollout_response_cap(self, batch: DataProto) -> Optional[int]:
188	+ if not self.rollout_early_stop_enable:	188	+ if not self.rollout_early_stop_enable:
189	+ return None	189	+ return None
	@@ -232,9 +232,9 @@ diff --git a/llm_rl/qwen3/verl/trainer/ppo/ray_trainer.py b/llm_rl/qwen3/verl/tr
232	+ metrics["rollout/response_max_tokens_cap"] = int(response_cap)	232	+ metrics["rollout/response_max_tokens_cap"] = int(response_cap)
233	+	233	+
234	# pass global_steps to trace	234	# pass global_steps to trace
235	-diff --git a/llm_rl/qwen3/verl/utils/dataset/rl_dataset.py b/llm_rl/qwen3/verl/utils/dataset/rl_dataset.py	235	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/utils/dataset/rl_dataset.py b/llm_rl/qwen3/verl-mindspeed/verl/utils/dataset/rl_dataset.py
236	---- a/llm_rl/qwen3/verl/utils/dataset/rl_dataset.py	236	+--- a/llm_rl/qwen3/verl-mindspeed/verl/utils/dataset/rl_dataset.py
237	-+++ b/llm_rl/qwen3/verl/utils/dataset/rl_dataset.py	237	++++ b/llm_rl/qwen3/verl-mindspeed/verl/utils/dataset/rl_dataset.py
238	@@ -368,6 +368,9 @@	238	@@ -368,6 +368,9 @@
239	if need_tools_kwargs and not tools_kwargs:	239	if need_tools_kwargs and not tools_kwargs:
240	logger.warning("tools_kwargs is empty for index {}, data source: {}", index, row_dict["data_source"])	240	logger.warning("tools_kwargs is empty for index {}, data source: {}", index, row_dict["data_source"])
	@@ -245,13 +245,13 @@ diff --git a/llm_rl/qwen3/verl/utils/dataset/rl_dataset.py b/llm_rl/qwen3/verl/u
245	row_dict["tools_kwargs"] = tools_kwargs	245	row_dict["tools_kwargs"] = tools_kwargs
246	row_dict["interaction_kwargs"] = interaction_kwargs	246	row_dict["interaction_kwargs"] = interaction_kwargs
247	return row_dict	247	return row_dict
248	-diff --git a/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py b/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	248	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py b/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
249	---- a/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	249	+--- a/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
250	-+++ b/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	250	++++ b/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
251	@@ -387,6 +387,17 @@	251	@@ -387,6 +387,17 @@
252	"n": 1, # if validate, already repeat in ray_trainer	252	"n": 1, # if validate, already repeat in ray_trainer
253	}	253	}
254	-	254	+
255	+ response_cap = prompts.meta_info.get("response_max_tokens_cap", None)	255	+ response_cap = prompts.meta_info.get("response_max_tokens_cap", None)
256	+ if response_cap is not None:	256	+ if response_cap is not None:
257	+ try:	257	+ try:

Rllm_rl/qwen3/patches/verl/features/__init__.py→llm_rl/qwen3/verl-mindspeed/patches/verl/features/__init__.py+0-0

文件重命名但无更改。

Rllm_rl/qwen3/patches/verl/features/rollout_optimize/__init__.py→llm_rl/qwen3/verl-mindspeed/patches/verl/features/rollout_optimize/__init__.py+0-0

文件重命名但无更改。

Rllm_rl/qwen3/patches/verl/features/rollout_optimize/config.py→llm_rl/qwen3/verl-mindspeed/patches/verl/features/rollout_optimize/config.py+0-0

文件重命名但无更改。

Rllm_rl/qwen3/patches/verl/features/rollout_optimize/rollout_rebalance.py→llm_rl/qwen3/verl-mindspeed/patches/verl/features/rollout_optimize/rollout_rebalance.py+0-0

文件重命名但无更改。

Rllm_rl/qwen3/patches/verl/features/rollout_optimize/utils.py→llm_rl/qwen3/verl-mindspeed/patches/verl/features/rollout_optimize/utils.py+0-0

文件重命名但无更改。

Rllm_rl/qwen3/patches/verl/utils/hybrid_data_parallel/hdp.py→llm_rl/qwen3/verl-mindspeed/patches/verl/utils/hybrid_data_parallel/hdp.py+28-28

	@@ -30,37 +30,37 @@ from patches.verl.utils.hybrid_data_parallel.utils import set_batch_hdp_group, g
30		30
31		31
32	def generate_hdp_group_from_batch(	32	def generate_hdp_group_from_batch(
33	- atten_mask: torch.Tensor,	33	+ atten_mask: torch.Tensor,
34	- per_rank_overload_threshold=0.1,	34	+ per_rank_overload_threshold=0.1,
35	fractional_roundup_threshold=0.9,	35	fractional_roundup_threshold=0.9,
36	max_fraction_threshold=0.5	36	max_fraction_threshold=0.5
37	):	37	):
38	"""	38	"""
39	Generate HDP groups for micro-batch to optimize Ring Attention communication in CP.	39	Generate HDP groups for micro-batch to optimize Ring Attention communication in CP.
40	-	40	+
41	Args:	41	Args:
42	atten_mask: Attention mask tensor of shape [micro_batch_size, sequence_length]	42	atten_mask: Attention mask tensor of shape [micro_batch_size, sequence_length]
43	- per_rank_overload_threshold: Maximum allowed overload per rank. For sequences already assigned	43	+ per_rank_overload_threshold: Maximum allowed overload per rank. For sequences already assigned
44	- to N ranks, if the fractional part is less than N * this threshold,	44	+ to N ranks, if the fractional part is less than N * this threshold,
45	the fractional tokens can be absorbed without adding a new rank.	45	the fractional tokens can be absorbed without adding a new rank.
46	- Example: sequence needs 2.1 ranks with threshold=0.1 → assigned to 2 ranks.	46	+ Example: sequence needs 2.1 ranks with threshold=0.1 -> assigned to 2 ranks.
47	- fractional_roundup_threshold: Threshold for rounding up fractional rank requirements. When the	47	+ fractional_roundup_threshold: Threshold for rounding up fractional rank requirements. When the
48	fractional part exceeds this value, round up to the next integer.	48	fractional part exceeds this value, round up to the next integer.
49	- Example: sequence needs 1.9 ranks with threshold=0.9 → assigned to 2 ranks.	49	+ Example: sequence needs 1.9 ranks with threshold=0.9 -> assigned to 2 ranks.
50	max_fraction_threshold: Maximum fractional overload allowed regardless of per-rank threshold.	50	max_fraction_threshold: Maximum fractional overload allowed regardless of per-rank threshold.
51	Prevents excessive overload when sequences are assigned many ranks.	51	Prevents excessive overload when sequences are assigned many ranks.
52	-	52	+
53	Returns:	53	Returns:
54	List of lists containing rank indices assigned to each sequence	54	List of lists containing rank indices assigned to each sequence
55	Example: [[0,1], [2], [3]] means sequence 0 uses 2 ranks, sequences 1&2 use 1 rank each	55	Example: [[0,1], [2], [3]] means sequence 0 uses 2 ranks, sequences 1&2 use 1 rank each
56	-	56	+
57	Example:	57	Example:
58	For sequences [1024, 512, 512] with 4 ranks:	58	For sequences [1024, 512, 512] with 4 ranks:
59	- - seq0: (1024/2048)*4 = 2.0 ranks → assigned to ranks [0,1]	59	+ - seq0: (1024/2048)*4 = 2.0 ranks -> assigned to ranks [0,1]
60	- - seq1: (512/2048)*4 = 1.0 rank → assigned to rank [2]	60	+ - seq1: (512/2048)*4 = 1.0 rank -> assigned to rank [2]
61	- - seq2: (512/2048)*4 = 1.0 rank → assigned to rank [3]	61	+ - seq2: (512/2048)*4 = 1.0 rank -> assigned to rank [3]
62	Result: [[0,1], [2], [3]]	62	Result: [[0,1], [2], [3]]
63	-	63	+
64	Benefits:	64	Benefits:
65	- seq0: communicates between ranks 0 and 1 only (internal Ring Attention)	65	- seq0: communicates between ranks 0 and 1 only (internal Ring Attention)
66	- seq1, seq2: no cross-rank communication needed (completely local to rank 2 and rank 3 respectively)	66	- seq1, seq2: no cross-rank communication needed (completely local to rank 2 and rank 3 respectively)
	@@ -92,7 +92,7 @@ def generate_hdp_group_from_batch(
92	if ranks >= 1:	92	if ranks >= 1:
93	batch_hdp_group[i] = list(range(start_rank, start_rank + int(ranks)))	93	batch_hdp_group[i] = list(range(start_rank, start_rank + int(ranks)))
94	start_rank += int(ranks)	94	start_rank += int(ranks)
95	- if frac < min(per_rank_overload_threshold * int(ranks), max_fraction_threshold):	95	+ if frac < min(per_rank_overload_threshold * int(ranks), max_fraction_threshold):
96	continue	96	continue
97	if frac > fractional_roundup_threshold:	97	if frac > fractional_roundup_threshold:
98	batch_hdp_group[i].append(start_rank)	98	batch_hdp_group[i].append(start_rank)
	@@ -126,13 +126,13 @@ def generate_hdp_group_from_batch(
126	def check_load_balance(batch_hdp_group, ranks_per_seq, max_load_imbalance_threshold=1.3):	126	def check_load_balance(batch_hdp_group, ranks_per_seq, max_load_imbalance_threshold=1.3):
127	"""	127	"""
128	Check load balance across ranks in HDP grouping	128	Check load balance across ranks in HDP grouping
129	-	129	+
130	Args:	130	Args:
131	batch_hdp_group: HDP batch grouping information, indicating which ranks process each sequence	131	batch_hdp_group: HDP batch grouping information, indicating which ranks process each sequence
132	ranks_per_seq: number of ranks per sequence, representing computational load	132	ranks_per_seq: number of ranks per sequence, representing computational load
133	cp_size: total number of ranks in model parallelism	133	cp_size: total number of ranks in model parallelism
134	max_load_imbalance_threshold: maximum load imbalance threshold, default 1.3	134	max_load_imbalance_threshold: maximum load imbalance threshold, default 1.3
135	-	135	+
136	Returns:	136	Returns:
137	bool: True if all rank loads are within threshold, False otherwise	137	bool: True if all rank loads are within threshold, False otherwise
138	"""	138	"""
	@@ -154,7 +154,7 @@ def pack_frac(pairs, start_rank, batch_hdp_group):
154	Routes to appropriate packing strategy based on available ranks.	154	Routes to appropriate packing strategy based on available ranks.
155	"""	155	"""
156	cp_size = mpu.get_context_parallel_world_size()	156	cp_size = mpu.get_context_parallel_world_size()
157	-	157	+
158	if start_rank >= cp_size:	158	if start_rank >= cp_size:
159	return _pack_into_existing_groups(pairs, start_rank, batch_hdp_group)	159	return _pack_into_existing_groups(pairs, start_rank, batch_hdp_group)
160	else:	160	else:
	@@ -164,7 +164,7 @@ def pack_frac(pairs, start_rank, batch_hdp_group):
164	def _pack_into_existing_groups(pairs, start_rank, batch_hdp_group):	164	def _pack_into_existing_groups(pairs, start_rank, batch_hdp_group):
165	# Create generator for non-empty HDP groups with their original indices	165	# Create generator for non-empty HDP groups with their original indices
166	non_empty_group_index = (	166	non_empty_group_index = (
167	- (hdp_group.copy(), index)	167	+ (hdp_group.copy(), index)
168	for index, hdp_group in enumerate(batch_hdp_group)	168	for index, hdp_group in enumerate(batch_hdp_group)
169	if hdp_group	169	if hdp_group
170	)	170	)
	@@ -217,19 +217,19 @@ def _pack_into_new_ranks(pairs, start_rank, batch_hdp_group):
217	def pack_sequences_into_buckets(seqlen_list: list[int], max_bucket_length: int, num_buckets: int) -> list[list[int]]:	217	def pack_sequences_into_buckets(seqlen_list: list[int], max_bucket_length: int, num_buckets: int) -> list[list[int]]:
218	"""	218	"""
219	Pack sequences into buckets using greedy algorithm for subsequent HDP grouping.	219	Pack sequences into buckets using greedy algorithm for subsequent HDP grouping.
220	-	220	+
221	Args:	221	Args:
222	seqlen_list: List of sequence lengths to be packed	222	seqlen_list: List of sequence lengths to be packed
223	max_bucket_length: Maximum allowed total sequence length per bucket	223	max_bucket_length: Maximum allowed total sequence length per bucket
224	num_buckets: Number of available buckets for distribution	224	num_buckets: Number of available buckets for distribution
225	-	225	+
226	Returns:	226	Returns:
227	List of buckets, where each bucket contains the original indices of sequences assigned to it	227	List of buckets, where each bucket contains the original indices of sequences assigned to it
228	-	228	+
229	"""	229	"""
230	# Sort by length in descending order, preserving original indices	230	# Sort by length in descending order, preserving original indices
231	indexed_seqlens = sorted(enumerate(seqlen_list), key=lambda x: -x[1])	231	indexed_seqlens = sorted(enumerate(seqlen_list), key=lambda x: -x[1])
232	-	232	+
233	buckets = [[] for _ in range(num_buckets)]	233	buckets = [[] for _ in range(num_buckets)]
234	bucket_sums = [0] * num_buckets	234	bucket_sums = [0] * num_buckets
235		235
	@@ -270,11 +270,11 @@ def preprocess_packed_seqs_hdp(
270	attention_mask: torch.Tensor,	270	attention_mask: torch.Tensor,
271	pre_process: bool = True	271	pre_process: bool = True
272	) -> tuple[torch.Tensor, PackedSeqParams]:	272	) -> tuple[torch.Tensor, PackedSeqParams]:
273	-	273	+
274	- batch_hdp_group = get_batch_hdp_group()	274	+ batch_hdp_group = get_batch_hdp_group()
275	if batch_hdp_group is None:	275	if batch_hdp_group is None:
276	return preprocess_packed_seqs(input_ids, attention_mask, pre_process)	276	return preprocess_packed_seqs(input_ids, attention_mask, pre_process)
277	-	277	+
278	cp_rank = mpu.get_context_parallel_rank()	278	cp_rank = mpu.get_context_parallel_rank()
279	indices, local_group = find_group(batch_hdp_group, cp_rank)	279	indices, local_group = find_group(batch_hdp_group, cp_rank)
280	batch_size = len(indices)	280	batch_size = len(indices)
	@@ -311,7 +311,7 @@ def preprocess_packed_seqs_hdp(
311	if pre_process:	311	if pre_process:
312	input_ids_rmpad = torch.zeros(shape, dtype=input_ids.dtype, device=input_ids.device)	312	input_ids_rmpad = torch.zeros(shape, dtype=input_ids.dtype, device=input_ids.device)
313	for i in range(batch_size):	313	for i in range(batch_size):
314	- # Use Python int, so no GPU→CPU sync in the loop	314	+ # Use Python int, so no GPU->CPU sync in the loop
315	if hdp_size <= 1:	315	if hdp_size <= 1:
316	seqlen = seqlens_in_batch_cpu[i]	316	seqlen = seqlens_in_batch_cpu[i]
317	start_idx = cu_seqlens_padded_cpu[i]	317	start_idx = cu_seqlens_padded_cpu[i]
	@@ -378,7 +378,7 @@ def postprocess_packed_seqs_hdp(
378	output_new = torch.zeros(shape, dtype=output.dtype, device=output.device)	378	output_new = torch.zeros(shape, dtype=output.dtype, device=output.device)
379	batch_hdp_group = get_batch_hdp_group()	379	batch_hdp_group = get_batch_hdp_group()
380	if batch_hdp_group is None:	380	if batch_hdp_group is None:
381	- return postprocess_packed_seqs(output, packed_seq_params, attention_mask, batch_size, seq_len,	381	+ return postprocess_packed_seqs(output, packed_seq_params, attention_mask, batch_size, seq_len,
382	post_process=post_process)	382	post_process=post_process)
383	cp_size = mpu.get_context_parallel_world_size()	383	cp_size = mpu.get_context_parallel_world_size()
384	if cp_size > 1:	384	if cp_size > 1:

Rllm_rl/qwen3/patches/verl/utils/hybrid_data_parallel/utils.py→llm_rl/qwen3/verl-mindspeed/patches/verl/utils/hybrid_data_parallel/utils.py+0-0

文件重命名但无更改。

Rllm_rl/qwen3/patches/vllm/0001-vllm-feature-disable_gc.patch→llm_rl/qwen3/verl-mindspeed/patches/vllm/0001-vllm-feature-disable_gc.patch+36-36

	@@ -1,36 +1,36 @@
1	-From d3f41b02ca14e0c0a6a40d5aec4ce59f1d4dab4e Mon Sep 17 00:00:00 2001	1	+From d3f41b02ca14e0c0a6a40d5aec4ce59f1d4dab4e Mon Sep 17 00:00:00 2001
2	-From: caojingyi <caojingyi@noreply.gitcode.com>	2	+From: caojingyi <caojingyi@noreply.gitcode.com>
3	-Date: Tue, 11 Nov 2025 11:40:55 +0800	3	+Date: Tue, 11 Nov 2025 11:40:55 +0800
4	-Subject: [PATCH 11/18] Update vllm: disable gc during the decode step	4	+Subject: [PATCH 11/18] Update vllm: disable gc during the decode step
5	-Disable garbage collect(gc) before decode steps to avoid host bound caused by memory management.	5	+Disable garbage collect(gc) before decode steps to avoid host bound caused by memory management.
6	-Restore gc after completion.	6	+Restore gc after completion.
7	----	7	+---
8	- llm_rl/qwen3/vllm/entrypoints/llm.py \| 4 ++++	8	+ llm_rl/qwen3/verl-mindspeed/vllm/entrypoints/llm.py \| 4 ++++
9	- 1 file changed, 4 insertions(+)	9	+ 1 file changed, 4 insertions(+)
10	-	10	+
11	-diff --git a/llm_rl/qwen3/vllm/entrypoints/llm.py b/llm_rl/qwen3/vllm/entrypoints/llm.py	11	+diff --git a/llm_rl/qwen3/verl-mindspeed/vllm/entrypoints/llm.py b/llm_rl/qwen3/verl-mindspeed/vllm/entrypoints/llm.py
12	-index 862f383..da97046 100644	12	+index 862f383..da97046 100644
13	---- a/llm_rl/qwen3/vllm/entrypoints/llm.py	13	+--- a/llm_rl/qwen3/verl-mindspeed/vllm/entrypoints/llm.py
14	-+++ b/llm_rl/qwen3/vllm/entrypoints/llm.py	14	++++ b/llm_rl/qwen3/verl-mindspeed/vllm/entrypoints/llm.py
15	-@@ -6,6 +6,7 @@ from collections.abc import Sequence	15	+@@ -6,6 +6,7 @@ from collections.abc import Sequence
16	- from typing import TYPE_CHECKING, Any, cast	16	+ from typing import TYPE_CHECKING, Any, cast
17	-	17	+
18	- import cloudpickle	18	+ import cloudpickle
19	-+import gc	19	++import gc
20	- import torch.nn as nn	20	+ import torch.nn as nn
21	- from pydantic import ValidationError	21	+ from pydantic import ValidationError
22	- from tqdm.auto import tqdm	22	+ from tqdm.auto import tqdm
23	-@@ -1761,7 +1762,10 @@ class LLM:	23	+@@ -1761,7 +1762,10 @@ class LLM:
24	- total_in_toks = 0	24	+ total_in_toks = 0
25	- total_out_toks = 0	25	+ total_out_toks = 0
26	- while self.llm_engine.has_unfinished_requests():	26	+ while self.llm_engine.has_unfinished_requests():
27	-+ original_threshold = gc.get_threshold()	27	++ original_threshold = gc.get_threshold()
28	-+ gc.set_threshold(0)	28	++ gc.set_threshold(0)
29	- step_outputs = self.llm_engine.step()	29	+ step_outputs = self.llm_engine.step()
30	-+ gc.set_threshold(*original_threshold)	30	++ gc.set_threshold(*original_threshold)
31	- for output in step_outputs:	31	+ for output in step_outputs:
32	- if output.finished:	32	+ if output.finished:
33	- outputs.append(output)	33	+ outputs.append(output)
34	---	34	+--
35	-2.50.1.windows.1	35	+2.50.1.windows.1
36	-	36	+

Rllm_rl/qwen3/patches/vllm/0002-vllm-feature-enable_sam_decoding.patch→llm_rl/qwen3/verl-mindspeed/patches/vllm/0002-vllm-feature-enable_sam_decoding.patch+7-7

	@@ -4,13 +4,13 @@ Date: Mon, 26 Jan 2026 19:50:53 +0800
4	Subject: [PATCH] enable sam decoding in vllm	4	Subject: [PATCH] enable sam decoding in vllm
5		5
6	---	6	---
7	- llm_rl/qwen3/vllm/config/speculative.py \| 5 ++++-	7	+ llm_rl/qwen3/verl-mindspeed/vllm/config/speculative.py \| 5 ++++-
8	1 file changed, 4 insertions(+), 1 deletion(-)	8	1 file changed, 4 insertions(+), 1 deletion(-)
9		9
10	-diff --git a/llm_rl/qwen3/vllm/config/speculative.py b/llm_rl/qwen3/vllm/config/speculative.py	10	+diff --git a/llm_rl/qwen3/verl-mindspeed/vllm/config/speculative.py b/llm_rl/qwen3/verl-mindspeed/vllm/config/speculative.py
11	index cba00da..cd4cced 100644	11	index cba00da..cd4cced 100644
12	---- a/llm_rl/qwen3/vllm/config/speculative.py	12	+--- a/llm_rl/qwen3/verl-mindspeed/vllm/config/speculative.py
13	-+++ b/llm_rl/qwen3/vllm/config/speculative.py	13	++++ b/llm_rl/qwen3/verl-mindspeed/vllm/config/speculative.py
14	@@ -46,6 +46,7 @@ SpeculativeMethod = Literal[	14	@@ -46,6 +46,7 @@ SpeculativeMethod = Literal[
15	"mlp_speculator",	15	"mlp_speculator",
16	"draft_model",	16	"draft_model",
	@@ -18,7 +18,7 @@ index cba00da..cd4cced 100644
18	+ "sam",	18	+ "sam",
19	EagleModelTypes,	19	EagleModelTypes,
20	]	20	]
21	-	21	+
22	@@ -274,6 +275,8 @@ class SpeculativeConfig:	22	@@ -274,6 +275,8 @@ class SpeculativeConfig:
23	self.model = "ngram"	23	self.model = "ngram"
24	elif self.method == "suffix":	24	elif self.method == "suffix":
	@@ -31,12 +31,12 @@ index cba00da..cd4cced 100644
31	@@ -286,7 +289,7 @@ class SpeculativeConfig:	31	@@ -286,7 +289,7 @@ class SpeculativeConfig:
32	):	32	):
33	self.method = "ngram"	33	self.method = "ngram"
34	-	34	+
35	- if self.method in ("ngram", "[ngram]"):	35	- if self.method in ("ngram", "[ngram]"):
36	+ if self.method in ("ngram", "[ngram]", "sam"):	36	+ if self.method in ("ngram", "[ngram]", "sam"):
37	# Unified to "ngram" internally	37	# Unified to "ngram" internally
38	self.method = "ngram"	38	self.method = "ngram"
39	# Set default values if not provided	39	# Set default values if not provided
40	---	40	+--
41	2.45.1.windows.1	41	2.45.1.windows.1
42		42

Rllm_rl/qwen3/patches/vllm/0003-vllm-bugfix-rope_registry.patch→llm_rl/qwen3/verl-mindspeed/patches/vllm/0003-vllm-bugfix-rope_registry.patch+25-25

Rllm_rl/qwen3/patches/vllm_ascend/0001-vllm_ascend-feature-bs_threshold_for_spec_decode.patch→llm_rl/qwen3/verl-mindspeed/patches/vllm_ascend/0001-vllm_ascend-feature-bs_threshold_for_spec_decode.patch+11-11

	@@ -7,24 +7,24 @@ Subject: [PATCH] Add batch size auto switch for spec decoding
7	.../vllm_ascend/worker/model_runner_v1.py \| 57 +++++++++++++++----	7	.../vllm_ascend/worker/model_runner_v1.py \| 57 +++++++++++++++----
8	1 file changed, 47 insertions(+), 10 deletions(-)	8	1 file changed, 47 insertions(+), 10 deletions(-)
9		9
10	-diff --git a/llm_rl/qwen3/vllm_ascend/worker/model_runner_v1.py b/llm_rl/qwen3/vllm_ascend/worker/model_runner_v1.py	10	+diff --git a/llm_rl/qwen3/verl-mindspeed/vllm_ascend/worker/model_runner_v1.py b/llm_rl/qwen3/verl-mindspeed/vllm_ascend/worker/model_runner_v1.py
11	index 6d82372..b1107ab 100644	11	index 6d82372..b1107ab 100644
12	---- a/llm_rl/qwen3/vllm_ascend/worker/model_runner_v1.py	12	+--- a/llm_rl/qwen3/verl-mindspeed/vllm_ascend/worker/model_runner_v1.py
13	-+++ b/llm_rl/qwen3/vllm_ascend/worker/model_runner_v1.py	13	++++ b/llm_rl/qwen3/verl-mindspeed/vllm_ascend/worker/model_runner_v1.py
14	@@ -17,6 +17,7 @@	14	@@ -17,6 +17,7 @@
15	# Adapted from vllm-project/vllm/vllm/worker/gpu_model_runner.py	15	# Adapted from vllm-project/vllm/vllm/worker/gpu_model_runner.py
16	#	16	#
17	-	17	+
18	+import os	18	+import os
19	import math	19	import math
20	import sys	20	import sys
21	from collections import defaultdict	21	from collections import defaultdict
22	@@ -362,6 +363,10 @@ class NPUModelRunner(GPUModelRunner):	22	@@ -362,6 +363,10 @@ class NPUModelRunner(GPUModelRunner):
23	-	23	+
24	def _set_up_drafter(self):	24	def _set_up_drafter(self):
25	# Set up speculative decoding.	25	# Set up speculative decoding.
26	+ self.speculative_auto_bs_thre = int(os.environ.get('VLLM_SPECULATIVE_BATCH_SIZE_THRE', "-1"))	26	+ self.speculative_auto_bs_thre = int(os.environ.get('VLLM_SPECULATIVE_BATCH_SIZE_THRE', "-1"))
27	-+ # Indicates whether speculative decoding is active,	27	++ # Indicates whether speculative decoding is active,
28	+ # set at runtime based on the batch-size threshold.	28	+ # set at runtime based on the batch-size threshold.
29	+ self.speculative_decoding_active = False	29	+ self.speculative_decoding_active = False
30	self.drafter: Optional[Union[NgramProposer, EagleProposer, MtpProposer,	30	self.drafter: Optional[Union[NgramProposer, EagleProposer, MtpProposer,
	@@ -33,7 +33,7 @@ index 6d82372..b1107ab 100644
33	@@ -514,8 +519,7 @@ class NPUModelRunner(GPUModelRunner):	33	@@ -514,8 +519,7 @@ class NPUModelRunner(GPUModelRunner):
34	tokens = [scheduler_output.num_scheduled_tokens[i] for i in req_ids]	34	tokens = [scheduler_output.num_scheduled_tokens[i] for i in req_ids]
35	num_scheduled_tokens = np.array(tokens, dtype=np.int32)	35	num_scheduled_tokens = np.array(tokens, dtype=np.int32)
36	-	36	+
37	- req_indices = np.repeat(self.arange_np[:num_reqs],	37	- req_indices = np.repeat(self.arange_np[:num_reqs],
38	- num_scheduled_tokens)	38	- num_scheduled_tokens)
39	+ max_num_scheduled_tokens = max(tokens)	39	+ max_num_scheduled_tokens = max(tokens)
	@@ -78,7 +78,7 @@ index 6d82372..b1107ab 100644
78	<= self.cudagraph_batch_sizes[-1]):	78	<= self.cudagraph_batch_sizes[-1]):
79	# Add padding to the batch size.	79	# Add padding to the batch size.
80	@@ -743,6 +768,9 @@ class NPUModelRunner(GPUModelRunner):	80	@@ -743,6 +768,9 @@ class NPUModelRunner(GPUModelRunner):
81	-	81	+
82	use_spec_decode = len(	82	use_spec_decode = len(
83	scheduler_output.scheduled_spec_decode_tokens) > 0	83	scheduler_output.scheduled_spec_decode_tokens) > 0
84	+ # Adaptive SD	84	+ # Adaptive SD
	@@ -117,10 +117,10 @@ index 6d82372..b1107ab 100644
117	+ num_actual_tokens=total_num_scheduled_tokens,	117	+ num_actual_tokens=total_num_scheduled_tokens,
118	model_instance=self.model):	118	model_instance=self.model):
119	self.maybe_setup_kv_connector(scheduler_output)	119	self.maybe_setup_kv_connector(scheduler_output)
120	-	120	+
121	@@ -1571,7 +1603,7 @@ class NPUModelRunner(GPUModelRunner):	121	@@ -1571,7 +1603,7 @@ class NPUModelRunner(GPUModelRunner):
122	)	122	)
123	-	123	+
124	with ProfileExecuteDuration().capture_async("Draft"):	124	with ProfileExecuteDuration().capture_async("Draft"):
125	- if self.speculative_config:	125	- if self.speculative_config:
126	+ if self.speculative_config and self.speculative_decoding_active and not self.with_prefill:	126	+ if self.speculative_config and self.speculative_decoding_active and not self.with_prefill:
	@@ -145,6 +145,6 @@ index 6d82372..b1107ab 100644
145	if self.dynamic_eplb:	145	if self.dynamic_eplb:
146	self.eplb_updator.forward_end()	146	self.eplb_updator.forward_end()
147	if not self.use_async_scheduling:	147	if not self.use_async_scheduling:
148	---	148	+--
149	2.45.1.windows.1	149	2.45.1.windows.1
150		150

Rllm_rl/qwen3/patches/vllm_ascend/0002-vllm_ascend-feature-enable_sam_decoding.patch→llm_rl/qwen3/verl-mindspeed/patches/vllm_ascend/0002-vllm_ascend-feature-enable_sam_decoding.patch+13-13

	@@ -4,21 +4,21 @@ Date: Mon, 26 Jan 2026 22:51:16 +0800
4	Subject: [PATCH] Integrate SAM decoding into vllm-ascend	4	Subject: [PATCH] Integrate SAM decoding into vllm-ascend
5		5
6	---	6	---
7	- llm_rl/qwen3/vllm_ascend/spec_decode/__init__.py \| 3 +++	7	+ llm_rl/qwen3/verl-mindspeed/vllm_ascend/spec_decode/__init__.py \| 3 +++
8	- llm_rl/qwen3/vllm_ascend/spec_decode/interface.py \| 1 +	8	+ llm_rl/qwen3/verl-mindspeed/vllm_ascend/spec_decode/interface.py \| 1 +
9	2 files changed, 4 insertions(+)	9	2 files changed, 4 insertions(+)
10		10
11	-diff --git a/llm_rl/qwen3/vllm_ascend/spec_decode/__init__.py b/llm_rl/qwen3/vllm_ascend/spec_decode/__init__.py	11	+diff --git a/llm_rl/qwen3/verl-mindspeed/vllm_ascend/spec_decode/__init__.py b/llm_rl/qwen3/verl-mindspeed/vllm_ascend/spec_decode/__init__.py
12	index 89f558d..fbefb77 100644	12	index 89f558d..fbefb77 100644
13	---- a/llm_rl/qwen3/vllm_ascend/spec_decode/__init__.py	13	+--- a/llm_rl/qwen3/verl-mindspeed/vllm_ascend/spec_decode/__init__.py
14	-+++ b/llm_rl/qwen3/vllm_ascend/spec_decode/__init__.py	14	++++ b/llm_rl/qwen3/verl-mindspeed/vllm_ascend/spec_decode/__init__.py
15	@@ -21,6 +21,7 @@ from vllm_ascend.spec_decode.medusa_proposer import MedusaProposer	15	@@ -21,6 +21,7 @@ from vllm_ascend.spec_decode.medusa_proposer import MedusaProposer
16	from vllm_ascend.spec_decode.mtp_proposer import MtpProposer	16	from vllm_ascend.spec_decode.mtp_proposer import MtpProposer
17	from vllm_ascend.spec_decode.ngram_proposer import NgramProposer	17	from vllm_ascend.spec_decode.ngram_proposer import NgramProposer
18	from vllm_ascend.spec_decode.suffix_proposer import SuffixDecodingProposer	18	from vllm_ascend.spec_decode.suffix_proposer import SuffixDecodingProposer
19	+from patches.vllm_ascend.spec_decode.sam_proposer import SAMDecodingProposer	19	+from patches.vllm_ascend.spec_decode.sam_proposer import SAMDecodingProposer
20	-	20	+
21	-	21	+
22	def get_spec_decode_method(method, vllm_config, device, runner):	22	def get_spec_decode_method(method, vllm_config, device, runner):
23	@@ -34,6 +35,8 @@ def get_spec_decode_method(method, vllm_config, device, runner):	23	@@ -34,6 +35,8 @@ def get_spec_decode_method(method, vllm_config, device, runner):
24	return SuffixDecodingProposer(vllm_config, device, runner)	24	return SuffixDecodingProposer(vllm_config, device, runner)
	@@ -29,18 +29,18 @@ index 89f558d..fbefb77 100644
29	else:	29	else:
30	raise ValueError("Unknown speculative decoding method: "	30	raise ValueError("Unknown speculative decoding method: "
31	f"{method}")	31	f"{method}")
32	-diff --git a/llm_rl/qwen3/vllm_ascend/spec_decode/interface.py b/llm_rl/qwen3/vllm_ascend/spec_decode/interface.py	32	+diff --git a/llm_rl/qwen3/verl-mindspeed/vllm_ascend/spec_decode/interface.py b/llm_rl/qwen3/verl-mindspeed/vllm_ascend/spec_decode/interface.py
33	index feec5bc..968f75b 100644	33	index feec5bc..968f75b 100644
34	---- a/llm_rl/qwen3/vllm_ascend/spec_decode/interface.py	34	+--- a/llm_rl/qwen3/verl-mindspeed/vllm_ascend/spec_decode/interface.py
35	-+++ b/llm_rl/qwen3/vllm_ascend/spec_decode/interface.py	35	++++ b/llm_rl/qwen3/verl-mindspeed/vllm_ascend/spec_decode/interface.py
36	@@ -15,6 +15,7 @@ class SpecDcodeType(enum.Enum):	36	@@ -15,6 +15,7 @@ class SpecDcodeType(enum.Enum):
37	MTP = 4	37	MTP = 4
38	SUFFIX = 5	38	SUFFIX = 5
39	MEDUSA = 6	39	MEDUSA = 6
40	+ SAM = 7	40	+ SAM = 7
41	-	41	+
42	-	42	+
43	class Proposer:	43	class Proposer:
44	---	44	+--
45	2.45.1.windows.1	45	2.45.1.windows.1
46		46

Rllm_rl/qwen3/patches/vllm_ascend/0003-vllm_ascend-bugfix-set_hccl_op_expansion_mode.patch→llm_rl/qwen3/verl-mindspeed/patches/vllm_ascend/0003-vllm_ascend-bugfix-set_hccl_op_expansion_mode.patch+6-6

	@@ -4,13 +4,13 @@ Date: Tue, 27 Jan 2026 09:25:28 +0800
4	Subject: [PATCH] Avoided hccl timeout of all_gather-ing from multiple	4	Subject: [PATCH] Avoided hccl timeout of all_gather-ing from multiple
5		5
6	---	6	---
7	- llm_rl/qwen3/vllm_ascend/utils.py \| 1 +	7	+ llm_rl/qwen3/verl-mindspeed/vllm_ascend/utils.py \| 1 +
8	1 file changed, 1 insertion(+)	8	1 file changed, 1 insertion(+)
9		9
10	-diff --git a/llm_rl/qwen3/vllm_ascend/utils.py b/llm_rl/qwen3/vllm_ascend/utils.py	10	+diff --git a/llm_rl/qwen3/verl-mindspeed/vllm_ascend/utils.py b/llm_rl/qwen3/verl-mindspeed/vllm_ascend/utils.py
11	index eb88afd..264ac91 100644	11	index eb88afd..264ac91 100644
12	---- a/llm_rl/qwen3/vllm_ascend/utils.py	12	+--- a/llm_rl/qwen3/verl-mindspeed/vllm_ascend/utils.py
13	-+++ b/llm_rl/qwen3/vllm_ascend/utils.py	13	++++ b/llm_rl/qwen3/verl-mindspeed/vllm_ascend/utils.py
14	@@ -999,6 +999,7 @@ def get_hccl_config_for_pg_options(group_name: str) -> dict \| None:	14	@@ -999,6 +999,7 @@ def get_hccl_config_for_pg_options(group_name: str) -> dict \| None:
15	return None	15	return None
16	hccl_config_map = {	16	hccl_config_map = {
	@@ -18,7 +18,7 @@ index eb88afd..264ac91 100644
18	+ "tp": {"hccl_op_expansion_mode": 3},	18	+ "tp": {"hccl_op_expansion_mode": 3},
19	}	19	}
20	return hccl_config_map.get(group_name, get_default_buffer_config())	20	return hccl_config_map.get(group_name, get_default_buffer_config())
21	-	21	+
22	---	22	+--
23	2.45.1.windows.1	23	2.45.1.windows.1
24		24

Rllm_rl/qwen3/patches/vllm_ascend/0004-vllm_ascend-bugfix-npugraph_ex_static_kernel_typo.patch→llm_rl/qwen3/verl-mindspeed/patches/vllm_ascend/0004-vllm_ascend-bugfix-npugraph_ex_static_kernel_typo.patch+5-5

Rllm_rl/qwen3/patches/vllm_ascend/0005-vllm_ascend-bugfix-align_FIA_input_for_TND_layout.patch→llm_rl/qwen3/verl-mindspeed/patches/vllm_ascend/0005-vllm_ascend-bugfix-align_FIA_input_for_TND_layout.patch+6-6

	@@ -7,14 +7,14 @@ Subject: [PATCH] Align actual_seq_lengths_q with runtime_shape for TND layout
7	.../vllm_ascend/attention/attention_v1.py \| 26 ++++++++++++++++++-	7	.../vllm_ascend/attention/attention_v1.py \| 26 ++++++++++++++++++-
8	1 file changed, 25 insertions(+), 1 deletion(-)	8	1 file changed, 25 insertions(+), 1 deletion(-)
9		9
10	-diff --git a/llm_rl/qwen3/vllm_ascend/attention/attention_v1.py b/llm_rl/qwen3/vllm_ascend/attention/attention_v1.py	10	+diff --git a/llm_rl/qwen3/verl-mindspeed/vllm_ascend/attention/attention_v1.py b/llm_rl/qwen3/verl-mindspeed/vllm_ascend/attention/attention_v1.py
11	index 5a8f6a1..31cd7be 100644	11	index 5a8f6a1..31cd7be 100644
12	---- a/llm_rl/qwen3/vllm_ascend/attention/attention_v1.py	12	+--- a/llm_rl/qwen3/verl-mindspeed/vllm_ascend/attention/attention_v1.py
13	-+++ b/llm_rl/qwen3/vllm_ascend/attention/attention_v1.py	13	++++ b/llm_rl/qwen3/verl-mindspeed/vllm_ascend/attention/attention_v1.py
14	@@ -62,6 +62,28 @@ from vllm_ascend.utils import weak_ref_tensors	14	@@ -62,6 +62,28 @@ from vllm_ascend.utils import weak_ref_tensors
15	SWA_INT_MAX = 2147483647	15	SWA_INT_MAX = 2147483647
16	-	16	+
17	-	17	+
18	+def _pad_attention_seq_params(	18	+def _pad_attention_seq_params(
19	+ actual_seq_lengths_q: list[int], seq_lens: list[int], runtime_shape: int	19	+ actual_seq_lengths_q: list[int], seq_lens: list[int], runtime_shape: int
20	+) -> tuple[list[int], list[int]]:	20	+) -> tuple[list[int], list[int]]:
	@@ -51,6 +51,6 @@ index 5a8f6a1..31cd7be 100644
51	torch.npu.graph_task_update_begin(update_stream, handle)	51	torch.npu.graph_task_update_begin(update_stream, handle)
52	torch_npu.npu_fused_infer_attention_score.out(	52	torch_npu.npu_fused_infer_attention_score.out(
53	query=query,	53	query=query,
54	---	54	+--
55	2.45.1.windows.1	55	2.45.1.windows.1
56		56

Rllm_rl/qwen3/patches/vllm_ascend/0006-vllm_ascend-feature-chunk_moe.patch→llm_rl/qwen3/verl-mindspeed/patches/vllm_ascend/0006-vllm_ascend-feature-chunk_moe.patch+124-124

	@@ -1,124 +1,124 @@
1	-From f45d5289e99793f169252879f92ee253273faac1 Mon Sep 17 00:00:00 2001	1	+From f45d5289e99793f169252879f92ee253273faac1 Mon Sep 17 00:00:00 2001
2	-From: huyuanquan1 <huyuanquan1@huawei.com>	2	+From: huyuanquan1 <huyuanquan1@huawei.com>
3	-Date: Sat, 28 Feb 2026 15:09:05 +0800	3	+Date: Sat, 28 Feb 2026 15:09:05 +0800
4	-Subject: [PATCH] feature chunk moe	4	+Subject: [PATCH] feature chunk moe
5	-	5	+
6	----	6	+---
7	- vllm_ascend/ops/fused_moe/moe_comm_method.py \| 80 ++++++++++++++++++++	7	+ vllm_ascend/ops/fused_moe/moe_comm_method.py \| 80 ++++++++++++++++++++
8	- 1 file changed, 80 insertions(+)	8	+ 1 file changed, 80 insertions(+)
9	-	9	+
10	-diff --git a/llm_rl/qwen3/vllm_ascend/ops/fused_moe/moe_comm_method.py b/llm_rl/qwen3/vllm_ascend/ops/fused_moe/moe_comm_method.py	10	+diff --git a/llm_rl/qwen3/verl-mindspeed/vllm_ascend/ops/fused_moe/moe_comm_method.py b/llm_rl/qwen3/verl-mindspeed/vllm_ascend/ops/fused_moe/moe_comm_method.py
11	-index 458557e9..ebb0790e 100644	11	+index 458557e9..ebb0790e 100644
12	---- a/llm_rl/qwen3/vllm_ascend/ops/fused_moe/moe_comm_method.py	12	+--- a/llm_rl/qwen3/verl-mindspeed/vllm_ascend/ops/fused_moe/moe_comm_method.py
13	-+++ b/llm_rl/qwen3/vllm_ascend/ops/fused_moe/moe_comm_method.py	13	++++ b/llm_rl/qwen3/verl-mindspeed/vllm_ascend/ops/fused_moe/moe_comm_method.py
14	-@@ -15,6 +15,7 @@	14	+@@ -15,6 +15,7 @@
15	- # This file is a part of the vllm-ascend project.	15	+ # This file is a part of the vllm-ascend project.
16	- from __future__ import annotations	16	+ from __future__ import annotations
17	-	17	+
18	-+import os	18	++import os
19	- from abc import ABC, abstractmethod	19	+ from abc import ABC, abstractmethod
20	- from dataclasses import dataclass	20	+ from dataclasses import dataclass
21	- from typing import Dict, Optional	21	+ from typing import Dict, Optional
22	-@@ -36,6 +37,83 @@ from vllm_ascend.ops.fused_moe.token_dispatcher import (	22	+@@ -36,6 +37,83 @@ from vllm_ascend.ops.fused_moe.token_dispatcher import (
23	- _MoECommMethods: Dict[Optional[MoECommType], MoECommMethod] = {}	23	+ _MoECommMethods: Dict[Optional[MoECommType], MoECommMethod] = {}
24	-	24	+
25	-	25	+
26	-+def chunk_moe_decorator(fused_experts_func):	26	++def chunk_moe_decorator(fused_experts_func):
27	-+ chunk_moe_size = int(os.environ.get('VLLM_CHUNK_MOE_SIZE', 512))	27	++ chunk_moe_size = int(os.environ.get('VLLM_CHUNK_MOE_SIZE', 512))
28	-+ def get_arg(name, kwargs):	28	++ def get_arg(name, kwargs):
29	-+ if name in kwargs:	29	++ if name in kwargs:
30	-+ return kwargs.pop(name)	30	++ return kwargs.pop(name)
31	-+ return None	31	++ return None
32	-+	32	++
33	-+ def wrapper(args, *kwargs):	33	++ def wrapper(args, *kwargs):
34	-+ hidden_states = get_arg('hidden_states', kwargs)	34	++ hidden_states = get_arg('hidden_states', kwargs)
35	-+ topk_weights = get_arg('topk_weights', kwargs)	35	++ topk_weights = get_arg('topk_weights', kwargs)
36	-+ topk_ids = get_arg('topk_ids', kwargs)	36	++ topk_ids = get_arg('topk_ids', kwargs)
37	-+	37	++
38	-+ chunk_start_index = 0	38	++ chunk_start_index = 0
39	-+ ctx = get_forward_context()	39	++ ctx = get_forward_context()
40	-+ from vllm.distributed import get_tensor_model_parallel_world_size	40	++ from vllm.distributed import get_tensor_model_parallel_world_size
41	-+ tp_size = get_tensor_model_parallel_world_size()	41	++ tp_size = get_tensor_model_parallel_world_size()
42	-+ max_tokens = (ctx.max_tokens_across_dp + tp_size - 1) // tp_size	42	++ max_tokens = (ctx.max_tokens_across_dp + tp_size - 1) // tp_size
43	-+	43	++
44	-+ if max_tokens < chunk_moe_size:	44	++ if max_tokens < chunk_moe_size:
45	-+ return fused_experts_func(	45	++ return fused_experts_func(
46	-+ hidden_states=hidden_states,	46	++ hidden_states=hidden_states,
47	-+ topk_weights=topk_weights,	47	++ topk_weights=topk_weights,
48	-+ topk_ids=topk_ids,	48	++ topk_ids=topk_ids,
49	-+ *args,	49	++ *args,
50	-+ **kwargs	50	++ **kwargs
51	-+ )	51	++ )
52	-+	52	++
53	-+ num_tokens = hidden_states.size(0)	53	++ num_tokens = hidden_states.size(0)
54	-+ final_routed_out = torch.zeros_like(hidden_states)	54	++ final_routed_out = torch.zeros_like(hidden_states)
55	-+ all_expert_tokens = []	55	++ all_expert_tokens = []
56	-+ last_before_dispatch_evt = None	56	++ last_before_dispatch_evt = None
57	-+ last_before_combine_evt = None	57	++ last_before_combine_evt = None
58	-+ group_list_type = None	58	++ group_list_type = None
59	-+ for chunk_start in range(0, max_tokens, chunk_moe_size):	59	++ for chunk_start in range(0, max_tokens, chunk_moe_size):
60	-+ skip_result_store = chunk_start >= num_tokens	60	++ skip_result_store = chunk_start >= num_tokens
61	-+ chunk_end = min(chunk_start + chunk_moe_size, num_tokens)	61	++ chunk_end = min(chunk_start + chunk_moe_size, num_tokens)
62	-+ chunk_start = min(chunk_start, num_tokens - 1)	62	++ chunk_start = min(chunk_start, num_tokens - 1)
63	-+ chunk_hidden_states = hidden_states[chunk_start:chunk_end]	63	++ chunk_hidden_states = hidden_states[chunk_start:chunk_end]
64	-+ chunk_topk_ids = topk_ids[chunk_start:chunk_end]	64	++ chunk_topk_ids = topk_ids[chunk_start:chunk_end]
65	-+ chunk_topk_weights = topk_weights[chunk_start:chunk_end]	65	++ chunk_topk_weights = topk_weights[chunk_start:chunk_end]
66	-+ update_kwargs = dict(**kwargs)	66	++ update_kwargs = dict(**kwargs)
67	-+ if update_kwargs.get('shared_experts'):	67	++ if update_kwargs.get('shared_experts'):
68	-+ update_kwargs['shared_experts'] = update_kwargs['shared_experts'][chunk_start:chunk_end]	68	++ update_kwargs['shared_experts'] = update_kwargs['shared_experts'][chunk_start:chunk_end]
69	-+	69	++
70	-+ res = fused_experts_func(	70	++ res = fused_experts_func(
71	-+ hidden_states=chunk_hidden_states,	71	++ hidden_states=chunk_hidden_states,
72	-+ topk_weights=chunk_topk_weights,	72	++ topk_weights=chunk_topk_weights,
73	-+ topk_ids=chunk_topk_ids,	73	++ topk_ids=chunk_topk_ids,
74	-+ *args,	74	++ *args,
75	-+ **update_kwargs	75	++ **update_kwargs
76	-+ )	76	++ )
77	-+	77	++
78	-+ if skip_result_store:	78	++ if skip_result_store:
79	-+ continue	79	++ continue
80	-+ chunk_end_idx = chunk_start_index + res.routed_out.shape[0]	80	++ chunk_end_idx = chunk_start_index + res.routed_out.shape[0]
81	-+ final_routed_out[chunk_start_index: chunk_end_idx, :] = res.routed_out	81	++ final_routed_out[chunk_start_index: chunk_end_idx, :] = res.routed_out
82	-+ if res.expert_tokens is not None:	82	++ if res.expert_tokens is not None:
83	-+ all_expert_tokens.append(res.expert_tokens)	83	++ all_expert_tokens.append(res.expert_tokens)
84	-+ last_before_dispatch_evt = res.before_dispatch_evt	84	++ last_before_dispatch_evt = res.before_dispatch_evt
85	-+ last_before_combine_evt = res.before_combine_evt	85	++ last_before_combine_evt = res.before_combine_evt
86	-+ group_list_type = res.group_list_type	86	++ group_list_type = res.group_list_type
87	-+ chunk_start_index = chunk_end_idx	87	++ chunk_start_index = chunk_end_idx
88	-+	88	++
89	-+ combine_expert_tokens = None	89	++ combine_expert_tokens = None
90	-+ if all_expert_tokens:	90	++ if all_expert_tokens:
91	-+ combined_expert_tokens = torch.cat(all_expert_tokens, dim=0)	91	++ combined_expert_tokens = torch.cat(all_expert_tokens, dim=0)
92	-+ return FusedExpertsResult(	92	++ return FusedExpertsResult(
93	-+ routed_out=final_routed_out,	93	++ routed_out=final_routed_out,
94	-+ before_dispatch_evt=last_before_dispatch_evt,	94	++ before_dispatch_evt=last_before_dispatch_evt,
95	-+ before_combine_evt=last_before_combine_evt,	95	++ before_combine_evt=last_before_combine_evt,
96	-+ group_list_type=group_list_type,	96	++ group_list_type=group_list_type,
97	-+ expert_tokens=combined_expert_tokens	97	++ expert_tokens=combined_expert_tokens
98	-+ )	98	++ )
99	-+	99	++
100	-+ return wrapper	100	++ return wrapper
101	-+	101	++
102	-+	102	++
103	- def get_moe_comm_method(	103	+ def get_moe_comm_method(
104	- moe_comm_type: Optional[MoECommType]) -> Optional[MoECommMethod]:	104	+ moe_comm_type: Optional[MoECommType]) -> Optional[MoECommMethod]:
105	- return _MoECommMethods.get(moe_comm_type, None)	105	+ return _MoECommMethods.get(moe_comm_type, None)
106	-@@ -100,6 +178,7 @@ class MoECommMethod(ABC):	106	+@@ -100,6 +178,7 @@ class MoECommMethod(ABC):
107	- context_metadata)	107	+ context_metadata)
108	- return hidden_states	108	+ return hidden_states
109	-	109	+
110	-+ @chunk_moe_decorator	110	++ @chunk_moe_decorator
111	- def fused_experts(	111	+ def fused_experts(
112	- self,	112	+ self,
113	- hidden_states: torch.Tensor,	113	+ hidden_states: torch.Tensor,
114	-@@ -275,6 +354,7 @@ class FusedMC2CommImpl(MoECommMethod):	114	+@@ -275,6 +354,7 @@ class FusedMC2CommImpl(MoECommMethod):
115	- def _get_prepare_finalize(self):	115	+ def _get_prepare_finalize(self):
116	- return PrepareAndFinalizeWithMC2(self.moe_config)	116	+ return PrepareAndFinalizeWithMC2(self.moe_config)
117	-	117	+
118	-+ @chunk_moe_decorator	118	++ @chunk_moe_decorator
119	- def fused_experts(	119	+ def fused_experts(
120	- self,	120	+ self,
121	- hidden_states: torch.Tensor,	121	+ hidden_states: torch.Tensor,
122	---	122	+--
123	-2.45.1.windows.1	123	+2.45.1.windows.1
124	-	124	+

Rllm_rl/qwen3/patches/vllm_ascend/spec_decode/sam_proposer.py→llm_rl/qwen3/verl-mindspeed/patches/vllm_ascend/spec_decode/sam_proposer.py+0-0

文件重命名但无更改。

Rllm_rl/qwen3/ray_start_npu.sh→llm_rl/qwen3/verl-mindspeed/ray_start_npu.sh+3-0

	@@ -12,6 +12,9 @@
12	# See the License for the specific language governing permissions and	12	# See the License for the specific language governing permissions and
13	# limitations under the License.	13	# limitations under the License.
14		14
		15	+SCRIPT_DIR=$(cd "$(dirname "$0")" && pwd)
		16	+cd "${SCRIPT_DIR}"
		17	+
15	pkill -9 python	18	pkill -9 python
16	ray stop --force	19	ray stop --force
17		20

Rllm_rl/qwen3/requirements.txt→llm_rl/qwen3/verl-mindspeed/requirements.txt+0-0

文件重命名但无更改。

Rllm_rl/qwen3/run_container.sh→llm_rl/qwen3/verl-mindspeed/run_container.sh+0-0

文件重命名但无更改。

Allm_rl/qwen3/verl-torchtitan/Dockerfile.vllm_ascend.torchtitan.qwen3+63-0

@@ -0,0 +1,63 @@

2 +#

3 +# Licensed under the Apache License, Version 2.0 (the "License");

4 +# you may not use this file except in compliance with the License.

5 +# You may obtain a copy of the License at

6 +#

7 +# http://www.apache.org/licenses/LICENSE-2.0

8 +#

9 +# Unless required by applicable law or agreed to in writing, software

10 +# distributed under the License is distributed on an "AS IS" BASIS,

11 +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

12 +# See the License for the specific language governing permissions and

13 +# limitations under the License.

14 +

15 +FROM quay.io/ascend/vllm-ascend:v0.15.0rc1-a3-openeuler

16 +

17 +ARG PIP_INDEX_URL="https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple"

18 +ARG VERL_COMMIT="e9aa879bc61821621a36881ea305eaa0785520c1"

19 +ARG TORCHTITAN_COMMIT="ac13e536c84e7f6647b14fa9375c3c8a8a2b8578"

20 +ARG TORCHTITAN_NPU_COMMIT="29bbc8ba5bee5daf63f8a0c09512038449ffaf37"

21 +ARG VLLM_ASCEND_TAG="v0.15.0rc1"

22 +

23 +ENV TASK_QUEUE_ENABLE=1 \

24 + OMP_NUM_THREADS=1

25 +

26 +SHELL ["/bin/bash", "-c"]

27 +

28 +RUN pip config set global.index-url ${PIP_INDEX_URL}

29 +

30 +WORKDIR /workspace

31 +

32 +# veRL

33 +RUN git clone https://gitcode.com/GitHub_Trending/ve/verl.git && \

34 + cd verl && \

35 + git checkout ${VERL_COMMIT}

36 +

37 +# TorchTitan

38 +RUN git clone https://gitcode.com/GitHub_Trending/to/torchtitan.git && \

39 + cd torchtitan && \

40 + git checkout ${TORCHTITAN_COMMIT}

41 +

X

X xuyujun6月22日

这里需要安装下pip install -r requirements.txt吗？不然依赖在容器里并不完整

zhangjianshe

6月22日评论：

42 +# TorchTitan-NPU

43 +RUN git clone https://gitcode.com/cann/torchtitan-npu.git && \

44 + cd torchtitan-npu && \

45 + git checkout ${TORCHTITAN_NPU_COMMIT}

46 +

47 +# vLLM-Ascend

48 +RUN git clone https://gitcode.com/gh_mirrors/vl/vllm-ascend.git && \

49 + cd vllm-ascend && \

50 + git checkout ${VLLM_ASCEND_TAG} && \

51 + git submodule update --init --recursive

52 +

53 +# Install common Python dependencies. The sample's build_project.sh applies

54 +# version patches and rebuilds vllm-ascend after the recipe source is mounted.

55 +RUN python3 -m pip install modelscope 'ray>=2.47.1,<=2.48.0' 'protobuf>3.20.0' && \

56 + python3 -m pip cache purge

57 +

58 +RUN yum install -y patch

59 +

60 +RUN echo "source /usr/local/Ascend/ascend-toolkit/set_env.sh" >> ~/.bashrc && \

61 + echo "export LD_PRELOAD=/usr/lib64/libjemalloc.so.2:$LD_PRELOAD" >> ~/.bashrc

atomgit-bot6月18日

🔵 Low Priority

Dockerfile 第 56 行：RUN echo "export LD_PRELOAD=/usr/lib64/libjemalloc.so.2:$LD_PRELOAD" >> ~/.bashrc

在双引号内的 $LD_PRELOAD 会在 Docker 构建时展开，而非运行时。构建环境中 LD_PRELOAD 通常未设置，因此展开为空字符串，实际写入 .bashrc 的内容为：末尾的冒号表示一个空的 LD_PRELOAD 条目，ld.so 会忽略它（功能上无害）。但如果构建环境恰好设置了 LD_PRELOAD，该值将被"固化"到镜像中，而不是在容器启动时动态拼接。

预期行为应为运行时拼接：保留容器启动时已有的 LD_PRELOAD 并前置 libjemalloc。应将 $ 转义为 \$LD_PRELOAD 或使用单引号。

建议：将双引号改为单引号，或将 $LD_PRELOAD 转义为 \$LD_PRELOAD，使变量在容器运行时展开而非构建时展开。可选方案：RUN echo 'export LD_PRELOAD=/usr/lib64/libjemalloc.so.2:$LD_PRELOAD' >> ~/.bashrc

改动建议

61	- echo "export LD_PRELOAD=/usr/lib64/libjemalloc.so.2:$LD_PRELOAD" >> ~/.bashrc
61	+ RUN echo 'export LD_PRELOAD=/usr/lib64/libjemalloc.so.2:$LD_PRELOAD' >> ~/.bashrc

应用建议

不准确？

zhangjianshe

6月22日评论：

62 +

63 +CMD ["/bin/bash"]

Allm_rl/qwen3/verl-torchtitan/README.md+220-0

@@ -0,0 +1,220 @@
	1	+# Qwen3 TorchTitan RL训练样例
	zhanghz16月26日用户关注点是"能否易懂地引导复现实验"。当前 README 把流程（建镜像→源码→patch→数据→权重→启动）讲得很清楚，但没有任何"成功长什么样"的描述：没有预期 loss/reward 走势、吞吐(TPS)、或一次成功运行的样例日志/产物位置说明。对比同目录的 mindspeed/README.md，其概述里给出了明确的性能基线（如"最优系统吞吐 120 TPS/卡"及性能表），可作为复现验收标准。补一个“预期结果”章节，把PR里的loss等成果展示放出来，让用户知道复现结果应该是什么样 zhangjianshe 7月1日评论： 7月1日评论：
	2	+
	3	+## 概述
	4	+
	5	+本样例面向 Qwen3 系列模型，基于 [veRL](https://github.com/volcengine/verl) 的 TorchTitan 训练引擎能力，提供 NPU 环境下的 TorchTitan 训练适配 patch 和训练启动脚本示例。
	6	+
	7	+当前目录的组织方式与 `llm_rl/qwen3/verl-mindspeed` 保持一致：
	8	+
	9	+- `patches/`：存放针对框架源码的 git patch，patch 内部路径统一以 `llm_rl/qwen3/verl-torchtitan/` 为前缀。
	10	+- `internal/`：存放训练启动脚本示例。
	11	+- `download_frameworks_source_code.sh`：下载并切换依赖框架源码版本。
	12	+- `build_project.sh`：拷贝依赖源码、应用 patch，并在拷贝 `vllm_ascend` 包代码前重新编译安装 `vllm-ascend`。
	13	+- `apply_all_patches.sh`：按文件名版本序统一应用当前样例下的所有 patch。
	14	+
	15	+当前提供的训练脚本以 Qwen3-30B-A3B、单机 16 卡 GRPO 训练为示例，使用 TorchTitan 作为 actor/ref 训练引擎，使用 vLLM 作为 rollout 引擎，具体训练性能及结果如下所示：
	16	+
	17	+GRPO算法RL训练：基于Atlas A3 16卡集群，加载真实权重，使用gsm8k数据集，Prefill/Decode阶段长度分别为256与4096，性能测试结果如下：
	18	+
	19	+ \| 基础模型 \| 机器型号 \| GBS \| n_samples \| step \| max_prompt_length(最大输入长度) \| max_response_length(最大输出长度) \| perf/time_per_step(首步总时间) \|
	20	+ \| --------------- \| ------------- \| --- \| --------- \| ---- \| ------------------------------- \| --------------------------------- \| ------------------ \|
	21	+ \| Qwen3-30B-A3B \| Atlas A3 16卡 \| 8 \| 2 \| 1 \| 256 \| 4096 \| 653 \|
	22	+
	23	+ 随迭代进行，gsm8k 验证集准确率变化如下：
	24	+
	25	+ <p align="center">
	26	+ <img src="../../../docs/llm_rl/figures/qwen3_figures/image43.png" alt="GSM8K 验证集准确率变化">
	27	+ </p>
	28	+
	29	+## 组件版本
	30	+
	31	+\| 组件 \| 版本/Commit ID \|
	32	+\|------\|----------------\|
	33	+\| verl \| e9aa879bc61821621a36881ea305eaa0785520c1 \|
	34	+\| torchtitan \| ac13e536c84e7f6647b14fa9375c3c8a8a2b8578 \|
	35	+\| torchtitan-npu \| 29bbc8ba5bee5daf63f8a0c09512038449ffaf37 \|
	36	+\| vllm \| 0.15.0 \|
	37	+\| vllm-ascend \| 0.15.0rc1 \|
	38	+\| torch \| 2.12.0 \|
	39	+\| torch_npu \| 2.12.0rc1 \|
	40	+
	41	+## 硬件和环境要求
	42	+
	43	+产品型号：Atlas A3 系列
	44	+
	45	+操作系统：Linux ARM
	46	+
	47	+镜像版本：cann:8.5.0-a3-openeuler24.03-py3.11
	48	+
	49	+建议在已安装 CANN、torch、torch_npu 以及基础编译工具链的镜像或容器中运行。请根据实际 CANN 安装目录 source 环境变量，例如：
	50	+
	51	+```bash
	52	+source /usr/local/Ascend/ascend-toolkit/set_env.sh
	53	+```
	54	+
	55	+运行前建议确认 `torch_npu`、`torchtitan`、`torchtitan_npu`、`vllm` 和 `vllm-ascend` 已安装或已加入 `PYTHONPATH`。
	56	+
	57	+## 基于Dockerfile构建环境
	58	+
	59	+环境搭建可以基于 Dockerfile 快速实现。本样例提供的 Dockerfile 会基于 vllm-ascend A3 镜像安装基础工具链，并拉取 `verl`、`torchtitan`、`torchtitan-npu` 和 `vllm-ascend` 源码到 `/workspace`。
	60	+
	61	+1. 基于Dockerfile创建镜像。
	62	+ ```bash
	63	+ # 下载本样例所在代码仓，以 master 分支为例
	64	+ git clone https://gitcode.com/cann/cann-recipes-train.git
	65	+
	66	+ cd ./cann-recipes-train/llm_rl/qwen3/verl-torchtitan
	67	+
	68	+ docker build -t qwen3-torchtitan-env -f Dockerfile.vllm_ascend.torchtitan.qwen3 .
	69	+ ```
	70	+
	71	+ 可通过当前目录 run_container.sh 创建容器。请传入容器名称和镜像名称：
	72	+ ```bash
	73	+ bash run_container.sh qwen3_torchtitan qwen3-torchtitan-env
	74	+ ```
	75	+ 该脚本会挂载常用 NPU 设备、驱动目录和数据目录，并在容器创建后自动进入容器。
	76	+ 请确保 `cann-recipes-train` 仓库位于容器可见的挂载目录中，例如 `/home` 或 `/data` 下；进入容器后需切换到该仓库的 `llm_rl/qwen3/verl-torchtitan` 目录继续执行后续步骤。
	77	+
	78	+2. 源码准备及安装所需的python依赖。
	79	+ ```bash
	80	+ # Dockerfile 已预置 /workspace 下依赖框架源码，基于 Dockerfile 创建环境时无需执行 download_frameworks_source_code.sh。
	81	+ # 进入容器后，拷贝依赖源码、应用 patch 并按需编译安装 vllm-ascend。
	82	+ bash build_project.sh
	83	+ ```
	84	+
	85	+ 若未基于 Dockerfile 创建环境，需要手动下载依赖框架源码后再构建：
	86	+ ```bash
	87	+ bash download_frameworks_source_code.sh
	88	+ bash build_project.sh
	89	+ ```
	90	+
	91	+ `vllm-ascend` 当前最高适配到 `torch_npu==2.10.0`，而本样例使用的 TorchTitan 版本依赖 `torch==2.12`。两者存在版本差异，可能导致 vLLM 原有融合算子在运行时报错。`build_project.sh` 会在拷贝 `vllm_ascend` 包代码前，先基于 `vllm-ascend` 源码重新编译安装。
	92	+
	93	+ `build_project.sh` 默认使用以下源码目录，若源码目录不同，可在执行前通过环境变量覆盖：
	94	+
	95	+ \| 环境变量 \| 默认值 \| 说明 \|
	96	+ \|----------\|--------\|------\|
	97	+ \| `VERL_SRC_DIR` \| `/workspace/verl` \| veRL 源码目录 \|
	98	+ \| `TORCHTITAN_SRC_DIR` \| `/workspace/torchtitan` \| TorchTitan 源码目录 \|
	99	+ \| `TORCHTITAN_NPU_SRC_DIR` \| `/workspace/torchtitan-npu` \| TorchTitan-NPU 源码目录 \|
	100	+ \| `VLLM_ASCEND_SRC_DIR` \| `/workspace/vllm-ascend` \| vllm-ascend 源码目录 \|
	101	+
	102	+3. 使能patch修改。
	103	+ `build_project.sh` 会自动调用 `apply_all_patches.sh` 完成 patch 应用。如果需要单独使能 patch，可在 `cann-recipes-train` 仓库根目录下运行：
	104	+ ```bash
	105	+ bash llm_rl/qwen3/verl-torchtitan/apply_all_patches.sh
	106	+ ```
	107	+
	108	+ `apply_all_patches.sh` 会在 `patches` 目录下查找所有 `*.patch` 文件，按文件名版本序排序后统一应用。大部分 patch 内部路径以 `llm_rl/qwen3/verl-torchtitan/` 为前缀；`patches/torchair` 下的 patch 会在 Python `site-packages` 中动态查找 `torch_npu/dynamo/torchair` 实际安装路径后应用。
	109	+
	110	+## 数据集准备
	111	+
	112	+本样例中使用的 gsm8k 数据集准备方法可参考 [verl官方文档](https://verl.readthedocs.io/en/latest/preparation/prepare_data.html)。
	113	+
	114	+数据格式可参考 veRL 对 RLHF/RL 数据集的要求，常见字段包括 prompt、response 或用于奖励计算的任务字段。若使用 GSM8K、DeepScaler、DAPO 等数据集，请先按对应数据处理脚本生成 parquet 文件。
	115	+
	116	+运行前请将训练脚本中的数据集路径替换为实际数据集路径。
	117	+
	118	+```bash
	119	+data.train_files=/path/to/gsm8k/train.parquet
	120	+data.val_files=/path/to/gsm8k/test.parquet
	121	+```
	122	+
	123	+## 模型权重准备
	124	+
	125	+本样例使用的模型权重准备方法如下：
	126	+
	127	+### Qwen3-30B-A3B
	128	+
	129	+```bash
	130	+# 下载 Qwen3-30B-A3B 及完整权重至当前样例目录的 ./Qwen3-30B-A3B 目录下。
	131	+mkdir ./Qwen3-30B-A3B
	132	+pip install modelscope
	133	+modelscope download --model Qwen/Qwen3-30B-A3B --local_dir ./Qwen3-30B-A3B
	134	+```
	135	+
	136	+下载完成后，将训练脚本中的 `MODEL_PATH` 修改为实际 HuggingFace 格式模型权重目录。若按下文从 `cann-recipes-train/llm_rl/qwen3/verl-torchtitan` 目录启动训练，路径可配置为：
	137	+
	138	+```bash
	139	+MODEL_PATH=./Qwen3-30B-A3B
	140	+```
	141	+
	142	+模型权重目录需至少包含模型配置、tokenizer 文件和完整权重文件，并确保当前环境可以被 veRL、TorchTitan 和 vLLM 正确加载。
	143	+
	144	+## RL后训练执行
	145	+
	146	+当前目录提供了一个 TorchTitan GRPO 训练脚本示例：
	147	+
	148	+```bash
	149	+internal/train_grpo_qwen3_30b_a3b_16die.sh
	150	+```
	151	+
	152	+在 `cann-recipes-train/llm_rl/qwen3/verl-torchtitan` 目录下启动示例：
	153	+
	154	+```bash
	155	+bash internal/train_grpo_qwen3_30b_a3b_16die.sh
	156	+```
	atomgit-bot6月23日 🔵 Low Priority README 中存在目录路径不一致：第 52 行引导用户在 `cann-recipes-train/llm_rl/qwen3/torchtitan` 目录下构建环境和执行 `build_project.sh`。第 68 行 `cd ./cann-recipes-train/llm_rl/qwen3/torchtitan` 延续上述目录。但第 138 行写"在 `cann-recipes-train/llm_rl/qwen3` 目录下启动示例"，并用 `bash torchtitan/internal/train_grpo_qwen3_30b_a3b_16die.sh`。如果用户严格按照步骤 1→2→3 的顺序从头执行，到第 138 行时当前目录应为 `torchtitan/`，此时执行 `bash torchtitan/internal/...` 会因路径错误而失败。虽然训练脚本本身不依赖当前目录（只执行 `python3 -m verl.trainer.main_ppo`），但文档不一致会导致用户困惑或执行失败。建议：统一 README 中的当前目录指引。建议将第 138 行改为"在 `cann-recipes-train/llm_rl/qwen3/torchtitan` 目录下启动示例"，并将启动命令改为 `bash internal/train_grpo_qwen3_30b_a3b_16die.sh`，与前面的步骤保持一致。不准确？ zhangjianshe 6月25日评论： 6月25日评论： atomgit-bot6月23日 🔵 Low Priority README 中存在目录路径不一致：第 52 行和 68 行引导用户在 `cann-recipes-train/llm_rl/qwen3/torchtitan` 目录下操作。但第 138 行引导用户切换到 `cann-recipes-train/llm_rl/qwen3` 目录（上一级）来启动训练。虽然第 141 行的命令 `bash torchtitan/internal/train_grpo_qwen3_30b_a3b_16die.sh` 确实需要在 `llm_rl/qwen3/` 下执行（因为脚本路径以 `torchtitan/` 为前缀），但 README 没有给出显式的 `cd ..` 步骤，用户容易在当前 `torchtitan/` 目录下错误执行 `bash torchtitan/internal/...`，导致"文件不存在"错误。上一轮审查已报告，本轮未修复。建议：统一 README 中的目录指引：在第 138 行前添加 `cd ..` 步骤，或将启动命令改为 `bash internal/train_grpo_qwen3_30b_a3b_16die.sh`（即保持用户在当前 `torchtitan/` 目录下执行），二者选其一。不准确？ zhangjianshe 6月25日评论： 6月25日评论：
	157	+
	158	+执行前请根据实际环境修改脚本中的以下配置：
	159	+
	160	+\| 配置项 \| 说明 \|
	161	+\|--------\|------\|
	162	+\| `MODEL_PATH` \| Qwen3 模型权重路径 \|
	163	+\| `data.train_files` \| 训练数据路径 \|
	164	+\| `data.val_files` \| 验证数据路径 \|
	165	+\| `ASCEND_RT_VISIBLE_DEVICES` \| 可见 NPU 设备 \|
	166	+\| `NUM_GPUS` \| 单节点参与训练的 NPU 数量 \|
	167	+\| `FSDP_SIZE` \| TorchTitan FSDP shard 并行规模 \|
	168	+\| `TP_SIZE` \| TorchTitan tensor parallel 并行规模 \|
	169	+\| `EP_SIZE` \| TorchTitan expert parallel 并行规模 \|
	170	+\| `actor_rollout_ref.rollout.tensor_model_parallel_size` \| vLLM rollout TP 并行规模 \|
	171	+\| `actor_rollout_ref.rollout.data_parallel_size` \| vLLM rollout DP 并行规模 \|
	172	+\| `actor_rollout_ref.rollout.max_model_len` \| rollout 最大模型上下文长度 \|
	173	+
	174	+训练脚本默认启用的关键环境变量包括：
	175	+
	176	+\| 环境变量 \| 说明 \|
	177	+\|----------\|------\|
	178	+\| `HYDRA_FULL_ERROR=1` \| 展示完整 Hydra 错误栈 \|
	179	+\| `RAY_EXPERIMENTAL_NOSET_ASCEND_RT_VISIBLE_DEVICES=1` \| 避免 Ray 自动覆盖 NPU 可见设备 \|
	180	+\| `HCCL_CONNECT_TIMEOUT=3600` \| 增大 HCCL 连接超时时间 \|
	181	+\| `PYTORCH_NPU_ALLOC_CONF=max_split_size_mb:256` \| 调整 NPU 显存分配策略 \|
	182	+
	183	+可通过命令行追加 Hydra 参数覆盖脚本默认配置，例如：
	184	+
	185	+```bash
	186	+bash internal/train_grpo_qwen3_30b_a3b_16die.sh \
	187	+ trainer.total_training_steps=10 \
	188	+ trainer.test_freq=5
	189	+```
	190	+
	191	+## 附录
	192	+
	193	+### 文件说明
	194	+
	195	+\| 上级目录 \| 文件路径 \| 说明 \|
	196	+\|----------\|----------\|------\|
	197	+\| verl \| [0001-verl-feature-add_torchtitan_npu.patch](patches/verl/0001-verl-feature-add_torchtitan_npu.patch) \| 适配 TorchTitan 在 NPU 场景下的运行逻辑，包括基于 `torchtitan_npu` 的融合算子优化、NPU expandable segments 设置、TorchTitan flavor 推导修正和训练 worker 结束后的显存清理。 \|
	198	+\| torchtitan \| [0001-torchtitan-bugfix-qwen3_init_for_npu.patch](patches/torchtitan/0001-torchtitan-bugfix-qwen3_init_for_npu.patch) \| 调整 Qwen3 初始化和 debug 配置，规避 NPU 场景下不兼容的初始化参数和 flash debug 配置。 \|
	199	+\| torchair \| [0001-torchair-bugfix-adapt_hint_int_import.patch](patches/torchair/0001-torchair-bugfix-adapt_hint_int_import.patch) \| 修改 `torch_npu` 内置 TorchAir 代码，兼容不同 torch 版本中 `hint_int` 与 `optimization_hint` 的符号导入差异。 \|
	200	+\| vllm_ascend \| [0001-vllm_ascend-feature-align_torch_2_12_build.patch](patches/vllm_ascend/0001-vllm_ascend-feature-align_torch_2_12_build.patch) \| 调整 `llm_rl/qwen3/verl-torchtitan/vllm-ascend` 源码目录下构建和依赖文件中的 `torch`、`torch-npu` 版本约束，用于和 TorchTitan 训练环境对齐。 \|
	201	+\| torchtitan \| [Dockerfile.vllm_ascend.torchtitan.qwen3](Dockerfile.vllm_ascend.torchtitan.qwen3) \| 构建 Qwen3 TorchTitan 样例基础运行镜像，预置基础工具链并拉取依赖框架源码。 \|
	202	+\| torchtitan \| [run_container.sh](run_container.sh) \| 基于指定镜像创建并进入容器，挂载 NPU 设备、驱动和常用宿主机目录。 \|
	203	+\| torchtitan \| [download_frameworks_source_code.sh](download_frameworks_source_code.sh) \| 下载并切换本样例依赖的 `verl`、`torchtitan`、`torchtitan-npu` 和 `vllm-ascend` 源码版本。 \|
	204	+\| torchtitan \| [build_project.sh](build_project.sh) \| 拷贝依赖源码、应用 patch、编译安装适配当前 torch/torch_npu 版本的 `vllm-ascend`。 \|
	205	+\| torchtitan \| [apply_all_patches.sh](apply_all_patches.sh) \| 一键应用本样例按仓库拆分存放的 `verl`、`torchtitan` 和 `vllm-ascend` patch。 \|
	206	+\| internal \| [train_grpo_qwen3_30b_a3b_16die.sh](internal/train_grpo_qwen3_30b_a3b_16die.sh) \| Qwen3-30B-A3B 使用 TorchTitan 作为 actor/ref 训练引擎、vLLM 作为 rollout 引擎的 GRPO 训练启动脚本示例。 \|
	207	+
	208	+### 注意事项
	209	+
	210	+1. 当前 patch 按源码仓库拆分存放，但应用路径统一对应 `llm_rl/qwen3` 下拷贝后的源码目录。
	211	+2. 训练脚本中的模型、数据和 profile 路径均为占位示例，运行前必须替换为实际路径。
	212	+3. 如果切换 CANN、torch 或 torch_npu 版本，建议清理编译和运行缓存后再启动训练：
	213	+
	214	+```bash
	215	+rm -rf kernel_meta
	216	+rm -rf .torchair_cache
	217	+rm -rf .cache
	218	+rm -rf /root/.cache
	219	+rm -rf /root/atc_data/
	220	+```

Allm_rl/qwen3/verl-torchtitan/apply_all_patches.sh+61-0

@@ -0,0 +1,61 @@

2 +#

3 +# Licensed under the Apache License, Version 2.0 (the "License");

4 +# you may not use this file except in compliance with the License.

5 +# You may obtain a copy of the License at

6 +#

7 +# http://www.apache.org/licenses/LICENSE-2.0

8 +#

9 +# Unless required by applicable law or agreed to in writing, software

10 +# distributed under the License is distributed on an "AS IS" BASIS,

11 +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

12 +# See the License for the specific language governing permissions and

13 +# limitations under the License.

14 +

15 +#!/bin/bash

atomgit-bot6月18日

🟡 Medium Priority

脚本的 shebang #!/bin/bash 位于第 15 行，但其上的第 1-13 行为 Apache License 注释。Unix/Linux 内核在直接执行脚本时（./apply_all_patches.sh）只会检查文件第一行的前两个字节是否为 #!。由于第一行是 # Copyright...，内核无法识别 shebang，将使用默认 shell（通常为 /bin/sh）解释执行脚本，可能导致 bash 特有语法（如 [[ ]]、set -o pipefail）执行失败。

同一仓库中的 build_project.sh、run_container.sh 也存在相同模式。README 中的使用方式为 bash script.sh，此时问题不会触发，但若用户直接执行 ./script.sh 则会出问题。

建议：将 #!/bin/bash 移到文件的第一行，版权注释放在 shebang 之后。

改动建议

15	- #!/bin/bash
15	+ #!/bin/bash
16	+ # Copyright (c) 2025 Huawei Technologies Co., Ltd. All Rights Reserved.

应用建议

不准确？

zhangjianshe

6月22日评论：

atomgit-bot6月23日

🟡 Medium Priority

shebang #!/bin/bash 位于第 15 行，其上方第 1-13 行为 Apache License 注释。Unix/Linux 内核在直接执行脚本时（./apply_all_patches.sh）只检查第一行的前两个字节是否为 #!。由于第一行是 # Copyright...，内核无法识别 shebang，将使用 /bin/sh 解释脚本，导致 [[ ]]、set -o pipefail 等 bash 特性语法错误。此问题在上一轮审查中已报告，本轮未修复。

触发条件：用户直接执行 ./apply_all_patches.sh 而非 bash apply_all_patches.sh。

建议：将 #!/bin/bash 移到文件第一行，版权注释放在 shebang 之后。

改动建议

-#!/bin/bash
+#!/bin/bash
+# Copyright (c) 2026 Huawei Technologies Co., Ltd. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# ...

应用建议

不准确？

zhangjianshe

6月25日评论：

atomgit-bot6月23日

🟡 Medium Priority

torchtitan 的 apply_all_patches.sh（新增文件）第 15 行为 #!/bin/bash，但其上方第 1-13 行为 Apache License 注释。Unix/Linux 内核在直接执行脚本（./apply_all_patches.sh）时只检查第一行的前两个字节是否为 #!。由于第一行是 # Copyright...，内核无法识别 shebang，将使用 /bin/sh 解释脚本。该脚本使用了 [[ ]]、set -eo pipefail 等 bash 特性，在 POSIX sh 下会执行失败或产生未预期行为。上一轮审查已报告，本轮未修复。

建议：将 #!/bin/bash 移到文件第一行，版权注释放在 shebang 之后。

改动建议

-#!/bin/bash
+#!/bin/bash
+# Copyright (c) 2026 Huawei Technologies Co., Ltd. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+...

应用建议

不准确？

zhangjianshe

6月25日评论：

16 +set -eo pipefail

17 +

18 +SCRIPT_DIR=$(cd "$(dirname "$0")" && pwd)

19 +REPO_DIR=$(cd "${SCRIPT_DIR}/../../.." && pwd)

20 +PATCH_ROOT="llm_rl/qwen3/verl-torchtitan/patches"

21 +

22 +cd "${REPO_DIR}"

23 +

24 +echo "Applying patches in numerical order..."

25 +

26 +find "./${PATCH_ROOT}" -type f -name "*.patch" | \

27 +sort -V | \

28 +while IFS= read -r PATCH_FILE; do

29 + [[ -z "$PATCH_FILE" ]] && continue

30 + PATCH_REL_PATH=$(realpath --relative-to=. "$PATCH_FILE")

31 +

32 + echo -n "Applying $PATCH_REL_PATH ... "

33 +

34 + if [[ "$PATCH_REL_PATH" == llm_rl/qwen3/verl-torchtitan/patches/torchair/* ]]; then

35 + if ! command -v patch >/dev/null 2>&1; then

36 + echo "[FAIL]: patch command not found" >&2

37 + exit 1

38 + fi

39 +

40 + PATCH_ABS_PATH="${REPO_DIR}/${PATCH_REL_PATH}"

41 + TORCHAIR_DIR=$(python3 -c 'import pathlib, site, sys, sysconfig; paths = [sysconfig.get_paths().get("purelib"), sysconfig.get_paths().get("platlib"), *site.getsitepackages()]; paths = [p for p in paths if p]; matches = [pathlib.Path(p) / "torch_npu" / "dynamo" / "torchair" for p in paths]; matches = [p.resolve() for p in matches if p.is_dir()]; sys.exit(1) if not matches else print(matches[0])') || {

42 + echo "[FAIL]: torch_npu torchair dir not found" >&2

43 + exit 1

44 + }

45 + if ! patch --dry-run -p1 -d "${TORCHAIR_DIR}" < "${PATCH_ABS_PATH}" >/dev/null; then

46 + echo "[FAIL]: $PATCH_REL_PATH dry run failed" >&2

47 + exit 1

48 + fi

49 +

50 + if ! patch -p1 -d "${TORCHAIR_DIR}" --backup --version-control=numbered < "${PATCH_ABS_PATH}" >/dev/null; then

51 + echo "[FAIL]: $PATCH_REL_PATH" >&2

52 + exit 1

53 + fi

54 + else

55 + if ! git apply -v --ignore-whitespace "$PATCH_REL_PATH"; then

56 + echo "[FAIL]: $PATCH_REL_PATH" >&2

57 + exit 1

58 + fi

59 + fi

60 + echo "[SUCCESS]: $PATCH_REL_PATH"

61 +done

Allm_rl/qwen3/verl-torchtitan/build_project.sh+49-0

@@ -0,0 +1,49 @@

2 +#

3 +# Licensed under the Apache License, Version 2.0 (the "License");

4 +# you may not use this file except in compliance with the License.

5 +# You may obtain a copy of the License at

6 +#

7 +# http://www.apache.org/licenses/LICENSE-2.0

8 +#

9 +# Unless required by applicable law or agreed to in writing, software

10 +# distributed under the License is distributed on an "AS IS" BASIS,

11 +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

12 +# See the License for the specific language governing permissions and

13 +# limitations under the License.

14 +

15 +#!/bin/bash

atomgit-bot6月18日

🟡 Medium Priority

build_project.sh 的 shebang 同样位于第 15 行而非第 1 行。直接 ./build_project.sh 执行时会使用 /bin/sh 而非 /bin/bash，虽该脚本主要使用 POSIX 兼容语法（set -ex、cp、cd），但 bash 调用 apply_all_patches.sh 时仍可能受环境差异影响。

建议：将 #!/bin/bash 移到文件第一行。

不准确？

zhangjianshe

6月22日评论：

atomgit-bot6月23日

🟡 Medium Priority

shebang #!/bin/bash 位于第 15 行，其上方第 1-13 行为 Apache License 注释。直接 ./build_project.sh 执行时内核无法识别 shebang，将使用 /bin/sh 解释脚本。该脚本使用了 set -ex、${VAR:-default} 等 bash 特性，在 POSIX sh 下行为不可预期。此问题在上一轮审查中已报告，本轮未修复。

建议：将 #!/bin/bash 移到文件第一行。

改动建议

-#!/bin/bash
+#!/bin/bash
+# Copyright (c) 2026 Huawei Technologies Co., Ltd. All Rights Reserved.
+# ...

应用建议

不准确？

zhangjianshe

6月25日评论：

atomgit-bot6月23日

🟡 Medium Priority

torchtitan 的 build_project.sh（新增文件）第 15 行为 #!/bin/bash，但其上方第 1-13 行为 Apache License 注释。直接 ./build_project.sh 执行时内核无法识别 shebang，将使用 /bin/sh 解释脚本。该脚本使用了 ${VAR:-default} 等 bash 特性，在 POSIX sh 下可能执行失败。上一轮审查已报告，本轮未修复。

建议：将 #!/bin/bash 移到文件第一行。

改动建议

-#!/bin/bash
+#!/bin/bash
+# Copyright (c) 2026 Huawei Technologies Co., Ltd. All Rights Reserved.
+...

应用建议

不准确？

zhangjianshe

6月25日评论：

16 +set -ex

17 +

18 +SCRIPT_DIR=$(cd "$(dirname "$0")" && pwd)

19 +

20 +VERL_SRC_DIR=${VERL_SRC_DIR:-/workspace/verl}

21 +TORCHTITAN_SRC_DIR=${TORCHTITAN_SRC_DIR:-/workspace/torchtitan}

22 +TORCHTITAN_NPU_SRC_DIR=${TORCHTITAN_NPU_SRC_DIR:-/workspace/torchtitan-npu}

23 +VLLM_ASCEND_SRC_DIR=${VLLM_ASCEND_SRC_DIR:-/workspace/vllm-ascend}

24 +

25 +cd "${SCRIPT_DIR}"

26 +cp -r "${VERL_SRC_DIR}/verl" ./

27 +cp -r "${TORCHTITAN_SRC_DIR}/torchtitan" ./

28 +cp -r "${TORCHTITAN_NPU_SRC_DIR}/torchtitan_npu" ./

29 +cp -r "${VLLM_ASCEND_SRC_DIR}" ./vllm-ascend

30 +

31 +python --version

32 +python3 --version

33 +python3 -m pip install torch==2.12.0 --index-url https://download.pytorch.org/whl/cpu --trusted-host download.pytorch.org --trusted-host download-r2.pytorch.org

34 +python3 -m pip install torchvision==0.27.0 --index-url https://download.pytorch.org/whl/cpu --trusted-host download.pytorch.org --trusted-host download-r2.pytorch.org

35 +python3 -m pip install -r "${SCRIPT_DIR}/requirements.txt"

36 +

37 +bash "${SCRIPT_DIR}/apply_all_patches.sh"

38 +

39 +if python3 -m pip list --format=freeze | grep -qx 'vllm==0.15.0+empty'; then

40 + cd "${SCRIPT_DIR}/vllm-ascend"

41 + pip install -e . --no-deps --no-build-isolation

42 +

43 + cd "${SCRIPT_DIR}"

44 + cp -r ./vllm-ascend/vllm_ascend ./

45 +else

46 + echo "Skip vllm-ascend build because vllm==0.15.0+empty is not installed."

47 +fi

48 +

49 +ls -l

Allm_rl/qwen3/verl-torchtitan/download_frameworks_source_code.sh+43-0

@@ -0,0 +1,43 @@
	1	+# Copyright (c) 2026 Huawei Technologies Co., Ltd. All Rights Reserved.
	2	+#
	3	+# Licensed under the Apache License, Version 2.0 (the "License");
	4	+# you may not use this file except in compliance with the License.
	5	+# You may obtain a copy of the License at
	6	+#
	7	+# http://www.apache.org/licenses/LICENSE-2.0
	8	+#
	9	+# Unless required by applicable law or agreed to in writing, software
	10	+# distributed under the License is distributed on an "AS IS" BASIS,
	11	+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
	12	+# See the License for the specific language governing permissions and
	13	+# limitations under the License.
	14	+
	15	+set -ex
	16	+
	17	+HOME_DIR=$(pwd)
	18	+
	19	+rm -rf /workspace
	20	+mkdir -p /workspace && cd /workspace
	21	+
	22	+git clone https://gitcode.com/GitHub_Trending/ve/verl.git
	23	+cd verl
	24	+git checkout e9aa879bc61821621a36881ea305eaa0785520c1
	atomgit-bot6月23日 🔵 Low Priority `download_frameworks_source_code.sh` 第 22 行从 `https://gitcode.com/GitHub_Trending/ve/verl.git` 克隆 verl，而 Dockerfile（第 32 行）从 `https://github.com/volcengine/verl.git` 克隆。两者 checkout 到相同 commit（`e9aa879bc61821621a36881ea305eaa0785520c1`），但 `gitcode.com` 是第三方镜像，存在以下风险：镜像可能未及时同步，导致该 commit 在镜像上不存在；镜像仓库可能被篡改（虽然 git commit hash 提供完整性保护，但仓库历史可被重写）。建议统一使用同一来源（推荐 Dockerfile 中的官方 github.com 仓库），避免因来源不一致导致的构建差异。建议：将 verl 克隆源统一为 Dockerfile 中使用的 `https://github.com/volcengine/verl.git`，或至少添加注释说明为何使用不同源。不准确？ zhangjianshe 6月25日评论： 6月25日评论：
	25	+cd -
	26	+
	27	+git clone https://gitcode.com/GitHub_Trending/to/torchtitan.git
	28	+cd torchtitan
	29	+git checkout ac13e536c84e7f6647b14fa9375c3c8a8a2b8578
	30	+cd -
	31	+
	32	+git clone https://gitcode.com/cann/torchtitan-npu.git
	33	+cd torchtitan-npu
	34	+git checkout 29bbc8ba5bee5daf63f8a0c09512038449ffaf37
	35	+cd -
	36	+
	37	+git clone https://gitcode.com/gh_mirrors/vl/vllm-ascend.git
	38	+cd vllm-ascend
	39	+git checkout v0.15.0rc1
	40	+git submodule update --init --recursive
	41	+cd -
	42	+
	43	+cd $HOME_DIR
	atomgit-bot6月23日 🟡 Medium Priority `download_frameworks_source_code.sh` 存在两个问题：完全缺少 shebang：文件从第 1 行版权注释开始，第 15 行直接是 `set -ex`，未声明 `#!/bin/bash`。直接执行 `./download_frameworks_source_code.sh` 时，系统将使用默认 shell（`/bin/sh`），而脚本使用了 `set -o pipefail` 等 bash 特性。此问题上一轮审查已报告，本轮未修复。第 40 行 `cd $HOME_DIR` 变量未引号：若 `HOME_DIR` 包含空格或特殊字符（极少见但理论上可能），`cd` 行为异常。`HOME_DIR` 来自第 17 行 `HOME_DIR=$(pwd)`，在正常路径下安全，但不符合 shell 最佳实践。建议：在文件首行添加 `#!/bin/bash`；将 `cd $HOME_DIR` 改为 `cd "$HOME_DIR"`。不准确？ zhangjianshe 6月25日评论： 6月25日评论： atomgit-bot6月23日 🟡 Medium Priority `download_frameworks_source_code.sh` 存在两个健壮性问题（上一轮审查已报告，本轮未修复）：完全缺少 shebang：文件从第 1 行版权注释开始，第 15 行直接是 `set -ex`，未声明 `#!/bin/bash`。直接执行 `./download_frameworks_source_code.sh` 时，系统将使用默认 shell（`/bin/sh`）解释，而脚本使用了 `cd -`（bash 扩展）等非 POSIX 特性，可能导致执行失败。 `cd $HOME_DIR` 未引号：第 40 行 `cd $HOME_DIR` 中 `$HOME_DIR` 未加双引号。若 `HOME_DIR` 路径包含空格（虽然较少见），会导致 cd 失败或进入错误目录。建议：在文件首行添加 `#!/bin/bash`；将 `cd $HOME_DIR` 改为 `cd "$HOME_DIR"`。不准确？

Allm_rl/qwen3/verl-torchtitan/internal/train_grpo_qwen3_30b_a3b_16die.sh+106-0

@@ -0,0 +1,106 @@
	1	+#!/bin/bash
	2	+# Copyright (c) 2026 Huawei Technologies Co., Ltd. All Rights Reserved.
	3	+# Copyright 2024 Bytedance Ltd. and/or its affiliates
	4	+#
	5	+# Licensed under the Apache License, Version 2.0 (the "License");
	6	+# you may not use this file except in compliance with the License.
	7	+# You may obtain a copy of the License at
	8	+#
	9	+# http://www.apache.org/licenses/LICENSE-2.0
	10	+#
	11	+# Unless required by applicable law or agreed to in writing, software
	12	+# distributed under the License is distributed on an "AS IS" BASIS,
	13	+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
	14	+# See the License for the specific language governing permissions and
	15	+# limitations under the License.
	16	+
	17	+set -x
	atomgit-bot6月18日 🟡 Medium Priority `train_grpo_qwen3_30b_a3b_16die.sh` 训练启动脚本：缺少 shebang：文件第一行直接是 `set -x`，无 `#!/bin/bash`。直接执行时使用 `/bin/sh`。缺少 `set -e` 和 `set -u`：脚本仅设置了 `set -x`。若 `python3` 命令失败，脚本不会因错误退出（尽管它是最后一行，但若有后续代码或管道操作则可能吞掉错误）。`set -u` 缺失意味着未定义的变量将被静默展开为空字符串。第 21 行 `MODEL_PATH=/path/to/Qwen3-30B-A3B` 和第 27-28 行的数据路径均为占位符，直接用 `/path/to/` 前缀。如果用户在未修改配置的情况下直接运行脚本，训练将因路径不存在而失败。虽然在 README 中已说明需要替换，但脚本本身没有任何防护措施（如路径存在性检查），增加了用户误操作的风险。建议：在首行添加 `#!/bin/bash`，将 `set -x` 改为 `set -exuo pipefail`。对 `MODEL_PATH` 和数据路径添加存在性检查或提前退出提示。不准确？ zhangjianshe 6月22日评论： 6月22日评论：
	18	+export HYDRA_FULL_ERROR=1
	19	+export ASCEND_RT_VISIBLE_DEVICES=0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15
	20	+export RAY_EXPERIMENTAL_NOSET_ASCEND_RT_VISIBLE_DEVICES=1
	21	+export CUDA_DEVICE_MAX_CONNECTIONS=1
	22	+export CPU_AFFINITY_CONF=1
	23	+export TASK_QUEUE_ENABLE=2
	24	+export HCCL_CONNECT_TIMEOUT=3600
	25	+export STREAMS_PER_DEVICE=32
	26	+export MULTI_STREAM_MEMORY_RESERVE=2
	27	+export PYTORCH_NPU_ALLOC_CONF=max_split_size_mb:256
	28	+export HCCL_ALLOW_ALL_GATHER_INCONSISTENT=0
	29	+
	30	+VAL_BEFORE_TRAIN=${VAL_BEFORE_TRAIN:-False}
	31	+NUM_GPUS=${NUM_GPUS:-16}
	32	+FSDP_SIZE=${FSDP_SIZE:-16}
	33	+TP_SIZE=${TP_SIZE:-1}
	34	+EP_SIZE=${EP_SIZE:-16}
	35	+VERL_EXP_NAME=${VERL_EXP_NAME:-qwen3-30B-A3B-GRPO-torchtitan}
	36	+
	37	+MODEL_PATH=${MODEL_PATH:-"${HOME}/Qwen3-30B-A3B"}
	38	+
	39	+python3 -m verl.trainer.main_ppo \
	40	+ model_engine=torchtitan \
	41	+ algorithm.adv_estimator=grpo \
	42	+ data.seed=42 \
	43	+ data.train_files=/path/to/gsm8k/train.parquet \
	44	+ data.val_files=/path/to/gsm8k/test.parquet \
	45	+ data.train_batch_size=8 \
	46	+ data.max_prompt_length=256 \
	47	+ data.max_response_length=4096 \
	48	+ data.filter_overlong_prompts=True \
	49	+ data.truncation='error' \
	50	+ actor_rollout_ref.model.path="${MODEL_PATH}" \
	51	+ actor_rollout_ref.model.use_remove_padding=True \
	52	+ actor_rollout_ref.model.enable_gradient_checkpointing=False \
	53	+ actor_rollout_ref.model.enable_activation_offload=True \
	54	+ actor_rollout_ref.actor.optim.lr=5e-7 \
	55	+ actor_rollout_ref.actor.optim.min_lr_factor=1.0 \
	56	+ actor_rollout_ref.actor.ppo_mini_batch_size=8 \
	57	+ actor_rollout_ref.actor.ppo_micro_batch_size_per_gpu=1 \
	58	+ actor_rollout_ref.actor.torchtitan.data_parallel_shard_size="${FSDP_SIZE}" \
	59	+ actor_rollout_ref.actor.torchtitan.tensor_parallel_size="${TP_SIZE}" \
	60	+ actor_rollout_ref.actor.torchtitan.expert_parallel_size="${EP_SIZE}" \
	61	+ actor_rollout_ref.actor.torchtitan.attn_type=sdpa \
	62	+ actor_rollout_ref.actor.torchtitan.use_torch_compile=False \
	63	+ actor_rollout_ref.actor.torchtitan.param_offload=True \
	64	+ actor_rollout_ref.actor.torchtitan.optimizer_offload=True \
	65	+ actor_rollout_ref.actor.torchtitan.reshard_after_forward="always" \
	66	+ actor_rollout_ref.actor.torchtitan.mixed_precision=True \
	67	+ actor_rollout_ref.actor.torchtitan.entropy_checkpointing=True \
	68	+ actor_rollout_ref.actor.torchtitan.forward_prefetch=True \
	69	+ actor_rollout_ref.actor.use_kl_loss=True \
	70	+ actor_rollout_ref.actor.kl_loss_coef=0.001 \
	71	+ actor_rollout_ref.actor.kl_loss_type=low_var_kl \
	72	+ actor_rollout_ref.ref.profiler.enable=False \
	73	+ actor_rollout_ref.ref.use_torch_compile=False \
	74	+ actor_rollout_ref.ref.log_prob_micro_batch_size_per_gpu=1 \
	75	+ actor_rollout_ref.ref.torchtitan.use_torch_compile=False \
	76	+ actor_rollout_ref.ref.torchtitan.param_offload=True \
	77	+ actor_rollout_ref.ref.torchtitan.optimizer_offload=True \
	78	+ actor_rollout_ref.ref.torchtitan.reshard_after_forward="always" \
	79	+ actor_rollout_ref.ref.torchtitan.mixed_precision=True \
	80	+ actor_rollout_ref.rollout.profiler.enable=False \
	81	+ actor_rollout_ref.rollout.enforce_eager=True \
	82	+ actor_rollout_ref.rollout.log_prob_micro_batch_size_per_gpu=1 \
	83	+ actor_rollout_ref.rollout.enable_chunked_prefill=False \
	84	+ actor_rollout_ref.rollout.tensor_model_parallel_size=4 \
	85	+ actor_rollout_ref.rollout.data_parallel_size=4 \
	86	+ actor_rollout_ref.rollout.expert_parallel_size=1 \
	87	+ actor_rollout_ref.rollout.name=vllm \
	88	+ actor_rollout_ref.rollout.gpu_memory_utilization=0.7 \
	89	+ actor_rollout_ref.rollout.max_model_len=40960 \
	90	+ actor_rollout_ref.rollout.free_cache_engine=True \
	91	+ actor_rollout_ref.rollout.n=2 \
	92	+ actor_rollout_ref.rollout.trace.token2text=True \
	93	+ actor_rollout_ref.rollout.trace.backend='mlflow' \
	94	+ algorithm.kl_ctrl.kl_coef=0.001 \
	95	+ trainer.use_legacy_worker_impl=disable \
	96	+ trainer.logger=['console','mlflow'] \
	97	+ trainer.rollout_data_dir="outputs/rollout_samples" \
	98	+ trainer.experiment_name="${VERL_EXP_NAME}" \
	99	+ trainer.val_before_train="${VAL_BEFORE_TRAIN}" \
	100	+ trainer.n_gpus_per_node="${NUM_GPUS}" \
	101	+ trainer.del_local_ckpt_after_load=True \
	102	+ trainer.nnodes=1 \
	103	+ trainer.save_freq=-1 \
	104	+ trainer.test_freq=10 \
	105	+ trainer.total_epochs=10 \
	106	+ trainer.total_training_steps=100 $@
	zhanghz16月26日以下AI说的，研究确认下改正确：问题：trainer.total_training_steps=100 $@ 中 $@ 未加引号，含空格的透传参数会被错误地词分割。说明：mindspeed 现有脚本也是未加引号的 $@，属仓库历史习惯，故非本 PR 引入；但既然新写，建议直接用 "$ @" 修正，避免后续踩坑。 zhangjianshe 7月1日评论： 7月1日评论：

Allm_rl/qwen3/verl-torchtitan/patches/torchair/0001-torchair-bugfix-adapt_hint_int_import.patch+46-0

@@ -0,0 +1,46 @@
	1	+diff --git a/_ge_concrete_graph/continguous.py b/_ge_concrete_graph/continguous.py
	2	+index 6c624392..200a40df 100644
	3	+--- a/_ge_concrete_graph/continguous.py
	4	++++ b/_ge_concrete_graph/continguous.py
	5	+@@ -1,4 +1,7 @@
	6	+-from torch.fx.experimental.symbolic_shapes import hint_int
	7	++try:
	8	++ from torch.fx.experimental.symbolic_shapes import hint_int
	9	++except ImportError:
	10	++ from torch.fx.experimental.symbolic_shapes import optimization_hint as hint_int
	11	+ from torchair.core._concrete_graph import ValuePack
	12	+ from torchair.ge._ge_graph import Tensor, is_sym, DataType
	13	+ from torchair._ge_concrete_graph.utils import is_host_data_tensor, force_op_unknown_shape
	14	+diff --git a/_ge_concrete_graph/infer_symbol_shape.py b/_ge_concrete_graph/infer_symbol_shape.py
	15	+index 3a43b794..3193cc77 100644
	16	+--- a/_ge_concrete_graph/infer_symbol_shape.py
	17	++++ b/_ge_concrete_graph/infer_symbol_shape.py
	18	+@@ -7,7 +7,11 @@ import warnings
	19	+
	20	+ import torch
	21	+ from torch._subclasses.fake_tensor import FakeTensorMode
	22	+-from torch.fx.experimental.symbolic_shapes import ShapeEnv, DimDynamic, hint_int
	23	++try:
	24	++ from torch.fx.experimental.symbolic_shapes import hint_int
	25	++except ImportError:
	26	++ from torch.fx.experimental.symbolic_shapes import optimization_hint as hint_int
	27	++from torch.fx.experimental.symbolic_shapes import ShapeEnv, DimDynamic
	28	+ from torch._dynamo.source import GlobalSource
	29	+ from torchair.ge._ge_graph import torch_type_to_ge_type, torch_type_to_ge_type
	30	+ from torchair.ge._ge_graph import is_sym, sym_to_ge_dtype, ge_type_to_torch_type, _ge_proto_dtype_to_ge_dtype
	31	+diff --git a/_ge_concrete_graph/utils.py b/_ge_concrete_graph/utils.py
	32	+index 6bfebdcd..7188acdd 100644
	33	+--- a/_ge_concrete_graph/utils.py
	34	++++ b/_ge_concrete_graph/utils.py
	35	+@@ -5,7 +5,10 @@ from typing import Any, Dict, List, Tuple, Union, Callable
	36	+ import sympy
	37	+ import torch
	38	+ from torch.utils._mode_utils import no_dispatch
	39	+-from torch.fx.experimental.symbolic_shapes import hint_int
	40	++try:
	41	++ from torch.fx.experimental.symbolic_shapes import hint_int
	42	++except ImportError:
	43	++ from torch.fx.experimental.symbolic_shapes import optimization_hint as hint_int
	44	+ from torchair.core.utils import logger
	45	+ from torchair._ge_concrete_graph.ge_ir_pb2 import GraphDef
	46	+ from torchair.ge._ge_graph import compat_as_bytes, DataType, is_sym, Tensor, \

Allm_rl/qwen3/verl-torchtitan/patches/torchtitan/0001-torchtitan-bugfix-qwen3_init_for_npu.patch+24-0

@@ -0,0 +1,24 @@
	1	+diff --git a/llm_rl/qwen3/verl-torchtitan/torchtitan/models/qwen3/__init__.py b/llm_rl/qwen3/verl-torchtitan/torchtitan/models/qwen3/__init__.py
	2	+index 63e5a7269..f637b7c2e 100644
	3	+--- a/llm_rl/qwen3/verl-torchtitan/torchtitan/models/qwen3/__init__.py
	4	++++ b/llm_rl/qwen3/verl-torchtitan/torchtitan/models/qwen3/__init__.py
	5	+@@ -55,7 +55,8 @@ _EPS = 1e-6
	6	+ def _output_linear_init(dim: int) -> dict[str, Callable]:
	7	+ s = dim**-0.5
	8	+ return {
	9	+- "weight": partial(nn.init.trunc_normal_, std=s, a=-3 * s, b=3 * s),
	10	++ # NPU init path is incompatible with bounded trunc_normal_ parameters.
	11	++ "weight": partial(nn.init.trunc_normal_, std=s), #, a=-3 * s, b=3 * s),
	atomgit-bot6月18日 🔵 Low Priority patch `0001-torchtitan-fix-qwen3-init-for-npu.patch` 第 9 行将： "weight": partial(nn.init.trunc_normal_, std=s, a=-3 * s, b=3 * s), 改为： "weight": partial(nn.init.trunc_normal_, std=s), #, a=-3 * s, b=3 * s), 这会将 Qwen3 模型 `output_linear_init` 的权重初始化从截断正态分布（限制在 ±3σ）变为无界正态分布。如果该初始化函数在加载预训练权重后被调用（例如新增层或特定代码路径），则权重可能取到极端值，导致训练不稳定或 NaN。虽对于从预训练 checkpoint 加载的模型，该初始化可能不会被触发，但 patch 和 README 均未说明此变更的原因（是否为 NPU 上 `trunc_normal_` 不支持 `a`/`b` 参数的兼容性变通方案）。建议：确认移除 `trunc_normal_` 的截断参数是有意的（NPU 上的 `trunc_normal_` 可能不支持 `a`/`b` 参数）。如果是兼容性变通方案，应在 patch 注释或 README 中说明原因，以免被误认为是意外的初始化变更。不准确？ zhangjianshe 6月23日评论： 6月23日评论： zhanghz16月26日这种容易引起困惑的改动，都要注释说明原因 zhangjianshe 7月1日评论： 7月1日评论：
	12	+ "bias": nn.init.zeros_,
	13	+ }
	14	+
	15	+@@ -638,7 +639,8 @@ def _235b_a22b() -> Qwen3Model.Config:
	16	+ qwen3_configs = {
	17	+ "debugmodel": _debugmodel,
	18	+ "debugmodel_flex": _debugmodel_flex,
	19	+- "debugmodel_flex_flash": _debugmodel_flex_flash,
	20	++ # flex_flash debug flavor depends on a flash path that is not enabled for NPU.
	21	++ # "debugmodel_flex_flash": _debugmodel_flex_flash,
	zhanghz16月26日这里为什么注释掉，添加注释说明原因 zhangjianshe 7月1日评论： 7月1日评论：
	22	+ "debugmodel_varlen": _debugmodel_varlen,
	23	+ "0.6B": _0_6b,
	24	+ "0.6B_varlen": _0_6b_varlen,

Allm_rl/qwen3/verl-torchtitan/patches/verl/0001-verl-feature-add_torchtitan_npu.patch+107-0

@@ -0,0 +1,107 @@
	1	+diff --git a/llm_rl/qwen3/verl-torchtitan/verl/utils/device.py b/llm_rl/qwen3/verl-torchtitan/verl/utils/device.py
	2	+index e0e917a..fd10a9d 100644
	3	+--- a/llm_rl/qwen3/verl-torchtitan/verl/utils/device.py
	4	++++ b/llm_rl/qwen3/verl-torchtitan/verl/utils/device.py
	5	+@@ -144,6 +144,8 @@ def set_expandable_segments(enable: bool) -> None:
	6	+ """
	7	+ if is_cuda_available:
	8	+ torch.cuda.memory._set_allocator_settings(f"expandable_segments:{enable}")
	9	++ if is_npu_available:
	10	++ torch.npu.memory._set_allocator_settings(f"expandable_segments:{enable}")
	11	+
	12	+
	13	+ def auto_set_device(config) -> None:
	14	+diff --git a/llm_rl/qwen3/verl-torchtitan/verl/utils/vllm/npu_vllm_patch.py b/llm_rl/qwen3/verl-torchtitan/verl/utils/vllm/npu_vllm_patch.py
	15	+index 727afc8..ccfc080 100644
	16	+--- a/llm_rl/qwen3/verl-torchtitan/verl/utils/vllm/npu_vllm_patch.py
	17	++++ b/llm_rl/qwen3/verl-torchtitan/verl/utils/vllm/npu_vllm_patch.py
	18	+@@ -169,7 +169,7 @@ def patch_vllm013_rotary_emb():
	19	+ is_neox_style: bool = True,
	20	+ enable_fp32_compute: bool = False,
	21	+ ) -> None:
	22	+- super(ApplyRotaryEmb, self).__init__(enforce_enable)
	23	++ super(ApplyRotaryEmb, self).__init__(enforce_enable=enforce_enable)
	24	+ self.is_neox_style = is_neox_style
	25	+ self.enable_fp32_compute = enable_fp32_compute
	26	+ self.apply_rotary_emb_flash_attn = None
	27	+diff --git a/llm_rl/qwen3/verl-torchtitan/verl/workers/engine/torchtitan/transformer_impl.py b/llm_rl/qwen3/verl-torchtitan/verl/workers/engine/torchtitan/transformer_impl.py
	28	+index 739ccf2..bc74755 100644
	29	+--- a/llm_rl/qwen3/verl-torchtitan/verl/workers/engine/torchtitan/transformer_impl.py
	30	++++ b/llm_rl/qwen3/verl-torchtitan/verl/workers/engine/torchtitan/transformer_impl.py
	31	+@@ -29,6 +29,14 @@ from tensordict import TensorDict
	32	+ from torch.distributed.checkpoint.state_dict import get_model_state_dict
	33	+ from torch.distributed.tensor import DTensor
	34	+ from torchtitan.components.checkpoint import CheckpointManager
	35	++try:
	36	++ import torchtitan_npu
	37	++except ImportError as exc:
	38	++ raise ImportError(
	39	++ "torchtitan_npu is required for the Qwen3 TorchTitan NPU recipe. "
	40	++ "Run build_project.sh to copy torchtitan_npu into the recipe directory."
	41	++ ) from exc
	42	++
	43	+ from torchtitan.components.lr_scheduler import LRSchedulersContainer
	44	+ from torchtitan.components.optimizer import OptimizersContainer
	45	+ from torchtitan.config import CompileConfig, ParallelismConfig, TrainingConfig
	46	+@@ -69,6 +77,22 @@ logger.setLevel(os.getenv("VERL_LOGGING_LEVEL", "WARN"))
	47	+ device_name = get_device_name()
	48	+
	49	+
	50	++def apply_npu_patcher(config: Trainer.Config):
	51	++ model_name = config.model_spec.name if config.model_spec else None
	52	++ flavor = config.model_spec.flavor if config.model_spec else None
	53	++
	54	++ from torchtitan.protocols.model_converter import ModelConvertersContainer
	55	++ from torchtitan_npu.converters.registry import get_model_converter_config
	56	++ config.model_converters = ModelConvertersContainer.Config(
	57	++ converters=[
	58	++ get_model_converter_config("npu_rms_norm"),
	59	++ get_model_converter_config("npu_rope"),
	60	++ get_model_converter_config("npu_permute"),
	61	++ get_model_converter_config("npu_gmm"),
	62	++ ],
	63	++ )
	64	++ config.activation_checkpoint.mode='full'
	65	++
	66	+ class TorchTitanEngine(BaseEngine):
	67	+ """
	68	+ Concrete Engine implementation using PyTorch TorchTitan parallelism.
	69	+@@ -171,6 +195,7 @@ class TorchTitanEngine(BaseEngine):
	70	+ # Use a no-op dataloader since verl has its own data loading
	71	+ dataloader=NoOpDataLoader.Config(),
	72	+ )
	73	++ apply_npu_patcher(self.config)
	74	+ self.trainer = Trainer(self.config)
	75	+
	76	+ self._init_device_mesh()
	77	+diff --git a/llm_rl/qwen3/verl-torchtitan/verl/workers/engine/torchtitan/utils.py b/llm_rl/qwen3/verl-torchtitan/verl/workers/engine/torchtitan/utils.py
	78	+index a88c82e..8de5d1a 100644
	79	+--- a/llm_rl/qwen3/verl-torchtitan/verl/workers/engine/torchtitan/utils.py
	80	++++ b/llm_rl/qwen3/verl-torchtitan/verl/workers/engine/torchtitan/utils.py
	81	+@@ -123,10 +123,11 @@ def derive_torchtitan_name_and_flavor(hf_config) -> tuple[str, str]:
	82	+ num_layers = hf_config.num_hidden_layers
	83	+ vocab_size = hf_config.vocab_size
	84	+
	85	+- for flavor_name, model_cfg in model_configs.items():
	86	++ for flavor_name, model_fun in model_configs.items():
	87	++ model_cfg=model_fun()
	88	+ if (
	89	+ getattr(model_cfg, "dim", None) == hidden_size
	90	+- and getattr(model_cfg, "n_layers", None) == num_layers
	91	++ and len(getattr(model_cfg, "layers", []) or []) == num_layers
	92	+ and getattr(model_cfg, "vocab_size", None) == vocab_size
	93	+ ):
	94	+ logger.info(
	95	+diff --git a/llm_rl/qwen3/verl-torchtitan/verl/workers/engine_workers.py b/llm_rl/qwen3/verl-torchtitan/verl/workers/engine_workers.py
	96	+index 8784d3a..df88fb2 100644
	97	+--- a/llm_rl/qwen3/verl-torchtitan/verl/workers/engine_workers.py
	98	++++ b/llm_rl/qwen3/verl-torchtitan/verl/workers/engine_workers.py
	99	+@@ -405,7 +405,7 @@ class TrainingWorker(Worker, DistProfilerExtension):
	100	+ ).cpu()
	101	+ else:
	102	+ final_output = None
	103	+-
	104	++ aggressive_empty_cache(force_sync=True)
	105	+ return final_output
	106	+
	107	+ @register(dispatch_mode=Dispatch.ONE_TO_ALL)

Allm_rl/qwen3/verl-torchtitan/patches/vllm_ascend/0001-vllm_ascend-feature-align_torch_2_12_build.patch+54-0

@@ -0,0 +1,54 @@
	1	+diff --git a/llm_rl/qwen3/verl-torchtitan/vllm-ascend/CMakeLists.txt b/llm_rl/qwen3/verl-torchtitan/vllm-ascend/CMakeLists.txt
	2	+index 9e3564e3..879318c4 100644
	3	+--- a/llm_rl/qwen3/verl-torchtitan/vllm-ascend/CMakeLists.txt
	4	++++ b/llm_rl/qwen3/verl-torchtitan/vllm-ascend/CMakeLists.txt
	5	+@@ -22,9 +22,9 @@ find_package(Torch REQUIRED)
	6	+
	7	+ run_python(TORCH_VERSION
	8	+ "import torch; print(torch.__version__)" "Failed to locate torch path")
	9	+-# check torch version is 2.9.0
	10	+-if(NOT ${TORCH_VERSION} VERSION_EQUAL "2.9.0")
	11	+- message(FATAL_ERROR "Expected PyTorch version 2.9.0, but found ${TORCH_VERSION}")
	12	++# check torch version is 2.12.0
	13	++if(NOT ${TORCH_VERSION} VERSION_EQUAL "2.12.0")
	14	++ message(FATAL_ERROR "Expected PyTorch version 2.12.0, but found ${TORCH_VERSION}")
	15	+ endif()
	16	+
	17	+ set(RUN_MODE "npu" CACHE STRING "cpu/sim/npu")
	18	+diff --git a/llm_rl/qwen3/verl-torchtitan/vllm-ascend/pyproject.toml b/llm_rl/qwen3/verl-torchtitan/vllm-ascend/pyproject.toml
	19	+index 061db046..e91a4bac 100644
	20	+--- a/llm_rl/qwen3/verl-torchtitan/vllm-ascend/pyproject.toml
	21	++++ b/llm_rl/qwen3/verl-torchtitan/vllm-ascend/pyproject.toml
	22	+@@ -18,8 +18,8 @@ requires = [
	23	+ "setuptools>=64",
	24	+ "setuptools-scm>=8",
	25	+ "transformers>=4.57.4",
	26	+- "torch-npu==2.9.0",
	27	+- "torch==2.9.0",
	28	++ "torch-npu==2.12.0rc1",
	29	++ "torch==2.12.0",
	30	+ "torchvision",
	31	+ "wheel",
	32	+ "msgpack",
	33	+diff --git a/llm_rl/qwen3/verl-torchtitan/vllm-ascend/requirements.txt b/llm_rl/qwen3/verl-torchtitan/vllm-ascend/requirements.txt
	34	+index b307c62e..90eb9295 100644
	35	+--- a/llm_rl/qwen3/verl-torchtitan/vllm-ascend/requirements.txt
	36	++++ b/llm_rl/qwen3/verl-torchtitan/vllm-ascend/requirements.txt
	37	+@@ -12,7 +12,7 @@ pandas
	38	+ psutil
	39	+ setuptools>=64
	40	+ setuptools-scm>=8
	41	+-torch==2.9.0
	42	++torch==2.12.0
	43	+ torchvision
	44	+ torchaudio
	45	+ wheel
	46	+@@ -31,7 +31,7 @@ numba
	47	+ # Install torch_npu
	48	+ #--pre
	49	+ #--extra-index-url https://mirrors.huaweicloud.com/ascend/repos/pypi
	50	+-torch-npu==2.9.0
	51	++torch-npu==2.12.0rc1
	52	+
	53	+ arctic-inference==0.1.1
	54	+ transformers>=4.57.4

Allm_rl/qwen3/verl-torchtitan/requirements.txt+12-0

@@ -0,0 +1,12 @@
	1	+torch-npu==2.12.0rc1
	2	+triton==3.5.0
	3	+tyro==1.0.13
	4	+tensorboard==2.20.0
	5	+torchdata==0.11.0
	6	+datasets==5.0.0
	7	+tensordict==0.13.0
	8	+omegaconf==2.3.1
	9	+hydra-core==1.3.3
	10	+codetiming==1.4.0
	11	+peft==0.19.1
	12	+mlflow==3.14.0

Allm_rl/qwen3/verl-torchtitan/run_container.sh+54-0

@@ -0,0 +1,54 @@
	1	+# Copyright (c) 2026 Huawei Technologies Co., Ltd. All Rights Reserved.
	2	+#
	3	+# Licensed under the Apache License, Version 2.0 (the "License");
	4	+# you may not use this file except in compliance with the License.
	5	+# You may obtain a copy of the License at
	6	+#
	7	+# http://www.apache.org/licenses/LICENSE-2.0
	8	+#
	9	+# Unless required by applicable law or agreed to in writing, software
	10	+# distributed under the License is distributed on an "AS IS" BASIS,
	11	+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
	12	+# See the License for the specific language governing permissions and
	13	+# limitations under the License.
	14	+
	15	+#!/bin/bash
	atomgit-bot6月18日 🔵 Low Priority `llm_rl/qwen3/torchtitan/` 下的三个脚本文件将版权声明头（第 1–14 行）放在了 `#!/bin/bash`（第 15 行）之前： `run_container.sh`（第 15 行） `apply_all_patches.sh`（第 15 行） `build_project.sh`（第 15 行） Unix 内核仅在文件第一行以 `#!` 开头时才会识别 shebang。当用户尝试以 `./run_container.sh` 的方式直接执行这些脚本时，内核无法找到 bash shebang，会回退到系统默认 shell（通常是 `/bin/sh`），从而导致行为不一致或脚本失败。虽然 README 中记录了应使用 `bash script.sh` 来调用，但作为可执行脚本，shebang 应当规范地放在第一行。另外，`download_frameworks_source_code.sh` 和 `internal/train_grpo_qwen3_30b_a3b_16die.sh` 完全缺少 shebang。建议：将 `#!/bin/bash` 移到每个文件的第 1 行，并将版权声明放在 shebang 之后。对于完全缺少 shebang 的脚本（`download_frameworks_source_code.sh` 和 `train_grpo_qwen3_30b_a3b_16die.sh`），在第 1 行添加 `#!/bin/bash`。不准确？ zhangjianshe 6月22日评论： 6月22日评论：
	16	+container_name=$1
	17	+image_name=$2
	18	+
	19	+# create
	20	+docker run -itd \
	21	+--device=/dev/davinci0 \
	22	+--device=/dev/davinci1 \
	23	+--device=/dev/davinci2 \
	24	+--device=/dev/davinci3 \
	25	+--device=/dev/davinci4 \
	26	+--device=/dev/davinci5 \
	27	+--device=/dev/davinci6 \
	28	+--device=/dev/davinci7 \
	29	+--device=/dev/davinci8 \
	30	+--device=/dev/davinci9 \
	31	+--device=/dev/davinci10 \
	32	+--device=/dev/davinci11 \
	33	+--device=/dev/davinci12 \
	34	+--device=/dev/davinci13 \
	35	+--device=/dev/davinci14 \
	36	+--device=/dev/davinci15 \
	37	+-v /usr/local/dcmi:/usr/local/dcmi \
	38	+-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
	39	+-v /var/log/npu/slog/slogd:/var/log/npu/slog/slogd \
	40	+-v /usr/local/sbin/:/usr/local/sbin/ \
	41	+-v /data/:/data/ \
	42	+-v /home/:/home/ \
	43	+-v /etc/localtime:/etc/localtime \
	44	+-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
	45	+-v /dev/shm:/dev/shm \
	46	+--device=/dev/davinci_manager \
	47	+--device=/dev/devmm_svm \
	48	+--device=/dev/hisi_hdc \
	49	+--net=host \
	50	+--name ${container_name} \
	51	+--privileged ${image_name} /bin/bash
	52	+
	53	+# execute
	54	+docker exec -it -u root ${container_name} bash
	atomgit-bot6月18日 🟡 Medium Priority `run_container.sh` 存在多个健壮性问题： shebang 不在首行（第 15 行而非第 1 行）：直接执行 `./run_container.sh` 时使用 `/bin/sh`，可能不支持 `[[ ]]`（不过本脚本未使用该语法，影响较小）。缺少 `set -e`：第 20 行 `docker run` 创建容器如果失败，第 54 行的 `docker exec` 仍会尝试执行，导致错误信息混乱。变量未加引号：第 50 行 `--name ${container_name}` 和第 51 行 `${image_name}` 未加引号。如果参数值包含空格或特殊字符，会引发命令解析错误。虽然容器名和镜像名通常不含空格，但作为脚本规范应加引号保护。 `--privileged` 模式：赋予容器几乎完整的宿主机权限，是一个安全风险点。对于 NPU 训练场景，部分特权确实是必要的，但应考虑是否可降级为更细粒度的 `--cap-add` 或 `--device` 控制。建议：1. 将 `#!/bin/bash` 移到第一行；2. 添加 `set -euo pipefail`；3. 对 `${container_name}` 和 `${image_name}` 加双引号；4. 评估 `--privileged` 是否可替换为更精细的权限控制。不准确？ zhangjianshe 6月22日评论： 6月22日评论：

	@@ -5,10 +5,10 @@
5	- [2026/04] 新增DeepSeek-V3模型[MXFP8/HiF8 低精度预训练](llm_pretrain/DeepSeekV3/README.md)样例。	5	- [2026/04] 新增DeepSeek-V3模型[MXFP8/HiF8 低精度预训练](llm_pretrain/DeepSeekV3/README.md)样例。
6	- [2026/04] 新增[DeepSeek-V4-Flash模型续训练0day支持](llm_pretrain/deepseekv4/README.md)（基于torchtitan框架）样例，支持训练入图、AutoFuse特性。	6	- [2026/04] 新增[DeepSeek-V4-Flash模型续训练0day支持](llm_pretrain/deepseekv4/README.md)（基于torchtitan框架）样例，支持训练入图、AutoFuse特性。
7	- [2026/02] 新增DeepSeek-V3.2模型[torchtitan框架预训练](llm_pretrain/deepseekv32/README.md)样例。	7	- [2026/02] 新增DeepSeek-V3.2模型[torchtitan框架预训练](llm_pretrain/deepseekv32/README.md)样例。
8	-- [2026/02] 新增Qwen3系列模型RL训练使能[npugraph_ex图模式](llm_rl/qwen3/README.md)样例。	8	+- [2026/02] 新增Qwen3系列模型RL训练使能[npugraph_ex图模式](llm_rl/qwen3/verl-mindspeed/README.md)样例。
9	- [2025/12] 新增Qwen2.5/Qwen3模型Code RL长上下文代码生成强化学习样例。	9	- [2025/12] 新增Qwen2.5/Qwen3模型Code RL长上下文代码生成强化学习样例。
10	-- [2025/12] 新增Qwen3系列模型RL训练使能[SAM投机推理](llm_rl/qwen3/README.md)、[tool agent RL](agent_rl/qwen3_tool_agent/README.md)样例。	10	+- [2025/12] 新增Qwen3系列模型RL训练使能[SAM投机推理](llm_rl/qwen3/verl-mindspeed/README.md)、[tool agent RL](agent_rl/qwen3_tool_agent/README.md)样例。
11	-- [2025/11] [Qwen3模型长序列RL](llm_rl/qwen3/README.md)样例首次上线。	11	+- [2025/11] [Qwen3模型长序列RL](llm_rl/qwen3/verl-mindspeed/README.md)样例首次上线。
12	- [2025/10] [DeepSeek-R1](llm_rl/deepseek/README.md)、[Qwen2.5模型](llm_rl/qwen2_5/verl_npu_demo/README.md)样例首次上线。	12	- [2025/10] [DeepSeek-R1](llm_rl/deepseek/README.md)、[Qwen2.5模型](llm_rl/qwen2_5/verl_npu_demo/README.md)样例首次上线。
13		13
14	## 🎉 概述	14	## 🎉 概述
	@@ -21,8 +21,8 @@ cann-recipes-train仓库旨在针对LLM与多模态模型训练业务中的典
21	\|-----\|-----\|	21	\|-----\|-----\|
22	\|[DeepSeek-R1 RL训练优化样例](llm_rl/deepseek/README.md) \|基于开源veRL框架，搭配MindSpeed+vLLM-Ascend框架，在Atlas A3集群实现GRPO算法的高吞吐RL训练，并达到120TPS/卡的系统吞吐量。\|	22	\|[DeepSeek-R1 RL训练优化样例](llm_rl/deepseek/README.md) \|基于开源veRL框架，搭配MindSpeed+vLLM-Ascend框架，在Atlas A3集群实现GRPO算法的高吞吐RL训练，并达到120TPS/卡的系统吞吐量。\|
23	\|[基于verl框架的Qwen2.5强化学习（入门样例）](llm_rl/qwen2_5/verl_npu_demo/README.md) \|基于Qwen2.5-1.5B-Instruct模型，采用verl强化学习框架，在MATH-lighteval数学推理数据集上进行了训练。本样例只需要单卡Atlas A2环境，帮助大家快速上手，使用昇腾NPU完成RL训练任务。\|	23	\|[基于verl框架的Qwen2.5强化学习（入门样例）](llm_rl/qwen2_5/verl_npu_demo/README.md) \|基于Qwen2.5-1.5B-Instruct模型，采用verl强化学习框架，在MATH-lighteval数学推理数据集上进行了训练。本样例只需要单卡Atlas A2环境，帮助大家快速上手，使用昇腾NPU完成RL训练任务。\|
24	-\|[Qwen3-235B-A22B RL训练优化样例](llm_rl/qwen3/README.md) \| 基于开源veRL框架，搭配MindSpeed+vLLM-Ascend框架，在Atlas A3集群实现GRPO/DAPO算法的长序列 2k+32k训练，GRPO达到120TPS/卡的系统吞吐量。\|	24	+\|[Qwen3-235B-A22B RL训练优化样例](llm_rl/qwen3/verl-mindspeed/README.md) \| 基于开源veRL框架，搭配MindSpeed+vLLM-Ascend框架，在Atlas A3集群实现GRPO/DAPO算法的长序列 2k+32k训练，GRPO达到120TPS/卡的系统吞吐量。\|
25	-\|[Qwen3-32B RL训练使能SAM投机推理样例](llm_rl/qwen3/README.md) \| 基于开源veRL框架，搭配MindSpeed+vLLM-Ascend框架，在Atlas A3集群，GRPO/DAPO算法的2k+32k训练场景下，使能SAM投机推理特性，达成10%性能提升。\|	25	+\|[Qwen3-32B RL训练使能SAM投机推理样例](llm_rl/qwen3/verl-mindspeed/README.md) \| 基于开源veRL框架，搭配MindSpeed+vLLM-Ascend框架，在Atlas A3集群，GRPO/DAPO算法的2k+32k训练场景下，使能SAM投机推理特性，达成10%性能提升。\|
26	\|[Qwen3 tool agent RL训练样例](agent_rl/qwen3_tool_agent/README.md) \|基于verl/recipe中的retool项目，调用Sandbox工具，使能`asyncLLM`和`agent_loop`特性，在昇腾NPU上完成端到端agent RL训练任务。\|	26	\|[Qwen3 tool agent RL训练样例](agent_rl/qwen3_tool_agent/README.md) \|基于verl/recipe中的retool项目，调用Sandbox工具，使能`asyncLLM`和`agent_loop`特性，在昇腾NPU上完成端到端agent RL训练任务。\|
27	\|[基于ScaleBox沙盒的Code RL训练样例](agent_rl/qwen2_code_rl/README.md) \|基于verl框架和ScaleBox代码沙盒，支持长上下文(2k+16k) Code RL训练，Qwen3-30B-A3B在LiveCodeBench上Pass@1从46.59提升至56.27。\|	27	\|[基于ScaleBox沙盒的Code RL训练样例](agent_rl/qwen2_code_rl/README.md) \|基于verl框架和ScaleBox代码沙盒，支持长上下文(2k+16k) Code RL训练，Qwen3-30B-A3B在LiveCodeBench上Pass@1从46.59提升至56.27。\|
28	\|[DeepSeek-V3.2 Pretrain训练样例](llm_pretrain/deepseekv32/README.md) \|基于torchtitan，在64卡Atlas A3集群上完成DeepSeek-V3.2模型32K长序列预训练复现，吞吐达成148 TPS/卡。\|	28	\|[DeepSeek-V3.2 Pretrain训练样例](llm_pretrain/deepseekv32/README.md) \|基于torchtitan，在64卡Atlas A3集群上完成DeepSeek-V3.2模型32K长序列预训练复现，吞吐达成148 TPS/卡。\|

	@@ -6,13 +6,13 @@ Add state_ten value check in the DistributedOptimizer class to prevent runtime e
6	thereby improving code robustness.	6	thereby improving code robustness.
7		7
8	---	8	---
9	- llm_rl/qwen3/megatron/core/optimizer/distrib_optimizer.py \| 2 ++	9	+ llm_rl/qwen3/verl-mindspeed/megatron/core/optimizer/distrib_optimizer.py \| 2 ++
10	1 file changed, 2 insertions(+)	10	1 file changed, 2 insertions(+)
11		11
12	-diff --git a/llm_rl/qwen3/megatron/core/optimizer/distrib_optimizer.py b/llm_rl/qwen3/megatron/core/optimizer/distrib_optimizer.py	12	+diff --git a/llm_rl/qwen3/verl-mindspeed/megatron/core/optimizer/distrib_optimizer.py b/llm_rl/qwen3/verl-mindspeed/megatron/core/optimizer/distrib_optimizer.py
13	index a00d1a3..62fe20f 100644	13	index a00d1a3..62fe20f 100644
14	---- a/llm_rl/qwen3/megatron/core/optimizer/distrib_optimizer.py	14	+--- a/llm_rl/qwen3/verl-mindspeed/megatron/core/optimizer/distrib_optimizer.py
15	-+++ b/llm_rl/qwen3/megatron/core/optimizer/distrib_optimizer.py	15	++++ b/llm_rl/qwen3/verl-mindspeed/megatron/core/optimizer/distrib_optimizer.py
16	@@ -1319,6 +1319,8 @@ class DistributedOptimizer(MixedPrecisionOptimizer):	16	@@ -1319,6 +1319,8 @@ class DistributedOptimizer(MixedPrecisionOptimizer):
17	# The optimizer state of STEP is handled	17	# The optimizer state of STEP is handled
18	# specifically and is read from param_groups.	18	# specifically and is read from param_groups.
	@@ -22,6 +22,6 @@ index a00d1a3..62fe20f 100644
22	replace_kwargs = dict(	22	replace_kwargs = dict(
23	key=f'{prefix}.{state_key}.{sharded_metadata.key}',	23	key=f'{prefix}.{state_key}.{sharded_metadata.key}',
24	data=state_ten,	24	data=state_ten,
25	---	25	+--
26	2.50.1.windows.1	26	2.50.1.windows.1
27		27

	@@ -7,21 +7,21 @@ Subject: [PATCH] megatron-feature-enable_hdp
7	.../megatron/core/models/common/embeddings/rope_utils.py \| 7 +++++++	7	.../megatron/core/models/common/embeddings/rope_utils.py \| 7 +++++++
8	1 file changed, 7 insertions(+)	8	1 file changed, 7 insertions(+)
9		9
10	-diff --git a/llm_rl/qwen3/megatron/core/models/common/embeddings/rope_utils.py b/llm_rl/qwen3/megatron/core/models/common/embeddings/rope_utils.py	10	+diff --git a/llm_rl/qwen3/verl-mindspeed/megatron/core/models/common/embeddings/rope_utils.py b/llm_rl/qwen3/verl-mindspeed/megatron/core/models/common/embeddings/rope_utils.py
11	index 08c9457..b38a6ca 100644	11	index 08c9457..b38a6ca 100644
12	---- a/llm_rl/qwen3/megatron/core/models/common/embeddings/rope_utils.py	12	+--- a/llm_rl/qwen3/verl-mindspeed/megatron/core/models/common/embeddings/rope_utils.py
13	-+++ b/llm_rl/qwen3/megatron/core/models/common/embeddings/rope_utils.py	13	++++ b/llm_rl/qwen3/verl-mindspeed/megatron/core/models/common/embeddings/rope_utils.py
14	@@ -43,6 +43,8 @@ try:	14	@@ -43,6 +43,8 @@ try:
15	except ImportError:	15	except ImportError:
16	apply_rotary_emb_flash = None	16	apply_rotary_emb_flash = None
17	-	17	+
18	+from patches.verl.utils.hybrid_data_parallel.utils import set_batch_hdp_group, get_batch_hdp_group	18	+from patches.verl.utils.hybrid_data_parallel.utils import set_batch_hdp_group, get_batch_hdp_group
19	+	19	+
20	-	20	+
21	__all__ = ['apply_rotary_emb_flash']	21	__all__ = ['apply_rotary_emb_flash']
22	-	22	+
23	@@ -159,6 +161,11 @@ def _apply_rotary_pos_emb_thd(	23	@@ -159,6 +161,11 @@ def _apply_rotary_pos_emb_thd(
24	-	24	+
25	cp_size = parallel_state.get_context_parallel_world_size()	25	cp_size = parallel_state.get_context_parallel_world_size()
26	cp_rank = parallel_state.get_context_parallel_rank()	26	cp_rank = parallel_state.get_context_parallel_rank()
27	+ batch_hdp_group = get_batch_hdp_group()	27	+ batch_hdp_group = get_batch_hdp_group()
	@@ -31,7 +31,7 @@ index 08c9457..b38a6ca 100644
31	+ cp_rank = hdp_group.index(cp_rank)	31	+ cp_rank = hdp_group.index(cp_rank)
32	cu_seqlens = cu_seqlens // cp_size	32	cu_seqlens = cu_seqlens // cp_size
33	seqlens = (cu_seqlens[1:] - cu_seqlens[:-1]).tolist()	33	seqlens = (cu_seqlens[1:] - cu_seqlens[:-1]).tolist()
34	-	34	+
35	---	35	+--
36	2.50.1.windows.1	36	2.50.1.windows.1
37		37

	@@ -5,23 +5,23 @@ Subject: [PATCH] mindspeed-feature-enable_hdp
5		5
6	---	6	---
7	.../ring_context_parallel/ring_context_parallel.py \| 12 ++++++++++++	7	.../ring_context_parallel/ring_context_parallel.py \| 12 ++++++++++++
8	- .../qwen3/mindspeed/core/context_parallel/utils.py \| 4 ++++	8	+ .../qwen3/verl-mindspeed/core/context_parallel/utils.py \| 4 ++++
9	2 files changed, 16 insertions(+)	9	2 files changed, 16 insertions(+)
10		10
11	-diff --git a/llm_rl/qwen3/mindspeed/core/context_parallel/ring_context_parallel/ring_context_parallel.py b/llm_rl/qwen3/mindspeed/core/context_parallel/ring_context_parallel/ring_context_parallel.py	11	+diff --git a/llm_rl/qwen3/verl-mindspeed/mindspeed/core/context_parallel/ring_context_parallel/ring_context_parallel.py b/llm_rl/qwen3/verl-mindspeed/mindspeed/core/context_parallel/ring_context_parallel/ring_context_parallel.py
12	index c545e00..cc4095c 100644	12	index c545e00..cc4095c 100644
13	---- a/llm_rl/qwen3/mindspeed/core/context_parallel/ring_context_parallel/ring_context_parallel.py	13	+--- a/llm_rl/qwen3/verl-mindspeed/mindspeed/core/context_parallel/ring_context_parallel/ring_context_parallel.py
14	-+++ b/llm_rl/qwen3/mindspeed/core/context_parallel/ring_context_parallel/ring_context_parallel.py	14	++++ b/llm_rl/qwen3/verl-mindspeed/mindspeed/core/context_parallel/ring_context_parallel/ring_context_parallel.py
15	@@ -11,6 +11,7 @@ from mindspeed.ops.fusion_attention_v2 import npu_fusion_attention, npu_fusion_a	15	@@ -11,6 +11,7 @@ from mindspeed.ops.fusion_attention_v2 import npu_fusion_attention, npu_fusion_a
16	from mindspeed.core.context_parallel import get_args	16	from mindspeed.core.context_parallel import get_args
17	from mindspeed.core.context_parallel.ring_context_parallel.context_parallel_kv_cache import ContextParallelKVCache	17	from mindspeed.core.context_parallel.ring_context_parallel.context_parallel_kv_cache import ContextParallelKVCache
18	from mindspeed.core.context_parallel.utils import RingP2P, tnd_out_update, causal_out_update, general_out_update, forward_update, unflatten_softmax, flatten_softmax, get_selection_indices_for_tnd_softmax_update	18	from mindspeed.core.context_parallel.utils import RingP2P, tnd_out_update, causal_out_update, general_out_update, forward_update, unflatten_softmax, flatten_softmax, get_selection_indices_for_tnd_softmax_update
19	+from patches.verl.utils.hybrid_data_parallel.utils import get_batch_hdp_group	19	+from patches.verl.utils.hybrid_data_parallel.utils import get_batch_hdp_group
20	-	20	+
21	-	21	+
22	def causal_forward_fetch(q_block_id, kv_block_id, q, cur_k, cur_v, attn_mask=None):	22	def causal_forward_fetch(q_block_id, kv_block_id, q, cur_k, cur_v, attn_mask=None):
23	@@ -1213,6 +1214,17 @@ class AttentionWithCp(torch.autograd.Function):	23	@@ -1213,6 +1214,17 @@ class AttentionWithCp(torch.autograd.Function):
24	-	24	+
25	def ringattn_context_parallel(q, k, v, n, cp_para, softmax_scale=None, attn_mask=None, dropout_p=0.,	25	def ringattn_context_parallel(q, k, v, n, cp_para, softmax_scale=None, attn_mask=None, dropout_p=0.,
26	packed_seq_params=None, shapes=None):	26	packed_seq_params=None, shapes=None):
27	+ batch_hdp_group = get_batch_hdp_group()	27	+ batch_hdp_group = get_batch_hdp_group()
	@@ -38,21 +38,21 @@ index c545e00..cc4095c 100644
38	AttentionWithCp.block_size = q.shape[0]	38	AttentionWithCp.block_size = q.shape[0]
39	AttentionWithCp.batch_size = q.shape[1]	39	AttentionWithCp.batch_size = q.shape[1]
40	out = AttentionWithCp.apply(	40	out = AttentionWithCp.apply(
41	-diff --git a/llm_rl/qwen3/mindspeed/core/context_parallel/utils.py b/llm_rl/qwen3/mindspeed/core/context_parallel/utils.py	41	+diff --git a/llm_rl/qwen3/verl-mindspeed/mindspeed/core/context_parallel/utils.py b/llm_rl/qwen3/verl-mindspeed/mindspeed/core/context_parallel/utils.py
42	index 4d96365..0046388 100644	42	index 4d96365..0046388 100644
43	---- a/llm_rl/qwen3/mindspeed/core/context_parallel/utils.py	43	+--- a/llm_rl/qwen3/verl-mindspeed/mindspeed/core/context_parallel/utils.py
44	-+++ b/llm_rl/qwen3/mindspeed/core/context_parallel/utils.py	44	++++ b/llm_rl/qwen3/verl-mindspeed/mindspeed/core/context_parallel/utils.py
45	@@ -136,6 +136,10 @@ class RingP2P:	45	@@ -136,6 +136,10 @@ class RingP2P:
46	self.next, self.prev = self.prev, self.next	46	self.next, self.prev = self.prev, self.next
47	-	47	+
48	self.send_recv_ops = []	48	self.send_recv_ops = []
49	-+	49	++
50	+ ring_rank_prev = ring_global_ranks.index(self.prev)	50	+ ring_rank_prev = ring_global_ranks.index(self.prev)
51	+ if self.ring_rank % 2 == 0 and ring_rank_prev % 2 == 0:	51	+ if self.ring_rank % 2 == 0 and ring_rank_prev % 2 == 0:
52	+ self.group_for_send_recv_overlap = group	52	+ self.group_for_send_recv_overlap = group
53	-	53	+
54	def async_send_recv(self, send_tensor, recv_tensor, shapes=None):	54	def async_send_recv(self, send_tensor, recv_tensor, shapes=None):
55	if self.ring_rank % 2 == 0:	55	if self.ring_rank % 2 == 0:
56	---	56	+--
57	2.50.1.windows.1	57	2.50.1.windows.1
58		58

	@@ -9,10 +9,10 @@ Helps monitor prgress during long or large-batch rollouts in real time.
9	.../verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py \| 2 +-	9	.../verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py \| 2 +-
10	1 file changed, 1 insertion(+), 1 deletion(-)	10	1 file changed, 1 insertion(+), 1 deletion(-)
11		11
12	-diff --git a/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py b/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	12	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py b/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
13	index 40613f1..990e90d 100644	13	index 40613f1..990e90d 100644
14	---- a/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	14	+--- a/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
15	-+++ b/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	15	++++ b/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
16	@@ -345,7 +345,7 @@ class vLLMRollout(BaseRollout):	16	@@ -345,7 +345,7 @@ class vLLMRollout(BaseRollout):
17	prompts=vllm_inputs, # because we have already convert it to prompt token id	17	prompts=vllm_inputs, # because we have already convert it to prompt token id
18	sampling_params=self.sampling_params,	18	sampling_params=self.sampling_params,
	@@ -20,8 +20,8 @@ index 40613f1..990e90d 100644
20	- use_tqdm=False,	20	- use_tqdm=False,
21	+ use_tqdm=True,	21	+ use_tqdm=True,
22	)	22	)
23	-	23	+
24	# TODO(sgm): disable logprob when recompute_log_prob is enable	24	# TODO(sgm): disable logprob when recompute_log_prob is enable
25	---	25	+--
26	2.50.1.windows.1	26	2.50.1.windows.1
27		27

	@@ -7,15 +7,15 @@ Disables repeat interleaving during batch repetition, and applies fixed reorderi
7	to reduce per-NPU latency variance.	7	to reduce per-NPU latency variance.
8		8
9	---	9	---
10	- llm_rl/qwen3/verl/trainer/ppo/ray_trainer.py \| 16 +++++++++++++++-	10	+ llm_rl/qwen3/verl-mindspeed/verl/trainer/ppo/ray_trainer.py \| 16 +++++++++++++++-
11	1 file changed, 15 insertions(+), 1 deletion(-)	11	1 file changed, 15 insertions(+), 1 deletion(-)
12		12
13	-diff --git a/llm_rl/qwen3/verl/trainer/ppo/ray_trainer.py b/llm_rl/qwen3/verl/trainer/ppo/ray_trainer.py	13	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/trainer/ppo/ray_trainer.py b/llm_rl/qwen3/verl-mindspeed/verl/trainer/ppo/ray_trainer.py
14	index 6f50a6e..0abdd34 100644	14	index 6f50a6e..0abdd34 100644
15	---- a/llm_rl/qwen3/verl/trainer/ppo/ray_trainer.py	15	+--- a/llm_rl/qwen3/verl-mindspeed/verl/trainer/ppo/ray_trainer.py
16	-+++ b/llm_rl/qwen3/verl/trainer/ppo/ray_trainer.py	16	++++ b/llm_rl/qwen3/verl-mindspeed/verl/trainer/ppo/ray_trainer.py
17	@@ -1037,7 +1037,14 @@ class RayPPOTrainer:	17	@@ -1037,7 +1037,14 @@ class RayPPOTrainer:
18	-	18	+
19	# pass global_steps to trace	19	# pass global_steps to trace
20	gen_batch.meta_info["global_steps"] = self.global_steps	20	gen_batch.meta_info["global_steps"] = self.global_steps
21	- gen_batch = gen_batch.repeat(repeat_times=self.config.actor_rollout_ref.rollout.n, interleave=True)	21	- gen_batch = gen_batch.repeat(repeat_times=self.config.actor_rollout_ref.rollout.n, interleave=True)
	@@ -27,30 +27,30 @@ index 6f50a6e..0abdd34 100644
27	+ if data_rebalance:	27	+ if data_rebalance:
28	+ interleave_indices = torch.arange(gen_batch.batch.batch_size[0]).view(	28	+ interleave_indices = torch.arange(gen_batch.batch.batch_size[0]).view(
29	+ -1, batch.batch.batch_size[0]).transpose(1, 0).reshape(-1)	29	+ -1, batch.batch.batch_size[0]).transpose(1, 0).reshape(-1)
30	-	30	+
31	is_last_step = self.global_steps >= self.total_training_steps	31	is_last_step = self.global_steps >= self.total_training_steps
32	with marked_timer("step", timing_raw):	32	with marked_timer("step", timing_raw):
33	@@ -1048,6 +1055,9 @@ class RayPPOTrainer:	33	@@ -1048,6 +1055,9 @@ class RayPPOTrainer:
34	else:	34	else:
35	gen_batch_output = self.async_rollout_manager.generate_sequences(gen_batch)	35	gen_batch_output = self.async_rollout_manager.generate_sequences(gen_batch)
36	-	36	+
37	+ if data_rebalance:	37	+ if data_rebalance:
38	+ gen_batch_output.reorder(interleave_indices)	38	+ gen_batch_output.reorder(interleave_indices)
39	-+	39	++
40	timing_raw.update(gen_batch_output.meta_info["timing"])	40	timing_raw.update(gen_batch_output.meta_info["timing"])
41	gen_batch_output.meta_info.pop("timing", None)	41	gen_batch_output.meta_info.pop("timing", None)
42	-	42	+
43	@@ -1062,6 +1072,10 @@ class RayPPOTrainer:	43	@@ -1062,6 +1072,10 @@ class RayPPOTrainer:
44	gen_baseline_output = self.actor_rollout_wg.generate_sequences(gen_baseline_batch)	44	gen_baseline_output = self.actor_rollout_wg.generate_sequences(gen_baseline_batch)
45	else:	45	else:
46	gen_baseline_output = self.async_rollout_manager.generate_sequences(gen_baseline_batch)	46	gen_baseline_output = self.async_rollout_manager.generate_sequences(gen_baseline_batch)
47	-+	47	++
48	+ if data_rebalance:	48	+ if data_rebalance:
49	+ gen_baseline_output.reorder(interleave_indices)	49	+ gen_baseline_output.reorder(interleave_indices)
50	+	50	+
51	batch = batch.union(gen_baseline_output)	51	batch = batch.union(gen_baseline_output)
52	reward_baseline_tensor = self.reward_fn(batch)	52	reward_baseline_tensor = self.reward_fn(batch)
53	reward_baseline_tensor = reward_baseline_tensor.sum(dim=-1)	53	reward_baseline_tensor = reward_baseline_tensor.sum(dim=-1)
54	---	54	+--
55	2.50.1.windows.1	55	2.50.1.windows.1
56		56

	@@ -6,24 +6,24 @@ Enable EP to reshard parameters with AllToAllV (without communication redundancy
6	Optimize memory usage and communication performance via the expert parameter direcyed routing scheme.	6	Optimize memory usage and communication performance via the expert parameter direcyed routing scheme.
7		7
8	---	8	---
9	- llm_rl/qwen3/verl/utils/megatron_utils.py \| 195 +++++++++++++++++++-	9	+ llm_rl/qwen3/verl-mindspeed/verl/utils/megatron_utils.py \| 195 +++++++++++++++++++-
10	1 file changed, 194 insertions(+), 1 deletion(-)	10	1 file changed, 194 insertions(+), 1 deletion(-)
11		11
12	-diff --git a/llm_rl/qwen3/verl/utils/megatron_utils.py b/llm_rl/qwen3/verl/utils/megatron_utils.py	12	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/utils/megatron_utils.py b/llm_rl/qwen3/verl-mindspeed/verl/utils/megatron_utils.py
13	index 3eab811..b5d44ac 100644	13	index 3eab811..b5d44ac 100644
14	---- a/llm_rl/qwen3/verl/utils/megatron_utils.py	14	+--- a/llm_rl/qwen3/verl-mindspeed/verl/utils/megatron_utils.py
15	-+++ b/llm_rl/qwen3/verl/utils/megatron_utils.py	15	++++ b/llm_rl/qwen3/verl-mindspeed/verl/utils/megatron_utils.py
16	@@ -40,6 +40,7 @@ from verl.utils.device import get_device_id, get_device_name, get_torch_device	16	@@ -40,6 +40,7 @@ from verl.utils.device import get_device_id, get_device_name, get_torch_device
17	from verl.utils.fs import local_mkdir_safe	17	from verl.utils.fs import local_mkdir_safe
18	from verl.utils.model import normalize_model_name	18	from verl.utils.model import normalize_model_name
19	from verl.utils.torch_dtypes import PrecisionType	19	from verl.utils.torch_dtypes import PrecisionType
20	+from vllm.distributed.parallel_state import get_ep_group	20	+from vllm.distributed.parallel_state import get_ep_group
21	-	21	+
22	-	22	+
23	def get_model_config(model):	23	def get_model_config(model):
24	@@ -750,6 +751,12 @@ def default_tp_concat_fn(	24	@@ -750,6 +751,12 @@ def default_tp_concat_fn(
25	from megatron.core import mpu	25	from megatron.core import mpu
26	-	26	+
27	train_tp_size = mpu.get_tensor_model_parallel_world_size()	27	train_tp_size = mpu.get_tensor_model_parallel_world_size()
28	+	28	+
29	+ if hasattr(model_config, 'n_routed_experts'):	29	+ if hasattr(model_config, 'n_routed_experts'):
	@@ -37,7 +37,7 @@ index 3eab811..b5d44ac 100644
37	@@ -801,6 +808,36 @@ def default_tp_concat_fn(	37	@@ -801,6 +808,36 @@ def default_tp_concat_fn(
38	up = torch.cat(up_lst, dim=0)	38	up = torch.cat(up_lst, dim=0)
39	infer_params = torch.cat((gate, up), dim=0) if not convert_qkv_gate_up_by_simple_split else [gate, up]	39	infer_params = torch.cat((gate, up), dim=0) if not convert_qkv_gate_up_by_simple_split else [gate, up]
40	-	40	+
41	+ elif "mlp.experts.weight1" in name: # for moe group matmul	41	+ elif "mlp.experts.weight1" in name: # for moe group matmul
42	+ gate_pp_lst = []	42	+ gate_pp_lst = []
43	+ up_pp_lst = []	43	+ up_pp_lst = []
	@@ -70,20 +70,20 @@ index 3eab811..b5d44ac 100644
70	+ infer_params = get_rollout_expert_after_resharding(infer_params, model_config, is_weight1=False)	70	+ infer_params = get_rollout_expert_after_resharding(infer_params, model_config, is_weight1=False)
71	elif "mlp.experts.linear_fc2.weight" in name: # moe	71	elif "mlp.experts.linear_fc2.weight" in name: # moe
72	infer_params = torch.cat(infer_params, dim=1)	72	infer_params = torch.cat(infer_params, dim=1)
73	-	73	+
74	@@ -829,6 +866,7 @@ def per_tensor_generator(	74	@@ -829,6 +866,7 @@ def per_tensor_generator(
75	vpp_size = len(actor_module)	75	vpp_size = len(actor_module)
76	all_gather_group = mpu.get_tensor_model_parallel_group()	76	all_gather_group = mpu.get_tensor_model_parallel_group()
77	all_gather_group_size = torch.distributed.get_world_size(group=all_gather_group)	77	all_gather_group_size = torch.distributed.get_world_size(group=all_gather_group)
78	+ etmp_group = mpu.get_expert_tensor_and_model_parallel_group()	78	+ etmp_group = mpu.get_expert_tensor_and_model_parallel_group()
79	-	79	+
80	def tensor_generator():	80	def tensor_generator():
81	for scan_vpp_idx in range(vpp_size):	81	for scan_vpp_idx in range(vpp_size):
82	@@ -928,6 +966,34 @@ def per_tensor_generator(	82	@@ -928,6 +966,34 @@ def per_tensor_generator(
83	-	83	+
84	yield from zip(converted_names, [param.detach() for param in converted_params], strict=True)	84	yield from zip(converted_names, [param.detach() for param in converted_params], strict=True)
85	continue	85	continue
86	-+	86	++
87	+ elif ".mlp.experts.weight" in cur_name and ep_size > 1:	87	+ elif ".mlp.experts.weight" in cur_name and ep_size > 1:
88	+ if etp_size > 1:	88	+ if etp_size > 1:
89	+ raise NotImplementedError("Reshard for ETP params when using MoE Group Matmul not supported for now.")	89	+ raise NotImplementedError("Reshard for ETP params when using MoE Group Matmul not supported for now.")
	@@ -111,13 +111,13 @@ index 3eab811..b5d44ac 100644
111	+	111	+
112	+ yield from zip(converted_names, converted_params)	112	+ yield from zip(converted_names, converted_params)
113	+ continue	113	+ continue
114	-	114	+
115	# tp all gather	115	# tp all gather
116	if tp_utils.is_tensor_parallel_param(broad_pp_tensor):	116	if tp_utils.is_tensor_parallel_param(broad_pp_tensor):
117	@@ -955,7 +1021,6 @@ def per_tensor_generator(	117	@@ -955,7 +1021,6 @@ def per_tensor_generator(
118	-	118	+
119	yield from zip(converted_names, [param.detach() for param in converted_params], strict=True)	119	yield from zip(converted_names, [param.detach() for param in converted_params], strict=True)
120	-	120	+
121	-	121	-
122	def get_transformer_layer_offset(pipeline_rank, vp_stage, config: TransformerConfig):	122	def get_transformer_layer_offset(pipeline_rank, vp_stage, config: TransformerConfig):
123	"""	123	"""
	@@ -254,6 +254,6 @@ index 3eab811..b5d44ac 100644
254	+ for local_idx, expert_id in enumerate(local_expert_ids):	254	+ for local_idx, expert_id in enumerate(local_expert_ids):
255	+ experts_down_pp[expert_id] = local_expert_params[local_idx]	255	+ experts_down_pp[expert_id] = local_expert_params[local_idx]
256	+ return experts_down_pp	256	+ return experts_down_pp
257	---	257	+--
258	2.50.1.windows.1	258	2.50.1.windows.1
259		259

	@@ -7,14 +7,14 @@ Subject: [PATCH] bugfix for loading moe models
7	.../rollout/vllm_rollout/vllm_rollout_spmd.py \| 18 +++++++++++++++++-	7	.../rollout/vllm_rollout/vllm_rollout_spmd.py \| 18 +++++++++++++++++-
8	1 file changed, 17 insertions(+), 1 deletion(-)	8	1 file changed, 17 insertions(+), 1 deletion(-)
9		9
10	-diff --git a/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py b/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	10	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py b/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
11	index 40613f1f..9a002de9 100644	11	index 40613f1f..9a002de9 100644
12	---- a/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	12	+--- a/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
13	-+++ b/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	13	++++ b/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
14	@@ -123,6 +123,11 @@ class vLLMRollout(BaseRollout):	14	@@ -123,6 +123,11 @@ class vLLMRollout(BaseRollout):
15	)	15	)
16	max_num_batched_tokens = self.config.get("max_num_batched_tokens", 8192)	16	max_num_batched_tokens = self.config.get("max_num_batched_tokens", 8192)
17	-	17	+
18	+ # If VLLM_DP_SIZE is configured, the DP communication domain needs to be explicitly initialized.	18	+ # If VLLM_DP_SIZE is configured, the DP communication domain needs to be explicitly initialized.
19	+ if int(os.environ.get("VLLM_DP_SIZE", "1")) > 1:	19	+ if int(os.environ.get("VLLM_DP_SIZE", "1")) > 1:
20	+ from r1_ascend.vllm_parallel_state import init_parallel_state	20	+ from r1_ascend.vllm_parallel_state import init_parallel_state
	@@ -28,13 +28,13 @@ index 40613f1f..9a002de9 100644
28	else:	28	else:
29	from verl.utils.vllm.patch import patch_vllm_moe_model_weight_loader	29	from verl.utils.vllm.patch import patch_vllm_moe_model_weight_loader
30	+ from vllm.model_executor.model_loader.utils import process_weights_after_loading	30	+ from vllm.model_executor.model_loader.utils import process_weights_after_loading
31	-	31	+
32	- model = self.inference_engine.llm_engine.model_executor.driver_worker.worker.model_runner.model	32	- model = self.inference_engine.llm_engine.model_executor.driver_worker.worker.model_runner.model
33	+ model_runner = self.inference_engine.llm_engine.model_executor.driver_worker.worker.model_runner	33	+ model_runner = self.inference_engine.llm_engine.model_executor.driver_worker.worker.model_runner
34	+ model = model_runner.get_model()	34	+ model = model_runner.get_model()
35	patch_vllm_moe_model_weight_loader(model)	35	patch_vllm_moe_model_weight_loader(model)
36	model.load_weights(weights)	36	model.load_weights(weights)
37	-	37	+
38	+ model_config = model_runner.vllm_config.model_config	38	+ model_config = model_runner.vllm_config.model_config
39	+ device_config = model_runner.vllm_config.device_config	39	+ device_config = model_runner.vllm_config.device_config
40	+ load_config = model_runner.vllm_config.load_config	40	+ load_config = model_runner.vllm_config.load_config
	@@ -44,8 +44,8 @@ index 40613f1f..9a002de9 100644
44	+ target_device = torch.device(load_device)	44	+ target_device = torch.device(load_device)
45	+ process_weights_after_loading(model, model_config, target_device)	45	+ process_weights_after_loading(model, model_config, target_device)
46	+	46	+
47	-	47	+
48	# https://github.com/vllm-project/vllm/issues/13175	48	# https://github.com/vllm-project/vllm/issues/13175
49	def _monkey_patch_compute_logits(model, vocab_size: int):	49	def _monkey_patch_compute_logits(model, vocab_size: int):
50	---	50	+--
51	2.45.1.windows.1	51	2.45.1.windows.1

	@@ -5,52 +5,52 @@ Subject: [PATCH] Update verl: megatron_workers
5	Fix the issue where MindSpeed framework disables `torch.compile` (causing in unavailable inference	5	Fix the issue where MindSpeed framework disables `torch.compile` (causing in unavailable inference
6	acceleration), enable the use of compiled model during rollout.	6	acceleration), enable the use of compiled model during rollout.
7	---	7	---
8	- llm_rl/qwen3/verl/workers/megatron_workers.py \| 11 +++++++++++	8	+ llm_rl/qwen3/verl-mindspeed/verl/workers/megatron_workers.py \| 11 +++++++++++
9	1 file changed, 11 insertions(+)	9	1 file changed, 11 insertions(+)
10		10
11	-diff --git a/llm_rl/qwen3/verl/workers/megatron_workers.py b/llm_rl/qwen3/verl/workers/megatron_workers.py	11	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/workers/megatron_workers.py b/llm_rl/qwen3/verl-mindspeed/verl/workers/megatron_workers.py
12	index 0d5fbc9..9d20ca6 100644	12	index 0d5fbc9..9d20ca6 100644
13	---- a/llm_rl/qwen3/verl/workers/megatron_workers.py	13	+--- a/llm_rl/qwen3/verl-mindspeed/verl/workers/megatron_workers.py
14	-+++ b/llm_rl/qwen3/verl/workers/megatron_workers.py	14	++++ b/llm_rl/qwen3/verl-mindspeed/verl/workers/megatron_workers.py
15	@@ -34,6 +34,8 @@ except ImportError:	15	@@ -34,6 +34,8 @@ except ImportError:
16	repatch = None	16	repatch = None
17	-	17	+
18	from megatron.core import parallel_state as mpu	18	from megatron.core import parallel_state as mpu
19	+from mindspeed.patch_utils import MindSpeedPatchesManager	19	+from mindspeed.patch_utils import MindSpeedPatchesManager
20	+from mindspeed.core.megatron_basic.requirements_basic import dummy_compile	20	+from mindspeed.core.megatron_basic.requirements_basic import dummy_compile
21	-	21	+
22	from verl import DataProto	22	from verl import DataProto
23	from verl.single_controller.base import Worker	23	from verl.single_controller.base import Worker
24	@@ -78,6 +80,10 @@ from verl.workers.rollout import get_rollout_class	24	@@ -78,6 +80,10 @@ from verl.workers.rollout import get_rollout_class
25	logger = logging.getLogger(__file__)	25	logger = logging.getLogger(__file__)
26	logger.setLevel(os.getenv("VERL_LOGGING_LEVEL", "WARN"))	26	logger.setLevel(os.getenv("VERL_LOGGING_LEVEL", "WARN"))
27	-	27	+
28	+MindSpeedPatchesManager.patches_info['torch.compile'].remove_patch()	28	+MindSpeedPatchesManager.patches_info['torch.compile'].remove_patch()
29	+TRUE_COMPILE = torch.compile	29	+TRUE_COMPILE = torch.compile
30	+DUMMY_COMPILE = dummy_compile	30	+DUMMY_COMPILE = dummy_compile
31	+	31	+
32	-	32	+
33	def set_random_seed(seed):	33	def set_random_seed(seed):
34	import random	34	import random
35	@@ -390,6 +396,8 @@ class ActorRolloutRefWorker(MegatronWorker, DistProfilerExtension):	35	@@ -390,6 +396,8 @@ class ActorRolloutRefWorker(MegatronWorker, DistProfilerExtension):
36	-	36	+
37	def _build_rollout(self, trust_remote_code=False):	37	def _build_rollout(self, trust_remote_code=False):
38	from torch.distributed.device_mesh import init_device_mesh	38	from torch.distributed.device_mesh import init_device_mesh
39	+ # Temporarily restore true torch.compile for the rollout build	39	+ # Temporarily restore true torch.compile for the rollout build
40	+ torch.compile = TRUE_COMPILE	40	+ torch.compile = TRUE_COMPILE
41	-	41	+
42	# 1. parse rollout and huggingface model config	42	# 1. parse rollout and huggingface model config
43	rollout_config: RolloutConfig = omega_conf_to_dataclass(self.config.rollout)	43	rollout_config: RolloutConfig = omega_conf_to_dataclass(self.config.rollout)
44	@@ -447,6 +455,9 @@ class ActorRolloutRefWorker(MegatronWorker, DistProfilerExtension):	44	@@ -447,6 +455,9 @@ class ActorRolloutRefWorker(MegatronWorker, DistProfilerExtension):
45	loop = asyncio.get_event_loop()	45	loop = asyncio.get_event_loop()
46	loop.run_until_complete(self.trainer_mode())	46	loop.run_until_complete(self.trainer_mode())
47	-	47	+
48	+ # Revert to dummy_compile after rollout is built	48	+ # Revert to dummy_compile after rollout is built
49	+ torch.compile = DUMMY_COMPILE	49	+ torch.compile = DUMMY_COMPILE
50	+	50	+
51	@register(dispatch_mode=Dispatch.ONE_TO_ALL)	51	@register(dispatch_mode=Dispatch.ONE_TO_ALL)
52	def init_model(self):	52	def init_model(self):
53	if self.config.model.get("external_lib", None) is not None:	53	if self.config.model.get("external_lib", None) is not None:
54	---	54	+--
55	2.50.1.windows.1	55	2.50.1.windows.1
56		56

	@@ -4,21 +4,21 @@ Date: Wed, 26 Nov 2025 15:16:55 +0800
4	Subject: [PATCH] verl-feature-enable_hdp	4	Subject: [PATCH] verl-feature-enable_hdp
5		5
6	---	6	---
7	- llm_rl/qwen3/verl/models/mcore/model_forward.py \| 12 ++++++++++++	7	+ llm_rl/qwen3/verl-mindspeed/verl/models/mcore/model_forward.py \| 12 ++++++++++++
8	- llm_rl/qwen3/verl/utils/seqlen_balancing.py \| 9 ++++++++-	8	+ llm_rl/qwen3/verl-mindspeed/verl/utils/seqlen_balancing.py \| 9 ++++++++-
9	2 files changed, 20 insertions(+), 1 deletion(-)	9	2 files changed, 20 insertions(+), 1 deletion(-)
10		10
11	-diff --git a/llm_rl/qwen3/verl/models/mcore/model_forward.py b/llm_rl/qwen3/verl/models/mcore/model_forward.py	11	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/models/mcore/model_forward.py b/llm_rl/qwen3/verl-mindspeed/verl/models/mcore/model_forward.py
12	index a6e5489..76330b2 100644	12	index a6e5489..76330b2 100644
13	---- a/llm_rl/qwen3/verl/models/mcore/model_forward.py	13	+--- a/llm_rl/qwen3/verl-mindspeed/verl/models/mcore/model_forward.py
14	-+++ b/llm_rl/qwen3/verl/models/mcore/model_forward.py	14	++++ b/llm_rl/qwen3/verl-mindspeed/verl/models/mcore/model_forward.py
15	@@ -14,6 +14,7 @@	15	@@ -14,6 +14,7 @@
16	# See the License for the specific language governing permissions and	16	# See the License for the specific language governing permissions and
17	# limitations under the License.	17	# limitations under the License.
18	-	18	+
19	+import os	19	+import os
20	from verl.utils.megatron_utils import unwrap_model	20	from verl.utils.megatron_utils import unwrap_model
21	-	21	+
22	from .util import (	22	from .util import (
23	@@ -41,6 +42,17 @@ def gptmodel_forward(	23	@@ -41,6 +42,17 @@ def gptmodel_forward(
24	"""Default forward pass for GPT models with optional sequence packing."""	24	"""Default forward pass for GPT models with optional sequence packing."""
	@@ -38,14 +38,14 @@ index a6e5489..76330b2 100644
38	if pack_seqs:	38	if pack_seqs:
39	batch_size, seq_len = attention_mask.shape[:2]	39	batch_size, seq_len = attention_mask.shape[:2]
40	input_ids_rmpad, packed_seq_params = preprocess_packed_seqs(input_ids, attention_mask, pre_process=pre_process)	40	input_ids_rmpad, packed_seq_params = preprocess_packed_seqs(input_ids, attention_mask, pre_process=pre_process)
41	-diff --git a/llm_rl/qwen3/verl/utils/seqlen_balancing.py b/llm_rl/qwen3/verl/utils/seqlen_balancing.py	41	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/utils/seqlen_balancing.py b/llm_rl/qwen3/verl-mindspeed/verl/utils/seqlen_balancing.py
42	index 5354d51..1f5d834 100644	42	index 5354d51..1f5d834 100644
43	---- a/llm_rl/qwen3/verl/utils/seqlen_balancing.py	43	+--- a/llm_rl/qwen3/verl-mindspeed/verl/utils/seqlen_balancing.py
44	-+++ b/llm_rl/qwen3/verl/utils/seqlen_balancing.py	44	++++ b/llm_rl/qwen3/verl-mindspeed/verl/utils/seqlen_balancing.py
45	@@ -12,6 +12,7 @@	45	@@ -12,6 +12,7 @@
46	# See the License for the specific language governing permissions and	46	# See the License for the specific language governing permissions and
47	# limitations under the License.	47	# limitations under the License.
48	-	48	+
49	+import os	49	+import os
50	import copy	50	import copy
51	import heapq	51	import heapq
	@@ -53,25 +53,25 @@ index 5354d51..1f5d834 100644
53	@@ -23,6 +24,8 @@ from verl.protocol import DataProto	53	@@ -23,6 +24,8 @@ from verl.protocol import DataProto
54	from verl.utils import tensordict_utils as tu	54	from verl.utils import tensordict_utils as tu
55	from verl.utils.device import get_device_name	55	from verl.utils.device import get_device_name
56	-	56	+
57	+from patches.verl.utils.hybrid_data_parallel.hdp import pack_sequences_into_buckets	57	+from patches.verl.utils.hybrid_data_parallel.hdp import pack_sequences_into_buckets
58	+	58	+
59	-	59	+
60	def karmarkar_karp(seqlen_list: list[int], k_partitions: int, equal_size: bool):	60	def karmarkar_karp(seqlen_list: list[int], k_partitions: int, equal_size: bool):
61	# see: https://en.wikipedia.org/wiki/Largest_differencing_method	61	# see: https://en.wikipedia.org/wiki/Largest_differencing_method
62	@@ -301,7 +304,11 @@ def rearrange_micro_batches(	62	@@ -301,7 +304,11 @@ def rearrange_micro_batches(
63	seq_len_effective = seq_len_effective.tolist()	63	seq_len_effective = seq_len_effective.tolist()
64	assert num_micro_batches <= len(seq_len_effective)	64	assert num_micro_batches <= len(seq_len_effective)
65	-	65	+
66	- micro_bsz_idx = get_seqlen_balanced_partitions(seq_len_effective, num_micro_batches, equal_size=False)	66	- micro_bsz_idx = get_seqlen_balanced_partitions(seq_len_effective, num_micro_batches, equal_size=False)
67	+ if os.environ.get("USE_HDP") == "1":	67	+ if os.environ.get("USE_HDP") == "1":
68	+ max_token_len = ceildiv(total_seqlen, num_micro_batches)	68	+ max_token_len = ceildiv(total_seqlen, num_micro_batches)
69	+ micro_bsz_idx = pack_sequences_into_buckets(seq_len_effective, max_token_len, num_micro_batches)	69	+ micro_bsz_idx = pack_sequences_into_buckets(seq_len_effective, max_token_len, num_micro_batches)
70	+ else:	70	+ else:
71	+ micro_bsz_idx = get_seqlen_balanced_partitions(seq_len_effective, num_micro_batches, equal_size=False)	71	+ micro_bsz_idx = get_seqlen_balanced_partitions(seq_len_effective, num_micro_batches, equal_size=False)
72	-	72	+
73	if use_dynamic_bsz_balance:	73	if use_dynamic_bsz_balance:
74	# Use the sum of squared sequence lengths to approximate attention computation workload	74	# Use the sum of squared sequence lengths to approximate attention computation workload
75	---	75	+--
76	2.50.1.windows.1	76	2.50.1.windows.1
77		77

	@@ -9,20 +9,20 @@ leverages multi-level compiled graphs, and greatly improves global throughput an
9	rl_train/qwen3/verl/workers/megatron_workers.py \| 3 +++	9	rl_train/qwen3/verl/workers/megatron_workers.py \| 3 +++
10	1 file changed, 3 insertions(+)	10	1 file changed, 3 insertions(+)
11		11
12	-diff --git a/llm_rl/qwen3/verl/workers/megatron_workers.py b/llm_rl/qwen3/verl/workers/megatron_workers.py	12	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/workers/megatron_workers.py b/llm_rl/qwen3/verl-mindspeed/verl/workers/megatron_workers.py
13	index 6cb8086..fe892b3 100644	13	index 6cb8086..fe892b3 100644
14	---- a/llm_rl/qwen3/verl/workers/megatron_workers.py	14	+--- a/llm_rl/qwen3/verl-mindspeed/verl/workers/megatron_workers.py
15	-+++ b/llm_rl/qwen3/verl/workers/megatron_workers.py	15	++++ b/llm_rl/qwen3/verl-mindspeed/verl/workers/megatron_workers.py
16	@@ -77,6 +77,9 @@ from verl.workers.critic.megatron_critic import MegatronPPOCritic	16	@@ -77,6 +77,9 @@ from verl.workers.critic.megatron_critic import MegatronPPOCritic
17	from verl.workers.reward_model.megatron.reward_model import MegatronRewardModel	17	from verl.workers.reward_model.megatron.reward_model import MegatronRewardModel
18	from verl.workers.rollout import get_rollout_class	18	from verl.workers.rollout import get_rollout_class
19	-	19	+
20	+from patches.verl.features.rollout_optimize import init_rollout_rebalance	20	+from patches.verl.features.rollout_optimize import init_rollout_rebalance
21	+init_rollout_rebalance()	21	+init_rollout_rebalance()
22	+	22	+
23	logger = logging.getLogger(__file__)	23	logger = logging.getLogger(__file__)
24	logger.setLevel(os.getenv("VERL_LOGGING_LEVEL", "WARN"))	24	logger.setLevel(os.getenv("VERL_LOGGING_LEVEL", "WARN"))
25	-	25	+
26	---	26	+--
27	2.50.1.windows.1	27	2.50.1.windows.1
28		28

	@@ -7,10 +7,10 @@ Subject: [PATCH] Update verl: support npugraph_ex for spec decode
7	.../rollout/vllm_rollout/vllm_rollout_spmd.py \| 21 ++++++++++++++-----	7	.../rollout/vllm_rollout/vllm_rollout_spmd.py \| 21 ++++++++++++++-----
8	1 file changed, 15 insertions(+), 6 deletions(-)	8	1 file changed, 15 insertions(+), 6 deletions(-)
9		9
10	-diff --git a/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py b/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	10	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py b/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
11	index c3adcaf..120a5ec 100644	11	index c3adcaf..120a5ec 100644
12	---- a/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	12	+--- a/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
13	-+++ b/llm_rl/qwen3/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py	13	++++ b/llm_rl/qwen3/verl-mindspeed/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py
14	@@ -188,12 +188,11 @@ class vLLMRollout(BaseRollout):	14	@@ -188,12 +188,11 @@ class vLLMRollout(BaseRollout):
15	cudagraph_capture_sizes = config.get("cudagraph_capture_sizes")	15	cudagraph_capture_sizes = config.get("cudagraph_capture_sizes")
16	# enforce_eager must be False to use cudagraph	16	# enforce_eager must be False to use cudagraph
	@@ -26,7 +26,7 @@ index c3adcaf..120a5ec 100644
26	+ "cudagraph_capture_sizes": cudagraph_capture_sizes,	26	+ "cudagraph_capture_sizes": cudagraph_capture_sizes,
27	+ "cudagraph_mode": "FULL",	27	+ "cudagraph_mode": "FULL",
28	+ }	28	+ }
29	-	29	+
30	self.dynamic_eplb = int(os.environ.get("VLLM_ENABLE_EPLB", "0")) == 1	30	self.dynamic_eplb = int(os.environ.get("VLLM_ENABLE_EPLB", "0")) == 1
31	self.inference_engine = LLM(	31	self.inference_engine = LLM(
32	@@ -205,6 +204,7 @@ class vLLMRollout(BaseRollout):	32	@@ -205,6 +204,7 @@ class vLLMRollout(BaseRollout):
	@@ -51,15 +51,15 @@ index c3adcaf..120a5ec 100644
51	@@ -238,6 +243,11 @@ class vLLMRollout(BaseRollout):	51	@@ -238,6 +243,11 @@ class vLLMRollout(BaseRollout):
52	repetition_penalty=config.get("repetition_penalty", 1.0),	52	repetition_penalty=config.get("repetition_penalty", 1.0),
53	)	53	)
54	-	54	+
55	+ # Patch: unset logprobs if speculative_config is enabled.	55	+ # Patch: unset logprobs if speculative_config is enabled.
56	+ if "speculative_config" in engine_kwargs:	56	+ if "speculative_config" in engine_kwargs:
57	+ logger.warning("The 'logprobs' parameter is incompatible with Speculative Decoding and has been disabled.")	57	+ logger.warning("The 'logprobs' parameter is incompatible with Speculative Decoding and has been disabled.")
58	+ del kwargs["logprobs"]	58	+ del kwargs["logprobs"]
59	+	59	+
60	kwargs["detokenize"] = False	60	kwargs["detokenize"] = False
61	-	61	+
62	# supporting adding any sampling params from the config file	62	# supporting adding any sampling params from the config file
63	---	63	+--
64	2.45.1.windows.1	64	2.45.1.windows.1
65		65

	@@ -7,13 +7,13 @@ Introduces `data_rebalance` to mitigate long-tail load imbalance in multi-NPU in
7	Disables repeat interleaving during batch repetition, and applies fixed reordering after generation	7	Disables repeat interleaving during batch repetition, and applies fixed reordering after generation
8	to reduce per-NPU latency variance.	8	to reduce per-NPU latency variance.
9	---	9	---
10	- llm_rl/qwen3/verl/trainer/dapo_ray_trainer.py \| 14 ++++++++++++--	10	+ llm_rl/qwen3/verl-mindspeed/verl/trainer/dapo_ray_trainer.py \| 14 ++++++++++++--
11	1 file changed, 12 insertions(+), 2 deletions(-)	11	1 file changed, 12 insertions(+), 2 deletions(-)
12		12
13	-diff --git a/llm_rl/qwen3/verl/trainer/dapo_ray_trainer.py b/llm_rl/qwen3/verl/trainer/dapo_ray_trainer.py	13	+diff --git a/llm_rl/qwen3/verl-mindspeed/verl/trainer/dapo_ray_trainer.py b/llm_rl/qwen3/verl-mindspeed/verl/trainer/dapo_ray_trainer.py
14	index 7ad7372..5420d0a 100644	14	index 7ad7372..5420d0a 100644
15	---- a/llm_rl/qwen3/verl/trainer/dapo_ray_trainer.py	15	+--- a/llm_rl/qwen3/verl-mindspeed/verl/trainer/dapo_ray_trainer.py
16	-+++ b/llm_rl/qwen3/verl/trainer/dapo_ray_trainer.py	16	++++ b/llm_rl/qwen3/verl-mindspeed/verl/trainer/dapo_ray_trainer.py
17	@@ -132,7 +132,14 @@ class RayDAPOTrainer(RayPPOTrainer):	17	@@ -132,7 +132,14 @@ class RayDAPOTrainer(RayPPOTrainer):
18	batch_keys=["input_ids", "attention_mask", "position_ids"],	18	batch_keys=["input_ids", "attention_mask", "position_ids"],
19	non_tensor_batch_keys=["raw_prompt_ids"],	19	non_tensor_batch_keys=["raw_prompt_ids"],
	@@ -27,9 +27,9 @@ index 7ad7372..5420d0a 100644
27	+ if data_rebalance:	27	+ if data_rebalance:
28	+ interleave_indices = torch.arange(gen_batch.batch.batch_size[0]).view(	28	+ interleave_indices = torch.arange(gen_batch.batch.batch_size[0]).view(
29	+ -1, new_batch.batch.batch_size[0]).transpose(1, 0).reshape(-1)	29	+ -1, new_batch.batch.batch_size[0]).transpose(1, 0).reshape(-1)
30	-	30	+
31	is_last_step = self.global_steps >= self.total_training_steps	31	is_last_step = self.global_steps >= self.total_training_steps
32	-	32	+
33	@@ -140,6 +147,8 @@ class RayDAPOTrainer(RayPPOTrainer):	33	@@ -140,6 +147,8 @@ class RayDAPOTrainer(RayPPOTrainer):
34	# generate a batch	34	# generate a batch
35	with marked_timer("gen", timing_raw, "red"):	35	with marked_timer("gen", timing_raw, "red"):
	@@ -38,7 +38,7 @@ index 7ad7372..5420d0a 100644
38	+ gen_batch_output.reorder(interleave_indices)	38	+ gen_batch_output.reorder(interleave_indices)
39	timing_raw.update(gen_batch_output.meta_info["timing"])	39	timing_raw.update(gen_batch_output.meta_info["timing"])
40	gen_batch_output.meta_info.pop("timing", None)	40	gen_batch_output.meta_info.pop("timing", None)
41	-	41	+
42	@@ -148,7 +157,8 @@ class RayDAPOTrainer(RayPPOTrainer):	42	@@ -148,7 +157,8 @@ class RayDAPOTrainer(RayPPOTrainer):
43	gen_baseline_batch = deepcopy(gen_batch)	43	gen_baseline_batch = deepcopy(gen_batch)
44	gen_baseline_batch.meta_info["do_sample"] = False	44	gen_baseline_batch.meta_info["do_sample"] = False
	@@ -49,6 +49,6 @@ index 7ad7372..5420d0a 100644
49	new_batch = new_batch.union(gen_baseline_output)	49	new_batch = new_batch.union(gen_baseline_output)
50	reward_baseline_tensor = self.reward_fn(new_batch)	50	reward_baseline_tensor = self.reward_fn(new_batch)
51	reward_baseline_tensor = reward_baseline_tensor.sum(dim=-1)	51	reward_baseline_tensor = reward_baseline_tensor.sum(dim=-1)
52	---	52	+--
53	2.50.1.windows.1	53	2.50.1.windows.1
54		54

	@@ -1,25 +1,25 @@
1	-From 0c17c27789cf55a4a3622aacf0bdb1172597fbd8 Mon Sep 17 00:00:00 2001	1	+From 0c17c27789cf55a4a3622aacf0bdb1172597fbd8 Mon Sep 17 00:00:00 2001
2	-From: huyuanquan1 <huyuanquan1@huawei.com>	2	+From: huyuanquan1 <huyuanquan1@huawei.com>
3	-Date: Mon, 2 Feb 2026 10:04:37 +0800	3	+Date: Mon, 2 Feb 2026 10:04:37 +0800
4	-Subject: [PATCH] bugfix ROPE registry	4	+Subject: [PATCH] bugfix ROPE registry
5	-	5	+
6	----	6	+---
7	- .../qwen3/vllm/model_executor/layers/rotary_embedding/common.py \| 2 +-	7	+ .../qwen3/vllm/model_executor/layers/rotary_embedding/common.py \| 2 +-
8	- 1 file changed, 1 insertion(+), 1 deletion(-)	8	+ 1 file changed, 1 insertion(+), 1 deletion(-)
9	-	9	+
10	-diff --git a/llm_rl/qwen3/vllm/model_executor/layers/rotary_embedding/common.py b/llm_rl/qwen3/vllm/model_executor/layers/rotary_embedding/common.py	10	+diff --git a/llm_rl/qwen3/verl-mindspeed/vllm/model_executor/layers/rotary_embedding/common.py b/llm_rl/qwen3/verl-mindspeed/vllm/model_executor/layers/rotary_embedding/common.py
11	-index 34de1da..96364aa 100644	11	+index 34de1da..96364aa 100644
12	---- a/llm_rl/qwen3/vllm/model_executor/layers/rotary_embedding/common.py	12	+--- a/llm_rl/qwen3/verl-mindspeed/vllm/model_executor/layers/rotary_embedding/common.py
13	-+++ b/llm_rl/qwen3/vllm/model_executor/layers/rotary_embedding/common.py	13	++++ b/llm_rl/qwen3/verl-mindspeed/vllm/model_executor/layers/rotary_embedding/common.py
14	-@@ -134,7 +134,7 @@ class ApplyRotaryEmb(CustomOp):	14	+@@ -134,7 +134,7 @@ class ApplyRotaryEmb(CustomOp):
15	- self.enable_fp32_compute = enable_fp32_compute	15	+ self.enable_fp32_compute = enable_fp32_compute
16	-	16	+
17	- self.apply_rotary_emb_flash_attn = None	17	+ self.apply_rotary_emb_flash_attn = None
18	-- if find_spec("flash_attn") is not None:	18	+- if find_spec("flash_attn") is not None:
19	-+ if find_spec("flash_attn") is not None and not hasattr(torch, "npu"):	19	++ if find_spec("flash_attn") is not None and not hasattr(torch, "npu"):
20	- from flash_attn.ops.triton.rotary import apply_rotary	20	+ from flash_attn.ops.triton.rotary import apply_rotary
21	-	21	+
22	- self.apply_rotary_emb_flash_attn = apply_rotary	22	+ self.apply_rotary_emb_flash_attn = apply_rotary
23	---	23	+--
24	-2.45.1.windows.1	24	+2.45.1.windows.1
25	-	25	+

	@@ -4,13 +4,13 @@ Date: Wed, 28 Jan 2026 10:24:24 +0800
4	Subject: [PATCH] Fix npugraph_ex static kernel bug	4	Subject: [PATCH] Fix npugraph_ex static kernel bug
5		5
6	---	6	---
7	- llm_rl/qwen3/vllm_ascend/compilation/compiler_interface.py \| 6 +++---	7	+ llm_rl/qwen3/verl-mindspeed/vllm_ascend/compilation/compiler_interface.py \| 6 +++---
8	1 file changed, 3 insertions(+), 3 deletions(-)	8	1 file changed, 3 insertions(+), 3 deletions(-)
9		9
10	-diff --git a/llm_rl/qwen3/vllm_ascend/compilation/compiler_interface.py b/llm_rl/qwen3/vllm_ascend/compilation/compiler_interface.py	10	+diff --git a/llm_rl/qwen3/verl-mindspeed/vllm_ascend/compilation/compiler_interface.py b/llm_rl/qwen3/verl-mindspeed/vllm_ascend/compilation/compiler_interface.py
11	index 7a02912..69c2a37 100644	11	index 7a02912..69c2a37 100644
12	---- a/llm_rl/qwen3/vllm_ascend/compilation/compiler_interface.py	12	+--- a/llm_rl/qwen3/verl-mindspeed/vllm_ascend/compilation/compiler_interface.py
13	-+++ b/llm_rl/qwen3/vllm_ascend/compilation/compiler_interface.py	13	++++ b/llm_rl/qwen3/verl-mindspeed/vllm_ascend/compilation/compiler_interface.py
14	@@ -88,12 +88,12 @@ def npugraph_ex_compile(	14	@@ -88,12 +88,12 @@ def npugraph_ex_compile(
15	# that can trigger the compilation of static kernel. If this configuration is	15	# that can trigger the compilation of static kernel. If this configuration is
16	# not applied, new shapes will trigger the compilation of static kernels,	16	# not applied, new shapes will trigger the compilation of static kernels,
	@@ -27,6 +27,6 @@ index 7a02912..69c2a37 100644
27	if max_num_tokens >= x >= uniform_decode_query_len	27	if max_num_tokens >= x >= uniform_decode_query_len
28	]	28	]
29	config.experimental_config.aclgraph._aclnn_static_shape_kernel_sym_value_range = decode_cudagraph_batch_sizes	29	config.experimental_config.aclgraph._aclnn_static_shape_kernel_sym_value_range = decode_cudagraph_batch_sizes
30	---	30	+--
31	2.45.1.windows.1	31	2.45.1.windows.1
32		32