master

文件	最后提交记录	最后更新时间
aclnn_torch_adapter	[Feature] Add Aclnn ops compilation and dependencies Co-authored-by: YuhanBai<baiyuhan1@h-partners.com> # message auto-generated for no-merge-commit merge: !52 merge aclnn-adapter into master [Feature] Add aclnn ops compiletion and dependencies Created-by: yuhanBai Commit-by: YuhanBai Merged-by: ascend-robot Description: ## What this PR does / why we need it? 本PR参考vllm-ascend中的算子工程，添加了aclnn算子的编译依赖 RFC: https://gitcode.com/Ascend/MindSpeed-Ops/issues/33 Co-Auther with: [@Liccol](https://gitcode.com/Liccol) [@qq_45570708](https://gitcode.com/qq_45570708) ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-Ops!52	4 个月前
add_rms_norm_bias	feat: Add add_rms_norm_bias compiletion Co-authored-by: Liccol<740821011@qq.com> # message auto-generated for no-merge-commit merge: !54 merge aclnn-case into master feat: Add add_rms_norm_bias compiletion Created-by: Liccol Commit-by: Liccol Merged-by: ascend-robot Description: ## What this PR does / why we need it? 新增 add_rms_norm_bias 自定义 ACLNN 算子，将逐元素加法（Add）与 RMS 归一化合并为单个 AscendC Kernel 执行，用于 Transformer 模型推理中残差连接 + RMSNorm 的融合优化。 主要变更： - 新增 add_rms_norm_bias 算子源码 - 新增 cann/ 下的 ACLNN 绑定及 torch_binding 算子注册 - 新增 Python 调用接口（api/aclnn/）及 JIT 构建器（op_builder/） - 引入 cmake/common/scripts/utils 等编译工具链及公共组件 - 新增算子使用文档及开发指南（docs/aclnn/） ## Does this PR introduce any user-facing change? 是，新增 npu_add_rms_norm_bias 接口 ## How was this patch tested? 通过单元测试与 CPU 参考实现对比验证： - 对比分离执行 Add + RMSNorm 与融合算子的数值精度 - 覆盖 float32 / float16 / bfloat16 三种数据类型 - 覆盖有 beta / 无 beta 两种场景  See merge request: Ascend/MindSpeed-Ops!54	4 个月前
atb	feat(atb): add MatmulAdd operators for fused wgrad accumulation Co-authored-by: Liz<lizhi166@huawei.com> # message auto-generated for no-merge-commit merge: !182 merge master into master feat(atb): add MatmulAdd operators for fused wgrad accumulation Created-by: Liz_ Commit-by: Liz Merged-by: ascend-robot Description: ## What this PR does / why we need it? Megatron gradient accumulation updates the main weight-gradient buffer as grad += grad_output.T @ total_input. The previous decomposed path materializes an [N, K] intermediate and launches a separate add kernel, increasing allocation and memory traffic. This PR adapts the MindSpeed MatmulAdd implementation to MindSpeed-Ops and provides: - npu_matmul_add_fp32 for bf16/fp16 operands accumulated into an fp32 grad; it uses the ATB fused MatmulAdd path on A2/A3 and addmm_ on A5. - npu_matmul_add_fp16 for bf16/fp16 operands and a matching half-precision grad; it replaces matmul + add_ with one in-place addmm_ call. - zero-sized-input protection, chip dispatch, builder integration, public documentation, and expanded unit coverage. - reproducible ATK case definitions and execution plugin for accuracy, device performance, and device memory validation. Generated cases, reports, and logs stay ignored; the final result summary is retained at tests/atk_tests/atb/matmul_add/atk_output/console/SUMMARY.md. This is feature enablement, not a bug fix. Related issue: N/A. ## Does this PR introduce any user-facing change? Yes. NPU callers can import npu_matmul_add_fp32 or npu_matmul_add_fp16 from mindspeed_ops.api.atb.npu_matmul_add. Both APIs mutate grad in place and return None. Inputs must be 2-D tensors with shapes total_input=[M,K], grad_output=[M,N], and grad=[N,K]. Operand dtypes must match and be bf16 or fp16. The fp32 API requires an fp32 grad buffer; the fp16 API requires grad to match the operand dtype. Empty inputs are a no-op. Usage and benchmark documentation: docs/zh/atb/matmul_add.md. ATK reproduction guide: tests/atk_tests/atb/matmul_add/README.md. ## How was this patch tested? Unit test in the 018 environment: - conda activate lz_py312_018 - python -m pytest tests/unit_tests/atb/test_npu_matmul_add.py -q -rs - Result: 8 passed, 7 skipped. The skipped cases require cann-nnal/ATB and were skipped because ATB_HOME_PATH was not set in this environment. ATK 26.4.30 was run on Ascend 910B3 (A2) with CANN 9.0.0 + ATB, Python 3.10, torch 2.7.1, and torch_npu 2.7.1.post4. Each API used 8 cases covering {bf16, fp16} operands across four shape tuples. - Accuracy, NPU vs CPU high-precision reference: fp32 8/8 Pass; fp16 8/8 Pass. - Device performance, fused dev0 vs decomposed dev1: fp32 average ratio 1.3662 with 100% case pass rate; fp16 average ratio 1.0007 with 87.5% case pass rate and overall Pass. - Device memory, fused dev0 vs decomposed dev1: fp32 and fp16 both 8/8 Pass with 100% memory pass rate under the fused/reference <= 1.1 criterion. The A2/A3 fp32 path requires cann-nnal and ATB_HOME_PATH. Its first invocation JIT-builds the C++ extension and takes about 30 seconds; performance measurements must use warmed, cache-hit runs. The public APIs are Ascend NPU-specific and do not provide a CUDA fallback. [RFC](https://gitcode.com/Ascend/MindSpeed-Ops/issues/80) See merge request: Ascend/MindSpeed-Ops!182	1 个月前
catlass	fix: 商发wheel安全编译扫描整改(rpath/RELRO/栈保护/strip) Co-authored-by: Raining__<wangruining5@huawei.com> # message auto-generated for no-merge-commit merge: !173 merge master into master fix: 商发wheel安全编译扫描整改(rpath/RELRO/栈保护/strip) Created-by: Raining__ Commit-by: Raining__ Merged-by: ascend-robot Description: ## 背景 商发 wheel 包华为安全编译扫描整改。本 PR 修复扫描命中的 4 类问题,覆盖 wheel 内全部 host 侧 ELF 产物(pybind 扩展 + CATLASS 子库 + 设备 kernel .o)。设备 kernel .o 的 PIE/RELRO/栈保护/FS/strip 项因 CANN 设备二进制格式所限不适用,单独走豁免说明。 ## 修改明细 ### 1. 禁止 rpath(c781b17,另含 037ba4c+a57dfd5 的 kernel 侧修复) - mindspeed_ops/csrc/CMakeLists.txt:OPS_COMPILE_OPTIONS 追加 -fno-rtlib-add-rpath,消除 ccec 驱动默认 -frtlib-add-rpath 给设备 kernel .o 写入的 DT_RPATH - CMakeLists.txt:删除 mindspeed_ops_C 的 BUILD_RPATH/INSTALL_RPATH "$ORIGIN:$ORIGIN/lib" - mindspeed_ops/csrc/catlass/{basic_matmul,chunk_loss}/CMakeLists.txt:删除 INSTALL_RPATH "$ORIGIN" - mindspeed_ops/__init__.py:新增 _preload_catlass_libraries()(置于引导序列首位)——以绝对路径 ctypes.CDLL(..., RTLD_GLOBAL) 预载 mindspeed_ops/lib/*.so,利用 SONAME 注册机制使扩展的 NEEDED 依赖在不带 RPATH 的情况下正常解析,功能与原 $ORIGIN/lib RPATH 等价;与文件内既有 _preload_custom_opapi_library 同一模式,CATLASS 未使能(无 lib/ 目录)时为空操作 ### 2. GOT 重定位只读 （全 RELRO）与立即绑定（BIND_NOW）(39354c4、0706d44) - CMakeLists.txt:mindspeed_ops_C 追加链接选项 -Wl,-z,relro,-z,now - 两个 CATLASS 子库 target_link_options 追加 -Wl,-z,relro,-z,now(bisheng 默认仅 -z relro 部分 RELRO,补 -z now 到全 RELRO) - 验证:产物均出现 GNU_RELRO 段 + DT_BIND_NOW + FLAGS_1 NOW ### 3. 栈保护(3de5382) - CMakeLists.txt:mindspeed_ops_C 追加编译选项 -fstack-protector-all(选 -all 而非 -strong,确保所有函数插桩、确定性产生 __stack_chk_fail 引用) - 验证:产物动态符号表出现 __stack_chk_fail@GLIBC_2.4 ### 4. strip(fe649b5) - 两个 CATLASS 子库 target_link_options 追加 -Wl,-s - 验证:.symtab/.strtab 去除,.dynsym 完整保留(动态链接不受影响),file 判定 stripped ## 验证 - 每轮均在 py3.12 + CANN 9.1.0 标准构建镜像内全量重建 wheel,解包对全部 10 个 ELF 逐个 readelf 核验,已修复项无回归(rpath 全为 0) - 容器内 pip install 后 import mindspeed_ops 正常,CATLASS 预载链路日志确认生效 ## 兼容性 - 全部为编译/链接选项与装载顺序调整,无 API/行为变更;-fstack-protector-all 有常规量级运行时开销(每函数 canary 检查),对 host 侧绑定层可忽略 - 不带 CATLASS 的构建(MINDSPEED_BUILD_CATLASS=0)不受影响 See merge request: Ascend/MindSpeed-Ops!173	1 个月前
cmake	feat: Add add_rms_norm_bias compiletion Co-authored-by: Liccol<740821011@qq.com> # message auto-generated for no-merge-commit merge: !54 merge aclnn-case into master feat: Add add_rms_norm_bias compiletion Created-by: Liccol Commit-by: Liccol Merged-by: ascend-robot Description: ## What this PR does / why we need it? 新增 add_rms_norm_bias 自定义 ACLNN 算子，将逐元素加法（Add）与 RMS 归一化合并为单个 AscendC Kernel 执行，用于 Transformer 模型推理中残差连接 + RMSNorm 的融合优化。 主要变更： - 新增 add_rms_norm_bias 算子源码 - 新增 cann/ 下的 ACLNN 绑定及 torch_binding 算子注册 - 新增 Python 调用接口（api/aclnn/）及 JIT 构建器（op_builder/） - 引入 cmake/common/scripts/utils 等编译工具链及公共组件 - 新增算子使用文档及开发指南（docs/aclnn/） ## Does this PR introduce any user-facing change? 是，新增 npu_add_rms_norm_bias 接口 ## How was this patch tested? 通过单元测试与 CPU 参考实现对比验证： - 对比分离执行 Add + RMSNorm 与融合算子的数值精度 - 覆盖 float32 / float16 / bfloat16 三种数据类型 - 覆盖有 beta / 无 beta 两种场景  See merge request: Ascend/MindSpeed-Ops!54	4 个月前
common	feat: Add add_rms_norm_bias compiletion Co-authored-by: Liccol<740821011@qq.com> # message auto-generated for no-merge-commit merge: !54 merge aclnn-case into master feat: Add add_rms_norm_bias compiletion Created-by: Liccol Commit-by: Liccol Merged-by: ascend-robot Description: ## What this PR does / why we need it? 新增 add_rms_norm_bias 自定义 ACLNN 算子，将逐元素加法（Add）与 RMS 归一化合并为单个 AscendC Kernel 执行，用于 Transformer 模型推理中残差连接 + RMSNorm 的融合优化。 主要变更： - 新增 add_rms_norm_bias 算子源码 - 新增 cann/ 下的 ACLNN 绑定及 torch_binding 算子注册 - 新增 Python 调用接口（api/aclnn/）及 JIT 构建器（op_builder/） - 引入 cmake/common/scripts/utils 等编译工具链及公共组件 - 新增算子使用文档及开发指南（docs/aclnn/） ## Does this PR introduce any user-facing change? 是，新增 npu_add_rms_norm_bias 接口 ## How was this patch tested? 通过单元测试与 CPU 参考实现对比验证： - 对比分离执行 Add + RMSNorm 与融合算子的数值精度 - 覆盖 float32 / float16 / bfloat16 三种数据类型 - 覆盖有 beta / 无 beta 两种场景  See merge request: Ascend/MindSpeed-Ops!54	4 个月前
flop_counter	feat(atb): add MatmulAdd operators for fused wgrad accumulation Co-authored-by: Liz<lizhi166@huawei.com> # message auto-generated for no-merge-commit merge: !182 merge master into master feat(atb): add MatmulAdd operators for fused wgrad accumulation Created-by: Liz_ Commit-by: Liz Merged-by: ascend-robot Description: ## What this PR does / why we need it? Megatron gradient accumulation updates the main weight-gradient buffer as grad += grad_output.T @ total_input. The previous decomposed path materializes an [N, K] intermediate and launches a separate add kernel, increasing allocation and memory traffic. This PR adapts the MindSpeed MatmulAdd implementation to MindSpeed-Ops and provides: - npu_matmul_add_fp32 for bf16/fp16 operands accumulated into an fp32 grad; it uses the ATB fused MatmulAdd path on A2/A3 and addmm_ on A5. - npu_matmul_add_fp16 for bf16/fp16 operands and a matching half-precision grad; it replaces matmul + add_ with one in-place addmm_ call. - zero-sized-input protection, chip dispatch, builder integration, public documentation, and expanded unit coverage. - reproducible ATK case definitions and execution plugin for accuracy, device performance, and device memory validation. Generated cases, reports, and logs stay ignored; the final result summary is retained at tests/atk_tests/atb/matmul_add/atk_output/console/SUMMARY.md. This is feature enablement, not a bug fix. Related issue: N/A. ## Does this PR introduce any user-facing change? Yes. NPU callers can import npu_matmul_add_fp32 or npu_matmul_add_fp16 from mindspeed_ops.api.atb.npu_matmul_add. Both APIs mutate grad in place and return None. Inputs must be 2-D tensors with shapes total_input=[M,K], grad_output=[M,N], and grad=[N,K]. Operand dtypes must match and be bf16 or fp16. The fp32 API requires an fp32 grad buffer; the fp16 API requires grad to match the operand dtype. Empty inputs are a no-op. Usage and benchmark documentation: docs/zh/atb/matmul_add.md. ATK reproduction guide: tests/atk_tests/atb/matmul_add/README.md. ## How was this patch tested? Unit test in the 018 environment: - conda activate lz_py312_018 - python -m pytest tests/unit_tests/atb/test_npu_matmul_add.py -q -rs - Result: 8 passed, 7 skipped. The skipped cases require cann-nnal/ATB and were skipped because ATB_HOME_PATH was not set in this environment. ATK 26.4.30 was run on Ascend 910B3 (A2) with CANN 9.0.0 + ATB, Python 3.10, torch 2.7.1, and torch_npu 2.7.1.post4. Each API used 8 cases covering {bf16, fp16} operands across four shape tuples. - Accuracy, NPU vs CPU high-precision reference: fp32 8/8 Pass; fp16 8/8 Pass. - Device performance, fused dev0 vs decomposed dev1: fp32 average ratio 1.3662 with 100% case pass rate; fp16 average ratio 1.0007 with 87.5% case pass rate and overall Pass. - Device memory, fused dev0 vs decomposed dev1: fp32 and fp16 both 8/8 Pass with 100% memory pass rate under the fused/reference <= 1.1 criterion. The A2/A3 fp32 path requires cann-nnal and ATB_HOME_PATH. Its first invocation JIT-builds the C++ extension and takes about 30 seconds; performance measurements must use warmed, cache-hit runs. The public APIs are Ascend NPU-specific and do not provide a CUDA fallback. [RFC](https://gitcode.com/Ascend/MindSpeed-Ops/issues/80) See merge request: Ascend/MindSpeed-Ops!182	1 个月前
scripts	feat: Add add_rms_norm_bias compiletion Co-authored-by: Liccol<740821011@qq.com> # message auto-generated for no-merge-commit merge: !54 merge aclnn-case into master feat: Add add_rms_norm_bias compiletion Created-by: Liccol Commit-by: Liccol Merged-by: ascend-robot Description: ## What this PR does / why we need it? 新增 add_rms_norm_bias 自定义 ACLNN 算子，将逐元素加法（Add）与 RMS 归一化合并为单个 AscendC Kernel 执行，用于 Transformer 模型推理中残差连接 + RMSNorm 的融合优化。 主要变更： - 新增 add_rms_norm_bias 算子源码 - 新增 cann/ 下的 ACLNN 绑定及 torch_binding 算子注册 - 新增 Python 调用接口（api/aclnn/）及 JIT 构建器（op_builder/） - 引入 cmake/common/scripts/utils 等编译工具链及公共组件 - 新增算子使用文档及开发指南（docs/aclnn/） ## Does this PR introduce any user-facing change? 是，新增 npu_add_rms_norm_bias 接口 ## How was this patch tested? 通过单元测试与 CPU 参考实现对比验证： - 对比分离执行 Add + RMSNorm 与融合算子的数值精度 - 覆盖 float32 / float16 / bfloat16 三种数据类型 - 覆盖有 beta / 无 beta 两种场景  See merge request: Ascend/MindSpeed-Ops!54	4 个月前
utils	feat: Add add_rms_norm_bias compiletion Co-authored-by: Liccol<740821011@qq.com> # message auto-generated for no-merge-commit merge: !54 merge aclnn-case into master feat: Add add_rms_norm_bias compiletion Created-by: Liccol Commit-by: Liccol Merged-by: ascend-robot Description: ## What this PR does / why we need it? 新增 add_rms_norm_bias 自定义 ACLNN 算子，将逐元素加法（Add）与 RMS 归一化合并为单个 AscendC Kernel 执行，用于 Transformer 模型推理中残差连接 + RMSNorm 的融合优化。 主要变更： - 新增 add_rms_norm_bias 算子源码 - 新增 cann/ 下的 ACLNN 绑定及 torch_binding 算子注册 - 新增 Python 调用接口（api/aclnn/）及 JIT 构建器（op_builder/） - 引入 cmake/common/scripts/utils 等编译工具链及公共组件 - 新增算子使用文档及开发指南（docs/aclnn/） ## Does this PR introduce any user-facing change? 是，新增 npu_add_rms_norm_bias 接口 ## How was this patch tested? 通过单元测试与 CPU 参考实现对比验证： - 对比分离执行 Add + RMSNorm 与融合算子的数值精度 - 覆盖 float32 / float16 / bfloat16 三种数据类型 - 覆盖有 beta / 无 beta 两种场景  See merge request: Ascend/MindSpeed-Ops!54	4 个月前
CMakeLists.txt	fix: 商发wheel安全编译扫描整改(rpath/RELRO/栈保护/strip) Co-authored-by: Raining__<wangruining5@huawei.com> # message auto-generated for no-merge-commit merge: !173 merge master into master fix: 商发wheel安全编译扫描整改(rpath/RELRO/栈保护/strip) Created-by: Raining__ Commit-by: Raining__ Merged-by: ascend-robot Description: ## 背景 商发 wheel 包华为安全编译扫描整改。本 PR 修复扫描命中的 4 类问题,覆盖 wheel 内全部 host 侧 ELF 产物(pybind 扩展 + CATLASS 子库 + 设备 kernel .o)。设备 kernel .o 的 PIE/RELRO/栈保护/FS/strip 项因 CANN 设备二进制格式所限不适用,单独走豁免说明。 ## 修改明细 ### 1. 禁止 rpath(c781b17,另含 037ba4c+a57dfd5 的 kernel 侧修复) - mindspeed_ops/csrc/CMakeLists.txt:OPS_COMPILE_OPTIONS 追加 -fno-rtlib-add-rpath,消除 ccec 驱动默认 -frtlib-add-rpath 给设备 kernel .o 写入的 DT_RPATH - CMakeLists.txt:删除 mindspeed_ops_C 的 BUILD_RPATH/INSTALL_RPATH "$ORIGIN:$ORIGIN/lib" - mindspeed_ops/csrc/catlass/{basic_matmul,chunk_loss}/CMakeLists.txt:删除 INSTALL_RPATH "$ORIGIN" - mindspeed_ops/__init__.py:新增 _preload_catlass_libraries()(置于引导序列首位)——以绝对路径 ctypes.CDLL(..., RTLD_GLOBAL) 预载 mindspeed_ops/lib/*.so,利用 SONAME 注册机制使扩展的 NEEDED 依赖在不带 RPATH 的情况下正常解析,功能与原 $ORIGIN/lib RPATH 等价;与文件内既有 _preload_custom_opapi_library 同一模式,CATLASS 未使能(无 lib/ 目录)时为空操作 ### 2. GOT 重定位只读 （全 RELRO）与立即绑定（BIND_NOW）(39354c4、0706d44) - CMakeLists.txt:mindspeed_ops_C 追加链接选项 -Wl,-z,relro,-z,now - 两个 CATLASS 子库 target_link_options 追加 -Wl,-z,relro,-z,now(bisheng 默认仅 -z relro 部分 RELRO,补 -z now 到全 RELRO) - 验证:产物均出现 GNU_RELRO 段 + DT_BIND_NOW + FLAGS_1 NOW ### 3. 栈保护(3de5382) - CMakeLists.txt:mindspeed_ops_C 追加编译选项 -fstack-protector-all(选 -all 而非 -strong,确保所有函数插桩、确定性产生 __stack_chk_fail 引用) - 验证:产物动态符号表出现 __stack_chk_fail@GLIBC_2.4 ### 4. strip(fe649b5) - 两个 CATLASS 子库 target_link_options 追加 -Wl,-s - 验证:.symtab/.strtab 去除,.dynsym 完整保留(动态链接不受影响),file 判定 stripped ## 验证 - 每轮均在 py3.12 + CANN 9.1.0 标准构建镜像内全量重建 wheel,解包对全部 10 个 ELF 逐个 readelf 核验,已修复项无回归(rpath 全为 0) - 容器内 pip install 后 import mindspeed_ops 正常,CATLASS 预载链路日志确认生效 ## 兼容性 - 全部为编译/链接选项与装载顺序调整,无 API/行为变更;-fstack-protector-all 有常规量级运行时开销(每函数 canary 检查),对 host 侧绑定层可忽略 - 不带 CATLASS 的构建(MINDSPEED_BUILD_CATLASS=0)不受影响 See merge request: Ascend/MindSpeed-Ops!173	1 个月前
build.sh	[Feature] Add Aclnn ops compilation and dependencies Co-authored-by: YuhanBai<baiyuhan1@h-partners.com> # message auto-generated for no-merge-commit merge: !52 merge aclnn-adapter into master [Feature] Add aclnn ops compiletion and dependencies Created-by: yuhanBai Commit-by: YuhanBai Merged-by: ascend-robot Description: ## What this PR does / why we need it? 本PR参考vllm-ascend中的算子工程，添加了aclnn算子的编译依赖 RFC: https://gitcode.com/Ascend/MindSpeed-Ops/issues/33 Co-Auther with: [@Liccol](https://gitcode.com/Liccol) [@qq_45570708](https://gitcode.com/qq_45570708) ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-Ops!52	4 个月前
build_aclnn.sh	feat: Add add_rms_norm_bias compiletion Co-authored-by: Liccol<740821011@qq.com> # message auto-generated for no-merge-commit merge: !54 merge aclnn-case into master feat: Add add_rms_norm_bias compiletion Created-by: Liccol Commit-by: Liccol Merged-by: ascend-robot Description: ## What this PR does / why we need it? 新增 add_rms_norm_bias 自定义 ACLNN 算子，将逐元素加法（Add）与 RMS 归一化合并为单个 AscendC Kernel 执行，用于 Transformer 模型推理中残差连接 + RMSNorm 的融合优化。 主要变更： - 新增 add_rms_norm_bias 算子源码 - 新增 cann/ 下的 ACLNN 绑定及 torch_binding 算子注册 - 新增 Python 调用接口（api/aclnn/）及 JIT 构建器（op_builder/） - 引入 cmake/common/scripts/utils 等编译工具链及公共组件 - 新增算子使用文档及开发指南（docs/aclnn/） ## Does this PR introduce any user-facing change? 是，新增 npu_add_rms_norm_bias 接口 ## How was this patch tested? 通过单元测试与 CPU 参考实现对比验证： - 对比分离执行 Add + RMSNorm 与融合算子的数值精度 - 覆盖 float32 / float16 / bfloat16 三种数据类型 - 覆盖有 beta / 无 beta 两种场景  See merge request: Ascend/MindSpeed-Ops!54	4 个月前
torch_binding.cpp	feat: add catlass chunk_loss Co-authored-by: Pr0Wh1teGive<calvinzhu.aiesec.unnc@gmail.com> # message auto-generated for no-merge-commit merge: !132 merge feat/catlass-chunk-loss into master feat: add catlass chunk_loss Created-by: Pr0Wh1teGive Commit-by: Pr0Wh1teGive Merged-by: ascend-robot Description: ## What this PR does / why we need it? 本 PR 新增基于 CATLASS 的 chunk_loss 分块语言模型交叉熵算子。 该算子支持 loss 前向计算，以及 hidden_states 和 head_weight 的反向梯度计算，功能上对齐 MindSpeed-MM 中对应的 chunk_loss 实现，可在支持的场景下直接替换使 用。 主要改动包括： - 新增 CATLASS chunk_loss 内核及 PyTorch 绑定。 - 新增 Python 接口 mindspeed_ops.api.catlass.chunk_loss_forward。 - 支持标量、batch 级和 token 级 alpha 归一化。 - 支持 ignore_index 以及 sum/none 两种 reduction 模式。 - 增加编译配置和单元测试。 - 增加算子文档：docs/catlass/chunk_loss.md。 ## Does this PR introduce any user-facing change? 是。本 PR 新增了可选的 CATLASS 接口： python from mindspeed_ops.api.catlass import chunk_loss_forward 只有在启用 CATLASS 和 chunk_loss 编译选项后，该算子才会被编译。默认构建行为和已有接口不受影响。 使用方式和输入约束请参考： docs/catlass/chunk_loss.md ## How was this patch tested? 本实现已在 Ascend 910B2、CANN 9.1 环境下，使用 BF16 输入完成验证。 测试内容包括： - 与 PyTorch 参考实现进行前向 loss 精度对比。 - 对 hidden_states 和 head_weight 的反向梯度进行精度对比。 - 验证标量、batch 级和 token 级 alpha。 - 验证 ignore_index、非有限 alpha 值及非法输入处理。 - 验证 2K、4K、8K、16K 和 32K 序列长度。 - 验证多种 batch size。 在测试配置下，CATLASS 实现相比 PyTorch 小算子实现取得约 1.5x～1.66x 的加速效果。 See merge request: Ascend/MindSpeed-Ops!132	2 个月前
torch_binding_meta.cpp	feat: Add add_rms_norm_bias compiletion Co-authored-by: Liccol<740821011@qq.com> # message auto-generated for no-merge-commit merge: !54 merge aclnn-case into master feat: Add add_rms_norm_bias compiletion Created-by: Liccol Commit-by: Liccol Merged-by: ascend-robot Description: ## What this PR does / why we need it? 新增 add_rms_norm_bias 自定义 ACLNN 算子，将逐元素加法（Add）与 RMS 归一化合并为单个 AscendC Kernel 执行，用于 Transformer 模型推理中残差连接 + RMSNorm 的融合优化。 主要变更： - 新增 add_rms_norm_bias 算子源码 - 新增 cann/ 下的 ACLNN 绑定及 torch_binding 算子注册 - 新增 Python 调用接口（api/aclnn/）及 JIT 构建器（op_builder/） - 引入 cmake/common/scripts/utils 等编译工具链及公共组件 - 新增算子使用文档及开发指南（docs/aclnn/） ## Does this PR introduce any user-facing change? 是，新增 npu_add_rms_norm_bias 接口 ## How was this patch tested? 通过单元测试与 CPU 参考实现对比验证： - 对比分离执行 Add + RMSNorm 与融合算子的数值精度 - 覆盖 float32 / float16 / bfloat16 三种数据类型 - 覆盖有 beta / 无 beta 两种场景  See merge request: Ascend/MindSpeed-Ops!54	4 个月前
utils.h	feat: Add add_rms_norm_bias compiletion Co-authored-by: Liccol<740821011@qq.com> # message auto-generated for no-merge-commit merge: !54 merge aclnn-case into master feat: Add add_rms_norm_bias compiletion Created-by: Liccol Commit-by: Liccol Merged-by: ascend-robot Description: ## What this PR does / why we need it? 新增 add_rms_norm_bias 自定义 ACLNN 算子，将逐元素加法（Add）与 RMS 归一化合并为单个 AscendC Kernel 执行，用于 Transformer 模型推理中残差连接 + RMSNorm 的融合优化。 主要变更： - 新增 add_rms_norm_bias 算子源码 - 新增 cann/ 下的 ACLNN 绑定及 torch_binding 算子注册 - 新增 Python 调用接口（api/aclnn/）及 JIT 构建器（op_builder/） - 引入 cmake/common/scripts/utils 等编译工具链及公共组件 - 新增算子使用文档及开发指南（docs/aclnn/） ## Does this PR introduce any user-facing change? 是，新增 npu_add_rms_norm_bias 接口 ## How was this patch tested? 通过单元测试与 CPU 参考实现对比验证： - 对比分离执行 Add + RMSNorm 与融合算子的数值精度 - 覆盖 float32 / float16 / bfloat16 三种数据类型 - 覆盖有 beta / 无 beta 两种场景  See merge request: Ascend/MindSpeed-Ops!54	4 个月前
version.info	feat: Add add_rms_norm_bias compiletion Co-authored-by: Liccol<740821011@qq.com> # message auto-generated for no-merge-commit merge: !54 merge aclnn-case into master feat: Add add_rms_norm_bias compiletion Created-by: Liccol Commit-by: Liccol Merged-by: ascend-robot Description: ## What this PR does / why we need it? 新增 add_rms_norm_bias 自定义 ACLNN 算子，将逐元素加法（Add）与 RMS 归一化合并为单个 AscendC Kernel 执行，用于 Transformer 模型推理中残差连接 + RMSNorm 的融合优化。 主要变更： - 新增 add_rms_norm_bias 算子源码 - 新增 cann/ 下的 ACLNN 绑定及 torch_binding 算子注册 - 新增 Python 调用接口（api/aclnn/）及 JIT 构建器（op_builder/） - 引入 cmake/common/scripts/utils 等编译工具链及公共组件 - 新增算子使用文档及开发指南（docs/aclnn/） ## Does this PR introduce any user-facing change? 是，新增 npu_add_rms_norm_bias 接口 ## How was this patch tested? 通过单元测试与 CPU 参考实现对比验证： - 对比分离执行 Add + RMSNorm 与融合算子的数值精度 - 覆盖 float32 / float16 / bfloat16 三种数据类型 - 覆盖有 beta / 无 beta 两种场景  See merge request: Ascend/MindSpeed-Ops!54	4 个月前