已合并
[sync] PR-37189: fix: unskip test_reductions test case in test_multiprocessing_api.py #37686
ascend-robot创建于 6月5日
[sync] PR-37189: fix: unskip test_reductions test case in test_multiprocessing_api.py #37686
已合并
Pull Request已成功合入, 合并人@ascend-robot
（感谢 ascend-robot 的贡献）
6月5日 创建了 pull request，commit 50564b88
ascend-robot
6月5日 评论：
6月5日 评论：
6月5日 添加了label：stat/needs-squash
6月5日 添加了label：ascend-cla/yes
ascend-robot
6月5日 评论：
6月5日 评论：
Thanks for your pull-request.
The full list of commands accepted by me can be found at here。
You can get sig-info at here
PR Approval Progress
✅ Congratulations! All modules have met the lgtm and approve requirements.
Module Approval Details
| module | lgtm status | approve status |
|---|---|---|
| test | ✅ liujunzhu, wanglijun55 (2/2) | ✅ liujunzhu (1/1) |
💡 Tip:
- Committer can comment
/approveor/lgtm- Commenting
/approveimplies both code review (lgtm) and intent to merge (approve)
CLA Signature Pass
ascend-ds-bot, thanks for your pull request. All authors of the commits have signed the CLA. 👍
wuyouqi1
6月5日 评论：
6月5日 评论：
compile
ascend-robot
6月5日 评论：
6月5日 评论：
Ascend docs pipeline is running...
6月5日 添加了label：docs-ci-pipeline-running
6月5日 添加了label：ci-pipeline-running
ascend-robot
6月5日 评论：
6月5日 评论：
✅ 跳过 docs ci 检查，没有需要检查的文档文件
6月5日 删除了label：docs-ci-pipeline-running
6月5日 添加了label：docs-ci-pipeline-success
ascend-robot
6月5日 评论：
6月5日 评论：
Ascend docs pipeline is running...
6月5日 删除了label：docs-ci-pipeline-success
6月5日 添加了label：docs-ci-pipeline-running
ascend-robot
6月5日 评论：
6月5日 评论：
✅ 跳过 docs ci 检查，没有需要检查的文档文件
6月5日 删除了label：docs-ci-pipeline-running
6月5日 添加了label：docs-ci-pipeline-success
6月5日 删除了label：ci-pipeline-running
6月5日 添加了label：ci-pipeline-passed
ascend-robot
6月5日 评论：
6月5日 评论：
流水线 PR-pipeline_pytorch#29909 已完成
| 阶段 | 任务名 | 状态 | 详情 |
|---|---|---|---|
| 编译构建 | Build_X86 | ✅ | >>> |
| Build_ARM | ✅ | >>> | |
| Build_LibTorch_x86 | ✅ | >>> | |
| Build_LibTorch_ARM | ✅ | >>> | |
| Build_X86_torchair | 🛑 | >>> | |
| Build_ARM_torchair | 🛑 | >>> | |
| patch_test | 🛑 | >>> | |
| 恶意代码检查 | Antipoison | ✅ | >>> |
| 编码安全与规范检查 | CodeCheck | ✅ | >>> |
| check_error | ✅ | >>> | |
| CodeCheck_lintrunner | ✅ | >>> | |
| 开源片段检查 | SCA | ✅ | >>> |
| 开发者测试 | UT_X86_Part_01 | 🛑 | >>> |
| UT_X86_Part_02 | 🛑 | >>> | |
| UT_ARM_A3_Part_01 | 🛑 | >>> | |
| UT_ARM_A3_Part_02 | 🛑 | >>> | |
| UT_ARM_A2_Part_01 | ✅ | >>> | |
| UT_ARM_A2_Part_02 | ✅ | >>> | |
| UT_ARM_A2_Part_03 | ✅ | >>> | |
| UT_inductor_Part_01 | 🛑 | >>> | |
| UT_inductor_Part_02 | 🛑 | >>> | |
| UT_inductor_Part_03 | 🛑 | >>> | |
| UT_inductor_Part_04 | 🛑 | >>> | |
| UT_DIST_ARM_Part_01 | 🛑 | >>> | |
| UT_DIST_ARM_Part_02 | 🛑 | >>> | |
| UT_DIST_ARM_Part_03 | 🛑 | >>> | |
| UT_DIST_ARM_Part_04 | 🛑 | >>> | |
| UT_ARM_A2_Select_Part_01 | ✅ | >>> | |
| UT_ARM_A2_Select_Part_02 | ✅ | >>> | |
| 流水线 | PR-pipeline_pytorch | ✅ | >>> |
liujunzhu
6月5日 评论：
6月5日 评论：
/approve
6月5日 添加了label：approved
wanglijun55
6月5日 评论：
6月5日 评论：
/lgtm
6月5日 添加了label：lgtm
6月5日 合入了pull request
ascend-robot
6月5日 评论：
6月5日 评论：
流水线 pytorch_gitcode_PR_multiVersion#10079 [ commitID：7de9c40e ] 已完成
1. Origin pull request:
https://gitcode.com/Ascend/pytorch/merge_requests/37189
2. Original pull request related issue(s):
3. Original pull request related commit(s):
Add @unittest.skipUnless to test_reductions to skip the test when
driver version < 25.3, as NPU IPC reduce/rebuild requires driver >= 25.3.
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
This reverts commit d1ee69dcbd891a48ced0f34471547b25710a5679.
Add torch_npu.multiprocessing.reductions debug logger to trace:
- _npu_reduce_tensor: which branch (NPU/meta/CPU), share_npu() return values, cache key format
- rebuild_npu_tensor: cache lookup key format, HIT/MISS, _new_shared_npu() result, rebuilt tensor data
- test worker: rebuild_fn args, rebuilt tensor data in child process
Purpose: verify cross-process IPC reduce→rebuild takes correct code path.
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
CPU tensor pickled through Queue uses rebuild_storage_fd which requires
resource_sharer Unix socket. Child process exits before parent can
establish connection. Convert tensor to plain list before sending.
Co-Authored-By: Claude noreply@anthropic.com
Address review comments:
- Worker now receives (rebuild_fn, args) from reduce_tensor, explicitly
calls rebuild_fn(*args) to reconstruct tensor
- Main process puts reduced_npu handle instead of raw tensor
- Main process compares result with original npu_tensor instead of
hardcoding expected values in worker
Co-Authored-By: Claude noreply@anthropic.com
Child process received all zeros because NPU write cache was not
flushed before IPC handle export. torch.npu.synchronize() ensures
data is written to physical memory before cross-process transfer.
Co-Authored-By: Claude noreply@anthropic.com
Include actual received data in failure message to diagnose IPC issue.
Co-Authored-By: Claude noreply@anthropic.com
Worker now sends result status back via Queue instead of relying on
exitcode. This propagates actual error messages from the child process
for diagnosis.
Co-Authored-By: Claude noreply@anthropic.com
Main process puts tensor to Queue, child process gets and validates.
Matches torch_npu multiprocessing test patterns. Use Queue instead of
SimpleQueue to avoid resource_sharer lifecycle issues.
Co-Authored-By: Claude Opus 4.7 noreply@anthropic.com
SimpleQueue.get() does not accept timeout parameter in Python 3.10.
Co-Authored-By: Claude Opus 4.7 noreply@anthropic.com
Combine NPU multiprocess rebuild into test_reductions instead of a separate
test case. CPU tensor uses same-process rebuild, NPU tensor uses cross-process
rebuild via Queue.
Co-Authored-By: Claude noreply@anthropic.com
spawn mode cannot pickle local functions. Move _npu_tensor_worker to
module top level.
Co-Authored-By: Claude noreply@anthropic.com
- Remove @unittest.skip decorator from test_reductions
- Remove unreliable same-process NPU tensor reconstruction assertion
(IPC handles cannot be imported in the same process that exported them)
- Add test_reductions_multiprocess to validate cross-process NPU tensor
sharing via Queue, aligned with PyTorch upstream CUDA test pattern
Co-Authored-By: Claude noreply@anthropic.com