从报错信息来看,capture存在流同步操作(capture期间不允许存在流同步操作),请提供运行脚本进一步分析


我遇到了同样的报错,编译模式为config.mode = "reduce-overhead"
[W108 06:23:36.660816180 compiler_depend.ts:250] Warning: CAUTION: The operator 'aten::gelu.out' is not currently supported on the NPU backend and will fall back to run on the CPU. This may have performance implications. (function npu_cpu_fallback)
Error with batch_size=1: copy_between_host_and_device_opapi:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:54 NPU function error: aclrtMemcpy, error code is 107030
[ERROR] 2026-01-08-06:23:36 (PID:3072093, Device:7, RankID:-1) ERR00100 PTA call acl api failed.
EE9999: Inner Error!
EE9999[PID: 3072093] 2026-01-08-06:23:36.451.921 (EE9999): Not allow to synchronize captured-stream, stream_id=46.[FUNC:StreamSynchronize][FILE:api_error.cc][LINE:962]
TraceBack (most recent call last):
rtStreamSynchronize execute failed, reason=[stream is captured][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
synchronize stream failed, runtime result = 107027[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:162]
rtMemcpy execute failed, reason=[the current capture mode does not support this operation][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
synchronized memcpy failed, kind = 2, runtime result = 107030[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:162]
详细报错日志,0a7ce077d5b14518a325b39f1336bc34.txt 对应的开源代码,https://github.com/KlingTeam/LivePortrait,设计的相关代码src/modules/convnextv2.py,Block类


通过实践分析,代码中,torch.arange(w).type(ref.dtype).to(ref.device) 是在计算图构建过程中创建 CPU 张量并试图传输到 NPU,这在捕获模式下是不允许的。改为 在 NPU 上直接创建张量:可以解决该问题
# 原来的代码:
# x = torch.arange(w).type(ref.dtype).to(ref.device)
# 改为:
x = torch.arange(w, dtype=ref.dtype, device=ref.device)
y = torch.arange(h, dtype=ref.dtype, device=ref.device)
z = torch.arange(d, dtype=ref.dtype, device=ref.device)
但是又遇到了同样的报错,
Error with batch_size=1: copy_between_host_and_device_opapi:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:54 NPU function error: aclrtMemcpy, error code is 107030
[ERROR] 2026-01-08-07:51:27 (PID:3125800, Device:7, RankID:-1) ERR00100 PTA call acl api failed.
EE9999: Inner Error!
EE9999[PID: 3125800] 2026-01-08-07:51:27.707.740 (EE9999): Not allow to synchronize captured-stream, stream_id=60.[FUNC:StreamSynchronize][FILE:api_error.cc][LINE:962]
TraceBack (most recent call last):
rtStreamSynchronize execute failed, reason=[stream is captured][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
synchronize stream failed, runtime result = 107027[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:162]
rtMemcpy execute failed, reason=[the current capture mode does not support this operation][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
synchronized memcpy failed, kind = 2, runtime result = 107030[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:162] Original traceback:
File ,
具体的代码为,
class SPADE(nn.Module):
def __init__(self, norm_nc, label_nc):
super().__init__()
self.param_free_norm = nn.InstanceNorm2d(norm_nc, affine=False)
nhidden = 128
self.mlp_shared = nn.Sequential(
nn.Conv2d(label_nc, nhidden, kernel_size=3, padding=1),
nn.ReLU())
self.mlp_gamma = nn.Conv2d(nhidden, norm_nc, kernel_size=3, padding=1)
self.mlp_beta = nn.Conv2d(nhidden, norm_nc, kernel_size=3, padding=1)
def forward(self, x, segmap):
normalized = self.param_free_norm(x)
segmap = F.interpolate(segmap, size=x.size()[2:], mode='nearest')
actv = self.mlp_shared(segmap)
gamma = self.mlp_gamma(actv)
beta = self.mlp_beta(actv)
out = normalized * (1 + gamma) + beta
return out


经分析脚本中存在copy_between_host_and_device算子操作,导致capture失败。(资料:https://www.hiascend.com/document/detail/zh/canncommercial/850/appdevg/acldevg/aclcppdevg_000519.html ---在捕获过程中,在ACL_MODEL_RI_CAPTURE_MODE_GLOBAL模式(全局禁止,所有线程都不可以调用非安全函数)下,调用内存同步操作类函数(例如aclrtMemset、aclrtMemcpy、aclrtMemcpy2d)是非法的,会校验报错导致捕获失败。)
建议尝试使用其他算子替代该功能
分析过程:
开启TORCH_COMPILE_DEBUG后,文件路径为torch_compile_debug/run_******/torchair/model__0/forward,文件列表如下:
├── 000_aot_forward_graph.txt---查看代码和算子对应关系
├── 001_aot_forward_graph_after_optimize_noop_ops.txt
├── 002_aot_forward_graph_after_recover_view_inplace_pattern.txt
├── 003_aot_forward_graph_after_apply_pattern_passes.txt
├── 004_aot_forward_graph_after_view_to_reshape.txt
├── 005_aot_forward_graph_after_apply_event_closure_with_multi_stream.txt
├── 006_aot_forward_graph_after_apply_event_record.txt
├── 007_aot_forward_graph_after_eliminate_dead_code.txt
├── 008_aot_forward_graph_after_reinplace_inplaceable_ops_pass.txt
├── 009_aot_forward_graph_after_reinplace_input_mutated_ops.txt
├── 010_aot_forward_graph_after_decompose_auto_functionalized.txt
├── 011_aot_forward_graph_after_replace_dynamic_workspace_ops.txt
├── 012_aot_forward_graph_after_replace_core_limit_nodes.txt
└── output_code.py ## 输出代码, 在22行存在该操作
报错日志中显示异常操作在output_code.py中22行
File "/torchair/_acl_concrete_graph/acl_graph.py", line 1127, in capture
captured_outputs = self.fx_forward(*args_list, node_info=self._updated_node_infos, is_capturing=True,
File "
output_code.py文件内容如下:
def forward(*args, node_info=[], is_capturing: bool = False):
primals_1, primals_2, primals_3, primals_4, primals_5, primals_6, primals_7, primals_8 = args
view = torch.ops.aten.reshape.default(primals_1, [1, 128, 256, 256]); primals_1 = None
_native_batch_norm_legit = torch.ops.aten._native_batch_norm_legit.no_stats(view, None, None, True, 0.1, 1e-05); view = None ### 该操作触发
getitem = _native_batch_norm_legit[0]; _native_batch_norm_legit = None
view_1 = torch.ops.aten.reshape.default(getitem, [2, 64, 256, 256]); getitem = None
upsample_nearest2d = torch.ops.aten.upsample_nearest2d.default(primals_2, [256, 256]); primals_2 = None
convolution = torch.ops.aten.convolution.default(upsample_nearest2d, primals_3, primals_4, [1, 1], [1, 1], [1, 1], False, [0, 0], 1); primals_4 = None
relu = torch.ops.aten.relu_.default(convolution); relu = None
convolution_1 = torch.ops.aten.convolution.default(convolution, primals_5, primals_6, [1, 1], [1, 1], [1, 1], False, [0, 0], 1); primals_6 = None
convolution_2 = torch.ops.aten.convolution.default(convolution, primals_7, primals_8, [1, 1], [1, 1], [1, 1], False, [0, 0], 1); primals_8 = None
add = torch.ops.aten.add_.Tensor(convolution_1, 1); add = None
mul = torch.ops.aten.mul.Tensor(view_1, convolution_1); convolution_1 = None
add_1 = torch.ops.aten.add_.Tensor(mul, convolution_2); convolution_2 = add_1 = None
return (mul, primals_3, primals_5, primals_7, view_1, upsample_nearest2d, convolution)
aot_forward_graph.txt文件内容: 代码对应算子位置:
class GraphModule(torch.nn.Module):
def forward(self, primals_1: "f32[2, 64, 256, 256]", primals_2: "f32[2, 12, 64, 64]", primals_3: "f32[128, 12, 3, 3]", primals_4: "f32[128]", primals_5: "f32[64, 128, 3, 3]", primals_6: "f32[64]", primals_7: "f32[64, 128, 3, 3]", primals_8: "f32[64]"):
# test.py:31 in forward, code: normalized = self.param_free_norm(x) ##该操作中存在host和device直接的内存拷贝导致capture失败。
view: "f32[1, 128, 256, 256]" = torch.ops.aten.view.default(primals_1, [1, 128, 256, 256]); primals_1 = None
_native_batch_norm_legit = torch.ops.aten._native_batch_norm_legit.no_stats(view, None, None, True, 0.1, 1e-05); view = None
getitem: "f32[1, 128, 256, 256]" = _native_batch_norm_legit[0]; _native_batch_norm_legit = None
view_1: "f32[2, 64, 256, 256]" = torch.ops.aten.view.default(getitem, [2, 64, 256, 256]); getitem = None


在提交问题之前,请通过搜索现有和历史问题确保该问题尚未被提出并解决。
您的环境信息
🐛 请描述bug
报错信息如下,请帮忙看下是什么原因: