已关闭
[Question|问题咨询]: 调试过程中建链报错 #26
youxiao创建于 2025年11月8日关闭于 2025年11月8日
youxiao
2025年11月12日 评论：
2025年11月12日 评论：
分析报错日志，底层报错日志信息，可以看出传入内存不合法，内存不合法通常有几种:
- 内存类型错误，传入内存类型和内存真实类型不匹配;
- 地址对齐错误：https://gitcode.com/cann/hixl/issues/14;
- 实际内存size和传入内存size不匹配。
排查传入的内存的大小和真实的内存大小，发现是size不匹配 - rdma注册内存会消耗系统内存，系统内存不足时会报注册MR失败，往往伴随着aicpu out of memory
wang-chunyang
6月26日 评论：
6月26日 评论：
你好，请问后续是怎么解决的呢？我们也遇到了类似问题。但是我们直接启动的vllm服务，自己没有去修改HCCL内存申请这块
Welcome to ask questions and discuss with other members.
问题描述
/root/ascend/log/debug/plog/plog-130245_20251108085519231.log:8852:[ERROR] HCCP(130245,python3):2025-11-08-08:55:21.692.085 [ra_hdc_rdma.c:609]tid:131014,ra_hdc_typical_mr_reg : [reg][ra_hdc_typical_mr]ra hdc message process failed ret(-13) phy_id(2)
/root/ascend/log/debug/plog/plog-130245_20251108085519231.log:8854:[ERROR] HCCL(130245,python3):2025-11-08-08:55:21.692.090 [adapter_hccp.cc:332] [131014][hrtRaRegGlobalMr]errNo[0x0000000005000013] ra reg global mr fail. return[128102], params: addr[0x12c085200000], size[143917056 Byte], access[7]
/root/ascend/log/debug/plog/plog-130245_20251108085519231.log:8855:[ERROR] HCCL(130245,python3):2025-11-08-08:55:21.692.093 [local_rdma_rma_buffer_impl.cc:120] [131014][Init]call trace: hcclRet -> 19
/root/ascend/log/debug/plog/plog-130245_20251108085519231.log:8856:[ERROR] HCCL(130245,python3):2025-11-08-08:55:21.692.096 [local_rdma_rma_buffer.cc:43] [131014][LocalRdmaRmaBuffer][Init]Init failed, ret[19]
/root/ascend/log/debug/plog/plog-130245_20251108085519231.log:8857:[ERROR] HCCL(130245,python3):2025-11-08-08:55:21.692.098 [hccl_mem.cc:115] [131014][HcclMemRegRoce]localbuffer init failed 19.
/root/ascend/log/debug/plog/plog-130245_20251108085519231.log:8858:[ERROR] HCCL(130245,python3):2025-11-08-08:55:21.692.100 [hccl_one_sided_service.cc:785] [131014][RegisterBoundMems]call trace: hcclRet -> 19
/root/ascend/log/debug/plog/plog-130245_20251108085519231.log:8859:[ERROR] HCCL(130245,python3):2025-11-08-08:55:21.692.102 [hccl_one_sided_service.cc:601] [131014][PrepareFullMesh]call trace: hcclRet -> 19
/root/ascend/log/debug/plog/plog-130245_20251108085519231.log:8873:[ERROR] HCCL(130245,python3):2025-11-08-08:55:21.693.377 [hccl_one_sided_service.cc:644] [131014][HcclOneSidedService][Prepare] Prepare failed. commIdentifier[7.150.15.164:20083_7.150.15.164:22407]
/root/ascend/log/debug/plog/plog-130245_20251108085519231.log:8874:[ERROR] HCCL(130245,python3):2025-11-08-08:55:21.693.380 [one_sided_service_adapt.cc:586] [131014][HcclCommPrepare]call trace: hcclRet -> 19