| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
feat(profiler): mspti backend for torch.profiler PrivateUse1 Co-authored-by: Dmitry Gladkov<gladkov.dmitry1@huawei.com> # message auto-generated for no-merge-commit merge: !44554 merge feat/mspti-kineto-backend into master feat(profiler): mspti backend for torch.profiler PrivateUse1 Created-by: gladkov_dmitry Commit-by: Dmitry Gladkov Merged-by: ascend-robot Description: <!-- PR描述模板更新日期：20260203 --> # 【合入来源】 - [x] Requirement / Feature - [ ] 问题单 - [ ] issue/工单 - [ ] 重构优化 - [ ] 资料更新 # 【Modification Scheme】 torch.profiler on Ascend collected CPU activity only. Nothing supplied device data for ProfilerActivity.PrivateUse1, so the NPU half of the timeline was empty and users had to reach for a second, incompatible profiler to see what the device did. This adds a libkineto plugin backed by mspti. NPU kernels now appear in a standard torch.profiler trace next to CPU operations, on one timeline. > PyTorch and libkineto are unchanged. The plugin registers itself through REGISTER_PRIVATEUSE1_PROFILER and is driven by libkineto through IActivityProfilerSession. > This is not torch_npu.profiler. That is a separate implementation over acl_prof with its own enums and configuration class; the two are independent front ends and can be used side by side. Files: > torch_npu/csrc/profiler/mspti_activity_profiler.{h,cpp}: the session - lifecycle, record conversion, device and stream lanes, flow emission. > torch_npu/csrc/framework/interface/MsptiInterface.{h,cpp}: wrappers over libmspti. Every symbol is resolved at runtime through FunctionLoader, so a missing or older libmspti.so gives an empty device lane rather than a link failure. > torch_npu/_init/registry/backend.py, CMakeLists.txt: registration and build. Kernels are linked back to the operations that launched them. A global RecordFunction callback pushes each operation's handle as an mspti external correlation id; mspti ties that handle to the correlation id of the launch call, which the kernel shares. processTrace turns the join into ac2g flows - the arrows a viewer draws from a CPU operation to the kernel it produced. > About three quarters of kernels link. The rest are launched outside any operation scope - allocator work, autograd internals, runtime housekeeping - and have nothing to point back to. The proportion is stable as the workload grows. > The flow tail is a PRIVATEUSE1_RUNTIME marker on the host process, mirroring CUDA where flows start on a cuda_runtime event rather than on a cpu_op. Two presentation decisions worth knowing when reading a trace: > Device operators export without an mspti:: prefix, so they read the way CUDA kernels do in a torch.profiler trace. > The NPU process takes a sort index above any real pid, so its lane renders below the CPU lane. Correlation ids on exported activities come from a private counter rather than from mspti: mspti's ids are not unique per event and collide with torch's low cpu-side ids, which would fail upstream's uniqueness check. stop() drains until records stop arriving rather than until a fixed deadline. mspti delivers asynchronously, roughly 200 ms behind execution, so a time cap truncated the tail of a recording. The loop now exits once several consecutive flushes add nothing, which captures more and returns in about 150 ms. # 【Documentation Change】 > Not involved. Behaviour and limitations are described in the module header. # 【Interface Change】 > No API signature change. Customer-visible: profiling with activities=[ProfilerActivity.CPU, ProfilerActivity.PrivateUse1] now yields NPU device kernels in the trace and in events() / key_averages(), where previously the device side was empty. Existing CPU-only behaviour is unchanged. # 【Functional Verification】 bash cd /tmp && python test/profiler/test_mspti_backend.py > 12 tests in three classes on Ascend 910B4, CANN 9.1.0, torch 2.13.0+cpu. > TestMsptiBackend: kernels reach the trace, carry type and streamId, export without the mspti:: prefix, the NPU lane sorts below the CPU lane, and every cycle of a schedule() run collects - so the backend re-arms between cycles. > TestMsptiAc2g: flows are emitted as start and finish pairs, every start has a finish, one start per operation, none points backwards in time, the ids match operations recorded by torch, the two ends land on different lanes, and the whole trace fits one timeline. > Kernel counts are never asserted in the tests: mspti delivers records asynchronously, so counts vary between runs. The assertions are structural. Trace view. Collected from bare tensor operations rather than a model, so the device lane holds a predictable set of kernels: python a = torch.randn(4096, 4096, device="npu") b = torch.randn(4096, 4096, device="npu") def phases(): with record_function("matmul"): for _ in range(5): torch.matmul(a, b) with record_function("add"): for _ in range(5): torch.add(a, b) with record_function("sum"): for _ in range(5): torch.sum(a) phases() # warm up outside profiling torch.npu.synchronize() with profile(activities=[ProfilerActivity.CPU, ProfilerActivity.PrivateUse1]) as prof: phases() torch.npu.synchronize() prof.export_chrome_trace("trace.json") Fifteen operations issued, <N> device kernels collected. 4096x4096 keeps each kernel millisecond-scale and visible. Opened at ui.perfetto.dev.  > The NPU lane sits below the CPU lane with its kernels named, against the matmul, add and sum phases above, and an ac2g arrow runs from the operation to the kernel it launched. > The flow tail is placed at the kernel's timestamp rather than the host launch time: the host clock reachable from this path has a different base. The link itself is exact; only the arrow's visual origin is approximate. > On CANN versions: only 9.1.0 is installed on the verification machine, so this was not built against an earlier CANN. The mspti dependency is resolved at runtime through FunctionLoader rather than linked, so a CANN without libmspti.so gives an empty device lane rather than a build or load failure. The kineto include added to CMakeLists.txt points at PyTorch's bundled headers, not CANN's, so it carries no CANN version dependency. # 【CheckList】 - [x] Comments complete - [x] Return-value / null-pointer checks done - [x] PR title uses type label (feat) - [ ] PR持续集成流水线（CI）执行通过，代码检查无异常 See merge request: Ascend/pytorch!44554 | 2 个月前 | |
chore(mspti): vendor only the mspti declarations torch_npu uses Co-authored-by: Dmitry Gladkov<gladkov.dmitry1@huawei.com> # message auto-generated for no-merge-commit merge: !43430 merge chore/mspti-vendor-headers into master chore(mspti): vendor only the mspti declarations torch_npu uses Created-by: gladkov_dmitry Commit-by: Dmitry Gladkov Merged-by: ascend-robot Description: <!-- PR描述模板更新日期：20260203 --> # 【合入来源】 - [x] Requirement / Feature - [ ] 问题单 - [ ] issue/工单 - [ ] 重构优化 - [ ] 资料更新 # 【Modification Scheme】 Replace the hand-written mspti_activity.h stub added by PR-37136 with the declarations actually needed, extracted verbatim from the CANN SDK headers. This is preparation for the mspti-backed kineto activity profiler; no code in this PR consumes the new declarations beyond what master already calls. > third_party/mspti/mspti_activity.h: the activity kinds, the external correlation kinds, the communication data types, the two record structures the backend parses (KERNEL and COMMUNICATION), the buffer request and complete callbacks, and the ten functions called through MsptiInterface. > third_party/mspti/mspti_result.h: added, because mspti_activity.h includes it for msptiResult. > Struct and enum bodies are copied from the SDK unchanged. Records are cast out of a raw buffer, so field order, field types and enum values are the wire format and must not be retyped. > Unused record kinds (MEMORY, MEMSET, MEMCPY, MARKER, API, HCCL), unused functions and mspti_cbid.h are not vendored. A full copy of the SDK headers would be 1257 lines; this is 435. Three deliberate differences from the SDK: > mspti_activity.h declares msptiActivityIsEnabled. It is a documented API of mspti 26.1.0 that the PR-37136 stub omitted, and torch_npu already calls it through MsptiInterface from mstx_mgr. https://gitcode.com/Ascend/mspti/blob/master/docs/zh/api_reference/c_api/context/msptiActivityIsEnabled.md > mspti_result.h uses the MSPTI_RESULT_H include guard rather than the SDK's MSPTI_BASE_H, per review, so the guard matches the filename. > The stub's ACL_FUNC_VISIBILITY decoration and its #include "acl/acl_base.h" are dropped. The current SDK headers use a #pragma GCC visibility block instead, and ACL_FUNC_VISIBILITY expands to nothing unless FUNC_VISIBILITY is defined. Adding support for another record kind means copying that struct in from the SDK at that time. # 【Documentation Change】 > Not involved. Each declaration keeps the SDK's own Doxygen comment. # 【Interface Change】 > No API signature change and no customer-visible change. These are vendored third-party declarations; the only consumer on master is MsptiInterface, whose existing call to msptiActivityIsEnabled continues to compile. # 【Functional Verification】 > The declarations can be checked against the installed SDK rather than read: bash for h in third_party/mspti/*.h; do diff "$h" "$ASCEND_HOME_PATH/tools/mspti/include/$(basename $h)" done > Every hunk is one of the three deviations listed above; there are no others. > Build and test: bash TORCH_DEVICE_BACKEND_AUTOLOAD=0 bash ci/build.sh --python=3.11 > A missing enum value or struct field is a hard compile error, so the build is the verification that the extracted subset is complete. # 【CheckList】 - [x] Comments complete; the SDK's own Doxygen is retained per declaration - [x] Return-value / null-pointer checks done (declarations only, no logic) - [x] PR title uses type label (chore) - [ ] PR持续集成流水线（CI）执行通过，代码检查无异常 See merge request: Ascend/pytorch!43430 | 2 个月前 | |
chore(mspti): vendor only the mspti declarations torch_npu uses Co-authored-by: Dmitry Gladkov<gladkov.dmitry1@huawei.com> # message auto-generated for no-merge-commit merge: !43430 merge chore/mspti-vendor-headers into master chore(mspti): vendor only the mspti declarations torch_npu uses Created-by: gladkov_dmitry Commit-by: Dmitry Gladkov Merged-by: ascend-robot Description: <!-- PR描述模板更新日期：20260203 --> # 【合入来源】 - [x] Requirement / Feature - [ ] 问题单 - [ ] issue/工单 - [ ] 重构优化 - [ ] 资料更新 # 【Modification Scheme】 Replace the hand-written mspti_activity.h stub added by PR-37136 with the declarations actually needed, extracted verbatim from the CANN SDK headers. This is preparation for the mspti-backed kineto activity profiler; no code in this PR consumes the new declarations beyond what master already calls. > third_party/mspti/mspti_activity.h: the activity kinds, the external correlation kinds, the communication data types, the two record structures the backend parses (KERNEL and COMMUNICATION), the buffer request and complete callbacks, and the ten functions called through MsptiInterface. > third_party/mspti/mspti_result.h: added, because mspti_activity.h includes it for msptiResult. > Struct and enum bodies are copied from the SDK unchanged. Records are cast out of a raw buffer, so field order, field types and enum values are the wire format and must not be retyped. > Unused record kinds (MEMORY, MEMSET, MEMCPY, MARKER, API, HCCL), unused functions and mspti_cbid.h are not vendored. A full copy of the SDK headers would be 1257 lines; this is 435. Three deliberate differences from the SDK: > mspti_activity.h declares msptiActivityIsEnabled. It is a documented API of mspti 26.1.0 that the PR-37136 stub omitted, and torch_npu already calls it through MsptiInterface from mstx_mgr. https://gitcode.com/Ascend/mspti/blob/master/docs/zh/api_reference/c_api/context/msptiActivityIsEnabled.md > mspti_result.h uses the MSPTI_RESULT_H include guard rather than the SDK's MSPTI_BASE_H, per review, so the guard matches the filename. > The stub's ACL_FUNC_VISIBILITY decoration and its #include "acl/acl_base.h" are dropped. The current SDK headers use a #pragma GCC visibility block instead, and ACL_FUNC_VISIBILITY expands to nothing unless FUNC_VISIBILITY is defined. Adding support for another record kind means copying that struct in from the SDK at that time. # 【Documentation Change】 > Not involved. Each declaration keeps the SDK's own Doxygen comment. # 【Interface Change】 > No API signature change and no customer-visible change. These are vendored third-party declarations; the only consumer on master is MsptiInterface, whose existing call to msptiActivityIsEnabled continues to compile. # 【Functional Verification】 > The declarations can be checked against the installed SDK rather than read: bash for h in third_party/mspti/*.h; do diff "$h" "$ASCEND_HOME_PATH/tools/mspti/include/$(basename $h)" done > Every hunk is one of the three deviations listed above; there are no others. > Build and test: bash TORCH_DEVICE_BACKEND_AUTOLOAD=0 bash ci/build.sh --python=3.11 > A missing enum value or struct field is a hard compile error, so the build is the verification that the extracted subset is complete. # 【CheckList】 - [x] Comments complete; the SDK's own Doxygen is retained per declaration - [x] Return-value / null-pointer checks done (declarations only, no logic) - [x] PR title uses type label (chore) - [ ] PR持续集成流水线（CI）执行通过，代码检查无异常 See merge request: Ascend/pytorch!43430 | 2 个月前 |