torchvision的nms可以在安装Torchvision Adapter后在npu上处理
可参考
https://gitcode.com/Ascend/vision

另外torch_npu这边提供了nms相关自定义算子供参考使用
https://www.hiascend.com/document/detail/zh/Pytorch/720/apiref/torchnpuCustomsapi/context/（beta）torch_npu-npu_nms_v4.md

你好，我按照您给的torchvision的nms可以在安装Torchvision Adapter后在npu上处理，出现了下面的错误，应该怎么解决呢？

Traceback (most recent call last):
File "/home/HwHiAiUser/.conda/envs/3dinference/lib/python3.10/site-packages/torch/init.py", line 2756, in _import_device_backends
entrypoint = backend_extension.load()
File "/home/HwHiAiUser/.conda/envs/3dinference/lib/python3.10/importlib/metadata/init.py", line 171, in load
module = import_module(match.group('module'))
File "/home/HwHiAiUser/.conda/envs/3dinference/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/home/HwHiAiUser/.conda/envs/3dinference/lib/python3.10/site-packages/torch_npu/init.py", line 41, in
import torch_npu.npu
File "/home/HwHiAiUser/.conda/envs/3dinference/lib/python3.10/site-packages/torch_npu/npu/init.py", line 128, in
from torch_npu.utils import _should_print_warning
File "/home/HwHiAiUser/.conda/envs/3dinference/lib/python3.10/site-packages/torch_npu/utils/init.py", line 22, in
from ._graph_tree import _apply_npugraph_tree_methods
File "/home/HwHiAiUser/.conda/envs/3dinference/lib/python3.10/site-packages/torch_npu/utils/_graph_tree.py", line 26, in
from torch._inductor.compile_fx import (
File "/home/HwHiAiUser/.conda/envs/3dinference/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 97, in
from .fx_passes.joint_graph import joint_graph_passes
File "/home/HwHiAiUser/.conda/envs/3dinference/lib/python3.10/site-packages/torch/_inductor/fx_passes/joint_graph.py", line 22, in
from ..pattern_matcher import (
File "/home/HwHiAiUser/.conda/envs/3dinference/lib/python3.10/site-packages/torch/_inductor/pattern_matcher.py", line 95, in
from .lowering import fallback_node_due_to_unsupported_type
File "/home/HwHiAiUser/.conda/envs/3dinference/lib/python3.10/site-packages/torch/_inductor/lowering.py", line 6518, in
import_submodule(kernel)
File "/home/HwHiAiUser/.conda/envs/3dinference/lib/python3.10/site-packages/torch/_dynamo/utils.py", line 2691, in import_submodule
importlib.import_module(f"{mod.name}.{filename[:-3]}")
File "/home/HwHiAiUser/.conda/envs/3dinference/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/home/HwHiAiUser/.conda/envs/3dinference/lib/python3.10/site-packages/torch/_inductor/kernel/mm_grouped.py", line 20, in
from ..utils import (
ImportError: cannot import name 'get_num_sms' from 'torch._inductor.utils' (/home/HwHiAiUser/.conda/envs/3dinference/lib/python3.10/site-packages/torch/_inductor/utils.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/HwHiAiUser/Documents/yolov8_web_inference/vision_npu/setup.py", line 23, in
import torch
File "/home/HwHiAiUser/.conda/envs/3dinference/lib/python3.10/site-packages/torch/init.py", line 2784, in
_import_device_backends()
File "/home/HwHiAiUser/.conda/envs/3dinference/lib/python3.10/site-packages/torch/init.py", line 2760, in _import_device_backends
raise RuntimeError(
RuntimeError: Failed to load the backend extension: torch_npu. You can disable extension auto-loading with TORCH_DEVICE_BACKEND_AUTOLOAD=0.

torchvision的nms可以在安装Torchvision Adapter后在npu上处理
可参考
https://gitcode.com/Ascend/vision

另外torch_npu这边提供了nms相关自定义算子供参考使用
https://www.hiascend.com/document/detail/zh/Pytorch/720/apiref/torchnpuCustomsapi/context/（beta）torch_npu-npu_nms_v4.md

@huangyunlong2022

因为我目前使用的设备是atlas 200i dk a2，他对于torch npu的处理速度非常慢，不知道有没有别的方案能提高他的速度？

import报错应该是版本不匹配，请安装正确配套的torch, torch_npu版本

性能的话，可以考虑安装配套的kernels包，配置torch.npu.set_compile_mode(jit_compile=False)走二进制减少编译耗时
https://www.hiascend.com/document/detail/zh/canncommercial/83RC1/softwareinst/instg/instg_0008.html?Mode=PmIns&InstallType=local&OS=openEuler&Software=cannToolKit
或者参考资料采集profiling进行进一步针对性优化
https://www.hiascend.com/document/detail/zh/Pytorch/720/ptmoddevg/trainingmigrguide/performance_tuning_0014.html

性能的话，可以考虑安装配套的kernels包，配置torch.npu.set_compile_mode(jit_compile=False)走二进制减少编译耗时
https://www.hiascend.com/document/detail/zh/canncommercial/83RC1/softwareinst/instg/instg_0008.html?Mode=PmIns&InstallType=local&OS=openEuler&Software=cannToolKit
或者参考资料采集profiling进行进一步针对性优化
https://www.hiascend.com/document/detail/zh/Pytorch/720/ptmoddevg/trainingmigrguide/performance_tuning_0014.html

@huangyunlong2022

你给的这个性能调优工具，他必须在模型训练中才能使用。
我现在是在模型推理过程，一部分直接用pyacl的接口实现，另一部分用的是numpy python的接口，这个也可以用上述工具进行测试吗？

你说的kernels包我安装了，但是速度并没有提升很快，达不到ms级别。

上述工具是采集torch_npu的，训练推理都可以采集，
其他组件的性能采集需要咨询对应组件，可以在cann性能调优工具中查找是否满足您的场景的
https://www.hiascend.com/document/detail/zh/canncommercial/83RC1/devaids/Profiling/atlasprofiling_16_0001.html