[Bug-Report|缺陷反馈]: v1版本的vllm 推理开启图模式后，无法使用msprobe进行性能采集-mstt-AtomGit

已关闭

[Bug-Report|缺陷反馈]: v1版本的vllm 推理开启图模式后，无法使用msprobe进行性能采集 #129

jingshenghang创建于 2025年10月28日关闭于 2025年10月31日

jingshenghang

2025年10月28日创建

Thanks for sending an issue! Please fill in the following template to help quickly solve your problem.

Describe the current behavior / 问题描述 (Mandatory / 必填)

根据文档，在vllm v1版本，使用msorobe测试性能，发现两个问题：

在vllm 0.11.0 中设置debugger = PrecisionDebugger之后，vllm中的LLM加载模型中，开启图模式会报错

(EngineCore_DP0 pid=109131) Process EngineCore_DP0:
(EngineCore_DP0 pid=109131) Traceback (most recent call last):
(EngineCore_DP0 pid=109131)   File "/usr/local/python3.11.13/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
(EngineCore_DP0 pid=109131)     self.run()
(EngineCore_DP0 pid=109131)   File "/usr/local/python3.11.13/lib/python3.11/multiprocessing/process.py", line 108, in run
(EngineCore_DP0 pid=109131)     self._target(*self._args, **self._kwargs)
(EngineCore_DP0 pid=109131)   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 712, in run_engine_core
(EngineCore_DP0 pid=109131)     raise e
(EngineCore_DP0 pid=109131)   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 699, in run_engine_core
(EngineCore_DP0 pid=109131)     engine_core = EngineCoreProc(*args, **kwargs)
(EngineCore_DP0 pid=109131)                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=109131)   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 498, in __init__
(EngineCore_DP0 pid=109131)     super().__init__(vllm_config, executor_class, log_stats,
(EngineCore_DP0 pid=109131)   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 92, in __init__
(EngineCore_DP0 pid=109131)     self._initialize_kv_caches(vllm_config)
(EngineCore_DP0 pid=109131)   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 190, in _initialize_kv_caches
(EngineCore_DP0 pid=109131)     self.model_executor.determine_available_memory())
(EngineCore_DP0 pid=109131)     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=109131)   File "/vllm-workspace/vllm/vllm/v1/executor/abstract.py", line 85, in determine_available_memory
(EngineCore_DP0 pid=109131)     return self.collective_rpc("determine_available_memory")
(EngineCore_DP0 pid=109131)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=109131)   File "/vllm-workspace/vllm/vllm/executor/uniproc_executor.py", line 83, in collective_rpc
(EngineCore_DP0 pid=109131)     return [run_method(self.driver_worker, method, args, kwargs)]
(EngineCore_DP0 pid=109131)             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=109131)   File "/vllm-workspace/vllm/vllm/utils/__init__.py", line 3122, in run_method
(EngineCore_DP0 pid=109131)     return func(*args, **kwargs)
(EngineCore_DP0 pid=109131)            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=109131)   File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/worker_v1.py", line 221, in determine_available_memory
(EngineCore_DP0 pid=109131)     self.model_runner.profile_run()
(EngineCore_DP0 pid=109131)   File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/model_runner_v1.py", line 2535, in profile_run
(EngineCore_DP0 pid=109131)     hidden_states = self._dummy_run(self.max_num_tokens,
(EngineCore_DP0 pid=109131)                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=109131)   File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
(EngineCore_DP0 pid=109131)     return func(*args, **kwargs)
(EngineCore_DP0 pid=109131)            ^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=109131)   File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/model_runner_v1.py", line 2499, in _dummy_run
(EngineCore_DP0 pid=109131)     hidden_states = self._generate_dummy_run_hidden_states(
(EngineCore_DP0 pid=109131)                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=109131)   File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/model_runner_v1.py", line 2320, in _generate_dummy_run_hidden_states
(EngineCore_DP0 pid=109131)     hidden_states = self.model(input_ids=input_ids,
(EngineCore_DP0 pid=109131)                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=109131)   File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
(EngineCore_DP0 pid=109131)     return self._call_impl(*args, **kwargs)
(EngineCore_DP0 pid=109131)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=109131)   File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
(EngineCore_DP0 pid=109131)     return forward_call(*args, **kwargs)
(EngineCore_DP0 pid=109131)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=109131)   File "/vllm-workspace/vllm/vllm/model_executor/models/qwen3.py", line 323, in forward
(EngineCore_DP0 pid=109131)     hidden_states = self.model(input_ids, positions, intermediate_tensors,
(EngineCore_DP0 pid=109131)                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=109131)   File "/vllm-workspace/vllm/vllm/compilation/decorators.py", line 310, in __call__
(EngineCore_DP0 pid=109131)     output = self.compiled_callable(*args, **kwargs)
(EngineCore_DP0 pid=109131)              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=109131)   File "/usr/local/python3.11.13/lib/python3.11/site-packages/torch/_dynamo/eval_frame.py", line 655, in _fn
(EngineCore_DP0 pid=109131)     return fn(*args, **kwargs)
(EngineCore_DP0 pid=109131)            ^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=109131) AttributeError: 'NoneType' object has no attribute 'frame_exec_strategy'

由于vllm v1版本接口修改，文档中采集推理的数据的函数无法执行

文档：debug/accuracy_tools/msprobe/docs/05.data_dump_PyTorch.md
章节：2.5 推理模型采集指定token_range

model = llm.llm_engine.model_executor.driver_worker.worker.model_runner.get_model()

Environment / 环境信息 (Mandatory / 必填)

A2 910B

Steps to reproduce the issue / 重现步骤 (Mandatory / 必填)

测试代码

from vllm import LLM, SamplingParams
from msprobe.pytorch import PrecisionDebugger, seed_all

seed_all()
debugger = PrecisionDebugger(task='tensor', dump_path="/home/ascend-vllm/msprobe")

# 模型定义及初始化等操作
prompts = ["Hello, my name is"]
sampling_params = SamplingParams(temperature=0.8, top_p=0.95)

# llm = LLM(model='/home/ascend-vllm/model/Qwen3-8B', enforce_eager=True) # Error, has to ensure enforce_eager=True
llm = LLM(model='/home/ascend-vllm/model/Qwen3-8B', enforce_eager=True) # OK

# model = llm.llm_engine.model_executor.driver_worker.worker.model_runner.get_model() # Error 

debugger.start()

output = llm.generate(prompts, sampling_params=sampling_params)

debugger.stop()
debugger.step()