已开启
[Bug]: cannot import name 'is_torchdynamo_compiling' from 'transformers.models.qwen3_vl_moe.modeling_qwen3_vl_moe' #302
tiancew创建于 7月23日
zjl5071
24 天前 评论：
24 天前 评论：
信息来源声明：以下分析基于 transformers（v4.57.3 / v5.5.4 双版本源码对照）与 hyper-parallel 仓库源码逐行核对，关键结论均附文件与行号。
结论：该问题已在 hyper-parallel master 修复（commit 34ab7da9b，2026-08-29），升级 hyper-parallel 即可解决，无需改业务代码。
1. 根因
报错链最后一环是旧版 hyper_parallel/integration/llamafactory/context_parallel/models/qwen3_vl/qwen3vl_forward.py 中的导入：
from transformers.models.qwen3_vl_moe.modeling_qwen3_vl_moe import (..., is_torchdynamo_compiling, ...)
这个符号在 transformers 5.x 已不从 modeling 子模块导出：
- v5.5.4
modeling_qwen3_vl_moe.pyL47 导入列表为TransformersKwargs, auto_docstring, can_return_tuple, torch_compilable_check，不含is_torchdynamo_compiling； - v4.57.3 同文件 L40 仍含该符号（4.57 线可用，版本边界在此）；
- 符号本体一直定义在
transformers/utils/import_utils.py（v5.5.4 为 L1399）。
即 is_torchdynamo_compiling 的规范导入位置是 transformers.utils；旧代码借道 modeling 子模块的 re-export，transformers 5.x 收紧导出后即断。您的 transformers==5.5.4 正好命中。
2. 修复状态
- master 已改为规范写法：
qwen3vl_forward.pyL27from transformers.utils import is_torchdynamo_compiling； - 引入 commit：
34ab7da9b（2026-08-29，"fix: update LlamaFactory integration compatibility"）；修复前版本（e8b6c9c62，06-18）确实还从 modeling 子模块导入； - 其后
79b3cf379（09-11）又做了重构，当前 master 无此问题。
3. 建议
- 升级 hyper-parallel 至包含
34ab7da9b（≥2026-08-29）的版本后，transformers 5.5.4 下该 ImportError 消失； - 若暂时无法升级，可临时手改环境内
qwen3vl_forward.py这一行 import（改为从transformers.utils导入），效果等价； - 若 LlamaFactory 侧是 vendored 的旧集成代码（
src/llamafactory/train/hyper_parallel/），请一并同步到新版本。
昇腾知识图谱接入方式：https://gitcode.com/agent0/kg-tools
Checklist
🐛 Describe the bug
[rank1]: File "/home/user/git/LlamaFactory/src/llamafactory/launcher.py", line 185, in
[rank1]: run_exp()
[rank1]: File "/home/user/git/LlamaFactory/src/llamafactory/train/tuner.py", line 156, in run_exp
[rank1]: _training_function(config={"args": args, "callbacks": callbacks})
[rank1]: File "/home/user/git/LlamaFactory/src/llamafactory/train/tuner.py", line 101, in _training_function
[rank1]: from .hyper_parallel import run_sft as run_sft_hp
[rank1]: File "/home/user/git/LlamaFactory/src/llamafactory/train/hyper_parallel/init.py", line 15, in
[rank1]: from .workflow import run_pt, run_sft
[rank1]: File "/home/user/git/LlamaFactory/src/llamafactory/train/hyper_parallel/workflow.py", line 29, in
[rank1]: from .trainer import HyperParallelTrainer
[rank1]: File "/home/user/git/LlamaFactory/src/llamafactory/train/hyper_parallel/trainer.py", line 25, in
[rank1]: from hyper_parallel.integration.llamafactory import (
[rank1]: File "/usr/local/python3.11.10/lib/python3.11/site-packages/hyper_parallel/integration/llamafactory/init.py", line 40, in
[rank1]: from hyper_parallel.integration.llamafactory.utils import (
[rank1]: File "/usr/local/python3.11.10/lib/python3.11/site-packages/hyper_parallel/integration/llamafactory/utils.py", line 46, in
[rank1]: from hyper_parallel.integration.llamafactory.context_parallel.inputs import (
[rank1]: File "/usr/local/python3.11.10/lib/python3.11/site-packages/hyper_parallel/integration/llamafactory/context_parallel/init.py", line 25, in
[rank1]: from .context_parallel_prepare import cp_prepare_model
[rank1]: File "/usr/local/python3.11.10/lib/python3.11/site-packages/hyper_parallel/integration/llamafactory/context_parallel/context_parallel_prepare.py", line 19, in
[rank1]: from .models import get_context_parallel_model_patches
[rank1]: File "/usr/local/python3.11.10/lib/python3.11/site-packages/hyper_parallel/integration/llamafactory/context_parallel/models/init.py", line 17, in
[rank1]: from .qwen3_vl import QWEN3VL_MOE_CONTEXT_PARALLEL_PATCH
[rank1]: File "/usr/local/python3.11.10/lib/python3.11/site-packages/hyper_parallel/integration/llamafactory/context_parallel/models/qwen3_vl/init.py", line 18, in
[rank1]: from .qwen3vl_forward import (
[rank1]: File "/usr/local/python3.11.10/lib/python3.11/site-packages/hyper_parallel/integration/llamafactory/context_parallel/models/qwen3_vl/qwen3vl_forward.py", line 20, in
[rank1]: from transformers.models.qwen3_vl_moe.modeling_qwen3_vl_moe import (
[rank1]: ImportError: cannot import name 'is_torchdynamo_compiling' from 'transformers.models.qwen3_vl_moe.modeling_qwen3_vl_moe' (/usr/local/python3.11.10/lib/python3.11/site-packages/transformers/models/qwen3_vl_moe/modeling_qwen3_vl_moe.py)
Expected behavior
Triggered when I was trying to run Llamafactory with context parallel. Llama-factory and hyper-parallel are built with the latest main branch. I'm using transformers==5.5.4, which may be an incompatible version.
Additional context
Environment info
Llama-factory and hyper-parallel are built with the latest main branch. I'm using transformers==5.5.4
Thanks for contributing 🎉!