注释掉bug触发代码后会有另一个错误：
/root/miniforge3/envs/florence2/lib/python3.11/site-packages/transformers/generation/utils.py

if "GenerationMixin" in str(cls.prepare_inputs_for_generation) and "GenerationMixin" in str(cls.generate)

File "/usr/local/lib64/python3.11/site-packages/torch/_dynamo/variables/builtin.py", line 1321, in _unimplemented
unimplemented(f"comparison {typestr(left)} {op} {typestr(right)}")
File "/usr/local/lib64/python3.11/site-packages/torch/_dynamo/exc.py", line 172, in unimplemented
raise Unsupported(msg)
torch._dynamo.exc.Unsupported: comparison ConstantVariable(NoneType) TensorVariable()

from user code:
File "/root/.cache/huggingface/modules/transformers_modules/Florence-2-base_msft/modeling_florence2.py", line 2805, in generate
return self.language_model.generate(
File "/usr/local/lib64/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/transformers/generation/utils.py", line 1519, in generate
generation_config, model_kwargs = self._prepare_generation_config(generation_config, kwargs)
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/transformers/generation/utils.py", line 1296, in _prepare_generation_config
generate_attributes_in_kwargs = [
** File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/transformers/generation/utils.py", line 1297, in
key for key, value in kwargs.items() if getattr(generation_config, key, None) != value

报错看着是pytorch原生2.1版本中不支持tranformers 4.41.0 代码中出现的str()方法和getattr(generation_config, key, None) != value的比较，建议在pytorch2.6及以上的版本尝试一下，看是否支持

升级torch和torchnpu版本后这个报错已不再触发，但后续有个新的报错。

torch版本更新结果：

报错信息：

I1210 09:13:55.913000 1543198 site-packages/torch/_dynamo/symbolic_convert.py:2706] [0/0] Step 1: torchdynamo start tracing generate /root/.cache/huggingface/modules/transformers_modules/Florence-2-base_msft/modeling_florence2.py:2788
I1210 09:13:55.915000 1543198 site-packages/torch/fx/experimental/symbolic_shapes.py:3192] [0/0] create_env
Traceback (most recent call last):
File "/home/florence2/migu/run_huawei910B_video_TorchAir_test.py", line 153, in
run_inference(pil_img, TASK_PROMPT)
File "/home/florence2/migu/run_huawei910B_video_TorchAir_test.py", line 91, in run_inference
generated_ids = model_generate(
^^^^^^^^^^^^^^^
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/eval_frame.py", line 574, in _fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/convert_frame.py", line 1380, in call
return self._torchdynamo_orig_callable(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/convert_frame.py", line 547, in call
return _compile(
^^^^^^^^^
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/convert_frame.py", line 986, in _compile
guarded_code = compile_inner(code, one_graph, hooks, transform)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/convert_frame.py", line 715, in compile_inner
return _compile_inner(code, one_graph, hooks, transform)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_utils_internal.py", line 95, in wrapper_function
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/convert_frame.py", line 750, in _compile_inner
out_code = transform_code_object(code, transform)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/bytecode_transformation.py", line 1361, in transform_code_object
transformations(instructions, code_options)
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/convert_frame.py", line 231, in _fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/convert_frame.py", line 662, in transform
tracer.run()
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 2868, in run
super().run()
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1052, in run
while self.step():
^^^^^^^^^^^
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 962, in step
self.dispatch_table[inst.opcode](self, inst)
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 659, in wrapper
return inner_fn(self, inst)
^^^^^^^^^^^^^^^^^^^^
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1736, in CALL_FUNCTION_EX
self.call_function(fn, argsvars.items, kwargsvars)
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 897, in call_function
self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/variables/functions.py", line 378, in call_function
return super().call_function(tx, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/variables/functions.py", line 317, in call_function
return super().call_function(tx, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/variables/functions.py", line 118, in call_function
return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 903, in inline_user_function_return
return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/dynamo/symbolic_convert.py", line 3072, in inline_call
return cls.inline_call(parent, func, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/dynamo/symbolic_convert.py", line 3198, in inline_call
tracer.run()
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 1052, in run
while self.step():
^^^^^^^^^^^
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 962, in step
self.dispatch_table[inst.opcode](self, inst)
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 774, in _missing
unimplemented(f"missing: {opname}")

** File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/_dynamo/exc.py", line 317, in unimplemented
raise Unsupported(msg, case_name=case_name)
torch._dynamo.exc.Unsupported: missing: WITH_EXCEPT_START**

from user code:
File "/root/.cache/huggingface/modules/transformers_modules/Florence-2-base_msft/modeling_florence2.py", line 2805, in generate
return self.language_model.generate(
File "/root/miniforge3/envs/florence2/lib/python3.11/site-packages/torch/utils/_contextlib.py**", line 115, in decorate_context
with ctx_factory()😗*

对应的python代码：

assert not (callable(ctx) and hasattr(ctx, '__enter__')), (
        f"Passed in {ctx} is both callable and also a valid context manager "
        "(has __enter__), making it ambiguous which interface to use.  If you "
        "intended to pass a context manager factory, rewrite your call as "
        "context_decorator(lambda: ctx()); if you intended to pass a context "
        "manager directly, rewrite your call as context_decorator(lambda: ctx)"
    )

    if not callable(ctx):
        def ctx_factory():
            return ctx
    else:
        ctx_factory = ctx

    if inspect.isclass(func):
        raise RuntimeError(
            "Cannot decorate classes; it is ambiguous whether or not only the "
            "constructor or all methods should have the context manager applied; "
            "additionally, decorating a class at definition-site will prevent "
            "use of the identifier as a conventional type.  "
            "To specify which methods to decorate, decorate each of them "
            "individually."
        )

    if inspect.isgeneratorfunction(func):
        return _wrap_generator(ctx_factory, func)

    @functools.wraps(func)
    def decorate_context(*args, **kwargs):
        with ctx_factory():
            return func(*args, **kwargs)

    return decorate_context

model_generate = torch.compile(model.generate, backend='aot_eager', fullgraph=True, dynamic=False) 我看你使用的一直是pytorch原生的dynamo后端的能力，并没有用到torchair，目前看dynamo后端有很多不支持的python语法，请问是在定位精度问题，所以使用aot_eager的能力的吗

我以为是先使用'aot_eager'先验证一下能不能跑通，避免数据类型的cpu数据和npu数据不一致报错。我的场景是想先试一下npu跑图能否成功，优先观察一下性能优化效果。
目前使用torch_npu距离性能距离目标还差一半，查看profiling结果如下：

2b69b9544b2341a8a75d2fb74e577035.png
且模型推理场景为单机单卡，希望通过torchair组图完成调度方面的性能优化。
我直接使用npu_backend也是一样的bug，的确有很多语法不支持，问大模型建议我只完成forward步骤的编译，跳过generate中不支持的Python语法。不知道这个思路是否可以参考？

import torch
import torch._dynamo
from functools import wraps

# ==================== 1. 配置 Dynamo ====================
torch._dynamo.config.allow_rnn = True
torch._dynamo.config.dynamic_shapes = True
torch._dynamo.config.cache_size_limit = 64

# ==================== 2. 定义混合执行装饰器 ====================
def compile_only_forward(model):
    """
    仅编译模型的 forward 方法，generate 方法保持 eager 模式
    """
    # 编译模型的 forward 方法
    model.forward = torch.compile(
        model.forward,
        mode="reduce-overhead",
        dynamic=True,
        options={"triton.cudagraphs": True}
    )
    
    # 保存原始的 generate 方法，避免被编译
    original_generate = model.generate
    
    @wraps(original_generate)
    def wrapper_generate(*args, **kwargs):
        # 临时禁用 Dynamo，让 generate 在 eager 模式下运行
        torch._dynamo.config.disable = True
        try:
            return original_generate(*args, **kwargs)
        finally:
            # 恢复 Dynamo 配置
            torch._dynamo.config.disable = False
    
    # 替换 generate 方法
    model.generate = wrapper_generate
    return model

# ==================== 3. 加载并处理模型 ====================
from transformers import AutoModelForCausalLM, AutoTokenizer

# 加载模型（不编译）
model_path = "/root/.cache/huggingface/modules/transformers_modules/Florence-2-base_msft"
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    trust_remote_code=True,
    torch_dtype=torch.float16,
    device_map="auto"
)
model.eval()

# 仅编译 forward 方法，generate 保持 eager
model = compile_only_forward(model)

# ==================== 4. 推理函数 ====================
def run_inference(pil_img, task_prompt):
    # 数据预处理（eager 模式）
    tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
    inputs = tokenizer(
        text=task_prompt,
        images=pil_img,
        return_tensors="pt",
        padding="max_length",
        max_length=1024
    ).to("cuda")
    
    # generate 方法会在 eager 模式下运行，但内部调用的 forward 是编译后的
    with torch.no_grad():
        generated_ids = model.generate(
            **inputs,
            max_new_tokens=100,
            num_beams=1,
            do_sample=False,
            pad_token_id=tokenizer.pad_token_id,
            eos_token_id=tokenizer.eos_token_id
        )
    
    return tokenizer.decode(generated_ids[0], skip_special_tokens=True)

可以的，把不支持的python语法用eager执行跳过，性能主要优化点应该还是集中在forward里面

参考大模型给出的代码尝试，感觉torchair使能未成功。查看日志发现都是skip
profiling查看还是超级多free：
e6c87e6beb62468ba469b7c8e84fe186.png

Python代码

print("====================================加载Florence-2-base模型 ====================================")
from transformers import AutoProcessor, AutoModelForCausalLM
print("正在加载模型和处理器...")
model = AutoModelForCausalLM.from_pretrained(
    MODEL_PATH,
    torch_dtype=torch_dtype,
    trust_remote_code=True
).to(DEVICE)
processor = AutoProcessor.from_pretrained(MODEL_PATH, trust_remote_code=True)
print("模型加载完成...")
# ==================== 模型编译核心函数 ====================
def compile_model(model):
    """仅编译forward核心方法，generate保持eager"""
    # 保存原始generate并设置为eager模式
    original_generate = model.generate
    @wraps(original_generate)
    def eager_generate(*args, **kwargs):
        torch._dynamo.config.disable = True
        try:
            return original_generate(*args, **kwargs)
        finally:
            torch._dynamo.config.disable = False
    model.generate = eager_generate
    # 使能torchair
    import logging
    import torchair
    from torchair import logger
    logger.setLevel(logging.DEBUG)
    torch._logging.set_logs(dynamo=logging.DEBUG,aot=logging.DEBUG,output_code=True,graph_code=True,recompiles=True)
    logger.info("正在进行TorchAir配置文件初始化...")
    config = torchair.CompilerConfig()
    config.mode = "max-autotune"
    npu_backend = torchair.get_npu_backend(compiler_config=config)
    # 编译forward和视觉编码核心方法
    logger.info("正在进行Florece2模型的forward方法的图编译...")
    model.forward = torch.compile(
        model.forward,
        backend=npu_backend,
        dynamic=False,
        fullgraph=True,
    )
    logger.info("正在进行Florence2的encoder_image图编译...")
    model._encode_image = torch.compile(
        model._encode_image,
        backend=npu_backend,
        dynamic=False,
        fullgraph=True
    )
    logger.info("Florence2-base模型TorchAir组图成功！！！")
    return model

# ==================== 推理核心函数 ========================
@torch.no_grad()
def run_inference(pil_image, prompt):
    """单张图片推理（适配TorchAir）"""
    # 预处理（eager模式）
    # 1. 预处理: 将文本和图像转换为模型输入, 并移动到 NPU
    inputs = processor(text=prompt, images=pil_image, return_tensors="pt").to(DEVICE, torch_dtype)
    # 推理
    generated_ids = model.generate(
        input_ids=inputs["input_ids"],
        pixel_values=inputs["pixel_values"],
        max_new_tokens=1024,
        num_beams=3
    )
    # 3. 解码
    generated_text = processor.batch_decode(generated_ids, skip_special_tokens=False)[0]

    # 4. 后处理
    parsed_answer = processor.post_process_generation(
        generated_text,
        task=prompt,
        image_size=(pil_image.width, pil_image.height)
    )

    return parsed_answer

DEBUG日志：
eade8973b3d44d1ab5e011a335bcfa62.log

pytorch原生不支持，通用做法是通过改模型里面不支持的地方，这里就是改tranformer里面的代码，使能走通

不同PyTorch版本触发的第一个不适配Bug

2.7.1

valueerror

I1210 19:37:39.102000 2166640 site-packages/torch/_dynamo/utils.py:1603] [0/0] ChromiumEventLogger initialized with id b5a848ec-4967-4c85-86b7-7b7661e5767e
I1210 19:37:39.106000 2166640 site-packages/torch/_dynamo/symbolic_convert.py:3322] [0/0] Step 1: torchdynamo start tracing generate /root/.cache/huggingface/modules/transformers_modules/Florence-2-base/modeling_florence2.py:2778
I1210 19:37:39.107000 2166640 site-packages/torch/fx/experimental/symbolic_shapes.py:3334] [0/0] create_env
I1210 19:37:45.022000 2166640 site-packages/torch/_dynamo/convert_frame.py:1121] [0/0] run_gc_after_compile: running gc
Warmup第1次失败: Observed exception
  Explanation: Dynamo found no exception handler at the top-level compiled function when encountering an exception. Exception will propagate outside the compiled region.
  Hint: Dynamo has detected that tracing the code will result in an error when running in eager. Please double check that your code doesn't contain a similar error when actually running eager/uncompiled.
  Hint: It may be possible to write Dynamo tracing rules for this code. Please report an issue to PyTorch if you encounter this graph break often and it is causing performance issues.
  <span style="color:#e60000;">**Developer debug context: raised exception ExceptionVariable(<class 'ValueError'>)**</span>
from user code:
   File "/root/.cache/huggingface/modules/transformers_modules/Florence-2-base/modeling_florence2.py", line 2795, in generate
    return self.language_model.generate(

2.7.0

ERROR: Could not find a version that satisfies the requirement torch-npu==2.7.0 (from versions: 2.1.0.post8, 2.1.0.post10, 2.1.0.post12, 2.1.0.post13, 2.1.0.post17, 2.3.1.post2, 2.3.1.post4, 2.3.1.post6, 2.4.0, 2.4.0.post2, 2.4.0.post4, 2.5.1rc1, 2.5.1, 2.5.1.post1, 2.6.0rc1, 2.6.0, 2.6.0.post3, 2.7.1rc1, 2.7.1, 2.8.0rc1, 2.8.0, 2.9.0rc1)

ERROR: No matching distribution found for torch-npu==2.7.0

2.8.0

报错一致同2.7.1一致

I1210 20:08:31.660000 2199777 site-packages/torch/_dynamo/convert_frame.py:1175] [0/0] run_gc_after_compile: running gc
Warmup第1次失败: Observed exception
Explanation: Dynamo found no exception handler at the top-level compiled function when encountering an exception. Exception will propagate outside the compiled region.
Hint: Dynamo has detected that tracing the code will result in an error when running in eager. Please double check that your code doesn't contain a similar error when actually running eager/uncompiled.
Hint: It may be possible to write Dynamo tracing rules for this code. Please report an issue to PyTorch if you encounter this graph break often and it is causing performance issues.

Developer debug context: raised exception ExceptionVariable(<class 'ValueError'>)


from user code:
 File "/root/.cache/huggingface/modules/transformers_modules/Florence-2-base/modeling_florence2.py", line 2795, in generate
  return self.language_model.generate(

2.9.0

Explanation: Dynamo found no exception handler at the top-level compiled function when encountering an exception. Exception will propagate outside the compiled region.
Hint: Dynamo has detected that tracing the code will result in an error when running in eager. Please double check that your code doesn't contain a similar error when actually running eager/uncompiled.
Hint: It may be possible to write Dynamo tracing rules for this code. Please report an issue to PyTorch if you encounter this graph break often and it is causing performance issues.

Developer debug context: raised exception ValueError([ConstantVariable(str: "torch.compile exception: all generation configuration attributes must be passed within a generation_config instance passed to generate (found: ['inputs_embeds', 'max_new_tokens']).")])
For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0088.html

补充：

不使用torchair，在2.7.1下执行性能最好，想基于这个torch版本修改transformers源码，使能TorchAir适配

感觉2.9.0是最难解决的，只能从中选择一个好解决的bug入手了

针对2.7.1版本下的valueerror尝试的工作

一、generation_config instance

报错信息

  File "/root/miniforge3/envs/base2air/lib/python3.11/site-packages/transformers/generation/utils.py", line 1300, in _prepare_generation_config
    raise ValueError(
ValueError: `torch.compile` exception: all generation configuration attributes must be passed within a `generation_config` instance passed to `generate` (found: ['inputs_embeds', 'max_new_tokens']).

解决方案

定位img_embds传入的位置：

/home/weights/Florence-2-base/modeling_florence2.py里的Florence2ForConditionalGeneration的generate方法。

同时按照提示传递参数。具体修改如下：

def generate(
        self,
        input_ids, 
        inputs_embeds=None,
        pixel_values=None,
        **kwargs
        ):

        if inputs_embeds is None:
            # 1. Extra the input embeddings
            if input_ids is not None:
                inputs_embeds = self.get_input_embeddings()(input_ids)
            # 2. Merge text and images
            if pixel_values is not None:
                image_features = self._encode_image(pixel_values)
                inputs_embeds, attention_mask = self._merge_input_ids_with_image_features(image_features, inputs_embeds)

        ''' Torchair使能代码，将compile的参数换成：self.language_model.generate'''

  
        # 1. 构建 GenerationConfig（核心修复）
        from transformers import GenerationConfig
        generation_config = GenerationConfig(
            inputs_embeds = inputs_embeds,
            max_new_tokens=100,  # 生成最大长度
            num_beams=3         # 束搜索数量
        )
  
  
        # 2. 调用编译后的 generate 方法
        results = compilied_generate(
            input_ids=None,
            generation_config=generation_config,  # 传入配置实例
            **kwargs
        )
        return results

二、transformers库参数检查

报错信息

  File "/root/miniforge3/envs/base2air/lib/python3.11/site-packages/transformers/generation/utils.py", line 446, in _maybe_initialize_input_ids_for_generation
    raise ValueError("`bos_token_id` has to be defined when no `input_ids` are provided.")
ValueError: `bos_token_id` has to be defined when no `input_ids` are provided.
[ERROR] 2025-12-18-20:00:20 (PID:3949784, Device:0, RankID:-1) ERR99999 UNKNOWN applicaiton exception

解决方案

在model的config.json里找到bos_token_id然后在该参数，

"bos_token_id": 0

三、C++头文件未找到

报错信息

/usr/local/Ascend/ascend-toolkit/8.2.RC2/aarch64-linux/tikcpp/tikcfw/impl/kernel_macros.h:24:10: fatal error: 'cstdint' file not found
#include <cstdint>
         ^~~~~~~~~
1 error generated.

], optype: [IsInf])
        Compile op[IsInf] failed, oppath[/usr/local/Ascend/ascend-toolkit/8.2.RC2/opp/built-in/op_impl/ai_core/tbe/impl/dynamic/is_inf.py], optype[IsInf], taskID[56]. Please check op's compilation error message.[FUNC:ReportBuildErrMessage][FILE:fusion_manager.cc][LINE:368]
        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281402666643872] recompile single op[LayerNormV4_1_LayerNormV3/AddLayerNorm] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:1135]
        [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281402666643872] recompile single op[IsInf] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:1135]
        [SubGraphOpt][Compile][ParalCompOp] Thread[281402666643872] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:1182]
        Call OptimizeFusedGraph failed, ret:4294967295, engine_name:AIcoreEngine, graph_name:partition1_rank32_new_sub_graph60[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:119]
        subgraph 23 optimize failed[FUNC:OptimizeSubGraphWithMultiThreads][FILE:graph_manager.cc][LINE:893]
        [Call][PreRun] Failed, graph_id:4, session_id:0.[FUNC:CompileGraph][FILE:graph_manager.cc][LINE:4654]
        [Compile][Graph]Compile graph failed, error code:1343225857, session_id:0, graph_id:4, isEnableSliceSchedule:0.[FUNC:CompileGraph][FILE:ge_api.cc][LINE:1365]

[ERROR] 2025-12-18-20:05:55 (PID:3953942, Device:0, RankID:-1) ERR03005 GRAPH internal error

解决方案

安装gcc、g++工具，并通过环境变量完成配置；

export CPLUS\_INCLUDE\_PATH=/usr/include/c++/12/aarch64-openEuler-linux:/usr/include/c++/12:\$CPLUS\_INCLUDE\_PATH

四、torch.ops.aten.isnan.default ge_converter is not implemented!

报错信息

File "/root/miniforge3/envs/base2air/lib/python3.11/site-packages/torch_npu/dynamo/torchair/_ge_concrete_graph/fx2ge_converter.py", line 272, in wrapped_converter
    ge_outputs = converter(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniforge3/envs/base2air/lib/python3.11/site-packages/torch_npu/dynamo/torchair/_ge_concrete_graph/ge_converter/aten/isnan.py", line 7, in conveter_aten_isnan_default
    raise NotImplementedError("torch.ops.aten.isnan.default ge_converter is not implemented!")
NotImplementedError: torch.ops.aten.isnan.default ge_converter is not implemented!

While executing %isnan : [num_users=1] = call_function[target=torch.ops.aten.isnan.default](args = (%arg0_1,), kwargs = {})
GraphModule: class GraphModule(torch.nn.Module):
    def forward(self, arg0_1: "f16[1, 1, 768][768, 768, 1]"):
         # File: /root/.cache/huggingface/modules/transformers_modules/Florence-2-base/modeling_florence2.py:1281 in torch_dynamo_resume_in_forward_at_1280, code: torch.isinf(hidden_states).any() or torch.isnan(hidden_states).any()
        isnan: "b8[1, 1, 768][768, 768, 1]" = torch.ops.aten.isnan.default(arg0_1);  arg0_1 = None
        any_1: "b8[][]" = torch.ops.aten.any.default(isnan);  isnan = None
        return (any_1,)
  

Original traceback:
  File "/root/.cache/huggingface/modules/transformers_modules/Florence-2-base/modeling_florence2.py", line 1281, in torch_dynamo_resume_in_forward_at_1280
    torch.isinf(hidden_states).any() or torch.isnan(hidden_states).any()



[ERROR] 2025-12-18-20:08:21 (PID:3959014, Device:0, RankID:-1) ERR03007 GRAPH feature not supported

解决方案

步骤1：在torch.compile之前将自定义算子注册入图：

        from torchair._ge_concrete_graph.fx2ge_converter import register_fx_node_ge_converter
        from torchair._ge_concrete_graph import ge_apis as ge
        from torchair.ge._ge_graph import Tensor, TensorSpec
        @register_fx_node_ge_converter(torch.ops.aten.isnan.default)
        def conveter_aten_isnan_Tensor(
                self: Tensor,
                *,
                meta_outputs: Union[TensorSpec, List[TensorSpec]] = None):
            return ge.IsNan(self)

步骤2：源码编译安装torchair

五、torch._dynamo不支持>=

报错信息

Traceback (most recent call last):
  File "/root/miniforge3/envs/base2air/lib/python3.11/site-packages/torch_npu/dynamo/torchair/_utils/error_code.py", line 43, in wapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniforge3/envs/base2air/lib/python3.11/site-packages/torch_npu/dynamo/torchair/npu_fx_compiler.py", line 393, in __call__
    return self._get_compiled_gm(gm, example_inputs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniforge3/envs/base2air/lib/python3.11/site-packages/torch_npu/dynamo/torchair/npu_fx_compiler.py", line 446, in _get_compiled_gm
    return _GmRunner(self._gen_compiled_gm(gm, example_inputs))
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniforge3/envs/base2air/lib/python3.11/site-packages/torch_npu/dynamo/torchair/npu_fx_compiler.py", line 474, in _gen_compiled_gm
    concrete_graph: ConcreteGraphBase = _NpuGraphConverter(
                                        ^^^^^^^^^^^^^^^^^^^
  File "/root/miniforge3/envs/base2air/lib/python3.11/site-packages/torch_npu/dynamo/torchair/npu_fx_compiler.py", line 138, in run
    super().run(*args, **kwargs)
  File "/root/miniforge3/envs/base2air/lib/python3.11/site-packages/torch/fx/interpreter.py", line 171, in run
    self.env[node] = self.run_node(node)
                     ^^^^^^^^^^^^^^^^^^^
  File "/root/miniforge3/envs/base2air/lib/python3.11/site-packages/torch_npu/dynamo/torchair/npu_fx_compiler.py", line 121, in run_node
    return super().run_node(n)
           ^^^^^^^^^^^^^^^^^^^
  File "/root/miniforge3/envs/base2air/lib/python3.11/site-packages/torch/fx/interpreter.py", line 240, in run_node
    return getattr(self, n.op)(n.target, args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniforge3/envs/base2air/lib/python3.11/site-packages/torch_npu/dynamo/torchair/npu_fx_compiler.py", line 91, in inner
    result = f(self, target, args, kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniforge3/envs/base2air/lib/python3.11/site-packages/torch_npu/dynamo/torchair/npu_fx_compiler.py", line 211, in call_function
    return self._wrap('call_function')(target, args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniforge3/envs/base2air/lib/python3.11/site-packages/torch_npu/dynamo/torchair/npu_fx_compiler.py", line 170, in inner
    return func(target, args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniforge3/envs/base2air/lib/python3.11/site-packages/torch/fx/interpreter.py", line 320, in call_function
    return target(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '>=' not supported between instances of 'ValuePack' and 'int'

While executing %ge : [num_users=1] = call_function[target=operator.ge](args = (%arg0_1, 101), kwargs = {})
GraphModule: class GraphModule(torch.nn.Module):
    def forward(self, arg0_1: "Sym(s0)"):
         # File: /root/miniforge3/envs/base2air/lib/python3.11/site-packages/transformers/generation/stopping_criteria.py:505 in __call__, code: is_done = torch.full((input_ids.shape[0],), False, device=input_ids.device)
        full: "b8[3][1]" = torch.ops.aten.full.default([3], False, device = device(type='npu', index=0), pin_memory = False)
  
         # File: /root/miniforge3/envs/base2air/lib/python3.11/site-packages/transformers/generation/stopping_criteria.py:76 in __call__, code: is_done = cur_len >= self.max_length
        ge: "Sym(s0 >= 101)" = arg0_1 >= 101;  arg0_1 = None
  
         # File: /root/miniforge3/envs/base2air/lib/python3.11/site-packages/transformers/generation/stopping_criteria.py:83 in __call__, code: return torch.full((input_ids.shape[0],), is_done, device=input_ids.device, dtype=torch.bool)
        full_1: "b8[3][1]" = torch.ops.aten.full.default([3], ge, dtype = torch.bool, device = device(type='npu', index=0), pin_memory = False);  ge = None
  
         # File: /root/miniforge3/envs/base2air/lib/python3.11/site-packages/transformers/generation/stopping_criteria.py:507 in __call__, code: is_done = is_done | criteria(input_ids, scores, **kwargs)
        bitwise_or: "b8[3][1]" = torch.ops.aten.bitwise_or.Tensor(full, full_1);  full = full_1 = None
        return (bitwise_or,)
  

Original traceback:
  File "/root/miniforge3/envs/base2air/lib/python3.11/site-packages/transformers/generation/stopping_criteria.py", line 507, in __call__
    is_done = is_done | criteria(input_ids, scores, **kwargs)
  File "/root/miniforge3/envs/base2air/lib/python3.11/site-packages/transformers/generation/stopping_criteria.py", line 76, in __call__
    is_done = cur_len >= self.max_length

解决方案

方案1：尝试强制类型转换，将valuepack转换成int支持比较

未能解决问题，报错继续

TypeError: '>=' not supported between instances of 'ValuePack' and 'int'


Original traceback:
File "/root/miniforge3/envs/base2air/lib/python3.11/site-packages/transformers/generation/stopping_criteria.py", line 508, in __call__
is_done = is_done | criteria(input_ids, scores, **kwargs)
File "/root/miniforge3/envs/base2air/lib/python3.11/site-packages/transformers/generation/stopping_criteria.py", line 77, in __call__
is_done = int(cur_len) >= self.max_length

方案2：缩小图编译范围，避免触发不支持的Python语法；

六、定位合适的TorchAir组图位置

原则：

• 避开host侧有多种可能情况的if判断
• 避开Embedding操作，不同的张量形状导致多次编图，重编译达到上限
• 避开异步流操作，导致断图；torch._dynamo原生不支持的Python语法：with语句、set等

建议：

走读代码：
1、缩小组图范围，规避不支持的Python操作；

2、深入理解模型，修改代码，使能适配；

由于该模型使用transformers架构，若选择建议2会修改transformers库源码，故选择建议1。

1、定位组图范围

取消在Florence2ForConditionalGeneration类中的generate方法中的组图，寻找模型定义代码中的Layer层定义和调用的位置；

Florence2EncoderLayer(nn.Module)以及Florence2DecoderLayer(nn.Module)，然后找到它们调用的位置如下：

2、编译Encoder、Decoder

组图执行时发现：推理结果为空，针对打屏的提示信息进行分析。具体如下：

问题根源分析

这个警告的核心原因是模型权重的键名不匹配：

• checkpoint 中权重键是 language_model.model.encoder.layers.xxx（无 _orig_mod 前缀）；
• 初始化 Florence2ForConditionalGeneration 时，模型结构中这些层被封装在了 _orig_mod 模块下（权重键多了 _orig_mod 前缀）；
• 最终导致原有权重未被加载，模型初始化了全新的随机权重（推理/训练都会受严重影响）。

_orig_mod 前缀通常是以下场景导致的：

1. 模型推理时使用了 torch.compile() 编译优化

解决方案

在加载模型前重命名权重键

# 1. 加载预训练权重（原始checkpoint）
ckpt_path = "/home/weights/Florence-2-base"
state_dict = torch.load(f"{ckpt_path}/pytorch_model.bin", map_location="cpu")

# 2. 重映射参数名：把预训练权重的key添加_orig_mod，匹配模型期望的参数名
# 初始化修正后的权重字典
fixed_state_dict = {}

# 遍历所有权重键，批量修正encoder/decoder的_orig_mod前缀
for old_key, weight in state_dict.items():
    # 修正encoder层：language_model.model.encoder.xxx → language_model.model.encoder._orig_mod.xxx
    if old_key.startswith("language_model.model.encoder."):
        new_key = old_key.replace(
            "language_model.model.encoder.",
            "language_model.model.encoder._orig_mod.",
            1  # 只替换第一个匹配项，避免多层嵌套错误
        )
        fixed_state_dict[new_key] = weight
  
    # 修正decoder层：language_model.model.decoder.xxx → language_model.model.decoder._orig_mod.xxx
    elif old_key.startswith("language_model.model.decoder."):
        new_key = old_key.replace(
            "language_model.model.decoder.",
            "language_model.model.decoder._orig_mod.",
            1
        )
        fixed_state_dict[new_key] = weight
  
    # 其他层（如视觉编码器、head等）保持原键名不变
    else:
        fixed_state_dict[old_key] = weight
print("正在加载模型和处理器...")

推理成功；

七、结论

1、推理性能出现劣化

ACL-Graph场景下组图成功进行推理，但优化点在计算上。此次尝试，针对Florence2-base模型的单算子调度问题无改善，因此在整体的推理性能上无优化效果。

亲爱的开发者朋友，欢迎添加TorchAir小助手，申请加入TorchAir的技术交流群哟~

操作过于频繁，可能得等一会儿才能加到

再试一下~，可以加了哈

还是不太行，微信扫码一直是操作过于频繁，请稍后再试