1. 背景

from_pretrained原子操作：

1. init model skeleton
2. parallize & 其他处理
3. 切分、mapping权重并加载

3依赖2的切分等处理，因此需要2输出 切分等（不需要输出sharding信息，这部分由param自己的placements, device_mesh属性来记录） 操作的信息，并规范其格式

2. 格式

from transformers.core_model_loading import (
    WeightConverter,
    WeightRenaming,
    convert_and_load_state_dict_in_model,
    revert_weight_conversion,
)

WeightMapping: list[WeightConverter | WeightRenaming] | None = None

3. 流程

  Stage 1: init_model()
    → canonical、未并行化 model skeleton
    → 已执行 model.tie_weights()

  Stage 1.5: init_build_ir()
    → checkpoint_mapping: checkpoint schema → canonical skeleton
    → model_mapping: canonical skeleton → current model（初始 identity）
    → tied alias groups

  Stage 2: model transform + parallelize
    → 累积 model WeightMappingDelta
    → 累积 ParamLayoutDelta: 挂载到param上的，由fsdp2manager管理，外界无需关注
    → 更新最终 Parameter binding/local chunk

  Stage 3: compile + load
    → compose(checkpoint_mapping, model_mapping)
    → combine with final Param Layout（param._placements, param._device_mesh）
    → 生成 rank-local LoadTargets
    → 读取 checkpoint 并写入 final model

3.1. 伪代码

def from_pretrained(...):
    # 1. init model skeleton
    with device_meta_init_ctx:
        model = _init_model(...)
    
    # 1.5. init ModelConversionMetadata
    # - WeightMapping: from transformers
    # - NO DTensor Metadata: as param attr managed by FSDP2Manager
    weights_mapping = init_weights_mapping(model, ...)

    # 2. parallize & ...
    # 2.1. perf module
    perf_model, weights_mapping = apply_perf_model(model, weights_mapping)
    # 2.1. sharding plan
    sharded_model = apply_sharding_plan(model, ...)
    # 2.2. fsdp
    final_model = fsdp2_manager.parallelize(sharded_model, ...)

    # 3. load checkpoint
    # Params of final_model have attr:
    # - placements
    # - device_mesh
    # for checkpoint sharding & loading
    load_checkpoint(final_model, ckpt_file, weights_mapping)