文件最后提交记录最后更新时间
【msmodelslim】刷新文件头部声明和版权 Co-authored-by: caishengcheng<caishengcheng@huawei.com> # message auto-generated for no-merge-commit merge: !28 merge license into master 【msmodelslim】刷新文件头部声明和版权 Created-by: caishengcheng Commit-by: caishengcheng Merged-by: ascend-robot Description: 【msmodelslim】刷新文件头部声明和版权 See merge request: Ascend/msmodelslim!284 个月前
【msmodelslim】刷新文件头部声明和版权 Co-authored-by: caishengcheng<caishengcheng@huawei.com> # message auto-generated for no-merge-commit merge: !28 merge license into master 【msmodelslim】刷新文件头部声明和版权 Created-by: caishengcheng Commit-by: caishengcheng Merged-by: ascend-robot Description: 【msmodelslim】刷新文件头部声明和版权 See merge request: Ascend/msmodelslim!284 个月前
feat(dts): add DFX capabilities for DTS runtime Co-authored-by: rookie_hongchuan<hongchuan6@h-partners.com> # message auto-generated for no-merge-commit merge: !396 merge feature/dts-collective-detection into master feat(dts): add DFX capabilities for DTS runtime Created-by: rookie_hongchuan Commit-by: rookie_hongchuan Merged-by: ascend-robot Description: ## Summary 本次 DTS DFX 能力拓展包含两项独立但互补的改进: ### 1. 集合通信检测 新增 _collective_op_guard() 上下文管理器,在 DTS 并行波次共享任务执行期间检测非法的跨 rank 集合通信操作。 该 guard **仅**作用于并行波次调度器,顺序波次不拦截——因为全体 rank 都参与时集合通信不会挂死。 检测到非法操作时抛出 SchemaValidateError,错误信息包含精确的调用位置(文件、行号、函数名)。 ### 2. 日志可审计 将多卡 DTS 运行时日志从多条 INFO 合并为一条 summary,包含完整性能信息: - tasks — 总任务数 - my_tasks — 本 rank 实际执行任务数 - exec_s / my_exec_s — 全部 / 本 rank 执行耗时 - sync_s / my_sync_s — 全部 / 本 rank 同步耗时 - t_run_wall_s — 墙钟耗时 - speedup — 加速比 = sum_exec / t_run_wall(修复了之前公式相反的错误) - queue — 共享策略(shared / static_rr) 同步耗时非零时,DEBUG 级别保留 exec_over_sync 比值和 not suitable for parallel 提示。 ## 典型场景 ### 集合通信检测 处理器提交的共享任务函数中意外使用了 torch.distributed 集合通信: ```python def my_calib_step(module): gathered = [torch.zeros_like(act) for _ in range(world_size)] dist.all_gather(gathered, act) return torch.stack(gathered).mean() ``` 报错信息: ``` Code: 203, Message: DTS shared task contains illegal cross-rank collective: torch.distributed.all_gather(). TIP: DTS shared tasks must be independently executable by any single rank; ... ``` ### DTS summary 日志 多卡并行(4 tasks, 2 rank, 共享队列): ``` [DTS] Summary: tasks=4 world_size=2 rank=0 my_tasks=2 exec_s=0.2403 ... speedup=1.86 queue=shared ``` ## 测试覆盖 - [x] 非分布式下 guard 不做拦截(4 用例) - [x] 分布式下 broadcast / all_reduce / barrier / all_gather 等 10 种集合通信各自抛出 SchemaValidateError(2 rank gloo,10 用例) - [x] 集成测试:并行调度器合法任务正常执行(2 rank) - [x] 结构化验证:并行调度器 _execute_local_task 包含 guard,顺序调度器不包含 - [x] DTS summary 日志格式契约测试(多卡并行 3 场景 + 单卡串行) - [x] 加速比公式正确性验证(total_exec / t_run_wall) - [x] 真实双进程 gloo 多 rank 日志输出验证 See merge request: Ascend/msmodelslim!3968 天前
【msmodelslim】刷新文件头部声明和版权 Co-authored-by: caishengcheng<caishengcheng@huawei.com> # message auto-generated for no-merge-commit merge: !28 merge license into master 【msmodelslim】刷新文件头部声明和版权 Created-by: caishengcheng Commit-by: caishengcheng Merged-by: ascend-robot Description: 【msmodelslim】刷新文件头部声明和版权 See merge request: Ascend/msmodelslim!284 个月前
【refactor】插件机制优化并应用于IQuantService Co-authored-by: joejoezhou<zhourongchen1@huawei.com> # message auto-generated for no-merge-commit merge: !105 merge refactor/plugin-infra into master 【refactor】插件机制优化并应用于IQuantService Created-by: joejoezhou Commit-by: joejoezhou Merged-by: ascend-robot Description: 1、插件机制重构,提高易用性,以函数注册<配置类,组件类>对,并允许通过config.ini注册 2、quant service使用插件机制 3、增加超时机制,插件加载超时报错 See merge request: Ascend/msmodelslim!1053 个月前
【UT】补充model/flux1和util/security单元测试 Co-authored-by: xyxin_006<xyxin_hit@163.com> # message auto-generated for no-merge-commit merge: !499 merge ut/add_ut into master 【UT】补充model/flux1和util/security单元测试 Created-by: xyxin_006 Commit-by: xyxin_006 Merged-by: ascend-robot Description: 感谢您贡献的Pull Request! 在提交之前,请务必阅读 [CONTRIBUTING.md](https://gitcode.com/Ascend/msmodelslim/blob/master/CONTRIBUTING.md)。 Thanks for sending a pull request! BEFORE SUBMITTING, PLEASE READ [CONTRIBUTING.md](https://gitcode.com/Ascend/msmodelslim/blob/master/CONTRIBUTING.md). ## PR描述 (What this PR does / why we need it?) - 请明确说明您提交PR的变更内容。本部分旨在概述所做的变更,以及此PR是如何解决该问题的。请尽可能地提供有助于评审人员更高效、更快速完成检视审查的实用说明。 - 请说明为何需要这些更改,例如具体的使用场景或bug描述。 - 关联issue号(如果有)。 - Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue. If possible, please consider writing useful notes for better and faster reviews in your PR. - Please clarify why the changes are needed. For instance, the use case and bug description. - Related issue number (if any) ## 面向用户的变更 (Does this PR introduce _any_ user-facing change)? - 请注意,这里指的是**任何**面向用户的变更,包括但不限于API、用户界面或其他使用方式上的变更。 - Note that it means *any* user-facing change including all aspects such as API, interface or other behavior changes. ## 功能验证 (How was this patch tested?) 请确认CI已通过增量及存量的单元测试用例。 如果本次测试方式与常规单元测试不同,请详细说明您的测试步骤(最好提供完整的可复现的操作路径及关键截图),以便Committer能够快速复现验证,也便于后续的维护。 如果未添加测试,请说明未添加的原因,以及为何难添加测试。 - [_] 功能自验 - [_] 本地自验截图(涉及个人标识符等敏感信息请注意脱敏) - [_] 新增/变更内容是否已新增/适配UT测试用例看护 CI passed with new added/existing test. If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future. If tests were not added, please describe why they were not added and/or why it was difficult to add. - [_] Self-verification of the feature. - [_] Screenshot of local self-verification (please anonymize any sensitive information such as personal identifiers) - [_] Have new or modified unit test (UT) cases been added or adapted to cover the newly added or changed content? See merge request: Ascend/msmodelslim!4995 小时前
[feature] 量化配置yaml校验 Co-authored-by: tanxiangyuu<tanxiangyu2@huawei.com> # message auto-generated for no-merge-commit merge: !380 merge dev into master [feature] 量化配置yaml校验 Created-by: tanxiangyuu Commit-by: tanxiangyuu Merged-by: ascend-robot Description: ## PR描述 (What this PR does / why we need it?) ### 背景 msmodelslim 的 YAML 量化配置解析链中,Metadata 模型缺少 extra='forbid' 设置,默认 extra='ignore' 导致字段名拼写错误(如 scroe→score、labll→label)、缩进错误(如 include/exclude 误入 metadata 层级)等低级失误被静默丢弃,用户无任何报错提示,量化流程以错误的默认参数继续执行。 此外,PyYAML 对重复键静默用后值覆盖前值,项目缺少相关检测。 ### 变更内容 1. Metadata extra='forbid' - msmodelslim/core/practice/interface.py:Metadata 添加字段验证器(validate_str_length、in_range),修复 verified_model_types 使用 dataclasses.field → pydantic.Field 的问题 - msmodelslim/utils/patch/pydantic.py:patch 后的 PatchedBaseModel 默认 extra='forbid',所有子类自动继承;需要允许额外字段的模型(如 BaseQuantConfig)已显式设置 extra='allow',不受影响 效果:Metadata 层级的拼写错误、缩进错误在配置加载阶段立即报错,不再静默丢弃。 2. YAML 重复键检测 - msmodelslim/utils/security/path.py:新增 _DuplicateKeyLoader(继承 yaml.SafeLoader),在 yaml_safe_load 中检测重复键,发现时抛出 SchemaValidateError 并提示行号 效果:config_id: a / config_id: b 重复键不再静默覆盖,立即报错。 3. Pydantic patch 优化 - msmodelslim/utils/patch/pydantic.py: - 错误信息包含完整路径(如 process.0.type 而非 value) - 一次性报出metadata或spec的所有校验错误,不截断 - __init__ 通过栈帧检测区分嵌套调用和直接调用,嵌套时不破坏 Pydantic 的错误收集 - model_config 默认 extra='forbid',所有新建模型自动拒绝未知字段 4. 校验工具函数 - msmodelslim/utils/validation/value.py:新增 allow_empty_list、allow_empty_dict、at_least_one_key、in_range 等通用校验函数 - msmodelslim/utils/validation/pydantic.py:新增对应的 Pydantic AfterValidator 包装 5. 代码清理 - 移除各 processor 文件中冗余的 model_config = ConfigDict(extra="forbid")(由 patch 默认提供) - 统一代码格式(缩进、引号、行尾空格) - 修复 greater_than_zero 支持 Decimal 类型 - 移除未使用的 is_boolean 校验函数 ### 面向用户的变更 | 场景 | 修复前 | 修复后 | |------|--------|--------| | Metadata 字段拼写错误(scroescore) | 静默丢弃,使用默认值 | 立即报错,提示完整路径 | | include/exclude 缩进到 metadata 下 | 静默丢弃,所有模块被量化 | 立即报错 | | YAML 重复键 | 静默覆盖 | 立即报错,提示行号 | | 任意 Pydantic 模型的未知字段 | 默认忽略 | 默认拒绝(extra='forbid') | | 多个校验错误同时存在 | 只报第一个 | 一次性全部报出 | ### 错误信息示例 #### 示例1 ![image.png](https://raw.gitcode.com/user-images/assets/8444818/d50dbb2a-bbcd-4cc6-95fb-c04d558c4d05/image.png 'image.png') ![image.png](https://raw.gitcode.com/user-images/assets/8444818/5b848056-63ad-423a-bce4-16188d5a587b/image.png 'image.png') #### 示例2 ![image.png](https://raw.gitcode.com/user-images/assets/8444818/8beba509-b986-4468-b018-d3183ad5a85f/image.png 'image.png') ![image.png](https://raw.gitcode.com/user-images/assets/8444818/e31bc2eb-9a21-46fc-b7b4-fd9835436410/image.png 'image.png') #### 示例3 ![image.png](https://raw.gitcode.com/user-images/assets/8444818/e51ea28e-c5ff-4df9-b199-b5ffa6f2d1c1/image.png 'image.png') ![image.png](https://raw.gitcode.com/user-images/assets/8444818/1bd0f623-1a60-4082-85be-36f01627a050/image.png 'image.png') ### 结构修改 | 文件 | 修改说明 | |------|---------| | 文件 | 修改说明 | | :--- | :--- | | msmodelslim/utils/patch/pydantic.py | 新增 _format_validation_error_in_model_validate;默认 extra='forbid'model_validate 合并 _additional_validation_errors | | msmodelslim/utils/validation/validation_state.py | 新增:管理校验上下文(路径前缀)和额外错误存储 | | msmodelslim/processor/base.py | validate_auto_processor_config_list 不再 raise,非 dict 项错误写入 validation_state;设置 spec. 路径前缀 | | msmodelslim/utils/security/path.py | 新增 _DuplicateKeyLoaderyaml_safe_load 使用自定义 Loader;添加 encoding='utf-8' | | msmodelslim/core/practice/interface.py | Metadata 添加验证器;verified_model_types 改用 pydantic.Field | | msmodelslim/utils/validation/value.py | 新增 allow_empty_listallow_empty_dictat_least_one_keyin_range | | msmodelslim/utils/validation/pydantic.py | 新增 _to_value_error 包装,Pydantic 能收集多个自定义验证器的错误 | | msmodelslim/processor/*.py | 移除冗余 model_config;统一代码格式 | See merge request: Ascend/msmodelslim!3802 天前
【msmodelslim】刷新文件头部声明和版权 Co-authored-by: caishengcheng<caishengcheng@huawei.com> # message auto-generated for no-merge-commit merge: !28 merge license into master 【msmodelslim】刷新文件头部声明和版权 Created-by: caishengcheng Commit-by: caishengcheng Merged-by: ascend-robot Description: 【msmodelslim】刷新文件头部声明和版权 See merge request: Ascend/msmodelslim!284 个月前
【msmodelslim】刷新文件头部声明和版权 Co-authored-by: caishengcheng<caishengcheng@huawei.com> # message auto-generated for no-merge-commit merge: !28 merge license into master 【msmodelslim】刷新文件头部声明和版权 Created-by: caishengcheng Commit-by: caishengcheng Merged-by: ascend-robot Description: 【msmodelslim】刷新文件头部声明和版权 See merge request: Ascend/msmodelslim!284 个月前
【msmodelslim】刷新文件头部声明和版权 Co-authored-by: caishengcheng<caishengcheng@huawei.com> # message auto-generated for no-merge-commit merge: !28 merge license into master 【msmodelslim】刷新文件头部声明和版权 Created-by: caishengcheng Commit-by: caishengcheng Merged-by: ascend-robot Description: 【msmodelslim】刷新文件头部声明和版权 See merge request: Ascend/msmodelslim!284 个月前
【msmodelslim】刷新文件头部声明和版权 Co-authored-by: caishengcheng<caishengcheng@huawei.com> # message auto-generated for no-merge-commit merge: !28 merge license into master 【msmodelslim】刷新文件头部声明和版权 Created-by: caishengcheng Commit-by: caishengcheng Merged-by: ascend-robot Description: 【msmodelslim】刷新文件头部声明和版权 See merge request: Ascend/msmodelslim!284 个月前
【feature】预检失败由error改为warning,后续适配器报错再提供版本依赖 Co-authored-by: libarry<870390541@qq.com> # message auto-generated for no-merge-commit merge: !485 merge bugfix/precheck into master 【feature】预检失败由error改为warning,后续适配器报错再提供版本依赖 Created-by: libarry Commit-by: libarry Merged-by: ascend-robot Description: ## PR描述 (What this PR does / why we need it?) 1、依赖检查报错改为警告,所有版本依赖改为==或范围约束 2、当依赖检查失败后,适配器方法增加装饰器exception_handler,确保后续使用过程的适配器内任何报错会提示安装推荐的依赖版本号。 3、由于TimeoutError和内置TimeoutError重名,改名为ModelslimTimeoutError 4、修改UnexpectedError,增加两个方法,分别是注入tips和清理tips,支持提前注入外部Tips的能力,应用于warning后报错场景 ## 面向用户的变更 (Does this PR introduce _any_ user-facing change)? 用户环境依赖不符时,直接报错改为警告 ## 功能验证 (How was this patch tested?) 模型适配器问题注入tips: ![image.png](https://raw.gitcode.com/user-images/assets/8444818/3874cd20-89b6-4da6-83a1-dfa7a4a63bd6/image.png 'image.png') 模型forward报错触发UnexpectedError注入tips: ![image.png](https://raw.gitcode.com/user-images/assets/8444818/969dacf1-0479-442e-a0af-ba397cabfb46/image.png 'image.png') 请确认CI已通过增量及存量的单元测试用例。 如果本次测试方式与常规单元测试不同,请详细说明您的测试步骤(最好提供完整的可复现的操作路径及关键截图),以便Committer能够快速复现验证,也便于后续的维护。 如果未添加测试,请说明未添加的原因,以及为何难添加测试。 - [_] 功能自验 - [_] 本地自验截图(涉及个人标识符等敏感信息请注意脱敏) - [_] 新增/变更内容是否已新增/适配UT测试用例看护 CI passed with new added/existing test. If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future. If tests were not added, please describe why they were not added and/or why it was difficult to add. - [_] Self-verification of the feature. - [_] Screenshot of local self-verification (please anonymize any sensitive information such as personal identifiers) - [_] Have new or modified unit test (UT) cases been added or adapted to cover the newly added or changed content? See merge request: Ascend/msmodelslim!4859 小时前
【msmodelslim】刷新文件头部声明和版权 Co-authored-by: caishengcheng<caishengcheng@huawei.com> # message auto-generated for no-merge-commit merge: !28 merge license into master 【msmodelslim】刷新文件头部声明和版权 Created-by: caishengcheng Commit-by: caishengcheng Merged-by: ascend-robot Description: 【msmodelslim】刷新文件头部声明和版权 See merge request: Ascend/msmodelslim!284 个月前
【msmodelslim】刷新文件头部声明和版权 Co-authored-by: caishengcheng<caishengcheng@huawei.com> # message auto-generated for no-merge-commit merge: !28 merge license into master 【msmodelslim】刷新文件头部声明和版权 Created-by: caishengcheng Commit-by: caishengcheng Merged-by: ascend-robot Description: 【msmodelslim】刷新文件头部声明和版权 See merge request: Ascend/msmodelslim!284 个月前
[Bugfix] 修复 Decimal 在 YAML 持久化与配置 MD5 计算中的序列化问题 Co-authored-by: joejoezhou<zhourongchen1@huawei.com> # message auto-generated for no-merge-commit merge: !256 merge bugfix/accuracy_type into master [Bugfix] 修复 Decimal 在 YAML 持久化与配置 MD5 计算中的序列化问题 Created-by: joejoezhou Commit-by: joejoezhou Merged-by: ascend-robot Description: 感谢您贡献的Pull Request! 在提交之前,请务必阅读 [CONTRIBUTING.md](https://gitcode.com/Ascend/msmodelslim/blob/master/CONTRIBUTING.md)。 Thanks for sending a pull request! BEFORE SUBMITTING, PLEASE READ [CONTRIBUTING.md](https://gitcode.com/Ascend/msmodelslim/blob/master/CONTRIBUTING.md). ## PR描述 (What this PR does / why we need it?) - **问题**:含 Decimal 字段的 Pydantic BaseModel 在使用默认 model_dump() 时,yaml.safe_dump 可能因无法表示 Decimal 而抛出 RepresenterErrorcalculate_md5 若对默认 dump 结果做 json.dumps,亦会因 Decimal 不可直接 JSON 序列化而失败。自动调优/精度相关配置的建模与落盘、配置哈希会受影响。 - **变更**:写入 YAML 或导出量化配置前统一使用 model_dump(mode="json")calculate_md5 改为基于 mode='json' 的 dict 再排序序列化。补充/调整单测;文档示例中将 evaluation.demand.expectationstarget/tolerance 改为与 JSON 编码一致的引号字符串并注明原因。 - **English**: Fixes YAML persistence and MD5 calculation for configs whose Pydantic models contain Decimal, by using JSON-compatible dumps (model_dump(mode="json")) before YAML/JSON serialization. Adds/adjusts UT; doc examples updated accordingly. - 关联 issue:无 / Related issue: N/A ## 面向用户的变更 (Does this PR introduce _any_ user-facing change)? - 修复此前在含 Decimal 的配置场景下可能出现的 YAML 写入失败;对外 Python API 签名未变更。文档示例中目标/容差建议按示例使用引号字符串,与内部 JSON 模式一致。 - **English**: Fixes persistence failures for Decimal-bearing configs; no public API signature changes. Doc examples use quoted strings for numeric expectations where needed. ## 功能验证 (How was this patch tested?) - 在 test_yaml_databasetest_hash 等用例中增加或适配对 Decimal 字段的覆盖;相关 infra 单测已同步调整。 - [x] 功能自验 - [ ] 本地自验截图(涉及个人标识符等敏感信息请注意脱敏) - [x] 新增/变更内容是否已新增/适配UT测试用例看护 - [x] Self-verification of the feature. - [ ] Screenshot of local self-verification (please anonymize any sensitive information such as personal identifiers) - [x] Have new or modified unit test (UT) cases been added or adapted to cover the newly added or changed content? 合入前请在 MR 中评论 compile 以触发 CI(见 CONTRIBUTING)。 See merge request: Ascend/msmodelslim!2562 个月前
【msmodelslim】刷新文件头部声明和版权 Co-authored-by: caishengcheng<caishengcheng@huawei.com> # message auto-generated for no-merge-commit merge: !28 merge license into master 【msmodelslim】刷新文件头部声明和版权 Created-by: caishengcheng Commit-by: caishengcheng Merged-by: ascend-robot Description: 【msmodelslim】刷新文件头部声明和版权 See merge request: Ascend/msmodelslim!284 个月前
【msmodelslim】刷新文件头部声明和版权 Co-authored-by: caishengcheng<caishengcheng@huawei.com> # message auto-generated for no-merge-commit merge: !28 merge license into master 【msmodelslim】刷新文件头部声明和版权 Created-by: caishengcheng Commit-by: caishengcheng Merged-by: ascend-robot Description: 【msmodelslim】刷新文件头部声明和版权 See merge request: Ascend/msmodelslim!284 个月前
【msmodelslim】刷新文件头部声明和版权 Co-authored-by: caishengcheng<caishengcheng@huawei.com> # message auto-generated for no-merge-commit merge: !28 merge license into master 【msmodelslim】刷新文件头部声明和版权 Created-by: caishengcheng Commit-by: caishengcheng Merged-by: ascend-robot Description: 【msmodelslim】刷新文件头部声明和版权 See merge request: Ascend/msmodelslim!284 个月前
【feature】预检失败由error改为warning,后续适配器报错再提供版本依赖 Co-authored-by: libarry<870390541@qq.com> # message auto-generated for no-merge-commit merge: !485 merge bugfix/precheck into master 【feature】预检失败由error改为warning,后续适配器报错再提供版本依赖 Created-by: libarry Commit-by: libarry Merged-by: ascend-robot Description: ## PR描述 (What this PR does / why we need it?) 1、依赖检查报错改为警告,所有版本依赖改为==或范围约束 2、当依赖检查失败后,适配器方法增加装饰器exception_handler,确保后续使用过程的适配器内任何报错会提示安装推荐的依赖版本号。 3、由于TimeoutError和内置TimeoutError重名,改名为ModelslimTimeoutError 4、修改UnexpectedError,增加两个方法,分别是注入tips和清理tips,支持提前注入外部Tips的能力,应用于warning后报错场景 ## 面向用户的变更 (Does this PR introduce _any_ user-facing change)? 用户环境依赖不符时,直接报错改为警告 ## 功能验证 (How was this patch tested?) 模型适配器问题注入tips: ![image.png](https://raw.gitcode.com/user-images/assets/8444818/3874cd20-89b6-4da6-83a1-dfa7a4a63bd6/image.png 'image.png') 模型forward报错触发UnexpectedError注入tips: ![image.png](https://raw.gitcode.com/user-images/assets/8444818/969dacf1-0479-442e-a0af-ba397cabfb46/image.png 'image.png') 请确认CI已通过增量及存量的单元测试用例。 如果本次测试方式与常规单元测试不同,请详细说明您的测试步骤(最好提供完整的可复现的操作路径及关键截图),以便Committer能够快速复现验证,也便于后续的维护。 如果未添加测试,请说明未添加的原因,以及为何难添加测试。 - [_] 功能自验 - [_] 本地自验截图(涉及个人标识符等敏感信息请注意脱敏) - [_] 新增/变更内容是否已新增/适配UT测试用例看护 CI passed with new added/existing test. If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future. If tests were not added, please describe why they were not added and/or why it was difficult to add. - [_] Self-verification of the feature. - [_] Screenshot of local self-verification (please anonymize any sensitive information such as personal identifiers) - [_] Have new or modified unit test (UT) cases been added or adapted to cover the newly added or changed content? See merge request: Ascend/msmodelslim!4859 小时前
【msmodelslim】刷新文件头部声明和版权 Co-authored-by: caishengcheng<caishengcheng@huawei.com> # message auto-generated for no-merge-commit merge: !28 merge license into master 【msmodelslim】刷新文件头部声明和版权 Created-by: caishengcheng Commit-by: caishengcheng Merged-by: ascend-robot Description: 【msmodelslim】刷新文件头部声明和版权 See merge request: Ascend/msmodelslim!284 个月前
[Bugfix] 修复 Decimal 在 YAML 持久化与配置 MD5 计算中的序列化问题 Co-authored-by: joejoezhou<zhourongchen1@huawei.com> # message auto-generated for no-merge-commit merge: !256 merge bugfix/accuracy_type into master [Bugfix] 修复 Decimal 在 YAML 持久化与配置 MD5 计算中的序列化问题 Created-by: joejoezhou Commit-by: joejoezhou Merged-by: ascend-robot Description: 感谢您贡献的Pull Request! 在提交之前,请务必阅读 [CONTRIBUTING.md](https://gitcode.com/Ascend/msmodelslim/blob/master/CONTRIBUTING.md)。 Thanks for sending a pull request! BEFORE SUBMITTING, PLEASE READ [CONTRIBUTING.md](https://gitcode.com/Ascend/msmodelslim/blob/master/CONTRIBUTING.md). ## PR描述 (What this PR does / why we need it?) - **问题**:含 Decimal 字段的 Pydantic BaseModel 在使用默认 model_dump() 时,yaml.safe_dump 可能因无法表示 Decimal 而抛出 RepresenterErrorcalculate_md5 若对默认 dump 结果做 json.dumps,亦会因 Decimal 不可直接 JSON 序列化而失败。自动调优/精度相关配置的建模与落盘、配置哈希会受影响。 - **变更**:写入 YAML 或导出量化配置前统一使用 model_dump(mode="json")calculate_md5 改为基于 mode='json' 的 dict 再排序序列化。补充/调整单测;文档示例中将 evaluation.demand.expectationstarget/tolerance 改为与 JSON 编码一致的引号字符串并注明原因。 - **English**: Fixes YAML persistence and MD5 calculation for configs whose Pydantic models contain Decimal, by using JSON-compatible dumps (model_dump(mode="json")) before YAML/JSON serialization. Adds/adjusts UT; doc examples updated accordingly. - 关联 issue:无 / Related issue: N/A ## 面向用户的变更 (Does this PR introduce _any_ user-facing change)? - 修复此前在含 Decimal 的配置场景下可能出现的 YAML 写入失败;对外 Python API 签名未变更。文档示例中目标/容差建议按示例使用引号字符串,与内部 JSON 模式一致。 - **English**: Fixes persistence failures for Decimal-bearing configs; no public API signature changes. Doc examples use quoted strings for numeric expectations where needed. ## 功能验证 (How was this patch tested?) - 在 test_yaml_databasetest_hash 等用例中增加或适配对 Decimal 字段的覆盖;相关 infra 单测已同步调整。 - [x] 功能自验 - [ ] 本地自验截图(涉及个人标识符等敏感信息请注意脱敏) - [x] 新增/变更内容是否已新增/适配UT测试用例看护 - [x] Self-verification of the feature. - [ ] Screenshot of local self-verification (please anonymize any sensitive information such as personal identifiers) - [x] Have new or modified unit test (UT) cases been added or adapted to cover the newly added or changed content? 合入前请在 MR 中评论 compile 以触发 CI(见 CONTRIBUTING)。 See merge request: Ascend/msmodelslim!2562 个月前