0
代码
代码
Issues
Pull Requests
讨论
Wiki
分析
0
ascend-robotascend-robotw8a8sc权重加载逻辑迁移至权重预加载路线
3345bbbe创建于 3月14日历史提交
文件最后提交记录最后更新时间
test_config_qwen3.py
w8a8sc权重加载逻辑迁移至权重预加载路线 Co-authored-by: frost_mourne<suyuxuan3@h-partners.com> # message auto-generated for no-merge-commit merge: !406 merge dev into dev w8a8sc权重加载逻辑迁移至权重预加载路线 Created-by: frost_mourne Commit-by: frost_mourne Merged-by: ascend-robot Description: <!-- PR描述模板更新日期：20251225 --> # 合入背景 w8a8sc量化权重加载逻辑从原有路线迁移至显存预分配路线。 Fixes #176 #238 # 修改内容 1.quant_method_adapter.py添加W8A8SCLinearMethod。 2.qwen2_config.py中增加权重文件与模型结构的双向映射表及格式转换的方法。 3.embedding.py处理lmhead分别对应linear和embedding的转换。 4.w8a8sc.py处理w8a8sc权重注册和格式判断。 5.default_model_loader.py和weight_utils.py增加w8a8sc格式权重加载逻辑。 6.前作https://gitcode.com/Ascend/MindIE-LLM/pull/266/在default_model_loader.py中误将get_mapper_cls函数放在w8a8sc判断之外，导致非Qwen3模型会误入get_mapper_cls函数并报NotImplementedError: The model type of qwen2 has not implemented W8A8SC quant method yet错误，本作将get_mapper_cls函数移入w8a8sc判断内，并测试其他模型功能正常。 # 资料变更 不涉及 # 接口变更 不涉及 # 测试结果 对比同一个版本原始代码与加上本PR后的代码，分别开启服务化用aisbench测gsm8k数据集。开启前与开启后精度分别为25.40、25.70。测试不影响Qwen3 w8a8、Qwen3 浮点、Qwen2.5浮点、Qwen2.5 w8a8sc功能。 # CheckList > PR提交人对以下CheckList自检项进行全量自检，自检通过或不涉及，均修改 [ ] 为 [x]。 - [x] 代码注释完备 - [x] 正确记录错误日志 - [x] 进行了返回值校验 (禁止使用void屏蔽安全函数、自研函数返回值；考虑接口的异常场景；调用底层组件接口时，需要进行返回值校验) - [x] 进行了空指针校验 - [x] 若存在资源申请，使用后资源被正确的释放了 - [x] 若涉及多线程场景，考虑了并发场景，不存在死锁问题 - [x] 按照[代码仓中提供的格式模板](https://gitcode.com/Ascend/MindIE-LLM/blob/master/.clang-format)，使用clang-format工具格式化代码 - [x] 符合Ascend社区的编码规范。[C++ 语言编程指导](https://gitcode.com/Ascend/community/blob/master/docs/contributor/Ascend-cpp-coding-style-guide.md) | [C++ 语言安全编程指导](https://gitcode.com/Ascend/community/blob/master/docs/contributor/Ascend-cpp-secure-coding-guide.md) See merge request: Ascend/MindIE-LLM!4062 个月前
test_input_builder_qwen3.py
[新需求]新增aclgraph model base + qwen3 部分内容 + tokenizer wrapper、json completor Co-authored-by: stanzzzzz<zonghaoxin@huawei.com> # message auto-generated for no-merge-commit merge: !178 merge 0108aclgraphTodev into dev [新需求]新增aclgraph model base + qwen3 部分内容 + tokenizer wrapper、json completor Created-by: stanzzzzz Commit-by: stanzzzzz Merged-by: ascend-robot Description: <!-- PR描述模板更新日期：20251225 --> # 合入背景 Fix part of https://gitcode.com/Ascend/MindIE-LLM/issues/103 # 修改内容 1. model input builder 基类实现和qwen3 子类实现，用于构建符合模型要求的输入格式。 2. model reasoning parser 基类实现和qwen3 子类实现，处理推理内容。 3. model tool calls processor 基类实现和qwen3 子类实现，处理工具调用。 4. tokenizer wrapper 封装类实现，提供输入编码和输出解码接口。 5. json 补全器 json completor 工具类实现 # 资料变更 不涉及 # 接口变更 不涉及 # 测试结果 deepseek v32 ,可正常拉起服务化，发送请求正常返回，精度正常： ``` curl -H "Accept: application/json" -H "Content-type: application/json" -X POST -d '{ "model": "ds_v3.2", "messages": [ {"role": "user", "content": "你是谁？"} ], "stream": false, "ignore_eos": false, "max_tokens": 64 }' http://127.0.0.1:10010/v1/chat/completions {"id":"endpoint_common_0","object":"chat.completion","created":1767838406,"model":"ds_v3.2","choices":[{"index":0,"message":{"role":"assistant","content":"我是DeepSeek，由深度求索公司创造的AI助手。很高兴为你解答问题，提供帮助！😊","tool_calls":[]},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":6,"prompt_tokens_details":{"cached_tokens":0},"completion_tokens":25,"completion_tokens_details":{"reasoning_tokens":0},"total_tokens":31,"batch_size":[1,1,1,1,1,1,1,1,1,1,1,1,1,1,1],"queue_wait_time":[536,682,422,1371,368,257,1247,781,830,815,859,1228,1592,670,433]},"prefill_time":2153,"decode_time_arr":[291,291,140,140,140,213,213,420,210,210,208,208,220,220,451,478,445,460,155,155,155,493,232,232]} ``` qwen3 32B 可以正常拉起服务化，发送请求可以正常返回，精度正常 ``` curl --request POST --url http://127.0.0.1:1025/v1/chat/completions --header 'Content-Type: application/json' --data '{ "model":"qwen", "messages":[{ "role": "system", "content": "以梦里花落知多少作为开头，续写一首七言律诗" }], "chat_template_kwargs":{"enable_thinking":true}, "stream": false, "temperature": 0.95, "max_tokens":2048 }' {"id":"endpoint_common_0","object":"chat.completion","created":1767797790,"model":"qwen","choices":[{"index":0,"message":{"role":"assistant","content":"<think>\n好的，用户让我以“梦里花落知多少”开头续写一首七言律诗。首先，我需要确认七言律诗的格式要求。七律通常有八句，每句七个字，讲究平仄对仗，中间两联需要对仗工整，押韵一般用平声韵。\n\n接下来，分析原句“梦里花落知多少”。这句诗带有淡淡的哀愁和回忆的感觉，可能涉及离别、时光流逝的主题。我需要延续这种意境，同时展开后续的内容。\n\n首先确定押韵的韵脚。原句的“少”在这里需要注意，因为“少”在平水韵中属于小韵，可能需要调整。不过用户可能更在意现代汉语的押韵，所以可能需要选择“少”对应的韵母，比如“ao”韵，但七律通常要求一韵到底，可能需要调整用词。或者可能用户没有严格遵循古韵，可以适当放宽。\n\n接下来考虑内容的发展。第一句是梦境中的花落，可能引出回忆或者对过去的感慨。中间两联需要对仗，比如第二联和第三联。比如可以写现实中的景象与梦中的对比，或者时间的流逝带来的变化。\n\n比如第二联可以写现实中的景物，比如柳絮、燕呢喃，与梦中的花落形成对比。第三联可以转到更深层次的情感，比如离别后的孤独，或者岁月的变迁。\n\n最后两句需要收束全诗，可能表达一种无奈或者希望。比如问碧海青天，或者寻找答案。\n\n然后检查对仗是否工整，平仄是否符合要求。可能需要调整用词，确保每联的对仗工整，比如“柳线摇风”对“燕声剪水”，“离舟”对“旧信”，“云外”对“灯前”。\n\n最后检查押韵是否一致，通常七律押平声韵，所以“少、娇、娇、聊、天”需要确认是否在同一韵部。可能需要调整最后一个韵脚，使其一致。比如“天”是否和之前的韵脚押韵，可能需要换成“遥”或者其他字。\n\n可能还需要润色诗句，使意境更连贯，情感更统一。例如，确保每句之间有逻辑联系，从梦到现实，再到情感的抒发，最后以问句或感叹结尾，增强余韵。\n</think>\n\n《七律","tool_calls":[]},"logprobs":null,"finish_reason":"length"}],"usage":{"prompt_tokens":25,"prompt_tokens_details":{"cached_tokens":0},"completion_tokens":512,"completion_tokens_details":{"reasoning_tokens":507},"total_tokens":537,"batch_size":[1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1],"queue_wait_time":[5441,401,774,1138,978,255,307,252,227,355,368,297,250,308,370,356,436,327,822,337,792,902,686,304,340,406,354,328,869,389,1066,302,102,149,291,346,269,131,585,1088,175,162,279,377,302,334,185,140,659,1007,933,847,255,246,730,819,513,341,252,304,341,454,111,362,359,536,1063,940,763,195,70,76,205,1073,1098,130,1062,242,147,136,814,560,82,945,638,1082,328,718,669,412,177,776,789,1073,1060,423,494,381,1048,165,666,629,175,1001,373,1100,267,167,1011,89,152,1103,1097,224,250,91,934,298,217,493,1108,149,643,1155,912,133,192,1040,476,341,773,1074,1080,984,581,918,866,878,1108,1095,525,951,903,170,124,79,83,1070,697,282,758,307,1150,249,1027,451,652,479,790,601,672,446,126,119,139,96,935,744,590,207,781,445,871,134,152,93,608,928,469,476,1014,710,1021,421,902,664,876,1000,89,195,180,166,1103,86,1070,130,644,152,49,76,1028,127,1087,155,132,88,1022,917,190,165,125,1096,764,1102,169,86,803,330,131,139,70,115,958,743,53,116,101,110,83,85,84,103,1074,185,53,125,69,142,81,113,76,990,70,833,761,1094,1026,714,62,993,843,660,498,71,1099,69,74,1123,978,72,123,125,1038,908,124,1006,899,173,154,113,131,135,1088,134,196,190,134,143,1081,1007,160,189,92,1072,1049,1012,102,841,841,163,148,148,211,231,141,753,592,99,251,194,182,210,132,742,368,154,1056,905,153,142,962,746,513,1076,784,801,225,66,1050,1015,973,1108,1032,1037,168,182,162,134,81,164,817,553,328,730,829,833,1026,865,577,924,999,804,942,1071,957,953,986,1071,57,128,496,964,960,995,957,1067,984,1067,828,1019,1026,1027,760,653,516,655,491,878,823,830,1006,508,824,812,148,831,555,935,836,647,377,804,911,602,276,868,724,862,838,883,1016,994,937,881,872,1102,1107,188,405,884,826,1029,1035,127,187,164,1106,1097,70,132,96,159,1084,1043,865,153,119,1135,668,446,1061,521,737,952,786,494,976,167,388,920,896,591,723,1002,684,441,898,766,923,1015,929,902,667,325,907,876,694,783,939,919,878,549,933,843,874,899,665,85,163,673,473,557,749,686,825,858,721,646,697,942,612,426,619,779,571,325,186,235,712,404,707,171,1134,444,225,146,227,255,368,909,670,576,1007,179,1034,769,953,252,411,207,960,1091,896,986,406,749,723,1015,262,680,310,284]},"prefill_time":166,"decode_time_arr":[56,37,38,37,36,36,36,36,36,37,36,36,36,36,36,37,37,38,37,37,38,37,36,36,36,36,37,38,37,37,37,36,36,36,36,37,37,37,37,36,36,36,37,36,37,38,38,37,38,38,37,36,37,38,38,37,37,36,36,37,37,36,37,37,37,39,38,37,38,36,36,37,37,37,37,38,37,36,37,38,37,37,38,38,39,37,38,38,38,37,37,38,39,38,37,38,38,38,37,39,38,37,38,37,37,37,37,37,36,37,37,37,36,37,37,37,37,37,37,37,37,38,38,37,36,37,38,38,37,37,38,38,38,37,38,38,37,38,38,37,38,39,37,37,36,37,38,39,37,38,38,39,38,39,38,39,40,38,38,39,38,37,36,37,40,38,38,37,37,38,37,37,36,36,37,38,38,38,37,38,38,38,37,38,37,37,37,36,36,36,37,37,37,37,37,37,37,36,37,37,37,37,36,36,37,38,37,36,36,37,38,37,37,37,37,38,37,36,37,36,37,38,37,36,36,36,36,36,36,36,37,38,37,36,36,36,36,36,36,37,37,37,38,37,38,38,37,37,38,38,38,37,36,37,36,36,38,37,36,36,37,38,37,37,38,37,36,36,36,36,37,37,36,36,36,36,37,38,37,36,37,37,38,38,37,37,38,37,36,36,36,36,37,37,38,37,36,36,36,36,37,37,38,37,37,38,37,36,37,38,38,37,38,37,37,37,37,38,38,37,38,37,37,36,36,36,36,36,37,38,38,38,38,39,38,39,39,38,38,39,38,38,39,39,39,38,39,37,37,37,38,37,38,37,38,37,38,37,38,37,38,39,39,38,39,38,39,40,39,39,38,39,39,38,38,37,38,38,38,37,37,38,39,37,38,37,38,37,37,38,38,38,38,38,38,38,37,37,38,37,38,37,36,36,37,38,37,36,36,36,37,38,38,37,36,37,38,38,38,40,38,38,39,39,39,39,37,37,38,38,37,37,38,38,37,38,37,37,38,38,38,38,37,38,38,37,37,38,38,38,37,38,37,37,38,38,37,37,38,37,37,38,37,37,38,38,37,37,39,38,37,37,38,38,38,37,37,39,39,38,39,39,3 ``` # CheckList > PR提交人对以下CheckList自检项进行全量自检，自检通过或不涉及，均修改 [ ] 为 [x]。 - [x] 代码注释完备 - [x] 正确记录错误日志 - [x] 进行了返回值校验 (禁止使用void屏蔽安全函数、自研函数返回值；考虑接口的异常场景；调用底层组件接口时，需要进行返回值校验) - [x] 进行了空指针校验 - [x] 若存在资源申请，使用后资源被正确的释放了 - [x] 若涉及多线程场景，考虑了并发场景，不存在死锁问题 - [ ] 按照[代码仓中提供的格式模板](https://gitcode.com/Ascend/MindIE-LLM/blob/master/.clang-format)，使用clang-format工具格式化代码 - [ ] 符合Ascend社区的编码规范。[C++ 语言编程指导](https://gitcode.com/Ascend/community/blob/master/docs/contributor/Ascend-cpp-coding-style-guide.md) | [C++ 语言安全编程指导](https://gitcode.com/Ascend/community/blob/master/docs/contributor/Ascend-cpp-secure-coding-guide.md) See merge request: Ascend/MindIE-LLM!1784 个月前
test_tool_calls_processor_qwen3.py
[新需求]新增aclgraph model base + qwen3 部分内容 + tokenizer wrapper、json completor Co-authored-by: stanzzzzz<zonghaoxin@huawei.com> # message auto-generated for no-merge-commit merge: !178 merge 0108aclgraphTodev into dev [新需求]新增aclgraph model base + qwen3 部分内容 + tokenizer wrapper、json completor Created-by: stanzzzzz Commit-by: stanzzzzz Merged-by: ascend-robot Description: <!-- PR描述模板更新日期：20251225 --> # 合入背景 Fix part of https://gitcode.com/Ascend/MindIE-LLM/issues/103 # 修改内容 1. model input builder 基类实现和qwen3 子类实现，用于构建符合模型要求的输入格式。 2. model reasoning parser 基类实现和qwen3 子类实现，处理推理内容。 3. model tool calls processor 基类实现和qwen3 子类实现，处理工具调用。 4. tokenizer wrapper 封装类实现，提供输入编码和输出解码接口。 5. json 补全器 json completor 工具类实现 # 资料变更 不涉及 # 接口变更 不涉及 # 测试结果 deepseek v32 ,可正常拉起服务化，发送请求正常返回，精度正常： ``` curl -H "Accept: application/json" -H "Content-type: application/json" -X POST -d '{ "model": "ds_v3.2", "messages": [ {"role": "user", "content": "你是谁？"} ], "stream": false, "ignore_eos": false, "max_tokens": 64 }' http://127.0.0.1:10010/v1/chat/completions {"id":"endpoint_common_0","object":"chat.completion","created":1767838406,"model":"ds_v3.2","choices":[{"index":0,"message":{"role":"assistant","content":"我是DeepSeek，由深度求索公司创造的AI助手。很高兴为你解答问题，提供帮助！😊","tool_calls":[]},"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":6,"prompt_tokens_details":{"cached_tokens":0},"completion_tokens":25,"completion_tokens_details":{"reasoning_tokens":0},"total_tokens":31,"batch_size":[1,1,1,1,1,1,1,1,1,1,1,1,1,1,1],"queue_wait_time":[536,682,422,1371,368,257,1247,781,830,815,859,1228,1592,670,433]},"prefill_time":2153,"decode_time_arr":[291,291,140,140,140,213,213,420,210,210,208,208,220,220,451,478,445,460,155,155,155,493,232,232]} ``` qwen3 32B 可以正常拉起服务化，发送请求可以正常返回，精度正常 ``` curl --request POST --url http://127.0.0.1:1025/v1/chat/completions --header 'Content-Type: application/json' --data '{ "model":"qwen", "messages":[{ "role": "system", "content": "以梦里花落知多少作为开头，续写一首七言律诗" }], "chat_template_kwargs":{"enable_thinking":true}, "stream": false, "temperature": 0.95, "max_tokens":2048 }' {"id":"endpoint_common_0","object":"chat.completion","created":1767797790,"model":"qwen","choices":[{"index":0,"message":{"role":"assistant","content":"<think>\n好的，用户让我以“梦里花落知多少”开头续写一首七言律诗。首先，我需要确认七言律诗的格式要求。七律通常有八句，每句七个字，讲究平仄对仗，中间两联需要对仗工整，押韵一般用平声韵。\n\n接下来，分析原句“梦里花落知多少”。这句诗带有淡淡的哀愁和回忆的感觉，可能涉及离别、时光流逝的主题。我需要延续这种意境，同时展开后续的内容。\n\n首先确定押韵的韵脚。原句的“少”在这里需要注意，因为“少”在平水韵中属于小韵，可能需要调整。不过用户可能更在意现代汉语的押韵，所以可能需要选择“少”对应的韵母，比如“ao”韵，但七律通常要求一韵到底，可能需要调整用词。或者可能用户没有严格遵循古韵，可以适当放宽。\n\n接下来考虑内容的发展。第一句是梦境中的花落，可能引出回忆或者对过去的感慨。中间两联需要对仗，比如第二联和第三联。比如可以写现实中的景象与梦中的对比，或者时间的流逝带来的变化。\n\n比如第二联可以写现实中的景物，比如柳絮、燕呢喃，与梦中的花落形成对比。第三联可以转到更深层次的情感，比如离别后的孤独，或者岁月的变迁。\n\n最后两句需要收束全诗，可能表达一种无奈或者希望。比如问碧海青天，或者寻找答案。\n\n然后检查对仗是否工整，平仄是否符合要求。可能需要调整用词，确保每联的对仗工整，比如“柳线摇风”对“燕声剪水”，“离舟”对“旧信”，“云外”对“灯前”。\n\n最后检查押韵是否一致，通常七律押平声韵，所以“少、娇、娇、聊、天”需要确认是否在同一韵部。可能需要调整最后一个韵脚，使其一致。比如“天”是否和之前的韵脚押韵，可能需要换成“遥”或者其他字。\n\n可能还需要润色诗句，使意境更连贯，情感更统一。例如，确保每句之间有逻辑联系，从梦到现实，再到情感的抒发，最后以问句或感叹结尾，增强余韵。\n</think>\n\n《七律","tool_calls":[]},"logprobs":null,"finish_reason":"length"}],"usage":{"prompt_tokens":25,"prompt_tokens_details":{"cached_tokens":0},"completion_tokens":512,"completion_tokens_details":{"reasoning_tokens":507},"total_tokens":537,"batch_size":[1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1],"queue_wait_time":[5441,401,774,1138,978,255,307,252,227,355,368,297,250,308,370,356,436,327,822,337,792,902,686,304,340,406,354,328,869,389,1066,302,102,149,291,346,269,131,585,1088,175,162,279,377,302,334,185,140,659,1007,933,847,255,246,730,819,513,341,252,304,341,454,111,362,359,536,1063,940,763,195,70,76,205,1073,1098,130,1062,242,147,136,814,560,82,945,638,1082,328,718,669,412,177,776,789,1073,1060,423,494,381,1048,165,666,629,175,1001,373,1100,267,167,1011,89,152,1103,1097,224,250,91,934,298,217,493,1108,149,643,1155,912,133,192,1040,476,341,773,1074,1080,984,581,918,866,878,1108,1095,525,951,903,170,124,79,83,1070,697,282,758,307,1150,249,1027,451,652,479,790,601,672,446,126,119,139,96,935,744,590,207,781,445,871,134,152,93,608,928,469,476,1014,710,1021,421,902,664,876,1000,89,195,180,166,1103,86,1070,130,644,152,49,76,1028,127,1087,155,132,88,1022,917,190,165,125,1096,764,1102,169,86,803,330,131,139,70,115,958,743,53,116,101,110,83,85,84,103,1074,185,53,125,69,142,81,113,76,990,70,833,761,1094,1026,714,62,993,843,660,498,71,1099,69,74,1123,978,72,123,125,1038,908,124,1006,899,173,154,113,131,135,1088,134,196,190,134,143,1081,1007,160,189,92,1072,1049,1012,102,841,841,163,148,148,211,231,141,753,592,99,251,194,182,210,132,742,368,154,1056,905,153,142,962,746,513,1076,784,801,225,66,1050,1015,973,1108,1032,1037,168,182,162,134,81,164,817,553,328,730,829,833,1026,865,577,924,999,804,942,1071,957,953,986,1071,57,128,496,964,960,995,957,1067,984,1067,828,1019,1026,1027,760,653,516,655,491,878,823,830,1006,508,824,812,148,831,555,935,836,647,377,804,911,602,276,868,724,862,838,883,1016,994,937,881,872,1102,1107,188,405,884,826,1029,1035,127,187,164,1106,1097,70,132,96,159,1084,1043,865,153,119,1135,668,446,1061,521,737,952,786,494,976,167,388,920,896,591,723,1002,684,441,898,766,923,1015,929,902,667,325,907,876,694,783,939,919,878,549,933,843,874,899,665,85,163,673,473,557,749,686,825,858,721,646,697,942,612,426,619,779,571,325,186,235,712,404,707,171,1134,444,225,146,227,255,368,909,670,576,1007,179,1034,769,953,252,411,207,960,1091,896,986,406,749,723,1015,262,680,310,284]},"prefill_time":166,"decode_time_arr":[56,37,38,37,36,36,36,36,36,37,36,36,36,36,36,37,37,38,37,37,38,37,36,36,36,36,37,38,37,37,37,36,36,36,36,37,37,37,37,36,36,36,37,36,37,38,38,37,38,38,37,36,37,38,38,37,37,36,36,37,37,36,37,37,37,39,38,37,38,36,36,37,37,37,37,38,37,36,37,38,37,37,38,38,39,37,38,38,38,37,37,38,39,38,37,38,38,38,37,39,38,37,38,37,37,37,37,37,36,37,37,37,36,37,37,37,37,37,37,37,37,38,38,37,36,37,38,38,37,37,38,38,38,37,38,38,37,38,38,37,38,39,37,37,36,37,38,39,37,38,38,39,38,39,38,39,40,38,38,39,38,37,36,37,40,38,38,37,37,38,37,37,36,36,37,38,38,38,37,38,38,38,37,38,37,37,37,36,36,36,37,37,37,37,37,37,37,36,37,37,37,37,36,36,37,38,37,36,36,37,38,37,37,37,37,38,37,36,37,36,37,38,37,36,36,36,36,36,36,36,37,38,37,36,36,36,36,36,36,37,37,37,38,37,38,38,37,37,38,38,38,37,36,37,36,36,38,37,36,36,37,38,37,37,38,37,36,36,36,36,37,37,36,36,36,36,37,38,37,36,37,37,38,38,37,37,38,37,36,36,36,36,37,37,38,37,36,36,36,36,37,37,38,37,37,38,37,36,37,38,38,37,38,37,37,37,37,38,38,37,38,37,37,36,36,36,36,36,37,38,38,38,38,39,38,39,39,38,38,39,38,38,39,39,39,38,39,37,37,37,38,37,38,37,38,37,38,37,38,37,38,39,39,38,39,38,39,40,39,39,38,39,39,38,38,37,38,38,38,37,37,38,39,37,38,37,38,37,37,38,38,38,38,38,38,38,37,37,38,37,38,37,36,36,37,38,37,36,36,36,37,38,38,37,36,37,38,38,38,40,38,38,39,39,39,39,37,37,38,38,37,37,38,38,37,38,37,37,38,38,38,38,37,38,38,37,37,38,38,38,37,38,37,37,38,38,37,37,38,37,37,38,37,37,38,38,37,37,39,38,37,37,38,38,38,37,37,39,39,38,39,39,3 ``` # CheckList > PR提交人对以下CheckList自检项进行全量自检，自检通过或不涉及，均修改 [ ] 为 [x]。 - [x] 代码注释完备 - [x] 正确记录错误日志 - [x] 进行了返回值校验 (禁止使用void屏蔽安全函数、自研函数返回值；考虑接口的异常场景；调用底层组件接口时，需要进行返回值校验) - [x] 进行了空指针校验 - [x] 若存在资源申请，使用后资源被正确的释放了 - [x] 若涉及多线程场景，考虑了并发场景，不存在死锁问题 - [ ] 按照[代码仓中提供的格式模板](https://gitcode.com/Ascend/MindIE-LLM/blob/master/.clang-format)，使用clang-format工具格式化代码 - [ ] 符合Ascend社区的编码规范。[C++ 语言编程指导](https://gitcode.com/Ascend/community/blob/master/docs/contributor/Ascend-cpp-coding-style-guide.md) | [C++ 语言安全编程指导](https://gitcode.com/Ascend/community/blob/master/docs/contributor/Ascend-cpp-secure-coding-guide.md) See merge request: Ascend/MindIE-LLM!1784 个月前