MindIE-LLM/mindie_llm/runtime/compilation · Ascend/MindIE-LLM

文件	最后提交记录	最后更新时间
__init__.py	compilation合入 Co-authored-by: LostFox11<wangziyue17@huawei.com> # message auto-generated for no-merge-commit merge: !209 merge compliation into dev compilation合入 Created-by: LostFox11 Commit-by: LostFox11 Merged-by: ascend-robot Description: <!-- PR描述模板更新日期：20251225 --> # 合入背景 > Fix part of #103 # 修改内容 > 添加acl_graph_wrapper.py文件当走aclgraph且decode情况下，在原来的model上包一层acl_graph_wrapper 其中包括实际选图、图的capture、replay、update等操作 # 资料变更 > 不涉及 # 接口变更 > 不涉及 # 测试结果 > 在A3机器上测试功能、性能、精度无误 # CheckList > PR提交人对以下CheckList自检项进行全量自检，自检通过或不涉及，均修改 [ ] 为 [x]。 - [x] 代码注释完备 - [ ] 正确记录错误日志 - [x] 进行了返回值校验 (禁止使用void屏蔽安全函数、自研函数返回值；考虑接口的异常场景；调用底层组件接口时，需要进行返回值校验) - [x] 进行了空指针校验 - [x] 若存在资源申请，使用后资源被正确的释放了 - [ ] 若涉及多线程场景，考虑了并发场景，不存在死锁问题 - [x] 按照[代码仓中提供的格式模板](https://gitcode.com/Ascend/MindIE-LLM/blob/master/.clang-format)，使用clang-format工具格式化代码 - [ ] 符合Ascend社区的编码规范。[C++ 语言编程指导](https://gitcode.com/Ascend/community/blob/master/docs/contributor/Ascend-cpp-coding-style-guide.md) \| [C++ 语言安全编程指导](https://gitcode.com/Ascend/community/blob/master/docs/contributor/Ascend-cpp-secure-coding-guide.md) See merge request: Ascend/MindIE-LLM!209	6 个月前
aclgraph_backend.py	修复dp负载不均时计算kv_len和q_len的问题 Co-authored-by: earthmanylf<yulinfeng2@huawei.com> # message auto-generated for no-merge-commit merge: !879 merge spec into dev 修复dp负载不均时计算kv_len和q_len的问题 Created-by: earthmanylf Commit-by: earthmanylf Merged-by: ascend-robot Description: <!-- PR描述模板更新日期：20251225 --> # 合入背景 > 请描述为什么要做这个PR内的改动。\ > 如涉及，请关联前序PR或同特性/需求下的其他PR。\ > 如果是修复之前PR引入的问题，请关联引入问题的PR。\ > 注意：`Fixes #ISSUE ID`会自动关闭issue，如问题部分解决请不要使用`Fixes`，可以用`Fix part of #ISSUE ID`替代. Fix part of [#103](https://gitcode.com/Ascend/MindIE-LLM/issues/103) # 修改内容 > 请描述修改内容的具体实现，涉及哪些组件之间进行交互，可以用1、2、3、...进行罗列。\ > 如果是需求或者重构类的PR，需要补充详细设计文档（说明上下游组件关系、时序图、类图、DFX能力等内容）。修复dp负载不均时计算kv_len和q_len的问题 # 资料变更 > 请确认是否涉及资料变更。如涉及，需要在PR中体现，并简要说明修改内容。如不涉及，需填写“不涉及”。不涉及 # 接口变更 > 请确认是否涉及跨代码仓或者客户面可见的接口变更。如涉及，需要详细说明接口以及对应的变更内容，同时需要在资料中体现。如不涉及，需填写“不涉及”。不涉及 # 测试结果 > 请说明测试场景，测试方法以及测试结果。\ > 测试用例设计时需考虑硬件、部署方式、功能、性能、精度、显存等维度。 curl通： curl http://141.61.81.21:3025/v1/chat/completions -X POST -d'{"model": "ds","messages": [{"role": "user","content": "You are a helpful assistant."}],"stream": true,"temperature": 0,"max_tokens": 10}' data: {"id":"endpoint_common_0","object":"chat.completion.chunk","created":1776348349,"model":"ds","choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}]} data: {"id":"endpoint_common_0","object":"chat.completion.chunk","created":1776348349,"model":"ds","choices":[{"index":0,"delta":{"role":"assistant","content":"Hello"},"logprobs":null,"finish_reason":null}]} data: {"id":"endpoint_common_0","object":"chat.completion.chunk","created":1776348349,"model":"ds","choices":[{"index":0,"delta":{"role":"assistant","content":"! I'm"},"logprobs":null,"finish_reason":null}]} data: {"id":"endpoint_common_0","object":"chat.completion.chunk","created":1776348349,"model":"ds","choices":[{"index":0,"delta":{"role":"assistant","content":" here to help"},"logprobs":null,"finish_reason":null}]} data: {"id":"endpoint_common_0","object":"chat.completion.chunk","created":1776348349,"model":"ds","choices":[{"index":0,"delta":{"role":"assistant","content":" you"},"logprobs":null,"finish_reason":null}]} data: {"id":"endpoint_common_0","object":"chat.completion.chunk","created":1776348349,"model":"ds","choices":[{"index":0,"delta":{"role":"assistant","content":"."},"logprobs":null,"finish_reason":null}]} data: {"id":"endpoint_common_0","object":"chat.completion.chunk","created":1776348349,"model":"ds","usage":{"prompt_tokens":10,"prompt_tokens_details":{"cached_tokens":0},"completion_tokens":10,"completion_tokens_details":{"reasoning_tokens":0},"total_tokens":20},"choices":[{"index":0,"delta":{"role":"assistant","content":" Whether"},"logprobs":null,"finish_reason":"length"}]} # CheckList > PR提交人对以下CheckList自检项进行全量自检，自检通过或不涉及，均修改 [ ] 为 [x]。 - [x] 代码注释完备 - [x] 正确记录错误日志 - [x] 进行了返回值校验 (禁止使用void屏蔽安全函数、自研函数返回值；考虑接口的异常场景；调用底层组件接口时，需要进行返回值校验) - [x] 进行了空指针校验 - [x] 若存在资源申请，使用后资源被正确的释放了 - [x] 若涉及多线程场景，考虑了并发场景，不存在死锁问题 - [x] 按照[代码仓中提供的格式模板](https://gitcode.com/Ascend/MindIE-LLM/blob/master/.clang-format)，使用clang-format工具格式化代码 - [x] 符合Ascend社区的编码规范。[C++ 语言编程指导](https://gitcode.com/Ascend/community/blob/master/docs/contributor/Ascend-cpp-coding-style-guide.md) \| [C++ 语言安全编程指导](https://gitcode.com/Ascend/community/blob/master/docs/contributor/Ascend-cpp-secure-coding-guide.md) See merge request: Ascend/MindIE-LLM!879	3 个月前