在提交新问题之前，请确保您已经在社区中搜索过相关问题，并使用了社区中提供的资源/工具后，仍未找到满意的解决方式。

⚠️ 安全信息提醒：请仔细检查提供的文本内容，确保其不包含敏感数据信息，包括但不限于：

• API 令牌或密钥
• 密码或身份验证凭证
• 私有网址或接口地址
• 个人或机密数据
• ...

在分享配置信息或代码示例时，请将敏感信息脱敏处理，或使用 <TOKEN> 等占位符替代原有内容。

环境信息

大EP的D节点一个dp下tp=2

🐛 问题描述

当D节点在pullkv时如果遇到tp0 pullkv成功，但是tp1 pullkv失败时没有考虑失败的结果，认为pullkv成功，这是不正确的，这时放到running队列中会导致报错The sequence 101 is not prefilled before decoding.

报错日志：
[2026-04-08 09:45:21,638] [440689] [281470492993248] [llm] [ERROR] [separate_deployment_engine.py-390] : Pull kv from remote_cluster_id: 0 failed, CANN status_code is: LLMStatusCode.LLM_NOT_YET_LINK.
[2026-04-08 09:45:21,638] [440689] [281470492993248] [llm] [ERROR] [separate_deployment_engine.py-519] : 124.0.14.44 pull kv from unknown failed, error code is MIE05E01001A.
[2026-04-08 09:45:21,638] [440689] [281470492993248] [llm] [ERROR] [generator.py-159] : Pull blocks from remote cluster id: 0 failed, error code is MIE05E01001A
[2026-04-08 09:45:21,638] [440689] [281470492993248] [llm] [ERROR] [router_impl.py-328] : global rank-11 cluster-0 pull kv cache failed! pull_kv_items: [(0, [72], [0]), (100, [7], [1]), (200, [7], [2]), (300, [7], [3]), (400, [7], [4]), (500, [7], [5]), (600, [7], [6]), (700, [7], [7]), (800, [4], [8])], request_ids: ['7308032266271034873']
[2026-04-08 09:45:21,639] [440689] [281470492993248] [llm] [INFO] [response_builder.py-136] : [ExecuteResponseBuilder] build_from_transfer_result proto_response=msg_type: 5
pull_kv_response {
pull_kv_results {
request_id: "7308032266271034873"
pd_error_code: PD_PULL_KV_ERROR
}
}

[2026-04-08 09:45:21,684] [440653] [281470472021728] [llm] [INFO] [response_builder.py-136] : [ExecuteResponseBuilder] build_from_transfer_result proto_response=msg_type: 5
pull_kv_response {
pull_kv_results {
request_id: "7308032266271034873"
}
}

[2026-04-08 09:45:21,687] [440689] [281459686368992] [llm] [DEBUG] [separate_deployment_engine.py-716] : Querying mem status for 124.0.13.31, attempt 3
[2026-04-08 09:45:21.687519] [414048] [534723] [llm] [INFO] [transfer_output_handler.cpp:27] [TransferOutputHandler] Entry4Executor PullKVResponse dpRank=0 pull_kv_response=pull_kv_results { request_id: "7308032266271034873" }
[2026-04-08 09:45:21.687680] [414048] [534723] [server] [DEBUG] [grpc_communication_mng.cpp:540] [endpoint] [GrpcCommunicationMng::SendKvReleaseMsg] send kv release request requestId: 7308032266271034873 to P node 192.168.117.15:1121
[2026-04-08 09:45:21.687712] [414048] [534723] [server] [INFO] [dmi_msg_sender.cpp:40] [endpoint] Client init without tls.
[2026-04-08 09:45:21.687796] [412741] [535317] [llm] [DEBUG] [executor.cpp:368] Ready to execute inference requests.
[2026-04-08 09:45:21,687] [440689] [281459686368992] [llm] [DEBUG] [separate_deployment_engine.py-724] : Query completed in 0.00s for 124.0.13.31
[2026-04-08 09:45:21.687871] [414048] [535321] [llm] [INFO] [scheduler.cpp:1409] [LlmEngine|Request-Enter running queue] DP RankId: 5. Pull kv ended, enter running queue. requestId: 7308032266271034873; seqId: 101; running size:1; waiting size: 0; swapped size:0; transferring size:0
[2026-04-08 09:45:21.687920] [414048] [534723] [server] [INFO] [grpc_communication_mng.cpp:427] [endpoint] Init kv sender success
[2026-04-08 09:45:21,687] [440689] [281459686368992] [llm] [DEBUG] [separate_deployment_engine.py-755] : Max query attempts reached for 124.0.13.31, requeueing
[2026-04-08 09:45:21.687990] [414048] [535321] [llm] [DEBUG] [executor.cpp:368] Ready to execute inference requests.
[2026-04-08 09:45:21.688080] [415774] [535326] [llm] [DEBUG] [executor.cpp:368] Ready to execute inference requests.
[separate_deployment_engine.py-724] : Query completed in 0.00s for 124.0.14.38
[2026-04-08 09:45:21,675] [440683] [281459444082400] [llm] [DEBUG] [separate_deployment_engine.py-716] : Querying mem status for 124.0.14.38, attempt 2
[2026-04-08 09:45:21,675] [440683] [281459444082400] [llm] [DEBUG] [separate_deployment_engine.py-724] : Query completed in 0.00s for 124.0.14.38
[2026-04-08 09:45:21.688196] [417377] [535328] [llm] [DEBUG] [executor.cpp:368] Ready to execute inference requests.
[2026-04-08 09:45:21,688] [440689] [281459686368992] [llm] [DEBUG] [separate_deployment_engine.py-716] : Querying mem status for 124.0.13.41, attempt 1
[2026-04-08 09:45:21,688] [440689] [281459686368992] [llm] [DEBUG] [separate_deployment_engine.py-724] : Query completed in 0.00s for 124.0.13.41
[2026-04-08 09:45:21,689] [440689] [281470507673312] [llm] [ERROR] [batch_context.py-542] : [MIE05E010010] There is no cached data for sequence 101. This could be due to one of the following reasons: 1. It has not gone through the prefilling stage. 2. The sequence was determined to have stopped but was still passed into the decoding process.
[2026-04-08 09:45:21,689] [440689] [281470507673312] [llm] [ERROR] [router_impl.py-608] : Unknown exception when handling normal batch, error: The sequence 101 is not prefilled before decoding.

欢迎加入社区，感谢您对社区的贡献 🎉!