Star41
139
代码
代码
Issues157
Pull Requests133
流水线
项目讨论
Wiki
分析
Star41
139
文件最后提交记录最后更新时间
roles
[feature] deployer NodePort 冲突检测与 user_config 重映射 Co-authored-by: yangan7<yangan7@h-partners.com> # message auto-generated for no-merge-commit merge: !676 merge nodeport into master [feature] deployer NodePort 冲突检测与 user_config 重映射 Created-by: mindie_yangan Commit-by: yangan7 Merged-by: tobking Description: ## 1. 合入背景 Motor 多服务同集群部署时，默认 NodePort（如 31015 / 31017 / 31027）若已被占用，kubectl apply 会因 nodePort already allocated 失败，相关 Pod 起不来。 本 PR 在 apply 前检测集群已占用 NodePort，支持交互式 remap，并使 user_config 中三个 NodePort 配置生效。 Related Issue：https://gitcode.com/Ascend/MindIE-Motor/issues/379 ## 2. 修改内容 **方案**：deployer apply 前扫描生成 yaml 中的 NodePort，与集群占用对比；冲突时按端口交互（y=自动分配、<port>=指定口、N=保持冲突口并给出修复指引）；非 TTY 视为 N。交互 remap **只回写本次** output_yamls，**不**自动改 user_config.json（需手动同步 motor_deploy_config.*_node_port）。 ### 2.1 业务代码 - examples/deployer/lib/nodeport_allocator.py（新增） - 采集计划口 / 集群占用、冲突判定（同 ns + 同 service 名视为自有，不算冲突；同端口只提示一次） - 交互 remap：y / <port> / N；dry-run 只打印建议映射（返回空 remapping），不改 yaml、不清已有告警文件 - N / 非 TTY：继续部署冲突口，写 coordinator/controller showlog 告警，并提示手动改 user_config - examples/deployer/lib/generator/k8s_utils.py - apply / configmap 前调用 resolve_nodeports_for_yaml_files；configmap 挂载冲突告警文件 - examples/deployer/deploy.py - dry-run 路径在生成 yaml 后做冲突检测（只打印） - examples/deployer/lib/utils.py + generator（controller / coordinator / infer_service） - coordinator_infer_node_port / coordinator_obs_node_port / controller_observability_node_port 生效 - is_observability_service_name() 避免 robust 一类误匹配（coordinator/controller/infer_service 复用） - examples/deployer/startup/roles/{coordinator,controller}.sh - 启动时打印 NodePort 冲突告警（若 configmap 中有内容） - examples/deployer/README.md - 补充三端口说明与冲突交互说明 ### 2.2 测试 - tests/examples/deployer/test_nodeport_allocator.py（新增） - 计划口采集、自有忽略、跨 ns 冲突、y/port/N、dry-run、非 TTY、showlog、obs 名称匹配、同端口只弹一次、拒绝占用其他 planned 端口等 ## 3. 自验证 ![image.png](https://raw.gitcode.com/user-images/assets/9072912/a2cc0b07-eb7f-4b86-9b31-daf55ff9a46b/image.png 'image.png') ![image.png](https://raw.gitcode.com/user-images/assets/9072912/777abb4c-3cc1-4be8-98ca-a788dd35d5c7/image.png 'image.png') See merge request: Ascend/MindIE-Motor!676
8 天前
boot.sh
[feature] 多kv池化后端支持 Co-authored-by: 吕有辉<lvyouhui@huawei.com> # message auto-generated for no-merge-commit merge: !382 merge memcache into master [feature] 多kv池化后端支持 Created-by: codeDogPro Commit-by: 吕有辉 Merged-by: towncharlie Description: ## **1. 合入背景** 适配多个池化后端【当前PR适配Memcache，yuanrong后端后续支持】 ## **2. 修改内容** 1、特性文档优化，多后端可拓展 2、startup脚本抽象kv_store_backends/，提升可维护性 3、添加memcache的metaservice，local service的脚本支持； 4、A2的聚合拉起；A3，A5的聚合、分离拉起local service的能力支持 ## **3. 资料变更** 设计 ## **4. 接口变更** 不涉及 ## **5. 测试结果** 能正常拉起memcache池化，压测请求无报错。 ## **6. CheckList** > PR提交人对以下CheckList自检项进行全量自检，自检通过或不涉及，均修改 [ ] 为 [x] [ ] 代码注释完备 [ ] 正确记录维测日志 [ ] 是否有UT用例 [ ] 若涉及多线程场景，考虑了并发场景，不存在死锁问题 See merge request: Ascend/MindIE-PyMotor!382
1 个月前
common.sh
[refractor]重构motor与memcache的交互逻辑 Co-authored-by: 吕有辉<lvyouhui@huawei.com> # message auto-generated for no-merge-commit merge: !530 merge feat/memcache_ssd into master [refractor]重构motor与memcache的交互逻辑 Created-by: codeDogPro Commit-by: 吕有辉 Merged-by: tobking Description: ## **1. 合入背景** https://gitcode.com/Ascend/MindIE-Motor/issues/353 ## **2. 修改内容** 1、完全重构motor与memcache的对接逻辑，解耦组件的交互 2、local service独立为单独的进程，由daemon拉起，提升拉起速度 ## **3. 资料变更** 涉及 ## **4. 接口变更** 不涉及 ## **5. 测试结果** 使用新拉起方式后A3拉起正常（standalone） HELP motor:prompt_tokens_per_second Prompt tokens per second computed from vllm:prompt_tokens_total counter deltas # TYPE motor:prompt_tokens_per_second gauge motor:prompt_tokens_per_second 0.0 # HELP motor:generation_tokens_per_second Generation tokens per second computed from vllm:generation_tokens_total counter deltas # TYPE motor:generation_tokens_per_second gauge motor:generation_tokens_per_second 0.0 # HELP vllm:prefix_cache_hit_rate Prefix cache hit rate (cached tokens / queried tokens). # TYPE vllm:prefix_cache_hit_rate gauge vllm:prefix_cache_hit_rate 0.0 # HELP motor:active_prefill_workers Number of active prefill instances # TYPE motor:active_prefill_workers gauge motor:active_prefill_workers 1 # HELP motor:active_decode_workers Number of active decode instances # TYPE motor:active_decode_workers gauge motor:active_decode_workers 1 # HELP motor:inactive_prefill_workers Number of inactive prefill instances # TYPE motor:inactive_prefill_workers gauge motor:inactive_prefill_workers 0 # HELP motor:inactive_decode_workers Number of inactive decode instances # TYPE motor:inactive_decode_workers gauge motor:inactive_decode_workers 0 # HELP kv_store_size KV store size in GB (layer=cpu|ssd|all, stat=usage|total) # TYPE kv_store_size gauge kv_store_size{layer="cpu",stat="usage"} 0.0 kv_store_size{layer="cpu",stat="total"} 64.0 kv_store_size{layer="ssd",stat="usage"} 0.0 kv_store_size{layer="ssd",stat="total"} 0.0 kv_store_size{layer="all",stat="usage"} 0.0 kv_store_size{layer="all",stat="total"} 64.0 # HELP kv_store_ratio KV store used ratio 0-1 (layer=cpu|ssd|all, stat=usage_rate) # TYPE kv_store_ratio gauge kv_store_ratio{layer="cpu",stat="usage_rate"} 0.0 kv_store_ratio{layer="ssd",stat="usage_rate"} 0.0 kv_store_ratio{layer="all",stat="usage_rate"} 0.0 # HELP kv_store_keys KV store number of stored keys # TYPE kv_store_keys gauge kv_store_keys 0.0 # HELP kv_store_eviction KV store eviction counters (stat=success|attempts) # TYPE kv_store_eviction gauge kv_store_eviction{stat="success"} 0.0 kv_store_eviction{stat="attempts"} 0.0 # HELP motor:memcache_segment_capacity_bytes Segment total capacity in bytes # TYPE motor:memcache_segment_capacity_bytes gauge motor:memcache_segment_capacity_bytes{segment="rank-0-dram"} 17179869184 motor:memcache_segment_capacity_bytes{segment="rank-1-dram"} 17179869184 # HELP motor:memcache_segment_allocated_bytes Segment allocated bytes # TYPE motor:memcache_segment_allocated_bytes gauge motor:memcache_segment_allocated_bytes{segment="rank-0-dram"} 0 motor:memcache_segment_allocated_bytes{segment="rank-1-dram"} 0 # HELP motor:memcache_total_capacity_bytes Total capacity by medium in bytes # TYPE motor:memcache_total_capacity_bytes gauge motor:memcache_total_capacity_bytes{medium="hbm"} 0 motor:memcache_total_capacity_bytes{medium="dram"} 34359738368 # HELP motor:memcache_allocated_bytes Allocated bytes by medium 压测性能： ╒══════════════════════════╤═════════╤═════════════════╤═════════════════╤═════════════════╤═════════════════╤═════════════════╤═════════════════╤═════════════════╤═════╕ │ Performance Parameters │ Stage │ Average │ Min │ Max │ Median │ P75 │ P90 │ P99 │ N │ ╞══════════════════════════╪═════════╪═════════════════╪═════════════════╪═════════════════╪═════════════════╪═════════════════╪═════════════════╪═════════════════╪═════╡ │ E2EL │ total │ 6848.4 ms │ 4251.6 ms │ 7985.9 ms │ 7590.0 ms │ 7717.7 ms │ 7879.3 ms │ 7975.2 ms │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────┤ │ TTFT │ total │ 422.2 ms │ 253.7 ms │ 733.2 ms │ 369.6 ms │ 486.8 ms │ 637.4 ms │ 723.6 ms │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────┤ │ TPOT │ total │ 14.4 ms │ 14.1 ms │ 14.5 ms │ 14.4 ms │ 14.5 ms │ 14.5 ms │ 14.5 ms │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────┤ │ ITL │ total │ 14.3 ms │ 0.0 ms │ 62.8 ms │ 13.8 ms │ 14.1 ms │ 15.3 ms │ 27.8 ms │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────┤ │ InputTokens │ total │ 1491.5 │ 1460.0 │ 1545.0 │ 1490.5 │ 1499.25 │ 1515.6 │ 1542.06 │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────┤ │ OutputTokens │ total │ 448.375 │ 245.0 │ 512.0 │ 512.0 │ 512.0 │ 512.0 │ 512.0 │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────┤ │ OutputTokenThroughput │ total │ 65.0576 token/s │ 57.6249 token/s │ 68.1729 token/s │ 65.8792 token/s │ 66.6954 token/s │ 67.1812 token/s │ 68.0737 token/s │ 8 │ ╘══════════════════════════╧═════════╧═════════════════╧═════════════════╧═════════════════╧═════════════════╧═════════════════╧═════════════════╧═════════════════╧═════╛ inprocess 模式测试如下： # HELP motor:memcache_segment_capacity_bytes Segment total capacity in bytes # TYPE motor:memcache_segment_capacity_bytes gauge motor:memcache_segment_capacity_bytes{segment="rank-0-dram"} 2147483648 motor:memcache_segment_capacity_bytes{segment="rank-1-dram"} 2147483648 motor:memcache_segment_capacity_bytes{segment="rank-2-dram"} 2147483648 motor:memcache_segment_capacity_bytes{segment="rank-3-dram"} 2147483648 motor:memcache_segment_capacity_bytes{segment="rank-4-dram"} 2147483648 motor:memcache_segment_capacity_bytes{segment="rank-5-dram"} 2147483648 motor:memcache_segment_capacity_bytes{segment="rank-6-dram"} 2147483648 motor:memcache_segment_capacity_bytes{segment="rank-7-dram"} 2147483648 # HELP motor:memcache_segment_allocated_bytes Segment allocated bytes # TYPE motor:memcache_segment_allocated_bytes gauge motor:memcache_segment_allocated_bytes{segment="rank-0-dram"} 12582912 motor:memcache_segment_allocated_bytes{segment="rank-1-dram"} 37748736 motor:memcache_segment_allocated_bytes{segment="rank-2-dram"} 37748736 motor:memcache_segment_allocated_bytes{segment="rank-3-dram"} 25165824 motor:memcache_segment_allocated_bytes{segment="rank-4-dram"} 50331648 motor:memcache_segment_allocated_bytes{segment="rank-5-dram"} 44040192 motor:memcache_segment_allocated_bytes{segment="rank-6-dram"} 56623104 motor:memcache_segment_allocated_bytes{segment="rank-7-dram"} 25165824 # HELP motor:memcache_total_capacity_bytes Total capacity by medium in by 压测性能一致 ``` ╒══════════════════════════╤═════════╤═════════════════╤═════════════════╤═════════════════╤═════════════════╤═════════════════╤════════════════╤═════════════════╤═════╕ │ Performance Parameters │ Stage │ Average │ Min │ Max │ Median │ P75 │ P90 │ P99 │ N │ ╞══════════════════════════╪═════════╪═════════════════╪═════════════════╪═════════════════╪═════════════════╪═════════════════╪════════════════╪═════════════════╪═════╡ │ E2EL │ total │ 6847.9 ms │ 4285.2 ms │ 8056.9 ms │ 7528.6 ms │ 7755.3 ms │ 7959.6 ms │ 8047.2 ms │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼────────────────┼─────────────────┼─────┤ │ TTFT │ total │ 431.8 ms │ 259.8 ms │ 723.0 ms │ 414.1 ms │ 458.9 ms │ 614.0 ms │ 712.1 ms │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼────────────────┼─────────────────┼─────┤ │ TPOT │ total │ 14.4 ms │ 13.9 ms │ 14.7 ms │ 14.4 ms │ 14.7 ms │ 14.7 ms │ 14.7 ms │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼────────────────┼─────────────────┼─────┤ │ ITL │ total │ 14.3 ms │ 0.0 ms │ 71.6 ms │ 13.8 ms │ 14.2 ms │ 15.4 ms │ 24.7 ms │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼────────────────┼─────────────────┼─────┤ │ InputTokens │ total │ 1491.5 │ 1460.0 │ 1545.0 │ 1490.5 │ 1499.25 │ 1515.6 │ 1542.06 │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼────────────────┼─────────────────┼─────┤ │ OutputTokens │ total │ 448.375 │ 245.0 │ 512.0 │ 512.0 │ 512.0 │ 512.0 │ 512.0 │ 8 │ ├──────────────────────────┼─────────┼── See merge request: Ascend/MindIE-Motor!530
25 天前
hccl_tools.py
[refractor]重构motor与memcache的交互逻辑 Co-authored-by: 吕有辉<lvyouhui@huawei.com> # message auto-generated for no-merge-commit merge: !530 merge feat/memcache_ssd into master [refractor]重构motor与memcache的交互逻辑 Created-by: codeDogPro Commit-by: 吕有辉 Merged-by: tobking Description: ## **1. 合入背景** https://gitcode.com/Ascend/MindIE-Motor/issues/353 ## **2. 修改内容** 1、完全重构motor与memcache的对接逻辑，解耦组件的交互 2、local service独立为单独的进程，由daemon拉起，提升拉起速度 ## **3. 资料变更** 涉及 ## **4. 接口变更** 不涉及 ## **5. 测试结果** 使用新拉起方式后A3拉起正常（standalone） HELP motor:prompt_tokens_per_second Prompt tokens per second computed from vllm:prompt_tokens_total counter deltas # TYPE motor:prompt_tokens_per_second gauge motor:prompt_tokens_per_second 0.0 # HELP motor:generation_tokens_per_second Generation tokens per second computed from vllm:generation_tokens_total counter deltas # TYPE motor:generation_tokens_per_second gauge motor:generation_tokens_per_second 0.0 # HELP vllm:prefix_cache_hit_rate Prefix cache hit rate (cached tokens / queried tokens). # TYPE vllm:prefix_cache_hit_rate gauge vllm:prefix_cache_hit_rate 0.0 # HELP motor:active_prefill_workers Number of active prefill instances # TYPE motor:active_prefill_workers gauge motor:active_prefill_workers 1 # HELP motor:active_decode_workers Number of active decode instances # TYPE motor:active_decode_workers gauge motor:active_decode_workers 1 # HELP motor:inactive_prefill_workers Number of inactive prefill instances # TYPE motor:inactive_prefill_workers gauge motor:inactive_prefill_workers 0 # HELP motor:inactive_decode_workers Number of inactive decode instances # TYPE motor:inactive_decode_workers gauge motor:inactive_decode_workers 0 # HELP kv_store_size KV store size in GB (layer=cpu|ssd|all, stat=usage|total) # TYPE kv_store_size gauge kv_store_size{layer="cpu",stat="usage"} 0.0 kv_store_size{layer="cpu",stat="total"} 64.0 kv_store_size{layer="ssd",stat="usage"} 0.0 kv_store_size{layer="ssd",stat="total"} 0.0 kv_store_size{layer="all",stat="usage"} 0.0 kv_store_size{layer="all",stat="total"} 64.0 # HELP kv_store_ratio KV store used ratio 0-1 (layer=cpu|ssd|all, stat=usage_rate) # TYPE kv_store_ratio gauge kv_store_ratio{layer="cpu",stat="usage_rate"} 0.0 kv_store_ratio{layer="ssd",stat="usage_rate"} 0.0 kv_store_ratio{layer="all",stat="usage_rate"} 0.0 # HELP kv_store_keys KV store number of stored keys # TYPE kv_store_keys gauge kv_store_keys 0.0 # HELP kv_store_eviction KV store eviction counters (stat=success|attempts) # TYPE kv_store_eviction gauge kv_store_eviction{stat="success"} 0.0 kv_store_eviction{stat="attempts"} 0.0 # HELP motor:memcache_segment_capacity_bytes Segment total capacity in bytes # TYPE motor:memcache_segment_capacity_bytes gauge motor:memcache_segment_capacity_bytes{segment="rank-0-dram"} 17179869184 motor:memcache_segment_capacity_bytes{segment="rank-1-dram"} 17179869184 # HELP motor:memcache_segment_allocated_bytes Segment allocated bytes # TYPE motor:memcache_segment_allocated_bytes gauge motor:memcache_segment_allocated_bytes{segment="rank-0-dram"} 0 motor:memcache_segment_allocated_bytes{segment="rank-1-dram"} 0 # HELP motor:memcache_total_capacity_bytes Total capacity by medium in bytes # TYPE motor:memcache_total_capacity_bytes gauge motor:memcache_total_capacity_bytes{medium="hbm"} 0 motor:memcache_total_capacity_bytes{medium="dram"} 34359738368 # HELP motor:memcache_allocated_bytes Allocated bytes by medium 压测性能： ╒══════════════════════════╤═════════╤═════════════════╤═════════════════╤═════════════════╤═════════════════╤═════════════════╤═════════════════╤═════════════════╤═════╕ │ Performance Parameters │ Stage │ Average │ Min │ Max │ Median │ P75 │ P90 │ P99 │ N │ ╞══════════════════════════╪═════════╪═════════════════╪═════════════════╪═════════════════╪═════════════════╪═════════════════╪═════════════════╪═════════════════╪═════╡ │ E2EL │ total │ 6848.4 ms │ 4251.6 ms │ 7985.9 ms │ 7590.0 ms │ 7717.7 ms │ 7879.3 ms │ 7975.2 ms │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────┤ │ TTFT │ total │ 422.2 ms │ 253.7 ms │ 733.2 ms │ 369.6 ms │ 486.8 ms │ 637.4 ms │ 723.6 ms │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────┤ │ TPOT │ total │ 14.4 ms │ 14.1 ms │ 14.5 ms │ 14.4 ms │ 14.5 ms │ 14.5 ms │ 14.5 ms │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────┤ │ ITL │ total │ 14.3 ms │ 0.0 ms │ 62.8 ms │ 13.8 ms │ 14.1 ms │ 15.3 ms │ 27.8 ms │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────┤ │ InputTokens │ total │ 1491.5 │ 1460.0 │ 1545.0 │ 1490.5 │ 1499.25 │ 1515.6 │ 1542.06 │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────┤ │ OutputTokens │ total │ 448.375 │ 245.0 │ 512.0 │ 512.0 │ 512.0 │ 512.0 │ 512.0 │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────┤ │ OutputTokenThroughput │ total │ 65.0576 token/s │ 57.6249 token/s │ 68.1729 token/s │ 65.8792 token/s │ 66.6954 token/s │ 67.1812 token/s │ 68.0737 token/s │ 8 │ ╘══════════════════════════╧═════════╧═════════════════╧═════════════════╧═════════════════╧═════════════════╧═════════════════╧═════════════════╧═════════════════╧═════╛ inprocess 模式测试如下： # HELP motor:memcache_segment_capacity_bytes Segment total capacity in bytes # TYPE motor:memcache_segment_capacity_bytes gauge motor:memcache_segment_capacity_bytes{segment="rank-0-dram"} 2147483648 motor:memcache_segment_capacity_bytes{segment="rank-1-dram"} 2147483648 motor:memcache_segment_capacity_bytes{segment="rank-2-dram"} 2147483648 motor:memcache_segment_capacity_bytes{segment="rank-3-dram"} 2147483648 motor:memcache_segment_capacity_bytes{segment="rank-4-dram"} 2147483648 motor:memcache_segment_capacity_bytes{segment="rank-5-dram"} 2147483648 motor:memcache_segment_capacity_bytes{segment="rank-6-dram"} 2147483648 motor:memcache_segment_capacity_bytes{segment="rank-7-dram"} 2147483648 # HELP motor:memcache_segment_allocated_bytes Segment allocated bytes # TYPE motor:memcache_segment_allocated_bytes gauge motor:memcache_segment_allocated_bytes{segment="rank-0-dram"} 12582912 motor:memcache_segment_allocated_bytes{segment="rank-1-dram"} 37748736 motor:memcache_segment_allocated_bytes{segment="rank-2-dram"} 37748736 motor:memcache_segment_allocated_bytes{segment="rank-3-dram"} 25165824 motor:memcache_segment_allocated_bytes{segment="rank-4-dram"} 50331648 motor:memcache_segment_allocated_bytes{segment="rank-5-dram"} 44040192 motor:memcache_segment_allocated_bytes{segment="rank-6-dram"} 56623104 motor:memcache_segment_allocated_bytes{segment="rank-7-dram"} 25165824 # HELP motor:memcache_total_capacity_bytes Total capacity by medium in by 压测性能一致 ``` ╒══════════════════════════╤═════════╤═════════════════╤═════════════════╤═════════════════╤═════════════════╤═════════════════╤════════════════╤═════════════════╤═════╕ │ Performance Parameters │ Stage │ Average │ Min │ Max │ Median │ P75 │ P90 │ P99 │ N │ ╞══════════════════════════╪═════════╪═════════════════╪═════════════════╪═════════════════╪═════════════════╪═════════════════╪════════════════╪═════════════════╪═════╡ │ E2EL │ total │ 6847.9 ms │ 4285.2 ms │ 8056.9 ms │ 7528.6 ms │ 7755.3 ms │ 7959.6 ms │ 8047.2 ms │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼────────────────┼─────────────────┼─────┤ │ TTFT │ total │ 431.8 ms │ 259.8 ms │ 723.0 ms │ 414.1 ms │ 458.9 ms │ 614.0 ms │ 712.1 ms │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼────────────────┼─────────────────┼─────┤ │ TPOT │ total │ 14.4 ms │ 13.9 ms │ 14.7 ms │ 14.4 ms │ 14.7 ms │ 14.7 ms │ 14.7 ms │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼────────────────┼─────────────────┼─────┤ │ ITL │ total │ 14.3 ms │ 0.0 ms │ 71.6 ms │ 13.8 ms │ 14.2 ms │ 15.4 ms │ 24.7 ms │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼────────────────┼─────────────────┼─────┤ │ InputTokens │ total │ 1491.5 │ 1460.0 │ 1545.0 │ 1490.5 │ 1499.25 │ 1515.6 │ 1542.06 │ 8 │ ├──────────────────────────┼─────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼────────────────┼─────────────────┼─────┤ │ OutputTokens │ total │ 448.375 │ 245.0 │ 512.0 │ 512.0 │ 512.0 │ 512.0 │ 512.0 │ 8 │ ├──────────────────────────┼─────────┼── See merge request: Ascend/MindIE-Motor!530
25 天前
set_env_docker.py
[feature] Docker-only PD 混部推理部署支持 Co-authored-by: Jechin<yuzechen1@huawei.com> # message auto-generated for no-merge-commit merge: !469 merge docs/docker-pd-aggregation-deployment into master [feature] Docker-only PD 混部推理部署支持 Created-by: Jechin Commit-by: Jechin Merged-by: towncharlie Description: ## **1. 合入背景** Fixes [#263](https://gitcode.com/Ascend/MindIE-PyMotor/issues/263) 当前 Docker-only 部署指南仅覆盖 PD 分离，缺少 PD 混部的单容器、多容器端到端部署方式；原有示例在单容器 union 实例拉起、环境变量注入、端口偏移和可选 KV 端口处理方面也不完整。本 PR 统一整理两种 PD 模式的部署指导，并补齐 PD 混部所需的启动与配置支持。 ## **2. 修改内容** 1. **统一 Docker 部署指南** - 将 PD 分离与 PD 混部整合到单容器、多容器两篇指南中，按“PD 分离在前、PD 混部在后”的顺序说明配置和启动差异。 - 增加 examples 获取方式、推荐端口规划、Coordinator 对外端口映射、服务验证及 A5 环境说明。 - 启动示例增加 CONFIGMAP_PATH 和 WEIGHT_MOUNT_PATH 挂载，模型权重使用只读挂载。 2. **支持单容器 union 实例拉起** - all_combine_in_single_container.sh 根据 motor_engine_union_config 识别 PD 混部模式。 - 校验 hybrid_instances_num，按实例设置 ROLE=union、INDEX、JOB_NAME 和 ranktable 路径并拉起 NodeManager。 3. **支持 union 环境变量注入** - set_env_docker.py 支持从 motor_engine_union_config 解析 engine 类型和模型名称。 - 为单容器、多容器启动脚本生成 set_union_env；未配置 motor_engine_union_env 时兼容复用 Prefill 环境配置。 4. **完善单容器混部端口与设备分配** - NodeManager 根据 union 实例索引计算管理端口、业务端口、设备及 DP RPC 端口偏移。 - EngineService 仅在端口配置有效时传递 --kv-port 和 --dp-rpc-port，避免生成 --kv-port None 导致 Engine Server 参数解析失败。 - EndpointConfig 将偏移后的 dp_rpc_port 应用到 union 并行配置，避免多 union 实例复用同一 DP RPC 端口。 5. **补充自动化测试** - 覆盖 union 环境注入、单容器 union 端口与设备偏移、混部/分离配置摘要、可选 KV 端口省略及 union DP RPC 端口覆盖。 6. **文档站点配置** - 修正 mkdocs.yml 中的站点地址。 ## **3. 资料变更** - docs/zh/user_guide/deployment/docker/single_container.md：统一单容器 PD 分离/混部部署流程，补充端口映射、挂载与服务验证。 - docs/zh/user_guide/deployment/docker/multi_container.md：统一多容器 PD 分离/混部部署流程，补充端口规划、挂载与服务验证。 - mkdocs.yml：更新文档站点地址。 ## **4. 接口变更** 不涉及跨代码仓 API 变更。Docker 启动流程新增对现有 motor_engine_union_config、motor_engine_union_env 和 hybrid_instances_num 配置的支持；文档启动示例新增 WEIGHT_MOUNT_PATH 变量及 31015:1025 服务端口映射。 ## **5. 测试结果** - 相关 NodeManager、EndpointConfig 测试：108 个用例通过。 - 相关文件 pre-commit 检查全部通过。 - 按部署文档验证服务能够成功拉起。 ## **6. CheckList** [x] 代码注释完备 [x] 正确记录维测日志 [x] 是否有UT用例 [x] 若涉及多线程场景，考虑了并发场景，不存在死锁问题 See merge request: Ascend/MindIE-PyMotor!469
1 个月前