支持引擎原生拉起：删除Engine Server冗余代码-MindIE-Motor-AtomGit

M.agent/skills/motor-dev/SKILL.md+0-4

	@@ -104,7 +104,6 @@ The skill body only covers core workflow. **When touching a specific module, rea
104	\| `motor/coordinator/` \| `references/coordinator.md` \|	104	\| `motor/coordinator/` \| `references/coordinator.md` \|
105	\| `motor/coordinator/metrics/` \| `references/metrics.md` \|	105	\| `motor/coordinator/metrics/` \| `references/metrics.md` \|
106	\| `motor/node_manager/` \| `references/nodeman.md` \|	106	\| `motor/node_manager/` \| `references/nodeman.md` \|
107	-\| `motor/engine_server/` \| `references/engine-server.md` \|
108	\| `tests/e2e/` \| `references/e2e-testing.md` \|	107	\| `tests/e2e/` \| `references/e2e-testing.md` \|
109	\| `motor/kv_conductor/` \| `references/kv-conductor.md` \|	108	\| `motor/kv_conductor/` \| `references/kv-conductor.md` \|
110	\| General testing / test options \| `references/testing-guide.md` \|	109	\| General testing / test options \| `references/testing-guide.md` \|
	@@ -153,9 +152,6 @@ bash tests/run_tests.sh tests/coordinator/
153	# NodeManager	152	# NodeManager
154	bash tests/run_tests.sh tests/node_manager/	153	bash tests/run_tests.sh tests/node_manager/
155		154
156	-# EngineServer
157	-bash tests/run_tests.sh tests/engine_server/
158	-
159	# E2E	155	# E2E
160	bash tests/run_tests.sh tests/e2e/	156	bash tests/run_tests.sh tests/e2e/
161	```	157	```

M.agent/skills/motor-dev/references/coordinator.md+8-5

D.agent/skills/motor-dev/references/engine-server.md+0-230

	@@ -1,230 +0,0 @@
1	-# EngineServer Module — Architecture & Implementation
2	-
3	-## Two-Endpoint Pattern
4	-
5	-Each `engine_server` is a separate OS process spawned by NodeManager's Daemon. It wraps the underlying engine framework (vLLM or SGLang) behind two FastAPI HTTP servers.
6	-
7	-``` text
8	-EngineServer Process
9	-│
10	-├── MgmtEndpoint (FastAPI on :mgmt_port)
11	-│ GET /status → engine health (polls InferEndpoint /health + SimInference)
12	-│ GET /metrics → Prometheus multiprocess metrics
13	-│
14	-└── InferEndpoint (FastAPI on :port)
15	- POST /v1/chat/completions → OpenAI chat completions (via dispatch adapter)
16	- POST /v1/completions → OpenAI completions (via dispatch adapter)
17	- POST /v1/metaserver → KV-transfer-aware dispatch control
18	- POST /v1/dispatch/stop → dispatch stop propagation (HTTP 499 to peer)
19	- GET /v1/models → model listing
20	- GET /health → app.state.health_checker()
21	- POST /suspend → snapshot: suspend engine to disk (model_save_path)
22	- POST /device_unlock → snapshot: unlock devices after suspend
23	- POST /resume → snapshot: resume engine (data_parallel_master_ip, model_path)
24	- POST /start_profile → start profiling (only when profiler is configured)
25	- POST /stop_profile → stop profiling (only when profiler is configured)
26	-```
27	-
28	-Why two ports: Management operations (health checks, metrics scraping) are separated from inference traffic. NodeManager polls mgmt_port for health without interfering with inference requests; Prometheus scrapes mgmt_port for metrics.
29	-
30	-## Engine Abstraction Layer
31	-
32	-ABC + factory pattern supporting multiple engine backends (vLLM primary, SGLang secondary):
33	-
34	-``` text
35	-IConfig (ABC) → VLLMConfig / SGLangConfig — CLI arg generation from deploy config
36	-Engine (ABC) → VLLMEngine / SGLangEngine — engine client creation + lifecycle
37	-Endpoint (ABC) → MgmtEndpoint / InferEndpoint — HTTP server lifecycle
38	-InferEndpoint.get_lifespan() → VLLMEndpoint / SGLangEndpoint — engine-specific startup
39	-```
40	-
41	-The transitional EngineServer reuses the CLI configuration adapters owned by
42	-`motor/node_manager/core/services/native_engine/backends/`. Process launch and supervision for the
43	-new direct-native path remain in `NativeEngineService`; the classes in this document describe only
44	-the legacy in-process EngineServer path.
45	-
46	-### Config → CLI Args Pipeline
47	-
48	-1. `ConfigFactory` looks up the engine's `IConfig` in an explicit `_ENGINE_CONFIG_MAP` dict (`vllm` / `sglang` → module paths); `parse()` runs the full `initialize()` → `convert()` → `validate()` pipeline
49	-2. `initialize()` — sets up engine-specific config (DP addresses, KV transfer, D2D config)
50	-3. `convert()` — replaces `sys.argv` with the generated args, builds `FlexibleArgumentParser`, calls `make_arg_parser()` + `parse_args()`
51	-4. `validate()` — for vLLM: calls `validate_parsed_serve_args()` on the parsed args
52	-
53	-Field mapping (`_get_default_mapping()` in `vllm_config.py` — 9 entries):
54	-
55	-```python
56	-{
57	- 'model_path': 'model',
58	- 'model_name': 'served_model_name',
59	- 'npu_mem_utils': 'gpu_memory_utilization',
60	- 'dp_size': 'data_parallel_size',
61	- 'tp_size': 'tensor_parallel_size',
62	- 'pp_size': 'pipeline_parallel_size',
63	- 'enable_ep': 'enable_expert_parallel',
64	- 'dp_rpc_port': 'data_parallel_rpc_port',
65	- 'cp_kv_cache_interleave_size': 'cp_kv_cache_interleave_size',
66	-}
67	-```
68	-
69	-Other fields are passed through as `--field value` with type-aware serialization (`bool`: `--flag` presence; `list`: repeated values; `dict`: JSON string).
70	-
71	-### PD Disaggregation (vLLM)
72	-
73	-In PD mode, `VLLMConfig.initialize()` sets up KV transfer based on the configured connector — there is no longer any "TransferEngine" concept:
74	-
75	-- MooncakeConnector (single, default): role sets `kv_role` (producer on prefill, consumer on decode), `engine_id` = instance_id, and injects prefill/decode parallel config into `kv_connector_extra_config`
76	-- MultiConnector: `connectors[0]` is processed as the transport (mooncake-style keys), `connectors[1]` as the store — may be a `UCMConnector` (kept `kv_both`, no injected rpc port) or `MoonCakeStoreV1` / `AscendStoreConnector` (injected `mooncake_rpc_port` / `lookup_rpc_port` = instance_id)
77	-- UCMConnector standalone: only supported in the `union` role (centralized-PD topology); prefill/decode raise a loud error instead of silently injecting mooncake-style keys
78	-- KV transfer config is serialized to JSON and passed as `--kv-transfer-config` CLI arg
79	-- D2D (decode-to-decode) config is set up separately via `_process_d2d_config()`
80	-
81	-### Lifespan-Based Initialization
82	-
83	-Heavy engine initialization (model loading, weight allocation) happens inside FastAPI's `lifespan` context manager — NOT at import time:
84	-
85	-```python
86	-@asynccontextmanager
87	-async def _vllm_lifespan(app: FastAPI):
88	- # Startup: build engine config, create AsyncLLM engine (loads model weights)
89	- engine_config = vllm.AsyncEngineArgs.from_cli_args(args)
90	- vllm_endpoint_config = engine_config.create_engine_config(
91	- usage_context=UsageContext.OPENAI_API_SERVER,
92	- headless=headless,
93	- )
94	- engine_client = AsyncLLM.from_vllm_config(
95	- vllm_config=vllm_endpoint_config,
96	- usage_context=UsageContext.OPENAI_API_SERVER,
97	- )
98	- app.state.engine_client = engine_client
99	- app.state.openai_serving_chat = OpenAIServingChat(...)
100	- yield # Server is ready
101	- # Shutdown: cleanup engine
102	- engine_client.shutdown()
103	-```
104	-
105	-Headless PCP follower mode: when `nnodes > 1` and `node_rank_within_dp > 0`, `_run_vllm()` skips the engine core entirely and starts only a `MultiprocExecutor` (workers only, following vLLM's `run_headless()` pattern); the headless flag also disables virtual inference.
106	-
107	-Why lifespan: Uvicorn starts the HTTP listener immediately on `uvicorn.run()`. If model loading happened at import time, the port wouldn't accept connections for 30-120 seconds (model load time), causing NodeManager's health probes to fail. With lifespan, Uvicorn binds the socket first, then loads the model — health probes get a connection-refused or 503 until loading completes, which NodeManager handles via the grace period.
108	-
109	-## Dispatch Adapter (Request Path Core)
110	-
111	-Every inference request on InferEndpoint flows through the dispatch adapter (`core/dispatch_adapter/`) before reaching the engine:
112	-
113	-``` text
114	-InferEndpoint handler
115	- → create_dispatch_adapter(config) — factory picks engine-specific subclass
116	- → adapt_request_body(body) — attach MotorDispatch, rewrite request body
117	- → maybe_prepare_response / should_finish_prepared_response
118	- → engine request (chat/completions/metaserver)
119	- → normalize_response / normalize_stream_chunk
120	- → error mapping: map_serving_exception / map_engine_error / map_stream_error
121	- → dispatch control: is_dispatch_stopped / stop_peer / finish_dispatch
122	-```
123	-
124	-Responsibilities:
125	-
126	-- Dispatch attach: wraps the request with `MotorDispatch` (prefill→decode handoff, request-body rewrite for the target role)
127	-- Stop propagation: `POST /v1/dispatch/stop` (`handle_stop`) stops a dispatch; peers are stopped with HTTP 499, in-flight requests are aborted and stream chunks normalized
128	-- Response normalization: `normalization.py` adapts engine responses/stream chunks to the unified OpenAI schema (e.g. completions-style bodies/chunks lifted to chat format, token_id stripping, request-id synthesis)
129	-- Error mapping: engine exceptions are mapped to serving HTTP errors (and vice versa) via registered error handlers
130	-- KV-aware metaserver: `POST /v1/metaserver` requests go through `prepare_metaserver_request` (engine_body + dispatch + KV params validation)
131	-
132	-Files: `base.py` (534, `DispatchAdapter` + `DispatchAttemptRegistry` + stop client), `vllm_adapter.py` (340, `VLLMDispatchAdapter`), `sglang_adapter.py` (49, `SGLangDispatchAdapter`), `normalization.py` (220), `factory.py` (23, `create_dispatch_adapter`).
133	-
134	-## Health Monitoring Stack (4 Layers)
135	-
136	-``` text
137	-Layer 1: NodeManager.HeartbeatManager
138	- → GET http://{ip}:{mgmt_port}/status (every heartbeat_interval, default 3s)
139	-
140	-Layer 2: MgmtEndpoint.HealthCollector
141	- → GET http://127.0.0.1:{port}/health (async HTTP, same process)
142	-
143	-Layer 3: InferEndpoint /health handler
144	- → app.state.health_checker() (infer_endpoint.py)
145	- → engine_client.check_health() (vLLM: AsyncLLM.check_health; SGLang: always True)
146	-
147	-Layer 4: SimInference (proactive health)
148	- → virtual inference request: POST /v1/completions {"prompt": "1", "max_tokens": 1}
149	- → npu-smi subprocess: poll NPU AICore usage percentage
150	- → Logic: if AICore usage is low AND virtual requests fail for max_failure_count
151	- consecutive times → ABNORMAL
152	- → Details:
153	- max_failure_count default 6 (HealthCheckConfig.max_failure_count)
154	- enable_virtual_inference default False — forced off for SGLang, headless
155	- mode, and DP rank != 0 (only DP0 performs virtual inference)
156	- 180s warmup stage (first virtual request timeout), then 5s interval —
157	- stretched to 20s when AI Cube peak >= 80%
158	-```
159	-
160	-SimInference rationale: An engine can pass basic health checks (process alive, port listening) but be unable to perform inference (e.g., NPU hang, driver issue, memory corruption). SimInference catches "silent failures" by running actual inference and monitoring hardware utilization.
161	-
162	-## Snapshot Support
163	-
164	-- `core/snapshot_sentinel.py` (200) — `SnapshotSentinel` thread: waits for InferEndpoint to be healthy, reaches the checkpoint, then drives `POST /suspend` / `POST /resume` against the engine
165	-- `core/snapshot_monitor.py` (51) — `SnapshotMonitor` (ThreadSafeSingleton) tracks suspend/unlock/resume completion states
166	-- Routes: `POST /suspend` (`model_save_path` query param), `POST /device_unlock`, `POST /resume` (`data_parallel_master_ip` + `model_path`); each 501/400s when the engine lacks `suspend`/`resume`/`device_unlock` support
167	-
168	-## Cross-Node PCP (nnodes > 1)
169	-
170	-- NodeManager injects `--node-rank` and `--master-dp-ip` into the engine command line (see nodeman.md)
171	-- In the engine, `--node-rank`/`--master-dp-ip` are parsed into `EndpointConfig.node_rank` / `master_dp_ip` and wired into the vLLM parallel config
172	-- Headless follower nodes (`node_rank_within_dp > 0`) run `MultiprocExecutor` workers only — no EngineCore, no AsyncLLM
173	-
174	-## TLS Support
175	-
176	-- `infer_tls_config` (InferEndpoint) and `mgmt_tls_config` (MgmtEndpoint) — when `enable_tls`, uvicorn gets an SSL context from `CertUtil.create_ssl_context(...)` and serves `https://`
177	-- NodeManagerAPI serves TLS from the same `mgmt_tls_config`
178	-
179	-## Key Files
180	-
181	-\| File \| Role \|
182	-\|------\|------\|
183	-\| `motor/engine_server/cli/main.py` \| Entry point: CLI arg parsing, factory wiring, start Mgmt+Infer endpoints \|
184	-\| `motor/node_manager/core/services/native_engine/backends/base.py` \| Shared `IConfig` ABC used by the transitional EngineServer and native backends \|
185	-\| `motor/engine_server/core/engine.py` \| `Engine` ABC: `launch()`, `shutdown()` \|
186	-\| `motor/engine_server/core/endpoint.py` \| `Endpoint` ABC: `run()`, uvicorn lifecycle \|
187	-\| `motor/engine_server/core/infer_endpoint.py` \| `InferEndpoint`: FastAPI app, uvicorn, route registration, lifespan, dispatch adapter wiring, TLS \|
188	-\| `motor/engine_server/core/mgmt_endpoint.py` \| `MgmtEndpoint`: `/status` + `/metrics`, HealthCollector + SimInference, TLS \|
189	-\| `motor/engine_server/core/health_collector.py` \| Async HTTP health polling (calls InferEndpoint `/health`) \|
190	-\| `motor/engine_server/core/sim_inference.py` \| Virtual inference requests + `npu-smi` AICore monitoring \|
191	-\| `motor/engine_server/core/dispatch_adapter/` \| Dispatch adapter subsystem: `base.py` (534), `vllm_adapter.py` (340), `normalization.py` (220), `sglang_adapter.py` (49), `factory.py` (23) \|
192	-\| `motor/engine_server/core/snapshot_sentinel.py` \| `SnapshotSentinel` thread: checkpoint wait + suspend/resume driving \|
193	-\| `motor/engine_server/core/snapshot_monitor.py` \| `SnapshotMonitor`: suspend/unlock/resume completion states \|
194	-\| `motor/node_manager/core/services/native_engine/backends/vllm/config.py` \| `VLLMConfig`: field mapping, DP address, KV transfer, D2D config \|
195	-\| `motor/engine_server/core/vllm/vllm_engine.py` \| `VLLMEngine`: `AsyncEngineArgs.from_cli_args` + `AsyncLLM.from_vllm_config`, headless PCP follower \|
196	-\| `motor/engine_server/core/vllm/vllm_endpoint.py` \| `VLLMEndpoint`: `_vllm_lifespan` context manager, route init \|
197	-\| `motor/engine_server/core/vllm/vllm_openai_compat.py` \| OpenAI-compat shims (model lists, request mapping) for the vLLM backend \|
198	-\| `motor/node_manager/core/services/native_engine/backends/sglang/config.py` \| `SGLangConfig`: native CLI conversion and validation \|
199	-\| `motor/engine_server/core/sglang/` \| Transitional `SGLangEngine` and `SGLangEndpoint` \|
200	-\| `motor/node_manager/core/services/native_engine/config_factory.py` \| `ConfigFactory`: explicit `_ENGINE_CONFIG_MAP` + `parse()` (initialize→convert→validate) \|
201	-\| `motor/engine_server/factory/endpoint_factory.py` \| InferEndpoint loading by engine name \|
202	-\| `motor/config/endpoint.py` \| `EndpointConfig`: engine_type, host, port, mgmt_port, role, dp_rank, master_dp_ip, node_rank, d2d_peer_ips, snapshot_metadata, deploy_config; `HealthCheckConfig`: timeout, retry attempts, npu_usage_threshold, enable_virtual_inference, max_failure_count \|
203	-
204	-## Development Rules
205	-
206	-### Adding a New Engine Backend
207	-
208	-To add support for a new engine (e.g., "trtllm"):
209	-
210	-1. Create `motor/node_manager/core/services/native_engine/backends/{engine}/config.py`
211	-2. Implement the engine config adapter as an `IConfig` subclass and register it in `ConfigFactory._ENGINE_CONFIG_MAP`
212	-3. For transitional EngineServer support, create `motor/engine_server/core/{engine}/`
213	-4. Implement `{engine}_engine.py`: subclass `Engine`, wrap engine client creation in `launch()`
214	-5. Implement `{engine}_endpoint.py`: subclass `InferEndpoint`, implement `get_lifespan()` + `init_request_handlers()`
215	-6. Register the transitional endpoint in `EndpointFactory`
216	-
217	-### Other Rules
218	-
219	-- New management endpoints → add routes to MgmtEndpoint; new inference endpoints → add routes to InferEndpoint (inference routes should go through the dispatch adapter)
220	-- Config flattening: `_flatten_config()` merges engine > model > parallel (engine takes precedence for conflicting keys). Add new fields to the appropriate source config class.
221	-- PD disaggregation: KV transfer config is set up during `IConfig.initialize()` based on the endpoint's role (prefill=producer, decode=consumer) and the selected connector (Mooncake/Multi/UCM/AscendStore).
222	-- Device pinning: handled by NodeManager via `ASCEND_RT_VISIBLE_DEVICES` env var. EngineServer must NOT manage device assignment — it inherits visibility from the parent process.
223	-- Prometheus multiprocess: `PROMETHEUS_MULTIPROC_DIR` must be set in `main()` before any metric import. Each engine_server writes its own metrics file to this directory.
224	-- Port ranges: `EndpointConfig.validate()` validates business and management ports before the engine process is spawned.
225	-
226	-## Testing
227	-
228	-```bash
229	-bash tests/run_tests.sh tests/engine_server/
230	-```

M.agent/skills/motor-dev/references/metrics.md+1-1

	@@ -75,7 +75,7 @@ Singleton (`ThreadSafeSingleton`) with a background daemon thread:
75	``` text	75	``` text
76	Periodic collect loop (every N seconds):	76	Periodic collect loop (every N seconds):
77		77
78	- 1. HTTP GET /metrics from every engine endpoint (EngineServerApiClient)	78	+ 1. HTTP GET /metrics from every native engine endpoint (`NativeEngineApiClient`)
79	2. Parse Prometheus exposition format → list[Metric]	79	2. Parse Prometheus exposition format → list[Metric]
80		80
81	- Manual parser (not prometheus_client library)	81	- Manual parser (not prometheus_client library)

M.agent/skills/motor-dev/references/nodeman.md+23-15

	@@ -71,7 +71,7 @@ Controller sends `POST /node-manager/start` with `StartCmdMsg`:
71	``` text	71	``` text
72	{	72	{
73	instance_id, job_name, role,	73	instance_id, job_name, role,
74	- endpoints: [{id, ip, business_port, mgmt_port, dp_rank, headless, ...}],	74	+ endpoints: [{id, ip, business_port, mgmt_port, bootstrap_port, dp_rank, headless, ...}],
75	master_dp_ip, node_rank, d2d_peer_ips, ranktable	75	master_dp_ip, node_rank, d2d_peer_ips, ranktable
76	}	76	}
77	```	77	```
	@@ -113,10 +113,11 @@ enabled, `MC_USE_IPV6=1` is supplied unless the environment already defines it.
113	\| `VllmBackend` \| Builds `vllm serve`; Native P/D only accepts the supported HANDOFF profile \|	113	\| `VllmBackend` \| Builds `vllm serve`; Native P/D only accepts the supported HANDOFF profile \|
114	\| `SGLangBackend` \| Builds `python3 -m sglang.launch_server`; encode role is rejected \|	114	\| `SGLangBackend` \| Builds `python3 -m sglang.launch_server`; encode role is rejected \|
115		115
116	-The backend configuration adapters flatten engine-specific JSON into native CLI arguments. They	116	+The backend configuration adapters initialize and flatten engine-specific JSON into native CLI
117	-must call the engine configuration converter and validator before returning a launch specification.	117	+arguments. Final argument validation is delegated to the native engine process; NodeManager does
118	-`None` values are omitted from CLI arguments. SGLang receives `enable-metrics=true` because	118	+not import engine parser implementations. `None` values are omitted from CLI arguments. SGLang
119	-Coordinator metrics and PreStop drain depend on the native metrics endpoint.	119	+receives `enable-metrics=true` because Coordinator metrics and PreStop drain depend on the native
		120	+metrics endpoint.
120		121
121	For SGLang P/D roles, the backend sets `disaggregation-mode=prefill\|decode`; union uses	122	For SGLang P/D roles, the backend sets `disaggregation-mode=prefill\|decode`; union uses
122	`disaggregation-mode=null`. Multi-node configurations validate `nnodes` and require	123	`disaggregation-mode=null`. Multi-node configurations validate `nnodes` and require
	@@ -195,14 +196,19 @@ Records are removed after cleanup so concurrent status reads cannot report a sto
195		196
196	### Snapshot Boundary	197	### Snapshot Boundary
197		198
198	-The Native Runtime path currently does not support container snapshot suspend/resume or native	199	+Native Snapshot is implemented for vLLM only. Enabling snapshot with another engine type is rejected
199	-engine restore. `snapshot_config.enable_snapshot=true` is rejected during NodeManager config	200	+during NodeManager config validation. Device snapshot save and restore are owned by the engine
200	-validation, and `snapshot_metadata_path` is not consumed by Native Runtime.	201	+image; NodeManager does not call `/suspend`, `/device_unlock`, or `/resume`.
201		202
202	-Legacy snapshot helpers and EngineServer compatibility code remain in the repository for the	203	+When `enable_snapshot` is true, NodeManager only:
203	-transition period, but they are not a supported Native Runtime launch path. Do not add new native	204	+
204	-runtime behavior that depends on `engine_server` management endpoints or snapshot metadata until a	205	+1. Refreshes framework state around restore (`job_name`, `pod_ip`, Controller DNS) so the restored
205	-separate runtime contract is defined.	206	+ pod can register with Controller.
		207	+2. Prepares snapshot metadata (`model_save_path`, `model_load_path`, `data_parallel_master_ip`).
		208	+3. Observes engine readiness and the host-side `checkpoint` marker to gate heartbeats and
		209	+ readiness.
		210	+
		211	+SGLang remains unsupported until the engine image provides the same snapshot capability.
206		212
207	## Key Files	213	## Key Files
208		214
	@@ -221,10 +227,10 @@ separate runtime contract is defined.
221	\| `motor/node_manager/core/services/registry.py` \| Service registration and backend discovery \|	227	\| `motor/node_manager/core/services/registry.py` \| Service registration and backend discovery \|
222	\| `motor/node_manager/core/services/memcache/` \| Optional KV-store service implementation \|	228	\| `motor/node_manager/core/services/memcache/` \| Optional KV-store service implementation \|
223	\| `motor/node_manager/core/heartbeat_manager.py` \| Native state polling, status mapping, heartbeat and suicide threshold \|	229	\| `motor/node_manager/core/heartbeat_manager.py` \| Native state polling, status mapping, heartbeat and suicide threshold \|
224	-\| `motor/node_manager/core/engine_manager.py` \| Controller registration, StartCmdMsg validation, ranktable and legacy transition hooks \|	230	+\| `motor/node_manager/core/engine_manager.py` \| Controller registration, StartCmdMsg validation, ranktable and snapshot metadata/restore helpers \|
225	\| `motor/node_manager/api_client/controller_api_client.py` \| Controller register, reregister and heartbeat HTTP client \|	231	\| `motor/node_manager/api_client/controller_api_client.py` \| Controller register, reregister and heartbeat HTTP client \|
226	\| `motor/node_manager/core/fault_reporter.py` \| Optional native engine software-fault polling \|	232	\| `motor/node_manager/core/fault_reporter.py` \| Optional native engine software-fault polling \|
227	-\| `motor/config/node_manager.py` \| NodeManager schema, endpoint derivation, ports and snapshot validation \|	233	+\| `motor/config/node_manager.py` \| NodeManager schema, endpoint derivation, ports and vLLM-only snapshot validation \|
228		234
229	## Port and Address Rules	235	## Port and Address Rules
230		236
	@@ -250,7 +256,9 @@ separate runtime contract is defined.
250	(`READY`/`NORMAL`).	256	(`READY`/`NORMAL`).
251	- New engine backends should add a backend/config package and tests without coupling `Daemon` to the	257	- New engine backends should add a backend/config package and tests without coupling `Daemon` to the
252	engine-specific CLI.	258	engine-specific CLI.
253	-- Native Runtime snapshot support is intentionally disabled until its lifecycle contract is defined.	259	+- Native Runtime snapshot support is vLLM-only. NodeManager must not orchestrate engine snapshot
		260	+ HTTP APIs; keep metadata preparation, registration refresh, and status observation on the
		261	+ NodeManager side.
254		262
255	## Testing	263	## Testing
256		264

M.agent/skills/motor-dev/references/testing-guide.md+0-1

	@@ -87,6 +87,5 @@ bash tests/run_tests.sh tests/config/ # Config changes
87	bash tests/run_tests.sh tests/controller/ # Controller	87	bash tests/run_tests.sh tests/controller/ # Controller
88	bash tests/run_tests.sh tests/coordinator/ # Coordinator	88	bash tests/run_tests.sh tests/coordinator/ # Coordinator
89	bash tests/run_tests.sh tests/node_manager/ # NodeManager	89	bash tests/run_tests.sh tests/node_manager/ # NodeManager
90	-bash tests/run_tests.sh tests/engine_server/ # EngineServer
91	bash tests/run_tests.sh tests/e2e/ # E2E integration	90	bash tests/run_tests.sh tests/e2e/ # E2E integration
92	```	91	```

MAGENTS.md+3-3

Mdocs/zh/.nav.yaml+0-3

	@@ -40,7 +40,6 @@ nav:
40	- ScaleP2D 故障恢复: user_guide/features/fault_tolerance/scale_p2d.md	40	- ScaleP2D 故障恢复: user_guide/features/fault_tolerance/scale_p2d.md
41	- 容器快照: user_guide/features/container_snapshot.md	41	- 容器快照: user_guide/features/container_snapshot.md
42	- D2D 权重加载加速: user_guide/features/startup_acceleration.md	42	- D2D 权重加载加速: user_guide/features/startup_acceleration.md
43	- - 虚推健康探测: user_guide/features/sim_inference.md
44	- 故障场景重调度: user_guide/features/fault_tolerance/rescheduler.md	43	- 故障场景重调度: user_guide/features/fault_tolerance/rescheduler.md
45	- 精度检测: user_guide/features/precision_detection.md	44	- 精度检测: user_guide/features/precision_detection.md
46	- 服务维护:	45	- 服务维护:
	@@ -53,7 +52,6 @@ nav:
53	- 管理接口: user_guide/api/management_interfaces.md	52	- 管理接口: user_guide/api/management_interfaces.md
54	- 指标接口: user_guide/api/metrics_interfaces.md	53	- 指标接口: user_guide/api/metrics_interfaces.md
55	- 服务运行状态观测接口: user_guide/api/observability_interface.md	54	- 服务运行状态观测接口: user_guide/api/observability_interface.md
56	- - 容器快照内部接口: user_guide/api/engine_server_interfaces.md
57	- 设计文档:	55	- 设计文档:
58	- 概述: design/README.md	56	- 概述: design/README.md
59	- PD 分离: design/pd_disaggregation.md	57	- PD 分离: design/pd_disaggregation.md
	@@ -75,5 +73,4 @@ nav:
75	- 组件设计:	73	- 组件设计:
76	- Controller: developer_guide/components/controller.md	74	- Controller: developer_guide/components/controller.md
77	- Coordinator: developer_guide/components/coordinator.md	75	- Coordinator: developer_guide/components/coordinator.md
78	- - Engine Server: developer_guide/components/engine_server.md
79	- Node Manager: developer_guide/components/node_manager.md	76	- Node Manager: developer_guide/components/node_manager.md

Mdocs/zh/architecture.md+4-7

	@@ -3,10 +3,10 @@
3		3
4	## MindIE Motor简介	4	## MindIE Motor简介
5		5
6	-MindIE Motor 是面向大语言模型（LLM）分布式推理，如PD分离推理（Prefill 与 Decode 阶段分离）的请求调度框架。它通过开放、可扩展的推理服务化平台架构，向下对接 [vLLM-Ascend](https://github.com/vllm-project/vllm-ascend)，旨在满足大语言模型的高性能推理需求。	6	+MindIE Motor 是面向大语言模型（LLM）分布式推理，如PD分离推理（Prefill 与 Decode 阶段分离）的请求调度框架。它通过开放、可扩展的推理服务化平台架构，对接 [vLLM-Ascend](https://github.com/vllm-project/vllm-ascend) 和 SGLang 原生运行时，旨在满足大语言模型的高性能推理需求。
7		7
8	>[!NOTE]说明	8	>[!NOTE]说明
9	->原 MindIE PyMotor 代码仓自 3.1.0 版本起更名为 MindIE Motor，后续版本将沿用该命名。其软件定位与基本功能保持不变，兼容 vLLM-Ascend 推理引擎。	9	+>原 MindIE PyMotor 代码仓自 3.1.0 版本起更名为 MindIE Motor，后续版本将沿用该命名。其软件定位与基本功能保持不变，当前以 vLLM-Ascend 为主，并支持 SGLang 原生引擎接入。
10		10
11	### 核心能力	11	### 核心能力
12		12
	@@ -55,19 +55,16 @@ MindIE Motor核心组件定义如下：
55	- LogCollector：k8s日志收集脚本。	55	- LogCollector：k8s日志收集脚本。
56	- BootHelper：容器启动脚本，自动配置环境变量。	56	- BootHelper：容器启动脚本，自动配置环境变量。
57		57
58	-### EngineServer
59	-
60	-节点推理服务入口，提供统一的RESTful EndPoints，包括OpenAI接口、Metrics等。北向对接Coordinator和Controller，南向对接vLLM/SGLang/MindIE框架。（当前版本仅支持vLLM）
61	-
62	### NodeManager	58	### NodeManager
63		59
64	节点级服务管理器，提供如下能力：	60	节点级服务管理器，提供如下能力：
65		61
66	-- 节点级服务进程启动：向Controller注册，获取实例身份，并拉起本节点的推理服务进程(EngineServer, vLLM等)。	62	+- 节点级服务进程启动：向Controller注册，获取实例身份，并直接拉起本节点的原生 vLLM 或 SGLang 进程。
67	- 节点级健康状态管理：监控推理服务子进程状态，并向Controller上报健康状态和心跳。	63	- 节点级健康状态管理：监控推理服务子进程状态，并向Controller上报健康状态和心跳。
68		64
69	### 周边组件	65	### 周边组件
70		66
71	- [vLLM-Ascend](https://github.com/vllm-project/vllm-ascend): vLLM加速引擎，提供模型实例加速能力。	67	- [vLLM-Ascend](https://github.com/vllm-project/vllm-ascend): vLLM加速引擎，提供模型实例加速能力。
		68	+- SGLang: 可通过 NodeManager 原生运行时接入的推理引擎，当前支持范围以[支持的推理引擎](./user_guide/features/supported_inference_engines.md)为准。
72	- [MindCluster](https://gitcode.com/Ascend/mind-cluster): 昇腾集群使能组件，提供Kubernetes底层支持能力，PD分离 CRD定义和配套Operator	69	- [MindCluster](https://gitcode.com/Ascend/mind-cluster): 昇腾集群使能组件，提供Kubernetes底层支持能力，PD分离 CRD定义和配套Operator
73	- [CCAE](https://www.hiascend.com/software/ccae)(可选)：华为算存网一体化运维可视化平台。	70	- [CCAE](https://www.hiascend.com/software/ccae)(可选)：华为算存网一体化运维可视化平台。

Mdocs/zh/design/fault_tolerance/fault_manager.md+15-1

	@@ -4,6 +4,9 @@
4		4
5	FaultManager 是 MindIE Motor Controller 中负责故障容错管理的核心组件。它通过观察者模式监听实例生命周期事件，统一管理硬件故障（ConfigMap 上报）和软件故障（引擎异常），协调 ResourceMonitor 进行故障检测，并与 InstanceManager 配合进行实例隔离和恢复。	5	FaultManager 是 MindIE Motor Controller 中负责故障容错管理的核心组件。它通过观察者模式监听实例生命周期事件，统一管理硬件故障（ConfigMap 上报）和软件故障（引擎异常），协调 ResourceMonitor 进行故障检测，并与 InstanceManager 配合进行实例隔离和恢复。
6		6
		7	+> 能力边界： Engine Server 已删除。NodeManager 直接轮询原生引擎的
		8	+> `/fault_tolerance/status` HTTP 接口；该接口必须由目标引擎版本明确提供。
		9	+
7	## 架构总览	10	## 架构总览
8		11
9	### 模块拆分	12	### 模块拆分
	@@ -45,6 +48,10 @@ Controller 侧:
45	├── 节点交换: _swap_node_ownership() → 跨 job 实例间节点所有权交换	48	├── 节点交换: _swap_node_ownership() → 跨 job 实例间节点所有权交换
46	└── 数据持久化: ETCD Client	49	└── 数据持久化: ETCD Client
47		50
		51	+跨组件通用基线:
		52	+ ├── NodeManager: 进程状态、原生 /health、Pod 级恢复
		53	+ └── Coordinator: 请求异常、熔断、实例隔离与恢复探测
		54	+
48	NodeManager 侧:	55	NodeManager 侧:
49	FaultReporter (EngineManager 聚合)	56	FaultReporter (EngineManager 聚合)
50	├── HTTP 轮询 → GET {endpoint.business_port}/fault_tolerance/status (vLLM FT API)	57	├── HTTP 轮询 → GET {endpoint.business_port}/fault_tolerance/status (vLLM FT API)
	@@ -53,7 +60,14 @@ NodeManager 侧:
53	└── HTTP POST → Controller /controller/report_software_fault	60	└── HTTP POST → Controller /controller/report_software_fault
54	```	61	```
55		62
56	-## 故障上报链路 (端到端)	63	+### 故障能力分层
		64	+
		65	+\| 层级 \| 来源 \| 目标态要求 \|
		66	+\|---\|---\|---\|
		67	+\| 通用基线 \| 原生进程状态、`/health`、请求 transport/5xx/协议异常、Coordinator 熔断、ConfigMap、Node Watch \| 所有原生引擎部署必须保留 \|
		68	+\| 引擎扩展 \| vLLM `/fault_tolerance/status` \| 启用 FaultReporter 时，目标引擎必须提供该 HTTP 接口 \|
		69	+
		70	+## 原生引擎故障上报链路（端到端）
57		71
58	```text	72	```text
59	vllm EngineCore 异常 → 引擎状态变为 unhealthy/dead	73	vllm EngineCore 异常 → 引擎状态变为 unhealthy/dead

Mdocs/zh/design/fault_tolerance/overview.md+1-1

	@@ -96,7 +96,7 @@ InstanceAssembler._start_command_sender 发送 StartCmdMsg
96	▼	96	▼
97	NodeManager 接收 StartCmdMsg	97	NodeManager 接收 StartCmdMsg
98	- parse_start_cmd(): 校验参数，存储 instance_id 和 endpoints	98	- parse_start_cmd(): 校验参数，存储 instance_id 和 endpoints
99	- - Daemon.pull_engine(): 启动 engine_server 推理进程	99	+ - Daemon.pull_engine(): 启动原生推理引擎进程
100	- HeartbeatManager.start(): 开始心跳上报	100	- HeartbeatManager.start(): 开始心跳上报
101	│	101	│
102	▼	102	▼

Mdocs/zh/design/pd_disaggregation.md+19-8

	@@ -24,7 +24,7 @@ P/D 实例的协同行为由引擎 Connector 推导出的 `dispatch_capabilities
24	\| `concurrent_engine_sync` \| P/D 并发执行，由引擎同步 KV \|	24	\| `concurrent_engine_sync` \| P/D 并发执行，由引擎同步 KV \|
25	\| `prefill_handoff_decode` \| Prefill 完成后将结果交给 Decode \|	25	\| `prefill_handoff_decode` \| Prefill 完成后将结果交给 Decode \|
26		26
27	-NodeManager 会从 vLLM 的 `kv_transfer_config.kv_connector` 或显式 `dispatch_profile` 推导 capability；SGLang 自动上报 `concurrent_engine_sync`。Coordinator 只选择 P/D 两端共同支持的能力，不再通过 `pd_separate`、`cpcd_separate` 等人工模式名称猜测行为。	27	+NodeManager 会从 vLLM 的 `kv_transfer_config.kv_connector` 或显式 `dispatch_profile` 推导 capability；SGLang 自动上报 `concurrent_engine_sync`。Coordinator 根据实例 `engine_type` 选择原生协议 Adapter，并使用 P/D 两端的兼容元数据进行保护性校验。
28		28
29	### vLLM Connector 识别白名单	29	### vLLM Connector 识别白名单
30		30
	@@ -41,15 +41,17 @@ vLLM 引擎按 `kv_connector` 名称（大小写不敏感）推导 capability，
41	- `MultiConnector` 只看 `connectors[0]`（传输层）。KV 池/存储类连接器（如 `AscendStoreConnector`、`MooncakeConnectorStoreV1`、`UCMConnector`、`LMCacheAscendConnector`）一般作为 `connectors[1]` 的后端使用，不参与 capability 判定，因此无需出现在白名单中。	41	- `MultiConnector` 只看 `connectors[0]`（传输层）。KV 池/存储类连接器（如 `AscendStoreConnector`、`MooncakeConnectorStoreV1`、`UCMConnector`、`LMCacheAscendConnector`）一般作为 `connectors[1]` 的后端使用，不参与 capability 判定，因此无需出现在白名单中。
42	- 不在上表内、且 `connectors[0]` 也无法识别的连接器会被判为 `unknown`，不产生任何 capability。	42	- 不在上表内、且 `connectors[0]` 也无法识别的连接器会被判为 `unknown`，不产生任何 capability。
43		43
44	-> ⚠️ fail-closed：当 P/D 两端没有共同 capability 时（例如顶层 `kv_connector` 或 `connectors[0]` 用了未识别的连接器），Coordinator 不会强行配对——`select_pair_and_allocate` 返回空、就绪判定为 `UNKNOWN`、路由返回 503。这是有意的保护，避免把不兼容的 P/D 配在一起、直到 KV 传输阶段才失败。	44	+> ⚠️ fail-closed：原生 vLLM P/D 启动只接受 handoff 语义；未知或不兼容 Connector 会在 NodeManager 构造启动命令时失败，避免把错误推迟到 KV 传输阶段。
45		45
46	-`dispatch_capabilities` 是 NodeManager 向 Coordinator 上报的内部字段，不支持在用户配置中显式填写。若需让未被识别的连接器作为 P/D 传输使用，请在 `motor_engine_prefill_config` / `motor_engine_decode_config` 顶层（与 `engine_type` 同级，不是 `engine_config` 内部）显式声明 `dispatch_profile` 作为逃生口：	46	+`dispatch_capabilities` 是 NodeManager 上报的兼容元数据，不支持在用户配置中直接填写。若需让未被识别的连接器作为 P/D 传输使用，请在 `motor_engine_prefill_config` / `motor_engine_decode_config` 顶层（与 `engine_type` 同级，不是 `engine_config` 内部）显式声明 `dispatch_profile`：
47		47
48	\| `dispatch_profile` \| 推导出的 capability \| 协同行为 \|	48	\| `dispatch_profile` \| 推导出的 capability \| 协同行为 \|
49	\|--------------------\|---------------------\|----------\|	49	\|--------------------\|---------------------\|----------\|
50	-\| `handoff` \| `prefill_handoff_decode` \| Prefill 完成后将结果交给 Decode \|	50	+\| `handoff` \| `prefill_handoff_decode` \| Prefill 完成后交给 Decode \|
51	\| `trigger` \| `concurrent_engine_sync` \| P/D 并发执行，由引擎同步 KV \|	51	\| `trigger` \| `concurrent_engine_sync` \| P/D 并发执行，由引擎同步 KV \|
52		52
		53	+当前原生 vLLM P/D 运行时只接受 `handoff`；`trigger` 仅作为兼容分类保留，不可用于原生 vLLM P/D 启动。
		54	+
53	配置示例（自定义 connector 不在白名单内时）：	55	配置示例（自定义 connector 不在白名单内时）：
54		56
55	```json	57	```json
	@@ -75,7 +77,16 @@ vLLM 引擎按 `kv_connector` 名称（大小写不敏感）推导 capability，
75	}	77	}
76	```	78	```
77		79
78	-> Prefill 与 Decode 两端 `dispatch_profile` 必须一致，且取值须与 connector 实际协同语义匹配，否则 Coordinator 仍无法配对（503）。字段说明见 [user_config 全量参数说明](../user_guide/configuration/config_reference.md#dispatch_profile)。	80	+> Prefill 与 Decode 两端 `dispatch_profile` 必须一致，且取值须与 Connector 实际协同语义匹配。字段说明见 [user_config 全量参数说明](../user_guide/configuration/config_reference.md#dispatch_profile)。
		81	+
		82	+### SGLang Bootstrap 元数据
		83	+
		84	+SGLang 使用原生 bootstrap 协议，不复用 vLLM 的 `kv_transfer_params`。NodeManager 从所选
		85	+引擎配置的 `engine_config.disaggregation_bootstrap_port`（兼容
		86	+`disaggregation-bootstrap-port`）派生每个 Pod 的 `bootstrap_port`，并在注册消息的 endpoint
		87	+元数据中上报。Coordinator 的 SGLang Adapter 将 Prefill endpoint 的 `bootstrap_host`、
		88	+`bootstrap_port` 和稳定的 `bootstrap_room` 注入 Prefill/Decode 请求；`business_port` 仍是
		89	+推理 HTTP 服务端口，`mgmt_port` 仅为注册协议兼容字段。
79		90
80	## 数据流	91	## 数据流
81		92
	@@ -85,8 +96,8 @@ flowchart LR
85	Coord --> Roles[Inspect instance roles]	96	Coord --> Roles[Inspect instance roles]
86	Roles -->\|P + D\| Unified[UnifiedPDRouter]	97	Roles -->\|P + D\| Unified[UnifiedPDRouter]
87	Roles -->\|Union or P only\| Hybrid[PDHybridRouter]	98	Roles -->\|Union or P only\| Hybrid[PDHybridRouter]
88	- Unified --> Capability[Select shared connector capability]	99	+ Unified --> Adapter[Select adapter by engine_type]
89	- Capability --> EngineP[Prefill instance]	100	+ Adapter --> EngineP[Prefill instance]
90	- Capability --> EngineD[Decode instance]	101	+ Adapter --> EngineD[Decode instance]
91	Hybrid --> EngineU[Union or fallback Prefill instance]	102	Hybrid --> EngineU[Union or fallback Prefill instance]
92	```	103	```

Mdocs/zh/developer_guide/README.md+1-1

	@@ -4,7 +4,7 @@
4		4
5	## 内容侧重	5	## 内容侧重
6		6
7	-- 组件内部实现（Controller、Coordinator、Engine Server、Node Manager）	7	+- 组件内部实现（Controller、Coordinator、Node Manager 与原生引擎适配）
8	- 类图、数据流、模块交互	8	- 类图、数据流、模块交互
9	- 构建工具（文档构建、镜像制作）	9	- 构建工具（文档构建、镜像制作）
10		10

Mdocs/zh/developer_guide/components/controller.md+8-4

	@@ -13,7 +13,7 @@ flowchart LR
13	Ctrl -->\|实例变更事件\| Coord[Coordinator<br/>调度面]	13	Ctrl -->\|实例变更事件\| Coord[Coordinator<br/>调度面]
14	Ctrl -->\|持久化\| ETCD[(ETCD)]	14	Ctrl -->\|持久化\| ETCD[(ETCD)]
15	Ctrl -->\|Watch ConfigMap/Node\| K8s[K8s API Server]	15	Ctrl -->\|Watch ConfigMap/Node\| K8s[K8s API Server]
16	- NM -->\|拉起/监控\| ES[EngineServer<br/>推理引擎]	16	+ NM -->\|拉起/监控\| ES[原生 vLLM/SGLang<br/>推理引擎]
17	Coord -->\|调度请求\| ES	17	Coord -->\|调度请求\| ES
18	```	18	```
19		19
	@@ -21,7 +21,7 @@ flowchart LR
21		21
22	- NodeManager：每个推理 Pod 的节点代理，注册自身到 Controller，接收启动指令，上报心跳与故障。	22	- NodeManager：每个推理 Pod 的节点代理，注册自身到 Controller，接收启动指令，上报心跳与故障。
23	- Coordinator：推理请求的调度入口，Consumer 接收 Controller 推送的实例状态变更，据此决定路由策略。	23	- Coordinator：推理请求的调度入口，Consumer 接收 Controller 推送的实例状态变更，据此决定路由策略。
24	-- EngineServer：实际执行推理的引擎进程（vLLM / SGLang），由 NodeManager 根据 Controller 下发的 StartCmd 拉起。	24	+- 原生推理引擎：实际执行推理的 vLLM / SGLang 进程，由 NodeManager 根据 Controller 下发的 StartCmd 直接拉起。
25	- ETCD：Controller 的持久化存储，支撑主备切换时状态恢复。	25	- ETCD：Controller 的持久化存储，支撑主备切换时状态恢复。
26	- K8s API Server：Controller 通过 Watch 机制感知硬件故障（ConfigMap）和节点状态变化。	26	- K8s API Server：Controller 通过 Watch 机制感知硬件故障（ConfigMap）和节点状态变化。
27		27
	@@ -237,7 +237,7 @@ sequenceDiagram
237	IA->>IM: add_instance(instance)	237	IA->>IM: add_instance(instance)
238	IM->>IM: notify(INSTANCE_INITIAL)	238	IM->>IM: notify(INSTANCE_INITIAL)
239	IA->>NM: NodeManagerApiClient.send_start_command()<br/>StartCmdMsg(job_name, instance_id, endpoints, master_dp_ip, ...)	239	IA->>NM: NodeManagerApiClient.send_start_command()<br/>StartCmdMsg(job_name, instance_id, endpoints, master_dp_ip, ...)
240	- NM->>NM: parse_start_cmd() + 拉起 EngineServer	240	+ NM->>NM: parse_start_cmd() + 拉起原生推理引擎
241	NM->>API: POST /controller/heartbeat<br/>HeartbeatMsg(ins_id, pod_ip, status)	241	NM->>API: POST /controller/heartbeat<br/>HeartbeatMsg(ins_id, pod_ip, status)
242	API->>IM: handle_heartbeat(msg)	242	API->>IM: handle_heartbeat(msg)
243	IM->>IM: 状态机: INITIAL → ACTIVE	243	IM->>IM: 状态机: INITIAL → ACTIVE
	@@ -529,7 +529,11 @@ flowchart LR
529	\|------\|----------\|----------\|----------\|	529	\|------\|----------\|----------\|----------\|
530	\| ConfigMap Watch \| NPU 卡故障 (`CardUnhealthy`)<br/>卡间网络故障 (`CardNetworkUnhealthy`)<br/>交换机故障 \| K8s Watch API，每个 Node 一个 ResourceMonitor \| `FaultInfo` → `NodeMetadata.hardware_fault_infos` \|	530	\| ConfigMap Watch \| NPU 卡故障 (`CardUnhealthy`)<br/>卡间网络故障 (`CardNetworkUnhealthy`)<br/>交换机故障 \| K8s Watch API，每个 Node 一个 ResourceMonitor \| `FaultInfo` → `NodeMetadata.hardware_fault_infos` \|
531	\| Node Watch \| 节点重启 / NotReady \| K8s Watch API \| `NODE_REBOOT` (fault_code: `0x0000001`) → L6 \|	531	\| Node Watch \| 节点重启 / NotReady \| K8s Watch API \| `NODE_REBOOT` (fault_code: `0x0000001`) → L6 \|
532	-\| 软件故障上报 \| Engine DEAD/UNHEALTHY \| NodeManager FaultReporter → HTTP \| `FaultInfo` → `NodeMetadata.software_fault_infos` \|	532	+\| 软件故障上报 \| Engine DEAD/UNHEALTHY \| 原生引擎 `/fault_tolerance/status` → NodeManager FaultReporter → HTTP \| `FaultInfo` → `NodeMetadata.software_fault_infos` \|
		533	+
		534	+> 原生引擎故障能力边界：启用 FaultReporter 时，目标引擎版本必须提供
		535	+> `/fault_tolerance/status`。进程状态、原生 health、请求异常与熔断仍由
		536	+> Node Manager/Coordinator 构成通用故障基线。
533		537
534	#### 3.3.2 故障等级体系	538	#### 3.3.2 故障等级体系
535		539

Ddocs/zh/developer_guide/components/engine_server.md+0-95

	@@ -1,95 +0,0 @@
1	-# Motor Engine Server（推理引擎侧进程）
2	-
3	-## 功能介绍
4	-
5	-在本仓库中，Engine Server 指可执行入口 `engine_server`（`setup.py` 中 entry point：`engine_server = motor.engine_server.cli.main:main`），实现该功能的脚本路径为 `motor/engine_server/cli/main.py`。
6	-
7	-> `EngineServer` 是过渡期兼容链路。Node Manager 的直接原生路径由
8	-> `motor/node_manager/core/services/native_engine/` 管理，并直接拉起 `vllm serve` 或
9	-> `python3 -m sglang.launch_server`，不会插入 `engine_server` 进程。
10	-
11	-主要功能如下：
12	-
13	-- 解析端点配置：`EndpointConfig.init_endpoint_config()`，经 `ConfigFactory` 得到具体引擎配置（`motor/node_manager/core/services/native_engine/config_factory.py` 中按 `vllm` / `sglang` 等类型选择配置类）。
14	-- 管理面 HTTP（MgmtEndpoint）：`motor/engine_server/core/mgmt_endpoint.py` 内在 `mgmt_port` 上启动 uvicorn，挂载 Prometheus 相关路由及 `GET /status`（路径常量 `STATUS_INTERFACE`，值为 `/status`）。状态字段键为 `STATUS_KEY` 对应常量（与同文件 `NORMAL_STATUS` / `ABNORMAL_STATUS` / `INIT_STATUS` 等配合使用）。可选 TLS（`mgmt_tls_config.enable_tls`）。
15	-- 推理面（InferEndpoint）：由 `EndpointFactory.get_infer_endpoint(config)` 按引擎类型构造（如 `VLLMEndpoint`、`SGLangEndpoint`，见 `motor/engine_server/factory/endpoint_factory.py`），与 `MgmtEndpoint` 并行 `run()`，主线程在 `infer_endpoint.wait()` 阻塞直至退出，再 `shutdown` 两端点。
16	-
17	-旧部署入口可通过子进程命令 `engine_server` 拉起本进程；新 Node Manager 原生路径不再组装该命令。以下 Engine Server 参数说明仅用于兼容链路和本地调试。
18	-
19	-## 与周边组件的关系
20	-
21	-Engine Server 同时处于数据面推理转发链路与管控面生命周期链路中：对外承接 Coordinator 转发的 OpenAI 请求；管控面上由 Node Manager 通过子进程拉起 Engine Server 并周期性探测其健康状态，再将结果经 heartbeat 上报 Controller。Controller 与 Engine Server 之间无直接 HTTP 或进程调用，启停均经 Node Manager 间接完成。
22	-
23	-![Engine Server 与周边组件关系图](../../imgs/engine_server_component_relations.png)
24	-
25	-PD 分离模式下，Coordinator 可能将 Prefill 与 Decode 分派至不同 role 的 Engine Server 实例；单节点 / Hybrid 模式则由同一实例完成推理全流程。
26	-
27	-\| 阶段 \| 方向 \| 接口/机制 \| 说明 \|
28	-\| -------- \| ----------------------------- \| -------------------------------- \| ------------------------------------------ \|
29	-\| 推理请求 \| Coordinator → Engine Server \| infer 端口`/v1/*` \| Coordinator 路由选中实例后转发 OpenAI 请求 \|
30	-\| 节点注册 \| Node Manager → Controller \| `POST /controller/register` \| Node Manager 启动后向 Controller 注册节点 \|
31	-\| 实例启动 \| Controller → Node Manager \| `POST /node-manager/start` \| Controller 完成实例组装后下发启动命令 \|
32	-\| 引擎拉起 \| Node Manager → Engine Server \| `subprocess` `engine_server` \| Node Manager 子进程拉起 engine_server \|
33	-\| 健康探测 \| Node Manager → Engine Server \| mgmt 端口`GET /status` \| Node Manager 周期性轮询引擎 mgmt 健康状态 \|
34	-\| 状态上报 \| Node Manager → Controller \| `POST /controller/heartbeat` \| 将各 endpoint 健康状态上报 Controller \|
35	-\| 实例停止 \| Controller → Node Manager \| `POST /node-manager/stop` \| Controller 下发停止指令（含故障恢复场景） \|
36	-\| 引擎停止 \| Node Manager → Engine Server \| `Daemon.stop` SIGKILL \| Node Manager 对引擎子进程发送 SIGKILL \|
37	-
38	-Node Manager 经 `engine_server_api_client` 对 `{ip}:{mgmt_port}` 发起 `GET /status`（TLS 取自 `motor_nodemanger_config.mgmt_tls_config`）。mgmt `/status` 综合推理面 `/health` 与可选虚推结果；虚推由 `health_check_config.enable_virtual_inference` 控制，默认关闭，机制见 [虚推健康探测](../../user_guide/features/sim_inference.md)，字段见 [配置参考 health_check_config](../../user_guide/configuration/config_reference.md#health_check_config)。
39	-
40	-## 环境准备
41	-
42	-- `--config-path` 指向包含 `motor_engine_prefill_config` / `motor_engine_decode_config` 的 `user_config.json`（与 Node Manager 挂载路径一致，见[配置文件说明](../../user_guide/configuration/config_reference.md)）。
43	-- Ascend NPU 驱动/HDK、模型权重路径等运行环境要求见 [环境准备](../../user_guide/environment_preparation.md)。
44	-
45	-## 配置说明
46	-
47	-- 引擎配置块：`motor_engine_prefill_config` / `motor_engine_decode_config`（含可选 `health_check_config`）。
48	-- 节点侧交叉项：`motor_nodemanger_config`（如 mgmt TLS，供 Node Manager 探测 Engine Server mgmt 端口）。
49	-- 字段权威说明：[配置参考](../../user_guide/configuration/config_reference.md) 中 `motor_engine_prefill_config` / `motor_engine_decode_config` 等章节。
50	-- CLI 侧定义：`motor/config/endpoint.py` 中 `EndpointConfig` 字段与校验逻辑。
51	-
52	-## 使用样例（本地调试）
53	-
54	-与 Node Manager 下发命令一致，本地调试需准备：
55	-
56	-- CLI 参数：`--host`、`--role`、`--port`、`--mgmt-port`、`--instance-id`、`--dp-rank`、`--master-dp-ip`、`--config-path`；单容器模式可能还需 `--kv-port`、`--dp-rpc-port`。
57	-- 配置文件：`--config-path` 指向的 `user_config.json` 须包含与 `--role` 对应的引擎配置块。
58	-- 运行环境：NPU/Ascend 运行时与模型权重路径由 `user_config` 内 `engine_config` 指定，见 [环境准备](../../user_guide/environment_preparation.md)。
59	-
60	-```bash
61	-engine_server --dp-rank 0 --instance-id 1 --role prefill \
62	- --host 127.0.0.1 --port 8000 --mgmt-port 8001 \
63	- --master-dp-ip 127.0.0.1 --config-path /path/to/user_config.json
64	-```
65	-
66	-下表说明示例命令中的 CLI 参数（完整定义与校验见 `motor/config/endpoint.py` 中 `EndpointConfig.parse_cli_args`；Node Manager 组装逻辑见 `motor/node_manager/core/daemon.py` 的 `pull_engine`）。
67	-
68	-\| 参数 \| 类型 \| 说明 \|
69	-\|------\|------\|------\|
70	-\| `--dp-rank` \| int \| 数据并行组内 endpoint 序号，默认 `0`。Node Manager 拉起时取 `endpoint.id`，并映射为 vLLM `data-parallel-rank`（见 [examples/deployer/README.md](https://gitcode.com/Ascend/MindIE-Motor/blob/master/examples/deployer/README.md)）。 \|
71	-\| `--instance-id` \| int \| 实例 ID，由 Controller 组装后随 `StartCmdMsg` 下发，默认 `0`。 \|
72	-\| `--role` \| string \| PD 分离角色：`prefill`、`decode` 或 `union`（混部）。 \|
73	-\| `--host` \| string \| 推理面与管理面监听地址（bind IP）。 \|
74	-\| `--port` \| int \| 推理业务端口，对外提供 `/v1/*`、`/health` 等 infer 接口。 \|
75	-\| `--mgmt-port` \| int \| 管理面端口，对外提供 `GET /status` 及 Prometheus 路由。 \|
76	-\| `--master-dp-ip` \| string \| DP master 节点 IP，用于分布式推理组网（对应 vLLM `data-parallel-address` 来源，见 [examples/deployer/README.md](https://gitcode.com/Ascend/MindIE-Motor/blob/master/examples/deployer/README.md)）。 \|
77	-\| `--config-path` \| string \| `user_config.json` 路径，须含与 `--role` 匹配的引擎配置块。 \|
78	-
79	-单容器或跨节点等场景可能额外传入下表参数（本地最小示例可不填）：
80	-
81	-\| 参数 \| 类型 \| 说明 \|
82	-\|------\|------\|------\|
83	-\| `--node-rank` \| int \| 跨节点 PCP 节点序号，由 Controller 按注册顺序分配；传递规则见 [Node Manager 组件文档](./node_manager.md#跨节点-pcp)。 \|
84	-\| `--kv-port` \| int \| 单容器模式下 KV 相关通信端口（`Daemon.pull_engine` 在 `single_container_flag` 时追加）。 \|
85	-\| `--dp-rpc-port` \| int \| 单容器模式下 DP RPC 端口（同上）。 \|
86	-\| `--lookup-rpc-port` \| int \| 可选 lookup RPC 端口（配置存在时追加）。 \|
87	-\| `--d2d-peer-ips` \| string \| D2D 权重传输对端 IP 列表，逗号分隔。 \|
88	-\| `--snapshot-metadata` \| string \| 容器快照元数据 JSON 路径，启用快照能力时传入。 \|
89	-
90	-实际端口与角色以调度结果为准；单机调试可参考测试与示例配置。
91	-
92	-## 报错与日志
93	-
94	-- 默认日志文件路径常量见 `motor/engine_server/constants/constants.py`（如 `LOG_DEFAULT_FILE` 相对 `./engine_server_log/`）。
95	-- Mgmt 面 `/status` 在健康检查异常时返回 `ABNORMAL_STATUS` 等（见 `mgmt_endpoint.get_status` 实现）；Node Manager 侧据此更新 endpoint 状态并可能参与自杀判断（见 `HeartbeatManager`）。

Mdocs/zh/developer_guide/components/node_manager.md+10-7

	@@ -94,7 +94,7 @@ Node Manager API 默认监听 `api_config.pod_ip:api_config.node_manager_port`
94	\| `job_name` \| string \| 是 \| 实例任务名，必须与本节点配置一致 \|	94	\| `job_name` \| string \| 是 \| 实例任务名，必须与本节点配置一致 \|
95	\| `role` \| string \| 是 \| 实例角色，如 `prefill`、`decode` 或 `union` \|	95	\| `role` \| string \| 是 \| 实例角色，如 `prefill`、`decode` 或 `union` \|
96	\| `instance_id` \| int \| 是 \| Controller 分配的实例 ID \|	96	\| `instance_id` \| int \| 是 \| Controller 分配的实例 ID \|
97	-\| `endpoints` \| array \| 是 \| 本节点管理的 endpoint；元素包含 `id`、`ip`、`business_port`、`mgmt_port`，SGLang PD endpoint 还可包含 `bootstrap_port` \|	97	+\| `endpoints` \| array \| 是 \| 本节点管理的 endpoint；元素包含 `id`、`ip`、`business_port`、`mgmt_port`，SGLang PD endpoint 还可包含 `bootstrap_port`。`mgmt_port` 为注册协议兼容字段，原生引擎健康探测和快照控制均使用 `business_port` \|
98	\| `master_dp_ip` \| string \| 是 \| 数据并行主节点 IP \|	98	\| `master_dp_ip` \| string \| 是 \| 数据并行主节点 IP \|
99	\| `ranktable` \| object/null \| 否 \| 实例级 ranktable，默认 `null` \|	99	\| `ranktable` \| object/null \| 否 \| 实例级 ranktable，默认 `null` \|
100	\| `d2d_peer_ips` \| array/null \| 否 \| D2D 权重传输对端，Controller 使用 `<endpoint_id>:<peer_ip>` 编码，默认 `null` \|	100	\| `d2d_peer_ips` \| array/null \| 否 \| D2D 权重传输对端，Controller 使用 `<endpoint_id>:<peer_ip>` 编码，默认 `null` \|
	@@ -187,8 +187,8 @@ Node Manager 从 `engine_config.nnodes` 推导每节点 `local_world_size`。当
187	\| `fault_tolerance_config.poll_interval_sec` \| `5.0` \| 轮询引擎 FT 状态的时间间隔（秒） \|	187	\| `fault_tolerance_config.poll_interval_sec` \| `5.0` \| 轮询引擎 FT 状态的时间间隔（秒） \|
188	\| `fault_tolerance_config.poll_timeout_sec` \| `5.0` \| 单次轮询的 HTTP 超时（秒） \|	188	\| `fault_tolerance_config.poll_timeout_sec` \| `5.0` \| 单次轮询的 HTTP 超时（秒） \|
189	\| `fault_tolerance_config.max_poll_failures` \| `3` \| 连续轮询失败阈值，达到后按 `dead` 上报 \|	189	\| `fault_tolerance_config.max_poll_failures` \| `3` \| 连续轮询失败阈值，达到后按 `dead` 上报 \|
190	-\| `snapshot_config.enable_snapshot` \| `false` \| 原生引擎运行时尚不支持容器快照；设为 `true` 会在配置校验阶段失败 \|	190	+\| `snapshot_config.enable_snapshot` \| `false` \| 是否启用容器快照；当前仅 vLLM 原生引擎支持，SGLang 配置为 `true` 会校验失败 \|
191	-\| `snapshot_config.snapshot_metadata_path` \| 空 \| 预留字段；当前原生引擎运行时不消费该路径 \|	191	+\| `snapshot_config.snapshot_metadata_path` \| 空 \| 容器快照元数据路径；为空时使用默认路径 `/snapshot/snapshot_metadata.json` \|
192	\| `port_allocator_config.enable` \| `true` \| 是否在启动时自动检查并调整端口 \|	192	\| `port_allocator_config.enable` \| `true` \| 是否在启动时自动检查并调整端口 \|
193		193
194	`endpoint_num`、`service_ports`、`mgmt_ports`、`device_num`、`parallel_config`、`model_name`、`engine_type` 和 `dispatch_capabilities` 主要由部署配置与引擎配置派生。`dispatch_capabilities` 不接受用户直接覆盖。	194	`endpoint_num`、`service_ports`、`mgmt_ports`、`device_num`、`parallel_config`、`model_name`、`engine_type` 和 `dispatch_capabilities` 主要由部署配置与引擎配置派生。`dispatch_capabilities` 不接受用户直接覆盖。
	@@ -321,11 +321,14 @@ GET http://{endpoint.ip}:{endpoint.business_port}/fault_tolerance/status
321		321
322	## 容器快照	322	## 容器快照
323		323
324	-当前 Node Manager 已切换为直接拉起原生 vLLM/SGLang，但原生 `suspend/resume/device-unlock`	324	+当前 Node Manager 直接拉起原生 vLLM/SGLang；显存快照的保存与恢复由引擎自闭环完成，
325	-控制接口尚未接入。为避免仅校验 metadata 文件、实际却未执行快照操作，配置	325	+仅支持提供快照能力的 vLLM 镜像。配置 `snapshot_config.enable_snapshot=true` 且引擎类型为
326	-`snapshot_config.enable_snapshot=true` 会在启动前明确失败。	326	+SGLang 时，配置校验会明确失败。
327		327
328	-待 Native Engine Backend 实现原生快照控制契约后，再恢复 metadata、checkpoint barrier 和恢复编排。	328	+启用容器快照后，Node Manager 只做框架侧编排：准备 `model_save_path` / `model_load_path` /
		329	+`data_parallel_master_ip` 等元数据，快照恢复后刷新 `job_name`、Pod IP 与 Controller DNS，
		330	+并通过引擎就绪状态和 Host 侧 `checkpoint` 标记感知保存/恢复是否完成。Node Manager 不调用
		331	+引擎的 `/suspend`、`/device_unlock`、`/resume`。
329		332
330	## 使用样例	333	## 使用样例
331		334

Ddocs/zh/imgs/engine_server_component_relations.png+0-0

Mdocs/zh/user_guide/api/README.md+2-20

	@@ -1,6 +1,6 @@
1	# 接口说明	1	# 接口说明
2		2
3	-MindIE Motor提供推理[业务接口](#业务接口)、[管理接口](#管理接口)、[指标接口](#指标接口)、[观测接口](#观测接口)和[内部接口](#内部接口)。	3	+MindIE Motor提供推理[业务接口](#业务接口)、[管理接口](#管理接口)、[指标接口](#指标接口)和[观测接口](#观测接口)。
4		4
5	## 业务接口	5	## 业务接口
6		6
	@@ -38,6 +38,7 @@ MindIE Motor提供下列管理接口：
38	- [存活探针接口](./management_interfaces.md#存活探针接口)：`/liveness`	38	- [存活探针接口](./management_interfaces.md#存活探针接口)：`/liveness`
39	- [就绪探针接口](./management_interfaces.md#就绪探针接口)：`/readiness`	39	- [就绪探针接口](./management_interfaces.md#就绪探针接口)：`/readiness`
40	- [实例刷新接口](./management_interfaces.md#实例刷新接口)：`/instances/refresh`	40	- [实例刷新接口](./management_interfaces.md#实例刷新接口)：`/instances/refresh`
		41	+- [精度告警状态清理接口](./management_interfaces.md#精度告警状态清理接口)：`/precision/alarm_cleared`
41	- [根路径服务信息接口](./management_interfaces.md#根路径服务信息接口)：`/`	42	- [根路径服务信息接口](./management_interfaces.md#根路径服务信息接口)：`/`
42	- [健康状态查询接口](./management_interfaces.md#健康状态查询接口)：`/health`	43	- [健康状态查询接口](./management_interfaces.md#健康状态查询接口)：`/health`
43		44
	@@ -124,25 +125,6 @@ Controller 观测接口提供模型服务清单与告警等运维观测数据（
124	- 在Kubernetes集群内，观测接口端口使用[`user_config.json`](../configuration/config_reference.md#motor_controller_config)配置文件中`observability_api_port`定义的端口。	125	- 在Kubernetes集群内，观测接口端口使用[`user_config.json`](../configuration/config_reference.md#motor_controller_config)配置文件中`observability_api_port`定义的端口。
125	- 当配置文件中无此配置项时，使用默认端口`1027`。	126	- 当配置文件中无此配置项时，使用默认端口`1027`。
126		127
127	-## 内部接口
128	-
129	-EngineServer提供下列内部接口：
130	-
131	-- [Engine Server 快照接口](./engine_server_interfaces.md#engine-server-快照接口)，包括：
132	- - [设备侧快照保存接口](./engine_server_interfaces.md#设备侧快照保存接口)：`/suspend`
133	- - [设备解锁接口](./engine_server_interfaces.md#设备解锁接口)：`/device_unlock`
134	- - [设备侧快照恢复接口](./engine_server_interfaces.md#设备侧快照恢复接口)：`/resume`
135	-- [MetaServer转发接口](./engine_server_interfaces.md#metaserver转发接口)：`/v1/metaserver`
136	-
137	->[!NOTE]说明
138	->
139	-> Engine Server 内部接口挂载在 Engine Server 推理面，不在 Coordinator 推理接口上提供服务。
140	-
141	-### 内部接口的IP/端口
142	-
143	-- 内部接口IP：Engine Server 所在节点的 IP 或 `engine_server --host` 绑定的地址。
144	-- 内部接口端口：`engine_server --port` 指定的端口。
145	-
146	## 安全、认证与限流	128	## 安全、认证与限流
147		129
148	- 安全协议：`infer_tls_config.enable_tls` / `mgmt_tls_config.enable_tls` 为 `true` 时，推理/管理接口端口使用 `https`	130	- 安全协议：`infer_tls_config.enable_tls` / `mgmt_tls_config.enable_tls` 为 `true` 时，推理/管理接口端口使用 `https`

Ddocs/zh/user_guide/api/engine_server_interfaces.md+0-291

	@@ -1,291 +0,0 @@
1	-# Engine Server 内部接口
2	-
3	-> [!NOTE]说明
4	->
5	-> Engine Server 内部接口挂载在 Engine Server 推理面，不在 Coordinator 推理接口上提供服务。
6	-
7	-## Engine Server 快照接口
8	-
9	-Engine Server 快照接口用于容器快照场景下的推理引擎保存设备侧快照、设备解锁与设备侧快照恢复。接口挂载在 Engine Server 推理面（InferEndpoint），与 `/v1/chat/completions`、`/health` 等推理接口共用 `engine_server` 启动参数 `--port` 指定的业务端口。
10	-
11	-典型调用顺序为：`suspend` →（可选）`device_unlock` → `resume`。具体时机由部署方决定。
12	-
13	-Engine Server 快照接口使用推理端口：
14	-
15	-- 基址：`http(s)://{EngineIP}:{推理端口}`
16	-- 安全协议：`infer_tls_config.enable_tls` 为 `true` 时使用 `https`，否则使用 `http`。
17	-
18	-IP与端口参见[内部接口的IP/端口](./README.md#内部接口的ip端口)
19	-
20	----
21	-
22	-### 设备侧快照保存接口
23	-
24	-接口功能
25	-
26	-通知推理引擎将模型运行时权重落盘到指定路径，锁定设备并保存设备侧快照。
27	-
28	-接口格式
29	-
30	-请求类型：POST
31	-
32	-> URL：`http(s)://{EngineIP}:{推理端口}/suspend?model_save_path={模型落盘路径}`
33	-
34	-IP与端口参见[内部接口的IP/端口](./README.md#内部接口的ip端口)
35	-
36	-请求参数
37	-
38	-\| 参数名 \| 类型 \| 说明 \|
39	-\| ----------------- \| ------ \| ------------------------- \|
40	-\| `model_save_path` \| string \| 必选；Query 参数。模型权重等数据的落盘目录。 \|
41	-
42	-使用样例
43	-
44	-```bash
45	-curl -X POST "http://{EngineIP}:{推理端口}/suspend?model_save_path=/snapshot/weight"
46	-```
47	-
48	-响应示例
49	-
50	-- 成功：HTTP `200`，响应体为空。
51	-- 失败：缺少必填参数 `model_save_path` 时返回 `400`；当前引擎未实现 `suspend` / `resume` 时返回 `501`。
52	-
53	----
54	-
55	-### 设备解锁接口
56	-
57	-接口功能
58	-
59	-在调用设备侧快照保存接口后，设备会处于锁定状态。本接口用于通知推理引擎解锁设备。
60	-
61	-接口格式
62	-
63	-请求类型：POST
64	-
65	-> URL：`http(s)://{EngineIP}:{推理端口}/device_unlock`
66	-
67	-IP与端口参见[内部接口的IP/端口](./README.md#内部接口的ip端口)
68	-
69	-请求参数
70	-
71	-无
72	-
73	-使用样例
74	-
75	-```bash
76	-curl -X POST "http://{EngineIP}:{推理端口}/device_unlock"
77	-```
78	-
79	-响应示例
80	-
81	-- 成功：HTTP `200`，响应体为空。
82	-- 失败：当前引擎未实现 `device_unlock` 时返回 `501`。
83	-
84	----
85	-
86	-### 设备侧快照恢复接口
87	-
88	-接口功能
89	-
90	-通知推理引擎恢复已保存的设备侧快照，从指定路径重新加载运行时模型权重并重建通信域等运行时状态。
91	-
92	-接口格式
93	-
94	-请求类型：POST
95	-
96	-> URL：`http(s)://{EngineIP}:{推理端口}/resume?data_parallel_master_ip={DP主节点IP}&model_path={模型路径}`
97	-
98	-IP与端口参见[内部接口的IP/端口](./README.md#内部接口的ip端口)
99	-
100	-请求参数
101	-
102	-\| 参数名 \| 类型 \| 说明 \|
103	-\| ------------------------- \| ------ \| --------------------------- \|
104	-\| `data_parallel_master_ip` \| string \| 必选；Query 参数。数据并行（DP）主节点 IP。 \|
105	-\| `model_path` \| string \| 必选；Query 参数。模型加载路径。 \|
106	-
107	-使用样例
108	-
109	-```bash
110	-curl -X POST "http://{EngineIP}:{推理端口}/resume?data_parallel_master_ip=10.0.0.1&model_path=/snapshot/weight"
111	-```
112	-
113	-响应示例
114	-
115	-- 成功：HTTP `200`，响应体为空。
116	-- 失败：缺少必填参数 `data_parallel_master_ip` 或 `model_path` 时返回 `400`；当前引擎未实现 `suspend` / `resume` 时返回 `501`。
117	-
118	----
119	-
120	-## MetaServer转发接口
121	-
122	-接口功能
123	-
124	-仅在PD/CDP分离部署场景使用，用于D节点将请求转发至P节点。
125	-
126	-接口格式
127	-
128	-请求类型：POST
129	-
130	-> URL：`http(s)://{EngineIP}:{推理端口}/v1/metaserver`
131	-
132	-IP与端口参见[内部接口的IP/端口](./README.md#内部接口的ip端口)
133	-
134	-请求参数
135	-
136	-\| 参数 \| 类型 \| 说明 \|
137	-\| -------------------------------------- \| ------- \| ------------------------------- \|
138	-\| `model` \| string \| 必选；模型名称，透传至目标节点。 \|
139	-\| `messages` \| array \| 与 `prompt` 二选一；Chat输入。 \|
140	-\| `prompt` \| string \| 与 `messages` 二选一；Completion 输入。 \|
141	-\| `stream` \| boolean \| 可选；是否流式返回，透传至目标节点。 \|
142	-\| `kv_transfer_params` \| object \| 必选；转发控制参数。 \|
143	-\| `kv_transfer_params.request_id` \| string \| 必选；请求标识，用于跨节点跟踪与关联。 \|
144	-\| `kv_transfer_params.do_remote_decode` \| boolean \| 可选；是否在目标节点执行 Decode。 \|
145	-\| `kv_transfer_params.do_remote_prefill` \| boolean \| 可选；是否在目标节点执行 Prefill。 \|
146	-\| `kv_transfer_params.remote_engine_id` \| string \| 必选；目标节点引擎 ID。 \|
147	-\| `kv_transfer_params.remote_host` \| string \| 必选；目标节点地址（IP 或域名）。 \|
148	-\| `kv_transfer_params.remote_port` \| string \| 必选；目标节点端口。 \|
149	-
150	-使用样例
151	-
152	-- CDP分离场景，D节点触发P节点Prefill：
153	-
154	- ```json
155	- curl -X POST "http://{EngineIP}:{推理端口}/v1/metaserver" \
156	- -H "Content-Type: application/json" \
157	- -d '{
158	- "model": "qwen3",
159	- "messages": [
160	- { "role": "user", "content": "Hello!" }
161	- ],
162	- "stream": false,
163	- "kv_transfer_params": {
164	- "request_id": "req-id",
165	- "do_remote_decode": false,
166	- "do_remote_prefill": true,
167	- "remote_engine_id": "engine-p-0",
168	- "remote_host": "10.0.0.12",
169	- "remote_port": "1000"
170	- }
171	- }'
172	- ```
173	-
174	-- PD分离场景，P节点触发D节点Decode：
175	-
176	- ```json
177	- curl -X POST "http://{EngineIP}:{推理端口}/v1/metaserver" \
178	- -H "Content-Type: application/json" \
179	- -d '{
180	- "model": "qwen3",
181	- "messages": [
182	- { "role": "user", "content": "Hello!" }
183	- ],
184	- "stream": false,
185	- "kv_transfer_params": {
186	- "request_id": "req-id",
187	- "do_remote_decode": true,
188	- "do_remote_prefill": false,
189	- "remote_engine_id": "engine-d-0",
190	- "remote_host": "10.0.0.21",
191	- "remote_port": "1001"
192	- }
193	- }'
194	- ```
195	-
196	-响应示例
197	-
198	-- CDP分离场景，透传P节点响应内容：
199	-
200	- ```json
201	- {
202	- "id": "chatcmpl-xxx12",
203	- "object": "chat.completion",
204	- "created": 1738828800,
205	- "model": "qwen3",
206	- "choices": [
207	- {
208	- "index": 0,
209	- "message": {
210	- "role": "assistant",
211	- "content": "Hello! How can I help you?"
212	- },
213	- "finish_reason": "stop"
214	- }
215	- ],
216	- "usage": {
217	- "prompt_tokens": 6,
218	- "completion_tokens": 7,
219	- "total_tokens": 13
220	- }
221	- }
222	- ```
223	-
224	-- PD分离场景，透传D节点响应内容：
225	-
226	- ```json
227	- {
228	- "id": "chatcmpl-xxx",
229	- "object": "chat.completion",
230	- "created": 1738828800,
231	- "model": "qwen3",
232	- "choices": [
233	- {
234	- "index": 0,
235	- "message": {
236	- "role": "assistant",
237	- "content": "Hello! How can I help you?"
238	- },
239	- "finish_reason": "stop"
240	- }
241	- ],
242	- "usage": {
243	- "prompt_tokens": 8,
244	- "completion_tokens": 9,
245	- "total_tokens": 17
246	- }
247	- }
248	- ```
249	-
250	-输出说明
251	-该示例为非流式 `chat.completion`的输出说明：
252	-
253	-\| 参数 \| 类型 \| 说明 \|
254	-\| --------------------------- \| ------- \| ----------------------------- \|
255	-\| `id` \| string \| 响应 ID。 \|
256	-\| `object` \| string \| 响应对象类型，示例为 `chat.completion`。 \|
257	-\| `created` \| integer \| 响应创建时间（Unix 时间戳）。 \|
258	-\| `model` \| string \| 实际使用的模型名称。 \|
259	-\| `choices` \| array \| 生成结果列表。 \|
260	-\| `choices[].index` \| integer \| 结果序号。 \|
261	-\| `choices[].message.role` \| string \| 角色，示例为 `assistant`。 \|
262	-\| `choices[].message.content` \| string \| 生成内容。 \|
263	-\| `choices[].finish_reason` \| string \| 结束原因，如 `stop`、`length` 等。 \|
264	-\| `usage` \| object \| Token 统计信息。 \|
265	-\| `usage.prompt_tokens` \| integer \| 输入 Token 数量。 \|
266	-\| `usage.completion_tokens` \| integer \| 输出 Token 数量。 \|
267	-\| `usage.total_tokens` \| integer \| 总 Token 数量。 \|
268	-
269	----
270	-
271	-## Native 拉起与 SGLang PD 约定
272	-
273	-当 EngineServer 启用 native CLI 拉起时：
274	-
275	-- 业务口由原生引擎进程占用（`vllm serve` / `sglang.launch_server`），不再经过 Motor InferEndpoint。
276	-- 管理口仍由 EngineServer 提供 `/status`、`/metrics`。
277	-
278	-### SGLang PD
279	-
280	-Coordinator 在识别到实例 `engine_type=sglang` 时，会在发往业务口的请求中直接注入原生字段：
281	-
282	-- `bootstrap_host`：Prefill 实例 IP
283	-- `bootstrap_port`：Prefill Endpoint 注册的原生 PD bootstrap 端口
284	-- `bootstrap_room`：由 `pair_id` + `attempt_seq` 稳定派生
285	-
286	-此时不会再附带 `_motor_dispatch`。SGLang 和 vLLM 的原生 PD 请求差异由 Coordinator 的协议适配器处理。
287	-
288	-### 取消 / stop
289	-
290	-- `/v1/dispatch/stop`：由 Motor InferEndpoint 提供；vLLM 等非 native 路径仍走该接口。
291	-- SGLang pure-native：业务口无 `/v1/dispatch/stop`。Coordinator 改为调用引擎原生 `POST /abort_request`，`rid` 与下发请求中的 `request_id`（`{root_request_id}#a{attempt_seq}`）对齐；失败时仅记录日志，不阻断 Coordinator 侧清理（best-effort）。

Mdocs/zh/user_guide/api/management_interfaces.md+54-3

	@@ -180,7 +180,8 @@ curl -X POST "http://{IP}:{Port}/instances/refresh" \
180	"id": 0,	180	"id": 0,
181	"ip": "192.168.1.1",	181	"ip": "192.168.1.1",
182	"business_port": "8080",	182	"business_port": "8080",
183	- "mgmt_port": "8081"	183	+ "mgmt_port": "8081",
		184	+ "bootstrap_port": 21000
184	}	185	}
185	}	186	}
186	}	187	}
	@@ -206,6 +207,9 @@ curl -X POST "http://{IP}:{Port}/instances/refresh" \
206		207
207	输出说明	208	输出说明
208		209
		210	+`instances[].endpoints` 中的 `bootstrap_port` 为可选字段，仅用于 SGLang PD 原生 bootstrap
		211	+对接；`mgmt_port` 仍是注册协议兼容字段，不代表原生引擎管理 HTTP 端口。
		212	+
209	\| 参数名 \| 类型 \| 说明 \|	213	\| 参数名 \| 类型 \| 说明 \|
210	\|---\|---\|---\|	214	\|---\|---\|---\|
211	\| request_id \| string \| 请求标识。 \|	215	\| request_id \| string \| 请求标识。 \|
	@@ -218,6 +222,52 @@ curl -X POST "http://{IP}:{Port}/instances/refresh" \
218		222
219	---	223	---
220		224
		225	+## 精度告警状态清理接口
		226	+
		227	+接口功能
		228	+
		229	+清理 Coordinator 调度器中指定 P/D 实例组的精度告警状态。该接口供 Controller/运维编排在
		230	+精度告警已处理后调用，不负责终止实例。
		231	+
		232	+接口格式
		233	+
		234	+请求类型：POST
		235	+> URL：`http(s)://{IP}:{Port}/precision/alarm_cleared`
		236	+
		237	+IP 与端口参见[管理接口的IP/端口与配置](./README.md#管理接口的ip端口与配置)
		238	+
		239	+请求头：
		240	+
		241	+- 必选：`Content-Type: application/json`
		242	+
		243	+请求参数
		244	+
		245	+\| 参数名 \| 类型 \| 必填 \| 说明 \|
		246	+\|---\|---\|---\|---\|
		247	+\| `d_instance_id` \| integer \| 是 \| Decode 实例 ID。 \|
		248	+\| `p_instance_id` \| integer \| 否 \| Prefill 实例 ID；不传表示仅按 Decode 实例清理。 \|
		249	+
		250	+使用样例
		251	+
		252	+```bash
		253	+curl -X POST "http://{IP}:{Port}/precision/alarm_cleared" \
		254	+ -H "Content-Type: application/json" \
		255	+ -d '{"d_instance_id": 2, "p_instance_id": 1}'
		256	+```
		257	+
		258	+响应示例
		259	+
		260	+```json
		261	+{
		262	+ "request_id": "precision_alarm_cleared",
		263	+ "status": "success",
		264	+ "message": "Precision alarm state cleared",
		265	+ "data": {"dismissed": true}
		266	+}
		267	+```
		268	+
		269	+---
		270	+
221	## 根路径服务信息接口	271	## 根路径服务信息接口
222		272
223	接口功能	273	接口功能
	@@ -246,12 +296,13 @@ curl -X GET "http://{IP}:{Port}/"
246	{	296	{
247	"service": "Motor Coordinator Management Server",	297	"service": "Motor Coordinator Management Server",
248	"version": "1.0.0",	298	"version": "1.0.0",
249	- "description": "Management plane: liveness, startup, readiness, instance refresh",	299	+ "description": "Management plane: liveness, startup, readiness, metrics, instance refresh",
250	"endpoints": {	300	"endpoints": {
251	"GET /liveness": "liveness check",	301	"GET /liveness": "liveness check",
252	"GET /startup": "startup probe",	302	"GET /startup": "startup probe",
253	"GET /readiness": "readiness check",	303	"GET /readiness": "readiness check",
254	- "POST /instances/refresh": "refresh instances"	304	+ "POST /instances/refresh": "refresh instances",
		305	+ "POST /precision/alarm_cleared": "clear precision alarm scheduler state"
255	}	306	}
256	}	307	}
257	```	308	```

Mdocs/zh/user_guide/configuration/config_reference.md+20-55

	@@ -157,7 +157,7 @@ motor_controller_config字段配置样例如下所示：
157	\| event_consumer_sleep_interval \| float \| 事件队列轮询间隔，即每次处理事件后的等待时间，单位：秒，默认值：1.0。 \|	157	\| event_consumer_sleep_interval \| float \| 事件队列轮询间隔，即每次处理事件后的等待时间，单位：秒，默认值：1.0。 \|
158	\| coordinator_heartbeat_interval \| float \| Controller 与 Coordinator 间心跳上报间隔，单位：秒，默认值：10.0。 \|	158	\| coordinator_heartbeat_interval \| float \| Controller 与 Coordinator 间心跳上报间隔，单位：秒，默认值：10.0。 \|
159	\|<a id="fault_tolerance_config"></a>fault_tolerance_config字段\|-\|-\|	159	\|<a id="fault_tolerance_config"></a>fault_tolerance_config字段\|-\|-\|
160	-\| enable_fault_tolerance \| bool \| 是否启用故障自愈（高级 RAS），默认值：true。取值如下：<ul><li>true：启用</li><li>false：不启用</li></ul> \|	160	+\| enable_fault_tolerance \| bool \| 是否启用 Motor Controller 故障自愈（高级 RAS），默认值：true。取值如下：<ul><li>true：启用</li><li>false：不启用</li></ul>该字段不等价于、也不会自动透传为 vLLM 开发分支的 `--enable-fault-tolerance`。 \|
161	\| strategy_center_check_interval \| int \| 策略中心轮询间隔，单位：秒，默认值：1。 \|	161	\| strategy_center_check_interval \| int \| 策略中心轮询间隔，单位：秒，默认值：1。 \|
162	\| configmap_namespace \|string\|configmap命名空间，默认值："kube-system"。\|	162	\| configmap_namespace \|string\|configmap命名空间，默认值："kube-system"。\|
163	\| configmap_prefix \|string\|configmap前缀，默认值："mindx-dl-deviceinfo-"。\|	163	\| configmap_prefix \|string\|configmap前缀，默认值："mindx-dl-deviceinfo-"。\|
	@@ -493,7 +493,7 @@ motor_coordinator_config字段配置样例如下所示：
493		493
494	## motor_engine_union_config	494	## motor_engine_union_config
495		495
496	-motor_engine_union_config字段用于PD混部场景，配置同一类union Engine Server实例。其结构与motor_engine_prefill_config/motor_engine_decode_config类似，但不区分P/D两套引擎配置，也无需配置 kv_transfer_config的producer/consumer角色。其配置样例如下所示。	496	+motor_engine_union_config字段用于PD混部场景，配置同一类 union 原生引擎实例。其结构与motor_engine_prefill_config/motor_engine_decode_config类似，但不区分P/D两套引擎配置，也无需配置 kv_transfer_config的producer/consumer角色。其配置样例如下所示。
497		497
498	```json	498	```json
499	"motor_engine_union_config": {	499	"motor_engine_union_config": {
	@@ -575,14 +575,15 @@ motor_engine_union_config字段用于PD混部场景，配置同一类union E
575	\| 配置项 \| 类型 \| 说明 \|	575	\| 配置项 \| 类型 \| 说明 \|
576	\|--------\|------\|------------------\|	576	\|--------\|------\|------------------\|
577	\| engine_type \| string \| 引擎类型，如 `vllm` \|	577	\| engine_type \| string \| 引擎类型，如 `vllm` \|
578	-\| engine_config字段 \| - \|engine_config字段中的参数说明详情请参见[vLLM官网参数配置](https://docs.vllm.ai/en/latest/api/vllm/config)\|	578	+\| engine_config字段 \| - \| `engine_config` 直接映射所选引擎的原生启动参数；请参阅对应 vLLM/SGLang 版本的官方参数文档。 \|
579	\| motor_nodemanger_config字段 \|-\|-\|	579	\| motor_nodemanger_config字段 \|-\|-\|
580	\| api_config.pod_ip \|string \| Pod IP（由环境或部署注入）。默认值：`127.0.0.1`（或 Env.pod_ip） \|	580	\| api_config.pod_ip \|string \| Pod IP（由环境或部署注入）。默认值：`127.0.0.1`（或 Env.pod_ip） \|
581	\| api_config.node_manager_port \|int \| NodeManager 端口。默认值：`1026` \|	581	\| api_config.node_manager_port \|int \| NodeManager 端口。默认值：`1026` \|
582	\| endpoint_config.endpoint_num \|int \| 引擎端点数量，通常由 HCCL/并行配置推导。默认值：`0` \|	582	\| endpoint_config.endpoint_num \|int \| 引擎端点数量，通常由 HCCL/并行配置推导。默认值：`0` \|
583	\| endpoint_config.base_port \|int \| 端点端口起始号。默认值：`10000` \|	583	\| endpoint_config.base_port \|int \| 端点端口起始号。默认值：`10000` \|
584	-\| endpoint_config.mgmt_ports \|array \| 各端点管控端口列表（整数数组）。默认值：`[]` \|	584	+\| endpoint_config.mgmt_ports \|array \| 各端点兼容管理端口列表（整数数组）。原生引擎不监听该端口，当前为注册协议兼容字段。默认值：`[]` \|
585	\| endpoint_config.service_ports \|array \| 各端点推理服务端口列表（整数数组）。默认值：`[]` \|	585	\| endpoint_config.service_ports \|array \| 各端点推理服务端口列表（整数数组）。默认值：`[]` \|
		586	+\| endpoint_config.bootstrap_port \|int/null \| SGLang PD 原生 bootstrap 端口。由所选引擎配置中的 `engine_config.disaggregation_bootstrap_port`（兼容 `disaggregation-bootstrap-port`）派生；vLLM 或未配置时为空。 \|
586	\| fault_tolerance_config.enable_fault_tolerance \|bool\|是否显式开启引擎软件故障轮询，默认值：false。<br>引擎 user config 检测到 FT 开关时自动开启，无需显式配置。\|	587	\| fault_tolerance_config.enable_fault_tolerance \|bool\|是否显式开启引擎软件故障轮询，默认值：false。<br>引擎 user config 检测到 FT 开关时自动开启，无需显式配置。\|
587	\| fault_tolerance_config.poll_interval_sec \|float\|轮询引擎 FT 状态的时间间隔，单位：秒，默认值：5.0。\|	588	\| fault_tolerance_config.poll_interval_sec \|float\|轮询引擎 FT 状态的时间间隔，单位：秒，默认值：5.0。\|
588	\| fault_tolerance_config.poll_timeout_sec \|float\|单次轮询的 HTTP 超时，单位：秒，默认值：5.0。\|	589	\| fault_tolerance_config.poll_timeout_sec \|float\|单次轮询的 HTTP 超时，单位：秒，默认值：5.0。\|
	@@ -612,7 +613,7 @@ motor_engine_union_config字段用于PD混部场景，配置同一类union E
612		613
613	## motor_engine_prefill_config/motor_engine_decode_config	614	## motor_engine_prefill_config/motor_engine_decode_config
614		615
615	-motor_engine_prefill_config和motor_engine_decode_config字段用于PD分离部署场景，这两个字段分别配置Prefill与Decode引擎。两者结构相同，均需指定engine_type与engine_config；可选配置dispatch_profile（PD协同语义）与health_check_config（虚推健康探测，见 [虚推健康探测](../features/sim_inference.md)）。配置示例如下所示。	616	+motor_engine_prefill_config和motor_engine_decode_config字段用于PD分离部署场景，这两个字段分别配置Prefill与Decode引擎。两者结构相同，均需指定engine_type与engine_config；`health_check_config` 用于配置原生 `/health` 超时与模型加载启动窗口。配置示例如下所示。
616		617
617	```json	618	```json
618	"motor_engine_prefill_config": {	619	"motor_engine_prefill_config": {
	@@ -773,14 +774,15 @@ motor_engine_prefill_config和motor_engine_decode_config字段用于**PD分离
773	\| 配置项 \| 类型 \| 说明 \|	774	\| 配置项 \| 类型 \| 说明 \|
774	\|--------\|------\|------------------\|	775	\|--------\|------\|------------------\|
775	\| engine_type \| string \| 引擎类型，如 `vllm` \|	776	\| engine_type \| string \| 引擎类型，如 `vllm` \|
776	-\| engine_config字段 \| - \| engine_config字段中的参数说明详情请参见[vLLM官网参数配置](https://docs.vllm.ai/en/latest/api/vllm/config) \|	777	+\| engine_config字段 \| - \| `engine_config` 直接映射所选引擎的原生启动参数；请参阅对应 vLLM/SGLang 版本的官方参数文档。 \|
777	\| motor_nodemanger_config字段 \|-\|-\|	778	\| motor_nodemanger_config字段 \|-\|-\|
778	\| api_config.pod_ip \|string \| Pod IP（由环境或部署注入）。默认值：`127.0.0.1`（或 Env.pod_ip） \|	779	\| api_config.pod_ip \|string \| Pod IP（由环境或部署注入）。默认值：`127.0.0.1`（或 Env.pod_ip） \|
779	\| api_config.node_manager_port \|int \| NodeManager 端口。默认值：`1026` \|	780	\| api_config.node_manager_port \|int \| NodeManager 端口。默认值：`1026` \|
780	\| endpoint_config.endpoint_num \|int \| 引擎端点数量，通常由 HCCL/并行配置推导。默认值：`0` \|	781	\| endpoint_config.endpoint_num \|int \| 引擎端点数量，通常由 HCCL/并行配置推导。默认值：`0` \|
781	\| endpoint_config.base_port \|int \| 端点端口起始号。默认值：`10000` \|	782	\| endpoint_config.base_port \|int \| 端点端口起始号。默认值：`10000` \|
782	-\| endpoint_config.mgmt_ports \|array \| 各端点管控端口列表（整数数组）。默认值：`[]` \|	783	+\| endpoint_config.mgmt_ports \|array \| 各端点兼容管理端口列表（整数数组）。原生引擎不监听该端口，当前为注册协议兼容字段。默认值：`[]` \|
783	\| endpoint_config.service_ports \|array \| 各端点推理服务端口列表（整数数组）。默认值：`[]` \|	784	\| endpoint_config.service_ports \|array \| 各端点推理服务端口列表（整数数组）。默认值：`[]` \|
		785	+\| endpoint_config.bootstrap_port \|int/null \| SGLang PD 原生 bootstrap 端口。由所选引擎配置中的 `engine_config.disaggregation_bootstrap_port`（兼容 `disaggregation-bootstrap-port`）派生；vLLM 或未配置时为空。 \|
784	\| fault_tolerance_config.enable_fault_tolerance \|bool\|是否显式开启引擎软件故障轮询，默认值：false。<br>引擎 user config 检测到 FT 开关时自动开启，无需显式配置。\|	786	\| fault_tolerance_config.enable_fault_tolerance \|bool\|是否显式开启引擎软件故障轮询，默认值：false。<br>引擎 user config 检测到 FT 开关时自动开启，无需显式配置。\|
785	\| fault_tolerance_config.poll_interval_sec \|float\|轮询引擎 FT 状态的时间间隔，单位：秒，默认值：5.0。\|	787	\| fault_tolerance_config.poll_interval_sec \|float\|轮询引擎 FT 状态的时间间隔，单位：秒，默认值：5.0。\|
786	\| fault_tolerance_config.poll_timeout_sec \|float\|单次轮询的 HTTP 超时，单位：秒，默认值：5.0。\|	788	\| fault_tolerance_config.poll_timeout_sec \|float\|单次轮询的 HTTP 超时，单位：秒，默认值：5.0。\|
	@@ -810,64 +812,28 @@ PD模式下P与D各自独立配置"health_check_config"，未配置时使用
810		812
811	### dispatch_profile	813	### dispatch_profile
812		814
813	-当engine_config.kv_transfer_config.kv_connector不在内置识别白名单内时，可在motor_engine_prefill_config/motor_engine_decode_config顶层显式声明 P/D 协同语义。NodeManager根据此推导并向Coordinator上报dispatch_capabilities。	815	+当 `engine_config.kv_transfer_config.kv_connector` 不在内置识别白名单内时，可在 `motor_engine_prefill_config` / `motor_engine_decode_config` 顶层显式声明 P/D 协同语义。NodeManager 根据该字段推导兼容元数据，并在构造原生 vLLM 启动命令时校验其语义。
814	-
815	-表6 dispatch_profile参数说明
816		816
817	\| 配置项 \| 类型 \| 说明 \|	817	\| 配置项 \| 类型 \| 说明 \|
818	-\|--------\|------\|--------\|	818	+\|--------\|------\|------\|
819	-\| dispatch_profile \| string \| P/D 协同语义。默认值：未配时由 kv_connector白名单推断。<br>可选值：<ul><li>handoff：Prefill完成后交给Decode，推导出的capability为prefill_handoff_decode。</li><li>trigger：P/D并发，引擎同步KV，推导出的capability为concurrent_engine_sync。</li></ul>Prefill与Decode两端须配置相同取值。 \|	819	+\| dispatch_profile \| string \| 可选值为 `handoff` 或 `trigger`。当前原生 vLLM P/D 启动仅接受 `handoff`；SGLang 使用自身 bootstrap 协议。Prefill 与 Decode 两端应保持一致。 \|
820	-
821	-vLLM内置识别的kv_connector白名单见[PD 分离特性说明](../../design/pd_disaggregation.md#vllm-connector-识别白名单)。白名单内connector无需手动配置dispatch_profile。
822		820
823	>[!NOTE]说明	821	>[!NOTE]说明
824	->	822	+> `dispatch_profile` 写在 `motor_engine_*_config` 顶层，不是在 `engine_config` 内部。`dispatch_capabilities` 为内部兼容字段，不支持用户直接填写。
825	->- dispatch_profile写在motor_engine_*_config顶层，不是在engine_config字段内部。
826	->- 不支持用户直接填写dispatch_capabilities，配置后会被NodeManager丢弃。
827	->- 取值须与connector实际协同语义一致；P/D不一致或无共同capability时，Coordinator路由返回503。
828	-
829	-配置示例（自定义connector）：
830	-
831	-```json
832	-"motor_engine_prefill_config": {
833	- "engine_type": "vllm",
834	- "dispatch_profile": "handoff",
835	- "engine_config": {
836	- ...
837	- "kv_transfer_config": {
838	- "kv_connector": "YourCustomConnector",
839	- "kv_role": "kv_producer",
840	- ...
841	- }
842	- }
843	-},
844	-"motor_engine_decode_config": {
845	- "engine_type": "vllm",
846	- "dispatch_profile": "handoff",
847	- "engine_config": {
848	- ...
849	- "kv_transfer_config": {
850	- "kv_connector": "YourCustomConnector",
851	- "kv_role": "kv_consumer",
852	- ...
853	- }
854	- }
855	-}
856	-```
857		823
858	### health_check_config	824	### health_check_config
859		825
860	-可选虚推（虚拟推理）健康探测配置，位于 `motor_engine_prefill_config` / `motor_engine_decode_config` 模块，默认关闭，机制说明见 [虚推健康探测](../features/sim_inference.md)。	826	+原生引擎健康探测配置，位于 `motor_engine_prefill_config` / `motor_engine_decode_config` 模块。
861		827
862	表7 health_check_config字段参数说明	828	表7 health_check_config字段参数说明
863		829
864	\| 配置项 \| 类型 \| 说明 \|	830	\| 配置项 \| 类型 \| 说明 \|
865	\|--------\|------\|--------\|	831	\|--------\|------\|--------\|
866	-\| enable_virtual_inference \| bool \| 虚推总开关，默认值：false。<br>取值为 `true` 时，在推理面 `/health` 正常后启动周期性虚推。<br>仅允许在 ERROR 日志级别下开启（`ASCEND_GLOBAL_LOG_LEVEL=3`，未配置默认 ERROR）；显式配置为非 ERROR 时 `deploy.py` 会强制关闭并打印 warning。 \|
867	-\| npu_usage_threshold \| int \| AI Cube 利用率阈值（%），默认值：3。<br>虚推仅在 `0 < npu_usage_threshold <= 100` 时启动；低于该阈值且虚推失败时累计失败次数加1。 \|
868	-\| max_failure_count \| int \| 连续虚推失败次数上限（在累计条件满足后），默认值：6。<br>达到后Engine Server `/status` 返回abnormal。 \|
869	\| health_collector_timeout \| int \| 推理面 `GET /health` 探测超时（秒），默认值：5。 \|	832	\| health_collector_timeout \| int \| 推理面 `GET /health` 探测超时（秒），默认值：5。 \|
870	-\| health_collector_timeout_retry_attempts \| int \| 推理面 `GET /health` 超时重试次数（含首次），默认值：3。<br>仅在探测超时时重试；连接失败、HTTP 错误等其它异常不重试。 \|	833	+\| health_collector_timeout_retry_attempts \| int \| 单次 `GET /health` 超时后的最大尝试次数（包含首次请求），默认值：3；仅超时重试。 \|
		834	+\| startup_timeout \| int \| 原生引擎模型加载启动窗口（秒），默认值：1800。窗口内连接失败保持 STARTING，不判定实例异常。 \|
		835	+
		836	+旧版 `enable_virtual_inference`、`npu_usage_threshold` 和 `max_failure_count` 字段仅为配置兼容保留，原生引擎运行时不消费。
871		837
872	---	838	---
873		839
	@@ -875,7 +841,7 @@ vLLM内置识别的kv_connector白名单见[PD 分离特性说明](../../design/
875		841
876	### motor_engine_union_env字段	842	### motor_engine_union_env字段
877		843
878	-PD混部场景下，union Engine Server 的环境变量配置在 `env.json` 的 `motor_engine_union_env` 中。示例可参考 `examples/infer_engines/vllm/pd_hybrid/env.json`。	844	+PD混部场景下，union 原生引擎的环境变量配置在 `env.json` 的 `motor_engine_union_env` 中。示例可参考 `examples/infer_engines/vllm/pd_hybrid/env.json`。
879		845
880	配置示例：	846	配置示例：
881		847
	@@ -900,8 +866,7 @@ PD混部场景下，union Engine Server 的环境变量配置在 `env.json` 的
900	### prefill_kv_event_config 自动推导	866	### prefill_kv_event_config 自动推导
901		867
902	该字段加载 `user_config.json` 时由 Coordinator 合并，一般无需手动添加。	868	该字段加载 `user_config.json` 时由 Coordinator 合并，一般无需手动添加。
903	-Coordinator 会根据实例角色自动识别 P/D 分离或 union 混部拓扑，并根据引擎 Connector 推导、由 NodeManager 内部上报的 `dispatch_capabilities` 选择并发或 handoff 行为。该字段不支持用户显式配置；自定义 Connector 可在 `motor_engine_prefill_config` / `motor_engine_decode_config` 顶层使用 `dispatch_profile` 声明语义，详情请参见[dispatch_profile](#dispatch_profile)。	869	+Coordinator 会根据实例角色自动识别 P/D 分离或 union 混部拓扑，并根据 `engine_type` 选择 vLLM handoff 或 SGLang bootstrap Adapter。vLLM Connector 白名单、`MultiConnector` 取 `connectors[0]` 的规则，以及未知 Connector 在启动期 fail-closed 的处理，详情请参见[PD 分离特性说明](../../design/pd_disaggregation.md#vllm-connector-识别白名单)与[PD 分离服务部署](../deployment/k8s/pd_disaggregation_deployment.md)。
904	-Connector 识别白名单、`MultiConnector` 取 `connectors[0]` 的规则，以及未识别连接器导致路由 503（fail-closed）的处理，详情请参见[PD 分离特性说明](../../design/pd_disaggregation.md#vllm-connector-识别白名单)与[PD 分离服务部署](../deployment/k8s/pd_disaggregation_deployment.md)。
905		870
906	表9 prefill_kv_event_config说明	871	表9 prefill_kv_event_config说明
907		872

Mdocs/zh/user_guide/deployment/cases/ModelArts&yuanrong/ModelArts_GLM5_yuanrong_PyMotor_8Node_BigEP_Deployment_Guide.md+3-3

	@@ -935,9 +935,9 @@ user_config.json内容如下
935	}	935	}
936	},	936	},
937	"health_check_config": {	937	"health_check_config": {
938	- "enable_virtual_inference": false,	938	+ "health_collector_timeout": 5,
939	- "npu_usage_threshold": 3,	939	+ "health_collector_timeout_retry_attempts": 3,
940	- "max_failure_count": 6	940	+ "startup_timeout": 1800
941	}	941	}
942	}	942	}
943	},	943	},

Mdocs/zh/user_guide/deployment/k8s/pd_aggregation_deployment.md+4-4

	@@ -4,7 +4,7 @@
4		4
5	### PD 混部介绍	5	### PD 混部介绍
6		6
7	-PD 混部将 Prefill 与 Decode 能力部署在同一类 Engine Server 实例中。部署时不再分别拉起 prefill、decode 两类角色，而是由 `union` 角色承载完整推理能力；Coordinator 以 `single_node` 调度模式将请求分发到可用的 union 实例。	7	+PD 混部将 Prefill 与 Decode 能力部署在同一类原生引擎实例中。部署时不再分别拉起 prefill、decode 两类角色，而是由 `union` 角色承载完整推理能力；Coordinator 以 `single_node` 调度模式将请求分发到可用的 union 实例。
8		8
9	与 [PD 分离部署](./pd_disaggregation_deployment.md) 相比，PD 混部减少了 P/D 角色拆分和 KV 跨角色传输配置，适用于快速验证、中小规模服务、资源规模较小或暂不需要独立规划 P/D 实例比例的场景。若业务需要针对 Prefill、Decode 两阶段分别规划资源、独立扩缩容或使用 PD 分离相关能力，建议使用 PD 分离部署。	9	与 [PD 分离部署](./pd_disaggregation_deployment.md) 相比，PD 混部减少了 P/D 角色拆分和 KV 跨角色传输配置，适用于快速验证、中小规模服务、资源规模较小或暂不需要独立规划 P/D 实例比例的场景。若业务需要针对 Prefill、Decode 两阶段分别规划资源、独立扩缩容或使用 PD 分离相关能力，建议使用 PD 分离部署。
10		10
	@@ -13,7 +13,7 @@
13	部署流程围绕三个入口展开：	13	部署流程围绕三个入口展开：
14		14
15	1. `user_config.json`：部署与业务的总配置，PD 混部重点配置 `hybrid_*` 字段、`motor_engine_union_config` 以及 Coordinator 的 `single_node` 调度模式。	15	1. `user_config.json`：部署与业务的总配置，PD 混部重点配置 `hybrid_*` 字段、`motor_engine_union_config` 以及 Coordinator 的 `single_node` 调度模式。
16	-2. `env.json`：各组件环境变量，PD 混部的 Engine Server 环境变量配置在 `motor_engine_union_env` 中。	16	+2. `env.json`：各组件环境变量，PD 混部的原生引擎环境变量配置在 `motor_engine_union_env` 中。
17	3. 部署脚本 `deploy.py`：读取上述配置，生成 K8s YAML、更新启动脚本、创建 ConfigMap 并执行 `kubectl apply`。	17	3. 部署脚本 `deploy.py`：读取上述配置，生成 K8s YAML、更新启动脚本、创建 ConfigMap 并执行 `kubectl apply`。
18		18
19	部署方式：PD 混部默认使用 CRD 方式（`infer_service_set`），由 InferServiceSet 中的 `union` 角色拉起混部实例。若需沿用传统多 YAML Deployment，可在 `motor_deploy_config.deploy_mode` 中显式配置为 `multi_deployment`；但推荐优先使用默认 CRD 方式。	19	部署方式：PD 混部默认使用 CRD 方式（`infer_service_set`），由 InferServiceSet 中的 `union` 角色拉起混部实例。若需沿用传统多 YAML Deployment，可在 `motor_deploy_config.deploy_mode` 中显式配置为 `multi_deployment`；但推荐优先使用默认 CRD 方式。
	@@ -167,7 +167,7 @@ PD 混部场景不再需要配置 Coordinator 调度模式。Coordinator 会根
167		167
168	### motor_engine_union_config（混部引擎）	168	### motor_engine_union_config（混部引擎）
169		169
170	-`motor_engine_union_config` 用于配置混部 Engine Server。其结构与 PD 分离中的 `motor_engine_prefill_config` / `motor_engine_decode_config` 类似，但无需分别配置 P/D 两套引擎，也无需配置 `kv_transfer_config` 的 producer/consumer 角色。	170	+`motor_engine_union_config` 用于配置混部原生引擎。其结构与 PD 分离中的 `motor_engine_prefill_config` / `motor_engine_decode_config` 类似，但无需分别配置 P/D 两套引擎，也无需配置 `kv_transfer_config` 的 producer/consumer 角色。
171		171
172	配置示例：	172	配置示例：
173		173
	@@ -212,7 +212,7 @@ PD 混部场景不再需要配置 Coordinator 调度模式。Coordinator 会根
212		212
213	## 配置 `env.json`	213	## 配置 `env.json`
214		214
215	-PD 混部可直接参考 `examples/infer_engines/vllm/pd_hybrid/env.json`。混部 Engine Server 的环境变量配置在 `motor_engine_union_env` 中。	215	+PD 混部可直接参考 `examples/infer_engines/vllm/pd_hybrid/env.json`。混部原生引擎的环境变量配置在 `motor_engine_union_env` 中。
216		216
217	配置示例：	217	配置示例：
218		218

Mdocs/zh/user_guide/deployment/k8s/pd_disaggregation_deployment.md+9-11

	@@ -136,7 +136,7 @@
136		136
137	## 特性配置指导	137	## 特性配置指导
138		138
139	-上文 `user_config.json` 与 `env.json` 全量示例已默认开启主备倒换、异常实例重启、服务限流、虚推、KV 亲和性调度、KV 池化等能力。若只需调整某项能力，可对照本节做最小配置修改。	139	+上文 `user_config.json` 与 `env.json` 全量示例已默认开启主备倒换、异常实例重启、服务限流、KV 亲和性调度、KV 池化等能力。若只需调整某项能力，可对照本节做最小配置修改。
140		140
141	### 主备倒换	141	### 主备倒换
142		142
	@@ -188,26 +188,24 @@ P/D 实例出现异常时，重启推理实例，避免实例长时间处于异
188	- 关闭：删除 `rate_limit_config` 配置块，或将 `enable_rate_limit` 设为 `false`。	188	- 关闭：删除 `rate_limit_config` 配置块，或将 `enable_rate_limit` 设为 `false`。
189	- 注意：字段详细说明请参见[motor_coordinator_config](../../configuration/config_reference.md#motor_coordinator_config)中的rate_limit_config字段。	189	- 注意：字段详细说明请参见[motor_coordinator_config](../../configuration/config_reference.md#motor_coordinator_config)中的rate_limit_config字段。
190		190
191	-### 虚推健康检查 (Virtual Inference Health Check)	191	+### 原生引擎健康探测
192		192
193	-探测服务健康状态，避免静默故障带来业务损失。静默故障表现为：部分进程卡死，服务看似无问题，但无法正常推理。	193	+Engine Server 删除后，NodeManager 不再执行虚拟推理或采集 AI Cube 利用率。旧配置中的 `enable_virtual_inference`、`npu_usage_threshold` 和 `max_failure_count` 仅为配置兼容字段，原生运行时不消费。
194		194
195	-- 原理：业务流量较小时发送轻量级推理请求；业务流量较大时查看 NPU 计算核心使用率。不健康的 P/D 实例会被重启以消除静默故障。	195	+- 就绪探测：NodeManager 轮询原生引擎业务端口的 `/health`，在 `startup_timeout` 窗口内保持 STARTING，成功后才允许调度。
196	-- 开启：	196	+- 进程存活：NodeManager 监管原生引擎进程组；主进程或工作进程异常退出时触发实例恢复。
197	- P 和 D 实例需要单独开启虚推功能：P 实例虚推健康检查开启方式如下，D 实例的开启方式相同。	197	+- 软件故障：引擎提供 `/fault_tolerance/status` 时，可由 FaultReporter 补充软件故障上报。
198		198
199	```json	199	```json
200	"motor_engine_prefill_config": {	200	"motor_engine_prefill_config": {
201	"health_check_config": {	201	"health_check_config": {
202	- "enable_virtual_inference": true,	202	+ "health_collector_timeout": 5,
203	- "npu_usage_threshold": 10	203	+ "health_collector_timeout_retry_attempts": 3,
		204	+ "startup_timeout": 1800
204	}	205	}
205	}	206	}
206	```	207	```
207		208
208	-- 关闭：删除 `health_check_config` 配置块，或将 `enable_virtual_inference` 设为 `false`。
209	-- 注意：虚推仅允许在 ERROR 日志级别下开启（`ASCEND_GLOBAL_LOG_LEVEL=3`，未配置默认即为 ERROR）。若在 `env.json` 中显式配置为非 ERROR，`deploy.py` 会强制关闭虚推并打印 warning。该功能使用详情请参见[虚推健康检查](../../features/sim_inference.md)。
210	-
211	### KV Cache 亲和调度	209	### KV Cache 亲和调度
212		210
213	将具有相同前缀的请求调度到同一实例，复用已有 KV Cache，减少 Prefill 耗时。	211	将具有相同前缀的请求调度到同一实例，复用已有 KV Cache，减少 Prefill 耗时。

Mdocs/zh/user_guide/features/container_snapshot.md+7-7

	@@ -2,7 +2,7 @@
2		2
3	## 特性介绍	3	## 特性介绍
4		4
5	-容器快照特性用于保存实例节点容器的运行状态，并在实例重调度等场景中快速恢复推理服务。Motor 服务框架负责 Device 侧的 suspend、resume 及恢复后的控制面注册；MindCluster 或用户负责对实例节点容器执行 Host 侧 checkpoint。	5	+容器快照特性用于保存实例节点容器的运行状态，并在实例重调度等场景中快速恢复推理服务。推理引擎负责 Device 侧的 suspend、resume；Motor 服务框架负责快照前后刷新控制面状态（Controller 域名、job_name、pod_ip）、准备快照元数据，并通过引擎状态感知保存/恢复是否完成。MindCluster 或用户负责对实例节点容器执行 Host 侧 checkpoint。
6		6
7	容器快照由以下两部分组成：	7	容器快照由以下两部分组成：
8		8
	@@ -24,12 +24,12 @@
24		24
25	容器快照制作流程如下：	25	容器快照制作流程如下：
26		26
27	-1. 实例冷启动并进入健康状态后，Engine Server 自动执行 suspend，锁定 Device 状态、保存 Device 快照，并将运行时模型权重写入 `model_save_path`。	27	+1. 实例冷启动并进入健康状态后，引擎完成 Device 侧 suspend：锁定 Device 状态、保存 Device 快照，并将运行时模型权重写入 `model_save_path`。Node Manager 仅准备快照元数据，不触发显存快照保存。
28	-2. 当本节点全部 Engine Server 均完成 suspend 后，实例节点容器到达稳态点。	28	+2. 当本节点全部原生引擎 Endpoint 均完成 suspend 后，实例节点容器到达稳态点。
29	- MindCluster 实例重调度场景：通过 Node Manager 的 `/readiness` 返回 `200` 判断。	29	- MindCluster 实例重调度场景：通过 Node Manager 的 `/readiness` 返回 `200` 判断。
30	- 用户自定义应用场景：通过 `/node-manager/status` 返回 `200 {"status": true}` 判断。	30	- 用户自定义应用场景：通过 `/node-manager/status` 返回 `200 {"status": true}` 判断。
31	3. 到达稳态点后，MindCluster 或用户使用 grus 对实例节点容器执行 checkpoint，保存容器 Host 快照镜像。	31	3. 到达稳态点后，MindCluster 或用户使用 grus 对实例节点容器执行 checkpoint，保存容器 Host 快照镜像。
32	-4. Host 侧 checkpoint 完成后，将元数据字段 `checkpoint` 更新为 `"done"`。Engine Server 检测到该状态后解锁 Device，冷启动实例恢复提供推理服务。	32	+4. Host 侧 checkpoint 完成后，将元数据字段 `checkpoint` 更新为 `"done"`。引擎检测到该状态后自行解锁 Device，冷启动实例恢复提供推理服务。
33		33
34	处于 checkpoint 过程中的实例无法提供推理服务。到达稳态点但 `checkpoint` 尚未完成时，Node Manager 暂停向 Controller 上报正常心跳。	34	处于 checkpoint 过程中的实例无法提供推理服务。到达稳态点但 `checkpoint` 尚未完成时，Node Manager 暂停向 Controller 上报正常心跳。
35		35
	@@ -39,8 +39,8 @@
39		39
40	1. MindCluster 或用户从 Host 快照镜像恢复实例节点容器，并挂载对应的运行时权重和快照元数据文件。	40	1. MindCluster 或用户从 Host 快照镜像恢复实例节点容器，并挂载对应的运行时权重和快照元数据文件。
41	2. Node Manager 从元数据读取 `job_name` 和 `namespace`，刷新 Pod IP 与 Controller DNS，然后向 Controller 重新注册。	41	2. Node Manager 从元数据读取 `job_name` 和 `namespace`，刷新 Pod IP 与 Controller DNS，然后向 Controller 重新注册。
42	-3. Controller 下发启动命令。Node Manager 更新快照元数据文件中的 `model_load_path` 与 `data_parallel_master_ip` 字段；快照恢复场景不会重新创建 Engine Server 进程。	42	+3. Controller 下发启动命令。Node Manager 更新快照元数据文件中的 `model_load_path` 与 `data_parallel_master_ip` 字段；快照恢复场景继续使用容器快照中恢复的原生引擎进程。
43	-4. Engine Server 使用元数据中的 `model_load_path` 与 `data_parallel_master_ip` 执行 resume。全部 endpoint 恢复为 `NORMAL` 后，实例重新进入就绪状态。	43	+4. 引擎读取元数据中的 `model_load_path` 与 `data_parallel_master_ip` 完成 resume。全部 Endpoint 恢复健康后，实例重新进入就绪状态。
44		44
45	## 启用制作容器快照配置	45	## 启用制作容器快照配置
46		46
	@@ -71,7 +71,7 @@
71	\| `job_name` \| 快照恢复 \| 从容器快照恢复前必须准备 \| 推理实例的唯一标识，恢复后注册时用于更新 Node Manager 的任务名 \|	71	\| `job_name` \| 快照恢复 \| 从容器快照恢复前必须准备 \| 推理实例的唯一标识，恢复后注册时用于更新 Node Manager 的任务名 \|
72	\| `namespace` \| 快照恢复 \| Controller 使用集群内 `.svc.cluster.local` DNS 时必须准备 \| 推理服务所属 namespace，用于更新 Controller DNS；非集群 DNS 场景可不配置 \|	72	\| `namespace` \| 快照恢复 \| Controller 使用集群内 `.svc.cluster.local` DNS 时必须准备 \| 推理服务所属 namespace，用于更新 Controller DNS；非集群 DNS 场景可不配置 \|
73	\| `data_parallel_master_ip` \| 快照恢复 \| 可不预先配置 \| 实例 Master DP 所在 Pod 的 IP；优先使用文件中的值，未配置时由 Node Manager 写入 Controller 下发值 \|	73	\| `data_parallel_master_ip` \| 快照恢复 \| 可不预先配置 \| 实例 Master DP 所在 Pod 的 IP；优先使用文件中的值，未配置时由 Node Manager 写入 Controller 下发值 \|
74	-\| `checkpoint` \| 快照制作 \| Host 侧 checkpoint 完成后写入 \| 更新为 `"done"` 后，Engine Server 解锁 Device，冷启动实例恢复推理服务 \|	74	+\| `checkpoint` \| 快照制作 \| Host 侧 checkpoint 完成后写入 \| 更新为 `"done"` 后，引擎解锁 Device，冷启动实例恢复推理服务 \|
75		75
76	用户自定义应用场景，制作容器快照前, 需要准备 `model_save_path` 字段; 从容器快照恢复前，需要准备 `model_load_path` 和 `job_name`；使用集群内 Controller DNS 时还需准备 `namespace`。	76	用户自定义应用场景，制作容器快照前, 需要准备 `model_save_path` 字段; 从容器快照恢复前，需要准备 `model_load_path` 和 `job_name`；使用集群内 Controller DNS 时还需准备 `namespace`。
77		77

Mdocs/zh/user_guide/features/kv_cache_store/README.md+1-1

	@@ -97,7 +97,7 @@ MindIE Motor开启KV池化能力只需修改`user_config.json`配置文件，其
97	\| [MemCache](backend/memcache.md) \| `memcache` \| 默认后端，天然支持，无需额外安装 \|	97	\| [MemCache](backend/memcache.md) \| `memcache` \| 默认后端，天然支持，无需额外安装 \|
98	\| Yuanrong \| `yuanrong` \| TODO：后续版本支持 \|	98	\| Yuanrong \| `yuanrong` \| TODO：后续版本支持 \|
99		99
100	-> 关于 Connector 的更多原理，以及识别白名单与 `dispatch_profile` 逃生口，请参见 [PD 分离特性说明](../../../design/pd_disaggregation.md#connector-驱动执行计划)。	100	+> 关于 Connector 的 handoff 白名单和 `MultiConnector` 传输层规则，请参见 [PD 分离特性说明](../../../design/pd_disaggregation.md#connector-驱动执行计划)。
101		101
102	### kv_cache_store_config（全局配置）	102	### kv_cache_store_config（全局配置）
103		103

Ddocs/zh/user_guide/features/sim_inference.md+0-66

	@@ -1,66 +0,0 @@
1	-# 虚推健康探测
2	-
3	-## 特性介绍
4	-
5	-虚推（虚拟推理，实现见 `motor/engine_server/core/sim_inference.py`）用于在业务低负载时主动向推理面发送轻量请求，结合 NPU AI Cube 利用率判断 Engine Server 推理引擎是否可用。配置位于 `user_config` 中 `motor_engine_prefill_config` / `motor_engine_decode_config` 的 `health_check_config` 子块，默认关闭。
6	-
7	-Node Manager 周期性请求 Engine Server mgmt 面 `GET /status`；返回值综合推理面 `/health` 与虚推结果。连续 abnormal 时 `HeartbeatManager` 可触发节点自杀重调度。
8	-
9	-版本要求：虚推仅支持 HDK 26.0.RC1 及以后版本。
10	-
11	-## 工作机制
12	-
13	-启用条件（须同时满足）：
14	-
15	-1. `engine_type` 为 `vllm`（SGLang 引擎即使配置 `enable_virtual_inference: true` 也会在运行时被自动关闭）
16	-2. `health_check_config.enable_virtual_inference` 为 `true`
17	-3. 该引擎角色生效的 `ASCEND_GLOBAL_LOG_LEVEL` 为 ERROR（`3`；0=DEBUG、1=INFO、2=WARNING、3=ERROR）。虚推仅允许在 ERROR 日志级别下开启；未配置时默认即为 ERROR。若显式配置为非 `3`，`deploy.py` 会强制将 `enable_virtual_inference` 设为 `false` 并打印 warning
18	-4. `0 < health_check_config.npu_usage_threshold <= 100`
19	-5. 推理面 `GET /health` 返回正常（由 `HealthCollector` 探测，`health_collector_timeout` 控制超时, `health_collector_timeout_retry_attempts` 控制超时重试次数）
20	-6. 仅 DP rank 0 执行虚推（非 DP0 节点运行时自动关闭）
21	-
22	-满足条件后，`mgmt_endpoint.py` 在首次 `/status` 请求时调用 `run_virtual_inference()` 启动虚推循环。
23	-
24	-虚推请求：向推理面 `POST /v1/completions`，请求体为 `prompt: "1"`、`max_tokens: 1`。vLLM layerwise decode（`dispatch_profile=trigger`）额外携带 `kv_transfer_params.do_virtual: true` 及 PD 分离相关字段；handoff decode 与 Prefill/Union 角色发送普通 completion 请求。
25	-
26	-NPU 负载采样：使用 `npu-smi info watch -s u` 采集 AI Cube 利用率（AI Cube Usage）。启动虚推前会通过 `npu-smi info watch -h` 检查 help 是否包含 `u - AI Cube Usage`；若当前 HDK 不支持该指标，Engine Server 会自动关闭虚推。
27	-
28	-动态探测间隔：
29	-
30	-\| AI Cube 利用率峰值（5 秒采样窗口） \| 下一轮间隔 \|
31	-\|-----------------------------------\|------------\|
32	-\| ≥ 80% \| 20 秒 \|
33	-\| < `npu_usage_threshold` \| 5 秒（默认） \|
34	-\| `[npu_usage_threshold, 80%)` \| 保持当前间隔不变 \|
35	-
36	-异常判定：当 AI Cube 利用率峰值低于 `npu_usage_threshold` 且虚推请求失败时，累计连续失败次数；达到 `max_failure_count` 后，`GET /status` 返回 `abnormal` 且虚推循环停止。Node Manager 的 `HeartbeatManager` 连续 5 次收到 abnormal 后触发自杀重调度。
37	-
38	-vLLM 指标过滤（v0.18+）：启用虚推时，Engine Server 会 patch vLLM `OutputProcessor._update_stats_from_finished`，在写入 per-request 指标前跳过 `external_req_id` 含 `_virtual` 后缀的虚推请求（对应虚推 `X-Request-Id: {timestamp}_virtual`）。仅过滤 `request_success_total` 等 per-request 指标；`prompt_tokens` / `generation_tokens` 等 iteration 级 counter 仍会累计。
39	-
40	-## 配置说明
41	-
42	-配置示例（未配置项使用下列默认值）：
43	-
44	-```json
45	-"health_check_config": {
46	- "enable_virtual_inference": false,
47	- "npu_usage_threshold": 3,
48	- "max_failure_count": 6,
49	- "health_collector_timeout": 5,
50	- "health_collector_timeout_retry_attempts": 3
51	-}
52	-```
53	-
54	-\| 配置项 \| 类型 \| 默认值 \| 说明 \|
55	-\|--------\|------\|--------\|------\|
56	-\| enable_virtual_inference \| bool \| `false` \| 虚推总开关。仅支持 vLLM；SGLang 配置为 `true` 时运行时会自动关闭。仅允许在 ERROR 日志级别下开启（`ASCEND_GLOBAL_LOG_LEVEL=3`，未配置默认 ERROR）；显式配置为非 ERROR 时 deploy 会强制关闭 \|
57	-\| npu_usage_threshold \| int \| `3` \| AI Cube 利用率阈值（%） \|
58	-\| max_failure_count \| int \| `6` \| 连续虚推失败次数上限 \|
59	-\| health_collector_timeout \| int \| `5` \| 推理面 `/health` 探测超时（秒） \|
60	-\| health_collector_timeout_retry_attempts \| int \| `3` \| 推理面 `/health` 超时重试次数（含首次，仅超时触发） \|
61	-
62	-完整字段说明见 [配置参考 health_check_config](../configuration/config_reference.md#health_check_config)。
63	-
64	-## 启用方式
65	-
66	-在 PD 分离部署的 `user_config.json` 中，将 Prefill 与 Decode 引擎配置的 `health_check_config.enable_virtual_inference` 设为 `true`，并按业务调整 `npu_usage_threshold`、`max_failure_count`。虚推要求 ERROR 日志级别：`ASCEND_GLOBAL_LOG_LEVEL` 未配置时默认即为 ERROR；若在 `env.json` 中显式配置为非 `3`，deploy 会强制关闭虚推。

Mdocs/zh/user_guide/features/supported_inference_engines.md+6-0

	@@ -52,3 +52,9 @@ SGLang 在多轮对话、Agent 搜索、Few-shot 等依赖前缀复用的场景
52	}	52	}
53	}	53	}
54	```	54	```
		55	+
		56	+SGLang PD 分离时，bootstrap 端口按 Pod/NodeManager 维度配置在
		57	+`engine_config.disaggregation_bootstrap_port`（也兼容原生 CLI 风格的
		58	+`disaggregation-bootstrap-port`）。NodeManager 将该端口作为 `bootstrap_port` 注册元数据，
		59	+并由 Coordinator 的 SGLang Adapter 用于 Prefill/Decode 对接；它与推理业务端口
		60	+`endpoint_config.service_ports` 是不同端口。未配置该字段时不生成 bootstrap 元数据。

Mdocs/zh/user_guide/menu_user_manual.md+1-2

	@@ -1,7 +1,7 @@
1	# MindIE Motor用户指南	1	# MindIE Motor用户指南
2		2
3	- [简介](../architecture.md)	3	- [简介](../architecture.md)
4	-- [版本说明](../release_note.md)	4	+- [版本说明](../release_note_motor.md)
5	- [快速入门](./quick_start_motor.md)	5	- [快速入门](./quick_start_motor.md)
6	- [安装指南](./maintenance/build_motor_image_from_vllm_ascend.md)	6	- [安装指南](./maintenance/build_motor_image_from_vllm_ascend.md)
7	- [基础环境准备](./environment_preparation.md)	7	- [基础环境准备](./environment_preparation.md)
	@@ -42,5 +42,4 @@
42	- [Controller](../developer_guide/components/controller.md)	42	- [Controller](../developer_guide/components/controller.md)
43	- [Coordinator](../developer_guide/components/coordinator.md)	43	- [Coordinator](../developer_guide/components/coordinator.md)
44	- [Coordinator支持的调度方式]()	44	- [Coordinator支持的调度方式]()
45	- - [Engine Server](../developer_guide/components/engine_server.md)
46	- [Node Manager](../developer_guide/components/node_manager.md)	45	- [Node Manager](../developer_guide/components/node_manager.md)

Mexamples/deployer/README.md+10-31

	@@ -10,11 +10,11 @@
10		10
11	建议在正式部署前先阅读上述文档，按文档完成环境准备与配置后再使用本目录中的工具进行部署。	11	建议在正式部署前先阅读上述文档，按文档完成环境准备与配置后再使用本目录中的工具进行部署。
12		12
13	-## deploy.py 使用方法	13	+## `deploy.py` 使用方法
14		14
15	### 参数说明	15	### 参数说明
16		16
17	-Motor服务部署参数说明	17	+MindIE Motor服务部署参数说明如下所示：
18		18
19	\| 参数 \| 简写 \| 说明 \|	19	\| 参数 \| 简写 \| 说明 \|
20	\|------\|------\|------\|	20	\|------\|------\|------\|
	@@ -206,34 +206,13 @@ Node selector 字段均为 JSON 对象。自定义标签会与 deployer 根据 `
206		206
207	\| 参数 \| 自动管理方式 \|	207	\| 参数 \| 自动管理方式 \|
208	\|------\|-------------\|	208	\|------\|-------------\|
209	-\| `data-parallel-address` \| Controller 根据组装结果确定 master DP 节点 IP，通过 `StartCmdMsg.master_dp_ip` → `--master-dp-ip` 传入 EngineServer \|	209	+\| `data-parallel-address` \| Controller 根据组装结果确定 master DP 节点 IP，通过 `StartCmdMsg.master_dp_ip` 传给 Node Manager；vLLM Adapter 生成原生参数 \|
210	-\| `data-parallel-rank` \| 由 Endpoint ID 决定，NodeManager Daemon 以 `--dp-rank` 传入 EngineServer \|	210	+\| `data-parallel-rank` \| 由 Endpoint ID 决定，Node Manager 的 vLLM Adapter 直接生成原生参数 \|
211	-\| `node-rank` \| Controller 按 NodeManager 注册先后顺序分配（先注册 = 主节点 rank 0），通过 `StartCmdMsg.node_rank` → `--node-rank` 传入 EngineServer \|	211	+\| `node-rank` \| Controller 按 Node Manager 注册先后顺序分配（先注册 = 主节点 rank 0），通过 `StartCmdMsg.node_rank` 传给 vLLM Adapter \|
212	-\| `master-addr` \| EngineServer 在检测到跨节点 PCP/PP 模式（`nnodes > 1` 且 `master-port` 存在）时，自动将 `master-dp-ip` 作为 `--master-addr` 注入 vLLM \|	212	+\| `master-addr` \| vLLM Adapter 检测到跨节点 PCP/PP 模式（`nnodes > 1` 且 `master-port` 存在）时，自动将 `master-dp-ip` 作为 `--master-addr` 注入原生 vLLM 命令 \|
213	-\| `headless` \| EngineServer 在跨节点 PCP/PP 模式下，对 `node-rank != 0` 的从节点自动追加 `--headless` \|	213	+\| `headless` \| vLLM Adapter 在跨节点 PCP/PP 模式下，对 `node-rank != 0` 的从节点自动追加 `--headless` \|
214		214
215	-> 注意：跨节点 PCP / Prefill 跨机 PP 场景下，用户仅需在 `engine_config` 中配置 `nnodes` 和 `master-port`，不要手写 `master-addr` / `node-rank`（由运行时注入）。当前不支持同一实例内 `data_parallel_size > 1` 且 `nnodes > 1`。	215	+>[!NOTE]说明
		216	+>跨节点 PCP/PP 场景下，用户仅需在 `engine_config` 中配置 `nnodes` 和 `master-port`，其余参数由 Motor 自动处理。当前不支持同一实例内 `data_parallel_size > 1` 且 `nnodes > 1`。
216		217
217	-### 跨机 PP 最小 `engine_config` 示例（Prefill）	218	+CLI 参数与 `engine_config` 键名的完整映射关系详见 [vLLM 原生配置适配器](../../motor/node_manager/core/services/native_engine/backends/vllm/config.py) 中的 `VLLMConfig`。
218	-
219	-以 Prefill `TP=16, PP=2, nnodes=2`（每节点 16 卡）为例，手写或经 `vllm_to_motor` 生成后，相关字段如下：
220	-
221	-```json
222	-{
223	- "data_parallel_size": 1,
224	- "tensor_parallel_size": 16,
225	- "pipeline_parallel_size": 2,
226	- "nnodes": 2,
227	- "master-port": 7060
228	-}
229	-```
230	-
231	-说明：
232	-
233	-- `nnodes` / `master-port`：用户（或 Deployer）写入；rendezvous 用。
234	-- `master-addr` / `node-rank` / `headless`：Motor 按上表自动注入，配置里不要写死。
235	-- 使用 `general_config` 从 vLLM 脚本转换时，若脚本中 `tp*pp` 跨多机，Deployer 会自动补上 `nnodes` 与默认 `master-port`（7060）。
236	-
237	-CLI 参数与 `engine_config` 键名的完整映射关系详见：
238	-
239	-👉 [CLI 参数与 engine_config 映射指南](CLI 参数完整定义与校验见 `motor/config/endpoint.py` 中 `EndpointConfig.parse_cli_args`)

Mexamples/deployer/lib/tui/step.py+1-1

	@@ -51,7 +51,7 @@ SEPARATOR_WIDTH = 80
51		51
52	KEY_STEPS = {	52	KEY_STEPS = {
53	'NodeManagerAPI server is ready': 10,	53	'NodeManagerAPI server is ready': 10,
54	- 'engine_server --dp-rank': 20,	54	+ 'vllm serve': 20,
55	'Loading safetensors': 30,	55	'Loading safetensors': 30,
56	'Loading model weights': 80,	56	'Loading model weights': 80,
57	'Graph capturing finished': 90,	57	'Graph capturing finished': 90,

Mexamples/features/agentic/context_parallelism.md+3-3

	@@ -64,9 +64,9 @@
64		64
65	支持 `data_parallel_size > 1` 与跨节点 PCP 组合使用。例如 DP=4、PCP=2、每节点 16 卡时，总共需要 4 × 2 = 8 个节点。Controller 会等待全部 8 个节点到齐后统一组装并下发启动命令。	65	支持 `data_parallel_size > 1` 与跨节点 PCP 组合使用。例如 DP=4、PCP=2、每节点 16 卡时，总共需要 4 × 2 = 8 个节点。Controller 会等待全部 8 个节点到齐后统一组装并下发启动命令。
66		66
67	-### 从节点 SimInference 处理	67	+### 从节点健康状态
68		68
69	-跨节点 PCP 的从节点不启动 API 服务器（headless 模式），MgmtEndpoint 的 `/status` 健康检查通过 AI Cube 利用率监控实现，虚拟推理请求自动禁用。	69	+跨节点 PCP 的从节点不启动 API 服务器（headless 模式），NodeManager 根据原生引擎进程状态维护从节点生命周期；只有主节点暴露业务端口并参与健康探测和请求调度。
70		70
71	### 调度模式（Coordinator 侧）	71	### 调度模式（Coordinator 侧）
72		72
	@@ -94,4 +94,4 @@
94		94
95	CLI 参数与 `engine_config` 键名的完整映射关系详见：	95	CLI 参数与 `engine_config` 键名的完整映射关系详见：
96		96
97	-👉 [CLI 参数与 engine_config 映射指南](CLI 参数完整定义与校验见 `motor/config/endpoint.py` 中 `EndpointConfig.parse_cli_args`)	97	+CLI 参数与 `engine_config` 的映射由 `motor/node_manager/core/services/native_engine/backends/vllm/config.py` 中的 `VLLMConfig` 维护。

Mmotor/common/constants.py+1-1

	@@ -10,7 +10,7 @@
10		10
11	"""	11	"""
12	Shared API format constants (OpenAI/vLLM compatible).	12	Shared API format constants (OpenAI/vLLM compatible).
13	-Used by coordinator and engine_server.	13	+Used by the coordinator and native-engine routing path.
14	"""	14	"""
15		15
16	__all__ = [	16	__all__ = [

Mmotor/common/engine_constants.py+0-60

	@@ -8,28 +8,8 @@
8	# MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.	8	# MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
9	# See the Mulan PSL v2 for more details.	9	# See the Mulan PSL v2 for more details.
10		10
11	-from motor.common.constants import (
12	- CHAT_COMPLETION_PREFIX as CHAT_COMPLETION_PREFIX,
13	- COMPLETION_PREFIX as COMPLETION_PREFIX,
14	-)
15	-
16	-# log dir permission
17	-MOTOR_CUSTOM_ZMQ_PRIVILEGE = 0o640
18	-MOTOR_CUSTOM_ZMQ_DIR_PRIVILEGE = 0o750
19	MUSK_PRIVILEGE = 0o777	11	MUSK_PRIVILEGE = 0o777
20		12
21	-# logger default config
22	-LOG_MAX_LINE_LENGTH = 1023
23	-LOG_SIMPLE_FORMAT = '[%(levelname)s] %(asctime)s.%(msecs)06d %(process)d %(filename)s:%(lineno)d %(message)s'
24	-LOG_DATE_FORMAT = '%Y/%m/%d %H:%M:%S'
25	-LOG_BACKUP_FORMAT = '%Y-%m-%dT%H-%M-%S.%f'
26	-LOG_BACKUP_PATTERN = '\\d{4}-\\d{2}-\\d{2}T\\d{2}-\\d{2}-\\d{2}\\.\\d{3}'
27	-LOG_DEFAULT_FILE = "./engine_server_log/engine_server.log"
28	-LOG_DEFAULT_FILE_PATH = "./engine_server_log/"
29	-LOG_DEFAULT_FILE_NAME = "engine_server.log"
30	-LOG_DEFAULT_BACKUP_COUNT = 30
31	-LOG_DEFAULT_MAX_BYTES = 1024 * 1024 * 20
32	-
33	# valid boundary value	13	# valid boundary value
34	MIN_RANK_SIZE = 0	14	MIN_RANK_SIZE = 0
35	MAX_RANK_SIZE = 4095	15	MAX_RANK_SIZE = 4095
	@@ -72,44 +52,4 @@ PP_SIZE = "pp_size"
72	# engine config	52	# engine config
73	ENGINE_ID = "engine_id"	53	ENGINE_ID = "engine_id"
74		54
75	-# server status
76	-INIT_STATUS = "initial"
77	-ABNORMAL_STATUS = "abnormal"
78	-NORMAL_STATUS = "normal"
79	-
80	-# response keys
81	-STATUS_KEY = "status"
82	-
83	-# content type
84	-TEXT_PLAIN = "text/plain"
85	-APPLICATION_JSON = "application/json"
86	-TEXT_EVENT_STREAM = "text/event-stream"
87	-
88	-# http headers
89	-CONTENT_TYPE = "content-type"
90	-CONTENT_LENGTH = "content-length"
91	-TRANSFER_ENCODING = "transfer-encoding"
92	-CHUNKED_ENCODING = "chunked"
93	-
94	-# json field names
95	-JSON_ID_FIELD = "id"
96	-
97	-# vllm related constants (shared with coordinator via motor.common.constants)
98	-
99	-# vllm api paths
100	-COMPLETIONS_PATH = "/v1/completions"
101	-CHAT_COMPLETIONS_PATH = "/v1/chat/completions"
102	-
103	-# vllm stream data format
104	-DATA_PREFIX = "data: "
105	-DATA_DONE = "data: [DONE]"
106	-
107	-# mgmt interface name
108	-METRICS_INTERFACE = "/metrics"
109	-STATUS_INTERFACE = "/status"
110	-
111	-# service type
112	-METRICS_SERVICE = "metrics_service"
113	-HEALTH_SERVICE = "health_service"
114	-
115	DISAGGREGATION_MODE = "disaggregation-mode"	55	DISAGGREGATION_MODE = "disaggregation-mode"

Mmotor/common/logger/logger.py+2-2

	@@ -98,8 +98,8 @@ env_log_dir = os.getenv('MOTOR_LOG_PATH')
98		98
99	_MODULE_LOGGER_NAME = "common.logger"	99	_MODULE_LOGGER_NAME = "common.logger"
100		100
101	-# Top-level packages that use only the first level (e.g. "engine_server", "node_manager", "config").	101	+# Top-level packages that use only the first level (e.g. "node_manager", "config").
102	-_TOPLEVEL_COMPONENTS = frozenset({"engine_server", "node_manager", "config"})	102	+_TOPLEVEL_COMPONENTS = frozenset({"node_manager", "config"})
103	# Top-level packages that use only the second level (e.g. "fault_tolerance", "domain", "http").	103	# Top-level packages that use only the second level (e.g. "fault_tolerance", "domain", "http").
104	_SECONDLEVEL_COMPONENTS = frozenset({"controller", "coordinator", "common"})	104	_SECONDLEVEL_COMPONENTS = frozenset({"controller", "coordinator", "common"})
105		105

Mmotor/common/resources/dispatch.py+1-252

	@@ -9,19 +9,7 @@
9	# See the Mulan PSL v2 for more details.	9	# See the Mulan PSL v2 for more details.
10		10
11	from enum import Enum	11	from enum import Enum
12	-from typing import Any, Literal	12	+from typing import Any
13	-
14	-from pydantic import BaseModel, Field, field_validator
15	-
16	-
17	-MOTOR_DISPATCH_KEY = "_motor_dispatch"
18	-MOTOR_PREFILL_RESULT_KEY = "_motor_prefill_result"
19	-MOTOR_DISPATCH_SCHEMA_VERSION = "1.0"
20	-
21	-
22	-DispatchRole = Literal["prefill", "decode", "single"]
23	-PrefillStatus = Literal["prepared", "completed", "skipped"]
24	-PrefillMode = Literal["trigger", "handoff", "bootstrap"]
25		13
26		14
27	class DispatchPlan(str, Enum):	15	class DispatchPlan(str, Enum):
	@@ -70,28 +58,6 @@ def classify_vllm_dispatch_profile(
70	return _classify_vllm_kv_transfer_config(kv_transfer_config)	58	return _classify_vllm_kv_transfer_config(kv_transfer_config)
71		59
72		60
73	-def infer_vllm_dispatch_profile_from_config(config: Any) -> DispatchProfile:
74	- """Resolve vLLM dispatch profile from an engine-server IConfig-like object."""
75	- get_endpoint_config = getattr(config, "get_endpoint_config", None)
76	- if get_endpoint_config is None:
77	- return DispatchProfile.UNKNOWN
78	-
79	- endpoint_config = get_endpoint_config()
80	- if endpoint_config is None:
81	- return DispatchProfile.UNKNOWN
82	-
83	- if _normalized(getattr(endpoint_config, "engine_type", None)) != "vllm":
84	- return DispatchProfile.UNKNOWN
85	-
86	- deploy_config = getattr(endpoint_config, "deploy_config", None)
87	- if deploy_config is None:
88	- return DispatchProfile.UNKNOWN
89	-
90	- engine_config = getattr(deploy_config, "engine_config", None)
91	- explicit_profile = getattr(deploy_config, "dispatch_profile", None)
92	- return classify_vllm_dispatch_profile(engine_config, explicit_profile=explicit_profile)
93	-
94	-
95	def dispatch_capabilities_for_profile(profile: DispatchProfile) -> list[str]:	61	def dispatch_capabilities_for_profile(profile: DispatchProfile) -> list[str]:
96	if profile == DispatchProfile.HANDOFF:	62	if profile == DispatchProfile.HANDOFF:
97	return [DispatchPlan.PREFILL_HANDOFF_DECODE.value]	63	return [DispatchPlan.PREFILL_HANDOFF_DECODE.value]
	@@ -162,220 +128,3 @@ def _config_get(config: Any, key: str, default: Any = None) -> Any:
162		128
163	def _normalized(value: Any) -> str:	129	def _normalized(value: Any) -> str:
164	return str(value or "").strip().lower()	130	return str(value or "").strip().lower()
165	-
166	-
167	-class DispatchEndpoint(BaseModel):
168	- """Network location of a scheduled engine endpoint for cross-engine dispatch."""
169	-
170	- instance_id: int = Field(..., ge=0, description="Scheduler instance identifier for the target engine")
171	- endpoint_id: int = Field(..., ge=0, description="Endpoint identifier within the instance")
172	- url: str = Field(..., min_length=1, description="Base HTTP URL for dispatch and stop calls to the engine")
173	- bootstrap_port: int \| None = Field(
174	- default=None,
175	- ge=1,
176	- le=65535,
177	- description="Engine-native PD bootstrap port when it differs from the inference URL port",
178	- )
179	-
180	-
181	-class DispatchEndpoints(BaseModel):
182	- """Paired prefill and decode endpoints for a single P/D dispatch attempt."""
183	-
184	- prefill: DispatchEndpoint \| None = Field(
185	- default=None,
186	- description="Prefill engine endpoint; omitted for decode-only or single-node roles",
187	- )
188	- decode: DispatchEndpoint \| None = Field(
189	- default=None,
190	- description="Decode engine endpoint; omitted for prefill-only or single-node roles",
191	- )
192	-
193	-
194	-class PrefillContextBudget(BaseModel):
195	- """Output budget used by the prefill post-tokenization context check."""
196	-
197	- max_output_tokens: int = Field(
198	- ...,
199	- ge=0,
200	- description="Remaining output-token budget for the current dispatch attempt",
201	- )
202	- parameter: Literal["max_tokens", "max_completion_tokens"] = Field(
203	- ...,
204	- description="Client request field that supplied the output-token budget",
205	- )
206	-
207	- def after_output_tokens(self, consumed_output_tokens: int) -> "PrefillContextBudget":
208	- """Return the budget remaining after replaying visible output tokens."""
209	- if consumed_output_tokens < 0:
210	- raise ValueError("consumed_output_tokens must be non-negative")
211	- if consumed_output_tokens == 0:
212	- return self
213	- # Resumed decode requests retain at least one output token so they can
214	- # recover a missing terminal chunk after the advertised budget was used.
215	- return self.model_copy(update={"max_output_tokens": max(1, self.max_output_tokens - consumed_output_tokens)})
216	-
217	-
218	-class MotorDispatch(BaseModel):
219	- """Motor metadata embedded in inference request bodies under ``_motor_dispatch``."""
220	-
221	- schema_version: str = Field(
222	- default=MOTOR_DISPATCH_SCHEMA_VERSION,
223	- description="Dispatch envelope schema version; major version must match coordinator support",
224	- )
225	- root_request_id: str = Field(..., min_length=1, description="Client-visible request id assigned by the coordinator")
226	- engine_request_id: str = Field(
227	- ...,
228	- min_length=1,
229	- description="Per-attempt engine request id, typically ``{root_request_id}#a{attempt_seq}``",
230	- )
231	- pair_id: str = Field(..., min_length=1, description="Stable id linking prefill and decode peers for one attempt")
232	- attempt_seq: int = Field(..., ge=1, description="Monotonic attempt index within a root request, starting at 1")
233	- role: DispatchRole = Field(..., description="Engine role handling this request: prefill, decode, or single")
234	- dispatch_mode: str = Field(
235	- ...,
236	- min_length=1,
237	- description="Coordinator dispatch plan name, e.g. concurrent_engine_sync or prefill_handoff_decode",
238	- )
239	- prefill_context_budget: PrefillContextBudget \| None = Field(
240	- default=None,
241	- description=(
242	- "Remaining client output-token budget and its source field; used for "
243	- "post-tokenization handoff-prefill validation"
244	- ),
245	- )
246	- endpoints: DispatchEndpoints = Field(
247	- ...,
248	- description="Peer endpoint addresses used for stop signals and handoff coordination",
249	- )
250	-
251	- @field_validator("schema_version")
252	- @classmethod
253	- def validate_schema_version(cls, value: str) -> str:
254	- major = value.split(".", 1)[0]
255	- supported_major = MOTOR_DISPATCH_SCHEMA_VERSION.split(".", 1)[0]
256	- if major != supported_major:
257	- raise ValueError(f"Unsupported motor dispatch schema major version: {value}")
258	- return value
259	-
260	-
261	-class PrefillResultStatus(str, Enum):
262	- """Lifecycle status of a prefill handoff result."""
263	-
264	- PREPARED = "prepared"
265	- COMPLETED = "completed"
266	- SKIPPED = "skipped"
267	-
268	-
269	-class PrefillHandoffMode(str, Enum):
270	- """KV handoff mechanism used between prefill and decode engines."""
271	-
272	- TRIGGER = "trigger"
273	- HANDOFF = "handoff"
274	- BOOTSTRAP = "bootstrap"
275	-
276	-
277	-class PrefillResult(BaseModel):
278	- """Prefill output envelope embedded under ``_motor_prefill_result`` for handoff decode."""
279	-
280	- object: str = Field(default="motor.prefill_result", description="Object type discriminator for prefill results")
281	- schema_version: str = Field(
282	- default=MOTOR_DISPATCH_SCHEMA_VERSION,
283	- description="Prefill result schema version; major version must match coordinator support",
284	- )
285	- root_request_id: str = Field(..., min_length=1, description="Client-visible request id assigned by the coordinator")
286	- engine_request_id: str = Field(
287	- ...,
288	- min_length=1,
289	- description="Per-attempt engine request id correlated with the paired MotorDispatch",
290	- )
291	- pair_id: str = Field(..., min_length=1, description="Stable id linking prefill and decode peers for one attempt")
292	- attempt_seq: int = Field(..., ge=1, description="Monotonic attempt index within a root request, starting at 1")
293	- status: PrefillStatus = Field(..., description="Whether prefill was prepared, completed, or skipped")
294	- handoff_mode: PrefillMode = Field(..., description="Trigger, handoff, or bootstrap coordination mode")
295	- payload: dict = Field(default_factory=dict, description="Engine-specific prefill handoff data, e.g. KV handles")
296	- usage: dict \| None = Field(
297	- default=None,
298	- description=(
299	- "Prefill usage block (carries prompt_tokens_details for cached-token reporting); "
300	- "kept separate from payload because payload is consumed verbatim as kv_transfer_params"
301	- ),
302	- )
303	- expires_at_ms: int \| None = Field(
304	- default=None,
305	- ge=0,
306	- description="Optional Unix timestamp in milliseconds after which the cached result is stale",
307	- )
308	-
309	- @field_validator("schema_version")
310	- @classmethod
311	- def validate_schema_version(cls, value: str) -> str:
312	- major = value.split(".", 1)[0]
313	- supported_major = MOTOR_DISPATCH_SCHEMA_VERSION.split(".", 1)[0]
314	- if major != supported_major:
315	- raise ValueError(f"Unsupported motor prefill result schema major version: {value}")
316	- return value
317	-
318	- def matches_dispatch(self, dispatch: MotorDispatch) -> bool:
319	- return (
320	- self.root_request_id == dispatch.root_request_id
321	- and self.pair_id == dispatch.pair_id
322	- and self.attempt_seq == dispatch.attempt_seq
323	- )
324	-
325	-
326	-class DispatchStopReason(str, Enum):
327	- """Reason the coordinator asked a peer engine to stop an in-flight dispatch attempt."""
328	-
329	- PEER_FAILED = "peer_failed"
330	- CLIENT_DISCONNECT = "client_disconnect"
331	- TIMEOUT = "timeout"
332	- RECOMPUTE = "recompute"
333	- RETRY_REPAIR = "retry_repair"
334	- OTHER = "other"
335	-
336	-
337	-class DispatchStopState(str, Enum):
338	- """Outcome of a ``/v1/dispatch/stop`` request."""
339	-
340	- STOPPED = "stopped"
341	- ALREADY_STOPPED = "already_stopped"
342	- ALREADY_DONE = "already_done"
343	- NOT_FOUND = "not_found"
344	- STALE = "stale"
345	-
346	-
347	-class DispatchStopRequest(BaseModel):
348	- """Request body for coordinator-initiated peer engine stop."""
349	-
350	- root_request_id: str = Field(..., min_length=1, description="Client-visible request id assigned by the coordinator")
351	- engine_request_id: str \| None = Field(
352	- default=None,
353	- description="Optional per-attempt engine request id for finer-grained stop matching",
354	- )
355	- attempt_seq: int = Field(..., ge=1, description="Attempt index within the root request to stop")
356	- pair_id: str = Field(..., min_length=1, description="Pair id linking the prefill and decode peers for the attempt")
357	- reason: str = Field(
358	- default=DispatchStopReason.OTHER.value,
359	- description="Why the stop was requested; normalized via normalized_reason()",
360	- )
361	- sent_at_ms: int \| None = Field(
362	- default=None,
363	- ge=0,
364	- description="Optional Unix timestamp in milliseconds when the stop request was sent",
365	- )
366	-
367	- def normalized_reason(self) -> DispatchStopReason:
368	- try:
369	- return DispatchStopReason(self.reason)
370	- except ValueError:
371	- return DispatchStopReason.OTHER
372	-
373	-
374	-class DispatchStopResponse(BaseModel):
375	- """Response body confirming whether a dispatch stop was accepted."""
376	-
377	- root_request_id: str = Field(..., description="Client-visible request id echoed from the stop request")
378	- attempt_seq: int = Field(..., description="Attempt index echoed from the stop request")
379	- accepted: bool = Field(..., description="Whether the engine accepted and processed the stop request")
380	- state: DispatchStopState = Field(..., description="Current attempt state after processing the stop request")
381	- message: str = Field(default="", description="Optional human-readable detail about the stop outcome")

Mmotor/common/resources/instance.py+3-3

	@@ -427,9 +427,9 @@ class Instance(BaseModel):
427	return True	427	return True
428	else:	428	else:
429	# Pod sending heartbeat for an IP not registered in this instance's	429	# Pod sending heartbeat for an IP not registered in this instance's
430	- # endpoints (e.g. stale engine-server after scale-down / pod eviction).	430	+ # endpoints (e.g. a stale NodeManager after scale-down / Pod eviction).
431	- # The caller raises HTTPException so the engine-server knows to	431	+ # The caller raises HTTPException so the NodeManager knows to
432	- # re-register. Throttle to avoid flooding the log every heartbeat cycle.	432	+ # re-register. Throttle to avoid flooding the log every heartbeat cycle.
433	_rl.error_window(	433	_rl.error_window(
434	f"hb_unknown_ip:{self.id}:{ip}",	434	f"hb_unknown_ip:{self.id}:{ip}",
435	"Instance %s not found endpoints for pod_ip %s" % (self.id, ip),	435	"Instance %s not found endpoints for pod_ip %s" % (self.id, ip),

Mmotor/common/utils/port_allocator.py+5-5

Mmotor/config/config_utils.py+1-1

	@@ -140,7 +140,7 @@ def _update_tls_config(
140	updated_config[tls_key] = tls_config[tls_key]	140	updated_config[tls_key] = tls_config[tls_key]
141		141
142		142
143	-def _update_engine_server_tls_config(	143	+def _update_native_engine_tls_config(
144	updated_config: dict[str, Any],	144	updated_config: dict[str, Any],
145	user_config_data: dict[str, Any],	145	user_config_data: dict[str, Any],
146	) -> None:	146	) -> None:

Mmotor/config/endpoint.py+2-61

	@@ -8,7 +8,6 @@
8	# MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.	8	# MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
9	# See the Mulan PSL v2 for more details.	9	# See the Mulan PSL v2 for more details.
10		10
11	-import argparse
12	import json	11	import json
13	import os	12	import os
14	from dataclasses import dataclass, field	13	from dataclasses import dataclass, field
	@@ -17,7 +16,7 @@ from typing import Any
17		16
18	from motor.common.logger import get_logger	17	from motor.common.logger import get_logger
19	from motor.common.resources.dispatch import DISPATCH_PROFILE_KEY	18	from motor.common.resources.dispatch import DISPATCH_PROFILE_KEY
20	-from motor.config.config_utils import _update_engine_server_tls_config	19	+from motor.config.config_utils import _update_native_engine_tls_config
21	from motor.config.resolver import ConfigResolver, normalize_keys	20	from motor.config.resolver import ConfigResolver, normalize_keys
22	from motor.config.tls_config import TLSConfig	21	from motor.config.tls_config import TLSConfig
23	from motor.common import engine_constants as constants	22	from motor.common import engine_constants as constants
	@@ -176,7 +175,7 @@ class DeployConfig:
176	"union": MOTOR_ENGINE_UNION_CONFIG_KEY,	175	"union": MOTOR_ENGINE_UNION_CONFIG_KEY,
177	}	176	}
178	data = raw_data.get(key_map.get(role, ""), {})	177	data = raw_data.get(key_map.get(role, ""), {})
179	- _update_engine_server_tls_config(data, raw_data)	178	+ _update_native_engine_tls_config(data, raw_data)
180		179
181	resolver = ConfigResolver(data)	180	resolver = ConfigResolver(data)
182		181
	@@ -274,59 +273,6 @@ class EndpointConfig:
274	d2d_peer_ips: str \| None = None	273	d2d_peer_ips: str \| None = None
275	deploy_config: DeployConfig = None	274	deploy_config: DeployConfig = None
276		275
277	- snapshot_metadata: str \| None = None
278	-
279	- @classmethod
280	- def parse_cli_args(cls) -> argparse.Namespace:
281	- parser = argparse.ArgumentParser(description="EngineServer - Universal Inference Engine Service")
282	- parser.add_argument("--host", help="EngineServer endpoint host")
283	- parser.add_argument("--role", help="PD separate role, prefill/decode/union")
284	- parser.add_argument("--kv-port", type=int, help="kv port")
285	- parser.add_argument("--lookup-rpc-port", type=int, help="lookup rpc port")
286	- parser.add_argument("--master-dp-ip", type=str, help="Master DP ip for distributed setup")
287	- parser.add_argument("--dp-rpc-port", type=int, help="dp rpc port")
288	- parser.add_argument("--port", type=int, help="EngineServer business interface port")
289	- parser.add_argument("--mgmt-port", type=int, dest="mgmt_port", help="EngineServer management interface port")
290	- parser.add_argument("--instance-id", type=int, default=0, help="Engine instance id")
291	- parser.add_argument("--dp-rank", type=int, default=0, help="DP parallel rank")
292	- parser.add_argument("--node-rank", type=int, default=0, help="PCP node rank (assigned by Motor Controller)")
293	- parser.add_argument("--config-path", help="Path to engine-specific configuration file (JSON format)")
294	- parser.add_argument(
295	- "--d2d-peer-ips",
296	- type=str,
297	- default=None,
298	- help="Comma-separated IPs of peer instances for D2D weight transfer",
299	- )
300	- parser.add_argument(
301	- "--snapshot-metadata",
302	- default=None,
303	- help="Snapshot metadata file (JSON format), enable snapshot function",
304	- )
305	- return parser.parse_args()
306	-
307	- @classmethod
308	- def init_endpoint_config(cls) -> 'EndpointConfig':
309	- cli_args = cls.parse_cli_args()
310	- endpoint_config = cls(
311	- host=cli_args.host,
312	- role=cli_args.role,
313	- kv_port=cli_args.kv_port,
314	- lookup_rpc_port=cli_args.lookup_rpc_port,
315	- master_dp_ip=cli_args.master_dp_ip,
316	- dp_rpc_port=cli_args.dp_rpc_port,
317	- port=cli_args.port,
318	- mgmt_port=cli_args.mgmt_port,
319	- instance_id=cli_args.instance_id,
320	- config_path=cli_args.config_path,
321	- dp_rank=cli_args.dp_rank,
322	- d2d_peer_ips=cli_args.d2d_peer_ips,
323	- node_rank=cli_args.node_rank,
324	- snapshot_metadata=cli_args.snapshot_metadata,
325	- )
326	- endpoint_config.validate()
327	- endpoint_config.load_deploy_config()
328	- return endpoint_config
329	-
330	def validate(self):	276	def validate(self):
331	if self.role not in supported_role:	277	if self.role not in supported_role:
332	raise ValueError(f"role {self.role} is not supported.")	278	raise ValueError(f"role {self.role} is not supported.")
	@@ -341,11 +287,6 @@ class EndpointConfig:
341	raise ValueError(f"config file {self.config_path} does not exist")	287	raise ValueError(f"config file {self.config_path} does not exist")
342	if not FileValidator(self.config_path).check_not_soft_link().check_file_size().check().is_valid():	288	if not FileValidator(self.config_path).check_not_soft_link().check_file_size().check().is_valid():
343	raise ValueError(f"{self.config_path} is not a valid file path.")	289	raise ValueError(f"{self.config_path} is not a valid file path.")
344	- if self.snapshot_metadata is not None:
345	- if not os.path.exists(self.snapshot_metadata):
346	- raise ValueError(f"snapshot metadata file {self.snapshot_metadata} does not exist")
347	- if not FileValidator(self.snapshot_metadata).check_not_soft_link().check_file_size().check().is_valid():
348	- raise ValueError(f"{self.snapshot_metadata} is not a valid file path")
349		290
350	def load_deploy_config(self):	291	def load_deploy_config(self):
351	self.deploy_config = DeployConfig.load(self.config_path, role=self.role)	292	self.deploy_config = DeployConfig.load(self.config_path, role=self.role)

Mmotor/config/node_manager.py+4-4

	@@ -149,10 +149,10 @@ class APIConfig:
149		149
150	@dataclass	150	@dataclass
151	class EndpointConfig:	151	class EndpointConfig:
152	- # EngineServer's number	152	+ # Native engine endpoint count
153	endpoint_num: int = 0	153	endpoint_num: int = 0
154		154
155	- # EngineServer's Port configuration	155	+ # Native engine endpoint port configuration
156	base_port: int = 10000	156	base_port: int = 10000
157	mgmt_ports: list[str] = field(default_factory=list)	157	mgmt_ports: list[str] = field(default_factory=list)
158	service_ports: list[str] = field(default_factory=list)	158	service_ports: list[str] = field(default_factory=list)
	@@ -793,8 +793,8 @@ class NodeManagerConfig:
793	if self.basic_config.heartbeat_interval_seconds <= 0:	793	if self.basic_config.heartbeat_interval_seconds <= 0:
794	errors.append("heartbeat_interval_seconds must be greater than 0")	794	errors.append("heartbeat_interval_seconds must be greater than 0")
795		795
796	- if self.snapshot_config.enable_snapshot:	796	+ if self.snapshot_config.enable_snapshot and self.basic_config.engine_type != ENGINE_TYPE_VLLM:
797	- errors.append("Native engine runtime does not support snapshot yet; enable_snapshot must be false")	797	+ errors.append("Native Snapshot currently supports only the vllm engine type")
798		798
799	# Validate logging configuration	799	# Validate logging configuration
800	valid_log_levels = ["DEBUG", "INFO", "WARNING", "ERROR"]	800	valid_log_levels = ["DEBUG", "INFO", "WARNING", "ERROR"]

Dmotor/engine_server/__init__.py+0-11

	@@ -1,11 +0,0 @@
1	-#!/usr/bin/env python3
2	-# -- coding: utf-8 --
3	-# Copyright (c) Huawei Technologies Co., Ltd. 2025-2026. All rights reserved.
4	-# MindIE is licensed under Mulan PSL v2.
5	-# You can use this software according to the terms and conditions of the Mulan PSL v2.
6	-# You may obtain a copy of Mulan PSL v2 at:
7	-# http://license.coscl.org.cn/MulanPSL2
8	-# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
9	-# EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
10	-# MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
11	-# See the Mulan PSL v2 for more details.

Dmotor/engine_server/cli/__init__.py+0-11

	@@ -1,11 +0,0 @@
1	-#!/usr/bin/env python3
2	-# -- coding: utf-8 --
3	-# Copyright (c) Huawei Technologies Co., Ltd. 2025-2026. All rights reserved.
4	-# MindIE is licensed under Mulan PSL v2.
5	-# You can use this software according to the terms and conditions of the Mulan PSL v2.
6	-# You may obtain a copy of Mulan PSL v2 at:
7	-# http://license.coscl.org.cn/MulanPSL2
8	-# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
9	-# EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
10	-# MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
11	-# See the Mulan PSL v2 for more details.

Dmotor/engine_server/cli/main.py+0-168

	@@ -1,168 +0,0 @@
1	-# Copyright (c) Huawei Technologies Co., Ltd. 2025-2026. All rights reserved.
2	-# MindIE is licensed under Mulan PSL v2.
3	-# You can use this software according to the terms and conditions of the Mulan PSL v2.
4	-# You may obtain a copy of Mulan PSL v2 at:
5	-# http://license.coscl.org.cn/MulanPSL2
6	-# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
7	-# EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
8	-# MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
9	-# See the Mulan PSL v2 for more details.
10	-
11	-import shlex
12	-import signal
13	-import subprocess
14	-import sys
15	-
16	-from motor.common.utils.process_utils import set_process_title
17	-
18	-
19	-# Set this to ``True`` to enable native CLI launch mode.
20	-# When enabled, the engine (vLLM / SGLang) is launched
21	-# via its native CLI command (e.g. ``vllm serve ...``) in a subprocess
22	-# instead of the default invasive in-process launch.
23	-NATIVE_LAUNCH_ENABLED = False
24	-
25	-
26	-def _dp_rank_from_argv() -> int:
27	- argv = sys.argv[1:]
28	- for idx, arg in enumerate(argv):
29	- if arg == "--dp-rank" and idx + 1 < len(argv):
30	- try:
31	- return int(argv[idx + 1])
32	- except ValueError:
33	- return 0
34	- if arg.startswith("--dp-rank="):
35	- try:
36	- return int(arg.split("=", 1)[1])
37	- except ValueError:
38	- return 0
39	- return 0
40	-
41	-
42	-set_process_title(f"EngineServer-DP{_dp_rank_from_argv()}")
43	-
44	-# ruff: noqa: E402
45	-from motor.common.logger import get_logger
46	-from motor.config.endpoint import EndpointConfig
47	-from motor.node_manager.core.services.native_engine.config_factory import ConfigFactory
48	-from motor.engine_server.factory.endpoint_factory import EndpointFactory
49	-from motor.engine_server.utils.prometheus import setup_multiprocess_prometheus
50	-
51	-logger = get_logger(__name__)
52	-
53	-
54	-def _log_safe_cmd(cmd: list[str]) -> str:
55	- """Format a command list for logging, escaping arguments that contain spaces."""
56	- return " ".join(shlex.quote(a) if " " in a else a for a in cmd)
57	-
58	-
59	-def _build_native_launch_cmd(config) -> list[str]:
60	- """Build the native CLI command for the engine from the parsed config.
61	-
62	- Returns a list suitable for ``subprocess.Popen``, e.g.
63	- ``["vllm", "serve", "--model", "...", "--host", "0.0.0.0", ...]``.
64	-
65	- Security:
66	- * ``engine_type`` is validated against a strict whitelist.
67	- * ``subprocess.Popen`` is called with a list (no ``shell=True``) so
68	- argument values (including JSON) are passed as literals.
69	- """
70	- engine_type = config.get_endpoint_config().engine_type
71	- cli_args = config.get_cli_args()
72	-
73	- if engine_type == "vllm":
74	- return ["vllm", "serve"] + cli_args
75	- elif engine_type == "sglang":
76	- return ["python3", "-m", "sglang.launch_server"] + cli_args
77	- else:
78	- raise ValueError(
79	- f"Unsupported engine type for native launch: {engine_type}. Supported types are: vllm, sglang."
80	- )
81	-
82	-
83	-def _run_native(config) -> None:
84	- """Launch the engine via native CLI command in a subprocess."""
85	- cmd = _build_native_launch_cmd(config)
86	- logger.info("Launching engine via native command: %s", _log_safe_cmd(cmd))
87	-
88	- with subprocess.Popen(cmd) as process:
89	-
90	- def _signal_handler(signum, frame):
91	- logger.info("Received signal %s, forwarding SIGTERM to native engine process", signum)
92	- process.send_signal(signal.SIGTERM)
93	-
94	- old_sigterm = signal.signal(signal.SIGTERM, _signal_handler)
95	- old_sigint = signal.signal(signal.SIGINT, _signal_handler)
96	-
97	- try:
98	- process.wait()
99	- finally:
100	- signal.signal(signal.SIGTERM, old_sigterm)
101	- signal.signal(signal.SIGINT, old_sigint)
102	-
103	- logger.info("Native engine process exited with code %s", process.returncode)
104	-
105	-
106	-def main():
107	- # Execute setup_multiprocess_prometheus before importing Endpoint to ensure
108	- # PROMETHEUS_MULTIPROC_DIR is detected when Prometheus low-level code creates ValueClass.
109	- setup_multiprocess_prometheus()
110	-
111	- from motor.engine_server.core.infer_endpoint import InferEndpoint
112	- from motor.engine_server.core.mgmt_endpoint import MgmtEndpoint
113	-
114	- endpoint_config = EndpointConfig.init_endpoint_config()
115	-
116	- mgmt_endpoint: MgmtEndpoint = MgmtEndpoint(endpoint_config)
117	- mgmt_endpoint.run()
118	-
119	- config_factory = ConfigFactory(endpoint_config=endpoint_config)
120	- config = config_factory.parse()
121	- logger.info("successfully parsed %s engine configuration", endpoint_config.engine_type)
122	-
123	- mgmt_endpoint.attach_engine(config)
124	-
125	- if NATIVE_LAUNCH_ENABLED:
126	- logger.info(
127	- "Native launch mode enabled (%s), launching engine via CLI subprocess.",
128	- NATIVE_LAUNCH_ENABLED,
129	- )
130	- if endpoint_config.snapshot_metadata is not None:
131	- logger.warning(
132	- "Snapshot metadata is provided, but native launch mode is enabled. "
133	- "Snapshot sentinel will not be started, and snapshot saving may not work as expected."
134	- )
135	- try:
136	- _run_native(config)
137	- finally:
138	- mgmt_endpoint.shutdown()
139	- return
140	-
141	- snapshot_sentinel = None
142	- if endpoint_config.snapshot_metadata is not None:
143	- from motor.engine_server.core.snapshot_sentinel import SnapshotSentinel
144	-
145	- snapshot_sentinel = SnapshotSentinel(endpoint_config)
146	- snapshot_sentinel.start()
147	- logger.info(
148	- "[snapshot] Snapshot metadata given, launching snapshot sentinel thread "
149	- "to save the device-side snapshot once the inference service is ready."
150	- )
151	-
152	- infer_endpoint: InferEndpoint = EndpointFactory().get_infer_endpoint(config)
153	- infer_endpoint.run()
154	- infer_endpoint.wait()
155	-
156	- logger.info("shutting down endpoints and child processes...")
157	- mgmt_endpoint.shutdown()
158	- infer_endpoint.shutdown()
159	- if snapshot_sentinel is not None:
160	- snapshot_sentinel.stop()
161	- snapshot_sentinel.join(timeout=5)
162	- if snapshot_sentinel.is_alive():
163	- logger.warning("[snapshot] snapshot sentinel thread did not exit within timeout")
164	- logger.info("endpoints and child processes shut down")
165	-
166	-
167	-if __name__ == "__main__":
168	- main()

Dmotor/engine_server/core/__init__.py+0-0

The file is empty

Dmotor/engine_server/core/dispatch_adapter/__init__.py+0-13

	@@ -1,13 +0,0 @@
1	-# Copyright (c) Huawei Technologies Co., Ltd. 2026. All rights reserved.
2	-# MindIE is licensed under Mulan PSL v2.
3	-# You can use this software according to the terms and conditions of the Mulan PSL v2.
4	-# You may obtain a copy of Mulan PSL v2 at:
5	-# http://license.coscl.org.cn/MulanPSL2
6	-# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
7	-# EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
8	-# MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
9	-# See the Mulan PSL v2 for more details.
10	-
11	-from motor.engine_server.core.dispatch_adapter.factory import create_dispatch_adapter
12	-
13	-__all__ = ["create_dispatch_adapter"]

Dmotor/engine_server/core/dispatch_adapter/base.py+0-534

	@@ -1,534 +0,0 @@
1	-# Copyright (c) Huawei Technologies Co., Ltd. 2026. All rights reserved.
2	-# MindIE is licensed under Mulan PSL v2.
3	-# You can use this software according to the terms and conditions of the Mulan PSL v2.
4	-# You may obtain a copy of Mulan PSL v2 at:
5	-# http://license.coscl.org.cn/MulanPSL2
6	-# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
7	-# EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
8	-# MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
9	-# See the Mulan PSL v2 for more details.
10	-
11	-import asyncio
12	-import json
13	-import time
14	-from dataclasses import dataclass
15	-from typing import Any
16	-from urllib.parse import urlparse
17	-
18	-import httpx
19	-from fastapi import HTTPException, status
20	-from fastapi.responses import JSONResponse, Response
21	-
22	-from motor.common.http.http_client import HTTPClientPool
23	-from motor.common.logger import get_logger
24	-from motor.common.resources.dispatch import (
25	- MOTOR_DISPATCH_KEY,
26	- MOTOR_PREFILL_RESULT_KEY,
27	- DispatchStopReason,
28	- DispatchStopRequest,
29	- DispatchStopResponse,
30	- DispatchStopState,
31	- MotorDispatch,
32	- PrefillResult,
33	-)
34	-from motor.node_manager.core.services.native_engine.backends.base import IConfig
35	-from motor.engine_server.core.vllm.prefill_context_validation import PrefillContextCheck
36	-from motor.engine_server.core.errors.sanitizer import sanitize_error_message
37	-
38	-logger = get_logger(__name__)
39	-
40	-
41	-@dataclass(frozen=True)
42	-class DispatchResponseContext:
43	- api: str
44	- raw_path: str
45	- request_body: dict[str, Any]
46	- dispatch: MotorDispatch \| None
47	- stream: bool
48	- client_return_token_ids: bool = False
49	- client_expects_chat_shape: bool = False
50	-
51	-
52	-@dataclass(frozen=True)
53	-class PrefillBodyCacheEntry:
54	- body: dict[str, Any]
55	- dispatch: MotorDispatch
56	- cached_at: float
57	-
58	-
59	-@dataclass(frozen=True)
60	-class DispatchMetaserverRequest:
61	- engine_body: dict[str, Any]
62	- dispatch: MotorDispatch \| None = None
63	-
64	-
65	-class DispatchPeerStopClient:
66	- def __init__(self, config: IConfig) -> None:
67	- self._tls_config = None
68	- endpoint_config = config.get_endpoint_config()
69	- deploy_config = getattr(endpoint_config, "deploy_config", None)
70	- if deploy_config is not None:
71	- self._tls_config = getattr(deploy_config, "infer_tls_config", None)
72	-
73	- async def stop_peer(
74	- self,
75	- dispatch: MotorDispatch,
76	- reason: DispatchStopReason = DispatchStopReason.PEER_FAILED,
77	- timeout: float = 1.0,
78	- ) -> DispatchStopResponse \| None:
79	- peer = dispatch.endpoints.decode if dispatch.role == "prefill" else dispatch.endpoints.prefill
80	- if peer is None:
81	- return None
82	-
83	- request = DispatchStopRequest(
84	- root_request_id=dispatch.root_request_id,
85	- engine_request_id=dispatch.engine_request_id,
86	- attempt_seq=dispatch.attempt_seq,
87	- pair_id=dispatch.pair_id,
88	- reason=reason.value,
89	- sent_at_ms=int(time.time() * 1000),
90	- )
91	- try:
92	- parsed = urlparse(peer.url)
93	- host = parsed.hostname
94	- port = parsed.port
95	- if not host or port is None:
96	- raise ValueError(f"Invalid dispatch peer url: {peer.url}")
97	- client = await HTTPClientPool().get_client(
98	- ip=host,
99	- port=str(port),
100	- tls_config=self._tls_config,
101	- )
102	- response = await client.post(
103	- "/v1/dispatch/stop",
104	- json=request.model_dump(mode="json"),
105	- timeout=timeout,
106	- )
107	- response.raise_for_status()
108	- return DispatchStopResponse.model_validate(response.json())
109	- except httpx.HTTPError as e:
110	- logger.warning(
111	- "Peer dispatch stop failed root_request_id=%s attempt_seq=%s peer=%s reason=%s error=%s",
112	- dispatch.root_request_id,
113	- dispatch.attempt_seq,
114	- peer.url,
115	- reason.value,
116	- e,
117	- )
118	- except Exception as e:
119	- logger.warning(
120	- "Peer dispatch stop response invalid root_request_id=%s attempt_seq=%s peer=%s error=%s",
121	- dispatch.root_request_id,
122	- dispatch.attempt_seq,
123	- peer.url,
124	- e,
125	- )
126	- return None
127	-
128	-
129	-class DispatchAttemptRegistry:
130	- """In-memory endpoint-side attempt state used by stop and stale guards."""
131	-
132	- def __init__(self, ttl_seconds: float = 300.0) -> None:
133	- self._lock = asyncio.Lock()
134	- self._states: dict[tuple[str, int], str] = {}
135	- self._dispatches: dict[tuple[str, int], MotorDispatch] = {}
136	- self._updated_at: dict[tuple[str, int], float] = {}
137	- self._prefill_entries: dict[str, PrefillBodyCacheEntry] = {}
138	- self._prefill_waiters: dict[str, asyncio.Event] = {}
139	- self._ttl_seconds = ttl_seconds
140	-
141	- async def activate(self, dispatch: MotorDispatch) -> None:
142	- async with self._lock:
143	- now = time.monotonic()
144	- self._cleanup_locked(now)
145	- key = (dispatch.root_request_id, dispatch.attempt_seq)
146	- self._states[key] = "active"
147	- self._dispatches[key] = dispatch
148	- self._updated_at[key] = now
149	-
150	- async def cache_prefill_body(self, dispatch: MotorDispatch, body: dict[str, Any]) -> None:
151	- async with self._lock:
152	- now = time.monotonic()
153	- self._cleanup_locked(now)
154	- self._prefill_entries[dispatch.engine_request_id] = PrefillBodyCacheEntry(
155	- body=body.copy(),
156	- dispatch=dispatch,
157	- cached_at=now,
158	- )
159	- waiter = self._prefill_waiters.pop(dispatch.engine_request_id, None)
160	- if waiter is not None:
161	- waiter.set()
162	-
163	- async def get_prefill_body(self, engine_request_id: str) -> dict[str, Any] \| None:
164	- entry = await self.get_prefill_entry(engine_request_id)
165	- if entry is None:
166	- return None
167	- return entry.body.copy()
168	-
169	- async def get_prefill_entry(self, engine_request_id: str) -> PrefillBodyCacheEntry \| None:
170	- async with self._lock:
171	- now = time.monotonic()
172	- self._cleanup_locked(now)
173	- entry = self._prefill_entries.get(engine_request_id)
174	- if entry is None:
175	- return None
176	- if now - entry.cached_at > self._ttl_seconds:
177	- self._prefill_entries.pop(engine_request_id, None)
178	- return None
179	- return PrefillBodyCacheEntry(
180	- body=entry.body.copy(),
181	- dispatch=entry.dispatch,
182	- cached_at=entry.cached_at,
183	- )
184	-
185	- async def wait_prefill_body(self, engine_request_id: str, timeout_seconds: float) -> dict[str, Any] \| None:
186	- entry = await self.wait_prefill_entry(engine_request_id, timeout_seconds)
187	- if entry is None:
188	- return None
189	- return entry.body.copy()
190	-
191	- async def wait_prefill_entry(self, engine_request_id: str, timeout_seconds: float) -> PrefillBodyCacheEntry \| None:
192	- cached = await self.get_prefill_entry(engine_request_id)
193	- if cached is not None:
194	- return cached
195	-
196	- async with self._lock:
197	- now = time.monotonic()
198	- self._cleanup_locked(now)
199	- entry = self._prefill_entries.get(engine_request_id)
200	- if entry is not None:
201	- if now - entry.cached_at <= self._ttl_seconds:
202	- return PrefillBodyCacheEntry(
203	- body=entry.body.copy(),
204	- dispatch=entry.dispatch,
205	- cached_at=entry.cached_at,
206	- )
207	- self._prefill_entries.pop(engine_request_id, None)
208	- event = self._prefill_waiters.get(engine_request_id)
209	- if event is None:
210	- event = asyncio.Event()
211	- self._prefill_waiters[engine_request_id] = event
212	-
213	- try:
214	- await asyncio.wait_for(event.wait(), timeout=timeout_seconds)
215	- except asyncio.TimeoutError:
216	- async with self._lock:
217	- if self._prefill_waiters.get(engine_request_id) is event:
218	- self._prefill_waiters.pop(engine_request_id, None)
219	- return None
220	- return await self.get_prefill_entry(engine_request_id)
221	-
222	- async def stop(self, stop_request: DispatchStopRequest) -> DispatchStopState:
223	- key = (stop_request.root_request_id, stop_request.attempt_seq)
224	- async with self._lock:
225	- now = time.monotonic()
226	- self._cleanup_locked(now)
227	- current = self._states.get(key)
228	- if current is None:
229	- return DispatchStopState.NOT_FOUND
230	- dispatch = self._dispatches.get(key)
231	- if dispatch is not None and not self._matches_stop_request(dispatch, stop_request):
232	- return DispatchStopState.STALE
233	- if dispatch is not None:
234	- self._prefill_entries.pop(dispatch.engine_request_id, None)
235	- waiter = self._prefill_waiters.pop(dispatch.engine_request_id, None)
236	- if waiter is not None:
237	- waiter.set()
238	- if current == "stopped":
239	- return DispatchStopState.ALREADY_STOPPED
240	- if current == "done":
241	- return DispatchStopState.ALREADY_DONE
242	- self._states[key] = "stopped"
243	- self._updated_at[key] = now
244	- return DispatchStopState.STOPPED
245	-
246	- async def is_stopped(self, dispatch: MotorDispatch) -> bool:
247	- key = (dispatch.root_request_id, dispatch.attempt_seq)
248	- async with self._lock:
249	- now = time.monotonic()
250	- self._cleanup_locked(now)
251	- return self._states.get(key) == "stopped"
252	-
253	- async def finish(self, dispatch: MotorDispatch) -> None:
254	- async with self._lock:
255	- now = time.monotonic()
256	- self._cleanup_locked(now)
257	- key = (dispatch.root_request_id, dispatch.attempt_seq)
258	- self._prefill_entries.pop(dispatch.engine_request_id, None)
259	- if self._states.get(key) != "stopped":
260	- self._states[key] = "done"
261	- self._dispatches[key] = dispatch
262	- self._updated_at[key] = now
263	-
264	- def _cleanup_locked(self, now: float) -> None:
265	- expired_keys = [
266	- key
267	- for key, updated_at in self._updated_at.items()
268	- if self._states.get(key) in ("done", "stopped") and now - updated_at > self._ttl_seconds
269	- ]
270	- for key in expired_keys:
271	- dispatch = self._dispatches.pop(key, None)
272	- self._states.pop(key, None)
273	- self._updated_at.pop(key, None)
274	- if dispatch is not None:
275	- self._prefill_entries.pop(dispatch.engine_request_id, None)
276	- waiter = self._prefill_waiters.pop(dispatch.engine_request_id, None)
277	- if waiter is not None:
278	- waiter.set()
279	-
280	- expired_prefill_keys = [
281	- engine_request_id
282	- for engine_request_id, entry in self._prefill_entries.items()
283	- if now - entry.cached_at > self._ttl_seconds
284	- ]
285	- for engine_request_id in expired_prefill_keys:
286	- self._prefill_entries.pop(engine_request_id, None)
287	-
288	- @staticmethod
289	- def _matches_stop_request(dispatch: MotorDispatch, stop_request: DispatchStopRequest) -> bool:
290	- if dispatch.pair_id != stop_request.pair_id:
291	- return False
292	- return stop_request.engine_request_id is None or stop_request.engine_request_id == dispatch.engine_request_id
293	-
294	-
295	-class DispatchAdapter:
296	- """EngineServer dispatch adapter base.
297	-
298	- The adapter validates and strips the internal dispatch envelope, injects
299	- engine-specific request fields, and normalizes dispatch responses before
300	- they leave the EngineServer.
301	- """
302	-
303	- def __init__(self, config: IConfig) -> None:
304	- self._config = config
305	- endpoint_config = config.get_endpoint_config()
306	- self._local_role = getattr(endpoint_config, "role", "union")
307	- self.engine_type = getattr(endpoint_config, "engine_type", "unknown")
308	- self._registry = DispatchAttemptRegistry()
309	- self._peer_stop_client = DispatchPeerStopClient(config)
310	-
311	- async def adapt_request_body(self, body: dict[str, Any]) -> tuple[dict[str, Any], MotorDispatch \| None]:
312	- dispatch_data = body.get(MOTOR_DISPATCH_KEY)
313	- if dispatch_data is None:
314	- return body, None
315	-
316	- self._reject_legacy_dispatch_fields(body)
317	-
318	- try:
319	- dispatch = MotorDispatch.model_validate(dispatch_data)
320	- except Exception as e:
321	- raise HTTPException(
322	- status_code=status.HTTP_400_BAD_REQUEST,
323	- detail=f"Invalid {MOTOR_DISPATCH_KEY}: {e}",
324	- ) from e
325	-
326	- self._validate_role(dispatch)
327	- await self._registry.activate(dispatch)
328	-
329	- try:
330	- engine_body = body.copy()
331	- engine_body.pop(MOTOR_DISPATCH_KEY, None)
332	- prefill_result = self._pop_and_validate_prefill_result(engine_body, dispatch)
333	- if prefill_result is not None:
334	- engine_body = await self._consume_prefill_result(engine_body, dispatch, prefill_result)
335	- engine_body = await self._adapt_engine_body(engine_body, dispatch)
336	- return engine_body, dispatch
337	- except Exception:
338	- await self.stop_peer(dispatch)
339	- await self.finish_dispatch(dispatch)
340	- raise
341	-
342	- def get_prefill_context_check(
343	- self,
344	- dispatch: MotorDispatch \| None,
345	- ) -> PrefillContextCheck \| None:
346	- """Return a post-tokenization context check for this request, if needed."""
347	- return None
348	-
349	- async def maybe_prepare_response(
350	- self, body: dict[str, Any], dispatch: MotorDispatch \| None
351	- ) -> dict[str, Any] \| None:
352	- return None
353	-
354	- async def should_finish_prepared_response(
355	- self,
356	- prepared: dict[str, Any],
357	- dispatch: MotorDispatch \| None,
358	- ) -> bool:
359	- return True
360	-
361	- async def prepare_metaserver_body(self, body: dict[str, Any]) -> dict[str, Any]:
362	- raise HTTPException(
363	- status_code=status.HTTP_404_NOT_FOUND,
364	- detail="Metaserver is not available for this endpoint.",
365	- )
366	-
367	- async def prepare_metaserver_request(self, body: dict[str, Any]) -> DispatchMetaserverRequest:
368	- return DispatchMetaserverRequest(engine_body=await self.prepare_metaserver_body(body))
369	-
370	- async def normalize_response(self, response: Response, context: DispatchResponseContext) -> Response:
371	- return response
372	-
373	- async def normalize_stream_chunk(
374	- self,
375	- chunk: bytes \| str,
376	- context: DispatchResponseContext,
377	- state: dict[str, Any],
378	- ) -> bytes \| str \| None:
379	- return chunk
380	-
381	- def register_error_handlers(self, app: Any) -> None:
382	- """Install engine-specific FastAPI handlers.
383	-
384	- The endpoint must not branch on engine type. Adapters own both the
385	- error format and the decision whether an application-level handler is
386	- required (for example, validation errors raised before a route runs).
387	- """
388	-
389	- def map_serving_exception(self, exc: Exception, *, has_dispatch: bool) -> Exception:
390	- """Apply the generic serving exception policy for this engine."""
391	- from motor.engine_server.core.serving_error import map_serving_exception
392	-
393	- return map_serving_exception(exc, map_unknown_to_http_500=not has_dispatch)
394	-
395	- def map_stream_error(self, exc: Exception, context: DispatchResponseContext) -> str \| None:
396	- """Return a serialized SSE error payload after response headers are sent.
397	-
398	- Non-vLLM engines must still emit a structured event instead of raising,
399	- because the HTTP status can no longer be changed once streaming starts.
400	- """
401	- if isinstance(exc, HTTPException):
402	- message = exc.detail if isinstance(exc.detail, str) else str(exc.detail)
403	- payload = {
404	- "error": {
405	- "message": sanitize_error_message(message),
406	- "type": "EngineError",
407	- "code": "engine_error",
408	- }
409	- }
410	- else:
411	- payload = {
412	- "error": {
413	- "message": sanitize_error_message(str(exc)),
414	- "type": "EngineError",
415	- "code": "engine_error",
416	- }
417	- }
418	- return json.dumps(payload, separators=(",", ":"))
419	-
420	- def map_engine_error(self, exc: Exception, context: DispatchResponseContext) -> Response \| HTTPException:
421	- """Return the engine-native error representation.
422	-
423	- The base adapter is shared by non-vLLM engines. Keep the historical
424	- FastAPI semantics here; vLLM overrides this to return its OpenAI error
425	- envelope.
426	- """
427	- if isinstance(exc, HTTPException):
428	- return exc
429	- return JSONResponse(
430	- status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
431	- content={
432	- "error": {
433	- "message": sanitize_error_message(str(exc)),
434	- "type": "EngineError",
435	- "code": "engine_error",
436	- }
437	- },
438	- )
439	-
440	- async def handle_stop(self, body: dict[str, Any]) -> DispatchStopResponse:
441	- try:
442	- stop_request = DispatchStopRequest.model_validate(body)
443	- except Exception as e:
444	- raise HTTPException(
445	- status_code=status.HTTP_400_BAD_REQUEST,
446	- detail=f"Invalid dispatch stop request: {e}",
447	- ) from e
448	-
449	- state = await self._registry.stop(stop_request)
450	- return DispatchStopResponse(
451	- root_request_id=stop_request.root_request_id,
452	- attempt_seq=stop_request.attempt_seq,
453	- accepted=state
454	- in (
455	- DispatchStopState.STOPPED,
456	- DispatchStopState.ALREADY_STOPPED,
457	- DispatchStopState.ALREADY_DONE,
458	- ),
459	- state=state,
460	- )
461	-
462	- async def finish_dispatch(self, dispatch: MotorDispatch \| None) -> None:
463	- if dispatch is not None:
464	- await self._registry.finish(dispatch)
465	-
466	- async def is_dispatch_stopped(self, dispatch: MotorDispatch \| None) -> bool:
467	- if dispatch is None:
468	- return False
469	- return await self._registry.is_stopped(dispatch)
470	-
471	- async def stop_peer(
472	- self,
473	- dispatch: MotorDispatch \| None,
474	- reason: DispatchStopReason = DispatchStopReason.PEER_FAILED,
475	- ) -> DispatchStopResponse \| None:
476	- if dispatch is None:
477	- return None
478	- return await self._peer_stop_client.stop_peer(dispatch, reason)
479	-
480	- async def _adapt_engine_body(self, body: dict[str, Any], dispatch: MotorDispatch) -> dict[str, Any]:
481	- return body
482	-
483	- async def _consume_prefill_result(
484	- self,
485	- body: dict[str, Any],
486	- dispatch: MotorDispatch,
487	- prefill_result: PrefillResult,
488	- ) -> dict[str, Any]:
489	- return body
490	-
491	- def _validate_role(self, dispatch: MotorDispatch) -> None:
492	- if self._local_role in ("union", "both"):
493	- return
494	- if self._local_role != dispatch.role:
495	- raise HTTPException(
496	- status_code=status.HTTP_400_BAD_REQUEST,
497	- detail=(f"Dispatch role {dispatch.role} does not match endpoint role {self._local_role}"),
498	- )
499	-
500	- @staticmethod
501	- def _reject_legacy_dispatch_fields(body: dict[str, Any]) -> None:
502	- legacy_fields = [key for key in body if key == "kv_transfer_params" or key.startswith("bootstrap_")]
503	- if legacy_fields:
504	- raise HTTPException(
505	- status_code=status.HTTP_400_BAD_REQUEST,
506	- detail=(
507	- "Legacy engine dispatch fields are not allowed with "
508	- f"{MOTOR_DISPATCH_KEY}: {', '.join(sorted(legacy_fields))}"
509	- ),
510	- )
511	-
512	- @staticmethod
513	- def _pop_and_validate_prefill_result(body: dict[str, Any], dispatch: MotorDispatch) -> PrefillResult \| None:
514	- result_data = body.pop(MOTOR_PREFILL_RESULT_KEY, None)
515	- if result_data is None:
516	- return None
517	- try:
518	- prefill_result = PrefillResult.model_validate(result_data)
519	- except Exception as e:
520	- raise HTTPException(
521	- status_code=status.HTTP_400_BAD_REQUEST,
522	- detail=f"Invalid {MOTOR_PREFILL_RESULT_KEY}: {e}",
523	- ) from e
524	- if dispatch.role != "decode":
525	- raise HTTPException(
526	- status_code=status.HTTP_400_BAD_REQUEST,
527	- detail=f"{MOTOR_PREFILL_RESULT_KEY} is only valid for decode dispatch.",
528	- )
529	- if not prefill_result.matches_dispatch(dispatch):
530	- raise HTTPException(
531	- status_code=status.HTTP_400_BAD_REQUEST,
532	- detail=f"{MOTOR_PREFILL_RESULT_KEY} does not match {MOTOR_DISPATCH_KEY}.",
533	- )
534	- return prefill_result

Dmotor/engine_server/core/dispatch_adapter/factory.py+0-23

	@@ -1,23 +0,0 @@
1	-# Copyright (c) Huawei Technologies Co., Ltd. 2026. All rights reserved.
2	-# MindIE is licensed under Mulan PSL v2.
3	-# You can use this software according to the terms and conditions of the Mulan PSL v2.
4	-# You may obtain a copy of Mulan PSL v2 at:
5	-# http://license.coscl.org.cn/MulanPSL2
6	-# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
7	-# EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
8	-# MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
9	-# See the Mulan PSL v2 for more details.
10	-
11	-from motor.node_manager.core.services.native_engine.backends.base import IConfig
12	-from motor.engine_server.core.dispatch_adapter.base import DispatchAdapter
13	-from motor.engine_server.core.dispatch_adapter.sglang_adapter import SGLangDispatchAdapter
14	-from motor.engine_server.core.dispatch_adapter.vllm_adapter import VLLMDispatchAdapter
15	-
16	-
17	-def create_dispatch_adapter(config: IConfig) -> DispatchAdapter:
18	- engine_type = config.get_endpoint_config().engine_type
19	- if engine_type == "vllm":
20	- return VLLMDispatchAdapter(config)
21	- if engine_type == "sglang":
22	- return SGLangDispatchAdapter(config)
23	- return DispatchAdapter(config)

Dmotor/engine_server/core/dispatch_adapter/normalization.py+0-220

	@@ -1,220 +0,0 @@
1	-# Copyright (c) Huawei Technologies Co., Ltd. 2026. All rights reserved.
2	-# MindIE is licensed under Mulan PSL v2.
3	-# You can use this software according to the terms and conditions of the Mulan PSL v2.
4	-# You may obtain a copy of Mulan PSL v2 at:
5	-# http://license.coscl.org.cn/MulanPSL2
6	-# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
7	-# EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
8	-# MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
9	-# See the Mulan PSL v2 for more details.
10	-
11	-"""Client-facing OpenAI response normalization for dispatch adapters."""
12	-
13	-from __future__ import annotations
14	-
15	-import json
16	-from typing import Any
17	-
18	-import msgspec
19	-
20	-
21	-CHOICES = "choices"
22	-CONTENT = "content"
23	-DELTA = "delta"
24	-MESSAGE = "message"
25	-PROMPT_TOKEN_IDS = "prompt_token_ids" # nosec B105 -- OpenAI response JSON field name
26	-ROLE = "role"
27	-TEXT = "text"
28	-TOKEN_IDS = "token_ids" # nosec B105 -- OpenAI response JSON field name
29	-KV_TRANSFER_PARAMS = "kv_transfer_params"
30	-
31	-
32	-def normalize_nonstream_body(
33	- body: dict[str, Any],
34	- *,
35	- client_expects_chat_shape: bool = False,
36	- req_id: str = "",
37	- client_return_token_ids: bool = False,
38	-) -> None:
39	- if client_expects_chat_shape and _is_completion_like_body(body):
40	- _adapt_completion_nonstream_to_chat(body, req_id=req_id)
41	- _strip_token_id_fields(body, client_return_token_ids=client_return_token_ids)
42	-
43	-
44	-def strip_engine_dispatch_fields(body: dict[str, Any]) -> None:
45	- """Remove engine-native dispatch fields before retrying or exposing a body."""
46	- body.pop(KV_TRANSFER_PARAMS, None)
47	- for key in list(body):
48	- if key.startswith("bootstrap_"):
49	- body.pop(key, None)
50	-
51	-
52	-def normalize_stream_chunk(
53	- chunk: bytes \| str,
54	- *,
55	- client_expects_chat_shape: bool = False,
56	- req_id: str = "",
57	- stream_state: dict[str, Any] \| None = None,
58	- client_return_token_ids: bool = False,
59	-) -> bytes \| str \| None:
60	- chunk_bytes = chunk.encode("utf-8") if isinstance(chunk, str) else chunk
61	- chunk_json = _parse_stream_chunk_json(chunk_bytes)
62	- if chunk_json is None:
63	- text = chunk_bytes.decode("utf-8", errors="replace").strip()
64	- if "[DONE]" in text:
65	- return chunk
66	- return b"" if isinstance(chunk, bytes) else ""
67	- if client_expects_chat_shape and _is_completion_like_stream_chunk(chunk_json):
68	- _adapt_completion_stream_chunk_to_chat(
69	- chunk_json,
70	- req_id=req_id,
71	- stream_state=stream_state if stream_state is not None else {},
72	- )
73	- _strip_token_id_fields(chunk_json, client_return_token_ids=client_return_token_ids)
74	- out = _encode_stream_chunk_bytes(chunk_bytes, chunk_json)
75	- return out if isinstance(chunk, bytes) else out.decode("utf-8")
76	-
77	-
78	-def _strip_token_id_fields(
79	- obj: dict[str, Any],
80	- *,
81	- client_return_token_ids: bool = False,
82	-) -> None:
83	- if not client_return_token_ids:
84	- obj.pop(PROMPT_TOKEN_IDS, None)
85	- for choice in obj.get(CHOICES) or []:
86	- if not isinstance(choice, dict):
87	- continue
88	- if not client_return_token_ids:
89	- choice.pop(TOKEN_IDS, None)
90	- choice.pop(PROMPT_TOKEN_IDS, None)
91	- if choice.get("stop_reason") == "recomputed":
92	- choice["stop_reason"] = "stop"
93	-
94	-
95	-def _parse_stream_chunk_json(chunk: bytes) -> dict[str, Any] \| None:
96	- try:
97	- chunk_str = chunk.decode("utf-8").strip()
98	- except UnicodeDecodeError:
99	- return None
100	- if not chunk_str:
101	- return None
102	- if chunk_str.startswith("data: "):
103	- chunk_str = chunk_str[len("data: ") :]
104	- try:
105	- parsed = json.loads(chunk_str)
106	- except json.JSONDecodeError:
107	- return None
108	- return parsed if isinstance(parsed, dict) else None
109	-
110	-
111	-def _encode_stream_chunk_bytes(original_chunk: bytes, chunk_json: dict[str, Any]) -> bytes:
112	- raw = original_chunk.decode("utf-8", errors="replace").strip()
113	- payload = _compact_json_bytes(chunk_json)
114	- line = b"data: " + payload if raw.startswith("data: ") else payload
115	- if original_chunk.endswith(b"\r\n\r\n"):
116	- suffix = b"\r\n\r\n"
117	- elif original_chunk.endswith(b"\n\n"):
118	- suffix = b"\n\n"
119	- elif original_chunk.endswith(b"\r\n"):
120	- suffix = b"\r\n"
121	- elif original_chunk.endswith(b"\n"):
122	- suffix = b"\n"
123	- else:
124	- suffix = b""
125	- return line + suffix
126	-
127	-
128	-def _compact_json_bytes(obj: Any) -> bytes:
129	- try:
130	- return msgspec.json.encode(obj)
131	- except Exception:
132	- return json.dumps(obj, separators=(",", ":")).encode("utf-8")
133	-
134	-
135	-def _is_completion_like_body(body: dict[str, Any]) -> bool:
136	- if body.get("object") == "text_completion":
137	- return True
138	- choices = body.get(CHOICES) or []
139	- return bool(choices and isinstance(choices[0], dict) and TEXT in choices[0])
140	-
141	-
142	-def _is_completion_like_stream_chunk(chunk_json: dict[str, Any]) -> bool:
143	- if chunk_json.get("object") == "text_completion":
144	- return True
145	- choices = chunk_json.get(CHOICES) or []
146	- if not choices or not isinstance(choices[0], dict):
147	- return False
148	- choice = choices[0]
149	- if choice.get(DELTA):
150	- return False
151	- return TEXT in choice
152	-
153	-
154	-def _adapt_completion_nonstream_to_chat(body: dict[str, Any], *, req_id: str) -> None:
155	- body["object"] = "chat.completion"
156	- body["id"] = _chat_completion_id(req_id)
157	- choices = body.get(CHOICES) or []
158	- if not choices or not isinstance(choices[0], dict):
159	- return
160	- choice = choices[0]
161	- text = choice.pop(TEXT, None) or ""
162	- finish_reason = choice.pop("finish_reason", None)
163	- stop_reason = choice.pop("stop_reason", None)
164	- token_ids = choice.pop(TOKEN_IDS, None)
165	- choice.pop("logprobs", None)
166	- choice.pop("prompt_logprobs", None)
167	- _lift_prompt_token_ids(choice, body)
168	- choice.clear()
169	- choice["index"] = 0
170	- choice[MESSAGE] = {ROLE: "assistant", CONTENT: text}
171	- if finish_reason is not None:
172	- choice["finish_reason"] = finish_reason
173	- if stop_reason is not None:
174	- choice["stop_reason"] = stop_reason
175	- if token_ids is not None:
176	- choice[TOKEN_IDS] = token_ids
177	-
178	-
179	-def _adapt_completion_stream_chunk_to_chat(
180	- chunk_json: dict[str, Any],
181	- *,
182	- req_id: str,
183	- stream_state: dict[str, Any],
184	-) -> None:
185	- chunk_json["object"] = "chat.completion.chunk"
186	- chunk_json["id"] = _chat_completion_id(req_id)
187	- choices = chunk_json.get(CHOICES) or []
188	- if not choices or not isinstance(choices[0], dict):
189	- return
190	- choice = choices[0]
191	- index = choice.get("index", 0)
192	- text = choice.pop(TEXT, None) or ""
193	- finish_reason = choice.pop("finish_reason", None)
194	- stop_reason = choice.pop("stop_reason", None)
195	- choice.pop("logprobs", None)
196	- _lift_prompt_token_ids(choice, chunk_json)
197	- delta: dict[str, Any] = {}
198	- if not stream_state.get("stream_role_sent"):
199	- delta[ROLE] = "assistant"
200	- stream_state["stream_role_sent"] = True
201	- if text:
202	- delta[CONTENT] = text
203	- choice.clear()
204	- choice["index"] = index
205	- choice[DELTA] = delta
206	- if finish_reason is not None:
207	- choice["finish_reason"] = finish_reason
208	- if stop_reason is not None:
209	- choice["stop_reason"] = stop_reason
210	-
211	-
212	-def _lift_prompt_token_ids(choice: dict[str, Any], response: dict[str, Any]) -> None:
213	- prompt_token_ids = choice.pop(PROMPT_TOKEN_IDS, None)
214	- if prompt_token_ids is not None and response.get(PROMPT_TOKEN_IDS) is None:
215	- response[PROMPT_TOKEN_IDS] = prompt_token_ids
216	-
217	-
218	-def _chat_completion_id(req_id: str) -> str:
219	- base = req_id.replace("cmpl-", "").replace("chatcmpl-", "")
220	- return f"chatcmpl-{base}"

Dmotor/engine_server/core/dispatch_adapter/sglang_adapter.py+0-48

	@@ -1,48 +0,0 @@
1	-# Copyright (c) Huawei Technologies Co., Ltd. 2026. All rights reserved.
2	-# MindIE is licensed under Mulan PSL v2.
3	-# You can use this software according to the terms and conditions of the Mulan PSL v2.
4	-# You may obtain a copy of Mulan PSL v2 at:
5	-# http://license.coscl.org.cn/MulanPSL2
6	-# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
7	-# EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
8	-# MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
9	-# See the Mulan PSL v2 for more details.
10	-
11	-import hashlib
12	-from typing import Any
13	-from urllib.parse import urlparse
14	-
15	-from fastapi import HTTPException, status
16	-
17	-from motor.common.resources.dispatch import MotorDispatch
18	-from motor.engine_server.core.dispatch_adapter.base import DispatchAdapter
19	-
20	-
21	-class SGLangDispatchAdapter(DispatchAdapter):
22	- async def _adapt_engine_body(self, body: dict[str, Any], dispatch: MotorDispatch) -> dict[str, Any]:
23	- body["request_id"] = dispatch.engine_request_id
24	- prefill = dispatch.endpoints.prefill
25	- if prefill is None:
26	- return body
27	-
28	- parsed = urlparse(prefill.url)
29	- bootstrap_port = prefill.bootstrap_port
30	- if bootstrap_port is None:
31	- raise HTTPException(
32	- status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
33	- detail=("Prefill endpoint bootstrap_port is required for SGLang dispatch requests."),
34	- )
35	- body.update(
36	- {
37	- "bootstrap_host": parsed.hostname or prefill.url,
38	- "bootstrap_port": bootstrap_port,
39	- "bootstrap_room": self._stable_bootstrap_room(dispatch),
40	- }
41	- )
42	- return body
43	-
44	- @staticmethod
45	- def _stable_bootstrap_room(dispatch: MotorDispatch) -> int:
46	- raw = f"{dispatch.pair_id}:{dispatch.attempt_seq}".encode("utf-8")
47	- digest = hashlib.blake2b(raw, digest_size=8).digest()
48	- return int.from_bytes(digest, "big") & ((1 << 63) - 1)

Dmotor/engine_server/core/dispatch_adapter/vllm_adapter.py+0-340

	@@ -1,340 +0,0 @@
1	-# Copyright (c) Huawei Technologies Co., Ltd. 2026. All rights reserved.
2	-# MindIE is licensed under Mulan PSL v2.
3	-# You can use this software according to the terms and conditions of the Mulan PSL v2.
4	-# You may obtain a copy of Mulan PSL v2 at:
5	-# http://license.coscl.org.cn/MulanPSL2
6	-# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
7	-# EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
8	-# MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
9	-# See the Mulan PSL v2 for more details.
10	-
11	-import json
12	-from typing import Any
13	-
14	-from fastapi import HTTPException, status
15	-from fastapi.responses import JSONResponse, Response
16	-
17	-from motor.common.constants import (
18	- CHAT_COMPLETION_PREFIX,
19	- COMPLETION_PREFIX,
20	- COMPLETION_SUFFIX,
21	-)
22	-from motor.common.resources.dispatch import (
23	- DispatchProfile,
24	- MotorDispatch,
25	- PrefillResult,
26	- infer_vllm_dispatch_profile_from_config,
27	-)
28	-from motor.engine_server.core.dispatch_adapter.base import (
29	- DispatchAdapter,
30	- DispatchMetaserverRequest,
31	- DispatchResponseContext,
32	-)
33	-from motor.engine_server.core.dispatch_adapter.normalization import (
34	- normalize_nonstream_body,
35	- normalize_stream_chunk,
36	-)
37	-from motor.engine_server.core.vllm.prefill_context_validation import (
38	- PrefillContextCheck,
39	-)
40	-
41	-
42	-class VLLMDispatchAdapter(DispatchAdapter):
43	- _METASERVER_PREFILL_WAIT_SECONDS = 30.0
44	- _LEGACY_CPCD_DISPATCH_MODE = "cpcd_separate"
45	-
46	- _METASERVER_REQUIRED_FIELDS = (
47	- "request_id",
48	- "do_remote_decode",
49	- "remote_block_ids",
50	- "remote_block_size",
51	- "remote_engine_id",
52	- "remote_host",
53	- "remote_port",
54	- "remote_cached_tokens",
55	- )
56	-
57	- def __init__(self, config) -> None:
58	- super().__init__(config)
59	- self._dispatch_profile = self._infer_dispatch_profile(config)
60	-
61	- def map_engine_error(self, exc: Exception, context: DispatchResponseContext) -> Response:
62	- from motor.engine_server.core.vllm.vllm_openai_compat import openai_http_response_from_exception
63	-
64	- return openai_http_response_from_exception(exc)
65	-
66	- def register_error_handlers(self, app: Any) -> None:
67	- from motor.engine_server.core.vllm.vllm_openai_compat import register_vllm_openai_error_handlers
68	-
69	- register_vllm_openai_error_handlers(app)
70	-
71	- def map_serving_exception(self, exc: Exception, *, has_dispatch: bool) -> Exception:
72	- # vLLM's native create_error_response() must see the original
73	- # exception. Pre-converting it to HTTPException changes both type and
74	- # message semantics compared with standalone vLLM.
75	- return exc
76	-
77	- def map_stream_error(self, exc: Exception, context: DispatchResponseContext) -> str \| None:
78	- from motor.engine_server.core.vllm.vllm_openai_compat import vllm_stream_error_json
79	-
80	- return vllm_stream_error_json(exc)
81	-
82	- async def _adapt_engine_body(self, body: dict[str, Any], dispatch: MotorDispatch) -> dict[str, Any]:
83	- body["request_id"] = dispatch.engine_request_id
84	- if dispatch.role == "decode":
85	- # In handoff the decode kv_transfer_params is the KV bootstrap threaded
86	- # in from the prefill result (_consume_prefill_result). The metaserver
87	- # callback is exclusive to the trigger profile, so a handoff connector
88	- # (e.g. MooncakeConnectorV1) must never be handed a metaserver URL -- it
89	- # expects remote_block_ids/remote_host/remote_port instead.
90	- if not self._uses_handoff(dispatch):
91	- prefill = dispatch.endpoints.prefill
92	- if prefill is not None and "kv_transfer_params" not in body:
93	- body["kv_transfer_params"] = {
94	- "do_remote_decode": False,
95	- "do_remote_prefill": True,
96	- "metaserver": f"{prefill.url.rstrip('/')}/v1/metaserver",
97	- }
98	- elif dispatch.role == "prefill":
99	- body.setdefault("return_token_ids", True)
100	- self._apply_prefill_generation_params(body)
101	- if self._uses_handoff(dispatch):
102	- # Tell the producer engine to generate KV for a remote decode so its
103	- # response carries the kv_transfer_params bootstrap. Without this the
104	- # connector's request_finished returns no bootstrap and PD silently
105	- # degrades. Mirrors the vLLM-ascend native proxy build_prefill_request.
106	- body.setdefault(
107	- "kv_transfer_params",
108	- {"do_remote_decode": True, "do_remote_prefill": False},
109	- )
110	- return body
111	-
112	- def get_prefill_context_check(
113	- self,
114	- dispatch: MotorDispatch \| None,
115	- ) -> PrefillContextCheck \| None:
116	- """Read the Coordinator-carried budget after P request rewriting."""
117	- if (
118	- dispatch is None
119	- or dispatch.role != "prefill"
120	- or not self._uses_handoff(dispatch)
121	- or dispatch.prefill_context_budget is None
122	- ):
123	- return None
124	- budget = dispatch.prefill_context_budget
125	- return PrefillContextCheck(
126	- max_output_tokens=budget.max_output_tokens,
127	- parameter=budget.parameter,
128	- )
129	-
130	- async def maybe_prepare_response(
131	- self, body: dict[str, Any], dispatch: MotorDispatch \| None
132	- ) -> dict[str, Any] \| None:
133	- if dispatch is None or dispatch.role != "prefill":
134	- return None
135	- if self._uses_handoff(dispatch):
136	- return None
137	- await self._registry.cache_prefill_body(dispatch, body)
138	- return PrefillResult(
139	- root_request_id=dispatch.root_request_id,
140	- engine_request_id=dispatch.engine_request_id,
141	- pair_id=dispatch.pair_id,
142	- attempt_seq=dispatch.attempt_seq,
143	- status="prepared",
144	- handoff_mode="trigger",
145	- ).model_dump(mode="json")
146	-
147	- async def should_finish_prepared_response(
148	- self,
149	- prepared: dict[str, Any],
150	- dispatch: MotorDispatch \| None,
151	- ) -> bool:
152	- if dispatch is None:
153	- return True
154	- return not (
155	- dispatch.role == "prefill"
156	- and prepared.get("status") == "prepared"
157	- and prepared.get("handoff_mode") == "trigger"
158	- )
159	-
160	- async def _consume_prefill_result(
161	- self,
162	- body: dict[str, Any],
163	- dispatch: MotorDispatch,
164	- prefill_result: PrefillResult,
165	- ) -> dict[str, Any]:
166	- if not self._uses_handoff(dispatch):
167	- return body
168	- if prefill_result.status != "completed":
169	- raise HTTPException(
170	- status_code=status.HTTP_400_BAD_REQUEST,
171	- detail="vLLM CPCD decode requires completed prefill result.",
172	- )
173	- if prefill_result.handoff_mode != "handoff":
174	- raise HTTPException(
175	- status_code=status.HTTP_400_BAD_REQUEST,
176	- detail="vLLM CPCD decode requires handoff prefill result.",
177	- )
178	- if prefill_result.payload:
179	- body["kv_transfer_params"] = prefill_result.payload.copy()
180	- return body
181	-
182	- async def prepare_metaserver_body(self, body: dict[str, Any]) -> dict[str, Any]:
183	- metaserver_request = await self.prepare_metaserver_request(body)
184	- return metaserver_request.engine_body
185	-
186	- async def prepare_metaserver_request(self, body: dict[str, Any]) -> DispatchMetaserverRequest:
187	- kv_transfer_params = self._extract_metaserver_kv_params(body)
188	- engine_request_id = self._extract_engine_request_id(kv_transfer_params)
189	- if not engine_request_id:
190	- raise HTTPException(
191	- status_code=status.HTTP_400_BAD_REQUEST,
192	- detail="Metaserver request missing request_id.",
193	- )
194	- cached_entry = await self._registry.wait_prefill_entry(engine_request_id, self._METASERVER_PREFILL_WAIT_SECONDS)
195	- if cached_entry is None:
196	- raise HTTPException(
197	- status_code=status.HTTP_404_NOT_FOUND,
198	- detail=f"Prefill context not found for request_id {engine_request_id}.",
199	- )
200	- try:
201	- self._validate_metaserver_kv_params(kv_transfer_params)
202	- except HTTPException:
203	- await self.stop_peer(cached_entry.dispatch)
204	- await self.finish_dispatch(cached_entry.dispatch)
205	- raise
206	- cached = cached_entry.body.copy()
207	- self._apply_prefill_generation_params(cached)
208	- # Preserve trigger-provided KV metadata while keeping the original
209	- # client request body cached by the prefill leg.
210	- cached["kv_transfer_params"] = kv_transfer_params
211	- return DispatchMetaserverRequest(
212	- engine_body=cached,
213	- dispatch=cached_entry.dispatch,
214	- )
215	-
216	- async def normalize_response(self, response: Response, context: DispatchResponseContext) -> Response:
217	- if context.dispatch is None:
218	- return response
219	- raw_body = getattr(response, "body", None)
220	- if not raw_body:
221	- return response
222	- try:
223	- body = json.loads(raw_body)
224	- except (TypeError, json.JSONDecodeError, UnicodeDecodeError):
225	- return response
226	- if not isinstance(body, dict):
227	- return response
228	- if context.dispatch is not None and context.dispatch.role == "prefill" and self._uses_handoff(context.dispatch):
229	- # Only successful prefill responses are valid handoff results.
230	- # Preserve validation and serving errors with their original HTTP
231	- # status and OpenAI-compatible error body.
232	- status_code = getattr(response, "status_code", 200)
233	- if not 200 <= status_code < 300 or body.get("error") is not None:
234	- return response
235	- # The decode leg consumes ``payload`` directly as its ``kv_transfer_params``
236	- # (see ``_consume_prefill_result``), and the engine connector reads the KV
237	- # bootstrap fields (do_remote_prefill, remote_block_ids, remote_host, ...) at
238	- # the top level of ``kv_transfer_params``.
239	- kv_transfer_params = body.get("kv_transfer_params")
240	- usage = body.get("usage")
241	- prefill_result = PrefillResult(
242	- root_request_id=context.dispatch.root_request_id,
243	- engine_request_id=context.dispatch.engine_request_id,
244	- pair_id=context.dispatch.pair_id,
245	- attempt_seq=context.dispatch.attempt_seq,
246	- status="completed",
247	- handoff_mode="handoff",
248	- payload=kv_transfer_params if isinstance(kv_transfer_params, dict) else {},
249	- # Preserve the prefill usage separately so the coordinator can still
250	- # capture prompt_tokens_details (cached tokens) -- payload now carries
251	- # only the KV bootstrap and no longer the full response body.
252	- usage=usage if isinstance(usage, dict) else None,
253	- )
254	- return JSONResponse(
255	- content=prefill_result.model_dump(mode="json"),
256	- status_code=response.status_code,
257	- )
258	- normalize_nonstream_body(
259	- body,
260	- client_expects_chat_shape=context.client_expects_chat_shape,
261	- req_id=context.dispatch.root_request_id,
262	- client_return_token_ids=context.client_return_token_ids,
263	- )
264	- headers = {
265	- key: value
266	- for key, value in response.headers.items()
267	- if key.lower() not in ("content-length", "content-type")
268	- }
269	- return JSONResponse(
270	- content=body,
271	- status_code=response.status_code,
272	- headers=headers,
273	- )
274	-
275	- async def normalize_stream_chunk(
276	- self,
277	- chunk: bytes \| str,
278	- context: DispatchResponseContext,
279	- state: dict[str, Any],
280	- ) -> bytes \| str \| None:
281	- if context.dispatch is None:
282	- return chunk
283	- return normalize_stream_chunk(
284	- chunk,
285	- client_expects_chat_shape=context.client_expects_chat_shape,
286	- req_id=context.dispatch.root_request_id,
287	- stream_state=state,
288	- client_return_token_ids=context.client_return_token_ids,
289	- )
290	-
291	- @staticmethod
292	- def _extract_engine_request_id(body: dict[str, Any]) -> str \| None:
293	- request_id = body.get("request_id")
294	- if not isinstance(request_id, str) or not request_id:
295	- return None
296	- if request_id.startswith(CHAT_COMPLETION_PREFIX):
297	- return request_id.removeprefix(CHAT_COMPLETION_PREFIX)
298	- if request_id.startswith(COMPLETION_PREFIX) and request_id.endswith(COMPLETION_SUFFIX):
299	- return request_id.removeprefix(COMPLETION_PREFIX).removesuffix(COMPLETION_SUFFIX)
300	- return request_id
301	-
302	- @staticmethod
303	- def _extract_metaserver_kv_params(body: dict[str, Any]) -> dict[str, Any]:
304	- nested = body.get("kv_transfer_params")
305	- if isinstance(nested, dict):
306	- return nested.copy()
307	- return body.copy()
308	-
309	- @classmethod
310	- def _validate_metaserver_kv_params(cls, kv_transfer_params: dict[str, Any]) -> None:
311	- missing = [field for field in cls._METASERVER_REQUIRED_FIELDS if field not in kv_transfer_params]
312	- if missing:
313	- raise HTTPException(
314	- status_code=status.HTTP_400_BAD_REQUEST,
315	- detail=f"Metaserver request missing KV fields: {', '.join(missing)}.",
316	- )
317	- if kv_transfer_params.get("do_remote_decode") is not True:
318	- raise HTTPException(
319	- status_code=status.HTTP_400_BAD_REQUEST,
320	- detail="Metaserver request must set do_remote_decode=true.",
321	- )
322	-
323	- @staticmethod
324	- def _apply_prefill_generation_params(body: dict[str, Any]) -> None:
325	- body["stream"] = False
326	- body["max_tokens"] = 1
327	- if "max_completion_tokens" in body:
328	- body["max_completion_tokens"] = 1
329	- body["min_tokens"] = 1
330	- body.pop("stream_options", None)
331	-
332	- def _uses_handoff(self, dispatch: MotorDispatch) -> bool:
333	- return (
334	- self._dispatch_profile == DispatchProfile.HANDOFF
335	- or dispatch.dispatch_mode == self._LEGACY_CPCD_DISPATCH_MODE
336	- )
337	-
338	- @classmethod
339	- def _infer_dispatch_profile(cls, config) -> DispatchProfile:
340	- return infer_vllm_dispatch_profile_from_config(config)

Dmotor/engine_server/core/endpoint.py+0-20

	@@ -1,20 +0,0 @@
1	-# Copyright (c) Huawei Technologies Co., Ltd. 2025-2026. All rights reserved.
2	-# MindIE is licensed under Mulan PSL v2.
3	-# You can use this software according to the terms and conditions of the Mulan PSL v2.
4	-# You may obtain a copy of Mulan PSL v2 at:
5	-# http://license.coscl.org.cn/MulanPSL2
6	-# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
7	-# EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
8	-# MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
9	-# See the Mulan PSL v2 for more details.
10	-
11	-from abc import ABC, abstractmethod
12	-
13	-
14	-class Endpoint(ABC):
15	- """Base interface for engine server HTTP listeners (inference API and management)."""
16	-
17	- @abstractmethod
18	- def run(self) -> None:
19	- """Start serving (may spawn a thread or process)."""
20	- pass

Dmotor/engine_server/core/engine.py+0-31

	@@ -1,31 +0,0 @@
1	-# Copyright (c) Huawei Technologies Co., Ltd. 2025-2026. All rights reserved.
2	-# MindIE is licensed under Mulan PSL v2.
3	-# You can use this software according to the terms and conditions of the Mulan PSL v2.
4	-# You may obtain a copy of Mulan PSL v2 at:
5	-# http://license.coscl.org.cn/MulanPSL2
6	-# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
7	-# EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
8	-# MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
9	-# See the Mulan PSL v2 for more details.
10	-
11	-from abc import abstractmethod, ABC
12	-from typing import Any
13	-
14	-from motor.common.logger import get_logger
15	-
16	-
17	-logger = get_logger(__name__)
18	-
19	-
20	-class Engine(ABC):
21	-
22	- @abstractmethod
23	- def launch(self) -> Any:
24	- """Launch the engine."""
25	- pass
26	-
27	-
28	- @abstractmethod
29	- def shutdown(self) -> None:
30	- """Shutdown the engine."""
31	- pass

Dmotor/engine_server/core/errors/__init__.py+0-15

	@@ -1,15 +0,0 @@
1	-# Copyright (c) Huawei Technologies Co., Ltd. 2025-2026. All rights reserved.
2	-# MindIE is licensed under Mulan PSL v2.
3	-# You can use this software according to the terms and conditions of the Mulan PSL v2.
4	-# You may obtain a copy of Mulan PSL v2 at:
5	-# http://license.coscl.org.cn/MulanPSL2
6	-# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
7	-# EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
8	-# MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
9	-# See the Mulan PSL v2 for more details.
10	-
11	-"""Engine-server error helpers that stay free of heavy HTTP/TLS imports."""
12	-
13	-from motor.engine_server.core.errors.sanitizer import sanitize_error_message
14	-
15	-__all__ = ["sanitize_error_message"]

Dmotor/engine_server/core/errors/sanitizer.py+0-43

	@@ -1,43 +0,0 @@
1	-# Copyright (c) Huawei Technologies Co., Ltd. 2025-2026. All rights reserved.
2	-# MindIE is licensed under Mulan PSL v2.
3	-# You can use this software according to the terms and conditions of the Mulan PSL v2.
4	-# You may obtain a copy of Mulan PSL v2 at:
5	-# http://license.coscl.org.cn/MulanPSL2
6	-# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
7	-# EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
8	-# MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
9	-# See the Mulan PSL v2 for more details.
10	-
11	-"""Lightweight error-message sanitization for engine error adapters.
12	-
13	-This module must stay free of TLS/HTTP client imports so error handling cannot
14	-fail again while trying to sanitize a message.
15	-"""
16	-
17	-from __future__ import annotations
18	-
19	-import re
20	-
21	-_FILE_PATH_RE = re.compile(r"[A-Za-z]:\\[^\s]+\|/[^\s]+")
22	-_FILE_LOCATION_RE = re.compile(r'File "[^"]+", line \d+')
23	-_TRACEBACK_RE = re.compile(r"Traceback \(most recent call last\):.*", flags=re.DOTALL)
24	-
25	-
26	-def sanitize_error_message(error_msg: str) -> str:
27	- """Strip paths, traceback fragments, and truncate overlong messages."""
28	- try:
29	- from motor.common.http.security_utils import sanitize_error_message as project_sanitize
30	-
31	- return project_sanitize(error_msg)
32	- except ImportError:
33	- pass
34	-
35	- message = _FILE_PATH_RE.sub("[FILE_PATH]", error_msg)
36	- message = _FILE_LOCATION_RE.sub("[FILE_LOCATION]", message)
37	- message = _TRACEBACK_RE.sub("", message)
38	- message = message.strip()
39	- if not message:
40	- return "An internal error occurred"
41	- if len(message) > 200:
42	- return message[:200] + "..."
43	- return message

Dmotor/engine_server/core/health_collector.py+0-66

Dmotor/engine_server/core/infer_endpoint.py+0-552

Dmotor/engine_server/core/mgmt_endpoint.py+0-180

Dmotor/engine_server/core/serving_error.py+0-188

Dmotor/engine_server/core/sglang/__init__.py+0-0

Dmotor/engine_server/core/sglang/openai/__init__.py+0-0

Dmotor/engine_server/core/sglang/openai/serving_chat.py+0-49

Dmotor/engine_server/core/sglang/openai/serving_completion.py+0-50

Dmotor/engine_server/core/sglang/openai/serving_models.py+0-68

Dmotor/engine_server/core/sglang/sglang_endpoint.py+0-68

Dmotor/engine_server/core/sglang/sglang_engine.py+0-87

Dmotor/engine_server/core/sim_inference.py+0-539

Dmotor/engine_server/core/snapshot_monitor.py+0-51

Dmotor/engine_server/core/snapshot_sentinel.py+0-200

Dmotor/engine_server/core/vllm/__init__.py+0-0

Dmotor/engine_server/core/vllm/openai/__init__.py+0-11

Dmotor/engine_server/core/vllm/openai/serving_chat.py+0-108

Dmotor/engine_server/core/vllm/openai/serving_completion.py+0-85

Dmotor/engine_server/core/vllm/prefill_context_validation.py+0-97

Dmotor/engine_server/core/vllm/vllm_endpoint.py+0-256

Dmotor/engine_server/core/vllm/vllm_engine.py+0-110

Dmotor/engine_server/core/vllm/vllm_openai_compat.py+0-486

Dmotor/engine_server/factory/__init__.py+0-11

Dmotor/engine_server/factory/endpoint_factory.py+0-48

Dmotor/engine_server/utils/__init__.py+0-11

Dmotor/engine_server/utils/ai_cube.py+0-122

Dmotor/engine_server/utils/cancellation.py+0-94

Dmotor/engine_server/utils/prometheus.py+0-53

Dmotor/engine_server/version.py+0-13

Mmotor/node_manager/core/engine_manager.py+4-4

Mmotor/node_manager/core/services/native_engine/backends/base.py+1-38

Mmotor/node_manager/core/services/native_engine/backends/sglang/config.py+1-24

Mmotor/node_manager/core/services/native_engine/backends/vllm/config.py+1-30

Mmotor/node_manager/core/services/native_engine/config_factory.py+15-13

Msetup.py+0-5

Mtests/common/logger/test_logger.py+4-6

Mtests/common/resources/test_dispatch_compatibility.py+11-50

Mtests/common/utils/test_process_title.py+1-68

Dtests/engine_server/__init__.py+0-11

Dtests/engine_server/cli/__init__.py+0-11

Dtests/engine_server/cli/conftest.py+0-40

Dtests/engine_server/cli/test_main_native_launch.py+0-302

Dtests/engine_server/config/__init__.py+0-11

Dtests/engine_server/config/test_endpoint.py+0-892

Dtests/engine_server/core/__init__.py+0-11

Dtests/engine_server/core/conftest.py+0-62

Dtests/engine_server/core/test_dispatch_adapter.py+0-896

Dtests/engine_server/core/test_health_collector.py+0-203

Dtests/engine_server/core/test_infer_endpoint_response_normalization.py+0-739

Dtests/engine_server/core/test_mgmt_endpoint.py+0-93

Dtests/engine_server/core/test_mgmt_endpoint_bootstrap.py+0-80

Dtests/engine_server/core/test_prefill_context_validation.py+0-94

Dtests/engine_server/core/test_serving_error.py+0-233

Dtests/engine_server/core/test_sim_inference.py+0-1153

Dtests/engine_server/core/test_snapshot_monitor.py+0-37

Dtests/engine_server/core/test_snapshot_sentinel.py+0-269

Dtests/engine_server/core/test_vllm_openai_compat.py+0-532

Dtests/engine_server/core/test_vllm_serving_render_compat.py+0-217

Dtests/engine_server/factory/__init__.py+0-11

Dtests/engine_server/utils/__init__.py+0-11

Dtests/engine_server/utils/test_ai_cube.py+0-177

Dtests/engine_server/utils/test_cancellation.py+0-66

Dtests/engine_server/utils/test_data/test.txt+0-0

Dtests/engine_server/utils/test_ip.py+0-185

Dtests/engine_server/utils/test_prometheus.py+0-66

Dtests/engine_server/utils/test_validators.py+0-216

Mtests/node_manager/core/services/native_engine/backends/test_config_contract.py+1-13

Mtests/node_manager/core/services/native_engine/test_backends.py+1-3

Mtests/node_manager/test_config.py+12-3

	@@ -195,9 +195,8 @@ Hot-reload is driven by a `ConfigWatcher` in the Mgmt process (not the daemo
195	\| `motor/coordinator/router/strategies/` \| \| `BaseRouter` + `PDHybridRouter` + `UnifiedPDRouter` implementations \|	195	\| `motor/coordinator/router/strategies/` \| \| `BaseRouter` + `PDHybridRouter` + `UnifiedPDRouter` implementations \|
196	\| `motor/coordinator/router/dispatch_session.py` \| \| Dispatch attempt session/state tracking \|	196	\| `motor/coordinator/router/dispatch_session.py` \| \| Dispatch attempt session/state tracking \|
197	\| `motor/coordinator/router/rescheduler/` \| \| `Rescheduler` (retry plans for failed requests) \|	197	\| `motor/coordinator/router/rescheduler/` \| \| `Rescheduler` (retry plans for failed requests) \|
198	-\| `motor/coordinator/router/dispatch_capability.py` \| \| P/D dispatch capability (kv_connector / dispatch_profile) checks \|	198	+\| `motor/coordinator/api_client/` \| \| `ConductorApiClient` / `ControllerApiClient` / `NativeEngineApiClient` (HTTP clients to kv-conductor, controller, engine) \|
199	-\| `motor/coordinator/api_client/` \| \| `ConductorApiClient` / `ControllerApiClient` / `EngineServerApiClient` (HTTP clients to kv-conductor, controller, engine) \|	199	+\| `motor/coordinator/api_server/management_server.py` \| \| Mgmt: `/liveness`, `/readiness`, `/instances/refresh`, `/precision/alarm_cleared` \|
200	-\| `motor/coordinator/api_server/management_server.py` \| \| Mgmt: `/liveness`, `/readiness`, `/instances/refresh` \|
201	\| `motor/coordinator/api_server/observability_server.py` \| \| Obs: `/metrics`, `/health` (`/instance/metrics` deprecated → `GET /metrics?type=instance`) \|	200	\| `motor/coordinator/api_server/observability_server.py` \| \| Obs: `/metrics`, `/health` (`/instance/metrics` deprecated → `GET /metrics?type=instance`) \|
202	\| `motor/coordinator/api_server/inference_server.py` \| \| Infer: `/v1/completions`, `/v1/chat/completions`, `/v1/models`, `/v1/messages` + `/v1/messages/count_tokens` (Anthropic) \|	201	\| `motor/coordinator/api_server/inference_server.py` \| \| Infer: `/v1/completions`, `/v1/chat/completions`, `/v1/models`, `/v1/messages` + `/v1/messages/count_tokens` (Anthropic) \|
203	\| `motor/coordinator/scheduler/runtime/scheduler_connection_manager.py` \| \| Shared Scheduler ZMQ connection (used by Mgmt/Obs/Infer) \|	202	\| `motor/coordinator/scheduler/runtime/scheduler_connection_manager.py` \| \| Shared Scheduler ZMQ connection (used by Mgmt/Obs/Infer) \|
	@@ -219,8 +218,12 @@ Controller detects instance change
219	→ ZMQ REFRESH_INSTANCES to Scheduler	218	→ ZMQ REFRESH_INSTANCES to Scheduler
220	→ Scheduler updates master InstanceManager, bumps version	219	→ Scheduler updates master InstanceManager, bumps version
221	→ PUB socket: INSTANCE_CHANGE_TOPIC notification (+ delta frame for ADD/DEL)	220	→ PUB socket: INSTANCE_CHANGE_TOPIC notification (+ delta frame for ADD/DEL)
222	- → Workload SHM: instance_version bump in header	221	+ → Workload SHM: instance_version bump in header
223	- → Workers: patch/invalidate caches, re-fetch on next scheduling call	222	+ → Workers: patch/invalidate caches, re-fetch on next scheduling call
		223	+
		224	+Controller clears a handled precision alarm
		225	+ → POST /precision/alarm_cleared (clear scheduler precision-alarm state for a P/D group)
		226	+ → Mgmt sends DISMISS_PRECISION_ALARM_STATE to Scheduler
224	```	227	```
225		228
226	## Fault Tolerance: Circuit Breaker & Precision Detection	229	## Fault Tolerance: Circuit Breaker & Precision Detection

	@@ -5,12 +5,12 @@
5		5
6	## 项目简介	6	## 项目简介
7		7
8	-MindIE-PyMotor 是面向大模型（LLM）分布式推理的控制器系统：Controller 管理实例生命周期，Coordinator 负责请求调度（PD 分离），NodeManager 管理引擎进程，EngineServer 封装 vLLM/SGLang，KV Conductor（Rust）提供 KV 缓存感知路由。	8	+MindIE-PyMotor 是面向大模型（LLM）分布式推理的控制器系统：Controller 管理实例生命周期，Coordinator 负责请求调度（PD 分离），NodeManager 直接管理 vLLM/SGLang 原生引擎进程，KV Conductor（Rust）提供 KV 缓存感知路由。
9		9
10	## 仓库结构	10	## 仓库结构
11		11
12	```text	12	```text
13	-motor/ Python 源码（coordinator / controller / node_manager / engine_server / common / config）	13	+motor/ Python 源码（coordinator / controller / node_manager / common / config）
14	motor/kv_conductor/ Rust KV Conductor（axum + tokio + ZMQ）	14	motor/kv_conductor/ Rust KV Conductor（axum + tokio + ZMQ）
15	tests/ 测试（目录镜像 motor/ 结构）	15	tests/ 测试（目录镜像 motor/ 结构）
16	examples/ 部署配置示例（user_config.json、deployer）	16	examples/ 部署配置示例（user_config.json、deployer）
	@@ -88,6 +88,6 @@ bash tests/run_tests.sh --cov tests/
88	深度开发规范在 `.agent/skills/motor-dev/`（Claude Code 中 `/motor-dev` 调用，其他 agent 直接读目录）：	88	深度开发规范在 `.agent/skills/motor-dev/`（Claude Code 中 `/motor-dev` 调用，其他 agent 直接读目录）：
89		89
90	- `SKILL.md` — 硬性约束（测试伴随改动、run_tests.sh、license、类型语法）、渐进式测试工作流、Skill Sync 铁律（发现文档与代码不符必须同步更新并同 PR 合入）	90	- `SKILL.md` — 硬性约束（测试伴随改动、run_tests.sh、license、类型语法）、渐进式测试工作流、Skill Sync 铁律（发现文档与代码不符必须同步更新并同 PR 合入）
91	-- `references/<module>.md` — 各模块架构（Coordinator/Controller/NodeManager/EngineServer/Metrics/KV Conductor）	91	+- `references/<module>.md` — 各模块架构（Coordinator/Controller/NodeManager/Metrics/KV Conductor）
92	- `bug-fix-history/INDEX.md` — 持续学习案例索引（调试前先查）	92	- `bug-fix-history/INDEX.md` — 持续学习案例索引（调试前先查）
93	- `references/issue-reporting.md` — 定位问题后按模板提交 ISSUE（仅用户同意后加载）	93	- `references/issue-reporting.md` — 定位问题后按模板提交 ISSUE（仅用户同意后加载）

	@@ -304,8 +304,8 @@ def apply_node_manager_ports(config: NodeManagerConfig) -> None:
304	mgmt_port = _auto(int(mgmt_pref), f"mgmt_ports[{idx}]")	304	mgmt_port = _auto(int(mgmt_pref), f"mgmt_ports[{idx}]")
305	new_service_ports.append(str(svc_port))	305	new_service_ports.append(str(svc_port))
306	new_mgmt_ports.append(str(mgmt_port))	306	new_mgmt_ports.append(str(mgmt_port))
307	- rows.append(_row("EngineServer", host, svc_port, "auto", f"DP{idx} business"))	307	+ rows.append(_row("NativeEngine", host, svc_port, "auto", f"DP{idx} business"))
308	- rows.append(_row("EngineServer", host, mgmt_port, "auto", f"DP{idx} mgmt"))	308	+ rows.append(_row("NodeManager", host, mgmt_port, "auto", f"DP{idx} reserved"))
309		309
310	ep.service_ports = new_service_ports	310	ep.service_ports = new_service_ports
311	ep.mgmt_ports = new_mgmt_ports	311	ep.mgmt_ports = new_mgmt_ports
	@@ -313,13 +313,13 @@ def apply_node_manager_ports(config: NodeManagerConfig) -> None:
313	if sc.single_container_flag:	313	if sc.single_container_flag:
314	if sc.kv_port is not None:	314	if sc.kv_port is not None:
315	sc.kv_port = _auto(sc.kv_port, "kv_port")	315	sc.kv_port = _auto(sc.kv_port, "kv_port")
316	- rows.append(_row("EngineServer", host, sc.kv_port, "auto", "KV transfer"))	316	+ rows.append(_row("NativeEngine", host, sc.kv_port, "auto", "KV transfer"))
317	if sc.lookup_rpc_port is not None:	317	if sc.lookup_rpc_port is not None:
318	sc.lookup_rpc_port = _auto(sc.lookup_rpc_port, "lookup_rpc_port")	318	sc.lookup_rpc_port = _auto(sc.lookup_rpc_port, "lookup_rpc_port")
319	- rows.append(_row("EngineServer", host, sc.lookup_rpc_port, "auto", "KV lookup RPC"))	319	+ rows.append(_row("NativeEngine", host, sc.lookup_rpc_port, "auto", "KV lookup RPC"))
320	if sc.dp_rpc_port is not None:	320	if sc.dp_rpc_port is not None:
321	sc.dp_rpc_port = _auto(sc.dp_rpc_port, "dp_rpc_port")	321	sc.dp_rpc_port = _auto(sc.dp_rpc_port, "dp_rpc_port")
322	- rows.append(_row("EngineServer", host, sc.dp_rpc_port, "auto", "DP RPC"))	322	+ rows.append(_row("NativeEngine", host, sc.dp_rpc_port, "auto", "DP RPC"))
323		323
324	PortAllocator.print_matrix(rows)	324	PortAllocator.print_matrix(rows)
325		325