---

Overview

InferNex Bridge is the adaptation layer (Controller + Webhook) for integrating InferNex with KServe. Based on LLMInferenceService (KServe's LLM inference CRD) / InferNexService (InferNex Bridge's native inference CRD) declarations, it automatically deploys and reconciles the InferNex inference suite.

Two deployment modes are available:

• KServe + InferNex Bridge: The inference engine / Hermes Router is deployed by KServe, while enhancement components are deployed by InferNex Bridge.
• InferNex Bridge: InferNex Bridge uniformly orchestrates the inference engine, Hermes Router, and enhancement components (Mooncake KVCache, cache-indexer, proxy-server, Elastic-Scaler, Tidal Controller, ResourceScalingGroup, Eagle-Eye Hardware Monitor, Eagle-Eye Hardware Diagnosis, Eagle-Eye Network Performance Exporter).

This provides a different entry point from the one-click Helm full-stack installation via the InferNex main chart.

The scope of enhancement component capabilities is consistent with the AI Inference Integrated Deployment. For architecture and design rationale, see OFEP-0040 InferNex KServe Adaptation Layer Proposal.

Specification

InferNex Bridge targets Ascend NPU full-stack inference, supporting dual entry points via LLMInferenceService and InferNexService. For version compatibility, Webhook patch behavior, default images, and responsibility boundaries, see InferNex Bridge Technical Specification.

Version compatibility

Component	Supported versions
KServe LLMISVC controller	0.17.0, 0.18.0, 0.19.0
InferNex	26.6.0

For KServe installation prerequisites, see LLMInferenceService Prerequisites; for InferNex overall deployment specifications, see InferNex Specification #42.

Quick Start

Prerequisites

• Kubernetes cluster, kubectl, Helm v3+.
• Inference cluster: NPU Operator, LWS, etc. See InferNex User Guide — Prerequisites.
• KServe: KServe must be installed (LLMISVC controller version 0.17.0–0.19.0).
• Gateway access: Envoy Gateway, Gateway API, and GIE-related CRDs.

Binary Deployment

Using namespace infernex-bridge-system and release name infernex-bridge as an example, execute:

helm upgrade --install infernex-bridge oci://cr.openfuyao.cn/charts/infernex-bridge \
  --version 0.0.0-latest \
  -n infernex-bridge-system --create-namespace --wait --timeout 10m

Replace 0.0.0-latest with the specific Chart version.

Source Deployment

1. Clone the project from the repository.

   git clone https://gitcode.com/openFuyao/InferNex.git
   

2. Install InferNex Bridge.

   Using namespace infernex-bridge-system and release name infernex-bridge as an example, execute the following command in the InferNex/component/InferNex-Bridge directory:

   cd InferNex/component/InferNex-Bridge
   helm upgrade --install infernex-bridge ./chart/infernex-bridge \
     -n infernex-bridge-system --create-namespace --wait --timeout 10m
   

Verify Deployment

1. Confirm that the Controller Pod and Service are ready.

   kubectl get pods,svc -n infernex-bridge-system
   

2. Confirm that the Mutating / Validating Webhooks are registered.

   kubectl get mutatingwebhookconfiguration,validatingwebhookconfiguration | grep infernex-bridge
   

3. Wait for Webhooks and certificates to be ready, then deploy InferNex.

For installation parameters, uninstallation, and InferNex deployment, see the AI Inference InferNex Bridge.

Deploy InferNex

After InferNex Bridge is installed and verified, select an example YAML from config/examples/ and submit it with kubectl apply to deploy InferNex.

1. Select an Example File

   LLMISVC

   Mode	Directory	Description
   Aggregate	llmisvc/aggregate/	ag-01-*.yaml .. ag-03-*.yaml
   Disaggregated (PD)	llmisvc/disaggregated/	pd-01-*.yaml .. pd-05-*.yaml

   Examples include the huggingface-download initContainer, which pulls models from Hugging Face Hub during deployment (e.g., ag-01-single-node-single-card.yaml).

   InferNexService (inference engine templates aligned with the same Spec IDs as llmisvc)

   Mode	Directory	Description
   Aggregate	insvc/aggregate/	ag-01-*.yaml .. ag-03-*.yaml
   Disaggregated (PD)	insvc/disaggregated/	pd-01-*.yaml .. pd-05-*.yaml

   Examples include the Mooncake initContainer, huggingface-download, and complete vllm serve startup parameters (consistent with the corresponding llmisvc examples); InferNexService.spec.model + baseRefs point to the inference engine templates in InferNexServiceConfig.

   • KServe LLMInferenceService + InferNex Bridge: Select the YAML with the corresponding Spec ID under llmisvc/ (contains infernex.io/runtime: "true").
   • InferNex Bridge directly managing InferNexService: Select the YAML with the same Spec ID under insvc/.

2. Submit the YAML

   cd InferNex/component/InferNex-Bridge/config/examples/insvc/aggregate
   kubectl apply -f ag-01-single-node-single-card.yaml -n kserve
   

   Replace the path and filename with your target scenario.