phyai:基于 FlashInfer 与 Humming 的 Physical AI 服务引擎项目

---

PhyAI (pronounced "phi") is a latency-first serving engine for Physical AI. It is designed first for latency critical workloads, such as policy and action models that run in interactive systems.

News

• [2026/10] Support PhyAI gateway for multi-host serving and LeRobot clients.
• [2026/09] 🚀 Support Qwen-Image 2.1! doc.
• [2026/09] 🤗 Support PI0.5 RL rollout for PPO. RLinf integration is in progress via RLinf PR #1587.
• [2026/07] 🚀 Day 0 support for MiniCPM-Robotic blog.
• [2026/07] 👏 Introducing PhyAI, a latency-first serving engine for Physical AI. Read the Blog.
• [2026/07] Support PI0.
• [2026/07] Support Cosmos3-Super with TP and CFG parallelism in the unified Cosmos3 generation path.
• [2026/06] Support Pi0.5 and Cosmos3-Nano's policy mode and generation mode.

Key Features

• 🚀 Runs on NVIDIA Jetson edge devices
• 🚀 Scales to GPU clusters with DP, TP, and CFG parallelism
• 🚀 Uses high-performance kernels from FlashInfer and Humming
• 🤗 Supports INT4, INT8, FP8, MXFP4, and NVFP4 quantization through kernel policies (PR under review)

Supported Models

VLA	π0, π0.5(w/ DP), GR00T N1.7, MiniCPM-Robot
WAM	Cosmos3-Nano-Policy-DROID(w/ TP, CFG Parallel)
Foundation Model	Cosmos3-Nano(w/ TP, CFG Parallel), Cosmos3-Super(w/ TP, CFG Parallel), Qwen3.5, Qwen3-VL

Performance Comparison

Installation

See the PhyAI installation guide for the latest source and nightly package instructions.

From source:

git clone https://github.com/mingti-org/phyai
cd phyai
uv sync

Nightly build:

uv pip install phyai phyai-ext \
  --extra-index-url https://mingti-org.github.io/phyai/simple/ \
  --prerelease=allow

Quantization

PhyAI supports post-training quantization (PTQ) when loading an unquantized checkpoint. One kernel policy YAML controls which layers are quantized, their weight and activation precision, and the execution kernels. Humming is included in the standard installation.

For example, run PI0.5 with INT4 weights and BF16 activations (W4A16, NVIDIA SM80+) from the repository root, with your PhyAI Python environment activated:

PHYAI_KERNEL_CONFIG=examples/pi05/kernel_policies/humming_w4a16.yaml \
python examples/pi05/run_pi05.py --checkpoint /path/to/pi05_base

This preset quantizes the language model and action expert's attention/MLP linear projections with group size 128. Vision and input/output heads keep their original precision. Point --checkpoint to your original, unquantized weights.

Choose another PI0.5 preset, or copy one and edit its quantization and kernel rules. See the quantization guide for matching layers, changing precision, and selecting kernels.

Multi-host serving

You can run model servers on separate machines and connect them through phyai-gateway. On each model host, prepare pi05.yaml with that host's checkpoint, tokenizer, and robot settings. You can start from the PI0.5 example, which serves pi05 on port 50063. Check the configuration, then start the server:

phyai server pi05.yaml

On the gateway host, register both servers at startup. Replace the IP addresses below with your server hosts; pi05 must match server.model_name in their YAML files. Repeat --backend MODEL HOST:PORT to add more servers:

phyai-gateway \
  --backend pi05 10.0.0.11:50063 \
  --backend pi05 10.0.0.12:50063

In another terminal on the gateway host, check that the backends report healthy: true:

curl http://127.0.0.1:30000/v1/backends

Clients use the gateway host's HTTP port 30000 or gRPC port 50111. The gateway distributes stateless requests across healthy replicas. See the server and gateway setup guide for client configuration and adding or removing servers while the gateway is running.

Contribution Guidelines

We thank the contributors below and welcome more developers to join us in building PhyAI.

Sponsors & Adoption

PhyAI is a latency-first, open-source serving engine for Physical AI. It is being adopted by companies working across AI infrastructure and robotics, including Mingti and ModelBest.

We are actively seeking partnerships with compute providers, chip vendors, and robotics companies. If you are interested in working with us, please contact us.

Citation

If you use PhyAI in research or production work, please cite the project:

@software{phyai2026,
  title = {PhyAI: Latency-First Serving Engine for Physical AI},
  author = {{PhyAI Team}},
  year = {2026},
  url = {https://github.com/mingti-org/phyai}
}

Contact:

We welcome PhD and master's students who want to help build Physical AGI, especially those interested in systems infrastructure. We also want to work with chip and compute companies, as well as robotics companies that plan to deploy models with PhyAI.

• PhyAI: Maintainer
• Mengwei Xu: mwx@bupt.edu.cn
• Daliang Xu: xudaliang@bupt.edu.cn

License

PhyAI is released under the MIT License. It uses FlashInfer, Humming, and FLA. We have also learned a great deal from SGLang, vLLM, and TokenSpeed. We thank the maintainers and contributors of all these projects.

Demos

Cosmos3-Nano-Policy-DROID, 260718 nightly version

https://github.com/user-attachments/assets/12a833ce-3b47-4f08-875c-b30cc2567bef

https://github.com/user-attachments/assets/8e7d433c-e3b1-4388-8488-196836a1ba51

https://github.com/user-attachments/assets/29067c88-f51a-4ca6-ab8c-b45f4d778923