.. production-stack documentation master file, created by
sphinx-quickstart on Mon Mar 3 12:36:28 2025.
You can adapt this file completely to your liking, but it should at least
contain the root toctree directive.
.. role:: raw-html(raw) :format: html
Welcome to production-stack!
.. figure:: ./assets/prodstack.png :width: 60% :align: center :alt: production-stack :class: no-scaled-link
.. raw:: html
K8S-native cluster-wide deployment for vLLM.
.. raw:: html
vLLM Production Stack project provides a reference implementation on how to build an inference stack on top of vLLM, which allows you to:
- 🚀 Scale from single vLLM instance to distributed vLLM deployment without changing any application code
- 💻 Monitor the metrics through a web dashboard
- 😄 Enjoy the performance benefits brought by request routing and KV cache offloading
- 📈 Easily deploy the stack on AWS, GCP, or any other cloud provider
Documentation
.. Add your content using reStructuredText syntax. See the
.. reStructuredText
.. documentation for details.
.. toctree:: :maxdepth: 2 :caption: Getting Started
getting_started/prerequisite getting_started/quickstart getting_started/faq
.. toctree:: :maxdepth: 2 :caption: Deployment
deployment/index
.. toctree:: :maxdepth: 2 :caption: Use Cases
use_cases/kv-cache-aware-routing use_cases/prefix-aware-routing use_cases/disaggregated-prefill use_cases/sharing-kv-cache use_cases/benchmarking use_cases/distributed-tracing use_cases/tool-enabled-installation use_cases/pipeline-parallelism-kuberay use_cases/sleep-wakeup-mode use_cases/autoscaling-keda use_cases/semantic-router-integration
.. toctree:: :maxdepth: 2 :caption: Developer Guide
developer_guide/contributing developer_guide/docker
.. toctree:: :maxdepth: 2 :caption: Community
community/meetings