vllm:基于 PyTorch 生态的 LLM 推理与服务库项目

Star0

Star0

A high-throughput and memory-efficient inference and serving engine for LLMs

项目介绍

A high-throughput and memory-efficient inference and serving engine for LLMs

Apache-2.0 Python 3.93 K提交数

定制我的领域