| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
[FEAT] add support for customiziable runtimeClass & podAnnotaions (#958) * [FEAT] Add runtimeClass for vllmRuntime Signed-off-by: Mahmoud Ayman <mahmoudk1000@gmail.com> * [FEAT] Add podAnnotaions for vllmRuntime Signed-off-by: Mahmoud Ayman <mahmoudk1000@gmail.com> --------- Signed-off-by: Mahmoud Ayman <mahmoudk1000@gmail.com> | 3 个月前 | |
[Feat][lora] add lora operator and modify vllm router to support (#446) * [FEAT][lora] lora controller with create and delete Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> * [FEAT][lora] add pod watch Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> * [Feat][lora] add router api support Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> * pre-commit Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> * fix conflixt Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> * fix lora for static discovery and pd Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> * add tutorial Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> --------- Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> Co-authored-by: Rui Zhang <zrfishnoodles@gmail.com> | 1 年前 | |
[FEAT] add support for customiziable runtimeClass & podAnnotaions (#958) * [FEAT] Add runtimeClass for vllmRuntime Signed-off-by: Mahmoud Ayman <mahmoudk1000@gmail.com> * [FEAT] Add podAnnotaions for vllmRuntime Signed-off-by: Mahmoud Ayman <mahmoudk1000@gmail.com> --------- Signed-off-by: Mahmoud Ayman <mahmoudk1000@gmail.com> | 3 个月前 | |
[Feat] Add initial CRD support for production stack (#415) * add CRD support for production stack Signed-off-by: royyhuang <roy.y.huang@gmail.com> * move opertor to a secondary dir instead of in root dir Signed-off-by: royyhuang <roy.y.huang@gmail.com> * rename api group from serving.vllm.ai to production-stack.vllm.ai Signed-off-by: royyhuang <roy.y.huang@gmail.com> * enable lmcache cpu offloading Signed-off-by: royyhuang <roy.y.huang@gmail.com> * enable lmcache remote cache server offloading Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix service discorvery issue by adding readiness probe to vllm pod Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix readiness probe Signed-off-by: royyhuang <roy.y.huang@gmail.com> --------- Signed-off-by: royyhuang <roy.y.huang@gmail.com> | 1 年前 | |
[FEAT] add support for customiziable runtimeClass & podAnnotaions (#958) * [FEAT] Add runtimeClass for vllmRuntime Signed-off-by: Mahmoud Ayman <mahmoudk1000@gmail.com> * [FEAT] Add podAnnotaions for vllmRuntime Signed-off-by: Mahmoud Ayman <mahmoudk1000@gmail.com> --------- Signed-off-by: Mahmoud Ayman <mahmoudk1000@gmail.com> | 3 个月前 | |
[Feat] Add initial CRD support for production stack (#415) * add CRD support for production stack Signed-off-by: royyhuang <roy.y.huang@gmail.com> * move opertor to a secondary dir instead of in root dir Signed-off-by: royyhuang <roy.y.huang@gmail.com> * rename api group from serving.vllm.ai to production-stack.vllm.ai Signed-off-by: royyhuang <roy.y.huang@gmail.com> * enable lmcache cpu offloading Signed-off-by: royyhuang <roy.y.huang@gmail.com> * enable lmcache remote cache server offloading Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix service discorvery issue by adding readiness probe to vllm pod Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix readiness probe Signed-off-by: royyhuang <roy.y.huang@gmail.com> --------- Signed-off-by: royyhuang <roy.y.huang@gmail.com> | 1 年前 | |
[Feat] Add initial CRD support for production stack (#415) * add CRD support for production stack Signed-off-by: royyhuang <roy.y.huang@gmail.com> * move opertor to a secondary dir instead of in root dir Signed-off-by: royyhuang <roy.y.huang@gmail.com> * rename api group from serving.vllm.ai to production-stack.vllm.ai Signed-off-by: royyhuang <roy.y.huang@gmail.com> * enable lmcache cpu offloading Signed-off-by: royyhuang <roy.y.huang@gmail.com> * enable lmcache remote cache server offloading Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix service discorvery issue by adding readiness probe to vllm pod Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix readiness probe Signed-off-by: royyhuang <roy.y.huang@gmail.com> --------- Signed-off-by: royyhuang <roy.y.huang@gmail.com> | 1 年前 | |
[Feat] Use sidecar to download lora model for helm deployment (#618) * Add sidecar dockerfile Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> * modify Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> * Integrate sidecar to helm deployment Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> * modify Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> --------- Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> Co-authored-by: Shaoting <shaotingf@uchicago.edu> | 1 年前 | |
[Feat] Add initial CRD support for production stack (#415) * add CRD support for production stack Signed-off-by: royyhuang <roy.y.huang@gmail.com> * move opertor to a secondary dir instead of in root dir Signed-off-by: royyhuang <roy.y.huang@gmail.com> * rename api group from serving.vllm.ai to production-stack.vllm.ai Signed-off-by: royyhuang <roy.y.huang@gmail.com> * enable lmcache cpu offloading Signed-off-by: royyhuang <roy.y.huang@gmail.com> * enable lmcache remote cache server offloading Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix service discorvery issue by adding readiness probe to vllm pod Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix readiness probe Signed-off-by: royyhuang <roy.y.huang@gmail.com> --------- Signed-off-by: royyhuang <roy.y.huang@gmail.com> | 1 年前 | |
[Misc] Refract the structure of VLLMRuntime CRD (#479) * add CRD support for production stack Signed-off-by: royyhuang <roy.y.huang@gmail.com> * move opertor to a secondary dir instead of in root dir Signed-off-by: royyhuang <roy.y.huang@gmail.com> * rename api group from serving.vllm.ai to production-stack.vllm.ai Signed-off-by: royyhuang <roy.y.huang@gmail.com> * enable lmcache cpu offloading Signed-off-by: royyhuang <roy.y.huang@gmail.com> * enable lmcache remote cache server offloading Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix service discorvery issue by adding readiness probe to vllm pod Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix readiness probe Signed-off-by: royyhuang <roy.y.huang@gmail.com> * restructure runtime crd spec Signed-off-by: royyhuang <roy.y.huang@gmail.com> * add default operator manifest Signed-off-by: royyhuang <roy.y.huang@gmail.com> * move rbac for vllm-router pod to controller Signed-off-by: royyhuang <roy.y.huang@gmail.com> * use service account name defined in vllm-router cr to create service account instead of hardcoding Signed-off-by: royyhuang <roy.y.huang@gmail.com> * update sample loraadapter manifest Signed-off-by: royyhuang <roy.y.huang@gmail.com> * update default controller image Signed-off-by: royyhuang <roy.y.huang@gmail.com> * remove old image name patch Signed-off-by: royyhuang <roy.y.huang@gmail.com> --------- Signed-off-by: royyhuang <roy.y.huang@gmail.com> | 1 年前 | |
[Feat] Add initial CRD support for production stack (#415) * add CRD support for production stack Signed-off-by: royyhuang <roy.y.huang@gmail.com> * move opertor to a secondary dir instead of in root dir Signed-off-by: royyhuang <roy.y.huang@gmail.com> * rename api group from serving.vllm.ai to production-stack.vllm.ai Signed-off-by: royyhuang <roy.y.huang@gmail.com> * enable lmcache cpu offloading Signed-off-by: royyhuang <roy.y.huang@gmail.com> * enable lmcache remote cache server offloading Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix service discorvery issue by adding readiness probe to vllm pod Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix readiness probe Signed-off-by: royyhuang <roy.y.huang@gmail.com> --------- Signed-off-by: royyhuang <roy.y.huang@gmail.com> | 1 年前 | |
[Feat] Add initial CRD support for production stack (#415) * add CRD support for production stack Signed-off-by: royyhuang <roy.y.huang@gmail.com> * move opertor to a secondary dir instead of in root dir Signed-off-by: royyhuang <roy.y.huang@gmail.com> * rename api group from serving.vllm.ai to production-stack.vllm.ai Signed-off-by: royyhuang <roy.y.huang@gmail.com> * enable lmcache cpu offloading Signed-off-by: royyhuang <roy.y.huang@gmail.com> * enable lmcache remote cache server offloading Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix service discorvery issue by adding readiness probe to vllm pod Signed-off-by: royyhuang <roy.y.huang@gmail.com> * fix readiness probe Signed-off-by: royyhuang <roy.y.huang@gmail.com> --------- Signed-off-by: royyhuang <roy.y.huang@gmail.com> | 1 年前 |