| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
[Bugfix] kv aware routing for lmcache 0.3.9 (#697) * [bugfix] fix kvaware routing Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> * Modify CI to be compatible Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> * Revert "fix dynamic config" This reverts commit 9ac02f992e58fa102a891d3dfe903142bf913f0b. Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> * modify CI Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> * bugfix: fix bug for kvaware routing to be compatiable with lmcache 0.3.9 Signed-off-by: Rui Zhang <rzhan229@ucsc.edu> * bugfix: fix ci Signed-off-by: Rui Zhang <rzhan229@ucsc.edu> * fix ci Signed-off-by: Rui Zhang <rzhan229@ucsc.edu> * modify ci Signed-off-by: Rui Zhang <rzhan229@ucsc.edu> * modify ci Signed-off-by: Rui Zhang <rzhan229@ucsc.edu> * modify ci Signed-off-by: Rui Zhang <rzhan229@ucsc.edu> * modify ci Signed-off-by: Rui Zhang <rzhan229@ucsc.edu> * update controller manager init Signed-off-by: Rui Zhang <rzhan229@ucsc.edu> * increase startup probe time Signed-off-by: Rui Zhang <rzhan229@ucsc.edu> * modify debug messaage Signed-off-by: Rui Zhang <rzhan229@ucsc.edu> --------- Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> Signed-off-by: Rui Zhang <rzhan229@ucsc.edu> | 9 个月前 | |
[Feat] Add Sentry Continuous Profiling Support to vLLM Router (#624) * Added profile lifecycle tracking and sampling rates for traces and sessions in Sentry initialization. Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * Added validation and command-line arguments for Sentry traces and profile session sample rates in the parser. Ensured sample rates are within the range of 0.0 to 1.0. Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * Update README.md to include new command-line options for Sentry traces and profiling session sample rates. Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * Refactor error message formatting in validate_args function for Sentry profile session sample rate validation. Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * Fix Dockerfile syntax by changing 'source' to '.' for activating the virtual environment. Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> --------- Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> | 1 年前 | |
[Feat] Use sidecar to download lora model for helm deployment (#618) * Add sidecar dockerfile Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> * modify Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> * Integrate sidecar to helm deployment Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> * modify Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> --------- Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> Co-authored-by: Shaoting <shaotingf@uchicago.edu> | 1 年前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 9 个月前 | ||
| 1 年前 | ||
| 1 年前 |