| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
[Doc] Add docs for router CRD config and dev, some small tweaks (#259) * Add docs for router CDR config, some small tweaks Signed-off-by: Siddhant Ray <siddhant.r98@gmail.com> * add initial dev guide Signed-off-by: Siddhant Ray <siddhant.r98@gmail.com> * add dev docs Signed-off-by: Siddhant Ray <siddhant.r98@gmail.com> --------- Signed-off-by: Siddhant Ray <siddhant.r98@gmail.com> | 1 年前 | |
Update calendar link for community meetings (#783) * Update calendar link for community meetings Signed-off-by: Rui Zhang <51696593+zerofishnoodles@users.noreply.github.com> * Update calendar link for community event Signed-off-by: Rui Zhang <51696593+zerofishnoodles@users.noreply.github.com> * Update calendar link for community meetings Signed-off-by: Rui Zhang <51696593+zerofishnoodles@users.noreply.github.com> --------- Signed-off-by: Rui Zhang <51696593+zerofishnoodles@users.noreply.github.com> | 7 个月前 | |
[bugfix] deprecate disable log request (#885) * bugfix: deprecate disable log request Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> * Update helm/README.md Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: Rui Zhang <51696593+ruizhang0101@users.noreply.github.com> --------- Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> Signed-off-by: Rui Zhang <51696593+ruizhang0101@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> | 6 个月前 | |
Update contact information in README.md (#821) * Update contact information in README.md Signed-off-by: Rui Zhang <51696593+ruizhang0101@users.noreply.github.com> * Update reviewer mentions in contributing guide Signed-off-by: Rui Zhang <51696593+ruizhang0101@users.noreply.github.com> * Update reviewer mentions in CONTRIBUTING.md Signed-off-by: Rui Zhang <51696593+ruizhang0101@users.noreply.github.com> --------- Signed-off-by: Rui Zhang <51696593+ruizhang0101@users.noreply.github.com> | 7 个月前 | |
[refactor] redesign RST documentation (#592) * initial reorg Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * pass CICD Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * fix Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * fix Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * fix: update helm.rst Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * fix Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * fix Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * Sharing KV Cache Across Instances Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * docker Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * community meeting Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * add deployment 3 options Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * remove A: Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * docker Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> --------- Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> Co-authored-by: Yuhan Liu <32589867+YuhanLiu11@users.noreply.github.com> | 1 年前 | |
[helm] document every values, udpate json schema and various fix (#886) * fix(helm) update sharedPvcStorage so it correctly create PVC and/or PV Signed-off-by: enneitex <etienne.divet@gmail.com> * fix(helm) various fix to align values, templates and README. update json schema Signed-off-by: enneitex <etienne.divet@gmail.com> * fix(helm) use correct type for lmcacheControllerPort Signed-off-by: enneitex <etienne.divet@gmail.com> * fix(helm) rayCluster: fix templating and add tests Signed-off-by: enneitex <etienne.divet@gmail.com> * fix(helm) deployment-router: simplify templating and add tests Signed-off-by: enneitex <etienne.divet@gmail.com> * feat(helm) improve schema, add tests and CI validation Signed-off-by: enneitex <etienne.divet@gmail.com> feat(helm) improve testing around rayCluster and engine deployment Signed-off-by: enneitex <etienne.divet@gmail.com> ci: retrigger Signed-off-by: enneitex <etienne.divet@gmail.com> fix(tutorials) rename ScaledObject file name Signed-off-by: enneitex <etienne.divet@gmail.com> fix pre-commit Signed-off-by: enneitex <etienne.divet@gmail.com> --------- Signed-off-by: enneitex <etienne.divet@gmail.com> Co-authored-by: Rui Zhang <51696593+ruizhang0101@users.noreply.github.com> | 5 个月前 | |
[Router] Replace httpx with aiohttp in vllm_router for enhanced high-concurrency performance (#589) * feat: update requirements to include aiohttp Signed-off-by: Ifta khairul Alam Adil <ikaadil007@gmail.com> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * feat: replace HTTPX with aiohttp for asynchronous client handling Signed-off-by: Ifta khairul Alam Adil <ikaadil007@gmail.com> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * feat: replace HTTPXClientWrapper with AiohttpClientWrapper for improved async handling Signed-off-by: Ifta khairul Alam Adil <ikaadil007@gmail.com> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * feat: replace HTTPXClientWrapper with AiohttpClientWrapper in lifespan management Signed-off-by: Ifta khairul Alam Adil <ikaadil007@gmail.com> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * feat: remove httpx from requirements for aiohttp integration Signed-off-by: Ifta khairul Alam Adil <ikaadil007@gmail.com> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * refactor: switch from httpx to aiohttp for asynchronous client handling in service discovery Signed-off-by: Ifta khairul Alam Adil <ikaadil007@gmail.com> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * feat: add AiohttpClientWrapper for asynchronous HTTP client handling Signed-off-by: Ifta khairul Alam Adil <ikaadil007@gmail.com> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * chore: pin aiohttp version to 3.9.5 in requirements Signed-off-by: Ifta khairul Alam Adil <ikaadil007@gmail.com> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * refactor: migrate from httpx to aiohttp for asynchronous HTTP requests in request service Signed-off-by: Ifta khairul Alam Adil <ikaadil007@gmail.com> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * refactor: update file upload implementation to use aiohttp for asynchronous requests Signed-off-by: Ifta khairul Alam Adil <ikaadil007@gmail.com> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * chore: replace httpx with aiohttp in project dependencies and test requirements Signed-off-by: Ifta khairul Alam Adil <ikaadil007@gmail.com> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * chore: format dependencies in pyproject.toml for consistency Signed-off-by: Ifta khairul Alam Adil <ikaadil007@gmail.com> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * chore: clean up imports and ensure consistent aiohttp version in requirements Signed-off-by: Ifta khairul Alam Adil <ikaadil007@gmail.com> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * fix: update aiohttp client closure method for consistency and accuracy in logging Signed-off-by: Ifta khairul Alam Adil <ikaadil007@gmail.com> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * fix: correct status code handling in request processing for improved response accuracy Signed-off-by: Ifta khairul Alam Adil <ikaadil007@gmail.com> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * bumping version (#593) Signed-off-by: YuhanLiu11 <yliu738@wisc.edu> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * Added option to specify priority class (#557) Signed-off-by: Fabijan Marič Vild <fm6209@student.uni-lj.si> Co-authored-by: Yuhan Liu <32589867+YuhanLiu11@users.noreply.github.com> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * [CI/Build] Change CI runner to L4 (#595) * Add env clean up before run Signed-off-by: Shaoting Feng <shaotingf@uchicago.edu> * Delete all pods Signed-off-by: Shaoting Feng <shaotingf@uchicago.edu> * Delete all release Signed-off-by: Shaoting Feng <shaotingf@uchicago.edu> * test ci runner Signed-off-by: Shaoting <shaotingf@uchicago.edu> * test ci runner Signed-off-by: Shaoting <shaotingf@uchicago.edu> * Test again Signed-off-by: Shaoting Feng <shaotingf@uchicago.edu> * Test again Signed-off-by: Shaoting Feng <shaotingf@uchicago.edu> * Change python version from 3.12 to 3.10 Signed-off-by: Shaoting Feng <shaotingf@uchicago.edu> * delete get helm Signed-off-by: Shaoting Feng <shaotingf@uchicago.edu> * change back python version Signed-off-by: Shaoting Feng <shaotingf@uchicago.edu> * trigger Signed-off-by: Shaoting Feng <shaotingf@uchicago.edu> * Add cache: pip Signed-off-by: Shaoting Feng <shaotingf@uchicago.edu> * Try check-latest: true Signed-off-by: Shaoting Feng <shaotingf@uchicago.edu> * Changing to 3.10 Signed-off-by: Shaoting Feng <shaotingf@uchicago.edu> * change to 3.9 Signed-off-by: Shaoting Feng <shaotingf@uchicago.edu> * Specify 3.9.2 Signed-off-by: Shaoting Feng <shaotingf@uchicago.edu> * Use conda python Signed-off-by: Shaoting Feng <shaotingf@uchicago.edu> * Fix vllm without conda Signed-off-by: Shaoting Feng <shaotingf@uchicago.edu> --------- Signed-off-by: Shaoting Feng <shaotingf@uchicago.edu> Signed-off-by: Shaoting <shaotingf@uchicago.edu> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * [Bugfix] fix dynamic config (#598) * [hotfix] fix dynamic config Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> * [Bugfix] Add missing field for reconfiguration Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> --------- Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * [refactor] redesign RST documentation (#592) * initial reorg Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * pass CICD Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * fix Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * fix Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * fix: update helm.rst Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * fix Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * fix Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * Sharing KV Cache Across Instances Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * docker Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * community meeting Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * add deployment 3 options Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * remove A: Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> * docker Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> --------- Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> Co-authored-by: Yuhan Liu <32589867+YuhanLiu11@users.noreply.github.com> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * [Misc] revert uv.lock (#604) Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * fix: update response content iteration method for improved performance Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * Replace httpx with aiohttp for prefill and decode clients in K8sServiceNameServiceDiscovery, updating timeout handling accordingly. Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * Update file upload function to include file parameter in aiohttp post request Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * Refactor response handling in request service to return JSON data and correctly set response status code in sleep/wakeup routing. Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * Add initialization of aiohttp ClientSession in service discovery during app startup - Implemented method in , , and classes to set up aiohttp ClientSession for prefill and decode endpoints. - Updated function in to call if available, ensuring proper session management during application startup. - Adjusted logging statements in to reference the attribute of the client sessions for accurate logging of routing requests. Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> * Refactor whitespace and formatting in app.py for improved readability Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> --------- Signed-off-by: Ifta khairul Alam Adil <ikaadil007@gmail.com> Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com> Signed-off-by: YuhanLiu11 <yliu738@wisc.edu> Signed-off-by: Fabijan Marič Vild <fm6209@student.uni-lj.si> Signed-off-by: Shaoting Feng <shaotingf@uchicago.edu> Signed-off-by: Shaoting <shaotingf@uchicago.edu> Signed-off-by: Rui Zhang <zrfishnoodles@gmail.com> Signed-off-by: Kobe Chen <xiaokunchen0@gmail.com> Co-authored-by: Yuhan Liu <32589867+YuhanLiu11@users.noreply.github.com> Co-authored-by: Fabhiahn <79161847+Fabhiahn@users.noreply.github.com> Co-authored-by: Shaoting <shaotingf@uchicago.edu> Co-authored-by: Rui Zhang <51696593+zerofishnoodles@users.noreply.github.com> Co-authored-by: Kobe Chen <xiaokunchen0@gmail.com> | 1 年前 | |
[Integration]: Add Intelligent Semantic Routing with vLLM-SR (#750) * [Integration]: Add Intelligent Semantic Routing with vLLM-SR Signed-off-by: bitliu <bitliu@tencent.com> * lint Signed-off-by: bitliu <bitliu@tencent.com> * lint Signed-off-by: bitliu <bitliu@tencent.com> --------- Signed-off-by: bitliu <bitliu@tencent.com> | 10 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 1 年前 | ||
| 7 个月前 | ||
| 6 个月前 | ||
| 7 个月前 | ||
| 1 年前 | ||
| 5 个月前 | ||
| 1 年前 | ||
| 10 个月前 |