| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
[serve] rename max concurrent queries (#43007) Rename max_concurrent_queries to max_ongoing_requests. All tests have been migrated to use max_ongoing_requests except for a few tests that are parametrized to ensure max_concurrent_queries still works: * test_max_ongoing_requests_set_to_one (e2e test) * test_update_config_max_ongoing_requests (e2e test) * test_valid_deployment_schema (unit test) **Rollout plan**: support both and deprecate max_concurrent_queries for 2.10. Remove max_concurrent_queries next release. Signed-off-by: Cindy Zhang <cindyzyx9@gmail.com> | 2 年前 | |
[docs][KubeRay] Update KubeRay doc for release v0.5.0 (#34178) index.md: No code here to test and verify. getting-started.ipynb: Test manually. user-guides.md: No code here to test and verify k8s-cluster-setup.md: No code here to test and verify config.md: No code here to test and verify configuring-autoscaling.md: Test manually. logging.md: Test manually. gpu.rst: I did not verify code snippets, but GPU usage will be verified in gpu-training-example.md. experimental.md: No code here to test and verify static-ray-cluster-without-kuberay.md: Skip this. This document has no relationship with KubeRay. examples.md ml-example.md: (Will update in [docs][KubeRay] Provide some GKE instructions in KubeRay example #33339) gpu-training-example.md (Will update in [docs][KubeRay] Provide some GKE instructions in KubeRay example #33339) references.md Ray Serve kubernetes.md: Test manually. fault-tolerance.md: I do not test all serve's recovery procedures. I make sure the RayService can be created as expected. helm repo add kuberay https://ray-project.github.io/kuberay-helm/ helm install kuberay-operator kuberay/kuberay-operator --version 0.5.0 # path: doc/ kubectl apply -f source/serve/doc_code/fault_tolerance/k8s_config.yaml # port forward kubectl port-forward service/rayservice-sample-serve-svc 8000 # Test the serve deployment curl localhost:8000 # Delete a worker Pod kubectl delete pod ${WORKER_POD} # Test the serve deployment again curl localhost:8000 run_gcs_ft_on_k8s.py | 3 年前 | |
[serve] Flip default for RAY_SERVE_ENABLE_NEW_HANDLE_API to 1 (#40625) Changes the default for the handle API to use DeploymentHandle. This will be a breaking API change. A warning was logged to inform users since Ray 2.7 (in addition to a warning on the documentation) and this will land in Ray 2.9. Users can continue to use the existing handle API by setting the environment variable RAY_SERVE_ENABLE_NEW_HANDLE_API=0 or using handle.options(use_new_handle_api=False). Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com> | 2 年前 | |
[Serve] Add gRPC context related docs (#41783) Adding example code and descriptions for using gRPC context in Serve. --------- Signed-off-by: Gene Su <e870252314@gmail.com> Signed-off-by: Gene Der Su <gdsu@ucdavis.edu> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> | 2 年前 | |
| 1 年前 | ||
Fix typos in doc directory (#51054) Signed-off-by: co63oc <co63oc@users.noreply.github.com> | 1 年前 | |
[Serve] update serve logging config docs (#43483) - Updated code examples to be complete with imports and requests so it works out of box - Added info on RAY_SERVE_ENABLE_JSON_LOGGING getting deprecated in the future release and using RAY_SERVE_LOG_ENCODING if need a global config - Run through all examples and update the existing logs --------- Signed-off-by: Gene Su <e870252314@gmail.com> Signed-off-by: Gene Der Su <gdsu@ucdavis.edu> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> | 2 年前 | |
[serve] Fix broken docs tests (#41023) Signed-off-by: Cindy Zhang <cindyzyx9@gmail.com> | 2 年前 | |
[serve] Fix app_builder doc code test (#35456) Missing import. | 3 年前 | |
[serve] Flip default for RAY_SERVE_ENABLE_NEW_HANDLE_API to 1 (#40625) Changes the default for the handle API to use DeploymentHandle. This will be a breaking API change. A warning was logged to inform users since Ray 2.7 (in addition to a warning on the documentation) and this will land in Ray 2.9. Users can continue to use the existing handle API by setting the environment variable RAY_SERVE_ENABLE_NEW_HANDLE_API=0 or using handle.options(use_new_handle_api=False). Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com> | 2 年前 | |
[serve] Flip default for RAY_SERVE_ENABLE_NEW_HANDLE_API to 1 (#40625) Changes the default for the handle API to use DeploymentHandle. This will be a breaking API change. A warning was logged to inform users since Ray 2.7 (in addition to a warning on the documentation) and this will land in Ray 2.9. Users can continue to use the existing handle API by setting the environment variable RAY_SERVE_ENABLE_NEW_HANDLE_API=0 or using handle.options(use_new_handle_api=False). Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com> | 2 年前 | |
[serve] Add inference serve example to run Stable Diffusion Inference using AWS Inferentia2 (#43046) This example showcases how to serve Stable Diffusion Inference using AWS Inferentia2 | 2 年前 | |
[serve] Flip default for RAY_SERVE_ENABLE_NEW_HANDLE_API to 1 (#40625) Changes the default for the handle API to use DeploymentHandle. This will be a breaking API change. A warning was logged to inform users since Ray 2.7 (in addition to a warning on the documentation) and this will land in Ray 2.9. Users can continue to use the existing handle API by setting the environment variable RAY_SERVE_ENABLE_NEW_HANDLE_API=0 or using handle.options(use_new_handle_api=False). Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com> | 2 年前 | |
[Serve][Doc] Add custom request router docs (#53511) Signed-off-by: Gene Su <e870252314@gmail.com> | 1 年前 | |
[Serve] Add RouterConfig field to DeploymentConfig to configure RequestRouter (#53870) Signed-off-by: Seiji Eicher <seiji@anyscale.com> | 1 年前 | |
[Serve] [Docs] Update serve status and serve.status() in docs (#39132) serve status now reports information about proxies and replica states. This change updates the serve status outputs in the documentation to match the current behavior. Additionally, this change documents serve.status(). Co-authored-by: Cindy Zhang <cindyzyx9@gmail.com> | 2 年前 | |
[serve][docs] Remove unused files in doc_code/ (#39326) | 2 年前 | |
[serve] Flip default for RAY_SERVE_ENABLE_NEW_HANDLE_API to 1 (#40625) Changes the default for the handle API to use DeploymentHandle. This will be a breaking API change. A warning was logged to inform users since Ray 2.7 (in addition to a warning on the documentation) and this will land in Ray 2.9. Users can continue to use the existing handle API by setting the environment variable RAY_SERVE_ENABLE_NEW_HANDLE_API=0 or using handle.options(use_new_handle_api=False). Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com> | 2 年前 | |
[serve] Fix broken docs tests (#41023) Signed-off-by: Cindy Zhang <cindyzyx9@gmail.com> | 2 年前 | |
[doc][kuberay] state rayStartParams is optional starting with KubeRay 1.4.0 (#53943) but that if autoscaling is used, the autoscaler image must have Ray 2.45.0 or later. closes https://github.com/ray-project/kuberay/issues/3580 ## Checks - [x] I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR. - [x] I've run scripts/format.sh to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in doc/source/tune/api/ under the corresponding .rst file. - [x] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [x] Unit tests - [ ] Release tests - [ ] This PR is not tested :( Signed-off-by: David Xia <david@davidxia.com> Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com> Co-authored-by: Dhyey Shah <dhyey2019@gmail.com> Co-authored-by: Kai-Hsun Chen <kaihsun@anyscale.com> | 1 年前 | |
[Serve] [Docs] Document how to build and deploy custom Docker images (#37370) In production, Serve users should put their Serve application code in a custom Docker image rather than a runtime_env. This change documents how to create custom Docker images and add them to a KubeRay config. Link to docs: https://anyscale-ray--37370.com.readthedocs.build/en/37370/serve/production-guide/docker.html Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com> | 3 年前 | |
[serve] Outdated API cleanup in docs (#40404) Clean up: - (removed) v1 api doc code - (Deprecated) using route prefix in serve deployment decorator | 2 年前 | |
[serve] Flip default for RAY_SERVE_ENABLE_NEW_HANDLE_API to 1 (#40625) Changes the default for the handle API to use DeploymentHandle. This will be a breaking API change. A warning was logged to inform users since Ray 2.7 (in addition to a warning on the documentation) and this will land in Ray 2.9. Users can continue to use the existing handle API by setting the environment variable RAY_SERVE_ENABLE_NEW_HANDLE_API=0 or using handle.options(use_new_handle_api=False). Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com> | 2 年前 | |
[serve] Gradio docs fixes (#32754) * The gradio example in the docs uses deprecated API, which spews deprecated warnings if a user runs it. * I updated the gradio version in the docs from 3.11 to 3.19, as I've verified the example runs fine with the most recent gradio version. * I switched out EleutherAI/gpt-neo-125M for distilgpt2 in the docs example because of weird errors that occur when pulling it from hugging face. (not sure why the errors are happening, it's not related to gradio or serve) Signed-off-by: Cindy Zhang <cindyzyx9@gmail.com> | 3 年前 | |
[serve] Gradio docs fixes (#32754) * The gradio example in the docs uses deprecated API, which spews deprecated warnings if a user runs it. * I updated the gradio version in the docs from 3.11 to 3.19, as I've verified the example runs fine with the most recent gradio version. * I switched out EleutherAI/gpt-neo-125M for distilgpt2 in the docs example because of weird errors that occur when pulling it from hugging face. (not sure why the errors are happening, it's not related to gradio or serve) Signed-off-by: Cindy Zhang <cindyzyx9@gmail.com> | 3 年前 | |
Revert "Revert "[serve] Remove ability to specify route_prefix at deployment level" (#48292)" (#48294) This reverts commit 703194a9a01969593772633eaed6c548061bccea. The original commit caused test_deploy_2.py to become flaky. It appears this was a timing issue due to the 10s timeout set on the test. I've removed this, as it looks like a very old test case that is no longer relevant. --------- Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com> | 1 年前 | |
Update Intel Gaudi documents (#44557) Signed-off-by: Jerome <janand@habana.ai> | 2 年前 | |
[Docs] update all hpu related docs (#51028) This is just update documents of Intel Gaudi framework from 1.14/1.15 to 1.20. Almost all examples python scripts can re-use just updating the docker image from 1.14/1.15 to 1.20. I also update the outputs of each examples. Signed-off-by: harborn <gangshengwu@habana.ai> | 1 年前 | |
[Docs] update all hpu related docs (#51028) This is just update documents of Intel Gaudi framework from 1.14/1.15 to 1.20. Almost all examples python scripts can re-use just updating the docker image from 1.14/1.15 to 1.20. I also update the outputs of each examples. Signed-off-by: harborn <gangshengwu@habana.ai> | 1 年前 | |
[serve] Flip default for RAY_SERVE_ENABLE_NEW_HANDLE_API to 1 (#40625) Changes the default for the handle API to use DeploymentHandle. This will be a breaking API change. A warning was logged to inform users since Ray 2.7 (in addition to a warning on the documentation) and this will land in Ray 2.9. Users can continue to use the existing handle API by setting the environment variable RAY_SERVE_ENABLE_NEW_HANDLE_API=0 or using handle.options(use_new_handle_api=False). Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com> | 2 年前 | |
[CI][Linter] Replace flake8 and isort with ruff (#49435) and use ruff inside pre-commit. | 1 年前 | |
Add docs for local_testing_mode (#48567) This PR adds docs to the dev-workflow.md complementing the changes in this PR: https://github.com/ray-project/ray/pull/48477 --------- Signed-off-by: Promptless Bot <hi@gopromptless.ai> Signed-off-by: Frances Liu <francestfls@gmail.com> Signed-off-by: frances720 <francestfls@gmail.com> Co-authored-by: promptless[bot] <179508745+promptless[bot]@users.noreply.github.com> Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com> | 1 年前 | |
[serve] rename target num ongoing requests per replica (#43454) Rename target_num_ongoing_requests_per_replica -> target_ongoing_requests. Signed-off-by: Cindy Zhang <cindyzyx9@gmail.com> | 2 年前 | |
[serve] Flip default for RAY_SERVE_ENABLE_NEW_HANDLE_API to 1 (#40625) Changes the default for the handle API to use DeploymentHandle. This will be a breaking API change. A warning was logged to inform users since Ray 2.7 (in addition to a warning on the documentation) and this will land in Ray 2.9. Users can continue to use the existing handle API by setting the environment variable RAY_SERVE_ENABLE_NEW_HANDLE_API=0 or using handle.options(use_new_handle_api=False). Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com> | 2 年前 | |
[serve] Fully remove old handle API (#42526) Fully remove the previously deprecated RayServeHandle and RayServeSyncHandle APIs. --------- Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com> | 2 年前 | |
[serve] Outdated API cleanup in docs (#40404) Clean up: - (removed) v1 api doc code - (Deprecated) using route prefix in serve deployment decorator | 2 年前 | |
[serve] Flip default for RAY_SERVE_ENABLE_NEW_HANDLE_API to 1 (#40625) Changes the default for the handle API to use DeploymentHandle. This will be a breaking API change. A warning was logged to inform users since Ray 2.7 (in addition to a warning on the documentation) and this will land in Ray 2.9. Users can continue to use the existing handle API by setting the environment variable RAY_SERVE_ENABLE_NEW_HANDLE_API=0 or using handle.options(use_new_handle_api=False). Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com> | 2 年前 | |
[Serve] [Docs] Document best practices for querying Serve over HTTP (#37160) This change adds best practices for querying Serve applications over HTTP in production. Related issue number Closes #36337. --------- Signed-off-by: Shreyas Krishnaswamy <shrekris@anyscale.com> Signed-off-by: shrekris-anyscale <92341594+shrekris-anyscale@users.noreply.github.com> Co-authored-by: Cindy Zhang <cindyzyx9@gmail.com> Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com> | 3 年前 | |
[serve] document auto num replicas (#43926) Document setting num_replicas="auto". Change example in basic autoscaling guide to use num_replicas=auto. Signed-off-by: Cindy Zhang <cindyzyx9@gmail.com> | 2 年前 | |
[serve] Outdated API cleanup in docs (#40404) Clean up: - (removed) v1 api doc code - (Deprecated) using route prefix in serve deployment decorator | 2 年前 | |
[serve][docs] Re-enable stable diffusion test and fix its deps (#52960) ## Why are these changes needed? This PR updates the dependencies needed for stable_diffusion.py test. Since the deps are fixed, we can re-enable the test. Signed-off-by: akyang-anyscale <alexyang@anyscale.com> | 1 年前 | |
[Serve] [Docs] Add end-to-end documentation for streaming (#36961) Serve has recently added streaming and WebSocket support. This change adds end-to-end examples to guide users through these features. Link to documentation: https://anyscale-ray--36961.com.readthedocs.build/en/36961/serve/tutorials/streaming.html Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> Co-authored-by: angelinalg <122562471+angelinalg@users.noreply.github.com> | 3 年前 | |
[Serve][Doc] Direct Ingress documentation (#29149) | 3 年前 | |
[Serve][Doc] Direct Ingress documentation (#29149) | 3 年前 | |
[serve] Outdated API cleanup in docs (#40404) Clean up: - (removed) v1 api doc code - (Deprecated) using route prefix in serve deployment decorator | 2 年前 | |
Revert "Revert "[serve] Remove ability to specify route_prefix at deployment level" (#48292)" (#48294) This reverts commit 703194a9a01969593772633eaed6c548061bccea. The original commit caused test_deploy_2.py to become flaky. It appears this was a timing issue due to the 10s timeout set on the test. I've removed this, as it looks like a very old test case that is no longer relevant. --------- Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com> | 1 年前 | |
[Serve][Doc] Fix batching tutorial code (#29467) | 3 年前 | |
[serve][doc] Update Serve API in tutorials code (#27579) | 3 年前 | |
[CI] Fix: flake8 rule B108: Found useless Tuple expression. Consider either assigning it to a variable or removing it. (#48118) ## Why are these changes needed? While running the pre-commit hook of flake8, the following error occurs if Python version is 3.12. It's because the version of flake8 is too old.  version: - python: 3.12.7 - flake8: 7.1.1 - flake8-bugbear: 24.8.19 <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number Closes #48065 <!-- For example: "Closes #1234" --> ## Checks - [x] I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR. - [x] I've run scripts/format.sh to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in doc/source/tune/api/ under the corresponding .rst file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :( Signed-off-by: win5923 <ken89@kimo.com> | 1 年前 | |
[serve][doc] Update Serve API in tutorials code (#27579) | 3 年前 | |
[serve] Flip default for RAY_SERVE_ENABLE_NEW_HANDLE_API to 1 (#40625) Changes the default for the handle API to use DeploymentHandle. This will be a breaking API change. A warning was logged to inform users since Ray 2.7 (in addition to a warning on the documentation) and this will land in Ray 2.9. Users can continue to use the existing handle API by setting the environment variable RAY_SERVE_ENABLE_NEW_HANDLE_API=0 or using handle.options(use_new_handle_api=False). Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com> | 2 年前 | |
Fix typos (#53048) Signed-off-by: omahs <73983677+omahs@users.noreply.github.com> | 1 年前 | |
Container runtime env docs (#41764) Document how to use the container runtime env feature. --------- Signed-off-by: Cindy Zhang <cindyzyx9@gmail.com> | 2 年前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 2 年前 | ||
| 3 年前 | ||
| 2 年前 | ||
| 2 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 2 年前 | ||
| 2 年前 | ||
| 3 年前 | ||
| 2 年前 | ||
| 2 年前 | ||
| 2 年前 | ||
| 2 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 2 年前 | ||
| 2 年前 | ||
| 2 年前 | ||
| 2 年前 | ||
| 1 年前 | ||
| 3 年前 | ||
| 2 年前 | ||
| 2 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 1 年前 | ||
| 2 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 2 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 2 年前 | ||
| 2 年前 | ||
| 2 年前 | ||
| 2 年前 | ||
| 2 年前 | ||
| 3 年前 | ||
| 2 年前 | ||
| 2 年前 | ||
| 1 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 2 年前 | ||
| 1 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 1 年前 | ||
| 3 年前 | ||
| 2 年前 | ||
| 1 年前 | ||
| 2 年前 |