| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
[deps] upgrade requests/starlette/jinja2 (#52951) also docker and mlflow; using more recent versions; this make pip-audit slightly happier Signed-off-by: Lonnie Liu <lonnie@anyscale.com> | 1 年前 | |
[data] adapt dask on ray to the new dask task class (#54108) "Dask on Ray" (DoR) is broken in dask==2024.11.0 or later as reported in https://github.com/ray-project/ray/issues/48689 because Dask removed a private function in https://github.com/dask/dask/pull/11378 that DoR has been relying on. Not only https://github.com/dask/dask/pull/11378, Dask has migrated their task data structure to a new format (the high-level motivation is described in https://github.com/dask/dask/issues/9969). Since this migration spans across a series of changes between 2024.11.0 and 2025.1.0, it's not realistic to copy what's been removed and paste them in Ray. This change adapts Dask on Ray to the change to keep its functionality. The change is compatible only with dask>=2024.11.0,<2025.1.0 because Dask made another major change in 2025.1.0, breaking the shuffle optimization introduced in https://github.com/ray-project/ray/pull/13951 Signed-off-by: Lonnie Liu <lonnie@anyscale.com> Co-authored-by: Hiromu Hota <hiromu.hota@gmail.com> | 1 年前 | |
[Data] Add reading from Delta Lake tables and from Unity Catalog (#53701) ## Why are these changes needed? The current methods for accessing data within Databricks is limited in scalability and governance features. This allows for large scale reads from Databricks that also integrates with Unity Catalog using credential vending. --------- Signed-off-by: soffer-anyscale <stephen.offer@anyscale.com> Signed-off-by: soffer-anyscale <173827098+soffer-anyscale@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Alexey Kudinkin <ak@anyscale.com> | 1 年前 | |
[CI] Remove tailing white space [1/N] (#50250) This is part of enabling pre-commit hook trailing-whitespace in lint Signed-off-by: fscnick <fscnick.dev@gmail.com> | 1 年前 | |
[py12] core tests (#46650) Add core unit-tests for python 3.12 version - all are passing now w00t w00t, thanks to all the great work from @hongchaodeng Test: - CI --------- Signed-off-by: can <can@anyscale.com> | 2 年前 | |
[rllib] removes tf2onnx from rllib dependencies (#53339) one step closer to allowing protobuf to upgrade t2fonnx has stopped updating in the past year, and it is pinning down protobuf to 3.x Signed-off-by: Lonnie Liu <lonnie@anyscale.com> | 1 年前 | |
[data] Adding in batch inference mock pipeline release test (#52616) ## Why are these changes needed? We want to add a release test that will be used to model performance on a batch inference image pipeline. This is meant to more accurately model a realistic user pipeline. This will be used to track our improvement on this workload and more broadly our improvement on batch inference with images with and without spot nodes. ## Related issue number <!-- For example: "Closes #1234" --> ## Checks - [ ] I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR. - [ ] I've run scripts/format.sh to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in doc/source/tune/api/ under the corresponding .rst file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :( --------- Signed-off-by: Matthew Owen <mowen@anyscale.com> Co-authored-by: Lonnie Liu <lonnie@anyscale.com> | 1 年前 | |
[ci] Upgrade datasets and huggingface-hub packages (#49490) huggingface-hub==0.19.4 and datasets==2.14.0 no longer work to load any datasets from HF Hub. Upgrade these packages to more up to date versions which work: huggingface-hub==0.24.0 datasets==2.19.1. These packages are not updated to the latest versions, since that requires many changes across examples / Ray library code. It also creates a chain of dependency resolution conflicts that should be tackled as a separate effort. This PR is meant to just unblock the CI which is failing. Example: python >>> import datasets >>> datasets.load_dataset("tweet_eval", "emotion", streaming=True) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/justin/miniforge3/envs/ray_dev_py311/lib/python3.11/site-packages/datasets/load.py", line 2106, in load_dataset builder_instance = load_dataset_builder( ^^^^^^^^^^^^^^^^^^^^^ File "/Users/justin/miniforge3/envs/ray_dev_py311/lib/python3.11/site-packages/datasets/load.py", line 1792, in load_dataset_builder dataset_module = dataset_module_factory( ^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/justin/miniforge3/envs/ray_dev_py311/lib/python3.11/site-packages/datasets/load.py", line 1492, in dataset_module_factory raise e1 from None File "/Users/justin/miniforge3/envs/ray_dev_py311/lib/python3.11/site-packages/datasets/load.py", line 1459, in dataset_module_factory raise e File "/Users/justin/miniforge3/envs/ray_dev_py311/lib/python3.11/site-packages/datasets/load.py", line 1433, in dataset_module_factory dataset_info = hf_api.dataset_info( ^^^^^^^^^^^^^^^^^^^^ File "/Users/justin/miniforge3/envs/ray_dev_py311/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/Users/justin/miniforge3/envs/ray_dev_py311/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 1987, in dataset_info return DatasetInfo(**data) ^^^^^^^^^^^^^^^^^^^ File "/Users/justin/miniforge3/envs/ray_dev_py311/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 626, in __init__ self.tags = kwargs.pop("tags") ^^^^^^^^^^^^^^^^^^ KeyError: 'tags' --------- Signed-off-by: Justin Yu <justinvyu@anyscale.com> Signed-off-by: Matthew Deng <matt@anyscale.com> Co-authored-by: matthewdeng <matt@anyscale.com> | 1 年前 | |
[ci] Upgrade datasets and huggingface-hub packages (#49490) huggingface-hub==0.19.4 and datasets==2.14.0 no longer work to load any datasets from HF Hub. Upgrade these packages to more up to date versions which work: huggingface-hub==0.24.0 datasets==2.19.1. These packages are not updated to the latest versions, since that requires many changes across examples / Ray library code. It also creates a chain of dependency resolution conflicts that should be tackled as a separate effort. This PR is meant to just unblock the CI which is failing. Example: python >>> import datasets >>> datasets.load_dataset("tweet_eval", "emotion", streaming=True) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/justin/miniforge3/envs/ray_dev_py311/lib/python3.11/site-packages/datasets/load.py", line 2106, in load_dataset builder_instance = load_dataset_builder( ^^^^^^^^^^^^^^^^^^^^^ File "/Users/justin/miniforge3/envs/ray_dev_py311/lib/python3.11/site-packages/datasets/load.py", line 1792, in load_dataset_builder dataset_module = dataset_module_factory( ^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/justin/miniforge3/envs/ray_dev_py311/lib/python3.11/site-packages/datasets/load.py", line 1492, in dataset_module_factory raise e1 from None File "/Users/justin/miniforge3/envs/ray_dev_py311/lib/python3.11/site-packages/datasets/load.py", line 1459, in dataset_module_factory raise e File "/Users/justin/miniforge3/envs/ray_dev_py311/lib/python3.11/site-packages/datasets/load.py", line 1433, in dataset_module_factory dataset_info = hf_api.dataset_info( ^^^^^^^^^^^^^^^^^^^^ File "/Users/justin/miniforge3/envs/ray_dev_py311/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/Users/justin/miniforge3/envs/ray_dev_py311/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 1987, in dataset_info return DatasetInfo(**data) ^^^^^^^^^^^^^^^^^^^ File "/Users/justin/miniforge3/envs/ray_dev_py311/lib/python3.11/site-packages/huggingface_hub/hf_api.py", line 626, in __init__ self.tags = kwargs.pop("tags") ^^^^^^^^^^^^^^^^^^ KeyError: 'tags' --------- Signed-off-by: Justin Yu <justinvyu@anyscale.com> Signed-off-by: Matthew Deng <matt@anyscale.com> Co-authored-by: matthewdeng <matt@anyscale.com> | 1 年前 | |
[Tune] update optuna_search to allow users to configure optuna storage (#48547) The OptunaSearch currently uses in-memory storage and does not provide a way to configure any other storages (e.g. database). This update add a configurable parameter to OptunaSearch that can be set to a valid Optuna Storage. Signed-off-by: Ravi Dalal <ravidalal@google.com> Signed-off-by: Ravi Dalal <12639199+ravi-dalal@users.noreply.github.com> Signed-off-by: Hongpeng Guo <hpguo@anyscale.com> Co-authored-by: Hongpeng Guo <hg5@illinois.edu> Co-authored-by: matthewdeng <matthew.j.deng@gmail.com> Co-authored-by: Hongpeng Guo <hpguo@anyscale.com> Co-authored-by: Justin Yu <justinvyu@anyscale.com> | 1 年前 | |
[deps] remove indirect referencing (#50742) it was creating weird paths when generating compiled dependency files locally. Signed-off-by: Lonnie Liu <lonnie@anyscale.com> | 1 年前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 2 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 |