| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
[Data] Replace AggregateFn with AggregateFnV2, cleaning up Aggregation infrastructure (#50585) ## Why are these changes needed? Cleaning up and streamlining existing aggregations Context --- AggregateFn is an unnecessary complicated API that could be streamlined. In addition to that it shifts the burden of implementing null-safe semantic on the user. AggregateFnV2 is meant to address that - Simplifying inputs to just 2 methods: aggregate_block and combine - Handles null-safety out of the box - Is compatible with AggregateFn so no new APIs are introduced AggregateFn is deprecated by this change but will be kept around for BWC. Changes --- - Deprecated AggregateFn, replacing with simpler AggregateFnV2 API (details captured in the py-doc) - Simplified and streamlined handling of NaNs/Nones - Deleted unnecessary code --------- Signed-off-by: Alexey Kudinkin <ak@anyscale.com> | 1 年前 | |
[data] Fix formatting, improve documentation, make finalize public (#52761) <!-- Thank you for your contribution! Please review https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? Currently the rendering is a bit odd: <img width="939" alt="Screenshot 2025-05-02 at 5 19 40 PM" src="https://github.com/user-attachments/assets/4f8ca220-b9cd-4858-ba5d-d22cb71c9b60" /> ## Related issue number <!-- For example: "Closes #1234" --> ## Checks - [ ] I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR. - [ ] I've run scripts/format.sh to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in doc/source/tune/api/ under the corresponding .rst file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :( --------- Signed-off-by: Richard Liaw <rliaw@berkeley.edu> Co-authored-by: Alexey Kudinkin <alexey.kudinkin@gmail.com> | 1 年前 | |
[Data] Mark aggregation functions as public APIs (#50202) https://github.com/ray-project/ray/pull/46193 moved AggregateFn subclasses like Count to the _internal subpackage. The rationale was that they're implementation details of methods like Dataset.count. However, if you want to perform multiple aggregations simultaneously, then you need to use the AggregateFn subclasses. So, this PR makes the subclasses public. --------- Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu> | 1 年前 | |
[Data] Document DataContext (#43578) To globally configure data, you need to set attributes of the DataContext object. However, none of the DataContext settings are documented. This PR documents them, and also simplifies the DataContext module. --------- Signed-off-by: Balaji Veeramani <balaji@anyscale.com> | 2 年前 | |
[data][doc] auto-generate dataset api documentation (#46557) This PR auto-generate dataset API documentation, by moving the declaration of API group to the code itself. Also few other fixes to bring consistency between pages. Test: - dataset page: https://anyscale-ray--46557.com.readthedocs.build/en/46557/data/api/dataset.html - navigation page: https://anyscale-ray--46557.com.readthedocs.build/en/46557/data/api/api.html Note that the documentation is now consistently documenting a class construction in its full form Signed-off-by: can <can@anyscale.com> | 2 年前 | |
[data] Fix formatting, improve documentation, make finalize public (#52761) <!-- Thank you for your contribution! Please review https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? Currently the rendering is a bit odd: <img width="939" alt="Screenshot 2025-05-02 at 5 19 40 PM" src="https://github.com/user-attachments/assets/4f8ca220-b9cd-4858-ba5d-d22cb71c9b60" /> ## Related issue number <!-- For example: "Closes #1234" --> ## Checks - [ ] I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR. - [ ] I've run scripts/format.sh to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in doc/source/tune/api/ under the corresponding .rst file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :( --------- Signed-off-by: Richard Liaw <rliaw@berkeley.edu> Co-authored-by: Alexey Kudinkin <alexey.kudinkin@gmail.com> | 1 年前 | |
[Docs] Remove signatures from API summaries (#39979) Our API summaries contain signatures like (fn, *[, compute, ...]). However, they aren't helpful, so it's better to exclude them. --------- Signed-off-by: Balaji Veeramani <balaji@anyscale.com> | 2 年前 | |
[docs/minor] Update Ray Data docs (#52082) ## Why are these changes needed? Update documentation wording for Ray Data. ## Related issue number <!-- For example: "Closes #1234" --> ## Checks - [ ] I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR. - [ ] I've run scripts/format.sh to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in doc/source/tune/api/ under the corresponding .rst file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :( --------- Signed-off-by: Richard Liaw <rliaw@berkeley.edu> | 1 年前 | |
[Docs] Added user-guide for Joins (#52987) <!-- Thank you for your contribution! Please review https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? 1. Added user-guide for joins 2. Added sections on different shuffling implementations available in Ray Data 3. Tidying up ## Related issue number <!-- For example: "Closes #1234" --> ## Checks - [ ] I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR. - [ ] I've run scripts/format.sh to lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in doc/source/tune/api/ under the corresponding .rst file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :( --------- Signed-off-by: Alexey Kudinkin <ak@anyscale.com> Co-authored-by: Richard Liaw <rliaw@berkeley.edu> Co-authored-by: Hao Chen <chenh1024@gmail.com> | 1 年前 | |
[Data] Add reading from Delta Lake tables and from Unity Catalog (#53701) ## Why are these changes needed? The current methods for accessing data within Databricks is limited in scalability and governance features. This allows for large scale reads from Databricks that also integrates with Unity Catalog using credential vending. --------- Signed-off-by: soffer-anyscale <stephen.offer@anyscale.com> Signed-off-by: soffer-anyscale <173827098+soffer-anyscale@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Alexey Kudinkin <ak@anyscale.com> | 1 年前 | |
[llm.batching] fix doc (#50774) Signed-off-by: Gene Su <e870252314@gmail.com> | 1 年前 | |
[data/docs] fix docs for Ray data vectorizers (#52456) <!-- Thank you for your contribution! Please review https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? It turns out a few preprocessors were missing in the documentation. The full list of APIs can be found in ray/data/preprocessors/__init__.py.  <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number Closes #52349 <!-- For example: "Closes #1234" --> ## Checks - [x] I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR. - [ ] I've run scripts/format.sh to lint the changes in this PR. - [x] I've included any doc changes needed for https://docs.ray.io/en/master/. - [ ] I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in doc/source/tune/api/ under the corresponding .rst file. - [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/ - Testing Strategy - [ ] Unit tests - [ ] Release tests - [ ] This PR is not tested :( Signed-off-by: Kit Lee <7000003+wingkitlee0@users.noreply.github.com> | 1 年前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 2 年前 | ||
| 2 年前 | ||
| 1 年前 | ||
| 2 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 |