| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
chore: minor cleanup to table-value functions (#3854) * removes unnecessary clones since aliases are now added in get_table https://github.com/Eventual-Inc/Daft/pull/3848 * uses factoring suggestion from https://github.com/Eventual-Inc/Daft/pull/3836#discussion_r1964181812 * adds additional table-value function tests | 1 年前 | |
fix(sql): Fix error in TPC-DS q23 (#6317) ## Changes Made Fixed duplicate alias error in TPC-DS Query 23 by updating UNION operation, and added a test. <!-- Describe what changes were made and why. Include implementation details if necessary. --> ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" --> Closes https://github.com/Eventual-Inc/Daft/issues/3561 _The original error "Loaded MicroPartition's tables' schema must match its own schema exactly" has been resolved, and TPC-DS Query 23 now runs successfully._ --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> | 6 个月前 | |
| 7 个月前 | ||
chore: bump mypy and ruff in pre-commit (#5836) ## Changes Made Bump mypy to 1.19.1 and ruff to 0.14.10 and resolve pre-commit errors. Rationale: my local ruff version is much higher than that in the project pre-commit, so I am getting a lot of linter warnings in my IDE. It is also about time to upgrade lints to utilize newer Python language features. ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" --> | 9 个月前 | |
fix: short-circuit evaluation for coalesce (#6525) ## Changes Made <!-- Describe what changes were made and why. Include implementation details if necessary. --> Fixes #4069 by adding proper short-circuit behavior to coalesce. - Added Expr::Coalesce with short-circuit logic - Implemented early-exit in record batch evaluation - Preserved semantics during UDF optimization - Added short-circuit tests ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" --> Closes #4069 --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> | 6 个月前 | |
fix(sql): reject duplicate CTE names in WITH clause (#6331) ## Changes Made Duplicate CTE names in a single WITH clause were silently shadowed — HashMap::insert in plan_ctes() overwrote the first definition without any error, returning incorrect query results. **Root cause:** Bindings::insert() delegates to HashMap::insert(), which silently replaces existing keys. The return value (Option<T>) was discarded, so the second CTE definition quietly replaced the first. **Fix:** Added a pre-insert check in plan_ctes() (src/daft-sql/src/planner.rs). If bound_ctes already contains the CTE name, an InvalidOperation error is raised before insertion. sql -- Previously: silently returned df2's result -- Now: raises "Duplicate CTE name: 'cte'" WITH cte AS (SELECT * FROM df1), cte AS (SELECT * FROM df2) SELECT * FROM cte **Tests:** Added 11 test cases in tests/sql/test_cte_duplicate_names.py covering: - Duplicate names raise error with correct message - Single / multiple distinct CTEs still work (regression) - Case sensitivity (cte vs CTE are distinct) - Duplicates among 3+ CTEs, with different queries / column aliases - CTE name reuse across separate daft.sql() calls ## Related Issues Closes #6325 | 6 个月前 | |
chore: Upgrade Ruff ruleset to 3.9 and add from __future__ import annotations (#4393) | 1 年前 | |
feat: Add a random shuffle operation (#6481) ## Changes Made Implement a global row-rearraging shuffle operation, similar to https://huggingface.co/docs/datasets/process#shuffle. All it does is generate a random integer per row and then sort by that integer. Rather than implementing a custom distributed and local operation, it is all done within expressions and the optimizer. Note, I still made it an explicit logical node because: 1. Its easier to understand in explain outputs 2. I don't think we should always assume that it optimizes the same way that sort does. For example, it might be reasonable to push limits through a shuffle, but not allowed for sorts. ## Related Issues First half of #2612. Second half to add a batch_size version. --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> | 6 个月前 | |
revert!: "revert: Temporarily revert "Remove deprecated APIs for 0.6" (#5084) ## Changes Made Unrevert https://github.com/Eventual-Inc/Daft/pull/5050 now that we've cut a new v0.5 | 1 年前 | |
feat: implement cross join in distributed flotilla engine (#5180) ## Changes Made Implements the cross_join join strategy in the flotilla distributed engine. Logic: Given a left and right pipeline node with N and M partitions respectively, creates N*M tasks with the previous plans that collect the cartesian product of all the inputs. | 1 年前 | |
feat: Add offset syntax support to SQL (#4707) ## Changes Made <!-- Describe what changes were made and why. Include implementation details if necessary. --> ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" --> #4547 ## Checklist - [ ] Documented in API Docs (if applicable) - [ ] Documented in User Guide (if applicable) - [ ] If adding a new documentation page, doc is added to docs/mkdocs.yml navigation - [ ] Documentation builds and is formatted properly (tag @/ccmao1130 for docs review) Signed-off-by: plotor <zhenchao.wang@hotmail.com> | 1 年前 | |
chore: Remove expression namespaces (#5619) ## Changes Made Bye bye ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" --> ## Checklist - [ ] Documented in API Docs (if applicable) - [ ] Documented in User Guide (if applicable) - [ ] If adding a new documentation page, doc is added to docs/mkdocs.yml navigation - [ ] Documentation builds and is formatted properly | 10 个月前 | |
chore: Remove expression namespaces (#5619) ## Changes Made Bye bye ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" --> ## Checklist - [ ] Documented in API Docs (if applicable) - [ ] Documented in User Guide (if applicable) - [ ] If adding a new documentation page, doc is added to docs/mkdocs.yml navigation - [ ] Documentation builds and is formatted properly | 10 个月前 | |
chore: Upgrade Ruff ruleset to 3.9 and add from __future__ import annotations (#4393) | 1 年前 | |
chore: Remove expression namespaces (#5619) ## Changes Made Bye bye ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" --> ## Checklist - [ ] Documented in API Docs (if applicable) - [ ] Documented in User Guide (if applicable) - [ ] If adding a new documentation page, doc is added to docs/mkdocs.yml navigation - [ ] Documentation builds and is formatted properly | 10 个月前 | |
feat: add support for SQL ORDER BY column position (#6211) ## Changes Made Add support for using positional indices in ORDER BY, similar to GROUP BY which already supports this. <!-- Describe what changes were made and why. Include implementation details if necessary. --> ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" --> Closes #3563 --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> | 7 个月前 | |
fix: ".*" not handled correctly in SQL planner (#5784) ## Changes Made <!-- Describe what changes were made and why. Include implementation details if necessary. --> 1. Fix handling of struct field wildcards (.*) in the SQL planner. 2. Add test case to validate. ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" --> https://github.com/Eventual-Inc/Daft/issues/4120 | 8 个月前 | |
fix: close connections in io tests and address other warnings (#6470) ## Changes Made * Iceberg tests weren't closing the sqlite connection which made our CI logs flooded with warnings. * Several other io tests had the same issue. * Also fixes some additional warnings which are making CI logs noisy. ## Related Issues N/A | 6 个月前 | |
chore: Remove expression namespaces (#5619) ## Changes Made Bye bye ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" --> ## Checklist - [ ] Documented in API Docs (if applicable) - [ ] Documented in User Guide (if applicable) - [ ] If adding a new documentation page, doc is added to docs/mkdocs.yml navigation - [ ] Documentation builds and is formatted properly | 10 个月前 | |
revert!: "revert: Temporarily revert "Remove deprecated APIs for 0.6" (#5084) ## Changes Made Unrevert https://github.com/Eventual-Inc/Daft/pull/5050 now that we've cut a new v0.5 | 1 年前 | |
chore: Upgrade Ruff ruleset to 3.9 and add from __future__ import annotations (#4393) | 1 年前 | |
revert!: "revert: Temporarily revert "Remove deprecated APIs for 0.6" (#5084) ## Changes Made Unrevert https://github.com/Eventual-Inc/Daft/pull/5050 now that we've cut a new v0.5 | 1 年前 | |
chore: Upgrade Ruff ruleset to 3.9 and add from __future__ import annotations (#4393) | 1 年前 | |
refactor(arrow2): utf8 comparison kernels (#6032) ## Changes Made Part of arrow2 -> arrow-rs migration. Migrates like, ilike, startswith, endswith, and contains to use arrow-rs compute kernels. Also fixes a bad implementation of like-to-regex in #5423 and removes some faulty tests. ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" --> | 8 个月前 | |
feat: add support for reading json arrays (#4844) ## Changes Made General approach is that we peek at the first byte of the file. if it's [ we use the array algorithm, if its { we use the default ndjson algo. ⚠️ It's worth noting that this is not highly optimized and _could_ OOM on very large JSON files as they are pulled **entirely** into memory. as a side note: Our JSON reading logic is a total mess and I didn't want to do any unnecessary refactors, so there was a bit of copy/paste in this PR to get things to work. ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" --> ## Checklist - [ ] Documented in API Docs (if applicable) - [ ] Documented in User Guide (if applicable) - [ ] If adding a new documentation page, doc is added to docs/mkdocs.yml navigation - [ ] Documentation builds and is formatted properly (tag @/ccmao1130 for docs review) | 1 年前 | |
chore(tests): migrate internal usages of daft.udf to cls/func (#6348) update tests to remove a bunch of deprecation warnings in CI/tests ## Changes Made <!-- Describe what changes were made and why. Include implementation details if necessary. --> ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" --> | 6 个月前 | |
feat(sql): add DATE_TRUNC function support (#6258) DATE_TRUNC('minute', ts) in SQL failed with Function date_trunc not found. The generic auto-registration loop in SQL_FUNCTIONS registers UDFs by calling get_function(empty_args, empty_schema), which succeeds for all functions but produces a passthrough wrapper that doesn't reorder SQL positional arguments into the named-arg format the Truncate UDF expects (it requires interval as a named string arg). This adds a custom SQLFunction wrapper (SQLDateTrunc) that: - Extracts the interval string from the first SQL argument and normalizes bare unit names ('minute' → '1 minute') - Maps the second argument to the input timestamp expression - Optionally maps a third argument to relative_to, defaulting to a null literal - Builds FunctionArgs with the correct positional/named layout the Truncate UDF expects The wrapper is registered as both date_trunc (standard SQL) and truncate (Daft name), after the auto-registration loop so it takes precedence over the generic passthrough. Closes #6257 --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> | 7 个月前 | |
chore: use dedicated OSS AWS account (#6442) ## Changes Made Update to use new dedicated OSS AWS account: - S3 buckets for GitHub artifacts, and public datasets - CloudFront distribution Requires: - Update ACTIONS_AWS_IAM_ROLE GitHub secret ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" --> | 6 个月前 | |
feat: Add string strip function (#6372) ## Changes Made <!-- Describe what changes were made and why. Include implementation details if necessary. --> Add strip() function to Daft's string functions. ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" --> https://github.com/Eventual-Inc/Daft/issues/3792 | 6 个月前 | |
fix: Remove flotilla fallback (#5114) ## Changes Made Removes the flotilla fallback. If the plan is not translatable raise a helpful error message. Additionally changes the flotilla execution config to be use_legacy_ray_runner instead of use_experimental_distributed_engine. ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" --> ## Checklist - [ ] Documented in API Docs (if applicable) - [ ] Documented in User Guide (if applicable) - [ ] If adding a new documentation page, doc is added to docs/mkdocs.yml navigation - [ ] Documentation builds and is formatted properly (tag @/ccmao1130 for docs review) | 1 年前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 1 年前 | ||
| 6 个月前 | ||
| 7 个月前 | ||
| 9 个月前 | ||
| 6 个月前 | ||
| 6 个月前 | ||
| 1 年前 | ||
| 6 个月前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 10 个月前 | ||
| 10 个月前 | ||
| 1 年前 | ||
| 10 个月前 | ||
| 7 个月前 | ||
| 8 个月前 | ||
| 6 个月前 | ||
| 10 个月前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 8 个月前 | ||
| 1 年前 | ||
| 6 个月前 | ||
| 7 个月前 | ||
| 6 个月前 | ||
| 6 个月前 | ||
| 1 年前 |