develop/ai_data_engineering_base

文件	最后提交记录	最后更新时间
observability	feat: add support for OTEL metrics and tracing (#4322) ## Changes Made - Add supports for OTEL metrics and traces via a OLTP GRPC endpoint. - Adds an example docker-compose file to setup a local OLTP collector that sends data to prometheus and jaeger instance bash DAFT_DEV_OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 python testing.py Current runtime stats metrics look like: <img width="628" alt="image" src="https://github.com/user-attachments/assets/0b78fb0d-eee9-4f41-8c73-8a6a7da7051f" /> ## Related Issues ## Checklist - [ ] Documented in API Docs (if applicable) - [ ] Documented in User Guide (if applicable) - [ ] If adding a new documentation page, doc is added to docs/mkdocs.yml navigation - [ ] Documentation builds and is formatted properly (tag @/ccmao1130 for docs review)	1 年前
__init__.py	Build Wheels in CI (#142) * Leverages cibuildwheel to build wheels for linux and macos targets in github workflow * script to correct macos target name x86_64 -> arm64 * uses semantic versioning for daft * change package name to getdaft * adds apache2 license * seperate out dependancies as groups	4 年前
aggregate_test_durations.py	chore: bump mypy and ruff in pre-commit (#5836) ## Changes Made Bump mypy to 1.19.1 and ruff to 0.14.10 and resolve pre-commit errors. Rationale: my local ruff version is much higher than that in the project pre-commit, so I am getting a lot of linter warnings in my IDE. It is also about time to upgrade lints to utilize newer Python language features. ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" -->	9 个月前
attach_debugger.py	docs: Specify what script the debug config uses in CONTRIBUTING.md (#4299) ## Changes Made <!-- Describe what changes were made and why. Include implementation details if necessary. --> The docs for setting up the debugger in CONTRIBUTING.md don't specify what python code it will debug. I quickly figured it out after skimming the given launch.json and tools/attach_debugger.py, but I can imagine others not figuring it out very quickly. This PR adds that small detail to the docs to make it immediately clear. json "configurations": [ { "name": "Debug Rust/Python", ... "args": [ "${file}" ], ... }, ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" -->	1 年前
capture-durations.sh	ci: Aggregate test durations (#5129) ## Changes Made Aggregate pytest durations so we can see what files / directories take the longest. See https://github.com/Eventual-Inc/Daft/actions/runs/17448222436/job/49547643604?pr=5129#step:12:2046 for example ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" --> ## Checklist - [ ] Documented in API Docs (if applicable) - [ ] Documented in User Guide (if applicable) - [ ] If adding a new documentation page, doc is added to docs/mkdocs.yml navigation - [ ] Documentation builds and is formatted properly (tag @/ccmao1130 for docs review)	1 年前
check_invalid_deps.sh	refactor(arrow-rs): Remove arrow2 from daft-sketch (#5967) ## Changes Made Remove arrow2 usage from the sketches code path --------- Co-authored-by: universalmind303 <cory.grinstead@gmail.com>	9 个月前
ci_bench_utils.py	ci: Fix ai benchmark workflow (#5363) ## Changes Made Successful run https://github.com/Eventual-Inc/Daft/actions/runs/18396830291 ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" --> ## Checklist - [ ] Documented in API Docs (if applicable) - [ ] Documented in User Guide (if applicable) - [ ] If adding a new documentation page, doc is added to docs/mkdocs.yml navigation - [ ] Documentation builds and is formatted properly	1 年前
convert_md_to_notebook.py	feat(tools): add markdown-to-notebook converter for documentation (#5691) Depends on https://github.com/Eventual-Inc/Daft/pull/5705. Also depends on #5774. ## Summary - Add a new tool tools/convert_md_to_notebook.py that converts Markdown documentation files to Jupyter notebooks - Replace the outdated docs/quickstart.ipynb with a newly generated version at docs/notebooks/quickstart.ipynb - The converter handles MkDocs-specific syntax (admonitions, icons, grid cards) and converts them to HTML that renders properly in Colab/Jupyter ### Key features of the converter: - Automatically adds Colab badge with correct GitHub links - Converts MkDocs admonitions (!!! note, !!! tip, etc.) to styled HTML divs - Converts relative markdown links to absolute docs.daft.ai URLs - Preserves code output blocks from source markdown - Handles pip install commands by prefixing with ! - Supports --branch option for testing Colab links on feature branches - Supports --update-source flag to add Colab badge to the source markdown file ## Test plan - [x] Run the converter on docs/quickstart.md and verify the output renders correctly in Jupyter/Colab - [x] Verify admonitions render as styled HTML boxes - [x] Verify links point to correct docs.daft.ai URLs - [x] Verify Colab badge links to the correct GitHub path ## Internal https://eventualgroup.slack.com/archives/C09GQPUTXQQ/p1765225846124549 Closes EVE-1327.	10 个月前
gha_run_cluster_job.py	ci: add a tool to launch workloads on cluster (#3516) Example usage: uv run tools/gha_run_cluster_job.py .github/working-dir/simple.py <img width="1589" alt="image" src="https://github.com/user-attachments/assets/26319681-9846-4c37-a630-12b4193478d8"> --------- Co-authored-by: Jay Chia <jaychia94@gmail.com@users.noreply.github.com>	1 年前
git_utils.py	chore: bump mypy and ruff in pre-commit (#5836) ## Changes Made Bump mypy to 1.19.1 and ruff to 0.14.10 and resolve pre-commit errors. Rationale: my local ruff version is much higher than that in the project pre-commit, so I am getting a lot of linter warnings in my IDE. It is also about time to upgrade lints to utilize newer Python language features. ## Related Issues <!-- Link to related GitHub issues, e.g., "Closes #123" -->	9 个月前