Vision infrastructure to turn complex documents into RAG/LLM-ready data
| Files | Last commit | Last update |
|---|---|---|
feat: added Open telemetry support for better analytics (#504) * set up otel collector and opentelemetry in server * added tracing to create task * removed api key from trace * updated crates * updated create_task span to use the same trace as actix * added user telemetry in middleware * propagated context to task worker * added spans for pipeline steps * added better messaging during file conversion * removed macro and updated error handling * added retry tracing * updated step span to have one for each retry * added llm level spans * fixed step level context issue * traces till open ai request added, need to fix parent-child relationships * fixed parent-child relationships * updated llm error * added better error messages for llm errors * updated attributes for better analytics * updated README for otel collector * fixed clippy warnings | 1 year ago | |
feat: created a stable and simple version by removing all extra/unused components (#570) * updated README * removed kube * removed clients * removed proxy and ngnix * removed unused sh files * removed CD steps from the workflow * updated docker compose * adding new compose files to reduce repeated code * reset README * removed vgt * removed package.lock | 1 year ago | |
ak/rate limit (#448) * added rate limit check with limit-lens * updated test to use the limit lens client * updated llm rate limit to have better errors and use model id * fix: LLM rate limits not initializing properly * Update settings.json | 1 year ago | |
chore: release main (#569) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> | 1 year ago | |
build: commit Cargo.lock for reproducible builds The .gitignore rule 'Ignore Cargo.lock if it is a library' was applied to binary crates, so neither server had a committed lockfile. Dependency versions therefore resolved fresh on every build. Commit the lockfiles and remove the ignore rule so builds are reproducible and the resolved dependency set is recorded. | 2 days ago | |
fix: replaced vgt with yolo model as it's more practical for consumer hardware | 1 year ago | |
updated readme | 1 year ago | |
feat: created a stable and simple version by removing all extra/unused components (#570) * updated README * removed kube * removed clients * removed proxy and ngnix * removed unused sh files * removed CD steps from the workflow * updated docker compose * adding new compose files to reduce repeated code * reset README * removed vgt * removed package.lock | 1 year ago | |
refactored | 1 year ago | |
chore: release main (#572) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> | 1 year ago | |
"update-codespellrc" | 1 year ago | |
feat: allow users to choose an LLM model to use for `segment_processing` by setting the `llm_processing.model_id` param in the POST and PATCH request. The available models can be configured using the `models.yaml` file. (#437) * exposed llm model and url to requester * llm processing custom models * llm model expose * compose * Update compose-cpu.yaml * Update llm_config.rs * addressed comments * addressed all comments except for .env and yaml merging in llmconfig - unsure about llm_ocr usage * fixed throttling and llm service * LLM working successfully * updated configuration to switch error_handling and llm_processing to have defaults instead of optional * added checks to create and upload routes to throw 400 erros if model_ids not in config * added route for llm models * updated llm processing deserializing to automatically replace defaults with actual model names and added check for outdated models in the update route * llm throttle is now model based rather than domain based * improved LLM fallbacks * all tests passed * formatted code * updated docker compose file replicas * updated models.yaml examples * added config map for models.yaml * updated README * removed example models.yaml from core --------- Co-authored-by: Akhilesh Sharma <sharmaninenine@gmail.com> | 1 year ago | |
chore: added to .env | 1 year ago | |
build: commit Cargo.lock for reproducible builds The .gitignore rule 'Ignore Cargo.lock if it is a library' was applied to binary crates, so neither server had a committed lockfile. Dependency versions therefore resolved fresh on every build. Commit the lockfiles and remove the ignore rule so builds are reproducible and the resolved dependency set is recorded. | 2 days ago | |
fixed pdf stuff | 2 years ago | |
chore: removed unused config for release please | 1 year ago | |
chore: release main (#572) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> | 1 year ago | |
chore: release main (#572) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> | 1 year ago | |
updated readme | 1 year ago | |
Rename CONTENT-MIGRATION-GUIDE to CONTENT-MIGRATION-GUIDE.md | 1 year ago | |
Create LICENSE | 1 year ago | |
Update README.md | 11 months ago | |
Add codespell support (config, workflow to detect/not fix) and make it fix few typos (#423) * Add github action to codespell main on push and PRs * Add rudimentary codespell config * [DATALAD RUNCMD] run codespell throughout fixing few left typos automagically === Do not change lines below === { "chain": [], "cmd": "codespell -w", "exit": 0, "extra_inputs": [], "inputs": [], "outputs": [], "pwd": "." } ^^^ Do not change lines above ^^^ | 1 year ago | |
chore: updated build context for cpu | 1 year ago | |
feat: created a stable and simple version by removing all extra/unused components (#570) * updated README * removed kube * removed clients * removed proxy and ngnix * removed unused sh files * removed CD steps from the workflow * updated docker compose * adding new compose files to reduce repeated code * reset README * removed vgt * removed package.lock | 1 year ago | |
feat: created a stable and simple version by removing all extra/unused components (#570) * updated README * removed kube * removed clients * removed proxy and ngnix * removed unused sh files * removed CD steps from the workflow * updated docker compose * adding new compose files to reduce repeated code * reset README * removed vgt * removed package.lock | 1 year ago | |
feat: allow users to choose an LLM model to use for `segment_processing` by setting the `llm_processing.model_id` param in the POST and PATCH request. The available models can be configured using the `models.yaml` file. (#437) * exposed llm model and url to requester * llm processing custom models * llm model expose * compose * Update compose-cpu.yaml * Update llm_config.rs * addressed comments * addressed all comments except for .env and yaml merging in llmconfig - unsure about llm_ocr usage * fixed throttling and llm service * LLM working successfully * updated configuration to switch error_handling and llm_processing to have defaults instead of optional * added checks to create and upload routes to throw 400 erros if model_ids not in config * added route for llm models * updated llm processing deserializing to automatically replace defaults with actual model names and added check for outdated models in the update route * llm throttle is now model based rather than domain based * improved LLM fallbacks * all tests passed * formatted code * updated docker compose file replicas * updated models.yaml examples * added config map for models.yaml * updated README * removed example models.yaml from core --------- Co-authored-by: Akhilesh Sharma <sharmaninenine@gmail.com> | 1 year ago | |
feat: https support for docker compose using nginx (#434) | 1 year ago |
Project README is empty