main

GGitHubGeneralize task chunker and add more logging (#423)

bd60469e创建于 3月5日历史提交

文件	最后提交记录	最后更新时间
postprocessing	create a chunking abstraction	2 年前
preprocessing	Generalize task chunker and add more logging (#423) * generalize task chunker and add more logging * Fix misleading variable name in chunking example (#429) * Initial plan * Fix variable name: rename hypothetical to summarizer in chunking.rs example Co-authored-by: ealmloff <66571940+ealmloff@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: ealmloff <66571940+ealmloff@users.noreply.github.com> * fix formatting --------- Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>	2 个月前
mod.rs	Remote chat, remote structured generation models, and single file gguf chat model loading (#319) * add chat template support and remove the VectorSpace trait * move sampling and chat templates to kalosm llama * update kalosm-llama unstructured generation to the new interface * restore structured generation module * Restore llama implementation of structured generation * clean up kalosm-llama clippy lints * restore llama chat and structured chat implementation * improve infer chat example * add support for remote chat models * support constraints for openai remote models * load the tokenizer from the gguf file if a huggingface tokenizer is not present * Fix tokenizer conversion * restore chat struct * Fix chat implementation with llama * remove tokio from language model * Create chat and text completion extension traits * add task helper to the chat extension trait * update kalosm-language to new task interface * make llama callable * add with_constraints method to task * fix task example * update examples to new chat and task api * set tools to none to fix llama chat template * Add helpers for the default parser for a specific type and model combo * simplify constrained rust type example * restore prompt annealing * fix structured example * document text completion model * document new chat api * update task documentation * Fix tokenizer gguf * fix custom llama source example * fix remaining tests * add logging to remote examples * Clippy fixes * More clippy fixes * use function call in docs more constantly * fix remaining doc tests	1 年前