0
代码介绍
代码
Issues
Pull Requests
流水线
Actions
讨论
Wiki
项目成员
分析
项目设置
0
  1. qmd
  2. /
  3. finetune
  4. /
  5. evals
TTobi Lutkefinetune: quoted phrases, negation, and entity preservation (#247)
39500557创建于 2月23日历史提交
文件最后提交记录最后更新时间
.gitignore
Refactor evals into separate run and score scripts New structure: - evals/run.py: Generate model outputs to JSONL - evals/score.py: Score outputs with detailed breakdown - evals/queries.txt: Test queries (26 total) Features: - Supports both HF Hub and local model paths - Named entity preservation scoring - Chat template leakage detection - Strict format validation (every line must be lex:/vec:/hyde:) - Generic phrase detection Usage: uv run evals/run.py --model tobil/qmd-query-expansion-0.6B-v4 uv run evals/score.py evals/results_*.jsonl Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> 3 个月前
queries.txt
finetune: quoted phrases, negation, and entity preservation (#247) Training data: - Expand lex phrases/negation examples from 12 to 74 with intent field - Add 50 personal entity examples (meetings, emails, projects with names) Reward function: - Detect entities at position 0 (fixes "Bob asked about deploy") - Per-entity coverage penalty: -20 per entity absent from all lex+vec - Phrase quoting bonus: +3 when lex uses quotes for multi-word terms - Expanded stopwords to reduce false positive entity detection Eval queries: add 21 test queries for personal entities, quoted phrases, and negation/disambiguation scenarios. 2 个月前