已合并
Fix Unicode case handling in literal query ranking #88
Fix Unicode case handling in literal query ranking #88
已合并
KaranocaVe创建于 27 天前
KaranocaVe
KaranocaVe
27 天前

Fix Unicode case handling in literal query ranking

Literal search ranks exact title matches ahead of title fragments, descriptions, and body mentions. Its scorer used SQLite lower(), which only handles ASCII case, while indexing and query normalization use Python's Unicode-aware str.lower(). A query such as Θ Exact title could therefore rank a description mention of θ Exact title above the exact title.

Use the same Unicode normalization in field scoring and add a CLI regression for the Greek case pair with an ASCII ranking control.

Validation: python3 -m unittest evals.test_retrieval_index.RetrievalIndexTests (10 tests passed) and bash check.sh (295 repository tests passed with 1 skipped, 36 trace tests passed, Ruff and shell checks passed).

关联 Issue: https://gitcode.com/cann/cannbot-knowledge/issues/33

likedislike
Pull Request已成功合入, 合并人@CANN-robot
(感谢 KaranocaVe 的贡献)
KaranocaVeKaranocaVe
27 天前 创建了 pull request,commit 5dc5b723
KaranocaVeKaranocaVe
27 天前 关联了issue:Unicode case handling can demote exact title matches in literal search
CANN-robotCANN-robot成员
27 天前 添加了label:cann-cla/yes
CANN-robot
CANN-robot成员
27 天前 评论:

Thanks for your pull-request.
The full list of commands accepted by me can be found at here.
You can get sig-info at here.
You can self-configure the PR merge rules for this repository. For more details, please refer to Here.
For more, you also can visit HICANN.


PR Approval Progress

✅ Congratulations! All modules have met the lgtm and approve requirements.

Module Approval Details

module lgtm status approve status
repo-cann/cannbot-knowledge ✅ qq_41667743, zhanghz1 (2/2) ✅ qq_41667743 (1/1)

💡 Tip:

  • Committer can comment /approve or /lgtm
  • Commenting /approve implies both code review (lgtm) and intent to merge (approve)

CLA Signature Pass

KaranocaVe, thanks for your pull request. All authors of the commits have signed the CLA. 👍

likedislike
KaranocaVe
KaranocaVe
27 天前 评论:

/compile

likedislike
KaranocaVeKaranocaVe
27 天前 预合并成功(commit_id: 6c4a3e3fd61fd1b011c845b68ca2285605efaf63)
CANN-robot
CANN-robot成员
27 天前 评论:
🚀 CI 流水线已启动
📋 执行详情: 点击查看流水线
likedislike
CANN-robotCANN-robot成员
27 天前 添加了label:ci-pipeline-running
CANN-robotCANN-robot成员
27 天前 删除了label:ci-pipeline-running
CANN-robotCANN-robot成员
27 天前 添加了label:ci-pipeline-passed
KaranocaVe
KaranocaVe
14 天前 评论:

@zhanghz1 @qq_41667743 @xfwang_ecnu @fafahan @zhengshencn_hwca @OpSphere @jon_jin @wangsj0012

麻烦各位抽空审核这个 PR;确认没有问题后,请按仓库规则补齐所需的 /lgtm 和 /approve。如果有需要修改的地方,请直接留言,我会跟进。谢谢!

likedislike
zhanghz1
zhanghz1成员
13 天前 评论:

/lgtm

likedislike
KaranocaVe
KaranocaVe
11 天前 评论:

补充提醒:按机器人当前审批进度,模块 repo-cann/cannbot-knowledge 还缺 approve。麻烦 @linghuis 抽空协助审批,并在条件满足后推进合并,谢谢!

likedislike
KaranocaVe
KaranocaVe
11 天前 评论:

再跟进一下:按机器人最新审批进度,模块 repo-cann/cannbot-knowledge 仍待 approve。麻烦 @zhanghz1 @xfwang_ecnu @linghuis @wangsj0012 @fafahan 再次抽空协助审批,并在审批条件满足后推进合并,谢谢!

likedislike
KaranocaVe
KaranocaVe
9 天前 评论:

/compile

likedislike
KaranocaVeKaranocaVe
9 天前 预合并成功(commit_id: 9ec24a7645db4ca68f8c307276420380c0ea4259)
CANN-robot
CANN-robot成员
9 天前 评论:
🚀 CI 流水线已启动
📋 执行详情: 点击查看流水线
likedislike
CANN-robotCANN-robot成员
9 天前 删除了label:ci-pipeline-passed
CANN-robotCANN-robot成员
9 天前 添加了label:ci-pipeline-running
CANN-robotCANN-robot成员
9 天前 删除了label:ci-pipeline-running
CANN-robotCANN-robot成员
9 天前 添加了label:ci-pipeline-passed
KaranocaVe
KaranocaVe
3 天前 评论:

再次提醒:根据 CANN-robot 当前审批进度,该 PR 仍缺少必要的代码检视或合入审批。麻烦 qq_41667743 OpSphere linghuis fafahan wangsj0012 zhanghz1 xfwang_ecnu 抽空协助审核;确认无误后请按仓库规则执行 /lgtm 和 /approve,谢谢!

likedislike
KaranocaVe
KaranocaVe
3 天前 评论:

再次提醒:根据 CANN-robot 当前审批进度,该 PR 仍缺少必要的代码检视或合入审批。麻烦 @qq_41667743 @OpSphere @linghuis @fafahan @wangsj0012 @zhanghz1 @xfwang_ecnu 抽空协助审核;确认无误后请按仓库规则执行 /lgtm 和 /approve,谢谢!

likedislike
zhanghz1
zhanghz1成员
3 天前 评论:

/lgtm

likedislike
汪超成员
3 天前 评论:

/approve

likedislike
CANN-robotCANN-robot成员
3 天前 添加了label:lgtmapproved
CANN-robotCANN-robot成员
3 天前 合入了pull request