已关闭
[Feature]: GLM DSA sparse attention and KPool infrastructure #87
guozhihua2创建于  6 天前关闭于  4 天前
guozhihua2成员
6 天前 创建

Background

GLM-5.3-Flash introduces hybrid attention (KDA linear attention alternating with DSA sparse attention) on top of the existing GLM5/GLM5.2 DSA stack. The sparse-attention groundwork (KPool indexer, kvallgather CP path for DSA, index-share reuse across shared decoder layers) is shared infrastructure that should land and be reviewed independently of the GLM53Flash model itself.

Expected Changes

  1. KPool DSA indexer (kpool.py, kpool_ascend.py) with fixed-width top-k layout and Ascend fused kernels.
  2. dsa_kvallgather_context_parallel.py: kvallgather CP path for DSA, including contiguous/lightning-indexer localization branches.
  3. dsa.py/dsa_utils/dsa_fused/dsa_absorption enhancements: index-share holder reuse across shared decoder layers, dense-TND fallback, metadata validation; glm_dsa_index_share.py holder module.
  4. Feature modules reorganized under features/megatron/{glm,glm_vl,qwen_vl,common} and features/megatron_bridge/{glm_vl,common}; OptimizeSendRecvCommFeature (dual-group ordinary P2P for PP send/recv) and Qwen3ASRAutoConfigFeature added.
  5. Reduced-layer packed-TND validation scripts under tests/poc/glm5 and tests/poc/glm52.

The GLM53Flash model, data pipeline and ST land in follow-up PRs that depend on this one.

Acceptance Criteria

  • glm5_744b / glm52_744b CI ST cases keep passing (no regression on the existing DSA stack).
  • python -m py_compile passes on all new/changed files; repository pre-commit hooks pass on the PR diff.
  • POC scripts run against the reduced-layer checkpoints with the documented parallel configurations.
likedislike
Gguozhihua2成员
6 天前 添加了label:feature
leizhenzhen23成员
6 天前 评论:

👋 您好,欢迎向 MindSpeed-Bridge 提交 Issue!

我们已收到您的反馈,感谢你对开源社区的支持。🎉

📅 处理时效: 维护团队将在 24 小时内 查看并回复您的问题(工作日)。
🔍 自助查询: 在等待期间,建议您先查阅社区文档,可能已有解决方案:

🚨 紧急联系: 如果您的问题非常紧急,可通过以下方式联系我们:

📧 邮箱:[ leizhenzhen4@huawei.com ]

请确保 Issue 描述清晰,包含复现步骤和日志,这将帮助我们更快定位问题。谢谢!

likedislike
Lleizhenzhen23成员
6 天前 添加了label:triaged
Gguozhihua2成员
6 天前 关联了pull request:feat(glm): add GLM DSA sparse attention and KPool infrastructure
Gguozhihua2成员
5 天前 关联了pull request:feat: support GLM-5.3-Flash
ascend-robotascend-robot成员
4 天前 关闭了 issue
ascend-robotascend-robot成员
4 天前 issue状态由 TODO 改变为 DONE
ascend-robotascend-robot成员
4 天前 添加了label:resolved