Closed
[Bug]: DSA sparse attention backward produces zero head dimension without RoPE #385
cuiyushiAdd on Sep 11Closed at Sep 11
Sep 11 Added label: bug
cuiyushi
Sep 11 Comments:
Sep 11 Comments:
Duplicate of mindspore/org-issues#43370. The fix is tracked by hyper-parallel PR #1395.


Sep 11 Close issue
Sep 11 changed custom state from TODO to DONE
Sep 11 Associated pull request
Checklist
🐛 Describe the bug
dsa_sparse_attention_rescalehandles an empty RoPE component asymmetrically between forward and backward.When
query_rope.size(-1) == 0, forward correctly skips padding the sparse-attention output. Backward neverthelessunconditionally executes:
grad_output = grad_output[:, :, :-query_rope.size(-1)] rescaled_output_tnd = rescaled_output_tnd[:, :, :-query_rope.size(-1)]In Python,
:-0is equivalent to:0. Both tensors are therefore reduced to a zero-sized last dimension beforebeing passed to
npu_sparse_flash_attention_grad_enhance.The issue is triggered when all of the following conditions hold:
qk_rope_head_dim == 0param_sink_number > 0A CPU unit test with mocked NPU backward operators reproduces the invalid shape:
Expected behavior
When the RoPE dimension is zero, backward should preserve the complete
grad_outputandrescaled_output_tndhead dimension. Cropping should only occur when
query_rope.size(-1) > 0.Additional context
The later
torch.split(..., [nope_dim, 0])call is valid PyTorch behavior and does not require special handling.Environment info
master