Closed
[Bug]: DSA sparse attention backward produces zero head dimension without RoPE #385
cuiyushiAdd on  Sep 11Closed at  Sep 11
cuiyushi
cuiyushiCollaborators
Sep 11 Create

Checklist

🐛 Describe the bug

dsa_sparse_attention_rescale handles an empty RoPE component asymmetrically between forward and backward.

When query_rope.size(-1) == 0, forward correctly skips padding the sparse-attention output. Backward nevertheless
unconditionally executes:

grad_output = grad_output[:, :, :-query_rope.size(-1)]
rescaled_output_tnd = rescaled_output_tnd[:, :, :-query_rope.size(-1)]

In Python, :-0 is equivalent to :0. Both tensors are therefore reduced to a zero-sized last dimension before
being passed to npu_sparse_flash_attention_grad_enhance.

The issue is triggered when all of the following conditions hold:

  • qk_rope_head_dim == 0
  • param_sink_number > 0
  • backward propagation is executed

A CPU unit test with mocked NPU backward operators reproduces the invalid shape:

expected sparse backward shape: (2, 2, 4)
actual sparse backward shape:   (2, 2, 0)

Expected behavior

When the RoPE dimension is zero, backward should preserve the complete grad_output and rescaled_output_tnd
head dimension. Cropping should only occur when query_rope.size(-1) > 0.

Additional context

The later torch.split(..., [nope_dim, 0]) call is valid PyTorch behavior and does not require special handling.

Environment info

  • HyperParallel: master
  • Python: 3.11
  • PyTorch: 2.6
  • Reproduction: CPU unit test with mocked Ascend NPU backward operators
likedislike
cuiyushicuiyushiCollaborators
Sep 11 Added label: bug
cuiyushi
cuiyushiCollaborators
Sep 11 Comments:

Duplicate of mindspore/org-issues#43370. The fix is tracked by hyper-parallel PR #1395.

likedislike
cuiyushicuiyushiCollaborators
Sep 11 Close issue
MindSpore-BotMindSpore-BotCollaborators
Sep 11 changed custom state from TODO to DONE
cuiyushicuiyushiCollaborators
Sep 11 Associated pull request