| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
Deprecate jax and tensorflow, remove unused files | 1 年前 | |
fix(processors): NumpyTensorAdapter.apply_mask doesn't broadcast a lower-rank mask like torch's masked_fill apply_mask used boolean fancy-index assignment (`result[mask] = value`), which requires the mask's shape to exactly match the tensor's shape -- numpy raises IndexError instead of broadcasting when the shapes only match under normal broadcasting rules (e.g. a 1D mask applied to a 2D tensor, a common pattern for reusing the same per-token mask across every row of a batch). TorchTensorAdapter.apply_mask already uses torch.masked_fill, and MLXTensorAdapter already uses mlx.where, both of which broadcast correctly. Only the numpy adapter lacked this, making numpy-backed generation crash on a mask shape that works fine on the other two backends. Fix: use numpy.where(mask, value, tensor), matching MLX's approach and torch's masked_fill's broadcasting semantics. Also non-mutating (like masked_fill), unlike the previous copy-then-index-assign, which is a behavior improvement consistent with the other two adapters. Added test_tensor_adapter_apply_mask_broadcasts_lower_rank_mask (parametrized across all three frameworks) using a 1D mask against a 2D tensor. TDD red->green verified: reverting only numpy.py reproduces `IndexError: boolean index did not match indexed array along axis 0`; reapplying passes. Full tests/processors/test_tensor_adapters.py: 24 passed, 12 skipped (MLX unavailable in this environment, pre-existing and unrelated). ruff/mypy clean via pre-commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> | 1 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 1 年前 | ||
| 1 个月前 |