| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
[mlir][linalg] Vectorize directly to a named contraction (#147296) Extends linalg vectorizer with a path to lower contraction ops directly into vector.contract. The direct rewriting preserves high-level op semantics and provides more progressive lowering compared to reconstructing contraction back from multi dimensional reduction. The added lowering focuses on named linalg ops and leverages their well defined semantics to avoid complex precondition verification. The new path is optional and disabled by default to avoid changing the default vectorizer behavior. | 1 年前 | |
[mlir][linalg][nfc] Move vectorization tests (#141656) Moves all the remaining Linalg vectorization tests from: * mlir/tests/Dialect/Linalg/* to: * mlir/tests/Dialect/Linalg/vectorization/* To maintain consistency within tests, vectorize-convolution.mlir was updated to use: * transform.structured.vectorize_children_and_apply_patterns instead of: * -test-linalg-transform-patterns=test-linalg-to-vector-patterns This change required minor updates to some CHECK lines, reflecting only reordering of ops due to an additional pattern being applied. Closes #141025 | 1 年前 | |
[mlir][linalg][nfc] Move vectorization tests (#141656) Moves all the remaining Linalg vectorization tests from: * mlir/tests/Dialect/Linalg/* to: * mlir/tests/Dialect/Linalg/vectorization/* To maintain consistency within tests, vectorize-convolution.mlir was updated to use: * transform.structured.vectorize_children_and_apply_patterns instead of: * -test-linalg-transform-patterns=test-linalg-to-vector-patterns This change required minor updates to some CHECK lines, reflecting only reordering of ops due to an additional pattern being applied. Closes #141025 | 1 年前 | |
[mlir][linalg][nfc] Move vectorization tests (#141656) Moves all the remaining Linalg vectorization tests from: * mlir/tests/Dialect/Linalg/* to: * mlir/tests/Dialect/Linalg/vectorization/* To maintain consistency within tests, vectorize-convolution.mlir was updated to use: * transform.structured.vectorize_children_and_apply_patterns instead of: * -test-linalg-transform-patterns=test-linalg-to-vector-patterns This change required minor updates to some CHECK lines, reflecting only reordering of ops due to an additional pattern being applied. Closes #141025 | 1 年前 | |
[mlir][Vector] Make elementwise-on-broadcast sinking handle splat consts (#150867) There is a pattern that rewrites elementwise_op(broadcast(x1 : T to U), broadcast(x2 : T to U), ...) to broadcast(elementwise_op(x1, x2, ...) : T to U). This pattern did not, however, account for the case where a broadcast constant is represented as a SplatElementsAttr, which can safely be reshaped or scalarized but is not a vector.broadcast or vector.splat operation. This patch fixes this oversight, prenting premature broadcasting. This did result in the need to update some linalg dialect tests, which now feature a less-broadcast computation and/or more constant folding. | 1 年前 | |
[mlir][linalg] Use ub.poison in linalg vectorizer instead of 0 for some transfer ops (#146544) This patch is a follow up to https://github.com/llvm/llvm-project/pull/146088 and changes the padding value in the linalg vectorizer from 0 to ub.poison in vector.transfer_reads created for extracting slices or when vectorizing a generic. Signed-off-by: Fabian Mora <fabian.mora-cordero@amd.com> | 1 年前 | |
[mlir][linalg] Move vectorization tests for Tensor Ops (nfc) (#140877) This patch reorganises vectorisation tests for tensor ops: * Tests for tensor.pad and tensor.insert_slice are extracted into dedicated files under a new vectorization/ subdirectory. * Test files for tensor.extract are renamed and moved to the same subdirectory. Goals: * Unify test file naming. * Better organise the growing set of tests, which are currently hard to navigate. This is also a preparatory step for upcoming changes. I’ll soon be updating the vectorisation logic for tensor.pad and tensor.insert_slice. With the new structure in place, follow-up changes will be easier to review: * Only tests related to those ops will be updated. * Changes (e.g., to masking logic) will be isolated to the relevant tests. This patch implements part of #141025 - please see the ticket for full context. | 1 年前 | |
[mlir][linalg] Refactor vectorization hooks to improve code reuse (#141244) This patch refactors two vectorization hooks in Vectorization.cpp: * createWriteOrMaskedWrite gains a new parameter for write indices, aligning it with its counterpart createReadOrMaskedRead. * vectorizeAsInsertSliceOp is updated to reuse both of the above hooks, rather than re-implementing similar logic. CONTEXT ------- This is effectively a refactoring of the logic for vectorizing tensor.insert_slice. Recent updates added masking support: * https://github.com/llvm/llvm-project/pull/122927 * https://github.com/llvm/llvm-project/pull/123031 At the time, reuse of the shared create* hooks wasn't feasible due to missing parameters and overly rigid assumptions. This patch resolves that and moves us closer to a more maintainable structure. CHANGES IN createWriteOrMaskedWrite ------------------------------------- * Introduces a clear distinction between the destination tensor and the vector to store, via named variables like destType/vecToStoreType, destShape/vecToStoreShape, etc. * Ensures the correct rank and shape are used for attributes like in_bounds. For example, the size of the in_bounds attr now matches the source vector rank, not the tensor rank. * Drops the assumption that vecToStoreRank == destRank - this doesn't hold in many real examples. * Deduces mask dimensions from vecToStoreShape (vector) instead of destShape (tensor). (Eventually we should not require inputVecSizesForLeadingDims at all - mask shape should be inferred.) NEW HELPER: isMaskTriviallyFoldable ------------------------------------- Adds a utility to detect when masking is unnecessary. This avoids inserting redundant masks and reduces the burden on canonicalization to clean them up later. Example where masking is provably unnecessary: mlir %2 = vector.mask %1 { vector.transfer_write %0, %arg1[%c0, %c0, %c0, %c0, %c0, %c0] {in_bounds = [true, true, true]} : vector<1x2x3xf32>, tensor<9x8x7x1x2x3xf32> } : vector<1x2x3xi1> -> tensor<9x8x7x1x2x3xf32> Also, without this hook, tests are more complicated and require more matching. VECTORIZATION BEHAVIOUR ----------------------- This patch preserves the current behaviour around masking and the use ofin_bounds attribute. Specifically: * useInBoundsInsteadOfMasking is set when no input vector sizes are available. * The vectorizer continues to infer vector sizes where needed. Note: the computation of the in_bounds attribute is not always correct. That issue is tracked here: * https://github.com/llvm/llvm-project/issues/142107 This will be addressed separately. TEST CHANGES ----------- Only affects vectorization of: * tensor.insert_slice (now refactored to use shared hooks) Test diffs involve additional arith.constant Ops due to increased reuse of shared helpers (which generate their own constants). This will be cleaned up via constant caching (see #138265). NOTE FOR REVIEWERS ------------------ This is a fairly substantial rewrite. You may find it easier to review createWriteOrMaskedWrite as a new method rather than diffing line-by-line. TODOs (future PRs) ------------------ Further alignment of createWriteOrMaskedWrite and createReadOrMaskedRead: * Move createWriteOrMaskedWrite next to createReadOrMaskedRead (in VectorUtils.cpp) * Make createReadOrMaskedRead leverage isMaskTriviallyFoldable. * Extend isMaskTriviallyFoldable with value-bounds-analysis. See the updated test in transform-vector.mlir for an example that would benefit from this. * Address #142107 (*) This method will eventually be moved out of Vectorization.cpp, which isn't the right long-term home for it. | 1 年前 | |
[mlir][vector] Missing indices on vectorization of 1-d reduction to 1-ranked memref (#166959) Vectorization of a 1-d reduction where the output variable is a 1-ranked memref can generate an invalid vector.transfer_write with no indices for the memref, e.g.: vector.transfer_write"(%vec, %buff) <{...}> : (vector<f32>, memref<1xf32>) -> () This patch solves the problem by providing the expected amount of indices (i.e. matching the rank of the memref). | 9 个月前 | |
[mlir][linalg] Update vectorization of linalg.pack (#163539) This patch changes vectorizeAsTensorPackOp to require users to specify **all** write-side vector sizes for linalg.pack (not just the outer dimensions). This makes linalg.pack vectorization consistent with linalg.unpack (see https://github.com/llvm/llvm-project/pull/149293 for a similar change). Conceptually, linalg.pack consists of these high-level steps: * **Read** from the source tensor using vector.transfer_read. * **Re-associate** dimensions of the read value, as specified by the op (via vector.shape_cast) * **Transpose** the re-associated value according to the permutation in the linalg.pack op (via vector.transpose). * **Write** the result into the destination tensor via vector.transfer_write. Previously, the vector sizes provided by the user were interpreted as write-vector-sizes for PackOp **_outer_** dims (i.e. the final step above). These were used to: * Infer read-vector-sizes using the inner_tiles attribute of PackOp. * Deduce vector sizes for the transpose and shape cast operations. * Ultimately determine the vector shape for the read. However, this logic breaks when one or more tile sizes are dynamic (*). In such cases, vectorizePackOpPrecondition would currently fail (see @pack_with_dynamic_dims_and_dynamic_inner_tile added in this PR - without this change it will crash). This patch updates the contract: users now directly specify _all_ the "write-vector-sizes", which inherently encode all inner tile sizes - including dynamic ones. It becomes the user's responsibility to provide valid sizes. In practice, since linalg.pack is typically constructed, tiled, and vectorized by the same transformation pipeline, the necessary "write-vector-sizes" should be recoverable. Notes for reviewers: * See test updates for user-facing impact. * Review vectorizeAsTensorPackOp as a new implementation rather than a diff. * Comments and variable names were updated to align with vectorizeAsTensorUnPackOp. (*) As a concrete example, "scalable" tile sizes are represent as dynamic values. Note, support for "scalable" vectorisation will be added in a separate PR. | 9 个月前 | |
[mlir][linalg][nfc] Move vectorization tests (#141656) Moves all the remaining Linalg vectorization tests from: * mlir/tests/Dialect/Linalg/* to: * mlir/tests/Dialect/Linalg/vectorization/* To maintain consistency within tests, vectorize-convolution.mlir was updated to use: * transform.structured.vectorize_children_and_apply_patterns instead of: * -test-linalg-transform-patterns=test-linalg-to-vector-patterns This change required minor updates to some CHECK lines, reflecting only reordering of ops due to an additional pattern being applied. Closes #141025 | 1 年前 | |
[mlir][linalg] Move vectorization tests for Tensor Ops (nfc) (#140877) This patch reorganises vectorisation tests for tensor ops: * Tests for tensor.pad and tensor.insert_slice are extracted into dedicated files under a new vectorization/ subdirectory. * Test files for tensor.extract are renamed and moved to the same subdirectory. Goals: * Unify test file naming. * Better organise the growing set of tests, which are currently hard to navigate. This is also a preparatory step for upcoming changes. I’ll soon be updating the vectorisation logic for tensor.pad and tensor.insert_slice. With the new structure in place, follow-up changes will be easier to review: * Only tests related to those ops will be updated. * Changes (e.g., to masking logic) will be isolated to the relevant tests. This patch implements part of #141025 - please see the ticket for full context. | 1 年前 | |
[mlir][linalg] Move vectorization tests for Tensor Ops (nfc) (#140877) This patch reorganises vectorisation tests for tensor ops: * Tests for tensor.pad and tensor.insert_slice are extracted into dedicated files under a new vectorization/ subdirectory. * Test files for tensor.extract are renamed and moved to the same subdirectory. Goals: * Unify test file naming. * Better organise the growing set of tests, which are currently hard to navigate. This is also a preparatory step for upcoming changes. I’ll soon be updating the vectorisation logic for tensor.pad and tensor.insert_slice. With the new structure in place, follow-up changes will be easier to review: * Only tests related to those ops will be updated. * Changes (e.g., to masking logic) will be isolated to the relevant tests. This patch implements part of #141025 - please see the ticket for full context. | 1 年前 | |
[mlir][linalg][nfc] Move vectorization tests (#141656) Moves all the remaining Linalg vectorization tests from: * mlir/tests/Dialect/Linalg/* to: * mlir/tests/Dialect/Linalg/vectorization/* To maintain consistency within tests, vectorize-convolution.mlir was updated to use: * transform.structured.vectorize_children_and_apply_patterns instead of: * -test-linalg-transform-patterns=test-linalg-to-vector-patterns This change required minor updates to some CHECK lines, reflecting only reordering of ops due to an additional pattern being applied. Closes #141025 | 1 年前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 9 个月前 | ||
| 9 个月前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 |