| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
[flang][hlfir] Implement hlfir.forall codegen when no temp is required The patch applies the schedule built with the utility added in the previous D150455 patch to generate the code for an ordered assignment tree. For now, it only supports forall that do not contain user defined assignments or assignments to vector subscripted entities, and for which the scheduling analysis does not require temporary storages. Support for temporary, WHERE, and user-defined/vector subscript assignment will be added in later patches. This enables end-to-end support with HLFIR for forall where the schedule analysis can prove there is no need to create temporary storage. Differential Revision: https://reviews.llvm.org/D150564 | 3 年前 | |
[flang][hlfir] Hoist forall bounds computation when possible When inner forall bound computations do not depend on previous forall indices, they can be hoisted. This is possible because: - bound computation are required to be pure (so evaluating them only once is possible). - If the bound computation depends on a value previously assigned, the forall scheduling analysis created different run for it: the assignment impacting the bounds value is not part of the current loop nest. The reason this optimization is done at that point and not as part of generic loop hoisting optimization is that having the all the loop bound computation hoisted will allow allocating simple temporary storages. The number of iteration can be pre-computed and used as the extent for the temporary. Differential Revision: https://reviews.llvm.org/D151110 | 3 年前 | |
[flang][hlfir] Implement the scheduling part of hlfir.forall codegen The lowering of hlfir.forall to loops (and later hlfir.where) requires doing a data dependency analysis to avoid creating temporary storage for every control/mask/rhs/lhs expressions. The added code implements a data dependency analysis for the hlfir ordered assignment trees (it is not specific to Forall since these nodes includes Where, user defined assignments, and assignment to vector subscripted entities, but the added code is only plugged and tested with hlfir.forall in this patch). This data dependency analysis returns a "schedule", which is a list of runs containing actions. Each runs will result in a single loop nest evaluating all its action "at the same time" inside the loop body. Actions may either evaluating an assignment, or saving some expression evaluation (the value yielded inside the ordered assignment hlfir operations) in a temporary storage before doing the assignment that requires this expression value but may "conflict" with it. A "conflict" is a read in an expression E to a variable that is, or may be (analysis is conservative), written by an assignment that depends on E. The analysis is based on MLIR SideEffectInterface and fir AliasAnalysis which makes it generic. For now, the codegen that will apply the schedule and rewrite the hlfir.forall into a set of loops is not implemented, but the scheduling is tested on its own (from Fortran, because it allows testing many cases in very readable fashions). The current scheduling has limitations, for instance "forall(i=1, 10) x(i)=2*x(i)" does not require saving the RHS values for all "i" before doing the assignments since the RHS does not depend on values computed during previous iterations. Any user call will also trigger a conservative assumption that there is a conflict. Finally, a lot of operations are missing memory effect interfaces (especially in HLFIR). This patch adds a few so that it can be tested, but more will be added in later patches. Differential Revision: https://reviews.llvm.org/D150455 | 3 年前 | |
[flang][HLFIR] fix FORALL issue 120190 (#120236) Fix #120190. The hlfir.forall lowering code was not properly checking for forall index reference in mask value computation before trying to hoist it: it was only looking at the ops directly nested in the hlfir.forall_mask region, but not the operation indirectly nested. This caused triggered bogus hoisting in #120190 leading to undefined behavior (reference to uinitialized data). The added regression test would die at compile time with a dominance error. Fix this by doing a deep walk of the region operation instead. Also clean-up the region cloning to use without_terminator. | 1 年前 | |
| 9 个月前 | ||
[flang] lower remaining cases of pointer assignments inside forall (#130772) Implement handling of NULL() RHS, polymorphic pointers, as well as lower bounds or bounds remapping in pointer assignment inside FORALL. These cases eventually do not require updating hlfir.region_assign, lowering can simply prepare the new descriptor for the LHS inside the RHS region. Looking more closely at the polymorphic cases, there is not need to call the runtime, fir.rebox and fir.embox do handle the dynamic type setting correctly. After this patch, the last remaining TODO is the allocatable assignment inside FORALL, which like some cases here, is more likely an accidental feature given FORALL was deprecated in F2003 at the same time than allocatable components where added. | 1 年前 | |
[flang] lower remaining cases of pointer assignments inside forall (#130772) Implement handling of NULL() RHS, polymorphic pointers, as well as lower bounds or bounds remapping in pointer assignment inside FORALL. These cases eventually do not require updating hlfir.region_assign, lowering can simply prepare the new descriptor for the LHS inside the RHS region. Looking more closely at the polymorphic cases, there is not need to call the runtime, fir.rebox and fir.embox do handle the dynamic type setting correctly. After this patch, the last remaining TODO is the allocatable assignment inside FORALL, which like some cases here, is more likely an accidental feature given FORALL was deprecated in F2003 at the same time than allocatable components where added. | 1 年前 | |
| 1 年前 | ||
[flang] add support for procedure pointer assignment inside FORALL (#130114) Very similar to object pointer assignment, the difference is the SSA types of the LHS (!fir.ref<!fir.boxproc<()->()>> and RHS (!fir.boxproc<()->()). The RHS must be saved as simple address, not descriptors (it is not possible to make CFI descriptor out of procedure entity). | 1 年前 | |
[flang][NFC] Strip trailing whitespace from tests (2 of N) Only the fortran source files in flang/test have been modified. The other files in the directory will be cleaned up in subsequent commits | 9 个月前 | |
[flang] lower remaining cases of pointer assignments inside forall (#130772) Implement handling of NULL() RHS, polymorphic pointers, as well as lower bounds or bounds remapping in pointer assignment inside FORALL. These cases eventually do not require updating hlfir.region_assign, lowering can simply prepare the new descriptor for the LHS inside the RHS region. Looking more closely at the polymorphic cases, there is not need to call the runtime, fir.rebox and fir.embox do handle the dynamic type setting correctly. After this patch, the last remaining TODO is the allocatable assignment inside FORALL, which like some cases here, is more likely an accidental feature given FORALL was deprecated in F2003 at the same time than allocatable components where added. | 1 年前 | |
[flang][hlfir] Fixed where/elsewhere mask saving in case of conflicts. The assignments inside where/elsewhere may affect variables participating in the mask expression, but execution of the assignments must not affect the established control mask(s) (F'18 10.2.3.2 p. 13). The following example must print all 42's: program test integer c(3) logical :: mask(3) = .true. where (mask) c = f() end where print *, c contains integer function f() mask = .false. f = 42 end function f end program test Reviewed By: tblah Differential Revision: https://reviews.llvm.org/D156959 | 2 年前 | |
[flang] Do not hoist all scalar sub-expressions from WHERE constructs (#91395) The HLFIR pass lowering WHERE (hlfir.where op) was too aggressive in its hoisting of scalar sub-expressions from LHS/RHS/MASKS outside of the loops generated for the WHERE construct. This violated F'2023 10.2.3.2 point 10 that stipulated that elemental operations must be evaluated only for elements corresponding to true values, because scalar operations are still elemental, and hoisting them is invalid if they could have side effects (e.g, division by zero) and if the MASK is always false (i.e., the loop body is never evaluated). The difficulty is that 10.2.3.2 point 9 mandates that nonelemental function must be evaluated before the loops. So it is not possible to simply stop hoisting non hlfir.elemental operations. Marking calls with an elemental/nonelemental attribute would not allow the pass to be correct if inlining is run before and drops this information, beside, extracting the argument tree that may have been CSE-ed with the rest of the expression evaluation would be a bit combursome. Instead, lower nonelemental calls into a new hlfir.exactly_once operation that will allow retaining the information that the operations contained inside its region must be hoisted. This allows inlining to operate before if desired in order to improve alias analysis. The LowerHLFIROrderedAssignments pass is updated to only hoist the operations contained inside hlfir.exactly_once bodies. | 2 年前 | |
[flang] Do not hoist all scalar sub-expressions from WHERE constructs (#91395) The HLFIR pass lowering WHERE (hlfir.where op) was too aggressive in its hoisting of scalar sub-expressions from LHS/RHS/MASKS outside of the loops generated for the WHERE construct. This violated F'2023 10.2.3.2 point 10 that stipulated that elemental operations must be evaluated only for elements corresponding to true values, because scalar operations are still elemental, and hoisting them is invalid if they could have side effects (e.g, division by zero) and if the MASK is always false (i.e., the loop body is never evaluated). The difficulty is that 10.2.3.2 point 9 mandates that nonelemental function must be evaluated before the loops. So it is not possible to simply stop hoisting non hlfir.elemental operations. Marking calls with an elemental/nonelemental attribute would not allow the pass to be correct if inlining is run before and drops this information, beside, extracting the argument tree that may have been CSE-ed with the rest of the expression evaluation would be a bit combursome. Instead, lower nonelemental calls into a new hlfir.exactly_once operation that will allow retaining the information that the operations contained inside its region must be hoisted. This allows inlining to operate before if desired in order to improve alias analysis. The LowerHLFIROrderedAssignments pass is updated to only hoist the operations contained inside hlfir.exactly_once bodies. | 2 年前 | |
[flang] Do not produce result for void runtime call (#123155) Runtime function call to a void function are producing a ssa value because the FunctionType result is set to NoneType with is later translated to a empty struct. This is not an issue when going to LLVM IR but it breaks when lowering a gpu module to PTX. This patch update the RTModel to correctly set the FunctionType result type to nothing. This is one runtime call before this patch at the LLVM IR dialect step. %45 = llvm.call @_FortranAAssign(%arg0, %1, %44, %4) : (!llvm.ptr, !llvm.ptr, !llvm.ptr, i32) -> !llvm.struct<()> After the patch the call would be correctly formed llvm.call @_FortranAAssign(%arg0, %1, %44, %4) : (!llvm.ptr, !llvm.ptr, !llvm.ptr, i32) -> () Without the patch it would lead to error like: ptxas /tmp/mlir-cuda_device_mod-nvptx64-nvidia-cuda-sm_60-e804b6.ptx, line 10; error : Output parameter cannot be an incomplete array. ptxas /tmp/mlir-cuda_device_mod-nvptx64-nvidia-cuda-sm_60-e804b6.ptx, line 125; error : Call has wrong number of parameters The change is pretty much mechanical. | 1 年前 | |
[flang] Do not produce result for void runtime call (#123155) Runtime function call to a void function are producing a ssa value because the FunctionType result is set to NoneType with is later translated to a empty struct. This is not an issue when going to LLVM IR but it breaks when lowering a gpu module to PTX. This patch update the RTModel to correctly set the FunctionType result type to nothing. This is one runtime call before this patch at the LLVM IR dialect step. %45 = llvm.call @_FortranAAssign(%arg0, %1, %44, %4) : (!llvm.ptr, !llvm.ptr, !llvm.ptr, i32) -> !llvm.struct<()> After the patch the call would be correctly formed llvm.call @_FortranAAssign(%arg0, %1, %44, %4) : (!llvm.ptr, !llvm.ptr, !llvm.ptr, i32) -> () Without the patch it would lead to error like: ptxas /tmp/mlir-cuda_device_mod-nvptx64-nvidia-cuda-sm_60-e804b6.ptx, line 10; error : Output parameter cannot be an incomplete array. ptxas /tmp/mlir-cuda_device_mod-nvptx64-nvidia-cuda-sm_60-e804b6.ptx, line 125; error : Call has wrong number of parameters The change is pretty much mechanical. | 1 年前 | |
[flang][hlfir] Do not reuse hlfir.expr mask when saving RHS. In WHERE and masked FORALL assignment, both the mask and the RHS may need to be saved in some temporary storage before evaluating the assignment. The code was trying to "optimize" that case when evaluating the RHS by not fetching the mask temporary that was just created, but in simple cases of WHERE construct where the evaluated mask is an hlfir.expr, this caused the hlfir.expr to be both used in an hlfir.associate and later in an hlfir.apply to create the fir.if to mask the RHS evaluation. This double usage prevents codegen from inlining the hlfir.expr at the hlfir.apply, and from "moving" the hlfir.expr storage into the temp during hlfir.associate bufferization. So this is pessimizing the code: this would lead to created two mask array temporary storages This was caught by the unexpectedly high number of "not yet implemented: hlfir.associate of hlfir.expr with more than one use" that were firing. Use the mask temporary instead (the hlfir.associate result) when possible. Some temporary (the "inlined stack") do not support fetching and pushing in the same run (a single counter is used to keep track of the fetching and pushing position). Add a canBeFetchedAfterPush() for safety, but this limitation is anyway not relevant for hlfir.expr since the inlined stack is only used to save "trivial" scalars. Also update the temporary storage name to only indicate "forall" if the top level construct is a FORALL. This is not a very precise name, but it should at least give a correct context to indicate in the IR why some temporary array storage was created. Differential Revision: https://reviews.llvm.org/D153880 | 3 年前 | |
[flang] Do not produce result for void runtime call (#123155) Runtime function call to a void function are producing a ssa value because the FunctionType result is set to NoneType with is later translated to a empty struct. This is not an issue when going to LLVM IR but it breaks when lowering a gpu module to PTX. This patch update the RTModel to correctly set the FunctionType result type to nothing. This is one runtime call before this patch at the LLVM IR dialect step. %45 = llvm.call @_FortranAAssign(%arg0, %1, %44, %4) : (!llvm.ptr, !llvm.ptr, !llvm.ptr, i32) -> !llvm.struct<()> After the patch the call would be correctly formed llvm.call @_FortranAAssign(%arg0, %1, %44, %4) : (!llvm.ptr, !llvm.ptr, !llvm.ptr, i32) -> () Without the patch it would lead to error like: ptxas /tmp/mlir-cuda_device_mod-nvptx64-nvidia-cuda-sm_60-e804b6.ptx, line 10; error : Output parameter cannot be an incomplete array. ptxas /tmp/mlir-cuda_device_mod-nvptx64-nvidia-cuda-sm_60-e804b6.ptx, line 125; error : Call has wrong number of parameters The change is pretty much mechanical. | 1 年前 | |
[flang][NFC] use hlfir.declare first result when both results are raw pointers (#132261) Currently, the helpers to get fir::ExtendedValue out of hlfir::Entity use hlfir.declare second result ( #1) in most cases. This is because this result is the same as the input and matches what FIR was getting before lowering to HLFIR. But this creates odd situations when both hlfir.declare are raw pointers and either result ends-up being used in the IR depending on whether the code was generated by a helper using fir::ExtendedValue, or via "pure HLFIR" helpers using the first result. This will typically prevent simple CSE and easy identification that two operation (e.g load/store) are touching the exact same memory location without using alias analysis or "manual detection" (looking for common hlfir.declare defining op). Hence, when hlfir.declare results are both raw pointers, use #0 when producing fir::ExtendedValue. When #0 is a fir.box, keep using #1 because these are not the same. The only code change is in HLFIRTools.cpp and is pretty small, but there is a big test fallout of #1 to #0. | 1 年前 | |
[flang] implement simple pointer assignments inside FORALL (#129522) The semantic of pointer assignments inside FORALL requires evaluating the targets (RHS) and pointer variables (LHS) of all iterations before evaluating the assignments. In practice, if the compiler can prove that the RHS and LHS evaluations are not impacted by the assignments, the evaluation of the FORALL assignment statement can be done in a single loop. However, if the compiler cannot prove this, it needs to "save" the addresses of the targets and/or the pointer descriptors of each iterations before doing the assignments. This patch implements the most common cases where there is no lower bound spec, no bounds remapping, the LHS is not polymorphic, and the RHS is not NULL. The HLFIR operation used to represent assignments inside FORALL can be used for pointer assignments to (the only difference being that the LHS is a descriptor address). The analysis for intrinsic assignment can be reused, with the distinction that the RHS data is not read during the assignment. The logic that is used to save LHS in intrinsic assignments inside FORALL is extracted to be used for the RHS of pointer assignments when needed (saving a descriptor value). Pointer assignment LHS are just descriptor addresses and are saved as int_ptr values. | 1 年前 | |
[flang][hlfir] do not consider local temps as conflicting in assignment (#113330) Last patch required to avoid creating a temporary for the LHS when dealing with x([a,b]) = y. The code dealing with "ordered assignments" (where, forall, user and vector subscripted assignments) is saving the evaluated RHS/LHS and masks if they have write effects because this write effects should not be evaluated when they affect entities that may be written to in other contexts after the evaluation and before the re-evaluation. But when dealing with write to storage allocated in the region for the expression being evluated, there is no problem to re-evaluate the write: it has no effect outside of the expression evaluation that owns the allocation. In the case of x([a,b]) = y, the temporary is created for the vector subscript. Raising the HLFIR abstraction for simple array constructors may be a good idea, but local temps are created in other contexts, so this fix is more generic. | 1 年前 | |
[flang] Undo the effects of CSE for hlfir.exactly_once. (#140190) CSE may delete operations from hlfir.exactly_once and reuse the equivalent results from the parent region(s), e.g. from the parent hlfir.region_assign. This makes it problematic to clone hlfir.exactly_once before the top-level hlfir.where. This patch adds a "canonicalizer" that pulls in such operations back into hlfir.exactly_once. | 1 年前 | |
[flang] Do not hoist all scalar sub-expressions from WHERE constructs (#91395) The HLFIR pass lowering WHERE (hlfir.where op) was too aggressive in its hoisting of scalar sub-expressions from LHS/RHS/MASKS outside of the loops generated for the WHERE construct. This violated F'2023 10.2.3.2 point 10 that stipulated that elemental operations must be evaluated only for elements corresponding to true values, because scalar operations are still elemental, and hoisting them is invalid if they could have side effects (e.g, division by zero) and if the MASK is always false (i.e., the loop body is never evaluated). The difficulty is that 10.2.3.2 point 9 mandates that nonelemental function must be evaluated before the loops. So it is not possible to simply stop hoisting non hlfir.elemental operations. Marking calls with an elemental/nonelemental attribute would not allow the pass to be correct if inlining is run before and drops this information, beside, extracting the argument tree that may have been CSE-ed with the rest of the expression evaluation would be a bit combursome. Instead, lower nonelemental calls into a new hlfir.exactly_once operation that will allow retaining the information that the operations contained inside its region must be hoisted. This allows inlining to operate before if desired in order to improve alias analysis. The LowerHLFIROrderedAssignments pass is updated to only hoist the operations contained inside hlfir.exactly_once bodies. | 2 年前 | |
[flang] Do not hoist all scalar sub-expressions from WHERE constructs (#91395) The HLFIR pass lowering WHERE (hlfir.where op) was too aggressive in its hoisting of scalar sub-expressions from LHS/RHS/MASKS outside of the loops generated for the WHERE construct. This violated F'2023 10.2.3.2 point 10 that stipulated that elemental operations must be evaluated only for elements corresponding to true values, because scalar operations are still elemental, and hoisting them is invalid if they could have side effects (e.g, division by zero) and if the MASK is always false (i.e., the loop body is never evaluated). The difficulty is that 10.2.3.2 point 9 mandates that nonelemental function must be evaluated before the loops. So it is not possible to simply stop hoisting non hlfir.elemental operations. Marking calls with an elemental/nonelemental attribute would not allow the pass to be correct if inlining is run before and drops this information, beside, extracting the argument tree that may have been CSE-ed with the rest of the expression evaluation would be a bit combursome. Instead, lower nonelemental calls into a new hlfir.exactly_once operation that will allow retaining the information that the operations contained inside its region must be hoisted. This allows inlining to operate before if desired in order to improve alias analysis. The LowerHLFIROrderedAssignments pass is updated to only hoist the operations contained inside hlfir.exactly_once bodies. | 2 年前 | |
[flang][hlfir] Enable WHERE scheduling in LowerHLFIROrderedAssignments Nothing special is needed, other than adding the logging code for where masks and to plug the pattern. This tests mainly adds test. Note that some of the justifications to create temps shows some lacks of side effect interface on operations (like hlfir.transpose), or on some transparent llvm intrinsic calls (llvm.stacksave/restore). I think we should as much as possible try to improve this on the ops generate code rather than special casing it here. Differential Revision: https://reviews.llvm.org/D150581 | 3 年前 | |
[flang] Do not hoist all scalar sub-expressions from WHERE constructs (#91395) The HLFIR pass lowering WHERE (hlfir.where op) was too aggressive in its hoisting of scalar sub-expressions from LHS/RHS/MASKS outside of the loops generated for the WHERE construct. This violated F'2023 10.2.3.2 point 10 that stipulated that elemental operations must be evaluated only for elements corresponding to true values, because scalar operations are still elemental, and hoisting them is invalid if they could have side effects (e.g, division by zero) and if the MASK is always false (i.e., the loop body is never evaluated). The difficulty is that 10.2.3.2 point 9 mandates that nonelemental function must be evaluated before the loops. So it is not possible to simply stop hoisting non hlfir.elemental operations. Marking calls with an elemental/nonelemental attribute would not allow the pass to be correct if inlining is run before and drops this information, beside, extracting the argument tree that may have been CSE-ed with the rest of the expression evaluation would be a bit combursome. Instead, lower nonelemental calls into a new hlfir.exactly_once operation that will allow retaining the information that the operations contained inside its region must be hoisted. This allows inlining to operate before if desired in order to improve alias analysis. The LowerHLFIROrderedAssignments pass is updated to only hoist the operations contained inside hlfir.exactly_once bodies. | 2 年前 | |
[flang] optimize array function calls using hlfir.eval_in_mem (#118070) This patch encapsulate array function call lowering into hlfir.eval_in_mem and allows directly evaluating the call into the LHS when possible. The conditions are: LHS is contiguous, not accessed inside the function, it is not a whole allocatable, and the function results needs not to be finalized. All these conditions are tested in the previous hlfir.eval_in_mem optimization (#118069) that is leveraging the extension of getModRef to handle function calls(#117164). This yields a 25% speed-up on polyhedron channel2 benchmark (from 1min to 45s measured on an X86-64 Zen 2). | 1 年前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 3 年前 | ||
| 3 年前 | ||
| 3 年前 | ||
| 1 年前 | ||
| 9 个月前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 9 个月前 | ||
| 1 年前 | ||
| 2 年前 | ||
| 2 年前 | ||
| 2 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 3 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 2 年前 | ||
| 2 年前 | ||
| 3 年前 | ||
| 2 年前 | ||
| 1 年前 |