aboutsummaryrefslogtreecommitdiffstats
path: root/lib/Transforms/Vectorize/LoopVectorize.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Fix PR14547. Handle induction variables of small sizes smaller than i32 (i8 ↵Nadav Rotem2013-01-131-0/+3
| | | | | | and i16). git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172348 91177308-0d34-0410-b5e6-96231b3b80d8
* ARM Cost Model: We need to detect the max bitwidth of types in the loop in ↵Nadav Rotem2013-01-111-6/+12
| | | | | | | | | | | | | order to select the max vectorization factor. We don't have a detailed analysis on which values are vectorized and which stay scalars in the vectorized loop so we use another method. We look at reduction variables, loads and stores, which are the only ways to get information in and out of loop iterations. If the data types are extended and truncated then the cost model will catch the cost of the vector zext/sext/trunc operations. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172178 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorizer: Fix a bug in the vectorization of BinaryOperators. The ↵Nadav Rotem2013-01-101-4/+4
| | | | | | | | | | BinaryOperator can be folded to an Undef, and we don't want to set NSW flags to undef vals. PR14878 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172079 91177308-0d34-0410-b5e6-96231b3b80d8
* ARM Cost model: Use the size of vector registers and widest vectorizable ↵Nadav Rotem2013-01-091-3/+48
| | | | | | instruction to determine the max vectorization factor. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@172010 91177308-0d34-0410-b5e6-96231b3b80d8
* Cost Model: Move the 'max unroll factor' variable to the TTI and add initial ↵Nadav Rotem2013-01-091-3/+2
| | | | | | Cost Model support on ARM. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171928 91177308-0d34-0410-b5e6-96231b3b80d8
* Code cleanup: refactor the switch statements in the generation of reduction ↵Nadav Rotem2013-01-081-59/+28
| | | | | | variables into an IR builder call. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171871 91177308-0d34-0410-b5e6-96231b3b80d8
* Rename the enum members to match the LLVM coding style.Nadav Rotem2013-01-081-25/+25
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171868 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorizer: Add support for floating point reductionsNadav Rotem2013-01-071-58/+91
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171812 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorizer: When we vectorizer and widen loops we process many elements ↵Nadav Rotem2013-01-071-3/+15
| | | | | | | | | | | at once. This is a good thing, except for small loops. On small loops post-loop that handles scalars (and runs slower) can take more time to execute than the rest of the loop. This patch disables widening of loops with a small static trip count. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171798 91177308-0d34-0410-b5e6-96231b3b80d8
* Simplify LoopVectorize to require target transform info and rely on itChandler Carruth2013-01-071-55/+43
| | | | | | | | | being present. Make a member of one of the helper classes a reference as part of this. Reformatting goodness brought to you by clang-format. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171726 91177308-0d34-0410-b5e6-96231b3b80d8
* Merge the unused header file for LoopVectorizer into the source file.Chandler Carruth2013-01-071-3/+519
| | | | | | | | | | | | | | This makes the loop vectorizer match the pattern followed by roughly all other passses. =] Notably, this header file was braken in several regards: it contained a using namespace directive, global #define's that aren't globaly appropriate, and global constants defined directly in the header file. As a side benefit, lots of the types in this file become internal, which will cause the optimizer to chew on this pass more effectively. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171723 91177308-0d34-0410-b5e6-96231b3b80d8
* Fix the enumerator names for ShuffleKind to match tho coding standards,Chandler Carruth2013-01-071-2/+2
| | | | | | and make its comments doxygen comments. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171688 91177308-0d34-0410-b5e6-96231b3b80d8
* Move TargetTransformInfo to live under the Analysis library. This noChandler Carruth2013-01-071-1/+1
| | | | | | | longer would violate any dependency layering and it is in fact an analysis. =] git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171686 91177308-0d34-0410-b5e6-96231b3b80d8
* Switch the loop vectorizer from VTTI to just use TTI directly.Chandler Carruth2013-01-051-49/+45
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171620 91177308-0d34-0410-b5e6-96231b3b80d8
* iLoopVectorize: Non commutative operators can be used as reduction variables ↵Nadav Rotem2013-01-051-4/+13
| | | | | | | | | | as long as the reduction chain is used in the LHS. PR14803. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171583 91177308-0d34-0410-b5e6-96231b3b80d8
* Do not vectorize loops with subtraction reductionsPaul Redmond2013-01-041-1/+0
| | | | | | | | | | Since subtraction does not commute the loop vectorizer incorrectly vectorizes reductions such as x = A[i] - x. Disabling for now. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171537 91177308-0d34-0410-b5e6-96231b3b80d8
* Fix a warningNadav Rotem2013-01-041-2/+2
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171525 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorizer:Nadav Rotem2013-01-041-4/+162
| | | | | | | | | | 1. Add code to estimate register pressure. 2. Add code to select the unroll factor based on register pressure. 3. Add bits to TargetTransformInfo to provide the number of registers. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171469 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorizer: Add support for loop-unrolling during vectorization for ↵Nadav Rotem2013-01-031-155/+267
| | | | | | increasing the ILP. At the moment this feature is disabled by default and this commit should not cause any functional changes. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171436 91177308-0d34-0410-b5e6-96231b3b80d8
* Avoid vectorization when the function has the "noimplicitflot" attribute.Nadav Rotem2013-01-021-3/+11
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171429 91177308-0d34-0410-b5e6-96231b3b80d8
* Move all of the header files which are involved in modelling the LLVM IRChandler Carruth2013-01-021-10/+10
| | | | | | | | | | | | | | | | | | | | | into their new header subdirectory: include/llvm/IR. This matches the directory structure of lib, and begins to correct a long standing point of file layout clutter in LLVM. There are still more header files to move here, but I wanted to handle them in separate commits to make tracking what files make sense at each layer easier. The only really questionable files here are the target intrinsic tablegen files. But that's a battle I'd rather not fight today. I've updated both CMake and Makefile build systems (I think, and my tests think, but I may have missed something). I've also re-sorted the includes throughout the project. I'll be committing updates to Clang, DragonEgg, and Polly momentarily. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171366 91177308-0d34-0410-b5e6-96231b3b80d8
* Add IRBuilder::CreateVectorSplat and use it to simplify code.Benjamin Kramer2013-01-011-12/+1
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171349 91177308-0d34-0410-b5e6-96231b3b80d8
* Remove the Function::getFnAttributes method in favor of using the AttributeSetBill Wendling2012-12-301-1/+2
| | | | | | | | | | directly. This is in preparation for removing the use of the 'Attribute' class as a collection of attributes. That will shift to the AttributeSet class instead. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171253 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorizer: Fix a bug in the code that updates the loop exiting block.Nadav Rotem2012-12-301-0/+13
| | | | | | | | | | | LCSSA PHIs may have undef values. The vectorizer updates values that are used by outside users such as PHIs. The bug happened because undefs are not loop values. This patch handles these PHIs. PR14725 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171251 91177308-0d34-0410-b5e6-96231b3b80d8
* If all of the write objects are identified then we can vectorize the loop ↵Nadav Rotem2012-12-261-1/+5
| | | | | | | | | | even if the read objects are unidentified. PR14719. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171124 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorizer: Optimize the vectorization of consecutive memory access when ↵Nadav Rotem2012-12-261-22/+63
| | | | | | the iteration step is -1 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171114 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorize: Enable vectorization of the fmuladd intrinsicHal Finkel2012-12-251-0/+1
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171076 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorizer: When checking for vectorizable types, also checkNadav Rotem2012-12-241-1/+8
| | | | | | | | | | the StoreInst operands. PR14705. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171023 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorizer: Fix an endless loop in the code that looks for reductions.Nadav Rotem2012-12-241-7/+8
| | | | | | | | | | The bug was in the code that detects PHIs in if-then-else block sequence. PR14701. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171008 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorize: Fix accidentaly inverted condition.Benjamin Kramer2012-12-231-1/+1
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171001 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorize: For scalars and void types there is no need to compute vector ↵Benjamin Kramer2012-12-231-12/+10
| | | | | | | | insert/extract costs. Fixes an assert during the build of oggenc in the test suite. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@171000 91177308-0d34-0410-b5e6-96231b3b80d8
* Loop Vectorizer: Update the cost model of scatter/gather operations and makeNadav Rotem2012-12-231-13/+29
| | | | | | | | them more expensive. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170995 91177308-0d34-0410-b5e6-96231b3b80d8
* Change 'AttrVal' to 'AttrKind' to better reflect that it's a kind of ↵Bill Wendling2012-12-221-1/+1
| | | | | | attribute instead of the value of the attribute. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170972 91177308-0d34-0410-b5e6-96231b3b80d8
* Remove duplicate includes.Roman Divacky2012-12-211-1/+0
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170902 91177308-0d34-0410-b5e6-96231b3b80d8
* Enable if-conversion.Nadav Rotem2012-12-211-1/+1
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170841 91177308-0d34-0410-b5e6-96231b3b80d8
* Fix a bug in the code that checks if we can vectorize loops while using dynamicNadav Rotem2012-12-211-19/+24
| | | | | | | | | | | | memory bound checks. Before the fix we were able to vectorize this loop from the Livermore Loops benchmark: for ( k=1 ; k<n ; k++ ) x[k] = x[k-1] + y[k]; git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170811 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorize: Fix a bug in the scalarization of instructions.Nadav Rotem2012-12-201-1/+1
| | | | | | | | | | | | Before if-conversion we could check if a value is loop invariant if it was declared inside the basic block. Now that loops have multiple blocks this check is incorrect. This fixes External/SPEC/CINT95/099_go/099_go git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170756 91177308-0d34-0410-b5e6-96231b3b80d8
* Loop Vectorizer: turn-off if-conversion.Nadav Rotem2012-12-201-1/+1
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170708 91177308-0d34-0410-b5e6-96231b3b80d8
* Loop Vectorizer: Enable if-conversion.Nadav Rotem2012-12-201-1/+1
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170632 91177308-0d34-0410-b5e6-96231b3b80d8
* Rename the 'Attributes' class to 'Attribute'. It's going to represent a ↵Bill Wendling2012-12-191-1/+1
| | | | | | single attribute in the future. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170502 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorize: Emit reductions as log2(vectorsize) shuffles + vector ops ↵Benjamin Kramer2012-12-181-12/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | instead of scalar operations. For example on x86 with SSE4.2 a <8 x i8> add reduction becomes movdqa %xmm0, %xmm1 movhlps %xmm1, %xmm1 ## xmm1 = xmm1[1,1] paddw %xmm0, %xmm1 pshufd $1, %xmm1, %xmm0 ## xmm0 = xmm1[1,0,0,0] paddw %xmm1, %xmm0 phaddw %xmm0, %xmm0 pextrb $0, %xmm0, %edx instead of pextrb $2, %xmm0, %esi pextrb $0, %xmm0, %edx addb %sil, %dl pextrb $4, %xmm0, %esi addb %dl, %sil pextrb $6, %xmm0, %edx addb %sil, %dl pextrb $8, %xmm0, %esi addb %dl, %sil pextrb $10, %xmm0, %edi pextrb $14, %xmm0, %edx addb %sil, %dil pextrb $12, %xmm0, %esi addb %dil, %sil addb %sil, %dl git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170439 91177308-0d34-0410-b5e6-96231b3b80d8
* Enable the Loop Vectorizer by default for O2 and O3. Disable if-conversion ↵Nadav Rotem2012-12-131-1/+1
| | | | | | by default. I plan to revert this patch later today. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170157 91177308-0d34-0410-b5e6-96231b3b80d8
* Teach the cost model about the optimization in r169904: Truncation of ↵Nadav Rotem2012-12-131-2/+17
| | | | | | induction variables costs the same as scalar trunc. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170051 91177308-0d34-0410-b5e6-96231b3b80d8
* Fix indentation.Nadav Rotem2012-12-121-4/+3
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170005 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorizer: Use the "optsize" attribute to decide if we are allowed to ↵Nadav Rotem2012-12-121-6/+11
| | | | | | increase the function size. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@170004 91177308-0d34-0410-b5e6-96231b3b80d8
* Fix the ascii drawing that was ruined when I split the H and CPPNadav Rotem2012-12-121-10/+10
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169955 91177308-0d34-0410-b5e6-96231b3b80d8
* fix a typo.Nadav Rotem2012-12-121-1/+1
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169953 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorizer: When -Os is used, vectorize only loops that dont require a ↵Nadav Rotem2012-12-121-22/+60
| | | | | | tail loop. There is no testcase because I dont know of a way to initialize the loop vectorizer pass without adding an additional hidden flag. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169950 91177308-0d34-0410-b5e6-96231b3b80d8
* PR14574. Fix a bug in the code that calculates the mask the converted PHIs ↵Nadav Rotem2012-12-111-1/+1
| | | | | | in if-conversion. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169916 91177308-0d34-0410-b5e6-96231b3b80d8
* Loop Vectorize: optimize the vectorization of trunc(induction_var). The ↵Nadav Rotem2012-12-111-1/+13
| | | | | | truncation is now done on scalars. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@169904 91177308-0d34-0410-b5e6-96231b3b80d8
generated by cgit v1.1 at 2025-07-06 17:38:11 +0000