aboutsummaryrefslogtreecommitdiffstats
path: root/test/Transforms/SLPVectorizer
Commit message (Collapse)AuthorAgeFilesLines
* Update aosp/master LLVM for rebase to r235153Pirama Arumuga Nainar2015-05-187-7/+7
| | | | | Change-Id: I9bf53792f9fc30570e81a8d80d296c681d005ea7 (cherry picked from commit 0c7f116bb6950ef819323d855415b2f2b0aad987)
* Update aosp/master llvm for rebase to r233350Pirama Arumuga Nainar2015-04-0975-1402/+1402
| | | | Change-Id: I07d935f8793ee8ec6b7da003f6483046594bca49
* Update aosp/master LLVM for rebase to r230699.Stephen Hines2015-03-2310-54/+434
| | | | Change-Id: I2b5be30509658cb8266be782de0ab24f9099f9b9
* Update aosp/master LLVM for rebase to r222494.Stephen Hines2014-12-0225-45/+1024
| | | | Change-Id: Ic787f5e0124df789bd26f3f24680f45e678eef2d
* Update LLVM for rebase to r212749.Stephen Hines2014-07-219-10/+567
| | | | | | | Includes a cherry-pick of: r212948 - fixes a small issue with atomic calls Change-Id: Ib97bd980b59f18142a69506400911a6009d9df18
* Update LLVM for 3.5 rebase (r209712).Stephen Hines2014-05-2911-2/+613
| | | | Change-Id: I149556c940fb7dc92d075273c87ff584f400941f
* Update to LLVM 3.5a.Stephen Hines2014-04-249-1/+293
| | | | Change-Id: Ifadecab779f128e62e430c2b4f6ddd84953ed617
* Merging r195504:Manman Ren2013-12-091-1/+2
| | | | | | | | | | | | | | | | ------------------------------------------------------------------------ r195504 | mren | 2013-11-22 13:49:45 -0800 (Fri, 22 Nov 2013) | 6 lines Debug Info: update testing cases to specify the debug info version number. We are going to drop debug info without a version number or with a different version number, to make sure we don't crash when we see bitcode files with different debug info metadata format. ------------------------------------------------------------------------ git-svn-id: https://llvm.org/svn/llvm-project/llvm/branches/release_34@196815 91177308-0d34-0410-b5e6-96231b3b80d8
* Merging r196294:Bill Wendling2013-12-071-0/+30
| | | | | | | | | | | | | | | | | ------------------------------------------------------------------------ r196294 | arnolds | 2013-12-03 08:33:06 -0800 (Tue, 03 Dec 2013) | 7 lines opt: Mirror vectorization presets of clang clang enables vectorization at optimization levels > 1 and size level < 2. opt should behave similarily. Loop vectorization and SLP vectorization can be disabled with the flags -disable-(loop/slp)-vectorization. ------------------------------------------------------------------------ git-svn-id: https://llvm.org/svn/llvm-project/llvm/branches/release_34@196649 91177308-0d34-0410-b5e6-96231b3b80d8
* Merging r196508:Bill Wendling2013-12-061-0/+35
| | | | | | | | | | | | | | | | | | | | | | ------------------------------------------------------------------------ r196508 | arnolds | 2013-12-05 07:14:40 -0800 (Thu, 05 Dec 2013) | 12 lines SLPVectorizer: An in-tree vectorized entry cannot also be a scalar external use We were creating external uses for scalar values in MustGather entries that also had a ScalarToTreeEntry (they also are present in a vectorized tuple). This meant we would keep a value 'alive' as a scalar and vectorized causing havoc. This is not necessary because when we create a MustGather vector we explicitly create external uses entries for the insertelement instructions of the MustGather vector elements. Fixes PR18129. radar://15582184 ------------------------------------------------------------------------ git-svn-id: https://llvm.org/svn/llvm-project/llvm/branches/release_34@196571 91177308-0d34-0410-b5e6-96231b3b80d8
* Merging r195791:Bill Wendling2013-11-271-1/+1
| | | | | | | | | | | | | | ------------------------------------------------------------------------ r195791 | nadav | 2013-11-26 14:24:25 -0800 (Tue, 26 Nov 2013) | 4 lines PR1860 - We can't save a list of ExtractElement instructions to CSE because some of these instructions may be removed and optimized in future iterations. Instead we save a list of basic blocks that we need to CSE. ------------------------------------------------------------------------ git-svn-id: https://llvm.org/svn/llvm-project/llvm/branches/release_34@195818 91177308-0d34-0410-b5e6-96231b3b80d8
* Merging r195773:Bill Wendling2013-11-271-0/+47
| | | | | | | | | | | | | | | | ------------------------------------------------------------------------ r195773 | nadav | 2013-11-26 09:29:19 -0800 (Tue, 26 Nov 2013) | 6 lines PR18060 - When we RAUW values with ExtractElement instructions in some cases we generate PHI nodes with multiple entries from the same basic block but with different values. Enabling CSE on ExtractElement instructions make sure that all of the RAUWed instructions are the same. ------------------------------------------------------------------------ git-svn-id: https://llvm.org/svn/llvm-project/llvm/branches/release_34@195817 91177308-0d34-0410-b5e6-96231b3b80d8
* Add llvm/test/Transforms/SLPVectorizer/ARM/lit.local.cfg. Tests there ↵NAKAMURA Takumi2013-10-291-0/+3
| | | | | | require ARM in targets. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193580 91177308-0d34-0410-b5e6-96231b3b80d8
* ARM cost model: Unaligned vectorized double stores are expensiveArnold Schwaighofer2013-10-291-0/+20
| | | | | | | | | Updated a test case that assumed that <2 x double> would vectorize to use <4 x float>. radar://15338229 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193574 91177308-0d34-0410-b5e6-96231b3b80d8
* ARM cost model: Account for zero cost scalar SROA instructionsArnold Schwaighofer2013-10-291-0/+52
| | | | | | | | | By vectorizing a series of srl, or, ... instructions we have obfuscated the intention so much that the backend does not know how to fold this code away. radar://15336950 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193573 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Don't vectorize volatile memory operationsArnold Schwaighofer2013-10-161-0/+43
| | | | | | | | | | radar://15231682 Reapply r192799, http://lab.llvm.org:8011/builders/lldb-x86_64-debian-clang/builds/8226 showed that the bot is still broken even with this out. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192820 91177308-0d34-0410-b5e6-96231b3b80d8
* Revert "SLPVectorizer: Don't vectorize volatile memory operations"Arnold Schwaighofer2013-10-161-43/+0
| | | | | | This speculatively reverts commit 192799. It might have broken a linux buildbot. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192816 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Don't vectorize volatile memory operationsArnold Schwaighofer2013-10-161-0/+43
| | | | | | radar://15231682 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192799 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Sort PHINodes based on their opcodeArnold Schwaighofer2013-10-121-2/+34
| | | | | | | | | | | | | | Before this patch we relied on the order of phi nodes when we looked for phi nodes of the same type. This could prevent vectorization of cases where there was a phi node of a second type in between phi nodes of some type. This is important for vectorization of an internal graphics kernel. On the test suite + external on x86_64 (and on a run on armv7s) it showed no impact on either performance or compile time. radar://15024459 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192537 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Sort inputs to commutative binary operationsArnold Schwaighofer2013-10-041-0/+234
| | | | | | | | | | | | | | | | | | | | | | | Sort the operands of the other entries in the current vectorization root according to the first entry's operands opcodes. %conv0 = uitofp ... %load0 = load float ... = fmul %conv0, %load0 = fmul %load0, %conv1 = fmul %load0, %conv2 Make sure that we recursively vectorize <%conv0, %conv1, %conv2> and <%load0, %load0, %load0>. This makes it more likely to obtain vectorizable trees. We have to be careful when we sort that we don't destroy 'good' existing ordering implied by source order. radar://15080067 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191977 91177308-0d34-0410-b5e6-96231b3b80d8
* Apply slp vectorization on fully-vectorizable tree of height 2Yi Jiang2013-10-021-17/+130
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191852 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Make store chain finding more aggressive with ↵Benjamin Kramer2013-10-021-0/+21
| | | | | | | | | GetUnderlyingObject. This recursively strips all GEPs like the existing code. It also handles bitcasts and other operations that do not change the pointer value. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191847 91177308-0d34-0410-b5e6-96231b3b80d8
* TBAA: update tbaa format from scalar format to struct-path aware format.Manman Ren2013-09-301-4/+5
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191690 91177308-0d34-0410-b5e6-96231b3b80d8
* TBAA: remove !tbaa from testing cases when they are not needed.Manman Ren2013-09-302-13/+6
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191689 91177308-0d34-0410-b5e6-96231b3b80d8
* IRBuilder: Add RAII objects to reset insertion points or fast math flags.Benjamin Kramer2013-09-301-1/+1
| | | | | | | | Inspired by the object from the SLPVectorizer. This found a minor bug in the debug loc restoration in the vectorizer where the location of a following instruction was attached instead of the location from the original instruction. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191673 91177308-0d34-0410-b5e6-96231b3b80d8
* Fix SLPVectorizer using wrong address space for load/storeMatt Arsenault2013-09-272-0/+69
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191564 91177308-0d34-0410-b5e6-96231b3b80d8
* Transforms: Use getFirstNonPHI to set the insertion point for PHIsJustin Bogner2013-09-271-0/+31
| | | | | | | | | | We were previously using getFirstInsertionPt to insert PHI instructions when vectorizing, but getFirstInsertionPt also skips past landingpads, causing this to generate invalid IR. We can avoid this issue by using getFirstNonPHI instead. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191526 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorize: Put horizontal reductions feeding a store under separate flagArnold Schwaighofer2013-09-251-8/+10
| | | | | | | Put them under a separate flag for experimentation. They are more likely to interfere with loop vectorization which happens later in the pass pipeline. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191371 91177308-0d34-0410-b5e6-96231b3b80d8
* Test case for r191314. Yi Jiang2013-09-241-0/+27
| | | | | | | | Some supplemental information for r191314: We would like to make sure SLP Vectorizer will not try to vectorize tiny trees even with a negative threshold so we set the cost to INT_MAX. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191327 91177308-0d34-0410-b5e6-96231b3b80d8
* Reapply "SLPVectorizer: Handle more horizontal reductions (disabled)""Arnold Schwaighofer2013-09-211-0/+415
| | | | | | | | | | | | | | Reapply r191108 with a fix for a memory corruption error I introduced. Of course, we can't reference the scalars that we replace by vectorizing and then call their eraseFromParent method. I only 'needed' the scalars to get the DebugLoc. Just store the DebugLoc before actually vectorizing instead. As a nice side effect, this also simplifies the interface between BoUpSLP and the HorizontalReduction class to returning a value pointer (the vectorized tree root). radar://14607682 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191123 91177308-0d34-0410-b5e6-96231b3b80d8
* Revert "SLPVectorizer: Handle more horizontal reductions (disabled)"Arnold Schwaighofer2013-09-211-415/+0
| | | | | | | | | This reverts commit r191108. The horizontal.ll test case fails under libgmalloc. Thanks Shuxin for pointing this out to me. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191121 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Handle more horizontal reductions (disabled)Arnold Schwaighofer2013-09-201-0/+415
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Match reductions starting at binary operation feeding into a phi. The code handles trees like r += v1 + v2 + v3 ... and r += v1 r += v2 ... and r *= v1 + v2 + ... We currently only handle associative operations (add, fadd fast). The code can now also handle reductions feeding into stores. a[i] = v1 + v2 + v3 + ... The code is currently disabled behind the flag "-slp-vectorize-hor". The cost model for most architectures is not there yet. I found one opportunity of a horizontal reduction feeding a phi in TSVC (LoopRerolling-flt) and there are several opportunities where reductions feed into stores. radar://14607682 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191108 91177308-0d34-0410-b5e6-96231b3b80d8
* Name the XCore target-specific subdirectories canonically.Chandler Carruth2013-09-182-0/+0
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190940 91177308-0d34-0410-b5e6-96231b3b80d8
* A couple of tests, in llvm/test/Transforms/*/xcore, are XCore-specific. They ↵NAKAMURA Takumi2013-09-181-0/+3
| | | | | | should be excluded when XCore is not built. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190938 91177308-0d34-0410-b5e6-96231b3b80d8
* Prevent LoopVectorizer and SLPVectorizer running if the target has no vector ↵Robert Lytton2013-09-181-0/+24
| | | | | | | | | | registers. XCore target: Add XCoreTargetTransformInfo This is where getNumberOfRegisters() resides, which in turn returns the number of vector registers (=0). git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190936 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Don't vectorize phi nodes that use invoke valuesArnold Schwaighofer2013-09-171-0/+62
| | | | | | | | | We can't insert an insertelement after an invoke. We would have to split a critical edge. So when we see a phi node that uses an invoke we just give up. radar://14990770 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190871 91177308-0d34-0410-b5e6-96231b3b80d8
* Debug Info Testing: updated to use NULL instead of "i32 0" in a few fields.Manman Ren2013-09-061-1/+1
| | | | | | | | | Field 2 of DIType (Context), field 9 of DIDerivedType (TypeDerivedFrom), field 12 of DICompositeType (ContainingType), fields 2, 7, 12 of DISubprogram (Context, Type, ContainingType). git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190205 91177308-0d34-0410-b5e6-96231b3b80d8
* In this patch we are trying to do two things:Yi Jiang2013-09-033-13/+141
| | | | | | | | | 1) If the width of vectorization list candidate is bigger than vector reg width, we will break it down to fit the vector reg. 2) We do not vectorize the width which is not power of two. The performance result shows it will help some spec benchmarks. mesa improved 6.97% and ammp improved 1.54%. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189830 91177308-0d34-0410-b5e6-96231b3b80d8
* Fix inserting instructions before last in bundle.Matt Arsenault2013-08-261-1/+1
| | | | | | | | | | | The builder inserts from before the insert point, not after, so this would insert before the last instruction in the bundle instead of after it. I'm not sure if this can actually be a problem with any of the current insertions. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189285 91177308-0d34-0410-b5e6-96231b3b80d8
* Debug Info: add an identifier field to DICompositeType.Manman Ren2013-08-261-1/+1
| | | | | | | | | | | | | | | | | | | DICompositeType will have an identifier field at position 14. For now, the field is set to null in DIBuilder. For DICompositeTypes where the template argument field (the 13th field) was optional, modify DIBuilder to make sure the template argument field is set. Now DICompositeType has 15 fields. Update DIBuilder to use NULL instead of "i32 0" for null value of a MDNode. Update verifier to check that DICompositeType has 15 fields and the last field is null or a MDString. Update testing cases to include an extra field for DICompositeType. The identifier field will be used by type uniquing so a front end can genearte a DICompositeType with a unique identifer. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189282 91177308-0d34-0410-b5e6-96231b3b80d8
* Forgot to add slp threshold to testMatt Arsenault2013-08-261-1/+2
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189248 91177308-0d34-0410-b5e6-96231b3b80d8
* Vectorize starting from insertelements building a vectorMatt Arsenault2013-08-261-0/+196
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189233 91177308-0d34-0410-b5e6-96231b3b80d8
* [Debug Info Tests] Update testing cases.Manman Ren2013-08-221-6/+6
| | | | | | | | | | A single metadata will not span multiple lines. This also helps me with my script to automatic update the testing cases. A debug info testing case should have a llvm.dbg.cu. Do not use hard-coded id for debug nodes. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189033 91177308-0d34-0410-b5e6-96231b3b80d8
* Teach the SLP vectorizer the correct way to check for consecutive accessChandler Carruth2013-08-221-1/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | using GEPs. Previously, it used a number of different heuristics for analyzing the GEPs. Several of these were conservatively correct, but failed to fall back to SCEV even when SCEV might have given a reasonable answer. One was simply incorrect in how it was formulated. There was good code already to recursively evaluate the constant offsets in GEPs, look through pointer casts, etc. I gathered this into a form code like the SLP code can use in a previous commit, which allows all of this code to become quite simple. There is some performance (compile time) concern here at first glance as we're directly attempting to walk both pointers constant GEP chains. However, a couple of thoughts: 1) The very common cases where there is a dynamic pointer, and a second pointer at a constant offset (usually a stride) from it, this code will actually not do any unnecessary work. 2) InstCombine and other passes work very hard to collapse constant GEPs, so it will be rare that we iterate here for a long time. That said, if there remain performance problems here, there are some obvious things that can improve the situation immensely. Doing a vectorizer-pass-wide memoizer for each individual layer of pointer values, their base values, and the constant offset is likely to be able to completely remove redundant work and strictly limit the scaling of the work to scrape these GEPs. Since this optimization was not done on the prior version (which would still benefit from it), I've not done it here. But if folks have benchmarks that slow down it should be straight forward for them to add. I've added a test case, but I'm not really confident of the amount of testing done for different access patterns, strides, and pointer manipulation. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189007 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Fix invalid iterator errorsArnold Schwaighofer2013-08-201-0/+30
| | | | | | | | | | | Update iterator when the SLP vectorizer changes the instructions in the basic block by restarting the traversal of the basic block. Patch by Yi Jiang! Fixes PR 16899. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188832 91177308-0d34-0410-b5e6-96231b3b80d8
* [tests] Cleanup initialization of test suffixes.Daniel Dunbar2013-08-162-3/+0
| | | | | | | | | | | | | | | | | - Instead of setting the suffixes in a bunch of places, just set one master list in the top-level config. We now only modify the suffix list in a few suites that have one particular unique suffix (.ml, .mc, .yaml, .td, .py). - Aside from removing the need for a bunch of lit.local.cfg files, this enables 4 tests that were inadvertently being skipped (one in Transforms/BranchFolding, a .s file each in DebugInfo/AArch64 and CodeGen/PowerPC, and one in CodeGen/SI which is now failing and has been XFAILED). - This commit also fixes a bunch of config files to use config.root instead of older copy-pasted code. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188513 91177308-0d34-0410-b5e6-96231b3b80d8
* Fix PR16797 - Support PHINodes with multiple inputs from the same basic block.Nadav Rotem2013-08-121-0/+41
| | | | | | | | | Do not generate new vector values for the same entries because we know that the incoming values from the same block must be identical. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188185 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorizer: Fix PR16777. PHInodes may use multiple extracted values that ↵Nadav Rotem2013-08-021-0/+35
| | | | | | | | | | come from different blocks. Thanks Alexey Samsonov. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187663 91177308-0d34-0410-b5e6-96231b3b80d8
* Add the C source code to the test to make it easier to update when debug ↵Nadav Rotem2013-07-291-0/+9
| | | | | | | | | | info changes. Thanks Eric. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187368 91177308-0d34-0410-b5e6-96231b3b80d8
* SLPVectorier: update the debug location for the new instructions.Nadav Rotem2013-07-291-0/+82
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187363 91177308-0d34-0410-b5e6-96231b3b80d8
generated by cgit v1.1 at 2024-06-03 19:18:12 +0000