aboutsummaryrefslogtreecommitdiffstats
path: root/test/Transforms/LoopVectorize
Commit message (Collapse)AuthorAgeFilesLines
* Update aosp/master LLVM for rebase to r235153Pirama Arumuga Nainar2015-05-182-2/+2
| | | | | Change-Id: I9bf53792f9fc30570e81a8d80d296c681d005ea7 (cherry picked from commit 0c7f116bb6950ef819323d855415b2f2b0aad987)
* Update aosp/master llvm for rebase to r233350Pirama Arumuga Nainar2015-04-09130-1616/+1865
| | | | Change-Id: I07d935f8793ee8ec6b7da003f6483046594bca49
* Update aosp/master LLVM for rebase to r230699.Stephen Hines2015-03-2340-473/+1207
| | | | Change-Id: I2b5be30509658cb8266be782de0ab24f9099f9b9
* Update aosp/master LLVM for rebase to r222494.Stephen Hines2014-12-0298-174/+945
| | | | Change-Id: Ic787f5e0124df789bd26f3f24680f45e678eef2d
* Update LLVM for rebase to r212749.Stephen Hines2014-07-2122-33/+558
| | | | | | | Includes a cherry-pick of: r212948 - fixes a small issue with atomic calls Change-Id: Ib97bd980b59f18142a69506400911a6009d9df18
* Update LLVM for 3.5 rebase (r209712).Stephen Hines2014-05-2911-22/+487
| | | | Change-Id: I149556c940fb7dc92d075273c87ff584f400941f
* Update to LLVM 3.5a.Stephen Hines2014-04-2424-28/+886
| | | | Change-Id: Ifadecab779f128e62e430c2b4f6ddd84953ed617
* Merging r197449:Bill Wendling2013-12-171-0/+63
| | | | | | | | | | | | | | | | | ------------------------------------------------------------------------ r197449 | arnolds | 2013-12-16 17:11:01 -0800 (Mon, 16 Dec 2013) | 7 lines LoopVectorizer: Don't if-convert constant expressions that can trap A phi node operand or an instruction operand could be a constant expression that can trap (division). Check that we don't vectorize such cases. PR16729 radar://15653590 ------------------------------------------------------------------------ git-svn-id: https://llvm.org/svn/llvm-project/llvm/branches/release_34@197453 91177308-0d34-0410-b5e6-96231b3b80d8
* Merging r195504:Manman Ren2013-12-092-1/+4
| | | | | | | | | | | | | | | | ------------------------------------------------------------------------ r195504 | mren | 2013-11-22 13:49:45 -0800 (Fri, 22 Nov 2013) | 6 lines Debug Info: update testing cases to specify the debug info version number. We are going to drop debug info without a version number or with a different version number, to make sure we don't crash when we see bitcode files with different debug info metadata format. ------------------------------------------------------------------------ git-svn-id: https://llvm.org/svn/llvm-project/llvm/branches/release_34@196815 91177308-0d34-0410-b5e6-96231b3b80d8
* Merging r196294:Bill Wendling2013-12-071-0/+28
| | | | | | | | | | | | | | | | | ------------------------------------------------------------------------ r196294 | arnolds | 2013-12-03 08:33:06 -0800 (Tue, 03 Dec 2013) | 7 lines opt: Mirror vectorization presets of clang clang enables vectorization at optimization levels > 1 and size level < 2. opt should behave similarily. Loop vectorization and SLP vectorization can be disabled with the flags -disable-(loop/slp)-vectorization. ------------------------------------------------------------------------ git-svn-id: https://llvm.org/svn/llvm-project/llvm/branches/release_34@196649 91177308-0d34-0410-b5e6-96231b3b80d8
* Merging r195787:Bill Wendling2013-12-011-0/+39
| | | | | | | | | | | | | | | | | | ------------------------------------------------------------------------ r195787 | arnolds | 2013-11-26 14:11:23 -0800 (Tue, 26 Nov 2013) | 8 lines LoopVectorizer: Truncate i64 trip counts of i32 phis if necessary In signed arithmetic we could end up with an i64 trip count for an i32 phi. Because it is signed arithmetic we know that this is only defined if the i32 does not wrap. It is therefore safe to truncate the i64 trip count to a i32 value. Fixes PR18049. ------------------------------------------------------------------------ git-svn-id: https://llvm.org/svn/llvm-project/llvm/branches/release_34@195991 91177308-0d34-0410-b5e6-96231b3b80d8
* Merging r195162:Bill Wendling2013-11-201-0/+33
| | | | | | | | | | | | | | | | | | | | | | ------------------------------------------------------------------------ r195162 | arnolds | 2013-11-19 14:20:20 -0800 (Tue, 19 Nov 2013) | 12 lines SLPVectorizer: Fix stale for Value pointer array We are slicing an array of Value pointers and process those slices in a loop. The problem is that we might invalidate a later slice by vectorizing a former slice. Use a WeakVH to track the pointer. If the pointer is deleted or RAUW'ed we can tell. The test case will only fail when running with libgmalloc. radar://15498655 ------------------------------------------------------------------------ git-svn-id: https://llvm.org/svn/llvm-project/llvm/branches/release_34@195222 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorizer: Extend the induction variable to a larger typeArnold Schwaighofer2013-11-181-0/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | In some case the loop exit count computation can overflow. Extend the type to prevent most of those cases. The problem is loops like: int main () { int a = 1; char b = 0; lbl: a &= 4; b--; if (b) goto lbl; return a; } The backedge count is 255. The induction variable type is i8. If we add one to 255 to get the exit count we overflow to zero. To work around this issue we extend the type of the induction variable to i32 in the case of i8 and i16. PR17532 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@195008 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorizer: Use abi alignment for accesses with no alignmentArnold Schwaighofer2013-11-151-0/+33
| | | | | | | | | | | When we vectorize a scalar access with no alignment specified, we have to set the target's abi alignment of the scalar access on the vectorized access. Using the same alignment of zero would be wrong because most targets will have a bigger abi alignment for vector types. This probably fixes PR17878. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194876 91177308-0d34-0410-b5e6-96231b3b80d8
* Scalarize select vector arguments when extracted.Matt Arsenault2013-11-041-29/+29
| | | | | | | | When the elements are extracted from a select on vectors or a vector select, do the select on the extracted scalars from the input if there is only one use. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@194013 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorizer: Perform redundancy elimination on induction variablesArnold Schwaighofer2013-11-013-5/+42
| | | | | | | | | | | | | | | | | | | | When the loop vectorizer was part of the SCC inliner pass manager gvn would run after the loop vectorizer followed by instcombine. This way redundancy (multiple uses) were removed and instcombine could perform scalarization on the induction variables. Having moved the loop vectorizer to later we no longer run any form of redundancy elimination before we perform instcombine. This caused vectorized induction variables to survive that did not before. On a recent iMac this helps linpack back from 6000Mflops to 7000Mflops. This should also help lpbench and paq8p. I ran a Release (without Asserts) build over the test-suite and did not see any negative impact on compile time. radar://15339680 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193891 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorize: Look for consecutive acces in GEPs with trailing zero indicesBenjamin Kramer2013-11-011-0/+42
| | | | | | | If we have a pointer to a single-element struct we can still build wide loads and stores to it (if there is no padding). git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193860 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorizer: If dependency checks fail try runtime checksArnold Schwaighofer2013-11-011-0/+28
| | | | | | | | | | | | When a dependence check fails we can still try to vectorize loops with runtime array bounds checks. This helps linpack to vectorize a loop in dgefa. And we are back to 2x of the scalar performance on a corei7-avx. radar://15339680 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193853 91177308-0d34-0410-b5e6-96231b3b80d8
* ARM cost model: Unaligned vectorized double stores are expensiveArnold Schwaighofer2013-10-291-9/+9
| | | | | | | | | Updated a test case that assumed that <2 x double> would vectorize to use <4 x float>. radar://15338229 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193574 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorizer: Don't attempt to vectorize extractelement instructionsHal Finkel2013-10-251-0/+35
| | | | | | | | | | | | | | | The loop vectorizer does not currently understand how to vectorize extractelement instructions. The existing check, which excluded all vector-valued instructions, did not catch extractelement instructions because it checked only the return value. As a result, vectorization would proceed, producing illegal instructions like this: %58 = extractelement <2 x i32> %15, i32 0 %59 = extractelement i32 %58, i32 0 where the second extractelement is illegal because its first operand is not a vector. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193434 91177308-0d34-0410-b5e6-96231b3b80d8
* I had to move and removeRenato Golin2013-10-241-46/+0
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193355 91177308-0d34-0410-b5e6-96231b3b80d8
* Fix broken builds by moving test to x86 dirRenato Golin2013-10-241-0/+46
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193351 91177308-0d34-0410-b5e6-96231b3b80d8
* Mark vector loops as already vectorizedRenato Golin2013-10-242-2/+49
| | | | | | | | Make sure we mark all loops (scalar and vector) when vectorizing, so that we don't try to vectorize them anymore. Also, set unroll to 1, since this is what we check for on early exit. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193349 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorize: External uses must use the last value in a reduction cycleArnold Schwaighofer2013-10-071-0/+27
| | | | | | | | | Otherwise, we don't perform operations that would have been performed on the scalar version. Fixes PR17498. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@192133 91177308-0d34-0410-b5e6-96231b3b80d8
* Don't use runtime bounds check between address spaces.Matt Arsenault2013-10-022-0/+377
| | | | | | | | | Don't vectorize with a runtime check if it requires a comparison between pointers with different address spaces. The values can't be assumed to be directly comparable. Previously it would create an illegal bitcast. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191862 91177308-0d34-0410-b5e6-96231b3b80d8
* Fix missing CHECK-LABELsMatt Arsenault2013-10-0217-33/+33
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191853 91177308-0d34-0410-b5e6-96231b3b80d8
* TBAA: update tbaa format from scalar format to struct-path aware format.Manman Ren2013-09-301-10/+11
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191690 91177308-0d34-0410-b5e6-96231b3b80d8
* TBAA: remove !tbaa from testing cases when they are not needed.Manman Ren2013-09-302-9/+3
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191689 91177308-0d34-0410-b5e6-96231b3b80d8
* Revert "LoopVectorizer: Only allow vectorization of intrinsics."Arnold Schwaighofer2013-09-231-20/+53
| | | | | | | | | | | | | | Revert 191122 - with extra checks we are allowed to vectorize math library function calls. Standard library indentifiers are reserved names so functions with external linkage must not overrided them. However, functions with internal linkage can. Therefore, we can vectorize calls to math library functions with a check for external linkage and matching signature. This matches what we do during SelectionDAG building. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191206 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorizer: Only allow vectorization of intrinsics. We can't know for ↵Nadav Rotem2013-09-211-3/+28
| | | | | | | | | | sure that the functions 'abs' or 'round' are the functions from libm. rdar://15012650 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@191122 91177308-0d34-0410-b5e6-96231b3b80d8
* Name the XCore target-specific subdirectories canonically.Chandler Carruth2013-09-182-0/+0
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190940 91177308-0d34-0410-b5e6-96231b3b80d8
* A couple of tests, in llvm/test/Transforms/*/xcore, are XCore-specific. They ↵NAKAMURA Takumi2013-09-181-0/+3
| | | | | | should be excluded when XCore is not built. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190938 91177308-0d34-0410-b5e6-96231b3b80d8
* Prevent LoopVectorizer and SLPVectorizer running if the target has no vector ↵Robert Lytton2013-09-181-0/+23
| | | | | | | | | | registers. XCore target: Add XCoreTargetTransformInfo This is where getNumberOfRegisters() resides, which in turn returns the number of vector registers (=0). git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190936 91177308-0d34-0410-b5e6-96231b3b80d8
* Don't vectorize if there are outside loop users of the induction variable.Arnold Schwaighofer2013-09-161-0/+30
| | | | | | | | | | | | We would have to compute the pre increment value, either by computing it on every loop iteration or by splitting the edge out of the loop and inserting a computation for it there. For now, just give up vectorizing such loops. Fixes PR17179. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190790 91177308-0d34-0410-b5e6-96231b3b80d8
* Fix missing CHECK-LABELsMatt Arsenault2013-09-101-5/+5
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190426 91177308-0d34-0410-b5e6-96231b3b80d8
* Debug Info Testing: update context from empty string to null.Manman Ren2013-09-081-2/+2
| | | | | | | Context should be either null or MDNode. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190267 91177308-0d34-0410-b5e6-96231b3b80d8
* Debug Info Testing: updated to use NULL instead of "i32 0" in a few fields.Manman Ren2013-09-062-3/+3
| | | | | | | | | Field 2 of DIType (Context), field 9 of DIDerivedType (TypeDerivedFrom), field 12 of DICompositeType (ContainingType), fields 2, 7, 12 of DISubprogram (Context, Type, ContainingType). git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190205 91177308-0d34-0410-b5e6-96231b3b80d8
* Disable unrolling in the loop vectorizer when disabled in the pass managerHal Finkel2013-08-282-1/+32
| | | | | | | | | | | | | | | | | When unrolling is disabled in the pass manager, the loop vectorizer should also not unroll loops. This will allow the -fno-unroll-loops option in Clang to behave as expected (even for vectorizable loops). The loop vectorizer's -force-vector-unroll option will (continue to) override the pass-manager setting (including -force-vector-unroll=0 to force use of the internal auto-selection logic). In order to test this, I added a flag to opt (-disable-loop-unrolling) to force disable unrolling through opt (the analog of -fno-unroll-loops in Clang). Also, this fixes a small bug in opt where the loop vectorizer was enabled only after the pass manager populated the queue of passes (the global_alias.ll test needed a slight update to the RUN line as a result of this fix). git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189499 91177308-0d34-0410-b5e6-96231b3b80d8
* Debug Info: add an identifier field to DICompositeType.Manman Ren2013-08-262-3/+3
| | | | | | | | | | | | | | | | | | | DICompositeType will have an identifier field at position 14. For now, the field is set to null in DIBuilder. For DICompositeTypes where the template argument field (the 13th field) was optional, modify DIBuilder to make sure the template argument field is set. Now DICompositeType has 15 fields. Update DIBuilder to use NULL instead of "i32 0" for null value of a MDNode. Update verifier to check that DICompositeType has 15 fields and the last field is null or a MDString. Update testing cases to include an extra field for DICompositeType. The identifier field will be used by type uniquing so a front end can genearte a DICompositeType with a unique identifer. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189282 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorize: Implement partial loop unrolling when vectorization is not ↵Nadav Rotem2013-08-261-0/+39
| | | | | | | | | | | | | | | | | | profitable. This patch enables unrolling of loops when vectorization is legal but not profitable. We add a new class InnerLoopUnroller, that extends InnerLoopVectorizer and replaces some of the vector-specific logic with scalars. This patch does not introduce any runtime regressions and improves the following workloads: SingleSource/Benchmarks/Shootout/matrix -22.64% SingleSource/Benchmarks/Shootout-C++/matrix -13.06% External/SPEC/CINT2006/464_h264ref/464_h264ref -3.99% SingleSource/Benchmarks/Adobe-C++/simple_types_constant_folding -1.95% git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189281 91177308-0d34-0410-b5e6-96231b3b80d8
* [Debug Info Tests] Update testing cases.Manman Ren2013-08-221-9/+9
| | | | | | | | | | A single metadata will not span multiple lines. This also helps me with my script to automatic update the testing cases. A debug info testing case should have a llvm.dbg.cu. Do not use hard-coded id for debug nodes. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189033 91177308-0d34-0410-b5e6-96231b3b80d8
* Teach LoopVectorize about address space sizesMatt Arsenault2013-08-221-2/+29
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188980 91177308-0d34-0410-b5e6-96231b3b80d8
* Add a llvm.copysign intrinsicHal Finkel2013-08-191-0/+53
| | | | | | | | | | | | | | | | | | | | | This adds a llvm.copysign intrinsic; We already have Libfunc recognition for copysign (which is turned into the FCOPYSIGN SDAG node). In order to autovectorize calls to copysign in the loop vectorizer, we need a corresponding intrinsic as well. In addition to the expected changes to the language reference, the loop vectorizer, BasicTTI, and the SDAG builder (the intrinsic is transformed into an FCOPYSIGN node, just like the function call), this also adds FCOPYSIGN to a few lists in LegalizeVector{Ops,Types} so that vector copysigns can be expanded. In TargetLoweringBase::initActions, I've made the default action for FCOPYSIGN be Expand for vector types. This seems correct for all in-tree targets, and I think is the right thing to do because, previously, there was no way to generate vector-values FCOPYSIGN nodes (and most targets don't specify an action for vector-typed FCOPYSIGN). git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188728 91177308-0d34-0410-b5e6-96231b3b80d8
* [tests] Cleanup initialization of test suffixes.Daniel Dunbar2013-08-163-5/+0
| | | | | | | | | | | | | | | | | - Instead of setting the suffixes in a bunch of places, just set one master list in the top-level config. We now only modify the suffix list in a few suites that have one particular unique suffix (.ml, .mc, .yaml, .td, .py). - Aside from removing the need for a bunch of lit.local.cfg files, this enables 4 tests that were inadvertently being skipped (one in Transforms/BranchFolding, a .s file each in DebugInfo/AArch64 and CodeGen/PowerPC, and one in CodeGen/SI which is now failing and has been XFAILED). - This commit also fixes a bunch of config files to use config.root instead of older copy-pasted code. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188513 91177308-0d34-0410-b5e6-96231b3b80d8
* Add ISD::FROUND for libm round()Hal Finkel2013-08-071-0/+52
| | | | | | | | | | | | | | | All libm floating-point rounding functions, except for round(), had their own ISD nodes. Recent PowerPC cores have an instruction for round(), and so here I'm adding ISD::FROUND so that round() can be custom lowered as well. For the most part, this is straightforward. I've added an intrinsic and a matching ISD node just like those for nearbyint() and friends. The SelectionDAG pattern I've named frnd (because ISD::FP_ROUND has already claimed fround). This will be used by the PowerPC backend in a follow-up commit. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187926 91177308-0d34-0410-b5e6-96231b3b80d8
* LoopVectorize: Allow vectorization of loops with lifetime markersArnold Schwaighofer2013-08-061-0/+96
| | | | | | Patch by Marc Jessome! git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187825 91177308-0d34-0410-b5e6-96231b3b80d8
* Use function attributes to indicate that we don't want to realign the stack.Bill Wendling2013-08-011-1/+1
| | | | | | | | | Function attributes are the future! So just query whether we want to realign the stack directly from the function instead of through a random target options structure. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187618 91177308-0d34-0410-b5e6-96231b3b80d8
* Debug Info: update testing cases to pass verifier.Manman Ren2013-07-291-7/+8
| | | | git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187362 91177308-0d34-0410-b5e6-96231b3b80d8
* Next batch of -disable-debug-info-verifier.Rafael Espindola2013-07-261-1/+1
| | | | | | These tests fail without it if pipefail is enabled. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187205 91177308-0d34-0410-b5e6-96231b3b80d8
* Catch more CHECK that can be converted to CHECK-LABEL in Transforms for ↵Stephen Lin2013-07-141-20/+20
| | | | | | | | | | | | | | | | | | | | | | | easier debugging. No functionality change. This conversion was done with the following bash script: find test/Transforms -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_]*\):\( *\)define\([^@]*\)@$FUNC\([( ]*\)\$/;\1\2-LABEL:\3define\4@$FUNC(/g" $TEMP done mv $TEMP $NAME fi done git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186269 91177308-0d34-0410-b5e6-96231b3b80d8
generated by cgit v1.1 at 2024-05-29 02:44:42 +0000