Commit r309730 moved the call to canForwardTree into an assert(), even
though this function has side-effects if its DoIt parameter is true. To
avoid a warning in release builds, do an (void)Execution of its result
instead.
To avoid such confusion in the future, rename
canForwardTree() to forwardTree().
llvm-svn: 309753
The previous algorithm was to search a writes and the sours of its value
operand, and see whether the write just stores the same read value back,
which includes a search whether there is another write access between
them. This is O(n^2) in the max number of accesses in a statement
(+ the complexity of isl comparing the access functions).
The new algorithm is more similar to the one used for searching for
overwrites and coalescable writes. It scans over all accesses in order
of execution while tracking which array elements still have the same
value since it was read. This is O(n), not counting the complexity
within isl. It should be more reliable than trying to catch all
non-conforming cases in the previous approach. It is also less code.
We now also support if the write is a partial write of the read's
domain, and to some extent non-affine subregions.
Differential Revision: https://reviews.llvm.org/D36137
llvm-svn: 309734
With a lot of reads and writes to the same array in a statement,
some isl sets that capture the state between access can become
complex such that isl takes more considerable time and memory
for operations on them.
The problems identified were:
- is_subset() takes considerable time with many disjoints in the
arguments. We limit the number of disjoints to 4, any additional
information is thrown away.
- subtract() can lead to many disjoints. We instead assume that any
array element is possibly accessed, which removes all disjoints.
- subtract_domain() may lead to considerable processing, even if all
elements are are to be removed. Instead, we remove determine and
remove the affected spaces manually. No behaviour is changed.
llvm-svn: 309728
This is necessary to get accurate traces from `nvprof` / `nvcc`.
Otherwise, we lose some profiling information.
Differential Revision: https://reviews.llvm.org/D35940
llvm-svn: 309682
It is possible that the `HostPtr` that coresponds to an array could be
invariant load hoisted. Make sure we use the invariant load hoisted
value by using `IslNodeBuilder::getLatestValue`.
Differential Revision: https://reviews.llvm.org/D36001
llvm-svn: 309681
We populate `IslNodeBuilder::ValueMap` which contains replacements for
`llvm::Value`s. There was no simple method to pick up a replacement if
it exists, otherwise fall back to the original.
Create a method `IslNodeBuilder::getLatestValue` which provides this
functionality.
This will be used in a later patch to fix bugs in `PPCGCodeGeneration`
where the latest value is not being used.
Differential Revision: https://reviews.llvm.org/D36000
llvm-svn: 309674
This allows -polly-optree to move instructions that depend on
synthesizable values.
The difficulty for synthesizable values is that their value depends on
the location. When it is moved over a loop header, and the SCEV
expression depends on the loop induction variable (SCEVAddRecExpr), it
would use the current induction variable instead of the last one.
At the moment we cannot forward PHI nodes such that crossing the header
of loops referenced by SCEVAddRecExpr is not possible (assuming the loop
header has at least two incoming blocks: for entering the loop and the
backedge, such any instruction to be forwarded must have a phi between
use and definition).
A remaining issue is when the forwarded value is used after the loop,
but is only synthesizable inside the loop. This happens e.g. if
ScalarEvolution is unable to determine the number of loop iterations or
the initial loop value. We do not forward in this situation.
Differential Revision: https://reviews.llvm.org/D36102
llvm-svn: 309609
In addition to array and PHI writes, also allow scalar value writes.
The only kind of write not allowed are writes by functions
(including memcpy/memmove/memset).
llvm-svn: 309582
Summary:
This allows us to map functions such as exp, expf, expl, for which no
LLVM intrinsics exist. Instead, we link to NVIDIA's libdevice which provides
high-performance implementations of a wide range of (math) functions. We
currently link only a small subset, the exp, cos and copysign functions. Other
functions will be enabled as needed.
Reviewers: bollu, singam-sanjay
Reviewed By: bollu
Subscribers: tstellar, tra, nemanjai, pollydev, mgorny, llvm-commits, kbarton
Tags: #polly
Differential Revision: https://reviews.llvm.org/D35703
llvm-svn: 309560
This reverts commit r309490 as it triggers on our AOSP buildbut error messages
of the form:
inlinable function call in a function with debug info must have a !dbg location
llvm-svn: 309556
Summary:
**Remove debug metadata from instruction to be copied to prevent the source file's debug metadata being copied into GPUModule and eventually failing Module verification and ASM string codegeneration.**
When copying the instruction onto the Module meant for the GPU, debug metadata attached to an instruction causes all related metadata to be pulled into the Module, including the DICompileUnit, which is not listed in llvm.dbg.cu of the Module. This fails the verification of the Module and generation of the ASM string.
The only debug metadata of the instruction, the DebugLoc, is unset by this patch.
Reviewers: grosser, bollu, Meinersbur
Reviewed By: grosser, bollu
Subscribers: pollydev
Tags: #polly
Differential Revision: https://reviews.llvm.org/D35630
llvm-svn: 309490
Write coalescing combines write accesses that
- Write the same llvm::Value.
- Write to the same array.
- Unless they do not write anything in a statement instance (partial
writes), write to the same element.
- There is no other access between them that accesses the same element.
This is particularly useful after DeLICM, which leaves partial writes to
disjoint domains.
Differential Revision: https://reviews.llvm.org/D36010
llvm-svn: 309489
ScopStmt::contains is currently implemented on the basis of BasicBlock
and does not take the instruction list into account. Therefore any
instruction copied by -polly-optree into another statement currently
triggers that assertion.
Remove that assertion for now. We might re-enable it when the
implementation of ScopStmt::contains changes.
llvm-svn: 309421
This fixes a bug in isl_flow where triggering the compute out could result in
undefined or unexpected behavior. This fixes some recent regressions we saw
in the android buildbots. Thanks Eli Friedman for reducing the corresponding
test cases.
llvm-svn: 309274
ScopBuilder and Simplify (through VirtualInstruction.cpp) previously
used this functionality in their own implementation. Refactor them
both into a common one into the Scop class.
BlockGenerator also makes use of a similiar functionality, but also
records outside users and takes place after region simplification.
Merging it as well would be more complicated.
llvm-svn: 309273
After region exit simplification, the incoming block of a phi node in
the SCoP region's exit block lands outside of the region. Since we
treat SCoPs as if this already happened, we need to account for that
when looking for outside uses of scalars (i.e. escaping scalars).
llvm-svn: 309271
A region statement's instruction list is always empty and ignored by the code
generator. Don't give the impression that it means anything.
llvm-svn: 309197
A PHI node's incoming block is the user of its operand, not the PHI's parent.
Assuming the PHINode's parent being the user lead to the removal of a
MemoryAccesses because its use was assumed to be inside of the SCoP.
llvm-svn: 309164
Translate the ScheduleOptimizer to use the new isl C++ bindings.
Reviewed-by: Michael Kruse <llvm@meinersbur.de>
Differential Revision: https://reviews.llvm.org/D35845
llvm-svn: 309119
Since there will be no more a 1:1 correspondence between statements and
basic blocks, we would like to get rid of the method getStmtFor(BB)
and its uses. Here we remove one of its uses in ScopInfo by fetching
the statement in which the call instruction lies.
Contributed-by: Nandini Singhal <cs15mtech01004@iith.ac.in>
Differential Revision: https://reviews.llvm.org/D35691
llvm-svn: 309110
In the following loop:
int i;
for (i = 0; i < func(); i+=1)
;
SCoP:
for (int j = 0; j<n; j+=1)
S(i, j)
The value i is synthesizable in the SCoP that includes only the j-loop.
This is because i is fixed within the SCoP, it is irrelevant whether
it originates from another loop.
This fixes a strange case where a PHI was synthesiable in a SCoP,
but not its incoming value, triggering an assertion.
This should fix MultiSource/Applications/sgefa/sgefa of the
perf-x86_64-penryn-O3-polly-before-vectorizer-unprofitable buildbot.
llvm-svn: 309109
It seems I still had some incomplete changes in the tree when committing.
In general, we only import changes from isl upstream. In this case, the
changes were especially unfortunate, as they broke the error management
in isl_flow.c and consequently caused regressions.
Thanks to Michael Kruse for spotting this mistake.
llvm-svn: 309039
In future, there will be no more a 1:1 correspondence between statements
and basic blocks, the name `contains` does not correctly capture their
relationship. A BB may infact comprise of multiple statements; hence we
describe a statement 'representing' a basic block.
Differential Revision: https://reviews.llvm.org/D35838
llvm-svn: 308982
Summary:
This consists instances of two changes:
- Accept any order of checks for a specific loop form, that appear in different order in the new vs legacy-PM.
- Remove checks for specific regions.
Reviewers: grosser
Reviewed By: grosser
Subscribers: pollydev, llvm-commits
Tags: #polly
Differential Revision: https://reviews.llvm.org/D35837
llvm-svn: 308976
Invariant load hoisted scalars, and arrays whose size we can statically compute
to be 0 do not need to be allocated as arrays.
Invariant load hoisted scalars are sent to the kernel directly as parameters.
Earlier, we used to allocate `0` bytes of memory for these because our
computation of size from `PPCGCodeGeneration::getArraySize` would result in `0`.
Now, since we don't invariant loads as arrays in PPCGCodeGeneration, this
problem does not occur anymore.
Differential Revision: https://reviews.llvm.org/D35795
llvm-svn: 308971
Michael Kruse