Some of the analyzer tests check the exact plist output, in order to
verify that the diagnostics produced is correct.
Current testing setup has many issues:
plist output clobbers tests, making them harder to read
it is impossible to debug test failures given error messages from FileCheck.
The only recourse is manually creating the files and using the diff
again, it is impossible to update the tests given the error message:
the only process is a tedious manual one,
going from a separate plist file to CHECK directives.
This patch offers a much better approach of using "diff" directly in place of FileCheck,
and moving tests to separate files.
Generated using the following script:
```
import os
import glob
import re
import subprocess
diagnostics_key = "// CHECK: <key>diagnostics</key>"
def process_file(f, data):
idx = data.index(diagnostics_key)
plist_out_f = 'ExpectedOutputs/plists/%s.plist' % f
plist_out_folder = os.path.join('ExpectedOutputs/plists/', os.path.dirname(f))
plist_data = data[idx:]
plist_data = plist_data.replace('// CHECK: ', '')
plist_data = plist_data.replace('// CHECK-NEXT: ', '')
plist_data += "</dict>\n</plist>\n"
data = data[:idx]
ptn = re.compile("FileCheck --?input-file(=| )(%t|%t\.plist) %s")
if not ptn.findall(data):
print "none found =/ skipping..."
return
data = ptn.sub(lambda m: "tail -n +11 %s | diff -u -w - %%S/../%s" % (m.group(2), plist_out_f), data)
with open(f, 'w') as out_f:
out_f.write(data)
subprocess.check_call(["mkdir", "-p", plist_out_folder])
with open(plist_out_f, 'w') as out_f:
out_f.write(plist_data)
def main():
files = glob.glob("**/*.*")
for f in files:
with open(f) as f_handler:
data = f_handler.read()
if diagnostics_key in data:
print "Converting %s" %f
process_file(f, data)
if __name__ == "__main__":
main()
```
Differential Revision: https://reviews.llvm.org/D50545
llvm-svn: 339475
Rename AlternateExtensive to Extensive.
In 2013, five years ago, we have switched to AlternateExtensive
diagnostics by default, and Extensive was available under unused,
undocumented flag.
This change remove the flag, renames the Alternate
diagnostic to Extensive (as it's no longer Alternate), and ports the
test.
Differential Revision: https://reviews.llvm.org/D47670
llvm-svn: 334524
In the analyzer's path-sensitive reports, when a report goes through a branch
and the branch condition cannot be decided to be definitely true or false
(based on the previous execution path), an event piece is added that tells the
user that a new assumption is added upon the symbolic value of the branch
condition. For example, "Assuming 'a' is equal to 3".
The text of the assumption is hand-crafted in various manners depending on
the AST expression. If the AST expression is too complex and the text of
the assumption fails to be constructed, the event piece is omitted.
This causes loss of information and misunderstanding of the report.
Do not omit the event piece even if the expression is too complex;
add a piece with a generic text instead.
Differential Revision: https://reviews.llvm.org/D23300
llvm-svn: 283301
This patch adds hashes to the plist and html output to be able to identfy bugs
for suppressing false positives or diff results against a baseline. This hash
aims to be resilient for code evolution and is usable to identify bugs in two
different snapshots of the same software. One missing piece however is a
permanent unique identifier of the checker that produces the warning. Once that
issue is resolved, the hashes generated are going to change. Until that point
this feature is marked experimental, but it is suitable for early adoption.
Differential Revision: http://reviews.llvm.org/D10305
Original patch by: Bence Babati!
llvm-svn: 251011
to the plist output. This check_name field does not guaranteed to be the
same as the name of the checker in the future.
Reviewer: Anna Zaks
Differential Revision: http://reviews.llvm.org/D6841
llvm-svn: 228624
...but don't yet migrate over the existing plist tests. Some of these
would be trivial to migrate; others could use a bit of inspection first.
In any case, though, the new edge algorithm seems to have proven itself,
and we'd like more coverage (and more usage) of it going forwards.
llvm-svn: 183165
Neither the compiler nor the analyzer are doing anything with non-VarDecl
decls in the CFG, and having them there creates extra nodes in the
analyzer's path diagnostics. Simplify the CFG (and the path edges) by
simply leaving them out. We can always add interesting decls back in when
they become relevant.
Note that this only affects decls declared in a DeclStmt, and then only
those that appear within a function body.
llvm-svn: 183157
The uniqueing location is the location which is part of the hash used to determine if two reports are
the same. This is used by the CmpRuns.py script to compare two analyzer runs and determine which
warnings are new.
llvm-svn: 180166
The 2 functions were computing the same location using different logic (each one had edge case bugs that the other
one did not). Refactor them to rely on the same logic.
The location of the warning reported in text/command line output format will now match that of the plist file.
There is one change in the plist output as well. When reporting an error on a BinaryOperator, we use the location of the
operator instead of the beginning of the BinaryOperator expression. This matches our output on command line and
looks better in most cases.
llvm-svn: 180165
Consider this case:
int *p = 0;
p = getPointerThatMayBeNull();
*p = 1;
If we inline 'getPointerThatMayBeNull', we might know that the value of 'p'
is NULL, and thus emit a null pointer dereference report. However, we
usually want to suppress such warnings as error paths, and we do so by using
FindLastStoreBRVisitor to see where the NULL came from. In this case, though,
because 'p' was NULL both before and after the assignment, the visitor
would decide that the "last store" was the initialization, not the
re-assignment.
This commit changes FindLastStoreBRVisitor to consider all PostStore nodes
that assign to this region. This still won't catches changes made directly
by checkers if they re-assign the same value, but it does handle the common
case in user-written code and will trigger ReturnVisitor's suppression
machinery as expected.
<rdar://problem/13299738>
llvm-svn: 176201
Fixes PR15358 and <rdar://problem/13295437>.
Along the way, shorten path diagnostics that say "Variable 'x'" to just
be "'x'". By the context, it is obvious that we have a variable,
and so this just consumes text space.
llvm-svn: 176115
This required more changes than I originally expected:
- ObjCIvarRegion implements "canPrintPretty" et al
- DereferenceChecker indicates the null pointer source is an ivar
- bugreporter::trackNullOrUndefValue() uses an alternate algorithm
to compute the location region to track by scouring the ExplodedGraph.
This allows us to get the actual MemRegion for variables, ivars,
fields, etc. We only hand construct a VarRegion for C++ references.
- ExplodedGraph no longer drops nodes for expressions that are marked
'lvalue'. This is to facilitate the logic in the previous bullet.
This may lead to a slight increase in size in the ExplodedGraph,
which I have not measured, but it is likely not to be a big deal.
I have validated each of the changed plist output.
Fixes <rdar://problem/12114812>
llvm-svn: 175988
path notes for cases where a value may be assumed to be null, etc.
Instead of having redundant diagnostics, do a pass over the generated
PathDiagnostic pieces and remove notes from TrackConstraintBRVisitor
that are already covered by ConditionBRVisitor, whose notes tend
to be better.
Fixes <rdar://problem/12252783>
llvm-svn: 166728
Some implicit statements, such as the implicit 'self' inserted for "free"
Objective-C ivar access, have invalid source locations. If one of these
statements is the location where an issue is reported, we'll now look at
the enclosing statements for a valid source location.
<rdar://problem/12446776>
llvm-svn: 165354
CmpRuns.py can be used to compare issues from different analyzer runs.
Since it uses the issue line number to unique 2 issues, adding a new
line to the beginning of a file makes all issues in the file reported as
new.
The hash will be an opaque value which could be used (along with the
function name) by CmpRuns to identify the same issues. This way, we only
fail to identify the same issue from two runs if the function it appears
in changes (not perfect, but much better than nothing).
llvm-svn: 158180
we use the same Expr* as the one being currently visited. This is preparation for transitioning to having
ProgramPoints refer to CFGStmts.
This required a bit of trickery. We wish to keep the old Expr* bindings in the Environment intact,
as plenty of logic relies on it and there is no reason to change it, but we sometimes want the Stmt* for
the ProgramPoint to be different than the Expr* being used for bindings. This requires adding an extra
argument for some functions (e.g., evalLocation). This looks a bit strange for some clients, but
it will look a lot cleaner when were start using CFGStmt* in the appropriate places.
As some fallout, the diagnostics arrows are a bit difference, since some of the node locations have changed.
I have audited these, and they look reasonable.
llvm-svn: 154214
Essentially, a bug centers around a story for various symbols and regions. We should only include
the path diagnostic events that relate to those symbols and regions.
The pruning is done by associating a set of interesting symbols and regions with a BugReporter, which
can be modified at BugReport creation or by BugReporterVisitors.
This patch reduces the diagnostics emitted in several of our test cases. I've vetted these as
having desired behavior. The only regression is a missing null check diagnostic for the return
value of realloc() in test/Analysis/malloc-plist.c. This will require some investigation to fix,
and I have added a FIXME to the test case.
llvm-svn: 152361
George Karpenkov