760204eb46
Fix variable and function names in benchmark script
2019-02-22 16:43:20 +00:00
3b2500bef9
Remove delete-failing-SMT-solver tests script
...
With the new tag introduced in the preceding commit this is no longer needed.
This actually simplifies our CI scripts.
2019-02-08 12:50:58 +00:00
9ca0dff0c1
README file on how to run benchmarks
...
This gives instruction on the installation and usage and an example of
command line to use.
2019-02-06 07:59:38 +00:00
044d5459aa
Add an example of method list for benchmarks
...
This are taken from the apache tika project. This can be used when the
apache tika package is compiled locally, otherwise this still serves as
an example of the syntax to use to provide a list of methods to the
script.
2019-02-06 07:59:38 +00:00
191410d05c
Gnuplot script for performance comparison
...
This can be used to compare performances of 2 runs of the benchmarks.
The data should be in 2 csv files named benchmark1.csv and
benchmark2.csv.
2019-02-06 07:59:38 +00:00
19c07d61e7
Script to convert json benchmarks to csv
...
Add a script which takes the output of the benchmark_java_project.js
script (in json) and format it as a csv file.
2019-02-06 07:59:38 +00:00
9d112e5980
Default JBMC arguments for benchmarks
...
Add a json file with some values for JBMC arguments.
2019-02-06 07:59:38 +00:00
9f7d891ca1
Add package.json
...
This is to be able to install dependencies easily, using npm.
2019-02-06 07:59:38 +00:00
2736c05978
Add a script to run Java benchmarks
2019-02-06 07:59:38 +00:00
321402d324
Maintain nondet counters across symex instances
...
Use the path storage, which maintains state across different symex
instantiations. While at it, also make the counters std::size_t as using limited
bitwidth is not a meaningful optimisation here.
2019-01-30 20:02:28 +00:00
291343b298
Silence all Visual Studio warnings when building Minisat or Glucose
...
We are not going to patch all the issues Visual Studio warns about in those code
bases.
2019-01-23 19:38:17 +00:00
4f384adf5c
Remove unmaintained coverage tooling
...
CMake rendered the home-grown vpath build script unnecessary, but the
get_coverage.sh script still referred to the vpath-setup.sh script. With the
preceding commit we can use CMake to get coverage reports, no need for shell
scripts.
2019-01-15 12:59:10 +00:00
1b14892c92
Check test cases as well as scenarios
2019-01-09 15:34:15 +00:00
e80eec3d35
Add test to linter that verifies that all tests have tags
2019-01-09 15:34:15 +00:00
5c03921143
Add a method that concatenates section of the code
...
Starting at elided_lines[start_line][linepos] take the end of the line
until the ending line.
2019-01-09 15:34:15 +00:00
5e82f32b3f
Add comment to CloseExpression
2019-01-09 15:34:15 +00:00
0f7497929b
added test for uninterpreted functions
2019-01-08 20:45:42 +00:00
e53d2b4d45
fix for uninterpreted functions
...
CBMC supports uninterpreted functions; this feature had no test, and thus
got broken with 5081310093Fixes #3561 .
2019-01-08 20:45:31 +00:00
b32649825b
Merge pull request #3662 from diffblue/shl-overflow
...
shift left overflow check now done in goto_check
2019-01-05 12:29:59 +00:00
3be9b5efc5
implement criteria for shl overflow in goto_check
...
The current implementation generates an overflow-shl predicate, which is
then interpreted by the solver APIs. This has the disadvantage that the
predicate has semantics that are both complicated and highly
language-dependent, which is not a good fit for a solver.
The new implementation defines the meaning of signed left shift overflows in
goto-check, similar as it is already done for division and unary minus.
This is covered by an existing test:
regression/cbmc/Overflow_Leftshift1/test.desc
2019-01-04 18:43:57 +00:00
667fd09842
Merge pull request #3604 from tautschnig/perf-test-cleanup
...
Performance test script: speed up build, support Nitro
2019-01-04 12:56:11 +01:00
aae639ab6b
smt2 frontend: let identifiers must be globally unique
...
The solver backend assumes that identifiers used in let expressions are
globally unique. The renamed identifiers are now retained in the id_map to
ensure that they are not re-used.
This enables 11 regression tests to pass.
2019-01-02 11:53:59 +00:00
786f7970fd
Merge pull request #3640 from diffblue/smt2-bound-symbols
...
smt2: fix exists/forall
2018-12-29 21:23:13 +00:00
9f1aa87289
Merge pull request #3639 from diffblue/smt2-with-full-size
...
smt2: fix encoding of with expressions
2018-12-29 21:22:33 +00:00
e70471bf06
smt2: fix exists/forall
2018-12-29 13:34:51 +00:00
fbc74decc9
smt2: fix with expressions for fixed-width arrays
2018-12-29 13:05:45 +00:00
75dd8bafb8
smt2 backend: member operator for non-byte offsets
2018-12-29 12:18:18 +00:00
c37290f8e4
perf-test: benchexec can use all NUMA nodes
...
Benchexec limits itself to the memory on a single NUMA node if cores==1.
Just permit use of all cores instead.
2018-12-28 07:34:13 +00:00
66d6934f4e
perf-test: support Nitro-based instances
...
These instances report EBS volumes as NVME devices.
2018-12-28 07:29:51 +00:00
7081be527c
perf-test: Use Ubuntu 18.04 as build image
...
This avoids having to fetch a recent GCC version from some PPA. Also use
AWS mirrors for faster download.
2018-12-28 07:29:51 +00:00
92f3f4fff6
Make __CPROVER_allocated_memory work with non-POD type objects
...
The precision of local_bitvector_analysist is not sufficient to rule out several
cases introduced in 732ce2aee6
2018-12-19 10:05:37 +00:00
1614a5fb35
perf-test: New option --compare-to to include past results in tables
...
This simplifies performance comparison.
2018-12-18 13:28:11 +00:00
03aa2b8cf7
perf-test: Use Ubuntu 18.04 as SV-COMP'19 does
2018-12-18 13:28:10 +00:00
3d55685228
perf-test: Switch sv-benchmarks branch to svcomp19
2018-12-18 13:28:10 +00:00
72129a4734
perf-test: Update the set of SV-COMP categories
2018-12-18 13:28:10 +00:00
7a2d2f4b0b
perf-test: Maintain shell scripts outside CloudFormation template
...
This simplifies editing. Just upload them to S3 when setting up benchmarking and
download them from S3 onto the instance. Also fixed several bugs in the script.
2018-12-18 13:28:10 +00:00
810bc88ae6
perf-test: Python2 compatible date formatting
2018-12-18 13:28:10 +00:00
5d9e0c3c85
perf-test: openjdk is not required as we only build cbmc/goto-cc
...
This reduces overhead during package install.
2018-12-18 13:28:10 +00:00
4857bf0354
perf-test: Enable ccache for CBMC builds
...
We need to manually upload to S3 rather than using CodeBuild's cache
configuration as each perf-test run creates new CodeBuild configurations and
thus also new keys to store the cache at.
2018-12-18 13:28:10 +00:00
0a21a31fba
Merge pull request #3570 from smowton/smowton/fix/value-set-member-type-safety
...
Fix value-set make_member function
2018-12-17 16:50:34 +00:00
8d88c9776d
Enable previously-failing test Linking7/member-name-mismatch
...
Now that value-set soundly simplifies member-of-cast-of-struct, this test works as expected.
This also improves the specificity of the two tests in this directory.
This is disabled under SMT2 for the same reason as the existing test.desc -- it involves a
struct-to-struct cast, which SMT2's `convert_typecast` doesn't handle yet.
2018-12-17 15:20:27 +00:00
53d786e513
smt2 backend: additional passing tests
2018-12-17 14:18:55 +00:00
77c52f5989
Fix broken doxygen download link
2018-12-13 13:01:22 +00:00
a4fe9b65e7
Merge pull request #3556 from peterschrammel/remove-old-cvc
...
Remove refs to obsolete solver modules
2018-12-12 13:20:00 +00:00
77df7eaa1a
Merge pull request #3552 from diffblue/smt2_floatbv_model_fix
...
smt2 backend: fix floating-point values
2018-12-11 13:35:49 -05:00
aab0b9bfdd
Merge pull request #3547 from diffblue/smt2_concat
...
smt2: concat
2018-12-11 11:28:49 -05:00
5169e7d346
smt2 backend: fix floating-point values
2018-12-11 16:08:25 +00:00
23deda58fe
Merge pull request #3550 from diffblue/smt2_constants
...
smt2 solver: constants
2018-12-11 11:06:01 -05:00
61f07f6579
Merge pull request #3548 from diffblue/smt2_sign_extend
...
smt2 parser: fix sign extension
2018-12-11 11:05:33 -05:00
24883f024a
smt2 parser: concat is multi-ary
2018-12-11 14:25:13 +00:00
Chris Smowton