History

Romain Brenguier 422ebff147 Make process_wrapper.sh executable Not having it executable can lead to the external script not executing as expected, and the reason would be difficult to track.		2019-03-18 10:16:21 +00:00
..
README.md	Benchmark script: improve file naming, readme consistency	2019-02-25 09:16:58 +00:00
benchmark_java_project.js	Print command line nicely in script	2019-03-18 10:16:06 +00:00
benchmark_to_spreadsheet.js	Add a --total option to benchmark_to_spreadsheet.js	2019-03-01 09:37:13 +00:00
draw_scatter.gp	Benchmark script: improve file naming, readme consistency	2019-02-25 09:16:58 +00:00
jbmc_arguments_example.json	Benchmark script: improve file naming, readme consistency	2019-02-25 09:16:58 +00:00
method_list_example.txt	Add an example of method list for benchmarks	2019-02-06 07:59:38 +00:00
package.json	Add package.json	2019-02-06 07:59:38 +00:00
process_wrapper.sh	Make process_wrapper.sh executable	2019-03-18 10:16:21 +00:00

README.md

Script to run an executable (for instance JBMC) on a list of Java methods.

Install

npm install

Usage

benchmark_java_project.js --help

Example

Assuming there is a java maven module installed in project-name/module-core/, and the jbmc main directory is in /path/to/jbmc, go to project-name/module-core/target/classes, and run:

/path/to/scripts/benchmark/benchmark_java_project.js /path/to/jbmc -l method_list_example.txt -a jbmc_arguments_example.json -m path/to/jbmc/lib/java-models-library/target/core-models.jar | tee result.json

This will display the results and save them into the file result.json.

The method_list_example.txt file provided here works for apache-tika/tika-core available from https://github.com/apache/tika

Converting the result to csv

./benchmark_to_spreadsheet.js result.json >result.csv

This will create a result.csv file containing the results.

Comparing two runs

Assuming the results have been converted to csv and saved into two files named develop.csv and branch.csv:

 gnuplot draw_scatter.gp

will create a png file perf_out.png with the time from the branch / changed run on the y axis and the develop / original run on the x axis.