SimAPR is a patch scheduling framework for the patch searching problem. It supports various patch-scheduling algorithm including
- Casino, our own patch-scheduling algorithm described in our ISSTA'23 paper
- The original algorithm of the six APR tools (AlphaRepair, Recoder, TBar, Avatar, FixMiner, and kPar)
- SeAPR, an existing deterministic patch scheduling algorithm
- Our implementation of the GenProg algorithm
This repository contains the implementation of SimAPR, modified APR tools to generate patch space, and scripts and tools to reproduce our experiments.
To reproduce our experiments, please refer to Detailed Instruction.
This section explains how to run SimAPR in a Docker container. We have already provided scripts to simplify the replication of our experiments. Please refer to the Detailed Instruction.
In this section, we'll explain how to use the TBar
and Closure-62
benchmarks with SimAPR.
If you want to run various APR tools and versions, make sure to change tbar
and TBar
to the appropriate APR tool, and Closure_62
to the appropriate version.
To pill our docker image, run the following command:
$ docker pull kyj1411/simapr:0.1-1.2
After that, create a container with the following command:
$ docker run -d --name simapr-1.2 -p 1001:22 kyj1411/simapr:0.1-1.2
Next, access the container with the following command:
$ ssh -p 1001 root@localhost
The password is root
.
Before running SimAPR, you need to create patch spaces and patch candidates. To do that, run the following command:
# cd SimAPR/experiments/tbar
# python3 tbar.py Closure_62
This will take about 2-3 minutes.
The generated patch space will be stored in ~/SimAPR/TBar/d4j/Closure_62, and the meta-information of the patch space will be stored in ~/SimAPR/TBar/d4j/Closure_62/switch-info.json.
After generating the patch space, run SimAPR by executing the following command:
# python3 ~/SimAPR/SimAPR/simapr.py -o ~/SimAPR/experiments/tbar/result/Closure_62-out -m <orig/casino/seapr/genprog> -k template -w ~/SimAPR/TBar/d4j/Closure_62 -t 180000 -T 600 -- python3 ~/SimAPR/SimAPR/script/d4j_run_test.py ~/SimAPR/TBar/buggy
SimAPR provides various scheduling algorithms: original, Casino, SeAPR and GenProg.
'Original' algorithm follows the order generated by the original APR tool.
Set -m
option as appropriate scheduling algorithm.
This command sets overall timeout as 10 minutes (It will take slightly more than 10 minutes).
If you want to change timeout, set -T
option in seconds.
The outputs of SimAPR are stored in ~/SimAPR/experiments/tbar/result/Closure_62-out
.
This directory contains three files: simapr-finished.txt
, simapr-result.json
and simapr-search.log
.
simapr-finished.txt
is generated after SimAPR finishes.
It contains overall time information.
Running time
: The overall time.Select time
The time taken to select patch candidates. This time is an overhead for each scheduling algorithm.Test time
: The time taken to execute test cases for each patch candidate.
Note that the select time
for the original algorithm is 0 because the original algorithm does not use dynamic scheduling.
simapr-result.json
contains the results of each patch candidate in JSON format.
It is a JSON array that contains the results of each patch candidate.
Each result contains the following information:
execution
: Actual test execution. (More details will be described later)iteration
: The number of iterations. It increments with each result.time
: The overall time until this result in seconds.result
: True if the patch passes at least one failing test. False if the patch fails all failing tests.pass_result
: True if the patch passes all test cases (valid patch).pass_all_neg_test
: True if the patch passes all failing test cases.compilable
: True if the patch is compilable.total_searched
: The number of tried patch candidates. It may be the same asiteration
.total_passed
: The number of patch candidates whoseresult
is true.total_plausible
: The number of patch candidates whosepass_result
is true (number of valid patches).config
: Patch ID.
SimAPR provides a simulation mode to reduce the running time.
Cached results are kept in experiments/tbar/result/cache/Closure_62-cache.json
in JSON format after SimAPR finishes.
It is a JSON object where the patch ID serves as the key, and the test execution result serves as the value.
Each cached result contains the following information:
basic
: True if the patch passes at least one failing test. False if the patch fails all failing tests (Same asresult
insimapr-result.json
).plausible
: True if the patch passes all test cases (valid patch) (Same aspass_result
insimapr-result.json
).pass_all_fail
: True if the patch passes all failing test cases (Same aspass_all_neg_test
insimapr-result.json
).compilable
: True if the patch is compilable (Same ascompilable
insimapr-result.json
).fail_time
: The time taken to execute failing tests.pass_time
: The time taken to execute passing tests.
Note that 'pass_time' is 0 if the patch fails the failing tests.
SimAPR does not run test cases and instead uses the cached result if the chosen patch candidate has already been cached.
As a result, the value of execution
in simapr-result.json
is not increased.
To reproduce our experiments, please refer to README_detailed.md.