Automated Regression Testing and Verification of Complex Code Changes

Abstract

In this dissertation, we answer how to determine the semantic impact of a complex change and just how complex a "complex change" really is. We answer whether the interaction of the simple changes constituting the complex change can result in regression errors, what the prevalence and nature of such (change interaction) errors is, and how to expose them. We answer how complex a "complex error" really is and whether regression errors due to change interaction are more complex than other regression errors. We make available an open-source tool, CyCC, to measure the complexity of Git source code commits, a test generation tool, Otter_Graph, for C programs that exposes change interaction errors, and a regression error subject suite, CoREBench, consisting of a large number of genuine regression errors in open-source C programs for the controlled study of regression testing, debugging, and repair techniques.