ANALYZING THE BEHAVIOR OF DEPLOYED SOFTWARE

Abstract

We present a method for collecting operational field data in deployed systems. The data is gathered from real executions with low overhead. The collected data reflects actual executions and thus does not contain any assumptions and approximations common for data gathered in-house. Field data can be used to achieve many different goals. First, we use the collected field data to mine performance specifications in the form of performance models and performance assertions. The mined models can aid in comprehension, verification, monitoring, and performance regression testing. Second, we demonstrate a method to evaluate the quality of a test suite by comparing field executions with runs observed during testing. Our technique increases confidence in the test suite and the testing process. The results of the technique can also be used to guide test suite augmentation or aid in comprehension.