Exploiting Similarity Patterns to Build Generic Test Case Templates for Software Product Line Testing

Abstract

Software product line testing (SPLT) is more complicated than the conventional testing. Since software product lines consist of several product variants, we need test each variant thereby causing test case explosion. In this thesis, we studied Android OS product line test libraries to understand the combinatorial test explosion problem. Our study revealed many similar test cases which we call ?test clones?. For example, when new versions of Android OS are created, test cases from earlier versions are copied and modified. This leads to test clones and problems of managing large test libraries with many redundancies. In this thesis, we propose a generic adaptation method to avoid test clones and therefore save effort of developing and maintaining SPL test libraries. A study of existing literature reveals that while some attempts have been made to address the test case explosion issue, most of these are heuristics, combinatorial selection or model based approaches which have known limitations when it comes to variability management and heterogeneity prevalent in the software product line executable test libraries. The approach proposed in this thesis solves the problem in a way that is effective (any type of test clones can be tackled) and practical (any test library can be addressed irrespective of programming platform). The proposed approach is based on test case reuse facilitated by test templates that represent groups of similar test cases. The Generic Adaptive Test Template (GATT) structure proposed in this thesis takes advantage of common aspects and predicted variability that are present among individual test cases. The process starts with detection and grouping of test clones, provisioning for variability and then constructing hierarchical templates. Subsequently, the process provides specifications to derive the test library by binding variant points with appropriate variant choices. This compile-time test template approach helps in test construction by adaptive generation without affecting the follow up test execution. The proposed template-based design and implementation approach helps the test engineers to handle key challenges namely variability, redundancy and heterogeneity in large scale test libraries. The results of the experiments conducted on Android OS test libraries demonstrate that a compressed, normalized, non-redundant test library can be achieved using our proposed approach. The results also confirm our hypothesis that test library construction using template-based approach will facilitate scalability and reuse from a software product line test evolution perspective.