Using Pre-Release Test Failures to Build Early Post-Release Defect Prediction Models

Software quality is one of the most pressing concerns for nearly all software developing companies. At the same time, software companies also seek to shorten their release cycles to meet market demands while maintaining their product quality. Identifying problematic code areas becomes more and more important. Defect prediction models became popular in recent years and many different code and process metrics have been studied. There has been minimal effort relating test executions during development with defect likelihood. This is surprising as test executions capture the stability and quality of a program during the development process. This paper presents an exploratory study investigating whether test execution metrics, e.g. test failure bursts, can be used as software quality indicators and used to build pre- and post-release defects prediction models. We show that test metrics collected during Windows 8 development can be used to build pre- and post-release defect prediction models early in the development process of a software product. Test metrics outperform pre-release defect counts when predicting post-release defects.