Cramer–von Mises Tests for the Compatibility of Two Software Operating Environments

Abstract

Higher-precision inferences about impending software failures can be achieved when the same software reliability model that fits failure data from the test interval also fits data from the field interval. If the test and field environments differ significantly in terms of how the software is used, then a single model for the pooled data may not be adequate. In this article we formulate the hypothesis of compatible test and field environments in terms of a statistical hypothesis and develop a Cramer–von Mises (CvM) test procedure within the context of a well-known nonhomogeneous Poisson process software reliability model. The CvM test has a compelling advantage over a previously proposed likelihood ratio test (LRT₀), because it does not require specification of the class of alternatives, which are frequently unknown for real-life problems. Moreover, although there are existence issues with LRT₀, the CvM test always exists. An asymptotic approximation for the p value of the CvM test is derived, and an algorithm for a small-sample bootstrap approximation is presented. A simulation study shows that the CvM test works well for the class of alternatives for which LRT₀ also would work well and continues to work well for other alternatives for which LRT₀ has no statistical motivation or otherwise has existence problems. Data from a real software project are used to illustrate the hypothesis testing procedures.

KEY WORDS :

• Cramer-von Mises statistic
• Likelihood ratio
• Software reliability