Comparing the Reliability of Related Populations With the Probability of Agreement

ABSTRACT

Combining information from different populations to improve precision, simplify future predictions, or improve underlying understanding of relationships can be advantageous when considering the reliability of several related sets of systems. Using the probability of agreement to help quantify the similarities of populations can help to give a realistic assessment of whether the systems have reliability that are sufficiently similar for practical purposes to be treated as a homogeneous population. The new method is described and illustrated with an example involving two generations of a complex system, where the reliability is modeled using either a logistic or probit regression model. Note that supplementary materials including code, datasets, and added discussion are available online.

KEYWORDS:

• Equivalence testing
• Generalized linear models
• Homogeneity of population characteristics
• Probability of agreement
• Reliability

Supplementary Materials

The supplementary materials consist of three parts: Supplemental Materials A provides technical details regarding maximum likelihood estimation of the probability of agreement, asymptotic normality, and construction of approximate confidence intervals; Supplemental Materials B provides detailed results and discussion of the simulation study described in Section 4; and Supplemental Materials C includes a discussion of a bootstrap alternative to the calculation of confidence intervals.