Computationally Efficient Comparison of Experimental Designs for System Reliability Studies With Binomial Data

Abstract

A common practical problem in experimental design is that of quantitatively determining how to best use a fixed amount of resources to supplement an existing analysis with additional data. We address this problem in the context of the second stage in Bayesian system reliability studies; these second-stage data are aimed at obtaining a more precise estimate of the system’s reliability. The current strategy for comparing potential experimental designs is computationally intensive and time-consuming. We present new, more computationally efficient methodology that can be used to quickly assess the anticipated improvements for candidate allocations, and demonstrate its effectiveness with a missile system application. While we find that there are some situations in which our methodology may provide a more optimistic estimate of the expected improvement to be gained from a candidate experiment than the current approach, the results of the two methods tend to closely match when the rankings of candidate experiments are considered. Our implementation of the algorithm (in C), along with a brief description and an example input set, is available online as supplementary materials.

KEY WORDS:

• Complex systems
• Markov chain Monte Carlo
• Resource allocation.

ACKNOWLEDGMENTS

This work was funded in part by an NSF grant DMS 0502347 EMSW21-RTG awarded to Iowa State University and an NSF award 0959713 awarded to St. Lawrence University. The authors thank the editor, associate editor, and two anonymous referees whose constructive feedback on earlier versions greatly improved the final article.