Estimating Percentiles in Computer Experiments: A Comparison of Sequential-Adaptive Designs and Fixed Designs

Abstract

Computer simulators are often used when it is impossible or infeasible to observe actual systems or processes. These simulators can be very complex, requiring many hours or days for a single simulation, and thus, the number of times we may run the code is small. Running such a simulator at a few chosen input settings comprises a computer experiment. Most of the work done in this area focuses on either estimating the unknown complex input–output relationship or optimizing the output. In this article, we consider the problem of percentile estimation in a computer experiments setting and propose the use of sequential-adaptive designs to estimate percentiles, as opposed to fixed designs. For estimating the value and location of the percentile, we present design criteria that can be used to adaptively select the design sites at which to run the simulator. A comparison of results from using sequential-adaptive designs and fixed designs is presented.

Keywords:

• 62L05
• 62K99

Keywords:

• Adaptive designs
• Computer experiments
• Gaussian stochastic process model
• Percentile estimation

Acknowledgment

We thank the editor and the referee for very helpful suggestions that improved this article.