A value-of-information based approach to simulation model refinement

Abstract

The appropriateness of a simulation model for engineering design is dependent on the trade-off between model accuracy and the computational expense for its development and execution. Since no simulation model is perfect, any simulation model for a system's physical behaviour can be refined further, although likely at an increased computational cost. Hence, the question faced by a designer is ‘How much refinement of a simulation model is appropriate for a particular design problem?’ The simplified nature of simulation models results in two types of uncertainty—variability, which can be modelled using probability distribution functions and imprecision, best modelled using intervals. Value-of-information has been used in the engineering design literature to decide whether to make a decision using the available information or to gather more information before making a decision. However, the main drawback of applying existing value-of-information based metrics for model refinement problems is that existing metrics only account for variability; they do not account for imprecision in simulation models and the impact of its reduction on design decisions. To overcome the limitation of existing metrics in the context of model refinement, this article presents a value-of-information based approach for determining the appropriate extent of refinement of simulation models. The approach consists of (i) a metric called improvement potential for quantifying the value-of-information obtained via refinement of simulation models and (ii) a method in which this metric is utilized for supporting model refinement decisions. The improvement potential measures the value-of-information by considering both imprecision and variability in simplified models. It quantifies the maximum possible improvement in a designer's decision that can be achieved by refining a simulation model. Specifically, we focus on multi-objective compromise decisions modelled using the compromise decision support problem construct, which is a hybrid formulation based on traditional optimization and goal programming. The method involves starting from a simple simulation model and gradually refining it until the value of further refinement on design decisions is small. The approach is presented using two examples—design of a pressure vessel and design of a multi-functional material. The pressure vessel problem is used to illustrate the benefits of using this approach shown by gradually refining its material parameters; the materials design problem is a comprehensive problem where a complex finite element model is gradually refined. The approach proposed in this article can be utilized by designers and analysts in developing effective simulation models for specific design problems while efficiently utilizing their model development resources.

Keywords:

• simulation models
• refinement
• value-of-information
• decision making

Acknowledgements

The authors gratefully acknowledge support from Air Force Office of Scientific Research (AFOSR) Multi-University Research Initiative Grant (1606U81), without which the research presented in this article would not have been possible. The authors also acknowledge the help of Ryan Austin and David McDowell for providing the shock simulation model for the materials design problem.

Notes

1. In this article, it is assumed that the decision maker uses utility functions to quantify the payoffs. Hence, the word ‘payoff’ is used synonymously with ‘utility’ in this article.