Incorporating Input Model Risk in Simulation Optimization and Uncertainty Quantification

Hunhye Song

doi:https://doi.org/10.21985/N2G106

Work

Incorporating Input Model Risk in Simulation Optimization and Uncertainty Quantification

Public Deposited

Download PDF

Download All Files (.zip)

Traditionally, simulation analysis has focused on designing a computationally efficient algorithm assuming a correct simulation model is given. As computation becomes cheaper, we are now able to perform more sophisticated simulation analyses involving extensive computation and consider all sources of errors in the simulation model and their effects to the conclusion that we draw from the model. Input model risk in computer simulation refers to the risk of inaccurate evaluation of the system performance that leads to a suboptimal decision due to the errors in the inputs to the simulation model. This dissertation proposes methods to quantify output uncertainty in computer simulation due to uncertain input models in the contexts of discrete-event simulation and complex computer models, and introduces a new discrete-event simulation optimization procedure that provides correct inference on the optimal solution in the presence of input model risk. In discrete-event simulation, input uncertainty refers to the uncertainty in simulation output caused by the estimation error in input probability distributions that are estimated from finite real-world data. We use a mean-variance effects model to propagate the impact of the estimation error in the input distributions to input uncertainty. We are the first to propose a contribution measure of each input distribution to the overall input uncertainty and provide guidance to decide from which input source to collect additional data. In the analysis of a complex computer model, global sensitivity analysis measures output variability due to the randomness in inputs is induced by natural variability or lack of information about a parameter. The current most popular global sensitivity measures, the first-order and total effects, fail to correctly capture the output sensitivity when there is dependence among inputs. We discuss mathematical properties of Shapley effects, a new global sensitivity measure based on the concept of Shapley value, focusing on its robustness to input dependence and propose an algorithm to estimate Shapley effects. While a complex computer model aims to understand the dynamics of the system, discrete-event simulation is often used to make an operational decision for the target system. However, most of the optimization via simulation procedures neglect the impact of input uncertainty on the optimal decision. By extending the traditional multiple comparisons with the best procedure, we created the input-output uncertainty comparisons procedure, which incorporates input uncertainty in finding a system with the best output mean performance by estimating the joint distribution of the impact of input uncertainty on all systems' simulation outputs and providing a set of likely optimal solutions with a high confidence guarantee.

Last modified