Measuring what's missing: practical estimates of coverage for stochastic simulations

References

• Fiacco AV. Introduction to sensitivity and stability analysis in nonlinear programming. New York, NY: Academic Press; 1983.
   Google Scholar
• Leamer E. Sensitivity analyses would help. Am. Econ. Rev. 1985;75(3):308–313.
   Web of Science ®Google Scholar
• Canova F. Sensitivity analysis and model evaluation in simulated dynamic general equilibrium economies. Int. Econ. Rev. 1995;36(2):477–501. doi: 10.2307/2527207
   Web of Science ®Google Scholar
• Spall JC. Introduction to stochastic search and optimization: estimation, simulation, and control. Hoboken, NJ: Wiley; 2003.
   Google Scholar
• Chib S. Markov Chain Monte Carlo technology. In: Gentle JE, Härdle WK, Mori Y, editors, Handbook of Computational Statistics. Springer Handbooks of Computational Statistics. Berlin Heidelberg: Springer; 2012. p. 73–104.
   Google Scholar
• Cowles MK, Carlin BP. Markov Chain Monte Carlo convergence diagnostics: a comparative review. J. Am. Stat. Assoc. 1996;91(434):883–904. doi: 10.1080/01621459.1996.10476956
   Web of Science ®Google Scholar
• Flegal JM, Jones GL. Implementing MCMC: Estimating with confidence. In: Brooks S, Gelman A, Jones G, Meng XL, editors. Handbook of Markov Chain Monte Carlo, chapter 7. Boca Raton, FL: CRC Press; 2011.
   Google Scholar
• Gelman A, Shirley K. Inference and monitoring convergence. In: Brooks S, Gelman A, Jones G, Meng XL, editors. Handbook of Markov Chain Monte Carlo, chapter 6. Boca Raton, FL: CRC Press; 2011.
   Google Scholar
• Gilman MJ. A brief survey of stopping rules in Monte Carlo simulations. In: Proceedings of the second conference on applications of simulations. Winter Simulation Conference; New York, NY; 1968. p. 16–20.
   Google Scholar
• Flegal JM, Haran M, Jones GL. Markov Chain Monte Carlo: Can we trust the third significant figure? Stat. Sci.. 2008;23(2):250–260. doi: 10.1214/08-STS257
   Web of Science ®Google Scholar
• Sala-i-Martin X. I just ran two million regressions. Am. Econ. Rev. 1997;87(2):178–183.
   Web of Science ®Google Scholar
• Cherchye L, Moesen W, Rogge N, et al. Creating composite indicators with DEA and robustness analysis: the case of the technology achievement index. J. Oper. Res. Soc. 2007;59(2):239–251. doi: 10.1057/palgrave.jors.2602445
   Web of Science ®Google Scholar
• Kovacevic M, Aguña CG. Uncertainty and sensitivity analysis of the Human Development Index. Human Development Research Paper 2010/47, Human Development Report Office (HDRO), United Nations Development Programme (UNDP); 2010.
   Google Scholar
• Nussbaumer P, Bazilian M, Modi V. Measuring energy poverty: focusing on what matters. Renew. Sustain. Ener. Rev. 2012;16(1):231–243. doi: 10.1016/j.rser.2011.07.150
   Web of Science ®Google Scholar
• WA Gale, Sampson G. Good-Turing frequency estimation without tears. J. Quant. Linguist. 1995;2(3):217–237. doi: 10.1080/09296179508590051
   Google Scholar
• IJ Good. The population frequencies of species and the estimation of population parameters. Biometrika. 1953;40(3–4):237–264. doi: 10.1093/biomet/40.3-4.237
   Web of Science ®Google Scholar
• Saisana M, Saltelli A, Tarantola S. Uncertainty and sensitivity analysis techniques as tools for the quality assessment of composite indicators. J. R. Stat. Soc. A Stat. 2005;168(2):307–323. doi: 10.1111/j.1467-985X.2005.00350.x
   Web of Science ®Google Scholar
• Saltelli A, Chan K, Scott EM. Sensitivity analysis. Hoboken, NJ: Wiley; 2009.
   Google Scholar
• James F. Monte Carlo theory and practice. Rep. Prog. Phys. 1980;43(9):1145. doi: 10.1088/0034-4885/43/9/002
   Web of Science ®Google Scholar
• Hastings WK. Monte Carlo sampling methods using Markov chains and their applications. Biometrika. 1970;57(1):97–109. doi: 10.1093/biomet/57.1.97
   Web of Science ®Google Scholar
• Niederreiter H. Quasi-Monte Carlo methods and pseudo-random numbers. B. Am. Math. Soc. 1978;84(6):957–1041. doi: 10.1090/S0002-9904-1978-14532-7
   Web of Science ®Google Scholar
• FJ Solis, RJ-B Wets. Minimization by random search techniques. Math. Oper. Res. 1981;6(1):19–30. doi: 10.1287/moor.6.1.19
   Web of Science ®Google Scholar
• Janssen H. Monte-Carlo based uncertainty analysis: sampling efficiency and sampling convergence. Reliab. Eng. Syst. Safe. 2013;109:123–132. doi: 10.1016/j.ress.2012.08.003
   Web of Science ®Google Scholar
• Hutchison DW, Spall JC. Stopping small-sample stochastic approximation. In: American Control Conference, IEEE; St. Louis, MO; 2009. p. 26–31.
   Google Scholar
• Chaudhuri S, Motwani R, Narasayya V. Random sampling for histogram construction: how much is enough? SIGMOD Rec. 1998;27(1):436–447. doi: 10.1145/276305.276343
   Google Scholar
• Efron B, Thisted R. Estimating the number of unseen species: how many words did Shakespeare know? Biometrika. 1976;63(3):435–447.
   Web of Science ®Google Scholar
• Berend D, Kontorovich A. The missing mass problem. Stat. Prob. Lett. 2012;82(6):1102–1110. doi: 10.1016/j.spl.2012.02.014
   Web of Science ®Google Scholar
• Bunge J, Fitzpatrick M. Estimating the number of species: A review. J. Am. Stat. Assoc. 1993;88(421):364–373.
   Web of Science ®Google Scholar
• Gandolfi A, Sastri CCA. Nonparametric estimations about species not observed in a random sample. Milan J. Math. 2004;72(1):81–105. doi: 10.1007/s00032-004-0031-8
   Google Scholar
• Bedeian AG, Sturman MC, Streiner DL. Decimal dust, significant digits, and the search for stars. Organ. Res. Methods. 2009;12(4):687–694. doi: 10.1177/1094428108321153
   Web of Science ®Google Scholar
• P-T Wilrich. Rounding of measurement values or derived values. Measurement. 2005;37(1):21–30. doi: 10.1016/j.measurement.2004.08.005
   Web of Science ®Google Scholar
• Cherchye L, Ooghe E, Van Puyenbroeck T. Robust human development rankings. J. Econ. Inequal. 2008;6(4):287–321. doi: 10.1007/s10888-007-9058-8
   Web of Science ®Google Scholar
• Ravallion M. Good and bad growth: the human development reports. World Dev. 1997;25(5):631–638. doi: 10.1016/S0305-750X(96)00141-6
   Web of Science ®Google Scholar
• Wolff H, Chong H, Auffhammer M. Classification, detection and consequences of data error: Evidence from the human development index. Economic J. 2011;121(553):843–870. doi: 10.1111/j.1468-0297.2010.02408.x
   Web of Science ®Google Scholar
• UNDP. Human Development Report 2014 – Sustaining Human Progress: Reducing Vulnerabilities and Building Resilience. New York, NY: United Nations Development Programme; 2014. Available from: http://hdr.undp.org/en/content/human-development-report-2014.
   Google Scholar
• McAllester DA, Schapire RE. On the convergence rate of Good-Turing estimators. In: Proceedings of the Thirteenth Annual Conference on Computational Learning Theory, COLT; Palo Alto, CA; 2000. p. 1–6.
   Google Scholar
• Auger A, Hansen N. Theory of evolution strategies: New perspectives. In: Auger A, Doerr B, editors. Theory of randomized search heuristics: foundations and recent developments, chapter 10. Singapore: World Scientific; 2011.
   Google Scholar
• Manski CF. Identification for prediction and decision. Cambridge, MA: Harvard University Press; 2009.
   Google Scholar
• Zabinsky ZB. Random search algorithms. In: Cochran JJ, Louis Anthony Cox J, Keskinocak P, Kharoufeh JP, Smith JC, editors. Wiley Encyclopedia of Operations Research and Management Science. Hoboken, NJ: Wiley; 2011.
   Google Scholar
• Orlitsky A, Santhanam NP, Zhang J. Always Good Turing: asymptotically optimal probability estimation. Science. 2003;302(5644):427–431. doi: 10.1126/science.1088284
   PubMed Web of Science ®Google Scholar
• Wagner A, Viswanath P, Kulkarni S. Strong consistency of the Good-Turing estimator. In: 2006 IEEE International Symposium on Information Theory. IEEE; 2006. p. 2526–2530. doi:10.1109/ISIT.2006.262066.
   Google Scholar