Expert Knowledge Elicitation: Subjective but Scientific

References

• Brownstein, N., Louis, T., O’Hagan, A., and Pendergast, J. (2018), “The Role of Expert Judgement in Statistical Inference and Evidence-Based Decision-Making,” The American Statistician, this issue, DOI: 10.1080/00031305.2018.1529623.
   PubMed Web of Science ®Google Scholar
• Burgman, M. A. (2015), Trusting Judgements: How to Get the Best Out of Experts, Cambridge, UK: Cambridge University Press.
   Google Scholar
• CFWI (2015), Horizon 2035 – Future Demand for Skills: Initial results. National Archives, available at https://www.gov.uk/government/publications/horizon-2035-future-demand-for-skills-initial-results
   Google Scholar
• Cooke, R. M. (1991), Experts in Uncertainty: Opinion and Subjective Probability in Science, Oxford, UK: Oxford University Press.
   Google Scholar
• Cooke, R. M., and Goossens, L. H. J. (2008), “TU Delft Expert Judgement Database,” Reliability Engineering and System Safety, 93, 745–756. DOI: 10.1016/j.ress.2007.03.017.
   Web of Science ®Google Scholar
• Cooke, R. M., Wittmann, M. E., Lodge, D. M., Rothlisberger, J. D., Rutherford, E. S., Zhang, H., and Mason, D. M. (2014), “Out-of-sample Validation for Structured Expert Judgement of Asian Carp Establishment in Lake Erie,” Integrated Environmental Assessment and Management, 10, 522–528. DOI: 10.1002/ieam.1559.
   PubMed Web of Science ®Google Scholar
• de Finetti, B. (1937), “Foresight: Its Logical Laws, its Subjective Sources,” Reprinted (and translated from the original French) in Studies in Subjective Probability, eds. H. E. Kyburg, Jr. and H. E. Smokler, New York: Wiley, 1964.
   Google Scholar
• de Finetti, B. (1970), Theory of Probability: A Critical Introductory Treatment (translation by A. Machi and A. F. M. Smith, 1974–5), 2 volumes. Wiley.
   Google Scholar
• European Food Safety Authority (2014), “Guidance on Expert Knowledge Elicitation in Food and Feed Safety Risk Assessment,” EFSA Journal 2014, 12 (6), 3734.
   Google Scholar
• Fischhoff, B., Slovic, P., and Lichtenstein, S. (1978), “Fault Trees: Sensitivity of Estimated Failure Probabilities to Problem Representation,” Journal of Experimental Psychology: Human Perception and Performance, 4, 330–344.
   Web of Science ®Google Scholar
• French, S. (1985), “Group Consensus Probability Distributions: A Critical Survey,” in Bayesian Statistics 2, J. M. Bernardo et al (eds.), Oxford, UK: Oxford University Press, pp. 183–202.
   Google Scholar
• Gosling, J. P. (2018), “SHELF: The Sheffield Elicitation Framework,” in Elicitation: The Science and Art of Structuring Judgement, eds. L. C. Dias, A. Morton and J. Quigley, Springer, pp. 61–93.
   Google Scholar
• Hanea, A. M., McBride, M. F., Burgman, M. A., and Wintle, B. C. (2018), “Classical Meets Modern in the IDEA Protocol for Structured Expert Judgement,” Journal of Risk Research, 4, 417–433. DOI: 10.1080/13669877.2016.1215346.
   Web of Science ®Google Scholar
• Hemming, V., Burgman, M. A., Hanea, A. M., McBride, M. F., and Wintle, B. C. (2018), “A Practical Guide to Structured Expert Elicitation using the IDEA Protocol,” Methods in Ecology and Evolution, 9, 169–1980. DOI: 10.1111/2041-210X.12857.
   Web of Science ®Google Scholar
• Janis, I. L. (1972), Victims of Groupthink: A Psychological Study of Foreign-Policy Decisions and Fiascoes, Boston, MA: Houghton Mifflin.
   Google Scholar
• Kahneman, D. (2011), Thinking, Fast and Slow, New York: Farrar, Straus and Giroux.
   Google Scholar
• Kurowicka, D., and Cooke, R. M. (2006), Uncertainty Analysis with High Dimensional Dependence Modelling, Chichester: Wiley.
   Google Scholar
• Oakley J. E., and O’Hagan, A. (2016), “SHELF: the Sheffield Elicitation Framework” (version 3.0), School of Mathematics and Statistics, University of Sheffield, UK, available at http://tonyohagan.co.uk/shelf.
   Google Scholar
• O’Hagan, A., Buck, C. E., Daneshkhah, A., Eiser, J. R., Garthwaite, P. H., Jenkinson, D. J., Oakley, J. E., and Rakow, T. (2006), Uncertain Judgements: Eliciting Expert Probabilities, Chichester, UK: Wiley.
   Google Scholar
• O’Hagan, A. (2012), “Probabilistic Uncertainty Specification: Overview, Elaboration Techniques and their Application to a Mechanistic Model of Carbon Flux,” Environmental Modelling and Software, 36, 35–48. DOI: 10.1016/j.envsoft.2011.03.003.
   Web of Science ®Google Scholar
• Parducci, A. (1963), “The Range-Frequency Compromise in Judgment,” Psychological Monographs, 77, 1–50. DOI: 10.1037/h0093829.
   Google Scholar
• Ramsey, F. P. (1926), “Truth and Probability,” reprinted in Studies in Subjective Probability, eds. H. E. Kyburg, Jr. and H. E. Smokler, 2nd edition, New York: R. E. Krieger Publishing Company, pp. 23–52.
   Google Scholar
• Rowe, G., and Wright, G. (1999), “The Delphi Technique as a Forecasting Tool; Issues and Analysis,” International Journal of Forecasting, 15, 353–375. DOI: 10.1016/S0169-2070(99)00018-7.
   Web of Science ®Google Scholar
• Savage, L. J. (1954), The Foundations of Statistics. New York: Wiley.
   Google Scholar
• Tversky, A., and Kahneman, D. (1974), “Judgments under Uncertainty: Heuristics and Biases,” Science, 185, 1124–1131. DOI: 10.1126/science.185.4157.1124.
   PubMed Web of Science ®Google Scholar