Reading list

Background

Our early thinking behind SHELF was based on the findings of a research project on best practice in elicitation, funded by the NHS’s Research Methodology Programme. The project was a collaboration between statisticians and psychologists, and the main output was this book:

O' Hagan, A., Buck, C. E., Daneshkhah, A., Eiser, J. E., Garthwaite, P. H., Jenkinson, D. J., Oakley, J. E. and Rakow, T. (2006) Uncertain judgements: Eliciting expert probabilities. Chichester: Wiley.

We have contributed to a more recent review/guidance document, which describes SHELF as well as Cooke’s classical method and the Delphi method:

European Food Safety Authority (2014) Guidance on expert knowledge elicitation in food and feed safety risk assessment. EFSA Journal 2014, 12(6): 3734, 278 pp.
DOI: 10.2903/j.efsa.2014.3734

John Paul Gosling has written about SHELF in an edited volume on elicitation methods:

Gosling, J. P. (2018) SHELF: The Sheffield Elicitation Framework. In Dias, L., Morton, A., Quigley, J. (eds.) Elicitation. International series in operations research and management science, vol. 261. Springer, Cham.
DOI: https://doi.org/10.1007/978-3-319-65052-4_4

Applications

These papers give examples of using SHELF:

• O’Hagan, A. (2018) Expert knowledge elicitation: Subjective but scientific. The American Statistician, 73:sup1, 69-81.
  DOI: 10.1080/00031305.2018.1518265

• Jurek, L., Balthazar, M., Gulati, S., Novakovic, N., Nunez, M., Oakley, J. and O’Hagan, A. (2021). Response (minimum clinically relevant change) in ASD symptoms after an intervention according to CARS-2: Consensus from an expert elicitation procedure. European Child and Adolescent Psychiatry.
  DOI: 10.1007/s00787-021-01772-z

• Holzhauer, B., Hampson, L. V., Gosling, J. P., Bornkamp, B., Kahn, J., Lange, M. R., Luo, W-L., Brindicci, C., Lawrence, D., Ballerstedt, S. and O’Hagan, A. (2022) Eliciting judgements about dependent quantities of interest: The Sheffield Elicitation Framework extension and copula methods illustrated using an asthma case study. Pharmaceutical Statistics, 1-17.
  DOI: 10.1002/pst.2212

This paper discusses the use of expert elicitation with SHELF at GlaxoSmithKline:

Dallow, N., Best, N., Montague, T. H. (2018) Better decision making in drug development through adoption of formal prior elicitation. Pharmaceutical Statistics, 17, 301–316.
DOI: https://doi.org/10.1002/pst.1854

This paper presents a case study in which SHELF is compared with Cooke’s classical method:

Williams, C. J., Wilson, K. J. and Wilson, N. (2021) A comparison of prior elicitation aggregation using the classical method and SHELF. J. R. Stat. Soc. Series A, 184, 920-940.
DOI: https://doi.org/10.1111/rssa.12691