Embedded Files
In-person course, University of Sheffield, Monday 7th September 2026, 10:00am - 16:00pm (BST)
About the course
Structured expert elicitation (SEE) is a rigorous and transparent approach for quantifying expert judgement as probabilistic distributions, enabling uncertainty to be explicitly incorporated into decision models. It is increasingly used in health technology assessment (HTA), particularly where evidence is immature, incomplete, or unavailable for key long-term outcomes.
A frequent application is survival extrapolation in cost-effectiveness modelling. In many HTA submissions, multiple statistical models may fit observed trial survival data equally well yet generate substantially different long-term survival predictions, leading to wide variation in incremental cost-effectiveness results. In these situations, SEE provides a structured way to formally elicit expert beliefs about long-term outcomes and use them to inform model selection, parameterisation, and sensitivity analysis in a transparent and decision-relevant way.
This one-day, in-person course provides a practical introduction to SEE in HTA, with a strong focus on survival modelling applications and implementation aligned with NICE Decision Support Unit (DSU) Technical Support Document (TSD) 26: Expert elicitation for long-term survival outcomes. Participants will learn how to design, conduct, and interpret elicitation exercises that are robust, reproducible, and suitable for use in regulatory and reimbursement settings. Participants will also gain hands-on experience using free software tools to implement structured expert elicitation in practice.
The course is grounded in real HTA challenges and reflects the growing reliance on immature clinical data in reimbursement submissions, where structured approaches to uncertainty are essential for credible decision making.
What you will learn
By the end of the course, participants will be able to:
Explain the principles of structured expert elicitation and its role in HTA decision-making
Understand key sources of uncertainty in survival extrapolation and why they matter for cost-effectiveness results
Describe best practice methods for eliciting expert judgement, including approaches to reduce cognitive bias and improve reliability
Translate elicited expert opinion into probabilistic distributions for use in economic models
Conduct structured expert elicitation using dedicated software tools
Critically appraise SEE studies in line with NICE DSU TSD 26 guidance
Recognise when SEE is appropriate and how it can strengthen model selection, scenario analysis, and decision uncertainty in HTA submissions
Course pre-requisites
Participants should have knowledge of basic concepts in probability theory and statistics. Familiarity with the definitions in the Glossary of NICE TSD 26 is recommended.
Software
Participants need to bring a laptop with R, R Studio and the SHELF R package installed. Knowledge of using R is not required for this course.
Course Fees
Academic, public sector: £280
Commercial sector: £400
Location
The Hicks Building is located on the University of Sheffield's central campus, a short distance from Sheffield city centre. Sheffield Railway Station is the nearest main train station, approximately a 20–25 minute walk from the venue, or a short tram or taxi ride away. Frequent tram services run from the station to the University tram stop, which is just one minutes' walk from the building.
Contacts
For further information please email Jess Forsyth (j.e.forsyth@sheffield.ac.uk).
For enquiries about course content contact Jeremy Oakley (j.oakley@sheffield.ac.uk) or Kate Ren (s.ren@sheffield.ac.uk).
Course programme
The course covers the following sessions.
09:30 registration
10:00 start
16:00 finish
Session 1: Introduction to structured expert elicitation (SEE)
The process of eliciting a probability distribution from one or more experts
Managing cognitive biases in probability judgements
Protocols for SEE
Session 2: NICE DSU TSD 26: SEE for survival extrapolation
What makes survival extrapolation distinct from other elicitation problems
Incorporating qualitative judgements about hazard trends
Adapting generic SEE protocols for survival extrapolation
Session 3: Software
Using the SHELF R package and supporting apps for SEE
General elicitation and distribution fitting
Incorporating individual patient data to support survival extrapolation
Session 4: SEE for HTA submissions
Planning and operating an SEE
Documenting the SEE process
Quality assurance and demonstrating validity
Course instructors
The instructors are all authors of NICE DSU TSD 26
Jeremy Oakley PhD is a Professor of Statistics at the University of Sheffield. He has over 25 years of experience in methodological research in structured expert elicitation, and has extensive practical experience across multiple domains, including HTA. Together with Professor Anthony O’Hagan, he co-developed the Sheffield Elicitation Framework (SHELF), a widely used framework for eliciting and combining expert judgements in situations where empirical data are limited. He is also the author of the SHELF R package.
Kate Ren PhD is a Professor of Statistical Health Technology Assessment at the Sheffield Centre for Health and Related Research (SCHARR) and a member of SCHARR Technology Assessment Group at the University of Sheffield. She is also Director of Statistics at ConnectHEOR. She has extensive expertise in structured expert elicitation, having designed and conducted multiple elicitation studies to inform health technology assessment and decision making. She is a member of SCHARR-TAG, and serves on committees for the National Institute for Health and Care Excellence. More broadly, her work spans indirect treatment comparisons and survival analysis, alongside advancing, refining, and critically evaluating statistical methodologies to support robust and transparent decision making in health technology assessment.
Jessica Forsyth PhD is a research associate in statistics specialising in HTA and has substantial expertise in HTA through her role at the University of Sheffield within SCHARR-TAG. Her work involves the critical evaluation of statistical approaches used in technology appraisal submissions, including expert elicitation, evidence synthesis methods, indirect treatment comparisons, survival modelling for extrapolation, and health-related quality-of-life analyses. She has also contributed extensively to advancing the use of structured expert elicitation within HTA, including developing and refining methodological recommendations, supporting the design and conduct of elicitation exercises.
Page updated
Report abuse