Skip to content


  • Poster presentation
  • Open Access

External validity of rct evidence in cost-effectiveness analyses. A review of recent technology appraisals for nice and proposed methods of adjustment

  • 1,
  • 1,
  • 1 and
  • 1
Trials201516 (Suppl 2) :P25

  • Published:


  • Public Health
  • Treatment Effect
  • Subgroup Analysis
  • Model Output
  • External Validity


Randomised Controlled Trials (RCTs) produce relative treatment effects that have high internal validity. However they may suffer from lack of external validity when used as inputs to cost-effectiveness analyses (CEA). If differences in RCT and CEA populations can be described by a set of observed covariates, then it is possible to adjust for these differences. A common approach is to estimate effects at the mean covariate values. For non-linear models this introduces bias, even if the RCT and CEA populations are comparable.


(i) To review recent technology appraisals (TAs) submitted to NICE to see how covariate adjustment in CEA is typically conducted and (ii) to explore the impact that failing to appropriately adjust for covariates can have on model outputs.


We reviewed all TAs issued by NICE in 2014. We explore, for binary outcomes, when the bias from using the covariate mean approach is likely to have an impact.


7/25(28%) reported important differences between RCT and CEA populations, and only 4 of those 7 adjusted for covariates in the CEA model. 17 of the 25 TAs adjusted for covariates, of which 3 used the mean of the covariates, 2 averaged over the covariate distribution, and 8 performed subgroup analyses, and the method of adjustment was unclear in 4. The bias introduced by using the mean of the covariates can be large and increases with the variance in the covariate distribution.


Covariate adjustment requires integration over the joint covariate distribution in the CEA population.

Authors’ Affiliations

University of Bristol, Bristol, UK


© Mantopoulos et al. 2015

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.


By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate. Please note that comments may be removed without notice if they are flagged by another user or do not comply with our community guidelines.