- Open Access
- Open Peer Review
Economic evaluation alongside pragmatic randomised trials: developing a standard operating procedure for clinical trials units
Trialsvolume 9, Article number: 64 (2008)
There is wide recognition that pragmatic randomised trials are the best vehicle for economic evaluation. This is because trials provide the best chance of ensuring internal validity, not least through the rigorous prospective collection of patient-specific data. Furthermore the marginal cost of collecting economic data alongside clinical data is typically modest. UK Clinical Research Collaboration (UKCRC) does not require a standard operating procedure (SOP) for economic evaluation as a prerequisite for trial unit registration. We judge that such a SOP facilitates the integration of health economics into trials.
A collaboration between health economists and trialists at Bangor University led to the development of a SOP for economic evaluation alongside pragmatic trials, in addition to the twenty SOPs required by UKCRC for registration, which include randomisation, data management and statistical analysis.
Our recent telephone survey suggests that no other UKCRC-registered trials unit currently has an economic SOP.
We argue that UKCRC should require, from all Trials Units undertaking economic evaluation and seeking registration or re-registration, a SOP for economic evaluation as one of their portfolio of supporting SOPs.
There is wide recognition that pragmatic randomised trials are the best vehicle for economic evaluation [1–4]. This is because trials provide the best chance of ensuring internal validity, not least through the rigorous prospective collection of patient-specific data. Furthermore the marginal cost of collecting economic data alongside clinical data is typically modest .
Several text books and journal articles define best practice for economic evaluation alongside clinical trials. However, meeting the defined standards and integrating health economics into trial protocols and procedures is still a challenge. For example Drummond et al add a survival guide for health economists to the latest edition of their text book !
The health economics team at Bangor University collaborated with the North Wales Organisation for Randomised Trials in Health [NWORTH], the regional trials unit, in its application to the UK Clinical Research Collaboration [UKCRC] in 2007 for full registration, offering health economics as an additional methodological capability. This collaboration led to the development of a standard operating procedure [SOP] for economic evaluation alongside pragmatic trials, in addition to the twenty SOPs required by UKCRC for registration, which include randomisation, data management and statistical analysis. Though UKCRC does not require a SOP for economic evaluation as a prerequisite for trial unit registration, we judge that such a SOP facilitates the integration of health economics into trials. Our recent telephone survey suggests that no other UKCRC-registered trials unit currently has a similar SOP available.
We had previously undertaken economic evaluations alongside pragmatic RCTs [5, 6]. So we based this SOP on previous experience and current health economics resources. Our SOP needed to cover the range of trials to which we currently contribute economic evaluation. These include trials in clinical medicine, for example COGNATE (trial of endoscopic ultrasonography in gastro-oesophageal cancer – ISRCTN 01444215) and FolATED (trial of folate as adjunct to anti-depressant therapy ISRCTN 37558856) ; trials of psycho-social interventions, notably RemCare (trial of reminiscence therapy for patients with dementia – ISRCTN 42430123); and public health trials, notably CHARISMA (trial of housing improvements for children with asthma – ISRCTN 13912429). The current trials all evaluate 'complex interventions' in many centres by adopting a pragmatic whole-system approach to trial design and conduct, and hence to the concomitant design and conduct of economic evaluation .
Economists and trialists work closely together to put health economics at the core of trials co-ordinated from Bangor. For example we appointed a health economist as trial manager and as health economist to the RemCare trial. This has enabled us to integrate economic evaluation into trial protocol preparation, design of information sheet and consent form, application for ethical approval, and early preparation of analysis plans and of publication policy. In this way we are meeting the defined standards for the design, conduct and reporting of economic evaluations [1–4].
The Bangor SOP [see Additional file 1] for economic evaluation alongside pragmatic trials, summarised in Figure 1, divides economic evaluation activities into three phases: those preceding funding, thus requiring the development and sustainability of a health economics research group, notably pre-funding protocol development; data collection and management; and economic and statistical analysis leading to publication. Phase I ensures that health economics is integral to the development of rigorous and comprehensive protocols. Phase II seeks to achieve a comprehensive data set in which clinical and economic data complement each other. Phase III completes the symbiotic process of clinical and economic evaluation. This SOP is being considered for adoption by other trials units in Wales. The SOP is also available electronically http://www.bangor.ac.uk/healtheconomics and http://www.bangor.ac.uk/imscar/nworth
SOPs cannot themselves overcome the two main limitations of conducting economic evaluations alongside clinical trials – namely the length of follow-up in a typical trial and the limited number of treatment options compared [2–4]. Nevertheless, Drummond et al have argued that standard reporting frameworks for economic evaluation promote transparency, improve comparability of published studies and stimulate economists to address challenging methodological issues [2, 9]. Similarly, we have found that the development of a SOP for economic evaluations alongside pragmatic trials managed by a registered trials unit improve trial conduct and hence the validity and generalisability of results. Above all, this process leads to a research culture in which clinical and economic findings receive equal credit.
Thus, we argue that UKCRC should require from all Trials Units undertaking economic evaluation, and seeking registration or re-registration, a SOP for economic evaluation as one of their portfolio of supporting SOPs.
Ramsey S, Willke R, Briggs A, Brown R, Buxton M, Chawla A: Good research practices for cost-effectiveness analysis alongside clinical trials: the ISPOR RCT-CEA task force report. Value in Health. 2005, 8: 521-533. 10.1111/j.1524-4733.2005.00045.x.
Drummond M, Sculpher MJ, Torrance GW, O'Brien BJ, Stoddart GL: Methods for the Economic Evaluation of Health Care Programmes. 2005, Oxford: Oxford University Press, 3
Glick HA, Doshi JA, Sonnad SS, Polsky D: Economic Evaluation in Clinical Trials (Handbooks for Health Economic Evaluation). 2007, Oxford: Oxford University Press
Brazier J, Ratcliffe J, Salomon J, Tsuchiya A: Measuring and Valuing Health Benefits for Economic Evaluation. 2007, Oxford: Oxford University Press
Williams NH, Edwards RT, Linck P, Muntz R, Hibbs R, Wilkinson C: Cost-utility analysis of osteopathy in primary care: results from a pragmatic randomised controlled trial. Family Practice. 2004, 21: 643-650. 10.1093/fampra/cmh612.
Edwards RT, Ó Céilleachair AJ, Bywater T, Hutchings J: Parenting programme for children at risk of developing conduct disorder: cost-effectiveness analysis. BMJ. 2007, 334: 682-687. 10.1136/bmj.39126.699421.55.
Roberts SH, Bedson E, Hughes DA, Lloyd K, Moat S, Pirmohamed M: Folated Augmentation of Treatment – Evaluation for Depression (FolATED): protocol of a randomised controlled trial. BMC Psychiatry. 2005, 7: 65-10.1186/1471-244X-7-65.
Medical Research Council: Framework for development and evaluation of RCTs for complex interventions to improve health. 2000, London: MRC
Drummond M, Manca A, Sculpher M: Increasing the generalizability of economic evaluations: recommendations for the design, analysis, and reporting of studies. Int J Technol Assess Health Care. 2005, 21: 165-171.
The authors would like to thank members of the NWORTH Standard Operating Procedure Development Group including, Ceri Bray, Angela Gliddon, Rashesh Mehta, Daphne Russell, Simant Westley, and Rhiannon Whitaker, for their helpful comments and suggestions. Thanks are also due to Dyfrig Hughes for his advice and comments. Emily Bacon and Alison Shaw helped with the preparation of the manuscript and provided administrative support.
The Centre for Economics and Policy in Health receives some funding from the National Public Health Service for Wales.
The authors declare that they have no competing interests.
RTE and BH conceived the drafted the standard operating procedure and manuscript. PL and ITR critically reviewed and revised the standard operating procedure and manuscript. All authors read and approved the final manuscript.
Electronic supplementary material
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.