Skip to main content

How can health economics be used in the design and analysis of adaptive clinical trials? A qualitative analysis



Adaptive designs offer a flexible approach, allowing changes to a trial based on examinations of the data as it progresses. Adaptive clinical trials are becoming a popular choice, as the prudent use of finite research budgets and accurate decision-making are priorities for healthcare providers around the world. The methods of health economics, which aim to maximise the health gained for money spent, could be incorporated into the design and analysis of adaptive clinical trials to make them more efficient. We aimed to understand the perspectives of stakeholders in health technology assessments to inform recommendations for the use of health economics in adaptive clinical trials.


A qualitative study explored the attitudes of key stakeholders—including researchers, decision-makers and members of the public—towards the use of health economics in the design and analysis of adaptive clinical trials. Data were collected using interviews and focus groups (29 participants). A framework analysis was used to identify themes in the transcripts.


It was considered that answering the clinical research question should be the priority in a clinical trial, notwithstanding the importance of cost-effectiveness for decision-making. Concerns raised by participants included handling the volatile nature of cost data at interim analyses; implementing this approach in global trials; resourcing adaptive trials which are designed and adapted based on health economic outcomes; and training stakeholders in these methods so that they can be implemented and appropriately interpreted.


The use of health economics in the design and analysis of adaptive clinical trials has the potential to increase the efficiency of health technology assessments worldwide. Recommendations are made concerning the development of methods allowing the use of health economics in adaptive clinical trials, and suggestions are given to facilitate their implementation in practice.

Peer Review reports


Decision-makers around the world are faced with limited budgets for funding health research, such as new clinical trials, and funding health technologies, such as new drugs [13]. Resources spent on funding new research may otherwise be spent directly on patient care, and so it is desirable to consider the cost-effective use of limited resources. The methods of health economics facilitate resource allocation decisions by evaluating the maximum health gained for the resources spent [4].

Cost-effectiveness plays an important role at two points in the health technology assessment (HTA) process: to determine whether it is cost-effective to fund a new health technology and whether it is cost-effective to fund a piece of research. For the first of these questions, cost-effectiveness analysis compares the costs and benefits (often measured using quality-adjusted life years (QALYs)) of competing health technologies to identify which represent value for money [4]. For the second type of evaluation, value of information analysis (VOIA) assesses whether it is worthwhile collecting further information, such as performing further clinical trials, to reduce decision uncertainty [5, 6]. Amongst other inputs, a reliable health economic analysis requires an accurate estimate of the treatment effect and associated confidence interval, often provided by a clinical trial.

Adaptive designs are one innovative approach to conducting a clinical trial. Unlike a traditional fixed sample size design, data are examined as the trial progresses to inform modifications to the trial. This can potentially save time and resources, as well as prevent patients from being needlessly randomised [79]. The number of trials using adaptive methods has increased from 11 per 10,000 registered trials between 2001 and 2005 to 38 per 10,000 registered trials between 2012 and 2013 [10].

Adaptive designs and their implementation are commonly based on demonstrating clinical effectiveness. Despite its importance, cost-effectiveness is often a secondary consideration [11]. It is currently unclear what impact the use of an adaptive design has on a health economic analysis. Additionally, opportunities are potentially being missed to incorporate health economics into the design and analysis of adaptive clinical trials. The objective of this paper is to report the views of stakeholders on the use of health economics in adaptive clinical trials.


We adopted a pragmatic approach to understanding issues around the use of health economics in adaptive designs [12]. An initial thematic framework was developed, based on the research aims, literature and discussions with our public advisory group (a group of members of the public advising on the research). Initial themes identified included the following: Pre-specification of Health Economic Analysis; Private Sector Attitude; Aims of a Clinical Trial and Advantages and Limitations of Adaptive Designs. Once data collection was under way, we amended the framework based on inferences about participant perspectives with no obvious place in the initial framework. The final framework in Table 1 condensed the themes into three main headings reflecting the key areas of interest: ethical, methodological and practical issues.

Table 1 Final thematic framework applied to the data collected from interview and focus groups HTA; health technology assessment

We used a purposive sample to identify the views of a range of key stakeholders in the HTA process. Members of the public were recruited regardless of their experiences as a patient, a user of the National Health Service (NHS) or a participant in clinical trials. We also approached researchers and decision-makers involved in HTA, especially those who might use the proposed methods.

Members of the public were first contacted through a public involvement co-ordinator at a clinical research unit at a local hospital, a departmental public involvement lead and the public advisory panel supporting the research project. After the initial data collection, to extend the reach beyond members of the public with research experience, a message was posted to an online discussion forum (Sheffield Forum). Researchers and decision-makers were identified by contacting researchers known to be working in relevant areas such as prominent health economists, statisticians and clinicians.

Semi-structured interviews and focus groups were used to collect data. Data collection with members of the public was conducted face to face at the University of Sheffield where possible; telephone interviews were available to avoid excluding members of the public who were unable to travel. Interviews were the preferred approach for researchers and decision-makers, to reduce the burden on participants, as previous related research reported difficulties in engaging health economists [13].

Separate study documentation and topic guides were produced for the public and researcher groups reflecting the technical language and information provided to the respective participants. The researcher topic guide was piloted with a colleague. We implemented recommendations from the project’s public advisory group on how to modify the topic guide for members of the public.

Prior to data collection, all participants were sent a short video introducing the key concepts of adaptive designs and health economic analysis in plain English to allow full participation in the study [14]. This video was created with the support of the public advisory group so as not include any of LF’s preconceived ideas which might influence the subsequent responses of the participant.

All interviews and focus groups were conducted by LF and lasted between 30 to 60 mins and 60 to 90 mins respectively. The interviews and focus groups were audio recorded. The interviews were anonymised and transcribed. Demographic data were collected using an online Google Form.

Based on the literature [1520], advice from the public advisory group, and keeping time constraints in mind, a target of a maximum of 20 researchers and 20 members of the public was set. Focus groups were planned to have a maximum of seven participants. At each stage the demographics of the sample were reviewed and subsequent participants selected to give a balanced and representative sample where possible, similar to the approach recommended by Francis et al. (2010) [21].

Transcriptions were imported into NVivo (QSR International v11, Melbourne, Australia). A framework analysis approach was adopted [22] and followed the recommended stages [23]:

  1. 1.

    Familiarisation with the data using the transcripts and any notes made during data collection

  2. 2.

    Coding the transcripts line by line and assigning a code from the a priori thematic framework or a new code

  3. 3.

    Updating the thematic framework with any new codes emerging from the data

  4. 4.

    Indexing transcripts using the categories and codes from the thematic framework

  5. 5.

    Charting data into a matrix summarising the data from each participant relevant to each of the categories from the thematic framework

  6. 6.

    Interpreting the results in the context of the research aims.

This included discussion of the findings with the public advisory panel. A table of recommendations was also sent to all study participants, who were given the opportunity to comment and suggest changes.

Ethics approval was granted by the University of Sheffield, School of Health and Related Research Ethics Committee (ref 009699). All participants were issued an information sheet and were required to sign a consent form or give verbal consent before data collection. The COREQ (Consolidated criteria for reporting qualitative research) checklist has been used in the reporting of this study [24] (see Additional file 1).


Description of participants

In total, 29 participants took part in the study between October 2017 and March 2018. At this point it was felt that few new themes were emerging and a range of participants had been included. We conducted 18 one-to-one interviews and two focus groups. There were no repeat interviews. The participant characteristics are summarised in Table 2. One researcher did not complete the demographic survey. There were \(\frac {2}{13}\) female researchers and \(\frac {8}{15}\) female members of the public. Researchers predominantly came from statistical or health economic areas of expertise from both the private and public sectors. Two of the 13 researchers were based in the USA. Ten of 15 members of the public had participated in research through public involvement groups and \(\frac {6}{15}\) had participated in a clinical trial.

Table 2 Demographics of participants who took part in interviews or focus groups. One researcher did not complete the demographics questionnaire

The health economists interviewed had limited experience with adaptive designs and reported that they are frequently not consulted in the design of clinical trials. Researcher participants suggested that health economists and statisticians frequently work independently, with their key point of contact often being the trial chief investigator. However, study participants did not feel this negatively impacted their work.

The knowledge and experience of members of the public who took part in the study came predominately from three main sources: the media, involvement in research as public advisors or participation in a clinical trial. Participants acknowledged that the media was likely to give biased views on healthcare decision-making for the general public and there was perhaps a lack of trustworthy resources available.

Current practice

Increasing the prominence of cost-effectiveness; improving the quality of health economic analyses; and preventing adaptive designs stopping before there is sufficient evidence for an accurate health economic analysis were suggested by study participants as some of the advantages of using health economics in adaptive designs. A study participant suggested that while formal VOIA calculations are rare, informal ‘rules of thumb’ based on the same ideas are commonplace. Developing decision rules for cost-effectiveness of the research similar to the rule for clinical effectiveness (p <0.05) and cost-effectiveness of the intervention (willingness-to-pay thresholds) would help the practical interpretation and implementation of this approach.

Few participants had seen the use of health economics in the design and analysis of adaptive clinical trials in practice. It was felt that further work is needed to extend the existing theory to the real-world context, and concerns were raised that combining these two complex methods might be deemed too challenging and hinder their uptake. It was also suggested that improvements to the methods of VOIA were required to provide accepted standards such as the time horizon over which value of information is calculated. The use of adaptive designs and VOIA in trial design and research prioritisation would need to be more commonplace before adding further complexity and combining the two approaches in a trial.

putting both together is going to be doubly complicated and time consuming and costly and so on …P35 Health economist

Ethical considerations

Participants in the qualitative study agreed that clinical effectiveness is the main aim of a clinical trial, whether it is adaptive or a fixed sample size design. Both researcher and public participants could appreciate the importance of demonstrating cost-effectiveness in a trial; however, they felt it was still important to answer the clinical question first.

It feels to me like it might be unfortunate to move into a world where, where clinical research, in general per se, was shaped too much, designed too much with economic considerations.…P19 Member of the public

On the other hand, some participants argued that cost-effectiveness decisions provide unseen benefits in healthcare. For example, if a trial stopped early on cost-effectiveness grounds, this would allow resources to be spent elsewhere in a more cost-effective way, conferring benefits to participants outside of the clinical trial. One researcher suggested that incorporating health economics into the adaptive design changes the focus of the trial from the individual to a population level. However, others felt this might be deemed unacceptable to some stakeholders in the HTA process.

A number of the participants agreed that a health economist should be included on a data monitoring and ethics committee (DMEC) if health economics is used as part of the design and analysis of an adaptive trial. On the other hand, one researcher argued that funding decisions should be made by the funder, and it would not be appropriate to delegate this responsibility to the DMEC.

…the DMEC have a very specific role looking at safety […]. But once you get into value of information that’s the funder’s decision and I don’t think the funder should delegate that to the DMEC. P1 Funding panel member

Methodological considerations

A number of participants were concerned about how the volatile nature of cost data would be handled if interim adaptations were made based on cost-effectiveness. This adds complexity when deciding whether to modify (or even stop) a trial on a given day when an interim analysis takes place, as this may be the wrong decision if, for example a drug price decreases.

There’s still this question of whether it is cost economic now as per tomorrow, so you might miss an opportunity. P6 Statistician

Study participants also highlighted that global multi-centre trials may encounter difficulties when using cost-effectiveness to inform interim adaptations. An interim analysis in one centre might demonstrate sufficient evidence to stop the trial early based on the cost-effectiveness rules in their jurisdiction, but this might not be reflected in the other centres around the world. Additionally, the extent to which adaptive trials incorporate health economics in the private sector might depend on whether the focus is on the US market or other countries where cost-effectiveness plays a more prominent role, such as the UK.

It was unappealing to many participants, that an adaptive design could stop early based on the cost-effectiveness of the intervention or the research itself. Alternative uses for the interim data suggested by participants included:

  1. 1.

    A hierarchy of stopping rules, where the clinical outcome is the first consideration in the adaptive decision-making and then, dependent on this result, cost-effectiveness may also be used to inform modifications to the trial.

  2. 2.

    The trial could be modified based on health economic grounds rather than the more extreme case of stopping a trial.

Practical considerations

Study participants felt that successfully implementing this approach would require sufficient resourcing, careful planning and also building a study team of researchers and clinicians who have sufficient training and are supportive of the approach. Adding the health economic layer to adaptive clinical trials raised a resourcing problem for many participants, as more time and resources will be required before a project has been funded. In the private sector, this type of impact was thought to vary depending on the size of the company.

Funding panels and HTA committees are composed of non-experts who will require sufficient training in the methods to allow them to assess their appropriateness in answering the research question and to interpret and assess the quality of the evidence. Study participants highlighted the value of developing software to help make methodology accessible and increase its use. Case studies were suggested as a way of demonstrating the value of using health economics in adaptive clinical trials. This could increase the understanding of the methods and appease the concerns of many participants regarding the complexity of this approach. The methodology would need to be presented in an accessible way for non-experts.

…the crucial thing is that the analysis has to be very understandable because […] in practice it will be unhelpful if a value of information analysis was updated and appeared to everyone involved in the trial to be a black box analysis. P1 Funding panel member

It will be increasingly important for the health economic analyses to be pre-specified if used to inform adaptive decision-making during a trial. There would need to be a greater level of scrutiny of the health economic analyses. However, concerns were also raised by participants about the potential challenge of pre-specifying everything for a health economic analysis prior to any trial results. Participants working in industry were concerned about the impact of early examinations of the data on cost-effectiveness grounds and the requirements to publicly report all analyses.

…it’s sort of …you’ve got chicken and egg here. […] Your model is developed as you look at the data but I think it would be quite difficult to do that because, yeah they sort of evolve alongside each otherP39 Health economist


Members of the public, researchers and decision-makers identified ethical, methodological and practical issues associated with the use of health economics in adaptive clinical trials. Recommendations and proposed actions were drawn from the themes identified in the data, reflecting the differing views of stakeholders in the HTA process and experiences of the research team and public advisory group. Recommendations are discussed under the three key themes—ethical, methodological and practical—and are summarised in Table 3.

Table 3 Summary of key recommendations and proposed actions from the interviews and focus groups with members of the public, researchers and decision-makers

Ethical considerations

Currently the use of health economics in the design and analysis of adaptive and non-adaptive clinical trials is largely secondary to clinical outcomes [11]. Study participants felt that for health economics to play a greater role there needs to be a shift of emphasis from the ethical perspective of beneficence—the duty to allocate resources to benefit the individual [25]—to the broader perspective of distributive justice: allocating resources to benefit the whole population [26, 27]. Changing the status quo requires the public and policy makers to find compelling ethical arguments showing that cost-effectiveness decisions provide an unseen benefit [28]. We recommend developing unbiased and simple materials on adaptive designs and health economics. Public advisors could be used to develop materials and consent forms that explain the methods of adaptive designs and health economics in sufficient detail for potential trial participants to make an informed decision to participate, as well as highlighting the broader implications of taking part in the research.

The acceptability of the role of cost-effectiveness to stakeholders is crucial when considering the direction of methods development. It is fruitless to develop statistical methods that incorporate health economics into an adaptive design if this is never implemented in practice due to concerns over the role of cost-effectiveness. For example, patients may be unwilling to be randomised, as they do not agree with decision-making centred on cost-effectiveness. Based on discussions with the study participants and the public advisory group, we recommend exploring alternative approaches, including the following:

  • Interim analyses used to check that all health economic data are being collected as required

  • Interim data used to update the health economic model

  • A hierarchy of interim decision rules where the health economic decisions are dependent on the outcome of the clinical effectiveness analysis

  • Health economics decision rules that are only considered at later interim analyses, giving the advantage of more mature data and also more weight to clinical outcomes at the start of the trial

  • Health economics used to make less extreme modifications such as increasing the sample size based on an expected value of sample information calculation.

The choice of approach could be made in collaboration with a public advisor so that the identified approach is acceptable to patients.

Methodological considerations

The advantages of using health economics in adaptive designs include saving resources and preventing patients from being needlessly randomised [11, 29, 30]. Such an approach will formalise the informal rules of thumb described by a study participant as commonly used by decision-makers and funding panels, so that they can be applied in an objective and consistent manner. We suggest that developing decision rules for cost-effectiveness of the research, similar to the rule for clinical effectiveness (p values) and cost-effectiveness of the intervention (willingness-to-pay thresholds), would help the practical interpretation and implementation of this approach. Public advisors could be used to ensure these rules are appropriate and reflect the views of patients and the public.

It will also be important to consider the limitations of both methodologies, including the potential for adaptive designs to introduce bias in the estimation of point estimates and confidence intervals and maintaining appropriate levels of blinding during interim analyses so as not to influence the conduct of the trial [31, 32].

The methods of adaptive designs and value of information analysis (VOIA) are not commonplace in HTA [10, 33, 34]. It will be vital to address the limitations of these methods to successfully implement them together in clinical trials. As identified by the study participants, these methods could be seen as more complex, potentially acting as a barrier to their use in practice. Work in the field of adaptive designs to promote their use and understanding amongst the health research community includes:

  • The Adaptive Designs Working Group, part of the Medical Research Council (MRC) Hubs for Trials Methodology Research, which aims to increase implementation of adaptive designs through tutorial papers in mainstream medical journals (see, e.g. [32]), development of software and presentations and lectures

  • The Adaptive CONSORT Extension (ACE) project, which aims to enhance transparency, credibility, reproducibility and replicability of adaptive trials by developing a consensus-driven extension to the CONSORT (Consolidated Standards of Reporting Trials) statement specific to adaptive designs [35].

Measures for increasing the use of value of information analyses include:

  • The Professional Society for Health Economics and Outcomes Research (ISPOR) Value of Information Analysis for Research Decisions task force, which aims to develop good practice guidance for using methods of value of information analysis to inform both technology reimbursement decisions and research prioritisation decisions [36]

  • Collaborative Network for Value of Information (ConVOI) group, an international network working to improve the calculation, adoption and application of value of information methods in clinical and public health research [37, 38].

We recommend the use of similar initiatives to promote the use of health economics in the design and analysis of adaptive clinical trials.

Practical considerations

If the use of health economics becomes common in the analysis of adaptive designs, then guidance on the composition of DMECs [39] should be updated, and lists which help identify DMEC members—such as StatLink [40]—could be extended to include health economists.

Such an approach will have cost consequences for clinical trials with the involvement of health economists throughout. There will be a requirement to conduct more analyses such as model conceptualisation prior to the trial being funded [13]. However, the development of a health economic model and cost-effectiveness analyses based on systematically reviewed current evidence is an essential step prior to undertaking any trial regardless of its design [41]. In the public sector, where there are few resources to bridge grant funding, researchers would need to include the costs of trial design within grant applications. Groups representing statisticians and health economists will need to persuade funders and regulators of alternative ways to fund adaptive clinical trials and value of information analyses.

If health economics is to be successfully implemented as part of an adaptive clinical trial, we recommend that trial sponsors need to ensure integrated working between health economists and statisticians throughout the trial process. More generally, across the research community links between statisticians and health economists could be encouraged through networking events and joint workshops to identify common ground and explore ways of working together more efficiently.

Pre-specification of health economic analyses will be crucial in maintaining the validity and integrity of adaptive designs that use health economics, with analysis plans including a description of the monitoring and adaptation plan, as well as pre-specification of methods used at interim analyses [29, 42]. The timing of interim analyses must be realistic given their complexity, and their blinding must be carefully considered. Cross-disciplinary training materials will be required, supplemented by case studies on the use of health economics in sequential trials [43], to raise awareness and advance methodological development by highlighting practical problems with their application.

Strengths and limitations

This is the first qualitative study to explore the ethical, methodological and practical issues of using health economics in the design and analysis of adaptive clinical trials. Participants came from different backgrounds and with experiences across the healthcare decision-making process. This study included a number of health economists, a group previously found to be hard to reach in adaptive designs research [13]. Members of the public were an important stakeholder group in this research, as the ultimate beneficiaries of clinical trials and healthcare decision-making that uses an adaptive design and health economic analysis. Additionally, members of the public are all potential participants in trials that use these methods. Failing to reflect their views in the design and analysis of clinical trials could, for example, make it difficult to recruit to trials that use an unpopular design based on cost-effectiveness.

Few female researchers took part in the study; however, it is not felt that the gender of the participants is likely to influence their responses. Instead, participants’ knowledge and experiences are more likely to inform their comments. Further work could consider a larger sample size and a broader range of experience of participants, such as programme managers at funders such as the National Institute for Health Research.

A public advisory group played an intrinsic part in the design and analysis of the qualitative study. The group were given training in adaptive designs and health economics by LF. They provided feedback on the information sheet, consent form and topic guide for members of the public to ensure the questions and proposed plan were suitable. The group developed the script for the short video sent to all study participants. After analysis was complete, the group met again to discuss the results and check the interpretation of findings. Where appropriate, their interpretation has been embedded in the paper.

A limitation of the research is that the members of the public and non-expert researchers and decision-makers were given a top-level understanding of the topics of value of information; thus, their responses are considering a broad view of cost-effectiveness considerations in clinical trials. However, the views of participants are still important in understanding the priorities of the stakeholder groups around the role of cost-effectiveness in adaptive clinical trials. Further work could consider a more detailed explanation of the methods of value of information analysis, providing scenarios of potential roles for the methods in adaptive clinical trials and using these to explore the views of stakeholders with a deeper understanding of the method.


The use of health economics in the design and analysis of adaptive clinical trials has the potential to increase the efficiency of health technology assessments worldwide. However, careful consideration is needed to ensure that the statistical methods reflect the importance of clinical effectiveness to stakeholders and that adequate training and resources are provided to facilitate the implementation of this approach.

Availability of data and materials

The data from the current study are available from the corresponding author on reasonable request.



Adaptive CONSORT Extension


Consolidated Standards of Reporting Trials


Consolidated criteria for reporting qualitative research


Data monitoring and ethics committee


Health technology assessment


The Professional Society for Health Economics and Outcomes Research


Medical Research Council


National Health Service


Practical Adaptive and Novel Designs and Analysis (Toolkit)


Quality-adjusted life year


Value of information analysis


  1. National Institute for Health and Care Excellence. Guide to the methods of technology appraisal. 2013. Accessed 12 July 2017.

  2. Ottawa: Canadian Agency for Drugs and Technologies in Health. Guidelines for the economic evaluation of health technologies: Canada. Technical report. 2006.

  3. Australian Government: Department of Health and Ageing. Guidelines for preparing submissions to the Pharmaceutical Benefits Advisory Committee (Version 4.3). Technical report. 2008.

  4. Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. Oxford: Oxford University Press; 2015.

    Google Scholar 

  5. Chilcott J, Brennan A, Booth A, Karnon J, Tappenden P. The role of modelling in prioritising and planning clinical trials. Health Technol Assess. 2003; 7(23).

  6. Strong M, Oakley JE, Brennan A, Breeze P. Estimating the expected value of sample information using the probabilistic sensitivity analysis sample: a fast, nonparametric regression-based method. Med Dec Making. 2015; 35(5):570–83.

    Article  Google Scholar 

  7. Bauer P, Brannath W. The advantages and disadvantages of adaptive designs for clinical trials. Drug Discov Today. 2004; 9(8):351–7.

    Article  PubMed  Google Scholar 

  8. Chow S, Chang M. Adaptive design methods in clinical trials - a review. Orphanet J Rare Dis. 2008; 3(11):11.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Jennison C, Turnbull BW. Group sequential methods with applications to clinical trials. Boca Raton: Chapman and Hall/CRC; 2000.

    Google Scholar 

  10. Hatfield I, Allison A, Flight L, Julious SA, Dimairo M. Adaptive designs undertaken in clinical research: a review of registered clinical trials. Trials. 2016; 17(1).

  11. Flight L, Arshad F, Barnsley R, Patel K, Julious S, Brennan A, Todd S. A review of clinical trials with an adaptive design and health economic analysis. Value Health. 2019; 22(4):391–8.

    Article  PubMed  Google Scholar 

  12. Creswell JW. Research design: qualitative, quantitative, and mixed methods approaches. Thousand Oaks: SAGE Publications; 2013.

    Google Scholar 

  13. Dimairo M, Boote J, Julious SA, Nicholl JP, Todd S. Missing steps in a staircase: a qualitative study of the perspectives of key stakeholders on the use of adaptive designs in confirmatory trials. Trials. 2015; 16(1).

  14. Flight L. Adaptive design clinical trials and their impact on the health economic analysis of healthcare technologies. Accessed 24 Feb 2020.

  15. Fugard AJB, Potts HWW. Supporting thinking on sample sizes for thematic analyses: a quantitative tool. Int J Soc Res Methodol. 2015; 18(6):669–84.

    Article  Google Scholar 

  16. Clarke V, Braun V. Successful qualitative research: a practical guide for beginners. London: SAGE Publications Inc.; 2013.

    Google Scholar 

  17. Guest G, Bunce A, Johnson L. How many interviews are enough? An experiment with data saturation and variability. Fam Health Int. 2006; 18(1):59–82.

    Google Scholar 

  18. Pope C, Mays N. Qualitative research in health care, 2nd ed. Oxford: BMJ Publishing; 2000.

    Google Scholar 

  19. Creswell JW, Hanson WE, Clark Plano VL, Morales A. Qualitative research designs: selection and implementation. Couns Psychol. 2007; 35(2):236–64.

    Article  Google Scholar 

  20. Carlsen B, Glenton C. What about N? A methodological study of sample-size reporting in focus group studies. BMC Med Res Methodol. 2011; 11(1):26.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Francis JJ, Johnston M, Robertson C, Glidewell L, Entwistle V, Eccles MP, Grimshaw JM. What is an adequate sample size? Operationalising data saturation for theory-based interview studies. Psychol Health. 2010; 25(10):1229–45.

    Article  PubMed  Google Scholar 

  22. Ritchie J, Spencer L. Qualitative data analysis for applied policy research. Analysing Qual Data. 1993:;173–94.

  23. Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research,. BMC Med Res Methodol. 2013; 13(1):117.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007; 19(6):349–57.

    Article  PubMed  Google Scholar 

  25. Beauchamp TL, Childress JF, et al.Principles of biomedical ethics. Oxford: Oxford University Press; 2001.

    Google Scholar 

  26. Rawls J. A theory of justice. Cambridge: Belknap; 1971.

    Google Scholar 

  27. Daniels N. Just health care. Cambridge: Cambridge University Press; 1985.

    Book  Google Scholar 

  28. Williams A. Cost-effectiveness analysis: is it ethical?. J Med Ethics. 1992; 18(1):7–11.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Food and Drug Administration. Adaptive designs for clinical trials of drugs and biologics. Center for Biologics Evaluation and Research (CBER). 2018. Accessed 24 Feb 2020.

  30. Chow S-C, Corey R, Lin M. On the independence of data monitoring committee in adaptive design clinical trials,. J Biopharm Stat. 2012; 22(4):853–67.

    Article  PubMed  Google Scholar 

  31. Whitehead J. The design and analysis of sequential clinical trials. Chichester: Wiley; 1997.

    Book  Google Scholar 

  32. Pallmann P, Bedding AW, Choodari-Oskooei B, Dimairo M, Flight L, Hampson LV, Holmes J, Mander AP, Sydes MR, Villar SS, et al.Adaptive designs in clinical trials: why use them, and how to run and report them. BMC Med. 2018; 16(1):29.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  33. Bothwell LE, Avorn J, Khan NF, Kesselheim AS. Adaptive design clinical trials: a review of the literature and ClinicalTrials. gov. BMJ Open. 2018; 8(2):018320.

    Article  Google Scholar 

  34. Mohiuddin S, Fenwick E, Payne K. Use of value of information in UK health technology assessments. Int J Technol Assess Health Care. 2014; 30(6):553–70.

    Article  PubMed  Google Scholar 

  35. Dimairo M, Coates E, Pallmann P, Todd S, Julious SA, Jaki T, Wason J, Mander AP, Weir CJ, Koenig F, et al.Development process of a consensus-driven CONSORT extension for randomised trials using an adaptive design. BMC Med. 2018; 16(1):210.

    Article  PubMed  PubMed Central  Google Scholar 

  36. ISPOR. Value of Information Analysis for Research Decisions Task Force. 2018. Accessed 15 Oct 2018.

  37. Heath A, Kunst NR, Jackson C, Strong M, Alarid-Escudero F, Goldhaber-Fiebert JD, Baio G, Menzies NA, Jalal H. Calculating the expected value of sample information in practice: considerations from three case studies. arXiv preprint arXiv:1905.12013. 2019.

  38. Kunst NR, Wilson E, Alarid-Escudero F, Baio G, Brennan A, Fairley M, Glynn D, Goldhaber-Fiebert JD, Jackson C, Jalal H, et al.Computing the expected value of sample information efficiently: expertise and skills required for four model-based methods. arXiv preprint arXiv:1910.03368. 2019.

  39. DAMOCLES Study Group. A proposed charter for clinical trial data monitoring committees: helping them to do their job well. Lancet. 2005; 365(9460):711–22.

    Article  Google Scholar 

  40. NIHR Statistics Group. Linking independent statisticians and investigators of clinical trials. 2018. Accessed 15 Oct 2018.

  41. Glasziou P, Chalmers I. How systematic reviews can reduce waste in research. Accessed 23 Dec 2019.

  42. Thorn J, Ridyard C, Hughes D, Wordsworth S, Mihaylova B, Noble SM, Hollingworth W. Health economics analysis plans: the current state of play. Trials. 2017; 18(Suppl 1):P144.

    Google Scholar 

  43. Forster M, Brealey S, Chick S, Keding A, Corbacho B, Alban A, Rangan A, et al.Cost-effective clinical trial design: application of a Bayesian sequential stopping rule to the ProFHER pragmatic trial. Technical report. 2019.

  44. Husbands SK. PhD Thesis, Model-building process. University of Birmingham. 2016; (February).

  45. Koerkamp B, Nikken J, Oei E, Stijnen T. Value of information analysis used to determine the necessity of additional research: MR imaging in acute knee trauma as an example. Radiology. 2008; 246(2):420–5.

    Article  Google Scholar 

  46. Legocki LJ, Meurer WJ, Frederiksen S, Lewis RJ, Durkalski VL, Berry DA, Barsan WG, Fetters MD. Clinical trialist perspectives on the ethics of adaptive clinical trials: a mixed-methods analysis. BMC Med Ethics. 2015; 16(1):27.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Meurer WJ, Legocki L, Mawocha S, Frederiksen SM, Guetterman TC, Barsan W, et al.Attitudes and opinions regarding confirmatory adaptive clinical trials: a mixed methods analysis from the Adaptive Designs Accelerating Promising Trials into Treatments (ADAPT-IT) project. Trials. 2016; 17(1):373.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Mullins CD, Vandigo J, Zheng Z, Wicks P. Patient-centeredness in the design of clinical trials. Value Health. 2014; 17(4):471–5.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Cook J, Drummond M, Heyse J. Multiple primary endpoints in clinical trials. Stat Methods Med Res. 2004; 13:157–76.

    Article  PubMed  Google Scholar 

  50. Koerkamp BG, Spronk S, Stijnen T, Hunink MGM. Value of information analyses of economic randomized controlled trials: the treatment of intermittent claudication. Value Health. 2010; 13(2):242–50.

    Article  Google Scholar 

  51. Tuffaha HW, Reynolds H, Gordon LG, Rickard CM, Scuffham P. Value of information analysis optimizing future trial design from a pilot study on catheter securement devices. Clin Trials (London, England). 2014; 11(6):648–56.

    Article  Google Scholar 

  52. Trotta F, Apolone G, Garattini S, Tafuri G. Stopping a trial early in oncology: for patients or for industry?Ann Oncol. 2008; 19(7):1347–53.

    Article  CAS  PubMed  Google Scholar 

  53. Ashby D, Tanb S-B. Where’s the utility of clinical trials? Bayesian data-monitoring. Clin Trials. 2005; 2:197–208.

    Article  PubMed  Google Scholar 

  54. Hollingworth W, McKell-Redwood D, Hampson L, Metcalfe C. Cost–utility analysis conducted alongside randomized controlled trials: are economic end points considered in sample size calculations and does it matter?Clin Trials. 2013; 10(1):43–53.

    Article  PubMed  Google Scholar 

  55. Briggs AH, Gray AM. Power and sample size calculations for stochastic cost-effectiveness analysis. Med Decis Making Int J Soc Med Decis Making. 1998; 18(2):81–92.

    Article  Google Scholar 

  56. O’Brien BJ, Drummond MF, Labelle RJ, Willan A. In search of power and significance: issues in the design and analysis of stochastic cost-effectiveness studies in health care. Medical Care. 1994; 32(2):150–63.

    Article  PubMed  Google Scholar 

  57. Briggs A. Economic evaluation and clinical trials: size matters,. BMJ. 2000; 321(7273):1362–3.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  58. Dimairo M, Pallmann P, Jaki T, et al.A Practical Adaptive & Novel Designs and Analysis (PANDA) Toolkit. 2019. Accessed 24 Feb 2020.

Download references


We would like to thank the Health Economic Evaluation of Adaptive Designs public involvement group, who have been involved throughout this study. They have played a crucial role in the success of this research, and we are grateful for their hard work and valuable contributions. Thank you to all study participants who gave up time from their busy schedules to take part in interviews and focus groups.


Laura Flight is funded by a National Institute for Health Research (NIHR) Doctoral Research Fellowship for this project. This article presents independent research funded by the NIHR. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR, the Department of Health and Social Care or the University of Sheffield.

Author information

Authors and Affiliations



LF designed the study, recruited participants, collected data, transcribed and analysed data and drafted the manuscript. DH contributed to the design of the study and supported data collection and analysis including co-coding of a sample of manuscripts. SAJ, AB and ST supported the design, data collection and analysis phases of the study. All authors critiqued the paper for important intellectual content and have read and approved the final manuscript.

Corresponding author

Correspondence to Laura Flight.

Ethics declarations

Ethics approval and consent to participate

Ethics approval was granted by the University of Sheffield, School of Health and related Research Ethics Committee (ref 009699). All participants were issued an information sheet and were required to sign a consent form or give verbal consent.

Consent for publication

Informed consent was obtained from participants to use their data for publication.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1

Completed COREQ checklist.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Flight, L., Julious, S., Brennan, A. et al. How can health economics be used in the design and analysis of adaptive clinical trials? A qualitative analysis. Trials 21, 252 (2020).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: