Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Advancing ‘real-world’ trials that take account of social context and human volition

Trials201718:531

https://doi.org/10.1186/s13063-017-2286-8

Received: 2 May 2017

Accepted: 3 October 2017

Published: 10 November 2017

Abstract

Background

The recent paper in Trials by Porter and colleagues highlights the utility of applying a critical realism approach in randomised trials, an approach central to the Medical Research Council’s (MRC) Framework for the Development and Evaluation of Complex Healthcare Interventions. The MRC framework offers a pragmatic step towards a more open systems approach that bridges randomised evaluation with social context and human agency in an effort to improve the generalisability of trial outcomes.

Main body

The MRC framework has contributed to the proliferation of a more open systems approach in health research; however, the broader acceptance of the realist approach to health research does not seem to be emulated by norms in research fund allocation, which largely prioritises laboratory-based research.

Conclusion

This commentary is simply a plea, to those who make the strategic decisions regarding allocation of research funding, to support all phases of health intervention research in complex systems that contribute to the development of effective, translational and sustainable interventions in the promotion of health.

Keywords

Complex interventionsExternal validityResearch funding

Background

The dissemination over a decade ago of the Medical Research Council’s (MRC) Framework for the Development and Evaluation of Complex Healthcare Interventions [1], and appeals by researchers for more research to be conducted on interventions in complex systems [2] has contributed to the proliferation of research in more open systems, i.e. in ‘real-world’ settings (for an example, see the INCLUSIVE study – [3]). The MRC framework advocates a mixed-method approach, including: qualitative research – such as process evaluation of implementation of an intervention and of contextual factors that could lead to variability in outcomes [4]; causal modelling approaches, that can provide information on the process and outcome of an intervention and that may lead to design changes prior to implementation, e.g. [5]; and a range of experimental methods for evaluating effectiveness, with preference given to randomised designs (randomised controlled trials – RCTs), wherever possible or appropriate, for optimisation of internal validity [6]. The mixed-method approach is an attempt to supplement the randomised trial with procedures embedded in realism that take the complexity of social context and human volition into account, and to shift the focus from identifying causality to explaining causality, or in other words, to explaining how intervention components interact with different populations and in different contexts [7]. Moore and Evans [8] demonstrate the importance of context by describing how an intervention nearly two decades ago, that effectively reduced smoking in young people by prohibiting smoking on school premises and thereby challenging norms surrounding smoking behaviour, may not prove as effective today. Norms surrounding smoking that existed 15–20 years ago (e.g. high prevalence rates and social acceptance) have changed, and consequently, so have the mechanisms that predict smoking behaviour in young people [8].

The combination of positivist and realist approaches found in the MRC framework, which Porter et al. [7] and others [9] refer to as realist RCTs, has given rise to debate [7, 913]. Central to the debate is whether randomised and realist evaluation can be consolidated. While critics argue that the realist RCT design found in the MRC framework does not fully take complexity into account [12], the MRC framework does offer a pragmatic step towards a more open systems (real-world) approach that bridges, to a degree, randomised evaluation with social context and human agency [14]. The recent paper in Trials by Porter and colleagues highlighting the utility of applying a critical realism approach [7] and the ongoing debate exploring the utility between realist and randomised evaluations [9, 12] are welcome and important steps towards the development and refinement of a usable open systems approach.

A plea for more research funding aimed at supporting health interventions in complex systems

While incremental steps are being made as evidenced by the MRC complex framework, the broader adoption of a more open systems approach in health research does not seem to have penetrated the orthodoxy of applying a traditional positivist epistemological and ontological (bio-medical) approach to evaluating research and, by extension, to deciding how research funds are allocated [15, 16]. For example, an overview of research-fund allocation from the Medical Research Council, The Wellcome Trust, British Heart Foundation, and Cancer Research UK, showed that across the four research-funding agencies, 85% of funds on average were allocated to laboratory-based research [17]. A recent review of funding trends within diabetes research illustrated that the proportion of funded studies that included social context and human volition was small compared to the proportion of funded bio-medical studies, with an estimated mean ratio of 17:1 [16]. The uptake of a non-linear, or real-world conception of causality by funding agencies seems, therefore, to be a slow burn. There is no doubt that the closed linear system approach, so pervasive in bio-medical and health research, has been effective in solving health-related and other issues. However, researchers, including those within bio-medicine, are increasingly asking whether the documented efficacy of an intervention, gained from identifying causality in a closed part of a larger or open system (traditional reductionist approach), is robust enough to reproduce the recorded effects reliably in real-world settings [18, 19]; or if successionist models of causality alone are adequate in solving the complex health issues of our time [2023].

It is estimated that as much as 85% of research investment can be categorised as waste (i.e. not benefitting society/patients) due largely to avoidable weaknesses in research design and production [2427]. Lack of external validity in the experimental research designs employed is one of the main reasons why laboratory-based and randomised clinical trial outcomes often fail to translate into benefits for the patient, as the robustness of probabilities in different contexts (what works best for whom and in which setting) is uncertain [12, 18, 19, 21, 28]. Laboratory and clinical trials that do not take higher-level concepts, such as social context and human volition, into account are, therefore, vulnerable when attempting to replicate intervention effects in real-world settings. As stated by Rothwell: “Government expenditure should provide value for money, and medical research is no exception” [17].

The MRC framework is an attempt to increase the external validity of evaluations while still preserving internal validity. The availability of more research funds that support health interventions and practice-orientated research in complex systems may result in more robust and sustainable interventions and lead to reduced waste in research investment.

Conclusion

Since the introduction of the MRC framework, complex interventions are increasingly being developed in an attempt to solve the continually expanding burden of health-related issues. This commentary is, therefore, a plea to policy-makers, and those who make the strategic decisions regarding allocation of research funding, to support to a greater degree all phases of real-world trials that take into account social context and human volition as advocated by the MRC framework, rather than to automatically allocate research funds on the basis of received wisdom.

Abbreviations

MRC: 

Medical Research Council

RCTs: 

Randomised controlled trials

Declarations

Acknowledgements

Not applicable

Funding

No sources of funding to declare.

Availability of data and materials

Not applicable (no empirical data involved)

Authors’ contributions

ABGH and AJ drafted the document, ABGH and AJ revised it critically for important intellectual content. Both authors have given final approval and agreed to be accountable for it.

Ethics approval and consent to participate

Not applicable (as this is not an empirical study, it involved no participants and does not require ethics approval).

Consent for publication

Not applicable (no individual data included).

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Strategic Research and Development Support, Metropolitan University College
(2)
Bachelor’s Degree in Global Nutrition and Health, Faculty of Health and Technology, Metropolitan University College

References

  1. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: new guidance. Medical Research Council: Prepared on behalf of the Medical Research Council; 2006. https://www.mrc.ac.uk/documents/pdf/complex-interventions-guidance/
  2. Rickles D. Causality in complex interventions. Med Health Care Philos. 2009;12:77–90. doi:10.1007/s11019-008-9140-4.View ArticlePubMedGoogle Scholar
  3. Bonell C, Allen E, Christie D, Elbourne D, Fletcher A, Grieve R, et al. Initiating change locally in bullying and aggression through the school environment (INCLUSIVE): study protocol for a cluster randomised controlled trial. Trials. 2014;15:1–14.View ArticleGoogle Scholar
  4. Moore G, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258. doi:10.1136/bmj.h1258.View ArticlePubMedPubMed CentralGoogle Scholar
  5. Hardeman W, Sutton S, Griffin S, Johnston M, White A, Wareham NJ, et al. A causal modelling approach to the development of theory-based behaviour change programmes for trial evaluation. Health Educ Res. 2005;20:676–87.View ArticlePubMedGoogle Scholar
  6. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655.View ArticlePubMedPubMed CentralGoogle Scholar
  7. Porter S, McConnell T, Reid J. The possibility of critical realist randomised controlled trials. Trials. 2017;18:133. doi:10.1186/s13063-017-1855-1.View ArticlePubMedPubMed CentralGoogle Scholar
  8. Moore GF, Evans RE. What theory, for whom and in which context? Reflections on the application of theory in the development and evaluation of complex population health interventions. SSM-Population Health. 2017;3:132–5.View ArticleGoogle Scholar
  9. Bonell C, Fletcher A, Morton M, Lorenc T, Moore L. Realist randomised controlled trials: a new approach to evaluating complex public health interventions. Social science medicine. 2012;75:2299–306. doi:10.1016/j.socscimed.2012.08.032.View ArticlePubMedGoogle Scholar
  10. Van Belle S, Wong G, Westhorp G, Pearson M, Emmel N, Manzano A, et al. Can ‘realist’ randomised controlled trials be genuinely realist? Trials. 2016;17:313. doi:10.1186/s13063-016-1407-0.View ArticlePubMedPubMed CentralGoogle Scholar
  11. Jamal F, Fletcher A, Shackleton N, Elbourne D, Viner R, Bonell C. The three stages of building and testing mid-level theories in a realist RCT: a theoretical and methodological case-example. Trials. 2015;16:466. doi:10.1186/s13063-015-0980-y.View ArticlePubMedPubMed CentralGoogle Scholar
  12. Marchal B, Westhorp G, Wong G, Van Belle S, Greenhalgh T, Kegels G, et al. Realist RCTs of complex interventions—an oxymoron. Social Science Medicine. 2013;94:124–8. doi:10.1016/j.socscimed.2013.06.025.View ArticlePubMedGoogle Scholar
  13. Bonell C, Warren E, Fletcher A, Viner R. Realist trials and the testing of context-mechanism-outcome configurations: a response to Van Belle et al. Trials. 2016;17:478. doi:10.1186/s13063-016-1613-9.View ArticlePubMedPubMed CentralGoogle Scholar
  14. Fletcher A, Jamal F, Moore G, Evans R, Murphy S, Bonell C. Realist complex intervention science: applying realist principles across all phases of the Medical Research Council framework for developing and evaluating complex interventions. Evaluation. 2016;22:286–303. doi:10.1177/1356389016652743.View ArticlePubMedPubMed CentralGoogle Scholar
  15. Rutter H, Glonti K. Towards a new model of evidence for public health. Lancet. 2016;388:S7. doi:10.1016/S0140-6736(16)32243-7.View ArticleGoogle Scholar
  16. Jones A, Vallis M, Cooke D, Pouwer F. Review of research grant allocation to psychosocial studies in diabetes research. Diabetic Med. 2016;33:1673–6. doi:10.1111/dme.13255.View ArticlePubMedGoogle Scholar
  17. Rothwell P. Funding for practice-oriented clinical research. Lancet. 2006;368:262–6. doi:10.1016/S0140-6736(06)69010-7.View ArticlePubMedGoogle Scholar
  18. Rothwell P. Commentary: external validity of results of randomized trials: disentangling a complex concept. Int J Epidemiol. 2010;39:94–6. doi:10.1093/ije/dyp305.View ArticlePubMedGoogle Scholar
  19. Rothwell P. External validity of randomised controlled trials: ‘to whom do the results of this trial apply?’. Lancet. 2005;365:82–93. doi:10.1016/S0140-6736(04)17670-8.View ArticlePubMedGoogle Scholar
  20. Van Regenmortel MHV. Reductionism and complexity in molecular biology. EMBO Rep. 2004;5:1016–20.View ArticlePubMedPubMed CentralGoogle Scholar
  21. Federoff H, Gostin L. Evolving from reductionism to holism: is there a future for systems medicine? JAMA: the Journal of the American Medical Association. 2009;302:994–6. doi:10.1001/jama.2009.1264.View ArticlePubMedGoogle Scholar
  22. Strange K. Revisiting the Krogh Principle in the post-genome era: Caenorhabditis elegans as a model system for integrative physiology research. J Exp Biol. 2007;210:1622–31. doi:10.1242/jeb.000125.View ArticlePubMedGoogle Scholar
  23. Levi-Montalcini R, Calissano P. The scientific challenge of the 21st century: from a reductionist to a holistic approach via systems biology. BMC Neuroscience. 2006;7:S1. doi:10.1186/1471-2202-7-S1-S1.View ArticlePubMed CentralGoogle Scholar
  24. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374:86–9. doi:10.1016/S0140-6736(09)60329-9.View ArticlePubMedGoogle Scholar
  25. Yordanov Y, Dechartres A, Porcher R, Boutron I, Altman D, Ravaud P. Avoidable waste of research related to inadequate methods in clinical trials. BMJ. 2015;350:h809. doi:10.1136/bmj.h809.View ArticlePubMedPubMed CentralGoogle Scholar
  26. Ioannidis JPA, Greenland S, Hlatky M, Khoury M, Macleod M, Moher D, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383:166–75. doi:10.1016/S0140-6736(13)62227-8.View ArticlePubMedPubMed CentralGoogle Scholar
  27. Heneghan C, Goldacre B, Mahtani K. Why clinical trial outcomes fail to translate into benefits for patients. Trials. 2017;18:122. doi:10.1186/s13063-017-1870-2.View ArticlePubMedPubMed CentralGoogle Scholar
  28. Steckler A, McLeroy K. The importance of external validity. Am J Public Health. 2008;98:9–10. doi:10.2105/AJPH.2007.126847.View ArticlePubMedPubMed CentralGoogle Scholar

Copyright

© The Author(s). 2017

Advertisement