Skip to main content

Incomplete reporting of complex interventions: a call to action for journal editors to review their submission guidelines

Abstract

Reporting of intervention research has been inadequate for many years. The development and promotion of freely available checklists aims to address this problem by providing researchers with a list of items that require reporting to enable study interpretation and replication. In this commentary, we present evidence from a recent systematic review of 51 randomised controlled trials published 2015–2020 that inadequate intervention reporting remains a widespread issue and that checklists are not being used to describe all intervention components. In 2022, we assessed the submission guidelines of 33 journals that published articles included in our review and found that just one at the time encouraged the use of reporting checklists for all intervention components. To drive progress, we contacted the editors of the other 32 journals and requested that they update their submission guidelines in response. We conclude by highlighting the waste associated with current practices and encourage journals from all fields to urgently review their submission guidelines. Only through collective action can we build an evidence base that is fit for purpose.

Peer Review reports

Reporting guidelines in research

Randomised controlled trials and other forms of intervention evaluations constitute a considerable proportion of public health research activity. For many years, researchers have typically provided inadequate descriptions of the intervention(s) they tested [1], which has limited the scientific and applied value of this activity [2]. From the 1990s, in response to evidence of poor reporting and its costs, bodies such as the International Committee of Medical Journal Editors (ICMJE) and the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network began supporting the development and promotion of evidence-based reporting guidelines [3]. There now exists a library of freely available reporting guidelines to support researchers in providing sufficient detail to enable study interpretation and replication [4]. The publication of the Template for Intervention Description and Replication (TIDieR) checklist in 2014 [5] aimed to improve reporting standards in intervention research specifically. The checklist outlines a minimum set of items considered essential for intervention description and replication. We present evidence in this commentary that the authors of complex interventions mostly use such checklists to describe only one part of the overall intervention. We draw on our experience of reviewing school-based physical activity research to present evidence of underreporting on interventions targeted at change agents.

Successful implementation of school-based interventions is often dependent on behaviour change by key actors or ‘change agents’ in students’ lives (e.g. teachers, parents/guardians, peers). For example, school staff may be required to change their teaching practices, and/or parents may be required to implement changes. Figure 1, adapted from the Medical Research Council guidance on the development and evaluation of complex interventions [6], illustrates that change agent-targeted interventions (sometimes referred to as ‘implementation strategies’ [7]) (e.g. staff training programmes, parent newsletters) play an important role in the logic model of the overall intervention. Nevertheless, they are frequently overlooked as behaviour change interventions in and of themselves [7] and their impact on process evaluation outcomes (e.g. percentage of intended educational sessions delivered) and trial effectiveness outcomes (e.g. students’ behaviour) is poorly understood.

Fig. 1
figure 1

Schematic overview of multi-actor interventions

Evidence of inadequate reporting on change agent interventions in school-based research

We recently completed a systematic review of school-based physical activity interventions to study if specific features of staff training programmes were associated with intervention fidelity and students’ study outcomes [8]. This was a significant research gap because the majority of children and adolescents worldwide are not sufficiently physically active [9, 10] and global intervention efforts, which largely focus on school settings, have mostly failed [11, 12]. Reasons for outcome failures were unknown as previous review efforts to address similar research questions were impeded by incomplete descriptions of interventions published 1999–2015 [13]. Given the greater availability and promotion of intervention reporting guidelines since 2015 (e.g. [5]), we solely reviewed interventions published since 2015, in anticipation that reporting quality had improved. We conducted a systematic, comprehensive and inclusive search for details on intervention descriptions of included randomised controlled trials, pooling information from multiple sources (e.g. protocols, process evaluations, outcome evaluations, trial registries, study websites; n = 183 in total). See Additional file 1 for details about methods and sources included. We aimed to identify descriptions of staff training programmes based on items in the TIDieR checklist [5]. Where incomplete reporting was identified, we contacted lead authors for further information.

We included 51 trials, reporting on 53 training programmes [8]. We found that, prior to contacting authors, complete information was only available for one of the training programmes we reviewed (2%). Basic details (e.g. use of theory, location of the intervention) were missing from the majority of interventions (see Additional file 2 for breakdown by TIDieR item before and after author contact). Descriptions across all available study sources largely focused on the intervention(s) aimed at the student, and while some study authors reported using checklists to describe student-targeted interventions, no checklists were reportedly used to describe change agent-targeted interventions.

At the time of writing this commentary, we checked the websites of all the journals that published articles included in our review (n = 33) and found that just one explicitly requested submission of reporting checklists for all intervention components. Our findings suggest that inadequate reporting remains a prevalent issue despite the availability of infrastructure (e.g. reporting checklists and online appendices/repositories to overcome manuscript word limits) and that current systems, including journal submission guidelines, are enabling wasteful practices to persist. Our findings are also particularly alarming given that adherence to reporting guidelines is reportedly better in larger and controlled studies [14]; the sample of publications we reviewed reported on medium-to-large-scale randomised controlled trials (median sample size: 779; interquartile range: 361–1397). While we found that the percentage of reported items improved considerably by contacting authors (see Additional file 2), this resource-intensive task is a poor replacement for Open Science practices. Moreover, readers of intervention outputs are not informed that they lack sufficient information to reliably replicate intervention(s) or interpret reported outcomes. As reliance upon change agents is common in many other intervention settings (e.g. hospitals, police custody suites, nursing homes [15,16,17]), our findings have wider relevance beyond school-based research.

Action taken

Current reporting practices are stifling learning opportunities within and beyond public health and limiting the types of knowledge that can be gained from costly evaluations. Without complete descriptions of all interventions under evaluation, the scientific community has no means of interpreting study outcomes or replicating effective interventions [5, 18]. This is resulting in poor outcomes for members of the public who have both funded and/or participated in intervention studies. Research communities must now move beyond describing poor reporting practices to taking action and pay equal attention to reporting of all change agent-targeted interventions. Although we urge authors of intervention papers to provide complete descriptions of all intervention components, we recognise the important gatekeeper role that journal editors play in setting research communication and reporting standards [14]. We contacted the editors of the other 32 journals (see Additional file 3), inviting them to update their submission guidelines in response to our findings. Specifically, we asked that they require intervention authors to submit separate reporting checklists for each of the interventions that have been delivered within a study, including interventions targeted at change agents. To date, we have received a reply from 27 journals, 26% of whom have updated their submission guidelines in response, including ‘British Journal of Sports Medicine’, ‘Journal of Physical Activity and Health’, ‘Sport, Exercise, and Performance Psychology’, 'Journal of Science and Medicine in Sport' and ‘Journal of Experimental Social Psychology’ (see Additional file 3 for a complete list).

Further action is needed

While we are pleased a number of journals have updated their guidelines, ongoing inaction from others will enable current practices to continue. The 2013 Declaration of Helsinki emphasises that journals have an ethical obligation to reject incomplete reports of research for publication [19]. We now call on all journals that publish intervention research, regardless of their field, to urgently review their submission policies. We also highlight the role that researchers, editorial teams, funders, and reviewers can all play in encouraging and supporting journals in this effort. Resources from the EQUATOR Network may be useful in guiding related discussion: https://www.equator-network.org/toolkits/using-guidelines-in-journals/creating-your-journals-reporting-guideline-policy/. As we approach the 10-year publication anniversary of the TIDieR intervention reporting guideline, we ask—how much more waste will be tolerated before action is taken?

Availability of data and materials

A summary of reviewed studies and their outputs is available in Additional file 1.

Abbreviations

TIDieR:

Template for Intervention Description and Replication

ICMJE:

International Committee of Medical Journal Editors

EQUATOR:

Enhancing the QUAlity and Transparency Of health Research

References

  1. Samaan Z, Mbuagbaw L, Kosa D, Debono VB, Dillenburg R, Zhang S, et al. A systematic scoping review of adherence to reporting guidelines in health care literature. J Multidiscip Healthc. 2013;6:169–88.

    PubMed  PubMed Central  Google Scholar 

  2. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86–9.

    Article  PubMed  Google Scholar 

  3. Altman DG, Simera I. A history of the evolution of guidelines for reporting medical research: the long road to the EQUATOR Network. J R Soc Med. 2016;109(2):67–77.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Library for health research reporting - EQUATOR Network. https://www.equator-network.org/library/

  5. Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348:g1687.

    Article  PubMed  Google Scholar 

  6. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: medical research council guidance. BMJ. 2015;350:h1258.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Wolfenden L, Foy R, Presseau J, Grimshaw JM, Ivers NM, Powell BJ, et al. Designing and undertaking randomised implementation trials: guide for researchers. BMJ. 2021;372:m3721.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Ryan M, Alliott O, Ikeda E, Ja L, Hofmann R, van Sluijs E. Features of effective staff training programmes within school-based interventions targeting student activity behaviour: a systematic review and meta-analysis. Int J Behav Nutr Phys Act. 2022;19(1):125.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Guthold R, Stevens GA, Riley LM, Bull FC. Worldwide trends in insufficient physical activity from 2001 to 2016: a pooled analysis of 358 population-based surveys with 1 9 million participants. Lancet Glob Health. 2018;6(10):e1077–86.

    Article  PubMed  Google Scholar 

  10. Aubert S, Barnes JD, Abdeta C, Abi Nader P, Adeniyi AF, Aguilar-Farias N, et al. Global matrix 3.0 physical activity report card grades for children and youth: results and analysis from 49 countries. J Phys Act Health. 2018;15(s2):S251–73.

    Article  PubMed  Google Scholar 

  11. Jones M, Defever E, Letsinger A, Steele J, Mackintosh KA. A mixed-studies systematic review and meta-analysis of school-based interventions to promote physical activity and/or reduce sedentary time in children. J Sport Health Sci. 2020;9(1):3–17.

    Article  PubMed  Google Scholar 

  12. Neil-Sztramko SE, Caldwell H, Dobbins M. School-based physical activity programs for promoting physical activity and fitness in children and adolescents aged 6 to 18. Cochrane Database Syst Rev. 2021;9(9):CD007651.

    PubMed  Google Scholar 

  13. Lander N, Eather N, Morgan PJ, Salmon J, Barnett LM. Characteristics of teacher training in school-based physical education interventions to improve fundamental movement skills and/or physical activity: a systematic review. Sports Med. 2017;47(1):135–61.

    Article  PubMed  Google Scholar 

  14. Jin Y, Sanger N, Shams I, Luo C, Shahid H, Li G, et al. Does the medical literature remain inadequately described despite having reporting guidelines for 21 years? – A systematic review of reviews: an update. J Multidiscip Healthc. 2018;11:495–510.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Addison M, McGovern R, Angus C, Becker F, Brennan A, Brown H, et al. Alcohol screening and brief intervention in police custody suites: pilot cluster randomised controlled trial (AcCePT). Alcohol Alcohol. 2018;53(5):548–59.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Geerligs L, Rankin NM, Shepherd HL, Butow P. Hospital-based interventions: a systematic review of staff-reported barriers and facilitators to implementation processes. Implement Sci. 2018;13(1):36.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Klingshirn H, Müller M, Beutner K, Hirt J, Strobl R, Grill E, et al. Implementation of a complex intervention to improve participation in older people with joint contractures living in nursing homes: a process evaluation of a cluster-randomised pilot trial. BMC Geriatr. 2020;20(1):270.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Skivington K, Matthews L, Simpson SA, Craig P, Baird J, Blazeby JM, et al. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ. 2021;374:n2061.

    Article  PubMed  PubMed Central  Google Scholar 

  19. World Medical Association. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310(20):2191–4.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This work was supported by the Economic and Social Research Council [grant number ES/P000738/1], the University of Cambridge, and the Medical Research Council [grant number: MC_UU_00006/5]. The funders had no role in informing the content of the manuscript.

Author information

Authors and Affiliations

Authors

Contributions

MR conceived the idea of the Commentary and contacted journal editors on behalf of all co-authors. MR and EvS drafted the main text. TH and RH reviewed and contributed additional content. All authors approved the final draft of the manuscript.

Corresponding author

Correspondence to Mairead Ryan.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

TH is the lead author of the TIDieR guideline and Co-Director of the Australasian EQUATOR Centre. EvS is a senior associate editor for the Journal of Physical Activity and Health. She played no role in the editor’s decision to update the journal’s submission guidelines. She is also a member of the Editorial Board at the International Journal of Behavioral Nutrition and Physical Activity.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1. 

Methods used to assess completeness of TIDieR checklist items in systematic review.

Additional file 2. 

List of TIDieR items reported before and after author contact.

Additional file 3. 

Complete list of journals contacted (n=32) and action taken (n=7).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ryan, M., Hoffmann, T., Hofmann, R. et al. Incomplete reporting of complex interventions: a call to action for journal editors to review their submission guidelines. Trials 24, 176 (2023). https://doi.org/10.1186/s13063-023-07215-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13063-023-07215-1

Keywords