Incomplete reporting of complex interventions: a call to action for journal editors to review their submission guidelines
Trials volume 24, Article number: 176 (2023)
Reporting of intervention research has been inadequate for many years. The development and promotion of freely available checklists aims to address this problem by providing researchers with a list of items that require reporting to enable study interpretation and replication. In this commentary, we present evidence from a recent systematic review of 51 randomised controlled trials published 2015–2020 that inadequate intervention reporting remains a widespread issue and that checklists are not being used to describe all intervention components. In 2022, we assessed the submission guidelines of 33 journals that published articles included in our review and found that just one at the time encouraged the use of reporting checklists for all intervention components. To drive progress, we contacted the editors of the other 32 journals and requested that they update their submission guidelines in response. We conclude by highlighting the waste associated with current practices and encourage journals from all fields to urgently review their submission guidelines. Only through collective action can we build an evidence base that is fit for purpose.
Reporting guidelines in research
Randomised controlled trials and other forms of intervention evaluations constitute a considerable proportion of public health research activity. For many years, researchers have typically provided inadequate descriptions of the intervention(s) they tested , which has limited the scientific and applied value of this activity . From the 1990s, in response to evidence of poor reporting and its costs, bodies such as the International Committee of Medical Journal Editors (ICMJE) and the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network began supporting the development and promotion of evidence-based reporting guidelines . There now exists a library of freely available reporting guidelines to support researchers in providing sufficient detail to enable study interpretation and replication . The publication of the Template for Intervention Description and Replication (TIDieR) checklist in 2014  aimed to improve reporting standards in intervention research specifically. The checklist outlines a minimum set of items considered essential for intervention description and replication. We present evidence in this commentary that the authors of complex interventions mostly use such checklists to describe only one part of the overall intervention. We draw on our experience of reviewing school-based physical activity research to present evidence of underreporting on interventions targeted at change agents.
Successful implementation of school-based interventions is often dependent on behaviour change by key actors or ‘change agents’ in students’ lives (e.g. teachers, parents/guardians, peers). For example, school staff may be required to change their teaching practices, and/or parents may be required to implement changes. Figure 1, adapted from the Medical Research Council guidance on the development and evaluation of complex interventions , illustrates that change agent-targeted interventions (sometimes referred to as ‘implementation strategies’ ) (e.g. staff training programmes, parent newsletters) play an important role in the logic model of the overall intervention. Nevertheless, they are frequently overlooked as behaviour change interventions in and of themselves  and their impact on process evaluation outcomes (e.g. percentage of intended educational sessions delivered) and trial effectiveness outcomes (e.g. students’ behaviour) is poorly understood.
Evidence of inadequate reporting on change agent interventions in school-based research
We recently completed a systematic review of school-based physical activity interventions to study if specific features of staff training programmes were associated with intervention fidelity and students’ study outcomes . This was a significant research gap because the majority of children and adolescents worldwide are not sufficiently physically active [9, 10] and global intervention efforts, which largely focus on school settings, have mostly failed [11, 12]. Reasons for outcome failures were unknown as previous review efforts to address similar research questions were impeded by incomplete descriptions of interventions published 1999–2015 . Given the greater availability and promotion of intervention reporting guidelines since 2015 (e.g. ), we solely reviewed interventions published since 2015, in anticipation that reporting quality had improved. We conducted a systematic, comprehensive and inclusive search for details on intervention descriptions of included randomised controlled trials, pooling information from multiple sources (e.g. protocols, process evaluations, outcome evaluations, trial registries, study websites; n = 183 in total). See Additional file 1 for details about methods and sources included. We aimed to identify descriptions of staff training programmes based on items in the TIDieR checklist . Where incomplete reporting was identified, we contacted lead authors for further information.
We included 51 trials, reporting on 53 training programmes . We found that, prior to contacting authors, complete information was only available for one of the training programmes we reviewed (2%). Basic details (e.g. use of theory, location of the intervention) were missing from the majority of interventions (see Additional file 2 for breakdown by TIDieR item before and after author contact). Descriptions across all available study sources largely focused on the intervention(s) aimed at the student, and while some study authors reported using checklists to describe student-targeted interventions, no checklists were reportedly used to describe change agent-targeted interventions.
At the time of writing this commentary, we checked the websites of all the journals that published articles included in our review (n = 33) and found that just one explicitly requested submission of reporting checklists for all intervention components. Our findings suggest that inadequate reporting remains a prevalent issue despite the availability of infrastructure (e.g. reporting checklists and online appendices/repositories to overcome manuscript word limits) and that current systems, including journal submission guidelines, are enabling wasteful practices to persist. Our findings are also particularly alarming given that adherence to reporting guidelines is reportedly better in larger and controlled studies ; the sample of publications we reviewed reported on medium-to-large-scale randomised controlled trials (median sample size: 779; interquartile range: 361–1397). While we found that the percentage of reported items improved considerably by contacting authors (see Additional file 2), this resource-intensive task is a poor replacement for Open Science practices. Moreover, readers of intervention outputs are not informed that they lack sufficient information to reliably replicate intervention(s) or interpret reported outcomes. As reliance upon change agents is common in many other intervention settings (e.g. hospitals, police custody suites, nursing homes [15,16,17]), our findings have wider relevance beyond school-based research.
Current reporting practices are stifling learning opportunities within and beyond public health and limiting the types of knowledge that can be gained from costly evaluations. Without complete descriptions of all interventions under evaluation, the scientific community has no means of interpreting study outcomes or replicating effective interventions [5, 18]. This is resulting in poor outcomes for members of the public who have both funded and/or participated in intervention studies. Research communities must now move beyond describing poor reporting practices to taking action and pay equal attention to reporting of all change agent-targeted interventions. Although we urge authors of intervention papers to provide complete descriptions of all intervention components, we recognise the important gatekeeper role that journal editors play in setting research communication and reporting standards . We contacted the editors of the other 32 journals (see Additional file 3), inviting them to update their submission guidelines in response to our findings. Specifically, we asked that they require intervention authors to submit separate reporting checklists for each of the interventions that have been delivered within a study, including interventions targeted at change agents. To date, we have received a reply from 27 journals, 26% of whom have updated their submission guidelines in response, including ‘British Journal of Sports Medicine’, ‘Journal of Physical Activity and Health’, ‘Sport, Exercise, and Performance Psychology’, 'Journal of Science and Medicine in Sport' and ‘Journal of Experimental Social Psychology’ (see Additional file 3 for a complete list).
Further action is needed
While we are pleased a number of journals have updated their guidelines, ongoing inaction from others will enable current practices to continue. The 2013 Declaration of Helsinki emphasises that journals have an ethical obligation to reject incomplete reports of research for publication . We now call on all journals that publish intervention research, regardless of their field, to urgently review their submission policies. We also highlight the role that researchers, editorial teams, funders, and reviewers can all play in encouraging and supporting journals in this effort. Resources from the EQUATOR Network may be useful in guiding related discussion: https://www.equator-network.org/toolkits/using-guidelines-in-journals/creating-your-journals-reporting-guideline-policy/. As we approach the 10-year publication anniversary of the TIDieR intervention reporting guideline, we ask—how much more waste will be tolerated before action is taken?
Availability of data and materials
A summary of reviewed studies and their outputs is available in Additional file 1.
Template for Intervention Description and Replication
International Committee of Medical Journal Editors
Enhancing the QUAlity and Transparency Of health Research
Samaan Z, Mbuagbaw L, Kosa D, Debono VB, Dillenburg R, Zhang S, et al. A systematic scoping review of adherence to reporting guidelines in health care literature. J Multidiscip Healthc. 2013;6:169–88.
Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86–9.
Altman DG, Simera I. A history of the evolution of guidelines for reporting medical research: the long road to the EQUATOR Network. J R Soc Med. 2016;109(2):67–77.
Library for health research reporting - EQUATOR Network. https://www.equator-network.org/library/
Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348:g1687.
Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: medical research council guidance. BMJ. 2015;350:h1258.
Wolfenden L, Foy R, Presseau J, Grimshaw JM, Ivers NM, Powell BJ, et al. Designing and undertaking randomised implementation trials: guide for researchers. BMJ. 2021;372:m3721.
Ryan M, Alliott O, Ikeda E, Ja L, Hofmann R, van Sluijs E. Features of effective staff training programmes within school-based interventions targeting student activity behaviour: a systematic review and meta-analysis. Int J Behav Nutr Phys Act. 2022;19(1):125.
Guthold R, Stevens GA, Riley LM, Bull FC. Worldwide trends in insufficient physical activity from 2001 to 2016: a pooled analysis of 358 population-based surveys with 1 9 million participants. Lancet Glob Health. 2018;6(10):e1077–86.
Aubert S, Barnes JD, Abdeta C, Abi Nader P, Adeniyi AF, Aguilar-Farias N, et al. Global matrix 3.0 physical activity report card grades for children and youth: results and analysis from 49 countries. J Phys Act Health. 2018;15(s2):S251–73.
Jones M, Defever E, Letsinger A, Steele J, Mackintosh KA. A mixed-studies systematic review and meta-analysis of school-based interventions to promote physical activity and/or reduce sedentary time in children. J Sport Health Sci. 2020;9(1):3–17.
Neil-Sztramko SE, Caldwell H, Dobbins M. School-based physical activity programs for promoting physical activity and fitness in children and adolescents aged 6 to 18. Cochrane Database Syst Rev. 2021;9(9):CD007651.
Lander N, Eather N, Morgan PJ, Salmon J, Barnett LM. Characteristics of teacher training in school-based physical education interventions to improve fundamental movement skills and/or physical activity: a systematic review. Sports Med. 2017;47(1):135–61.
Jin Y, Sanger N, Shams I, Luo C, Shahid H, Li G, et al. Does the medical literature remain inadequately described despite having reporting guidelines for 21 years? – A systematic review of reviews: an update. J Multidiscip Healthc. 2018;11:495–510.
Addison M, McGovern R, Angus C, Becker F, Brennan A, Brown H, et al. Alcohol screening and brief intervention in police custody suites: pilot cluster randomised controlled trial (AcCePT). Alcohol Alcohol. 2018;53(5):548–59.
Geerligs L, Rankin NM, Shepherd HL, Butow P. Hospital-based interventions: a systematic review of staff-reported barriers and facilitators to implementation processes. Implement Sci. 2018;13(1):36.
Klingshirn H, Müller M, Beutner K, Hirt J, Strobl R, Grill E, et al. Implementation of a complex intervention to improve participation in older people with joint contractures living in nursing homes: a process evaluation of a cluster-randomised pilot trial. BMC Geriatr. 2020;20(1):270.
Skivington K, Matthews L, Simpson SA, Craig P, Baird J, Blazeby JM, et al. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ. 2021;374:n2061.
World Medical Association. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310(20):2191–4.
This work was supported by the Economic and Social Research Council [grant number ES/P000738/1], the University of Cambridge, and the Medical Research Council [grant number: MC_UU_00006/5]. The funders had no role in informing the content of the manuscript.
Ethics approval and consent to participate
Consent for publication
TH is the lead author of the TIDieR guideline and Co-Director of the Australasian EQUATOR Centre. EvS is a senior associate editor for the Journal of Physical Activity and Health. She played no role in the editor’s decision to update the journal’s submission guidelines. She is also a member of the Editorial Board at the International Journal of Behavioral Nutrition and Physical Activity.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Ryan, M., Hoffmann, T., Hofmann, R. et al. Incomplete reporting of complex interventions: a call to action for journal editors to review their submission guidelines. Trials 24, 176 (2023). https://doi.org/10.1186/s13063-023-07215-1
- Reporting checklists
- Complex interventions
- Journal editors
- Research integrity
- Submission guidelines