Active ingredients are reported more often for pharmacologic than non-pharmacologic interventions: an illustrative review of reporting practices in titles and abstracts

Well designed and executed randomized controlled trials (RCTs) are the gold standard for assessing intervention efficacy and effectiveness, but study synthesis, comparison and reproducibility are compromised if interventions are poorly reported [1, 2]. Articles should contain details of key study components (Table 1) including ‘active ingredients’ [1, 3]. The term ‘active ingredient’ is frequently used to refer to the element within a pharmacologic intervention (PI) that is responsible for its therapeutic action. In contrast with PIs, non-pharmacologic interventions (NPIs) are usually complex, containing several interacting components that are all necessary for the intervention to be effective [4]. Such interventions may [5]:

The term ‘active ingredient’ was adopted in the UK complex intervention literature because early guidelines for the evaluation of complex interventions were based on the phases used to evaluate PIs [6]. The term ‘active ingredient’ refers to the components within an intervention that can be specifically linked to its effect on outcomes such that, if they were omitted, the intervention would be ineffective. For example, a cardiac rehabilitation intervention aimed at improving health-related behaviour associated with heart disease might focus on supporting smoking cessation, regular physical activity and a healthy diet [7]. Specific techniques used might include goal setting, providing information on consequences of behaviour and prompting self-monitoring of behaviour [8]. Provided these techniques have the potential to influence the health-related behaviour associated with heart disease (that is, causally influence outcomes) [7], they can be described as active ingredients. Therefore, throughout this article, the term ‘active ingredients’ is used to refer to these components in both PIs and NPIs. It is important to note that this is distinct from the mechanisms of action of interventions (the underlying reasons why the active ingredients have their particular effects) [7].

Accurate and comprehensive reporting of these components in titles and abstracts is essential [1, 9]. Abstracts are more widely circulated than full-text articles [10] and are generally the most widely available parts of articles [11]. Consequently, their content can have a greater than anticipated impact [9]. Inadequate specification of key components in titles and abstracts can have serious implications for systematic reviews, which are considered the best sources of evidence about the effectiveness of interventions [12]. Since reviewers base their initial inclusion decisions on abstracts [9], inadequate specification may result in studies being inappropriately rejected from the review, thus compromising review validity. Additionally, many readers use abstract content to determine whether full-text retrieval is worthwhile [1, 11, 13], while in certain countries, many healthcare professionals have easy access to abstracts but not to full texts [1, 9]. Therefore, it is important to investigate the quality of reporting in RCT abstracts.

The Consolidated Standards of Reporting Trials (CONSORT) guidelines [14] were developed to improve the quality of reporting of RCTs, and specific CONSORT guidelines (Table 1) have been developed for abstracts [1] and for NPIs [2], including surgical interventions, devices, rehabilitation packages and behaviour change interventions (BCIs) [15]. Owing to their complexity, NPIs are typically more difficult to standardize, describe and administer consistently than PIs while intervention success is often dependent on the expertise of the intervention providers [2, 15].

It is plausible that PIs may be reported more precisely than NPIs in titles and abstracts, owing to the complexity of NPIs and the restrictive word limits of titles and abstracts. The active ingredients of a PI can usually be specified clearly in one phrase within a sentence (for example, ‘Zoledronic acid’ [16]). However, an NPI often cannot be specified so succinctly: as a result, intervention objectives are often specified, while active ingredients are often not reported (for example, ‘Self-management programme. The programme teaches patients medical, social and emotional self-management skills.’ [17]). Comparisons of full-text articles have found that PIs are more often described with enough detail to be faithfully reproduced [18], and that descriptions of the key components of NPIs are reported less often and less precisely than those of PIs [19]. However, we are not aware of any reviews that have systematically evaluated the reporting of NPIs as compared with PIs in titles and abstracts.

Given the importance of accurate intervention reporting in titles and abstracts, an explicit comparison of reporting practices for PIs and NPIs would be useful for a number of reasons. Firstly, to establish whether there are similar disparities in the quality of abstract reporting between PIs and NPIs as there are for full-text articles. Reviews of full-text articles have illustrated how reporting of NPIs can be improved based on PI reporting practices; a comparative review of abstracts could similarly illustrate how the reporting of abstracts describing NPIs could be improved.

Title and abstract reporting practices for BCIs, a subset of NPIs that aim to modify health-related behaviour [20, 21], require special scrutiny. The CONSORT reporting guidelines have been adopted by most behavioural research journals [22]. Although Davidson and colleagues [3] proposed additional guidelines for reporting BCIs, these do not include specific recommendations for abstracts. It has been argued that full-text reports of BCIs usually do not fully comply with these guidelines [22]. However, to our knowledge, title and abstract reporting practices for BCIs have not been reviewed systematically.

These issues highlight that an explicit comparison of title and abstract reporting practices of PIs, NPIs and BCIs is necessary to clarify how reporting of NPIs and BCIs can be improved. This illustrative review compares the specification of interventions in titles and abstracts of published reports of RCTs. It was hypothesized that active ingredients would be reported more often for PIs than NPIs, and for other NPIs than BCIs.

Active ingredients were reported significantly less often in titles and abstracts for NPIs than PIs, and for BCIs than other types of NPIs. These results are in accordance with those described previously from reviews comparing full-text PI and NPI reports [18, 19]. This indicates that improvements are required in the reporting of NPIs in titles and abstracts, and that, in accord with much of the literature [35], titles and abstracts of BCI reports in particular lack a detailed intervention description. Without this key information, readers are left with an incomplete picture of what the intervention involved [14]. On the basis of our findings, we recommend that all authors routinely report active ingredients in titles and abstracts, and that all journal editors routinely require this.

Furthermore, abstract space limitations might contribute to these reporting differences. The CONSORT guidelines for abstracts state that 250 to 300 words are sufficient to report all recommended items [1]. However, it is likely that specifying PIs requires fewer words than NPIs. Another study of abstract reporting quality found that a greater abstract word count was associated with higher reporting quality for structured abstracts [13]. The 150-word limit for abstracts imposed by ABM therefore seems unsuitable. Journals could consider increasing abstract word limits to facilitate higher-quality abstract reporting, particularly for NPIs.

The exploratory analyses suggested that when intervention objectives and settings are reported for NPIs, active ingredients are often missing. This concurs particularly with previous behavioural research, where greater focus is often placed on reporting such aspects as the intervention objectives, trial setting, method of administration and intervention providers [3, 8]. Although these are important, describing interventions in this way can make different intervention content indistinguishable and thus make it difficult to determine the specific techniques that might be critical to intervention effectiveness [8].

Only 59% of the NPI reports mentioned the setting in the abstract. This seems low, given that the settings in which NPIs are administered are likely to interact with the intervention’s active ingredients to influence its effectiveness [40]. In addition, the effectiveness of an NPI can be influenced by the skills, experience and enthusiasm of the providers [18]. The CONSORT guidance for NPIs states that the number of providers involved in intervention delivery and details of their expertise should be reported in the abstract [2]. However, only 19% of the NPI reports contained any information about intervention providers in the abstract. Clearly, reporting of these components also requires improvement.

This is the first review to compare reporting of specific components of PIs and NPIs in titles and abstracts, and is therefore the first to show that active ingredients are reported more often in titles and abstracts for PIs than NPIs. There are, however, important limitations of this work. It was assumed that all included studies were of a high quality because they were published in high-impact journals: this cannot be confirmed, since a full quality assessment was not conducted. This was beyond the scope of this project. Our findings may be generalizable to other general medical journals of similar quality; however, the generalizability of our findings to other types of journal or conference abstracts may be limited. Journal recommendations for abstract word length were not verified at the outset of the study, and so there was imbalance between the journals targeted. Abstract limits were not checked because, as specified previously, our criterion for journal selection was high journal quality.

Data extraction and analysis were carried out by one reviewer. However, coding the active ingredients of NPIs was conducted by two reviewers, and there was general consensus regarding the data extracted during piloting. Study dropout must also be highlighted; one duplicate, one non-primary RCT report and four reports describing both intervention types were removed from the analysis. This resulted in unbalanced numbers of papers in the PI and NPI categories; these excluded papers could have been replaced, such that an equal number of papers in each group were analyzed.

The many differences between PIs and NPIs might make comparison difficult to interpret; however, existing abstract reporting guidelines were designed to apply to both intervention types, implying that both abstract types should be written to a similar standard. We have shown that this is not the case in these journals. Owing to the proportional sampling strategy used, only six articles published in ABM were included. The sample reflected the relative frequency of publication: fewer RCTs are published in ABM than in the other journals; therefore, fewer were retrieved and sampled. Although this reflects the relative frequency of publication, this is a limitation, given that this journal was specifically selected for inclusion of high-quality reports of BCIs.

One final important limitation concerns the link between abstract and full-text reporting. Reviews of full-text articles [18, 19] and reviews of abstracts [9, 10, 13, 41] have highlighted deficiencies in reporting. However, more research is required to verify whether the quality of abstract reporting is linked to that of full-text reporting. It is therefore not clear whether findings similar to ours would be obtained if full-text articles were reviewed in the same way: this is an important follow-up evaluation.

This review has several strengths. No intervention types were excluded, so the results apply broadly to reporting of PIs and NPIs. Included studies were recently published in high-impact peer-reviewed journals, so it appears that there are problems with very recent reporting practice even for high-quality reports. Finally, this review is unique in that it comprises an explicit comparison of the reporting of specific intervention components in titles and abstracts. The study of abstract quality is a fairly recent development [9], with most reviews focused on reporting of trial components [9, 10, 13]. Many studies use the CONSORT abstract reporting guidelines to evaluate quality. CONSORT specifies that intervention details should be reported, but gives no guidance on how they should be reported: this is the first review of abstracts to focus specifically on the level of detail provided regarding different types of intervention.

Active ingredients were more frequently reported in titles and abstracts for PIs than for NPIs, and more frequently for other NPIs than for BCIs. This review identifies the need for improved reporting of NPIs, and BCIs in particular. The creation of agreed labels and definitions for active ingredients of complex interventions, particularly BCIs, would contribute towards clearer reporting.