Skip to main content
Download PDF

Comparison of registered and published intervention fidelity assessment in cluster randomised trials of public health interventions in low- and middle-income countries: systematic review

Trials volume 19, Article number: 410 (2018) Cite this article

Abstract

Background

Cluster randomised trials (CRTs) are a key instrument to evaluate public health interventions. Fidelity assessment examines study processes to gauge whether an intervention was delivered as initially planned. Evaluation of implementation fidelity (IF) is required to establish whether the measured effects of a trial are due to the intervention itself and may be particularly important for CRTs of complex interventions conducted in low- and middle-income countries (LMICs). However, current CRT reporting guidelines offer no guidance on IF assessment. The objective of this review was to study current practices concerning the assessment of IF in CRTs of public health interventions in LMICs.

Methods

CRTs of public health interventions in LMICs that planned or reported IF assessment in either the trial protocol or the main trial report were included. The MEDLINE/PubMed, CINAHL and EMBASE databases were queried from January 2012 to May 2016. To ensure availability of a study protocol, CRTs reporting a registration number in the abstract were included. Relevant data were extracted from each study protocol and trial report by two researchers using a predefined screening sheet. Risk of bias for individual studies was assessed.

Results

We identified 90 CRTs of public health interventions in LMICs with a study protocol in a publicly available trial registry published from January 2012 to May 2016. Among these 90 studies, 25 (28%) did not plan or report assessing IF; the remaining 65 studies (72%) addressed at least one IF dimension. IF assessment was planned in 40% (36/90) of trial protocols and reported in 71.1% (64/90) of trial reports. The proportion of overall agreement between the trial protocol and trial report concerning occurrence of IF assessment was 66.7% (60/90). Most studies had low to moderate risk of bias.

Conclusions

IF assessment is not currently a systematic practice in CRTs of public health interventions carried out in LMICs. In the absence of IF assessment, it may be difficult to determine if CRT results are due to the intervention design, to its implementation, or to unknown or external factors that may influence results. CRT reporting guidelines should promote IF assessment.

Trial Registration

Protocol published and available at: https://doi.org/10.1186/s13643-016-0351-0

Peer Review reports

Background

Cluster randomised trials (CRTs) have become a key instrument to evaluate public health interventions [1,2,3,4,5,6,7], particularly in low- and middle-income countries (LMICs) [3, 8]. Randomised controlled trials (RCTs) are widely considered to provide the highest quality of evidence on the effectiveness of health interventions [9,10,11,12], and CRTs are a form of randomised trial in which clusters of individuals (such as families, villages, hospital services or schools), rather than independent individuals, are randomly allocated to intervention or control groups [2]. Increasingly, public health researchers recognise the importance of developing health interventions that target populations rather than individuals and can address the wide range of social and environmental factors influencing health [13, 14]. CRTs offer an appropriate design to assess such public health interventions as well as to measure the overall effect of an intervention at the population level [3, 5, 8, 13, 15], heterogeneity of impact among population subgroups and equity [16, 17].

Intervention fidelity in CRTs of public health interventions

Recent methodologically oriented systematic reviews have found evidence of improvements in the design and analysis of CRTs, while also noting deficiencies in trial implementation [18, 19]. Previous systematic reviews have emphasised the potential importance of process evaluation to mitigate these methodological problems, which can affect the internal and external validity of trial results [3, 18, 20,21,22].

‘Intervention fidelity’ refers to the degree to which an intervention is delivered as initially planned [23]. Fidelity assessment is the practice of evaluating intervention fidelity. Considered an aspect of process evaluation, fidelity assessment aims to understand and measure to what extent the intervention is being implemented as intended and identify specific reasons for the success or failure of the intervention [9, 23, 24]. The benefits of evaluating intervention fidelity include increased confidence in scientific findings, increased power to control for confounding factors and increased ability to evaluate the performance of theory-based interventions [25]. Fidelity assessment can improve internal and external validity of CRTs [18] by providing evidence that trial results are due to interventions themselves rather than to external factors and facilitating generalisation of results to contexts that may differ substantially from the original trial setting [9, 23]. Fidelity assessment may be particularly important for public health interventions, as these interventions tend to be complex and constituted by multiple components [10, 26] that act independently or interdependently [27], allowing greater potential for variation during implementation [23].

Framework for the evaluation of implementation fidelity

Table 1 outlines the conceptual framework used in this review, based on the work of Carroll et al. [23] and refined by Hasson [28]. We selected this framework to guide the review because it provides a comprehensive synthesis of previous work on intervention fidelity and has been widely influential.

Table 1 Conceptual framework for intervention fidelity used in this review
Full size table

Fidelity assessment in CRT reporting guidelines

The Consolidated Standards of Reporting Trials (CONSORT) group was created to provide guidance to improve the quality and transparency of RCT reporting [29], including CRTs [2, 30]. The CONSORT statement recognises that the trial protocol for a given study may not have been followed fully by some trial participants for a wide variety of reasons, including failure to receive the entire intervention as planned [29]. Cases of protocol non-adherence may influence interpretation and credibility of the results and thus the validity of the conclusions [18, 25, 31, 32]. To preserve the ability to make strong inferences about intervention effects, CONSORT recommends that all participants randomised be retained in the analysis and analysed according to their original assigned groups, an approach known as ‘intention-to-treat’ (ITT) analysis.

Reasons for protocol non-adherence in individually randomised RCTs may differ from those in CRTs. In a clinical trial setting, non-adherence depends largely on the actions of the trial participant and the treatment provider, which may in turn be related to issues such as treatment side effects and safety. In CRTs of public health interventions, protocol non-adherence may occur because complex interventions that include multiple components are delivered with poor fidelity. However, both for individually randomised parallel group trials [29] and CRTs [2, 30], current CONSORT guidelines offer no advice on methods to assess protocol non-adherence.

Rationale for undertaking this review

As earlier methodologically oriented systematic reviews have demonstrated, CRTs of complex public health interventions may be particularly at risk of experiencing protocol deviations and non-adherence, and these may compromise the validity of their findings [18, 19]. Although process evaluation techniques, such as evaluation of intervention fidelity, can help to identify these problems and mitigate their negative effects, current reporting guidelines for CRTs offer no specific guidance on intervention fidelity assessment. Wide divergence in current practices is therefore likely. We undertook a methodologically oriented systematic review of current practices related to assessment of intervention fidelity within CRTs of public health interventions in LMICs, so as to inform best practices for CRTs. To our knowledge, no other systematic review addresses this question. Although CRTs have been widely implemented to evaluate public health interventions in both high-income countries and LMIC contexts, interventions, approaches and outcomes may differ substantially between settings. Given the limitations of the resources, these challenges may be even more important in CRTs of public health interventions in LMICs. We therefore limit our focus to LMICs.

Objective

We conducted a methodologically oriented systematic review to address the following research questions:

  1. 1.

    What proportion of recent CRTs of public health interventions in LMICs planned to assess intervention fidelity (IF)?

  2. 2.

    What proportion of recent CRTs of public health interventions in LMICs reported assessing IF?

  3. 3.

    Which fidelity components were examined and which data collection methods were employed to assess each component?

  4. 4.

    Is there evidence of divergent practices between planned and reported studies?

Methods

We report results according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [33]; the PRISMA checklist is available in an online supplement (Additional file 1). For a detailed description of the methods, see the published protocol [34] (doi:https://doi.org/10.1186/s13643-016-0351-0). Some technical modifications were made while implementing the review. We report the date for each change in italics, along with the rationale.

Study eligibility

Study designs

We included all CRTs. For the purposes of this review, a CRT is defined as a trial in which “intact social units or clusters of individuals, rather than independent individuals, are randomly allocated to intervention groups” [30]. CRTs may include trials employing parallel group, stepped wedge, factorial, adaptive or pragmatic designs, among others.

Participants

Study participants were humans living in LMICs according to the World Bank country classifications [35].

Interventions

This review focuses on ‘public health interventions’. Following Rychetnik et al. [36, 37], we define a public health intervention as a disease prevention or health promotion intervention applied to many, most or all members in a community, which aims to deliver a net benefit to the community or population, as well as benefits to individuals. Assessment of intervention fidelity may be especially important for public health interventions, which are inherently “complex, programmatic, and context dependent” [36]. In order to operationalise this definition and guide study inclusion decisions, we used the Minnesota Department of Health ‘Intervention Wheel’ [38].

Comparators

Comparators were defined as per the original CRT.

Outcomes

We included CRTs that assessed IF in the trial protocol, the main trial report or both. We also included CRTs reporting assessment of IF in a complementary document such as a published article, an online appendix to the main paper, or a grey literature report, in lieu of reporting assessment of IF in the main trial report only. We considered studies as evaluating IF if they proposed or implemented methods to assess at least one of the four key fidelity components, namely (1) content, (2) coverage, (3) frequency and (4) duration.

Report characteristics

Availability of the study protocol

To ensure availability of a study protocol, we included CRTs reporting a registration number in the abstract for any registry meeting WHO Trial Registration Data Set criteria [39]. The Trial Registration Data Set was used in this review to evaluate planned assessment of intervention fidelity, either alone or in conjunction with a published study protocol.

Exclusion criteria

We excluded studies that (1) were not cluster randomised trials, (2) did not plan or report assessment of IF, (3) were not public health interventions, (4) were conducted in high-income countries [35], (5) were published before 2012, (6) did not have a publicly available protocol or (7) for which only the protocol but not the main trial report has been published. Manuscripts were also excluded if they were (8) pilot studies, (9) secondary reports of a main study for which the relevant findings were published prior to 2012, date of the last update to the CONSORT Statement to improve reporting of CRTs, (10) studies published in a language other than English, Spanish or French, or (11) CRT trials published only in the grey literature.

Information sources and search strategy

Literature search strategies were developed in collaboration with an academic librarian. The search strategy was limited to the period from January 2012 to May 2016 and, due to resource constraints, was not updated towards the end of the review (12-07-2017). Search strategies used MESH and text words related to cluster randomised trials, developing countries and public health interventions. The electronic database search was developed first for MEDLINE (ovid) and then adapted for EMBASE (ovid), CINAHL (ovid), PubMed and EMB Reviews (ovid). The full search strategy is available in an online supplement (Additional file 2). Searches were filtered to articles concerning humans, written in English, French or Spanish. We added relevant studies as suggested by the review team. Identified records were uploaded into EndNote software and duplicates were eliminated.

Study screening and selection

Study screening and selection were performed manually within EndNote. To ensure the availability of study protocols, we limited the search to CRTs that have the word “regist*” in the abstract and used these results to begin the process of screening and selection. We validated this procedure by examining a subset of excluded articles. Screening and selection were performed in two stages by two independent reviewers (MCP and NM). In the first stage, reviewers independently screened the titles and abstracts of each identified reference against the inclusion criteria to eliminate irrelevant publications. In the second stage, each reviewer independently screened the full text of all studies that appeared to meet the inclusion criteria, or for which eligibility was uncertain. For each study, reviewers identified additional articles of potential relevance by reviewing references from the main trial report, consulting the trial registry record and searching the PubMed database for new publications by the lead trial author or using trial identifiers. To aid in article screening and selection, the team developed and tested a full text screening sheet. Any disagreements between reviewers were resolved through discussion and, as necessary, through arbitration by a third author (MJ). A summary table describing studies excluded with reasons is available in an online supplement (Additional file 3).

Data extraction and data items

According to our protocol, two review authors would extract data independently. Given the volume and complexity of materials to review, it was not practical for both reviewers to abstract data independently. We adopted the following procedure (20-09-2016): included studies were randomly assigned to each of the two reviewers. Relevant data were extracted from each study protocol and trial report by one reviewer (MCP or NM) and cross-checked by the other reviewer (MCP or NM). To ensure high quality data extraction and to reach consensus, frequent meetings were held by the two reviewers (MCP and NM), including a third author (MJ) when necessary.

Outcomes and prioritisation

The search and selection process for this review was designed to identify two quantities required for calculation of outcomes based on proportions, namely (1) numerator: these are studies that meet all inclusion and exclusion criteria. As for all systematic reviews, these studies were our principal focus and were included in the review. (2) Denominator: this is the total of studies for this review (N), which we defined as all studies that satisfy all inclusion and exclusion criteria, with the exception of the outcome criterion (planned or report IF assessment).

Primary outcome

The primary outcome for this study was the proportion of overall agreement between the protocol and trial report concerning occurrence of IF assessment. This corresponds to research question 4. Data were summarised in a two-by-two table comparing assessment of intervention fidelity in the trial report to that in the protocol. ‘N’ represents the set of recently published CRTs of public health interventions in LMICs that have registered the study protocol in a publicly availably trial registry. For each CRT in N, we determined whether IF was assessed in the registered (or published) protocol or in the trial report (or associated documents). Studies judged to have assessed IF were coded as “1”; others were coded as “0”. The proportion of overall agreement is defined as the proportion of eligible CRTs for which the protocol and the trial report agree in the practice of intervention fidelity assessment (i.e. both positive or both negative). It was computed as (a+d) /N.

 

Protocol

 

+

–

Trial Report

+ a

b

a + b

– c

d

c + d

a + c

b + d

N

Secondary outcomes

To address research questions 1, 2 and 3, we also calculated:

  • The frequency and proportion of trial protocols reporting assessment of intervention fidelity, out of N

  • The frequency and proportion of trial reports reporting assessment of intervention fidelity, out of N

  • The proportion of positive agreement among those that agree, computed as a/(a+d)

  • Frequency counts and percentages summarising fidelity components examined and data collection methods proposed or employed

Risk of bias in individual studies

We used the Cochrane Collaboration tool to assess the risk of bias in individually randomised trials [40] and included additional criteria relevant to assessing risk of bias in CRTs [40,41,42], which are available in an online supplement (Additional file 4). We assessed each criterion as low, high or uncertain risk, and provided a sample text that illustrates the reasons for this judgment. The initial evaluation was performed by the primary reviewer (MCP or NM) and verified by the other reviewer (MCP or NM). In case of doubt, a third reviewer (MJ) was consulted. To investigate whether study quality influences IF practices, we stratified the sample to create categories of low risk studies (7 or more of the 10 criteria were evaluated as low risk, n = 36 studies), versus uncertain or high-risk studies (6 or fewer of the 10 criteria were evaluated as low risk, n = 29 studies). Cut-points were chosen empirically to create groups of roughly equal size.

Data synthesis

A narrative synthesis was used to explore relationships and findings within and between the included studies. As our review focuses on methodological issues, we did not assess potential publication bias.

Results

Study selection

The electronic search identified 6876 publications, of which 226 potentially relevant studies were assessed in full text. Stage 1 of the full text review was used to define the study denominator, as follows: of the 226 articles screened, 90 were CRTs of public health interventions in LMICs with a study protocol in a publicly available trial registry published from January 2012 to May 2016. In Stage 2 of the full text review, we reviewed these 90 studies to identify those that in addition considered IF in either the protocol, the trial report or both. We excluded 25 studies (28%) that did not plan or report assessing IF. The remaining 65 studies addressed at least one IF dimension [43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,50,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107]. Figure 1 shows the PRISMA flow chart.

Fig. 1
figure 1

Flow diagram summarising study selection process. 1There are 65 studies comprising 65 protocols from trial registries, 21 published peer-reviewed protocols, 15 grey literature protocols, 65 published peer-reviewed main trial reports, 13 secondary peer-reviewed publications reporting trial results, 6 complementary grey literature documents

Full size image

Study characteristics

The 65 studies were published in English between January 2012 and May 2016. The included studies are very diverse in terms of sample size and target population, ranging from 190 to more than 280,000 eligible participants per study. Table 2 presents general characteristics of the studies included in the review. Characteristics of the individual studies included are summarised in an online supplement (Additional file 5). Interventions reviewed addressed diverse public health aims; a detailed classification is available in an online supplement (Additional file 6).

Table 2 General characteristics of included studies (n = 65)
Full size table

Outcome

We identified 90 CRTs of public health interventions in LMICs with a study protocol in a publicly available trial registry published from January 2012 to May 2016. IF assessment was planned in 40% (36/90) of trial protocols and reported in 71.1% (64/90) of trial reports. Notably, 27.8% (25/90) of studies did not assess any IF dimension in either the trial or the protocol. Practices concerning IF assessment diverged between trial protocols and reports (Table 3). The proportion of overall agreement between the trial protocol and trial report concerning occurrence of IF assessment was 66.7% (60/90). The proportion of positive agreement among those that agree was 58.3% (35/60). We were unable to compute outcomes 4b, 4c and 4d as specified in the original protocol [34].

Table 3 Intervention fidelity assessment among cluster randomised trials of public health interventions in low- and middle-income countries (n = 90)
Full size table

Practices of intervention fidelity assessment in protocols and trial reports

Of 65 studies assessing IF, 71% (46/65) described fidelity assessment only in the main trial report, 20% (13/65) of studies published a process evaluation, and 9% (6/65) of studies detailed IF in a complementary document. We found that fidelity components and moderating factors were not systematically assessed in CRTs (Table 4). In addition, only 28% (18/65) of studies monitored events taking place in control groups.

Table 4 Fidelity components and moderating factors identifieda (n = 65)
Full size table

Methods employed to assess fidelity components and moderating factors

Among the 65 the CRTs reviewed, a variety of procedures were used to assess, maintain and monitor intervention fidelity, including (1) training in intervention delivery, (2) consultation of manuals and standardised guidelines, (3) regular meetings between personnel involved in intervention delivery and researchers to provide feedback, support and monitoring during implementation, (4) logistical and management support for trial coordinators, (5) support teams composed of local area and site managers, (6) supervision and monitoring to ensure data quality, and (7) field notes, self-reported evaluation or checklists to verify implementation progress. Researchers often reported the strategies used to promote intervention fidelity but failed to describe methods used to assess IF [108].

Risk of bias within studies

All studies reported accounting for clustering in the analysis. Due to the nature of the interventions, 74% (48/65) of the studies presented a high risk of bias in the category of ‘blinding of participants and personnel’. Only one study (1.5%) had low risk of bias for all dimensions evaluated [46]. The risk of bias graph for individual studies is available in an online supplement (Additional file 7). The overall risk of bias summary is reported in Fig. 2.

Fig. 2
figure 2

Summary of risk of bias in included randomised controlled trials

Full size image

Subgroup analyses

To better understand heterogeneity of results concerning IF assessment, we compared studies at low risk of bias versus those at uncertain or high risk of bias (combined). We found no clear differences in IF assessment between subgroups. Results are available in an online supplement (Additional file 8).

Discussion

Fidelity assessment may contribute to making studies more reliable, internally valid and externally generalisable [109]. Our review shows that fidelity assessment is not currently a systematic practice in CRTs of public health interventions carried out in LMICs. In addition, the amount of detail given and data collection methods used for each component were variable between studies. In the absence of fidelity assessment, it may be difficult to determine if CRT results are due to the intervention design, to its implementation, or to unknown or external factors that may influence results.

Strengths and limitations of the study

The strengths of this review include publication of the study protocol, adherence to the PRISMA guidelines, the use of five bibliographic databases providing comprehensive information on randomised trials [110], and recognising that word limits for scientific journal articles are highly constrained and that the current CONSORT reporting guidelines for CRTs do not require a description of elements related to IF, extending the search for IF assessment to associated documents beyond the main trial report.

Our study also has limitations. First, we included only CRTs that have the word “regist*” in the abstract. Therefore, we may not have identified all CRTs. However, we reviewed 16 excluded studies to assess this risk but none met our inclusion criteria. Thus, the effect of this limitation is likely to be small. Second, according to the original protocol [34], we were unable to compute outcomes for research questions 4b (Are trial reports with negative findings for the ITT analysis more likely to report a per-protocol (PP) analysis?), 4c (What is the overall agreement between ITT and PP analyses concerning intervention effectiveness?), and 4d (Does the magnitude of the intervention effect differ for PP as compared to ITT analyses?) for two main reasons. Firstly, it was not always possible to determine whether articles had performed an ITT analysis to assess the intervention effect. Although the CONSORT statement requires that authors indicate whether analysis were performed on an ITT basis, 16 (25%) studies did not explicitly declare the analysis as ITT or PP. These findings confirm that ITT analysis is “often inadequately described and inadequately applied” [111] in relation to missing outcome data in RCTs [111, 112]. Secondly, many articles reported more than one primary outcome; 58% (38/65) of the studies had more than one primary outcome, while 42% (27/65) of studies have results for one primary outcome. Within studies, conclusions regarding the effectiveness of interventions sometimes diverged; 55% (36/65) of studies reported at least one outcome effective.

Strengths and weaknesses in relation to other studies

This is the first published systematic review focussing on methodological issues related to fidelity assessment in CRTs. Previous reviews have shown that the information required to ensure internal and external validity is often poorly reported in CRTs [18, 113]. This study complements previous reviews by documenting considerable variation in the practice of fidelity assessment for CRTs of public health interventions in LMICs and suggesting the potential need for methodological improvement in this area.

Interpretation of findings

This review demonstrates that fidelity assessment is not currently a systematic practice in CRTs. Public health interventions are complex and may be affected by multiple contextual factors affecting the internal and external validity of the results. In CRTs, contextual differences at the cluster level could favour local adaptation of the intervention and differences in participants’ exposure to intervention components [114, 115]. This variability can lead to the heterogeneity of fidelity, reducing power to detect significant effects where they might be present. Fidelity assessment could help to fill this knowledge gap to aid researchers in attributing outcomes to the actual intervention and identify contextual factors that influence intervention in CRTs [18, 20, 27]. However, current reporting guidelines for CRTs offer no specific guidance on intervention fidelity assessment.

Failure to consider fidelity assessment in CRTs can be explained by several factors. First, approaches to assess intervention fidelity within randomised trials have not yet been made accessible to practitioners. Carroll et al. synthesise previous work [23, 25, 28, 109, 116], establishing a conceptual framework to evaluate intervention fidelity [116,117,118,119]. However, this framework does not provide guidance on methods to assess specific fidelity dimensions in practice. Second, fidelity assessment is not required at present by the CONSORT guidelines for CRT reporting. This does not encourage researchers to think about the importance of assessing intervention fidelity, nor to report their evaluations if conducted. Third, scientific journals limit the number of words in the final report, which may restrict authors’ ability to report their evaluations.

Conclusions

Randomised trials are generally viewed as the gold standard for establishing evidence of intervention effectiveness. Public health interventions require high-quality evaluations to determine whether the programmes work or not and to know how to improve them [120, 121]. Results of this methodologically oriented systematic review demonstrate considerable heterogeneity in the practice of fidelity assessment in CRTs of public health interventions carried out in LMICs. Failure to assess intervention fidelity may weaken the internal and external validity of a study. We offer two recommendations to enhance the quality of future CRTs.

First, CRT reporting guidelines should promote fidelity assessment as a means to facilitate valid causal inference and replication of results. Guidelines, such as those produced by the Cochrane network and by CONSORT, play an important role in helping authors to strengthen study methods and reporting and thereby in improving the quality of health research [2, 29, 30, 40, 122, 123]. The most recent Cochrane risk of bias guidelines include, for the first time, consideration of the potential biases arising from deviations from intended interventions [122, 123]. Based on the results of this review, we make two suggestions to strengthen CRT methods and reporting. CRT guidelines such as those produced by CONSORT should promote fidelity assessment and reporting to improve the transparency of reports and to help draw firm conclusions about the outcomes of interventions. By the same principle, scientific journals should encourage researchers to conduct and report IF assessment in CRTs.

Second, a practical tool is required to guide trialists involved in the design and implementation of CRTs on how to assess IF. The TIDieR checklist encourages researchers to improve the reporting of interventions and suggests, within its 12 items, that authors describe how the fidelity of the intervention was assessed [124]. However, to our knowledge, neither the TIDieR checklist nor any other resource offers researchers a practical guide on how to approach interventionfidelity. This is a fundamental gap that should be remedied.

Abbreviations

CONSORT:

Consolidated Standards of Reporting Trials

CRTs:

cluster randomisation trials

IF:

Intervention fidelity

ITT:

intention-to-treat

LMICs:

low- and middle-income countries

PP:

per protocol

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

RCTs:

Randomised controlled trials

WHO:

World Health Organization

References

  1. Bland JM. Cluster randomised trials in the medical literature: two bibliometric surveys. BMC Med Res Methodol. 2004;4(1):21.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Campbell MK, Elbourne DR, Altman DG, CONSORT Group. CONSORT statement: extension to cluster randomised trials. BMJ. 2004;328(7441):702–8.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Isaakidis P, Ioannidis JP. Evaluation of cluster randomized controlled trials in sub-Saharan Africa. Am J Epidemiol. 2003;158(9):921–6.

    Article  PubMed  Google Scholar 

  4. Campbell MJ, Donner A, Klar N. Developments in cluster randomized trials and statistics in medicine. Stat Med. 2007;26(1):2–19.

    Article  PubMed  CAS  Google Scholar 

  5. Moberg J, Kramer M. A brief history of the cluster randomised trial design. J R Soc Med. 2015;108(5):192–8.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Campbell MK, Mollison J, Steen N, Grimshaw JM, Eccles M. Analysis of cluster randomized trials in primary care: a practical approach. Fam Pract. 2000;17:192–6.

    Article  PubMed  CAS  Google Scholar 

  7. Osrin D, Azad K, Fernandez A, et al. Ethical challenges in cluster randomized controlled trials: experiences from public health interventions in Africa and Asia. Bull World Health Org. 2009;87(10):772–9.

    Article  PubMed  Google Scholar 

  8. Handlos LN, Chakraborty H, Sen PK. Evaluation of cluster-randomized trials on maternal and child health research in developing countries. Trop Med Int Health. 2009;14(8):947–56.

    Article  PubMed  Google Scholar 

  9. Richards DA, Hallberg IR. Complex Interventions in Health: An Overview of Research Methods. London & New York: Routledge; 2015.

    Book  Google Scholar 

  10. Campbell M, Fitzpatrick R, Haines A, et al. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000;321(7262):694–6.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  11. Craig P, Cooper C, Gunnell D, et al. Using natural experiments to evaluate population health interventions: new Medical Research Council guidance. J Epidemiol Commun Health. 2012;66(12):1182–6.

    Article  Google Scholar 

  12. Evidence-Based, Medicine Working, Group. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA. 1992;268(17):2420.

    Article  Google Scholar 

  13. Sanson-Fisher RW, D’Este CA, Carey ML, Noble N, Paul CL. Evaluation of systems-oriented public health interventions: alternative research designs. Annu Rev Public Health. 2014;35:9–27.

    Article  PubMed  Google Scholar 

  14. Upshur RE. Principles for the justification of public health intervention. Can J Public Health. 2002;93(2):101–3.

    PubMed  CAS  Google Scholar 

  15. Hayes RJ, Alexander ND, Bennett S, Cousens SN. Design and analysis issues in cluster-randomized trials of interventions against infectious diseases. Stat Methods Med Res. 2000;9(2):95–116.

    Article  PubMed  CAS  Google Scholar 

  16. Morris SS, Ranson MK, Sinha T, Mills AJ. Measuring improved targeting of health interventions to the poor in the context of a community-randomised trial in rural India. Contemp Clin Trials. 2007;28(4):382–90.

    Article  PubMed  Google Scholar 

  17. Ranson MK, Sinha T, Morris SS, Mills AJ. CRTs–cluster randomized trials or “courting real troubles”: challenges of running a CRT in rural Gujarat, India. Can J Public Health. 2006;97(1):72.

    PubMed  PubMed Central  Google Scholar 

  18. Eldridge S, Ashby D, Bennett C, Wakelin M, Feder G. Internal and external validity of cluster randomised trials: systematic review of recent trials. BMJ. 2008;336(7649):876–80.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Bonell C, Oakley A, Hargreaves J, Strange V, Rees R. Assessment of generalisability in trials of health interventions: suggested framework and systematic review. Br Med J. 2006;333:346–9.

    Article  CAS  Google Scholar 

  20. Oakley A, Strange V, Bonell C, Allen E, Stephenson J. Process evaluation in randomised controlled trials of complex interventions. BMJ. 2006;332(7583):413–6.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Brierley G, Brabyn S, Torgerson D, Watson J. Bias in recruitment to cluster randomized trials: a review of recent publications. J Eval Clin Pract. 2012;18(4):878–86.

    Article  PubMed  Google Scholar 

  22. Puffer S, Torgerson D, Watson J. Evidence for risk of bias in cluster randomised trials: review of recent trials published in three general medical journals. BMJ. 2003;327(7418):785–9.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual framework for implementation fidelity. Implement Sci. 2007;2:40.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Medical Research Council. Developing and Evaluating Complex Interventions: New Guidance. London: Medical Research Council; 2008.

    Google Scholar 

  25. Borrelli B. The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. J Public Health Dent. 2011;71(s1):S52–63.

    Article  PubMed Central  PubMed  Google Scholar 

  26. Hawe P. Lessons from complex interventions to improve health. Annu Rev Public Health. 2015;36:307–23.

    Article  PubMed  Google Scholar 

  27. Moore GF, Audrey S, Barker M, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Hasson H. Systematic evaluation of implementation fidelity of complex interventions in health and social care. Implement Sci. 2010;5:67.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol. 2010;63(8):e1–e37.

    Article  PubMed  Google Scholar 

  30. Campbell MK, Piaggio G, Elbourne DR, Altman DG, CONSORT Group. CONSORT 2010 Statement: extension to cluster randomised trials. BMJ. 2012;345:e5661.

    Article  PubMed  Google Scholar 

  31. Schneeweiss S. Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiol Drug Saf. 2006;15(5):291–303.

    Article  PubMed  Google Scholar 

  32. Thabane L, Mbuagbaw L, Zhang S, Samaan Z, Marcucci M, Ye C, Debono VB. A tutorial on sensitivity analyses in clinical trials: the what, why, when and how. BMC Med Res Methodol. 2013;13(1):92.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Medicine. 2009;6(7):e1000097.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Perez MC, Minoyan N, Ridde V, Sylvestre MP, Johri M. Comparison of registered and published intervention fidelity assessment in cluster randomised trials of public health interventions in low- and middle-income countries: systematic review protocol. Syst Rev. 2016;5(1):177.

    Article  PubMed  PubMed Central  Google Scholar 

  35. The World, Bank. Country and Lending Groups. Data & Statistics: Country Classification (2016). Washington, DC. https://datahelpdesk.worldbank.org/knowledgebase/articles/906519. Accessed 3 Jul 2016.

  36. Rychetnik L, Frommer M, Hawe P, Shiell A. Criteria for evaluating evidence on public health interventions. J Epidemiol Community Health. 2002;56:119–27.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  37. Rychetnik L, Hawe P, Waters E, Barratt A, Frommer M. A glossary for evidence based public health. J Epidemiol Community Health. 2004;58(7):538–45.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Keller LO, Strohschein S, Lia-Hoagberg B, Schaffer MA. Population-based public health interventions: practice-based and evidence-supported. Part I. Public Health Nurs. 2004;21(5):453–68.

    Article  PubMed  Google Scholar 

  39. World Health Organization. WHO International Clinical Trials Registry Platform (ICRP). New Standards for Registration of Human Medical Research. Trial Registration Data Set (version 1.2.1), 2016. http://www.who.int/ictrp/network/trds/en/. Accessed 15 May 2016.

  40. Higgins JP, Altman DG, Gøtzsche PC, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Hayes R, Moulton L. Cluster randomised trials. Taylor & Francis; 2009.

  42. Flynn TN. Design and Analysis of Cluster Randomization Trials in Health Research. Donner A and Klar N. London: Arnold; 2000. pp.178, £35.00. ISBN: 0-340-69153-0. Int J Epidemiol. 2001;30(2)407–8. https://doi.org/10.1093/ije/30.2.407-a.

  43. Abramsky T, Devries K, Kiss L, et al. Findings from the SASA! Study: a cluster randomized controlled trial to assess the impact of a community mobilization intervention to prevent violence against women and reduce HIV risk in Kampala, Uganda. BMC Med. 2014;12:122.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Althabe F, Belizan JM, McClure EM, et al. A population-based, multifaceted strategy to implement antenatal corticosteroid treatment versus standard care for the reduction of neonatal mortality due to preterm birth in low-income and middle-income countries: the ACT cluster-randomised trial. Lancet. 2015;385(9968):629–39.

    Article  PubMed  Google Scholar 

  45. Awasthi S, Peto R, Read S, Richards SM, Pande V, Bundy D. Population deworming every 6 months with albendazole in 1 million pre-school children in North India: DEVTA, a cluster-randomised trial. Lancet. 2013;381(9876):1478–86.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  46. Boisson S, Stevenson M, Shapiro L, et al. Effect of household-based drinking water chlorination on diarrhoea among children under five in Orissa, India: a double-blind randomised placebo-controlled trial. PLoS Med. 2013;10(8):e1001497.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  47. Boone P, Elbourne D, Fazzio I, et al. Effects of community health interventions on under-5 mortality in rural Guinea-Bissau (EPICS): a cluster-randomised controlled trial. Lancet Glob Health. 2016;4(5):e328–35.

    Article  PubMed  Google Scholar 

  48. Bousema T, Stresman G, Baidjoe AY, et al. The impact of hotspot-targeted interventions on malaria transmission in rachuonyo south district in the western Kenyan highlands: a cluster-randomized controlled trial. PLoS Med. 2016;13(4):e1001993.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Chen-Hussey V, Carneiro I, Keomanila H, et al. Can topical insect repellents reduce malaria? A cluster-randomised controlled trial of the insect repellent N,N-diethyl-m-toluamide (DEET) in Lao PDR. PLoS One. 2013;8(8):e70664.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  50. Clasen T, Boisson S, Routray P, et al. Effectiveness of a rural sanitation programme on diarrhoea, soil-transmitted helminth infection, and child malnutrition in Odisha, India: a cluster-randomised trial. Lancet Glob Health. 2014;2(11):e645–53.

    Article  PubMed  Google Scholar 

  51. Coates TJ, Kulich M, Celentano DD, et al. Effect of community-based voluntary counselling and testing on HIV incidence and social and behavioural outcomes (NIMH Project Accept; HPTN 043): a cluster-randomised trial. Lancet Glob Health. 2014;2(5):e267–77.

    Article  PubMed  PubMed Central  Google Scholar 

  52. Colbourn T, Nambiar B, Bondo A, et al. Effects of quality improvement in health facilities and community mobilization through women's groups on maternal, neonatal and perinatal mortality in three districts of Malawi: MaiKhanda, a cluster randomized controlled effectiveness trial. International Health. 2013;5(3):180–95.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Corbel V, Akogbeto M, Damien GB, et al. Combination of malaria vector control interventions in pyrethroid resistance area in Benin: a cluster randomised controlled trial. Lancet Infect Dis. 2012;12(8):617–26.

    Article  PubMed  Google Scholar 

  54. Deressa W, Yihdego YY, Kebede Z, Batisso E, Tekalegne A, Dagne GA. Effect of combining mosquito repellent and insecticide treated net on malaria prevalence in Southern Ethiopia: a cluster-randomised trial. Parasit Vectors. 2014;7:132.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  55. Devries KM, Knight L, Child JC, et al. The Good School Toolkit for reducing physical violence from school staff to primary school students: a cluster-randomised controlled trial in Uganda. Lancet Glob Health. 2015;3(7):e378–86.

    Article  PubMed  PubMed Central  Google Scholar 

  56. Feikin DR, Bigogo G, Audi A, et al. Village-randomized clinical trial of home distribution of zinc for treatment of childhood diarrhea in rural Western kenya. PLoS One. 2014;9(5):e94436.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  57. Go VF, Frangakis C, Minh NL, et al. Efficacy of a multi-level intervention to reduce injecting and sexual risk behaviors among HIV-infected people who inject drugs in Vietnam: a four-arm randomized controlled trial. PLoS One. 2015;10(5):e0125909.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  58. Halliday KE, Okello G, Turner EL, et al. Impact of intermittent screening and treatment for malaria among school children in Kenya: a cluster randomised trial. PLoS Med. 2014;11(1):e1001594.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Hanson C, Manzi F, Mkumbo E, et al. Effectiveness of a home-based counselling strategy on neonatal care and survival: a cluster-randomised trial in six districts of rural southern Tanzania. PLoS Med. 2015;12(9):e1001881.

    Article  PubMed  PubMed Central  Google Scholar 

  60. He FJ, Wu Y, Feng XX, et al. School based education programme to reduce salt intake in children and their families (School-EduSalt): cluster randomised controlled trial. BMJ. 2015;350:h770.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Huybregts L, Houngbe F, Salpeteur C, et al. The effect of adding ready-to-use supplementary food to a general food distribution on child nutritional status and morbidity: a cluster-randomized controlled trial. PLoS Med. 2012;9(9):e1001313.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  62. Kirkwood BR, Manu A, ten Asbroek AH, et al. Effect of the Newhints home-visits intervention on neonatal mortality rate and care practices in Ghana: a cluster randomised controlled trial. Lancet. 2013;381(9884):2184–92.

    Article  PubMed  Google Scholar 

  63. Lewycka S, Mwansambo C, Rosato M, et al. Effect of women's groups and volunteer peer counselling on rates of mortality, morbidity, and health behaviours in mothers and children in rural Malawi (MaiMwana): a factorial, cluster-randomised controlled trial. Lancet. 2013;381(9879):1721–35.

    Article  PubMed  PubMed Central  Google Scholar 

  64. More NS, Bapat U, Das S, et al. Community mobilization in Mumbai slums to improve perinatal care and outcomes: a cluster randomized controlled trial. PLoS Med. 2012;9(7):e1001257.

    Article  PubMed  PubMed Central  Google Scholar 

  65. Nikiema L, Huybregts L, Kolsteren P, et al. Treating moderate acute malnutrition in first-line health services: an effectiveness cluster-randomized trial in Burkina Faso. Am J Clin Nutr. 2014;100(1):241–9.

    Article  PubMed  CAS  Google Scholar 

  66. Onwujekwe O, Mangham-Jefferies L, Cundill B, et al. Effectiveness of provider and community interventions to improve treatment of uncomplicated malaria in Nigeria: a cluster randomized controlled trial. PLoS One. 2015;10(8):e0133832.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  67. Pasha O, McClure EM, Wright LL, et al. A combined community- and facility-based approach to improve pregnancy outcomes in low-resource settings: a global network cluster randomized trial. BMC Med. 2013;11:215.

    Article  PubMed  PubMed Central  Google Scholar 

  68. Pinder M, Jawara M, Jarju LB, et al. Efficacy of indoor residual spraying with dichlorodiphenyltrichloroethane against malaria in Gambian communities with high usage of long-lasting insecticidal mosquito nets: a cluster-randomised controlled trial. Lancet. 2015;385(9976):1436–46.

    Article  PubMed  CAS  Google Scholar 

  69. Raj A, Ghule M, Ritter J, et al. Cluster randomized controlled trial evaluation of a gender equity and family planning intervention for married men and couples in rural India. PLoS One. 2016;11(5):e0153190.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  70. Rotheram-Borus MJ, Richter LM, van Heerden A, et al. A cluster randomized controlled trial evaluating the efficacy of peer mentors to support South African women living with HIV and their infants. PLoS One. 2014;9(1):e84867.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  71. Soofi S, Ahmed S, Fox MP, et al. Effectiveness of community case management of severe pneumonia with oral amoxicillin in children aged 2-59 months in Matiari district, rural Pakistan: a cluster-randomised controlled trial. Lancet. 2012;379(9817):729–37.

    Article  PubMed  Google Scholar 

  72. Tian M, Ajay VS, Dunzhu D, et al. A cluster-randomized, controlled trial of a simplified multifaceted management program for individuals at high cardiovascular risk (SimCard Trial) in rural Tibet, China, and Haryana, India. Circulation. 2015;132(9):815–24.

    Article  PubMed  PubMed Central  Google Scholar 

  73. Tripathy P, Nair N, Sinha R, et al. Effect of participatory women's groups facilitated by Accredited Social Health Activists on birth outcomes in rural eastern India: a cluster-randomised controlled trial. Lancet Glob Health. 2016;4(2):e119–28.

    Article  PubMed  Google Scholar 

  74. Waiswa P, Pariyo G, Kallander K, et al. Effect of the Uganda Newborn Study on care-seeking and care practices: a cluster-randomised controlled trial. Glob Health Action. 2015;8:24584.

    Article  PubMed  Google Scholar 

  75. Wang PC, Mwango A, Moberley S, et al. A cluster randomised trial on the impact of integrating early infant HIV diagnosis with the expanded programme on immunization and HIV testing rates in rural health facilities in southern Zambia. PLoS One. 2015;10(10):e0141455.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  76. Yotebieng M, Labbok M, Soeters HM, et al. Ten Steps to Successful Breastfeeding programme to promote early initiation and exclusive breastfeeding in DR Congo: a cluster-randomised controlled trial. Lancet Glob Health. 2015;3(9):e546–55.

    Article  PubMed  Google Scholar 

  77. Suchdev PS, Ruth LJ, Woodruff BA, et al. Selling Sprinkles micronutrient powder reduces anemia, iron deficiency, and vitamin A deficiency in young children in western Kenya: a cluster-randomized controlled trial. Am J Clin Nutr. 2012;95(5):1223–30.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  78. Andersson N, Nava-Aguilera E, Arostegui J, et al. Evidence based community mobilization for dengue prevention in Nicaragua and Mexico (Camino Verde, the Green Way): cluster randomized controlled trial. BMJ. 2015;351:h3267.

    Article  PubMed  PubMed Central  Google Scholar 

  79. Andrade S, Lachat C, Ochoa-Aviles A, et al. A school-based intervention improves physical fitness in Ecuadorian adolescents: a cluster-randomized controlled trial. Int J Behav Nutr Phys Act. 2014;11:153.

    Article  PubMed  PubMed Central  Google Scholar 

  80. Ansah EK, Narh-Bana S, Affran-Bonful H, et al. The impact of providing rapid diagnostic malaria tests on fever management in the private retail sector in Ghana: a cluster randomized trial. BMJ. 2015;350:h1019.

    Article  PubMed  PubMed Central  Google Scholar 

  81. Attanasio OP, Fernandez C, Fitzsimons EO, Grantham-McGregor SM, Meghir C, Rubio-Codina M. Using the infrastructure of a conditional cash transfer program to deliver a scalable integrated early child development program in Colombia: cluster randomized controlled trial.[Erratum appears in BMJ. 2014;349:g6126]. BMJ. 2014;349:g5785.

    Article  PubMed  PubMed Central  Google Scholar 

  82. Aung T, Montagu D, Su Su Khin H, Win Z, San AK, McFarland W. Impact of a social franchising program on uptake of oral rehydration solution plus zinc for childhood diarrhea in myanmar: a community-level randomized controlled trial. J Trop Pediatr. 2014;60(3):189–97.

    Article  PubMed  Google Scholar 

  83. Ayles H, Muyoyeta M, Du Toit E, et al. Effect of household and community interventions on the burden of tuberculosis in southern Africa: the ZAMSTAR community-randomised trial. Lancet. 2013;382(9899):1183–94.

    Article  PubMed  Google Scholar 

  84. Baker-Henningham H, Scott S, Jones K, Walker S. Reducing child conduct problems and promoting social skills in a middle-income country: cluster randomised controlled trial. Br J Psychiatry. 2012;201:101–8.

    Article  PubMed  PubMed Central  Google Scholar 

  85. Bhandari N, Mazumder S, Taneja S, Sommerfelt H, Strand TA. Effect of implementation of Integrated Management of Neonatal and Childhood Illness (IMNCI) programme on neonatal and infant mortality: cluster randomised controlled trial. BMJ. 2012;344:e1634.

    Article  PubMed  PubMed Central  Google Scholar 

  86. Bird C, Ame S, Albonico M, Bickle Q. Do shoes reduce hookworm infection in school-aged children on Pemba Island, Zanzibar? A pragmatic trial. Trans R Soc Trop Med Hyg. 2014;108(5):297–304.

    Article  PubMed  CAS  Google Scholar 

  87. Burnhams NH, London L, Laubscher R, Nel E, Parry C. Results of a cluster randomised controlled trial to reduce risky use of alcohol, alcohol-related HIV risks and improve help-seeking behaviour among safety and security employees in the Western Cape, South Africa. Subst Abuse Treat Prev Policy. 2015;10:18.

    Article  PubMed  PubMed Central  Google Scholar 

  88. Dave PV, Shah AN, Nimavat PB, et al. Direct observation of treatment provided by a family member as compared to non-family member among children with new tuberculosis: a pragmatic, non-inferiority, cluster-randomized trial in Gujarat, India. PLoS One. 2016;11(2):e0148488.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  89. Durovni B, Saraceni V, Moulton LH, et al. Effect of improved tuberculosis screening and isoniazid preventive therapy on incidence of tuberculosis and death in patients with HIV in clinics in Rio de Janeiro, Brazil: a stepped wedge, cluster-randomised trial. Lancet Infect Dis. 2013;13(10):852–8.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  90. Fottrell E, Azad K, Kuddus A, et al. The effect of increased coverage of participatory women's groups on neonatal mortality in Bangladesh: a cluster randomized trial. JAMA Pediatrics. 2013;167(9):816–25.

    Article  PubMed  PubMed Central  Google Scholar 

  91. Fylkesnes K, Sandoy IF, Jurgensen M, Chipimo PJ, Mwangala S, Michelo C. Strong effects of home-based voluntary HIV counselling and testing on acceptance and equity: a cluster randomised trial in Zambia. Soc Sci Med. 2013;86:9–16.

    Article  PubMed  Google Scholar 

  92. Gong J, Chen X, Li S. Efficacy of a community-based physical activity program KM2H2 for stroke and heart attack prevention among senior hypertensive patients: a cluster randomized controlled phase-II trial. PLoS One. 2015;10(10):e0139442.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  93. Jack SJ, Ou K, Chea M, et al. Effect of micronutrient sprinkles on reducing anemia a cluster-randomized effectiveness trial. Arch Pediatr Adoles Med. 2012;166(9):842–50.

    Article  Google Scholar 

  94. Lutge E, Lewin S, Volmink J. Economic support to improve tuberculosis treatment outcomes in South Africa: a qualitative process evaluation of a cluster randomized controlled trial. Trials. 2014;15:236.

    Article  PubMed  PubMed Central  Google Scholar 

  95. Mengistie B, Berhane Y, Worku A. Household water chlorination reduces incidence of diarrhea among under-five children in rural Ethiopia: a cluster randomized controlled trial. PLoS One. 2013;8(10):e77887.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  96. Olney DK, Pedehombga A, Ruel MT, Dillon A. A 2-year integrated agriculture and nutrition and health behavior change communication program targeted to women in Burkina Faso reduces anemia, wasting, and diarrhea in children 3-12.9 months of age at baseline: a cluster-randomized controlled trial. J Nutr. 2015;145(6):1317–24.

    Article  PubMed  CAS  Google Scholar 

  97. Ononge S, Campbell OM, Kaharuza F, Lewis JJ, Fielding K, Mirembe F. Effectiveness and safety of misoprostol distributed to antenatal women to prevent postpartum haemorrhage after child-births: a stepped-wedge cluster-randomized trial. BMC Pregnancy Childbirth. 2015;15:315.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  98. Patil SR, Arnold BF, Salvatore AL, et al. The effect of India's total sanitation campaign on defecation behaviors and child health in rural Madhya Pradesh: a cluster randomized controlled trial. PLoS Med. 2014;11(8):e1001709.

    Article  PubMed  PubMed Central  Google Scholar 

  99. Sangoro O, Turner E, Simfukwe E, Miller JE, Moore SJ. A cluster-randomized controlled trial to assess the effectiveness of using 15% DEET topical repellent with long-lasting insecticidal nets (LLINs) compared to a placebo lotion on malaria transmission. Malar J. 2014;13:324.

    Article  PubMed  PubMed Central  Google Scholar 

  100. Singla DR, Kumbakumba E, Aboud FE. Effects of a parenting intervention to address maternal psychological wellbeing and child development and growth in rural Uganda: a community-based, cluster randomised trial. Lancet Glob Health. 2015;3(8):e458–69.

    Article  PubMed  Google Scholar 

  101. Sorensen G, Pednekar M, Cordeira LS, et al. Effects of a worksite tobacco control intervention in India: the Mumbai worksite tobacco control study, a cluster-randomised trial. Tob Control. 2017;26(2):210–6.

    Article  PubMed  Google Scholar 

  102. Weiss SM, Zulu R, Jones DL, Redding CA, Cook R, Chitalu N. The Spear and Shield intervention to increase the availability and acceptability of voluntary medical male circumcision in Zambia: a cluster randomised controlled trial. Lancet HIV. 2015;2(5):e181–9.

    Article  PubMed  PubMed Central  Google Scholar 

  103. West PA, Protopopoff N, Wright A, et al. Indoor residual spraying in combination with insecticide-treated nets compared to insecticide-treated nets alone for protection against malaria: a cluster randomised trial in Tanzania. PLoS Med. 2014;11(4):e1001630.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  104. Young SD, Cumberland WG, Nianogo R, Menacho LA, Galea JT, Coates T. The HOPE social media intervention for global HIV prevention in Peru: a cluster randomised controlled trial. Lancet HIV. 2015;2(1):e27–32.

    Article  PubMed  Google Scholar 

  105. Yousafzai AK, Rasheed MA, Rizvi A, Armstrong R, Bhutta ZA. Effect of integrated responsive stimulation and nutrition interventions in the Lady Health Worker programme in Pakistan on child development, growth, and health outcomes: a cluster-randomised factorial effectiveness trial. Lancet. 2014;384(9950):1282–93.

    Article  PubMed  Google Scholar 

  106. Fairall L, Bachmann MO, Lombard C, et al. Task shifting of antiretroviral treatment from doctors to primary-care nurses in South Africa (STRETCH): a pragmatic, parallel, cluster-randomised trial. Lancet. 2012;380(9845):889–98.

    Article  PubMed  PubMed Central  Google Scholar 

  107. Gunawardena N, Kurotani K, Indrawansa S, Nonaka D, Mizoue T, Samarasinghe D. School-based intervention to enable school children to act as change agents on weight, physical activity and diet of their mothers: a cluster randomized controlled trial. Int J Behav Nutr Phys Act. 2016;13:45.

    Article  PubMed  PubMed Central  Google Scholar 

  108. Dusenbury L, Brannigan R, Falco M, Hansen WB. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003;18(2):237–56.

    Article  PubMed  Google Scholar 

  109. Bellg AJ, Borrelli B, Resnick B, et al. Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH Behavior Change Consortium. Health Psychol. 2004;23(5):443–51.

    Article  PubMed  Google Scholar 

  110. Gasparyan AY, Ayvazyan L, Kitas GD. Multidisciplinary bibliographic databases. J Korean Med Sci. 2013;28(9):1270–5.

    Article  PubMed  PubMed Central  Google Scholar 

  111. Hollis S, Campbell F. What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ. 1999;319(7211):670–4.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  112. Alshurafa M, Briel M, Akl EA, et al. Inconsistent definitions for intention-to-treat in relation to missing outcome data: systematic review of the methods literature. PLoS One. 2012;7(11):e49163.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  113. Ivers NM, Taljaard M, Dixon S, et al. Impact of CONSORT extension for cluster randomised trials on quality of reporting and study methodology: review of random sample of 300 trials, 2000-8. BMJ. 2011;343:d5886.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  114. Ibrahim S, Sidani S. Fidelity of intervention implementation: a review of instruments. Health. 2015;2015:1687.

    Article  Google Scholar 

  115. O'Donnell CL. Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K-12 curriculum intervention research. Rev Educ Res. 2008;78(1):33–84.

    Article  Google Scholar 

  116. Perez D, Van der Stuyft P, Zabala MC, Castro M, Lefevre P. A modified theoretical framework to assess implementation fidelity of adaptive public health interventions. Implementation Sci. 2016;11(1):91.

    Article  Google Scholar 

  117. Schinckus L, Van den Broucke S, Housiaux M, Diabetes Literacy C. Assessment of implementation fidelity in diabetes self-management education programs: a systematic review. Patient Educ Couns. 2014;96(1):13–21.

    Article  PubMed  Google Scholar 

  118. Hasson H, Blomberg S, Dunér A. Fidelity and moderating factors in complex interventions: a case study of a continuum of care program for frail elderly people in health and social care. Implementation Sci. 2012;7(23):1–11.

    Google Scholar 

  119. Morello RT, Barker AL, Ayton DR, et al. Implementation fidelity of a nurse-led falls prevention program in acute hospitals during the 6-PACK trial. BMC Health Serv Res. 2017;17(1):383.

    Article  PubMed  PubMed Central  Google Scholar 

  120. Duflo E, Glennerster R, Kremer M. Using randomization in development economics research: A toolkit. Handbook Develop Econ. 2007;4:3895–962.

    Article  Google Scholar 

  121. Abhijit V. Banerjee and Esther Duflo, Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty. New York: Public Affairs; 2011. pp. 303. ISBN: 978-1-58648-798-0.

  122. Eldridge S, Campbell M, Campbell M, et al. Revised Cochrane risk of bias tool for randomized trials (RoB 2.0): Additional considerations for cluster-randomized trials. In:Chandler J, McKenzie J, Boutron I, Welch V (editors). Cochrane Methods. Cochrane Database Syst Rev. 2016;10(Suppl 1). https://sites.google.com/site/riskofbiastool/welcome/rob-2-0-tool. Accessed 22 June 2018.

  123. Higgins JPT, Sterne JAC, Savović J, Page MJ, Hróbjartsson A, Boutron I, Reeves B, Eldridge S. A revised tool for assessing risk of bias in randomized trials In: Chandler J, McKenzie J, Boutron I, Welch V (editors). Cochrane Methods. Cochrane Database Syst Rev. 2016;10(Suppl 1). https://sites.google.com/site/riskofbiastool/welcome/rob-2-0-tool. Accessed 22 June 2018.

  124. Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348:g1687.

    Article  PubMed  Google Scholar 

Download references

Funding

IC-IMPACTS (the India-Canada Centre for Innovative Multidisciplinary Partnerships to Accelerate Community Transformation and Sustainability) provided funding for this study in the form of doctoral scholarships for MCP and NM. VR holds a CIHR-funded Research Chair in Applied Public Health (CPP-137901). MPS holds a Research Scholars – Junior 1 award from the Fonds de recherche du Québec (FRSQ). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Availability of data and materials

All data from this study, including literature searches and data extraction forms, are available from the corresponding author.

Author information

Authors and Affiliations

  1. Centre de Recherche du Centre Hospitalier de l’Université de Montréal (CRCHUM), 900, rue Saint-Denis, Pavillon R, Tour Saint-Antoine Porte S03.414, Montréal, Québec, H2X 0A9, Canada

    Myriam Cielo Pérez, Nanor Minoyan, Marie-Pierre Sylvestre & Mira Johri

  2. Département de médicine sociale et préventive, École de santé publique (ESPUM), Université de Montréal, 7101, avenue du Parc, 3e étage, Montréal, Québec, H3N 1X9, Canada

    Myriam Cielo Pérez, Nanor Minoyan & Marie-Pierre Sylvestre

  3. Institut de Recherche en Santé Publique Université de Montréal (IRSPUM), Pavillon 7101 Avenue du Parc, P.O. Box 6128, Centre-ville Station, Montréal, Québec, H3C 3J7, Canada

    Valéry Ridde

  4. Institut de Recherche Pour le Développement (IRD), Le Sextant 44, bd de Dunkerque, CS 90009 13572, Cedex 02, Marseille, France

    Valéry Ridde

  5. Département de gestion, d’évaluation, et de politique de santé, École de santé publique, Université de Montréal, 7101, avenue du Parc, 3e étage, Montréal, Québec, H3N 1X9, Canada

    Mira Johri

Authors
  1. Myriam Cielo Pérez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nanor Minoyan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Valéry Ridde
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marie-Pierre Sylvestre
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mira Johri
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

MJ and MCP conceived the study. MCP, NM and MJ performed the data abstraction and the data analysis. VR and MPS contributed to study design and interpretation of the results. MCP drafted the manuscript in collaboration with MJ, and all authors contributed to the critical revision and final approval of the manuscript. All authors had full access to all of the data (including statistical reports and tables) in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis. All authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. MCP is guarantor.

Corresponding author

Correspondence to Mira Johri.

Ethics declarations

Ethics approval and consent to participate

This research does not involve human subjects. It is exempted from research ethics board review as it relies exclusively on publicly available information for which there is no reasonable expectation of privacy.

Consent for publication

Not applicable.

Competing interests

The authors declare that they no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

PRISMA Checklist: Comparison of registered and published intervention fidelity assessment in cluster randomised trials of public health interventions in low- and middle-income countries: systematic review. (DOC 64 kb)

Additional file 2:

Full search strategy for MEDLINE. (DOCX 32 kb)

Additional file 3:

Articles excluded with reasons. (DOCX 38 kb)

Additional file 4:

Tool for assessing risk of bias(a). (DOCX 42 kb)

Additional file 5:

Characteristics of included studies. (DOCX 55 kb)

Additional file 6:

Classification of Public Health Interventions*. (DOCX 322 kb)

Additional file 7:

Risk of bias graph for individual studies. (DOCX 136 kb)

Additional file 8:

Subgroup analyses. (DOCX 19 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pérez, M.C., Minoyan, N., Ridde, V. et al. Comparison of registered and published intervention fidelity assessment in cluster randomised trials of public health interventions in low- and middle-income countries: systematic review. Trials 19, 410 (2018). https://doi.org/10.1186/s13063-018-2796-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13063-018-2796-z

Keywords

  • Cluster randomised trials
  • Intervention fidelity
  • Public health interventions
  • Process evaluation
  • Developing countries
  • Systematic review

Trials

ISSN: 1745-6215