Reporting of key methodological and ethical aspects of cluster trials in hemodialysis require improvement: a systematic review

Background The hemodialysis setting is suitable for trials that use cluster randomization, where intact groups of individuals are randomized. However, cluster randomized trials (CRTs) are complicated in their design, analysis, and reporting and can pose ethical challenges. We reviewed CRTs in the hemodialysis setting with respect to reporting of key methodological and ethical issues. Methods We conducted a systematic review of CRTs in the hemodialysis setting, published in English, between 2000 and 2019, and indexed in MEDLINE or Embase. Two reviewers extracted data, and study results were summarized using descriptive statistics. Results We identified 26 completed CRTs and five study protocols of CRTs. These studies randomized hemodialysis centers (n = 17, 55%), hemodialysis shifts (n = 12, 39%), healthcare providers (n = 1, 3%), and nephrology units (n = 1, 3%). Trials included a median of 28 clusters with a median cluster size of 20 patients. Justification for using a clustered design was provided by 15 trials (48%). Methods that accounted for clustering were used during sample size calculation in 14 (45%), during analyses in 22 (71%), and during both sample size calculation and analyses in 13 trials (42%). Among all CRTs, 26 (84%) reported receiving research ethics committee approval; patient consent was reported in 22 trials: 10 (32%) reported the method of consent for trial participation and 12 (39%) reported no details about how consent was obtained or its purpose. Four trials (13%) reported receiving waivers of consent, and the remaining 5 (16%) provided no or unclear information about the consent process. Conclusion There is an opportunity to improve the conduct and reporting of essential methodological and ethical issues in future CRTs in hemodialysis. Review Registration We conducted this systematic review using a pre-specified protocol that was not registered.


Introduction
Patients on hemodialysis are often excluded from clinical trials, and many trials in the hemodialysis setting suffer from poor recruitment, inadequate sample sizes, and poor adherence to allocated treatment and treatment contamination [1][2][3][4][5]. Cluster randomized trials (CRTs) randomize intact groups of individuals (rather than independent individuals) to different arms. This design can offer a logistically convenient method to produce highquality evidence, can be effective in avoiding treatment contamination, and may be better received by participants and healthcare staff when delivered to a group of individuals rather than select patients. The CRT is an attractive design in the hemodialysis setting, where interventions are often delivered at the center-level and where staff follow the same protocol for patients under their care.
Cluster randomization, however, introduces methodological issues that need to be addressed during the design and analysis stages [6,7]. First, it may not be possible to identify and recruit participants until after the cluster has been randomized. This increases the risk of selection bias because knowledge of the allocated arm can influence both the identification of potential participants and their decisions to participate. Second, because outcomes are usually correlated within clusters, CRTs are statistically less efficient than individual-level randomized trials. As such, the CONSORT Statement for Cluster Randomized Trials requires that studies report how clustering was considered in both sample size calculation and analysis. Failing to account for clustering in the sample size calculation implies that the study may not have adequate power to detect meaningful differences between the groups, while failing to account for clustering in the analysis implies that standard errors of treatment effects will be under-estimated, increasing the risk of spurious statistical significance [6][7][8][9].
The CRT design also raises complex ethical issues. The Ottawa Statement on the Ethical Design and Conduct of Cluster Randomized Trials offers ethical guidance, providing 15 recommendations for those who design, conduct, and review CRTs [10][11][12][13][14]. For example, ethical issues that may challenge researchers including the following: When is a study considered research? Who is the research subject? And from whom, how, and when must informed consent be obtained? A summary of the Ottawa Statement recommendations and applicability of these recommendation for CRTs conducted in the hemodialysis setting is provided in Additional file 1: Appendix 1.
In the present study, we conducted a descriptive analysis of how CRTs in hemodialysis report key methodological (with regards to accounting for clustering effects and reporting of intra-class correlation coefficient) and ethical issues (with regards to the elements highlighted in the Ottawa Statement). This review will serve as a foundational step in a multi-year initiative that seeks to develop recommendations for the ethical design, conduct, and reporting of CRTs in the hemodialysis setting.

Protocol and registration
We conducted this systematic review using a prespecified protocol and reported our results according to published guidelines (PRISMA Checklist: Additional file 1: Appendix 2) [15].

Studies eligible for review
We did not set any limits on country of study and included published primary reports of CRTs or study protocols of CRTs with an unpublished primary report. We aimed to include English-language reports published between January 2000 and November 2019 that involved (1) patients on in-center hemodialysis or( 2) patients on in-center hemodialysis as a subgroup in a larger study of non-in-center hemodialysis patients. When we found a study protocol of a CRT with an identified completed trial, we used the protocol to supplement any missing information from the final published report. Other reports such as secondary analyses, conference abstracts, and pilot or feasibility CRTs were excluded. We excluded feasibility and pilot trials because they have different methodological [16] and ethical considerations than full scale CRTs.

Information sources
We implemented a search syntax on November 30, 2019, to identify published reports in MEDLINE and Embase.

Search
Our search strategy combined two published search filters designed to identify publications related to CRT [17] and dialysis [18] studies (Additional file 1: Appendix 3). Two reviewers (AAA and KC) screened titles and abstracts of articles. AAA manually searched for additional articles in bibliographies of all included articles, list of articles that cited the included studies in Google Scholar, and "Similar articles" feature in PubMed. The complete list of included studies was also reviewed by an expert in the field (AXG) to capture additional studies that may have been missed.

Study selection
We retrieved the full text of any article considered potentially relevant by any reviewer. Full-text articles were assessed for study eligibility by two reviewers (AAA and KC), with disagreements resolved through discussion. Agreement between the two reviewers was evaluated using the Kappa statistic [19].

Data collection process
We utilized a data abstraction form that was pilot tested on three studies by three reviewers (AAA, KC, and CEG). Thereafter, two reviewers (AAA and either KC or CEG) independently extracted data from each manuscript. Extracted details on trials considering the effect of clustering during sample size estimation and analysis were completed by AAA and either MT or SND. After each set of three studies, data extractions were compared within the pair and disagreements resolved by consensus. Details of extracted data are highlighted in Additional file 1: Appendix 4. We extracted data on study characteristics, methodological characteristics, data collection method, justification for using a CRT design, type of intervention, information regarding research ethics committee review, gatekeepers (i.e., an individual or body that represents the interests of cluster members, clusters, or organizations [20]), informed consent procedures, and any information about harm-benefit assessment or protection of vulnerable populations. Kidney disease disproportionally affects individuals traditionally considered vulnerable (e.g., patients with dementia). We defined vulnerable participants as any research participants who "may have increased likelihood of being wronged or of incurring additional harm," as per the CIOMS international ethics guidelines [21]. This includes persons who have "impairments in decisional capacity, education, resources, strength, or other attributes needed to protect their own interests [21]". We coded the presence of any vulnerable participants as clearly present, potentially present or unclear, and clearly absent or not relevant. If a vulnerability was clearly or potentially present, we looked for any reporting of additional protections provided by the authors.

Analysis
We summarized results using frequencies for categorical variables and medians with interquartile ranges for continuous variables. Given the small number of included studies, we did not test changes in reporting over time nor association between reporting of ethical elements and study characteristics. For all our analyses, we used R (Version 3.6.2) [22].

Characteristics of included studies
The study flow diagram is presented in Fig. 1. We screened 777 citations and retrieved 29 full-text articles Fig. 1 Flow diagram of study selection. **Other: One manuscript described the statistical plan for a main publication not related to cluster randomized trials, two described a program of research not related to the target population, and two were duplicate records not previously removed. Abbreviation: RCT, randomized controlled trial; CRT, cluster randomized trial to assess eligibility. We identified another seven articles by reviewing citation links (n total = 36). We had almost perfect between-reviewer agreement on which studies met the criteria for review (kappa statistic 0.96, 95% confidence interval: 0.91 to 1.00). Five articles were excluded after full-text review [23][24][25][26][27]. Thus, 31 articles were included in this review: 26 completed studies and 5 study protocols .

Reporting of methodological characteristics
The median (25th, 75th percentile) number of clusters included per trial was 28 (12,43), and all trials used 1:1 randomization. One trial (3%) had one cluster per arm, and six trials (19%) had fewer than the minimum recommendation of four clusters per arm [7,8]. The median number of participants per trial was 228 (120, 1723). All trials included patients (as opposed to providers alone) as the research participants with a median number of 20 (8,32) patients per cluster.
One study (3%) reported the intra-class correlation coefficient (ICC) for their primary outcome. Table 3 describes whether and how clustering was accounted for during sample size estimation and analysis. Fourteen trials (45%) accounted for clustering during sample size estimation for the primary outcome, three (10%) did not account for clustering, two (6%) accounted for clustering but using a different outcome measure than the primary outcome, one (3%) was unclear, and 11 (35%) did not report a sample size or power estimate. At the analysis stage, 22 trials (71%) accounted for clustering using either an individuallevel analysis adjusting for clustering or using a clusterlevel summary method. The remaining nine trials (29%) either did not account for clustering in their primary analysis or it was unclear if clustering was accounted for in the analysis. A total of 13 trials (42%) accounted for clustering in both the sample size calculation and analysis.

Reporting of justification for cluster randomization
Of all 31 trials, 15 trials (48%) reported a justification for using a cluster randomized design (Table 3). Thirteen trials (42%) reported using a CRT design to avoid contamination and two trials (6%) reported using a CRT design to avoid contamination and for logistical/ administrative convenience. Table 4 lists the types of intervention used in each arm of included trials. The most common type of study intervention was health promotion or an educational intervention (n = 22 trials; 71%) for which patients were the intended recipients. Six trials (19%) examined a direct patient therapeutic intervention-for example, intradialytic resistance training or antimicrobial barrier caps for central venous catheters. Among all trials, the intervention was necessarily administered at the cluster-level (e.g., education of providers) for 18 trials (58%). In the control arm, 23 trials (74%) utilized "usual care," four (13%) used some form of augmented care (usual care plus some minimal elements of active intervention), three (10%) used an active control, and one (3%) used an attention-placebo. Four trials (13%) utilized interventions that included an educational or quality improvement component targeting health professionals (e.g., transplant education and engagement activities). Both prevalent and incident patients on hemodialysis were included in 22 trials (71%), eight trials (26%) included only prevalent patients, and one trial (3%) included only incident patients on hemodialysis.

Data collection procedures
Data collection procedures in the intervention and control arm were similar for most trials (Table 4). In the intervention arm, 30 (97%) trials used local routinely collected data (e.g., medical charts or electronic medical records) as the primary source for data collection. Eleven trials (35%) used clinical registry data and 24 (77%) supplemented routinely collected data with additional sources: self-administered questionnaires (n = 18; 58%), interviewer administered questionnaires (n = 9; 29%), specimen collection or physical examination not required for usual patient care (n = 4; 13%), as well as active data collection (n = 5; 16%), for example, using case report forms.

Gatekeepers
Five trials (16%) reported that a gatekeeper provided permission for clusters to participate in the study (Table 5). For the remaining trials (84%), no information about gatekeepers was provided.

Reporting of research ethics review
We found that 26 trials (84%) reported research ethics committee approval, one (3%) reported that the study was exempt from review, and four (13%) did not report whether the study was reviewed by a research ethics committee ( Table 5).

Reporting of consent procedures
One trial (3%) reported they received an exemption from ethics review, three (10%) received a waiver of consent from the research ethics committee (see Additional file 1: Appendix 5), 22 trials (71%) reported obtaining consent from patients, and five (16%) trials either did not discuss the consent process or it was unclear if patients provided informed consent. For the 22 trials (71%) that reported obtaining consent from patients, written or verbal informed consent was reported in 10 trials (45%) for the study intervention and eight (36%) for data collection; the remaining trials provided no details about the method of consent for study intervention and/or for data collection (Table 4). Among the remaining 27 trials that did not receive an exemption from ethics review or had a waiver of consent, the timing of consent took place before randomization for seven trials (26%), after randomization for 10 (37%), and was unclear for the remaining 10 trials (37%).
The ability for participants to opt out of the data collection was reported in seven trials (23%); three trials (10%) reported patients could not opt out of data collection, and ability to opt out was unclear for the remaining 21 trials (68%) ( Table 5).

Assessment of benefit-harm and protections for vulnerable groups
Vulnerable participants were clearly present in 6 trials (20%); in another 24 (77%), vulnerable participants were considered to be potentially present or their presence could not be  Logistical or administrative convenience 2 (6%) One study presented power calculation, but it was a post hoc power analysis ¥ This may have included using an inappropriate method for the proposed primary outcome, or the study accounted for clustering but not based on the primary outcome measure (e.g., they assumed a continuous outcome, but the primary endpoint was a proportion) ₱ One study accounted for repeated events within patients but did not report accounting for within-cluster correlation; another study reported using a generalized linear mixed model but did not specify whether they accounted for the effect of the cluster as random effect clearly ruled out ( Table 5). None of the trials reported additional protections for vulnerable patients.

Discussion
The hemodialysis population is suitable for the CRT design, especially for interventions that are implemented at the center-level; in our review, approximately 60% of trials utilized an intervention that was necessarily administered at the cluster-level. This review presents a descriptive analysis of the reporting of key methodological and ethical characteristics of CRTs involving hemodialysis patients. Guidance on the reporting of CRTs is provided in the Table 4 Summary of results for type(s) of interventions, data collection procedures, reporting of participant consent procedures for study interventions and data collection, timing of any participant consent, and whether participants can opt out of the intervention or data collection Direct patient therapeutic intervention (e.g., intradialytic resistance training, antimicrobial barrier caps for catheters, etc.) Types of Data collection procedures ¥ N total = 31 N total = 31 Routinely collected outcomes extracted locally from existing patient medical records (physical charts or electronic records)

(97%) (97%)
Data query from clinical data registry or other central source of routinely collected data (e.g., administrative data)

(35%) (35%)
Specimen collection or physical examination that were not required for usual patient care 4 (13%) 4 (13%) Interviewer-administered patient questionnaires done face-to-face or by telephone that were not required for usual patient care 9 (29%) 9 (29%) The responses to these questions were not mutually exclusive Active data collection, including using case report form € Surrogate decision-maker educational intervention in the intervention arm; audit feedback from previous year in the control arm **These questions were not applicable when the comparator arm was usual care CONSORT extension for CRTs, while the Ottawa Statement is currently the only guidance document specific to the ethical design and conduct of CRTs in health research [8,14]. While several studies were published prior to the dissemination of the CONSORT, the Ottawa Statement or both, the interpretation of our results would not change had we presented our results based on the period pre-and post-publication of these statements. We found that cluster randomized trials in hemodialysis have poor methodological quality and sub-optimally report ethical considerations around this design. While many of the identified issues are not unique to the hemodialysis setting, we consider three issues that require special attention: (1) taking clustering into account at the sample size estimation and analysis stages, (2) methodological and contamination issues around designs that randomize shifts within hemodialysis centers, and (3) reporting on how the rights of vulnerable participants are protected.
First, patients on hemodialysis within the same center have similar characteristics compared to patients from other centers. For example, small satellite hemodialysis centers might have patients that are medically stable compared to large academic centers that might treat sicker patients requiring close medical monitoring. It is concerning that more than half of included trials did not report a method that appropriately accounts for withincluster correlation when estimating sample size and more than a quarter of trials did not account for clustering in the analysis, putting the study results at an increased risk of spurious statistical significance [7][8][9]14]. Adjusting for clustering is especially important in this setting because there is generally high practice variation between hemodialysis centers and low variation with-in centers [59][60][61], factors that increase the ICC [62].
Currently, there is limited information in the literature to inform estimates of the ICC for outcomes of patients on hemodialysis; thus, researchers in hemodialysis must rely on estimates from other disciplines or historic data. As such, it is important for completed trials to report the observed ICC or design effect estimates for their outcomes so that the community can begin to build a repository that might help in the design of future trials. In our review, only one trial reported an ICC [47].
Second, a common experimental design was to randomize shifts within hemodialysis centers (e.g., Mon, Wed, Fri versus Tues, Thu, Sat). This type of randomization requires additional considerations in design and conduct. For example, the same healthcare staff will care for patients dialyzing in a single center in both arms of the trial. Contamination of the two arms of the trial can still occur if staff observe better patient outcomes in one arm and then begin to implement the treatment in clusters (i.e., shifts) in the other arm. This type of design also requires additional considerations in the analysis because clustering can occur at two levels, i.e., center and shift. It was beyond this review to assess whether authors reported the appropriate analyses accounting for this type of experimental design.
Third, authors should report how the rights of vulnerable participants are protected, especially those that have limited health literacy or may not be capable to provide informed consent. When including these subgroups, it raises ethical concerns about the extent to which these participants are truly informed. There are no clear standards for "how much" understanding is adequate [63]. Additionally, lower education levels, lower health literacy, and a participant's primary language are all associated with poor comprehension of the informed consent process [64]. These characteristics are particularly important in the hemodialysis setting, where vulnerable participants are overrepresented [65][66][67][68]. In general, trials in this setting were small with both a limited number of clusters and patients within clusters. One trial randomized only one cluster to each arm of the trial and a fifth of reviewed trials had four or fewer clusters [43,56]. Randomizing two clusters effectively precludes any inferences about the intervention because it is impossible to disentangle natural variation between clusters from the effect of the intervention [69]. While some have suggested that parallel arm CRTs should have at least four clusters per study arm [7,8], with such a small number of clusters, the study may be severely under-powered, parametric statistical tests (e.g., t tests) may not meet the assumption of normality, and there is a high risk of baseline imbalances between trial arms that might complicate the interpretation of the trial results [70].
There is room for improved reporting of consent procedures. When consent is required, study authors ought to report adequate details to assess what consent was for (e.g., enrollment, receiving the interventions, data collection), as well as from whom (e.g., patients, providers, etc.), when (before or after randomization), and how (e.g., written, verbal) consent was obtained [8,14]. The timing of informed consent was either not reported or took place after randomization for 20 trials. Postrandomization consent, especially when the study is unblinded, is a key risk of bias that can introduce selection bias through differential recruitment [8]. When applicable, researchers must justify how their study meets the criteria for a waiver or alteration of informed consent as outlined by national regulations or international guidelines [21,71,72].
Our study has several limitations. We were unable to examine changes in quality of reporting over time, or factors associated with better reporting due to the small number of trials. When a study protocol was published for one of the completed trials, we used both references to complete study extraction; however, we did not have access to the original research ethics submissions or nonpeer-reviewed study protocols, did not follow-up with study authors, and did not conduct a search of any trial registries or Green Open Access options (e.g., Research-Gate). Thus, our results are based exclusively on what was reported in peer-reviewed published articles; for example, we are aware of other ongoing CRTs not included here because no study protocol or a primary report was available at the time of our search [73,74].
Our study also has several strengths. We utilized an abstraction tool that has been developed and refined over several studies [75][76][77][78]. It is unlikely that a substantial number of relevant primary trials were missed, as we combined two validated search strategies supplemented with an extensive manual search of reference resources [17,18]. To reduce the risk of misclassification of trial characteristics and reporting practices, we used consensus between two reviewers who independently extracted information from published articles.

Conclusion
There is suboptimal conduct and reporting of methodological issues of CRTs in the hemodialysis setting and incomplete reporting of key ethical issues. The Ottawa Statement on the Ethical Design and Conduct of Cluster Randomized Trials provides specific recommendations for CRTs, but did not consider unique characteristics of the hemodialysis setting [14]. This systematic review was conducted as a first step to describe key study design characteristics and document reporting of ethical practices in CRTs in the hemodialysis setting. Our future work builds on the information from this review to explore the views/perceptions of researchers and patients with regard to the ethical issues for CRTs in the hemodialysis setting.