Defining key design elements of registry-based randomised controlled trials: a scoping review

Abstract

Background

Traditional randomised controlled trials remain the gold standard for improving clinical care but they do have their limitations, including their associated high costs, high failure rate and limited external validity. An alternative methodology is the newly defined, prospective, registry-based randomised controlled trial (RRCT), where treatment and outcome data is collected in an existing registry. This scoping review explores the current literature regarding RRCTs to help identify the key design elements of RRCTs and the characteristics of clinical registries on which they are reliant on.

Methods

A scoping review methodology conducted in accordance with the Joanna Briggs Institute guidelines was performed. Four databases were searched for articles published from inception to June 2018: Medline; Embase; the Cumulative Index to Nursing and Allied Health Literature and; Scopus. The search strategy included MeSH and text words related to RRCT.

Results

We identified 2369 articles of which 75 were selected for full-text screening. Of these, only 17 articles satisfied our inclusion criteria. All studies were published between 1996 and 2017 and all were investigator-initiated. Study designs were mainly multi-site comparative/effectiveness studies incorporating the use of disease registries (n = 8), procedure registries (n = 8) and a health services registry (n = 1). The low cost, reduced administrative burden and enhanced external validity of RRCTs make them an attractive research methodology which can be used to address questions of public health importance. We identified that that there are variable definitions of what constituted a RRCT and that issues related to ethical conduct and data integrity, completeness, timeliness, validation and endpoint adjudication need to be carefully addressed.

Conclusion

RRCTs potentially have an important role to play in informing best clinical practice and health policy. There are a number of issues that need to be addressed to optimise the utility of this approach, including establishing universally accepted criteria for the definition of a RRCT.

Background

Randomised controlled trials (RCTs) are considered the gold standard for evaluating the effectiveness of medical interventions [1]. Despite their privileged status in the hierarchy of clinical evidence, the limitations of RCTs need to be acknowledged [2]. Traditional RCTs are complex and expensive to perform, they enrol a highly selected population, a high proportion fail to meet recruitment goals and they have limited external validity [3, 4], making it difficult to apply any learnings to the real-world patient population. Researchers are, therefore, turning their attention to alternative research methodologies in pursuit of more affordable and generalisable, high-quality clinical evidence [3, 5, 6].

One such alternative methodology is the registry-based randomised controlled trial (RRCT). RRCTs are generally considered under the broader umbrella of pragmatic trials [4, 7], although the definition of RRCTs is variable depending on how patients are recruited and whether clinical registries or routinely collected data (RCD) is used to capture outcomes [2, 8, 9]. One potential definition for a RRCT by Li et al. is a trial where eligible patients are identified and recruited from the registry, the patients’ existing baseline medical history is recorded in the registry and data related to the intervention and the outcomes are captured in the registry [2]. A randomisation module may also be incorporated within the registry [2, 10, 11].

Clinical registries can be disease, health services or product specific [12]. They collect clinical information for a specific area of interest and given the level of clinical detail that they capture, can support a variety of research questions. In contrast, RCD such as electronic health records (EHR) and administrative/claims data, support clinical care and administrative or billing activities and, as such, there are limits to how this data can be used to support research. RCD may lack detailed information on clinical indications, patient characteristics, type of treatment, and outcome events and may also be less structured (e.g. free text) [13].

RRCTs are best suited for testing hypotheses involving pharmaceutical interventions, devices, and any other intervention already available in the real-world clinical setting but where there is variable implementation or uncertainty regarding optimal treatment combination, sequencing or duration, or where multiple standard-of-care options are available. Hard endpoints, such as overall survival, are preferred. Given that the adverse events associated with the intervention under study are already well-established, less detail regarding adverse events needs to be captured [14, 15]. The benefits of a RRCT to assess the comparative effectiveness of treatments in a real-world setting have been demonstrated through the pioneering and landmark Thrombus Aspiration during ST-Segment-Elevation Myocardial Infarction (TASTE) [16] and the SAFE-PCI for Women studies [17].

Despite these benefits, there are several challenges in the design and implementation of RRCTs that need to be addressed, including: data quality, regulatory and ethical issues, adjudication of study outcomes, choice of methodology and study design, and operational challenges emanating from the type of clinical registry and RCD being used [2, 8, 18]. The diversity of elements that may underpin a RRCT study adds further complexity to this subject matter. As the RRCT is a new and evolving clinical trial methodology, the need to reach consensus on a set of elements, that should comprise and define RRCTs, is of high importance. To date, one review of RRCTs has been conducted but the identification and recruitment of eligible patients from a clinical registry was not a prerequisite [8]. As such, it is possible that the review included post-trial extension of RCTs to assess long-term outcomes using clinical registries.

The aims of this scoping review are to describe the literature covering RCTs embedded within clinical registries, and to identify the key design elements of RRCTs and the characteristics of clinical registries that enabled them to support RRCTs. We elected to use the RRCT definition of Li et al. [2] as we consider clinical registries as playing a pivotal role in the conduct of RRCTs. We acknowledge the variability of data quality and completeness across registries. However, clinical registries are more likely to contain greater depth of clinical information and exhibit a greater level of data validation than RCD [19]. Supplementary RCD may help strengthen the internal validity of RRCTs by reducing loss to follow-up and non-random missing data. As the RRCT methodology is still in its infancy, it is opportune that an attempt is made to better elucidate the essential elements required for a study to be considered a RRCT, and the characteristics that clinical registries must have in order to adequately support RRCTs.

Methods

The scoping review methodology was selected to map the literature in this emerging area. The scoping review was conducted in accordance with Joanna Briggs Institute methodology guidelines for conducting scoping studies [20] and reported using the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) [21]. The methodology draws on the Arksey and O’Malley framework [22]: (1) identifying the research question; (2) identifying relevant studies; (3) study selection; (4) charting the data; and (5) collating, summarising and reporting the results. The objectives, inclusion criteria and methods for this scoping review were specified in advance and documented in an unpublished protocol which can be made available upon request. The protocol was strictly followed with one minor deviation from the search strategy, in which articles were also identified from a RRCT review article [8].

Data sources and searches

Four databases were searched for articles published from inception to June 2018: Medline, Embase; the Cumulative Index to Nursing and Allied Health Literature (CINHAL); and Scopus. Articles were also screened from a review on RRCTs [8]. The reference list of all identified articles was searched for additional studies (hand search). The search strategy was conducted with the assistance of a librarian. The search strategy included MeSH and text words related to RRCT (Appendix 1).

Study selection

Inclusion and exclusion criteria

Articles were included if they met all of the following criteria: (1) limited to clinical registry trials that involved selection of participants from a registry or simultaneously enrolled participants into the registry and a trial; (2) randomisation of participants to an intervention or a control group; and (3) collection of at least one outcome measure from a clinical registry. Articles were excluded if they met any of the following criteria: (1) RRCT protocols; (2) registries of clinical trials; (3) trials using electronic health records and administrative databases to select participants; (4) non-randomised registry trials; (5) observational studies/retrospective cohort studies using registries; (6) registry-based follow up of RCT studies; (7) studies that did not derive any outcomes from a clinical registry and (8) published in languages other than English.

The search results from each database were imported into Endnote X8 and duplicates were excluded. Two authors (BK and KP) independently screened the titles and abstracts for relevance and then assessed the eligibility of the full-text articles. Discrepancies between authors were discussed between them and if they remained unresolved, a third author made the final decision.

Data extraction and synthesis

A standardised data charting form to record key information was created based on the protocol. The following information was extracted from the articles by two authors (BK and KP) independently: authors; year of publication; country; study design; aim; population; sample size; registry name; randomisation; intervention; trial duration and follow-up; and outcomes. Additional information pertaining to the clinical registries was extracted: purpose; time-period coverage; population coverage; consent; funding; variables; validity and reliability. Descriptive information of studies was reported. A narrative synthesis of the findings from the studies was conducted, with a focus on summarising the key design elements of a RRCT (recruitment, randomisation and outcomes) and characteristics of clinical registries (operating infrastructure and data quality) that enable them to support a RRCT.

Results

Study selection

We identified 2369 articles from four databases and one review article (Fig. 1). Following title and abstract screening, we excluded 2294 articles, leaving 75 articles for full-text screening. Following full-text screening, we excluded 59 articles of which: 21 were non-randomised registry trials; 12 were protocols; nine involved recruitment of participants from either administrative databases or population registries; six were observational studies; four did not collect any outcome from a registry; three were a registry-based follow-up RCT; two were a review; one was a RCT that did not involve a registry; and one we were unable to obtain full text. The remaining 16 articles satisfied the review inclusion criteria. An additional article was included via hand search. We included 17 articles in our synthesis.

Fig. 1

Flow diagram for retrieval of articles

Study characteristics

The characteristics of the 17 studies that satisfied the inclusion criteria of our review are described in Table 1. All studies were published between 1996 and 2017. Studies were predominantly conducted in the USA (n = 8), followed by Sweden (n = 4), Denmark (n = 2), Australia (n = 1), Italy (n = 1) and The Netherlands (n = 1). All studies were investigator-initiated, supported by a combination of research grants and cooperative agreements. In four studies, investigators also received unrestricted grants from commercial entities [6, 16, 23, 24]. Study designs were predominantly multi-site comparative/effectiveness studies, incorporating the use of disease registries (n = 8), procedure registries (n = 8) and a health services registry (i.e. ICU) (n = 1). Two studies were a registry-based follow-up of RRCT studies in percutaneous intervention and breast cancer conducted a year [24] and 12 years earlier [25], respectively. The total sample size across all studies was 59,330. The sample size per study ranged from 112 to 10,175. Study populations included patients with heart conditions (n = 8), unvaccinated children (n = 6), cancer patients (n = 2) and patients in intensive care units (ICUs) (n = 1). Interventions comprised patient reminders (n = 7), surgical procedures (n = 4), non-surgical procedures (n = 2), drug treatments (n = 2) and performance feedback (n = 2).

Table 1 Registry-based randomised controlled trial (RRCT) characteristics

RRCT study features

Recruitment

Ten studies used clinical registries to identify eligible participants for enrolment in RRCTs. Of these, six studies identified eligible children from immunisation registries [26,27,28,29,30,31]; two studies identified eligible participants from an Australian state-based cancer registry [32], and a registry of percutaneous aortic valve replacement procedures (REPLACE) [33], respectively; and two studies identified potential ICUs and hospitals from the Dutch National Intensive Care Evaluation (NICE) registry [34] and the National Cardiovascular Data Registry (NCDR) ACTION Registry® GWTG^TM [35], respectively. Seven studies simultaneously enrolled participants in the clinical registries and a RRCT. That is, participants who were enrolled in the clinical registries were also prospectively screened for RRCT eligibility. Of these, four studies utilised the Swedish Web System for Enhancement and Development of Evidence-based Care in Heart Disease Evaluated According to Recommended Therapy (SWEDEHEART) registry [6, 16, 24, 36]. Other registries included the National Cardiovascular Data Registry CathPCI [17], the Western Denmark Heart Registry [23] and the Danish Breast Cancer Cooperate Group (DBCG) registry [25].

Randomisation

Randomisation was commonly performed at the individual level, except for two studies where randomisation occurred at the hospital [35] and department (ICU) level [34]. Eight studies used a computer-generated code or random number for randomisation of participants into arms [23, 26,27,28, 30, 32,33,34]. In five studies, randomisation was performed within the clinical registry using an online randomisation module [6, 16, 17, 24, 36]. Four studies did not explicitly describe their randomisation processes, merely noting that participants were randomised [25, 29, 31, 35].

Interventions

Studies commonly involved two arms, except for two studies which consisted of three [31] and four arms [30], respectively. The interventions were comprised of clinical and non-clinical interventions. Clinical intervention studies included four surgical procedures [16, 17, 23, 24], two non-surgical procedures [33, 36] and two drug treatments [6, 25]. Surgical and non-surgical procedure studies investigated procedures/devices where the short- and long-term efficacy appeared to be uncertain in sub-groups of the population. For example, Frobert and Lagerqvist [16] and Lagerqvist and Frobert [24] examined the effect of thrombus aspiration on mortality within 30 days and 1 year among participants with ST-segment-elevation myocardial infarction. Rao and Hess [17] compared the impact of radial with femoral artery access on bleeding/vascular complications within 72 h of the procedure among women undergoing percutaneous coronary intervention (PCI). Similarly, the pharmacological studies investigated drug treatments where the relative short- and long-term efficacy were unclear. Erlinge and Omerovic [6] compared the effect of bivalirudin versus heparin monotherapy in reducing mortality within 180 days among participants with acute coronary syndrome treated with PCI. Kristensen and Ejlertsen [25] investigated the long-term impact of adjuvant tamoxifen and local radiotherapy versus local radiotherapy alone on femoral fractures among postmenopausal women with breast cancer.

Non-clinical studies included seven studies where the interventions were patient reminders [26,27,28,29,30,31,32] and two performance feedback surveys [34, 35]. Patient-reminder interventions generally compared various modes of reminders (e.g. letter, telephone) with standard/no reminder (control group) on immunisation rates. The performance feedback studies compared targeted/activated feedback with standard feedback/benchmark reports on patient outcomes.

Outcomes and loss to follow-up

Primary outcomes in clinical intervention studies focussed on well-defined clinical endpoints: all-cause mortality at 30 days [16] and within 1 year [24, 36]; a composite of death from any cause, myocardial infarction, or major bleeding during 180 days of follow-up [6]; a composite of cardiac death, myocardial infarction, or target-lesion revascularisation within 1 year [23]; bleeding/vascular complications requiring intervention occurring within 72 h of the procedure/hospital discharge [17]; incidence of acute kidney injury occurring within 72 h after the procedure [33]; and occurrence of fractures [25].

Secondary outcomes in clinical intervention studies included: rehospitalisation [16, 24, 36]; target-vessel/lesion-revascularisation [16, 23, 24]; and complications and length of stay [16]. Primary outcomes in non-clinical interventions were: immunisation doses as recorded in the clinical registry [26,27,28,29, 31]; improvement in the overall composite of all metrics (i.e. acute measures, discharge, excess dosing and reperfusion measures) [35]; ICU length of stay [34]; and response rate [32].

Primary outcomes were predominantly collected from the clinical registry. In some studies, additional outcomes were collected via data linkage with other registries and/or administrative datasets [6, 16, 23,24,25, 31, 36] and case report forms [33].

Trial duration across all studies ranged from 2 months to 2 years and 9 months and follow-up period ranged from 72 h to 12 years. Loss to follow-up for clinical intervention studies were minimal: four studies had no patient loss to follow-up [16, 24, 25, 33]; three had < 2% loss to follow-up [6, 17, 23]; and one had 6% loss to follow-up [36]. For non-clinical intervention studies, loss to follow-up was rarely reported or was not applicable to the study design.

Only three studies provided any commentary on the cost-effectiveness of their study [17, 28, 30] Two of the studies pertained to vaccination reminders and associated labour costs [28, 30] whilst the third study provided a cursory cost comparison of a cardiology intervention RRCT against the potential cost of a similar conventional RCT [17].

Registry features

Table 2 describes the characteristics of the clinical registries. The studies involved 13 clinical registries: disease registries (n = 6); procedure(s) registries (n = 6); and a health services registry (n = 1). Several studies utilised the same clinical registries: SWEDEHEART registry (n = 4) [6, 16, 24, 36]; the Michigan Care Improvement Registry (n = 2) [27, 28]; the NCDR (n = 2) [17, 35]. The primary purpose of all clinical registries was to monitor and improve patient care. Clinical registries covered a wide range of populations: children; patients with myocardial infarction; and patients with cancer. Population coverage ranged from one hospital in Italy [33] to 86% of all ICUs in The Netherlands [34].

Table 2 Characteristics of clinical registries

Period coverage was reported in seven clinical registries. Most of the clinical registries were established in the 1990s, with the earliest registry, the DBCG in Denmark developed in 1977. The most recently established registry was the SWEDEHEART in Sweden which was developed in 2009.

Variables collected in the clinical registries were dependent on the type of clinical registry, with the most common variables collected being patient demographics, medical history and clinical outcomes. Data quality was briefly reported in seven clinical registries. The approach to data-quality assurance varied across the studies and included: validation rules (e.g. out-of-range values); random review of data entered in the clinical registry with hospital records; and data record completeness. Data validity was noted in 10 studies and the method of achievement varied from study to study and involved fully automated processes utilising algorithms or manual spot-checking of registry data against the medical record.

Funding sources for clinical registries were not disclosed, except for three clinical registries. The NCDR ACTION Registry® GWTG^TM, SWEDEHEART and the NCDR CathPCI were supported by multiple sources of funding including government agencies, non-profit/charitable-fundraising organisations and pharmaceutical companies. The approaches to patient consent (e.g. waiver of consent, opt in or opt out) to have data included in the clinical registries were not described.

Discussion

Our scoping review yielded a total of 17 studies. Of these, eight were in cardiology, six in immunisation, two in oncology and one in critical care. The clinical interventions ranged across comparative studies of drugs, devices or procedures. These interventions were mostly confined to the cardiology and oncology studies. For the non-clinical interventions, such as the immunisation and critical care studies, the intervention was a vaccination reminder and performance feedback, respectively. The majority of the studies were multi-centre, involved large sample sizes and included long follow-up periods with minimal loss to follow-up.

Most clinical registries were already relatively well-established, having been operational for a number of years prior to their utilisation in a RRCT. Furthermore, most of the registries included in our review were either national registries or at minimum State- or district-based. The Nordic countries exhibited the most comprehensive national registries that facilitated the enrolment of patients onto the registry upon confirmation of their disease, with the Swedish national cardiac registry – SWEDEHEART being an online registry that supported four RRCTs.

Data validity and data integrity of registries are critical elements in realising the full potential and scope of RRCTs [14]. In our review, 10 studies provided information on registry-data validity. Only three studies commented on missing data [29, 31, 36], whilst 14 studies remained silent on this matter. In countries with well-established national registries, data validation appears robust with minimal data missing. This, coupled with recruitment of large patient cohorts, enables RRCTs using such registries to not only have strong external validity but to also afford good internal validity. This confers them properties more akin to RCTs and makes them a viable alternative for obtaining high-quality clinical evidence. Whereas, RRCTs that are reliant on registries that are not robust or not subject to adequate data-validation processes may produce findings which cannot reliably inform clinical practice or health policy.

Therefore, data validation of registries and other data sources used in a RRCT is imperative if RRCTs are to move up the hierarchy of clinical evidence to position themselves as a valid alternative to the conventional RCT. Before embarking on the conduct of a RRCT, researchers should have a clear appreciation of the data collected in the registry(ies) and any supplementary RCD that their study will need to rely on, and the quality and validation of all data sources. Understanding how data is collected in the clinical registry is important to avoid misinterpreting results that are the consequence of data-entry error or bias. Based on this, the study design should be such that it adequately compensates for any deficiencies that such data sources may present, with researchers fully aware of these limitations in advance and actively looking to appropriately address them [2].

The ability of RRCTs to identify and recruit more effectively than conventional RCTs, due to the availability of searchable clinical information in the clinical registry enabling screening for eligible patients, is well-documented [36, 37]. In the TASTE trial, 76.9% of all eligible patients were randomised within 2 years and 9 months [2]. However, where participant registration onto the registry is not timely, or the registry has limited catchment coverage, this can pose a number of challenges. This was most evident in the vaccination studies, resulting in the non -recording of patients who sought their intervention outside of the registry catchment area. Invitations for participation were also issued to individuals who were no longer eligible [26, 31]. This becomes even more important for clinical intervention studies conducted within acute settings, as enrolment onto the registry and randomisation must be close to real time or at least concurrent with commencement of an intervention, so that the conduct of the study does not impact on the provision of best care. This was evident in several cardiology studies [16, 17].

The embedding of a randomisation module into the registry expedites the recruitment and randomisation of patients into a RRCT. The online nature of the SWEDEHEART registry allows for immediate enrolment of patients into the registry upon hospital admission and identification of the need for a PCI. As this registry is a nationally supported and funded initiative, capture of clinical data and validation of outcome measures is well-managed. This results in the conduct of RRCTs, such as the TASTE trial, providing high-quality clinical evidence; thus, demonstrating that RRCTs are a viable alternative to the more expensive standard RCT. It is apparent that for registries to accommodate RRCTs within acute clinical settings, an online registry platform that provides real-time registry enrolment of potential participants, is essential. Furthermore, the embedding of a randomisation module within a registry may help to address the intervention time constraints in such settings.

All of the six clinical intervention studies were open-label, multi-centre RRCTs. These studies examined two standard-of-care interventions in clinical settings [6, 16, 17, 23, 33, 36]. Only one of these six studies provided an explanation as to why a blinded design was not pursued. In this study, the choice for an open-label design was justified by feasibility and ethical considerations and the unavailability of a suitable sham comparator [36]. The remaining five interventional studies did not comment on the reasons for their open-label design. However, as reduced cost is a driving force behind conducting RRCTs we surmise that the cost of making the study double blinded may have been prohibitive. It is appreciated that there are increased costs associated with the manufacture and provision of a placebo control aimed to mimic in appearance or application the active intervention. It is also acknowledged that blinding is not always practically possible when two different standards of care are being compared. Most studies did acknowledge that the open-label design of the study was a study limitation, potentially biasing study outcomes. A double-blinded study design was not applicable for the non-clinical intervention studies.

Given the broad inclusion criteria of a RRCT, adequately powered trials can potentially be conducted at a single site, dependent on the type of event being investigated and the population size the site services. Multi-centre studies allow for the recruitment of a much larger number of participants into a trial, and this is necessary where either the disease being investigated and/or the outcome event is rare [3]. It is acknowledged that despite the large participant numbers and broad inclusion criteria, RRCTs may be subject to reduced external validity if the intended study population is geographically or socioeconomically restricted. For example, some of the vaccination-reminder studies identified this as a potential limitation of their study, as some of the populations targeted were of a restricted socioeconomic status [26, 29, 30].

Ethical and governance considerations are aspects of RRCTs that remain active areas of work. Given the breadth of research activity that can fall under the classification of a RRCT and the varied jurisdictional requirements, there can be no ‘one rule fits all’ approach. Six studies that involved both ethical approval and at a minimum oral consent involved a clinical intervention whereby randomisation determined the standard-of-care intervention to be provided. Consent was obtained in most studies prior to randomisation, and the inability to provide consent was a study exclusion criterion. For studies that had time constraints in relation to the delivery of an intervention, oral consent was deemed acceptable. This was then followed by written consent at a later and more appropriate time. In contrast, seven studies did not mention that they had ethical approval and made no reference to any form of consent from the participants. These studies were either vaccination participation invitation letters or a quality improvement study, whereby ethical approval and, by default, active participant consent were not considered to be necessary [26,27,28,29, 31, 34, 35]. Two studies that did not directly involve a clinical intervention [25, 32] obtained ethical approval but did not involve consent for participation. Consent for participation on a registry and/or RRCT is an area that requires further exploration and the approach will be informed by the ethical and governance requirements of the jurisdiction in which the registry resides and where the trial is being conducted.

Central adjudication of study endpoints, along with dedicated follow-up and systematic monitoring in RRCTs is critical to ensure the quality of the data related to the study outcome measures [38]. In our review, most studies involving a clinical intervention involved hard endpoints, such as mortality, in addition to other intervention-related study outcome measures. Five studies commented on the adjudication of their study outcome measures [6, 16, 17, 23, 36]. Of these, two confirmed a blinded endpoint adjudication process [6, 23]. One study confirmed that there was no adjudication of their study outcomes but relied on strict diagnostic indicators for defining the primary endpoint [33]. It is appreciated that hard endpoints, such as death from any cause, do not require adjudication [36]. For studies that utilised the SWEDEHEART registry, most relied on that registry for adjudication of their study outcomes, with no further study-specific adjudication of outcomes being made. The lack of central adjudication of study outcome measures in RRCTs has been a well-acknowledged limitation and becomes even more critical when RRCTs are multi-centre or, particularly, if there is intention for multinational involvement. Furthermore, the lack of adjudication, coupled with lower quality or missing data does necessitate more complex statistical methods to be utilised, which may inadvertently intimidate the reader [36].

In most of the clinical intervention studies, the hard endpoints included death from any cause, and the required outcomes were collected via data linkage of clinical registries with administrative population/claims data. This was most evident in the studies conducted in the Nordic countries where unique patient identification numbers facilitate complete tracking of patients across registries and other sources of databases; thus, allowing near complete follow-up of all participants [2]. Furthermore, the use of a registry in a RRCT allows for the long-term follow-up of participants. In our review, we identified two follow-up studies of a RRCT. The first was the 1-year post-TASTE follow-up study and the second, a 12-year post-follow-up study of a retrospective RRCT looking at bone fractures in women treated with tamoxifen for breast cancer[25]. In the TASTE study, which had more than 7244 participants, there was no single patient lost to follow-up; again highlighting the advantages of well-established registries and the ability to easily link to supplementary datasets using a unique patient identification number [9].

For RRCTs to provide high-quality clinical evidence, the challenges of outcome adjudication and data integrity and quality need to be addressed through the establishment of registries and/or datasets that have integrated quality assurance processes embedded into their administration. The use of supplementary datasets in conjunction with a registry can help to minimise the occurrence of missing or inaccurate data by facilitating data triangulation and providing a better understanding around data validity and integrity. However, in countries without a unique patient identifier, data linkage to enable the collection of primary/secondary outcomes may not always be possible and other means to collect such outcomes must be explored. In such circumstances, researchers would need to rely on data linkage across a number of records through appropriate data-linkage software. In Victoria, Australia, the Centre for Victorian Data Linkage (CVDL) utilises a deterministic data-linkage method whereby records across a number of registries and datasets are determined to belong to the one person on the basis of returning an exact match for a set of fields [39]. Probabilistic data linkage is also an option with obvious inherent limitations. However, the pursuit of enhanced RRCT internal validity must be carefully balanced so that the administrative and economic benefits that make RRCTs a viable alternative to conventional RCTs are not progressively eroded.

RRCTs are considered to be a cost-effective way of obtaining quality clinical evidence compared to conventional RCTs. Three studies provided a cost-benefit assessment of conducting a RRCT [17, 28, 30]. The SAFE-PCI study cost approximately US$5 million to conduct due to the utilisation of the NCDR CathPCI Registry for streamlined data collection and randomisation. A comparably sized trial not underpinned around a registry would have cost considerably more [17]. For the TASTE trial, costs were estimated at 10% or less of a conventional RCT [38, 40]. The increased cost-effectiveness of RRCTs can be attributed to RRCTs obtaining their outcome data from registries or RCD, reducing requirements for follow-up visits, monitoring and audits. Furthermore, as RRCTs utilise and rely on existing infrastructure and human resources, the need for new equipment and training of staff is limited. Given that 9–14% of a RCT’s total cost can be attributed to site monitoring, it is not surprising that the reduction or even elimination of many of the activities that comprise the essential compliance aspects of a RCT would result in substantive cost savings [9].

Despite the cost benefits of RRCTs, they are not yet readily afforded commercial support in comparison to conventional RCTs. Of all the studies, two cardiology studies had unrestricted commercial support in addition to academic grant funding. The support of RRCTs by industry should be a welcomed involvement, as it will allow for the conduct of adequately funded studies and the introduction of investment that is essential in building the requisite infrastructure and processes required to help overcome the challenges of RRCTs and to enhance their internal validity. Furthermore, industry participation in RRCT studies would result in RCTs not necessarily underpinned on commercial imperatives but ones that address questions of public health importance. Ideally, to alleviate any concerns in relation to undue industry influence on the topic of investigation, namely study design and result reporting, any industry support in investigator-initiated RRCTs should be prefaced around the provision of unrestricted grants or like funding. The prospect of industry embracing RRCTs in lieu of conventional RCTs does not appear to be an imminent prospect, but one which, over time, will evolve and increase in occurrence as industry better appreciates how RRCTs can complement RCTs, and the academic establishment learns to work alongside commercial entities in a synergistic and complementary manner and feels comfortable accepting and pursuing such arrangements. Future research is warranted to investigate industry’s perspective of RRCTs and to further explore the barriers that have limited their involvement to date.

To our knowledge, there are currently no guidelines for the reporting of a RRCT and this presents several challenges. It is acknowledged that RRCTs should be underpinned by the Consolidated Standards of Reporting Trials Statement (CONSORT Statement). RRCTs should provide information on the quality of the registry itself, particularly around elements of data quality which should include, but not be limited to: accuracy, completeness, timeliness, population coverage and study endpoint adjudication. The reporting of consent into the registry, and subsequently into a RRCT, need to be improved, as does the financial disclosure for both the registry and the RRCT. An extension of the CONSORT Statement for RCTs using cohorts and routinely collected health data is currently underway to improve the quality of reporting [41].

Limitations

Whilst the search that we conducted was extensive and included a wide range of relevant electronic databases, it did not include studies in languages other than English and of the grey literature. Given that RRCTs are a novel research design, the absence of indexing terms for RRCTs increases the possibility that some studies may not have been captured by our search terms. Furthermore, a lack of a precise definition for a RRCT makes it challenging to ascertain the research activity in this space and its impact. Depending on the criteria used to define a RRCT, the number of studies captured will vary considerably. This is evident in the review conducted by Mathes and Buehn [8] which used a broader definition to define RRCT, resulting in 71 studies being included. RRCTs underpinned around RCD, such as electronic health records and administrative claims data, were excluded from our review, but we recognise the role of RCD in supplementing information required in the conduct of a RRCT and helping to address some of their inherent limitations. A combination of RCD and actively collected data, such as a clinical registry, may make a trial more feasible [9]. Future research is warranted to assess the feasibility of using RCD in RCTs. Furthermore, most studies only briefly described the quality of their registries and provided limited information about ethical approval and the consent process. It is unclear whether this represents reporting bias or merely highlights the lack of emphasis placed on these aspects, given that there is an inherent expectation of lower data quality and integrity for such trials compared to conventional RCTs. We also acknowledge that this review did not explicitly explore qualitative barriers and enablers to the use of RRCTs. We believe that further research in this area is warranted to help increase the implementation of RRCTs.

Conclusion

RRCTs have an important role to play in informing best clinical practice and health policy. Their low cost, reduced administrative burden and enhanced external validity make them an attractive research methodology to be used to address questions of public health importance. However, for RRCTs to be considered a viable alternative to a RCT in certain clinical settings, the issues of data integrity, completeness, timeliness, validation and adjudication of endpoints need to be carefully addressed. It would be our recommendation that RRCTs should be registered as is the case for RCTs and that RRCTs should be underpinned by the CONSORT Statement. Our review also highlights the variable definitions being used for a RRCT and reinforces the need for universally accepted criteria to be established, such that the current broad criteria that are in use do not dilute the influence and impact of studies that carry the real hallmarks of a RRCT.

Appendices

Appendix 1

Search strategy

registry-based randomised clinical trial* or

registry-based randomized clinical trial* or

registry-based randomised control* clinical trial* or

registry-based randomized control* clinical trial* or

registry-based randomised control* trial*or

registry-based randomized control* trial* or

registry embedded clinical trial* or

registry trial* or

registry-based randomized trial* or

registry-based randomised trial* or

pragmatic trial* or

pragmatic clinical trial*

Appendix 2

Table 3 Data extraction of included studies (N = 17)

Appendix 3

Table 4 Characteristics of clinical registries (N = 13)