Our scoping review yielded a total of 17 studies. Of these, eight were in cardiology, six in immunisation, two in oncology and one in critical care. The clinical interventions ranged across comparative studies of drugs, devices or procedures. These interventions were mostly confined to the cardiology and oncology studies. For the non-clinical interventions, such as the immunisation and critical care studies, the intervention was a vaccination reminder and performance feedback, respectively. The majority of the studies were multi-centre, involved large sample sizes and included long follow-up periods with minimal loss to follow-up.
Most clinical registries were already relatively well-established, having been operational for a number of years prior to their utilisation in a RRCT. Furthermore, most of the registries included in our review were either national registries or at minimum State- or district-based. The Nordic countries exhibited the most comprehensive national registries that facilitated the enrolment of patients onto the registry upon confirmation of their disease, with the Swedish national cardiac registry – SWEDEHEART being an online registry that supported four RRCTs.
Data validity and data integrity of registries are critical elements in realising the full potential and scope of RRCTs [14]. In our review, 10 studies provided information on registry-data validity. Only three studies commented on missing data [29, 31, 36], whilst 14 studies remained silent on this matter. In countries with well-established national registries, data validation appears robust with minimal data missing. This, coupled with recruitment of large patient cohorts, enables RRCTs using such registries to not only have strong external validity but to also afford good internal validity. This confers them properties more akin to RCTs and makes them a viable alternative for obtaining high-quality clinical evidence. Whereas, RRCTs that are reliant on registries that are not robust or not subject to adequate data-validation processes may produce findings which cannot reliably inform clinical practice or health policy.
Therefore, data validation of registries and other data sources used in a RRCT is imperative if RRCTs are to move up the hierarchy of clinical evidence to position themselves as a valid alternative to the conventional RCT. Before embarking on the conduct of a RRCT, researchers should have a clear appreciation of the data collected in the registry(ies) and any supplementary RCD that their study will need to rely on, and the quality and validation of all data sources. Understanding how data is collected in the clinical registry is important to avoid misinterpreting results that are the consequence of data-entry error or bias. Based on this, the study design should be such that it adequately compensates for any deficiencies that such data sources may present, with researchers fully aware of these limitations in advance and actively looking to appropriately address them [2].
The ability of RRCTs to identify and recruit more effectively than conventional RCTs, due to the availability of searchable clinical information in the clinical registry enabling screening for eligible patients, is well-documented [36, 37]. In the TASTE trial, 76.9% of all eligible patients were randomised within 2 years and 9 months [2]. However, where participant registration onto the registry is not timely, or the registry has limited catchment coverage, this can pose a number of challenges. This was most evident in the vaccination studies, resulting in the non -recording of patients who sought their intervention outside of the registry catchment area. Invitations for participation were also issued to individuals who were no longer eligible [26, 31]. This becomes even more important for clinical intervention studies conducted within acute settings, as enrolment onto the registry and randomisation must be close to real time or at least concurrent with commencement of an intervention, so that the conduct of the study does not impact on the provision of best care. This was evident in several cardiology studies [16, 17].
The embedding of a randomisation module into the registry expedites the recruitment and randomisation of patients into a RRCT. The online nature of the SWEDEHEART registry allows for immediate enrolment of patients into the registry upon hospital admission and identification of the need for a PCI. As this registry is a nationally supported and funded initiative, capture of clinical data and validation of outcome measures is well-managed. This results in the conduct of RRCTs, such as the TASTE trial, providing high-quality clinical evidence; thus, demonstrating that RRCTs are a viable alternative to the more expensive standard RCT. It is apparent that for registries to accommodate RRCTs within acute clinical settings, an online registry platform that provides real-time registry enrolment of potential participants, is essential. Furthermore, the embedding of a randomisation module within a registry may help to address the intervention time constraints in such settings.
All of the six clinical intervention studies were open-label, multi-centre RRCTs. These studies examined two standard-of-care interventions in clinical settings [6, 16, 17, 23, 33, 36]. Only one of these six studies provided an explanation as to why a blinded design was not pursued. In this study, the choice for an open-label design was justified by feasibility and ethical considerations and the unavailability of a suitable sham comparator [36]. The remaining five interventional studies did not comment on the reasons for their open-label design. However, as reduced cost is a driving force behind conducting RRCTs we surmise that the cost of making the study double blinded may have been prohibitive. It is appreciated that there are increased costs associated with the manufacture and provision of a placebo control aimed to mimic in appearance or application the active intervention. It is also acknowledged that blinding is not always practically possible when two different standards of care are being compared. Most studies did acknowledge that the open-label design of the study was a study limitation, potentially biasing study outcomes. A double-blinded study design was not applicable for the non-clinical intervention studies.
Given the broad inclusion criteria of a RRCT, adequately powered trials can potentially be conducted at a single site, dependent on the type of event being investigated and the population size the site services. Multi-centre studies allow for the recruitment of a much larger number of participants into a trial, and this is necessary where either the disease being investigated and/or the outcome event is rare [3]. It is acknowledged that despite the large participant numbers and broad inclusion criteria, RRCTs may be subject to reduced external validity if the intended study population is geographically or socioeconomically restricted. For example, some of the vaccination-reminder studies identified this as a potential limitation of their study, as some of the populations targeted were of a restricted socioeconomic status [26, 29, 30].
Ethical and governance considerations are aspects of RRCTs that remain active areas of work. Given the breadth of research activity that can fall under the classification of a RRCT and the varied jurisdictional requirements, there can be no ‘one rule fits all’ approach. Six studies that involved both ethical approval and at a minimum oral consent involved a clinical intervention whereby randomisation determined the standard-of-care intervention to be provided. Consent was obtained in most studies prior to randomisation, and the inability to provide consent was a study exclusion criterion. For studies that had time constraints in relation to the delivery of an intervention, oral consent was deemed acceptable. This was then followed by written consent at a later and more appropriate time. In contrast, seven studies did not mention that they had ethical approval and made no reference to any form of consent from the participants. These studies were either vaccination participation invitation letters or a quality improvement study, whereby ethical approval and, by default, active participant consent were not considered to be necessary [26,27,28,29, 31, 34, 35]. Two studies that did not directly involve a clinical intervention [25, 32] obtained ethical approval but did not involve consent for participation. Consent for participation on a registry and/or RRCT is an area that requires further exploration and the approach will be informed by the ethical and governance requirements of the jurisdiction in which the registry resides and where the trial is being conducted.
Central adjudication of study endpoints, along with dedicated follow-up and systematic monitoring in RRCTs is critical to ensure the quality of the data related to the study outcome measures [38]. In our review, most studies involving a clinical intervention involved hard endpoints, such as mortality, in addition to other intervention-related study outcome measures. Five studies commented on the adjudication of their study outcome measures [6, 16, 17, 23, 36]. Of these, two confirmed a blinded endpoint adjudication process [6, 23]. One study confirmed that there was no adjudication of their study outcomes but relied on strict diagnostic indicators for defining the primary endpoint [33]. It is appreciated that hard endpoints, such as death from any cause, do not require adjudication [36]. For studies that utilised the SWEDEHEART registry, most relied on that registry for adjudication of their study outcomes, with no further study-specific adjudication of outcomes being made. The lack of central adjudication of study outcome measures in RRCTs has been a well-acknowledged limitation and becomes even more critical when RRCTs are multi-centre or, particularly, if there is intention for multinational involvement. Furthermore, the lack of adjudication, coupled with lower quality or missing data does necessitate more complex statistical methods to be utilised, which may inadvertently intimidate the reader [36].
In most of the clinical intervention studies, the hard endpoints included death from any cause, and the required outcomes were collected via data linkage of clinical registries with administrative population/claims data. This was most evident in the studies conducted in the Nordic countries where unique patient identification numbers facilitate complete tracking of patients across registries and other sources of databases; thus, allowing near complete follow-up of all participants [2]. Furthermore, the use of a registry in a RRCT allows for the long-term follow-up of participants. In our review, we identified two follow-up studies of a RRCT. The first was the 1-year post-TASTE follow-up study and the second, a 12-year post-follow-up study of a retrospective RRCT looking at bone fractures in women treated with tamoxifen for breast cancer[25]. In the TASTE study, which had more than 7244 participants, there was no single patient lost to follow-up; again highlighting the advantages of well-established registries and the ability to easily link to supplementary datasets using a unique patient identification number [9].
For RRCTs to provide high-quality clinical evidence, the challenges of outcome adjudication and data integrity and quality need to be addressed through the establishment of registries and/or datasets that have integrated quality assurance processes embedded into their administration. The use of supplementary datasets in conjunction with a registry can help to minimise the occurrence of missing or inaccurate data by facilitating data triangulation and providing a better understanding around data validity and integrity. However, in countries without a unique patient identifier, data linkage to enable the collection of primary/secondary outcomes may not always be possible and other means to collect such outcomes must be explored. In such circumstances, researchers would need to rely on data linkage across a number of records through appropriate data-linkage software. In Victoria, Australia, the Centre for Victorian Data Linkage (CVDL) utilises a deterministic data-linkage method whereby records across a number of registries and datasets are determined to belong to the one person on the basis of returning an exact match for a set of fields [39]. Probabilistic data linkage is also an option with obvious inherent limitations. However, the pursuit of enhanced RRCT internal validity must be carefully balanced so that the administrative and economic benefits that make RRCTs a viable alternative to conventional RCTs are not progressively eroded.
RRCTs are considered to be a cost-effective way of obtaining quality clinical evidence compared to conventional RCTs. Three studies provided a cost-benefit assessment of conducting a RRCT [17, 28, 30]. The SAFE-PCI study cost approximately US$5 million to conduct due to the utilisation of the NCDR CathPCI Registry for streamlined data collection and randomisation. A comparably sized trial not underpinned around a registry would have cost considerably more [17]. For the TASTE trial, costs were estimated at 10% or less of a conventional RCT [38, 40]. The increased cost-effectiveness of RRCTs can be attributed to RRCTs obtaining their outcome data from registries or RCD, reducing requirements for follow-up visits, monitoring and audits. Furthermore, as RRCTs utilise and rely on existing infrastructure and human resources, the need for new equipment and training of staff is limited. Given that 9–14% of a RCT’s total cost can be attributed to site monitoring, it is not surprising that the reduction or even elimination of many of the activities that comprise the essential compliance aspects of a RCT would result in substantive cost savings [9].
Despite the cost benefits of RRCTs, they are not yet readily afforded commercial support in comparison to conventional RCTs. Of all the studies, two cardiology studies had unrestricted commercial support in addition to academic grant funding. The support of RRCTs by industry should be a welcomed involvement, as it will allow for the conduct of adequately funded studies and the introduction of investment that is essential in building the requisite infrastructure and processes required to help overcome the challenges of RRCTs and to enhance their internal validity. Furthermore, industry participation in RRCT studies would result in RCTs not necessarily underpinned on commercial imperatives but ones that address questions of public health importance. Ideally, to alleviate any concerns in relation to undue industry influence on the topic of investigation, namely study design and result reporting, any industry support in investigator-initiated RRCTs should be prefaced around the provision of unrestricted grants or like funding. The prospect of industry embracing RRCTs in lieu of conventional RCTs does not appear to be an imminent prospect, but one which, over time, will evolve and increase in occurrence as industry better appreciates how RRCTs can complement RCTs, and the academic establishment learns to work alongside commercial entities in a synergistic and complementary manner and feels comfortable accepting and pursuing such arrangements. Future research is warranted to investigate industry’s perspective of RRCTs and to further explore the barriers that have limited their involvement to date.
To our knowledge, there are currently no guidelines for the reporting of a RRCT and this presents several challenges. It is acknowledged that RRCTs should be underpinned by the Consolidated Standards of Reporting Trials Statement (CONSORT Statement). RRCTs should provide information on the quality of the registry itself, particularly around elements of data quality which should include, but not be limited to: accuracy, completeness, timeliness, population coverage and study endpoint adjudication. The reporting of consent into the registry, and subsequently into a RRCT, need to be improved, as does the financial disclosure for both the registry and the RRCT. An extension of the CONSORT Statement for RCTs using cohorts and routinely collected health data is currently underway to improve the quality of reporting [41].
Limitations
Whilst the search that we conducted was extensive and included a wide range of relevant electronic databases, it did not include studies in languages other than English and of the grey literature. Given that RRCTs are a novel research design, the absence of indexing terms for RRCTs increases the possibility that some studies may not have been captured by our search terms. Furthermore, a lack of a precise definition for a RRCT makes it challenging to ascertain the research activity in this space and its impact. Depending on the criteria used to define a RRCT, the number of studies captured will vary considerably. This is evident in the review conducted by Mathes and Buehn [8] which used a broader definition to define RRCT, resulting in 71 studies being included. RRCTs underpinned around RCD, such as electronic health records and administrative claims data, were excluded from our review, but we recognise the role of RCD in supplementing information required in the conduct of a RRCT and helping to address some of their inherent limitations. A combination of RCD and actively collected data, such as a clinical registry, may make a trial more feasible [9]. Future research is warranted to assess the feasibility of using RCD in RCTs. Furthermore, most studies only briefly described the quality of their registries and provided limited information about ethical approval and the consent process. It is unclear whether this represents reporting bias or merely highlights the lack of emphasis placed on these aspects, given that there is an inherent expectation of lower data quality and integrity for such trials compared to conventional RCTs. We also acknowledge that this review did not explicitly explore qualitative barriers and enablers to the use of RRCTs. We believe that further research in this area is warranted to help increase the implementation of RRCTs.
