We conducted a comprehensive and thorough search to identify studies evaluating the effectiveness of techniques or interventions designed to prevent or avoid enrolment errors during patient recruitment into RCTs. This included an attempt to search the grey literature (conference abstracts and presentations). No primary publications on this topic were found. We identified one study reported in the grey literature presenting the results of an evaluation of the effectiveness of an intervention (a dummy enrolment run-in phase) designed to prevent or avoid enrolment errors in a clinical trial. In the context of a multicentre clinical trial, use of such an intervention could significantly reduce enrolment errors; in this report, from 16.1% during the run-in phase to < 1% after trial initiation (P < 0.001).
The conduct of multicentre clinical trials is expensive, and the costs are increasing at an alarming rate. Accounting for all costs of development (including the costs of therapies that are not successful at the FDA phase I, II or III licensing stages, but not including marketing costs), the 1999 costs of successfully licensing one new drug or therapy under the FDA scheme have been estimated at US$500 million . In 2009, discounting for inflation, these costs were estimated at more than US$1,000 million dollars . Interestingly, over the same time period, the rate at which FDA phase III clinical trials failed to demonstrate expected benefits (negative trials) has risen from 20% in the 1990s to nearly 50% now. Excessive protocol violations may be one reason why trials fail to demonstrate expected benefits, even if benefits truly exist .
Excessive protocol violations may result in safety issues that can cause multicentre clinical trials to be stopped prematurely . Data from multicentre FDA phase III licensing trials show that patients enrolled with protocol violations may experience harm, even death . Furthermore, this harm may dilute treatment effects, leading to negative results or early trial cessation . Examples of protocol violations include failure of the researchers to deliver the study intervention according to the study protocol, noncompliance of patient participants with the study protocol, issues relating to informed consent, and inappropriate enrolment of patients into the trial who did not meet the trial eligibility criteria. Of these various types of protocol violations, evidence suggests that enrolment in contradiction of key eligibility criteria (enrolment errors) can result in the greatest patient harm . For example, if a protocol violation occurs because investigators fail to deliver any of the active study treatment, no direct patient harm is caused; however if a patient with a known contraindication to the study treatment is enrolled, the risk of harm is real.
Although it is recommended that study eligibility criteria should be clear, objective and precise , they are often complex and open to interpretation. In our own personal experiences, a CCC must dedicate considerable time during the early stages of a multicentre trial to ensure that interpretation is consistent between sites. Indeed, compelling evidence demonstrates that the risk of enrolment errors is highest early in a trial, and decreases as study sites gain experience in enrolling patients [2, 9]. This apparent learning curve, combined with the possibility of harm from inappropriate enrolment, begs the question: how can we get to the flat end of the learning curve faster?
The dummy enrolment run-in phase
Medical simulation exercises, such as performing cardiopulmonary resuscitation on a mannequin, can create a realistic learning environment without patient risk. Simulations have been shown to change practice, increase compliance with guidelines, and improve team dynamics . Like a simulation exercise, a dummy enrolment run-in phase in an RCT replicates the screening and patient identification process in a protected and interactive learning environment, without patient risk.
The dummy enrolment run-in phase is conducted after the provision of education at a study start-up meeting but before an enrolment website is made 'live'. During the dummy enrolment run-in phase, sites gain practical experience applying the new study eligibility criteria by attempting to correctly identify patients they believe to be eligible for the trial and submitting these patients to the trial's CCC for adjudication on appropriateness. Because these patients are not formally enrolled or randomised, risk of harm is avoided. The dummy enrolment run-in phase allows participating sites to learn, by direct first-hand experience, how to apply and interpret novel inclusion and exclusion criteria .
If a site submits a patient that is inappropriate to enrol, the CCC is provided with an opportunity to provide immediate nonpunitive positive feedback and education on that particular patient. The dummy enrolment run-in phase also allows the CCC to identify sites that may have major problems with trial execution. These sites can be targeted and provided with more detailed multifaceted educational strategies, including educational outreach visits and formal one to one academic detailing [12, 13], before making the study live. Furthermore, the dummy enrolment run-in phase allows the CCC to learn as well.
The CCC can use the dummy enrolment run-in phase to identify inclusion or exclusion criteria that are too complex, open to different interpretation or poorly worded. These problematic criteria can be addressed by providing additional information to all sites using a 'frequently asked questions' (FAQ) publication. The CCC may also elect to rewrite problematic criteria or to change the focus or content of future education on the topic. The dummy enrolment run-in phase provides the opportunity to refine CCC processes and address eligibility criteria issues before the trial goes live.
The conduct of a dummy enrolment run-in phase increases the overall duration, and therefore the costs, of a clinical trial. The ideal duration of the run-in phase must therefore be balanced against these costs. The publications by Macias et al. and Laterre et al. both present evidence of a 'learning curve', whereby more protocol violations occur early during a clinical trial, and suggest that error rates are minimised after a site recruits at least four patients [2, 9]. The presence of learning curves, describing how complications are reduced as experience increases, have been reported for many laparoscopic and other surgical procedures [14–16]. It is likely that the ideal duration of a dummy enrolment run-in phase varies based on the complexity of the eligibility criteria and on other trial factors. Publications of enrolment errors and other protocol violations by patient recruitment numbers should be encouraged to assist those conducting trials in determining the appropriate duration of a dummy enrolment run-in for their given speciality, discipline or type of trial.
Strengths and limitations
The primary literature search, conducted using PubMed, EMBASE and Cochrane database, was designed to be 'sensitive' to the presence of articles on the topic of interest. We are reasonably certain that our negative search results are reliable. Because we acknowledged that it was possible the primary literature search would yield few results, we believed it was important to search the grey literature as well, and, for the purposes of this study, focused this 'grey literature search towards identifying conference abstracts and presentations .
Hand-searching for conference abstracts is known to be time-consuming, expensive and difficult . Our grey literature search for online abstract repositories was unrewarding. Because conference websites are not indexed using standardised terms, it is likely that our search missed important meetings. Online abstract repositories were uncommon, even though the technology for providing access to abstracts in a searchable format is readily available. For example, posting the abstract to the conference website in a portable document format (PDF) that is indexed by Google Scholar (http://scholar.google.com.au, accessed 26 October 2009) would effectively create an online searchable repository. We strongly recommend that academic conferences publish accepted abstracts on their conference website in a format that allows indexing and thus searching, by search engines such as Google Scholar.