Recruitment patterns in a large international randomized controlled trial of perioperative care in cancer patients
Trials volume 22, Article number: 219 (2021)
The Prophylactic Antibiotic Regimens in Tumor Surgery (PARITY) randomized controlled trial (RCT) was the first study to prospectively enroll and randomize orthopedic oncology patients in multiple centers internationally. The objective of this study was to describe recruitment patterns, to examine the differences in enrollment across different PARITY sites, and to identify variables associated with differing levels of recruitment.
Data from this study was obtained from the PARITY trial Methods Center and records of correspondence between the Methods Center and recruiting sites. We performed descriptive statistics to report the recruitment patterns over time. We compared recruitment, time to set up, and time to enroll the first patient between North American and international sites, private and public healthcare models, and the presence or absence of research personnel. Two-tailed non-paired t tests were performed to test average monthly recruitment rates between groups.
A total of 602 patients from 36 North American and 12 international sites were recruited from 2013 to 2019. North American sites were able to become fully enrollment-ready at an average of 19.5 months and international sites at an average of 27 months. Once enrolling, international sites were able to enroll 0.59 patients per/month whereas North American sites averaged a monthly recruitment rate of 0.2 patients/month once enrolling. Sites with research personnel reached enrollment-ready status at an average of 19.3 months and sites without research support at an average of 30.3 months. Once enrolling, the recruitment rate was 0.28 patients/month and 0.2 patients per month for sites with and without research support, respectively. Publicly funded sites had a monthly enrollment of 0.4 patients/month whereas privately funded sites had a monthly enrollment rate of 0.17 patients/month.
As a collaborative group, the PARITY investigators increased the pace of recruitment throughout the trial, likely by increasing the number of active sites. The longer time to start-up at international sites may be due to the complex governing regulations of pharmaceutical trials. Nevertheless, international sites should be considered essential as they recruited significantly more patients per month once active. The absence of research support personnel may lead to delays in the time to start-up. The results of the current study will provide guidance for choosing which sites to recruit for participation in future collaborative clinical trials in orthopedic oncology and other surgical specialties.
ClinicalTrials.gov NCT01479283. Prospectively registered on November 24, 2011
Randomized controlled trials (RCTs) are widely recognized as the study design that can provide the highest level of clinical effectiveness evidence with the minimal amount of systematic bias . When compared to the medical field, surgical specialties often lack high-quality RCT evidence on which to base clinical decisions [2, 3]. Surgically based RCTs present with a unique set of challenges that can lead to systematic bias such as lack of blinding, surgeon preferences, poor patient compliance, and loss to follow-up [3,4,5].
In order to reach the target sample size and to increase the generalizability of results, RCTs are often multicenter and international . However, international trials are burdened with significant challenges such as differences in regulatory bodies, varying sources of funding, protocol standardization, language barriers, and language translation errors . These challenges can make timely recruitment and enrollment difficult to achieve. It is well documented that a significant number of clinical trials fail to meet enrollment goals, which may lead to underpowered studies and imprecise findings, extended study duration, or even trial abandonment [8,9,10,11,12]. A variety of site-specific variables have been found to impact enrollment performance in clinical trials including patient volume, previous clinical research involvement, lead investigator enthusiasm, and dedicated support staff [13,14,15,16].
Within the surgical subspecialty of orthopedic oncology, there is very little high-level evidence to inform clinical decision-making—including a lack of international multicenter surgical RCTs [17, 18]. Given the rarity of soft tissue and bone sarcomas, a collaborative multicenter approach is needed. However, due to the extreme heterogeneity of both case complexity as well as treatment protocols, trials in orthopedic oncology represent a unique challenge in RCT implementation. The Prophylactic Antibiotic Regimens in Tumor Surgery (PARITY) trial was the first international multicenter trial in orthopedic oncology and aimed to provide high-quality evidence for prophylactic antibiotic therapy in patients undergoing surgical management for lower extremity bone tumors [19, 20]. The trial completed enrollment in 2019. Given that this was the first RCT of its kind in the field, it is important to reflect upon the challenges encountered and experiences gained in order to inform future trials. The objective of this study was to describe recruitment patterns, to examine the differences in enrollment across different PARITY sites, and to identify variables associated with differing levels of recruitment.
Materials and methods
Data for this study was derived from the PARITY trial (NCT01479283), the protocol for which has been previously published . PARITY was a blinded, parallel two-arm multicenter international RCT. Study enrollment began in January 2013 and was completed in October 2019. Target enrollment was 600 participants based on a pre hoc sample size calculation. Inclusion criteria consisted of patients 12 years of age or older who had a lower extremity primary bone malignancy, a benign aggressive bone tumor, a soft-tissue sarcoma which had invaded the bone, or oligometastatic bone disease and required surgical excision and endoprosthetic reconstruction of the femur and/or tibia were included.
Study participants were randomized to either short- (24 h) or long-duration (5 days) post-operative prophylactic antibiotics. A centralized, Internet-based computerized randomization system was utilized to conceal allocation. The primary outcome for the trial was surgical site infection.
We obtained several variables that have been previously associated with differing levels of recruitment in RCTs directly from local site investigators [21,22,23,24]. We compared recruitment levels between North American and international sites, private or public healthcare models, and the presence or absence of research personnel. We defined the presence of research personnel as a dedicated research nurse, coordinator, or assistant available to assist the local Principal Investigator in the conduct of the trial. Differentiating between North American and international sites was undertaken for a number of reasons. This trial was funded through the Musculoskeletal Tumor Society, a largely North American-based society, making North American sites the initial focus of the trial. The decision to add international sites was based on the growing international interest in collaboration. Given that the majority of these sites operated in languages outside of the English language, they were considered operationally different with unique language-based challenges .
We defined the rate of recruitment as patients recruited per month. We defined the time to set-up (TTSU) as the time from when an interested site first contacted the PARITY Methods Center to the time when the site was deemed able to begin enrolling—after having secured ethics approval, completed training, and signed all clinical trial agreements. We defined time to first patient (TTFP) as the interval between when a site was ready to enroll patients and when the first patient was enrolled at that site.
We performed descriptive statistics to evaluate the recruitment patterns over time. We used two-tailed non-paired t tests to explore differences in average monthly recruitment rates between North American and international sites, between private or public healthcare systems, by the presence or absence of dedicated research personnel, and if monthly enrollment varied by TTSU or TTFP. We also compared TTSU and TTFP based on location (North America vs international), healthcare funding model, and research personnel. We also calculated the Pearson R correlation coefficient between TTSU, TTFP, and average enrollment rate. We used the chi-square test to assess for the relationship between categorical variables, when applicable. Significance for all tests was set at alpha = 0.05. We performed all analyses in Microsoft Excel .
A total of 55 sites obtained ethics approval and signed clinical trial agreements. Forty-eight (87%) of these open sites recruited at least one patient during the study period and were deemed active, while seven sites did not enroll any patients in the study. There were 44 other sites that expressed interest and/or received study contracts but never completed the process of becoming an open site. Table 1 and Fig. 1 demonstrate the number of active sites and patients enrolled per month over the study period. A total of 36 active sites were North American and 12 were international. When characterized by the healthcare model, 19 sites were predominately publicly funded and the remaining 29 were predominately privately funded. Of the privately funded sites, 28 were based in the USA and one site was based in Argentina.
Patient enrollment rate
A total of 602 patients from 48 clinical sites in 12 countries were recruited from January 2013 through to October 2019. While the average number of patients enrolled throughout the study was 7.3 patients/month (SD ± 5.2), average monthly enrollment increased every year until the last year of the study and enrollment was greater than 10 patients/month in 2017, 2018, and 2019 (Table 1 and Fig. 1). While 58 months were required to reach 50% of target enrollment (January 2013 to October 2017), only an additional 24 months was required to achieve 100% enrollment. The top 20% of sites in terms of patient enrollment accounted for almost 60% of all included patients (355/602). The highest enrolling single site for the trial accounted for 14% of all patients (84/602).
Site opening and enrollment
There was no correlation between TTSU and the site’s average monthly enrollment (Pearson R = 0.27, p = 0.069) (Fig. 2). However, once set up, the time needed for a site to recruit their first patient (TTFP) was predictive of the number of total patients recruited and the average recruitment rate of that site. Generally, the time needed to recruit the first patient was inversely correlated to the average monthly enrollment rate of that site (Pearson R = − 0.44, p = 0.0033). In fact, sites that required longer than 1 year to recruit their first patient had an average recruitment rate three and a half times lower than that of sites that were able to recruit their first patient within 1 year (0.09/month vs 0.32/month, p = 0.04).
Site location and healthcare system
There were 36 (75%) North American and 12 (25%) international active sites that enrolled patients. North American sites were able to become fully enrollment-ready at an average of 19.5 months and international sites at an average of 27 months. Once enrolling, international sites were able to enroll 0.59 patients per/month whereas North American sites averaged a monthly recruitment rate of 0.2 patients/month once enrolling (Table 2). The TTFP of North American and international sites was 8.7 and 4.5 months, respectively. The proportion of sites that demonstrated interest but did not become active or recruit a single patient was 25/61 and 26/38 for international and North American sites, respectively. When considering the healthcare model, the TTSU was 20.1 months in privately funded sites and 23.4 months in publicly funded sites. Publicly funded sites had a monthly enrollment of 0.4 patients/month whereas privately funded sites had a monthly enrollment rate of 0.17 patients/month.
Of active enrolling sites, 40 (83%) had research support personnel and 8 (17%) sites did not. Sites with research personnel reached enrollment-ready status at an average of 19.3 months and sites without research support at an average of 30.3 months. Once enrolling, the recruitment rate was 0.28 patients/month and 0.2 patients per month for sites with and without research support, respectively.
The PARITY trial is the first multicenter surgical RCT in orthopedic oncology and successfully achieved target recruitment. Increasing the number of open sites each year led to the continually increasing monthly enrollment rates, which was crucial to reaching target enrollment in a reasonable time frame. Recruitment of the second half of the patient cohort required less than 50% of the time required to recruit the first half (24 vs 58 months). While international sites required more time to become enrollment-ready (TTSU), a higher average monthly enrollment was observed at these sites once they were active. Sites without dedicated research personnel also demonstrated a longer TTSU, but enrolled at equivalent rates once active. Whether a site was publicly or privately funded did not affect TTSU; however, recruitment rates were faster in publicly funded sites. Sites that took longer than 1 year to recruit their first patient once active were unlikely to recover their recruitment rate thereafter and demonstrated an average recruitment rate of less than one third that of sites that were quicker to enroll.
Compared to medical trials, surgical trials have high rates of failure and are often discontinued due to slow recruitment [27, 28]. Incomplete or under-recruited RCTs pose significant problems through the waste of public funding and the reporting of inaccurate results [29, 30]. The presence of trial fatigue can slow recruitment and decrease the likelihood that a trial will be adequately powered . The PARITY Methods Center employed a number of strategies to maintain a high trial profile to encourage ongoing recruitment throughout the recruitment phase of the trial. Attending and presenting updates at national and international conferences allowed co-investigators to stay engaged in the trial. The trial website was a platform that allowed all investigators to remain up to date on the trial progress (www.PARITYtrial.com). The website also highlighted new sites and invited all potential investigators to join the trial. The PARITY team also delivered several newsletters annually via email to keep investigators up to date on trial progress. These strategies have been employed widely in RCTs in the past; however, due to the heterogeneity of recruitment strategies utilized, no formal comparisons to assess their efficacy have been undertaken . Although we were unable to quantify the effects of these interventions, the PARITY investigators felt that these interventions were important in increasing the year-by-year enrollment over the duration of the study.
International trials present unique challenges and benefits . Although they have the potential to increase recruitment and provide generalizable data, international trials are also burdened by increased regulatory burden, data quality, safety monitoring, and language barriers [25, 32,33,34]. The PARITY trial demonstrated that international sites outside of North America took significantly longer to achieve regulatory approval. In addition, it was more likely for international sites to fail to become enrollment-ready. However, once approved, international sites contributed significantly to recruitment. Delays in regulatory approvals in international trials are not unique to PARITY and have been encountered by several previous RCTs [21, 22]. Aban et al. conducted a US-based international trial determining the efficacy of steroids in Duchenne-muscular dystrophy. The authors demonstrated longer start-up times by international sites and cited delays in ethics approval, language barriers, and varying regulatory bodies as potential causes . Similarly, Crow et al. cited language barriers and the need for translation of key study documents prior to submission to local ethics committees as a major unintended financial cost and delay in site start-up in a multinational RCT .
We have decided not to publish the list of contributing sites, as requested by the reviewer, until the PARITY trial results themselves are published in 2021. The publication of this data would violate the proprietary rights of the PARITY investigators who contributed to the trial but did not agree to have identifiable individual site data published outside of the trial itself. In addition, the identity of the sites and investigators are beyond the scope of the current study.
Our findings also suggest ongoing challenges with clinical trial recruitment in insurance-based healthcare systems. It is possible that the lower volume of patients seen in a privatized healthcare system is due to the lack of centralized care. Some private hospitals have a single orthopedic oncologist and significantly lower volume than publicly funded hospitals that provide centralized care for a larger population and therefore treat a significantly higher volume of patients. The resources required to start up low-volume sites can result in recruitment delays, increased start-up costs, and increased yearly funding required to keep a large number of low-volume sites open. Alternatively, lower recruitment in privatized healthcare centers may have been due to a lack of insurance coverage for participation. Klamerus et al. demonstrated that 13.6% of patients consented for cancer clinical trials were unable to participate due to lack of insurance coverage .
Clinical research coordinators and research assistants are considered integral to running and completing a successful clinical trial [36,37,38]. Survey-based research has determined that the majority of Principal Investigators believe that clinical research personnel are able to improve recruitment . Results from our study demonstrate that sites with research personnel were able to start up significantly faster than sites without research personnel. However, once active, there was no difference in the monthly recruitment between sites that had research personnel and those that did not. A lack of difference in recruitment may have been secondary to reasons unrelated to research personnel such as low patient volumes or challenges with insurance companies. Based on our findings, we conclude that in orthopedic oncology research personnel are valuable and significantly improve trial efficiency by speeding up site start-up time.
A major strength of this study is the novelty of the findings. This is the first paper to our knowledge to examine variables associated with varying rates of recruitment in RCTs within orthopedic surgery. It provides important information about expected challenges and patterns of recruitment for future RCTs in the field. A limitation of the current paper is that the underlying reasons for differing levels of recruitment could not be fully explored with the available data. Although we can make hypotheses about differences among sites, without in-depth interviews with the site investigators, we cannot make conclusions about site-specific facilitators or barriers to successful recruitment. Future considerations for qualitative analysis of interviews with local principal investigators will allow for a deeper understanding of causative factors of varying levels of recruitment.
As a collaborative group, the PARITY investigators increased the pace of recruitment throughout the trial. The longer time to start-up at international sites is likely due to the complex governing regulations of pharmaceutical trials. Nevertheless, international sites should be considered essential as they recruited significantly more patients per month once active. The PARITY trial also highlights the ongoing challenges with recruitment in insurance-based healthcare systems. The absence of research support personnel is likely to result in delays in the time to start-up. The results of the current study provide important information to guide participation and recruitment in future collaborative clinical trials in orthopedic oncology and other surgical specialties.
Availability of data and materials
The data sets used and analyzed during the current study are available from the corresponding author on reasonable request.
Wright JG. A practical guide to assigning levels of evidence. J Bone Joint Surg Series A. 2007;89:1128–30.
Farrokhyar F, Karanicolas PJ, Thoma A, Simunovic M, Bhandari M, Devereaux PJ, et al. Randomized controlled trials of surgical interventions. Ann Surg. 2010;251:409–16.
Cook JA. The challenges faced in the design, conduct and analysis of surgical randomised controlled trials. Trials. 2009;10:1–9.
Jeray KJ, Tanner S, Schemitsch EH, Bhandari M. Are large fracture trials possible? J Orthopaedic Trauma. 2010;24:S87–92.
Sibai T, Carlisle H, Tornetta P. The darker side of randomized trials: recruitment challenges. J Bone Joint Surg Series A. 2012;94:49–55.
Søreide K, Alderson D, Bergenfelz A, Beynon J, Connor S, Deckelbaum DL, et al. Strategies to improve clinical research in surgery through international collaboration. Lancet. 2013;382:1140–51.
Kenyon GM, David Mendelow A, Gregson BA, Rowan E. Obtaining regulatory approval for multicentre randomised controlled trials: experiences in the STICH II trial. Br J Neurosurg. 2011;25:352–6.
Charlson ME, Horwitz RI. Applying results of randomised trials to clinical practice: impact of losses before randomisation. Br Med J. 1984;289:1281–4.
Easterbrook PJ, Matthews DR. Fate of research studies. J R Soc Med. 1992;85:2.
Toerien M, Brookes ST, Metcalfe C, de Salis I, Tomlin Z, Peters TJ, et al. A review of reporting of participant recruitment and retention in RCTs in six major journals. Trials. 2009;85:71.
Fletcher B, Gheorghe A, Moore D, Wilson S, Damery S. Improving the recruitment activity of clinicians in randomised controlled trials: a systematic review. BMJ Open. 2012;2:1.
Schroen AT, Petroni GR, Wang H, Gray R, Wang XF, Cronin W, et al. Preliminary evaluation of factors associated with premature trial closure and feasibility of accrual benchmarks in phase III oncology trials. Clin Trials. 2010;7:312–21.
Dickson S, Logan J, Hagen S, Stark D, Glazener C, McDonald AM, et al. Reflecting on the methodological challenges of recruiting to a United Kingdom-wide, multi-centre, randomised controlled trial in gynaecology outpatient settings. Trials. 2013;14:1–8.
Fogel DB. Factors associated with clinical trials that fail and opportunities for improving the likelihood of success: a review. Contemp Clin Trials Commun. 2018;11:156–64.
Kenyon G, Gregson B, Mendelow AD, Rowan E. What does it take to deliver an international neurosurgical trial? Experiences from STICH II. Br J Neurosurg. 2014;28:461–6.
Csimma C, Swiontkowski MF. Large clinical trials in musculoskeletal trauma: are they possible? Lessons learned from the international study of the use of rhBMP-2 in open tibial fractures. J Bone Joint Surg Series A. 2005;87:218–22.
Evaniew N, Nuttall J, Farrokhyar F, Bhandari M, Ghert M. What are the levels of evidence on which we base decisions for surgical management of lower extremity bone tumors? Tumor. Clin Orthopaedics Relat Res. 2014;472:8–15.
Lerman DM, Cable MG, Thornley P, Evaniew N, Slobogean GP, Bhandari M, et al. Has the level of evidence of podium presentations at the musculoskeletal tumor society annual meeting changed over time? Clin Orthopaedics Relat Res. 2017;475:853–60.
Ghert M, Deheshi B, Holt G, Randall RL, Ferguson P, Wunder J, et al. Prophylactic antibiotic regimens in tumour surgery (PARITY): protocol for a multicentre randomised controlled study. BMJ Open. 2012;2:6.
Ghert M. Prophylactic antibiotic regimens in tumour surgery (PARITY): a pilot multicentre randomised controlled trial. Bone Joint Res. 2015;4:154–62.
Metzger-Filho O, de Azambuja E, Bradbury I, Saini KS, Bines J, Simon SD, et al. Analysis of regional timelines to set up a global phase III clinical trial in breast cancer: the adjuvant lapatinib and/or trastuzumab treatment optimization experience. Oncologist. 2013.
Aban IB, Wolfe GI, Cutter GR, Kaminski HJ, Jaretzki A, Minisman G, et al. The MGTX experience: challenges in planning and executing an international, multicenter clinical trial. J Neuroimmunol. 2008;18:134.
Klamerus JF, Bruinooge SS, Ye X, Klamerus ML, Damron D, Lansey D, et al. The impact of insurance on access to cancer clinical trials at a comprehensive cancer center. Clin Cancer Res. 2010;16:5997–6003.
Hamel LM, Penner LA, Albrecht TL, Heath E, Gwede CK, Eggly S. Barriers to clinical trial enrollment in racial and ethnic minority patients with cancer. Cancer Control. 2016;23:327–37.
Minisman G, Bhanushali M, Conwit R, Wolfe GI, Aban I, Kaminski HJ, et al. Implementing clinical trials on an international platform: challenges and perspectives. J Neurol Sci. 2012;313:1–6.
Furlan P, Mcfedries P. Excel 2010. compare a journal of comparative education. 2010.
Rosenthal R, Kasenda B, Dell-Kuster S, Von Elm E, You J, Blümle A, et al. Completion and publication rates of randomized controlled trials in surgery: an empirical study. Ann Surg. 2015;262:68–73.
Mouw TJ, Hong SW, Sarwar S, Fondaw AE, Walling AD, Al-Kasspooles M, et al. Discontinuation of surgical versus nonsurgical clinical trials: an analysis of 88,498 trials. J Surg Res. 2018;227:151–7.
Kasenda B, Von Elm E, You J, Blümle A, Tomonaga Y, Saccilotto R, et al. Prevalence, characteristics, and publication of discontinued randomized trials. JAMA. 2014;311:1045–52.
Abdullah L, Davis DE, Fabricant PD, Baldwin K, Namdari S. Is there truly “no significant difference”? Underpowered randomized controlled trials in the Orthopaedic literature. J Bone Joint Surg (Am Vol). 2014;97:2068–73.
McDonald AM, Knight RC, Campbell MK, Entwistle VA, Grant AM, Cook JA, et al. What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies. Trials. 2006;7:9.
Al-Shahi Salman R, Beller E, Kagan J, Hemminki E, Phillips RS, Savulescu J, et al. Increasing value and reducing waste in biomedical research regulation and management. Lancet. 2014;383:101–4.
Gentry KR, Arnup SJ, Disma N, Dorris L, de Graaff JC, Hunyady A, et al. Enrollment challenges in multicenter, international studies: the example of the GAS trial. Paediatric Anaesthesia. 2019;29:51–8.
Hemminki A, Kellokumpu-Lehtinen PL. Harmful impact of EU clinical trials directive. Br Med J. 2006;332:501–2.
Crow RA, Hart KA, McDermott MP, Tawil R, Martens WB, Herr BE, et al. A checklist for clinical trials in rare disease: obstacles and anticipatory actions—lessons learned from the FOR-DMD trial. Trials. 2018;19:291.
Yanagawa H, Akaishi A, Miyamoto T, Takai S, Nakanishi R, Irahara M. Role of clinical research coordinators in promoting clinical trials of drugs for surgical patients. Int Arch Med. 2008;1:1–4.
Katz S, Dufficy H, John C. Keys to success with clinical trials. Gastroenterol Hepatol. 2011:7;100.
Thoma A, Farrokhyar F, McKnight L, Bhandari M. Practical tips for surgical research: how to optimize patient recruitment. Can J Surg. 2010;53:205.
Kadam R, Borde S, Madas S, Salvi S, Limaye S. Challenges in recruitment and retention of clinical trial subjects. Perspect Clin Res. 2016;7:137.
We would like to acknowledge all of the PARITY co-investigators for their contributions and efforts in patient recruitment.
No external funding was received for the preparation of this manuscript. The PARITY trial was funded in part by the Physicians’ Services Incorporated, Canadian Cancer Society, Canadian Institutes of Health Research Operating Grant and the Orthopaedic Research and Education Foundation in partnership with the Musculoskeletal Tumor Society. The aforementioned funding bodies did not play a role in study design, data collection and interpretation, or writing of the manuscript.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Gazendam, A., Bozzo, A., Schneider, P. et al. Recruitment patterns in a large international randomized controlled trial of perioperative care in cancer patients. Trials 22, 219 (2021). https://doi.org/10.1186/s13063-021-05149-0