Consideration of factors of low accrual and methods for setting appropriate accrual periods: Japan Clinical Oncology Group study

Abstract

Background

Poor patient accrual can delay reporting of clinical trials and, consequently, the development of new treatments. For reducing the risk of additional resource requirements, a method for setting planned accrual periods with minimal deviation from the actual accrual periods is desirable. Risk factors for poor patient accrual and the appropriate method of estimating the required accrual period for timely completion of clinical trials were evaluated using the data of trials conducted by the Japan Clinical Oncology Group.

Methods

The study included 199 trials that started patient accrual between January 1, 1990, and June 30, 2021. The explanatory variables included factors that could be evaluated prior to trial commencement. We also evaluated whether the estimation methods for accrual pace could lead to completion within the planned accrual period.

Results

Approximately 23.6% of trials were completed within the planned accrual period. The risk factors for trial extension included planned accrual periods > 3 years (reference group: ≤ 3 years, odds ratio [OR] 0.37, 95% confidence interval [CI]: 0.15–0.92, P = 0.033) and stratified trial design (reference group: nonrandomized phase II trials, nonrandomized phase III trial [OR: 3.28, 95% CI: 0.99–10.9, P = 0.051], randomized phase II trial [OR: 3.91, 95% CI: 0.75–20.30, P = 0.105], and randomized phase III trial [OR: 9.29, 95% CI: 3.39–25.40, P < 0.001]). The method of estimating the accrual pace based on past clinical trials facilitated timely completion of the trial (OR: 3.51; 95% CI: 1.73–7.10, P < 0.001), unlike the estimation method based on survey evaluation of the accrual pace for participating institutions (OR: 1.12, 95% CI: 0.56–2.26, P = 0.751). Furthermore, the discrepancy between planned and actual accrual periods was minimal when using the methods of considering the accrual pace of past clinical trials.

Conclusions

Considering the accrual pace of past clinical trials is useful for estimating the required accrual period if data from past trials are available. When conducting a survey, it is necessary to be cautious of overestimating the cases at each facility.

Background

The extension of the clinical trial period and delayed reporting of findings can significantly burden researchers, increase the resources required, postpone new therapy development, and potentially diminish the value of the trial results because of changes in standardized treatment or the emergence of newer, better remedies. Patient accrual period is the period during which patients are enrolled in the clinical trial to reach the planned sample size. Poor patient accrual is the most common reason for delayed reporting of clinical trial results [1], leading to study discontinuation or sample size downsizing, insufficient statistical power, and compromised reliability of the findings [2]. Thus, adequately estimating accrual periods and developing appropriate recruitment strategies is crucial [3].

In their study examining clinical trials conducted by the National Clinical Trials Network (NCTN) and funded by the National Cancer Institute (NCI), Bennette et al. [4] found that factors increasing the risk of poor accrual included phase III trial designs (compared with those of phase II); the number of trials competing for eligible patients in the same population per year; studies examining nontargeted drugs, radiotherapy, trials without the use of investigational new drug; those evaluating multiple cancer types; and those focusing on major cancers. However, these findings were based on trials funded by a giant organization such as NCI [5], preventing extrapolation to those run by groups that did not adopt a similar implementation structure. Therefore, the current study investigated the risk factors associated with poor patient accrual in non-US clinical trials.

The Japan Clinical Oncology Group (JCOG), the largest clinical trial group in Japan, has conducted over 300 studies since its inception in 1990, many of which were prospective multicenter clinical trials for cancer [6]. The primary aim of the JCOG is to establish better standard treatments using a combination of drugs, surgery, endoscopy, and radiotherapy. It includes 16 research subgroups and focuses on all types of cancer, except for leukemia and pediatric cancer. The JCOG conducts clinical trials on cancer treatments in collaboration with approximately 190 medical institutions in Japan, with around 100 trials being currently active.

Adequate estimation of the sample size and required accrual period is essential for minimizing the risk of poor accrual and trial extension. Inaccurate sample size estimation may lead to patient enrollment difficulties, requirement of additional resources, or termination of the clinical trial contract. A method for setting planned accrual periods with minimal deviations from the actual accrual periods is needed to reduce the risk of inaccurate sample size estimation. There are several reported methods for the calculation of accrual pace, including (i) survey methods where each participating institution reports the number of patients they expect to enroll, (ii) estimating the accrual pace for the current trial based on the actual accrual pace in past trials in the same community, and (iii) estimating the accrual pace from data in cancer registries [7,8,9]. However, there is limited evidence on the most effective method. Therefore, the current study investigated the association between the method of estimating the planned accrual period and the likelihood of completing patient accrual as planned using data from the JCOG trials.

Methods

The current study included phase II, II/III, and III trials that began patient accrual between January 1, 1990, and June 30, 2021. Relevant data were collected from the study protocols, monitoring reports, and clinical study reports stored in the JCOG Data Center database. Trials for which protocols were not stored in the database or those that did not report the patient accrual period were excluded. The patient accrual pace of ongoing trials was evaluated up to December 31, 2021, allowing for a minimum grace period of 6 months from registration.

Risk factors for prolonged patient accrual period

The outcome measure of interest was whether patient accrual could be completed within the planned period (defined at the start of the clinical trial) without any extension. The accrual period was defined as the duration between the beginning and end of patient enrollment. For ongoing trials still recruiting patients on December 31, 2021, the accrual period was considered “not exceeded” if the patient accrual number matched or exceeded the number planned for that particular time point in the study. Additionally, for trials terminated early for any reason, the accrual period was considered “not extended” if the number of enrolled patients at the time of termination exceeded the originally planned enrollment. Despite protocol revisions or sample size downsizing, we proceeded with evaluating whether patient accrual was completed within the planned accrual period set at the start of the trial. Factors that could be estimated prior to the study commencement, including clinical trial phase, randomization, design type (superiority, noninferiority, or other), single- or multi-modal treatment, intervention type (surgery, drug, radiotherapy, or examination), presence of a competing trial at the start of patient accrual, intergroup trial, metastatic or nonmetastatic carcinoma, cancer type (“common and solid” or “rare and liquid”), planned sample size (≤ 100, 101–300, or > 300), planned accrual periods (≤ 3 or > 3 years), and the number of institutions at the start of patient accrual (≤ 30 or > 30), were set as explanatory variables as per previous reports published by the NCTN [4]. Variables that were difficult to assess or quantify in the JCOG trials, such as the annual number of eligible patients, number of clinical research coordinators, and complexity of eligibility criteria and treatment schedules, were excluded. Given that the JCOG trials typically do not provide case registration fees to institutions, research incentives were not included as a factor.

Supplementary risk factor analyses by the definition of prolonged patient accrual period used

Paul et al. defined low accrual trials as those that did not meet the planned accrual pace, whereas Bennette et al. defined them as those exhibiting < 50% of the planned accrual pace [4, 10]. Because these variations in definitions can affect the interpretation of the risk factors, supplementary risk factor analyses were conducted using the definition of low accrual trials as those completed at twice the planned pace (i.e., requiring accrual period extension), as per the definition proposed by Bennette et al. [4].

Method of estimating planned accrual periods

The JCOG trial protocols typically provide information on the methods used to estimate the planned accrual periods. These include survey evaluation of the accrual paces of the participating institutions and the consideration of patient accrual based on the actual accrual pace in past trials (both JCOG trials and domestic clinical trials, including some JCOG participating institutions) conducted in similar populations. It was evaluated whether the estimation methods for accrual pace could determine if accrual could be completed within the planned period. For example, researchers estimated an annual patient accrual of 50 cases through a “survey” and set the planned accrual period to 5 years with a sample size of 250 cases. We retrospectively investigated whether the accrual of 250 cases was completed within 5 years and what methods were used.

Additionally, the discrepancies between the planned and actual accrual periods were examined for each method. For trials that ended early, the actual accrual period was determined based on the accrual pace up to the point of termination.

Statistical analysis

Descriptive statistics were generated, with continuous variables reported as medians and interquartile ranges (IQR) and categorical variables reported as proportions. Intergroup comparisons of whether accrual could be completed within the planned period were performed using the Mann–Whitney U test for continuous values and the Fisher’s exact test or chi-square test with Bonferroni adjustment for categorical variables (two-sided P values reported). Risk factor analyses included multivariate logistic regression models developed using the backward elimination method [11]. The threshold in the model was α = 0.05.

Analysis of the methods of estimating the planned accrual periods included “consideration of the patient accrual of past trials” and “survey evaluation of the accrual pace of the participating institutions” as explanatory variables. Univariate and multivariate analyses were used to examine whether trials were successfully completed within the planned accrual period. All statistical analyses were conducted using EZR (version 1.55), which is the graphical user interface of the statistical software R (The R Foundation for Statistical Computing, Vienna, Austria) [12].

Results

The current study included 199 JCOG trials, of which 150 were completed and 49 were in progress. Among JCOG trials, 47 (23.6%) were completed within the planned accrual period or at the planned accrual pace, including 26 (17.7%) phase III trials. Additionally, 162 (81.4%) were completed within twice the planned accrual period or pace (Fig. 1). Thirty-one (15.6%) trials were terminated early for various reasons, with 10 (5.0%) due to low accrual (Fig. 2). Seventeen (8.5%) trials were forced to downsize their sample, requiring an accrual period extension in all cases. The median (IQR) values of the planned sample size, planned accrual periods, and number of institutions at the start of patient accrual were 230 (IQR: 105–355), 3.5 years (IQR: 3.0–5.0), and 34 (IQR: 25–44), respectively.

Fig. 1

Percentage of trials within the planned accrual period under each definition of “low accrual”

Fig. 2

Percentage of early termination trials and the corresponding reason

Risk factors for prolonged patient accrual period

The proportion of accrual completion in various factors is shown in Table 1. The median (IQR) values of the planned sample size, planned accrual periods, and number of institutions at the start of patient accrual for trials that were completed as planned were 140 (IQR: 58–310), 4.0 years (IQR: 2.3–5.0), and 32 (IQR: 24–44), respectively, while the corresponding values for those that were not completed as planned were 240 (IQR: 123–360), 3.2 years (IQR: 3.0–4.5), and 34 (IQR: 25–43), respectively.

Table 1 Characteristics for all phase II or III oncology trials conducted between 1990 and 2021 in JCOG

Because a large number of phase II trials were nonrandomized and the majority of phase III trials were randomized, stratified trial design was considered an explanatory variable and grouped into nonrandomized phase II (n = 40), nonrandomized phase III (n = 25), randomized phase II (n = 12), and randomized phase III trials (n = 122). The multivariate analyses identified planned accrual periods > 3 years (reference group: ≤ 3 years; odds ratio [OR]: 0.37; 95% confidence interval [CI]: 0.15–0.92, P = 0.033) and a stratified trial design (reference group: nonrandomized phase II trial; nonrandomized phase III trial OR: 3.28, 95% CI: 0.99–10.9, P = 0.051; randomized phase II trial OR: 3.91, 95% CI: 0.75–20.30, P = 0.105; and randomized phase III trial OR: 9.29, 95% CI: 3.39–25.40, P < 0.001) as significant risk factors (Table 2).

Table 2 Multivariable logistic regression model using backward elimination method

The supplementary multivariate analyses of trials in which patient accrual was completed within twice the planned period identified no significant risk factors.

Method used to estimate planned accrual periods

Table 3 shows the univariate and multivariate analyses of explanatory factors for timely completion of the trial. Sixty-three trials properly considered the accrual pace of past trials, whereas 100 trials used a survey evaluation of the accrual pace of institutions. Approximately 40% of trials that considered the accrual pace of past trials completed within the planned accrual period, whereas 16% did not consider the accrual pace of past trials. Additionally, 22% used survey evaluations of the accrual pace of institutions, whereas 25% did not. There were only 21 trials that adopted both methods, of which 8 completed accrual within the planned period (38%).

Table 3 Analysis of methods for estimating appropriate planned accrual periods

Consideration of the accrual pace of past trials was a significant determinant of adequately estimating the planned accrual period (n = 63, OR: 3.51, 95% CI: 1.73–7.10, P < 0.001). This was particularly applicable to studies considering the accrual pace of past JCOG trials (n = 50, OR: 3.46, 95% CI: 1.68–7.11, P < 0.001) but not for those considering the accrual pace of other community trials with some JCOG participating institutions (n = 13, OR: 1.48, 95% CI: 0.43–5.04, P = 0.532). Conversely, survey evaluation of the accrual pace of participating institutions was an insignificant determinant of adequate estimation of the accrual period (n = 100, OR: 1.12, 95% CI: 0.56–2.26, P = 0.751).

Table 4 shows the differences between the planned and actual accrual periods based on the calculation method used. The planned accrual periods for studies that did and did not consider the accrual pace of past trials were 3.3 and 3.0 years (P = 0.131), respectively, indicating that this method did not result in excessively prolonged planned accrual periods. The planned and actual accrual periods were 3.3 and 3.9 years, respectively, in trials considering the accrual pace of past trials, with a discrepancy of 0.6 years. The corresponding values for trials that did not consider the accrual pace of past trials were 3.0 and 5.1 years, with discrepancies of 2.1 years. Similarly, the planned and actual accrual periods were 3.0 and 4.8 years, respectively, in trials that included survey evaluation of the accrual pace of the participating institutions, with a discrepancy of 1.8 years. The corresponding values for those that did not include survey evaluation of the accrual pace of the participating institutions were 3.0 and 4.4 years, with discrepancies of 2.1 years (Table 4).

Table 4 Planned/actual accrual periods by method

The discrepancy between the planned and actual accrual periods was examined separately for phase II and III trials. Among the phase II trials, the discrepancy was 0.3 years for trials that considered the accrual pace of past trials and 1.0 years for those that did not. Among the phase III trials, the discrepancy was 0.7 years for trials that considered the accrual pace of past trials and 2.5 years for those that did not (Table 4).

Discussion

Study summary

In the current study, 23.6% of the examined JCOG trials were completed within the planned accrual period. Shorter planned accrual periods and the stratification of the trial design by phase and randomization were significant risk factors for trial extensions, whereas considering the accrual pace of past trials when calculating the required accrual period was a significant factor in ensuring the trial was completed in a timely manner.

Evaluation of trials that completed accrual in a timely manner

A previous study of adult phase III clinical trials conducted by the NCI-sponsored Clinical Trials Cooperative Group found that 27–31% of trials were terminated early because of poor accrual [2, 13]. Among trials supported by NCI-CTEP (n = 764), 81.5% (n = 623) failed to complete their target accrual within the planned period [14]. In the current study, 76.4% (152/199) of all included trials and 82.3% (121/147) of phase III trials did not complete accrual in a timely manner, and these findings are consistent with the trends reported by NCI-CTEP. Furthermore, 81.4% of JCOG trials were completed within twice the planned accrual period, consistent with the findings of Bennette et al. [4], where 81.6% of trials were not classified as poor accrual trials (defined as an accrual pace < 50% of the planned pace).

Approximately 5% of JCOG trials were terminated early because of poor accrual, which is a lower proportion than that reported by the NCI-sponsored Clinical Trials Cooperative Group (27–31%) [2, 13]. This may be attributed to the fact that the Data and Safety Monitoring Committee proposed a countermeasure for poor accrual and did not mandate termination because of poor accrual until recently. However, an early trial termination rule for trials with poor accrual was enforced by the JCOG in 2019 (Table 5). Additionally, over 20% of JCOG trials conducted between 2000 and 2018 would be subject to the termination rule.

Table 5 Early trial termination rule for trials with poor accrual in JCOG

Risk factors for poor accrual

The risk factor analysis showed that stratified trial designs (i.e., by randomization and phase) were a risk factor for poor accrual, and this finding was consistent with that reported by previous studies [4]. A phase-by-phase analysis showed that randomized phase III trials had a significantly higher risk of poor accrual than nonrandomized phase II trials. A key disadvantage of randomized trials compared with nonrandomized trials was that the former did not allow patients to choose their treatment, making obtaining consent difficult [15]. In the current study, nonrandomized phase III trials also showed a tendency toward poor accrual compared with nonrandomized phase II trials. As for the reason, in our study, a nonrandomized phase III trial design was often to be adopted in studies examining rare cancers, thereby making patient accrual challenging.

A shorter planned accrual period was another significant risk factor for trial extension, with underestimation of the planned accrual period being considered a possible cause. Extending the planned accrual period increases the chances of trial completion within the planned period. However, overestimating the planned accrual period may delay the publication of trial findings, potentially reducing its value, while underestimating it may make maintaining the necessary accrual pace challenging, thereby leading to trial extensions and an increase in associated costs. Therefore, exploring methods for accurately estimating the accrual pace is essential.

Certainty of the planned accrual period

Planned accrual period estimation should be determined based on the study sample size, feasibility of the clinical trial group, and participating institutions. The current study findings suggest that the actual accrual pace of past trials conducted by the same group should also be considered when calculating the required accrual period. Discrepancies between the planned and actual accrual periods may result in increased associated costs and potential delays in treatment development, emphasizing the importance of minimizing this discrepancy. In the current study, this discrepancy was only 0.6 and 2.1 years in trials that considered and did not consider the accrual pace of past trials, respectively, and this trend was particularly noticeable in phase III trials. Therefore, this method may be considered useful as it minimizes the discrepancies between the planned and actual accrual periods.

In contrast, considering survey evaluations of the accrual pace at participating institutions did not improve the accuracy of estimating planned accrual periods. Furthermore, discrepancies between planned and actual accrual periods were observed regardless of whether survey evaluation of the accrual pace of participating institutions was considered or not. This could include the possibility that the survey was not clearly conducted, such as not indicating the eligibility criteria using those trials, borderline patients who would not be enrolled in the actual study may have been counted, and the differences in the quality of each answer of institutions.

Future outlook

The results of this study could be applied to future clinical trials as follows. First, proactive interventions may be necessary for monitoring enrollment progress, especially in randomized phase III trials. Specifically, this involves investigating reasons for poor accrual, monitoring enrollment progress at participant institutions, and conducting meetings with researchers and research groups. Additionally, employing electronic methods and offering incentives may boost patient accrual.

Second, it is recommended the method of consideration of the accrual pace of past trials. However, if similar trials have not been conducted within the community, it may be beneficial to use accrual pace data of clinical trials from clinical trial registration systems, conduct surveys with rigorous and detailed methods, or refer to registration records from trials in other communities. Further research is needed to assess whether these methods can be effective for patient accrual and to better understand the factors contributing to accrual rate overestimation.

Limitation

The current study has several limitations. First, it was conducted in a single clinical trial group, which prevents generalization of the findings to wider groups. Second, the methods used for determining the planned accrual period could not be examined for trials that did not report this information in the study protocols, monitoring reports, or clinical study reports. Factors that were difficult to quantify in JCOG trials, as mentioned in the methods, were excluded. Third, this study did not examine the specific interventions being implemented. Finally, poor accrual can be partially addressed by modifying the eligibility criteria, but this study did not consider that approach.

Conclusion

The findings of the current study indicate that shorter planned accrual periods and randomized phase III trials increase the risk of trial extension. Furthermore, consideration of the accrual pace of past trials contributed to an accurate estimation of the required accrual period, unlike the use of survey evaluation of the accrual pace of the participating institutions that did not facilitate this. As such, caution should be taken against overestimating the number of cases at each facility when conducting a survey.