Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Quality of transurethral resection of bladder tumor procedure influenced a phase III trial comparing the effect of KLH and mitomycin C

  • Narasimha N. G. Prasad1Email author and
  • Shammana N. Muddukrishna2
Trials201718:123

DOI: 10.1186/s13063-017-1843-5

Received: 9 August 2016

Accepted: 16 February 2017

Published: 14 March 2017

Abstract

Background

Retrospective analysis of Center effect of the multi-center trial conducted to compare Immucothel (KLH Immunotherapy drug product) with Mytomycin-C (MM) concluded that efficacy evaluation of the drug product may be impacted by physician’s subjective performance of Transurethral resection of bladder tumor (TURBT).

Methods

A randomized trial was performed in 18 hospitals (clinical centers) and a total of 553 recruited, 283 patients under KLH arm and 270 patients under MM. An initial statistical analysis of efficacy comparisons between KLH and MM based on log-rank test performed for each center (hospital) showed 6 hospitals out of 18 hospitals a p-value of <0.05 and remaining 12 hospitals showed a p-values of >0.05. No association was observed between number of patients analysed and the associated p-values across hospitals. Final statistical analyses were carried out under each drug product using Kaplan-Meier survival analysis along with log-rank test after combining all eligible patients data for 6 hospital group and 12 hospital group.

Results

Median recurrence free survival (RFS) times (in weeks) showed statistical significance (p-value = 0.03) between two groups of hospitals under KLH arm, while similar median values showed no statistical significance (p-value = 0.05).

Conclusion

Center effect with respect to median RFS values under KLH was more pronounced than under MM. Under the presence of such center effect, for reasons other than product related effects, concluding superiority of one drug product over another may create confounding bias conclusions in multi-center clinical trials. In the above cited clinical trial study, physician’s prior experience on TURBT might have contributed to center effect in examining efficacies of KLH and MM. Similar observation was also noted in the literature on studies dealing with TURBT and in other clinical studies.

Trial registration

Data set used in this study is based on previously documented clinical trial in the literature: See (Lammers et al., J Clin Oncol 30:2273–9, 2012) and Acknowledgments.

Background

Intravesical bacillus Calmette-Guérin (BCG) instillation remains the standard treatment among intravesical anticancer therapies after transurethral resection (TURBT) for bladder cancer, to suppress disease recurrence and progression [1]. Since the toxicity of BCG is a limitation in some patients, other alternative intravesical chemotherapeutic agents have been developed. One such cytostatic agent is mitomycin C (MM). IMMUCOTHEL® (Keyhole Limpet Hemocyanin (KLH) Immunotherapy drug product, biosyn, Fellbach, Germany) is an approved drug product for the prevention of bladder carcinoma recurrence after TURBT and after failure of other established therapies, like BCG and MM.

A randomized phase III trial based on 553 patients (283 patients under KLH and 270 patients under MM), with intermediate- and high-risk non-muscle-invasive bladder cancer (NMIBC) without carcinoma in situ has been performed and the study details are published [2]. The multicenter trial spanned over 18 clinical centers (hospitals). In the phase III trial study patients were randomized into the KLH arm or the MM arm after TURBT. In the KLH arm, patients received 1 mg KLH preimmunization followed by instillations with 20 mg KLH once per week for 6 weeks and once per month for 10 months for a total targeted 16 instillations as part of the KLH treatment. In the MM arm, patients received 40 mg MM once per week for 4 weeks and followed by five monthly instillations, and then at 9 and 12 months for a total targeted 11 instillations. The efficacy comparison between KLH and MM was based on median recurrence-free survival (RFS) and recurrence rate [2].

The present work focus is on exploratory evaluation of a treatment-by-centre interaction effect through statistical significance tests for treatment effects within each centre based on ICH Guidance [3].

Material and Methods

Patient demographic and clinical characteristics of the multicenter phase III clinical trial have already been published by Lammers et al. (Table 1) [2]. Initial analysis of the 18 individual hospitals’ data was performed using the by Kaplan-Meier method and hazard ratio (HR) using the Cox PH model. The statistical analysis is summarized in Table 1.
Table 1

Summary of recurrences, p value (log-rank) and hazard ratio (HR) values (Cox PH) for 18 trial centers

Hospitals with p < 0.05 highlighted in gray

*HR value larger than 2.26 (over all HR value based on all 18 hospitals)

**HR value not available since no recurrences in the mitomycin C group

KLH Keyhole Limpet Hemocyanin, MM mitomycin C, TURBT transurethral resection of bladder tumor

The overall HR (18 centers) is 2.26 as reported by Lammers et al. [2] and confirmed in our analysis. The hospital grouping based on HR and p values shows a fairly good overall correlation. All the hospitals (5 out of 5, 100%) with p < 0.05 have a HR value greater than 2.26 and majority of hospital (9 out of 11, 81%) with p > 0.05 have a HR value less than 2.26. The HR values for two hospitals, 17 and 18, are not reported as there was no recurrence in MM treatment arm (Table 1). In view of these observations subsequent analysis was performed using hospital grouping based on p values.

In Table 1, the p value for six hospitals (shaded in gray) was statistically significant; p value < 0.05. Those six centers were designated as group 1. The remaining 12 hospitals had a p value > 0.05 and were pooled in group 2. The patient demographic and clinical characteristics of all patients for group 1 and group 2 are summarized in Table 2.
Table 2

Patient demographic and clinical characteristics of all patients within the two hospital groups

Characteristic

KLH

MM

Total

Group 2

Group 1

Group 2

Group 1

KLH

MM

N

%

N

%

N

%

N

%

  

Number of patients

172

64.4

95

35.6

166

64.8

90

35.2

267

256

Sex

 Male

141

82

77

81.1

133

80.1

77

85.6

218

210

 Female

31

18

18

18.9

33

19.9

13

14.4

49

46

 Mean age in years

37.6

 

36.8

 

35.6

 

47.9

   

Tumor status

 Primary

100

58.1

44

46.3

102

61.4

44

48.9

144

146

 Recurrent

72

41.9

51

53.7

64

38.6

46

51.1

123

110

Previous treatment in patients with recurrent tumors at baseline

 Current single post-TURBT intravesical chemotherapy

2

 

3

 

1

 

1

 

5

2

 Previous single post-TURBT intravesical chemotherapy

8

 

2

 

2

 

1

 

10

3

 Local adjuvant intravesical therapy

20

 

11

 

12

 

15

 

31

27

 Other therapy

3

       

3

3

 No therapy other than TURBT

39

 

35

 

49

 

29

 

74

78

Tumor stage

 pTa

132

76.7

84

88.4

113

68.1

71

78.9

216

184

 pT1

40

23.3

11

11.6

53

31.9

19

21.1

51

72

Tumor grade

 1

54

31.4

34

35.8

47

28.3

32

35.6

88

79

 2

95

55.2

53

55.8

87

52.4

47

52.2

148

134

 3

23

13.4

8

8.4

32

19.3

11

12.2

31

43

Number of tumors

 Single

60

34.9

29

34.9

57

34.3

26

28.9

89

83

 Multiple

112

65.1

66

69.5

108

65.1

64

71.1

178

172

 Unknown

    

1

    

1

KLH Keyhole Limpet Hemocyanin, MM mitomycin C, TURBT transurethral resection of bladder tumor

Group 1 represents six hospitals, which show a statistically significant difference (p value < 0.05) between the two treatment groups. The data in group 1 consists of 95 patients in the KLH treatment arm and 90 patients for MM. Group 2 represents the remaining 12 hospitals, which displayed no statistically significant difference (p value > 0.05). Group 2 includes 172 patients for KLH and 166 for MM.

The median RFS and recurrence rates for KLH and MM were derived by Kaplan-Meier survival analysis. Comparison of median RFS using the log-rank test was performed for the same drug product (KLH or MM) between group 1 and group 2.

Results and Discussion

Significant differences between trial centers

The analysis of the 18 individual trial centers showed significant differences for six hospitals (p < 0.05) for both treatment arms. Therefore, those six centers were designated as group 1 and the remaining 12 hospitals were pooled in group 2 (Table 1). In total, 95 patients (35.6%) received KLH and 90 (35.2%) patients were treated with MM in group 1. In group 2, 172 patients (64.4%) were treated with KLH and 166 (64.8%) received MM (Table 3). Patient demographic and clinical characteristics are shown in Table 2. The data suggests that the distribution of patients for KLH and MM arms is similar between group 1 and group 2.
Table 3

Frequency of subjects and number of recurrences in the two hospital groups

Treatment received

Hospital groupa

Total N

N of events (# of recurrences)

Recurrences in [%]

KLH

Group 1

95

70

74

Group 2

172

93

54

Overall

267

163

61

Mitomycin C

Group 1

90

23

26

Group 2

166

64

39

Overall

256

87

34

aGroup 1 = 6 hospitals, Group 2 = 12 hospitals

KLH Keyhole Limpet Hemocyanin

Recurrence rate and median RFS were significantly different within treatment arms

The recurrence rate and median RFS for KLH and MM in group 1 and group 2 are summarized in Table 4. For the KLH treatment arm, 74% (70 out of 95 patients) developed a recurrence in group 1 as compared to 54% (93 out of 172 patients) in group 2 (p value = 0.03, chi-square test). For the MM treatment arm, 26% (23 out of 90 patients) developed a recurrence in group 1 as compared to 39% (64 out of 166 patients) in group 2 (p value = 0.05, chi-squared test).
Table 4

Recurrence rate and median recurrence-free survival (RFS) for Keyhole Limpet Hemocyanin (KLH) and mitomycin C (MM) for group 1 and group 2

Treatment received

Hospital groupa

Recurrence rate (%)

Median (weeks)

95% confidence interval

Lower bound

Upper bound

KLH

Group 1

74

84

64

103

Group 2

54

139

100

178

Mitomycin C

Group 1

26

N/Ab

N/Ab

N/Ab

Group 2

39

297

152

442

aGroup 1 = 6 hospitals, Group 2 = 12 hospitals

bNot attainable

The median RFS for KLH in group 1 is 84 weeks compared to 139 weeks for median RFS for KLH in group 2. The median RFS for group 1 is significantly lower than group 2 (p = 0.004, log-rank test). Similarly, the median RFS for group 1 is not attained for MM compared to a median RFS value of 297 for group 2 (approaching significance p = 0.05, log-rank test).

Similar results for recurrence rate and medium RFS were obtained when the hospital grouping was performed based on HR values (low <2.26 and high >2.26) (details not reported; see Additional files 1 and 2).

These results imply that the same drug product has better efficacy in one group compared to the other. Logically, this would be an unlikely scenario and suggests that other center-related effects may be involved in this study.

Significant influence of prior TURBT in multi-center trial

The study design called for TURBT prior to randomizing the patients into each treatment arm. Existing medical literature data suggests that outcome of TURBT with regard to recurrence is significantly influenced by the operative experience of the surgeon [410]. The results of this analysis provide additional evidence to the referenced existing medical literature data. The observed differences between group 1 and group 2 hospitals were possibly due to differences in the operative experience of the surgeons.

What are the possible approaches that can be taken in the design of clinical trials to overcome this TURBT-related variability in evaluating new drug product efficacies? One approach could be recruiting patients who have undergone at least one TURBT procedure. In this regard, we have performed a subgroup analysis of patients receiving 16 KLH instillations or 11 MM instillations and who had prior recurrent tumors (n = 156; KLH = 67 and MM = 89). The results showed that the percentage of recurrences for KLH-treated patients was 54% while the percentage of recurrence for MM treated patients was 38%. Statistical evaluation of product efficacy (chi-squared test) indicated that the two groups are not significantly different (p = 0.107).

Further supporting data has been reported by Lammers et al. [2]. Subgroup analysis performed for patients with recurrent disease and a history of intravesical treatments, the number of patients with recurrences and time to recurrence was not statistically different for the KLH and MM treatment groups.

Different efficacy profile for KLH and mitomycin C exacerbate influence of TURBT on therapy outcome

KLH is a nonspecific immunotherapeutic agent and unlike chemotherapeutics, such as mitomycin C, the immunotherapeutic effect to take hold requires some lead time to produce a positive outcome in patients and, as such, the treatment course includes 16 instillations over a 12-month time period. KLH (IMMUCOTHEL®) is approved as a second-line treatment. Therefore, subgroup analysis was performed for patients with recurrent disease and a complete treatment course who had received an additional treatment after TURBT for prior tumor. The number of patients with recurrences (45% versus 52%) and time to recurrence (235 versus 98 weeks) was not statistically different for the KLH and MM treatment groups (p ~ 1.0, respectively, p = 0.549) (Table 5).
Table 5

Recurrence rate and median recurrence-free survival (RFS) for Keyhole Limpet Hemocyanin (KLH) and mitomycin C (MM) in patients with recurrence and complete treatment course who received additional treatment after transurethral resection of bladder tumor (TURBT) for prior tumor (KLH second-line treatment)

 

Recurrence rate (%)

Median (weeks)

95% confidence interval

Lower bound

Upper bound

KLH

48

235

35

435

Mitomycin C

52

98

75

121

Overall

50

210

69

351

In general, it appears that the recurrence rate between institutions or multiple clinical studies for bladder cancer patients treated with any of the currently used drug products (BCG and MM included) has a large variability [2, 4, 9].

Conclusion

The results of the prospective, randomized, multicenter, phase III clinical trial, “Intracutaneous and Intravesical Immunotherapy with Keyhole Limpet Hemocyanin Compared with Intravesical Mitomycin in Patients with Non-Muscle-Invasive Bladder Cancer,” have been published [2]. The authors concluded that KLH has a different safety profile and is inferior to MM in preventing non-muscle-invasive bladder cancer.

This retrospective analysis of the same clinical data focused on evaluating center effects on product efficacy. From the statistical analysis of the individual centers, we identified two sets of hospital groups based on the p values (p < 0.05 and p > 0.05). The results indicate that the majority of the hospitals showed no statistical difference between KLH and MM with respect to median RFS. This suggests the presence of center effects in this clinical trial as evidenced by statistical comparison of the same drug product (KLH or MM) between the two hospital groups. The conclusion of this data would be that the same drug product is superior to itself between the two centers. As this is an illogical conclusion; among other possible determining factors, the TURBT procedure is suspected to be the main contributing factor. The patient data summary in Tables 1 and 2 suggests that there is no selection bias arising due to clinical and demographic patient characteristics. Such selection bias would normally be a prime determining factor but is not present in this well-designed clinical trial. Similar suspicions have been raised by other reports as it relates to variable observed recurrence rates between institutions. This report specifically expands on that concept to show that it has an effect on the efficacy determination of drug products.

This suggests that when performing new drug efficacy evaluation studies involving surgery as a prerequisite step, as is the case with bladder cancer, the operative experience of the surgeon [10, 11] is an additional important factor for consideration in designing future clinical studies.

In view of such differences in the efficacy of the same product used on patients in two groups of hospitals, proper care and procedures need to be taken in generalizing product efficacies. As suggested by Richsterstetter et al. [8] and Brausi et al. [9], suitable statistical procedures need to be utilized to account for interaction effect between product efficacy and multicenter effects before generalizing results of the clinical study. This additional confounding effect has to be taken into consideration when designing clinical trials with new drug products involving surgery as a prerequisite treatment step.

Once a product is approved, surgeons optimize surgical technique for a given drug product to provide improved clinical outcomes for their patients. Such modifications may be done unintentionally. As a result, when surgeons perform clinical studies using new product candidates, there may be an inherent bias in their surgical techniques toward their preferred choice of drug product.

Abbreviations

BCG: 

Bacillus Calmette-Guérin

KLH: 

Keyhole Limpet Hemocyanin

MM: 

Mytomycin C

RFS: 

Recurrence-free survival

TURBT: 

Transurethral resection of bladder tumor

Declarations

Acknowledgements

Authors thank Wim P.J. Witjes, MD, PhD, Scientific and Clinical Research Director, CuraTrial SMO and Research, EAU Central Research Office, Radboud University, Nijmegen Medical Center, Nijmegen, The Netherlands and Christien T.M. Caris for providing the clinical trial data in SPSS format.

Funding

There is no direct funding for the project. However, a statistical consulting fee was given to NP.

Availability of data and materials

This is an independent secondary analysis of the dataset from Lammers RJM, Witjes WPJ, Janzing-Pastors MHD, Caris CTM, Witjes JA. Intracutaneous and Intravesical Immunotherapy with Keyhole Limpet Hemocyanin Compared with Intravesical Mitomycin in Patients with Non-Muscle-Invasive Bladder Cancer: Results From a Prospective Randomized Phase III Trial. J Clin Oncol. 2012 Jun 20;30(18):2273-9.

Authors’ contributions

NP performed statistical analysis and manuscript preparation. SM was responsible for clinical interpretation, coordination and manuscript preparation. Both authors have read and approved the final manuscript related to this submission.

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Yes.

Ethics approval and consent to participate

Not applicable.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Department of Mathematical and Statistical Sciences, University of Alberta
(2)
Biosyn Corporation

References

  1. Babjuk M, Burger M, Zigeuner R, Shariat SF, van Rhijn BWG, Compérat E, et al. EAU guidelines on non-muscle-invasive urothelial carcinoma of the bladder: update 2013. Eur Urol. 2013;64(4):639–53.View ArticlePubMedGoogle Scholar
  2. Lammers RJM, Witjes WPJ, Janzing-Pastors MHD, Caris CTM, Witjes JA. Intracutaneous and intravesical immunotherapy with keyhole limpet hemocyanin compared with intravesical mitomycin in patients with non-muscle-invasive bladder cancer: results from a prospective randomized phase III trial. J Clin Oncol. 2012;30(18):2273–9.View ArticlePubMedGoogle Scholar
  3. Guidance for Industry – E9 Statistical Principles for Clinical Trials. U.S. Department of Health and Human Services Food and Drug Administration, Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research. Rockville: ICH;1998.
  4. Jurincic CD, Engelmann U, Gasch J, Klippel KF. Immunotherapy in bladder cancer with keyhole-limpet hemocyanin: a randomized study. J Urol. 1988;139:723–6.PubMedGoogle Scholar
  5. Allard CB, Meyer CP, Gandaglia G, Chang SL, Chun FKH, Gelpi-Hammerschmidt F, et al. The effect of resident involvement on perioperative outcomes in transurethral urologic surgeries. J Surg Educ. 2015;72(5):1018–25.
  6. Jancke G, Rosell J, Jahnson S. Impact of surgical experience on recurrence and progression after transurethral resection of bladder tumour in non-muscle-invasive bladder cancer. Scand J Urol. 2014;48(3):276–83.View ArticlePubMedGoogle Scholar
  7. Mariappan P, Finney SM, Head E, Somani BK, Zachou A, Smith G, et al. Good quality white-light transurethral resection of bladder tumours (GQ-WLTURBT) with experienced surgeons performing complete resections and obtaining detrusor muscle reduces early recurrence in new non-muscle-invasive bladder cancer: validation across time and place and recommendation for benchmarking. BJU Int. 2012;109(11):1666–73.View ArticlePubMedGoogle Scholar
  8. Richterstetter M, Wullich B, Amann K, Haeberle L, Engehausen DG, Goebell PJ, et al. The value of extended transurethral resection of bladder tumour (TURBT) in the treatment of bladder cancer: the value of extended TURBT for treating bladder cancer. BJU Int. 2012;110(2b):E76–9.View ArticlePubMedGoogle Scholar
  9. Brausi M, Collette L, Kurth K, van der Meijden AP, Oosterlinck W, Witjes JA, et al. Variability in the recurrence rate at first follow-up cystoscopy after TUR in stage Ta T1 transitional cell carcinoma of the bladder: a combined analysis of seven EORTC studies. Eur Urol. 2002;41(5):523–31.View ArticlePubMedGoogle Scholar
  10. Rolevich AI, Minich AA, Nabebina TI, Polyakov SL, Krasny SA, Sukonko OG. Surgeon has a major impact on long-term recurrence risk in patients with non-muscle invasive bladder cancer. Cent European J Urol. 2016;69:170–7.PubMedPubMed CentralGoogle Scholar
  11. Hsu C-E, Lin T-K, Lee M-H, Lee S-T, Chang C-N, Lin C-L, et al. The impact of surgical experience on major intraoperative aneurysm rupture and their consequences on outcome: a multivariate analysis of 538 microsurgical clipping cases. PLoS One. 2016;11(3):e0151805. doi:10.1371/journal.pone.0151805.View ArticlePubMedPubMed CentralGoogle Scholar

Copyright

© The Author(s). 2017