- Open Access
Importance of competing risks in the analysis of anti-epileptic drug failure
Trials volume 8, Article number: 12 (2007)
Retention time (time to treatment failure) is a commonly used outcome in antiepileptic drug (AED) studies.
Two datasets are used to demonstrate the issues in a competing risks analysis of AEDs. First, data collection and follow-up considerations are discussed with reference to information from 15 monotherapy trials. Recommendations for improved data collection and cumulative incidence analysis are then illustrated using the SANAD trial dataset. The results are compared to the more common approach using standard survival analysis methods.
A non-significant difference in overall treatment failure time between gabapentin and topiramate (logrank test statistic = 0.01, 1 degree of freedom, p-value = 0.91) masked highly significant differences in opposite directions with gabapentin resulting in fewer withdrawals due to side effects (Gray's test statistic = 11.60, 1 degree of freedom, p = 0.0007) but more due to poor seizure control (Gray's test statistic = 14.47, 1 degree of freedom, p-value = 0.0001). The significant difference in overall treatment failure time between lamotrigine and carbamazepine (logrank test statistic = 5.6, 1 degree of freedom, p-value = 0.018) was due entirely to a significant benefit of lamotrigine in terms of side effects (Gray's test statistic = 10.27, 1 degree of freedom, p = 0.001).
Treatment failure time can be measured reliably but care is needed to collect sufficient information on reasons for drug withdrawal to allow a competing risks analysis. Important differences between the profiles of AEDs may be missed unless appropriate statistical methods are used to fully investigate treatment failure time. Cumulative incidence analysis allows comparison of the probability of failure between two AEDs and is likely to be a more powerful approach than logrank analysis for most comparisons of standard and new anti-epileptic drugs.
Patients with a new diagnosis of epilepsy are usually prescribed an antiepileptic drug (AED) as monotherapy [1, 2]. An AED would be considered successful if the person taking it becomes seizure free with few side effects. Treatments that cause unacceptable side effects are likely to be changed to an alternative, whilst a treatment that fails to control seizures will either be changed to an alternative, or a second drug will be added. In order to assess this measure of failure in randomised trials (RCTs) of AEDs, a primary outcome recommended by the International League Against Epilepsy is retention time on AED , a composite outcome [4, 5], defined as the time from randomisation to the withdrawal of the randomised AED or addition of another.
Overall retention on treatment, i.e. time to treatment failure for any reason, may be analysed using standard survival analysis methods  and such analyses are common in the AED field in both studies of monotherapy [7–14] and add-on therapy [15, 16]. However this approach fails to examine different reasons for treatment failure (such as inadequate seizure control or adverse events), and assumes that reasons for failure are of equal importance which may not be the case. For example the loss of a driving license due to continued seizures will have differing social and economic consequences to the more common side effects such as nausea, dizziness or rash. A situation could arise in which two AEDs are considered equivalent as a result of similar overall treatment failure when in fact the drugs have very different effects on withdrawal due to side effects and poor seizure control.
In the statistical literature, the situation where there are several reasons why an event can occur is known as 'competing risks'. Ignoring this aspect of an outcome by analysing events overall can result in misleading conclusions . Some authors have examined separately the time to withdrawal due to side effects  and have censored patients whose allocated treatment is changed due to inadequate seizure control, which may give misleading results as analyses assume that the competing risks of withdrawal are independent . This assumption is questionable for AED treatment where there may be an association between these two causes of treatment failure. Thus a full investigation of retention time should include statistical methods that do not assume the competing risks of withdrawal are independent.
This paper considers the implications of competing risks of treatment failure for the design, data collection and analysis of AED studies motivated by analysis of the SANAD (Standard And New Anti-epileptic Drugs) trial  and drawing on our experience of datasets from our programme of individual patient data meta-analyses (IPDMA). We make recommendations for improving research practice.
Treatment failure time
The definition of treatment failure time used in this paper is the time from randomisation to the intention to withdraw the randomised drug due to lack of efficacy (poor seizure control) and/or intolerable side-effects; or the addition of other AEDs whichever is the earliest. The date of intention to withdraw rather than completion of withdrawal was chosen to reflect the point at which the treatment policy had been changed. Although retention time is the usual term for this outcome, a more appropriate alternative is time to treatment failure since it better reflects the event of interest. It also encompasses the situation when an AED is added as a result of poor seizure control and not just treatment failures resulting in the withdrawal of the original drug. Patients who achieve a period of seizure freedom (usually 2 to 3 years) may decide to withdraw treatment, in this circumstance the withdrawal reflects a successful outcome (seizure remission) rather than a treatment failure and the withdrawal is not counted as an event. Since remissions occur later in the follow-up period than withdrawals due to treatment failure, such censoring has no effect on the analysis.
Two datasets are used to demonstrate the issues in a competing risks analysis of AEDs. First, data collection, terminology and follow-up considerations are discussed with reference to information from a monotherapy meta-analysis. Recommendations for improved data collection and analysis methods are then illustrated using the SANAD dataset.
Individual patient data meta-analysis dataset
Information on treatment failure time was available from 15 randomised trials [18–31] involving 3883 individuals, collected as part of a suite of individual patient data meta-analyses concerning six different AEDs [7–14].
The design and analysis of this trial have been published elsewhere . For illustration here, we present the results from a competing risks analysis of the data for two pairwise comparisons, (i) gabapentin (GBP) versus topiramate (TPM) and (ii) lamotrigine (LTG) versus carbamazepine (CBZ).
Table 1 summarises the information collected on reason for treatment failure in the 15 trials where we had obtained the individual patient data. There was no standard approach for collecting this information and the level of detail recorded varied across the studies. For this reason, no analysis of competing risks has been undertaken for these data in the relevant meta-analyses. In some trials, only the primary reason for AED withdrawal was recorded. In others, several reasons were recorded along with order of importance. Occasionally, a further variable was recorded which indicated whether the clinician considered the reason for withdrawal to be related to the AED or not.
In some trials patients may have been recorded as having withdrawn for both side effects and poor seizure control. In clinical practice, if a patient is still experiencing seizures the dose of medication is increased until either seizures are controlled or side effects occur that prevent further dose increases. In the latter case, treatment will be withdrawn because of both inadequate seizure control and side effects. Since the former is the primary reason for failure, patients are classified as failures due to inadequate seizure control. Some patients will reach maximum doses without control of seizures and without significant side effects. Pragmatically, the clinician would call this inadequate seizure control.
Table 1 shows that the percentage of patients for whom the reason for treatment failure was unclear varied from 0 to 15%, but as a proportion of the number of withdrawals ranged as high as 70%. Recording the reason for AED withdrawal as non-compliance is not sufficient. Non-compliance may be as a result of side effects or possibly poor seizure control. Alternatively it may be due to a reason unrelated to the efficacy or tolerability of the drug. Thus to classify correctly the information in both an overall treatment failure analysis and a competing risks analysis requires more information to be obtained.
As a result of the problems identified in data collection in previous trials, extra detail regarding treatment failure reason was sought in the SANAD study. Table 2 shows the categories for reason for withdrawal recorded in SANAD, together with whether the withdrawal was coded as an event, and if so the reason for withdrawal (ISC or UAE), or as a censored observation in the competing risks analysis of treatment failure time.
Standard survival analysis methods such as Kaplan-Meier survival curve estimation, logrank hypothesis testing and Cox regression modelling can all be applied validly to the analysis of overall treatment failure time. However Kaplan-Meier estimates of being event free for a specific cause will be biased if the assumption that the competing risks are independent is violated. Kaplan-Meier curves are presented in this paper for illustration only however they cannot be interpreted in terms of survival probabilities in the presence of dependent censoring. We describe an alternative approach, cumulative incidence analysis, which makes no such assumption and allows the assessment of cause-specific withdrawal in the presence of other competing risks in the Appendix.
If all events of one type occur after the last event of another type, the standard survival analysis methods such as the logrank test and the Cox model will give identical results to those obtained following the cumulative incidence approach. Thus as a first stage of data analysis, it is important to understand the time distribution of the various causes for withdrawal.
Cumulative incidence analysis, including hypothesis testing, is available as part of the 'cmprsk' module  within the R software package. Version 2.1–5 was used for the following analyses.
Figure 1 shows how the distributions of withdrawal times due to side effects and poor seizure control in the SANAD trial overlap thus indicating the need to consider a cumulative incidence approach.
Figure 2 gives results for overall treatment failure for CBZ and LTG when the two causes, side effects and poor seizure control, are not distinguished as events. The logrank test statistic is 5.6 (1 df, p-value = 0.018) indicating LTG is better retained than CBZ. Figure 3 considers withdrawal for side effects as the only event, but censors withdrawals for poor seizure control at the time of withdrawal. The logrank test statistic is 10.4 (1 df, p-value = 0.001). There is clear evidence that LTG is better tolerated than CBZ with differences emerging very early on. Figure 4 considers withdrawal for poor seizure control as the only event, but censors withdrawals for side effects at the time of withdrawal. The logrank test statistic is 0.02 (1 df, p-value = 0.89) and shows LTG to be equivalent to CBZ in terms of seizure control.
Figure 5 gives results for overall treatment failure for GBP and TPM. The logrank test statistic is 0.01 (1 df, p-value = 0.91) indicating no overall difference between GBP and TPM. Figure 6 considers withdrawal for side effects as the only event, but censors withdrawals for poor seizure control at the time of withdrawal. The logrank test statistic is 9.6 (1 df, p-value = 0.002). There is clear evidence that GBP is better tolerated than TPM with differences emerging very early on. Figure 7 considers withdrawal for poor seizure control as the only event, but censors withdrawals for side effects at the time of withdrawal. The logrank test statistic is 9.3 (1 df, p-value = 0.002). There is clear evidence that TPM improves seizure control compared to GBP with differences emerging very early on.
Figures 8 and 9 show cumulative incidence plots for the two competing risks. The test statistic  comparing withdrawals due to side effects is 10.27 (1 df, p-value = 0.001). The test statistic comparing withdrawals due to poor seizure control is 0.78 (1 df, p-value = 0.38). The plots show the probabilities of suffering a treatment failure due to an adverse event and of suffering a failure due to inadequate seizure control in the setting where competing risks are acknowledged to exist. Thus at two years, for LTG the chance of withdrawal due to adverse events is 20% and of withdrawal due to inadequate seizure control is 15%, whilst for CBZ these probabilities are 30% and 13% respectively.
Figures 10 and 11 show cumulative incidence plots for the two competing risks. The test statistic  comparing withdrawals due to side effects is 11.60 (1 df, p-value = 0.0007). The test statistic comparing withdrawals due to poor seizure control is 14.47 (1 df, p-value = 0.0001). The cumulative incidence analysis provides stronger evidence of the advantage to GBP for side effects and TPM for seizure control compared to the logrank approach. The plots show the probabilities of suffering a treatment failure due to an adverse event and of suffering a failure due to inadequate seizure control in the setting where competing risks are acknowledged to exist. Thus at two years, for GBP the chance of withdrawal due to adverse events is 19% and of withdrawal due to inadequate seizure control is 32%, whilst for TPM these probabilities are 30% and 21% respectively.
Failure to investigate the competing risks of anti-epileptic drug withdrawal can lead to important differences in the clinical effect profiles of AEDs being missed. Thus overall treatment failure could be similar for two AEDs when the drugs have very different effects on withdrawal due to side effects and poor seizure control. For a particular patient, it is best to summarise each of these two risks for AEDs individually, as well as considering the overall risk of withdrawing for any reason. Further research is needed however to ascertain the relative importance of these different outcomes to individuals with epilepsy.
Prior to analysis, the timescale for competing causes should always be investigated. Risks operating over different time periods are less of an issue for the analysis. However, as demonstrated here, there is overlap of the timescale for withdrawals due to the different reasons and this is likely to be true for all AED studies.
More attention is required in the choice of statistical methods employed for the analysis of the competing risks of drug withdrawal. Cumulative incidence estimation allows appropriate inference about the probability of failure of AEDs allowing for the presence of competing reasons for such failure. This contrasts with the inappropriate inference that is often made from one minus the Kaplan-Meier estimate for a single failure reason which can only be interpreted as a probability of failure in the hypothetical situation that other failure reasons are not possible. Further, as suggested by a recent simulation study , the cumulative incidence approach has greater power to detect treatment differences than a logrank analysis in particular circumstances. This holds true for a difference in one direction for one cause and no or an opposite difference for the other cause, a situation worthy of note for anti-epileptic drug trials particularly when new and standard AEDs are being compared. Although conclusions from the two methods of analysis were similar for both pairwise comparisons shown here, this will not always be the case.
The reliability of classification of the cause of drug failure must be assured before a competing risks analysis is undertaken. The analysis of overall withdrawal is safe since the information that a drug has been withdrawn is usually reliable. For a competing risks analysis however, the analyst must consider whether the classification of the reason for withdrawal is reliable. The trials reviewed in this paper varied in the level of detail collected. In particular, data on the reason for withdrawal needs to be as accurate as possible for more explanatory research questions such as those posed in pharmacogenetic studies.
The dataset used for illustration here contained only two patients for whom the reason for drug withdrawal was recorded as non-compliance. Censoring the outcome for these two individuals implicitly assumes that the reason for non-compliance was not related to either seizure control or side effects. The results are likely to be robust to such an assumption in this example due to the low number of cases involved. However in other datasets where this reason is recorded more frequently, some sort of sensitivity analysis would be required to establish the robustness of the conclusions to this assumption. One approach would be to first code the withdrawal due to non-compliance as an event then as a censored observation and assess the robustness of the conclusions to such extreme assumptions. Of course ideally, as is the thrust of this paper, one would wish to minimise such problems through improved data collection methods.
The work presented here leads to the following recommendations for future studies which include treatment failure time as an outcome:
the outcome should be defined as the time to the intention to withdraw the randomised AED or add in another (i.e. the point at which the treatment policy has been changed),
sufficient detail should be collected on the primary reason for drug withdrawal/addition to allow classification into one of the following categories: withdrawal due to unacceptable side effects, withdrawal due to inadequate seizure control, withdrawal due to remission, withdrawal due to a reason confirmed to be unrelated to either side effects or seizure control.
The majority of drug withdrawals occurred by one year in the SANAD dataset although two-year follow-up provides greater power since almost all drug withdrawals occurred by this time. Monotherapy studies need longer follow-up however to investigate seizure control. Studies with both treatment failure and remission outcomes should intend to follow up each patient for a minimum of one year, and ensure that a reasonable number of patients will provide longer term follow-up, particularly important for seizure outcomes. Finally, we recommend that this outcome be termed 'time to treatment failure' rather than retention time since the former better reflects the event of interest.
The non-parametric cause-specific hazard function for cause l is estimated by maximum likelihood via
where d lj represents the number of failures of type l at time t j and n j is the number at risk at this time. The maximum likelihood estimator of the cause-specific incidence at time t j is
I l (t j ) = P(T <t j , L = l) = S(t j-1)h l (t j )
where P(T <t j , L = l) is the probability that an individual withdraws from drug due to cause l before t j and S(t j-1) denotes the overall survival function at time t j-1i.e. the probability that an individual does not withdraw for any reason before t j-1.
The cause-specific cumulative incidence function at time t is then
where the summation is over each cause-specific event time up to time t.
French JA, Kanner AM, Bautista J, Abou-Khalil B, Browne T, Harden CL, Theodore WH, Bazil C, Stern J, Schachter SC, Bergen D, Hirtz D, Montouris GC, Nespeca M, Gidal B, Marks WJ, Turk MR, Fischer JH, Bourgeois B, Wilner A, Faught RE, Sachdeo RC, Beydoun A, Glauser TA: Efficacy and tolerability of the new antiepileptic drugs in treatment of new onset epilepsy: report of the therapeutics and technology assessment subcommittee and quality standards subcommittee of the American academy of neurology and the American epilepsy society. Neurology. 2004, 62: 1252-60.
National Collaborating Centre for Primary Care: The epilepsies: the diagnosis and management of the epilepsies in adults and children in primary and secondary care – NICE guideline. 2004, London: National Institute for Clinical Excellence
Commission on antiepileptic drugs: Considerations on designing clinical trials to evaluate the place of new antiepileptic drugs in the treatment of newly diagnosed and chronic patients with epilepsy. Epilepsia. 1998, 39: 799-803. 10.1111/j.1528-1157.1998.tb01167.x.
Chadwick D: Better comparisons of antiepileptic drugs: what measures of efficacy?. Pharm World Sci. 1997, 19: 214-216. 10.1023/A:1008642623460.
Perucca E: Evaluation of drug treatment outcome in epilepsy: a clinical perspective. Pharm World Sci. 1997, 19: 217-222. 10.1023/A:1008698807530.
Kalbfleisch JD, Prentice RL: The statistical analysis of failure time data. 1980, Wiley, New York
Marson AG, Williamson PR, Hutton JL, Clough HE, Chadwick DW: Carbamazepine versus valproate monotherapy for epilepsy (Cochrane Review). The Cochrane Library. 2000, Oxford: Update Software, 3
Taylor S, Tudur Smith C, Williamson PR, Marson AG: Phenobarbitone versus phenytoin monotherapy for partial onset seizures and generalized onset tonic-clonic seizures (Cochrane Review). The Cochrane Library. 2001, Oxford: Update Software, 4
Tudur Smith C, Marson AG, Williamson PR: Phenytoin versus valproate monotherapy for partial onset seizures and generalized onset tonic-clonic seizures (Cochrane Review). The Cochrane Library. 2001, Oxford: Update Software, 4
Tudur Smith C, Marson AG, Clough HE, Williamson PR: Carbamazepine versus phenytoin monotherapy for epilepsy (Cochrane Review). The Cochrane Library. 2002, Oxford: Update Software, 2
Tudur Smith C, Marson AG, Williamson PR: Carbamazepine versus phenobarbitone monotherapy for epilepsy (Cochrane Review). The Cochrane Library. 2003, Oxford: Update Software, 1
Gamble CL, Marson AG, Williamson PR: Lamotrigine versus carbamazepine monotherapy for epilepsy (Protocol for a Cochrane Review). The Cochrane Library. 2004, Oxford: Update Software, 2
Muller MM, Marson AG, Williamson PR: Oxcarbazepine versus phenytoin monotherapy for epilepsy (Protocol for a Cochrane Review). The Cochrane Library. 2004, Oxford: Update Software, 2
Tudur Smith C, Marson AG, Williamson PR: Valproate versus phenobarbitone monotherapy for epilepsy [Cochrane protocol]. The Cochrane Library. 2004, Oxford: Update Software, 1
Lhatoo SD, Wong ICK, Polizzi G, Sander JWAS: Long-term retention rates of lamotrigine, gabapentin, and topiramate in chronic epilepsy. Epilepsia. 2000, 41: 1592-1596. 10.1111/j.1528-1157.2000.tb00165.x.
Lhatoo SD, Wong ICK, Sander JWAS: Prognostic factors affecting long-term retention of topiramate in patients with chronic epilepsy. Epilepsia. 2000, 41: 338-341. 10.1111/j.1528-1157.2000.tb00165.x.
Marson AG, Al-Kharusi AM, Alwaidh M, Appleton R, Baker GA, Chadwick DW, Cramp C, Cockerell OC, Cooper PN, Doughty J, Eaton B, Gamble C, Goulding PJ, Howell SJL, Hughes A, Jackson M, Jacoby A, Kellett M, Lawson GR, Leach JP, Nicolaides P, Roberts R, Shackley P, Shen J, Smith DF, Smith PEM, Tudur Smith C, Vanoli A, Williamson PR, on behalf of the SANAD Study group: Carbamazepine, gabapentin, lamotrigine, oxcarbazepine or topiramate for partial epilepsy: results from the SANAD trial. Lancet.
Heller AJ, Chesterman P, Elwes RD, Crawford P, Chadwick D, Johnson AL, Reynolds EH: Phenobarbitone, phenytoin, carbamazepine, or sodium valproate for newly diagnosed adult epilepsy: a randomised comparative monotherapy trial. JNNP. 1995, 58: 44-50.
de Silva M, MacArdle B, McGowan M, Hughes E, Stewart J, Reynolds EH, Hughes E, Neville BGR, Johnson AL: Randomised comparative monotherapy trial of phenobarbitone, phenytoin, carbamazepine, or sodium valproate for newly diagnosed childhood epilepsy. The Lancet. 1996, 347: 709-13. 10.1016/S0140-6736(96)90074-4.
Mattson RH, Cramer JA, Collins JF, Smith DB, Delgado-Escueta AV, Browne TR, Williamson PD, Treiman DM, McNamara JO, McCutchen CB, Homan RW, Crill WE, Lubozynski MF, Rosenthal NP, Mayersdorf A: Comparison of carbamazepine, phenobarbital, phenytoin, and primidone in partial and secondarily generalized tonic-clonic seizures. NEJM. 1985, 313: 145-51.
Mattson RH, Cramer JA, Collins JF: A comparison of valproate with carbamazepine for the treatment of complex partial seizures and secondarily generalized tonic-clonic seizures in adults. NEJM. 1992, 327: 765-71.
Richens A, Davidson DLW, Cartlidge NEF, Easter DJ: A multicentre comparative trial of sodium valproate and carbamazepine in adult onset epilepsy. JNNP. 1994, 57: 682-687.
Verity CM, Hosking G, Easter DJ: A multicentre comparative trial of sodium valproate and carbamazepine in paediatric epilepsy. Dev Med Child Neur. 1995, 37: 97-108.
Brodie MJ, Richens A, Yuen AWC: Double-blind comparison of lamotrigine and carbamazepine in newly diagnosed epilepsy. The Lancet. 1995, 345: 476-79. 10.1016/S0140-6736(95)90581-2.
Ramsey RE, Wilder BJ, Murphy JV, Holmes GL, Uthman B: Efficacy and safety of valproic acid versus phenytoin as sole therapy for newly diagnosed primary generalized tonic-clonic seizures. Epilepsy. 1992, 5: 55-60. 10.1016/S0896-6974(05)80021-0.
Turnbull DM, Howel D, Rawlins MD, Chadwick DW: Which drug for the adult epileptic patient: phenytoin or valproate?. BMJ. 1985, 290: 815-819.
Placencia M, Sander JWAS, Shorvon SD, Roman M, Alarcon F, Bimos C, Cascante S: Antiepileptic drug treatment in a community health care setting in northern Ecuador: a prospective 12-month assessment. Epilepsy Res. 1993, 14: 237-244. 10.1016/0920-1211(93)90048-C.
Brodie MJ, Overstall PW, Giorgi L: Multicentre, double-blind, randomised comparison between lamotrigine and carbamazepine in elderly patients with newly diagnosed epilepsy. Epilepsy Res. 1999, 37: 81-87. 10.1016/S0920-1211(99)00039-X.
Bill PA, Vigonius U, Pohlmann H, Guerreiro CA, Kochen S, Saffer D, Moore A: A double-blind controlled clinical trial of oxcarbazepine versus phenytoin in adults with previously untreated epilepsy. Epilepsy Res. 1997, 27: 195-204. 10.1016/S0920-1211(97)00024-7.
Guerreiro MM, Vigonius U, Pohlmann H, de Manreza ML, Fejerman N, Antoniuk SA, Moore A: A double-blind controlled trial of oxcarbazepine versus phenytoin in children and adolescents with epilepsy. Epilepsy Res. 1997, 27: 205-213. 10.1016/S0920-1211(97)00025-9.
Nieto-Barrera M, Brozmanova M, Capovilla G, Christe W, Pedersen B, Kane K, O'Neill F: A comparison of monotherapy with lamotrigine or carbamazepine in patients with newly diagnosed partial epilepsy. Epilepsy Res. 2001, 46: 145-155. 10.1016/S0920-1211(01)00271-6.
Gray RJ: Subdistribution analysis of competing risks. Version 2.1–5. 2004, http://www.r-project.org
Williamson PR, Kolamnunnage-Dona R, Tudur Smith C: The influence of competing risks setting on the choice of hypothesis test for treatment effect. Biostatistics. 2006, doi: 10.1093/biostatistics/kxl040
The authors thank the IPDMA monotherapy trialists' group and the SANAD collaborators for providing their data.
PRW identified the issue of competing risks in epilepsy trials, investigated appropriate analysis methods, directed the analyses, interpreted results and wrote the manuscript. CTS carried out the statistical analyses, interpreted results and contributed to writing the manuscript. JWS provided data for analysis, provided clinical input and commented on the manuscript. AGM provided data for analysis, provided clinical input and commented on the manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
About this article
Cite this article
Williamson, P.R., Smith, C.T., Sander, J.W. et al. Importance of competing risks in the analysis of anti-epileptic drug failure. Trials 8, 12 (2007). https://doi.org/10.1186/1745-6215-8-12
- Treatment Failure
- Cumulative Incidence
- Seizure Control
- Compete Risk Analysis