This article has Open Peer Review reports available.
Importance of competing risks in the analysis of anti-epileptic drug failure
© Williamson et al; licensee BioMed Central Ltd. 2007
Received: 21 November 2006
Accepted: 29 March 2007
Published: 29 March 2007
Retention time (time to treatment failure) is a commonly used outcome in antiepileptic drug (AED) studies.
Two datasets are used to demonstrate the issues in a competing risks analysis of AEDs. First, data collection and follow-up considerations are discussed with reference to information from 15 monotherapy trials. Recommendations for improved data collection and cumulative incidence analysis are then illustrated using the SANAD trial dataset. The results are compared to the more common approach using standard survival analysis methods.
A non-significant difference in overall treatment failure time between gabapentin and topiramate (logrank test statistic = 0.01, 1 degree of freedom, p-value = 0.91) masked highly significant differences in opposite directions with gabapentin resulting in fewer withdrawals due to side effects (Gray's test statistic = 11.60, 1 degree of freedom, p = 0.0007) but more due to poor seizure control (Gray's test statistic = 14.47, 1 degree of freedom, p-value = 0.0001). The significant difference in overall treatment failure time between lamotrigine and carbamazepine (logrank test statistic = 5.6, 1 degree of freedom, p-value = 0.018) was due entirely to a significant benefit of lamotrigine in terms of side effects (Gray's test statistic = 10.27, 1 degree of freedom, p = 0.001).
Treatment failure time can be measured reliably but care is needed to collect sufficient information on reasons for drug withdrawal to allow a competing risks analysis. Important differences between the profiles of AEDs may be missed unless appropriate statistical methods are used to fully investigate treatment failure time. Cumulative incidence analysis allows comparison of the probability of failure between two AEDs and is likely to be a more powerful approach than logrank analysis for most comparisons of standard and new anti-epileptic drugs.
Patients with a new diagnosis of epilepsy are usually prescribed an antiepileptic drug (AED) as monotherapy [1, 2]. An AED would be considered successful if the person taking it becomes seizure free with few side effects. Treatments that cause unacceptable side effects are likely to be changed to an alternative, whilst a treatment that fails to control seizures will either be changed to an alternative, or a second drug will be added. In order to assess this measure of failure in randomised trials (RCTs) of AEDs, a primary outcome recommended by the International League Against Epilepsy is retention time on AED , a composite outcome [4, 5], defined as the time from randomisation to the withdrawal of the randomised AED or addition of another.
Overall retention on treatment, i.e. time to treatment failure for any reason, may be analysed using standard survival analysis methods  and such analyses are common in the AED field in both studies of monotherapy [7–14] and add-on therapy [15, 16]. However this approach fails to examine different reasons for treatment failure (such as inadequate seizure control or adverse events), and assumes that reasons for failure are of equal importance which may not be the case. For example the loss of a driving license due to continued seizures will have differing social and economic consequences to the more common side effects such as nausea, dizziness or rash. A situation could arise in which two AEDs are considered equivalent as a result of similar overall treatment failure when in fact the drugs have very different effects on withdrawal due to side effects and poor seizure control.
In the statistical literature, the situation where there are several reasons why an event can occur is known as 'competing risks'. Ignoring this aspect of an outcome by analysing events overall can result in misleading conclusions . Some authors have examined separately the time to withdrawal due to side effects  and have censored patients whose allocated treatment is changed due to inadequate seizure control, which may give misleading results as analyses assume that the competing risks of withdrawal are independent . This assumption is questionable for AED treatment where there may be an association between these two causes of treatment failure. Thus a full investigation of retention time should include statistical methods that do not assume the competing risks of withdrawal are independent.
This paper considers the implications of competing risks of treatment failure for the design, data collection and analysis of AED studies motivated by analysis of the SANAD (Standard And New Anti-epileptic Drugs) trial  and drawing on our experience of datasets from our programme of individual patient data meta-analyses (IPDMA). We make recommendations for improving research practice.
Treatment failure time
The definition of treatment failure time used in this paper is the time from randomisation to the intention to withdraw the randomised drug due to lack of efficacy (poor seizure control) and/or intolerable side-effects; or the addition of other AEDs whichever is the earliest. The date of intention to withdraw rather than completion of withdrawal was chosen to reflect the point at which the treatment policy had been changed. Although retention time is the usual term for this outcome, a more appropriate alternative is time to treatment failure since it better reflects the event of interest. It also encompasses the situation when an AED is added as a result of poor seizure control and not just treatment failures resulting in the withdrawal of the original drug. Patients who achieve a period of seizure freedom (usually 2 to 3 years) may decide to withdraw treatment, in this circumstance the withdrawal reflects a successful outcome (seizure remission) rather than a treatment failure and the withdrawal is not counted as an event. Since remissions occur later in the follow-up period than withdrawals due to treatment failure, such censoring has no effect on the analysis.
Two datasets are used to demonstrate the issues in a competing risks analysis of AEDs. First, data collection, terminology and follow-up considerations are discussed with reference to information from a monotherapy meta-analysis. Recommendations for improved data collection and analysis methods are then illustrated using the SANAD dataset.
Individual patient data meta-analysis dataset
Information on treatment failure time was available from 15 randomised trials [18–31] involving 3883 individuals, collected as part of a suite of individual patient data meta-analyses concerning six different AEDs [7–14].
The design and analysis of this trial have been published elsewhere . For illustration here, we present the results from a competing risks analysis of the data for two pairwise comparisons, (i) gabapentin (GBP) versus topiramate (TPM) and (ii) lamotrigine (LTG) versus carbamazepine (CBZ).
Classification of reasons for treatment change in 15 monotherapy trials
Still on drug at end of follow-up (censored)
Reason for withdrawal
Seizure control and adverse events
Total number of patients in trial
Heller et al., 199518
De Silva et al., 199619
Mattson et al., 198520
Mattson et al., 199221
Richens et al., 199422
Verity et al., 199523
Brodie et al., 1995a24
Brodie et al., 1995b24
Ramsay et al., 199225
Turnbull et al., 198526
Placencia et al., 199327
Brodie et al., 199928
Bill et al., 199729
Guerreiro et al., 199730
Nieto-Barrera et al., 200131
In some trials patients may have been recorded as having withdrawn for both side effects and poor seizure control. In clinical practice, if a patient is still experiencing seizures the dose of medication is increased until either seizures are controlled or side effects occur that prevent further dose increases. In the latter case, treatment will be withdrawn because of both inadequate seizure control and side effects. Since the former is the primary reason for failure, patients are classified as failures due to inadequate seizure control. Some patients will reach maximum doses without control of seizures and without significant side effects. Pragmatically, the clinician would call this inadequate seizure control.
Table 1 shows that the percentage of patients for whom the reason for treatment failure was unclear varied from 0 to 15%, but as a proportion of the number of withdrawals ranged as high as 70%. Recording the reason for AED withdrawal as non-compliance is not sufficient. Non-compliance may be as a result of side effects or possibly poor seizure control. Alternatively it may be due to a reason unrelated to the efficacy or tolerability of the drug. Thus to classify correctly the information in both an overall treatment failure analysis and a competing risks analysis requires more information to be obtained.
Classification of reasons for treatment change used in the SANAD trial
Reason for withdrawal from drug or other drug added (earliest event)
Categorised as event (type classified as ISC or UAE) or censored in 'time to treatment failure' analysis
Number of patients (CBZ, LTG, TPM and GBP)
Inadequate seizure control
Unacceptable adverse events
Perceived risk of adverse effect
Remission of epilepsy categorised by patient 1 (less than 12 months' remission from seizures)
Remission of epilepsy categorised by clinician (any length) or patient (more than 12 months' remission from seizures)
Study withdrawal – consent withdrawn 2
Death (unrelated to epilepsy) 3
Death (related to epilepsy) 3
Moved from area
Patient non-compliant or patient decision 4
Pregnant or planning pregnancy
Still on drug at last follow-up
Standard survival analysis methods such as Kaplan-Meier survival curve estimation, logrank hypothesis testing and Cox regression modelling can all be applied validly to the analysis of overall treatment failure time. However Kaplan-Meier estimates of being event free for a specific cause will be biased if the assumption that the competing risks are independent is violated. Kaplan-Meier curves are presented in this paper for illustration only however they cannot be interpreted in terms of survival probabilities in the presence of dependent censoring. We describe an alternative approach, cumulative incidence analysis, which makes no such assumption and allows the assessment of cause-specific withdrawal in the presence of other competing risks in the Appendix.
If all events of one type occur after the last event of another type, the standard survival analysis methods such as the logrank test and the Cox model will give identical results to those obtained following the cumulative incidence approach. Thus as a first stage of data analysis, it is important to understand the time distribution of the various causes for withdrawal.
Cumulative incidence analysis, including hypothesis testing, is available as part of the 'cmprsk' module  within the R software package. Version 2.1–5 was used for the following analyses.
Failure to investigate the competing risks of anti-epileptic drug withdrawal can lead to important differences in the clinical effect profiles of AEDs being missed. Thus overall treatment failure could be similar for two AEDs when the drugs have very different effects on withdrawal due to side effects and poor seizure control. For a particular patient, it is best to summarise each of these two risks for AEDs individually, as well as considering the overall risk of withdrawing for any reason. Further research is needed however to ascertain the relative importance of these different outcomes to individuals with epilepsy.
Prior to analysis, the timescale for competing causes should always be investigated. Risks operating over different time periods are less of an issue for the analysis. However, as demonstrated here, there is overlap of the timescale for withdrawals due to the different reasons and this is likely to be true for all AED studies.
More attention is required in the choice of statistical methods employed for the analysis of the competing risks of drug withdrawal. Cumulative incidence estimation allows appropriate inference about the probability of failure of AEDs allowing for the presence of competing reasons for such failure. This contrasts with the inappropriate inference that is often made from one minus the Kaplan-Meier estimate for a single failure reason which can only be interpreted as a probability of failure in the hypothetical situation that other failure reasons are not possible. Further, as suggested by a recent simulation study , the cumulative incidence approach has greater power to detect treatment differences than a logrank analysis in particular circumstances. This holds true for a difference in one direction for one cause and no or an opposite difference for the other cause, a situation worthy of note for anti-epileptic drug trials particularly when new and standard AEDs are being compared. Although conclusions from the two methods of analysis were similar for both pairwise comparisons shown here, this will not always be the case.
The reliability of classification of the cause of drug failure must be assured before a competing risks analysis is undertaken. The analysis of overall withdrawal is safe since the information that a drug has been withdrawn is usually reliable. For a competing risks analysis however, the analyst must consider whether the classification of the reason for withdrawal is reliable. The trials reviewed in this paper varied in the level of detail collected. In particular, data on the reason for withdrawal needs to be as accurate as possible for more explanatory research questions such as those posed in pharmacogenetic studies.
The dataset used for illustration here contained only two patients for whom the reason for drug withdrawal was recorded as non-compliance. Censoring the outcome for these two individuals implicitly assumes that the reason for non-compliance was not related to either seizure control or side effects. The results are likely to be robust to such an assumption in this example due to the low number of cases involved. However in other datasets where this reason is recorded more frequently, some sort of sensitivity analysis would be required to establish the robustness of the conclusions to this assumption. One approach would be to first code the withdrawal due to non-compliance as an event then as a censored observation and assess the robustness of the conclusions to such extreme assumptions. Of course ideally, as is the thrust of this paper, one would wish to minimise such problems through improved data collection methods.
the outcome should be defined as the time to the intention to withdraw the randomised AED or add in another (i.e. the point at which the treatment policy has been changed),
sufficient detail should be collected on the primary reason for drug withdrawal/addition to allow classification into one of the following categories: withdrawal due to unacceptable side effects, withdrawal due to inadequate seizure control, withdrawal due to remission, withdrawal due to a reason confirmed to be unrelated to either side effects or seizure control.
The majority of drug withdrawals occurred by one year in the SANAD dataset although two-year follow-up provides greater power since almost all drug withdrawals occurred by this time. Monotherapy studies need longer follow-up however to investigate seizure control. Studies with both treatment failure and remission outcomes should intend to follow up each patient for a minimum of one year, and ensure that a reasonable number of patients will provide longer term follow-up, particularly important for seizure outcomes. Finally, we recommend that this outcome be termed 'time to treatment failure' rather than retention time since the former better reflects the event of interest.
The non-parametric cause-specific hazard function for cause l is estimated by maximum likelihood via
where d lj represents the number of failures of type l at time t j and n j is the number at risk at this time. The maximum likelihood estimator of the cause-specific incidence at time t j is
I l (t j ) = P(T <t j , L = l) = S(t j-1)h l (t j )
where P(T <t j , L = l) is the probability that an individual withdraws from drug due to cause l before t j and S(t j-1) denotes the overall survival function at time t j-1i.e. the probability that an individual does not withdraw for any reason before t j-1.
The cause-specific cumulative incidence function at time t is then
where the summation is over each cause-specific event time up to time t.
The authors thank the IPDMA monotherapy trialists' group and the SANAD collaborators for providing their data.
- French JA, Kanner AM, Bautista J, Abou-Khalil B, Browne T, Harden CL, Theodore WH, Bazil C, Stern J, Schachter SC, Bergen D, Hirtz D, Montouris GC, Nespeca M, Gidal B, Marks WJ, Turk MR, Fischer JH, Bourgeois B, Wilner A, Faught RE, Sachdeo RC, Beydoun A, Glauser TA: Efficacy and tolerability of the new antiepileptic drugs in treatment of new onset epilepsy: report of the therapeutics and technology assessment subcommittee and quality standards subcommittee of the American academy of neurology and the American epilepsy society. Neurology. 2004, 62: 1252-60.View ArticlePubMedGoogle Scholar
- National Collaborating Centre for Primary Care: The epilepsies: the diagnosis and management of the epilepsies in adults and children in primary and secondary care – NICE guideline. 2004, London: National Institute for Clinical ExcellenceGoogle Scholar
- Commission on antiepileptic drugs: Considerations on designing clinical trials to evaluate the place of new antiepileptic drugs in the treatment of newly diagnosed and chronic patients with epilepsy. Epilepsia. 1998, 39: 799-803. 10.1111/j.1528-1157.1998.tb01167.x.View ArticleGoogle Scholar
- Chadwick D: Better comparisons of antiepileptic drugs: what measures of efficacy?. Pharm World Sci. 1997, 19: 214-216. 10.1023/A:1008642623460.View ArticlePubMedGoogle Scholar
- Perucca E: Evaluation of drug treatment outcome in epilepsy: a clinical perspective. Pharm World Sci. 1997, 19: 217-222. 10.1023/A:1008698807530.View ArticlePubMedGoogle Scholar
- Kalbfleisch JD, Prentice RL: The statistical analysis of failure time data. 1980, Wiley, New YorkGoogle Scholar
- Marson AG, Williamson PR, Hutton JL, Clough HE, Chadwick DW: Carbamazepine versus valproate monotherapy for epilepsy (Cochrane Review). The Cochrane Library. 2000, Oxford: Update Software, 3Google Scholar
- Taylor S, Tudur Smith C, Williamson PR, Marson AG: Phenobarbitone versus phenytoin monotherapy for partial onset seizures and generalized onset tonic-clonic seizures (Cochrane Review). The Cochrane Library. 2001, Oxford: Update Software, 4Google Scholar
- Tudur Smith C, Marson AG, Williamson PR: Phenytoin versus valproate monotherapy for partial onset seizures and generalized onset tonic-clonic seizures (Cochrane Review). The Cochrane Library. 2001, Oxford: Update Software, 4Google Scholar
- Tudur Smith C, Marson AG, Clough HE, Williamson PR: Carbamazepine versus phenytoin monotherapy for epilepsy (Cochrane Review). The Cochrane Library. 2002, Oxford: Update Software, 2Google Scholar
- Tudur Smith C, Marson AG, Williamson PR: Carbamazepine versus phenobarbitone monotherapy for epilepsy (Cochrane Review). The Cochrane Library. 2003, Oxford: Update Software, 1Google Scholar
- Gamble CL, Marson AG, Williamson PR: Lamotrigine versus carbamazepine monotherapy for epilepsy (Protocol for a Cochrane Review). The Cochrane Library. 2004, Oxford: Update Software, 2Google Scholar
- Muller MM, Marson AG, Williamson PR: Oxcarbazepine versus phenytoin monotherapy for epilepsy (Protocol for a Cochrane Review). The Cochrane Library. 2004, Oxford: Update Software, 2Google Scholar
- Tudur Smith C, Marson AG, Williamson PR: Valproate versus phenobarbitone monotherapy for epilepsy [Cochrane protocol]. The Cochrane Library. 2004, Oxford: Update Software, 1Google Scholar
- Lhatoo SD, Wong ICK, Polizzi G, Sander JWAS: Long-term retention rates of lamotrigine, gabapentin, and topiramate in chronic epilepsy. Epilepsia. 2000, 41: 1592-1596. 10.1111/j.1528-1157.2000.tb00165.x.View ArticlePubMedGoogle Scholar
- Lhatoo SD, Wong ICK, Sander JWAS: Prognostic factors affecting long-term retention of topiramate in patients with chronic epilepsy. Epilepsia. 2000, 41: 338-341. 10.1111/j.1528-1157.2000.tb00165.x.View ArticlePubMedGoogle Scholar
- Marson AG, Al-Kharusi AM, Alwaidh M, Appleton R, Baker GA, Chadwick DW, Cramp C, Cockerell OC, Cooper PN, Doughty J, Eaton B, Gamble C, Goulding PJ, Howell SJL, Hughes A, Jackson M, Jacoby A, Kellett M, Lawson GR, Leach JP, Nicolaides P, Roberts R, Shackley P, Shen J, Smith DF, Smith PEM, Tudur Smith C, Vanoli A, Williamson PR, on behalf of the SANAD Study group: Carbamazepine, gabapentin, lamotrigine, oxcarbazepine or topiramate for partial epilepsy: results from the SANAD trial. Lancet.Google Scholar
- Heller AJ, Chesterman P, Elwes RD, Crawford P, Chadwick D, Johnson AL, Reynolds EH: Phenobarbitone, phenytoin, carbamazepine, or sodium valproate for newly diagnosed adult epilepsy: a randomised comparative monotherapy trial. JNNP. 1995, 58: 44-50.Google Scholar
- de Silva M, MacArdle B, McGowan M, Hughes E, Stewart J, Reynolds EH, Hughes E, Neville BGR, Johnson AL: Randomised comparative monotherapy trial of phenobarbitone, phenytoin, carbamazepine, or sodium valproate for newly diagnosed childhood epilepsy. The Lancet. 1996, 347: 709-13. 10.1016/S0140-6736(96)90074-4.View ArticleGoogle Scholar
- Mattson RH, Cramer JA, Collins JF, Smith DB, Delgado-Escueta AV, Browne TR, Williamson PD, Treiman DM, McNamara JO, McCutchen CB, Homan RW, Crill WE, Lubozynski MF, Rosenthal NP, Mayersdorf A: Comparison of carbamazepine, phenobarbital, phenytoin, and primidone in partial and secondarily generalized tonic-clonic seizures. NEJM. 1985, 313: 145-51.View ArticlePubMedGoogle Scholar
- Mattson RH, Cramer JA, Collins JF: A comparison of valproate with carbamazepine for the treatment of complex partial seizures and secondarily generalized tonic-clonic seizures in adults. NEJM. 1992, 327: 765-71.View ArticlePubMedGoogle Scholar
- Richens A, Davidson DLW, Cartlidge NEF, Easter DJ: A multicentre comparative trial of sodium valproate and carbamazepine in adult onset epilepsy. JNNP. 1994, 57: 682-687.Google Scholar
- Verity CM, Hosking G, Easter DJ: A multicentre comparative trial of sodium valproate and carbamazepine in paediatric epilepsy. Dev Med Child Neur. 1995, 37: 97-108.View ArticleGoogle Scholar
- Brodie MJ, Richens A, Yuen AWC: Double-blind comparison of lamotrigine and carbamazepine in newly diagnosed epilepsy. The Lancet. 1995, 345: 476-79. 10.1016/S0140-6736(95)90581-2.View ArticleGoogle Scholar
- Ramsey RE, Wilder BJ, Murphy JV, Holmes GL, Uthman B: Efficacy and safety of valproic acid versus phenytoin as sole therapy for newly diagnosed primary generalized tonic-clonic seizures. Epilepsy. 1992, 5: 55-60. 10.1016/S0896-6974(05)80021-0.View ArticleGoogle Scholar
- Turnbull DM, Howel D, Rawlins MD, Chadwick DW: Which drug for the adult epileptic patient: phenytoin or valproate?. BMJ. 1985, 290: 815-819.View ArticlePubMedPubMed CentralGoogle Scholar
- Placencia M, Sander JWAS, Shorvon SD, Roman M, Alarcon F, Bimos C, Cascante S: Antiepileptic drug treatment in a community health care setting in northern Ecuador: a prospective 12-month assessment. Epilepsy Res. 1993, 14: 237-244. 10.1016/0920-1211(93)90048-C.View ArticlePubMedGoogle Scholar
- Brodie MJ, Overstall PW, Giorgi L: Multicentre, double-blind, randomised comparison between lamotrigine and carbamazepine in elderly patients with newly diagnosed epilepsy. Epilepsy Res. 1999, 37: 81-87. 10.1016/S0920-1211(99)00039-X.View ArticlePubMedGoogle Scholar
- Bill PA, Vigonius U, Pohlmann H, Guerreiro CA, Kochen S, Saffer D, Moore A: A double-blind controlled clinical trial of oxcarbazepine versus phenytoin in adults with previously untreated epilepsy. Epilepsy Res. 1997, 27: 195-204. 10.1016/S0920-1211(97)00024-7.View ArticlePubMedGoogle Scholar
- Guerreiro MM, Vigonius U, Pohlmann H, de Manreza ML, Fejerman N, Antoniuk SA, Moore A: A double-blind controlled trial of oxcarbazepine versus phenytoin in children and adolescents with epilepsy. Epilepsy Res. 1997, 27: 205-213. 10.1016/S0920-1211(97)00025-9.View ArticlePubMedGoogle Scholar
- Nieto-Barrera M, Brozmanova M, Capovilla G, Christe W, Pedersen B, Kane K, O'Neill F: A comparison of monotherapy with lamotrigine or carbamazepine in patients with newly diagnosed partial epilepsy. Epilepsy Res. 2001, 46: 145-155. 10.1016/S0920-1211(01)00271-6.View ArticlePubMedGoogle Scholar
- Gray RJ: Subdistribution analysis of competing risks. Version 2.1–5. 2004, http://www.r-project.org Google Scholar
- Williamson PR, Kolamnunnage-Dona R, Tudur Smith C: The influence of competing risks setting on the choice of hypothesis test for treatment effect. Biostatistics. 2006, doi: 10.1093/biostatistics/kxl040Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.