Study protocol | Open | Published:
European COMPARative Effectiveness research on blended Depression treatment versus treatment-as-usual (E-COMPARED): study protocol for a randomized controlled, non-inferiority trial in eight European countries
Trialsvolume 17, Article number: 387 (2016)
Effective, accessible, and affordable depression treatment is of high importance considering the large personal and economic burden of depression. Internet-based treatment is considered a promising clinical and cost-effective alternative to current routine depression treatment strategies such as face-to-face psychotherapy. However, it is not clear whether research findings translate to routine clinical practice such as primary or specialized mental health care. The E-COMPARED project aims to gain knowledge on the clinical and cost-effectiveness of blended depression treatment compared to treatment-as-usual in routine care.
E-COMPARED will employ a pragmatic, multinational, randomized controlled, non-inferiority trial in eight European countries. Adults diagnosed with major depressive disorder (MDD) will be recruited in primary care (Germany, Poland, Spain, Sweden, and the United Kingdom) or specialized mental health care (France, The Netherlands, and Switzerland). Regular care for depression is compared to “blended” service delivery combining mobile and Internet technologies with face-to-face treatment in one treatment protocol. Participants will be followed up at 3, 6, and 12 months after baseline to determine clinical improvements in symptoms of depression (primary outcome: Patient Health Questionnaire-9), remission of depression, and cost-effectiveness. Main analyses will be conducted on the pooled data from the eight countries (n = 1200 in total, 150 participants in each country).
The E-COMPARED project will provide mental health care stakeholders with evidence-based information and recommendations on the clinical and cost-effectiveness of blended depression treatment.
France: ClinicalTrials.gov NCT02542891. Registered on 4 September 2015; Germany: German Clinical Trials Register DRKS00006866. Registered on 2 December 2014; The Netherlands: Netherlands Trials Register NTR4962. Registered on 5 January 2015; Poland: ClinicalTrials.Gov NCT02389660. Registered on 18 February 2015; Spain: ClinicalTrials.gov NCT02361684. Registered on 8 January 2015; Sweden: ClinicalTrials.gov NCT02449447. Registered on 30 March 2015; Switzerland: ClinicalTrials.gov NCT02410616. Registered on 2 April 2015; United Kingdom: ISRCTN registry, ISRCTN12388725. Registered on 20 March 2015.
Depression is a common mental disorder with a negative impact on mental well-being, quality of life, and social and work-related functioning both in the short and longer term . Additionally, depression is associated with increased morbidity, mortality, health care utilization, and health care costs [2–4]. On a population level, depression is one of the most costly diseases. In the European Union, 30 million citizens are affected by depression and the economic costs of depression were estimated at €92 billion in 2010 and are still rising [5, 6]. The majority of these costs is caused by lost productivity . In combination with the fact that depression is highly prevalent among the working-age population , depression poses a significant societal and economic burden to European society.
Depression can be treated effectively with pharmacological treatment, psychotherapy, especially cognitive-behavioral therapy (CBT), or a combination of both [8–12]. Antidepressants are widely prescribed, both in primary and specialized mental health care settings, while non-pharmacologic treatments, such as psychotherapeutic interventions, are offered to a lesser degree in most of Europe, and are virtually absent in primary care in some countries such as Portugal . Moreover, despite the availability of effective treatments, the proportion of adults with depression who actually seek or receive treatment is limited with estimates ranging between 35 and 45 % in higher-income countries [14, 15] indicating that the treatment of depression leaves ample room for improvement. Thus, European health care systems face the challenge of improving access to cost-effective treatments to relieve the burden of depression .
Internet-based depression treatment is considered a promising clinical and cost-effective alternative to current routine depression treatment strategies such as face-to-face psychotherapy. Ample research has demonstrated the clinical effectiveness of Internet-based treatment for depression when delivered with minimal guidance in controlled research settings  with effect sizes that are comparable to face-to-face interventions . It is also unclear whether Internet-based depression treatments are cost-effective, which is considered an important next step in research . In addition, it is not yet clear to what extent these findings translate to routine clinical practice such as primary care or specialized mental health care. The patient populations in these settings are more heterogeneous than those in controlled research settings and the delivery of services in these settings is more complex.
Blended depression treatment is a relatively new treatment format where online and face-to-face interventions are integrated into one treatment protocol [20, 21]. Blended depression treatment formats are of particular interest to primary and specialized treatment centers for several reasons [22, 23]. Patients often present with complex clinical problems and may need more intensive guidance than Internet-based treatments alone are able to offer . Blended treatment delivery would allow therapists to focus on process-related treatment components during the face-to-face sessions, such as patient-specific needs, discussion of thoughts and feelings, and treatment evaluation, while more practical therapy aspects can be delivered in the online sessions (e.g., psychoeducation, homework, and symptom monitoring) [25, 26]. In addition, treatment in specialized mental health care settings is costly. Both in primary and specialized mental health care, patients often return for follow-up consultations which provides an opportunity to introduce blended treatments. It is expected that implementation of blended Internet-based treatments may reduce the number of face-to-face sessions needed while mental health gains remain the same or even increase. Thus, there are a number of advantages associated with blended depression treatment which may improve the efficiency of the health care system on the whole if these are implemented on a wider scale.
Comparative effectiveness research (CER) is designed to inform health care decisions with the aim of improving patient outcomes, quality of life, and cost-containment. The overall objective of E-COMPARED is to provide mental health care stakeholders, including patients, health care professionals, health insurers, mental health service providers, and policy-makers, with evidence-based information and recommendations about the clinical effectiveness and cost-effectiveness of blended depression treatment in comparison with treatment-as-usual (TAU) in Europe. The study will be conducted in eight European countries with different diversities and geographical spread including countries that are frontrunners in the field of Internet-based treatments (The Netherlands, Sweden, and the United Kingdom (UK)), countries in which the field has been evolving rapidly (Germany, Spain, and Switzerland), and countries with very little expertise and experience in this area (France and Poland). Thus, the outcome of the project will take into account these different situations and the results will inform the routine practice of specific mental health service organizations in Europe. The aim of this paper is to describe the protocol of the multinational study that evaluates the effectiveness and cost-effectiveness of blended cognitive behavioral therapy (bCBT) for adults with major depressive disorder (MDD) in comparison with TAU in Europe both in primary and specialized mental health care settings. It is hypothesized that bCBT is clinically non-inferior (i.e., not less effective) as compared to TAU, but that it is cost-effective since we expect that less therapist time is needed to deliver treatment resulting in lower treatment costs.
This is a two-arm, randomized controlled, non-inferiority trial, with an economic evaluation alongside, in eight countries in Europe (France, Germany, The Netherlands, Poland, Spain, Sweden, Switzerland, and the UK). The non-inferiority design was chosen to test whether bCBT is not less effective than TAU . The trials will be conducted in routine primary care (sites: Germany, Poland, Spain, Sweden, and the UK) or specialized mental health care (sites: France, The Netherlands, and Switzerland). Respondents in both conditions will be followed until 12 months after baseline (measures will be taken at baseline, 3 months, 6 months, and 12 months).
Recruitment procedures are country- and setting-specific (see Table 1). See Fig. 1 for the flow of patients through the study and attachments (Additional files 1 and 2) for an overview of study procedures and a checklist in line with Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) . In all countries, the following inclusion and exclusion criteria will be applied uniformly.
Inclusion criteria: (1) being 18 years of age or older, (2) meeting Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) diagnostic criteria for MDD as confirmed by the telephone-administered MINI International Neuropsychiatric Interview (M.I.N.I) version 5.0 , and (3) minimal to severe symptoms of depression based on a score of 5 or higher on the Patient Health Questionnaire-9 (PHQ-9) screening questionnaire .
Exclusion criteria: (1) current high risk for suicide according to the M.I.N.I. interview section C, (2) psychiatric comorbidity: substance dependence, bipolar affective disorder, psychotic illness, obsessive compulsive disorder, as established during the M.I.N.I. interview, (3) currently receiving psychological treatment for depression in primary or specialized mental health care, (4) being unable to comprehend the spoken and written language of the country where the study is conducted (i.e., Dutch in The Netherlands, German in Germany, English in the UK, Polish in Poland, Swedish in Sweden, German in Switzerland, French in France, and Spanish in Spain), (5) not having access to a computer with fast Internet connection (i.e., broadband or comparable), and (6) not having a smartphone that is compatible with the mobile component of the intervention that is offered, or not willing to carry a smartphone if one is provided with one by the research team for the study duration.
Sample size calculation
The sample size calculation is based on the non-inferiority design and calculated for the primary clinical outcome, symptoms of depression at 3 months after baseline. The non-inferiority margin was set at Cohen’s d = 0.20 which is a conservative estimate of the subjective minimal important difference that is noticeable by depressed patients . To test the hypothesis that bCBT is not inferior to TAU, a total of 1052 patients is required. With this number of patients we are 90 % certain (power of .90) that the lower limit of the two-sided 95 % confidence interval will truly be above the non-inferiority limit Cohen’s d = −0.20. To allow for expected study dropout and variation between settings the total number of participants to be recruited will be 1200, 150 patients in each country.
Randomization will be conducted centrally by the (PI) organization (VU Amsterdam) by a team of independent researchers (the randomization team). The randomization team allocates patients to treatment in all countries except for France where the allocation process will be automated and the UK where the allocation process will be outsourced to a local independent researcher as for logistical reasons the randomization outcome should be instantly available. Randomization will take place at an individual level, stratified by country, after the eligibility and baseline assessment. Further stratification on recruitment location will be applied in France (11 centers), Switzerland (3 centers), The Netherlands (3 centers), and Sweden (two cities). The randomization team will create the allocation scheme with a computerized random number generator (Random Allocation Software) at an allocation ratio of 1:1. Block randomization will be used with variable block sizes that vary between 8 and 14 allocations per block (in France block sizes of 2 and 4 will be applied because of the large number of strata). Subjects will be randomized into two groups: bCBT or TAU. All investigators and clinicians will be unknown to the randomization scheme. Blinding for treatment allocation is not possible as it will be clear to both therapists and patients whether the treatment is blended or not. However, outcome assessors will be blinded.
An overview of the measures and the time of assessment is provided in Table 2. All questionnaire measures are completed online.
The primary outcome measure is symptoms of depression as assessed with the Patient Health Questionnaire-9 (PHQ-9) at 3 months . The PHQ-9 is a multipurpose instrument that was developed for use in primary care but has been validated in different patient populations such as primary care, the general population, specialized mental health care, and patients with somatic disease [32, 33]. The questionnaire is used frequently in clinical trials to assess the outcomes of treatment . The nine items are each scored on a 0–3 scale with the total score ranging from 0–27 and higher scores indicating more severe depression. The PHQ-9 has been shown to have good psychometric properties [32, 33].
The 16-item self-report version of the Quick Inventory of Depressive Symptomatology Self-Report (QIDS-SR)  will be used in addition to the PHQ-9 to measure symptoms of depression. The patient QIDS-SR consists of 16 items (each item scores 0–3) and includes symptom domains of MDD based on the DSM-IV. The QIDS-SR has shown good psychometric properties in several clinical populations .
A diagnosis of depression will be established with the M.I.N.I. International Neuropsychiatric Interview (M.I.N.I.) version 5.0. The M.I.N.I. is a structured diagnostic interview based on the DSM-IV and the International Classification of Diseases (ICD-10) criteria. The M.I.N.I. has been translated to 65 languages and is used for both clinical and research practice. The interview compares well with the Structural Clinical Interview for DSM-IV Disorders (SCID)  and the Composite International Diagnostic Interview (CIDI) [29, 36]. The full M.I.N.I. 5.0, with exception of sections M (Anorexia nervosa), N (Bulimia nervosa), and P (Antisocial personality disorder), will be administered at baseline to assess lifetime and current depression, and current comorbid disorders that often co-occur with and predict the onset of depression (anxiety disorders and post-traumatic stress disorder; PTSD), and other comorbid disorders that are an exclusion criteria in this study (i.e., substance dependence, bipolar affective disorder, psychotic illness, and obsessive compulsive disorder). At the 12-month follow-up, the depression, anxiety and PTSD sections will be administered again to assess recovery of depression and status of the frequently co-occurring disorders of anxiety and PTSD.
Quality of life will be assessed with the EQ-5D-5L . The EQ-5D-5L is a self-report questionnaire which measures health-related quality of life and enables conversion to utility scores to calculate quality-adjusted life years (QALYs). The EQ-5D-5L consists of five dimensions: mobility, self-care, ordinary activities, discomfort, and mood state related to anxiety or depression. For each dimension there are five severity levels defined ranging from no problems to many problems . The EQ-5D-5L has been translated into more than 100 different language versions. The EQ-5D-5L health states will be converted to utility scores using country-specific preference weights if available. Otherwise, the UK preference weights will be used. QALYs will be calculated by multiplying the utilities with the amount of time a patient has spent in a particular health state. Transitions between health states will be linearly interpolated.
Costs will be measured from a societal perspective. Health service uptake, use of informal care, and lost productivity due to illness will be measured with an adapted version of the Trimbos and iMTA Questionnaires on Costs Associated with Psychiatric Illness (TiC-P) . The TiC-P is a self-report questionnaire and consists of two different parts that can be administered separately. Part I will be used to assess the participants’ health care utilization and medication use. Part II (short form Health and Labor Questionnaire; SF-HLQ) measures lost productivity costs resulting from absenteeism (being absent from work because of illness) and presenteeism (being present at work while ill which may lead to reduced efficiency), and consists of 11 items. Health care utilization, use of informal care, and productivity losses will be valued using country-specific opportunity costs.
Several demographic variables, history of treatment for mental health problems, and treatment preferences will be measured at baseline.
The therapeutic alliance between therapists and patients will be assessed with the short version of the Working Alliance Inventory short form (WAI-SF). The WAI-SF is a 12-item self-report questionnaire with responses on a 5-point Likert scale ranging from 1 (never) to 5 (always) . The questionnaire covers three dimensions of working alliance: (1) therapeutic goals, (2) tasks, and (3) bond, and the subscales have been shown to have good internal consistencies. Both the patient and the 10-item therapist version of the questionnaire will be administered at 3-month follow-up. The alliance between the patient and technologies will be assessed with the WAI Online Therapy questionnaire developed by Labpsitec (http://www.labpsitec.uji.es/esp/index.php) at 3-month follow-up.
Patients’ expectancy of treatment will be assessed with the credibility and expectancy questionnaire (CEQ) of Devilly and Borkovec  at baseline. Both factors (credibility and expectancy) have been shown to be stable across different populations with high internal consistency within each factor. The scale consists of six questions, with answer options rated on a 10-point scale and on a 1–100 % scale.
Patient’s satisfaction with the treatment will be assessed with the Client Satisfaction Questionnaire-8 (CSQ-8) . This questionnaire has been translated into multiple languages and is used to measure global patient satisfaction. The questionnaire consists of eight items that are measured on a 4-point scale with total scores ranging from 8 to 32 and has shown good psychometric properties.
Satisfaction with the platform will be evaluated with the System Usability Scale (SUS) . The SUS is a 10-item questionnaire giving a global view of subjective assessments of usability of a technology system. All items are measured on a 5-point scale ranging from strongly disagree to strongly agree. Total SUS scores have a range from 0–100. The questionnaire was found to be reliable and robust .
The blended depression treatment in this study combines individual face-to-face CBT with CBT delivered through an Internet-based treatment platform with mobile phone components (either integrated in the treatment platform or as a separate system). The core components of the bCBT treatment are: (1) psychoeducation, (2) cognitive restructuring, (3) behavioral activation, and (4) relapse prevention. These will be delivered over 11–20 sessions (see Table 3). Traditional CBT treatment consists mostly of face-to-face sessions. In our bCBT treatment, the number of face-to-face sessions is reduced and replaced by online treatment modules. In this study, the ratio between the number of face-to-face sessions and the number of online modules may vary according to practice in participating countries, but a minimum of one third of the sessions should be face-to-face and a minimum of one third of the sessions should be provided online. Thus, for example, when the number of face-to-face sessions is 6, the number of online sessions lies between 3 and 12. bCBT may also include additional components such as mindfulness, coping skills training or problem solving, but these additional components cannot take up more than a quarter of the total treatment (no more than about 25 % of the face-to-face and online sessions combined). This is to prevent too much heterogeneity in the treatment programs. bCBT will be provided by CBT therapists who will receive training on how to deliver the treatment. CBT therapists will either be licensed CBT therapists, CBT therapists in training who work under supervision of an experienced licensed CBT therapist in specialized mental health care, or licensed psychologists, or psychologists in training who work under supervision of a licensed psychologist with a CBT orientation in primary care.
Internet platforms applied in this study
Various platforms are used in this study, tailored to the needs of local sites (see Table 3). All Internet platforms include: (1) a web-based interface providing the patients’ access to online CBT, (2) a mobile phone component which enables daily monitoring of mood state, cognitions, activities, social interaction, and sleep (Ecological Momentary Assessment; EMA). The mobile measures will be time and date stamped.
To ensure treatment fidelity: (1) a detailed bCBT treatment manual is available at each site to guide therapists through the treatment, (2) regular contacts are organized between the therapists and the research team to prevent drift of the treatment protocols within each country, and (3) therapists register the number of sessions, the frequency of the sessions, the main strategies used in each session, the duration of each contact, and whether they have referred the patient to the online part of the treatment. Therapist activities on the platform will be assessed through track and change functionalities to measure, for example, the number of logins and type of activity in each country (log files). In some countries, therapists deliver both treatments (bCBT and TAU), because there are not sufficient therapists available in the partaking country (i.e., France) or the intake procedure does not allow a change of therapist after randomization (i.e., The Netherlands, and Switzerland). Thus, in these countries the therapist conducting the intake will also be the therapist providing the treatment. Contamination between blended depression treatment and TAU is not an insurmountable problem in this study as the contents of the face-to-face treatments may be similar and the participants in the TAU condition do not have access to the Internet-based treatment. Patients are allowed to receive pharmacotherapy in addition to the bCBT if deemed necessary by the health care professional since this reflects routine clinical practice in both treatment conditions.
TAU is defined as the routine care that subjects receive when they are diagnosed with depression in the specific treatment setting where they are recruited. In practice, this means that TAU may vary between countries, treatment setting, and among patients (see Table 1) and is likely to include pharmacologic treatment, psychotherapy or a combination of both. We will not interfere with TAU but we will monitor carefully which health care services are utilized by TAU patients using patient records and through self-report (including TIC-P measurements).
Data from the eight trials will be combined and multiple imputation will be used to deal with missing data. Intention-to-treat analyses (ITT) increase the risk of type I errors in non-inferiority trials and per-protocol analyses are preferred over ITT analyses . Therefore, the main analysis will consist of a per-protocol analysis meaning that only those patients who have completed the treatment will be included in the analyses. ITT analyses will be used in sensitivity analyses to increase confidence in the results obtained by including all participants in the analyses independent of whether they completed the treatment or not. Multilevel regression analyses will be applied for both types of analyses, taking into account the differences between countries. Time of assessment will be treated as a fixed variable to examine the difference between groups on each occasion of measurement. Regarding the primary outcome, symptoms of depression, bCBT is considered non-inferior as compared with TAU when the two-sided 95 % confidence interval (the range of plausible differences between the two treatments) lies entirely above the standard mean difference of −0.20 which is the non-inferiority margin and the smallest clinically acceptable difference. Cohen’s d will be calculated to determine the magnitude of the treatment effects for continuous outcome measures, both within groups for each timepoint compared to baseline, and between groups. Effect sizes under <0.2 are deemed to be small, between 0.2 and 0.5 are deemed to be moderate, and >0.8 are deemed to be large. For dichotomous outcome measures, we will calculate the relative risk ratio.
The economic evaluation will be first conducted on a time horizon corresponding to the trial horizon. In a second, time modeling cost-effectiveness on a longer period (a 5-year horizon) will be considered. The analysis will be performed both from the societal perspective and a health care cost perspective. Multiple imputation will be used to impute missing cost and effect data. Bivariate regression models will be used to estimate cost and effect differences while adjusting for potential confounders . Incremental cost-effectiveness ratios (ICERs) will be calculated by dividing the mean difference in costs between bCBT and TAU by the difference in effects. To account for the typically skewed distribution of costs, bias-corrected and accelerated bootstrapping (5000 replications) will be used to estimate the 95 % confidence intervals around the mean cost differences and the uncertainty surrounding the ICERs. The bootstrapped ICERs will be graphically presented in cost-effectiveness planes . Cost-effectiveness acceptability curves  will be estimated to show the probability that bCBT is cost-effective in comparison with TAU for a range of different ceiling ratios, thereby showing decision uncertainty.
Ethical approval for the trials has been obtained locally in each country. All participants provide written informed consent before taking part in the trial and are asked for their permission to share their (anonymized) data across participating E-COMPARED partners and to report on the results in publications. When data is shared among the collaborating sites and countries it will be encrypted and it will not contain identifiable information. The studies will be conducted in line with the declaration of Helsinki  and each trial is registered in a local clinical trial register. All researchers will follow the guidelines for Good Clinical Practice  and the trial outcomes will be reported in line with the Consolidated Standards of Reporting Trials (CONSORT) guidelines .
European health care systems face the challenge of improving access to cost-effective treatments to relieve the burden of depression. Comparative effectiveness research (CER) may be an appropriate method to provide evidence for improved informed decision on depression treatment with the aim to increase patient outcomes, quality of life, and cost-containment. The E-COMPARED project aims to examine the clinical and cost-effectiveness of bCBT for adults with depression compared to TAU in eight countries across Europe. It is expected that bCBT is clinically non-inferior compared to TAU, but that it is cost-effective as less therapist time is needed to deliver treatment.
This is one of the first trials to examine the effectiveness of bCBT in routine primary and specialized mental health care throughout Europe. So far, most studies have been conducted in controlled research settings in individual countries, and it is not clear how these findings translate to routine “real-world” practice or how they differ between health care systems. The E-COMPARED study is a large, multinational, randomized controlled trial with sufficient statistical power to take into account the heterogeneity between countries by pooling data. Comparison of bCBT with TAU will inform how blended depression treatment is related to current routine practice in different settings and under different circumstances.
Research in routine practice is difficult as research procedures are often constrained by the complexity of the service delivery centers and the patient population. Therefore, pragmatic decisions (i.e., external validity) have to be balanced against stringent research design requirements (i.e., internal validity). We expect that research will be even more complicated in countries that do not have much experience with Internet-based treatment or psychotherapy for depression, such as Poland and France, as we do not know to what extent this type of treatment is acceptable to patients and therapists in those countries. From a research design perspective, the differences in TAU between countries are challenging. In some countries, TAU may be primarily pharmacotherapy whereas in other countries this may be CBT or combined treatment. This is a pragmatic study, however, and where possible we have standardized study procedures, inclusion and exclusion criteria, outcome measures, and intervention components while not restricting the content of TAU in any way. The results will provide evidence-based recommendations for national and European mental health policy and decision-makers allowing them to make informed decisions regarding the dissemination and implementation of Internet-based treatments in primary and specialized care, while taking into account the different health care systems.
The study commenced recruitment in February 2015 and is currently recruiting.
bCBT, blended cognitive behavioral therapy; CBT, cognitive behavioral therapy; CEQ, Credibility and Expectancy Questionnaire; CER, comparative effectiveness research; CIDI, Composite International Diagnostic Interview; CSQ-8, Client Satisfaction Questionnaire-8; DSM-IV, Diagnostic and Statistical Manual of Mental disorders; EMA, Ecological Momentary Assessment; EQ-5D-5L, EuroQol 5 dimensions 5 levels; ICD-10, International Classification of Diseases-10; ICER, incremental cost-effectiveness ratio; ITT, intention-to-treat; M.I.N.I, MINI International Neuropsychiatric Interview; MDD, major depressive disorder; PHQ-9, Patient Health Questionnaire-9; PTSD, post-traumatic stress disorder; QALY, quality-adjusted life years; QIDS, Quick Inventory of Depressive Symptomatology; RCT, randomized controlled trial; SCID, Structural Clinical Interview for DSM-IV; SF-HLQ, short form Health and Labor Questionnaire; SUS, System Usability Scale; TAU, treatment as usual; TIC-P, Trimbos and iMTA Questionnaires on Costs Associated with Psychiatric Illness; WAI-SF, Work Alliance Inventory short form
Richards D. Prevalence and clinical course of depression: a review. Clin Psychol Rev. 2011. doi:10.1016/j.cpr.2011.07.004.
Cuijpers P, Vogelzangs N, Twisk J, Kleiboer A, Li J, Penninx B. Is excess mortality in depression generic or specific? A comprehensive meta-analysis of community and patient studies. Am J Psychiatry. 2014. doi:10.1176/appi.ajp.2013.13030325.
Olesen J, Gustavsson A, Svensson M, Wittchen HU, Jönsson B. The economic cost of brain disorders in Europe. Eur J Neurol. 2012;19(1):155–62.
Smit F, Cuijpers P, Oostenbrink J, Batelaan N, de Graaf R, Beekman A. Costs of nine common mental disorders: implications for curative and preventive psychiatry. J Ment Health Policy Econ. 2006;9:193–200.
Gustavsson A, Svensson M, Jacobi F, et al. Costs of disorders of the brain in Europe 2010. Eur Neuropsychopharmacol. 2011;21:718–79.
Sobocki P, Jönsson B, Angst J, Rehnberg C. Cost of depression in Europe. J Ment Health Policy Econ. 2006;9:87–98.
Evans-Lacko S, Knapp M. Importance of social and cultural factors for attitudes, disclosure and time off work for depression: findings from a seven country European study on depression in the workplace. PLoS One. 2014;9, e91053. doi:10.1371/journal.pone.0091053.
Cuijpers P, Berking M, Andersson G, Quigley L, Kleiboer A, Dobson KS. A meta-analysis of cognitive-behavioral therapy for adult depression, alone and in comparison with other treatments. Can J Psychiatry. 2013;58:376–85.
Cuijpers P, van Straten A, van Oppen P, Andersson G. Are psychological and pharmacologic interventions equally effective in the treatment of adult depressive disorders? A meta-analysis of comparative studies. J Clin Psychol. 2008;69(11):1675–85.
Cuijpers P, van Straten A, Schuurmans J, van Oppen P, Hollon SD, Andersson G. Psychotherapy for chronic major depression and dysthymia: a meta-analysis. Clin Psychol Rev. 2010;30:51–62.
Cuijpers P, van Straten A, Warmerdam L, Andersson G. Psychotherapy versus the combination of psychotherapy and pharmacotherapy in the treatment of depression: a meta-analysis. Depress Anxiety. 2009;26:279–88.
NICE. The treatment and management of depression in adults. London: National Institute for Clinical Excellence; 2009.
Woelbert E. Psychotherapy for mental illness in Europe: an exploration of the evidence base and status quo. European Commission Joint Research Centre; 2015.
Andrews G, Issakidis C, Carter G. Shortfall in mental health service utilisation. Br J Psychiatry. 2001;179:417–25. doi:10.1192/bjp.179.5.417.
Spijker J, Bijl RV, De Graaf R, Nolen WA. Care utilization and outcome of DSM-III-R major depression in the general population. Results from the Netherlands Mental Health Survey and Incidence Study (NEMESIS). Acta Psychiatr Scand. 2001;104:19–24. doi:10.1034/j.1600-0447.2001.00363.x.
Kazdin AE, Blase SL. Rebooting psychotherapy research and practice to reduce the burden of mental illness. Perspect Psychol Sci. 2011;6:21–37.
Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012;32(4):329e342.
Andersson G, Topooco N, Havik OE, Nordgreen T. Internet-supported versus face-to-face cognitive behavior therapy for depression. Expert Rev Neurother. 2016;16:55–60.
Wykes T, Haro JM, Belli SR, Obradors-Tarragó C, Arango C, Ayuso-Mateos JL, Bitter I, Brunn M, Chevreul K, Demotes-Mainard J, Elfeddali I, Evans-Lacko S, Fiorillo A, Forsman AK, Hazo JB, Kuepper R, Knappe S, Leboyer M, McDaid D, Miret M, Papp S, Park AL, Schumann G, Thornicroft G, Feltz-Cornelis CM, van Os J, Wahlbeck K, Walker-Tilley T, Wittchen HU. Mental health research priorities for Europe. Lancet Psychiatry. 2015;2(22):1036–42.
Kooistra L, Wiersma JE, Ruwaard J, van Oppen P, Smit F, Lokkerbol J, Cuijpers P, Riper H. Blended vs. face-to-face cognitive behavioural treatment for major depression in specialized mental health care: study protocol of a randomized controlled cost-effectiveness trial. BMC Psychiatry. 2014;14:290.
Krieger T, Meyer B, Sude K, Urech A, Maercker A, Berger T. Evaluating an e-mental health program (“deprexis”) as adjunctive treatment tool in psychotherapy for depression: design of a pragmatic randomized controlled trial. BMC Psychiatry. 2014;14(1):285.
López del Hoyo Y, Olivan B, Luciano JV, Mayoral F, Roca M, Gili M, Andres E, Serrano A, Collazo F, Araya R, Baños RM, Botella C, Magallón R, y García-Campayo J. Low intensity vs. self-guided Internet-delivered psychotherapy for major depression: a multicenter, controlled, randomized study. BMC Psychiatry. 2013;13(21). doi: 10.1186/1471-244X-13-21.
Wentzel J, van der Vaart R, Bohlmeijer ET, van Gemert-Pijnen JE. Mixing online and face-to-face therapy: how to benefit from blended care in mental health care. JMIR Mental Health. 2016;3(1), e9.
Beattie A, Shaw A, Kaur S, Kessler D. Primary-care patients’ expectations and experiences of online cognitive behavioural therapy for depression: a qualitative study. Health Expect. 2009;12:45–59.
Montero-Marín J, Carrasco JM, Roca M, Serrano-Blanco A, Gili M, Mayoral F, Luciano JV, Lopez-del-Hoyo Y, Olivan B, Collazo F, Araya R, Baños RM, Botella C, García-Campayo J. Expectations, experiences, and attitudes of patients and primary care health professionals regarding online psychotherapeutic interventions for depression: protocol for a qualitative study. BMC Psychiatry. 2013;13:64.
van der Vaart R, Witting M, Riper H, Kooistra L, Bohlmeijer ET, van Gemert-Pijnen L. Blending online therapy into regular face-to-face therapy for depression: content, ratio and preconditions according to patients and therapists using a Delphi study. BMC Psychiatry. 2014;14:355.
Schumi J, Wittes JT. Through the looking glass: understanding non-inferiority. Trials. 2011;12:106. doi:10.1186/1745-6215-12-106.
Chan A-W, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin J, Dickersin K, Hróbjartsson A, Schulz KF, Parulekar WR, Krleža-Jerić K, Laupacis A, Moher D. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;346, e7586.
Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, Dunbar GC. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59 Suppl 20:22–33.
Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16:606–13.
Cuijpers P, Turner EH, Koole SL, van Dijke A, Smit F. What is the threshold for a clinically relevant effect? The case of major depressive disorders. Depress Anxiety. 2014;31:374–8.
Gilbody S, Richards D, Brealey S, Hewitt C. Screening for depression in medical settings with the Patient Health Questionnaire (PHQ): a diagnostic meta-analysis. J Gen Intern Med. 2007;22:1596–602.
Wittkampf KA, Naeije L, Schene AH, Huyser J, van Weert HC. Diagnostic accuracy of the mood module of The Patient Health Questionnaire: a systematic review. Gen Hosp Psychiatry. 2007;29:388–95.
Löwe B, Unutzer J, Callahan CM, Perkins AJ, Kroenke K. Monitoring depression treatment outcomes with the Patient Health Questionnaire-9. Med Care. 2004;42:1194–201.
Rush AJ, Trivedi MH, Ibrahim HM, Carmody TJ, Arnow B, Klein DN, Markowitz JC, Ninan PT, Kornstein S, Manber R, Thase ME, Kocsis JH, Keller MB. The 16-item Quick Inventory of Depressive Symptomatology (QIDS) Clinician Rating (QIDS-C) and Self-Report (QIDS-SR): a psychometric evaluation in patients with chronic major depression. Biol Psychiatry. 2003;54:573–83.
Lecrubier Y, Sheehan DV, Weiller E, Amorim P, Bonora I, Harnett Sheehan K, Dunbar GC. The Mini International Neuropsychiatric Interview (MINI). A short diagnostic structured interview: reliability and validity according to the CIDI. European Psychiatry. 1997;12:224–31.
Group EQ. EuroQol—a new facility for the measurement of health-related quality of life. Health Policy. 1990;16:199–208.
van Agt HME, Essink-Bot ML, Krabbe PFM, Bonsel GJ. Test-retest reliability of health state valuations collected with the EuroQol questionnaire. Soc Sci Med. 1994;39:1537–44.
Hakkaart-van Rooijen L, van Straten A, Donker M, Tiemans B. Manual Trimbos/iMTA questionnaire for costs associated with psychiatric illness (TIC-P). Rotterdam: Institute for Medical Technology Assessment; 2002.
Hatcher RL, Gillaspy JA. Development and validation of a revised short version of the working alliance inventory. Psychother Res. 2007;16:12–25.
Devilly GJ, Borkovec TD. Psychometric properties of the credibility/expectancy questionnaire. J Behav Ther Exp Psychiatry. 2000;31:73–86.
Nguyen TD, Attkinson CC, Stegner BL. Assessment of patient satisfaction: development and refinement of a service evaluation questionnaire. Eval Program Plann. 1983;6:299–314.
Brooke J. SUS-a quick and dirty usability scale. In: Jordan PW, Thomas B, Weerdmeester BA, McClelland IL, editors. Usability Eval Ind. Volume 189. London: Taylor & Francis Ltd; 1996. p. 189–94.
Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. Int J Hum Comput Interact. 2008;24:574–94.
Willan AR, Briggs AH, Hoch JS. Regression methods for covariate adjustment and subgroup analysis for non-censored cost-effectiveness data. Health Econ. 2004;13(5):461–75.
Black WC. The CE, plane: a graphic representation of cost-effectiveness. Med Decis Making. 1990;10(3):212–4.
Fenwick E, O’Brien BJ, Briggs A. Cost-effectiveness acceptability curves—facts, fallacies and frequently asked questions. Health Econ. 2004;13:405–15.
World Medical Association. Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310(20):2191–4.
Food Drug Administration (FDA). International conference on harmonization, good clinical practice: consolidated guidelines. Fed Regist. 1997;62:25692–709.
Schulz KF, Altman DG, Moher D, for the CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332.
This project is funded by the European Commission FP7-Health-2013-Innovation-1 program, grant agreement number: 603098.
Availability of data and materials
HR received the funding for the project. AK, JS, JB, GA, TB, TK, CB, RB, KC, RA, RC, AvS, DE, MB, BF, and PC were all co-applicants. AK drafted the paper. All authors contributed to the design of the study and have commented on and approved the final version of the paper.
The authors declare that they have no competing interests.
Consent for publication
All participants are asked for their permission to share their (anonymized) data across participating E-COMPARED partners and to report on the results in publications.
Ethical approval and consent to participate
Ethical approval for the trials has been obtained locally in each country (France: Comité de protection des personnes, Ile de France V; 15033-n° 2015-A00565-44; Germany: Ethik Kommison DGPsychologie, Universitat Trier; MB 102014; The Netherlands: METC VUMC; 2015.078; Poland: Komisja ds. Etyki Badan Naukowych; 10/2014; Spain: Comision Deontologica/Comite Etico de Investigacion en Humanos; H1414775276823; Sweden: Regionala etikprovningsnamnden; 2014/428-31; Switzerland: Kantonale Ethikkomission Bern; 001/2015; United Kingdom: NRES Committee London-Camden and King’s Cross; 15/LO/0511). All participants provide written informed consent before taking part in the trial.
About this article
- Internet-based treatment
- Cognitive behavioral treatment (CBT)
- Blended treatment
- Comparative effectiveness research (CER)
- Randomized controlled trial (RCT)