This article has Open Peer Review reports available.
Effectiveness and cost effectiveness of guided online treatment for patients with major depressive disorder on a waiting list for psychotherapy: study protocol of a randomized controlled trial
© Kenter et al.; licensee BioMed Central Ltd. 2013
Received: 11 July 2013
Accepted: 18 November 2013
Published: 1 December 2013
Depressive disorders are highly prevalent and result in negative consequences for both patients and society. It is therefore important that these disorders are treated adequately. However, due to increased demand for mental healthcare and subsequent increased costs, it would be desirable to reduce costs associated with major depressive disorder while maintaining or improving the quality of care within the healthcare system. Introducing evidence-based online self-help interventions in mental healthcare might be the way to maintain clinical effects while minimizing costs by reducing the number of face-to-face sessions. This study aims to evaluate the clinical and economical effects of a guided online self-help intervention when offered to patients with major depressive disorder on a waiting list for psychotherapy in specialized mental health centers (MHCs).
Patients at mental health centers identified with a Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV) diagnosis of major depression who are awaiting face-to-face treatment are studied in a randomized controlled trial. During this waiting list period, patients are randomized and either (1) receive an internet-based guided self-help treatment or (2) receive a self-help book. The 5-week internet-based guided self-help intervention and the self-help booklet are based on problem solving treatment. After the intervention, patients are allowed to start regular face-to-face treatment at MHCs. Costs and effects are measured at baseline, after the intervention at 6 to 8 weeks, 6 months and 12 months. The primary outcome measure is symptoms of depression. Secondary outcome measures are diagnosis of depression, number of face-to-face sessions, absence of work and healthcare uptake in general. Additional outcome measures are anxiety, insomnia, quality of life and mastery.
This study evaluates the effectiveness and cost effectiveness of internet-based guided self-help in patients at specialized mental health centers. The aim is to demonstrate whether the introduction of internet-based self-help interventions in regular mental healthcare for depressive disorders can maintain clinical effects and reduce costs. Strengths and limitations of this study are discussed.
Netherlands Trial Register NTR2824
Depressive disorders are highly prevalent [1–3], and are associated with high costs in the professional, social, personal and financial realms [1, 3, 4]. Those suffering from depression are more often absent from work, and have higher levels of healthcare consumption compared to healthy individuals . It is desirable from a clinical point of view to reduce the burden of depression, as it is from a societal point of view to reduce the economic costs related to increased healthcare uptake, and reduced work productivity. In addition, given the increased demand for mental healthcare and limitation of recourses, it is important for mental health centers to optimize the efficient and effective use of resources.
In The Netherlands, a person with symptoms of depression who is seeking help is most likely to be seen first by a general practitioner (GP) , who acts as a gatekeeper for referral to specialized mental health services. After registration at a mental health service, patients usually have an assessment and are then assigned to a specific treatment. The time between registration and the first treatment session is normally at least 6 weeks. Long waiting lists caused by low workforce numbers are common. This time might be used to deploy internet-based self-help treatments [6, 7], as previous studies have indicated that internet-based self-help therapies are clinically effective in diverse populations [8–11]. Internet-based treatments require less therapist time in comparison with standard face to face treatments and can therefore offer potential solutions as they are immediately accessible, less costly and put less strain on therapeutic resources. They might serve first step in a stepped delivery of care. This means that only those who do not respond adequately step up to a treatment of higher intensity, in this case the regular treatment in mental health centers. This stepped-care approach is suggested in several guidelines such as the Australian and National Institute for Health and Care Excellence (NICE) guidelines for depression, which recommend that patients receive the least burdensome treatment. Psychological treatments such as computerized cognitive behavioral therapy (cCBT) and individual guided self-help programs are recommended as low intensity treatments. After receiving such treatments, patients possibly need fewer or no face-to-face sessions.
With the current cuts in healthcare budgets, it would be desirable to reduce costs associated with major depressive disorder while maintaining or improving the quality of care. Introducing evidence-based internet-based self-help interventions in mental healthcare might be the way to speed up clinical recovery while minimizing costs by reducing the number of face-to-face sessions. However, internet-based guided self-help interventions are currently not offered as a first step to those waiting for specialized mental healthcare.
In the current study, we offer an internet-based guided self-help program to patients on a waitlist for psychological treatment. The self-help program is based on problem-solving therapy and uses self-examination therapy as a general framework . This method has been found to be effective in several studies in the US , and for this study we used a guided self-help program that has been examined in two earlier trials, and has proved its effectiveness in reducing depressive symptoms in community samples [14, 15].
Aims and hypotheses
This study aims to establish the effectiveness of an internet-based guided self-help intervention for patients with major depressive disorder before face-to-face treatment in specialized mental healthcare in comparison to a control group who are on a waitlist for face-to-face treatment. Furthermore, we expect that offering internet-based treatment as a first step of care reduces costs as we predict the internet group to take up fewer regular treatment sessions.
The study is a randomized controlled trial with an economic evaluation. We are currently enrolling 248 patients over 2 large mental health centers (MHCs) in 10 different locations. We will recruit patients directly after registration for regular face-to-face mental health services who need to wait for at least 6 weeks before their first treatment session. They will be randomized to either an internet-based guided self-help intervention or the regular waitlist before the first treatment session. People on the waitlist will receive a self-help book without additional guidance in order to motivate them to participate in the randomize, controlled trial (RCT). Both groups are allowed to receive regular face-to-face (FTF) therapy at the MHCs after the waitlist period. Participants in both groups complete assessment at baseline, at intervention or waiting list (WL) completion (6 to 8 weeks), at a 6-month follow-up, and at 12 months. The protocol of this study has been approved by the Medical Ethics Committee of the VU University Medical Center (registration number 2011/223).
Inclusion and exclusion criteria
Eligible participants are adults, aged 18 or over, registering for regular treatment at one of the participating MHCs who meet the criteria for a Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV) diagnosis of major depression as measured with the Clinical International Diagnostic Interview (CIDI) by a trained research assistant, have access to the internet, and adequate proficiency in Dutch. Exclusion criteria are starting or changing type of dosage of antidepressant medication, the presence of a bipolar or psychotic disorder, and an increased risk of suicide. Comorbid disorders other than bipolar or psychotic disorders are allowed.
Participants will be recruited while registering at the participating MHCs. In routine care patients are briefly scanned by the MHCs (according to MHCs protocol to screen out patients who are in crisis and need immediate treatment), and for the purpose of this study, patients with mood problems who are eligible for an intake assessment are asked by the MHC to share their contact details with the researchers. Those who are willing to do so, will then be called by a member of the research team who further explains the aim of this study and performs an additional check of the inclusion and exclusion criteria. If eligible, patients receive a study brochure and an informed consent form. Patients will only be included in the study if they meet all the criteria, and sign the informed consent form. In order to confirm the diagnoses Major Depressive Disorder trained interviewers conduct a clinical interview (CIDI).
The trial is powered to detect an effect size of d ≥0.40 [10, 16] as statistically significant in a two-tailed test with α = 0.05 and power of (1 - β =) 0.80 with N = 99 per condition. To compensate for loss of follow-up of 20%, the trial requires starting with 99/0.8 = 124 participants per condition at baseline. The dropout rate of 20% is conservative, but from a power perspective a cautious approach. As this trial has two conditions, a total of N = 248 will be necessary for the complete trial.
Mental health centers characteristics
Two MHCs will be participating in this research. They each offer services at a number of different locations. In this trial, a total of ten locations will participate. These centers are chosen for pragmatic purposes, as they have a high number of patient enrolment, and have participated in prior research of the VU University. In general, patients are referred by their general practitioner to the mental health centers where the patients is screened and placed on a waiting list. Within 6 weeks an initial meeting with a therapist takes place in which the patient’s needs and preferences regarding to treatment are determined. At these participating MHCs it normally takes between 7 to 16 weeks for patients to have a first treatment session, depending on therapist workload, treatment modality and other factors. Treatment in both centers can consist of psychological therapies such as CBT, sometimes in combination with medication. The researchers are not involved in the face-to-face treatment, and neither in the decision-making process. However, prior to the start of this study all therapists at the participating MHCs will be informed about the goals of this study and the internet-based intervention, they will attend a Q&A session with the first author and receive the self-help book containing the intervention. Prior to the start of a patients’ face-to-face treatment at the MHC the therapists will be informed by the aforementioned note in electronic patient record and through email that their patients are participating in this study. One of the objectives of the internet-based intervention is that the subsequent face-to-face treatment can be adjusted to fewer sessions. The number and type of therapeutic sessions patients will receive is based on the needs of the patient, the judgment of the therapist, and protocol available at the MHC. The type and number of sessions will therefore vary per participant, and are outside the control of the researchers.
The internet intervention that will be used is called 'Taking Control’ (original title: 'Alles Onder Controle’). This intervention uses the self-examination treatment model developed by Bowman and colleagues [12, 13], which is based on problem-solving therapy and uses self-examination therapy (SET) as a general framework. We translated it into Dutch, elaborated on it, and added information and exercises. We built a website for this intervention and developed a system for email support. This intervention has been described in more detail in several other studies [12–14].
In brief, this intervention is short, structured and manualized. It consists of five weekly sessions. Each session contains a structured homework assignment to be completed by the participant. The first session requires participants to consider what is important in their lives. Next to this, participants will make a list of current problems and worries in their lives. This first session helps participants to divide their problems into the following three categories: not important (if the problem is not related to the list of important things), important and unsolvable (for example, permanent loss of health or a loved one), or important and solvable. In the second, third and fourth session participants will be offered various coping skills related to each of the categories of problems; the main focus is placed on adopting a structured six-step approach when encountering important, solvable problems. This structured approach is divided in the following steps: identifying the current problem; finding possible solutions; selecting one solution; create a plan to solve the problem with this solution; execute the plan and evaluate the plan. The last week of the intervention is reserved for both the reflection on long-term goals and the development of a structure to achieve these goals. Participants can only move on to the next session once the exercise in the current session has been submitted and when feedback on this session has been released by the research team.
The participants are supported by a coach, who gives feedback to the homework assignments of the participants in brief, weekly emails. The total amount of time spent on each patient is about 1.5 h (estimate based on our earlier trials). The writing of these emails takes about 15 or 20 minutes per week, and will be performed by a coach. The coaches will be trained by the psychologists who have developed the intervention and also wrote the protocol for giving feedback to ensure the consistency and integrity. An independent psychologist will verify whether the coaches have followed the protocol sufficiently by reading a random selection of the feedback emails.
The feedback has two purposes. Firstly, coaches will help participants to become familiar with the presented techniques. The second purpose consists of motivating the participant to continue with the intervention. Feedback will be received by participants within 3 working days after a session has been completed and submitted. When participants pose content related questions to their coaches via the website, email or phone, their coaches will provide additional guidance after receiving the question. After 5 weeks of guidance, patients can continue to use the internet-based treatment but will not receive any feedback on their assignments.
To increase participation rates in the control group, this group receives an unguided self-help book posted to their home address. This control group will not receive any feedback from coaches, nor will it have the opportunity to pose questions. Earlier research shows that self-help without any form of guidance only results in a small effect on participants with increased levels of depressive symptoms .
Participants in both conditions will be referred for regular FTF treatment after registration at the MHC. FTF treatment might consist of additional waiting time; the duration of the waiting time is highly variable with a minimum of 7 weeks and a maximum of 16 weeks depending on the location of MHC. Variation fluctuates both between locations and over time within the centers, due to availability of therapists. In case the waiting time for any MHC falls below 8 weeks, new participants from this MHC will be temporarily excluded from participating in the research until the waiting time for new patients at this MHC exceeds 8 weeks again. This has been decided in order to not measure the effects of active treatment by the MHCs at the first post-intervention test at 8 weeks.
The researchers of this study do not influence the waiting time at the MHCs, nor will participation in this study influence the waiting time for the participants.
Randomization and blinding
T0 Baseline (pre-test)
T1 Post-test (8 weeks)
T2 Follow-up I (6 months)
T3 Follow-up II (12 months)
Symptoms of depression
CIDI (Section E)
Diagnosis of depression
HADS (Anxiety section)
Symptoms of anxiety
Level of insomnia
Quality of life
Number of face-to-face appointments in MHC
General health service uptake and productivity losses
NEO-FFI (sections N + C)
Neuroticism and conscientiousness traits
Expectancy and treatment
Expectations of the internet intervention
Client satisfaction with treatment
Satisfaction with type of intervention
Our primary outcome will be symptoms of depression. Secondary outcomes are DSM-IV diagnosis of depression, number of face-to-face sessions in MHC, costs (in terms of all healthcare use and work absenteeism), anxiety, mastery, insomnia, quality of life, and satisfaction. Plausible moderators measured in this study will be alcohol use, personality, demographic factors and client expectations.
Because the Composite International Diagnostic Interview (CIDI) is part of the questionnaires at baseline and at post-intervention assessment (Table 1), the baseline questionnaires and the 6 to 8 week assessment will be administered by phone. Follow-up assessments at 6 and 12 months will be administered online. All questionnaires will be in Dutch.
Symptoms of depression
Symptoms of depression will be registered by using the Center for Epidemiological Studies Depression scale (CES-D) . This scale consists of 20 items, the total score ranges between 0 and 60; higher scores indicate higher levels of depression and a score of 16 and above indicates a clinical level of depression. This questionnaire has been tested in various populations and has been found valid and reliable . A Dutch version of the CES-D has been validated for internet administration .
Diagnosis of depression
The CIDI will be used to assess whether the DSM-IV diagnosis of depression has been met. The CIDI has been developed by the World Health Organization to assess DSM-IV Axis-I diagnoses . For the purposes of this study, only section E will be administered, which allows for screening for depressive disorders. The CIDI will be conducted by phone by a trained interviewer.
Symptoms of anxiety
The subscale Anxiety of the Hospital Anxiety and Depression Scale (HADS) will be used to measure symptoms of anxiety. The depression scale will not be utilized in this research because depression will be measured by both CIDI and CES-D. The Anxiety subscale consists of 7 items, which scores range from 0 to 21; higher scores indicate higher levels of anxiety . The HADS has shown to be reliable in Dutch populations .
Symptoms of insomnia
The perceived level of insomnia will be measured by the Insomnia Severity Index (ISI) . This questionnaire measures both the concerns associated with the perceived level of insomnia, as well as symptoms and consequences of insomnia. Each item is rated on a 0 to 4 scale; a higher score indicates more severe insomnia. ISI has been found to be internally consistent and reliable .
Quality of life
The EuroQol 5-Dimensions questionnaire (EQ-5D), which consists of five items, will be used to measure quality of life. It registers the level of perceived problems (no, some or extreme) in five domains (mobility, self-care, usual activities, pain/discomfort and anxiety/depression). A total of 486 distinct health states can be scored which are located between 0, which indicates worst health possible, and 1, which indicates perfect health .
Costs related to the intervention
Costs related to the intervention will be calculated based on the costs of running the intervention in both conditions. To calculate costs related to the internet intervention, the costs of running the website platform and the costs of the hours of coaching invested in the intervention will be taken into account. Additional costs in the control group are the publishing of the self-help books.
Costs related to mental healthcare
Costs related to mental healthcare will be measured as the reported number and type of sessions by the MHC. This information is sent to the first author 12 months after participants have registered at the MHCs. The direct costs of the sessions will be calculated based on the Dutch standard cost prices .
Costs from a societal perspective
Healthcare costs in general as well as productivity losses will be measured with the revised version of the Trimbos and iMTA Questionnaire on Costs Associated with Psychiatric Illness (TiC-P). Direct costs will be measured by investigating which contact with health professionals has occurred and which type of medication has been prescribed. Indirect costs will be measured by work absenteeism and reduced productivity. The baseline TiC-P measures the care consumption 4 weeks prior to the intervention. The follow-up assessments at 6 and 12 months use a 14 and 26 weeks recall period respectively. Previous research has shown that up until half a year later, patients can reliably recall their consumption of health services . The TiC-P has been used previously in a population with depressive symptoms in The Netherlands .
The amount of perceived control in a person’s life will be measured by the Pearlin Mastery Scale . The scale consists of seven distinct items that are rated on a four-point scale. Higher scores indicate more perceived control; scores range from 7 to 35. The scale has good reliability .
Demographic variables such as age, gender, parental nationality, family composition, family income and educational level will be screened for in a general questionnaire that measures user characteristics.
The use of alcohol will be monitored using the four questions that make up the acronym CAGE: 'Cutting down, Annoyance by criticism, Guilty feeling, and Eye-openers’ . The CAGE is a widely used concise screening tool for problematic alcohol consumption .
In order to measure the constructs neuroticism and conscientiousness, the NEO-Five Factor Inventory (NEO-FFI) will be administered . Previous research has indicated that neuroticism often coincides with depression (for example, ). Those scoring higher on conscientiousness are expected to adhere better to the homework exercises and consequently benefit more from internet-based therapies. Only these two domains will be tested for to not exhaust participants more than necessary. A total of 12 questions on each of the domains will be answered on a 5-point Likert scale.
The Credibility/Expectancy Questionnaire of Devilly and Borkovec (CEQ) will be used to measure the expected change and credibility of proposed treatment . It consists of six questions; four questions measure the 'thinking’ aspects about the treatment, two questions measure the 'feeling’ aspects of the questions. One question in both the 'feeling’ and 'thinking’ domain is rated from 0% to 100%; the remaining four questions are rated on a Likert-type scale from 1 to 9. Both factors have been found to have a high internal consistency and to have high test-retest reliability .
Expectancy with internet intervention
The 'Alles Onder Controle’ Expectancy Questionnaire (AOCEQ) has been designed for this research to ask what participants’ expectations are about the course. An open question investigates in which ways participants expect to gain benefits from this course. Three five-point Likert scale questions measure to which degree participants appreciate starting with the intervention immediately, that personalized feedback will be given in the internet group and to which degree they expect the course to help them feel less miserable.
General satisfaction with treatment
The satisfaction with the internet intervention will be measured by the Client Satisfactory Questionnaire-8 (CSQ-8), which consists of eight questions, each question is scored on a Likert-type scale from 1 to 4 . The questionnaire addresses several elements that contribute to overall service satisfaction and is reported in a single dimension of overall satisfaction. A high internal consistency has been reported .
Satisfaction with the internet intervention
The 'Alles Onder Controle’ Satisfaction Questionnaire (AOCSQ) has been designed specifically to investigate to what degree participants are satisfied with this internet-based intervention. It includes questions on the number of sessions completed and, if applicable, the reasons for not finishing the course. The questionnaire further researches satisfaction with the separate elements of the intervention, such as the quality of the feedback, the clarity of the website and the appropriateness of the examples. Lastly, it explores whether participants were satisfied with the provided alternative to waiting, to which degree internet interventions are preferred over book interventions and to which degree feedback is preferred over non-feedback.
The analyses will be conducted in agreement with the intention to treat (ITT) principle, as per the CONSORT statement . Therefore missing endpoints will be imputed using state of the art imputation methods, as a reliable method for handling missing values . Imputation allows for analyzing all participants in the condition to which they have been randomized, which contributes to guarantee the integrity of the randomization and restores loss of power due to dropout.
To answer the research questions we will first look at post-test differences between the two groups. We will use analysis of variance (ANOVA), with the baseline values and waiting time as covariates. Subgroup analyses will be performed for different characteristics. The difference in scores between the intervention group and the control group will also be expressed in effect sizes. We use Cohen’s d which is calculated by dividing the difference in mean scores of the two groups by their pooled standard deviation (Xexp-Xctrl/SDpooled). Effect sizes under 0.2 are considered to be small, those of 0.5 are moderate and effect sizes of 0.8 are considered to be large . Furthermore, we will compare the rate of DSM-IV diagnosis of depression in both groups with logistic regression analysis. The clinical effects will also be calculated using reliable change . The long-term outcomes will be analyzed with longitudinal analyses.
The economic evaluation will be conducted from a societal perspective, therefore it will include not only the intervention costs and costs stemming from healthcare uptake (direct medical costs), but also the patients' out of pocket costs (direct non-medical costs) and costs stemming from productivity losses due to absenteeism and work cutback days (indirect non-medical costs). Costs will be based on the Dutch standard cost prices  and productivity losses will be valued using the friction costs method, as per the Dutch guideline for economic evaluation. Quality-adjusted life years (QALYs) will be calculated on the basis of the EQ-5D. Having calculated the costs and QALYs allows for a cost-utility analysis, which can be used to compare this intervention’s gains against those of other interventions for depressive disorders. A cost-effectiveness analysis can be carried out by dividing the difference in costs by the difference in effect, as is customary in the field of mental health. Bootstrapping will be used to ascertain the amount of uncertainty surrounding the incremental cost effectiveness ratio (ICER) estimates and graphically depicted on the ICER plane and in the acceptability curve.
This study will examine the effectiveness of offering an internet-based guided self-help intervention to patients before to start of face-to-face treatment in comparison with patients who have to wait for face-to-face treatment. We will also compare the uptake of regular treatment in both groups. An economic evaluation will determine whether a guided internet intervention followed by face-to-face treatment is economically more sensible compared to waiting for face-to-face treatment. A number of strengths and challenges have been identified by the researchers in this study.
Existing evidence shows that internet-based treatments are effective in treating depressive disorders in general populations [9, 38]. This trial however, will shed new light on the question whether patients with major depressive disorder waiting for specialized care in MHCs can benefit from this type of internet-based guided self-help intervention. This population has, to the best of our knowledge, not previously been studied in this manner, while the relevance of conducting a study on people with major depressive disorder who receive an internet-based intervention as a first step of treatment is high for a number of reasons. As demonstrated previously, depression is a widely prevalent, invasive disorder, which affects the patients, their direct environments, and the society as a whole in multiple ways. Any measures which contribute to more people receiving better care for their depression in specialized mental healthcare against a lower costs will be welcomed by all stakeholders. Stepped care is suggested for treating depression in multiple guidelines, for example the NICE guidelines in the UK (2009) Although there is some supporting evidence, there are few RCTs to demonstrate the (cost) effectiveness of this program in specialized mental healthcare. Furthermore, considering the increase of online interventions to the standard treatment in more and more MHCs, this study will contribute valuable information on the effects of adding online treatment as a first step towards treating major depressive disorder.
An additional strength of this intervention is the online delivery. The online intervention allows participants not only to access the intervention any convenient hour, with the rise of portable internet there are hardly any limitations related to the location of a participant. In addition, offering guided online treatment during the time otherwise lost to waiting may result in benefits for the patient as well as for the MHCs in terms of patient satisfaction and clinical effectiveness.
A final strength of this study is the possibility of applicability. If the study indicates that it is economically and clinically beneficial to deliver internet interventions to those with a major depressive disorder as part of their treatment, the intervention could easily be integrated as a standard component in the treatment of depression, as only short training is necessary to become a successful coach for this intervention. This intervention could be applied widely in case the intervention is beneficial for treating people with major depressive disorder.
One of the possible challenges of this study concerns the general attrition from and non-adherence to internet interventions. Internet treatments require a degree of motivation of the participant. This study includes depressed patients who, by definition, have impaired motivation and lack of energy that may make it more difficult for patients adhere to the intervention. To prevent dropouts and maximize the uptake rates the participants will receive emails and phone calls stating the importance adhering to the intervention.
Another challenge for the trial could be the prevailing attitude of psychologists at several MHCs who fear redundancy due to internet interventions. However, when MHCs would start treating patients online as a first step towards better health, more patients could be reached as online treatment seems to be less time consuming for health professionals compared to face-to-face treatment.
A further challenge is the extent the therapists take into account that patients might already have acquired skills and knowledge due to the internet intervention, so that there might be no need to start patients a fixed number of face-to-face sessions as prescribed by the standard treatment protocol. We aim that only those patients who require additional treatment due to a more complex psychological problem will be seen face-to-face. Therefore, if internet interventions are proven to be more clinically effective, it is feasible that psychologists at MHCs will be able to treat more patients in a better manner because of the implementation of internet interventions.
In summary, existing internet-based guided self-help treatments focus mainly on depression in the general population. This trial, however, is to the best of our knowledge the first effectiveness study of an internet-based guided self-help intervention for major depressive disorder in specialized mental healthcare that also focuses on reduction of face-to-face sessions and costs in general. The findings of this study will contribute to the body of knowledge on the additional value of internet-based treatments for depression. And if the low intensity internet intervention shows to be (cost) effective, it might serve as a first step towards the treatment of major depressive disorder.
The status of the trial is ongoing recruitment.
This study is funded by ZonMw (project number 80-82310-97-11047).
- Mathers CD, Loncar D: Projections of global mortality and burden of disease from 2002 to 2030. PLoS Med. 2006, 3: e442-10.1371/journal.pmed.0030442.View ArticlePubMedPubMed CentralGoogle Scholar
- Sobocki P, Lekander I, Borgström F, Ström O, Runeson B: The economic burden of depression in Sweden from 1997 to 2005. Eur Psychiatry. 2007, 22: 146-152. 10.1016/j.eurpsy.2006.10.006.View ArticlePubMedGoogle Scholar
- Watkins KE, Burnam MA, Orlando M, Escarce JJ, Huskamp HA, Goldman HH: The health value and cost of care for major depression. Value Health. 2009, 12: 65-72. 10.1111/j.1524-4733.2008.00388.x.View ArticlePubMedGoogle Scholar
- Berto P, D’Ilario D, Ruffo P, Di Virgilio R, Rizzo F: Depression: cost-of-illness studies in the international literature, a review. J Ment Health Policy Econ. 2000, 3: 3-10. 10.1002/1099-176X(200003)3:1<3::AID-MHP68>3.0.CO;2-H.View ArticlePubMedGoogle Scholar
- Nuijen J: Tweedelijns GGZ voor volwassenen: ontwikkelingen in toegang en gebruik. Trendrapportage GGZ 2010 deel 2. Edited by: Nuijen J. 2010, Utrecht, The Netherlands: Trimbos Instituut, 59-114. 1Google Scholar
- Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N: Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One. 2010, 5: e13196-10.1371/journal.pone.0013196.View ArticlePubMedPubMed CentralGoogle Scholar
- McCrone P: Cost-effectiveness of computerised cognitive-behavioural therapy for anxiety and depression in primary care: randomised controlled trial. Br J Psychiatry. 2004, 185: 55-62. 10.1192/bjp.185.1.55.View ArticlePubMedGoogle Scholar
- Cuijpers P, Van Straten A, Schaik A, Andersson G: Psychological treatment of depression in primary care: a meta-analysis. Br J Gen Pract. 2009, 59: e51-e60.View ArticlePubMedPubMed CentralGoogle Scholar
- Richards D, Richardson T: Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012, 32: 329-342. 10.1016/j.cpr.2012.02.004.View ArticlePubMedGoogle Scholar
- Andersson G, Cuijpers P: Internet-based and other computerized psychological treatments for adult depression: a meta analysis. Cogn Behav Ther. 2009, 38: 196-205. 10.1080/16506070903318960.View ArticlePubMedGoogle Scholar
- Tate DF, Finkelstein EA, Khavjou O, Gustafson A: Cost effectiveness of internet interventions: review and recommendations. Ann Behav Med. 2009, 38: 40-45. 10.1007/s12160-009-9131-6.View ArticlePubMedPubMed CentralGoogle Scholar
- Bowman DG: Self-examination therapy: treatment for anxiety and depression. Innovations in Clinical Practice: A Source Book. Edited by: VandeCreek L. 1995, Sarasota, FL: Professional Resource PressGoogle Scholar
- Bowman D, Scogin F, Lyrene B: The efficacy of self-examination therapy and cognitive bibliotherapy in the treatment of mild to moderate depression. Psychother Res. 1995, 5: 131-140. 10.1080/10503309512331331256.View ArticleGoogle Scholar
- Van Straten A, Cuijpers P, Smits N: Effectiveness of a web-based self-help intervention for symptoms of depression, anxiety, and stress: randomized controlled trial. J Med Internet Res. 2008, 10: e7-10.2196/jmir.954.View ArticlePubMedPubMed CentralGoogle Scholar
- Warmerdam L, Van Straten A, Twisk J, Riper H, Cuijpers P: Internet-based treatment for adults with depressive symptoms: randomized controlled trial. J Med Internet Res. 2008, 10: e44-10.2196/jmir.1094.View ArticlePubMedPubMed CentralGoogle Scholar
- Cuijpers P, Donker T, Johansson R, Mohr DC, Van Straten A, Andersson G: Self-guided psychological treatment for depressive symptoms: a meta-analysis. PLoS One. 2011, 6: e21274-10.1371/journal.pone.0021274.View ArticlePubMedPubMed CentralGoogle Scholar
- Radloff LS: The CES-D scale: a self-report depression scale for research in the general population. Appl Psychol Meas. 1997, 1: 385-401.View ArticleGoogle Scholar
- Beekman AT, Deeg DJ, Van Limbeek J, Braam AW, De Vries MZ, Van Tilburg W: Criterion validity of the center for epidemiologic studies depression scale (CES-D): results from a community-based sample of older subjects in The Netherlands. Psychol Med. 1997, 27: 231-235. 10.1017/S0033291796003510.View ArticlePubMedGoogle Scholar
- Donker T, Van Straten A, Marks I, Cuijpers P: Brief self rated screening for depression on the internet. J of Affect Disord. 2010, 122 (3): 253-259. 10.1016/j.jad.2009.07.013.View ArticleGoogle Scholar
- Kessler RC, Abelson J, Demler O, Escobar JI, Gibbon M, Guyer ME, Howes MJ: Clinical calibration of DSM-IV diagnoses in the World Mental Health (WMH) version of the World Health Organization (WHO) Composite International Diagnostic Interview (WMH-CIDI). Int J Methods Psychiatr Res. 2004, 13: 122-139. 10.1002/mpr.169.View ArticlePubMedGoogle Scholar
- Spinhoven P, Ormel J, Sloekers PP, Kempen GI, Speckens AE, Van Hemert AM: A validation study of the Hospital Anxiety and Depression Scale (HADS) in different groups of Dutch subjects. Psychol Med. 1997, 27: 363-370. 10.1017/S0033291796004382.View ArticlePubMedGoogle Scholar
- Bastien CH, Morin CM, Ouellet M-C, Blais FC, Bouchard S: Cognitive-behavioral therapy for insomnia: comparison of individual therapy, group therapy, and telephone consultations. J Consult Clin Psychol. 2004, 72: 653-659.View ArticlePubMedGoogle Scholar
- Brooks R: EuroQol: the current state of play. Health Policy. 1996, 37: 53-72. 10.1016/0168-8510(96)00822-6.View ArticlePubMedGoogle Scholar
- Hakkaart-Van Roijen L, Tan SS, Bouwmans CAM: Handleiding voor kostenonderzoek: methoden en standaard kostprijzen voor economische evaluaties in de gezondheidszorg. 2010, College voor zorgverzekeringen. Geactualiseerde versie,http://www.cvz.nl/binaries/content/documents/zinl-www/documenten/publicaties/overige-publicaties/1007-handleiding-voor-kostenonderzoek/Handleiding+voor+kostenonderzoek.pdf,Google Scholar
- Van Den Brink M, Van Den Hout WB, Stiggelbout AM, Putter H, Van De Velde CJH, Kievit J: Self-reports of health-care utilization: diary or questionnaire?. Int J Technol Assess Health Care. 2005, 21: 298-304.View ArticlePubMedGoogle Scholar
- Smit F, Willemse G, Koopmanschap M, Onrust S, Cuijpers P, Beekman A: Cost-effectiveness of preventing depression in primary care patients: randomised trial. Br J Psychiatry. 2006, 188: 330-336. 10.1192/bjp.188.4.330.View ArticlePubMedGoogle Scholar
- Pearlin LI, Schooler C: The structure of coping. J Health Soc Behav. 1978, 19: 2-21. 10.2307/2136319.View ArticlePubMedGoogle Scholar
- Ewing JA: Detecting alcoholism: the CAGE questionnaire. JAMA. 1984, 252: 1905-1907. 10.1001/jama.1984.03350140051025.View ArticlePubMedGoogle Scholar
- O’Brien CP: The CAGE questionnaire for detection of alcoholism: a remarkably useful but simple tool. JAMA. 2008, 300: 2054-2056. 10.1001/jama.2008.570.View ArticlePubMedGoogle Scholar
- Costa PT, Mccrae RR: Normal personality assessment in clinical practice: the NEO personality inventory. Psychol Assess. 1992, 4: 5-13.View ArticleGoogle Scholar
- Clark LA, Watson D, Mineka S: Temperament, personality, and the mood and anxiety disorders. J Abnorm Psychol. 1994, 103: 103-116.View ArticlePubMedGoogle Scholar
- Devilly GJ, Borkovec TD: Psychometric properties of the credibility/expectancy questionnaire. J Behav Ther Exp Psychiatry. 2000, 31: 73-86. 10.1016/S0005-7916(00)00012-4.View ArticlePubMedGoogle Scholar
- De Brey H: A cross-national validation of the client satisfaction questionnaire: the Dutch experience. Eval Program Plann. 1983, 6: 395-400. 10.1016/0149-7189(83)90018-6.View ArticlePubMedGoogle Scholar
- Schulz KF, Altman DG, Moher D, for the CONSORT Group: CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. Br Med J. 2010, 340: c332-10.1136/bmj.c332.View ArticleGoogle Scholar
- Little RJA, Rubin DB: The analysis of social science data with missing values. Sociol Methods Res. 1989, 18: 292-326. 10.1177/0049124189018002004.View ArticleGoogle Scholar
- Cohen J: Statistical Power Analysis for the Behavioral Sciences. 1988, Hillsdale NJ: ErlbaumGoogle Scholar
- Koopmanschap MA, Van Exel NJ, Van Den Bos GA, Van Den Berg B, Brouwer WB: The desire for support and respite care preferences of Dutch informal caregivers. Health Policy. 2004, 68: 309-320. 10.1016/j.healthpol.2003.10.002.View ArticlePubMedGoogle Scholar
- Sarrami Foroushani P, Schneider J, Assareh N: Meta-review of the effectiveness of computerised CBT in treating depression. BMC Psychiatry. 2011, 11: 131-10.1186/1471-244X-11-131.View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.