Skip to main content
Download PDF
  • Study protocol
  • Open access
  • Published:

The clinical and cost-effectiveness of a self-management intervention for patients with persistent depressive disorder and their partners/caregivers: study protocol of a multicenter pragmatic randomized controlled trial

Trials volume 22, Article number: 731 (2021) Cite this article

Abstract

Background

After regular treatment, patients with persistent depressive disorder (PDD) may remain in specialized psychiatric outpatient care without achieving remission. Lacking other options, these patients often receive long-term, non-protocolized care as usual (CAU) that does not involve the partner/caregiver of the patient. Although the revised depression treatment guidelines suggest focusing on psychiatric rehabilitation and self-management as the next treatment step for PDD, an evidence-based cost-effective self-management protocol for PDD is lacking. This study investigates the “Patient and Partner Education Program for All Chronic Illnesses” (PPEP4All) as a brief self-management protocol that could lead to lower costs, higher quality of life, and less disease burden in PDD patients and their partners/caregivers.

Methods

Presented is the rationale and methods of a multicenter pragmatic randomized controlled trial to evaluate the clinical efficacy and cost-effectiveness of PPEP4All for patients with PDD and their partners/caregivers. In accordance with current recommendations, a mixed methods research approach is used with both quantitative and qualitative data. A total of 178 eligible outpatients with PDD and their partners/caregivers are recruited and randomized to either PPEP4All or CAU. Those assigned to PPEP4All receive nine weekly self-management sessions with a trained PPEP4All therapist. Primary and secondary outcome measurements are at 0, 3, 6, and 12 months.

Discussion

This project will result in the implementation of a self-management intervention for patients with PDD, meeting an urgent need in mental healthcare. Using PPEP4All can optimize the quality and efficiency of care for both patients with PDD and their partners/caregivers.

Trial registration

Netherlands Trial Register Identifier NTR5973. Registered on 20 July 2016.

Peer Review reports

Background

Approximately 10–17% of persons with major depressive disorder (MDD) suffer a chronic course, or a continuous period of at least 2 years [1]. Two types of depressive disorders have a prolonged duration: chronic major depressive disorder and dysthymic disorder [2]. These were combined into one syndrome, persistent depressive disorder (PDD), in the latest Diagnostic and Statistical Manual of Mental Disorders (DSM-5) [3]. Compared to less persistent forms of depression, PDD has a 3–6% lifetime prevalence [3,4,5,6,7] and adverse consequences for patients, such as impaired functioning in social and family relationships, high disease burden for both patient and partner/caregiver, reduced performance in work, comorbid psychiatric and somatic problems, and an increased suicide risk [8, 9].

Many patients with PDD remain in specialized outpatient care without achieving remission, even after having been treated according to the international multidisciplinary depression guidelines [4, 10,11,12]. Clinicians are resigned to providing long-term supportive non-specific treatment (i.e., care as usual), due to the considerable suffering of patients and the absence of alternative treatment options [13]. More intensive prolonged care, however, often fails to improve therapeutic response; previous research has shown that patients with increasing depression symptom levels have a poor treatment prognosis despite receiving more intensive care [14,15,16]. The ongoing unsuccessful treatment leaves both patients and clinicians feeling powerless and frustrated [13]. Moreover, this treatment often disregards the partner/caregiver of the patient, although involvement of the partner/caregiver has been shown to be beneficial in improving treatment results of the patient and reducing the partner’s psychosocial burden concerning patient’s disease [17,18,19,20,21].

The revised multidisciplinary guidelines for depression treatment [11, 22,23,24,25,26,27] advise focusing on psychiatric rehabilitation as the next therapeutic step for PDD patients with inadequate treatment results. Psychiatric rehabilitation using self-management interventions focuses less on symptom recovery and more on restoring psychosocial functioning and enhancing patient autonomy. Individuals with PDD learn to set realistic goals (e.g., regarding work reintegration) and to cope by adjusting their activities to compensate for restrictions caused by the chronic illness. To date, there is still an urgent need in specialized mental healthcare for a brief evidence-based, cost-effective treatment protocol concerning self-management for treatment-resisting persons with PDD that also involves the partner/caregiver and replaces long-term non-specific usual care.

As a possible solution, we propose the use of the self-management program named the “Patient and Partner Education Program for All Chronic Illnesses” (PPEP4All). This program was originally named the “Patient and Partner Education Program” (PPEP) and developed for homogenous groups of patients with chronic somatic diseases, like Parkinson’s disease and Huntington’s disease, and tested by EduPark, a research consortium with seven participating European countries [18, 28,29,30]. Later, PPEP was tested in a heterogeneous group of patients with various chronic somatic disorders and comorbid long-term depression and/or anxiety, with positive results, after which the program name was changed to PPEP4All [18]. The structure and substance of the eight sessions are the same; however, the wording of the program and workbooks was altered for inclusivity of various diseases [31,32,33], and the present PPEP4All program includes one additional session regarding coping with chronic depression (i.e., “Suicide prevention & relapse plan”).

Research has already shown that PPEP is effective: patients with Parkinson’s disease showed a significant improvement in quality of life after completion of PPEP [19]. Moreover, patients with symptom-present Huntington’s disease showed a significant improvement in active coping and seeking social support and reported fewer behavioral problems and less anxiety. After 6 months, they also showed an improvement in psychosocial burden related to their chronic illness [34]. Patients with chronic pituitary disorders showed significant higher self-efficacy after following PPEP [35]. Moreover, after following PPEP4All, patients with various chronic somatic disorders and comorbid chronic depression showed an improvement in depression scores [18]; an indication that PPEP4All could also be applied to mental healthcare, which was the starting point for the present study. The cost-effectiveness of PPEP/PPEP4All has not yet been investigated.

The present study evaluates the clinical- and cost-effectiveness of PPEP4All in patients with PDD and their partners/caregivers. Additionally, this project aims to further our understanding of the healthcare needs for patients with PDD and elicit direct feedback about PPEP4All from patients, partners/caregivers, and clinicians via qualitative interviews. Compared to usual care for PDD, we expect that PPEP4All will lead to an improved quality of life, fewer psychiatric symptoms, and more mental resilience in patients (using superiority testing). Second, we expect that PPEP4All will lead to lower psychosocial burden from chronic depression for both patients and their partner/caregiver. Third, due to the briefness and anticipated higher effectiveness of PPEP4All, we expect lower societal and healthcare costs, compared to care as usual.

Methods

Study design

In this multicenter, pragmatic randomized controlled trial (RCT), the protocolized PPEP4All self-management intervention is evaluated against care as usual (CAU) in specialized outpatient mental health care clinics in the Netherlands. Measurements are at 0 (pre-treatment), 3 (post-treatment), 6 (first follow-up), and 12 months (second follow-up), with assessors/data-analysts blind to randomization status. In accordance with current recommendations (in the context of RCTs) [36,37,38,39,40,41], we use a mixed methods research approach with both quantitative and qualitative data, where the qualitative study is nested in the main RCT project. The qualitative data complements the quantitative data and can facilitate the practical implementation of the intervention, strengthen the clinical relevance of our results, and optimize the PPEP4All intervention by adding the perspectives of patients, partners/caregivers, and clinicians. Moreover, the study protocol conforms to the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guidelines (see Additional file 1).

Setting and participants

Eligible patients are recruited from five Dutch mental healthcare organizations, with 11 individual locations, that offer outpatient treatment for depressive disorders (see Acknowledgements). The primary/lead research center of the study is the Department of Psychiatry of the Leiden University Medical Center (LUMC). The inclusion criteria for participating patients are as follows: (1) diagnosis of recurrent or chronic major depressive disorder (duration of at least 2 years) according to the DSM-IV [2] or PDD according to the DSM-5 (non-severe comorbidity is allowed, e.g., anxiety disorder) [3]; (2) age of 18 years or older; (3) treatment indication for rehabilitation as specified by the clinician, with a minimum of at least one previous psychological treatment and at least two medication trials, with unsatisfactory results, according to the international multidisciplinary guidelines for depressive disorders [11, 22,23,24,25,26,27]. Patients with recurrent or chronic depression who do not wish to complete all treatment steps of the depression guideline (e.g., refuse medication) can also enroll in the study. Patients with bipolar disorder type II may also participate considering they generally spend much more time in a depressed state than in a hypomanic state, making it difficult to differentiate from recurrent depression [42]. Exclusion criteria for patients are as follows: severe psychopathology (e.g., schizophrenia, current psychotic state, severe substance addiction, bipolar disorder type I); acute and severe risk of suicide; severe somatic disorders that are too disabling or render the patient immobile; severe cognitive problems such as dementia; expected dosage adjustments of medication during PPEP4All; and insufficient fluency in the Dutch language.

The inclusion criteria for partner/caregiver to participate in the study and PPEP4All program for partners/caregivers are as follows: (1) ability to participate in at least three sessions of the intervention, (2) not currently receiving active psychotherapy, and (3) agreement from the patient for the partner/caregiver to participate.

Recruitment, enrollment, and allocation

Psychiatric nurses and psychiatrists of the participating mental healthcare locations identify patients eligible for participation according to the above inclusion/exclusion criteria, and they introduce research participation and provide information letters. Patients who are interested and consent to participate in the study sign an informed consent form, which is then sent to the primary research center (LUMC). On this informed consent form, participants also indicate whether they want to be invited to a follow-up nested qualitative interview study in which we evaluate their satisfaction with PPEP4All and their process of implementing coping strategies for chronic depression (see section “Qualitative data”).

To check the inclusion and exclusion criteria formally, a trained research assistant checks the DSM-IV depressive disorders with the Dutch translation of the Mini-International Neuropsychiatric Interview (MINI interview, modules A, B, and C regarding depression, dysthymia, and suicidality). The MINI interview is a well-validated diagnostic interview used to identify psychiatric disorders, with excellent interrater and test-retest reliability [43, 44].

Allocation of participants is performed by an independent data coordinator/data manager, who is not involved in the data collection/analysis (GC; see Acknowledgements). Participants are allocated to either PPEP4All or CAU, using a stratified randomization schedule (stratified by gender and research center) designed by an independent statistician of the Department of Medical Statistics and Bioinformatics of the LUMC.

For the patients allocated to PPEP4All, their main caregiver (e.g., partner, close family member, or close friend) is invited to participate in the project. The data manager makes an appointment with the partner/caregiver, and they sign an informed consent before the first measurement. All participants are informed that participation is voluntary and that they can withdraw from the study at any time without consequences. Participants receive a 20-euro gift card to thank them for their efforts. An overview of the study design and patient flow is provided in Fig. 1.

Fig. 1
figure 1

Study design and flow. Notes: MINI = Mini-International Neuropsychiatric Interview. PDD = Persistent depressive disorder. PPEP4All = Patient and Partner Education Program for All Chronic Illnesses. CAU = care as usual

Full size image

Assessment and instruments

As mentioned, the measurement timepoints are at 0, 3, 6, and 12 months; the latter two are follow-up measurements. The first measurement is completed within 1 month prior to the start of the treatment. The study is conducted using a Routine Outcome Monitoring (ROM) system, which periodically measures the presence and severity of psychiatric symptoms in patients with a battery of psychometric instruments and thus monitors therapeutic progress [45,46,47]. To assist in the administration of the ROM measurements, a web-based application called QuestManager is available. At each timepoint, the questionnaires are completed online via QuestManager (see Fig. 2 for the study schedule and overview of the questionnaires). If preferred or necessary (e.g., computer problems), questionnaires may be completed on paper. The participant may choose to complete the questionnaires at the location of their choice: at their home, the mental health clinic, the research center, or by telephone.

Fig. 2
figure 2

Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) schedule of enrollment, interventions, and assessment of the study. Notes: MINI = Mini-International Neuropsychiatric Interview [35, 36]. EQ-5D-5L = EuroQoL-5 with visual analogue scale [40,41,42,43,44]. TiC-P = Trimbos Medical Technology Assessment questionnaire for psychiatric illness-associated costs [45]. IDS-SR = Inventory of Depressive Symptomatology, self-rated [46,47,48]. SQ-48 = Symptom Questionnaire-48 [49,50,51,52]. SRH = self-rated happiness [53,54,55]. BRSnl = Brief Resilience Scale [56, 57]. B4CZ = Questionnaire on Burden of Chronic Disease for Patients (or Partners) [58, 59]. aThe PPEP4All intervention is mainly provided as a group treatment but can also be offered as an individual treatment. bThese questionnaires are administered to partners/caregivers in the study

Full size image

At the first measurement (i.e., 0 months), participant demographics are registered, and the primary and secondary outcome questionnaires are completed. At the second, third, and fourth measurements (i.e., 3, 6, and 12 months), the outcome questionnaires are again completed.

The primary outcome measures pertain to quality of life and healthcare costs (see Fig. 2). Quality of life is measured for both patients and partners/caregivers using the EuroQoL-5 (EQ-5D-5L, including the visual analogue scale, 5 items) [48,49,50,51,52]. Using the EQ-5D-5L, we calculate the mean quality-adjusted life years (QALYs) gained, which is used in the cost-effectiveness analysis (CEA; see “Economic evaluation”). Healthcare consumption (such as contact with healthcare professionals, medical costs, and productivity loss) are measured with the Trimbos Medical Technology Assessment questionnaire for psychiatric illness-associated costs (TiC-P; 30 items) [53].

The secondary outcome measures are shown in Fig. 2. Symptoms, psychopathology, and well-being, for patients only, are measured using the Inventory of Depressive Symptomatology, Self-Report (IDS-SR; 30 items) [54,55,56]; the Symptom Questionnaire-48 (SQ-48; 48 items) [57,58,59,60]; and the Self-Rated Happiness survey (SRH; 1 item) [61,62,63], respectively. In addition, mental resilience of patients is measured using the Brief Resilience Scale (BRSnl; 6 items) [64, 65]. The burden of chronic disease for both the patient and partner/caregiver is measured by the Questionnaire on Burden of Chronic Disease for Patients (B4CZ; 20 items), and the Questionnaire on Burden of Chronic Disease for Partners (B4CZ partners; 16 items), respectively [66, 67]. Directly after the final measurement, patients complete an additional evaluation survey pertaining to the following: (1) treatment modality (group or individual treatment), (2) treatment satisfaction, (3) number of therapy sessions, (4) medication changes (yes/no); and (5) PPEP4All treatment adherence.

Interventions

Self-management intervention (PPEP4All)

Each of the nine PPEP4All sessions focuses on specific self-management themes, such as psychoeducation about chronic depression, stress-management using cognitive restructuring and relaxation techniques, reactivation with positive activity planning, social skills building, and mobilization of one’s social network. Although the original PPEP4All program had eight sessions, it has been suggested that an extra PPEP4All session may help sustain enhanced quality of life [19]. PPEP4All in this project includes a ninth session focusing on suicidality, dealing with crises, and relapse prevention. Regarding the latter, participants make a recovery or self-care action plan together with the PPEP4All therapist, using the skills and information learned during the program. For an overview of the PPEP4All self-management themes, see Fig. 3.

Fig. 3
figure 3

An overview of the self-management themes of the PPEP4All sessions for the patient and the partner/caregiver. Notes: PPEP4All = Patient and Partner Education Program for All Chronic Illnesses. Patient and partner/caregiver programs include the same themes with the exception of sessions 5 and 8. Each session has a general structure of discussing previous homework, presenting active information (i.e., the theme), performing an active exercise, then providing homework and a preview of the next session. The trainer is the PPEP4All therapist. aParticipants also gain insight by using a body awareness exercise. bIn partner/caregiver program, session 5 focuses on caregiver burden and stress (overload) and relevant challenges. cIn partner/caregiver program, session 8 focuses on making a self-care plan

Full size image

The partner/caregiver program (separate partner group) is the same as the patient program with only one difference: session five focuses on dealing with caregiver burden and patient suicidality (see Fig. 3). The patient and partner/caregiver receive PPEP4All workbooks with homework assignments after each session to practice skills and integrate information. Although the original protocol describes a minimum of 5 and a maximum of 10 participants per group [17], we allowed groups to start with as few as 3–4 participants to avoid undue waiting times.

The PPEP4All program is eclectic, including theoretical influences of system theory (“patient system”) [68, 69], cognitive behavioral theory (“(dys) functional cognitions”) [70], social cognitive theory (“self-efficacy”) [71], stress-coping model (“coping strategies”) [72, 73], transtheoretical model (“motivation to change”) [74]; bio-psycho-social model (quality of life) [75,76,77], and generic model of self-management [78].

At the time of this project, there are other self-management programs available, such as the Wellness Recovery Action Plan (WRAP; a self-designed prevention and wellness process, mainly used for severe mental illness) [79, 80]; Illness Management and Recovery (IMR; a standardized psychosocial intervention for severe mental illness like schizophrenia) [81]; and Self-management for Chronic Anxiety and Depression (SemCAD; for chronic depressive and anxiety disorders) [82,83,84]. PPEP4All, however, has advantages when compared to these other interventions. First, PPEP4All involves and engages the partner/caregiver in the treatment process, which has been shown to be effective in reducing the partners’ psychosocial burden concerning the patients’ illness as well as the patients’ outcome [17,18,19,20,21]. A key feature of PPEP4All is that the patient and partner/caregiver receive separate sessions. Maintaining separate sessions allows both patient and partner/caregiver to speak freely about their personal situations. Second, the program can be offered as a group or individual intervention. The treatment modality is determined by the patient and his or her therapist in a process of shared decision making. Third, patients with depression often have comorbid psychiatric or somatic disorders [14, 85]; the PPEP4All program provides a general toolkit with which all these issues can be addressed. Fourth, compared to the other self-management programs, each session of PPEP4All is highly structured, with both patients and partner/caregivers working according to their PPEP4All workbook.

The clinicians of the participating mental health clinics who administer the program are called PPEP4All therapists. Each therapist receives a treatment manual and completes a 3-day certified course with the PPEP4All founder [17, 18, 31, 32, 86, 87]. The training combines homework/self-study with lectures, assignments, group discussions, role play, and self-assessment. The 34 PPEP4All therapists in this project are mainly psychiatric nurses or psychiatric nurse specialists and three are psychologists. After completing their PPEP4All training, all PPEP4All therapists in this study could receive continuous support with regular booster sessions, intervision, consultation with the PPEP4All founder, and an annual national PPEP4ALL symposium.

Care as usual (CAU)

To compare PPEP4All with what is currently being offered in mental health clinics, the participants randomized to the control group receive care as usual (CAU), thus increasing the ecological validity of the present study. This standard treatment for patients with PDD concerns mostly long-term, non-protocolized supportive care from a psychiatric nurse, with pharmacological maintenance therapy from a psychiatrist. CAU mostly has an individual treatment modality and does not include the partner/caregiver of the patient. Patients allocated to CAU in this study generally continue the standard treatment with their own therapist.

Treatment integrity

To evaluate the treatment integrity of PPEP4All, each PPEP4All therapist completes a therapy protocol checklist at the end of each group or individual session. On this protocol checklist, the PPEP4All therapist indicates whether and how the provided treatment deviated from the treatment protocol (e.g., the themes mentioned in Fig. 3). It is not possible to record therapy sessions with a video or audio recorder in participating mental health clinics due to privacy law and regulations. Morever, participant attendance is recorded on a separate attendance list and provided to the researchers by the PPEP4All therapist. For CAU, without a specific treatment protocol, we cannot assess its treatment integrity. However, after the final fourth measurement, we collect information about treatment type and modality within CAU, including treatment satisfaction and general medication changes during the course of the research project.

Sample size and power calculation

We based our sample size on the effect size of previously conducted randomized controlled trials that evaluated relevant interventions in a similar setting in patients with chronic or treatment-resistant depression and that reported a quality of life outcome measure [83, 84, 88,89,90]. We can expect an effect size around 0.35 [90]. Considering a power level of 80%, alpha of 95%, and equal allocation ratio, we would require 81 participants per group [91,92,93,94,95]. With an additional 16 persons to account for an expected attrition of 10% [96], we expect to include a total sample of 178 participants (2 × 89).

Data analyses

In this project, we examine both quantitative and qualitative data.

Quantitative data

Using the data collected from the questionnaires, we examine the clinical and economic effectiveness of PPEP4All (in collaboration with the LUMC Department of Medical Decision Making). This is further specified below.

Clinical effectiveness

The mean quality-adjusted life years (QALYs) are estimated using EQ-5D-5L data from four assessments. For the primary data analysis, we perform an analysis of covariance for the QALY’s using the post-test EQ-5D-5L results with the pre-test scores as covariate [92,93,94]. In this way, we test whether there is a difference between PPEP4All and CAU, accounting for the variation where the patients started at the first measurement. For the secondary data analyses, secondary parameters are tested using weighted generalized estimating equation (GEE) analyses [97, 98]. Clinical effectiveness analyses are examined using superiority testing.

Moreover, we examine tailored care by further investigating for which groups PPEP4All is the most effective. If possible, subgroup analyses are performed for age, gender, ethnicity, and intervention modality (group/individual). In this context, we explore relevant socio-demographics (e.g., education, living situation, and work) and clinical variables (e.g., comorbidity, baseline severity of disease).

For all outcome analyses, an intention-to-treat (ITT) approach is used, according to the CONSORT statement [99]. Analyses are conducted in SPSS, Stata or R.

Economic evaluation

We perform two economic evaluation analyses (see below). The first is a cost-effectiveness analysis (CEA) to determine whether PPEP4All (compared to CAU) is favorably cost-effective at 1-year follow-up. The second is a budget impact analysis (BIA). The economic evaluation is based on the TiC-P questionnaire (see Fig. 2). The TiC-P questionnaire assesses healthcare utilization such as general practitioner (GP) visits and non-healthcare use such as absence from work, reduced efficiency at work, difficulties with job performance (absenteeism from paid work), and production losses without absenteeism from paid work (e.g., presenteeism). Using the area-under-the-curve method for the utility scores results in QALY outcome per patient.

Cost-effectiveness analysis (CEA)

Direct costs per patient in PPEP4All versus the costs per patient in CAU are compared. The differences in mean costs and effects between strategies are compared with two-sided bootstrapping. In a net-benefit analysis, costs are related to patient-reported outcomes and presented in a cost-effectiveness acceptability curve (CEAC), which indicates the probability of PPEP4All being cost-effective. No discounting is applied considering the time horizon of 1 year. This economic evaluation is approached from a societal perspective and includes costs due to healthcare resource utilization (i.e., healthcare costs) and costs attributable to production losses [100]. Sensitivity analysis will test whether estimates are sensitive to plausible changes in perspectives (societal or healthcare perspective) and indirect costs (friction cost method and human capital approach). Furthermore, in an additional analysis, we extrapolate cost and effect estimates to a lifetime time horizon, in accordance with the recent Dutch directive for conducting economic evaluations in healthcare [101, 102]. In the base-case analysis, costs of absenteeism from paid work are calculated using the friction cost method, which estimates the indirect costs of disease, mainly occurring during the time it takes to replace a worker [103,104,105,106].

In this project, costs are calculated using standard unit prices published in the most recent Dutch Costing Manual [101, 102]. If references are not available, costs are estimated by cost research experts (i.e., LUMC Department of Medical Decision Making). The costs are divided into healthcare costs, costs of patients and partner/caregiver, and costs in other sectors (i.e., non-healthcare costs such as productivity costs). Healthcare costs include the costs of PPEP4All and other healthcare use during the first year of follow-up (e.g., GP visits, outpatient visits, hospital days, medication, home care, informal care). Costs of PPEP4All are based on micro-costing including time of caregivers and materials used. Costs of patients and partners/caregivers consist of time lost or productivity loss from paid/unpaid work of the patient, and time required for the PPEP4All meetings by partners/care givers.

Budget impact analysis (BIA)

In addition to the CEA, a BIA is performed according to the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Task Force principles [107, 108]. The BIA estimates the financial impact of implementation (i.e., adoption and diffusion) of PPEP4All in patients with PDD at the national level. The analysis is based on the costs estimated during the study and the expected number of patients eligible for this treatment in the Netherlands. Costs of the treatment and other consequences of it will be included in the BIA. The BIA is conducted from the perspective of the different healthcare payers (public purse or budgetary framework [in Dutch: Budgetair Kader Zorg (BKZ)]; health insurers; health care providers) and from the societal perspective (i.e., including productivity costs). If the PPEP4All intervention appears to be cost-effective in patients with PDD, the following scenarios will be compared: (a) intervention not yet implemented, (b) intervention implemented in 100% of the target group, (c) intervention gradually introduced over a period of 3 years. Sensitivity analysis is performed on the costs of the intervention and the diffusion rate. Factors that determine the budget impact are determined. Costs are estimated per budget period (1 year) for a time horizon of 3 years. In the societal perspective and healthcare perspective, standard prices will be used [101]. For the BKZ and health insurer’s perspective, tariffs established by the Dutch Healthcare Authority (Nederlandse Zorgautoriteit (NZA)) are used.

Qualitative data

All patients and partners/caregivers who give consent to be approached for the nested follow-up qualitative study are invited by telephone to participate. The participant may choose where the in-depth semi-structured interview is planned: at the participant’s home, at the main research location (LUMC), or over telephone. Participants are compensated for travel and given a 20-euro gift card to thank them for their time. Participants are allowed to withdraw at any moment of the qualitative study.

For each interview, we use topic lists, or interview guides (see Supplementary Topic Lists in Additional Files 2 and 3), which were evaluated in a pilot study of patients with PDD and revised as necessary [109, 110]. The topics and questions are presented to the participant as they naturally arise during the interview, to maintain the flow of the interview. The qualitative study includes two parts that examined: (a) use of coping strategies of PDD patients and their partners/caregivers for chronic depression/suicidality and their specific needs for care; (b) satisfaction of patients, partners/caregivers, and therapists with PPEP4All (or CAU). The two parts are discussed in further detail below (see “Coping with chronic depression” and “Satisfaction”).

The interviews are recorded using a digital audio recorder and then transcribed by research assistants using the dictation feature of the online Transcribe application (https://transcribe.wreally.com). This application does not store documents; audio files are deleted after the window is closed, making it a secure method that maintains confidentiality after transcription work. Personal data that can be traced back to the participant (e.g., names of participant, family members, or employer) are removed, and transcripts are saved under a participant code. The verified transcripts are then checked and analyzed by the researcher (ES) and a research assistant. Each interview and the resulting themes are discussed with the research assistant and tracked using an audit trail, to provide systematic and detailed documentation of analytical decision making during the process. Memo writing is utilized throughout the analytic process. Discrepancies in coding are discussed until consensus on the data is established. Because we used a qualitative emergent approach focused on gaining new insights, rather than developing a theory, our analytical approach in this study can be described as using “Grounded Theory lite” [111,112,113,114,115,116]. For the analysis, using Atlas.ti version 9, transcripts are first coded line-by-line, then coded axially into emergent conceptual categories [113, 116, 117]. We also use constant comparison, a method in which we continuously compare the concepts or themes between data and emergent coded categories and the data of the interviews [117, 118]. The main goal of the study is greater conceptual clarity and to build a conceptual framework of the experiences of participants with chronic depression regarding their treatment needs and use of self-management to meet these needs [119].

Coping with chronic depression

First, we aim to answer the following research questions: how do individuals with chronic depression manage their chronic condition, and what are their experiences in relation to potential facilitators and barriers, personal strengths and hindrances, and use of self-management coping strategies? (see Supplementary Topic List A, or Additional File 2, which is partly based on a previous relevant topic list [110]). For this part of the qualitative study, we predetermined the number of required participants to ensure a sufficient sample. According to guidelines, we aim to include between 25 and 40 patients from the PPEP4All or CAU groups and 15 PPEP4All partners/caregivers to reach saturation of the themes [120,121,122].

Satisfaction

Second, we aim to evaluate satisfaction with PPEP4All and investigate how we can optimize it to meet the needs of the patients with PDD and their partners/caregivers. Our approach is multi-faceted: we conduct in-depth interviews with patients, partners/caregivers, and PPEP4All therapists (see Supplementary Topic List B, or Additional File 3, which is partly based on the Mental Healthcare Thermometer [109]), and we collect data using an evaluation questionnaire at the end of the fourth quantitative measurement to elucidate the positive and negative aspects of PPEP4All or CAU. During the interview, participants may provide feedback and suggestions regarding PPEP4All, and they rate the PPEP4All program on a scale from 1 to 10. We will provide the average rating for each group. This satisfaction rating reflects an indirect measurement of the effectiveness of PPEP4All. For this part of the qualitative study, we aim to include between 25 PPEP4All patients, 15 PPEP4All partners/caregivers, and 10 PPEP4All therapists [120,121,122]. In addition, we strive to measure satisfaction of participants of CAU. The procedure for evaluating CAU is the same as PPEP4All: we use an evaluation questionnaire and a qualitative interview. During the interview, participants can rate CAU on a scale from 1 to 10 and provide additional information regarding their experience with CAU, such as suggestions regarding what kind of treatment techniques they would prefer as CAU.

Ethics, data management, and dissemination

The study protocol, informed consent forms, participant education, and recruitment materials were approved by the Medical Ethics Committee (MEC) of the LUMC. If there are any subsequent modifications to the protocol which may impact the conduct of the study, including changes of study objectives, patient population, sample sizes, study procedures, or significant administrative aspects, these will be submitted as an amendment and reviewed by the MEC of the LUMC for approval. The approved protocol modifications will be communicated to research locations and participants by email or newsletter. Subsequent to initial review and approval, the MEC will review the protocol annually and the researcher will make safety and progress reports to the MEC annually and within 3 months of study termination. These reports will include the total number of participants enrolled and any particularities that should be reported to the MEC. Due to the low risk associated with study participation, annual audit of the study, data monitoring committee, and interim analyses are not required. Additionally, a data safety monitoring board is not required because we do not investigate a medical product and we do not foresee any major risks associated with study participation or with participation in the intervention provided by trained therapists.

Written informed consent will be obtained from all participants prior to the baseline assessment. All informed consent forms will be arranged and co-signed by the researcher or trained research assistant who provided information regarding the study. For the nested qualitative study, each participant gave written consent to be approached for participation in this additional study. Participants thereafter gave verbal consent to the study including consent to audio recording and transcribe their interviews.

In general, we followed the guidelines for Good Research Practice (GRP) [123,124,125,126]. To protect the confidentiality of data and privacy of patients, we use a unique research code for each participant on all research-related participant forms, such as contact forms or paper questionnaires. This enables the research team to identify individuals without using names. The list linking the participant code to personal information is kept in a secure electronic database with access limited to the designated researcher (ES) and independent data coordinator (GC). The list may be assessed and the main mental health clinician of the patient informed, if required for the sake of the safety of the participant. All electronic research-related participant information is stored on a protected network in a secure file with limited access. The hardcopy paper research information is kept in a locked cabinet in a secure area of the LUMC department. Any documents with information traceable to the participant are kept in locked files separate from the patient data with limited access. To uphold high standards of research, all researchers, research coordinators, and research assistants successfully completed the research certification for Good Research Practice (GRP). Additionally, all research team members completed a training workshop regarding the trial procedures and questionnaires, use of QuestManager, and administration of the MINI screening interview.

Moreover, after the data analysis phase, patient privacy will be further maintained. The results obtained from the questionnaires will be described on a group level to prevent the data being traced back to a single person. Data will be kept on the secure network for maximum 3 months at the end of the research project as the embargo period, for the sole purpose of completing data analysis, as per the policy of the funding sponsor. Once the final report of the study is available, the study results will be extensively disseminated to the international scientific community in the form of peer-reviewed journal articles, giving preference to open-access journals.

After the embargo period, the pseudonymized data will be stored for 15 years on an online Dutch meta-data catalogue called the Data Archiving and Networked Services (DANS, www.dans.knaw.nl) together with a data dictionary, as is common practice for mental healthcare research in the Netherlands. Only the designated researchers will have access to the final data set. External researchers may get access to the final trial dataset from the designated team on reasonable request.

Discussion

Scientific and clinical importance of this study

The present study examines the clinical- and cost-effectiveness of a brief self-management intervention, namely PPEP4All, in patients with PDD and their partners/caregivers. Although self-management is more common in the treatment of somatic illnesses, it is gaining popularity in mental healthcare. Psychiatric rehabilitation via self-management is the next step in the revised international multidisciplinary treatment guidelines for chronic depression after psychotherapy and pharmacotherapy have been attempted with inadequate results. In this context, there is an urgent need in mental healthcare for a brief specific self-management protocol that could replace the long-term non-specific care for patients with PDD (care as usual).

Study implications

The findings of this study will have potential implications for the implementation of self-management on a national level in the Netherlands and could be further incorporated into the multidisciplinary treatment guidelines for depression. The Netherlands has recently seen a shift in the mental healthcare system due to tighter healthcare budgets. Mental healthcare clinics are looking for empirically supported protocolized treatments that can reduce healthcare costs. With this in mind, PPEP4All is a brief self-management intervention that can be provided when depression treatment leads to unsatisfactory reduction of symptoms, typically after a 2-year treatment period. The expectation is that empowering patients via PPEP4All will decrease the number of ongoing supportive sessions of CAU, for which there is currently little to no empirical evidence [13].

The present study has several strengths. First, in line with the pragmatic nature of the project, we focus on external validity: we evaluate the standard care (CAU) already provided by healthcare professionals in multiple mental healthcare clinics. Moreover, the broad inclusion criteria allow the recruitment of patients with a wide range of comorbidity, as typically seen among persons with chronic or recurrent depression. We expect the resulting sample to be broadly representative of outpatients and generalizable to the population of persons with chronic depression. We selected only the most crucial research questionnaires to limit the burden on the patients; therefore, we expect limited effect of the test battery on patient drop-out. Additionally, by using a mixed methods research approach, we are able to evaluate both quantitative and qualitative aspects of PPEP4All from multiple perspectives (patient, partner/caregiver, clinician).

Challenges concerning trial recruitment and implementation

We wish to report some early challenges we faced in the recruitment of participants and implementation of our study design and PPEP4All in the participating research centers. After all, patient recruitment is a key determinant of success or failure for clinical trials [127,128,129].

Firstly, we learned that recruiting only patients with a primary diagnosis of chronic depression/PDD limited our ability to also recruit potential participants with secondary, or comorbid, PDD (e.g., bipolar disorder type II participants with comorbid PDD). After careful deliberation, we allowed these participants to enroll in our study considering that PPEP4All is a generic self-management program that could potentially be used for a wide range of psychiatric disorders and its focus is on recovery, stabilization, and reintegration into the community [130].

Secondly, we had to make the following adjustments to reduce participant drop-out [93]. Initially, we had proposed providing PPEP4All as group treatment, because a group could be the most cost-effective and efficient modality for mental healthcare. However, several patients with PDD preferred individual PPEP4All and therefore refused to participate. This possible preference for individual PPEP4All was also reported to us by the Client Council of the participating centers. For this reason, although PPEP4All is still primarily offered as a group treatment, we also allow the option to provide PPEP4All as an individual treatment (same protocol). In addition, for patients with a preference for group PPEP4All, we lowered the minimum threshold of required group participants to three rather than the original five, which allowed us to start PPEP4All groups without a long waiting period. Moreover, some patients who explicitly wanted PPEP4All withdrew their study participation after they were randomized to CAU. We remedied this with the message that, if necessary, these patients could receive PPEP4All after CAU and completing at least their first or second follow-up.

Conclusion

This self-management intervention, PPEP4All, could fill an urgent need in mental healthcare and provide further support for the use of psychiatric rehabilitation via self-management for patients with PDD and their partner/caregiver. Using psychiatric rehabilitation as the next step in the treatment guideline for PDD without remission could replace the non-protocolized usual care and optimize the quality and efficiency of depression care. PPEP4All is expected to reduce costs for the mental health organizations and empower patients to achieve a higher quality of life.

Trial status

This trial is currently recruiting patients, which started on 1 April 2017 and is anticipated to end on 1 April 2021. The trial is expected to end on 31 December 2022. The protocol is Version 7 dated 1 February 2021.

Availability of data and materials

Datasets generated and/or analyzed during the current study will be pseudonymized and stored on an online Dutch meta-data catalogue called the Data Archiving and networked Services (DANS, www.dans.knaw.nl), according to the funding sponsor policy, with access limited to a designated team within the Department of Psychiatry of the Leiden University Medical Center. External researchers may get access to the final trial dataset from the designated team on reasonable request. The (intellectual) property rights with regard to the generated data will reside at the Leiden University Medical Center. Anonymized results will be published in peer-reviewed journals and presented in international conferences.

Abbreviations

B4CZ:

Questionnaire on Burden of Chronic Disease, version for Patients or Partners

BIA:

Budget impact analysis

BRSnl:

Brief Resilience Scale-Dutch translation, 6 items

CAU:

Care as usual

CEA:

Cost-effectiveness analysis

CEAC:

Cost-effectiveness acceptability curve

DANS:

Data Archiving and Networked Services

DSM:

Diagnostic and Statistical Manual of Mental Disorders

EQ-5D-5L:

EuroQoL-5, including the visual analogue scale, 5 items

GEE:

Generalized estimating equation

GRP:

Good Research Practice research certification

IDS-SR:

Inventory of Depressive Symptomatology, Self-Report, 30-items

IMR:

Illness Management and Recovery

ISPOR:

International Society for Pharmacoeconomics and Outcomes Research-Task Force principles

LUMC:

Leiden University Medical Center

MDD:

Major depressive disorder

MINI:

Mini-International Neuropsychiatric Interview-Dutch translation

NZA:

Dutch Healthcare Authority (Nederlandse Zorgautoriteit)

PDD:

Persistent depressive disorder

PPEP:

Patient and Partner Education Program

PPEP4All:

Patient and Partner Education Program for All Chronic Illnesses

QALYs:

Quality-adjusted life years

ROM:

Routine Outcome Monitoring

SemCAD:

Self-Management For Chronic Anxiety and Depression

SPIRIT:

Standard Protocol Items: Recommendations for Interventional Trials

SQ-48:

Symptom Questionnaire-48, 48 items

SRH:

Self-Rated Happiness survey, 1 item

TiC-P:

Trimbos Medical Technology Assessment questionnaire for psychiatric illness-associated costs, 30 items

WRAP:

Wellness Recovery Action Plan

References

  1. Steinert C, Hofmann M, Kruse J, Leichsenring F. The prospective long-term course of adult depression in general practice and the community. A systematic literature review. J Affect Disord. 2014;152-154(1):65–75.

    Article  PubMed  Google Scholar 

  2. American Psychiatric A. Diagnostic and statistical manual of mental disorders, fourth edition (DSM-IV). Washington, DC: American Psychiatric Publishing; 2001.

    Google Scholar 

  3. American Psychiatric A. Diagnostic and statistical manual of mental disorders, fifth edition (DSM-5). Washington, DC: American Psychiatric Publishing; 2013.

    Book  Google Scholar 

  4. Kriston L, Wolff A, Westphal A, Hölzel LP, Härter M. Efficacy and acceptability of acute treatments for persistent depressive disorder: a network meta-analysis. Depress Anxiety. 2014;31(8):621–30.

    Article  CAS  PubMed  Google Scholar 

  5. Kessler RC, Angermeyer M, Anthony J, de Graaf R, Demyttenaere K, Gasquet I, et al. Lifetime prevalence and age-of-onset distributions of mental disorders in the World Health Organization’s World Mental Health Survey Initiative. World Psychiatry. 2007;6(3):168–76.

    PubMed  PubMed Central  Google Scholar 

  6. Murphy JA, Byrne GJ. Prevalence and correlates of the proposed DSM-5 diagnosis of Chronic Depressive Disorder. J Affect Disord. 2012;139(2):172–80.

    Article  PubMed  Google Scholar 

  7. Klein DN. Chronic depression: diagnosis and classification. Curr Dir Psychol Sci. 2010;19(2):96–100.

    Article  Google Scholar 

  8. Gilmer WS, Trivedi MH, Rush AJ, Wisniewski SR, Luther J, Howland RH, et al. Factors associated with chronic depressive episodes: a preliminary report from the STAR-D project. Acta Psychiatr Scand. 2005;112(6):425–33.

    Article  CAS  PubMed  Google Scholar 

  9. Ten Have M, de Graaf R, van Dorsselaer S, Tuithof M, Kleinjan M, Penninx BWJH. Recurrence and chronicity of major depressive disorder in the general population: results from the Netherlands Mental Health Survey and Incidence Study-2. Tijdschr Psychiatr. 2019;61(1):22-31.

  10. Spijker J, Bockting CLH, Meeuwissen JAC, van Vliet IM, Emmelkamp PMG, Hermens MLM, et al. Multidisciplinaire richtlijn Depressie (Derde revisie): Richtlijn voor de diagnostiek, behandeling en begeleiding van volwassen patiënten met een depressieve stoornis [Multidisciplinary guidelines depression (Third revision): Guideline for the diagnosis, treatment, and guidance of adults with a depressive disorder]. Utrecht: Trimbos-institute; 2013.

  11. Spijker J, van Vliet IM, Meeuwissen JAC, van Balkom AJLM. Update of the multidisciplinary guidelines for anxiety and depression. Tijdschr Psychiatr. 2010;52(10):715-8.

  12. Rush AJ, Laux G, Giles DE, Jarrett RB, Weissenburger J, Feldman-Koffler F, et al. Clinical characteristics of outpatients with chronic major depression. J Affect Disord. 1995;34(1):25–32.

    Article  CAS  PubMed  Google Scholar 

  13. Koekkoek B, Van Meijel B, Schene A, Hutschemaekers G. Clinical problems in the long-term care of patients with chronic depression. J Adv Nurs. 2008;62(6):689–97.

    Article  PubMed  Google Scholar 

  14. Katon W, Unützer J, Russo J. Major depression: the importance of clinical characteristics and treatment response to prognosis. Depress Anxiety. 2010;27(1):19–26.

    Article  PubMed  Google Scholar 

  15. Boer S, Dekkers OM, Cessie SL, Carlier IVE, van Hemert AM. Prediction of prolonged treatment course for depressive and anxiety disorders in an outpatient setting: the Leiden routine outcome monitoring study. J Affect Disord. 2019;247:81–7.

  16. Richards D. Prevalence and clinical course of depression: a review. Clin Psychol Rev. 2011;31(7):1117–25.

    Article  PubMed  Google Scholar 

  17. A'Campo LEI, Wekking EM, Spliethoff-Kamminga NGA, Le Cessie S, Roos RAC. The benefits of a standardized patient education program for patients with Parkinson’s disease and their caregivers. Parkinsonism Relat Disord. 2010;16(2):89–95.

    Article  CAS  PubMed  Google Scholar 

  18. A'Campo LEI, Spliethoff-Kamminga NGA, Macht M, Roos RAC. Caregiver education in Parkinson’s disease: formative evaluation of a standardized program in seven European countries. Qual Life Res. 2010;19(1):55–64.

    Article  CAS  PubMed  Google Scholar 

  19. A'Campo LEI, Spliethoff-Kamminga NGA, Roos RAC. An evaluation of the patient education programme for Parkinson’s disease in clinical practice. Int J Clin Pract. 2011;65(11):1173–9.

    Article  CAS  PubMed  Google Scholar 

  20. Henken T, Huibers MJH, Churchill R, Restifo KK, Roelofs JJ. Family therapy for depression. Cochrane Depression, Anxiety and Neurosis. Group. 2007;3.

  21. Shimodera S, Furukawa T, Mino Y, Shimazu K, Nishida A, Inoue S. Cost-effectiveness of family psychoeducation to prevent relapse in major depression: results from a randomized controlled trial. BMC Psychiatry. 2012;12(1):40.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Cuijpers P, Berking M, Andersson G, Quigley L, Kleiboer A, Dobson KS. A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments. Can J Psychiat-Rev Can Psychiat. 2013;58(7):376–85.

    Article  Google Scholar 

  23. Cuijpers P, Huibers M, Daniel Ebert D, Koole SL, Andersson G. How much psychotherapy is needed to treat depression? A metaregression analysis. J Affect Disord. 2013;149(1-3):1–13.

    Article  PubMed  Google Scholar 

  24. Cuijpers P, Andersson G, Donker T, van Straten A. Psychological treatment of depression: results of a series of meta-analyses. Nord J Psychiatry. 2011;65(6):354–64.

    Article  PubMed  Google Scholar 

  25. Cuijpers P. Four decades of outcome research on psychotherapies for adult depression: an overview of a series of meta-analyses. Can Psychol. 2017;58(1):7–19.

    Article  Google Scholar 

  26. Jobst A, Brakemeier EL, Buchheim A, Caspar F, Cuijpers P, Ebmeier KP, et al. European Psychiatric Association Guidance on psychotherapy in chronic depression across Europe. Eur Psychiatry. 2016;33:18–36.

    Article  CAS  PubMed  Google Scholar 

  27. Karyotaki E, Smit Y, Beurs DP, Henningsen KH, Robays J, Huibers MJH, et al. The long-term efficacy of acute-phase psychotherapy for depression: a meta-analysis of randomized trials. Depress Anxiety. 2016;33(5):370–83.

    Article  PubMed  Google Scholar 

  28. Macht M, Gerlich C, Ellgring H, Schradi M, Rusiñol ÀB, Crespo M, et al. Patient education in Parkinson’s disease: formative evaluation of a standardized programme in seven European countries. Patient Educ Couns. 2007;65(2):245–52.

    Article  PubMed  Google Scholar 

  29. Simons G, Thompson SBN, Smith Pasqualini MC. An innovative education programme for people with Parkinson’s disease and their carers. Parkinsonism Relat Disord. 2006;12(8):478–85.

    Article  PubMed  Google Scholar 

  30. Tiihonen S, Lankinen A, Viemerö V. An evaluation of a cognitive-behavioral patient education program for persons with Parkinson’s disease in Finland. Nordic Psychology. 2008;60(4):316–31.

    Article  Google Scholar 

  31. Kamminga NGA. PPEP4All: Zelfmanagementprogramma voor chronisch zieken en hun partner- Patientenwerkboek [Self-management program for persons with chronic illness and their partner- Patient workbook]. Amsterdam: Boom; 2013.

    Google Scholar 

  32. Kamminga NGA. PPEP4All: Zelfmanagementprogramma voor chronisch zieken en hun partner- Partnerwerkboek [Self-management program for persons with chronic illness and their partner- Partner workbook]. Amsterdam: Boom; 2013.

  33. Spliethoff-Kamminga NGA, Bustraan J. PPEP4All: Zelfmanagementprogramma voor chronisch zieken en hun partner- Hulpverlenershandleiding [Self-management program for persons with chronic illness and their partners- Clinician Handbook]. Amsterdam: Boom; 2014.

    Google Scholar 

  34. A'Campo LEI, Spliethoff-Kamminga NGA, Roos RAC. The Patient Education Program for Huntington’s Disease (PEP-HD). J Huntingtons Dis. 2012;1(1):47–56.

    Article  PubMed  Google Scholar 

  35. Andela CD, Repping-Wuts H, Stikkelbroeck N, Pronk MC, Tiemensma J, Hermus A, et al. Enhanced self-efficacy after a selfmanagement programme in pituitary disease: a randomized controlled trial. Eur J Endocrinol. 2017;177(1):59–72.

    Article  CAS  PubMed  Google Scholar 

  36. Nelson G, Macnaughton E, Goering P. What qualitative research can contribute to a randomized controlled trial of a complex community intervention. Contemp Clin Trials. 2015;45(Pt B):377–84.

    Article  PubMed  Google Scholar 

  37. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. Int J Nurs Stud. 2013;50(5):587–92.

    Article  PubMed  Google Scholar 

  38. Kluger BM, Katz M, Galifianakis N, Pantilat SZ, Kutner JS, Sillau S, et al. Does outpatient palliative care improve patient-centered outcomes in Parkinson’s disease: rationale, design, and implementation of a pragmatic comparative effectiveness trial. Contemp Clin Trials. 2019;79:28–36.

    Article  PubMed  Google Scholar 

  39. Griffin C, Phoenix A. The relationship between qualitative and quantitative research: lessons from feminist psychology. J Community Appl Soc Psychol. 1994;4(4):287–98.

    Article  Google Scholar 

  40. Michell J. The place of qualitative research in psychology. Qual Res Psychol. 2004;1(4):307–19.

    Article  Google Scholar 

  41. Powell H, Mihalas S, Onwuegbuzie AJ, Suldo S, Daley CE. Mixed methods research in school psychology: a mixed methods investigation of trends in the literature. Psychol Schools. 2008;45(4):291–309.

    Article  Google Scholar 

  42. Phillips ML, Kupfer DJ. Bipolar disorder diagnosis: challenges and future directions. Lancet (London, England). 2013;381(9878):1663-71.

  43. Lecrubier Y, Sheehan DV, Weiller E, Amorim P, Bonora I, Harnett Sheehan K, et al. The Mini International Neuropsychiatric Interview (MINI). A short diagnostic structured interview: reliability and validity according to the CIDI. Eur Psychiatry. 1997;12(5):224–31.

    Article  Google Scholar 

  44. Sheehan DV, Lecrubier Y, Sheehan K, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (MINI): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59:22–33.

    PubMed  Google Scholar 

  45. Meuldijk D, Carlier IVE, van Vliet IM, van Den Akker-Marle ME, Zitman FG. A randomized controlled trial of the efficacy and cost-effectiveness of a brief intensified cognitive behavioral therapy and/or pharmacotherapy for mood and anxiety disorders: Design and methods. Contemp Clin Trials. 2012;33(5):983–92.

    Article  PubMed  Google Scholar 

  46. de Beurs E, Den Hollander-Gijsman M, van Rood Y, Giltay E, van Noorden M, van Der Lem R, et al. Routine outcome monitoring in the Netherlands: practical experiences with a web-based strategy for the assessment of treatment outcome in clinical practice. Clin Psychol Psychother. 2011;18(1):1–12.

    Article  PubMed  Google Scholar 

  47. Carlier IVE, Andree Wiltens DH, Van Rood YR, Van Veen T, Dekker J, Van Hemert AM. Treatment course and its predictors in patients with somatoform disorders: a routine outcome monitoring study in secondary psychiatric care. Clin Psychol Psychother. 2018; 25(4):550-64.

  48. Herdman M, Gudex C, Lloyd A, Janssen MF, Kind P, Parkin D, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res. 2011;20(10):1727–36.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. van Hout B, Janssen MF, Feng Y-S, Kohlmann T, Busschbach J, Golicki D, et al. Interim Scoring for the EQ-5D-5L: mapping the EQ-5D-5L to EQ-5D-3L Value Sets. Value Health. 2012;15(5):708–15.

    Article  PubMed  Google Scholar 

  50. Janssen M, Pickard A, Golicki D, Gudex C, Niewada M, Scalone L, et al. Measurement properties of the EQ-5D-5L compared to the EQ-5D-3L across eight patient groups: a multi-country study. Qual Life Res. 2013;22(7):1717–27.

    Article  CAS  PubMed  Google Scholar 

  51. Mulhern B, Feng Y, Shah K, Janssen MF, Herdman M, Van Hout B, et al. Comparing the UK EQ-5D-3L and English EQ-5D-5L value sets. PharmacoEconomics. 2018;36(6):699–713.

  52. Stolk E, Ludwig K, Rand K, van Hout B, Ramos-Goñi JM. Overview, update, and lessons learned from the International EQ-5D-5L Valuation Work: Version 2 of the EQ-5D-5L Valuation Protocol. Value Health. 2019;22(1):23–30.

    Article  PubMed  Google Scholar 

  53. Bouwmans C, De Jong K, Timman R, Zijlstra-Vlasveld M, der Feltz-Cornelis C, Tan SS, et al. Feasibility, reliability and validity of a questionnaire on healthcare consumption and productivity loss in patients with a psychiatric disorder (TiC-P).(Report). BMC Health Serv Res. 2013;13(1):217.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Rush AJ, Gullion CM, Basco MR, Jarrett RB, Trivedi MH. The Inventory of Depressive Symptomatology (IDS): psychometric properties. Psychol Med. 1996;26(3):477–86.

    Article  CAS  PubMed  Google Scholar 

  55. Rush AJ, Carmody T, Reimitz PE. The Inventory of Depressive Symptomatology (IDS): Clinician (IDS-C) and Self-Report (IDS-SR) ratings of depressive symptoms. Int J Methods Psychiatr Res. 2000;9(2):45–59.

    Article  Google Scholar 

  56. Trivedi MH, Rush AJ, Ibrahim HM, Carmody TJ, Biggs MM, Suppes T, et al. The Inventory of Depressive Symptomatology, Clinician Rating (IDS-C) and Self-Report (IDS-SR), and the Quick Inventory of Depressive Symptomatology, Clinician Rating (QIDS-C) and Self-Report (QIDS-SR) in public sector patients with mood disorders: a psychometric evaluation. Psychol Med. 2004;34(1):73–82.

    Article  CAS  PubMed  Google Scholar 

  57. Carlier IVE, Schulte-van Maaren Y, Wardenaar K, Giltay E, Van Noorden M, Vergeer P, et al. Corrigendum to “Development and validation of the 48-item symptom questionnaire (SQ-48) in patients with depressive, anxiety and somatoform disorders”. Psychiatry Res. 2012;200:904–10.

    Article  PubMed  Google Scholar 

  58. Carlier IVE, Schulte-Van Maaren Y, Wardenaar K, Giltay E, Van Noorden M, Vergeer P, et al. Development and validation of the 48-item Symptom Questionnaire (SQ-48) in patients with depressive, anxiety and somatoform disorders. Psychiatry Res. 2012;200(2-3):904–10.

    Article  PubMed  Google Scholar 

  59. Carlier IVE, Kovács V, Noorden MS, Feltz-Cornelis C, Mooij N, Schulte-Van Maaren YWM, et al. Evaluating the responsiveness to therapeutic change with routine outcome monitoring: a comparison of the Symptom Questionnaire-48 (SQ-48) with the Brief Symptom Inventory (BSI) and the Outcome Questionnaire-45 (OQ-45). Clin Psychol Psychother. 2017;24(1):61–71.

    Article  PubMed  Google Scholar 

  60. Carlier IVE, van Eeden WA, de Jong K, Giltay EJ, van Noorden MS, van Der Feltz-Cornelis C, et al. Testing for response shift in treatment evaluation of change in self-reported psychopathology amongst secondary psychiatric care outpatients. Int J Methods Psychiatr Res. 2019;28:e1785.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Spinhoven P, Elzinga BM, Giltay E, Penninx BWJH. Anxious or depressed and still happy? PLoS One. 2015;10(10):e0139912.

  62. Abdel-Khalek A. Measuring happiness with a single-item scale. Soc Behav Pers. 2006;34(2):139–49.

    Article  Google Scholar 

  63. Veenhoven R, Ehrhardt J. The cross-national pattern of happiness: test of predictions implied in three theories of happiness. Inter Interdisc J Qual Life Measur. 1995;34(1):33–68.

    Google Scholar 

  64. Leontjevas R, de Beek WO, Lataster J, Jacobs N. Resilience to affective disorders: a comparative validation of two resilience scales. J Affect Disord. 2014;168:262–8.

    Article  PubMed  Google Scholar 

  65. Smith B, Dalen J, Wiggins K, Tooley E, Christopher P, Bernard J. The brief resilience scale: assessing the ability to bounce back. Int J Behav Med. 2008;15(3):194–200.

    Article  PubMed  Google Scholar 

  66. Spliethoff-Kamminga NGA, Zwinderman AH, Springer MP, Roos RAC. Psychosocial problems in Parkinson’s disease: evaluation of a disease-specific questionnaire. Mov Disord. 2003;18(5):503–9.

    Article  PubMed  Google Scholar 

  67. Spliethoff-Kamminga NGA, Zwinderman A, Springer M, Roos R. A disease-specific psychosocial questionnaire for Parkinson’s disease caregivers. J Neurol. 2003;250(10):1162–8.

    Article  PubMed  Google Scholar 

  68. Satir V. Conjoint family therapy. 3rd ed. Palo Alto: Science and Behavior Books; 1983.

  69. Bamm EL, Rosenbaum P. Family-centered theory: origins, development, barriers, and supports to implementation in rehabilitation medicine. Arch Phys Med Rehabil. 2008;89(8):1618–24.

    Article  PubMed  Google Scholar 

  70. Beck AT. Cognitive therapy and the emotional disorders. New York: International Universities Press; 1976.

    Google Scholar 

  71. Bandura A, Cervone D. Self-evaluative and self-efficacy mechanisms governing the motivational effects of goal systems. J Pers Soc Psychol. 1983;45(5):1017–28.

    Article  Google Scholar 

  72. Lazarus RS. Psychological stress and the coping process. New York: McGraw-Hill; 1966.

  73. Lazarus RS. Stress, appraisal, and coping. New York: Springer Pub. Co.; 1984.

    Google Scholar 

  74. Prochaska JO, Velicer WF. The transtheoretical model of health behavior change. Am J Health Promot. 1997;12(1):38–48.

    Article  CAS  PubMed  Google Scholar 

  75. Cohen SI. Updating the model for psychotherapy of psychosomatic problems. Psychother Psychosom. 1979;32(1-4):72–90.

    Article  CAS  PubMed  Google Scholar 

  76. Zimmermann C, Tansella M. Psychosocial factors and physical illness in primary care: promoting the biopsychosocial model in medical practice. J Psychosom Res. 1996;40(4):351–8.

    Article  CAS  PubMed  Google Scholar 

  77. Engel G. The need for a new medical model: a challenge for biomedicine. Psychodyn Psychiatry. 2012;40(3):377–96.

  78. Liberman RFD. Psychiatric rehabilitation for chronic mental patients. Psychiatr Ann. 1983;13:539–45.

    Article  Google Scholar 

  79. Pratt R, Macgregor A, Reid S, Given L. Wellness Recovery Action Planning (WRAP) in self-help and mutual support groups. Psychiatr Rehabil J. 2012;35(5):403–5.

    Article  PubMed  Google Scholar 

  80. Pratt R, MacGregor A, Reid S, Given L. Experience of Wellness Recovery Action Planning in self-help and mutual support groups for people with lived experience of mental health difficulties. Scientific World Journal. 2013;2013:180587.

    Article  PubMed  PubMed Central  Google Scholar 

  81. McGuire AB, Kukla M, Green A, Gilbride D, Mueser KT, Salyers MP. Illness management and recovery: a review of the literature. Psychiatr Serv. 2014;65(2):171–9.

    Article  PubMed  PubMed Central  Google Scholar 

  82. Sinnema H, Schene A, Smit F, Spijker J. Effectiveness of a self-management training for patients with chronic and treatment resistant anxiety or depressive disorders on quality of life, symptoms, and empowerment: results of a randomized controlled trial. BMC Psychiatry. 2019;19(1):46.

    Article  PubMed  PubMed Central  Google Scholar 

  83. Muntingh A, Laheij M, Sinnema H, Zoun M, van Balkom A, Schene A, et al. Quality of life and symptoms in patients with chronic depression and anxiety after a self-management training: a randomised controlled trial. Tijdschr Psychiatr. 2016;58(7):504–12.

    CAS  PubMed  Google Scholar 

  84. Zoun MHH, Koekkoek B, Sinnema H, Muntingh ADT, van Balkom AJLM, Schene AH, et al. Effectiveness and cost-effectiveness of a self-management training for patients with chronic and treatment resistant anxiety or depressive disorders: design of a multicenter randomized controlled trial. BMC Psychiatry. 2016;16(1):216.

  85. Gaspersz R, Lamers F, Beekman ATF, van Hemert AM, Schoevers RA, Penninx BWJH. The impact of depressive disorder symptoms and subtypes on 6-year incidence of somatic diseases. Psychother Psychosom. 2018;87(5):308–10.

  86. Spliethoff-Kamminga NGA, de Vreugd J. Patiënt Educatie Programma Parkinson (PEPP). Amsterdam: Harcourt; 2006.

  87. Spliethoff-Kamminga NGA, de Vreugd J. Session 6: Social competence for people with PD and for Carers. In: Smith Pasqualini M, Simons, G, editors. Patient education for people with Parkinson's disease and their carers. Chichester: John Wiley & Sons, Ltd; 2006. p. 191-233.

  88. McCrone P, Rost F, Koeser L, Koutoufa I, Stephanou S, Knapp M, et al. The economic cost of treatment-resistant depression in patients referred to a specialist service. J Ment Health. 2018;27(6):567–73.

    Article  PubMed  Google Scholar 

  89. Kuyken W, Byford S, Taylor RS, Watkins E, Holden E, White K, et al. Mindfulness-based cognitive therapy to prevent relapse in recurrent depression. J Consult Clin Psychol. 2008;76(6):966–78.

    Article  PubMed  Google Scholar 

  90. Fonagy P, Rost F, Carlyle JA, McPherson S, Thomas R, Pasco Fearon RM, et al. Pragmatic randomized controlled trial of long-term psychoanalytic psychotherapy for treatment-resistant depression: the Tavistock Adult Depression Study (TADS). World Psychiatry. 2015;14(3):312–21.

    Article  PubMed  PubMed Central  Google Scholar 

  91. Altman DG. Practical statistics for medical research. London: Chapman and Hall; 1991.

  92. Maxwell SE, Delaney, HD. Designing experiments and analyzing data: a model comparison perspective. 2nd ed. Mahwah: Lawrence Erlbaum Associates; 2004.

  93. Schafer WD. Analysis of pretest-posttest designs. Measure Ev Counsel Dev. 1992;25(1):2.

    Google Scholar 

  94. Morgan GA, Gliner JA, Harmon RJ. Understanding and evaluating research in applied and clinical settings. Mahwah, NJ [etc.]: Lawrence Erlbaum Associates; 2006.

  95. Bausell RB, Li, Y-F. Power analysis for experimental research: a practical guide for the biological, medical and social sciences. Cambridge: Cambridge University Press; 2002.

  96. Zeeck A, Wietersheim JV, Weiss H, Beutel M, Hartmann A. The INDDEP study: inpatient and day hospital treatment for depression - symptom course and predictors of change. BMC Psychiatry. 2013;13(1):100.

    Article  PubMed  PubMed Central  Google Scholar 

  97. Twisk J. Longitudinal Data Analysis. A comparison between generalized estimating equations and random coefficient analysis. Eur J Epidemiol. 2004;19(8):769–76.

    Article  PubMed  Google Scholar 

  98. Zeger SL, Liang K-Y. Longitudinal data analysis for discrete and continuous outcomes. Biometrics. 1986;42(1):121–30.

    Article  CAS  PubMed  Google Scholar 

  99. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials. Ann Intern Med. 2010;152(11):726–32.

    Article  PubMed  Google Scholar 

  100. Meuldijk D, Carlier IVE, van Vliet IM, van Hemert AM, Zitman FG, van den Aker-van Marle ME. Economic evaluation of concise cognitive behavioural therapy and/or pharmacotherapy for depressive and anxiety disorders. J Ment Health Policy Econ. 2015;18:175–83.

    PubMed  Google Scholar 

  101. Hakkart-van Rooijen L, van der Linden N, Bouwmans C, Kanters TA, Tan SS. Kostenhandleiding: Methodologie van kostenonderzoek en referentieprijzen voor economische evaluaties in de gezondheidszorg [Costing manual: methods of cost-related studies and reference prices for economic evaluations of mental healthcare]. Rotterdam: Institute for Medical Technology Assessment, Erasmus University; 2015.

  102. Kanters TA, Bouwmans CAM, van der Linden N, Tan SS, Hakkaart-van Roijen L, Prinja S. Update of the Dutch manual for costing studies in health care. PLoS One. 2017;12(11):e0187477.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  103. Koopmanschap MA, Rutten FFH, van Ineveld BM, van Roijen L. The friction cost for measuring indirect costs of disease. J Health Econ. 1995;14(2):171.

    Article  CAS  PubMed  Google Scholar 

  104. Johannesson M, Karlsson G. The friction cost method: a comment. J Health Econ. 1997;16(2):249–55.

    Article  CAS  PubMed  Google Scholar 

  105. Brouwer WBF, Meerding W-J, Lamers LM, Severens JL. The relationship between productivity and health-related QOL: an exploration. (Current Opinion). PharmacoEconomics. 2005;23(3):209.

    Article  PubMed  Google Scholar 

  106. Rappange D, van Baal P, van Exel N, Feenstra T, Rutten F, Brouwer W. Unrelated medical costs in life-years gained: should they be included in economic evaluations of healthcare interventions? PharmacoEconomics. 2008;26(10):815–30.

    Article  PubMed  Google Scholar 

  107. Sullivan SD, Mauskopf JA, Augustovski F, Jaime Caro J, Lee KM, Minchin M, et al. Budget Impact Analysis—Principles of Good Practice: Report of the ISPOR 2012 Budget Impact Analysis Good Practice II Task Force. Value Health. 2014;17(1):5–14.

    Article  PubMed  Google Scholar 

  108. Goettsch WG, Enzing J. Review: Report of the ISPOR 2012 Budget Impact Analysis Good Practice II Task Force. Value Health. 2014;17(1):1–2.

    Article  PubMed  Google Scholar 

  109. Kok I, van Wijngaarden B. Client appreciation in mental health care: manual of the Dutch mental healthcare thermometer of appreciation by clients. Utrecht: Trimbos-institute/GGZ Nederland; 2003.

  110. Chambers E, Cook S, Thake A, Foster A, Shaw S, Hutten R, et al. The self-management of longer-term depression: learning from the patient, a qualitative study. BMC Psychiatry. 2015;15(1):172.

    Article  PubMed  PubMed Central  Google Scholar 

  111. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.

    Article  Google Scholar 

  112. Braun V, Clarke V. Successful qualitative research: a practical guide for beginners. London: SAGE; 2013.

  113. Servant-Miklos VFC, Noordzij G. Investigating the impact of problem-oriented sustainability education on students’ identity: a comparative study of planning and liberal arts students. J Clean Prod. 2021;280.

  114. Mc Sharry J, Bishop FL, Moss-Morris R, Kendrick T. “The chicken and egg thing”: Cognitive representations and self-management of multimorbidity in people with diabetes and depression. Psychol Health. 2013;28(1):103–19.

    Article  PubMed  Google Scholar 

  115. Glaser BG, Strauss AL. The discovery of grounded theory: strategies for qualitative research. Chicago: Aldine Pub. Co; 1967.

  116. Strauss AL, Corbin JM. Basics of qualitative research: techniques and procedures for developing grounded theory. 2nd ed. Thousand Oaks: Sage Publications; 1998.

  117. Kolb SM. Grounded theory and the constant comparative method: valid research strategies for educators. J Emerg Trends Educ Res Policy Studies. 2012;3(1):83–6.

    Google Scholar 

  118. Sullivan C, Forrester MA. Doing qualitative research in psychology: a practical guide. 2nd ed. Los Angeles: SAGE; 2019.

    Google Scholar 

  119. Timonen V, Foley G, Conlon C. Challenges when using grounded theory: a pragmatic introduction to doing GT research. Int J Qual Methods. 2018;17(1):160940691875808.

    Article  Google Scholar 

  120. Dworkin SL. Sample size policy for qualitative studies using in-depth interviews. Arch Sex Behav. 2012;41(6):1319.

    Article  PubMed  Google Scholar 

  121. Hennink MM, Kaiser BN, Marconi VC. Code saturation versus meaning saturation: how many interviews are enough? Qual Health Res. 2017;27(4):591–608.

    Article  PubMed  Google Scholar 

  122. Evers J, de Boer, F. The qualitative interview: features, types, and preparation. In: Evers J, editor. The qualitative interview: Art and skill. Den Haag: LEMMA; 2007. p. 23–47.

  123. KNAW; NFU; NWO; TO2-federatie; Vereniging Hogescholen; VSNU. Netherlands Code of Conduct for Research Integrity [Nederlandse Gedragscode Wetenschappelijke Integriteit]. DANS database. 2018. https://doi.org/10.17026/dans-2cj-nvwu.

  124. Glick JL, Shamoo AE. A call for the development of “Good Research Practices” (GRP) guidelines. Account Res. 1993;2(4):231–5.

    Article  CAS  PubMed  Google Scholar 

  125. Poolman RW, Hanson B, Marti RK, Bhandari M. Conducting a clinical study: a guide for good research practice. Indian Jf Orthopaedics. 2007;41(1):27–31.

    Article  Google Scholar 

  126. Vijayananthan A, Nawawi O. The importance of Good Clinical Practice guidelines and its role in clinical trials. Biomed Imaging Inter J. 2008;4(1):e5.

  127. Huang GD, Bull J, Johnston Mckee K, Mahon E, Harper B, Roberts JN. Clinical trials recruitment planning: a proposed framework from the Clinical Trials Transformation Initiative. Contemp Clin Trials. 2018;66:74–9.

    Article  PubMed  Google Scholar 

  128. Treweek S, Lockhart P, Pitkethly M, Cook JA, Kjeldstrøm M, Johansen M, et al. Methods to improve recruitment to randomised controlled trials: Cochrane systematic review and meta-analysis. BMJ Open. 2013;3(2):e002360.

  129. Ross S, Grant A, Counsell C, Gillespie W, Russell I, Prescott R. Barriers to participation in randomised controlled trials: a systematic review. J Clin Epidemiol. 1999;52(12):1143–56.

    Article  CAS  PubMed  Google Scholar 

  130. Rössler W. Psychiatric rehabilitation today: an overview. World Psychiatry. 2006;5(3):151.

    PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge the contributions made by the participants of this study and the participating Dutch mental healthcare providers (GGZ Rivierduinen, Parnassia Group/PsyQ, GGZ Transparant, the Psychiatry Department of Maastricht University Medical Center, and the Praktijk voor de Psychotherapie/Psychotherapy Practice “De Lindehoeve”). We also thank the local coordinators, advisors, and managers of the participating centers for their recruiting efforts. In addition, special thanks are offered to the following persons within the LUMC who supported this project: Nic van der Wee, Max de Leeuw, Irene van Vliet, Gea Corton, Mirjam van der Hoorn, Maximilian Schenk, Marieke de Wit, Marian Lucas, Saskia Engelander, Wessel van Eeden, Sander Muntz, Marjolein Schenk, Karin de Winter, and Hanneke Blom. Finally, we thank all research assistants of this project for their contribution to the quantitative and qualitative data collection.

Funding

This work was supported by the Dutch ZonMw Doelmatigheidsonderzoek program [843002709] and GGZ Rivierduinen. The funding agency had no role in the design of this study and will not have any role during its execution or analyses, interpretation or the data, or decision to submit results.

Author information

Authors and Affiliations

  1. Department of Psychiatry, Leiden University Medical Center (LUMC), Albinusdreef 2, 2333 ZA, Leiden, The Netherlands

    Ericka C. Solis, Ingrid V. E. Carlier, Nic J. A. van der Wee & Albert M. van Hemert

Authors
  1. Ericka C. Solis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ingrid V. E. Carlier
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nic J. A. van der Wee
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Albert M. van Hemert
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

IC conceived and developed the study design and coordinates the study. ES is conducting this study in fulfillment of a doctoral degree and was able to contribute to the study design. NvdW is involved in the oversight of the data collection. AvH is the department head LUMC Psychiatry and departmental responsible person. ES will perform the data analysis. ES drafted the manuscript and IC, NvdW, and AvH critically revised the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Ericka C. Solis.

Ethics declarations

Ethics approval and consent to participate

This study protocol has been approved by the Medical Ethics Committee of the Leiden University Medical Center. All participants in the study will provide documented informed consent.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Solis, E.C., Carlier, I.V.E., van der Wee, N.J.A. et al. The clinical and cost-effectiveness of a self-management intervention for patients with persistent depressive disorder and their partners/caregivers: study protocol of a multicenter pragmatic randomized controlled trial. Trials 22, 731 (2021). https://doi.org/10.1186/s13063-021-05666-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13063-021-05666-y

Keywords

Trials

ISSN: 1745-6215