Skip to main content

Advertisement

We're creating a new version of this page. See preview

  • Study protocol
  • Open Access
  • Open Peer Review

Efficacy of a hybrid online training for panic symptoms and agoraphobia: study protocol for a randomized controlled trial

  • 1Email author,
  • 1,
  • 1,
  • 1, 2,
  • 3,
  • 4,
  • 1,
  • 1, 4, 5 and
  • 1, 2
Trials201415:427

https://doi.org/10.1186/1745-6215-15-427

©  Ebenfeld et al.; licensee BioMed Central Ltd. 2014

  • Received: 6 August 2014
  • Accepted: 14 October 2014
  • Published:
Open Peer Review reports

Abstract

Background

Recently, internet-based interventions have been proposed as effective treatments for people with panic disorder (PD). However, little is known about the clinical effects of integrating mobile technology into these interventions. Because users carry their smartphones with them throughout the day, we hypothesize that this technology can be used to significantly support individuals with monitoring and overcoming their PD symptoms. The aim of the present study is to evaluate the efficacy and cost-effectiveness of a newly developed hybrid intervention that combines internet/PC with smartphone delivery to treat the symptoms of PD. The intervention is based on cognitive behavioral therapy and consists of six modules over a total of six weeks.

Methods/Design

A two-arm randomized controlled trial (RCT) will be conducted to evaluate the effects of a hybrid online training module for PD. Based on a power calculation (d =0.60; 1-β of 80%; α =0.05), 90 participants with mild to moderate panic symptoms with or without agoraphobia (as assessed by the Panic and Agoraphobia Scale) will be recruited from the general population and randomly assigned to either the intervention group or a six-month waitlist control group. The primary outcome measure will be the severity of panic symptoms. Secondary outcomes will include depression, quality of life, and an observer-based rating of panic severity. Furthermore, data regarding acceptance and the usability of the smartphone app will be assessed. Assessments will take place at baseline as well as eight weeks, three months, and six months after randomization. Moreover, a cost-effectiveness analysis will be performed from a societal perspective. Data will be analyzed on an intention-to-treat basis and per protocol.

Discussion

To our knowledge, this RCT is one of the first to examine the efficacy of a hybrid online training for adult PD. This study seeks to contribute to the emerging field of hybrid online training. If the intervention is efficacious, then research on this hybrid online training should be extended. The cost-effectiveness analysis will also indicate whether online training is an economical tool for treating PD among adults.

Trial registration

German Clinical Trial Register:DRKS00005223 (registered on 15 August 2013).

Keywords

  • Panic disorder
  • Agoraphobia
  • Sub-clinical
  • Internet
  • Mobile
  • Smartphone
  • Hybrid

Background

Panic disorder (PD) is characterized by recurrent, unexpected panic attacks and a persistent worry about future panic attacks[1]. With a 12-month prevalence of 1.8% among adults, PD is one of the most common anxiety disorders[2]. In addition, sub-threshold cases have been estimated to be even more prevalent, with between 10 and 16% of the population experiencing a panic attack at some point during their lifetimes[3]. PD (with or without agoraphobia) is associated with a high psychiatric comorbidity, lower quality of life, and severe work impairment[4, 5], and also places a significant burden on healthcare systems[2, 6].

Psychotherapeutic treatments such as cognitive behavioral therapy (CBT) are effective for PD[7]. However, only 16.7% of people who suffer from anxiety disorders seek help from a mental health professional, and of these individuals, only 21.3% receive CBT, which is arguably the most effective treatment[8]. The reasons for this low endorsement of effective PD treatments include a lack of psychoeducation, a fear of stigmatization, and structural barriers such as a lack of access to adequate treatments[9, 10].

Interventions delivered via the internet have been proposed as a helpful method to overcome such barriers and facilitate access to empirically validated treatments[11, 12]. Although internet-based interventions based on CBT are effective at treating adult PD[13–16], one drawback of this method (compared with traditional face-to-face treatments) is the lack of direct support from a therapist throughout the course of treatment, particularly during exposure exercises. This lack of direct support from a therapist might increase the risk of clients not fully complying with the treatment, or dropping out of treatment completely. Studies that evaluate online trainings with self-exposure elements consistently report particularly high dropout rates[17, 18].

One potentially promising method to enhance the adherence to, and efficacy of, internet-based interventions for PD might be to complement interventions with components that are delivered through clients’ smartphones. Mobile components might assist patients in overcoming several limitations of traditional desktop- and laptop-based interventions. For example, these barriers might include situations in which the clients start to engage in exposure exercises in their natural environment (as is typical for in vivo exposure exercises) and are subsequently required to leave their desktop PCs or laptop. This results in a dilemma - either exposure exercises are exclusively conducted at the client’s desk or they are conducted without the device and therefore lack support during the exposure exercises. Consequently, clients are more likely to disengage from the intervention. In contrast, clients often carry their smartphones with them in almost any situation, and they might support clients in successfully completing the intended exposure exercise[19–21]. Moreover, when desktop PCs or laptops are used to monitor symptoms of PD to identify factors that cue panic attacks and avoidance, clients typically use daily or weekly electronic diary entries, which are likely biased by memory effects because clients must complete these diaries retrospectively and not in real-time[22–28]. In contrast, a mobile-based PD intervention tool can be used to assess the symptoms of PD in real-time through ecological momentary assessment (EMA) approaches[29–33]. Finally, a PC-based program can only prompt appropriate coping responses when the PC is on and clients are near it. Therefore, it is not available in other situations of their daily lives in which they inevitably encounter stimuli that trigger the symptoms of PD. In contrast, a mobile device that is nearly always on, or near, the client can be used as an ecological mobile intervention (EMI) device and prompt coping responses, potentially those that have been specifically identified as effective for the individual based on the EMA function of the device. Despite these advantages, no data are currently available regarding the efficacy of online-based interventions for PD that integrate a mobile component.

We developed the GET.ON PANIC intervention for adult PD. This intervention integrates desktop and mobile components into a hybrid online intervention based on CBT for PD. The desktop component is primarily used to provide text- and video-based psychoeducation, as well as exercises that require participants to write extensive texts (for example, in a cognitive restructuring module), which is difficult to do on smartphones. The mobile component is used to guide clients through self-monitoring and self-exposure tasks. To evaluate the efficacy and cost-effectiveness of GET.ON PANIC, we will conduct a randomized controlled trial (RCT).

Methods/Design

Study design

We will conduct an RCT with two arms: an internet-based self-help intervention supported by a mobile application with minimal guidance from a coach (GET.ON PANIC), and a waitlist control group who will receive the intervention after a six-month follow-up assessment. Assessments will be conducted prior to randomization, at post-treatment (eight weeks), as well as at the three- and six-month follow-up assessments (see Figure 1). The Ethical Committee of Marburg approved this study (number: 2013-23 K), and it was registered with the German Clinical Trial Register (registration number: DRKS00005223).
Figure 1
Figure 1

Overview of study procedure. BDI-II, Beck Depression Inventory II; PAS, Panic and agoraphobia scale; SCID-I, Structured clinical interview for DSM-IV Axis I Disorders.

Study population

The study population will consist of a community sample of adults who suffer from mild to moderate panic symptoms. The inclusion criteria are the following: experiencing mild to moderate panic symptoms as assessed by the Panic and Agoraphobia Scale (PAS, score range: 9 to 28)[34, 35], being 18 years or older, having panic as the primary concern for seeking help, and having internet and smartphone access. Although both iOS™ and Android™ devices will be supported, the GET.ON PANIC APP will not run on entry-level smartphones with small screens and low memories. Therefore, the minimum system requirements are an iPhone™ 3GS or a comparable Android device. With respect to the operating system, iOS 6, iOS 7, and Android 2.3 or newer will be supported. Furthermore, because the mobile application periodically uploads data to our servers, we recommend that participants have a data plan to avoid unnecessary costs. The exclusion criteria for this study are the following: experiencing too mild (PAS score 0 to 8) or too severe (PAS score 29 to 52) panic symptoms; receiving current psychological help for anxiety problems or being on a waitlist for psychotherapy; having physical health problems assessed via self-report that prevents participants from engaging in self-exposure, as recommended by an established German guideline for treating people with PD and agoraphobia[36]; currently having posttraumatic stress disorder or psychotic or dissociative disorders assessed via self-report and clinical interview; and having current suicidality, as measured by a score above 1 on item 9 of the Beck Depression Inventory II (BDI-II)[37, 38] and question A9 of the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I)[39]. In the event that potential participants are excluded because of suicidal ideation or intention, the exclusion procedure will be managed as determined by an established suicide protocol. All excluded participants will be contacted via email and provided with information regarding where they can obtain appropriate help.

Sample size

The sample size for this study is based on the meta-analysis of self-help treatments for anxiety disorders by Haug et al.[13]. An effect size of d =0.83 was derived for PD after comparing the self-help group with the waitlist/placebo group. Because hybrid online training was not integrated into the meta-analysis, we chose a conservative estimation of d =0.60. To examine the efficacy of GET.ON PANIC with a two-tailed t-test (α =0.05; 1-β =0.80), a sample of 45 participants will be needed for each group. Thus, the total sample size for this study will be 90 participants.

Randomization

We will use the computer program DatInf RandList Version 1.2 (DatInf GmbH, Tübingen, Germany)[40] to randomize participants into either the intervention group or the waitlist control condition. This random assignment will be stratified for clinical or subclinical symptomatology, as well as the presence or absence of agoraphobia.

Procedure

Participants will be recruited from the general population via an online health center website postings in anxiety- and panic-related online forums, and newspaper articles about the research project. Participants will receive an email with detailed information about the study. Afterwards, participants will be invited to complete a screening questionnaire to evaluate their study eligibility. Participants will then have access to the online training platform via their email address (as their username) and a self-selected password. If participants meet the eligibility criteria, they will receive an ID number. Participants can then opt into study participation by reading, signing, and returning the informed consent document. Participants will then receive a link to complete the baseline questionnaires. After completing the baseline questionnaires (t0), participants will be invited to take part in a telephone interview. This interview has two purposes. The first is for a trained interviewer to conduct a diagnostic interview (SCID-I) to provide a detailed sample description[41, 42]. The second is to conduct an observer rating of anxiety symptoms using the Hamilton Anxiety Scale (HAM-A)[43] to strengthen the robustness of the self-report measures. Assessors blind to the participant treatment condition will conduct all observer-based ratings. The post-treatment measurement (t1) will be assessed eight weeks after randomization. Follow-up measures will be conducted at three months (t2) and six months (t3) after randomization. All questionnaires at the baseline, post- and follow-up assessments are self-reported and conducted via the internet, with the exception of the observer rating HAM-A, which will be performed at t0 and t1 via telephone. The waitlist control group will receive the treatment after t3.

Intervention

GET.ON PANIC is a hybrid internet-based self-help intervention with minimal therapeutic guidance based on CBT principles[18, 44–46]. The hybrid online training consists of two components: a browser-based section (desktop PC or laptop) and a mobile application (smartphone app). The intervention is divided into six modules: psychoeducation, interoceptive exposure, in vivo exposure, two modules of cognitive restructuring, and relapse prevention (see Table 1). Using responsive web design, participants can use the program on a desktop PC, a laptop, a tablet, or a smartphone. An integrated read-aloud function allows participants to follow the lessons via audio narration. The app addresses interoceptive and in vivo exercises as well as diary and relaxation exercises. Detailed information about the development of the app can be found in the paper by Kleine Stegemann et al.[47].In the first module, participants will receive an overview of the different modules and the practical procedure of the online training. Information about PD will be provided, and personal goals will be defined. In addition, a mobile diary will be introduced to the participants. The emphasis of the second module is interoceptive exposure. The theoretical background of the relationship between bodily symptoms and anxiety will be provided in an interactive way with videos and writing exercises. The app contains three interoceptive exposure packages, each consisting of four different tasks. In module three, participants will rank their individual anxiety provoking situations in a hierarchy before beginning the app-assisted in vivo exposures (see Figure 2). In addition, participants will continue with the second of the interceptive exposure exercises, which address dizziness.
Table 1

Overview of sessions

Week

Content and homework

1

Browser:

Psychoeducation:

 

Information about panic

 

Defining goals of training

 

Setting up a reward list

 

Mobile:

Daily diary

 

Registration of current panic event (event-based)

 

Daily summary of panic, avoidance, and mood

2

Browser:

Interoceptive exposure:

 

Bodily symptoms in panic

 

Avoidance

 

Safety behaviors

 

Mobile:

Respiratory interoceptive exposure exercises

 

Daily diary

3

Browser:

In vivo exposure:

 

Defining an anxiety hierarchy

 

Mobile:

In vivo exposures

 

Dizziness interoceptive exposure exercises

 

Daily diary

4

Browser:

Cognitive restructuring I:

 

Negative automatic thoughts

 

Defining anxiety project (training schedule for exposures)

 

Mobile:

In vivo exposures

 

Further interoceptive exposure exercises

 

Daily diary

5

Browser:

Cognitive restructuring II:

 

Reality testing of automatic negative thoughts

 

Mobile:

In vivo exposures

 

Further interoceptive exposure exercises

 

Daily diary

6

Browser:

Relapse prevention:

 

Early warning signs

 

Critical life events

 

Evaluation of training and its aims

 

Mobile:

Breathing and muscle relaxation exercises

Figure 2
Figure 2

App screenshots of GET.ON PANICAPP. The app supports in vivo exposures. Participants start with ranking their anxiety provoking situations in a hierarchical order. Furthermore, they are asked to answer questions about their anxiety before and after performing the exposure exercise (for example the level of anxiety, the degree of avoidance and the severity of body symptoms). In addition, they have the ability to take a photo after an in vivo exposure exercise is completed. The bar chart in the end offers graphical feedback to the participants regarding the exposure performance.

The fourth module concerns cognitive restructuring. Participants will deal with the maladaptive relationships among situations, cognitions, and emotions. In addition, participants will continue with interoceptive and in vivo exposures. In module five, participants will engage in advanced cognitive restructuring, where they analyze their thinking patterns, identify their cognitive distortions and thinking errors, challenge their thoughts associated with panic, and replace their maladaptive thoughts with more constructive cognitions. Participants will continue to practice interoceptive and in vivo exposures for homework. The sixth and final module is related to relapse prevention. Participants will have the opportunity to reflect, summarize, and evaluate the online training as well as their progress toward achieving their predefined goals. Participants will work towards coping prior to experiencing anxiety-provoking events that might occur in the future by making two plans: one to identify early warning signs and one for dealing with difficult life circumstances. Furthermore, participants will be trained in breathing and muscle relaxation exercises to learn adequate coping strategies for daily stressors. Audio-based relaxation exercises will be available to the participants in both short and long formats on the app.

Throughout the study, participants will receive online technical support from an IT specialist to install the app, access to a personal online coach (a trained psychologist) to answer any questions, reminder messages for homework assignments, and brief homework feedback from the online coach based on a manual written by the first author (LE). All online coaches will receive online training with regard to GET.ON PANIC by the first and second authors (LE and SKSt), and will be supervised by licensed and experienced psychotherapists when offering guidance. The total amount of coaching time per participant will be approximately three hours for the duration of the six-week online training.

Instruments

For an overview of the instruments at screening, baseline, after treatment, and at the follow-up assessments, see Table 2.
Table 2

Overview of instruments per time of assessment

Assessments

Screening

T0

T1

T2

T3

Sociodemographic data

x

-

-

-

-

Suicidality (Item 9, BDI-II)

x

-

-

-

-

Diagnosis (SCID-I, sections for anxiety disorders and current depressive episode)

-

x

-

-

 

Panic and agoraphobia severity, self-rating (PAS)

x

x

x

x

x

Panic and agoraphobia severity, observer-rating (HAM-A)

-

x

x

-

-

Agoraphobic cognitions (ACQ)

-

x

x

x

x

Body sensations (BSQ)

-

x

x

x

x

Agoraphobic avoidance (MI)

-

x

x

x

x

Depressive symptoms (CES-D)

-

x

x

x

x

Quality of life (EQ-5D, SF-12)

-

x

x

x

x

Economic evaluations (TiC-P)

-

x

-

-

x

Negative effects of online health trainings

-

-

(x)

(x)

(x)

Attitudes towards seeking psychological help

-

-

(x)

-

-

User satisfactory

-

-

(x)

-

-

Technology acceptance of smartphone app

-

-

(x)

-

-

Usability of smartphone app (SUS)

-

-

(x)

-

-

T0 = Baseline, T1 = 8 weeks, T2 = 3 months, T4 = 6 months.

Assessments: x = intervention and control group, (x) = intervention group only.

ACQ, Agoraphobic cognitions questionnaire; BDI-II, Beck Depression Inventory II; BSQ, Body sensations questionnaire; CES-D, Center for epidemiological studies depression scale; EQ-5D, EuroQol; HAM-A, Hamilton anxiety scale; MI, Mobility inventory; PAS, Panic and agoraphobia scale; SCID-I, Structured clinical interview for DSM-IV Axis I Disorders; SF-12, Short form 12; SUS, System Usability Scale; TiC-P, Trimbos and the Institute of Medical Technology Assessment Cost Questionnaire for Psychiatry.

Screening and diagnostic interview

The preliminary screening will collect demographic data from participants, including their age, gender, education, therapeutic experience, and medication use, and the PAS will be administered (for a detailed description see the section regarding the primary outcome measure)[34, 35]. Item 9 of the BDI-II[37, 38] will be used to screen for suicidality. The BDI-II is a 21-item assessment of depressive symptoms that has demonstrated high internal consistency using outpatient samples[37].

The SCID-I[39] will be used to assess the presence of PD, agoraphobia, other anxiety disorders, and current depressive episodes. A trained interviewer will perform the interview via telephone. Previous studies have demonstrated the validity of telephone-based SCID-I interviews[41, 42].

Primary outcome measure

Panic severity

The primary outcome will be the severity of panic and agoraphobia symptoms as assessed by the total PAS score[34, 35, 48]. This questionnaire was originally developed as a self- and observer-rating scale. In this study, we will use the self-rating questionnaire that was adapted into an online version. The PAS consists of 13 items grouped into five subscales, and an extra item regarding unexpected versus expected panic attacks. The five subscales assess the following areas: panic attacks, agoraphobic avoidance, anticipatory anxiety, daily life limitations, and health concerns (for example, the fear of physical harm or the fear of an organic cause). These subscales can be evaluated separately or as a total score that combines all subscales, ranging from 0 to 52 points. A higher score on the PAS indicates more panic symptoms. The psychometric properties of the scale are satisfactory, with a Cronbach’s alpha of 0.86[49]. A score of between 0 and 8 indicates no clinically relevant symptoms, scores of between 9 and 28 indicate moderate symptoms, and a score of 29 or higher indicates a severe level of symptoms[49].

Secondary outcome measures

Depression

Depressive symptoms will be measured using the Allgemeine Depressions-Skala (ADS)[50], the German adaptation of the Center for Epidemiological Studies Depression Scale (CES-D)[51]. The ADS consists of 20 items that refer to the previous week and are answered using a four-point Likert scale. The total score ranges from 0 to 60. Its internal consistency is α =0.89, and its split-half reliability is r =0.91[50].

Quality of life

Quality of life will be measured using the EuroQol (EQ-5D)[52] and the Short Form 12 (SF-12)[53]. The EQ-5D is a well-established measurement of quality of life that consists of five items assessing mobility, self-care, common activities, pain and/or discomfort, and anxiety and/or depression, as well as a visual analogue scale concerning health status. We will also use the SF-12, which consists of 12 items that assess eight health domains: physical functioning, role limitations, pain, general health perception, vitality, mental health, emotional role, and social functioning. The SF-12 generates two summary scores: physical health and mental health.

Agoraphobic cognitions

Agoraphobic cognitions will be measured using the Agoraphobic Cognitions Questionnaire (ACQ)[54, 55]. The ACQ consists of 14 items, and its total score ranges from 14 to 70. Craske et al. reported that the ACQ has an internal reliability of α =0.80[56].

Bodily sensations

The Body Sensation Questionnaire (BSQ)[54, 55] is a 17-item self-report questionnaire that measures bodily sensations. The BSQ ranges from 17 to 85 points and has a satisfactory internal reliability of α =0.87[54].

Agoraphobic avoidance

The Mobility Inventory (MI)[54, 55] measures agoraphobic avoidance. The MI consists of 27 items that address the most important agoraphobic situations. Each item is rated both for when patients are alone and when they are accompanied. These two scales have a summed score ranging from 27 to 135 points, respectively. The internal consistencies are α =0.94 (alone) and α =0.91 (accompanied)[57].

Observer rating anxiety symptoms

In addition, the secondary outcome measures will include the observer-rated HAM-A[43, 58] to obtain a more detailed understanding of symptom severity. The HAM-A is a 14-item clinician-reported rating scale with a total score ranging from 0 to 30. Thus, we will use the structured interview guide for the HAM-A (SIGH-A)[59]. The interview has shown satisfactory inter-rater and test-retest reliabilities of Intraclass Correlation Coefficients of respectively 0.99 and 0.89[59].

Economic evaluation

We will use an adaption of the Trimbos/iMTA questionnaire to measure the costs associated with psychiatric illness (TiC-P)[60] with regard to the German healthcare system.

Diary data

EMA data will be collected via the mobile application GET.ON PANIC APP[47]. This app contains a diary where clients can record their panic attacks and monitor their progress using daily summaries. With regard to the latter, clients will summarize their general anxiety levels, their degree of avoidance, and their moods each evening. The application also records the type and number of exposure exercises performed by the client.

Additional measures

We will also collect data concerning technology acceptance (via a questionnaire based on the technology acceptance model; TAM)[61, 62], the usability of the smartphone app GET.ON PANIC APP (via the System Usability Scale; SUS)[63, 64], user satisfaction of the online training (a self-designed questionnaire based on the German version of the Client Satisfaction Questionnaire[65, 66], and the adverse effects of psychotherapy[67].

Statistical analyses

The analyses will be performed based on the Consolidated Standards of Reporting Trials (CONSORT) statement regarding eHealth[68]. The data will be analyzed on an intention-to-treat basis. We will also conduct per protocol and completers-only investigations as secondary analyses.

Treatment efficacy

Group differences in the baseline values of the primary outcome will be compared using t-tests to assess whether randomization was successful. Missing data will be addressed following the recommendations of Little and Rubin[69] and Schafer[70]. We will analyze the PAS data at eight weeks post-treatment using between-group analyses of covariance with regard to the individual baseline PAS scores, adjusted for sex, age, and socioeconomic status. We will use Cohen’s d to measure the between-group effect size. Cohen’s d will be calculated as the difference between the mean post-test scores of the intervention group and the control group divided by the pooled standard deviation[71]. All other secondary outcomes will be analyzed in a similar manner.

We will also conduct clinical significance change analyses as described by Jacobson and Truax[72]. In the first step, we will test whether the changes from pre-test to post-test are statistically reliable and build a reliable change index (RCI). In the second step, we will calculate clinical significance. Based on the RCI, the participants who display a reliable positive change, no change, or a reliable deteriorated change will be classified as responders, non-responders, or deteriorated, respectively[72]. We will use mixed-model regressions to examine the long-term effect on the primary outcome after three and six months.

Economic analyses

We will conduct an economic evaluation by performing a cost-effectiveness analysis (CEA) and a cost-utility analysis (CUA) from a societal perspective. The clinical outcome of the CEA will be the severity of panic symptoms as assessed by the PAS. Quality-adjusted life years (QALYs) will be calculated for the CUA. A non-parametric bootstrapping method with 95% confidence intervals will be used to assess the differences between the intervention and control groups. The intervention and waitlist control groups will be compared in terms of incremental costs and incremental effects. Thus, we will calculate the incremental cost-effectiveness ratio (ICER). Bootstrapping via 5,000 iterations will be used to quantify the uncertainty in the ratios and to test the robustness of the ICER. The results will be displayed on a cost-effectiveness plane and via a cost-effective acceptability curve. A multi-way sensitivity analysis will be conducted to test the robustness of the base-case findings. For instance, these analyses will integrate data based on the EMA to reduce the retrospective bias of self-reported panic symptom severity.

Discussion

Given the availability and the daily use of smartphones, mobile-based health interventions have become increasingly popular[73–78]. Evidence for the efficacy of these studies comes from attempts to promote physical activity[79], cope with schizophrenia[25], and overcome child anxiety[80]. Despite the obvious advantages of integrating mobile components into online-based treatments (for example, mobile components can be used for EMA, supporting exposure exercises away from home, and EMI), no data are available regarding the efficacy of such an intervention for PD. Thus, we developed the hybrid online training GET.ON PANIC based on CBT, and we will test the efficacy and cost-effectiveness of this online training among people with clinical or subclinical PD with or without agoraphobia at post-assessment, as well as at three- and six-month follow-ups, compared with a waitlist control group who will receive the online training after the last assessment, but who will have free access to the usual treatments.

This study has several limitations. First, the absence of an active control group does not allow us to clarify the mechanisms responsible for changes in treatment. However, because this study is one of the first to test the efficacy of hybrid online training for PD, evaluating the efficacy of the intervention is an important research question in and of itself. If the present study provides evidence regarding the efficacy of GET.ON PANIC, then future research should detect the mechanisms that drive the change within the intervention. Such research should also include dismantling studies that compare online-based interventions with or without a mobile component to explain the additional benefits of including a mobile component. Second, using a waitlist control group design does not enable a comparison between newly developed hybrid online training and the current gold standard (face-to-face CBT to treat people with PD and agoraphobia). Future studies should compare hybrid online training with the gold standard. Finally, this study is not designed to test the long-term efficacy of GET.ON PANIC. With an assessment time of six months, we will only be able to make conclusions over a relatively short period of time. A 12-month follow-up assessment is desirable.

Trial status

The study is currently ongoing. Recruitment began in August 2013 and will conclude in October 2014.

Abbreviations

ACQ: 

Agoraphobic cognitions questionnaire

BDI-II: 

Beck Depression Inventory II

BSQ: 

Body sensations questionnaire

CBT: 

Cognitive behavioral therapy

CEA: 

Cost-effectiveness analysis

CES-D: 

Center for epidemiological studies depression scale

CONSORT: 

Consolidated standards of reporting trials

CUA: 

Cost-utility analysis

DRKS: 

German Clinical Trial Register

EMA: 

Ecological momentary assessment

EMGO: 

Institute for Health and Care Research

EMI: 

Ecologically momentary intervention

EQ-5D: 

EuroQol

HAM-A: 

Hamilton anxiety scale

ICER: 

Incremental cost-effectiveness ratio

INEP: 

Adverse effects of psychotherapy

MI: 

Mobility inventory

PAS: 

Panic and agoraphobia scale

PD: 

Panic disorder

QALYs: 

Quality-adjusted life years

RCI: 

Reliable change index

RCT: 

Randomized controlled trial

SCID-I: 

Structured clinical interview for DSM-IV Axis I Disorders

SF-12: 

Short form 12

SUS: 

System Usability Scale

TAM: 

Technology acceptance model

TiC-P: 

Trimbos and the Institute of Medical Technology Assessment Cost Questionnaire for Psychiatry

VU: 

Vrije Universiteit.

Declarations

Acknowledgements

The European Union funded all authors of the associated project (project number: EFRE: CCI 2007DE161PR001).

Authors’ Affiliations

(1)
Division of Health Training Online, Leuphana University Lueneburg, Innovation Incubator, Rotenbleicher Weg 67, 21335 Lueneburg, Germany
(2)
Chair of Clinical Psychology and Psychotherapy, Friedrich-Alexander-University Erlangen-Nuremberg, Bismarckstraße 1, 91054 Erlangen, Germany
(3)
Department of Psychology, Institute of Personality and Social Research, University of California, Berkeley, 4152 Tolman Hall, Berkeley, California 94720-1650, USA
(4)
Institute for Health and Care Research (EMGO), VU University Medical Center, Van der Boechorststraat 7, 1081, BT Amsterdam, The Netherlands
(5)
Faculty of Psychology and Education, VU University Amsterdam, Van der Boechorststraat 1, 1081, BT Amsterdam, The Netherlands

References

  1. American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders: DSM-5. 2013, Washington, DC: American Psychiatric Publishing, 5Google Scholar
  2. Wittchen H, Jacobi F, Rehm J, Gustavsson A, Svensson M, Jönsson B, Olesen J, Allgulander C, Alonso J, Faravelli C, Fratiglioni L, Jennum P, Lieb R, Maercker A, van Os J, Preisig M, Salvador-Carulla L, Simon R, Steinhausen H: The size and burden of mental disorders and other disorders of the brain in Europe 2010. Eur Neuropsychopharmacol. 2011, 21: 655-679. 10.1016/j.euroneuro.2011.07.018.View ArticlePubMedGoogle Scholar
  3. Goodwin R, Faravelli C, Rosi S, Cosci F, Truglia E, de Graaf R, Wittchen H: The epidemiology of panic disorder and agoraphobia in Europe. Eur Neuropsychopharmacol. 2005, 15: 435-443. 10.1016/j.euroneuro.2005.04.006.View ArticlePubMedGoogle Scholar
  4. Barrera TL, Norton PJ: Quality of life impairment in generalized anxiety disorder, social phobia, and panic disorder. J Anxiety Disord. 2009, 23: 1086-1090. 10.1016/j.janxdis.2009.07.011.View ArticlePubMedPubMed CentralGoogle Scholar
  5. Bystritsky A, Kerwin L, Niv N, Natoli JL, Abrahami N, Klap R, Wells K, Young AS: Clinical and subthreshold panic disorder. Depress Anxiety. 2010, 27: 381-389. 10.1002/da.20622.View ArticlePubMedPubMed CentralGoogle Scholar
  6. Batelaan N, Smit F, de Graaf R, van Balkom A, Vollebergh W, Beekman A: Economic costs of full-blown and subthreshold panic disorder. J Affect Disord. 2007, 104: 127-136. 10.1016/j.jad.2007.03.013.View ArticlePubMedGoogle Scholar
  7. Sánchez-Meca J, Rosa-Alcázar AI, Marín-Martínez F, Gómez-Conesa A: Psychological treatment of panic disorder with or without agoraphobia: a meta-analysis. Clin Psychol Rev. 2010, 30: 37-50. 10.1016/j.cpr.2009.08.011.View ArticlePubMedGoogle Scholar
  8. Andrews G: Shortfall in mental health service utilisation. Br J Psychiatry. 2001, 179: 417-425. 10.1192/bjp.179.5.417.View ArticlePubMedGoogle Scholar
  9. Coles ME, Coleman SL: Barriers to treatment seeking for anxiety disorders: initial data on the role of mental health literacy. Depress Anxiety. 2010, 27: 63-71. 10.1002/da.20620.View ArticlePubMedGoogle Scholar
  10. Mojtabai R, Olfson M, Sampson NA, Jin R, Druss B, Wang PS, Wells KB, Pincus HA, Kessler RC: Barriers to mental health treatment: results from the national comorbidity survey replication. Psychol Med. 2011, 41: 1751-1761. 10.1017/S0033291710002291.View ArticlePubMedGoogle Scholar
  11. Andersson G, Cuijpers P: Pros and cons of online cognitive-behavioural therapy. Br J Psychiatry. 2008, 193: 270-271. 10.1192/bjp.bp.108.054080.View ArticlePubMedGoogle Scholar
  12. Marks IM, Cavanagh K, Gega L: Hands-on help: Computer-aided psychotherapy. Maudsley monographs, (David AS (Series editor): volume 49). Edited by: David AS. 2007, New York, NY: Psychology PressGoogle Scholar
  13. Haug T, Nordgreen T, Öst LG, Havik OE: Self-help treatment of anxiety disorders: a meta-analysis and meta-regression of effects and potential moderators. Clin Psychol Rev. 2012, 32: 425-445. 10.1016/j.cpr.2012.04.002.View ArticlePubMedGoogle Scholar
  14. Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N, Baune BT: Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One. 2010, 5: e13196-10.1371/journal.pone.0013196.View ArticlePubMedPubMed CentralGoogle Scholar
  15. Cuijpers P, Donker T, van Straten A, Li J, Andersson G: Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies. Psychol Med. 2010, 40: 1943-1957. 10.1017/S0033291710000772.View ArticlePubMedGoogle Scholar
  16. Mewton L, Smith J, Rossouw P, Andrews G: Current perspectives on Internet delivered cognitive behavioral therapy for adults with anxiety and related disorders. PRBM. 2014, 7: 37-46.View ArticleGoogle Scholar
  17. Marks IM, Kenwright M, McDonough M, Whittaker M, Mataix-Cols D: Saving clinicians’ time by delegating routine aspects of therapy to a computer: a randomized controlled trial in phobia/panic disorder. Psychol Med. 2004, 34: 9-18. 10.1017/S003329170300878X.View ArticlePubMedGoogle Scholar
  18. van Ballegooijen W, Riper H, Klein B, Ebert DD, Kramer J, Meulenbeek P, Cuijpers P: An Internet-based guided self-help intervention for panic symptoms: randomized controlled trial. J Med Internet Res. 2013, 15: e154-10.2196/jmir.2362.View ArticlePubMedPubMed CentralGoogle Scholar
  19. Gloster AT, Wittchen H, Einsle F, Lang T, Helbig-Lang S, Fydrich T, Fehm L, Hamm AO, Richter J, Alpers GW, Gerlach AL, Ströhle A, Kircher T, Deckert J, Zwanzger P, Höfler M, Arolt V: Psychological treatment for panic disorder with agoraphobia: A randomized controlled trial to examine the role of therapist-guided exposure in situ in CBT. J Consult Clin Psychol. 2011, 79: 406-420.View ArticlePubMedGoogle Scholar
  20. Lind C, Boschen MJ, Morrissey S: Technological advances in psychotherapy: implications for the assessment and treatment of obsessive compulsive disorder. J Anxiety Disord. 2013, 27: 47-55. 10.1016/j.janxdis.2012.09.004.View ArticlePubMedGoogle Scholar
  21. Kelders SM, Kok RN, Ossebaard HC, van Gemert-Pijnen JE: Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res. 2012, 14: e152-10.2196/jmir.2104.View ArticlePubMedPubMed CentralGoogle Scholar
  22. Fahrenberg J, Myrtek M, Pawlik K, Perrez M: Ambulatory assessment - monitoring behavior in daily life settings. Eur J Psychol Assess. 2007, 23: 206-213. 10.1027/1015-5759.23.4.206.View ArticleGoogle Scholar
  23. Solhan MB, Trull TJ, Jahng S, Wood PK: Clinical assessment of affective instability: comparing EMA indices, questionnaire reports, and retrospective recall. Psychol Assess. 2009, 21: 425-436.View ArticlePubMedPubMed CentralGoogle Scholar
  24. Stone AA, Broderick JE: Real-time data collection for pain: appraisal and current status. Pain Med. 2007, 8: 85-93. 10.1111/j.1526-4637.2007.00372.x.View ArticleGoogle Scholar
  25. Ben-Zeev D, Brenner CJ, Begale M, Duffecy J, Mohr DC, Mueser KT: Feasibility, acceptability, and preliminary efficacy of a smartphone intervention for schizophrenia. Schizophr Bull. 2014, 40: 1244-1253. 10.1093/schbul/sbu033.View ArticlePubMedPubMed CentralGoogle Scholar
  26. Ebner-Priemer UW, Kuo J, Welch SS, Thielgen T, Witte S, Bohus M, Linehan MM: A valence-dependent group-specific recall bias of retrospective self-reports: a study of borderline personality disorder in everyday life. J Nerv Ment Dis. 2006, 194: 774-779. 10.1097/01.nmd.0000239900.46595.72.View ArticlePubMedGoogle Scholar
  27. Ebner-Priemer UW, Bohus M, Kuo J: Can retrospective interviews assess instability? A comparison of ambulatory assessment and expert interviews regarding DSM-IV criteria for borderline personality disorder. Frontiers in Stepped eCare. Edited by: Sorbi MJ, Rüddel H, Bühring M. 2007, Utrecht: University Utrecht, 25-37.Google Scholar
  28. Ebner-Priemer UW, Trull TJ: Ecological momentary assessment of mood disorders and mood dysregulation. Psychol Assess. 2009, 21: 463-475.View ArticlePubMedGoogle Scholar
  29. Alpers GW: Ambulatory assessment in panic disorder and specific phobia. Psychol Assess. 2009, 21: 476-485.View ArticlePubMedGoogle Scholar
  30. Myin-Germeys I, Oorschot M, Collip D, Lataster J, Delespaul P, van Os J: Experience sampling research in psychopathology: opening the black box of daily life. Psychol Med. 2009, 39: 1533-1547. 10.1017/S0033291708004947.View ArticlePubMedGoogle Scholar
  31. Moskowitz DS, Russell JJ, Sadikaj G, Sutton R: Measuring people intensively. Can Psychol. 2009, 50: 131-140.View ArticleGoogle Scholar
  32. Helbig S, Lang T, Swendsen J, Hoyer J, Wittchen H: Implementierung, Akzeptanz und Informationsgehalt eines Ecological Momentary Assessment (EMA)-Ansatzes bei Patienten mit Panikstörung und Agoraphobie. Z Kl Psych Psychoth. 2009, 38: 108-117. 10.1026/1616-3443.38.2.108.View ArticleGoogle Scholar
  33. Trull TJ, Ebner-Priemer UW: Using experience sampling methods/ecological momentary assessment (ESM/EMA) in clinical assessment and clinical research: introduction to the special section. Psychol Assess. 2009, 21: 457-462.View ArticlePubMedPubMed CentralGoogle Scholar
  34. Bandelow B: Panik- und Agoraphobieskala (PAS). 1997, Göttingen/Bern/Toronto/Seattle: HogrefeGoogle Scholar
  35. Bandelow B: Assessing the efficacy of treatments for panic disorder and agoraphobia. II. The panic and agoraphobia scale. Int Clin Psychopharmacol. 1995, 10: 73-81.View ArticlePubMedGoogle Scholar
  36. Schneider S, Margraf J: Agoraphobie und Panikstörung. Fortschritte der Psychotherapie, Volume 3. Edited by: Schulte D, Hahlweg K, Margraf J, Vaitl D. 1998, Göttingen: HogrefeGoogle Scholar
  37. Beck AT, Steer RA, Brown GK: Manual for the Beck Depression Inventory-II: Manual. 1996, San Antonio, TX, Boston: Psychological Corp, 2Google Scholar
  38. Hautzinger M, Keller F, Kühner C: BDI-II. Beck Depressions-Inventar Revision. 2006, Frankfurt am Main: Harcourt Test ServicesGoogle Scholar
  39. Wittchen HU, Zaudig M, Fydich T: Strukturiertes Klinisches Interview für DSM-IV Achse I und II [Structured Clinical Interview for DSM-IV Axis I and II]. 1997, Göttingen: HogrefeGoogle Scholar
  40. DatInf GmbH: DatInf RandList Version 1.2. 2007, Tübingen: DatInf GmbHGoogle Scholar
  41. Rohde P, Lewinsohn PM, Seeley JR: Comparability of telephone and face-to-face interviews in assessing axis I and II disorders. Am J Psychiatry. 1997, 154: 1593-1598.View ArticlePubMedGoogle Scholar
  42. Irvine A, Drew P, Sainsbury R: ‘Am I not answering your questions properly?’ Clarification, adequacy and responsiveness in semi-structured telephone and face-to-face interviews. J Qual Res. 2013, 13: 87-106. 10.1177/1468794112439086.View ArticleGoogle Scholar
  43. Hamilton M: The assessment of anxiety states by rating. Brit J Med Psychology. 1959, 32: 50-55. 10.1111/j.2044-8341.1959.tb00467.x.View ArticleGoogle Scholar
  44. Carlbring P: Remote treatment of panic disorder: a randomized trial of internet-based cognitive behavior therapy supplemented with telephone calls. Am J Psychiatry. 2006, 163: 2119-2125. 10.1176/appi.ajp.163.12.2119.View ArticlePubMedGoogle Scholar
  45. Klein B, Richards JC, Austin DW: Efficacy of internet therapy for panic disorder. J Behav Ther Exp Psy. 2006, 37: 213-238. 10.1016/j.jbtep.2005.07.001.View ArticleGoogle Scholar
  46. Lindner P, Ivanova E, Ly K, Andersson G, Carlbring P: Guided and unguided CBT for social anxiety disorder and/or panic disorder via the internet and a smartphone application: study protocol for a randomised controlled trial. Trials. 2013, 14: 437-10.1186/1745-6215-14-437.View ArticlePubMedPubMed CentralGoogle Scholar
  47. Kleine Stegemann S, Ebenfeld L, Lehr D, Berking M, Funk B, Kleine Stegemann S, Ebenfeld L, Lehr D, Berking M, Funk B: Development of a mobile application for people with panic disorder as augmentation for an internet-based intervention. Federated Conference on Computer Science and Information Systems: 8–11 September 2013; Kraków, Poland. Edited by: Ganzha M, Maciaszek L, Paprzycki M. 2013, New York City: The Institute of Electrical and Electronics Engineers, 1319-1325.Google Scholar
  48. Bandelow B, Broocks A, Pekrun G, George A, Meyer T, Pralle L, Bartmann U, Hillmer-Vogel U, Rüther E: The use of the panic and agoraphobia scale (P & A) in a controlled clinical trial. Pharmacopsychiatry. 2000, 33: 174-181. 10.1055/s-2000-12982.View ArticlePubMedGoogle Scholar
  49. Bandelow B, Hajak G, Holzrichter S, Kunert HJ, Rüther E: Assessing the efficacy of treatments for panic disorder and agoraphobia. I Methodological problems. Int Clin Psychopharmacol. 1995, 10: 83-93. 10.1097/00004850-199506000-00004.View ArticlePubMedGoogle Scholar
  50. Hautzinger M, Bailer M: Allgemeine Depressions Skala: Manual. 1993, Göttingen: Beltz-Test-GmbHGoogle Scholar
  51. Radloff LS: The CES-D Scale: a self-report depression scale for research in the general population. Appl Psychol Meas. 1977, 1: 385-401. 10.1177/014662167700100306.View ArticleGoogle Scholar
  52. Group EQ: EuroQol. A new facility for the measurement of health-related quality of life. Health Policy. 1990, 16: 199-208.View ArticleGoogle Scholar
  53. Ware J, Kosinski M, Keller SD: A 12-item short-form health survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996, 34: 220-233. 10.1097/00005650-199603000-00003.View ArticlePubMedGoogle Scholar
  54. Chambless DL, Caputo GC, Bright P, Gallagher R: Assessment of fear of fear in agoraphobics: the body sensations questionnaire and the agoraphobic cognitions questionnaire. J Consult Clin Psychol. 1984, 52: 1090-1097.View ArticlePubMedGoogle Scholar
  55. Ehlers A, Margraf J: Fragebogen zu körperbezogenen Ängsten, Kognitionen und Vermeidung (AKV). 2001, Göttingen: Beltz-Test-GmbH, 2Google Scholar
  56. Craske MG, Rachman SJ, Tallman K: Mobility, cognitions, and panic. J Psychopathol Behav Assess. 1986, 8: 199-210. 10.1007/BF00959832.View ArticleGoogle Scholar
  57. Chambless DL, Caputo G, Jasin SE, Gracely EJ, Williams C: The mobility inventory for agoraphobia. Beh Res Ther. 1985, 23: 35-44. 10.1016/0005-7967(85)90140-8.View ArticleGoogle Scholar
  58. Weyer G: Internationale Skalen für Psychiatrie. 2005, Göttingen: Beltz-Test-GmbH, 5Google Scholar
  59. Shear MK, Vander Bilt J, Rucci P, Endicott J, Lydiard B, Otto MW, Pollack MH, Chandler L, Williams J, Ali A, Frank DM: Reliability and validity of a structured interview guide for the Hamilton Anxiety Rating Scale (SIGH-A). Depress Anxiety. 2001, 13: 166-178. 10.1002/da.1033.View ArticlePubMedGoogle Scholar
  60. Hakkaart-van Roijen L: Manual Trimbos/iMTA Questionnaire for Costs Associated with Psychiatric Illness (in Dutch). 2002, Rotterdam: Institute for Medical Technology AssessmentGoogle Scholar
  61. Davis FD, Bagozzi RP, Warshaw PR: User acceptance of computer technology: a comparison of two theoretical models. Manag Sci. 1989, 35: 982-1003. 10.1287/mnsc.35.8.982.View ArticleGoogle Scholar
  62. Venkatesh V, Davis FD: A theoretical extension of the technology acceptance model: four longitudinal field studies. Manag Sci. 2000, 46: 186-204. 10.1287/mnsc.46.2.186.11926.View ArticleGoogle Scholar
  63. Brooke J: SUS: a "quick and dirty" usability scale. Usability Evaluation in Industry. Edited by: Jordan PW, Thomas B, Weerdmeester BA, McClelland IL. 1996, London: Taylor and Francis, 189-194. 1Google Scholar
  64. Bangor A, Kortum PT, Miller JT: An empirical evaluation of the system usability scale. Int J Hum-Comp Int. 2008, 24: 574-594. 10.1080/10447310802205776.View ArticleGoogle Scholar
  65. Schmidt J, Lamprecht F, Wittmann WW: Zufriedenheit mit der stationären Versorgung. Entwicklung eines Fragebogens und erste Validitätsuntersuchungen. Psychother Psychosom Med Psychol. 1989, 39: 248-255.PubMedGoogle Scholar
  66. Attkisson CC, Zwick R: The client satisfaction questionnaire. Psychometric properties and correlations with service utilization and psychotherapy outcome. Eval Program Plann. 1982, 5: 233-237. 10.1016/0149-7189(82)90074-X.View ArticlePubMedGoogle Scholar
  67. Nestoriuc Y, Rief W: Nebenwirkungen von Verhaltenstherapie. Erfassung, Bewältigung, Risikovermeidung. Risiken und Nebenwirkungen von Psychotherapie. Edited by: Linden M, Strauss B. 2012, Berlin: Medizinisch Wissenschaftliche Verlagsgesellschaft, 59-74.Google Scholar
  68. Eysenbach G: CONSORT-EHEALTH: improving and standardizing evaluation reports of web-based and mobile health interventions. J Med Internet Res. 2011, 13: e126-10.2196/jmir.1923.View ArticlePubMedPubMed CentralGoogle Scholar
  69. Little RJA, Rubin DB: Statistical analysis with missing data. Wiley Series in Probability and Statistics. Edited by: Balding DJ, Cressie NA, Fitzmaurice G, Goldstein H, Johnstone IM, Molenberghs G, Scott DW, Tsay RS, Smith AFM, Weisberg S. 2002, Hoboken, NJ: Wiley, 2Google Scholar
  70. Schafer JL: Analysis of incomplete multivariate data. Monographs on Statistics and Applied Probability. Volume 72. Edited by: Smith RL, Isham V, Tong H, Keiding N, Louis TA, Bunea F, Fan J. 2000, Boca Raton: Chapman & Hall/CRC, 1Google Scholar
  71. Cohen J: Statistical Power Analysis for the Behavioral Sciences. 1988, Hillsdale, NJ: Lawrence Erlbaum Associates, 2Google Scholar
  72. Jacobson NS, Truax P: Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol. 1991, 59: 12-19.View ArticlePubMedGoogle Scholar
  73. Boschen MJ, Casey LM: The use of mobile telephones as adjuncts to cognitive behavioral psychotherapy. Prof Psychol Res Pract. 2008, 39: 546-552.View ArticleGoogle Scholar
  74. Eonta AM, Christon LM, Hourigan SE, Ravindran N, Vrana SR, Southam-Gerow MA: Using everyday technology to enhance evidence-based treatments. Prof Psychol Res Pract. 2011, 42: 513-520.View ArticleGoogle Scholar
  75. Ehrenreich B, Righter B, Rocke DA, Dixon L, Himelhoch S: Are mobile phones and handheld computers being used to enhance delivery of psychiatric treatment?. J Nerv Ment Dis. 2011, 199: 886-891. 10.1097/NMD.0b013e3182349e90.View ArticlePubMedGoogle Scholar
  76. Proudfoot J: The future is in our hands: the role of mobile phones in the prevention and management of mental disorders. Aust N Z J Psychiatry. 2013, 47: 111-113. 10.1177/0004867412471441.View ArticlePubMedGoogle Scholar
  77. Reger MA, Gahm GA: A meta-analysis of the effects of internet- and computer-based cognitive-behavioral treatments for anxiety. J Clin Psychol. 2009, 65: 53-75. 10.1002/jclp.20536.View ArticlePubMedGoogle Scholar
  78. Donker T, Petrie K, Proudfoot J, Clarke J, Birch M, Christensen H: Smartphones for smarter delivery of mental health programs: a systematic review. J Med Internet Res. 2013, 15: e247-10.2196/jmir.2791.View ArticlePubMedPubMed CentralGoogle Scholar
  79. Antypas K, Wangberg SC: An internet- and mobile-based tailored intervention to enhance maintenance of physical activity after cardiac rehabilitation: short-term results of a randomized controlled trial. J Med Internet Res. 2014, 16: e77-10.2196/jmir.3132.View ArticlePubMedPubMed CentralGoogle Scholar
  80. Pramana G, Parmanto B, Kendall PC, Silk JS: The SmartCAT: an m-health platform for ecological momentary intervention in child anxiety treatment. Telemed J E Health. 2014, 20: 419-427. 10.1089/tmj.2013.0214.View ArticlePubMedPubMed CentralGoogle Scholar

Copyright

© Ebenfeld et al.; licensee BioMed Central Ltd. 2014

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate. Please note that comments may be removed without notice if they are flagged by another user or do not comply with our community guidelines.

Advertisement

Trials

ISSN: 1745-6215