Evaluation of a smartphone application for self-help for patients with social anxiety disorder: a randomized controlled study—SMASH
Trials volume 24, Article number: 154 (2023)
There is growing evidence that Internet-based cognitive behavioral therapy (ICBT) is as effective as a stand-alone treatment and helps facilitating access to treatment. Given the complexity of the treatment, we argue that the effect of ICBT could be even greater if guided by a therapist, as this could increase treatment adherence. We modified an established and well-evaluated treatment approach and developed a mobile application for treating social anxiety disorder (SAD). In the present study, we compare the efficacy of app use alone (APP) with video-based, therapist-guided app use (TG-APP) and with a wait-list control group (WLC) in terms of symptom reduction, and various secondary outcomes such as increase in quality of life or decrease of general psychological distress.
A within-between interaction design with randomization to one of three conditions will be used. In the APP condition, patients receive only the app without any additional contact with therapists, while in the TG-APP condition, therapists provide 8 sessions of video-based treatment in addition to using the app. The study will be conducted in two university outpatient treatment centers with reliably diagnosed SAD patients. The primary outcome will be defined as change in SAD symptoms, as measured by the Liebowitz Social Anxiety Scale (expert rating). Furthermore, a wide range of self-reports and clinician ratings for other symptoms (depression, general psychopathology) or quality of life will be used. A simulation-based power analysis for a 3 × 2 interaction effect (group × time) on the primary outcome in a linear mixed model resulted in a total sample size of N = 165.
The present study will be one of the first to examine the additional benefit of therapist-guided video sessions regarding the use of an app treating SAD. Study results are pivotal to future treatment application in SAD.
Evaluation of a smartphone application for self-help for patients with social anxiety disorder: a randomized controlled study (SMASH)
Trial registration: ClinicalTrials.gov, NCT05554718. Registered 16 September 2022, https://clinicaltrials.gov/show/NCT05554718.
The pilot study was registered in the German Clinical Trials Register (DRKS), DRKS00029701. Registered 01 August 2022. https://www.drks.de/drks_web/navigate.do?navigationId=trial.HTML&TRIAL_ID=DRKS00029701
Version 2 of 16-12-2022
This study is funded by Mindable Health GmbH, Germany. Contact: email@example.com
J. M. Schittenhelm: Goethe Universitaet Frankfurt, Germany
C. von Borell: Technische Universitaet Dresden, Germany
C. Clément: Goethe Universitaet Frankfurt, Germany
J. Schüller: Goethe Universitaet Frankfurt, Germany
U. Stangier: Goethe Universitaet Frankfurt, Germany
J. Hoyer: Technische Universitaet Dresden, Germany
Name and contact information for the trial sponsor
Role of sponsor
The funders do not have a role in collection, analysis, and interpretation of data.
Social anxiety disorder (SAD) is characterized by the fear of behaving in a way that is negatively perceived by others . In the USA, the disorder has a lifetime prevalence of 10.7% and a 12-month prevalence of 7.4% ; in Europe, the estimated lifetime prevalence is 6.7% . The disorder is associated with high psychosocial impairment and disability .
On the one hand, there are ways to treat SAD. For example, a recent meta-analysis demonstrated that individualized cognitive therapy (CT) based on the model of Clark and Wells  was among the most effective treatments, outperforming pharmacotherapy and other psychotherapeutic approaches in several studies . On the other hand, although treatment is recommended in international guidelines, access to effective treatment for SAD patients remains difficult, in part because SAD-related anxiety and impairment exacerbate difficulties and delays in initial treatment contact [6,7,8].
Addressing these treatment barriers, Lee and Stapinski  found evidence that social anxiety is related to a perception of decreased risk of negative evaluation and greater control in online communication. Therefore, Internet-delivered cognitive-behavioral treatments (ICBTs) may be a promising approach to the treatment of SAD. ICBTs may reduce the threshold to seek treatment and provide cost-efficient [10, 11] and highly scalable access to scientifically evaluated treatment programs . Recent empirical data shows that internet-based cognitive behavioral treatments can be as effective as traditional face-to-face therapy [12, 13] and also reach favorable long-term outcomes [12, 14]. Stott et al.  for example were able to show that an online treatment program, based on the Clark and Wells approach, offered through a website, was significantly reducing SAD-related symptoms.
Previous studies that have examined the effectiveness of ICBTs have largely used treatment programs delivered via a computer. Today, however, the use of a smartphone including its apps is even more widespread, rendering mobiles a promising option for the treatment of mental disorders. It might even have more advantages than ICBTs delivered via a computer, as apps seem to be easily integrated into daily routines  and are convenient for both the patient and the practitioner . Furthermore, using a mobile phone has the advantage that patients can use protocols or other forms of assessments when the situation occurs so that some forms of perception and reporting biases can be minimized . As an example, Stolz et al.  have been able to show that an app based on the Clark and Wells  treatment approach with text-based guidance during the process can significantly reduce symptoms of social anxiety. However, it remains unclear whether their app would also work without additional guidance.
This raises the important research question of whether the effectiveness of apps can be enhanced by accompanying face-to-face sessions with a therapist. To date, few studies have addressed this question. The authors assume that therapist guidance could additionally increase compliance with ICBT treatment as well as the motivation to complete it. Another advantage of therapist-guided ICBT over nonguided ICBT could be higher patient safety, as the therapist can assist when questions or ambiguities arise, or when adverse side effects of ICBT occur, such as worsening of symptoms, negative well-being, or noncompliance .
Particularly as a result of the COVID-19 pandemic, new ways of providing therapeutic support have been developed: Due to the limitations of physical contact, the use of videoconferencing has increased dramatically. A meta-analysis has shown that video-based psychotherapy typically has large effect sizes and that the difference to in-person therapy is negligible . The use of this modality to treat mental disorders could be an important step in increasing accessibility to psychotherapy, not only because of the ongoing (or possibly recurring) pandemic situation, but also to overcome other barriers to in-person treatment, such as illnesses that limit the scope of activities, specialized treatment programs that exist only in distant locations, or difficulty finding a therapist in rural areas.
In summary, the use of ICBT delivered via a smartphone application could improve the accessibility of psychotherapy and integrate interventions into everyday life, making it a promising approach for the future treatment of SAD. However, few studies have attempted to implement Clark and Wells’  well-evaluated approach into a treatment program in form of an app, and even fewer have examined whether the app works without further guidance. Therefore, our goal was to develop and evaluate an app that could effectively help patients without therapeutic support. In addition, questions remain about how the app and face-to-face therapy can be combined and how this combined treatment compares to using the app alone. We aim to investigate these questions in a randomized controlled trial focusing on (1) the overall efficacy of a newly developed app for SAD and (2) the potential benefit of additional therapeutic support delivered via videoconferencing.
The main goal of the present study is to evaluate whether the use of the app-based intervention helps to decrease symptoms of social anxiety disorder and has positive effects on mental health. There will be two experimental treatment conditions: App-based CT (APP) and app-based CT plus 8 therapist-guided video-delivered therapy sessions (TG-APP). Both will be compared to a wait-list control group (WLC). The therapists will be trained in Cognitive Therapy based on the model of Clark and Wells , which has been proven to be effective in face-to-face therapy  and in ICBT . In a randomized controlled trial (RCT), the following hypotheses will be tested:
Hypothesis 1a: In the APP and TG-APP condition, SAD-related symptoms, as measured by the LSAS, will decrease significantly more from baseline to post-treatment than in the WLC condition.
Hypothesis 1b: The TG-APP condition results in a significantly greater reduction in SAD-related symptoms, as measured by the LSAS, than the APP and the WLC conditions, from baseline to post-treatment.
Hypothesis 1c: In the APP and the TG-APP condition, there will be no significant deterioration of SAD-related symptoms, as measured by the LSAS, from post-treatment to follow-up.
Hypothesis 2a: In the APP and TG-APP conditions, significant improvements in secondary outcome variables, including social phobic cognitions, depression symptoms, general psychological distress, psychological impairment, quality of life, and interpersonal pleasure, are achieved from baseline to post-treatment compared to the WLC condition.
Hypothesis 2b: In the TG-APP condition, improvements in secondary outcome variables from baseline to post-treatment will be significantly larger than in the APP condition.
Hypothesis 2c: In the APP and TG-APP condition, there will be no significant deterioration in secondary outcome variables from post-treatment to follow-up.
Hypothesis 3: Individual learning styles contribute significantly to predicting success in both treatment conditions.
Hypothesis 4: The increase in social approach behavior will mediate the effect of the TG-APP versus APP condition.
The design of the planned bi-centric randomized controlled superiority trial will be based on a within-between interaction group design with three groups (WLC, APP, TG-APP) and parallel group assignment. The primary outcome criterion is the change in symptom severity of SAD, as measured by the Liebowitz Social Anxiety Scale (LSAS) [21, 22]. It will be assessed by trained, blinded, independent raters at pre-treatment, post-treatment, and 6 months after post-treatment (follow-up). The planned allocation ratio for TG-APP vs. APP only vs. WLC will be 1:1:0.7.
Design and sample size
We conducted a simulation-based a priori power analysis for a 3 × 2 interaction (group ( time) in a linear mixed model for the primary outcome of SAD symptom severity (according to the LSAS), using the R package “simr” . The analysis is based on a model including a random intercept for participants, a moderate effect size of Cohen’s d = 0.5 for the treatment effect in the APP condition, and a large treatment effect of Cohen’s d = 0.8 for the TG-APP condition compared to the control group (implying a small effect in favor of TG-APP between both treatment conditions).
We chose these effect sizes as conservative estimates based on the results of a recent meta-analysis reporting moderate effect sizes in ICBTs for SAD . Comparable to Stolz et al. , we decided to recruit different sample sizes for the control and treatment conditions, because treatment effects compared to the control group were assumed to be larger than between the two treatment conditions.
We based our power analysis on an iterative simulation of increasing sample sizes with starting values of n = 30 for the control condition and n = 50 for each of the treatment conditions, increasing stepwise by one participant per condition. The variance of the random intercepts and residual variance was set to 0.5 respectively . Because missing values due to the dropout of participants were planned to be imputed by the jump to reference method (J2R) , the reference being the control condition, we reduced our expected effected sizes by multiplying them with the term (1—dropout-rate). Given a dropout rate of 26% at post-treatment evaluation in the study by Stolz et al.  and a pooled post-treatment dropout rate of 26.2% in a meta-analysis of studies using smartphone apps by Torous et al. , we chose a conservative estimate of 30% post-treatment dropout for the APP condition and multiplied the expected effect size of 0.5 by 0.7, equaling to d = 0.35. For the TG-APP condition, we expected a lower post-treatment dropout rate of 20% due to the additional guidance by a therapist and multiplied the expected effect size of 0.8 by 0.8, equaling to d = 0.64. Based on 500 simulations of the linear mixed model fitted with the expected effect sizes, we obtained the power of finding a statistically significant interaction of group × time based on a likelihood ratio test, compared to finding only main effects, and the 95% confidence interval for this estimate, for increasing sample sizes. We located the minimum sample size in our results after which the lower bound of the 95% confidence interval did not fall below 80% power, given α = 0.05. This condition was met at n = 41 for the control group and n = 61 for both treatment conditions (power estimate = 84%, 95% confidence interval = 80.0, 87.0). In addition to our study purposes, the app provider intends to use the results of the pilot study and the main study for the licensing process of the app as a digital health product. For this licensing process, a separate statistical analysis will be conducted comparing only the APP condition with the WLC condition, using a t-test for statistical analysis. A corresponding additional sample size calculation revealed the need to recruit 43 subjects in the wait-list control group (WLC). To establish conformity across the planned analyses we adjusted the number of participants for WLC upward to n = 43, resulting in a total sample size of N =165 (see also Fig. 1).
As a robustness check, we additionally conducted a power analysis based on the simulation of a 3 × 3 interaction (group × time) that included the estimated effect sizes at t3 (follow-up assessment), for which we obtained a highly similar sample size.
The study will be conducted in Germany at the outpatient psychotherapy unit of the Clinical Psychology department of the Goethe University of Frankfurt as well as the outpatient psychotherapy unit of the Clinical Psychology department of the Technische Universitaet Dresden and two further CBT training centers in Dresden. For reporting purposes, we followed the SPIRIT reporting guidelines .
In the first step, all persons that register for participating in the study will be screened using an online self-report assessment. If participants exceed the cut-off of 25-points or find themselves within the criteria for social anxiety disorder, they are referred to another online self-assessment and then invited for an interview with a clinical rater. Beforehand, they will be informed about the study and provided with written informed consent (either personally or via email if necessary). Interviews to determine the patient's formal diagnosis are conducted by blinded and independent psychologists (with at least a master’s degree) who have been trained in the appropriate interviews. Eligible patients are then randomly assigned to one of three groups (WLC, APP, TG-APP). Participation in each of the groups lasts 3 months. Participants in the WLC group then have the option of using the app if they wish. The outcome measures will be assessed by blinded and independent raters at baseline, post-treatment (3 months after start), as well as 6 months after the post-treatment interview (follow-up). Furthermore, participants will complete self-report questionnaires online at baseline, at mid-treatment (at 6 weeks), post-treatment (at 12 weeks), and follow-up (at 36 weeks; see Fig. 1 for the study flowchart). Interviewers will remind patients to fill out the self-report questionnaires and if necessary, patients will be contacted by the study coordinator to complete the missing questionnaires.
Prior to the start of the study, we will conduct a pilot study to improve usability and evaluate the mobile application and the applicability of therapist-guided video therapy and the videoconferencing tool.
The study was approved by the Ethics Committee of the Department of Psychology at Goethe University Frankfurt. If post-trial care is required (e.g., in case of serious adverse effects), this will be provided by the outpatient clinics in Frankfurt and Dresden, and participants will be referred to a hospital or psychiatric clinic if necessary.
Advertisements for participation include print media (e.g., brochures or text in print media), posts on social media, and outreach to mental health professionals, inpatient and outpatient facilities, and general practitioners to identify appropriate participants.
Current diagnosis of Social Anxiety Disorder
Written informed consent before the start of the study
Age: 18 to 65 years
Possession of smartphone
Familiarity with using apps
Active substance abuse or dependence
Severe medical conditions (e.g., chronic cardiovascular disease)
Borderline personality disorder
Current psychotherapeutic treatment
Current psychopharmacological treatment
No proficient skills in the German language
Furthermore, patients may be withdrawn from the study for the following reasons:
Request of patient
Post-hoc observation of criteria for exclusion
Threat to patient’s mental or physical health due to participating in the study
Allocation and randomization
Randomization is performed using a randomization list in which, based on 165 randomly generated numbers sorted by size, each of the available options is assigned according to the planned sample size (WLC = 43, APP = 61, TG-APP = 61). The list is then sorted by sequential number. A study coordinator who is not involved in the diagnosis or treatment process will enter the subjects into the randomization list according to the order of inclusion. The result is communicated to the patients via email and remains hidden from the clinical evaluators. The email to the participant contains the result and, if the person belongs to one of the two intervention groups, all the necessary information to use the app.
The app offers an adaptation of scientifically proven techniques in the treatment of SAD based on Clark and Wells  and is intended for use as a mobile-based intervention. This intervention is designed to reduce the wait time for treatment and help therapists reduce the number of sessions while maintaining or increasing the effectiveness of treatment. The general idea is that using the app will facilitate the treatment process for both the patient and the therapist, as patients can practice the protocolized treatment steps in their environment and therapists can then incorporate these experiences into therapist-led change processes. The entire treatment program will last three months.
Clark and Wells’ model  aims to change the factors that maintain social anxiety. Treatment includes derivation of a model and behavioral experiments to test the effect of safety behaviors and self-focused attention, attention training, video feedback, further behavioral experiments, and cognitive restructuring. Based on this model and its German adaptations by Stangier et al.  and Hoyer and Härtling , the treatment includes the following modules:
Module 1—Learning and Understanding: The goal of this module of the app is to provide the user with key information about SAD and to create an individualized model based on Clark and Wells’ approach. The information used to derive the model is based on patients’ self-reports of anxiety-provoking situations. The model is developed in a stepwise manner, prompted by feedback from the app.
Module 2—Attention Training: The goal of the second module is to change self-focused attention. The training uses recorded audio exercises that train the ability to focus attention on external stimuli.
Module 3—Behavioral experiments: This key module of the app consists of four phases. Each phase includes specific types of behavioral experiments, as well as various forms and questions for planning the experiments and recording the results and key findings.
The first phase provides an experiment on how increasing or decreasing safety behaviors and self-focused attention affects anxiety. The experiment is supplemented by video feedback, in which patients are asked to record themselves in a simulated social situation (e.g., a presentation) and subsequently correct dysfunctional beliefs about their appearance or behavior.
The second phase involves testing the dysfunctional beliefs that emerge in real social situations while simultaneously reducing safety behaviors and directing their attention outward.
The third phase focuses on testing beliefs about the assumed negative impact of supposedly embarrassing behaviors on how they are evaluated by others (e.g., “If I pause during a presentation, I will appear awkward, and others will think I am an idiot”). Patients are instructed to explicitly demonstrate certain critical behaviors (e.g., pausing during a presentation) and to question the validity of their (possibly erroneous) beliefs.
The fourth phase involves integrating behavioral experiments into daily life by continually testing SAD-related beliefs that interfere with personal goals.
In the therapist-guided app condition (TG-APP), use of the app is accompanied by eight therapist-guided video sessions. Based on the manual by Stangier et al. , the therapist assists patients in conducting behavioral experiments and provides a better understanding of the underlying processes. The goals of the sessions are as follows:
First session: establishing a therapeutic alliance, exploring the symptoms and giving an introduction to the app
Second session: answering questions based on the individualized model of the app, deriving a second cognitive model by exploring safety behaviors and establishing motivation for change
Third session: performing a video recorded behavioral experiment to modify safety behaviors and self-focused attention
Fourth session: analyzing the recorded video to test dysfunctional self-evaluation (video feedback)
Fifth session: conducting a therapist-guided behavioral experiment testing a dysfunctional belief
Sixth session: conducting a therapist-guided behavioral experiment deliberately showing critical behavior and testing the evaluation by others
Seventh session: motivating the patient to implement behavioral experiments into the everyday life and debriefing of previous behavioral experiments
Eighth session: relapse prevention (summary of helpful techniques, pursuing personal goals, cognitive interventions for strengthening the self-esteem)
Therapists will be recruited mainly from the outpatient units of the Universities of Dresden and Frankfurt. All therapists must be either licensed psychotherapists or in advanced post-graduate CBT training (at least 1.5 years of training). Study-specific training will focus on how to use the app, how to conduct and analyze behavioral experiments, and how to use techniques of cognitive restructuring. Therapists are required to apply the TG-APP procedures with a pilot patient to gain experience and expertise in this particular form of blended therapy. All treatments will be supervised by psychotherapists with at least five years of clinical experience under their state licensure to ensure adherence to intervention protocols.
Primary outcome measure
We defined the clinician-rated Liebowitz Social Anxiety Scale (LSAS) [21, 22], as the primary outcome measure, a measure widely considered the gold standard in controlled studies on SAD. The LSAS is designed as an interview to assess fear and avoidance in 24 different social situations (rated on a 4-point scale from “not at all” or “never” to “severe” or “most of the time”). The LSAS demonstrated good internal consistency (Cronbach’s α = 0.96), validity and treatment sensitivity .
As the secondary outcome measure for SAD, we use the Social Phobia Inventory (SPIN) . The SPIN is a self-report questionnaire designed to assess the amount of discomfort in different situations during the last week. The total score can range between 0 and 68. Its psychometric properties are satisfactory [32, 33]. The Mini-SPIN , a short version with 3 items, is used as a screening instrument.
We also use the German version of the Social Cognitions Questionnaire (SCQ, German version: SPK) [35, 36], which lists 22 beliefs typically related to social anxiety (e.g., “People will see I’m nervous”) that are rated on two scales: the mean thought frequency, ranging from 1 (“thought never occurs”) to 5 (“thought always occurs”) and a mean belief rating, ranging from 0 (“I do not believe this thought”) to 100 (“I am completely convinced this thought is true”).
Symptoms of depression and general psychological distress will be assessed using the Beck Depression Inventory Fast Screen (BDI-FS)  and the 18-item version of the Brief Symptom Scale (BSI-18) . The BDI-FS is the abbreviated version of the widely used Beck’s Depression Inventory (BDI-II) , which consists of 7 items, including for example “Sadness,” rated on a 4-point-scale. The questionnaire shows good reliability and validity ratings . The BSI-18 is the short version of the original BSI , a questionnaire measuring general psychological distress rating items like “feelings of loneliness” on a scale from 0 (“not at all”) to 4 (“a lot”). The German version of the BSI-18 shows satisfactory internal consistency and validity . Sensitivity to rejection in social situations are assessed with the Social Pain Questionnaire (SPQ-5) . The questionnaire consists of 5 items (e.g., “If I have the feeling that a colleague pulls back from me, I feel rejected”) rating from 0 (“describes me perfectly”) to 4 (“not at all”). Initial psychometric properties show good reliability and validity .
To assess the quality of life, we will use specific domains of the WHO-QOL-BREF  that include the overall quality of life, as well as psychological quality, and quality of social relationships. The WHO-QOL-BREF asks for ratings on varying 5-point scales from 1 (e.g., “not at all” or “very dissatisfied”) to 5 (e.g., “completely” or “very satisfied”) for items like “How satisfied are you with your sleep?”. We excluded the domains of physical health and environment because they are not relevant to our study. The questionnaire shows good to excellent reliability and good validity .
To measure the amount of distress caused by the mental illness, we used the Pain and Disability Index (PDI) . It consists of 7 different categories of life (like “Social Activity”), each of which is rated on a scale from 0 (“no disability”) to 10 (“total disability”). Reliability and validity seem adequate . Additionally, the Anticipatory and Consummatory Interpersonal Pleasure Scale (ACIPS)  will be used to measure the pleasure patients experience in interpersonal situations with 17 items. It demonstrated good reliability and validity .
To assess if individual learning styles influence treatment effects, we included Kolb’s Learning Style Inventory (LSI) , where participants have to match different sentences (e.g., “when I learn…”) with four proposed items (e.g., “… I like to be active”) that are arranged on a scale from 1 (“describes me the worst”) to 4 (“describes me best”), so that each item for each sentence has a unique number from 1 to 4. It demonstrates satisfactory internal reliability and evidence for the suggested internal structure .
Treatment satisfaction will be measured by using the Client Satisfaction Questionnaire (CSQ) , which we adapted slightly to match the use of the app. This 8-item questionnaire uses questions like “How satisfied are you with the amount of help you received?“. It showed satisfying internal consistency and concurrent validity [51, 52]. Social anxiety and social approach behavior will be measured using a modified version of the Social Phobia Weekly Summary Scale (SPWSS) , which will be assessed by participants weekly (for the APP and the TG-APP condition) and at pre, mid, post and follow-up assessments (for all conditions). The SPWSS covers ratings of anxiety, avoidance, self-focused attention, anticipatory worry, post-event rumination, and social approach behavior. We added questions asking for feelings of pleasure and pursuit of personal goals and changed the scale to a 0–100 scale. The SPWSS is sensitive to treatment effects and has good internal consistency .
Finally, we will also assess negative side effects which are specific to patients with SAD. Drawing on the results of Boettcher et al. , we developed a brief (7-item) scale with items covering the negative side effects that were considered most relevant by a group of experts (see Additional file 1). The items are rated on a scale from 0 (“do not agree at all”) to 5 (“fully agree”). Please note that the descriptive list of possible side effects is preliminary and may be slightly revised after the pilot study. As the list contains an open item, responses to this item will be evaluated after the pilot study and inclusion of further side effects in the list is possible should they be mentioned frequently.
To screen for borderline personality disorder (BPD), we use a set of 5 questions suggested by Wongpakaran et al.  that screen for symptoms of BPD based on the DSM-V and a Rasch analysis. It showed good quality for assessing BPD . If screening indicates that BPD may be present, we use the appropriate category of the SCID-5 to check for the other criteria of BPD.
We further use the Clinical Global Impression (CGI)  modified for patients with SAD. The CGI-Severity scale provides information about the current severity of social phobic symptoms which are rated on a 7-step scale from “normal” or “not ill at all” to “among the most severely ill patients” by a clinician. The CGI-Improvement scale is a 7-point scale rating the change in symptom severity from ‘improved by a lot’ to “a lot worse.” The use of the CGI, as a measure for symptom-specific improvement for patients with SAD, is supported by adequate psychometric properties  and its practicality.
Finally, the Quick Inventory of Depressive Symptomatology (QIDS-C)  will be applied, a 16-item rating instrument for the assessment of depressive symptoms by an independent interviewer, which showed acceptable internal consistency and treatment sensitivity . Figure 2 gives an overview of the measures and assessments used in this study.
All analyses will be conducted in the intent-to-treat (ITT) sample, including all participants that were randomized into one of the three groups.
For our primary analysis, we plan to fit a linear mixed model with random intercepts for participants in the R package lme4 . We take the z-standardized values of the Liebowitz Social Anxiety Scale (LSAS) as the outcome variable, and time and treatment condition as categorical predictors, for which we specify an interaction term. The estimated effects of the predictor variables indicate the change in LSAS from baseline at week 0, which we choose as the reference category, to post-treatment.
As secondary analyses, we will evaluate treatment effects (from baseline to post-treatment, as described in hypothesis 2a and from post-treatment to follow-up, as described in hypothesis 2c) on quality of life, as measured by the WHO-QOL-BREF, interpersonal pleasure, as measured by the ACIPS, further SAD-related symptoms, as measured by the SPIN and the CGI, social phobic cognitions, as measured by the SPK, symptoms of depression, as measured by the BDI-FS and the QIDS-C, general psychological distress, as measured by the BSI, and limitations in everyday life, as measured by the PDI. We will also assess adverse effects measured by the self-developed adverse effect scale. Moreover, we will compare posttreatment LSAS scores with scores at follow-up, as stated in hypothesis 1c, using the linear mixed model described earlier.
To control for moderating effects on decreases in social anxiety symptoms, as indicated in hypothesis 3, we include the scores of the individual learning style, as measured by Kolb’s LSI and experienced pain in social situations, as measured by the SPQ-5, as predictors in the model and specify three-way interactions with treatment conditions and time for all these factors. We will conduct a similar analysis to test whether social approach behavior measured by the SPWSS has a mediating effect on social anxiety symptoms, as we hypothesized in hypothesis 4. We will test all interactions by comparing the full to a reduced model without the respective interaction term via likelihood ratio tests (LRTs)  and remove all non-significant interactions from the final model.
To evaluate effects between two conditions at a given time, e.g., between the APP and TG-APP conditions at post-treatment, as indicated in hypothesis 1b and 2b, we will calculate the statistical significance of pairwise differences in LSAS based on the linear mixed model using the R package “emmeans” .
We will fit all measures as outcome variables in linear mixed models including the same predictors as in the primary analysis and add the average weekly time participants spent using the app and the number of behavioral experiments conducted as additional explanatory predictors.
Besides that, we will analyze and compare responder rates between the different treatment conditions at post-treatment using a chi-squared test. Response to treatment will be defined as a 29% score decrease of the clinician-rated LSAS from baseline to post-treatment according to Glischinski et al. .
We define as “drop-out” any participant who did not complete the post-treatment measurement. Systematic differences in dropout rates between the three groups will be analyzed by a Fisher exact test. In addition, differences between participants and drop-outs in terms of sociodemographic and clinical variables will be considered.
In case of a rejection of the licensing process by the inspecting federal institute for drugs and medical devices, an interim analysis of the treatment effect of the APP condition on outcomes of the main study (LSAS, WHO-QOL-BREF, PDI) could turn out to be necessary. Such interim analysis will be blinded, that is, performed by different researchers than the respective research teams of the study and after reaching a sample size of n = 30 for the APP and for the WLC group at post-treatment (based on a one-tailed power analysis simulation in R for a mean difference between two independent groups with unequal variance and effect size of d = 0.35, a standard deviation of outcome variables of 0.7 for both groups and 80% power). To account for multiple hypothesis testing, the alpha error level will be adjusted according to Pocock . The interim analysis will not affect the continuation of the study.
Based on a recent meta-analysis, there is evidence that internet- or app-based interventions have moderate effect sizes in the treatment of SAD . There is also preliminary evidence from previous studies that internet-based cognitive behavioral therapies (ICBTs), based on the model of Clark and Wells , may achieve large effect sizes [15, 16, 66]. These findings are consistent with results demonstrating the superiority of CBT based on Clark and Wells’ model in regular face-to-face therapy . In our study, we plan to extend these findings by conducting an RCT comparing the efficacy of a mobile app based on Clark and Wells’ model with the efficacy of the app plus additional therapist-guided sessions delivered via video and a wait-list control group.
We hypothesize that the addition of therapist-guided sessions will result in larger effect sizes than using the app alone. In agreement with Stott et al. , we assume that contact with a therapist has a critical impact on engagement in specific CT interventions, particularly behavioral experiments. Similar to exposure-based treatments, the presence of a therapist assists the patient in overcoming avoidance, shifting attention outward, and identifying and discontinuing safety behaviors and ensures that feared negative evaluation is challenged. Therefore, therapist-guided sessions can prevent the underuse of behavioral experimentation, an important effective component of CT according to Clark and Wells’ model.
To summarize, our results will demonstrate (a) the efficacy of a new app-based intervention using the Clark and Wells approach in another well-designed and preregistered independent trial and (b) whether additional face-to-face therapy is beneficial to patients in terms of increased symptom reduction of SAD compared to app use alone. The results will be critical for service delivery and treatment planning for social anxiety disorder.
Coordinating center and trial steering committee
The present study is a bicentric study which is designed and coordinated in a cooperation between the Goethe University Frankfurt and the Technische Universitaet Dresden. The study team has weekly meetings. There is no additional steering committee.
Support for the trial is provided by:
Principal investigator: provides supervision for the treatment, has medical responsibility of the patients, designs and plans the study
Study coordinator: coordinates study procedures, responsible for administrative tasks, reviews study progress
Study therapists: responsible for the implementation of the therapies
Clinical raters: responsible for conducting independent clinical evaluations with respect to outcome criteria and inclusion and exclusion criteria
Data monitoring committee
There is no data monitoring committee and no independent auditing process. The quality of the data is assured by the blinded clinical raters and there are no known serious adverse effects. If serious adverse effects occur, they will be reported to the principal investigator, who will then interview the patient in question and refer them to a psychiatry if needed. Adherence to therapeutic measures is ensured by regular supervision by the principal investigators. Completeness of data will be regularly verified by the study coordinators.
Data management and confidentiality
Questionnaires will be answered online. The screening is the only time participants enter personal contact information. In the subsequent assessments and questionnaires, participants are identified by a random computer-generated code. Other than the study coordinators, only therapists and clinical raters know the participant’s name. The anonymized data is stored in an encrypted folder on a protected research server at Goethe University Frankfurt. There is also a separate encrypted list that contains only contact information and the code so that participants can be contacted if needed. This list is deleted after the study, so that only the codes with the assigned data remain. The program R is used to analyze the data. The results of the study will be published only in peer-reviewed journals. The informed consent form will be kept in a locker at the outpatient clinics of the universities of Frankfurt and Dresden.
Relevant concomitant care and interventions that are permitted or prohibited during the trial
There is no concomitant care additional to the intervention groups that we already described. If there are medical conditions of the patients, that need to be treated right away, patients will be send to a physician. If patients start a different psychotherapy while being part of the study, they will drop out.
Recruitment began in July 2022. The current protocol is version 2 of 16 December 2022. There have been no patients enrolled into the trial so far (16 September 2022). Approximate date of completed recruitment: 31 December 2023.
Availability of data and materials
The data used in this study, including for example, the R code for randomization, are available from the corresponding authors upon reasonable request.
Liebowitz Social Anxiety Scale
Social Phobia Inventory
Fragebogen zu sozialphobischen Kognitionen
Pain and Disability Index
Beck Depression Inventory - Fast Screen
World Health Organization Quality of Life
Brief Symptom Inventory - 18-item version
Anticipatory and Consummatory Interpersonal Pleasure Scale
Social Phobia Weekly Summary Scale
Clinical Global Impression
Quick Inventory of Depressive Symptoms
Structured Clinical Interview for DSM-5
- Kolb’s LSI:
Kolb’s Learning Style Inventory
Social Pain Questionnaire
Client Satisfaction Questionnaire
American Psychiatric Association. Diagnostic and statistical manual of mental disorders: DSM-5. 5th ed. Arlington: American Psychiatric Publishing, Inc.; 2013.
Kessler RC, Petukhova M, Sampson NA, Zaslavsky AM, Wittchen HU. Twelve-month and lifetime prevalence and lifetime morbid risk of anxiety and mood disorders in the United States. Int J Methods Psychiatr Res. 2012;21(3):169–84.
Fehm L, Pelissolo A, Furmark T, Wittchen HU. Size and burden of social phobia in Europe. Eur Neuropsychopharmacol. 2005;15(4):453–62.
Clark DM, Wells A. A cognitive model of social phobia. In: Heimberg RG, Liebowitz MR, Hope DA, Schneier FR, editors. Social phobia: diagnosis, assessment, and treatment. New York: The Guilford Press; 1995. p. 69–93.
Mayo-Wilson E, Dias S, Mavranezouli I, Kew K, Clark DM, Ades AE, et al. Psychological and pharmacological interventions for social anxiety disorder in adults: a systematic review and network meta-analysis. Lancet Psychiatry. 2014;1(5):368–76.
Hähnel A, Weigel B, Hoyer J, Helbig S. Wartezeit für Psychotherapiepatienten – und wie sie zu nutzen ist. Verhaltenstherapie. 2004;14(4):294–302.
Olfson M, Guardino M, Struening E, Schneier FR, Hellman F, Klein DF. Barriers to the treatment of social anxiety. Am J Psychiatry. 2000;157(4):521–7.
Wang PS, Berglund P, Olfson M, Pincus HA, Wells KB, Kessler RC. Failure and delay in initial treatment contact after first onset of mental disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6):603–13.
Lee BW, Stapinski LA. Seeking safety on the internet: relationship between social anxiety and problematic internet use. J Anxiety Disord. 2012;26(1):197–205.
Berger T, Urech A, Krieger T, Stolz T, Schulz A, Vincent A, et al. Effects of a transdiagnostic unguided Internet intervention (‘velibra’) for anxiety disorders in primary care: results of a randomized controlled trial. Psychol Med. 2017;47(1):67–80.
Hedman E, Andersson E, Ljótsson B, Andersson G, Rück C, Lindefors N. Cost-effectiveness of Internet-based cognitive behavior therapy vs. cognitive behavioral group therapy for social anxiety disorder: results from a randomized controlled trial. Behav Res Ther. 2011;49(11):729–36.
Guo S, Deng W, Wang H, Liu J, Liu X, Yang X, et al. The efficacy of internet-based cognitive behavioural therapy for social anxiety disorder: a systematic review and meta-analysis. Clin Psychol Psychother. 2021;28(3):656–68.
Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlöf E. Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cogn Behav Ther. 2018;47(1):1–18.
Andersson G, Rozental A, Shafran R, Carlbring P. Long-term effects of internet-supported cognitive behaviour therapy. Expert Rev Neurother. 2018;18(1):21–8.
Stott R, Wild J, Grey N, Liness S, Warnock-Parkes E, Commins S, et al. Internet-delivered cognitive therapy for social anxiety disorder: a development pilot series. Behav Cogn Psychother. 2013;41(4):383–97.
Stolz T, Schulz A, Krieger T, Vincent A, Urech A, Moser C, et al. A mobile app for social anxiety disorder: a three-arm randomized controlled trial comparing mobile and PC-based guided self-help interventions. J Consult Clin Psychol. 2018;86(6):493–504.
Reid SC, Kauer SD, Hearps SJ, Crooke AH, Khor AS, Sanci LA, et al. A mobile phone application for the assessment and management of youth mental health problems in primary care: a randomised controlled trial. BMC Fam Pract. 2011;12(1):131.
Heron KE, Smyth JM. Ecological momentary interventions: Incorporating mobile technology into psychosocial and health behaviour treatments. Br J Health Psychol. 2010;15(1):1–39.
Boettcher J, Rozental A, Andersson G, Carlbring P. Side effects in Internet-based interventions for Social Anxiety Disorder. Internet Interv. 2014;1(1):3–11.
Fernandez E, Woldgabreal Y, Day A, Pham T, Gleich B, Aboujaoude E. Live psychotherapy by video versus in-person: a meta-analysis of efficacy and its relationship to types and targets of treatment. Clin Psychol Psychother. 2021;28(6):1535–49.
Liebowitz MR. Social phobia. Mod Probl Pharmacopsychiatry. 1987;22:141–73.
Stangier U, Heidenreich T. Liebowitz Social Anxiety Scale. In: Collegium Internationale Psychiatriae Scalarum, editor. Internationale Skalen für Psychiatrie. 5th ed. Weinheim: Beltz; 2005.
Green P, MacLeod CJ. SIMR: an R package for power analysis of generalized linear mixed models by simulation. Methods Ecol Evol. 2016;7(4):493–8.
Schielzeth H, Dingemanse NJ, Nakagawa S, Westneat DF, Allegue H, Teplitsky C, et al. Robustness of linear mixed-effects models to violations of distributional assumptions. Methods Ecol Evol. 2020;11(9):1141–52.
Cro S, Morris TP, Kenward MG, Carpenter JR. Reference-based sensitivity analysis via multiple imputation for longitudinal trials with protocol deviation. Stata J. 2016;16(2):443–63.
Torous J, Lipschitz J, Ng M, Firth J. Dropout rates in clinical trials of smartphone apps for depressive symptoms: a systematic review and meta-analysis. J Affect Disord. 2020;263:413–9.
Chan AW, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin JA, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;346:e7586.
Stangier U, Clark DM, Ginzburg DM, Ehlers A. Soziale Angststörung. 2. überarbeitete und erweiterte Auflage. Fortschritte der Psychotherapie, Band 28. Göttingen: Hogrefe Verlag; 2016.
Hoyer J, Härtling S. Soziale Angst verstehen und verändern. 2nd ed. Berlin: Springer; 2019.
Heimberg RG, Horner KJ, Juster HR, Safren SA, Brown EJ, Schneier FR, et al. Psychometric properties of the Liebowitz Social Anxiety Scale. Psychol Med. 1999;29(1):199–212.
Sosic Z, Gieler U, Stangier U. Screening for social phobia in medical in- and outpatients with the German version of the Social Phobia Inventory (SPIN). J Anxiety Disord. 2008;22(5):849–59.
Antony MM, Coons MJ, McCabe RE, Ashbaugh A, Swinson RP. Psychometric properties of the social phobia inventory: further evaluation. Behav Res Ther. 2006;44(8):1177–85.
Connor KM, Davidson JRT, Churchill LE, Sherwood A, Weisler RH, Foa E. Psychometric properties of the Social Phobia Inventory (SPIN): New self-rating scale. Br J Psychiatry. 2000;176(4):379–86.
Connor KM, Kobak KA, Churchill LE, Katzelnick D, Davidson JRT. Mini-SPIN: a brief screening assessment for generalized social anxiety disorder. Depress Anxiety. 2001;14(2):137–40.
Stangier U, Liefke S, Heidenreich T. Fragebogen zu sozialphobischen Kognitionen (SPK). In Hoyer J, Margraf J, editors. Angstdiagnostik. Berlin: Springer; 2002.
McManus F, Sacadura C, Clark DM. Social Cognitions Questionnaire--Modified Version (SCQ), APA PsycTests. 2008. https://doi.org/10.1037/t14514-000.
Beck AT, Steer RA, Brown GK. Beck Depressions-Inventar-FS: Deutsche Bearbeitung. Frankfurt: Pearson; 2013.
Spitzer C, Hammer S, Löwe B, Grabe HJ, Barnow S, Rose M, et al. Die Kurzform des Brief Symptom Inventory (BSI-18): Erste Befunde zu den psychometrischen Kennwerten der deutschen Version. [The short version of the Brief Symptom Inventory (BSI-18): preliminary psychometric properties of the German translation.]. Fortschritte Neurol Psychiatr. 2011;79(9):517–23.
Beck AT, Steer RA, Brown GK. Manual for the Beck Depression Inventory–II. San Antonio: Psychological Corp; 1996.
Kliem S, Mößle T, Zenger M, Brähler E. Reliability and validity of the beck depression inventory-fast screen for medical patients in the general German population. J Affect Disord. 2014;156:236–9.
Derogatis LR, Spencer PM. The Brief Symptom Inventory: administration, scoring, and procedures manual-I. Baltimore: Clinical Psychometric Research; 1982.
Stangier U, Schüller J, Brähler E. Development and validation of a new instrument to measure social pain. Sci Rep. 2021;11(1):8283.
THE WHOQOL GROUP. Development of the World Health Organization WHOQOL-BREF Quality of Life Assessment. Psychol Med. 1998;28(3):551–8.
Skevington SM, Lotfy M, O’Connell KA. The World Health Organization’s WHOQOL-BREF quality of life assessment: psychometric properties and results of the international field trial. A Report from the WHOQOL Group. Qual Life Res. 2004;13(2):299–310.
Pollard CA. Preliminary validity study of the Pain Disability Index. Percept Mot Skills. 1984;59(3):974.
Tait RC, Chibnall JT, Krause S. The Pain Disability Index: psychometric properties. Pain. 1990;40(2):171–82.
Gooding DC, Pflum MJ. The assessment of interpersonal pleasure: introduction of the Anticipatory and Consummatory Interpersonal Pleasure Scale (ACIPS) and preliminary findings. Psychiatry Res. 2014;215(1):237–43.
Kolb AY, Kolb DA. The Kolb Learning Style Inventory—Version 3.1 2005 technical specifications. Boston: HayGroup; 2005.
Kayes DC. Internal validity and reliability of Kolb’s Learning Style Inventory Version 3 (1999). J Bus Psychol. 2005;20(2):249–57.
Larsen DL, Attkisson CC, Hargreaves WA, Nguyen TD. Assessment of client/patient satisfaction: development of a general scale. Eval Program Plann. 1979;2(3):197–207.
Attkisson CC, Zwick R. The client satisfaction questionnaire: psychometric properties and correlations with service utilization and psychotherapy outcome. Eval Program Plann. 1982;5(3):233–7.
Kelly PJ, Kyngdon F, Ingram I, Deane FP, Baker AL, Osborne BA. The Client Satisfaction Questionnaire-8: psychometric properties in a cross-sectional survey of people attending residential substance abuse treatment. Drug Alcohol Rev. 2018;37(1):79–86.
Clark DM, Ehlers A, McManus F, Hackmann A, Fennell M, Campbell H, et al. Cognitive therapy versus fluoxetine in generalized social phobia: a randomized placebo-controlled trial. J Consult Clin Psychol. 2003;71(6):1058–67.
First MB, Williams JBW, Karg RS, Spitzer RL. User’s guide for the SCID-5-CV: structured clinical interview for DSM-5 disorders: clinical version. Arlington: American Psychiatric Publishing, Inc.; 2016.
Osório FL, Loureiro SR, Hallak JEC, Machado-de-Sousa JP, Ushirohira JM, Baes CVW, et al. Clinical validity and intrarater and test–retest reliability of the Structured Clinical Interview for DSM-5 – Clinician Version (SCID-5-CV). Psychiatry Clin Neurosci. 2019;73(12):754–60.
Wongpakaran N, Wongpakaran T, Kuntawong P. A short screening tool for borderline personality disorder (Short-Bord): validated by Rasch analysis. Asian J Psychiatry. 2019;44:195–9.
Guy W. ECDEU assessment manual for psychopharmacology. Rockville: US Department of Health, Education, and Welfare, Public Health Service, Alcohol, Drug Abuse, and Mental Health Administration; 1976.
Zaider TI, Heimberg RG, Fresco DM, Schneier FR, Liebowitz MR. Evaluation of the Clinical Global Impression Scale among individuals with social anxiety disorder. Psychol Med. 2003;33(4):611–22.
Rush AJ, Trivedi MH, Ibrahim HM, Carmody TJ, Arnow B, Klein DN, et al. The 16-Item quick inventory of depressive symptomatology (QIDS), clinician rating (QIDS-C), and self-report (QIDS-SR): a psychometric evaluation in patients with chronic major depression. Biol Psychiatry. 2003;54(5):573–83.
Trivedi MH, Rush AJ, Ibrahim HM, Carmody TJ, Biggs MM, Suppes T, et al. The Inventory of Depressive Symptomatology, Clinician Rating (IDS-C) and Self-Report (IDS-SR), and the Quick Inventory of Depressive Symptomatology, Clinician Rating (QIDS-C) and Self-Report (QIDS-SR) in public sector patients with mood disorders: a psychometric evaluation. Psychol Med. 2004;34(1):73–82.
Bates D, Mächler M, Bolker B, Walker S. Fitting linear mixed-effects models using lme4. J Stat Softw. 2015;67(1):1–48.
Dobson S, Ramlogan C. Economic growth and convergence in Latin America. J Dev Stud. 2002;38(6):83–104.
Lenth R, Buerkner P, Herve M, Love J, Miguez F, Riebl H, Singmann H. Emmeans: estimated marginal means, aka least-squares means, R package version 1.7.2. 2022. https://CRAN.R-project.org/package=emmeans. Accessed 4 Mar 2022.
von Glischinski M, Willutzki U, Stangier U, Hiller W, Hoyer J, Leibing E, et al. Liebowitz Social Anxiety Scale (LSAS): optimal cut points for remission and response in a German sample. Clin Psychol Psychother. 2018;25(3):465–73.
Pocock SJ. Group sequential methods in the design and analysis of clinical trials. Biometrika. 1977;64(2):191–9.
McCall HC, Richardson CG, Helgadottir FD, Chen FS. Evaluating a web-based social anxiety intervention among university students: randomized controlled trial. J Med Internet Res. 2018;20(3):e91.
Open Access funding enabled and organized by Projekt DEAL. This study is funded by Mindable Health GmbH, Berlin, Germany. The funders do not have a role in collection, analysis, and interpretation of data.
Ethics approval and consent to participate
The study has been approved by the ethics committee of the psychology department of the Goethe University Frankfurt. All participants provide written informed consent. All substantial amendments will be reported to the responsible ethics committee, e.g., if there are changes that need to be made because it might affect the mental or physical health of participants.
Consent for publication
Not applicable—no identifying images or other personal or clinical details of participants are presented here or will be presented in reports of the trial results. Informed consent materials are available from the corresponding author on request.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Schittenhelm, J.M., von Borell, C., Clément, C. et al. Evaluation of a smartphone application for self-help for patients with social anxiety disorder: a randomized controlled study—SMASH. Trials 24, 154 (2023). https://doi.org/10.1186/s13063-023-07168-5