An Internet-based emotion regulation intervention versus no intervention for non-suicidal self-injury in adolescents: a statistical analysis plan for a feasibility randomised clinical trial
Trials volume 22, Article number: 456 (2021)
Non-suicidal self-injury (NSSI) has a lifetime prevalence of 17% in adolescents in the general population and up to 74% in adolescents with psychiatric disorders. NSSI is one of the most important predictors of later suicidal behaviour and death by suicide. The TEENS feasibility trial was initiated to assess the feasibility and safety of Internet-based Emotion Regulation Individual Therapy for Adolescents (ERITA) as an add-on to treatment as usual in 13–17-year-old patients with NSSI referred to the Child and Adolescent Mental Health Services.
The TEENS feasibility trial is a randomised clinical trial with a parallel-group design. The trial intervention is an 11-week online therapy which is tested as an add-on to treatment as usual versus treatment as usual. The primary feasibility outcomes are the fraction of participants who (1) completed 12 weeks of follow-up interview or assessment, (2) consented to inclusion and randomisation out of all eligible participants, and (3) were compliant with the experimental intervention, assessed as completion of at least six out of eleven modules in the programme. Since this is a feasibility trial, we did not predefine a required sample size. The exploratory clinical outcome, the frequency of NSSI episodes, assessed using Deliberate Self-Harm Inventory – Youth version (DSHI-Y), at the end of intervention, is planned to be the future primary outcome in a larger pragmatic definitive randomised clinical trial. After completion of the feasibility trial, blinded data will be analysed by two independent statisticians blinded to the intervention, where ‘A’ and ‘B’ refer to the two groups. A third party will compare these reports, and discrepancies will be discussed. The statistical report with the analyses chosen for the manuscript is being tracked using a version control system, and both statistical reports will be published as a supplementary material. Based on the final statistical report, two blinded conclusions will be drawn by the steering group.
We present a pre-defined statistical analysis plan for the TEENS feasibility trial, which limits bias, p-hacking, data-driven interpretations. This statistical analysis plan is accompanied by a pre-programmed version-controlled statistical report with simulated data, which increases transparency and reproducibility.
Non-suicidal self-injury (NSSI) has a lifetime prevalence of 17% in adolescents in the general population and up to 74% in adolescents with psychiatric disorders [1, 2]. NSSI is one of the most important predictors of later suicidal behaviour and death by suicide [2,3,4,5]. The treatment as usual for NSSI includes a variety of clinical treatments and assessments offered by the Child and Adolescent Mental Health Services (CAMHS). Treatment of NSSI is heterogeneous, with different interventions such as pharmacological, family-based, and cognitive-behavioural therapy (CBT); supportive counselling; and psychoeducation depending on the patients’ primary mental health problem [6,7,8].
No treatment has been found superior for NSSI in adolescents . The stigmatisation of NSSI may lower the tendency to seek help and adhere to treatment, while Internet-based interventions are assumed to be more easily accepted [10,11,12,13]. Internet-based interventions for adolescents guided by a therapist are effective for several psychiatric disorders . Only one previous feasibility study has investigated the potential of an Internet-based intervention (Emotion Regulation Individual Therapy for Adolescents (ERITA)) for NSSI in youth; this study found promising trends towards reduced NSSI frequency at the end of treatment provision; however, the results should be interpreted with caution based on single-arm design [10, 15, 16]. There is a need for randomised clinical trials assessing the effect of specialised Internet-based interventions for NSSI .
The TEENS feasibility trial was initiated to assess the feasibility and safety of Internet-based ERITA as an add-on to treatment as usual in 13–17-year-old patients with NSSI referred to the Child and Adolescent Mental Health Services . This paper describes the plan for statistical analyses of the feasibility and exploratory clinical outcomes in the TEENS feasibility trial .
The TEENS feasibility trial is a randomised feasibility trial with a parallel-group design . The trial methodology in general has been described previously . Patients are recruited from the Child and Adolescent Mental Health Services, the Capital Region, Denmark. The experimental intervention is therapist-guided Internet-based ERITA as an add-on to treatment as usual [10, 11, 18]. A detailed description of the Internet-based ERITA intervention can be found elsewhere . Treatment as usual is provided by multidisciplinary teams in nine outpatient clinics within the Child and Adolescent Mental Health Services in the Capital Region of Denmark. The trial was registered on ClinicalTrials.gov (identification no. NCT04243603) before the inclusion of the first participant. Consent for inclusion and randomisation is carried out only if a patient fulfils all inclusion criteria and none of the exclusion criteria. The study design was based on the CONSORT extension for randomised pilot and feasibility trials , and our statistical analysis plan is based on the recommendation from Gamble et al. .
Age 13 to 17 years, both inclusive
≥ 5 NSSI episodes during the past year and ≥ 1 NSSI episodes during the past month assessed by the Deliberate Self-Harm Inventory, Youth version (DSHI-Y) 
Age-appropriate Danish literacy assessed by referring clinicians and the self-injury team
At least one parent committing to participate in the parent programme
Informed consent from parents or legal caretakers
Informed consent from the participant above 15 years of age
Elevated or imminent suicidal risk assessed by clinicians during routine screening (that can be rated as no risk, elevated risk, or imminent risk). In the latter two cases, the patient needs close supervision and possibly hospitalisation.
Randomisation and blinding
Randomisation is, after informed consent, performed by a member of the self-injury team using the central web-based randomisation system managed by the Copenhagen Trial Unit (CTU, Copenhagen, Denmark). As this is a feasibility trial, we will not stratify the randomisation. Due to the nature of the intervention, blinding of participants and clinicians is not possible. The researchers are, however, blinded to the allocation. During the final phone and video interviews, the participants are instructed not to disclose the allocation outcome.
The trial intervention is an add-on to treatment as usual and consists of 11-week, manualised online therapy based on the methods of CBT, dialectical behaviour therapy, and acceptance and commitment therapy, the so-called ERITA [10, 11]. The programme consists of 11 modules following an initial introduction. The intervention also provides six modules for parents’ involvement focusing on the information about NSSI and other destructive behaviours, emotional awareness, effective communication skills (e.g. validation), and strategies to cope with their child’s negative emotions in an appropriate way. ERITA is provided online, meaning that the adolescents and the parents have online contact with an assigned clinical therapist during the 11-week intervention period. The adolescents are expected to complete a new module every week, i.e. eleven modules, while the parents must complete a module every second week, i.e. six modules. In addition, they are encouraged to review the youth modules every week. Through the course of the modules, the therapist will review the participant’s responses and provide written feedback through the Internet platform.
The primary feasibility outcomes are the fraction of all participants who (1) completed 12 weeks of follow-up interview or assessment, (2) consented and were randomised of all eligible patients, and (3) were compliant with the experimental intervention, assessed as completion of at least six out of eleven modules in the programme and their parents completed at least 3 modules out of 6. Furthermore, several exploratory clinical outcomes are assessed (Table 1).
Sample size and power justification
Since this is a feasibility trial, we have not predefined a required sample size. The exploratory clinical outcome, the frequency of self-injury episodes, assessed using DSHI-Y , at the end of the intervention, is planned to be the primary future outcome in a larger pragmatic clinical trial. We plan to take the results of the exploratory clinical outcomes into consideration when estimating the required sample size and power estimation of both the primary and non-primary outcomes for the planned pragmatic randomised clinical trial. We have pragmatically chosen to include 15 participants in each group to provide acceptable robustness for the sample size calculation for a future larger pragmatic trial [19, 28] and to assess the proportion of missing data we could expect. Based on a reference of 13% missingness, this trial with the inclusion of 30 participants would be able to, by using a proportion power calculation for the binomial distribution with a power of 80% and an alpha of 0.05, dismiss an expected missingness of more than 32% in a future larger pragmatic trial.
General analysis principles
Statistical analyses will be handled using R version 4.0.3 (R Core Team, Vienna, Austria) and Stata (StataCop LLC, TX, USA). All randomised participants will be included in all analyses. The baseline characteristics will be presented for each group (Table 2). For nationality, Danish citizens will be presented in their own group, and participants from other countries will be presented based on cultural properties, e.g. other European/North American or Middle Eastern. The threshold for significance in all analyses will be below 0.05 and will not be corrected for multiple comparisons, as the clinical outcomes are all exploratory and will be interpreted as such.
Analysis of primary feasibility outcomes
The feasibility outcomes were decided based on consensus and agreement between the investigators and were based on clinical expertise (Morthorst B, Pagsberg AK, Møhl B, and Rubæk L) and trial experience from previous pragmatic and feasibility trials (Lindschou J, Gluud C, and Jakobsen JC). These are all seen as relevant for carrying out a definitive large-scale trial. The primary feasibility outcomes are all based on the fraction of participants who fulfil our predefined criteria (see the ‘Inclusion criteria’ section). The fractions will be presented together with the confidence intervals using a 1-sample proportions test with continuity correction, with an adjusted maximum confidence limit of 100% (Fig. 1). The trial will be deemed feasible if (1) ≥ 87% (95% confidence intervals (95% CI) 68 to 96%) of participants completed at least one clinical outcome assessment (NSSI events) at the end of the intervention, (2) ≥ 40% (95% CI 29 to 52%) of all eligible patients were randomised, and (3) ≥ 73% (95% CI 45 to 91%) of the participants completed at least 6 modules out of 11 and their parents completed at least 3 modules out of 6. We have, after publishing the protocol , changed the statistical analyses for calculating the confidence intervals for proportions. The estimate of the lower confidence interval, using 1-sample proportions test with continuity correction, will serve as the least feasible indicator to conduct a large-scale trial. For instance, a lower confidence limit of 68% from the analysis will be interpreted as a feasible estimate for the first outcome.
Analysis of exploratory clinical outcomes
We plan to analyse the exploratory clinical outcomes as we plan to analyse these outcomes in the planned larger pragmatic trial, by choosing the analyses which fulfils the assumptions. The results will be interpreted with caution as this trial is not powered to investigate clinical outcomes, but the signals will help inform which outcomes we might choose for the larger pragmatic trial.
Continuous exploratory clinical outcomes will be presented as means and standard deviations (SD) for each group, with an annotation in the tables of the percentage of missing data per group (Fig. 1A; Table 3). As previously recommended, we will use linear regression analyses adjusted for the baseline value for the continuous exploratory clinical outcomes .
Count data outcomes
Count data exploratory clinical outcomes will be presented as medians and interquartile ranges for each group, with an annotation in the tables of the percentage of missing data per group (Fig. 2B). Count data exploratory clinical outcomes will be analysed using the Mann-Whitney U test. Hodges-Lehmann confidence intervals will be presented to demonstrate the uncertainty of the results .
Dichotomous exploratory clinical outcomes will be presented as proportions for each group with an annotation in the tables of the percentage of missing data per group. Dichotomous exploratory clinical outcomes will be analysed using logistic regression. We will estimate the marginal effects to obtain RRs and confidence intervals of the RRs (based on ‘nlcom’ from Stata (StataCorp LLC, TX, USA)).
Handling of missing data
No specific methodology, including multiple imputations, will be used to handle missing data, but missingness will be listed in detail in the tables in the statistical reports (see below) as a tool to adapt the design of a larger pragmatic randomised trial.
Assessments of underlying statistical assumptions
The chosen analyses have few assumptions, with the main assumptions being related to the linear and logistic regressions [29, 31]. The variables included in the linear regression models will be visually assessed for normal distribution using histograms and quantile-quantile plots of the residuals and for homogeneity using residuals plotted against covariates and fitted values, with the possibility of a logarithmic transformation or applying robust standard errors to minimise deviations from the model .
The deviance divided by the degrees of freedom for logistic regression model will be calculated to assess relevant overdispersion. The logistic regression used will be univariable, i.e. with no covariates, and if few or zero events are identified (substantially lower than the rule of thumb of 10 events), the analyses will be carried out using Fisher’s exact test. The robustness of the confidence intervals and p-values might be affected by the small sample size, and these will be interpreted with caution .
A pre-programmed statistical report based on simulated data is publicly available on Zenodo (https://zenodo.org/record/4643529; European Organization for Nuclear Research, Genevé, Switzerland) and submitted as a supplemental material. After completion of the trial, blinded data will be analysed by two independent statisticians blinded to the intervention, where ‘A’ and ‘B’ refer to the two groups. The two statisticians will independently analyse all data and present the results in two independent reports. The two independent reports will be based on the agreed-upon pre-programmed statistical report. The coordinating investigator, the two statisticians and the Steering Committee will compare these reports and discrepancies will be discussed. The statistical report with consensus on the definitive analyses in the manuscript is being tracked using a version control system (https://github.com/lilleoel/CTU_TEENS, GitHub, San Francisco, CA, USA), and both statistical reports will, furthermore, be published as a supplementary material. Based on the final statistical report, two blinded conclusions will be drawn by the Steering Committee: one assuming ‘A’ is the experimental group and ‘B’ is the control group and one assuming the opposite. These abstracts will utilise the results from the blinded reports, and when the blinding is broken, the ‘correct’ abstract will be chosen and the conclusions in this abstract will not be revised. This described the process of analysing data, and interpreting data will also be used in the future large randomised clinical trial.
We present a detailed predefined description of the statistical analysis of the TEENS feasibility trial. The primary aim of this statistical analysis plan is to limit bias, p-hacking, and data-driven interpretations.
The primary strengths are the predefined statistical analysis plan and publication of a version-controlled pre-programmed statistical report before any data were available. This secures methodological transparency and enables the reproducibility of our results. Completion of a feasibility trial with three independent feasibility outcomes and multiple exploratory clinical outcomes will contribute with important data for the future randomised clinical trial we have planned.
Since no correction for multiplicity will be applied to the exploratory outcomes, any significance must be interpreted with caution. We assess multiple outcomes which increase the risk of false-positive results (type I error); any difference between the groups might be explained by random errors (‘play of chance’).
We present a pre-defined statistical analysis plan for the TEENS feasibility trial, which limits bias, p-hacking, and data-driven interpretations. This statistical analysis plan is, furthermore, accompanied by a pre-programmed version-controlled statistical report with simulated data, which increases transparency and reproducibility.
Borderline Symptom List
Child and Adolescent Mental Health Services
Cognitive behavioural therapy
Coping with Children’s Negative Emotions Scale Adolescent
Coping with Children’s Negative Emotions Scale
Columbia-Suicide Severity Rating Scale
Copenhagen Trial Unit
Depression Anxiety Stress Scale
Difficulties in Emotion Regulation Scale
Deliberate Self-Harm Inventory
Experience in Close Relationships
Emotion Regulation Individual Therapy for Adolescents
Negative Effects Questionnaire
Strengths and Difficulties Questionnaire
Working Alliance Inventory
Swannell SV, Martin GE, Page A, Hasking P, St John NJ. Prevalence of nonsuicidal self-injury in nonclinical samples: systematic review, meta-analysis and meta-regression. Suicide Life Threat Behav. 2014;44(3):273–303. https://doi.org/10.1111/sltb.12070.
Asarnow JR, Porta G, Spirito A, Emslie G, Clarke G, Wagner KD, et al. Suicide attempts and nonsuicidal self-injury in the treatment of resistant depression in adolescents: findings from the TORDIA study. J Am Acad Child Adolesc Psychiatry. 2011;50(8):772–81. https://doi.org/10.1016/j.jaac.2011.04.003.
Whitlock J, Knox KL. The relationship between self-injurious behavior and suicide in a young adult population. Arch Pediatr Adolesc Med. 2007;161(7):634–40. https://doi.org/10.1001/archpedi.161.7.634.
Wilkinson P, Kelvin R, Roberts C, Dubicka B, Goodyer I. Clinical and psychosocial predictors of suicide attempts and nonsuicidal self-injury in the Adolescent Depression Antidepressants and Psychotherapy Trial (ADAPT). Am J Psychiatry. 2011;168(5):495–501. https://doi.org/10.1176/appi.ajp.2010.10050718.
Nock MK, Kessler RC. Prevalence of and risk factors for suicide attempts versus suicide gestures: analysis of the National Comorbidity Survey. J Abnorm Psychol. 2006;115(3):616–23. https://doi.org/10.1037/0021-843X.115.3.616.
Hawton K, Witt KG, Taylor Salisbury TL, Arensman E, Gunnell D, Townsend E, van Heeringen K, Hazell P. Interventions for self-harm in children and adolescents. Cochrane Database Syst Rev. 2015;(12):CD012013. https://doi.org/10.1002/14651858.CD012013.
Hawton K, Witt KG, Taylor Salisbury TL, Arensman E, Gunnell D, Hazell P, Townsend E, van Heeringen K. Psychosocial interventions for self-harm in adults. Cochrane Database Syst Rev. 2016;(5):CD012189. https://doi.org/10.1002/14651858.CD012189.
Turner BJ, Austin SB, Chapman AL. Treating nonsuicidal self-injury: a systematic review of psychological and pharmacological interventions. Can J Psychiatr. 2014;59(11):576–85. https://doi.org/10.1177/070674371405901103.
Ougrin D, Tranah T, Stahl D, Moran P, Asarnow JR. Therapeutic interventions for suicide attempts and self-harm in adolescents: systematic review and meta-analysis. J Am Acad Child Adolesc Psychiatry. 2015;54:97–107.e2.
Bjureberg J, Sahlin H, Hedman-Lagerlöf E, Gratz KL, Tull MT, Jokinen J, et al. Extending research on Emotion Regulation Individual Therapy for Adolescents (ERITA) with nonsuicidal self-injury disorder: open pilot trial and mediation analysis of a novel online version. BMC Psychiatry. 2018;18(1):326. https://doi.org/10.1186/s12888-018-1885-6.
Bjureberg J, Sahlin H, Hellner C, Hedman-Lagerlöf E, Gratz KL, Bjärehed J, et al. Emotion regulation individual therapy for adolescents with nonsuicidal self-injury disorder: a feasibility study. BMC Psychiatry. 2017;17(1):411. https://doi.org/10.1186/s12888-017-1527-4.
Gratz KL, Tull MT, Levy R. Randomized controlled trial and uncontrolled 9-month follow-up of an adjunctive emotion regulation group therapy for deliberate self-harm among women with borderline personality disorder. Psychol Med. 2014;44(10):2099–112. https://doi.org/10.1017/S0033291713002134.
Miller AL. Introduction to a special issue dialectical behavior therapy: evolution and adaptations in the 21st century. Am J Psychother. 2015;69(2):91–5. https://doi.org/10.1176/appi.psychotherapy.2015.69.2.91.
Vigerland S, Lenhard F, Bonnert M, Lalouni M, Hedman E, Ahlen J, et al. Internet-delivered cognitive behavior therapy for children and adolescents: a systematic review and meta-analysis. Clin Psychol Rev. 2016;50:1–10. https://doi.org/10.1016/j.cpr.2016.09.005.
Collins R, Bowman L, Landray M, Peto R. The magic of randomization versus the myth of real-world evidence. N Engl J Med. 2020;382(7):674–8. https://doi.org/10.1056/NEJMsb1901642.
Jakobsen J, Gluud C. The necessity of randomized clinical trials. Br J Med Med Res. 2013;3(4):1453–68. https://doi.org/10.9734/BJMMR/2013/3208.
Arshad U, Farhat-Ul-Ain GJ, Husain N, Chaudhry N, Taylor PJ. A systematic review of the evidence supporting mobile- and Internet-based psychological interventions for self-harm. Suicide Life Threat Behav. 2020;50(1):151–79. https://doi.org/10.1111/sltb.12583.
Morthorst B, Rubæk L, Lindschou J, Jakobsen JC, Gluud C, Bjureberg J, et al. An Internet-based emotion regulation intervention versus no intervention for nonsuicidal self-injury in adolescents: study protocol for a feasibility trial. Pilot Feasibility Stud. 2021;7:1–10.
Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, Thabane L, et al. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. BMJ. 2016;355:i5239.
Gamble C, Krishan A, Stocken D, Lewis S, Juszczak E, Doré C, et al. Guidelines for the content of statistical analysis plans in clinical trials. JAMA. 2017;318(23):2337–43. https://doi.org/10.1001/jama.2017.18556.
Gratz KL. Measurement of deliberate self-harm: preliminary data on the deliberate self-harm inventory. J Psychopathol Behav Assess. 2001;23(4):253–63. https://doi.org/10.1023/A:1012779403943.
Ravens-Sieberer U, Erhart M, Rajmil L, Herdman M, Auquier P, Bruil J, et al. Reliability, construct and criterion validity of the KIDSCREEN-10 score: a short measure for children and adolescents’ well-being and health-related quality of life. Qual Life Res. 2010;19(10):1487–500. https://doi.org/10.1007/s11136-010-9706-5.
Henry JD, Crawford JR. The short-form version of the Depression Anxiety Stress Scales (DASS-21): construct validity and normative data in a large non-clinical sample. Br J Clin Psychol. 2005;44(2):227–39. https://doi.org/10.1348/014466505X29657.
Bjureberg J, Ljótsson B, Tull MT, Hedman E, Sahlin H, Lundh L-G, et al. Development and validation of a brief version of the Difficulties in Emotion Regulation Scale: the DERS-16. J Psychopathol Behav Assess. 2016;38(2):284–96. https://doi.org/10.1007/s10862-015-9514-x.
Conway PM, Erlangsen A, Teasdale TW, Jakobsen IS, Larsen KJ. Predictive validity of the Columbia-Suicide Severity Rating Scale for Short-Term Suicidal Behavior: a Danish study of adolescents at a high risk of suicide. Arch Suicide Res. 2017;21(3):455–69. https://doi.org/10.1080/13811118.2016.1222318.
Fabes RA, Eisenberg N, Bernzweig J. The Coping with Children’s Negative Emotions Scale: procedures and scoring. Arizona State Univ. 1990; Available from: http://www.public.asu.edu/~rafabes/cnesall.pdf.
Rozental A, Kottorp A, Boettcher J, Andersson G, Carlbring P. Negative effects of psychological treatments: an exploratory factor analysis of the negative effects questionnaire for monitoring and reporting adverse and unwanted events. PLoS One. 2016;11(6):e0157503. https://doi.org/10.1371/journal.pone.0157503.
Julious SA. Sample size of 12 per group rule of thumb for a pilot study. Pharm Stat. 2005;4(4):287–91. https://doi.org/10.1002/pst.185.
Nørskov AK, Lange T, Nielsen EE, Gluud C, Winkel P, Beyersmann J, de Uña-Álvarez J, Torri V, Billot L, Putter H, Wetterslev J, Thabane L, Jakobsen JC. Assessment of assumptions of statistical analysis methods in randomised clinical trials: the what and how. BMJ Evid Based Med. 2021;26(3):121-126. https://doi.org/10.1136/bmjebm-2019-111268. Epub 2020 Jan 27.
Visalakshi J, Jeyaseelan L. Confidence interval for skewed distribution in outcome of change or difference between methods. Clin Epidemiol Glob Heal. 2014;2(3):117–20. https://doi.org/10.1016/j.cegh.2013.07.006. Accessed 1 May 2021.
Nielsen EE, Nørskov AK, Lange T, Thabane L, Wetterslev J, Beyersmann J, et al. Assessing assumptions for statistical analyses in randomised clinical trials. BMJ Evid Based Med. 2019;24(5):185–9. https://doi.org/10.1136/bmjebm-2019-111174.
Copenhagen Trial Unit, Centre for Clinical Intervention Research, Copenhagen University Hospital − Rigshospitalet, Copenhagen, Denmark
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Olsen, M.H., Morthorst, B., Pagsberg, A.K. et al. An Internet-based emotion regulation intervention versus no intervention for non-suicidal self-injury in adolescents: a statistical analysis plan for a feasibility randomised clinical trial. Trials 22, 456 (2021). https://doi.org/10.1186/s13063-021-05406-2