Skip to main content

The BEACON study: an update to the protocol for a cohort study as part of an evaluation of the effectiveness of smartphone-assisted problem-solving therapy in men who present with intentional self-harm to emergency departments in Ontario

A Study protocol to this article was published on 13 November 2020

Abstract

Background

Men who present to the emergency department (ED) with self-harm are at high risk of dying by suicide, with 2.7% of men dying in the year following their presentation, more than double the rate for women (1.2%). Despite this, care received after an ED visit is highly variable and many are not assessed for psychological needs. Furthermore, the limited psychological care that is available is often not covered by provincial health insurance. Even when referrals for follow-up care are made, engagement rates are low. Previous recommendations to improve engagement include written discharge plans, caring contacts, and focused interventions targeting middle-aged men at elevated risk of dying by suicide. Blended care, the incorporation of technology into traditional care, has also been proposed as a method to increase engagement in and clinical benefits from psychotherapy. This project aims to determine whether the delivery of an evidence based treatment (problem-solving therapy (PST)) is enhanced by the addition of a custom smartphone application (BEACON) compared to usual care. Due to the impact of the COVID-19 pandemic on site participation and the planned implementation, we have made several changes to the study design, primary outcome, and implementation.

Method

We originally proposed a cohort study nested within a larger cluster randomized trial wherein intervention sites would deliver the blended care, and control sites, whose personnel were not aware of their participation, would continue delivering usual care. The cohort study evaluated participant level outcomes as previously described by Hatcher et al. (2020). Due to pandemic-related constraints, our number of participating sites dropped to five potential sites which left the cohort study underpowered. As such, we changed the study design to a multi-site, individual randomized controlled trial (RCT) among the five remaining sites. Participants will be randomized to six sessions of therapy (PST) alone, or to the therapy plus BEACON, and followed up for 6 months. Our primary outcome was changed to evaluate feasibility and acceptability with the aim of designing a definitive RCT. Study implementation was reimagined to allow for completely virtual/online conduct to comply with local COVID-19 and institutional restrictions on in-person activities.

Conclusion

This updated protocol will provide strong results for the planning of a definitive RCT of the blended care intervention in the future, addressing areas of difficulty and concern prior to its implementation. We will evaluate the feasibility of the study intervention, assess recruitment and retention of participants, and address challenges with implementing the protocol. Lastly, we will evaluate the appropriateness of our primary outcome measure and accurately determine a sample size for a definitive RCT.

Trial registration

ClinicalTrials.gov, NCT03473535. Registered on March 22, 2018.

Peer Review reports

Update

This update describes amendments to study design, recruitment, implementation, and study outcomes and should be read alongside the previously published protocol [1]. The changes were employed due to the COVID-19 pandemic and the resulting impact on sites’ ability to conduct study activities. Consequently, several sites ultimately withdrew their participation, leaving the original study underpowered. All changes in the protocol have been approved by the Board of Record, the Ottawa Health Sciences Network Research Ethics Board (OHSN-REB) via Clinical Trials Ontario (https://www.ctontario.ca/), a centralized REB platform developed to facilitate multi-site clinical trial review in Ontario, Canada.

Study design

The original study design was a cohort study nested within a larger pragmatic multicenter pre- and post-design cluster randomized trial [1]. The COVID-19 pandemic resulted in a priority shift away from research at our randomized community sites resulting in their complete drop-out from the trial, leaving the cohort study underpowered with only 5 potential sites. As a result, we changed the design to a pilot individual randomized controlled trial (RCT). Instead of all individuals enrolled at a participating site receiving problem-solving therapy (PST) together with access to the BEACON smartphone application (BEACON), sites will now randomize participants to PST alone or PST + BEACON. The length of participation has also been changed with study follow-up decreased from 12 to 6 months. Visits will now occur as per the updated time and events schedule outlined below in Table 1. Eligibility criteria remains largely unchanged, with only the duration from ED presentation to recruitment extended from 2 to 4 weeks to accommodate potential pandemic-related delays in recruitment.

Table 1 Revised time and events schedule

Study outcome

In the original study, we had planned to evaluate change in suicidal ideation as measured on the Beck Scale for Suicide Ideation (BSS) [2,3,4]; however, the study is no longer powered for efficacy outcomes. In order to inform a future definitive trial, the updated primary outcome examines feasibility and acceptability.

Feasibility and acceptability will be evaluated using the following four indicators: (i) eligibility, recruitment, and retention; (ii) patient use and acceptability of the blended intervention; (iii) the primary outcome measure and sample size for a definitive RCT; and (iv) adherence to the protocol.

  1. i)

    For eligibility, we will retain screen failure data from those participants who have consented to be in the study to assess the frequency at which each inclusion/exclusion criterion are not met. We will assess recruitment by comparing group level demographics at each hospital of men who presented to the ED with self-harm compared with those enrolled in the study. Lastly, for retention, we will assess the characteristics of those who complete 0–2 sessions, 3–6 sessions but not the 6 month assessments, and those who complete 3–6 sessions and all follow-up assessments.

  2. ii)

    We will assess patient use of the BEACON application using de-identified usage statistics including number of BEACON presses and red pins activated, as well as any periods of app inactivity (more than 7 days). We will also conduct qualitative interviews with participants to assess the use of the BEACON application and the acceptability of the blended therapy, as well as any other treatments used by the participants.

  3. iii)

    To inform determination of a primary outcome in a definitive trial, we will measure the severity of suicide ideas at six months as measured by the BSS, as described in the original paper. We will use the change in responses as well as the qualitative interviews to determine whether the BSS is an appropriate outcome measure for the definitive RCT. We will also use the change in responses on the BSS to inform sample size calculations for the larger planned RCT.

  4. iv)

    We will evaluate any protocol deviations, planned or unplanned, as well as modifications requested by sites for the conduct of the study at their site in their REB submission. We will evaluate site level frequency of completion of a Therapy Adherence Form that is completed by the therapist at each study visit documenting which activities were completed.

With the exception of timing of administration, the secondary outcome measures remain unchanged. In the original protocol, many secondary outcome measures were administered weekly. This has been updated to baseline visit, final PST visit (week 6), and follow-up assessments (week 12 and week 24). The updated timeline and outcome administration is outlined in Table 1.

Implementation

To accommodate the ongoing work from home orders, we adjusted the delivery of study activities, including delivery of therapy, to allow for completely remote visits. For completion of study visits and therapy sessions, a videoconferencing platform will be used to facilitate “face-to-face” interactions. We also worked with the Ottawa Methods Centre to develop and implement a study Electronic Data Capture System (EDCS) that is capable of randomization and data capture. Upon enrollment of a participant, a study staff member enters the individual’s information into the EDCS, which then randomizes each new participant, assigning a unique identifier and study allocation.

Whereas the original protocol utilized paper-based questionnaires, the updated study uses the EDCS to provide a secure link for participants to complete questionnaires in their own home and at their own pace, a change required to facilitate remote study visits. The EDCS also captures staff entered data such as medications and adverse events. Entries are reviewed regularly by study staff and participants are informed that responses may not be reviewed immediately.

Recruitment

Recruitment has had to adapt to accommodate institutional restrictions for onsite and in-person activities. In many instances, research staff have not been allowed onsite to complete study activities to decrease the burden on the hospital and reduce the risk of transmission and outbreaks. Additionally, most sites are no longer allowed to leave recruitment materials that would be handled by multiple individuals (such as bookmarks and posters) in waiting areas due to infection prevention policies. As such, we have had to adapt our recruitment approach to accommodate for these changing scenarios. We have allowed sites to develop their own recruitment plans to reflect their own internal practices. As sites are based across Ontario, the implementation of regional restrictions has often meant that there is no consistent process across sites. The source of recruitment has not changed; participants must still have been seen in the ED, but recruitment efforts have been expanded to include scanning of medical records to recruit individuals who have consented to be contacted for research after their discharge from ED and recruitment among inpatient units where patients have been transferred to after their ED presentation.

Sample size

Sample size has been updated to reflect the change in design and outcomes. Calculating sample size for pilot studies is controversial. Calculations may be based on estimation of important parameters with sufficient precision [5], the likelihood of unforeseen problems [6] or rules of thumb such as 12 participants per group [7], at least 9% of the main trial’s sample size [8], or at least 50 participants [9]. Further, there is a lack of guidance on calculating sample size for pilots of multi-center trials where clustering at the different sites may be a factor. Based on previous rigorous randomized controlled trials of interventions in this population, we expect that in the main trial the effect size will be small and the sample size large. We have designed the pilot to estimate the proportion of patients who would meet our feasibility criteria, using confidence intervals. Based on our experience with previous studies conducted in this population, we estimate that enrolling 100 patients across participating sites would allow us to assess our feasibility outcomes and maximize the chance of identifying unexpected barriers to carrying out a larger trial across multiple centers.

Randomization

The original protocol was a cluster randomized trial where all sites whose personnel were aware of their involvement would receive access to the intervention. With the update to an individual RCT, the randomization criteria had to be redesigned. The updated randomization for this study will occur with 2:1 (67:33) allocation in favor of the blended therapy model across a maximum of five sites. Given the small sample size, there will be no stratification across sites to ensure an equitable allocation to the conditions.

Statistical analysis

The statistical analysis plan remains unchanged from the initial protocol, except for removal of an interim analysis.

Open science

In our original publication, we stated our intent to provide datasets on an open access platform (Open Science Framework; https://osf.io/). However, due to concerns around data sharing and privacy, and differing implementations of regulations between sites, this will no longer be possible. All publications resulting from the study will be available in an Open Access format, and de-identified datasets will be available from the principal investigator on reasonable request at the end of the study.

Current status of the study

This adapted of the protocol was approved and implemented in January 2021. Three sites have been activated and are enrolling participants. The first participant was enrolled in May 2021, and 34 participants have been enrolled as of July 2022. We will continue recruiting through 2022 with a planned end date of March 2023.

Availability of data and materials

Not applicable.

Abbreviations

AUDIT:

Alcohol-Use Disorder Identification Test

BDI:

Beck Depression Inventory

BSS:

Beck Scale for Suicide Ideation

CIHR:

Canadian Institutes of Health Research

CMNI:

Conformity to Masculine Norms Inventory

C-SSRS:

Columbia-Suicide Severity Rating Scale

DAST-10:

Drug Abuse Screening Test Short Form 10

ED:

Emergency Department

EDCS:

Electronic Data Capture System

EMIL:

Experienced Meaning in Life Questionnaire

EQ-5D-5L:

EuroQol 5 Dimensions (5 levels) Questionnaire

GAD-7:

Generalized Anxiety Disorder Questionnaire

MSPSS:

Multidimensional Scale of Perceived Social Support Questionnaire

OHSN-REB:

Ottawa Health Sciences Network Research Ethics Board

PC-PTSD:

Primary Care Post-Traumatic Stress Disorder Screening Questionnaire

PHQ-9:

Patient Health Questionnaire

PST:

Problem-Solving Therapy

RCT:

Randomized Controlled Trial

REB:

Research Ethics Board

SPSI-R:S:

Social Problem-Solving Inventory-Revised Short Form

tiC-P:

Questionnaire on Healthcare Consumption and Productivity losses for patients with a Psychiatric Disorder

References

  1. Hatcher S, Heisel M, Ayonrinde O, Campbell JK, Colman I, Corsi DJ, et al. The BEACON study: protocol for a cohort study as part of an evaluation of the effectiveness of smartphone-assisted problem-solving therapy in men who present with intentional self-harm to emergency departments in Ontario. Trials. 2020;21(1):925.

    Article  Google Scholar 

  2. Cochrane-Brink KA, Phil D, Lofchy JS, Sakinofsky I, Psych FRC. Clinical rating scales in suicide risk assessment. 2000. Available from: https://journals-scholarsportal-info.proxy.bib.uottawa.ca/pdf/01638343/v22i0006/445_crsisra.xml

  3. Beck A, Steer R. Manual for the Beck Scale for Suicide Ideation. Antonio, TX: Psychological Corporation; 1991.

    Google Scholar 

  4. Beck AT, Brown GK, Steer RA. Psychometric characteristics of the scale for suicide with psychiatric outpatients. Behav Res Ther. 1997;35(11):1039–46.

    Article  CAS  Google Scholar 

  5. Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, et al. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol. 2010;10:1.

    Article  Google Scholar 

  6. Viechtbauer W, Smits L, Kotz D, Budé L, Spigt M, Serroyen J, et al. A simple formula for the calculation of sample size in pilot studies. J Clin Epidemiol. 2015;68(11):1375–9.

    Article  Google Scholar 

  7. Julious SA. Sample size of 12 per group rule of thumb for a pilot study. Pharm Stat. 2005;4(4):287–91.

    Article  Google Scholar 

  8. Cocks K, Torgerson DJ. Sample size calculations for pilot randomized trials: a confidence interval approach. J Clin Epidemiol. 2013;66(2):197–201. https://doi.org/10.1016/j.jclinepi.2012.09.002.

    Article  PubMed  Google Scholar 

  9. Sim J, Lewis M. The size of a pilot study for a clinical trial should be calculated in relation to considerations of precision and efficiency. J Clin Epidemiol. 2012;65(3):301–8.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

The trial is sponsored by the Ottawa Hospital Research Institute (OHRI) and funded by the Ontario Strategy for Patient Oriented Research (SPOR) Support Unit (OSSU), funded through the Canadian Institutes of Health Research (CIHR) and the Government of Ontario. The funders had no role in the study, including the study design; collection, analysis, and interpretation of study data; or the drafting and review of the manuscript for publication.

Author information

Authors and Affiliations

Authors

Contributions

SH: Conceptualization, Funding Acquisition, Methodology, Resources, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing; MH: Conceptualization, funding acquisition, methodology, Writing – Review & Editing; OA: Supervision, Writing – Review & Editing; DC: Methodology, Writing – Review & Editing; NEE: Conceptualization, Methodology, Project Administration, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing; SHK: Supervision, Writing – Review & Editing; SJR: Supervision, Writing – Review & Editing; AS: Supervision, Writing – Review & Editing; MS: Supervision, Writing – Review & Editing.

Corresponding author

Correspondence to Simon Hatcher.

Ethics declarations

Ethics approval and consent to participate

This protocol has received research ethics board (REB) approval from the OHSN-REB as the board of record via Clinical Trials Ontario (CTO reference CTO-790). All active study sites have received approval.

Consent for publication

Not applicable.

Competing interests

The study investigators have the following competing interests to declare: SJR is a coinvestigator with the Canadian Biomarker Integration Network in Depression (CAN-BIND), funded by the Ontario Brain Institute (OBI). She also received research grant funding from Pfizer Canada. The other authors declare that they have no competing interests. Dr. Sidney H. Kennedy has received funding for Consulting or Speaking engagements from Abbvie, Boehringer-Ingelheim, Janssen, Lundbeck, Lundbeck Institute, Merck, Otsuka Pfizer, Sunovion, and Servier. He has received Research Support from Abbott, Brain Canada, CIHR (Canadian Institutes of Health Research), Janssen, Lundbeck, Ontario Brain Institute, Otsuka, Pfizer, and SPOR (Canada’s Strategy for Patient-Oriented Research). He has stock/stock options in Field Trip Health.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hatcher, S., Heisel, M.J., Ayonrinde, O. et al. The BEACON study: an update to the protocol for a cohort study as part of an evaluation of the effectiveness of smartphone-assisted problem-solving therapy in men who present with intentional self-harm to emergency departments in Ontario. Trials 23, 849 (2022). https://doi.org/10.1186/s13063-022-06788-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13063-022-06788-7