- Study protocol
- Open Access
Population-based outreach versus care as usual to prevent suicide attempt: study protocol for a randomized controlled trial
Trials volume 17, Article number: 452 (2016)
Suicide remains the 10th-ranked most frequent cause of death in the United States, accounting for over 40,000 deaths per year. Nonfatal suicide attempts lead to over 200,000 hospitalizations and 600,000 emergency department visits annually. Recent evidence indicates that responses to the commonly used Patient Health Questionnaire (PHQ9) can identify outpatients who are at risk of suicide attempt and suicide death and that specific psychotherapy or Care Management programs can prevent suicide attempts in high-risk patients. Motivated by these developments, the NIMH-funded Mental Health Research Network has undertaken a multisite trial of two outreach programs to prevent suicide attempts among outpatients identified by routinely administered PHQ9 questionnaires.
Outpatients who are at risk of suicide attempt are automatically identified using data from electronic health records (EHRs). Following a modified Zelen design, all those identified are assigned to continued usual care (i.e., no contact) or to be offered one of two population-based outreach programs. A Care Management intervention includes systematic outreach to assess suicide risk, EHR-based tools to implement risk-based care pathways, and care management to facilitate recommended follow-up. A Skills Training intervention includes interactive online training in Dialectical Behavior Therapy skills, supported by reminder and reinforcement messages from a skills coach. Each intervention supplements, rather than replaces, usual care; participants may receive any other services normally available. Interventions are delivered primarily by secure messaging through EHR patient portals. Suicide attempts and deaths following randomization are identified using state vital statistics data and health system EHR and insurance claim data. Primary evaluation will compare risk of suicide attempt or death over 18 months according to the initial assignment, regardless of intervention participation. Recruitment is underway in three health systems (Group Health Cooperative, HealthPartners, and Kaiser Permanente Colorado). Over 2500 participants have been randomized as of 1 March 2016, with enrollment averaging approximately 100 per week.
Assessing the effectiveness of population-based suicide prevention requires adherence to the principles of pragmatic trials: population-based enrollment, accepting variable treatment participation, assessing outcomes using health record data, and analyses based on intent-to-treat.
ClinicalTrials.gov registration #NCT02326883, registered on 23 December 2014.
Suicide remains the 10th-ranked most frequent cause of death in the United States, accounting for over 40,000 deaths per year . Nonfatal suicide attempts lead to over 200,000 hospitalizations and 600,000 emergency department visits each year [2, 3]. In contrast with other common causes of death, suicide mortality has not decreased over the last 25 years.
While prevention of suicide attempts and suicide death is a public health priority, existing evidence does not clearly support selective or secondary prevention programs. In 2013, the US Preventive Services Task Force found insufficient evidence to support the benefits of screening for suicide risk in general medical outpatients . That review found insufficient evidence both for the accuracy of screening tests and for the effectiveness of interventions in those identified by screening.
More recent evidence indicates that responses to the commonly used Patient Health Questionnaire (PHQ9) can identify outpatients who are at increased risk of suicide attempt and suicide death [5, 6]. Patients reporting frequent thoughts of death or self-injury (the ninth item of the PHQ9) show a sustained increase in risk, with cumulative hazard approaching 4 % over 12 months. Reflecting this new evidence, the Joint Commission recently issued a Sentinel Event Alert  regarding detection of suicidal ideation in health care settings.
In addition, growing evidence supports the effectiveness of tertiary or indicated prevention interventions for high-risk patients. Structured psychotherapy emphasizing specific behavioral and cognitive skills has been proven to reduce risk among people who have made recent suicide attempts [8–10]. Outreach and Care Management programs appear to reduce risk among people who have made recent suicide attempts or high-risk patients treated in mental health specialty clinics . This evidence for tertiary prevention suggests that similar interventions could reduce risk in the broader (secondary prevention) population of patients who are experiencing frequent suicidal ideation.
Motivated by these developments, the National Institute of Mental Health-funded Mental Health Research Network has undertaken a multisite trial of two population-based programs to prevent suicide attempts among outpatients identified by routinely administered depression questionnaires. Both programs include systematic outreach and regular supportive contact. One focuses on risk assessment and care management , while the other includes online training in specific skills from Dialectical Behavior Therapy (DBT) . This pragmatic trial will examine whether either program can reduce long-term risk compared to care as usual.
At participating health systems, outpatients who are at increased risk of suicide attempt are identified using data from electronic health records (EHRs). Following a modified Zelen [13–15] design, all those identified are automatically assigned to continue in usual care (i.e., no contact) or to be offered one of two population-based prevention programs:
A Care Management intervention including: systematic outreach to assess risk of suicidal behavior, EHR-based tools to implement risk-based care pathways, and care management to facilitate and monitor recommended follow-up care
A Skills Training intervention including: interactive online training in DBT skills , supported by reminder and reinforcement messages from a skills coach
Each intervention is intended to supplement, rather than replace, usual care provided by specialty mental health or primary care providers. Participants in all three treatment groups are free to receive any other services that are normally available, including pharmacotherapy, individual or group psychotherapy, or inpatient care. Intervention services are delivered primarily by online secure messaging through EHR patient portals [16, 17]. Nonfatal and fatal suicide attempts following randomization are identified using state vital statistics data and diagnoses of self-inflicted injury from health system clinical and insurance claim records [18, 19]. Primary evaluation will compare risk of first suicide attempt (nonfatal or fatal) over the 18 months following randomization. Groups will be compared according to initial treatment assignment, regardless of level of participation in either intervention program.
The study sites include three members of the Mental Health Research Network: Group Health Cooperative, HealthPartners, and Kaiser Permanente of Colorado. These health systems provide general medical and mental health specialty care as well as insurance coverage to defined member/patient populations. Patients served are representative of each health system’s geographic service area in terms of race, ethnicity, educational attainment, and household income.
All three participating health care systems recommend the routine use of the PHQ9 depression questionnaire  at all mental health specialty visits and all primary care visits for treatment of depression . All three sites participated in previous research  demonstrating that the response to item 9 of the PHQ9 (regarding thoughts of death or self-harm) predicts markedly elevated risk of suicide attempt over the following 18 months (Fig. 1).
Eligibility criteria for automatic inclusion in the trial include:
Completion of a PHQ9 questionnaire  at an outpatient mental health or general medical visit
Age 18 years or older on the visit date
Response of “more than half the days” or “nearly every day” to item 9 of the PHQ9
Use of EHR patient portal secure messaging during the prior year
Currently enrolled in participating health system insurance plan (to ensure adequate ascertainment of subsequent suicide attempts)
Exclusion criteria include:
Recorded diagnosis of dementia or developmental delay in the previous 2 years
Limited English proficiency (as indicated by “need for interpreter” recorded in the EHRs)
Previous request to be excluded from research invitations
Already enrolled and randomized via a previous visit
Enrollment and randomization
Each week, EHR and insurance claim databases at each study site are used to identify all patients who meet eligibility criteria during the previous week. Immediately after sampling, all eligible patients are randomly assigned in equal proportions (1/3:1/3:1/3) to continue in usual care or to be offered one of the two intervention programs. At each site, randomization occurs automatically within the sampling computer program, stratified by eligible PHQ9 response (“more than half the days” or “nearly every day”) and site. A computer-generated concealed allocation table at each site provides randomly generated assignments in block sizes of either six or nine.
Invitation and consent
Participants assigned to the Care Management intervention receive an initial invitation message from the study care manager via the EHR-based online secure messaging system [16, 17]. This invitation includes a brief description of the Care Management program and abbreviated information regarding required elements of informed consent (study purpose, study procedures, potential risks or harms, and right to refuse or withdraw at any time). Each participant can decline participation by replying to the invitation message or can consent to receive intervention services by either replying to the message or returning the attached risk assessment questionnaire. Participants who neither decline nor consent receive a reminder (either by online message or telephone message) after 1 week. Participants who neither decline nor consent after that reminder message receive a second invitation message (with possible reminder) 4 weeks later and may receive a third invitation message after an additional 4 weeks. Those who decline at any point are not contacted again. Those who do not respond after three cycles of invitation are not contacted again.
The invitation and consent process for participants assigned to the Skills Training program parallels that for the Care Management program: an invitation message including abbreviated informed consent information and up to three cycles of invitation and reminder for those not responding. A participant can consent to receive intervention services by replying to an invitation message or by making an initial visit to the online intervention program. Those who decline at any point are not contacted again. Those who do not respond after three cycles of invitation are not contacted again.
Participants assigned to continue in usual care are never contacted by study staff. Providers are not notified regarding participants’ assignment to continued usual care.
Care Management intervention
Following the design of previous Care Management interventions , including the Perfect Depression Care program at Henry Ford Health System , this program aims to reduce risk of suicide attempt by monitoring and maintaining engagement in effective mental health treatment.
In collaboration with developers of the Columbia Suicide Severity Rating Scale  (CSSRS), study investigators developed a simplified CSSRS for self-report administration via online secure messaging. This abbreviated CSSRS provides a 6-point ordinal rating of current suicide risk based on frequency and intensity of suicidal thoughts, presence and specificity of suicidal plans, and clarity of suicidal intent during the last week. For example, a score of 0 would indicate no recent thoughts of self-injury or suicide, a score of 3 would indicate suicidal ideation with some recent thoughts about specific means, and a score of 5 would indicate a current and specific suicidal plan.
In collaboration with health system stakeholders (see below), investigators developed rules for risk-based care pathways specifying appropriate level of care, minimum standards for follow-up visit intervals and timing of outreach messages. For example: a CSSRS score of 1 would lead to a recommendation for follow-up within 1 month (sooner if clinically appropriate) in either primary care or specialty mental health care, a score of 3 would lead to a recommendation for follow-up in specialty mental health care within 2 weeks, and a score of 5 would lead to a recommendation for specialty mental health follow-up within two business days (or sooner as clinically indicated).
Care manager role
At each site, one or more care managers are responsible for:
Initial and follow-up invitations to all participants assigned to be offered Care Management
Periodic outreach to assess current risk (using an online version of the CSSRS)
Application of follow-up algorithms, supported by informatics tools (see below)
Regular feedback to treating providers regarding risk assessments and follow-up plans
As-needed communication with participants and providers to facilitate follow-up care
Care managers communicate with participants primarily by online secure messaging through EHR patient portals, but may communicate by telephone as needed. Care managers are expected to consider individual patients’ clinical circumstances and providers’ treatment plans when applying algorithms regarding outreach and visit frequency. At all sites, care managers are Master’s-prepared mental health clinicians.
Intervention delivery is supported by existing functions of health system EHRs:
Online administration of structured questionnaires such as the CSSRS
Secure provider-to-provider messaging for care managers’ communications with primary care and mental health specialty providers
Population management and reporting tools to apply follow-up algorithms and deliver algorithm-based recommendations to care managers regarding outreach and follow-up
Engagement with health system stakeholders
During the pilot phase, regular meetings with clinical leaders from all sites (representing both primary care and mental health specialty care) developed consensus regarding content and workflow of the Care Management program, including:
Language for outreach messages
Recommended follow-up intervals
Criteria for referral from primary care to mental health specialty care
Processes for communicating with primary care and mental health specialty providers
Procedures for urgent assessment and referral
Training and supervision
Training of care managers across sites was conducted by videoconference and teleconference, led by clinical investigators from the Group Health site. Initial training included:
6 h of clinical training regarding suicide risk assessment
2 h of general orientation to project aims and procedures
6 h of specific training regarding care management aims, tools, and procedures
Ongoing teleconference supervision for all care managers is led by clinical investigators from the Group Health site. Supervision meetings were scheduled weekly for 6 months and twice monthly thereafter. Consistent with the principles of pragmatic trials , no detailed monitoring of intervention fidelity (e.g., review of content of online messaging or phone contacts) is conducted.
Skills Training intervention
Following the content and structure of proven Dialectical Behavior Therapy (DBT) treatments [8, 12], this program aimed to reduce risk of suicide attempt through training in specific DBT skills shown to mediate the beneficial effects of traditional in-person DBT .
Specific skills content
The online program and coaching support focuses on four specific skills:
Mindfulness – Introduction to mindfulness skills, emphasizing nonjudgmental observation
Mindfulness of Current Emotion – Nonjudgmental observation of sensations associated with difficult or painful emotions
Opposite Action – Acting in opposition to urges associated with painful or difficult emotions
Paced Breathing – Use of breathing techniques to manage overwhelming emotions or crises
Adaptation for online delivery
The web-based interactive program includes an introductory section, personal profiles of team members (including coaches and contributing peer experts described below), and four skill modules (one for each DBT skill listed above). Each module includes:
A brief video introduction to the skill concept
A longer teaching video describing the skill, including in vivo practice
Example videos of peers (see below) describing use of the skill in daily life
Interactive exercises for use during the online session
Customizable worksheets to support between-session practice
Each participant is free to visit skills modules in any order and use the program at any pace, returning as frequently as desired. Coaches send reinforcement and outreach messages (see below) to encourage regular use.
Skills coach role
At each site, one or more skills coaches is responsible for:
Initial and follow-up invitations to all participants assigned to be offered Skills Training
For participants visiting the online program, messages to reinforce use of the program and practice of specific skills
For participants not visiting the program, periodic outreach messages to encourage return visits
As-needed communication with treating providers regarding participants’ progress
Skills coaches communicate with participants primarily by online secure messaging through EHR patient portals, but may communicate by telephone as needed. At all sites, skills coaches are Master’s-prepared mental health clinicians.
The online program is delivered through the DatStat survey platform (DatStat Inc., Seattle, WA, USA). This platform supports secure access, detailed tracking of participant activity, and participant-level reports to guide the timing and content of coaches’ reinforcement and reminder messages. Participants access the online program via secure personalized links embedded in messages from skills coaches.
Engagement with patient stakeholders
Peer experts (people with lived experience of suicidal ideation and suicide attempts) were essential collaborators in the development of the Skills Training intervention and continue to support intervention delivery . Design of outreach messages and content of the online program were informed by anonymous online surveys and focus group interviews with peer experts . People with lived experience contributed video descriptions of the use of DBT skills, and continue to contribute to development of training and support materials for skills coaches.
Training and supervision
Training of skills coaches was conducted by videoconference and teleconference, led by clinical investigators from the Group Health site. Initial training included:
6 h of clinical training regarding suicide risk assessment
2 h of general orientation to project aims and procedures
6 h of specific training regarding skills coaching aims, tools, and procedures
Ongoing teleconference supervision for all skills coaches is led by clinical investigators from the Group Health site. Supervision meetings were scheduled weekly for 6 months and twice monthly thereafter. Consistent with the principles of pragmatic trials , no detailed monitoring of intervention fidelity (e.g., review of content of online messaging or phone contacts) is conducted.
The primary study outcome is the time to first suicide attempt following randomization. Fatal suicide attempts will be identified by death certificate diagnoses of self-inflicted injury or poisoning. All three study sites routinely link membership files to state vital record data to ascertain cause of death for all enrolled members. Nonfatal suicide attempts will be identified from EHRs (for care delivered by participating health systems) and insurance claim data (for care received outside of participating health systems) using three criteria:
Any outpatient or inpatient diagnosis of definite self-inflicted injury or poisoning
Any outpatient or inpatient diagnosis of possible self-inflicted injury or poisoning
Any outpatient or inpatient diagnosis of other injury or poisoning associated with a diagnosis of suicidal ideation during the same encounter
For these three criteria, review of full-text medical records documented high positive predictive value for self-inflicted injury with suicidal intent [18, 25]. Because this validation work was completed prior to health care systems’ transition from the International Classification of Diseases, version 9 (ICD-9) to ICD-10 diagnoses, additional work will be necessary in 2016 to revalidate outcome definitions based on ICD-10 diagnostic codes. Given that participants may seek care for self-injury at external facilities, ascertainment will include both insurance claim and EHR data, and the sample is limited to patients who are enrolled in a health system insurance plan.
Primary analyses will use the log-rank test to compare risk of diagnosed suicide attempt (defined above) over 18 months following randomization. For each intervention condition, risk among those assigned to the intervention will be compared to risk among those assigned to usual care – regardless of level of participation in either intervention. Individuals will be censored at time of health system disenrollment, death from cause other than suicide, or administratively, at 18 months following randomization. We evaluate the effect of each of the interventions compared to usual care using a log-rank test stratified by site and initial response to PHQ9 item 9 (2 versus 3). Sensitivity analyses will use weighted log-rank tests to account for a possible association between prerandomization characteristics (ascertained from EHRs) and censoring. In censoring weights, we will include sex, age group (18–29, 30–64, and 65 or more), race/ethnicity (Black American, Asian American, Hispanic, other), and visit type at which the initial PHQ9 questionnaire was completed (primary care versus mental health specialty).
Original sample size estimates were based on previous research  suggesting an approximately 4 % risk of suicide attempt over 18 months among those meeting study eligibility criteria. Consultation with health system stakeholders indicated that implementation of a systematic outreach program was unlikely unless a program could be expected to reduce that risk to 3 % (relative risk reduction of 25 %). We used PASS software  to estimate the sample size required for a log-rank test  with 90 % power to compare the survival curves assuming a 3.8 % risk over 18 months in the comparison group and a 25 % risk reduction in the intervention group. We assumed 2 % disenrollment rate each month, resulting in approximately 25 % censorship over the 18 months of follow-up. The primary comparison for this trial is each of the interventions compared to usual care; we use a Bonferroni correction to account for the two tests in our primary analysis. Assuming a two-sided log-rank test, with a type-1 error rate of 0.025 and 90 %, we plan on enrolling 6500 patients per arm (total n = 19,500).
The trial was funded through the NIH Health Care Systems Collaboratory as one of the initial Pragmatic Clinical Trials Demonstration Projects . Following a pilot phase to validate outcome definitions and to demonstrate the feasibility and acceptability of the intervention programs, participant enrollment and randomization began at the Group Health site in March 2015, expanding to three sites in July 2015. Approximately 4000 participants have been enrolled and randomized as of 1 July 2016. Approximately 100 participants are enrolled and randomized each week, and that rate is expected to increase to approximately 150 in the fall of 2016.
Ethical and regulatory approval
Study design and procedures were reviewed and approved by Institutional Review Boards at all three health system. That review process addressed several issues common to pragmatic trials of prevention interventions.
Waiver of informed consent
Limiting a randomized trial of outreach to those who actively consent to receive outreach would yield a result of questionable validity and generalizability. Consequently, a modified Zelen design [13–15], randomizing all eligible patients without first obtaining consent, is necessary for valid test of the study question. This design, however, requires a waiver of the usual requirement for informed consent prior to enrollment or randomization. While it is not practicable to obtain informed consent prior to randomization, it is practicable to provide appropriate information to participants at the time intervention services are offered. As described above, invitation messages to participants assigned to either intervention include a brief description of the study purpose, study procedures, potential risks, and the right to decline participation. This design, therefore, includes a waiver of consent for enrollment/randomization and a modified consent procedure for receipt of intervention services.
Defining minimal risk
Current regulations regarding protection of human research participants allow waiver of consent for research involving no more than minimal risk. Our proposal to waive the requirement for informed consent in patients who are at risk of suicide attempt led to extensive discussions with health system Institutional Review Boards, leadership of the NIH Healthcare Systems Research Collaboratory, and the federal Office for Human Research Protections . Those discussions helped to clarify four issues:
Research risk versus preexisting risk – Given that the trial enrolls patients who are at risk of suicide attempt, we encountered concern that study procedures could not be classified as having minimal risk. To address this concern, we relied on regulatory guidance distinguishing between preexisting risk due to a research participant’s health state (i.e., increased risk of suicide attempt) and incremental risk created by study procedures. This distinction led to the appreciation of this trial as evaluating minimal-risk interventions in a high-risk population
Risk of assignment to continued usual care – We also encountered concern regarding the ethical acceptability of randomly assigning patients who are at risk of suicide attempt to a usual care control condition. We clarified that a participant assigned to usual care will, by definition, receive the same treatment that she or he would have received if the study were not occurring
Risk of assignment to offer of intervention programs – We also encountered concern that assignment to either intervention group might increase risk. Both intervention programs are based on effective interventions and are intended to reduce risk of suicide attempt. Participants are free to receive any other services that are normally available. Nevertheless, it is possible that some participants will experience negative effects from either program. We addressed this concern by clarifying that participants are assigned to the offer of an intervention, with no obligation to participate. Invitation messages clearly identify intervention programs as research activities, make no promise of benefit, and advise that participation is completely voluntary
Intrusiveness or invasion of privacy – Different stakeholders expressed concern regarding both inappropriate intrusiveness of repeated outreach and inadequate vigor of outreach given the known increased risk of suicide attempt. In consultation with peer experts, we designed the outreach strategy described above, including up to three cycles of invitation, as a reasonable compromise between these two concerns.
At all participating health systems, notices regarding privacy practices specifically advise members regarding the use of health records for research. Members who have previously requested exclusion from research contact are excluded from the study sample.
Monitoring for adverse events
In most clinical trials of mental health treatments, a suicide attempt would be considered a serious adverse event, subject to immediate reporting and review to determine the “relatedness” of an individual suicide attempt to study participation. This traditional approach was clearly not appropriate for a large-scale trial of population-based prevention programs . First, record data regarding suicide attempts may not be available for 3 months or more following an event. Second, several hundred suicide attempts are expected to occur among study participants, and review of individual events could not determine causal relationship to study participation. While it is possible that either intervention could paradoxically increase risk of suicide attempt, that could only be determined by comparison to suicide attempt rates in usual care (see below).
Interim analyses of benefit or harm
We do not plan any interim analyses to evaluate benefit of the intervention programs. First, early detection of a benefit of either intervention is extremely unlikely. We project that randomization will be complete before complete outcome data are available for half of the participants. Second, early termination of randomization or intervention delivery would not offer any additional protection to current or future study participants. Premature termination would instead return all current and potential participants to care as usual.
We do, however, plan interim analyses testing for evidence of significant harm (increased risk of suicide attempt) in either intervention group compared to usual care. Clear evidence that either intervention resulted in significantly increased risk of suicide attempt would certainly warrant suspending assignment of patients to that program or suspending delivery of that program to participants already assigned. Interim analyses comparing risk of suicide attempt in each intervention group to that in usual care will be conducted three times per year, beginning 12 months after start of enrollment. Interim analyses will be reported to the National Institute of Mental Health Data and Safety Monitoring Board by the study statistician, but all other study staff will be blinded to these results.
Data and resource sharing
A deidentified version of the analytic dataset will be made available at the time of the initial publication of primary study findings. Consistent with policies of the NIH Collaboratory, all resources (intervention materials, specifications, computer code, etc.) will be shared at or before the publication of study results.
This suicide prevention outreach trial addresses a practical question that is relevant to practicing clinicians or health system leaders: Will population-based outreach programs reduce risk of suicide attempt among patients identified as being at risk by routinely administered depression questionnaires? This focus on a practical or pragmatic question has several implications for trial design. Table 1 describes how specific aspects of this trial conform to the characteristics of pragmatic trials described by Thorpe . Central pragmatic trial features include:
Population-based enrollment – If we hope to inform policy or implementation decisions by health system leaders, then it is necessary to evaluate program effectiveness in the full population of those to whom the program would eventually be offered
Allowing variable participation or compliance – Restricting enrollment to those willing to participate in outreach or prevention programs would not allow a valid assessment of program effectiveness. Low participation or high rates of dropout should be considered essential indicators of effectiveness rather than threats to internal validity
Analysis by intent-to-treat – A valid evaluation of prevention program effectiveness must examine risk among all those offered the prevention service, rather than those who accept or participate. Any “as treated” or “completers” analysis (limited to those who participate in prevention services) would certainly be biased
Our trial differs from a purely pragmatic design in one aspect: the training and supervision of clinical staff delivering the prevention interventions. All care managers and skills coaches complete approximately 14 h of initial training followed by weekly or bi-weekly supervision teleconferences. This training and supervision was necessary because both of these clinical roles required implementation of new clinical work processes and the use of new informatics tools. If either program is proven effective, we would recommend that any subsequent implementation include a similar level of training as well as a period of regular supervision.
Assessing the effectiveness of any population-based prevention program requires a clinical trial following the core principles of pragmatic trials: population-based enrollment, accepting variable treatment participation, assessing outcomes using health record data, and analyses based on intent-to-treat. We describe the design and implementation of such a trial, now underway in three large integrated health systems.
Enrollment is ongoing and is expected to be complete in early 2018.
Columbia Suicide Severity Rating Scale
Dialectical Behavior Therapy
Electronic health record
Patient Health Questionnaire depression scale
National Center for Health Statistics: Deaths: final data for 2013. In: National Vital Statistics Reports. vol. 64. Hyattsville 2015.
National Hospital Ambulatory Medical Care Survey: 2008 Emergency Department Summary Tables. http://www.cdc.gov/nchs/data/ahcd/nhamcs_emergency/2011_ed_web_tables.pdf. Accessed 11 Sept 2016.
Crosby AE, Han B, Ortega LAG, Parks SE, Gfroerer J. Suicidal thoughts and behaviors among adults aged ≥18 years - United States, 2008–2009. MMWR Morb Mortal Wkly Rep. 2011;60(13):1–31.
O’Connor E, Gaynes BN, Burda BU, Soh C, Whitlock EP. Screening for and treatment of suicide risk relevant to primary care: a systematic review for the U.S. Preventive Services Task Force. Ann Intern Med. 2013;158(10):741–54.
Simon GE, Coleman KJ, Rossom RC, Beck A, Oliver M, Johnson E, Whiteside U, Operskalski B, Penfold RB, Shortreed SM, et al. Risk of suicide attempt and suicide death following completion of the Patient Health Questionnaire depression module in community practice. J Clin Psychiatry. 2016;77(2):221–7.
Simon GE, Rutter CM, Peterson D, Oliver M, Whiteside U, Operskalski B, Ludman EJ. Does response on the PHQ-9 Depression Questionnaire predict subsequent suicide attempt or suicide death? Psychiatr Serv. 2013;64(12):1195–202.
Patient Safety Advisory Group: detecting and treating suicidal ideation in all settings. The Joint Commission Sentinel Event Alerts 2016;56:1-7.
Comtois KA, Linehan MM. Psychosocial treatments of suicidal behaviors: a practice-friendly review. J Clin Psychol. 2006;62(2):161–70.
Linehan MM, Comtois KA, Murray AM, Brown MZ, Gallop RJ, Heard HL, Korslund KE, Tutek DA, Reynolds SK, Lindenboim N. Two-year randomized controlled trial and follow-up of dialectical behavior therapy vs therapy by experts for suicidal behaviors and borderline personality disorder. Arch Gen Psychiatry. 2006;63(7):757–66.
Rudd MD, Bryan CJ, Wertenberger EG, Peterson AL, Young-McCaughan S, Mintz J, Williams SR, Arne KA, Breitbach J, Delano K, et al. Brief cognitive-behavioral therapy effects on post-treatment suicide attempts in a military sample: results of a randomized clinical trial with 2-year follow-up. Am J Psychiatry. 2015;172(5):441–9.
Hampton T. Depression care effort brings dramatic drop in large HMO population’s suicide rate. JAMA. 2010;303(19):1903–5.
Neacsiu AD, Rizvi SL, Linehan MM. Dialectical behavior therapy skills use as a mediator and outcome of treatment for borderline personality disorder. Behav Res Ther. 2010;48(9):832–9.
Adamson J, Cockayne S, Puffer S, Torgerson DJ. Review of randomised trials using the post-randomised consent (Zelen’s) design. Contemp Clin Trials. 2006;27(4):305–19.
Zelen M. A new design for randomized clinical trials. N Engl J Med. 1979;300(22):1242–5.
Zelen M. Randomized consent designs for clinical trials: an update. Stat Med. 1990;9(6):645–56.
Ralston JD, Rutter CM, Carrell D, Hecht J, Rubanowice D, Simon GE. Patient use of secure electronic messaging within a shared medical record: a cross-sectional study. J Gen Intern Med. 2009;24(3):349–55.
Simon GE, Ralston JD, Savarino J, Pabiniak C, Wentzel C, Operskalski BH. Randomized trial of depression follow-up care by online messaging. J Gen Intern Med. 2011;26(7):698–704.
Simon GE, Savarino J. Suicide attempts among patients starting depression treatment with medications or psychotherapy. Am J Psychiatry. 2007;164(7):1029–34.
Lu CY, Stewart C, Ahmed AT, Ahmedani BK, Coleman K, Copeland LA, Hunkeler EM, Lakoma MD, Madden JM, Penfold RB, et al. How complete are E-codes in commercial plan claims databases? Pharmacoepidemiol Drug Saf. 2014;23:218–20.
Kroenke K, Spitzer RL, Williams JB, Lowe B. The Patient Health Questionnaire Somatic, Anxiety, and Depressive Symptom Scales: a systematic review. Gen Hosp Psychiatry. 2010;32(4):345–59.
Simon GE, Rossom RC, Beck A, Waitzfelder BE, Coleman KJ, Stewart C, Operskalski B, Penfold RB, Shortreed SM. Antidepressants are not overprescribed for mild depression. J Clin Psychiatry. 2015;76(12):1627–32.
Posner K, Brown GK, Stanley B, Brent DA, Yershova KV, Oquendo MA, Currier GW, Melvin GA, Greenhill L, Shen S, et al. The Columbia-Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry. 2011;168(12):1266–77.
Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, Tunis S, Bergel E, Harvey I, Magid DJ, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol. 2009;62(5):464–75.
Whiteside U, Lungu A, Richards J, Simon GE, Clingan S, Siler J, Snyder L, Ludman E. Designing messaging to engage patients in an online suicide prevention intervention: survey results from patients with current suicidal ideation. J Med Internet Res. 2014;16(2):e42.
Simon G, Savarino J, Operskalski B, Wang P. Suicide risk during antidepressant treatment. Am J Psychiatry. 2006;163:41–7.
PASS 14 Power Analysis and Sample Size Software. In: Kaysville, Utah: NCSS; 2015
Lakatos E. Sample sizes based on the log-rank statistic in complex clinical trials. Biometrics. 1988;44(1):229–41.
Johnson KE, Neta G, Dember LM, Coronado GD, Suls J, Chambers DA, Rundell S, Smith DH, Liu B, Taplin S, et al. Use of PRECIS ratings in the National Institutes of Health (NIH) Health Care Systems Research Collaboratory. Trials. 2016;17(1):32.
Demonstration project ethics and regulatory documentation. https://www.nihcollaboratory.org/demonstration-projects/Pages/regulatoryethics.aspx. Accessed 11 Sept 2016.
Data and safety monitoring in pragmatic trials. https://www.nihcollaboratory.org/Pages/Grand-Rounds-02-13-15.aspx. Accessed 11 Sept 2016.
Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, Tunis S, Bergel E, Harvey I, Magid DJ, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. CMAJ. 2009;180(10):E47–57.
Drs. Brian Ahmedani, Marsha Linehan, Anita Lungu, and Maria Oquendo contributed to the development and pilot-testing of intervention programs.
Funded by Grant UH3 MH007755 from the NIH Common Fund Health Care Systems Research Collaboratory and the National Institute of Mental Health.
Availability of supporting data
Outcome data will not be available until 12 months following completion of recruitment. A deidentified analytic dataset will be made available for public use following publication of the main results.
GS led the design of the trial and pilot testing of interventions and had primary responsibility for drafting of this manuscript. AB participated in trial design and led testing and implementation of trial procedures at the Colorado site. RR participated in trial design and led testing and implementation of trial procedures at the Minnesota site. JR organized the implementation of the Care Management intervention. BK organized the implementation of the Skills Training intervention. DK led pilot-testing and co-led training and supervision for the Care Management intervention. LS co-led training and supervision for the Skills Training intervention EL co-led training and supervision for the Skills Training and Care Management interventions. RP participated in trial design and the development of informatics specification and tools. SS participate in trial design and led the development of sample size estimation and analysis plans. UW led the development and pilot testing of the Skills Training intervention. All authors participated in critical revision of this manuscript for intellectual content. All authors read and approved the final manuscript.
Additional authors’ information
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
All study procedures were reviewed and approved by the Institutional Review Boards at Group Health Cooperative, HealthPartners, and Kaiser Permanente of Colorado. All three boards approved a waiver of consent for research use of health records, a waiver of consent for randomization of all eligible participants to usual care or intervention conditions, and an abbreviated consent procedure for receipt of intervention programs.
About this article
Cite this article
Simon, G.E., Beck, A., Rossom, R. et al. Population-based outreach versus care as usual to prevent suicide attempt: study protocol for a randomized controlled trial. Trials 17, 452 (2016). https://doi.org/10.1186/s13063-016-1566-z
- Pragmatic trial
- Care management
- Consent waiver