Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Population-based outreach versus care as usual to prevent suicide attempt: study protocol for a randomized controlled trial

  • Gregory E. Simon1Email author,
  • Arne Beck2,
  • Rebecca Rossom3,
  • Julie Richards1,
  • Beth Kirlin1,
  • Deborah King1,
  • Lisa Shulman1,
  • Evette J. Ludman1,
  • Robert Penfold1,
  • Susan M. Shortreed1 and
  • Ursula Whiteside1
Trials201617:452

https://doi.org/10.1186/s13063-016-1566-z

Received: 22 April 2016

Accepted: 20 August 2016

Published: 15 September 2016

Abstract

Background

Suicide remains the 10th-ranked most frequent cause of death in the United States, accounting for over 40,000 deaths per year. Nonfatal suicide attempts lead to over 200,000 hospitalizations and 600,000 emergency department visits annually. Recent evidence indicates that responses to the commonly used Patient Health Questionnaire (PHQ9) can identify outpatients who are at risk of suicide attempt and suicide death and that specific psychotherapy or Care Management programs can prevent suicide attempts in high-risk patients. Motivated by these developments, the NIMH-funded Mental Health Research Network has undertaken a multisite trial of two outreach programs to prevent suicide attempts among outpatients identified by routinely administered PHQ9 questionnaires.

Methods/design

Outpatients who are at risk of suicide attempt are automatically identified using data from electronic health records (EHRs). Following a modified Zelen design, all those identified are assigned to continued usual care (i.e., no contact) or to be offered one of two population-based outreach programs. A Care Management intervention includes systematic outreach to assess suicide risk, EHR-based tools to implement risk-based care pathways, and care management to facilitate recommended follow-up. A Skills Training intervention includes interactive online training in Dialectical Behavior Therapy skills, supported by reminder and reinforcement messages from a skills coach. Each intervention supplements, rather than replaces, usual care; participants may receive any other services normally available. Interventions are delivered primarily by secure messaging through EHR patient portals. Suicide attempts and deaths following randomization are identified using state vital statistics data and health system EHR and insurance claim data. Primary evaluation will compare risk of suicide attempt or death over 18 months according to the initial assignment, regardless of intervention participation. Recruitment is underway in three health systems (Group Health Cooperative, HealthPartners, and Kaiser Permanente Colorado). Over 2500 participants have been randomized as of 1 March 2016, with enrollment averaging approximately 100 per week.

Discussion

Assessing the effectiveness of population-based suicide prevention requires adherence to the principles of pragmatic trials: population-based enrollment, accepting variable treatment participation, assessing outcomes using health record data, and analyses based on intent-to-treat.

Trial registration

ClinicalTrials.gov registration #NCT02326883, registered on 23 December 2014.

Keywords

Suicide Prevention Pragmatic trial Care management Consent waiver

Background

Suicide remains the 10th-ranked most frequent cause of death in the United States, accounting for over 40,000 deaths per year [1]. Nonfatal suicide attempts lead to over 200,000 hospitalizations and 600,000 emergency department visits each year [2, 3]. In contrast with other common causes of death, suicide mortality has not decreased over the last 25 years.

While prevention of suicide attempts and suicide death is a public health priority, existing evidence does not clearly support selective or secondary prevention programs. In 2013, the US Preventive Services Task Force found insufficient evidence to support the benefits of screening for suicide risk in general medical outpatients [4]. That review found insufficient evidence both for the accuracy of screening tests and for the effectiveness of interventions in those identified by screening.

More recent evidence indicates that responses to the commonly used Patient Health Questionnaire (PHQ9) can identify outpatients who are at increased risk of suicide attempt and suicide death [5, 6]. Patients reporting frequent thoughts of death or self-injury (the ninth item of the PHQ9) show a sustained increase in risk, with cumulative hazard approaching 4 % over 12 months. Reflecting this new evidence, the Joint Commission recently issued a Sentinel Event Alert [7] regarding detection of suicidal ideation in health care settings.

In addition, growing evidence supports the effectiveness of tertiary or indicated prevention interventions for high-risk patients. Structured psychotherapy emphasizing specific behavioral and cognitive skills has been proven to reduce risk among people who have made recent suicide attempts [810]. Outreach and Care Management programs appear to reduce risk among people who have made recent suicide attempts or high-risk patients treated in mental health specialty clinics [11]. This evidence for tertiary prevention suggests that similar interventions could reduce risk in the broader (secondary prevention) population of patients who are experiencing frequent suicidal ideation.

Motivated by these developments, the National Institute of Mental Health-funded Mental Health Research Network has undertaken a multisite trial of two population-based programs to prevent suicide attempts among outpatients identified by routinely administered depression questionnaires. Both programs include systematic outreach and regular supportive contact. One focuses on risk assessment and care management [11], while the other includes online training in specific skills from Dialectical Behavior Therapy (DBT) [12]. This pragmatic trial will examine whether either program can reduce long-term risk compared to care as usual.

Methods

Overview

At participating health systems, outpatients who are at increased risk of suicide attempt are identified using data from electronic health records (EHRs). Following a modified Zelen [1315] design, all those identified are automatically assigned to continue in usual care (i.e., no contact) or to be offered one of two population-based prevention programs:
  • A Care Management intervention including: systematic outreach to assess risk of suicidal behavior, EHR-based tools to implement risk-based care pathways, and care management to facilitate and monitor recommended follow-up care

  • A Skills Training intervention including: interactive online training in DBT skills [12], supported by reminder and reinforcement messages from a skills coach

Each intervention is intended to supplement, rather than replace, usual care provided by specialty mental health or primary care providers. Participants in all three treatment groups are free to receive any other services that are normally available, including pharmacotherapy, individual or group psychotherapy, or inpatient care. Intervention services are delivered primarily by online secure messaging through EHR patient portals [16, 17]. Nonfatal and fatal suicide attempts following randomization are identified using state vital statistics data and diagnoses of self-inflicted injury from health system clinical and insurance claim records [18, 19]. Primary evaluation will compare risk of first suicide attempt (nonfatal or fatal) over the 18 months following randomization. Groups will be compared according to initial treatment assignment, regardless of level of participation in either intervention program.

Study settings

The study sites include three members of the Mental Health Research Network: Group Health Cooperative, HealthPartners, and Kaiser Permanente of Colorado. These health systems provide general medical and mental health specialty care as well as insurance coverage to defined member/patient populations. Patients served are representative of each health system’s geographic service area in terms of race, ethnicity, educational attainment, and household income.

All three participating health care systems recommend the routine use of the PHQ9 depression questionnaire [20] at all mental health specialty visits and all primary care visits for treatment of depression [21]. All three sites participated in previous research [5] demonstrating that the response to item 9 of the PHQ9 (regarding thoughts of death or self-harm) predicts markedly elevated risk of suicide attempt over the following 18 months (Fig. 1).
Fig. 1

Trial flow chart

Eligibility

Eligibility criteria for automatic inclusion in the trial include:
  • Completion of a PHQ9 questionnaire [20] at an outpatient mental health or general medical visit

  • Age 18 years or older on the visit date

  • Response of “more than half the days” or “nearly every day” to item 9 of the PHQ9

  • Use of EHR patient portal secure messaging during the prior year

  • Currently enrolled in participating health system insurance plan (to ensure adequate ascertainment of subsequent suicide attempts)

Exclusion criteria include:
  • Recorded diagnosis of dementia or developmental delay in the previous 2 years

  • Limited English proficiency (as indicated by “need for interpreter” recorded in the EHRs)

  • Previous request to be excluded from research invitations

  • Already enrolled and randomized via a previous visit

Enrollment and randomization

Each week, EHR and insurance claim databases at each study site are used to identify all patients who meet eligibility criteria during the previous week. Immediately after sampling, all eligible patients are randomly assigned in equal proportions (1/3:1/3:1/3) to continue in usual care or to be offered one of the two intervention programs. At each site, randomization occurs automatically within the sampling computer program, stratified by eligible PHQ9 response (“more than half the days” or “nearly every day”) and site. A computer-generated concealed allocation table at each site provides randomly generated assignments in block sizes of either six or nine.

Invitation and consent

Participants assigned to the Care Management intervention receive an initial invitation message from the study care manager via the EHR-based online secure messaging system [16, 17]. This invitation includes a brief description of the Care Management program and abbreviated information regarding required elements of informed consent (study purpose, study procedures, potential risks or harms, and right to refuse or withdraw at any time). Each participant can decline participation by replying to the invitation message or can consent to receive intervention services by either replying to the message or returning the attached risk assessment questionnaire. Participants who neither decline nor consent receive a reminder (either by online message or telephone message) after 1 week. Participants who neither decline nor consent after that reminder message receive a second invitation message (with possible reminder) 4 weeks later and may receive a third invitation message after an additional 4 weeks. Those who decline at any point are not contacted again. Those who do not respond after three cycles of invitation are not contacted again.

The invitation and consent process for participants assigned to the Skills Training program parallels that for the Care Management program: an invitation message including abbreviated informed consent information and up to three cycles of invitation and reminder for those not responding. A participant can consent to receive intervention services by replying to an invitation message or by making an initial visit to the online intervention program. Those who decline at any point are not contacted again. Those who do not respond after three cycles of invitation are not contacted again.

Participants assigned to continue in usual care are never contacted by study staff. Providers are not notified regarding participants’ assignment to continued usual care.

Care Management intervention

Rationale

Following the design of previous Care Management interventions [17], including the Perfect Depression Care program at Henry Ford Health System [11], this program aims to reduce risk of suicide attempt by monitoring and maintaining engagement in effective mental health treatment.

Assessment tools

In collaboration with developers of the Columbia Suicide Severity Rating Scale [22] (CSSRS), study investigators developed a simplified CSSRS for self-report administration via online secure messaging. This abbreviated CSSRS provides a 6-point ordinal rating of current suicide risk based on frequency and intensity of suicidal thoughts, presence and specificity of suicidal plans, and clarity of suicidal intent during the last week. For example, a score of 0 would indicate no recent thoughts of self-injury or suicide, a score of 3 would indicate suicidal ideation with some recent thoughts about specific means, and a score of 5 would indicate a current and specific suicidal plan.

Follow-up algorithms

In collaboration with health system stakeholders (see below), investigators developed rules for risk-based care pathways specifying appropriate level of care, minimum standards for follow-up visit intervals and timing of outreach messages. For example: a CSSRS score of 1 would lead to a recommendation for follow-up within 1 month (sooner if clinically appropriate) in either primary care or specialty mental health care, a score of 3 would lead to a recommendation for follow-up in specialty mental health care within 2 weeks, and a score of 5 would lead to a recommendation for specialty mental health follow-up within two business days (or sooner as clinically indicated).

Care manager role

At each site, one or more care managers are responsible for:
  • Initial and follow-up invitations to all participants assigned to be offered Care Management

  • Periodic outreach to assess current risk (using an online version of the CSSRS)

  • Application of follow-up algorithms, supported by informatics tools (see below)

  • Regular feedback to treating providers regarding risk assessments and follow-up plans

  • As-needed communication with participants and providers to facilitate follow-up care

Care managers communicate with participants primarily by online secure messaging through EHR patient portals, but may communicate by telephone as needed. Care managers are expected to consider individual patients’ clinical circumstances and providers’ treatment plans when applying algorithms regarding outreach and visit frequency. At all sites, care managers are Master’s-prepared mental health clinicians.

Informatics tools

Intervention delivery is supported by existing functions of health system EHRs:
  • Online patient-provider secure messaging via the EHR patient portal [16, 17] for invitation and outreach to participants

  • Online administration of structured questionnaires such as the CSSRS

  • Secure provider-to-provider messaging for care managers’ communications with primary care and mental health specialty providers

  • Population management and reporting tools to apply follow-up algorithms and deliver algorithm-based recommendations to care managers regarding outreach and follow-up

Engagement with health system stakeholders

During the pilot phase, regular meetings with clinical leaders from all sites (representing both primary care and mental health specialty care) developed consensus regarding content and workflow of the Care Management program, including:
  • Language for outreach messages

  • Recommended follow-up intervals

  • Criteria for referral from primary care to mental health specialty care

  • Processes for communicating with primary care and mental health specialty providers

  • Procedures for urgent assessment and referral

Training and supervision

Training of care managers across sites was conducted by videoconference and teleconference, led by clinical investigators from the Group Health site. Initial training included:
  • 6 h of clinical training regarding suicide risk assessment

  • 2 h of general orientation to project aims and procedures

  • 6 h of specific training regarding care management aims, tools, and procedures

Ongoing teleconference supervision for all care managers is led by clinical investigators from the Group Health site. Supervision meetings were scheduled weekly for 6 months and twice monthly thereafter. Consistent with the principles of pragmatic trials [23], no detailed monitoring of intervention fidelity (e.g., review of content of online messaging or phone contacts) is conducted.

Skills Training intervention

Rationale

Following the content and structure of proven Dialectical Behavior Therapy (DBT) treatments [8, 12], this program aimed to reduce risk of suicide attempt through training in specific DBT skills shown to mediate the beneficial effects of traditional in-person DBT [12].

Specific skills content

The online program and coaching support focuses on four specific skills:
  • Mindfulness – Introduction to mindfulness skills, emphasizing nonjudgmental observation

  • Mindfulness of Current Emotion – Nonjudgmental observation of sensations associated with difficult or painful emotions

  • Opposite Action – Acting in opposition to urges associated with painful or difficult emotions

  • Paced Breathing – Use of breathing techniques to manage overwhelming emotions or crises

Adaptation for online delivery

The web-based interactive program includes an introductory section, personal profiles of team members (including coaches and contributing peer experts described below), and four skill modules (one for each DBT skill listed above). Each module includes:
  • A brief video introduction to the skill concept

  • A longer teaching video describing the skill, including in vivo practice

  • Example videos of peers (see below) describing use of the skill in daily life

  • Interactive exercises for use during the online session

  • Customizable worksheets to support between-session practice

Each participant is free to visit skills modules in any order and use the program at any pace, returning as frequently as desired. Coaches send reinforcement and outreach messages (see below) to encourage regular use.

Skills coach role

At each site, one or more skills coaches is responsible for:
  • Initial and follow-up invitations to all participants assigned to be offered Skills Training

  • For participants visiting the online program, messages to reinforce use of the program and practice of specific skills

  • For participants not visiting the program, periodic outreach messages to encourage return visits

  • As-needed communication with treating providers regarding participants’ progress

Skills coaches communicate with participants primarily by online secure messaging through EHR patient portals, but may communicate by telephone as needed. At all sites, skills coaches are Master’s-prepared mental health clinicians.

Informatics tools

The online program is delivered through the DatStat survey platform (DatStat Inc., Seattle, WA, USA). This platform supports secure access, detailed tracking of participant activity, and participant-level reports to guide the timing and content of coaches’ reinforcement and reminder messages. Participants access the online program via secure personalized links embedded in messages from skills coaches.

Engagement with patient stakeholders

Peer experts (people with lived experience of suicidal ideation and suicide attempts) were essential collaborators in the development of the Skills Training intervention and continue to support intervention delivery [24]. Design of outreach messages and content of the online program were informed by anonymous online surveys and focus group interviews with peer experts [24]. People with lived experience contributed video descriptions of the use of DBT skills, and continue to contribute to development of training and support materials for skills coaches.

Training and supervision

Training of skills coaches was conducted by videoconference and teleconference, led by clinical investigators from the Group Health site. Initial training included:
  • 6 h of clinical training regarding suicide risk assessment

  • 2 h of general orientation to project aims and procedures

  • 6 h of specific training regarding skills coaching aims, tools, and procedures

Ongoing teleconference supervision for all skills coaches is led by clinical investigators from the Group Health site. Supervision meetings were scheduled weekly for 6 months and twice monthly thereafter. Consistent with the principles of pragmatic trials [23], no detailed monitoring of intervention fidelity (e.g., review of content of online messaging or phone contacts) is conducted.

Outcome definitions

The primary study outcome is the time to first suicide attempt following randomization. Fatal suicide attempts will be identified by death certificate diagnoses of self-inflicted injury or poisoning. All three study sites routinely link membership files to state vital record data to ascertain cause of death for all enrolled members. Nonfatal suicide attempts will be identified from EHRs (for care delivered by participating health systems) and insurance claim data (for care received outside of participating health systems) using three criteria:
  • Any outpatient or inpatient diagnosis of definite self-inflicted injury or poisoning

  • Any outpatient or inpatient diagnosis of possible self-inflicted injury or poisoning

  • Any outpatient or inpatient diagnosis of other injury or poisoning associated with a diagnosis of suicidal ideation during the same encounter

For these three criteria, review of full-text medical records documented high positive predictive value for self-inflicted injury with suicidal intent [18, 25]. Because this validation work was completed prior to health care systems’ transition from the International Classification of Diseases, version 9 (ICD-9) to ICD-10 diagnoses, additional work will be necessary in 2016 to revalidate outcome definitions based on ICD-10 diagnostic codes. Given that participants may seek care for self-injury at external facilities, ascertainment will include both insurance claim and EHR data, and the sample is limited to patients who are enrolled in a health system insurance plan.

Analysis plan

Primary analyses will use the log-rank test to compare risk of diagnosed suicide attempt (defined above) over 18 months following randomization. For each intervention condition, risk among those assigned to the intervention will be compared to risk among those assigned to usual care – regardless of level of participation in either intervention. Individuals will be censored at time of health system disenrollment, death from cause other than suicide, or administratively, at 18 months following randomization. We evaluate the effect of each of the interventions compared to usual care using a log-rank test stratified by site and initial response to PHQ9 item 9 (2 versus 3). Sensitivity analyses will use weighted log-rank tests to account for a possible association between prerandomization characteristics (ascertained from EHRs) and censoring. In censoring weights, we will include sex, age group (18–29, 30–64, and 65 or more), race/ethnicity (Black American, Asian American, Hispanic, other), and visit type at which the initial PHQ9 questionnaire was completed (primary care versus mental health specialty).

Sample size

Original sample size estimates were based on previous research [5] suggesting an approximately 4 % risk of suicide attempt over 18 months among those meeting study eligibility criteria. Consultation with health system stakeholders indicated that implementation of a systematic outreach program was unlikely unless a program could be expected to reduce that risk to 3 % (relative risk reduction of 25 %). We used PASS software [26] to estimate the sample size required for a log-rank test [27] with 90 % power to compare the survival curves assuming a 3.8 % risk over 18 months in the comparison group and a 25 % risk reduction in the intervention group. We assumed 2 % disenrollment rate each month, resulting in approximately 25 % censorship over the 18 months of follow-up. The primary comparison for this trial is each of the interventions compared to usual care; we use a Bonferroni correction to account for the two tests in our primary analysis. Assuming a two-sided log-rank test, with a type-1 error rate of 0.025 and 90 %, we plan on enrolling 6500 patients per arm (total n = 19,500).

Enrollment progress

The trial was funded through the NIH Health Care Systems Collaboratory as one of the initial Pragmatic Clinical Trials Demonstration Projects [28]. Following a pilot phase to validate outcome definitions and to demonstrate the feasibility and acceptability of the intervention programs, participant enrollment and randomization began at the Group Health site in March 2015, expanding to three sites in July 2015. Approximately 4000 participants have been enrolled and randomized as of 1 July 2016. Approximately 100 participants are enrolled and randomized each week, and that rate is expected to increase to approximately 150 in the fall of 2016.

Ethical and regulatory approval

Study design and procedures were reviewed and approved by Institutional Review Boards at all three health system. That review process addressed several issues common to pragmatic trials of prevention interventions.

Waiver of informed consent

Limiting a randomized trial of outreach to those who actively consent to receive outreach would yield a result of questionable validity and generalizability. Consequently, a modified Zelen design [1315], randomizing all eligible patients without first obtaining consent, is necessary for valid test of the study question. This design, however, requires a waiver of the usual requirement for informed consent prior to enrollment or randomization. While it is not practicable to obtain informed consent prior to randomization, it is practicable to provide appropriate information to participants at the time intervention services are offered. As described above, invitation messages to participants assigned to either intervention include a brief description of the study purpose, study procedures, potential risks, and the right to decline participation. This design, therefore, includes a waiver of consent for enrollment/randomization and a modified consent procedure for receipt of intervention services.

Defining minimal risk

Current regulations regarding protection of human research participants allow waiver of consent for research involving no more than minimal risk. Our proposal to waive the requirement for informed consent in patients who are at risk of suicide attempt led to extensive discussions with health system Institutional Review Boards, leadership of the NIH Healthcare Systems Research Collaboratory, and the federal Office for Human Research Protections [29]. Those discussions helped to clarify four issues:
  • Research risk versus preexisting risk – Given that the trial enrolls patients who are at risk of suicide attempt, we encountered concern that study procedures could not be classified as having minimal risk. To address this concern, we relied on regulatory guidance distinguishing between preexisting risk due to a research participant’s health state (i.e., increased risk of suicide attempt) and incremental risk created by study procedures. This distinction led to the appreciation of this trial as evaluating minimal-risk interventions in a high-risk population

  • Risk of assignment to continued usual care – We also encountered concern regarding the ethical acceptability of randomly assigning patients who are at risk of suicide attempt to a usual care control condition. We clarified that a participant assigned to usual care will, by definition, receive the same treatment that she or he would have received if the study were not occurring

  • Risk of assignment to offer of intervention programs – We also encountered concern that assignment to either intervention group might increase risk. Both intervention programs are based on effective interventions and are intended to reduce risk of suicide attempt. Participants are free to receive any other services that are normally available. Nevertheless, it is possible that some participants will experience negative effects from either program. We addressed this concern by clarifying that participants are assigned to the offer of an intervention, with no obligation to participate. Invitation messages clearly identify intervention programs as research activities, make no promise of benefit, and advise that participation is completely voluntary

  • Intrusiveness or invasion of privacy – Different stakeholders expressed concern regarding both inappropriate intrusiveness of repeated outreach and inadequate vigor of outreach given the known increased risk of suicide attempt. In consultation with peer experts, we designed the outreach strategy described above, including up to three cycles of invitation, as a reasonable compromise between these two concerns.

At all participating health systems, notices regarding privacy practices specifically advise members regarding the use of health records for research. Members who have previously requested exclusion from research contact are excluded from the study sample.

Monitoring for adverse events

In most clinical trials of mental health treatments, a suicide attempt would be considered a serious adverse event, subject to immediate reporting and review to determine the “relatedness” of an individual suicide attempt to study participation. This traditional approach was clearly not appropriate for a large-scale trial of population-based prevention programs [30]. First, record data regarding suicide attempts may not be available for 3 months or more following an event. Second, several hundred suicide attempts are expected to occur among study participants, and review of individual events could not determine causal relationship to study participation. While it is possible that either intervention could paradoxically increase risk of suicide attempt, that could only be determined by comparison to suicide attempt rates in usual care (see below).

Interim analyses of benefit or harm

We do not plan any interim analyses to evaluate benefit of the intervention programs. First, early detection of a benefit of either intervention is extremely unlikely. We project that randomization will be complete before complete outcome data are available for half of the participants. Second, early termination of randomization or intervention delivery would not offer any additional protection to current or future study participants. Premature termination would instead return all current and potential participants to care as usual.

We do, however, plan interim analyses testing for evidence of significant harm (increased risk of suicide attempt) in either intervention group compared to usual care. Clear evidence that either intervention resulted in significantly increased risk of suicide attempt would certainly warrant suspending assignment of patients to that program or suspending delivery of that program to participants already assigned. Interim analyses comparing risk of suicide attempt in each intervention group to that in usual care will be conducted three times per year, beginning 12 months after start of enrollment. Interim analyses will be reported to the National Institute of Mental Health Data and Safety Monitoring Board by the study statistician, but all other study staff will be blinded to these results.

Data and resource sharing

A deidentified version of the analytic dataset will be made available at the time of the initial publication of primary study findings. Consistent with policies of the NIH Collaboratory, all resources (intervention materials, specifications, computer code, etc.) will be shared at or before the publication of study results.

Discussion

This suicide prevention outreach trial addresses a practical question that is relevant to practicing clinicians or health system leaders: Will population-based outreach programs reduce risk of suicide attempt among patients identified as being at risk by routinely administered depression questionnaires? This focus on a practical or pragmatic question has several implications for trial design. Table 1 describes how specific aspects of this trial conform to the characteristics of pragmatic trials described by Thorpe [31]. Central pragmatic trial features include:
Table 1

PRECIS domains defining pragmatic trials

 

PRECIS criteria for pragmatic trials

Design of suicide prevention outreach trial

Participants

All eligible participants enrolled, regardless of risk, responsiveness, comorbidities or past compliance

Adult health plan members reporting frequent suicidal ideation on routine depression questionnaires are automatically enrolled

Intervention condition

Interventions are highly flexible, offering providers leeway in formulation and application

Both interventions allow personalization to patients’ needs and preferences. Varying levels of participation are expected

Intervention practitioners

Interventions are applied by the full range of practitioners in the full range of settings with only ordinary attention to dose and side effects

Intervention clinicians will be recruited from existing local workforces. Each site will be responsible for selection and supervision of clinicians (using standard quality control tools)

Comparison condition

“Usual practice” (or the best alternative), offering practitioners considerable leeway in application

Each prevention program will be compared to usual care

Comparison practitioners

The control intervention is applied by the full range of clinicians in the full range of settings, with only ordinary attention to training, experience, and performance

Usual care will be provided by real-world providers (mental health and general medical clinicians) under usual practice conditions – with no additional training or supervision

Follow-up assessments

There are no research assessments; administrative databases are searched for outcomes

All outcome data are collected from EHR, insurance claim data, and death certificate data

Outcome definition

The primary outcome is objectively measured, meaningful to study participants, and does not depend on central adjudication

Primary and secondary outcomes are defined by specific ICD-9/ICD-10 diagnosis codes – no clinical assessment is required

Intervention compliance

There are no special strategies to improve compliance, and compliance is unobtrusively measured

Patients assigned to interventions are free to participate (or not participate) at any level. Participation or compliance is assessed passively using EHRs and online intervention databases

Practitioner adherence

There are no special strategies to maintain practitioner adherence, and adherence is unobtrusively measured

Care managers and skills coaches work independently at each site, but receive initial training and regular supervision from study investigators

Primary comparison

The analysis includes all patients regardless of compliance, eligibility, or others

All outcomes will be analyzed according to initial assignment – regardless of intervention participation or compliance

EHR electronic health record, ICD International Classification of Diseases

  • Population-based enrollment – If we hope to inform policy or implementation decisions by health system leaders, then it is necessary to evaluate program effectiveness in the full population of those to whom the program would eventually be offered

  • Allowing variable participation or compliance – Restricting enrollment to those willing to participate in outreach or prevention programs would not allow a valid assessment of program effectiveness. Low participation or high rates of dropout should be considered essential indicators of effectiveness rather than threats to internal validity

  • Analysis by intent-to-treat – A valid evaluation of prevention program effectiveness must examine risk among all those offered the prevention service, rather than those who accept or participate. Any “as treated” or “completers” analysis (limited to those who participate in prevention services) would certainly be biased

Our trial differs from a purely pragmatic design in one aspect: the training and supervision of clinical staff delivering the prevention interventions. All care managers and skills coaches complete approximately 14 h of initial training followed by weekly or bi-weekly supervision teleconferences. This training and supervision was necessary because both of these clinical roles required implementation of new clinical work processes and the use of new informatics tools. If either program is proven effective, we would recommend that any subsequent implementation include a similar level of training as well as a period of regular supervision.

Assessing the effectiveness of any population-based prevention program requires a clinical trial following the core principles of pragmatic trials: population-based enrollment, accepting variable treatment participation, assessing outcomes using health record data, and analyses based on intent-to-treat. We describe the design and implementation of such a trial, now underway in three large integrated health systems.

Trial status

Enrollment is ongoing and is expected to be complete in early 2018.

Abbreviations

CSSRS: 

Columbia Suicide Severity Rating Scale

DBT: 

Dialectical Behavior Therapy

EHR: 

Electronic health record

PHQ9: 

Patient Health Questionnaire depression scale

Declarations

Acknowledgements

Drs. Brian Ahmedani, Marsha Linehan, Anita Lungu, and Maria Oquendo contributed to the development and pilot-testing of intervention programs.

Funding

Funded by Grant UH3 MH007755 from the NIH Common Fund Health Care Systems Research Collaboratory and the National Institute of Mental Health.

Availability of supporting data

Outcome data will not be available until 12 months following completion of recruitment. A deidentified analytic dataset will be made available for public use following publication of the main results.

Authors’ contributions

GS led the design of the trial and pilot testing of interventions and had primary responsibility for drafting of this manuscript. AB participated in trial design and led testing and implementation of trial procedures at the Colorado site. RR participated in trial design and led testing and implementation of trial procedures at the Minnesota site. JR organized the implementation of the Care Management intervention. BK organized the implementation of the Skills Training intervention. DK led pilot-testing and co-led training and supervision for the Care Management intervention. LS co-led training and supervision for the Skills Training intervention EL co-led training and supervision for the Skills Training and Care Management interventions. RP participated in trial design and the development of informatics specification and tools. SS participate in trial design and led the development of sample size estimation and analysis plans. UW led the development and pilot testing of the Skills Training intervention. All authors participated in critical revision of this manuscript for intellectual content. All authors read and approved the final manuscript.

Additional authors’ information

None.

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

All study procedures were reviewed and approved by the Institutional Review Boards at Group Health Cooperative, HealthPartners, and Kaiser Permanente of Colorado. All three boards approved a waiver of consent for research use of health records, a waiver of consent for randomization of all eligible participants to usual care or intervention conditions, and an abbreviated consent procedure for receipt of intervention programs.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Group Health Research Institute
(2)
Institute for Health Research, Kaiser Permanente Colorado
(3)
HealthPartners Institute

References

  1. National Center for Health Statistics: Deaths: final data for 2013. In: National Vital Statistics Reports. vol. 64. Hyattsville 2015.Google Scholar
  2. National Hospital Ambulatory Medical Care Survey: 2008 Emergency Department Summary Tables. http://www.cdc.gov/nchs/data/ahcd/nhamcs_emergency/2011_ed_web_tables.pdf. Accessed 11 Sept 2016.
  3. Crosby AE, Han B, Ortega LAG, Parks SE, Gfroerer J. Suicidal thoughts and behaviors among adults aged ≥18 years - United States, 2008–2009. MMWR Morb Mortal Wkly Rep. 2011;60(13):1–31.Google Scholar
  4. O’Connor E, Gaynes BN, Burda BU, Soh C, Whitlock EP. Screening for and treatment of suicide risk relevant to primary care: a systematic review for the U.S. Preventive Services Task Force. Ann Intern Med. 2013;158(10):741–54.View ArticlePubMedGoogle Scholar
  5. Simon GE, Coleman KJ, Rossom RC, Beck A, Oliver M, Johnson E, Whiteside U, Operskalski B, Penfold RB, Shortreed SM, et al. Risk of suicide attempt and suicide death following completion of the Patient Health Questionnaire depression module in community practice. J Clin Psychiatry. 2016;77(2):221–7.View ArticlePubMedPubMed CentralGoogle Scholar
  6. Simon GE, Rutter CM, Peterson D, Oliver M, Whiteside U, Operskalski B, Ludman EJ. Does response on the PHQ-9 Depression Questionnaire predict subsequent suicide attempt or suicide death? Psychiatr Serv. 2013;64(12):1195–202.View ArticlePubMedPubMed CentralGoogle Scholar
  7. Patient Safety Advisory Group: detecting and treating suicidal ideation in all settings. The Joint Commission Sentinel Event Alerts 2016;56:1-7.Google Scholar
  8. Comtois KA, Linehan MM. Psychosocial treatments of suicidal behaviors: a practice-friendly review. J Clin Psychol. 2006;62(2):161–70.View ArticlePubMedGoogle Scholar
  9. Linehan MM, Comtois KA, Murray AM, Brown MZ, Gallop RJ, Heard HL, Korslund KE, Tutek DA, Reynolds SK, Lindenboim N. Two-year randomized controlled trial and follow-up of dialectical behavior therapy vs therapy by experts for suicidal behaviors and borderline personality disorder. Arch Gen Psychiatry. 2006;63(7):757–66.View ArticlePubMedGoogle Scholar
  10. Rudd MD, Bryan CJ, Wertenberger EG, Peterson AL, Young-McCaughan S, Mintz J, Williams SR, Arne KA, Breitbach J, Delano K, et al. Brief cognitive-behavioral therapy effects on post-treatment suicide attempts in a military sample: results of a randomized clinical trial with 2-year follow-up. Am J Psychiatry. 2015;172(5):441–9.View ArticlePubMedGoogle Scholar
  11. Hampton T. Depression care effort brings dramatic drop in large HMO population’s suicide rate. JAMA. 2010;303(19):1903–5.View ArticlePubMedGoogle Scholar
  12. Neacsiu AD, Rizvi SL, Linehan MM. Dialectical behavior therapy skills use as a mediator and outcome of treatment for borderline personality disorder. Behav Res Ther. 2010;48(9):832–9.View ArticlePubMedPubMed CentralGoogle Scholar
  13. Adamson J, Cockayne S, Puffer S, Torgerson DJ. Review of randomised trials using the post-randomised consent (Zelen’s) design. Contemp Clin Trials. 2006;27(4):305–19.View ArticlePubMedGoogle Scholar
  14. Zelen M. A new design for randomized clinical trials. N Engl J Med. 1979;300(22):1242–5.View ArticlePubMedGoogle Scholar
  15. Zelen M. Randomized consent designs for clinical trials: an update. Stat Med. 1990;9(6):645–56.View ArticlePubMedGoogle Scholar
  16. Ralston JD, Rutter CM, Carrell D, Hecht J, Rubanowice D, Simon GE. Patient use of secure electronic messaging within a shared medical record: a cross-sectional study. J Gen Intern Med. 2009;24(3):349–55.View ArticlePubMedPubMed CentralGoogle Scholar
  17. Simon GE, Ralston JD, Savarino J, Pabiniak C, Wentzel C, Operskalski BH. Randomized trial of depression follow-up care by online messaging. J Gen Intern Med. 2011;26(7):698–704.View ArticlePubMedPubMed CentralGoogle Scholar
  18. Simon GE, Savarino J. Suicide attempts among patients starting depression treatment with medications or psychotherapy. Am J Psychiatry. 2007;164(7):1029–34.View ArticlePubMedGoogle Scholar
  19. Lu CY, Stewart C, Ahmed AT, Ahmedani BK, Coleman K, Copeland LA, Hunkeler EM, Lakoma MD, Madden JM, Penfold RB, et al. How complete are E-codes in commercial plan claims databases? Pharmacoepidemiol Drug Saf. 2014;23:218–20.View ArticlePubMedGoogle Scholar
  20. Kroenke K, Spitzer RL, Williams JB, Lowe B. The Patient Health Questionnaire Somatic, Anxiety, and Depressive Symptom Scales: a systematic review. Gen Hosp Psychiatry. 2010;32(4):345–59.View ArticlePubMedGoogle Scholar
  21. Simon GE, Rossom RC, Beck A, Waitzfelder BE, Coleman KJ, Stewart C, Operskalski B, Penfold RB, Shortreed SM. Antidepressants are not overprescribed for mild depression. J Clin Psychiatry. 2015;76(12):1627–32.View ArticlePubMedPubMed CentralGoogle Scholar
  22. Posner K, Brown GK, Stanley B, Brent DA, Yershova KV, Oquendo MA, Currier GW, Melvin GA, Greenhill L, Shen S, et al. The Columbia-Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry. 2011;168(12):1266–77.View ArticlePubMedPubMed CentralGoogle Scholar
  23. Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, Tunis S, Bergel E, Harvey I, Magid DJ, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol. 2009;62(5):464–75.View ArticlePubMedGoogle Scholar
  24. Whiteside U, Lungu A, Richards J, Simon GE, Clingan S, Siler J, Snyder L, Ludman E. Designing messaging to engage patients in an online suicide prevention intervention: survey results from patients with current suicidal ideation. J Med Internet Res. 2014;16(2):e42.View ArticlePubMedPubMed CentralGoogle Scholar
  25. Simon G, Savarino J, Operskalski B, Wang P. Suicide risk during antidepressant treatment. Am J Psychiatry. 2006;163:41–7.View ArticlePubMedGoogle Scholar
  26. PASS 14 Power Analysis and Sample Size Software. In: Kaysville, Utah: NCSS; 2015Google Scholar
  27. Lakatos E. Sample sizes based on the log-rank statistic in complex clinical trials. Biometrics. 1988;44(1):229–41.View ArticlePubMedGoogle Scholar
  28. Johnson KE, Neta G, Dember LM, Coronado GD, Suls J, Chambers DA, Rundell S, Smith DH, Liu B, Taplin S, et al. Use of PRECIS ratings in the National Institutes of Health (NIH) Health Care Systems Research Collaboratory. Trials. 2016;17(1):32.View ArticlePubMedPubMed CentralGoogle Scholar
  29. Demonstration project ethics and regulatory documentation. https://www.nihcollaboratory.org/demonstration-projects/Pages/regulatoryethics.aspx. Accessed 11 Sept 2016.
  30. Data and safety monitoring in pragmatic trials. https://www.nihcollaboratory.org/Pages/Grand-Rounds-02-13-15.aspx. Accessed 11 Sept 2016.
  31. Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, Tunis S, Bergel E, Harvey I, Magid DJ, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. CMAJ. 2009;180(10):E47–57.View ArticlePubMedPubMed CentralGoogle Scholar

Copyright

© The Author(s). 2016

Advertisement