Table 2 RE-AIM outcomes

Reach

Degree to which target population is impacted

1. Number of eligible patients (% patients with a 7-day gap)

2. % of patients who did not opt out

3. Representativeness of study participants compared to overall patients within each respective health system

4. Reasons why patients decline

Study database derived from EHR clinical and pharmacy data

EHR data; also, brief phone interview with those who decline (if permission given on opt-out form?)

Effectiveness

Success of the intervention in changing patient outcomes

1. Improvement in medication adherence (PDC) and reduction in utilization/clinical outcomes/costs

2. Generalization (heterogeneity) of effects across patient subgroups

3. Unintended consequences—either positive or negative

Study database. Analytic plan for the primary outcome of interest is further discussed in the analytic plan

Adoption

Degree to which interventions are taken up by organizations, clinics, providers, and pharmacists

1. Records of clinics, physicians, and pharmacists approached and willingness to participate in the intervention

2. Clinic, physician, and pharmacist characteristics of those participating vs. not—if < 90% participate

3. Reasons for declining

Study database

Brief phone interviews with subset of those who decline

Implementation

Degree to which interventions are implemented as intended (fidelity).

(a) Adaptations made; (b) costs; and (c) contextual factors associated with outcomes

1. Among patients with gap, how many interventions were delivered per patient

2. Proportion and representativeness of those reached and by method (text message versus IVR)

3. Among patients in arm #4, proportion where AI chat bot was used, and the barrier identified

4. Mixed methods assessment of adaptions (see above)

5. Budget impact/cost of the program and replication costs (see below)

6. Qualitative interviews focused on PRISM factors of (1) organizational and participants characteristics, (2) intervention characteristics from the organizational (healthcare system and providers) and participants’ perspectives (i.e., patients), (3) implementation and sustainability infrastructure (training and support), and (4) external environment

1. Qualitative interviews

2. Study database

3. Health economics plan

Maintenance (sustainment)

Can the program be sustained over time? Across (a) settings and (b) patients

1. HCS Intent to continue or modify intervention following grant support

2. Patient medication adherence status 12 months after intervention is stopped

3. Can intervention be extended to other patient populations with different conditions and other settings?

1. Post-implementation qualitative interviews

2. Study database

3. D&I plan

4. Input from stakeholder panel