Table 4 Thematic responses and clarification by domain (N = 11)
Domain (N) | Interpretation and PI responses | Domain clarification |
|---|---|---|
Eligibility | 1. Less pragmatic due to additional effort needed to identify appropriate patients and to validate correct identification: “It’s a little more work to figure out patients who are chronically vented who are excluded…I just thought it was going to be very, very easy and you don’t have to think about it. But it turns out actually I have to have my staff validate it.” (PI) | Eligibility refers to the extent to which the trial population matches the population intended for the intervention. The issue of effort to engage participants is more relevant to Recruitment Path, which addresses whether effort to recruit participants is greater than for patient engagement in usual care |
2. Less pragmatic due to a higher proportion of patients excluded than originally anticipated: “Once we started applying the criteria, we recognized…there are some people who we’ve excluded and I think they’re for good reasons… we haven’t changed the criteria, it’s just that as we’ve been applying them, we realized that it excludes a larger percentage of people perhaps than we thought.” (PI) | For the Eligibility domain one should consider the extent to which trial participants are similar to those who would receive the intervention if it were part of usual care (rather than volume of participants excluded) | |
Setting | 3. Setting is similar to usual care. “The setting is really the identical setting to usual care. But I probably scored it a bit down [at T1] it’s very representative of a usual setting.” (PI) | Setting receives a more explanatory score if there is only a single center, or only a specialized trial or academic center. Multi-center trials can be rated 3–5 |
Flexibility-Delivery | 4. More pragmatic because clinician notification (re: eligibility) was more automated than anticipated. “When we were in the planning phase…not clear how we were going to notify the [___]. It turns out the hospital itself had an outside vendor trying to figure out actually how to link …that [mechanism] is part of the hospital infrastructure now.” (PI) | Resource requirements are addressed under the Organization; the issue of resources required to conduct the study is not relevant to intervention delivery or adherence |
Flexibility-Adherence | 5. More pragmatic because notification (re: patient eligibility) was more automated than anticipateda: “When we were in the planning phase, [it wasn’t] clear to us exactly how we were going to notify the ___ – ” (PI) | Resource requirements are addressed under the Organization (see above). This domain should not have been rated as there is no monitoring of patient adherence |
6. More pragmatic because no participants are excluded due to adherence: “We’re not excluding anybody based on adherence, but we are encouraging adherence and are providing feedback on adherence.” (PI) | This domain addresses how flexibly participants in the trial are monitored and encouraged compared to usual care. This domain is not applicable to 2 of the trials as there is no compliance issue after consent has been given. The domain should be left blank (unrated) | |
7. More pragmatic because the intervention is executed in emergency care and adherence is minimal: “Our intervention really is executed and then it’s done, so the adherence of it is actually very minimal and the remainder of care given thereafter is just standard of care.” (PI) | ||
8. Less pragmatic because there is no usual care comparison: “There’s no way to know what would happen in usual care because the intervention’s never been tried in usual care. But I would foresee if our results prove favorable that the implementation in the real world would be identical to what we’re testing.” (PI) | The issue of usual care comparison is relevant to Flexibility of Intervention Delivery rather than Adherence | |
Follow-up | 9. Less pragmatic as collection of follow-up requires more effort than anticipated: “In clinical care, one would not necessarily seek out follow-up on patients,…what made us think that it was less pragmatic was the manner by which you seek out that information.” (PI) | Does not apply to this domain, which is concerned only with burden of follow-up on the participants, not whether the follow-up data are routinely collected |
10. Less pragmatic as collection of follow-up is less automatic than anticipated: “I have to apply in a separate IRB to a statistics department to get that long term follow-up. And that requires linking of the patient’s data. So that’s just a little less automatic…more work for me. For patients it’s the same.” (PI) | This domain is concerned only with burden of follow-up on the participants, not burden on research team or effort needed to collect the follow-up data | |
Primary Analysis | 11. Less pragmatic because the primary outcomes are not a standard measure: “The analysis is a standard analysis one would do for this type of a trial, but [not] a standard comparison that one would make on a daily basis.” (PI) | Pragmatism of primary analysis is based only on the degree to which all data are included in the analysis of the primary outcome |