Table 2 Exemplar quotes for identified themes

1. Identification of relevant risks from trial participation and determination of what constitutes minimal risk

1.1 “[A] nother issue would be around things like adverse events. […] [T]hat’s another thing about pragmatic trials, relatively low risk […] but you’re putting it [the intervention] into a high-risk situation. […] [Y] our intervention is actually very low risk, but it’s a question of what do you do then about your adverse events? And I do know examples where — none of the ones I’ve been involved in — but examples where regulators have said you’ve got to collect all your adverse events which could be really, really expensive.” TE010, Methodologist, UK

1.2 “I think the, the biggest issue is why something is a risk. I would say it’s a why question. It’s sort of like saying […] both of these are standard treatments, for example, if you’re comparing standard treatments. And if you went to one doctor this would be an accepted treatment. You go to another doctor this is an accepted treatment it’s all kind of haphazard out in the real world anyway so all we’re doing is randomizing so where’s the risks? Right.[…] So it’s, it’s more of a question of conceptualizing how people are affected because of the pragmatic trial #1, and #2 because of that is there any change in their welfare prospect that’s directly attributable to the research? I think that’s the question that’s difficult to answer and not sufficiently explored.” TE018, Ethicist, USA

1.3 “And then there’s the issue of should we be going to treat some of them as minimal risk which, at least in the US context, opens up the possibility of doing an expedited review of a clinical trial.” TE014, Ethicist, USA

1.4 “So, so here [in Australia] it’s reasonably clear. Here it’s absolute risk, so our system defines risk not in terms of the likelihood or probability of the risk but instead purely in terms of the magnitude.” TE024, Ethicist, Australia

2. Determining when alternative consent practices are appropriate

2.1 “[T] he trial that we’re running now, we focused the argument of consent for basically giving consent to follow your data, to have access to your PHI [Personal Health Information] and not actually for the intervention because again, you were going to get one of these two interventions regardless. […] So, we focused our argument on that and we haven’t had any problems with the IRB side of that but what I found interesting was again the consent form’s still 9 pages. […] I find that kind of stuff really takes away from the patient receiving informed consent. They just end up with a huge stack of paper.” TE005, Clinical investigator, USA

2.2 “[T] raditional people tend to argue about whether you need to include the detailed long consent in written form versus a more abbreviated consent. The more adventuresome people argue about whether you need consent at all if it’s really answering a question about two modes of practice both of which are used commonly and are in play where there’s no particular reason to pick one over the other.” TE006, Knowledge user, USA

2.3 “I think one of the problems that doesn’t afflict investigator-initiated pragmatic studies that does afflict a commercially sponsored trial is the lack of a financial incentive for investigators to recruit patients. … [T] his is where a pragmatic trial is beneficial if you’re doing a large trial with limited data collection, you’re not paying investigators a lot of money so they can generate a huge surplus then that that to me is a good thing.” TE025, Clinical investigator, Australia

2.4 “I wonder if that [consent to data, not intervention] might not be the right way to think about this kind of trial [pragmatic trials]. So, it’s not that you’re necessarily consenting to whichever treatment practice is the opted-for treatment practice, I mean if both treatments are standard treatment that might be given in this case. Instead what you’re giving access to is your information and the results of that, how you’re feeling about it and all those kinds of things. I just think that might be a way of thinking about how you could do a consent process for this.” TE024, Ethicist, Australia

2.5 “And then there’s the issue of should we be going to treat some of them as minimal risk which, at least in the US context, opens up the possibility of doing an expedited review of a clinical trial which is pretty uncommon otherwise and opens up the possibility of a waiver or modification of informed consent on US regs.” TE014, Ethicist, USA,

2.6 “[Y] es, we still quite often face problems with consent. So especially in emergency departments if we’re needing to invoke or trying to invoke emergency consent we still get a lot of kickback about whether we need to go out for a proxy consent at that point in time or whether we must go and find family members who can consent. So there are layers of people who can consent on their behalf. And, you know, very often our argument has been that you need to include these people for whom this is potentially at that stage lifesaving, then we can’t wait. So there is still a lot of discussion around that and a lot of work needs to go into the crafting of the writing of the rationale for why we want to go ahead without consent.” TE008, Methodologist, UK

2.7 “And the concerning issues really revolve around the ability to waive consent if the intervention is time critical..” TE023, Clinical investigator, Australia

2.8 “I can easily see scandals occurring. But people really have to realize that, if a particular clinic or a research program screws up, they could say wow that was a problem with that researcher and so forth. If you do a pragmatic trial where the entire practice infrastructure of a healthcare system is involved and which endorses what happened [waiver of consent], then from a patient subject’s perspective they’re think this entire healthcare system conspired against me. [laugh] That could happen, you know, if something happened. So I think that I think it’s a big issue.” TE018, Ethicist, USA

2.9 “But my worry about all this kind of stuff where you’re doing stuff without getting consent and, and so on is that is that you’ll get these cases where we get used to a practice... Ethics committees become quite comfortable with it. It’s standard practice and then somebody says something to someone and somebody goes hey I’m in a research trial? What? I thought I was getting standard care? And if that’s not handled really, really carefully and it becomes a big media story it hurts recruitment for research. It hurts funding for research and so on.” TE024, Ethicist, Australia

2.10 “I think that the, the legal challenges, the ethical challenges, the consent challenges are often under played, you know. It’s recognized but you have the situation where you’ll have researchers having sort of this internal consensus […] and it’s almost like they think that well we [will] resolve this issue without understanding that there’s this legal framework outside, these legal norms outside, that you actually have to satisfy. That researchers or even research policy people can’t do it with a workshop [laugh] and you can’t do it with a consensus meeting. Those things are not going to resolve this issue, because the norms that determine how you’re supposed to proceed are actually designed or actually exist within liberal democracies, […] all it’s going to take is one controversy and the whole thing is gonna unravel. And we saw that with Henrietta Lacks in the United States. We saw that with the Texas blood spot situation where they destroyed was it 6, 7, 8 million samples as a result of a couple of families complaining. So, this matters, right … you can’t forge ahead on wishful thinking and a consensus of researchers. On the contrary you’ve got to resolve it more broadly.” TE021, Lawyer, Canada

3. The distinction between research, quality improvement, and practice

3.1 “However, there are huge regulations around anything that you label clinical trial. So, I can go off and behave in the most bizarre fashion as an independent clinician […], I get there will be peer review etc. but basically, I can do what I want. If I want to include people in a clinical trial I have to go through all manner of checks and balances which is fair enough when you’re studying experimental treatments. But when you’re doing more clinical effectiveness trials I think it’s possibly an ethical issue that these things are putting enormous barriers in the way of actually doing the research that needs to be done.” TE025, Clinical investigator, Australia

3.2 “[…] there’s this sort of double standard between—people are only interested in research ethics and not practice ethics. And, you know, doctors do unethical things all the time … [S] omeone once said in the New England Medical Journal: if I want to give half my patients a new treatment I have to get ethics committee approval. If I want to give all my patients a new treatment I don’t.” TE001, Methodologist, UK

3.3 “I’m quite sceptical about the kind of general case of “oh we need a completely new system”; “oh we need to completely change everything” and so on. I’m more happy with the kind of approach that places like Australia take where they just say it’s all research” TE024, Ethicist, Australia

3.4 “[T] he first thing is what kind of ethical framework [should be used with pragmatic trials]? Is it a research ethical framework? Is it a systems or healthcare improvement framework or is it both? […] So that’s for me the biggest question and the most important and the one that this project really needs to sort out clearly.” TE002, Clinical Investigator, Canada

3.5 “It’s true of any trial [that discussion happens between patients in and outside the trial] […] so the notion of the kind of often hard line like you can’t talk about this [research outside the trial]; [this distinction], makes no sense on the recruiting side from my perspective. […].” LM002, Patient partner, USA

4. The potential for broader populations to be affected by the trial and establishing what protections they might be owed

4.1 “I sometimes get the sense that, if you’re doing a trial on women with post-partum haemorrhage, people think it’s a vulnerable group ─ we’ve got to be extra vigilant, but actually what they do is by making it harder to enrol these women in clinical trials or making it longer (the assessment process), they actually disadvantage them instead of doing anything good for them.” TE001, Clinical Investigator, UK

4.2 “[W] here everybody is potentially involved, how do we make sure from a language barrier standpoint or an economic standpoint that people aren’t excluded because they’re not asking the right questions or they don’t understand the material. So I think the medical things [exclusion criteria] made perfect sense to me and I think we dealt with it more on an inclusion side than an exclusion side.” LM006, Patient partner, USA

4.3 “The one issue is, how do we trade off, or do we need to trade off, or even, do we need to think about the ethics of those in the trial, those in the control group versus those in the intervention group and those outside the trials who might one day be eligible for these things. […] I think it might be an idea to think about that question against a number of different groups: patients offered the intervention, patients receiving the intervention, patients in the control group, patients who might one day benefit from the information from this trial.” TE002, Clinical investigator, Canada

5. The broader range of trial stakeholders in pragmatic RCTs, and determining their roles and responsibilities

5.1 “I guess thinking more expansively, more recently there has been a lot of suggestion or proposals to increase the efficiency of trials by doing registry-based randomized trials, which is again an extension of the same practice [...] And who’s the guardian of that data and who should make these approaches and who should allow potentially a trial to sit overly on that dataset. So guardianship issues, permission issues, consent issues, approach issues, privacy issues.” TE008, Methodologist, UK

5.2 “[…] there have been trials that have kind of got both the scientific regulator and the payer’s points built in […] But the objectives I think are quite different. I mean the regulator is looking purely at efficacy and safety so the regulator’s coming to it from the more explanatory end. And the payer is looking at quality of life or qualities or effectiveness. That’s what’s coming into it in the more pragmatic end.” TE015, Knowledge user, UK

5.3 “The most interesting IRB ethics thing that I’ve been involved in over the last couple of trials is essentially this off label use […] the FDA or Health Canada they don’t necessarily want to review every single clinical study being done in the country, yet the rules are that if it’s off label they need to. And, at least in the US, the FDA has written very clear guidance documents that basically say the best people to understand whether this off-label use increases the risk to the patients are actually the practitioners and the local IRB. But the local IRBs still don’t want to take on all the liability per se so then they still make you go through the FDA. And that again has been very frustrating because that really is the pragmatic side, right?” TE005, Clinical investigator, USA

5.4 “The one thing that I am very clear on and my PPI group were very clear on was that we were not the medical side of the trial. We were the public trial and we took advice from the professors etc. as to what things like the placebo should be.” LM007, Community member, UK

6. Determining what constitutes “usual care” and implications for trial reporting

6.1 “[…] obviously it’s one of those things [usual care] where there’s no agreement and I think the most you can do is explain what you mean every time for a given situation. I mean the standard of care is actually more often used in the legal setting in terms of, if you did something wrong. If you’re accused of malpractice, you could say no, this was standard practice. This has been the standard within the community of physicians and for this kind of disorder and so forth. And then there might be instances where an intervention might not be used very often, in fact rarely used for purely practical or accidental historical reasons, but no one would think that that was substandard if you used it, right?” TE018, Ethicist, USA

6.2 “usual care is how patients are treated in routine practice if you can profile usually from EHR data sources. So, for example, a patient with diabetes in a given healthcare system, how frequently are they seeing their primary care doctor and their endocrinologist on a yearly basis? What proportion of them are on insulin or other diabetes agents? And how does that conform to recommended practice guidelines? So guidelines define some of that but your local care pattern is going to differ a little bit from the guideline. But you can define that and it will differ by region and environment.” TE007, Clinical investigator, USA

6.3 “I think that [usual care] fits in nicely with the whole pragmatic/explanatory thing. So on the pragmatic side my interpretation would be usual care is at the individual practitioner level […]And on the explanatory side usual care will be some national, regional standard that everybody can agree on and then therefore you can standardize usual care to sort of a standard of care.” TE005, Clinical investigator, USA

6.4 “Well usual care’s not a single thing. Usual care’s whatever that patient would have got in contact with whatever clinician they happened to be in contact with in whatever clinic they happened to be in. And so, the problem with usual care which explanatory trialists really struggle with and, and so they should and so should we as more pragmatic trialists, is the care that the control group gets in a usual care trial is going to be pretty varied” TE002, Clinical Investigator, Canada

6.5 “I think others have argued well once you get consent from someone you’re obligated in the usual care arm to give them the standard of care which is almost always higher than the usual care that people actually get. I don’t subscribe to that, but I also think there are many trials in which to answer the scientific question and, and the pragmatic question you actually do need to give standard of care.” TE006, Knowledge user, USA

6.6 “[…] are you compromising standard of care by having them [specific patient groups] in the study? Meaning that if you withhold standard of care therapies or treatments as part of a patient’s participation in the study, that’s not ethical. The study should actually look at things on top of that or in comparison to it, but withholding it is not proper.” TE007, Clinical investigator, USA

6.7 “the major concern, of course, is that routine practice is changing all the time. Well we hope it is, I mean, there is all this money going in for infrastructure and for […] capacity building on a routine basis. It’s much harder, then, to think of a pragmatic trial that will exactly mirror routine practice that is itself flexible. So rather than the intervention being flexible it’s […] whether the intervention evaluated in a pragmatic trial is […] going to map on to the intervention as it appears in routine practice [and] to how relevant the data will be.” TE020, Ethicist, UK

6.8 “The other part of pragmatic trial is that the intervention could be less described or some part of the intervention could be different from one physician to another in pragmatic trials. You could either explain very well all the different components of your intervention or you could decide in your pragmatic trials that some of the components of the interventions are to the discretion of the physician […] Even if it is stuff like that it must be at least reported in the paper and it’s not always reported.” TE016, Clinical Investigator, France