Beyond Interviews and Focus Groups: Mapping New Ways to Integrate Innovative Qualitative Methods into Randomised Controlled Trials of Complex Public Health Interventions

Background: Randomised controlled trials (RCTs) are widely used for establishing evidence of the effectiveness of interventions, yet public health interventions are often complex, posing specic challenges for RCTs. While there is increasing recognition that qualitative methods can and should be integrated into RCTs, few frameworks and practical guidance highlight which qualitative methods should be integrated and for what purposes. As a result, qualitative methods are often poorly or haphazardly integrated into existing trials and researchers rely heavily on standard interviews and focus group discussions. To improve current practice, we propose a framework for innovative qualitative research methods that can help address the challenges of RCTs for complex public health interventions. Methods: We used a stepped approach to develop a practical framework for researchers, which included: (1) a systematic review of the innovative qualitative methods mentioned in the health literature, (2) in-depth interviews with 23 academics from different methodological backgrounds working on RCTs of public health interventions in 11 different countries, and (3) framework development and group consensus-building process. Results: The ndings are presented according to the CONSORT Statement categories for ease of use. The main challenges of RCTs for public health interventions are identied alongside each of the CONSORT categories and potential innovative qualitative methods that overcome each challenge are listed as part of a Framework for the Integration of Innovative Qualitative Methods into RCTs of Complex Health Interventions. Innovative qualitative methods raised in the interviews and discussed in detail include: rapid ethnographic appraisals, document analysis, diary methods, interactive voice responses and SMS, community mapping, spiral walks, public randomisation, pair interviews, visual participatory analysis, among others. Conclusions: The ndings of this study point to the usefulness of observational and participatory methods for trials of complex public health interventions, offering a novel contribution to the broader literature about the need for mixed methods approaches. Integrating a diverse toolkit of qualitative methods can enable appropriate adjustments during RCTs, which in turn create more sustainable and effective interventions. However, this will require a cultural shift including the adoption of method-neutral research approaches, transdisciplinary collaborations, and a shift in publishing regimes.


Background
In this article, we argue that many of the challenges facing randomised controlled trials (RCTs) of complex public health interventions can be addressed through the integration of a diverse toolbox of qualitative methods, and propose a framework of potential methods. RCTs are widely used for establishing evidence of the effectiveness of interventions, yet public health interventions are often complex, posing speci c challenges for RCTs that qualitative research can help address. Qualitative research methods offer important insights for the evaluation of health interventions and are increasingly being used as part of RCTs [1][2][3]. Research investigating the potential value of qualitative methods has highlighted their role in facilitating the transferability of interventions, improving external validity, providing a more nuanced understanding of contexts and processes, and improving delivery of interventions. All of these ultimately increase the utility of evidence generated during RCTs [2][3][4][5].
Pope and Mays [6] claim that qualitative research is a "prerequisite of good quantitative research, particularly in areas that have received little previous investigation" (p42). It can provide insight into the contextual circumstances of the implementation, delivery and evaluation of interventions [7]. Qualitative research is particularly valuable for evaluating complex health interventions, which are often found in public health. A complex intervention is composed of different interacting components that are multifaceted and socially mediated [8,9]. The complexity resides in the variety of behaviours required by participants in the intervention; the groups or organisational levels targeted; the potentially large number of outcomes; and the degree of exibility permitted by the intervention [10]; all of which are associated with emergent phenomena that are di cult to predict. All these factors mean that complex health interventions are often challenging to de ne and therefore reproduce [11,12].
Qualitative methods can help address these challenges by "reaching the parts that other methods cannot reach," including aspects of behaviours, attitudes and interactions that affect the external validity and reproducibility of health interventions [6].
While there is increasing recognition that qualitative methods can and should be integrated into RCTs, few frameworks and practical guidance highlight which qualitative methods should be integrated and for what purposes. As a result, qualitative methods are often poorly or haphazardly integrated into existing trials [1,5]. This is evidenced by the lack of explicit reference to how qualitative ndings have been used to interpret or help explain quantitative trial results in published articles [3,13]. It contributes in turn to variation in the quality of qualitative research used alongside trials [5], and is compounded by what O'Cathain and colleagues [3] refer to as the "add-on status of qualitative research" (p121) in quantitative health research.

Challenges of using RCTs to evaluate complex health interventions
Although RCTs are currently perceived as the 'gold standard' in global health evaluation, their limitations are welldocumented [14,15]. Power and colleagues [11] point to several implementation trials that have lost internal validity when failing to consider the broader contexts in which they were carried out [16][17][18]. Other scholars acknowledge a disconnect between the recommendation that interventions be standardised to ensure valid measurement of trial outcomes and the very concept of complex health care systems [14].
These critiques of RCTs and standardised approaches to intervention are well recognised [10]. Recent Medical Research Council (MRC) guidelines on developing and evaluating complex interventions suggest a range of designs to address the challenges of conventional experimental trials, including cluster randomised and stepped wedge designs; preference trials including 'Wennberg' and 'Rucker' designs, which base randomisation on individuals' preferences; and randomised consent 'Zelen' designs, which randomise individuals prior to taking consent [19]. Others have discussed the need for 'adaptive' trial designs whereby changes are made to an intervention or trial based on information collected during the experiment phase [20]. An example of such adaptations is the use of 'pragmatic trials,' which attempt to establish the effectiveness of interventions under routine conditions instead of rigid experimental design [21]. Hawe and colleagues [22] argue that, in pragmatic trials the evaluation process and function can be standardised, while the form of the intervention is made adaptable to context. Pragmatic trials are designed to re ect the real-world heterogeneity of participants and situations, often keeping exclusion criteria to a minimum and not necessarily requiring blinding [23]. A pragmatic RCT approach is seen as providing an opportunity to evaluate not only the intervention, but also its interaction with the complex social, cultural, legal and ethical context, which may be as important as the intervention itself [23]. This is unlike conventional experimental designs in which the intervention is normally de ned at the outset with little room for adaptation [21].
However, as large-scale experimental studies, both experimental and pragmatic RCTs remain resource intensive, posing challenges to produce reliable and accessible evidence of generalisable impact and relevance to the targeted population [24][25][26]. The persistent use of trials to evaluate complex interventions has been critiqued as 'consistently becoming one of expensive failure -that is weak or non-signi cant ndings at huge cost' [22] (p1561). Chalmers and Glasziou argue that there is 85% cumulative waste in the production and reporting of research evidence, generating unnecessary losses in investments as a result of entirely correctable problems [24]. An important aspect of this waste relates to inappropriate methods, which contribute to incomplete reporting of trial outcomes and the need to measure new outcomes not part of the original plan [26].
In an attempt to address these limitations and drawing on the advantages of qualitative methods discussed previously, we propose a framework for integrating qualitative methods into quantitative evaluations of complex health interventions. We aim to answer the question: how can innovative qualitative methods address the challenges of RCTs as an evaluation methodology? In this article, we draw on key informant interviews with 23 researchers involved in RCTs to identify how qualitative methods can be used to address the challenges that the evaluation of complex health interventions raises for RCT methodologies.
Using qualitative methods to address the challenges Methodological frameworks for the use of qualitative methods alongside quantitative intervention evaluation methods are currently limited in justifying the need for qualitative methods. Mixed methods evidence about the use of qualitative methods alongside trials centres on time-related or 'temporal' frameworks. Along these lines, Sandelowski [27] highlights how qualitative research can be incorporated either as formative evaluations before the trial begins, process evaluations during the trial, or impact evaluations after an intervention [27]. Creswell and colleagues build on Sandelowski's work and argue that it is the purpose of the qualitative data collection that de nes whether the "before," "during" or "after" model should be used. In the stages before a trial begins, qualitative research is said to be most useful for de ning research topics while ensuring the intervention's relevance and appropriateness to the populations of interest [28]. Nested within a trial, qualitative process evaluations may indicate the reasons behind an intervention failure, unexpected consequences, or success [8].
Finally, after a trial is complete, the use of qualitative methods is said to help "interpret quantitative ndings and question underlying theory and assumptions to better inform future hypotheses and intervention design" [29] (p714). Frameworks within this group include the MRC Framework [8], which also divides the process of developing and evaluating an intervention in a trial into time-related phases.
There are different approaches. Flemming et al. [28] have developed a framework that focuses on the contribution of qualitative research to speci c RCT processes, including planning, recruitment, randomisation, conduct, attrition and implementation. In line with Flemming's work, O'Cathain and colleagues [3] argue that temporal frameworks contribute to a lack of clarity about when the qualitative research actually takes place as part of a RCT. Synthesising the use of qualitative research within speci c trials across 296 studies, they produce an alternative framework that includes ve broad categories: the intervention being tested, the trial design and conduct, outcomes, measures used within the trial, and the intervention studied.
However, neither temporal nor process frameworks go much beyond a description of qualitative research and its characteristics. Despite methodological discussions about when to collect qualitative data and for what purpose, there has been little discussion to date of what qualitative methods might be most useful for complementing quantitative RCT data in the evaluation of complex health interventions. As such, existing frameworks have not contributed to broadening the discussion around the inherent value of having a diverse selection of qualitative methods to draw on for RCTs, and how much greater innovation and diversity of methods could be drawn on to strengthen what are often unrealised attempts at mixed methods. Thus there is a need for trialists to 'look beyond focus groups and interviews' and expand the extremely limited diversity of qualitative methods currently being used [3].
In this article, we offer a new framework developed from expert researchers' accounts of using qualitative methods as a component of trial. Rather than offering an alternative to temporal or process frameworks, our aim is to highlight the diversity of qualitative methods that can be used to address many of the major challenges complex health interventions pose for RCTs. We suggest innovative examples of qualitative methods to demonstrate the adaptive and extensive nature of qualitative research as part and parcel of RCTs.

Methods
We followed a stepped approach to develop a practical framework for researchers selecting qualitative methods to use within RCTs of public health interventions. Our aim was to cover a range of potential trial designs, including individually randomised, cluster randomised and stepped wedge. Our approach included (1) a review of the qualitative methods mentioned in the health literature, (2) in-depth interviews with academics from different methodological backgrounds (quantitative and qualitative) working on RCTs of public health interventions, and (3) framework development and group consensus-building.
The inclusion of qualitative and quantitative researchers in the study provided an opportunity to gather methodological perspectives on the inherent value of using qualitative methods within trials. The use of in-depth interviews to gather academics' opinions and experiences provided a means of overcoming some of the power dynamics involved in qualitative and quantitative research paradigms and minimising the in uence of group dynamics [30]. The participants in the study were also invited to be co-authors on the nal paper to ensure the framework was developed and re ned through a group decision-making process.
Step 1: Review of Qualitative Methods used in RCTs An initial list of innovative qualitative methods was derived through searching the published literature using the databases Pubmed and Scopus. We used the term 'innovative' to refer to qualitative methods other than standard interviews and focus group discussions. The "sources" section of Scopus was searched on 08/11/2017 for journals that contained "qualitative" (n=23) and "method" (n=160) in their titles. Journals were assessed for relevance based on their focus on qualitative methodologies and English language. This search produced a list of 25 journals (Table 1). Two further journals were added that had been identi ed by Wiles et al. [28]. The website catalogues of these journals were then searched with the terms "innov*, new, novel, emerg*" as previously used in the methodology by Wiles and colleagues [31]. These search terms were used in order to identify methods considered as innovative or new to qualitative methodologists. This produced 654 search results which were exported to Endnote (Thomson Reuters). Journal articles from before 2008 were removed to capture methodological discussions over the past 10 years. Abstracts were screened for mention of qualitative methodologies beyond interviews and focus group discussions, or for discussions of methodological innovations. This produced 127 full text articles of interest, which were further screened for mention or discussion of methods. A list of methods was subsequently compiled ( Table 2). Step 2: Key informant identi cation and in-depth interviews Identifying key informants: The list of methods generated during Step 1 was adapted to produce a search strategy to identify scholars working in mixed-methods. We searched Scopus and Medline for articles that mentioned any of the qualitative methods from Step 1 alongside RCTs. Emphasis was placed on methods that were not basic interviews or focus group discussions without a theoretical rationale mentioned in their design. This produced a nal selection of 9 journal articles ( Figure 1). Authors of these articles were reviewed to categorise them as potential key informants according to whether they were primarily publishing qualitative or quantitative research, their discipline, geographic location, and research position [32].
The initial list of scholars identi ed in the literature (n=85) was reduced to rst and last authors only (n=18), and supplemented by professional research networks (n=25) and snowballing (n=15). The nal list of experts (n=65) and their categorisation was used to ensure maximum heterogeneity in the group composition as a means of encouraging and heightening creativity in the decision-making process that followed [33].
We invited 58 experts to participate in the study. In the invitation sent by email, they were given an information sheet and consent form, and asked to con rm their consent to participate with an explicit email response. Interview participants were offered the opportunity to contribute as authors of this paper, under the condition that they provide input into each draft. A nal tally of 23 experts agreed to participate in the in-depth interviews (Table 3). Table 3. Demographic characteristics of respondents Participants had experience working in China, Ethiopia, India, Malawi, Nepal, Rwanda, South Africa, Uganda, UK, US, and Zambia. Anonymity was maintained between participants throughout the study up until the paper writing stage, and participant co-authors were not involved in the analysis of the raw data. The study received ethical approval from the UCL Research Ethics Committee number 12449/001. Data collection through in-depth Interviews: A topic guide was used for the interview with a broad set of questions about the inherent value of using qualitative methods within RCTs, methods being used to address intervention or evaluation challenges, and some of the challenges faced when mixing methods. Interviews were conducted by n % phone or Skype and lasted 50 minutes on average. The interviews were audio recorded and transcribed. Transcripts were entered into NVivo (QSR International) for qualitative data analysis (QDA).
Step 3: Framework development and group consensus process We conducted a thematic analysis of the interview data using a framework approach [34]. We adapted a version of the Consolidated Standards of Reporting Trials (CONSORT) Statement [35] as our framework in an attempt to move beyond current discussions of how qualitative research can add value to trials and highlight the indispensable nature of qualitative methods as a complement to quantitative research methodologies. CONSORT is recognised as an essential tool for reporting trial outcomes, and widely used by medical journals and international editorial groups [36]. This provided an ideal starting point for appealing to quantitative researchers using a well-recognised framework. To adapt the CONSORT Statement for our purposes, our research team (KD, NM, JM) conducted a rapid review of the literature on the main methodological challenges of using RCTs to evaluate complex public health interventions and organised these challenges according to the CONSORT Statement's main headings (Table  4). We then merged them into categories in terms of their key challenges. This provided the adapted framework for our thematic analysis of the in-depth interviews with scholars. Using this framework, we completed a thematic analysis in NVivo of the interview transcripts, aligning what scholars had said about the inherent usefulness of particular qualitative methods for the RCTs they had worked on with the RCT challenges in our framework. We then shared the draft framework with participants in two separate rounds in order to elicit feedback and facilitate group consensus on the alignment between RCT challenges and speci c qualitative methods.

Results
The aim of our analysis was to generate a list of innovative qualitative methods that help to address common methodological challenges of RCTs (Table 4). We present our results following the adapted CONSORT statement framework used to guide the analysis. Common methodological challenges are organised according to the merged categories, and potential qualitative methods that overcome each challenge are listed. Our results are presented according to seven categories: 1) Background and Setting; 2) Intervention design and compliance; 3) Participant recruitment and enrollment; 3) Allocation of the intervention and randomisation; 5) Participant follow-up; 6) Data collection; and 7) Analysis and results. Under each category, we have combined ndings from the in-depth interviews and evidence from the literature review as a means of expanding the scope of the results and describing a more comprehensive methodological landscape.

1) Background and Setting
Challenge 1: Ensure relevance of the intervention to the context.
Interviewees suggested that qualitative methods had an important role to play in ensuring that the design and adaptation of interventions corresponds with local needs. They suggested a number of activities that could be completed either (a) before testing an intervention through a trial or (b) during the trial itself, to ensure there is delity and that the intervention remains relevant and acceptable. For instance, an assessment to identify the local health, social, cultural and political landscape was suggested as a means of providing important information about the need for an intervention within a particular context: In global public health, most interventions are … delivered in quite complex settings and trying to understand the setting -how people go about their daily lives and then how that intervention is going to t into those daily livesis really important. I16 In the qualitative literature, direct observation is frequently mentioned as a particularly useful method for developing cultural understanding of context by enabling data to be gathered through rst-hand or eyewitness experience [43]. Drawing on ethnographic techniques in anthropology, observation as a method often involves sustained immersion in the research setting for a long period [44], which is not always possible as part of formative research leading up to a trial or within the constraints of a process evaluation.
As an alternative, quicker approaches to ethnography were mentioned by interview participants, including social mapping, non-participant observation, and 'Broad Brush Surveys', which systematically gather data on communities in a period of 5-12 days [45]: The point of [Broad Brush surveys] is to provide a narrow and rapid impression of the visible features of the communities and get a sense of the more invisible characteristics of the communities and how they might affect the particular research and interventions planned. I11 These rapid techniques draw on the advantages of ethnographic techniques while adhering to the limitations of a trial and the need for rapid results.
Challenge 2: Obtain buy-in from stakeholders including practitioners and policy makers.
If public health interventions undergoing trials are not informed by key stakeholders, including communities, community leaders, local policy makers and health providers at national and regional levels, they risk reduced sustainability and rejection by local actors. It is particularly important to spend time engaging communities to increase participation, recruitment and to promote ownership of the problem as well as the solution, as discussed during the interviews: The ideal would be that we engage communities even before the trial is designed in order to decide what their priorities are and then use those priorities to develop the intervention.i18 The literature points to trials that have been stopped due to signi cant stakeholder ethical opposition [46]. Even interventions that are effective may not translate into improved practices or policies if they are not aligned with the current agenda of local governments or policy-makers [47]. Therefore, it is important to consider and adapt to stakeholders' concerns early on to increase subsequent dissemination and uptake of results.
Qualitative methods mentioned by interviewees to overcome these risks and promote take-up of the intervention include in-depth interviews with policy makers, observation and document analysis. This was not always done formally, but often informally as part of the early stages of a trial: The literature similarly supports the use of in-depth semi-structured interviews as a means of understanding stakeholder perspectives because of the potential to produce data on people's knowledge, understandings, interpretations, experiences and interactions [48]. With this information one can enable the design of optimal, scalable interventions that could feasibly be translated into policy.

2) Intervention design and compliance
Challenge 3: Ensure cultural acceptability and feasibility.
Interventions undergoing an RCT that are not acceptable to the population or feasible in the local context will likely result in poor adherence and drop-out of participants, affecting both the magnitude of the effect and representativeness of the nal sample. Tailoring the intervention to increase its acceptability can also be challenging due to a reluctance of participants to share relevant information, social desirability bias [49] and power-dynamics between researchers and participants [50]. An effective means of accessing crucial qualitative data on the acceptability and feasibility of an intervention mentioned by interviewees is diary methods: They [ eldworkers] kept diaries because they were the people that were going on the wards every day and noticing the culture of the wards and able to see pertinent issues about the wards. I3 Recognised in the literature as a useful technique for obtaining observational data, diaries can be both qualitative and quantitative, and are used to capture and record the details of time-sensitive and context-speci c phenomena [51]. As they allow events to be recorded in their natural setting, diaries reduce the delay between an event and its recording and provide a level of privacy that should reduce potential bias [52].
Interviewees also mentioned the potential for mobile phone technology to be used to collect audio diaries through an Interactive Voice Response process where participants record brief voice clips about their activities or perspectives when prompted: So, [interactive voice response, or Short Message Service (SMS)] has been, I would say, key in helping us to track change over time but also gives us very practical information about what we need to do in terms of orienting the intervention to be most impactful. I15 Most of the research now is done on mobile phones […] it's probably the best way to do research, to get large samples of anything. You use mobile phones here. I20 Such methods were also highlighted by interviewees as particularly useful with geographically hard-to-reach populations, tracking change in an intervention over time, or as a means of obtaining large sample sizes for data collection.

3) Participant recruitment and enrolment
Challenge 4: Guarantee representativeness and improve recruitment.
Developing recruitment techniques that reach the required sample size, while re ecting the characteristics of the broader population, is challenging: We've recently been doing work in shing communities, here, in Uganda, where it has been very much trying to get some sense of who spends time where in terms of, if you're going to reach them with particular interventions, where are you going to nd them? I10 Qualitative methods that capitalise on local knowledge can be invaluable for understanding complex social characteristics and differences that may lead to recruitment bias. To address this, both the literature and interviewees mentioned the usefulness of mapping methods, which bring together groups or communities to 'map' assets, processes, or physical environments. These can be extremely useful in collecting information about resources, activities, and potential recruitment locations in a relatively short period [53]. A particularly effective technique mentioned by interviewees is mapping physical spaces and resources within a community by using transect or 'spiral' walk methods to record how resources are distributed: If you don't walk through that area, you may not realise quite what's going on. So the idea is to walk in a circle, facilitated, trying to come into contact with as many different people from as many different parts of a settlement as possible. You walk in a spiral. I10 Spiral walks use observation methods to identify people and places of signi cance to the research. The walk is usually in concentric circles from the centre to the outskirts of an area and can involve interviews en route [54]. The purpose of the circles is to cover all sections of a community -a transect walk in a straight line may leave out an area where a particular group of people live or work. Spiral walks were perceived by interviewees to provide a greater understanding of social structures in the community. However, in order to maximise the potential for recruitment, mapping methods should be combined with data from those implementing the recruitment procedures to assess the t between the understanding of the social context and the way recruitment is currently being done within the trial.

4) Randomisation and allocation of the intervention Challenge 5: Guarantee comparability between groups
One of the main challenges of trials is to ensure balance across intervention arms, in order to ensure groups are comparable and that the difference between them can still be used to achieve causal interpretation [55]. Different underpinning cluster or individual characteristics can reduce the comparability between groups, affecting the effect size and directionality of the results in unmeasurable ways. Therefore, it is important to tease out contextual factors such as community characteristics or locations that can be strati ed on during randomisation. As suggested by an interviewee: We put these communities into different types and then allow that to be part of the randomisation process so rather than just randomising according to the number of HIV prevalence or geographical location and so on. I11 Although considering contextual factors during randomisation can be di cult to achieve, it might also save a trial from losing validity. Participant observation at early stages of the study might allow researchers to witness a community's dynamics and observe relevant information that can be used to identify strata for randomisation.
A participatory approach to identifying strata in collaboration with communities suggested during the interviews was to do public randomisation: You know, usually it's just the scientists going into a room and using a random number generator. But putting the clusters together and also making sure that the clusters were balanced really required the community participation because they understand the agencies involved. Then even on the randomisation inside, they helped ... we had a really great statistician who did a group facilitated exercise to come up with the randomisation. I4 Involving communities in the process of identifying strata to be used in randomisation in this way can help to ensure the randomisation process considers potential similarities and differences in groups that are not directly observable by the research team. This can be a particularly useful method when there are a relatively small number of clusters to be randomised.

5) Participant follow-up
Challenge 6: Minimise the number of subjects leaving the study. Improve adherence to treatment or intervention.
Complex health interventions often address highly challenging health problems among marginalised populations through multiple intervention components. The challenging lives of targeted populations combined with the complexity of the intervention itself often magnify the number of people leaving the study and reduce potential adherence for those who stay. To effectively gather data on loss to follow-up and adherence, an interviewee recommended monitoring participants and adapting the intervention over time, consistent with a pragmatic trial approach: …we always explicitly said that we're going to continually measure how our intervention is working or not working in terms of whether people like it, whether people are using it, so that we can adapt our intervention as we go. I8 Interviewees also mentioned the need to maintain constant communication with participants and the advantages of doing so through highly available methods, such as SMS or WhatsApp technologies: Wherever the population is widely using smart phones I would say, it's worth [collecting data] on smart phones… it's just easy to access people that way, and you can access people who you wouldn't otherwise be able to access… you know, very speci c groups. I20 The advantages of mobile communication technologies are also emphasised in the literature for their ability to provide a closer follow-up with participants, as well as acting as an alternative data collection source [56].

6) Data collection
Challenge 7: Enhance reliability and quality of the data.
Data reliability in RCTs can be affected through several mechanisms, some related to the process and others to the intrinsic nature of the data gathered. On one hand, collecting data under real-life conditions can affect the prestandardized process of data collection. On the other hand, trials that evaluate complex public health interventions may also rely on outcomes that are subject to social desirability bias, which compromises objectivity in the measurement of outcomes.
One means of addressing this and increasing the reliability and validity of qualitative data mentioned by interviewees is the use of community-based participatory research (CBPR) methods. CBPR collaboratively involves community stakeholders, such as community groups and organisations and members of the community, at all stages of the research process [57]. The advantages of the deep involvement of participants is the potential for participants to forget they are being researched: If they [people] are involved in an activity with a task that's group-based, I think that they forget that they're involved in research, and so you might get slightly more accurate insights into people's perceptions and behaviours because they don't feel quite so much like they're on show. I21 The literature also emphasises the potential of CBPR methods and in particular measurement tools that are codesigned with participants to ensure content validity, construct validity and consensual validity [58]. However, iterative tool development is understood as a long-term process, which should take place over multiple sessions and consultations with stakeholder groups within the community [58].
Challenge 8: Maintain objectivity during the data collection process Many complex interventions are aimed at changing behaviours at an individual level and collect data based on self-reports, which can hinder the collection of valid data when there are complex social structures in uencing people's behaviours. An example of this challenge was introduced in one of the interviews: So, our quantitative data showed very high rates of handwashing, but our observations showed that the reality was quite different. People often didn't have soap in the house or only washed their hands with water, but they actually said something else in the quantitative interview. So, it gave us a better contextual understanding of the lives of those families and the structural barriers that they faced to changing their behaviour, and also, the limitations of our quantitative data, which was useful. I12 Although many RCTs select "hard" endpoint variables to ensure the identi cation of effect, measuring the process remains predominantly "soft". It is a challenge for RCTs to nd strategies to collect high quality, valid process data. The susceptibility to potential biases such as social desirability or recall bias increases in complex public health interventions, and identifying the best qualitative methodology to capture issues with validity becomes imperative to maintaining the objectivity of the study.
To prevent socially desirable responses from participants, interviewees mentioned the potential for embedding well-structured activities into focus group discussions to help access more reliable data. For instance, ashcards or images-as-prompts can be used to access the immediate reactions and understandings of participants, to identify social taboos, and to understand misconceptions about the intervention that could create challenges for the RCT. This same idea is evident in literature that mentions the advantages of having participants organise or "pile sort" images to assess the importance of certain ideas or issues for a group or to gain an understanding of local priorities [59]. Asking participants to order events as they believe they would or should happen can gauge a sense of people's daily lives or processes of care-seeking. Interviewees also mentioned the potential of interviewing young people in pairs or with friends, or using role-play or song, to address shyness or to obtain valuable information from younger participants: We've tried friendship pair interviews where we've tried to get two friends to talk things through with the idea that if they're with somebody who they feel very comfortable with the social desirability bias might be less and the friend might have some insights into the person's behaviour. I2 All these activities offer an opportunity to more effectively access the psychological processes underlying perceptions and opinions of participants.
Challenge 9: Identify the mechanisms underpinning the effect of the intervention.
While trials are useful for evaluating the effect of a designed intervention, they fall short in explaining the mechanisms behind an intervention's effect. Qualitative data can help address this, as acknowledged in the MRC guidance on process evaluation [39] and discussed in an interview: …the quantitative piece will really help us understand the scale of the impact but the qualitative we're feeding in to understanding kind of the practices and if it worked, how it worked and why it worked and we're de nitely very interested once we have the endline [data], both quantitative and qualitative to do a lot of cross-triangulation so that those two datasets kind of speak to each other. I1 Without qualitative data, trials that show little effect of an intervention will not have an explanation of why this may have occurred or which component failed. This is especially important in trials evaluating behaviour change, in which interpretations of what has changed may differ between participants and researchers. One interviewee mentioned the use of "visual participatory analysis" to understand community perceptions of the results of the trial. This method involves engaging participants in a community-based focus group discussion about the meaning of the quantitative results. During the focus groups, the participants discuss a series of graphic representation of the study's results. This qualitative data then provided additional data that was used to inform the interpretation of the quantitative results. Where an intervention is complex and the impacts on people's lives are likely to be wide-ranging, this method offers a potential means of eliciting different perspectives on the intervention and its effects. It moves beyond recommendations that results should be disseminated to local communities [60] by offering a means of ensuring the qualitative ndings from this process are integrated with the analysis of the quantitative ndings from the trial.

Discussion
Our ndings present a new framework for integrating qualitative methods into RCTs of complex interventions with the aim of contributing to methodological discussions about the value of diverse qualitative methods for understanding complex interventions in global health and addressing the challenges of RCTs. Similar to others, our framework is a process rather than temporal framework, but with the added value of situating the bene ts of using qualitative methods against the speci c challenges of RCTs. This contributes directly to calls for innovation in the use of RCT methodologies to evaluate complex health interventions and the challenges they face in maintaining internal and external validity [61].
The ndings of this study point speci cally to the usefulness of observational and participatory methods for trials of complex public health interventions, offering a novel contribution to the broader literature about the need for mixed methods approaches. The ndings highlight the bene ts of using methods such as diaries, mapping techniques, and observational spiral walks to understand the social, political, and cultural context surrounding both interventions and trials. The ndings also outline the bene ts of using community participation to inform key trial design decisions drawing on methods such as public randomisation and 'community-based participatory research' as part and parcel of trials. Finally, the ndings highlight the advantages of using observation and participatory approaches, such as 'visual participatory analysis,' to understand the effects of an intervention. This is a radical departure from the use of endline interviews with participants or focus group discussions as part of a mixed methods process evaluation. Scholars have recognised the constraints of collecting brief qualitative data during a trial and the limited understanding of the process of the intervention this generates [62]. The methods suggested as part of this study provide a means of harnessing the rich potential of qualitative research to understand context, to subsequently ensure the intervention is the best for this context, and nally to understand its effects.
In a study of how qualitative and quantitative data are combined in RCTs, The QUAlitative Research in Trials study (QUART)), researchers found that those who take an "integrated methods approach" also see qualitative research as essential to the trial and as producing evidence related to the "real world" [3]. The QUART study therefore recommends that researchers design and implement "studies not trials," with the outcomes of the qualitative research being "central to the team's thinking" (p124). In practice, the implementation of 'studies' rather than 'trials' requires researchers to adopt a neutral approach to methods. This essentially means selecting the best method for the research question posed rather than making presumptions about which methods are best based on a hierarchy of evidence [63]. For example, if the question is about whether people have changed health-related beliefs, then the research team potentially has multiple methods to choose from, which may include qualitative interviews, focus group discussions, photovoice, quantitative surveys, or a combination of these methods. The selection of which method is best should be driven by identifying the method that is best for answering the speci c question for a particular context [64]. This approach to method neutrality may be particularly useful for complex public health interventions with strong theoretical models. Trials that are driven by theories of change or logic models have a series of research questions that may be posed alongside each stage of the model in an enumerated fashion. In a method-neutral approach, each of these stages require the research team to identify the most appropriate methods based on the question and the context. The diverse toolbox of qualitative methods suggested as part of our Framework helps to foster such an approach.
The ndings presented in this article have also shown how a diverse toolkit of qualitative methods can enable appropriate adjustments during intervention development to create more sustainable interventions [7]. It can determine the relevance of a trial to participants, hence ensuring successful recruitment and follow-up, and can also identify behaviours and biases of those recruiting participants to a trial and perceptions of participants that may explain differences in outcome [28]. Finally, qualitative research can indicate why successful interventions might not work in the real world and nd the best ways to ensure ndings are translated to policy by establishing how results may be received by relevant parties, enhancing the utility, effectiveness and applicability of ndings from RCTs [28,65]. Regardless of whether we continue to use standardised randomised trials or seek out more pragmatic designs, we need to move beyond the division between quantitative and qualitative methods and see public health evaluation itself as a form of mixed methods research. To accomplish this, the purpose and value of qualitative methods need to be highlighted within both trial protocols and published studies, and the integration of qualitative research needs to be considered at the earliest stages of grant proposal development [13]. Implicit in the integration of quantitative and qualitative research in the evaluation of complex public health interventions is a need for greater engagement between disciplines with strong qualitative or quantitative traditions, such as anthropology and epidemiology. This is already happening to a certain extent with interdisciplinary relationships between anthropologists and epidemiologists contributing to more nuanced understandings of human behaviour and interventions better suited for local contexts [66]. However, experiences of working collaboratively across disciplines is not without its challenges and adopting a method-neutral approach as discussed above requires an acceptance of methodology as a socially-constructed tool used to 'interpret' rather than 'observe' reality [67]. This will require changes by all research actors including scholars, universities, funders and those publishing research results.
Along similar lines, a fundamental change in the publishing bias of medical journals that favours brief, quantitative articles is needed to bring method-neutral and transdisciplinary approaches to public health evaluation research [68]. In 2016, the current bias towards publishing quantitative articles led to an open letter to the British Medical Journal calling for it to "move beyond a 'quantitative strong, qualitative weak' stance" and to stop "rejecting qualitative research on the grounds of low priority" [69] (p2). Both the QUART study (in interviews) and our own qualitative interviews found a culture in which qualitative research is not cited as frequently as quantitative research, and is often not published in high-impact journals [69].

Conclusion
We argue that it is through dialogue and recognition of the complementarity of disciplines that the expansion of successful health-related interventions will be achieved. We have contributed to this dialogue by presenting innovative ways in which qualitative research can be integrated into RCTs to improve research quality and to increase health impact. We hope this encourages researchers to enter into new studies with a broader understanding of what counts as 'evidence' for impact evaluations and how qualitative research can strengthen the validity and rigour of RCT research for the future.

Limitations of this study
Given the foundation of this study in expert opinions and experiences, our ndings are naturally in uenced by the types of complex health interventions the experts interviewed were involved in. They may not be equally applicable to the wide variety of complex interventions currently being tested using different RCT designs, including those testing systems versus drugs or devices, treatment-focused versus prevention-focused interventions, clusterrandomised versus individually-randomised designs, etc. The methods we have discussed as part of our framework are also not an exhaustive list, and we would encourage researchers to investigate alternative methods from different disciplinary perspectives that meet their speci c trial needs. It is also worth mentioning that our presentation of different qualitative methods does not consider the ethical implications of using these methods, and that this would need to be assessed as part of the speci c aims and objectives of the trial being undertaken.