Systematic review of the use of translated patient-reported outcome measures in cancer trials

Background Patient-reported outcomes (PROs) are used in clinical trials to assess the effectiveness and tolerability of interventions. Inclusion of participants from different ethnic backgrounds is essential for generalisability of cancer trial results. PRO data collection should include appropriately translated patient-reported outcome measures (PROMs) to minimise missing data and sample attrition. Methods Protocols and/or publications from cancer clinical trials using a PRO endpoint and registered on the National Institute for Health Research Portfolio were systematically reviewed for information on recruitment, inclusion of ethnicity data, and use of appropriately translated PROMs. Semi-structured interviews were conducted with key stakeholders to explore barriers and facilitators for optimal PRO trial design, diverse recruitment and reporting, and use of appropriately translated PROMs. Results Eighty-four trials met the inclusion criteria, only 14 (17%) (n = 4754) reported ethnic group data, and ethnic group recruitment was low, 611 (13%). Although 8 (57%) studies were multi-centred and multi-national, none reported using translated PROMs, although available for 7 (88%) of the studies. Interviews with 44 international stakeholders identified a number of perceived barriers to ethnically diverse recruitment including diverse participant engagement, relevance of ethnicity to research question, prominence of PROs, and need to minimise investigator burden. Stakeholders had differing opinions on the use of translated PROMs, the impact of trial designs, and recruitment strategies on diverse recruitment. Facilitators of inclusive research were described and examples of good practice identified. Conclusions Greater transparency is required when PROs are used as primary or secondary outcomes in clinical trials. Protocols and publications should demonstrate that recruitment was accessible to diverse populations and facilitated by trial design, recruitment strategies, and appropriate PROM usage. The use of translated PROMs should be made explicit when used in cancer clinical trials.


Introduction
Under-representation of biologically and culturally diverse populations in clinical trials can have implications for the generalisability of findings to practice [1]. Studies have shown that people from ethnic groups often have poorer healthcare experiences and face additional barriers when accessing services. Cancer incidence rates and mortality outcomes often differ, with ethnic groups presenting later leading to poorer health outcomes [2].
Patient-reported outcomes (PROs) are increasingly used to evaluate primary, secondary or exploratory outcomes in cancer clinical trials [3][4][5]. Patient-reported outcome measures (PROMs) are patient-completed measurement tools that capture PRO data and provide the patient's perspective on their health and impairment status and the impact of symptoms and side effects on their health-related quality of life (HRQoL) [6,7]. PROMs may identify additional patient impairments, treatment benefits, or harms that may be missed using clinical measures alone [8][9][10].
Measures which reflect the patient's health and cultural experiences have been found to be more sensitive to change and more reliable measurement tools [11,12]. The use of translated and culturally validated PROMs facilitates the inclusion of a wider range of participants. Culturally inappropriate PROMs may contribute to sample attrition and missing data due to misunderstood or culturally irrelevant items, limiting the ability to establish conceptual equivalence or the wisdom in aggregating patient-centred data in clinical trial results [13]. Guidelines have been developed to ensure that translation processes are rigorous and reflect required cultural perspectives [11,13]. However, it is unclear if translated and culturally validated PROMs are being used in clinical trials with PRO endpoints [14] or to what extent this guidance is being adhered to in contemporary cancer clinical trials [4,15].
This review aimed to establish reporting of ethnicity in cancer trials listed on the National Institute for Health Research (NIHR) Portfolio Database [16] and collecting PROM data. We also reviewed whether PRO data were captured using translated and culturally validated PROMs. The rationale is that to meaningfully accommodate linguistically and culturally diverse participants, translated and culturally validated PROMs should be used to facilitate data collection. We also undertook qualitative interviews with international stakeholders to understand current practice and associated barriers and facilitators to the use of translated and culturally validated PROMs in clinical trials [16].

Data collection
We conducted secondary data analysis of the Systematic Evaluation of Patient-Reported Outcome Protocol Content and Reporting in Cancer Trials (EPiC) study database. EPiC systematically evaluated completed cancer clinical trials collecting a PRO as a primary or secondary endpoint, identified on the NIHR Portfolio between January 2001 and April 2014 [16,17]. The most up-todate ethically approved trial protocol and any arising publications were retrieved. Two independent investigators (i) extracted trial characteristics, (ii) determined the availability of PRO results, (iii) evaluated general protocol and reporting quality using the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) 2013 and CONSORT (CONsolidated Standards of Reporting Trials) checklists respectively, and (iv) evaluated PRO protocol content and reporting using a bespoke protocol checklist and the CONSORT-PRO Extension [16,17].

Review of protocols and publications
The current study included trials within the EPiC database that included ethnicity data in their protocols and/ or publications. Two independent reviewers (KA/AS) screened the studies for information relating to recruitment, inclusion/exclusion criteria, use of culturally and linguistically appropriate consent forms, information sheets, and translated and culturally validated PROMs, stratified by ethnicity. Disagreements were resolved via discussion with a third reviewer where required (MC/DK).

Qualitative study
Ethics approval was obtained from the University of Birmingham Ethics Committee (ERN_17-0085A) and participants consented to take part in interviews. Participants were purposively sampled and had experience of PROMs, through either participation in a trial or professional experience of designing and reviewing clinical trials [16]. Participants were drawn from four groups: (1) trialists and chief investigators with experience of cancer trials collecting a PRO as a primary or secondary endpoint, (2) individuals with lived experience of cancer, (3) international experts in oncology PRO trial design, and (4) journal editors, funding panellists, and regulatory board members [16].
Semi-structured face-to-face or telephone interviews were conducted with individuals from the stakeholder groups. An experienced qualitative researcher (AR) conducted the interviews using a topic guide; questions relating to the use of PRO and translated and culturally validated PROMs were explored, subsequent probes depended on the individuals' experience. Barriers and facilitators to optimal PRO trial design, recruitment, collection, and reporting, in relation to the use of translated and culturally validated PROMs, were discussed.
Interviews were audio recorded and transcribed verbatim by a professional transcription service and analysed by AR using directed thematic analysis [16]. Findings from previous qualitative and review work, coupled with analysis of the protocols and publications, underpinned the development of an initial coding framework. A broad code to extract items pertaining to the collection and reporting of PRO data using translated and culturally validated PROMs was used. These were coded and subcodes were then formulated iteratively. Formal triangulation of coding was conducted by DK/MC.

EPiC review results
The EPiC review identified 1141 trials across the NIHR Portfolio Database. Of these, 228 met the eligibility criteria, 160 studies had published their primary results within the review timeframe. In total, there were 84 trials for which a matching protocol and arising publication were retrieved (see Fig. 1 and Kyte et al.) [18]. Of these 84 trials, only 14 (17%) reported any type of ethnic group profile data (

Reporting ethnicity
Ethnicity profile data were generally reported in relation to baseline participant characteristics. The most frequently described ethnic groups were African, African-American, Afro-Caribbean, and Asian/Indian (Table 1). Three trials specifically excluded non-English speakers [29,30,33]. The numbers of participants represented by ethnicity data were small (611, 13%) in comparison to the total numbers of participants recruited across the 14 studies (4754), although in the UK this would have reflected current population norms (Table 1). Only three multi-national studies reported ethnicity numbers above 15% of their total population, primarily categorised as East Asian or Asian participants (Table 1) [20,24,26].
One study stratified survival results by ethnicity [20].

Use of translated and culturally validated PROMs
Eight (57%) of the 14 included studies were multicentre, multi-national trials. The use of translated PROMs was not reported in any of the trial protocols or publications despite 7 (88%) using PROMs that have been translated into other languages ( Table 2). It was not clear if missing data were related to the availability of translated and culturally validated PROMs. One study stated in the protocol that translated patient documents would be available if required. However, it was not clear if this included translated PROMs [26,27].

Qualitative results
Forty-four interviews were undertaken with a broad range of international stakeholders (Table 3). Issues highlighted by the interviewees were similar despite the diversity of professional and national backgrounds. Three main themes emerged from the qualitative data: (1) recruitment, (2) development of research questions and study design, and (3) implementing translated and culturally validated PROMs (Table 4). Key elements with interview quotations are illustrated in Table 4. Discussions relating to the use of translated and culturally appropriate PROMs referred to the inclusion of linguistically and culturally diverse participants within a single-country trial, rather than the inclusion of international trial participants in multi-national trials, collecting PROMs.

(3) Black Caribbean (3) Others 4 (5)
NS not stated a Primary paper, b Secondary paper    (8) Canada (2) USA (1) Austria (1) Regulatory (3), research ethics (2), research (7), clinical (7), funding panellist (5)  Working with peer-researchers "They had some funding … to do some research with the non-English speaking Chinese community. And they recruited several non-English speaking Chinese … and through interpreters worked with them and developed a structure for the study. But then those same non-English speaking Chinese actually worked as the researchers on the study, they actually did the field work, with support and supervision obviously, but they actually did the field work. Perceived relevance of ethnicity and PROs to oncology "I think there is a little bit less concern in oncology with the demographic makeup of our sample. When you look at other therapeutic areas … PROs are more Prominence of PRO to study "I think there is a general concern with making sure that we have a very good representative sample, but it's not specific to PROs." [010] Limited detail in study protocol "It's not typically something that really is talked about in the protocol, we will use a validated linguistic and cultural adaptation of each instrument but there is not any language in the protocols usually about any differences in how the instruments are administered or anything like that … It's just the same procedures but same instrument in a different language. Using English as the default language "One of our eligibility criteria was that patients had to be able to converse in English to a high enough level to be able to complete the questionnaires." [016] Incompatible study design "The General Practices were randomised to be invited to [the study] … so we did not have any choice over the practices … There wasn't really the opportunity to do anything different. I think if we realised at the outset that we were not including as many ethnic minority men as we would have wished, we might have targeted some areas with higher levels of ethnic minorities but we did not notice that really until we looked at the final baseline figures." [028] Facilitators Ensuring diversity through review process "Ethics committees are often the ones where this is discussed … and so you might put an eligibility criteria (sic) in there which the ethics committee or others question and say, 'Could you broaden that out? Why does a patient need to be able to speak English to enter the trial? Could the questionnaires be translated?'" [016] Design study depending on target population "Generally, also our scientific questions, you'd want those kind of geared towards the population that you are registering. So, in a clinical trial that's focused on minority enrolment, our scientific questions for PROs would be geared towards those specific populations and the issues that are important to those patient groups but also kind of reflected in PROs that way." [022] Include relevant groups in study design "Researchers were told they had to go, to the minority groups who were likely to be involved in the work. These were nursing studies by the way not major medical treatment studies, they were told they had to go to members of the communities and recruit them to be involved in helping define the objectives of the study and the measures which the study was going to use.  Administrative challenges "We did have some centres abroad … but I think the quality of life data was more sporadic, less likely to come back from centres abroad and I think that that's probably an administrative thing, sending actual paper, quality of life returns from abroad is probably more than the centres abroad could cope with." [014] Reluctance of research organisations "What we found … was how using many of the common depressed mood surveys desperately under-reported the amount of depression and depressed mood in African American women. They tried to push [the funding body] to allow them to use different surveys that were more appropriate, they were newer ones though. They were not like your classic ones. They received a lot of pushback back then. That was early mid-90s and they said, 'No, these are the ones you use'." [011] Facilitators Pilot PRO instruments with target populations "Within our regular clinical trials, when we do a PRO component it is administered to all patients, including minorities. We have specifically targeted minorities for a couple of our studies. We've done a couple of feasibility studies for our new, newer, questionnaires … We did an enrichment strategy where we were purposefully trying to enrol minority folks because they are generally underrepresented on our clinical trials. And so to make sure that our tools and our message were applicable across a wide range of the population we did kind of purposefully try to enrol as many as minorities as we could." [022] Undertake research in diverse localities "The minority representation should be a consideration and I would think in large urban settings, there are numerous minorities and that would be a good base upon which to recruit minorities." [029] Monitor recruitment and address shortcomings in real time "That could be a nurse practitioner, a member of the investigative staff. A concerted effort. Maybe they'll say "Well we do not have enough representation with respect to minorities. What can we do to reach out?" So maybe having a member of the investigative team who is also a minority may reach out to fellow minorities in order to make

Theme 1: Recruitment
Several participants identified challenges regarding the recruitment of ethnically diverse samples within the countries in which the trials operate. They described the lack of resources for recruitment and inclusive recruitment strategies as flaws in the trial design and a barrier to recruitment. Some participants were pessimistic, stating that certain communities were wary of engaging with research (Table 4); however, it was considered that dynamic and flexible recruitment methods would overcome this. Participants described how communities are often receptive to efforts by researchers to meaningfully engage and better understand research priorities and concerns; this may be through working with community leaders and using peer-researchers. However, interviewees acknowledged that their recruitment efforts were sometimes ineffective, and more effort could be made to design better recruitment strategies when engaging with ethnically diverse groups. Participants also described how unsuccessful attempts to recruit diverse samples often meant that translated and culturally validated PROMs were rendered unnecessary.

Theme 2: Development of research questions and study design
The rationale and feasibility for a cancer clinical trial to purposively sample based on ethnicity and the impact on subsequent trial design was discussed. Participants described the incidence of cancer by ethnicity and questioned whether ethnicity was a necessary factor for exploration in these trials. It was considered that if participant samples were representative of the general population, there was no need to stratify results by ethnicity.
Participants described the extent to which translated and culturally validated PROMs were considered during the trial design process (and detailed in the protocol). This depended on the prominence of the PRO within the study and whether targeted recruitment of specific groups was compatible with the study design. It was also noted that fluency in English is often used as an eligibility criterion for PRO components. Participants described using English as the default language and this was considered standard practice in many studies, rendering the use of translated and culturally validated measures unnecessary.
Concerns were voiced about balancing the need for inclusivity without additionally burdening the investigator. However, participants also described how research questions formulated with consideration of the target population promotes the use of study design and PRO strategy that is appropriate and reflects the priorities of the groups. Nursing studies were given as examples of good practice in this regard, whereby researchers engaged actively with target populations to recruit community members to define study objectives and oversee the selection of measures. Participants described experiences of research ethics committees ensuring that eligibility criteria were not arbitrarily restrictive and ensuring trial designs accommodated the monitoring of recruitment to ensure a diverse sample was captured during trials.

Theme 3: Implementing inclusive PRO research
A recurring theme was the difficulty of ensuring translated and culturally validated PROMs were available and the time-consuming, expensive, labour-intensive nature of their use. One participant described how uptake of new measures that are validated with population subgroups can be undermined by the preference for older, more commonly used measures. Another explained how inclusive PRO strategies detailed in ethics committee applications may not be implemented in practice. One participant reported having used translated PROMs in other studies; however, we were unable to identify evidence of similar practices in any of the protocols or trial publications in our sample. Administrative difficulties and capacity restraints involved in collecting different versions of PROMs in multi-national trials were considered challenging. Where translations were not available, it was suggested researchers were dependent on translations of the questions by the recruiter for the participants. Several facilitators of inclusive PRO research were described. It was suggested that recruitment aims should be clear at the beginning of the design process and monitored throughout the study in real time to ensure a representative sample; making use of existing instruments that have been extensively translated and validated;

Discussion
In this review, we identified the extent to which ethnicity was reported in a cohort of cancer clinical trials and whether translated and culturally appropriate measures were used to capture PRO data. We examined the barriers and facilitators to using appropriate PROMs with ethnically diverse groups. Several findings emerged (Fig.  2). First, few trials reported the collection of data by ethnic groups despite many of the studies being multicentred and multi-national. Second, no trials including the multi-national studies reported using translated PROMs. Third, qualitative interviews highlighted significant barriers to the use of translated and culturally validated PROMs, including availability of measures, insufficient resources and training, investigator burden, and administrative difficulties associated with collecting different versions. The dearth of reporting in both protocols and publications raises several issues and questions: firstly, the extent to which patients are excluded because of language/ cultural barriers is not transparent, and secondly, failure to report the use of translated and culturally validated PROMs. This has a number of important implications for cancer clinical trials.

Recruitment
Where ethnicity data was reported, it was generally in relation to baseline characteristics. Most studies did not report ethnicity data, this may be due to small numbers being recruited or because it was not considered necessary to report this information. Eight of the included studies were multi-national, yet numbers reported for ethnicity were still low in comparison to the overall number recruited. Only three multinational studies recruited more than 15% from identified ethnic groups, and these were made up of predominantly East Asian and Asian populations. Interviewees identified a number of barriers including design and recruitment issues, limited time, training and resources, and staff preconceptions about recruiting ethnic group participants.

Development of research questions and study design
Respondents in the qualitative interviews raised several issues relating to the research design and study aims. Participants had differing views on whether stratification by ethnicity was relevant if samples were representative of the population. Trial designs can contribute to sample attrition and lack of transparency in relation to data collection and analysis. The recent development of the SPIRIT-PRO guidelines for trial protocol writers should help improve the design and transparency of clinical trials with a PRO element, ensuring future clinical trials will be more transparent and rigorously designed [35].
It was suggested that pre-testing of PROMs with ethnic groups during the development phase might identify potential translation issues and facilitate recruitment. Examples of successful facilitation of ethnic group recruitment included the use of peer-researchers and engagement with ethnic groups and community leaders.

Implementing inclusive PRO research
Studies have shown that PROMs provide a rich source of data and, alongside clinical data, can identify adverse events and the impact of therapeutics on quality of life [36]. Side effects of treatments may have a negative impact on quality of life. It is important these effects are captured for all participants, via their access to linguistically translated and culturally validated PROMs, preventing additional issues and potentially improving survival outcomes [37,38]. PROM data can then be used to assess the benefit-risk assessment for all patients in regulatory labelling claims [5,39].
Details of administration and use of culturally relevant PROMs in the protocols or publications were limited. Identified barriers included uncertainty about which translations would be required, difficulties with administration, and time and resources required to translate and utilise culturally validate PROMs. Availability of translated and culturally validated PROMs was a concern raised in some of the interviews. However, most of the studies in this review used measures such as the EORTC-C30, which has a wide range of validated translated versions and is freely available [13,15]. Whilst administration of different versions can be complex, details of their implementation should be reflected in the protocol [4]. There was no indication that translated versions were being used in the studies, one study stated that translated patient documents were available if required, but it was not clear if this included the PROMs. One participant in the interviews intimated that translated PROMs were being used but not reported in protocols or papers. Another suggested that even if it was stated that translations would be used it did not necessarily happen in practice.
Using culturally insensitive PROMs can add to attrition rates and high levels of missing data. Many researchers assume it is sufficient to use a linguistically validated PROM without ensuring cross-cultural equivalence [40]. Recent recommendations to ensure cross-cultural validity are important to consider when adapting PROMs [4,13,15,41]. If data is being pooled from different translations, it is important to establish conceptual and psychometric equivalence of data and this should be transparent in protocols and publications [40,41]. Psychometric validation of cross-cultural equivalence can be costly and time consuming to achieve [40]. There is also an issue around the sample sizes required to evaluate cross-cultural equivalence [4,40]. This may not be feasible in all cancers, especially rarer forms [42]. Another concern is the suggestion that researchers administering the PROM are translating questions for participants. This could potentially invalidate the data, and methods of administration should be made explicit in the protocol, as recommended by the SPIRIT-PRO extension guidelines [35]. Recent studies have demonstrated that the inclusion and implementation of PRO data in clinical trials is generally poor, and PRO data results are often not published [37]. We can only identify the differences or advantages of therapeutic options by assessing the therapeutic impacts of cancer treatments on quality of life and symptoms for all participants. The additional issues relating to exclusion of some participants either by design or by default threaten the external validity of clinical trials.
This study has demonstrated that reporting of ethnicity data in relation to PRO in clinical trials was limited, and transparency around the use of translated PROMs needs addressing. The SPIRIT-PRO extension provides international consensus-based guidance on protocol content for clinical trials including PRO. Use of these guidelines moving forward may improve the overall detail of trial protocols in relation to PRO data collection and administration. Indeed, some of the issues raised in discussion with stakeholders are addressed by many of the SPIRIT-PRO guidelines [35]. These include the importance of specifying PRO-specific language requirements, domains and concepts being used, sample size requirements for PRO endpoints, administration of PROMs, and whether more than one language version is being used [35]. PRO analysis plans may require adjustments based on ethnicity, with specific mention to include reference to exclusion or inclusion on the basis of language, and reporting of ethnicity data [35].
Reviewers and funders may need to be more cognisant about the use of translated and culturally validated PROMs especially when trials being reviewed are multinational. More transparency in the reporting of trials may also help judgements to be made on the applicability of clinical trials to all populations, thus helping to reduce health inequalities [43]. Research ethics committees could also provide a steer on the recruitment of ethnic groups and the use of translated PROMs, when reviewing clinical trial protocols. Ensuring transparency of recruitment procedures in relation to ethnic groups and monitoring recruitment and use of translations might also facilitate better engagement with ethnic group communities and the use of translated PROMs in the future.
This study is not without limitations. The review was reliant upon clinical trial protocols and publications based in an English-speaking country, and therefore, the use of English is standard as key sources of information pertaining to the operations and processes within the trial sample. The lack of transparency around the use of culturally and linguistically validated PROMs in the multi-national trials demonstrates a concerning pattern of non-inclusion of international trial participants in PRO components of clinical trials. A limitation of the qualitative component was that discussion largely related to the use of translated and culturally appropriate PROMs in the inclusion of linguistically and culturally diverse participants within a single-country trial, rather than the inclusion of international trial participants in multi-national trials, collecting PRO data. This may be an area to consider for future research.
Strengths of this study include the use of both an evaluation of trial protocols and publications and qualitative interviews with international stakeholders. Reviews of the protocols and publication identified the current state of play regarding recruitment and the use and reporting of translated and culturally validated PROMs in cancer clinical trials. Qualitative interviews allowed an in-depth exploration of the existing issues, as perceived by a range of international stakeholders.

Conclusions
Cancer clinical trials should be transparent in their recruitment strategies and demonstrate that recruitment is accessible to all representative populations and facilitated by the trial design, recruitment strategies, and the use of translated and culturally validated PROMs. Transparency in reporting and use of culturally as well as linguistically adapted PROMs needs to be demonstrated. Where data from different translated versions are aggregated, the validity of this approach should be demonstrated, including information around psychometric equivalence. The current level of reporting makes it difficult to quantify precisely the shortcomings in this area due to widespread omissions in reporting of ethnicity data and the use of translated and culturally validated PROMs in trial protocols and publications.
Funding bodies and research reviewers should consider the access of trial participants to appropriate PROMs. Reviewers and journal editors need to be aware of the need for appropriate reporting on the use of translated and culturally validated PROMs in multinational clinical trials. Future use of the SPIRIT-PRO guidelines may help prevent some of the omissions seen in the current use and reporting of appropriately translated PROMs. Increasing the transparency of PROM usage in future clinical trial reports and protocols will enable us to identify the extent to which cancer clinical trials are inclusive and patient centred.