Skip to main content

Systematic review of the use of translated patient-reported outcome measures in cancer trials

Abstract

Background

Patient-reported outcomes (PROs) are used in clinical trials to assess the effectiveness and tolerability of interventions. Inclusion of participants from different ethnic backgrounds is essential for generalisability of cancer trial results. PRO data collection should include appropriately translated patient-reported outcome measures (PROMs) to minimise missing data and sample attrition.

Methods

Protocols and/or publications from cancer clinical trials using a PRO endpoint and registered on the National Institute for Health Research Portfolio were systematically reviewed for information on recruitment, inclusion of ethnicity data, and use of appropriately translated PROMs. Semi-structured interviews were conducted with key stakeholders to explore barriers and facilitators for optimal PRO trial design, diverse recruitment and reporting, and use of appropriately translated PROMs.

Results

Eighty-four trials met the inclusion criteria, only 14 (17%) (n = 4754) reported ethnic group data, and ethnic group recruitment was low, 611 (13%). Although 8 (57%) studies were multi-centred and multi-national, none reported using translated PROMs, although available for 7 (88%) of the studies.

Interviews with 44 international stakeholders identified a number of perceived barriers to ethnically diverse recruitment including diverse participant engagement, relevance of ethnicity to research question, prominence of PROs, and need to minimise investigator burden. Stakeholders had differing opinions on the use of translated PROMs, the impact of trial designs, and recruitment strategies on diverse recruitment. Facilitators of inclusive research were described and examples of good practice identified.

Conclusions

Greater transparency is required when PROs are used as primary or secondary outcomes in clinical trials. Protocols and publications should demonstrate that recruitment was accessible to diverse populations and facilitated by trial design, recruitment strategies, and appropriate PROM usage. The use of translated PROMs should be made explicit when used in cancer clinical trials.

Peer Review reports

Highlights

  • Wide variation exists in the use of translated patient-reported outcome measures (PROMs) in multi-national clinical trials.

  • There is a lack of transparency around publishing/reporting ethnic group data and use of appropriately translated PROMs.

  • We identify key facilitators and barriers to using translated culturally sensitive PROMs in cancer clinical trials.

Introduction

Under-representation of biologically and culturally diverse populations in clinical trials can have implications for the generalisability of findings to practice [1]. Studies have shown that people from ethnic groups often have poorer healthcare experiences and face additional barriers when accessing services. Cancer incidence rates and mortality outcomes often differ, with ethnic groups presenting later leading to poorer health outcomes [2].

Patient-reported outcomes (PROs) are increasingly used to evaluate primary, secondary or exploratory outcomes in cancer clinical trials [3,4,5]. Patient-reported outcome measures (PROMs) are patient-completed measurement tools that capture PRO data and provide the patient’s perspective on their health and impairment status and the impact of symptoms and side effects on their health-related quality of life (HRQoL) [6, 7]. PROMs may identify additional patient impairments, treatment benefits, or harms that may be missed using clinical measures alone [8,9,10].

Measures which reflect the patient’s health and cultural experiences have been found to be more sensitive to change and more reliable measurement tools [11, 12]. The use of translated and culturally validated PROMs facilitates the inclusion of a wider range of participants. Culturally inappropriate PROMs may contribute to sample attrition and missing data due to misunderstood or culturally irrelevant items, limiting the ability to establish conceptual equivalence or the wisdom in aggregating patient-centred data in clinical trial results [13]. Guidelines have been developed to ensure that translation processes are rigorous and reflect required cultural perspectives [11, 13]. However, it is unclear if translated and culturally validated PROMs are being used in clinical trials with PRO endpoints [14] or to what extent this guidance is being adhered to in contemporary cancer clinical trials [4, 15].

This review aimed to establish reporting of ethnicity in cancer trials listed on the National Institute for Health Research (NIHR) Portfolio Database [16] and collecting PROM data. We also reviewed whether PRO data were captured using translated and culturally validated PROMs. The rationale is that to meaningfully accommodate linguistically and culturally diverse participants, translated and culturally validated PROMs should be used to facilitate data collection. We also undertook qualitative interviews with international stakeholders to understand current practice and associated barriers and facilitators to the use of translated and culturally validated PROMs in clinical trials [16].

Data collection

We conducted secondary data analysis of the Systematic Evaluation of Patient-Reported Outcome Protocol Content and Reporting in Cancer Trials (EPiC) study database. EPiC systematically evaluated completed cancer clinical trials collecting a PRO as a primary or secondary endpoint, identified on the NIHR Portfolio between January 2001 and April 2014 [16, 17]. The most up-to-date ethically approved trial protocol and any arising publications were retrieved. Two independent investigators (i) extracted trial characteristics, (ii) determined the availability of PRO results, (iii) evaluated general protocol and reporting quality using the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) 2013 and CONSORT (CONsolidated Standards of Reporting Trials) checklists respectively, and (iv) evaluated PRO protocol content and reporting using a bespoke protocol checklist and the CONSORT-PRO Extension [16, 17].

Review of protocols and publications

The current study included trials within the EPiC database that included ethnicity data in their protocols and/or publications. Two independent reviewers (KA/AS) screened the studies for information relating to recruitment, inclusion/exclusion criteria, use of culturally and linguistically appropriate consent forms, information sheets, and translated and culturally validated PROMs, stratified by ethnicity. Disagreements were resolved via discussion with a third reviewer where required (MC/DK).

Qualitative study

Ethics approval was obtained from the University of Birmingham Ethics Committee (ERN_17-0085A) and participants consented to take part in interviews. Participants were purposively sampled and had experience of PROMs, through either participation in a trial or professional experience of designing and reviewing clinical trials [16]. Participants were drawn from four groups: (1) trialists and chief investigators with experience of cancer trials collecting a PRO as a primary or secondary endpoint, (2) individuals with lived experience of cancer, (3) international experts in oncology PRO trial design, and (4) journal editors, funding panellists, and regulatory board members [16].

Semi-structured face-to-face or telephone interviews were conducted with individuals from the stakeholder groups. An experienced qualitative researcher (AR) conducted the interviews using a topic guide; questions relating to the use of PRO and translated and culturally validated PROMs were explored, subsequent probes depended on the individuals’ experience. Barriers and facilitators to optimal PRO trial design, recruitment, collection, and reporting, in relation to the use of translated and culturally validated PROMs, were discussed.

Interviews were audio recorded and transcribed verbatim by a professional transcription service and analysed by AR using directed thematic analysis [16]. Findings from previous qualitative and review work, coupled with analysis of the protocols and publications, underpinned the development of an initial coding framework. A broad code to extract items pertaining to the collection and reporting of PRO data using translated and culturally validated PROMs was used. These were coded and sub-codes were then formulated iteratively. Formal triangulation of coding was conducted by DK/MC.

Results

EPiC review results

The EPiC review identified 1141 trials across the NIHR Portfolio Database. Of these, 228 met the eligibility criteria, 160 studies had published their primary results within the review timeframe. In total, there were 84 trials for which a matching protocol and arising publication were retrieved (see Fig. 1 and Kyte et al.) [18]. Of these 84 trials, only 14 (17%) reported any type of ethnic group profile data (Table 1), 8 out of 14 (57%) were multi-national clinical trials, and the remainder were United Kingdom (UK) based.

Fig. 1
figure 1

Study selection of cancer clinical trials using PROMs and reporting ethnic group data

Table 1 Type of trial and reporting of ethnic group profile data

Reporting ethnicity

Ethnicity profile data were generally reported in relation to baseline participant characteristics. The most frequently described ethnic groups were African, African-American, Afro-Caribbean, and Asian/Indian (Table 1). Three trials specifically excluded non-English speakers [29, 30, 33]. The numbers of participants represented by ethnicity data were small (611, 13%) in comparison to the total numbers of participants recruited across the 14 studies (4754), although in the UK this would have reflected current population norms (Table 1). Only three multi-national studies reported ethnicity numbers above 15% of their total population, primarily categorised as East Asian or Asian participants (Table 1) [20, 24, 26]. One study stratified survival results by ethnicity [20].

Use of translated and culturally validated PROMs

Eight (57%) of the 14 included studies were multi-centre, multi-national trials. The use of translated PROMs was not reported in any of the trial protocols or publications despite 7 (88%) using PROMs that have been translated into other languages (Table 2). It was not clear if missing data were related to the availability of translated and culturally validated PROMs. One study stated in the protocol that translated patient documents would be available if required. However, it was not clear if this included translated PROMs [26, 27].

Table 2 PROM usage and translations available in the multi-national studies

Qualitative results

Forty-four interviews were undertaken with a broad range of international stakeholders (Table 3). Issues highlighted by the interviewees were similar despite the diversity of professional and national backgrounds. Three main themes emerged from the qualitative data: (1) recruitment, (2) development of research questions and study design, and (3) implementing translated and culturally validated PROMs (Table 4). Key elements with interview quotations are illustrated in Table 4. Discussions relating to the use of translated and culturally appropriate PROMs referred to the inclusion of linguistically and culturally diverse participants within a single-country trial, rather than the inclusion of international trial participants in multi-national trials, collecting PROMs.

Table 3 Characteristics of qualitative study participants
Table 4 Research stage and themes

Theme 1: Recruitment

Several participants identified challenges regarding the recruitment of ethnically diverse samples within the countries in which the trials operate. They described the lack of resources for recruitment and inclusive recruitment strategies as flaws in the trial design and a barrier to recruitment.

Some participants were pessimistic, stating that certain communities were wary of engaging with research (Table 4); however, it was considered that dynamic and flexible recruitment methods would overcome this. Participants described how communities are often receptive to efforts by researchers to meaningfully engage and better understand research priorities and concerns; this may be through working with community leaders and using peer-researchers. However, interviewees acknowledged that their recruitment efforts were sometimes ineffective, and more effort could be made to design better recruitment strategies when engaging with ethnically diverse groups. Participants also described how unsuccessful attempts to recruit diverse samples often meant that translated and culturally validated PROMs were rendered unnecessary.

Theme 2: Development of research questions and study design

The rationale and feasibility for a cancer clinical trial to purposively sample based on ethnicity and the impact on subsequent trial design was discussed. Participants described the incidence of cancer by ethnicity and questioned whether ethnicity was a necessary factor for exploration in these trials. It was considered that if participant samples were representative of the general population, there was no need to stratify results by ethnicity.

Participants described the extent to which translated and culturally validated PROMs were considered during the trial design process (and detailed in the protocol). This depended on the prominence of the PRO within the study and whether targeted recruitment of specific groups was compatible with the study design. It was also noted that fluency in English is often used as an eligibility criterion for PRO components. Participants described using English as the default language and this was considered standard practice in many studies, rendering the use of translated and culturally validated measures unnecessary.

Concerns were voiced about balancing the need for inclusivity without additionally burdening the investigator. However, participants also described how research questions formulated with consideration of the target population promotes the use of study design and PRO strategy that is appropriate and reflects the priorities of the groups. Nursing studies were given as examples of good practice in this regard, whereby researchers engaged actively with target populations to recruit community members to define study objectives and oversee the selection of measures. Participants described experiences of research ethics committees ensuring that eligibility criteria were not arbitrarily restrictive and ensuring trial designs accommodated the monitoring of recruitment to ensure a diverse sample was captured during trials.

Theme 3: Implementing inclusive PRO research

A recurring theme was the difficulty of ensuring translated and culturally validated PROMs were available and the time-consuming, expensive, labour-intensive nature of their use. One participant described how uptake of new measures that are validated with population subgroups can be undermined by the preference for older, more commonly used measures. Another explained how inclusive PRO strategies detailed in ethics committee applications may not be implemented in practice. One participant reported having used translated PROMs in other studies; however, we were unable to identify evidence of similar practices in any of the protocols or trial publications in our sample. Administrative difficulties and capacity restraints involved in collecting different versions of PROMs in multi-national trials were considered challenging. Where translations were not available, it was suggested researchers were dependent on translations of the questions by the recruiter for the participants. Several facilitators of inclusive PRO research were described. It was suggested that recruitment aims should be clear at the beginning of the design process and monitored throughout the study in real time to ensure a representative sample; making use of existing instruments that have been extensively translated and validated; piloting PROMs with the target community groups, adopting an “enrichment” strategy to promote recruitment; and recruitment in diverse localities. Participants described the role of key research institutions and suggested ethics committees could request details pertaining to diverse recruitment and diversity targets could be linked to funding. Thus, following successful examples, such as that of Patient and Public Involvement (PPI) initiatives, enables diverse trial participation in clinical trials.

Discussion

In this review, we identified the extent to which ethnicity was reported in a cohort of cancer clinical trials and whether translated and culturally appropriate measures were used to capture PRO data. We examined the barriers and facilitators to using appropriate PROMs with ethnically diverse groups. Several findings emerged (Fig. 2). First, few trials reported the collection of data by ethnic groups despite many of the studies being multi-centred and multi-national. Second, no trials including the multi-national studies reported using translated PROMs. Third, qualitative interviews highlighted significant barriers to the use of translated and culturally validated PROMs, including availability of measures, insufficient resources and training, investigator burden, and administrative difficulties associated with collecting different versions.

Fig. 2
figure 2

Resources required and barriers and facilitators to engaging with ethnic group participants in clinical trials

The dearth of reporting in both protocols and publications raises several issues and questions: firstly, the extent to which patients are excluded because of language/cultural barriers is not transparent, and secondly, failure to report the use of translated and culturally validated PROMs. This has a number of important implications for cancer clinical trials.

Recruitment

Where ethnicity data was reported, it was generally in relation to baseline characteristics. Most studies did not report ethnicity data, this may be due to small numbers being recruited or because it was not considered necessary to report this information. Eight of the included studies were multi-national, yet numbers reported for ethnicity were still low in comparison to the overall number recruited. Only three multi-national studies recruited more than 15% from identified ethnic groups, and these were made up of predominantly East Asian and Asian populations. Interviewees identified a number of barriers including design and recruitment issues, limited time, training and resources, and staff preconceptions about recruiting ethnic group participants.

Development of research questions and study design

Respondents in the qualitative interviews raised several issues relating to the research design and study aims. Participants had differing views on whether stratification by ethnicity was relevant if samples were representative of the population. Trial designs can contribute to sample attrition and lack of transparency in relation to data collection and analysis. The recent development of the SPIRIT-PRO guidelines for trial protocol writers should help improve the design and transparency of clinical trials with a PRO element, ensuring future clinical trials will be more transparent and rigorously designed [35].

It was suggested that pre-testing of PROMs with ethnic groups during the development phase might identify potential translation issues and facilitate recruitment. Examples of successful facilitation of ethnic group recruitment included the use of peer-researchers and engagement with ethnic groups and community leaders.

Implementing inclusive PRO research

Studies have shown that PROMs provide a rich source of data and, alongside clinical data, can identify adverse events and the impact of therapeutics on quality of life [36]. Side effects of treatments may have a negative impact on quality of life. It is important these effects are captured for all participants, via their access to linguistically translated and culturally validated PROMs, preventing additional issues and potentially improving survival outcomes [37, 38]. PROM data can then be used to assess the benefit-risk assessment for all patients in regulatory labelling claims [5, 39].

Details of administration and use of culturally relevant PROMs in the protocols or publications were limited. Identified barriers included uncertainty about which translations would be required, difficulties with administration, and time and resources required to translate and utilise culturally validate PROMs. Availability of translated and culturally validated PROMs was a concern raised in some of the interviews. However, most of the studies in this review used measures such as the EORTC-C30, which has a wide range of validated translated versions and is freely available [13, 15]. Whilst administration of different versions can be complex, details of their implementation should be reflected in the protocol [4]. There was no indication that translated versions were being used in the studies, one study stated that translated patient documents were available if required, but it was not clear if this included the PROMs. One participant in the interviews intimated that translated PROMs were being used but not reported in protocols or papers. Another suggested that even if it was stated that translations would be used it did not necessarily happen in practice.

Using culturally insensitive PROMs can add to attrition rates and high levels of missing data. Many researchers assume it is sufficient to use a linguistically validated PROM without ensuring cross-cultural equivalence [40]. Recent recommendations to ensure cross-cultural validity are important to consider when adapting PROMs [4, 13, 15, 41]. If data is being pooled from different translations, it is important to establish conceptual and psychometric equivalence of data and this should be transparent in protocols and publications [40, 41]. Psychometric validation of cross-cultural equivalence can be costly and time consuming to achieve [40]. There is also an issue around the sample sizes required to evaluate cross-cultural equivalence [4, 40]. This may not be feasible in all cancers, especially rarer forms [42]. Another concern is the suggestion that researchers administering the PROM are translating questions for participants. This could potentially invalidate the data, and methods of administration should be made explicit in the protocol, as recommended by the SPIRIT-PRO extension guidelines [35].

Recent studies have demonstrated that the inclusion and implementation of PRO data in clinical trials is generally poor, and PRO data results are often not published [37]. We can only identify the differences or advantages of therapeutic options by assessing the therapeutic impacts of cancer treatments on quality of life and symptoms for all participants. The additional issues relating to exclusion of some participants either by design or by default threaten the external validity of clinical trials.

This study has demonstrated that reporting of ethnicity data in relation to PRO in clinical trials was limited, and transparency around the use of translated PROMs needs addressing. The SPIRIT-PRO extension provides international consensus-based guidance on protocol content for clinical trials including PRO. Use of these guidelines moving forward may improve the overall detail of trial protocols in relation to PRO data collection and administration. Indeed, some of the issues raised in discussion with stakeholders are addressed by many of the SPIRIT-PRO guidelines [35]. These include the importance of specifying PRO-specific language requirements, domains and concepts being used, sample size requirements for PRO endpoints, administration of PROMs, and whether more than one language version is being used [35]. PRO analysis plans may require adjustments based on ethnicity, with specific mention to include reference to exclusion or inclusion on the basis of language, and reporting of ethnicity data [35].

Reviewers and funders may need to be more cognisant about the use of translated and culturally validated PROMs especially when trials being reviewed are multi-national. More transparency in the reporting of trials may also help judgements to be made on the applicability of clinical trials to all populations, thus helping to reduce health inequalities [43]. Research ethics committees could also provide a steer on the recruitment of ethnic groups and the use of translated PROMs, when reviewing clinical trial protocols. Ensuring transparency of recruitment procedures in relation to ethnic groups and monitoring recruitment and use of translations might also facilitate better engagement with ethnic group communities and the use of translated PROMs in the future.

This study is not without limitations. The review was reliant upon clinical trial protocols and publications based in an English-speaking country, and therefore, the use of English is standard as key sources of information pertaining to the operations and processes within the trial sample. The lack of transparency around the use of culturally and linguistically validated PROMs in the multi-national trials demonstrates a concerning pattern of non-inclusion of international trial participants in PRO components of clinical trials. A limitation of the qualitative component was that discussion largely related to the use of translated and culturally appropriate PROMs in the inclusion of linguistically and culturally diverse participants within a single-country trial, rather than the inclusion of international trial participants in multi-national trials, collecting PRO data. This may be an area to consider for future research.

Strengths of this study include the use of both an evaluation of trial protocols and publications and qualitative interviews with international stakeholders. Reviews of the protocols and publication identified the current state of play regarding recruitment and the use and reporting of translated and culturally validated PROMs in cancer clinical trials. Qualitative interviews allowed an in-depth exploration of the existing issues, as perceived by a range of international stakeholders.

Conclusions

Cancer clinical trials should be transparent in their recruitment strategies and demonstrate that recruitment is accessible to all representative populations and facilitated by the trial design, recruitment strategies, and the use of translated and culturally validated PROMs. Transparency in reporting and use of culturally as well as linguistically adapted PROMs needs to be demonstrated. Where data from different translated versions are aggregated, the validity of this approach should be demonstrated, including information around psychometric equivalence. The current level of reporting makes it difficult to quantify precisely the shortcomings in this area due to widespread omissions in reporting of ethnicity data and the use of translated and culturally validated PROMs in trial protocols and publications.

Funding bodies and research reviewers should consider the access of trial participants to appropriate PROMs. Reviewers and journal editors need to be aware of the need for appropriate reporting on the use of translated and culturally validated PROMs in multi-national clinical trials. Future use of the SPIRIT-PRO guidelines may help prevent some of the omissions seen in the current use and reporting of appropriately translated PROMs. Increasing the transparency of PROM usage in future clinical trial reports and protocols will enable us to identify the extent to which cancer clinical trials are inclusive and patient centred.

Availability of data and materials

Data will be available from the author on reasonable request. Please contact the corresponding author Dr. Anita Slade (a.l.slade@bham.ac.uk).

Abbreviations

PROs:

Patient-reported outcomes

PROMs:

Patient-reported outcome measures

NIHR:

The National Institute for Health Research

EPiC:

Systematic Evaluation of Patient-Reported Outcome Protocol Content and Reporting in Cancer Trials

PPI:

Patient and Public Involvement

HRQoL:

Health-related quality of life

BPI-SF:

Brief Pain Inventory – Short Form

DAS:

Diarrhoea Assessment Scale

EORTC QLQ-C30:

European Organisation for Research and Treatment of Cancer core quality of life—additional specific cancer modules include QLQ-EN24 (endometrial cancer) and QLQ-HDC29 (high-dose chemotherapy)

EQ-5D-5L:

Euro-Qol quality of life questionnaire

FACIT:

Functional Assessment of Chronic Illness Therapy includes a range of measures including FACT-B, Breast Cancer module Scale; FACT-B TOI-PFB, Trial Outcome Index Physical/Functional/Breast; FACIT-Fatigue, Fatigue; FACIT-G, General; and FACT-KSI, Kidney Symptom Index

HAD:

Hospital Anxiety and Depression

KPS:

Karnofksy Performance Status scale

OMDQ:

Oral Mucositis Daily Questionnaire

MAT:

MASCC Antiemesis Tool

MFI:

Multidimensional Fatigue Inventory

RI:

Rhodes Index of Nausea, Vomiting and Retching SDI-21 Social Difficulties Index

STAI-S:

State Trait Anxiety Inventory

SQLQ:

Supplementary quality of life questionnaire

References

  1. Bartlett C, Doyal L, Ebrahim S, et al. The causes and effects of socio-demographic exclusions from clinical trials. Health Technol Assess. 2005;9(38)

  2. Cancer Research UK. In: Gordon-Dseagu V, editor. Cancer and health inequalities: an introduction to current evidence. London: Cancer Research UK; 2008.

    Google Scholar 

  3. Mercieca-Bebber R, Rouette J, Calvert M, et al. Preliminary evidence on the uptake, use and benefits of the CONSORT-PRO extension. Qual Life Res. 2017;26(6):1427–37. https://doi.org/10.1007/s11136-017-1508-6.

    Article  PubMed  Google Scholar 

  4. Wild D, Eremenco S, Mear I, Martin M, Houchin C, Gawlicki M, Hareendran A, Wiklund I, Chong LY, von Maltzahn R, Cohen L, Molsen E. Multinational trials—recommendations on the translations required, approaches to using the same language in different countries, and the approaches to support pooling the data: the ISPOR patient-reported outcomes translation and linguistic validation good research practices task force report. Value Health. 2009;12(4):430–40. https://doi.org/10.1111/j.1524-4733.2008.00471.x.

    Article  PubMed  Google Scholar 

  5. Basch E, Iasonos A, McDonough T, Barz A, Culkin A, Kris MG, Scher HI, Schrag D. Patient versus clinician symptom reporting using the National Cancer Institute Common Terminology Criteria for Adverse Events: results of a questionnaire-based study. Lancet Oncol. 2006;7(11):903–9. https://doi.org/10.1016/S1470-2045(06)70910-X.

    Article  PubMed  Google Scholar 

  6. Weldring T, Smith SMS. Patient-reported outcomes (PROs) and patient-reported outcome measures (PROMs). Health Serv Insights. 2013;6:61–8. https://doi.org/10.4137/HSI.S11093.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Calvert M, Blazeby J, Altman DG, Revicki DA, Moher D, Brundage MD, CONSORT PRO Group. Reporting of patient reported outcomes in randomised trials: the CONSORT PRO extension. JAMA. 2013;309(8):814–22. https://doi.org/10.1001/jama.2013.879.

    Article  CAS  PubMed  Google Scholar 

  8. Ahmed S, Berzon RA, Revicki DA, Lenderking WR, Moinpour CM, Basch E, Reeve BB, Wu AW, International Society for Quality of Life Research. The use of patient-reported outcomes (PRO) within comparative effectiveness research: implications for clinical practice and health care policy. Med Care. 2012;50(12):1060–70. https://doi.org/10.1097/MLR.0b013e318268aaff.

    Article  PubMed  Google Scholar 

  9. Wilson R. Patient led PROMs must take centre stage in cancer research. Res Involve Engage. 2018;4(1):7. https://doi.org/10.1186/s40900-018-0092-4.

    Article  Google Scholar 

  10. Avery K, Francischetto EO, Velikova G, et al. Assessment of patient-reported adverse events after discharge from hospital in RCTs in gastrointestinal cancer surgery: is there sufficient coverage in existing EORTC questionnaires? Trials. 2015;16(2):48.

    Article  Google Scholar 

  11. Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E, Ring L. Content validity—establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 2—assessing respondent understanding. Value Health. 2011;14(8):978–88. https://doi.org/10.1016/j.jval.2011.06.013.

    Article  PubMed  Google Scholar 

  12. Patrick DL, Burke LB, Gwaltney CJ, et al. Establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 1. Value in Health. 2014;14(8):967–77.

    Article  Google Scholar 

  13. Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, Erikson P, ISPOR Task Force for Translation and Cultural Adaptation. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR task force for translation and cultural adaptation. Value Health. 2005;8(2):94–104. https://doi.org/10.1111/j.1524-4733.2005.04054.x.

    Article  PubMed  Google Scholar 

  14. Calvert M, Kyte D, Duffy H, Gheorghe A, Mercieca-Bebber R, Ives J, Draper H, Brundage M, Blazeby J, King M. Patient-reported outcome (PRO) assessment in clinical trials: a systematic review of guidance for trial protocol writers. PLoS One. 2014;9(10):e110216. https://doi.org/10.1371/journal.pone.0110216.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. EORTC. EORTC quality of life group translation procedure. In. Edited by Group EQoL, Fourth edn. Brussels: European Organisation for Research and Treatment of Cancer; 2017: 26.

  16. Retzer A, Keeley T, Ahmed K, et al. Evaluation of patient-reported outcome protocol content and reporting in UK cancer clinical trials: the EPiC study qualitative protocol. BMJ Open. 2018;8(2)

  17. Ahmed K, Kyte D, Keeley T, Efficace F, Armes J, Brown JM, Calman L, Copland C, Gavin A, Glaser A, Greenfield DM, Lanceley A, Taylor R, Velikova G, Brundage M, Mercieca-Bebber R, King MT, Calvert M. Systematic evaluation of patient-reported outcome (PRO) protocol content and reporting in UK cancer clinical trials: the EPiC study protocol. BMJ Open. 2016;6(9):e012863. https://doi.org/10.1136/bmjopen-2016-012863.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Kyte D, Retzer A, Ahmed K, Keeley T, Armes J, Brown JM, Calman L, Gavin A, Glaser AW, Greenfield DM, Lanceley A, Taylor RM, Velikova G, Brundage M, Efficace F, Mercieca-Bebber R, King MT, Turner G, Calvert M. Systematic evaluation of patient-reported outcome protocol content and reporting in cancer trials. J Natl Cancer Inst. 2019;111(11):1170–8. https://doi.org/10.1093/jnci/djz038.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Armstrong AJ, Halabi S, Eisen T, Broderick S, Stadler WM, Jones RJ, Garcia JA, Vaishampayan UN, Picus J, Hawkins RE, Hainsworth JD, Kollmannsberger CK, Logan TF, Puzanov I, Pickering LM, Ryan CW, Protheroe A, Lusk CM, Oberg S, George DJ. Everolimus versus sunitinib for patients with metastatic non-clear cell renal cell carcinoma (ASPEN): a multicentre, open-label, randomised phase 2 trial. Lancet Oncol. 2016;17(3):378–88. https://doi.org/10.1016/S1470-2045(15)00515-X.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. Yao JC, Fazio N, Singh S, Buzzoni R, Carnaghi C, Wolin E, Tomasek J, Raderer M, Lahner H, Voi M, Pacaud LB, Rouyrre N, Sachs C, Valle JW, Fave GD, van Cutsem E, Tesselaar M, Shimada Y, Oh DY, Strosberg J, Kulke MH, Pavel ME, RAD001 in Advanced Neuroendocrine Tumours, Fourth Trial (RADIANT-4) Study Group. Everolimus for the treatment of advanced, non-functional neuroendocrine tumours of the lung or gastrointestinal tract (RADIANT-4): a randomised, placebo-controlled, phase 3 study. Lancet. 2016;387(10022):968–77. https://doi.org/10.1016/S0140-6736(15)00817-X.

    Article  CAS  PubMed  Google Scholar 

  21. Schöffski P, Chawla S, Maki RG, Italiano A, Gelderblom H, Choy E, Grignani G, Camargo V, Bauer S, Rha SY, Blay JY, Hohenberger P, D’Adamo D, Guo M, Chmielowski B, le Cesne A, Demetri GD, Patel SR. Eribulin versus dacarbazine in previously treated patients with advanced liposarcoma or leiomyosarcoma: a randomised, open-label, multicentre, phase 3 trial. Lancet. 2016;387(10028):1629–37. https://doi.org/10.1016/S0140-6736(15)01283-0.

    Article  CAS  PubMed  Google Scholar 

  22. Hudgens S, Forsythe A, Kontoudis I, et al. Evaluation of quality of life at progression in patients with soft tissue sarcoma. Sarcoma. 2017;2017:8.

    Article  Google Scholar 

  23. O'Shaughnessy J, Campone M, Brain E, Neven P, Hayes D, Bondarenko I, Griffin TW, Martin J, de Porre P, Kheoh T, Yu MK, Peng W, Johnston S. Abiraterone acetate, exemestane or the combination in postmenopausal patients with estrogen receptor-positive metastatic breast cancer†. Ann Oncol. 2015;27(1):106–13. https://doi.org/10.1093/annonc/mdv487.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Senan S, Brade A, Wang L-h, et al. PROCLAIM: randomized phase III trial of pemetrexed-cisplatin or etoposide-cisplatin plus thoracic radiation therapy followed by consolidation chemotherapy in locally advanced nonsquamous non-small-cell lung cancer. J Clin Oncol. 2016;34(9):953–62. https://doi.org/10.1200/JCO.2015.64.8824.

    Article  CAS  PubMed  Google Scholar 

  25. Ryan CW, Merimsky O, Agulnik M, Blay JY, Schuetze SM, van Tine BA, Jones RL, Elias AD, Choy E, Alcindor T, Keedy VL, Reed DR, Taub RN, Italiano A, Garcia del Muro X, Judson IR, Buck JY, Lebel F, Lewis JJ, Maki RG, Schöffski P. PICASSO III: a phase III, placebo-controlled study of doxorubicin with or without palifosfamide in patients with metastatic soft tissue sarcoma. J Clin Oncol. 2016;34(32):3898–905. https://doi.org/10.1200/JCO.2016.67.6684.

    Article  CAS  PubMed  Google Scholar 

  26. Verma S, Miles D, Gianni L, Krop IE, Welslau M, Baselga J, Pegram M, Oh DY, Diéras V, Guardino E, Fang L, Lu MW, Olsen S, Blackwell K, EMILIA Study Group. Trastuzumab emtansine for HER2-positive advanced breast cancer. N Engl J Med. 2012;367(19):1783–91. https://doi.org/10.1056/NEJMoa1209124.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Welslau M, Diéras V, Sohn J-H, Hurvitz SA, Lalla D, Fang L, Althaus B, Guardino E, Miles D. Patient-reported outcomes from EMILIA, a randomized phase 3 study of trastuzumab emtansine (T-DM1) versus capecitabine and lapatinib in human epidermal growth factor receptor 2–positive locally advanced or metastatic breast cancer. Cancer. 2014;120(5):642–51. https://doi.org/10.1002/cncr.28465.

    Article  CAS  PubMed  Google Scholar 

  28. Escudier B, Porta C, Bono P, Powles T, Eisen T, Sternberg CN, Gschwend JE, de Giorgi U, Parikh O, Hawkins R, Sevin E, Négrier S, Khan S, Diaz J, Redhu S, Mehmud F, Cella D. Randomized, controlled, double-blind, cross-over trial assessing treatment preference for pazopanib versus sunitinib in patients with metastatic renal cell carcinoma: PISCES study. J Clin Oncol. 2014;32(14):1412–8. https://doi.org/10.1200/JCO.2013.50.8267.

    Article  CAS  PubMed  Google Scholar 

  29. Beaver K, Williamson S, Sutton C, Hollingworth W, Gardner A, Allton B, Abdel-Aty M, Blackwood K, Burns S, Curwen D, Ghani R, Keating P, Murray S, Tomlinson A, Walker B, Willett M, Wood N, Martin-Hirsch P. Comparing hospital and telephone follow-up for patients treated for stage–I endometrial cancer (ENDCAT trial): a randomised, multicentre, non-inferiority trial. BJOG Int J Obstet Gynaecol. 2016;124(1):150–60. https://doi.org/10.1111/1471-0528.14000.

    Article  Google Scholar 

  30. Horne B, Newsham A, Velikova G, Liebersbach S, Gilleece M, Wright P. Development and evaluation of a specifically designed website for haematopoietic stem cell transplant patients in Leeds. Eur J Cancer Care. 2015;25(3):402–18. https://doi.org/10.1111/ecc.12352.

    Article  Google Scholar 

  31. Molassiotis A, Bardy J, Finnegan-John J, Mackereth P, Ryder DW, Filshie J, Ream E, Richardson A. Acupuncture for cancer-related fatigue in patients with breast cancer: a pragmatic randomized controlled trial. J Clin Oncol. 2012;30(36):4470–6. https://doi.org/10.1200/JCO.2012.41.6222.

    Article  PubMed  Google Scholar 

  32. Molassiotis A, Russell W, Hughes J, Breckons M, Lloyd-Williams M, Richardson J, Hulme C, Brearley SG, Campbell M, Garrow A, Ryder WD. The effectiveness of acupressure for the control and management of chemotherapy-related acute and delayed nausea: a randomized controlled trial. J Pain Symptom Manag. 2014;47(1):12–25. https://doi.org/10.1016/j.jpainsymman.2013.03.007.

    Article  Google Scholar 

  33. Neal RD, Nafees S, Pasterfield D, Hood K, Hendry M, Gollins S, Makin M, Stuart N, Turner J, Carter B, Wilkinson C, Williams N, Robling M. Patient-reported measurement of time to diagnosis in cancer: development of the Cancer Symptom Interval Measure (C-SIM) and randomised controlled trial of method of delivery. BMC Health Serv Res. 2014;14(1):3. https://doi.org/10.1186/1472-6963-14-3.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Jones L, Harrington J, Barlow CA, Tookman A, Drake R, Barnes K, King M. Advance care planning in advanced cancer: can it be achieved? An exploratory randomized patient preference trial of a care planning discussion. Palliat Support Care. 2011;9(1):3–13. https://doi.org/10.1017/S1478951510000490.

    Article  PubMed  Google Scholar 

  35. Calvert M, Kyte D, Mercieca-Bebber R, Slade A, Chan AW, King MT, and the SPIRIT-PRO Group, Hunn A, Bottomley A, Regnault A, Chan AW, Ells C, O’Connor D, Revicki D, Patrick D, Altman D, Basch E, Velikova G, Price G, Draper H, Blazeby J, Scott J, Coast J, Norquist J, Brown J, Haywood K, Johnson LL, Campbell L, Frank L, von Hildebrand M, Brundage M, Palmer M, Kluetz P, Stephens R, Golub RM, Mitchell S, Groves T. Guidelines for inclusion of patient-reported outcomes in clinical trial protocols: the SPIRIT-PRO extension. JAMA. 2018;319(5):483–94. https://doi.org/10.1001/jama.2017.21903.

    Article  PubMed  Google Scholar 

  36. Nissen A, Bager L, Pappot H. The use of PRO in adverse event identification during cancer therapy – choosing the right questions to ask. Acta Oncol. 2019;58(5):596–602. https://doi.org/10.1080/0284186X.2018.1560496.

    Article  PubMed  Google Scholar 

  37. Wilson MK, Collyar D, Chingos DT, Friedlander M, Ho TW, Karakasis K, Kaye S, Parmar MKB, Sydes MR, Tannock IF, Oza AM. Outcomes and endpoints in cancer trials: bridging the divide. Lancet Oncol. 2015;16(1):e43–52. https://doi.org/10.1016/S1470-2045(14)70380-8.

    Article  PubMed  Google Scholar 

  38. Dueck AC, Denicoff AM, O’Mara AM, et al. Recommended patient-reported core set of symptoms to measure in adult cancer treatment trials. J Natl Cancer Instit. 2014;106(7)

  39. King-Kallimanis BL, Howie LJ, Roydhouse JK, Singh H, Theoret MR, Blumenthal GM, Kluetz PG. Patient reported outcomes in anti-PD-1/PD-L1 inhibitor immunotherapy registration trials: FDA analysis of data submitted and future directions. Clinical Trials. 2019;16(3):322–6. https://doi.org/10.1177/1740774519836991.

    Article  PubMed  Google Scholar 

  40. Tennant A, Penta M, Tesio L, Grimby G, Thonnard JL, Slade A, Lawton G, Simone A, Carter J, Lundgren-Nilsson Å, Tripolski M, Ring H, Biering-Sørensen F, Marincek Č, Burger H, Phillips S. Assessing and adjusting for cross-cultural validity of impairment and activity limitation scales through differential item functioning within the framework of the Rasch model - the PRO-ESOR project. Med Care. 2004;42(1):37–48. https://doi.org/10.1097/01.mlr.0000103529.63132.77.

    Article  Google Scholar 

  41. Breugelmans R. Dangers in using translated medical questionnaires: the importance of conceptual equivalence across languages and cultures in patient-reported outcome measures. Chest. 2009;136(4):1175–7. https://doi.org/10.1378/chest.09-1684.

    Article  PubMed  Google Scholar 

  42. Slade A, Isa F, Kyte D, Pankhurst T, Kerecuk L, Ferguson J, Lipkin G, Calvert M. Patient reported outcome measures in rare diseases: a narrative review. Orphanet J Rare Dis. 2018;13(1):61. https://doi.org/10.1186/s13023-018-0810-x.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Welch VA, Norheim OF, Jull J, Cookson R, Sommerfelt H, Tugwell P. CONSORT-Equity 2017 extension and elaboration for better reporting of health equity in randomised trials. BMJ. 2017;359:j5085.

Download references

Acknowledgements

We wish to thank Anita Walker for her administrative support.

Disclaimers

The views expressed in this article are those of the author(s) and not necessarily those of the National Institute for Health Research or the Department of Health and Social Care.

Funding

This work was supported by funding from the National Institute for Health Research, Clinical Research Network funding (Grant number RHJK20013), and Macmillan Cancer Support (Grant number 5592105). This paper presents independent research supported by the National Institute of Health Research Birmingham Biomedical Research Centre at the University Hospitals Birmingham National Health Service Foundation Trust and the University of Birmingham. The study funders did not have any role in the study design; the collection, analysis, and interpretation of the data; the writing of the report; or the decision to submit the article for publication.

ALS was supported by funding from Macmillan Cancer Support and reports funding by British Heart Foundation, National Institute for Health Research Invention for Innovation, Medical Research Council, National Institute for Health Research MedTech and INvitro diagnostics Co-operative – Trauma management, Eye Hope Foundation, Association du Syndrome de Wolfram, Snow Foundation; Pfizer Health Research Foundation; National Institute for Health Research Clinical Research Network; and National Institute For Health Research Birmingham Biomedical Research Centre and National Institute For Health Research Surgical Reconstruction and Microbiology Research Centre at the University of Birmingham and University Hospitals Birmingham National Health Service Foundation Trust outside the submitted work.

AR was supported by funding from Macmillan Cancer Support and reports funding by the National Institute for Health Research Clinical Research Network and Innovate UK (part of UK Research and Innovation) outside the submitted work.

KA was supported by funding from Macmillan Cancer Support and reports funding by the National Institute for Health Research Clinical Research Network outside the submitted work.

DK was supported by project funding from Macmillan Cancer Support and reports grants from Innovate UK, the National Institute For Health Research, Birmingham Biomedical Research Centre, and National Institute For Health Research Surgical Reconstruction and Microbiology Research Centre at the University of Birmingham and University Hospitals Birmingham National Health Service Foundation Trust, and personal fees from Merck and GlaxoSmithKline outside the submitted work.

TK was supported by funding from Macmillan Cancer Support.

JA is a member of the National Cancer Research Institute Psychosocial Oncology and Survivorship Clinical Studies Group subgroup: Understanding and measuring the consequences of cancer and its treatment.

JMB report grants from the National Institute for Health Research and Yorkshire Cancer Research.

LC is a member of the National Cancer Research Institute Psychosocial Oncology and Survivorship Clinical Studies Group subgroup: Understanding and measuring the consequences of cancer and its treatment.

AG is a member of the National Cancer Research Institute Psychosocial Oncology and Survivorship Clinical Studies Group subgroup: Understanding and measuring the consequences of cancer and its treatment.

AWG reports research grant funding from Macmillan Cancer Support, Prostate Cancer UK, Yorkshire Cancer Research. He is a member of the Teenage and Young Adult Clinical study Group.

DMG is a National Institute for Health Research Senior Nurse Research Leader and member of the National Cancer Research Institute Psychosocial Oncology and Survivorship Clinical Studies Group subgroup: Understanding and measuring the consequences of cancer and its treatment.

AL is a member of the National Cancer Research Institute Psychosocial Oncology and Survivorship Clinical Studies Group subgroup: Understanding and measuring the consequences of cancer and its treatment.

RMT is a National Institute for Health Research Senior Nurse Research Leader and a member of the National Cancer Research Institute Psychosocial Oncology and Survivorship Clinical Studies Group subgroup: Understanding and measuring the consequences of cancer and its treatment.

GV reports grants from Pfizer, Breast Cancer NOW, Yorkshire Cancer Research, and European Organisation for Research and Treatment of Cancer and personal fees from Roche, Eisai, and Novartis.

GT is funded by the National Institute for Health Research Postdoctoral Fellowship programme (PDF-2017-10-047) and reports funding by the National Institute for Health Research Birmingham Biomedical Research Centre and National Institute for Health Research Surgical Reconstruction and Microbiology Research Centre at the University of Birmingham and University Hospitals Birmingham National Health Service Foundation Trust outside the submitted work.

MC is a National Institute for Health Research Senior Investigator and receives funding from the National Institute for Health Research Birmingham Biomedical Research Centre, the National Institute for Health Research Surgical Reconstruction and Microbiology Research Centre, and National Institute for Health Research Applied Research Collaboration West Midlands at the University of Birmingham and University Hospitals Birmingham National Health Service Foundation Trust, Health Data Research United Kingdom, Innovate United Kingdom (part of United Kingdom Research and Innovation), Macmillan Cancer Support, Health Foundation, UCB Pharma and GSK. MC has received personal fees from Astellas, Takeda, Merck, Daiichi Sankyo, Glaukos, GlaxoSmithKline, and the Patient-Centred Outcomes Research Institute (PCORI) outside the submitted work.

Author information

Authors and Affiliations

Authors

Contributions

The original study was initially conceived by members of the United Kingdom National Cancer Research Institute (NCRI) Psychosocial Oncology and Survivorship Clinical Studies Group subgroup: Understanding and measuring the consequences of cancer and its treatment (JA, LC [chair], AWG, AG, DMG, DK, AL, and RMT). DK, MC, JA, LC, AWG, AG, DG, AL, and RMT acquired funding. DK and MC designed the original EPiC study. DK, MC, KA, and ALS designed this review. KA and ALS undertook data extraction and analysis with input from DK and MC. AR conducted the qualitative interviews and data analysis, and ALS drafted the first manuscript, with all authors contributing to subsequent iterations. All authors provided critical input on the manuscript and approved the final version for publication. DK or MC is the guarantor.

Corresponding author

Correspondence to A. L. Slade.

Ethics declarations

Ethics approval and consent to participate

The Science, Technology, Engineering and Mathematics Ethical Review Committee at the University of Birmingham (Ref: ERN_17–0085) gave ethical approval for this study. All participants gave written consent to take part in the qualitative interviews.

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Slade, A.L., Retzer, A., Ahmed, K. et al. Systematic review of the use of translated patient-reported outcome measures in cancer trials. Trials 22, 306 (2021). https://doi.org/10.1186/s13063-021-05255-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13063-021-05255-z

Keywords