Skip to main content

Approaches to prioritising research for clinical trial networks: a scoping review

Abstract

Background

Prioritisation of clinical trials ensures that the research conducted meets the needs of stakeholders, makes the best use of resources and avoids duplication. The aim of this review was to identify and critically appraise approaches to research prioritisation applicable to clinical trials, to inform best practice guidelines for clinical trial networks and funders.

Methods

A scoping review of English-language published literature and research organisation websites (January 2000 to January 2020) was undertaken to identify primary studies, approaches and criteria for research prioritisation. Data were extracted and tabulated, and a narrative synthesis was employed.

Results

Seventy-eight primary studies and 18 websites were included. The majority of research prioritisation occurred in oncology and neurology disciplines. The main reasons for prioritisation were to address a knowledge gap (51 of 78 studies [65%]) and to define patient-important topics (28 studies, [35%]). In addition, research organisations prioritised in order to support their institution’s mission, invest strategically, and identify best return on investment. Fifty-seven of 78 (73%) studies used interpretative prioritisation approaches (including Delphi surveys, James Lind Alliance and consensus workshops); six studies used quantitative approaches (8%) such as prospective payback or value of information (VOI) analyses; and 14 studies used blended approaches (18%) such as nominal group technique and Child Health Nutritional Research Initiative. Main criteria for prioritisation included relevance, appropriateness, significance, feasibility and cost-effectiveness.

Conclusion

Current research prioritisation approaches for groups conducting and funding clinical trials are largely interpretative. There is an opportunity to improve the transparency of prioritisation through the inclusion of quantitative approaches.

Peer Review reports

Introduction

Clinical trials networks (CTNs) conduct investigator-initiated research and public good trials, largely funded by charities, universities and governments. Examples of CTNs in Australia include the Australian Kidney Trials Network (AKTN https://aktn.org.au/), the Australia and New Zealand College of Anaesthetists clinical trials network (ANZCA https://www.anzca.edu.au/research/anzca-clinical-trials-network) and the Cooperative Trials Group for Neuro-Oncology (COGNO https://www.cogno.org.au/default.aspx). Example CTNs in Europe include the European Society of Anaesthesiology and Intensive Care (ESAIC https://www.esaic.org/research/clinical-trial-network/) and in the USA include the HIV Prevention Trials Network (HPTN https://www.hptn.org/). However, with limited available resources including trained research personnel, trial participants and funds, decisions need to be made about which trials are a priority. The desired result of successful prioritisation is funded trials that generate important information, help inform clinical and policy decision-making and improve health outcomes. In reality, research prioritisation is not easy, and many organisations wrestle with competing criteria and the multiple interests of stakeholder groups.

Three main approaches to prioritisation have emerged in health and medicine research, namely interpretive, quantitative and blended methods. Interpretive approaches utilise consensus views of informed participants and include James Lind Alliance (JLA) and Delphi surveys [1, 2]. These approaches can reflect emerging patterns in the future and engage consumers; however, they do not provide methodology for identifying participants, often lack criteria transparency and have the potential for investigators and facilitators to bias opinions. Quantitative approaches utilise epidemiological, clinical or economic data. Examples include burden of disease, prospective payback and value of information (VOI) analyses [3, 4]. These approaches provide an objective assessment of value for money; however, they do not consider other criteria such as equity and broad stakeholder’s involvement; furthermore, they can be technically demanding. Blended approaches utilise and combine both interpretive and quantitative assessments and include the Child Health Nutrition Research Initiative (CHNRI) and multi-criteria decision analysis (MCDA [5, 6].

The aim of this scoping review was to identify approaches for priority setting in health and medical research useful to clinical trial networks (CTNs) in Australia and internationally and research funders. Specifically to answer the following research questions: what models, approaches or methods are used by CTNs to prioritise clinical trials; how have these models, approaches or methods been developed and validated; and what is the best practice for prioritising clinical trials? The findings will then be used to develop best practice guidance for CTNs and research funders.

Methods

A scoping review of published literature and working documents, as well as websites from research funding organisations and CTNs, was undertaken to identify research prioritisation tools and criteria. Digital databases including Ovid MEDLINE, Embase and the WHO library database (WHOLIS) were searched for publications about guidelines for prioritising research questions relevant to CTNs. Search terms included ([prioritization OR prioritisation OR setting priorities OR priority setting OR research priority*] AND [clinical trials OR clinical trial networks OR clinical trial group]). The search was limited to studies in English published from year 2000 onwards. The search was updated on 30 January 2020.

Titles and abstracts were screened, and eligible studies were selected by a single reviewer (VBN) for the following inclusion criteria: original studies, systematic reviews, guidelines, recommendations, and tools for research prioritisation. Both qualitative and quantitative methods of prioritisation were accepted. Studies not relevant to CTNs, duplicate publications, guidelines written from the perspective of funders, opinion articles, letters to editors and abstracts only were excluded. A manual search of key references cited in the retrieved papers and reports was also undertaken to identify additional publications not encountered by the electronic searches. A second reviewer (RLM) was consulted when in doubt regarding study selection, and any discrepancies were resolved by consensus with a third reviewer (MC).

A second search of key Australian and international CTNs/clinical disciplines/clinical specialties websites was then undertaken. Organisations were selected by the author team as likely to provide guidance on prioritisation and selection of clinical trials, and websites were searched by two authors (VBN, AB). Searched websites are listed in the Appendix. Searching included exploration of the website menu structure for relevant documents and searching within the sites using the terms “clinical trials”, “priorities”, “prioritisation” or “prioritization” (depending on the nationality of the website).

The following types of documents were selected for inclusion: guidance on prioritisation; case studies or examples of prioritisation exercises that reported the methods used; guidance on criteria for the assessment, selection or prioritization of clinical trials (e.g. for funding purposes). Documents that did not constitute current guidance or were superseded by later versions of current guidance (e.g. prioritisation processes to inform past priorities or strategic plans or discussion documents that appeared to be older than current guidance), were excluded. Documents with URLs that were no longer accessible in January 2020 were also excluded.

Data from studies and websites were extracted and tabulated into an Excel file according to a predefined codebook. Data extraction variables comprised author name, author group (e.g. CTN, funder), clinical discipline, country, year of publication, participants or stakeholders in the prioritisation process (e.g. health professionals, researchers, policy/decision makers, funders, patients, carers/consumers), intended audience (e.g. government/policymakers, clinicians, researchers, funders, the public), brief reason for prioritisation (e.g. knowledge gap, important to patients, return on investment, feasibility of methodology), type of research (e.g. trials), research prioritisation tools (e.g. Delphi, CHNRI, JLA, payback, MCDA, forced ranking, workshop/consensus meeting, other), prioritisation method (e.g. quantitative scoring, nominal group technique, weighted scores, monetary, other), research prioritisation criteria (e.g. relevance, appropriateness, significance, feasibility, cost-effectiveness [7]) and the URLs (for websites). Data from published articles and websites were summarised and tabulated separately. Critical appraisals of included studies, guidance documents or websites were not undertaken. Reporting of this scoping review was consistent with items in the PRISMA-ScR checklist [8].

Results

Literature search

The results of the literature search and study selection process are depicted in Fig. 1. A table of the seventy-eight primary studies included in this review is presented in Table 1.

Fig. 1
figure 1

PRISMA 2009 diagram

Table 1 Included studies

Most research prioritisation exercises were conducted either in Europe (n = 32; 41%) or North America (n = 25; 32%); six prioritisation studies (8%) originated in Australia and New Zealand. Two studies were conducted in South Asia (India; 3%), one in South-East Asia (1%), one in Africa (1%) and 11 studies were international (14%). Included studies were published between 2000 and 2019 (see Fig. 2a). Clinical specialties most frequently involved in research prioritisation were oncology (n = 11 studies; 14%), neurology (n = 11; 14%), paediatrics (n = 8; 10%), maternal and child health (n = 4; 5%), infectious diseases and HIV/AIDS (n = 5; 6%), nephrology (n = 4; 5%), respiratory medicine (n = 4; 5%), mental health (n = 3; 4%) and ophthalmology (n = 3; 4%).

Fig. 2
figure 2

Prioritisation study numbers, participants, reason and intended. a Number of studies published by year. b Participants in the prioritisation exercise. c Reason for prioritisation. d Intended audience of the prioritisation process

The stakeholders most frequently involved in the prioritisation process were health professionals (n = 65 studies; 83.3%), patients and carers/consumers (each n = 42 studies; 53.8%), researchers (n = 34 studies, 44%), policy or decision makers (n = 20 studies, 26%), and funders (n = 15 studies, 19%; see Fig. 2b). Stakeholders were not stated in four studies (5%).

The reasons for conducting the research prioritisation exercise included a knowledge gap in 51 studies (65%), ascertaining what was important to patients in 28 studies (36%), assessing the feasibility of a particular prioritisation methodology in 12 studies (15%) and estimating a return on investment in 10 studies (13%; see Fig. 2c; Table 2).

Table 2 Prioritisation methodologies

The intended audience for the outcomes of the prioritisation exercise were the funders in 40 studies (51%), researchers in 34 studies (44%), government or policymakers in 13 studies (17%), clinicians in 12 studies (15%), CTNs in 11 studies (14%), the general public in 4 studies (5.1%) and not stated in 20 studies (26%; see Fig. 2d).

A table of the prioritisation approaches is presented in Table 2. Fifty-seven studies used interpretative prioritisation approaches (73%), 14 studies used blended approaches (18%) and six studies used quantitative approaches (8%). Twenty-two studies used the JLA prioritisation tool or a modification thereof (28%), 19 studies used the Delphi methodology (24%) and 11 studies used a workshop or consensus meeting to establish their priorities (18%; see Fig. 3; Table 2). The “Payback” category included quantitative methods such as prospective payback of research (PPoR), expected value of information (EVI), return on investment (ROI) and the “Other” category included methods such as online surveys/questionnaires, focus groups, World Café and mixed methods. Forty-five studies (58%) employed quantitative scoring as a prioritisation method, frequently in the form of nominal group technique (n = 11 studies; 14%). Six studies used weighted scores (8%) and five studies used monetary value (6%). One study each used Agency for Healthcare Research and Quality (AHRQ) criteria, Dotmocracy, forced ranking, red-amber-green light and no prioritisation (1%).

Fig. 3
figure 3

Prioritisation tools used. CHNRI, Child Health and Nutrition Research Initiative; JLA, James Lind Alliance; MCDA, Multiple-criteria Decision Analysis

Over two-thirds of the identified studies (n=53, 68%) did not describe any formal prioritisation criteria. In those that did describe prioritisation criteria, multiple criteria were mentioned. Relevance (i.e. why should we do it? including the burden of disease, equity, and knowledge gaps) was cited in 14 of the 78 included studies (18%). Seven studies (9%) cited criteria related to appropriateness (i.e. should we do it? including scientific rigour and suitability to answer the research question); 17 studies (22%) considered criteria related to significance of research outcomes (i.e. what will we get out of it? including impact, innovation, capacity building); 12 studies (15%) cited feasibility among their prioritisation criteria (i.e. can we do it? including team quality and research environment). Cost-effectiveness was considered by fifteen studies (19%). Five studies cited other prioritisation criteria (6%).

Organisational websites

Thirty-nine websites of research funding organisations and CTNs were reviewed (Appendix), and 18 were found to contain research prioritisation information: one from Australia (6%), two from New Zealand (11%), one from Ireland (6%), eight from Canada (44%) and six from the USA (33%; see Table 3). A table of the clinical disciplines involved is depicted listed in Table 3. The stakeholders most frequently involved in priority setting were researchers (n = 14 websites; 78%) followed by health professionals (n = 12 websites; 67%) and policy/decision makers (n = 11 websites; 61%; see Fig. 4a). Funders and patients were mentioned in seven processes each (39%) and carers/consumers were mentioned six times (33%). Participants or stakeholders were not stated in three occasions (17%; see Fig. 4a).

Table 3 Websites searched
Fig. 4
figure 4

Research prioritisation participants, reasons, intended audience and funding criteria from websites of research funders and clinical trial networks. a Participants in the prioritisation exercise. b Reason for prioritisation. c Intended audience for prioritisation. d Funding criteria as reported in organisational websites

A “knowledge gap” was the reason for developing a prioritisation guideline among 10 websites (56%), followed by “wanting to know what was important to patients” (n = 8 websites; 44%). Six organisations mentioned the reason for the priority-setting exercise was to support their vision and mission or to invest strategically and in a balanced way (33%) and five organisations wanted to find the best return on investment (32%; see Fig. 4b). Feasibility of the methodologies used was mentioned once (6%).

The intended audience was in all but one case the general public (n = 17 websites; 94%), followed by the researchers (n = 6 websites; 33%), the government and policymakers (n = 5 websites; 28%) and clinicians or funders (n = 4 websites each; 22%; see Fig. 4c).

As for the prioritisation tools used, three organisations used workshop/consensus meeting (17%), one used the JLA tool, one used payback (VOI) and one used MCDA (6%). Five organisations used other tools (surveys, working groups; 28%) and seven did not describe the tool used (39%). In general, few details were provided in organisational websites to further describe the prioritisation approaches undertaken.

The prioritisation criteria included relevance on 16 occasions (89%), appropriateness on 10 occasions (56%), significance on 14 occasions (78%), feasibility on 9 occasions (50%) and cost-effectiveness on 4 occasions (22%). Two websites did not state any prioritisation criteria (11%; see Fig. 4d).

Discussion

This extensive scoping review summarises findings from international agencies about current methods and approaches to prioritisation of clinical trials undertaken by CTNs and research funders. The main reasons for prioritisation were to address a knowledge gap in clinical decision making, and to define patient-important topics. More than two thirds used an interpretive approach (e.g. James Lind Alliance); a small proportion used a quantitative approach (e.g. prospective payback); and one fifth used a blended approach combining qualitative and quantitative methods (e.g. CHNRI). The most common criteria for prioritisation were significance, relevance and cost-effectiveness.

The rationale for prioritisation of trials on the basis of generating new knowledge to improve clinical decision-making is not surprising, as efficacy and effectiveness trials are designed to answer important questions in patient management [27, 50, 88]. What was less clear, however, was how these trials all with “good” questions were then ranked in order of priority. Consensus-based methods that use an interpretative approach are appealing because of their broad stakeholder engagement; however, the trade-offs between criteria, such as significance versus feasibility, and the subsequent processes for overall ranking of trials are not transparent [89, 90]. This is where blended approaches that include a quantitative component that facilitates objective scoring of trial proposals can assist.

The infrequent use of pure quantitative approaches for prioritisation of trials including burden of disease or value for money is likely due to few standardised methods to view competing claims side by side, or knowing how to weight such criteria. It may also be related to low technical knowledge or expertise within the trials community to generate this information. For example, many clinical trial funders suggest the relevance of the problem to be stated, which is typically reported as burden of disease, incidence or prevalence. When different metrics are used across trial proposals they become difficult to compare, which may lead to grant reviewers considering whether the criterion is satisfied (i.e. is there a substantial burden, [yes/no]), rather than comparing those burdens. Sometimes, the burden is presented as disability-adjusted life years (DALYs), and sometimes, the disease burden is monetized to provide an overview of health system or societal costs. While this provides a common metric on which trial applications can be compared, these estimates are limited to quantifying the current situation; they do not provide insight into the value of the proposed trial in reducing that burden (i.e. the significance), otherwise known as the impact or net benefit.

Value of information (VOI) analysis has emerged as a new framework for quantifying the net benefit of proposed randomised trials. VOI uses a cost-effectiveness modelling approach and takes into account the cost of running the trial and the value of the new trial information to reduce uncertainty with the current clinical decision. The benefit of the health outcomes for the better decision (e.g. using drug A over drug B) is then multiplied across the population at risk using assumptions about post-trial implementation. A VOI analysis can be undertaken for most randomised trials enabling studies in a given portfolio to be ranked from most to least value. This requires capacity building in the health economics and statistics workforce. Efficient methods to calculate VOI are currently underway [91, 92].

An encouraging sign from this review was the emphasis placed on patient-important topics through consumer-generated questions and topic ranking, from both published literature and organisational websites. This ensures that not only are questions important and of interest to clinicians or trialists, but that they also address issues, problems or concerns that are bothering those with the disease and/or undergoing specific treatments. This is especially important for government and non-profit charity funders where the funding for research originates from the general public (i.e. tax-payers), or donors.

The strengths of this review include the dual searching of published and unpublished literature, including organisational websites of international clinical trial networks and trial funders. This approach was likely to identify prioritisation processes that were operational, yet had not been formally described in the peer-reviewed literature. It is a strength that we were able to locate and extract research prioritisation approaches and methods as well as the prioritisation criteria used, as this provides sufficient detail for clinical trials networks and funders to replicate. Our scoping review was limited to studies and websites published in English and therefore may omit relevant studies published in other languages. It was not a systematic review and therefore may not have identified all studies of research prioritisation in the published literature. In addition, we could only tabulate methods where they were clearly described.

Further research consulting consumers, researchers and policy-makers is now needed to develop specific criteria weights for clinical trials networks and coordinating centre members of the Australian Clinical Trials Alliance (ACTA), of international CTNs and funders of clinical trials. Development of tools to aid clinicians and researchers in using quantitative approaches is also needed. Following implementation of a formalised prioritisation process, clinical trials networks and funders will need to then evaluate the process and assess whether the “best” trials are subsequently funded and deliver on their expected benefits [93].

Conclusion

Research prioritisation approaches for groups conducting and funding clinical trials are predominantly interpretative. Given the strengths of a blended approach to prioritisation, there is an opportunity to improve the transparency of process through the inclusion of quantitative techniques.

Availability of data and materials

Data generated and analysed during this study are included in this published article and its supplementary information files.

Abbreviations

ACTA:

Australian Clinical Trials Alliance

AHRQ:

Agency for Healthcare Research and Quality

AKTN:

Australasian Kidney Trials Network

ANZCA:

Australian and New Zealand College for Anaesthetists

ANZIC-RC:

Australian and New Zealand Intensive Care Research Centre

ANZICS:

Australian and New Zealand Intensive Care Society

CHNRI:

Child Health Nutrition Research Initiative

CTN:

Clinical Trial Network

DALY:

Disability-adjusted life years

EVI:

Expected Value of Information

JLA:

James Lind Alliance

MCDA:

Multi-criteria Decision Analysis

NHMRC:

National Health and Medical Research Council

PPoR:

Prospective Payback of Research

ROI:

Return on investment

VOI:

Value of information

WHOLIS:

World Health Organization Library Database

References

  1. Cowan K, Oliver S. The James Lind Alliance guidebook. Southampton: National Institute for Health Research Evaluation, Trials and Studies Coordinating Centre; 2021.

    Google Scholar 

  2. Linstone HA, Turoff M. The Delphi method. Reading: Addison-Wesley; 1975.

    Google Scholar 

  3. Williams A. Calculating the global burden of disease: time for a strategic reappraisal? Chichester: John Wiley & Sons, Ltd; 1999.

    Google Scholar 

  4. Davies L, Drummond M, Papanikolaou P. Prioritizing investments in health technology assessment: can we assess potential value for money? Int J Technol Assess Health Care. 2000;16(1):73–91.

    Article  CAS  Google Scholar 

  5. Rudan I, Gibson JL, Ameratunga S, Arifeen SE, Bhutta ZA, Black M, et al. Setting priorities in global child health research investments: guidelines for implementation of CHNRI method. Croatian Med J. 2008;49(6):720–33.

    Article  Google Scholar 

  6. Belton V, Stewart T. Multiple criteria decision analysis: an integrated approach: Springer Science & Business Media; 2002.

    Book  Google Scholar 

  7. Tuffaha HWESN, Chambers SK, Scuffham PA. Directing research funds to the right research projects: a review of criteria used by research organisations in Australia in prioritising health research projects for funding. BMJ Open. 2018;8(12):e026207.

    Article  Google Scholar 

  8. Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73. https://doi.org/10.7326/M18-0850.

    Article  Google Scholar 

  9. Folayan MO, Haire B, Allman D, Yakubu A, Afolabi MO. Research priorities during infectious disease emergencies in West Africa. BMC Res Notes. 2018;11(1):159.

    Article  Google Scholar 

  10. Middleton JW, Piccenna L, Lindsay Gruen R, Williams S, Creasey G, Dunlop S, et al. Developing a spinal cord injury research strategy using a structured process of evidence review and stakeholder dialogue. Part III: Outcomes. Spinal Cord. 2015;53(10):729–37.

    Article  CAS  Google Scholar 

  11. Sangvatanakul P, Hillege S, Lalor E, Levi C, Hill K, Middleton S. Setting stroke research priorities: The consumer perspective. J Vasc Nurs. 2010;28(4):121–31.

    Article  Google Scholar 

  12. Sawford K, Dhand NK, Toribio JA, Taylor MR. The use of a modified Delphi approach to engage stakeholders in zoonotic disease research priority setting. BMC Public Health. 2014;14:182.

    Article  Google Scholar 

  13. Thom O, Keijzers G, Davies S, Mc DTD, Knott J, Middleton PM. Clinical research priorities in emergency medicine: results of a consensus meeting and development of a weighting method for assessment of clinical research priorities. Emerg Med Australas. 2014;26(1):28–33.

    Article  Google Scholar 

  14. Tong A, Crowe S, Chando S, Cass A, Chadban SJ, Chapman JR, et al. Research priorities in CKD: report of a national workshop conducted in Australia. Am J Kidney Dis. 2015;66(2):212–22.

    Article  Google Scholar 

  15. Taylor WJ, Green SE. Use of multi-attribute decision-making to inform prioritization of Cochrane review topics relevant to rehabilitation. Eur J Phys Rehabil Med. 2019;55(3):322–30.

    Article  Google Scholar 

  16. Barnieh L, Jun M, Laupacis A, Manns B, Hemmelgarn B. Determining research priorities through partnership with patients: an overview. Semin Dial. 2015;28(2):141–6.

    Article  Google Scholar 

  17. Hayes KC, Bassett-Spiers K, Das R, Ethans KD, Kagan C, Kramer JL, et al. Research priorities for urological care following spinal cord injury: recommendations of an expert panel. Can J Urol. 2007;14(1):3416–23.

    CAS  Google Scholar 

  18. Lavigne M, Birken CS, Maguire JL, Straus S, Laupacis A. Priority setting in paediatric preventive care research. Arch Dis Child. 2017;102(8):748–53.

    Article  Google Scholar 

  19. Manns B, Hemmelgarn B, Lillie E, Dip SC, Cyr A, Gladish M, et al. Setting research priorities for patients on or nearing dialysis. Clin J Am Soc Nephrol. 2014;9(10):1813–21.

    Article  Google Scholar 

  20. Ota S, Cron RQ, Schanberg LE, O’Neil K, Mellins ED, Fuhlbrigge RC, et al. Research priorities in pediatric rheumatology: The Childhood Arthritis and Rheumatology Research Alliance (CARRA) consensus. Pediatr Rheumatol. 2008;6:5.

    Article  Google Scholar 

  21. Restall GJ, Carnochan TN, Roger KS, Sullivan TM, Etcheverry EJ, Roddy P. Collaborative priority setting for human immunodeficiency virus rehabilitation research: a case report. Can J Occup Ther. 2016;83(1):7–13.

    Article  Google Scholar 

  22. Schneider P, Evaniew N, Rendon JS, McKay P, Randall RL, Turcotte R, et al. Moving forward through consensus: protocol for a modified Delphi approach to determine the top research priorities in the field of orthopaedic oncology. BMJ Open. 2016;6(5):e011780.

    Article  Google Scholar 

  23. Sivananthan SN, Chambers LW. A method for identifying research priorities for health systems research on health and aging. Healthc Manage Forum. 2013;26(1):33–6. https://doi.org/10.1016/j.hcmf.2012.08.002.

  24. Wu J, Bezjak A, Chow E, Cross P, Genest P, Grant N, et al. A consensus development approach to define national research priorities in bone metastases: proceedings from NCIC CTG workshop. Clin Oncol (R Coll Radiol). 2003;15(8):496–9.

    Article  CAS  Google Scholar 

  25. Al-Khatib SM, Gierisch JM, Crowley MJ, Coeytaux RR, Myers ER, Kendrick A, et al. Future research prioritization: implantable cardioverter-defibrillator therapy in older patients. J Gen Intern Med. 2015;30(12):1812–20.

    Article  Google Scholar 

  26. Ardoin SP, Daly RP, Merzoug L, Tse K, Ardalan K, Arkin L, et al. Research priorities in childhood-onset lupus: results of a multidisciplinary prioritization exercise. Pediatr. 2019;17(1):32.

    Google Scholar 

  27. Bennette CS, Veenstra DL, Basu A, Baker LH, Ramsey SD, Carlson JJ. Development and evaluation of an approach to using value of information analyses for real-time prioritization decisions within SWOG, a large cancer clinical trials cooperative group. Med Decis Making. 2016;36(5):641–51.

    Article  Google Scholar 

  28. Bousvaros A, Sylvester F, Kugathasan S, Szigethy E, Fiocchi C, Colletti R, et al. Challenges in pediatric inflammatory bowel disease. Inflamm Bowel Dis. 2006;12(9):885–913.

    Article  Google Scholar 

  29. Carlson JJ, Kim DD, Guzauskas GF, Bennette CS, Veenstra DL, Basu A, et al. Integrating value of research into NCI Clinical Trials Cooperative Group research review and prioritization: a pilot study. Cancer Med. 2018;7(9):4251–60.

    Article  Google Scholar 

  30. Duong DN, Schempp C, Barker E, Cupples S, Pierce P, Ryan-Wenger N, et al. Developing military nursing research priorities. Mil Med. 2005;170(5):362–5.

    Article  Google Scholar 

  31. Esmail LC, Roth J, Rangarao S, Carlson JJ, Thariani R, Ramsey SD, et al. Getting our priorities straight: a novel framework for stakeholder-informed prioritization of cancer genomics research. Genet Med. 2013;15(2):115–22.

    Article  Google Scholar 

  32. Fochtman D, Hinds PS. Identifying nursing research priorities in a pediatric clinical trials cooperative group: the Pediatric Oncology Group experience. J Pediatr Oncol Nurs. 2000;17(2):83–7.

    Article  CAS  Google Scholar 

  33. Henkle E, Aksamit T, Barker A, Daley CL, Griffith D, Leitman P, et al. Patient-centered research priorities for pulmonary nontuberculous mycobacteria (NTM) infection an NTM research consortium workshop report. Ann Am Thor Soc. 2016;13(9):S379–S84.

    Article  Google Scholar 

  34. Henkle E, Aksamit TR, Daley CL, Griffith DE, O’Donnell AE, Quittner AL, et al. US Patient-Centered Research Priorities and Roadmap for Bronchiectasis. Chest. 2018;154(5):1016–23.

    Article  Google Scholar 

  35. Higginbotham SL. The SFP research priority setting process. Contraception. 2015;92(4):282–8.

    Article  Google Scholar 

  36. Roach ES, Abreu N, Acosta M, Ball KL, Berrocal A, Bischoff J, et al. Leveraging a Sturge-Weber gene discovery: an agenda for future research. Pediatr Neurol. 2016;58:12–24.

    Article  Google Scholar 

  37. Safdar B, Greenberg MR. Organization, execution and evaluation of the 2014 academic emergency medicine consensus conference on gender-specific research in emergency care - An executive summary. Acad Emerg Med. 2014;21(12):1307–17.

    Article  Google Scholar 

  38. Saldanha IJ, Dickersin K, Hutfless ST, Akpek EK. Gaps in current knowledge and priorities for future research in dry eye. Cornea. 2017;36(12):1584–91.

    Article  Google Scholar 

  39. Thariani R, Wong W, Carlson JJ, Garrison L, Ramsey S, Deverka PA, et al. Prioritization in comparative effectiveness research: the CANCERGEN Experience. Med Care. 2012;50(5):388–93.

    Article  Google Scholar 

  40. Vickrey BG, Brott TG, Koroshetz WJ. Stroke Research Priorities Meeting Steering C, the National Advisory Neurological D, Stroke C, et al. Research priority setting: a summary of the 2012 NINDS Stroke Planning Meeting Report. Stroke. 2013;44(8):2338–42.

    Article  Google Scholar 

  41. Aliberti S, Masefield S, Polverino E, De Soyza A, Loebinger MR, Menendez R, et al. Research priorities in bronchiectasis: a consensus statement from the EMBARC Clinical Research Collaboration. Eur Respir J. 2016;48(3):632–47.

    Article  Google Scholar 

  42. Forsman AK, Wahlbeck K, Aaro LE, Alonso J, Barry MM, Brunn M, et al. Research priorities for public mental health in Europe: recommendations of the ROAMER project. Eur J Public Health. 2015;25(2):249–54.

    Article  Google Scholar 

  43. van der Feltz-Cornelis CM, van Os J, Knappe S, Schumann G, Vieta E, Wittchen HU, et al. Towards Horizon 2020: Challenges and advances for clinical mental health research - Outcome of an expert survey. Neuropsychiatr Dis Treatment. 2014;10:1057–68.

    Article  Google Scholar 

  44. de Graaf G, Postmus D, Buskens E. Using multicriteria decision analysis to support research priority setting in biomedical translational research projects. BioMed Research Int. 2015;2015:191809.

    Article  Google Scholar 

  45. Aldiss S, Fern LA, Phillips RS, Callaghan A, Dyker K, Gravestock H, et al. Research priorities for young people with cancer: a UK priority setting partnership with the James Lind Alliance. BMJ Open. 2019;9(8):e028119.

    Article  Google Scholar 

  46. Andronis L, Billingham LJ, Bryan S, James ND, Barton PM. A practical application of value of information and prospective payback of research to prioritize evaluative research. Med Decis Making. 2016;36(3):321–34.

    Article  Google Scholar 

  47. Boney O, Bell M, Bell N, Conquest A, Cumbers M, Drake S, et al. Identifying research priorities in anaesthesia and perioperative care: final report of the joint National Institute of Academic Anaesthesia/James Lind Alliance Research Priority Setting Partnership. BMJ Open. 2015;5(12):e010006.

    Article  Google Scholar 

  48. Cox A, Arber A, Gallagher A, MacKenzie M, Ream E. Establishing priorities for oncology nursing research: nurse and patient collaboration. Oncol Nurs Forum. 2017;44(2):192–203.

    Google Scholar 

  49. Deane KH, Flaherty H, Daley DJ, Pascoe R, Penhale B, Clarke CE, et al. Priority setting partnership to identify the top 10 research priorities for the management of Parkinson’s disease. BMJ Open. 2014;4(12):e006434.

    Article  Google Scholar 

  50. Fleurence RL. Setting priorities for research: a practical application of ‘payback’ and expected value of information. Health Econ. 2007;16(12):1345–57.

    Article  Google Scholar 

  51. Gadsby R, Snow R, Daly AC, Crowe S, Matyka K, Hall B, et al. Setting research priorities for type 1 diabetes. Diabet Med. 2012;29(10):1321–6.

    Article  CAS  Google Scholar 

  52. Hall DA, Mohamad N, Firkins L, Fenton M, Stockdale D. Identifying and prioritizing unmet research questions for people with tinnitus: The James Lind Alliance Tinnitus Priority Setting Partnership. Clin Investig. 2013;3(1):21–8.

    Article  CAS  Google Scholar 

  53. Hart AL, Lomer M, Verjee A, Kemp K, Faiz O, Daly A, et al. What are the top 10 research questions in the treatment of inflammatory bowel disease? A priority setting partnership with the James Lind Alliance. J Crohn’s Colitis. 2017;11(2):204–11.

    Article  Google Scholar 

  54. Heazell AE, Whitworth MK, Whitcombe J, Glover SW, Bevan C, Brewin J, et al. Research priorities for stillbirth: process overview and results from UK Stillbirth Priority Setting Partnership. Ultrasound Obstet Gynecol. 2015;46(6):641–7.

    Article  CAS  Google Scholar 

  55. Howell SJ, Pandit JJ, Rowbotham DJ. Research Council of the National Institute of Academic A. National Institute of Academic Anaesthesia research priority setting exercise. Br J Anaesthesia. 2012;108(1):42–52.

    Article  CAS  Google Scholar 

  56. Ingram JR, Abbott R, Ghazavi M, Alexandroff AB, McPhee M, Burton T, et al. The Hidradenitis Suppurativa Priority Setting Partnership. Br J Dermatol. 2014;171(6):1422–7.

    Article  CAS  Google Scholar 

  57. Kelly S, Lafortune L, Hart N, Cowan K, Fenton M, Brayne C, et al. Dementia priority setting partnership with the James Lind Alliance: using patient and public involvement and the evidence base to inform the research agenda. Age & Ageing. 2015;44(6):985–93.

    Article  Google Scholar 

  58. Knight SR, Metcalfe L, O’Donoghue K, Ball ST, Beale A, Beale W, et al. Defining priorities for future research: results of the UK kidney transplant priority setting partnership. PLoS ONE. 2016;11(10):e0162136.

    Article  Google Scholar 

  59. Macbeth AE, Tomlinson J, Messenger AG, Moore-Millar K, Michaelides C, Shipman AR, et al. Establishing and prioritizing research questions for the treatment of alopecia areata: the Alopecia Areata Priority Setting Partnership. Br J Dermatol. 2017;176(5):1316–20.

    Article  CAS  Google Scholar 

  60. McKenna C, Griffin S, Koffijberg H, Claxton K. Methods to place a value on additional evidence are illustrated using a case study of corticosteroids after traumatic brain injury. J Clin Epidemiol. 2016;70:183–90.

    Article  Google Scholar 

  61. Morris C, Simkiss D, Busk M, Morris M, Allard A, Denness J, et al. Setting research priorities to improve the health of children and young people with neurodisability: a British Academy of Childhood Disability-James Lind Alliance Research Priority Setting Partnership. BMJ Open. 2015;5(1):e006233.

    Article  Google Scholar 

  62. Owens C, Ley A, Aitken P. Do different stakeholder groups share mental health research priorities? A four-arm Delphi study. Health Expect. 2008;11(4):418–31.

    Article  Google Scholar 

  63. Perry DC, Wright JG, Cooke S, Roposch A, Gaston MS, Nicolaou N, et al. A consensus exercise identifying priorities for research into clinical effectiveness among children’s orthopaedic surgeons in the United Kingdom. Bone Joint J. 2018;100-B(5):680–4.

    Article  CAS  Google Scholar 

  64. Pollock A, St George B, Fenton M, Firkins L. Top 10 research priorities relating to life after stroke--consensus from stroke survivors, caregivers, and health professionals. Int J Stroke. 2014;9(3):313–20.

    Article  Google Scholar 

  65. Rangan A, Upadhaya S, Regan S, Toye F, Rees JL. Research priorities for shoulder surgery: results of the 2015 James Lind Alliance patient and clinician priority setting partnership. BMJ Open. 2016;6(4):e010412.

    Article  Google Scholar 

  66. Rowat A, Pollock A, St George B, Cowey E, Booth J, Lawrence M, et al. Top 10 research priorities relating to stroke nursing: a rigorous approach to establish a national nurse-led research agenda. J Adv Nurs. 2016;72(11):2831–43.

    Article  Google Scholar 

  67. Rowe F, Wormald R, Cable R, Acton M, Bonstein K, Bowen M, et al. The Sight Loss and Vision Priority Setting Partnership (SLV-PSP): overview and results of the research prioritisation survey process. BMJ Open. 2014;4(7):e004905.

    Article  Google Scholar 

  68. Shepherd V, Wood F, Hood K. Establishing a set of research priorities in care homes for older people in the UK: a modified Delphi consensus study with care home staff. Age Ageing. 2017;46(2):284–90. https://doi.org/10.1093/ageing/afw204.

  69. Stephens RJ, Whiting C, Cowan K, James Lind Alliance Mesothelioma Priority Setting Partnership Steering C. Research priorities in mesothelioma: A James Lind Alliance Priority Setting Partnership. Lung Cancer. 2015;89(2):175–80.

    Article  CAS  Google Scholar 

  70. van Middendorp JJ, Allison HC, Ahuja S, Bracher D, Dyson C, Fairbank J, et al. Top ten research priorities for spinal cord injury: the methodology and results of a British priority setting partnership. Spinal Cord. 2016;54(5):341–6.

    Article  Google Scholar 

  71. Wan YL, Beverley-Stevenson R, Carlisle D, Clarke S, Edmondson RJ, Glover S, et al. Working together to shape the endometrial cancer research agenda: The top ten unanswered research questions. Gynecol Oncol. 2016;143(2):287–93.

    Article  Google Scholar 

  72. Willett KM, Gray B, Moran CG, Giannoudis PV, Pallister I. Orthopaedic trauma research priority-setting exercise and development of a research network. Injury. 2010;41(7):763–7.

    Article  CAS  Google Scholar 

  73. Arora NK, Mohapatra A, Gopalan HS, Wazny K, Thavaraj V, Rasaily R, et al. Setting research priorities for maternal, newborn, child health and nutrition in India by engaging experts from 256 indigenous institutions contributing over 4000 research ideas: a CHNRI exercise by ICMR and INCLEN. J Global Health. 2017;7(1):011003.

    Article  Google Scholar 

  74. Ravindran TKS, Seshadri T. A health equity research agenda for India: results of a consultative exercise. Health Res Policy Syst. 2018;16(Suppl 1):94.

    Article  Google Scholar 

  75. Allotey J, Matei A, Husain S, Newton S, Dodds J, Armson AB, et al. Research prioritization of interventions for the primary prevention of preterm birth: an international survey. Eur J Obstet Gynecol Reprod Biol. 2019;236:240–8.

    Article  Google Scholar 

  76. Bahl R, Martines J, Ali N, Bhan MK, Carlo W, Chan KY, et al. Research priorities to reduce global mortality from newborn infections by 2015. Pediatr Infect Dis J. 2009;28(1 Suppl):S43–8.

    Article  Google Scholar 

  77. Brown KH, Hess SY, Boy E, Gibson RS, Horton S, Osendarp SJ, et al. Setting priorities for zinc-related health research to reduce children’s disease burden worldwide: an application of the Child Health and Nutrition Research Initiative’s research priority-setting method. Public Health Nutrition. 2008;12(3):389–96.

    Google Scholar 

  78. Brundin P, Barkerb RA, Connc PJ, Dawsond TM, Kieburtze K, Lees AJ, et al. Linked clinical trials-the development of new clinical learning studies in Parkinson’s disease using screening of multiple prospective new treatments. J Parkinson’s Dis. 2013;3(3):231–9.

    Article  Google Scholar 

  79. Foster NE, Dziedzic KS, van der Windt DA, Fritz JM, Hay EM. Research priorities for non-pharmacological therapies for common musculoskeletal problems: nationally and internationally agreed recommendations. BMC Musculoskelet Disord. 2009;10:3.

    Article  Google Scholar 

  80. Prescott HC, Iwashyna TJ, Blackwood B, Calandra T, Chlan LL, Choong K, et al. Understanding and enhancing sepsis survivorship. Priorities for Research and Practice. Am J Respir Crit Care Med. 2019;200(8):972–81.

    Article  Google Scholar 

  81. Robinson N, Lorenc A, Ding W, Jia J, Bovey M, Wang XM. Exploring practice characteristics and research priorities of practitioners of traditional acupuncture in China and the EU-A survey. J Ethnopharmacol. 2012;140(3):604–13.

    Article  Google Scholar 

  82. Rowbotham NJ, Smith SJ, Elliott ZC, Leighton PA, Rayner OC, Morley R, et al. Adapting the James Lind Alliance priority setting process to better support patient participation: an example from cystic fibrosis. Res Involv Engagem. 2019;5:24.

    Article  CAS  Google Scholar 

  83. Ruhl AP, Sadreameli SC, Allen JL, Bennett DP, Campbell AD, Coates TD, et al. Identifying Clinical and Research Priorities in Sickle Cell Lung Disease. An Official American Thoracic Society Workshop Report. Ann Am Thor Soc. 2019;16(9):e17–32.

    Article  Google Scholar 

  84. Thompson W, Reeve E, Moriarty F, Maclure M, Turner J, Steinman MA, et al. Deprescribing: Future directions for research. Res Social Adm Pharm. 2019;15(6):801–5.

    Article  Google Scholar 

  85. Viergever RF, Olifson S, Ghaffar A, Terry RF. A checklist for health research priority setting: nine common themes of good practice. Health Res Policy Syst. 2010;8(1):36.

    Article  Google Scholar 

  86. Yu T, Li T, Lee KJ, Friedman DS, Dickersin K, Puhan MA. Setting priorities for comparative effectiveness research on management of primary angle closure: a survey of Asia-Pacific clinicians. J Glaucoma. 2015;24(5):348–55.

    Article  Google Scholar 

  87. Comi G, Freedman MS, Kappos L, Olsson TP, Miller AE, Wolinsky JS, et al. Pooled safety and tolerability data from four placebo-controlled teriflunomide studies and extensions. Mult Scler Relat Disord. 2016;5:97–104. https://doi.org/10.1016/j.msard.2015.11.006. Epub 2015 Nov 10.

  88. Singal AG, Higgins PD, Waljee AK. A primer on effectiveness and efficacy trials. Clin Transl Gastroenterol. 2014;5(1):e45.

    Article  CAS  Google Scholar 

  89. Tuffaha HW, Gordon LG, Scuffham PA. Value of information analysis informing adoption and research decisions in a portfolio of health care interventions. MDM Policy Pract. 2016;1(1):2381468316642238.

    Google Scholar 

  90. Carlson JJ, Thariani R, Roth J, Gralow J, Henry NL, Esmail L, et al. Value-of-information analysis within a stakeholder-driven research prioritization process in a US setting: an application in cancer genomics. Med Decis Making. 2013;33(4):463–71.

    Article  Google Scholar 

  91. Fenwick E, Steuten L, Knies S, Ghabri S, Basu A, Murray JF, et al. Value of information analysis for research decisions—an introduction: report 1 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force. Value Health. 2020;23(2):139–50.

    Article  Google Scholar 

  92. Rothery C, Strong M, Koffijberg HE, Basu A, Ghabri S, Knies S, et al. Value of information analytical methods: report 2 of the ISPOR Value of information analysis emerging good practices task force. Value Health. 2020;23(3):277–86.

    Article  Google Scholar 

  93. Australian Clinical Trials Alliance (ACTA). Research Prioritisation Framework. 2020. Accessed 21 Jan 2021 from: https://clinicaltrialsalliance.org.au/resource/acta-research-prioritisation-framework/.

    Google Scholar 

Download references

Acknowledgements

We thank ACTA’s Research Prioritisation Reference Group.

Funding

This project received grant funding from the Australian Government. Salary support for RLM was provided through an Australian NHMRC TRIP Fellowship #1150989 and University of Sydney Robinson Fellowship. HT was supported by an NHMRC ECR Fellowship #1121232.

Author information

Authors and Affiliations

Authors

Contributions

All authors conceived the idea. RM wrote the first draft. All authors gave intellectual input into the article and approved the final manuscript.

Corresponding author

Correspondence to Rachael L. Morton.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Key Australian and international clinical trials networks (CTNs) /clinical disciplines/ clinical specialties and Funders’ websites

Clinical trials network websites

▪ Australia & New Zealand Musculoskeletal (ANZMUSC) Clinical Trials Network

▪ Australasian College for Emergency Medicine (ACEM) Clinical Trials Group

▪ Australasian Gastro-Intestinal Trials Group (AGITG)

▪ Australasian Lung Cancer Trials Group (ALTG)

▪ Australasian Radiopharmaceutical Trials Network (ARTNET)

▪ Australasian Sarcoma Study Group (ASSG)

▪ Australasian Society for Infectious Diseases (ASID) Clinical Research Network

▪ Australasian Stroke Trials Network (ASTN)

▪ Australia & New Zealand Breast Cancer Trials Group (ANZBCTG)

▪ Australia & New Zealand Melanoma Trials Group (ANZMTG)

▪ Australia New Zealand Gynaecological Oncology Group (ANZGOG)

▪ Australian & New Zealand Children’s Haematology/Oncology Group (ANZCHOG)

▪ Australian & New Zealand College of Anaesthetists (ANZCA) Clinical Trials Network

▪ Australian & New Zealand Intensive Care Society (ANZICS) Clinical Trials Group

▪ Australian & New Zealand Urogenital & Prostate (ANZUP) Cancer Trials Group

▪ Australian Epilepsy Clinical Trials Network (AECTN)

▪ Australian Paediatric Research Network (APRN)

▪ Australian Primary Care Research Network (APCReN)

▪ Cooperative Trials Group for Neuro-Oncology (COGNO)

▪ Multiple Sclerosis Research Australia Clinical Trials Network (MS Australia)

▪ NSW Better Treatments 4 Kids (BT4K)

▪ Paediatric Research in Emergency Departments International Collaborative (PREDICT)

▪ Paediatric Trials Network Australia (PTNA)

▪ Palliative Care Clinical Studies Collaborative (PaCCSC)

▪ Primary Care Collaborative Cancer Clinical Trial Group (PC4)

▪ Psycho-Oncology Co-operative Research Group (PoCoG)

▪ The Australasian Consortium of Centres for Clinical Cognitive Research (AC4R)

▪ The Australasian Kidney Trials Network (AKTN)

▪ The Australasian Sleep Trials Network (ASTN)

▪ The Spinal Cord Injury Network (SSCIS)

▪ The Australian Type 1 Diabetes Clinical Research Network (T1DCRN)

▪ The Interdisciplinary Maternal Perinatal Australasian Collaborative Trials Network (IMPACT)

▪ Therapeutic and Vaccine Research Program, Kirby Institute

▪ Trans-Tasman Radiation Oncology Group (TROG)

Funders’ websites

National Health and Medical Research Council (NHMRC)—Australia

Medical Research Future Fund (MRFF)—Australia

Medical Research Council (MRC)—United Kingdom

National Institute for Health Research (NIHR)—United Kingdom

Health Research Board (HRB)—Ireland

Canadian Institutes for Health Research (CIHR)—Canada

National Science Challenges (NSCs)—New Zealand

Patient-Centered Outcomes Research Institute (PCORI)—United States James Lind Alliance Evidence Gap Maps

Health Research Council of New Zealand—New Zealand

National Institutes of Health (NIH)—United States

Department of Business, Enterprise and Innovation—Ireland

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Morton, R.L., Tuffaha, H., Blaya-Novakova, V. et al. Approaches to prioritising research for clinical trial networks: a scoping review. Trials 23, 1000 (2022). https://doi.org/10.1186/s13063-022-06928-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13063-022-06928-z

Keywords