Activities critical to success and growth of clinical trials networks. What is needed and how are we doing? An Australian and New Zealand perspective

Abstract

Background

Clinical trial evidence underpins evidence-based medicine and the improvement of healthcare worldwide. In Australasia, a significant proportion of clinical trials are conducted by geographically dispersed and multidisciplinary clinical researchers under the auspices of Clinical Trials Networks (CTNs). These groups play an important role in contributing to evidence-based medicine, primarily by conducting investigator-initiated clinical trials. Despite their clear benefits in terms of return on investment, CTNs suffer significant challenges.

Methods

We conducted surveys and focus groups with Australian and New Zealand CTNs to identifying the activities and attributes that enable CTNs to operate successfully. Based on our findings, we then conducted further surveys of Australian and New Zealand CTNs to identify the prevalence of these success factors in existing CTNs.

Results

Our focus groups identified three key themes associated with success and growth of a CTN: engaged membership, established infrastructure, and sustainability; and thirteen critical success factors: shared vision and motivation; strong leaders, governance and succession planning; an executive officer; sustainable funding for operations; effective communication; diverse representation and consumer input; transparent processes; a strong pipeline of trials; a reputable and recognised CTN brand; innovation and adaption; an effective group of network sites with a skilled workforce; embedded trials and prioritisation of research. These key themes and the relevant key areas were presented to 30 CTNs. Two factors were almost universally present in CTNs, reflecting the importance of these attributes: the presence of an executive officer, and a strong pipeline of trials. Three factors had a particularly low prevalence: sustainable funding for operations, effective communication, and embedded trials.

Conclusions

By supporting both emerging and established CTNs to achieve critical success factors, we can improve the efficiency of CTNs to continue to contribute and expand their clinical trial activities. Particular focus needs to be on finding sustainable funding for CTNs, and raising awareness of the critical role undertaken by CTNs to improve healthcare and health outcomes.

Background

Evidence-based medicine is critical to improving patient care and health outcomes [1,2,3]. Underpinning evidence-based medicine is the “individual clinical expertise, the values and desires of the patient, and the best available research” [4]. Such research may be commercially sponsored or publicly funded research. Clinical Trials Networks (CTNs) bring together communities of clinical researchers that are active in defined areas of clinical trials research and often geographically dispersed and multidisciplinary. While some CTNs may facilitate commercially sponsored research, their primary function is to design, conduct, and publish investigator-initiated (and often publicly funded) clinical research. Thus, CTNs play an important role in contributing to evidence-based medicine.

CTNs have a broad and diverse membership encompassing triallists, healthcare providers, and consumer or patient representatives. Through the reach and expertise of their membership, successful CTNs facilitate the delivery of multicentre trials, include peer and consumer review and endorsement processes to develop high-quality patient and clinician relevant clinical trial proposals, have a successful track record of practice-changing trials, have established central and site infrastructure and skilled workforce to support multiple multi-site clinical trials, have capacity for enhanced translation and implementation of clinical trial findings beyond the trial sites, and have a wide variety of stakeholders including international partnerships.

Dependent upon their core activities and responsibilities, CTNs are often defined as either “facilitating” or “coordinating” networks (see Table 1).

Table 1 Core activities and responsibilities of Clinical Trials Networks

The economic benefits of investigator-initiated clinical trials conducted through CTNs have recently been published [5]. In the report, the benefit to cost ratio of investigator-initiated clinical trials was 5.8:1; that is, for every $1 spent on trials in Australia, a return on investment of $5.80 is achieved. Similar economic benefits have been reported internationally, albeit in the context of the cost of trials and not necessarily the cost of the network [6, 7]. This is a remarkable achievement and return on investment for CTNs.

However, the same Australian report noted there remain significant challenges for CTNs, not-the-least their reliance on in-kind support [5]. With this in mind, the Australian Clinical Trials Alliance (ACTA), a national body that supports and represents CTNs, Clinical Quality Registries (CQRs) and Clinical Trial Coordinating Centres (CTCCs) undertook an extensive consultation process with CTNs operating across Australia and New Zealand to identify activities and attributes that enable CTNs to operate successfully (via surveys and focus groups) and then assessed the prevalence of each to support their core remit of maintaining membership to conduct clinical trials to contribute to evidence-based medicine (using surveys). The CTNs were considered as three groups: facilitating, coordinating, and newly established or establishing CTNs.

Methods

Aim

The aim of this study was to identify activities critical to success and growth of clinical trials networks and to assess what activities CTNs currently undertake.

Focus group questions development

An email was sent to the administrative contact of 38 Australian and New Zealand CTNs who were associated with ACTA requesting they review the matrix of activities (see Table 1) and identify any additional structural components or operational activities of CTNs. This matrix was used to develop the focus group question guide. The focus group question guide consisted of semi-structured, open-ended questions, informed by the results of this sector-wide consultation (see Additional file 1: Supplementary Text).

Focus groups

Three focus groups were convened, one for each of the coordinating, facilitating, and newly established/establishing CTNs. Participation in a focus group was primarily a sample of convenience, with the majority of attendees being based in Melbourne, Australia, although all represented national Australian or bi-national Australian and New Zealand networks. A balance between cancer and non-cancer CTNs was considered, given that cancer CTNs have a dedicated funding source (via Cancer Australia) for some CTN infrastructure. Apart from the newly established/establishing CTNs, all invited CTNs were full members of ACTA. Each CTN Chairperson and Executive Officer was invited to join the focus group.

Focus groups were held in September and November 2018, in Melbourne, Australia. All focus groups were recorded and transcribed verbatim.

Focus group analysis

Transcripts of focus groups were analysed and relevant points collated under themes by the primary researcher. Themes were refined and discussion points reclassified, where necessary. The results of this thematic analysis were reviewed by a working party to prevent potential bias from occurring due to a sole individual undertaking the thematic coding. Discussions pertaining to clinical trial coordination were not included in the thematic analysis. Leading themes and critical success factors were identified.

Survey development and implementation for prevalence of success factors in Australasian CTNs

A survey was developed to assess CTNs on the prevalence of thirteen critical success factors identified to operate a CTN successfully. These factors aligned with the findings of our focus groups and international literature on success factors for CTNs. Additional questions were based upon generic organisation capacity surveys covering governance, strategic planning, and succession planning. The survey instrument was approved by the Monash University Human Research Ethics Committee (Project ID 23794), and piloted with two CTNs in February 2020, and a further three CTNs in April 2020. Surveys were sent to all 38 CTNs who were members of ACTA. CTNs were asked to self-assess against a range of questions relevant to each of the 13 success factors (Table 2). Surveys were completed between April and June 2020.

Table 2 Thirteen success factors for CTNs

Statistical considerations for survey

No formal sample size calculation was conducted, and participation was limited to CTNs that were members of ACTA. Sub-group analyses compared the self-assessed ratings of “facilitating CTNs” compared to “coordinating CTNs” and “newly established/establishing CTNs” compared to “established CTNs”. All analyses were conducted using Minitab 17.3.1, with values of p < 0.05 considered statistically significant.

Results

Focus group survey development

Five of 38 CTNs responded with additional items for both structural and operational activities of CTNs. The consolidated list of items appears in Table 3.

Table 3 The structural components and operational activities of CTNs identified during sector-wide consultation

Focus group participants

The “coordinating CTNs” focus group included representatives from five CTNs (Palliative Care Clinical Studies Collaborative, PaCCSC; Australasian Leukaemia and Lymphoma Group, ALLG; Australasian Gastro-Intestinal Trials Group, AGITG; The Australian Kidney Trials Network, AKTN; Australian and New Zealand College of Anaesthetist Clinical Trials Network, ANZCA) with a median of 13 years (range 12 to 47 years) since establishment. There were six attendees (one Chair and five Executive Officers), four of whom attended virtually. The focus group ran for 180 min. The “facilitating CTNs” focus group included representatives from four CTNs (Australasian Stroke Trials Network, ASTN; Australasian Lung Cancer Trials Group, ALTG; Paediatric Research in Emergency Departments International Collaborative, PREDICT; Interdisciplinary Maternal Perinatal Australasian Collaborative Trials Network, IMPACT) with a median of 18 years (14 to 23 years) since establishment. There were five attendees (three Chairs and three Executive Officers), all of those attended in person. This focus group ran for 165 min. The “newly established/establishing CTNs” included three CTNs (Australia and New Zealand Musculoskeletal Clinical Trials Network, ANZMUSC; Australian and New Zealand Alliance for Cardiovascular Trials, ANZACT; Child and Youth Mental Health, CYMH), with three Executive Officers, and was offered virtually. Two CTNs in this focus group had commenced operations during the preceding year, and one had been in operation for 5 years. This focus group ran for 100 min.

Factors identified as critical to CTN success

Three key themes were identified as critical to CTN success and growth: engaged membership, established infrastructure, and sustainability. Within each theme, key areas were identified (Fig. 1). A number of key tools and resources that supported each theme and area were identified as being critical to CTN operations (Table 4).

Fig. 1

Clinical trials networks themes and activities

Table 4 Key tools and resources to facilitate CTN operations

Engaged membership

The most consistent theme associated with success was an engaged membership:

“One of our enablers is engagement and in broader terms one of our barriers is lack of engagement of investigators”

Member engagement is improved if each member may have the opportunity to become a leader by ensuring governance committee members serve a finite number of terms, and the processes for “election” to governance committees is clear. When engagement is high, key operations flourish, and in a circular relationship, when the CTN is operating successfully, membership engagement remains high. A key factor is the intangible network culture that promotes passion, goodwill, and research.

“I think it’s the culture of the people, the enthusiasm, the goodwill. The network runs 90 percent, like all the other networks, on goodwill of clinicians. Without that it would be non-existent”

Governance is also key, and this typically falls to a small number of members that comprise a Steering or Executive Committee. This group needs to be structured, engaged, and transparent. Tools that facilitate transparent and accountable processes are important. Engagement in CTNs is often driven by champions or Executive Committee members, and their communication with the rest of the membership is critical to maintaining engagement.

Uniting members with a clear vision and mission is also important: typically, this mission is to improve healthcare by generating evidence through clinical trials. The vision and mission of a CTN should be clearly developed with a strategic plan that can be operationalised by governance committees.

“… strong motivation among the members to improve outcomes for their patients. … this is partly because outcomes are so poor for many diseases, that gives that group of physicians a very strong imperative to try and improve outcomes.”

Activities that support member engagement such as scientific meetings, workshops and social events, and communication directly to members were also seen as important. Communication with individual members who are contributing at a high level to the CTN must also be considered.

“It’s crucial to keep that communication going and it’s often at the operational level easy to think things are flowing fairly nicely, but then if investigators are not up to date and they suddenly get a question about X or Y or they get an update that there’s a problem, that can cause issues. Communication is important.”

Social media platforms such as LinkedIn, Twitter, and Facebook, that are not specifically targeted to members, were considered less important for communication as they did not necessarily return high membership engagement. However, when using social media, CTNs felt that different platforms should be targeted for different stakeholder groups, for example, CTN members are most likely to access Twitter, but Facebook is more likely to be effective in raising community awareness. From an advocacy perspective, the CTNs noted that their resources were stretched and suggested “trying to get your message out there among everybody else’s message” remained a barrier in the face of limited resource.

An established infrastructure

A CTN requires a structure with effective central network activity that can support and facilitate a pipeline of clinical trials across multiple sites. An Executive Officer is central to operations. The Executive Officer performs a range of duties including supporting governance committees; coordinating events, smaller interest groups and meetings; implementing and measuring strategic objectives; and managing budgets. In addition, they are expected to maintain membership and stakeholder engagement through communication. CTNs must prioritise research, and various methods were employed by CTNs to do this including Delphi processes (a method of formally obtaining consensus), or holding concept development workshops where members indicate support for concepts usually centred around feasibility, lack of existing evidence and unmet need and scientific merit. However, some noted these processes were not perfect.

“We do want to think more strategically about prioritisation though; where are the issues and where are we going to get our greatest value from.”

There were differing models for engaging recruiting sites: either though a site accreditation process or directly with CTN members, who would become clinical trial site principal investigators. CTNs felt there was a burden associated with introducing new sites to a CTN, and their associated processes, which was overcome somewhat in the setting of a coordinating CTN or by a well-resourced central network administration.

“… having the CTN office being able to provide good infrastructure is very helpful because by far the majority of sites don’t have the infrastructure to actually run a multisite trial.”

Trial coordination services were offered by some CTNs, but for others, they relied on the services of the clinical trial coordinating centres.

Consumer engagement was also seen as vital, ensuring the development of patient-relevant clinical trials—however, such engagement is sometimes challenging. Setting up appropriate processes and infrastructure for consumer review is an ongoing need.

“Often the consumers that engage are semi-professional consumers who are representing groups who have very much got their own agenda... I think guidelines or leadership of the consumer group is really important to make them effective.”

Sustainability

Most CTNs had a membership that included a variety of diseases and/or disciplines, which enhanced sustainability, as different trials in different areas could be run concurrently, ensuring a pipeline and trial program was maintained, without exhausting the patient population available for clinical trials. Some CTNs had successfully formed partnerships with Clinical Trials Networks overseas, including in Canada, Europe, UK, and Asia.

As the CTNs rely heavily on committed investigators and members for, among other items, clinical trial ideas and conduct, membership engagement and executive oversight, succession planning for key roles is also important and can be achieved through formal mentorship programs and other activities. This is particularly important for early-career researchers who can benefit from the reputation that the CTN has built based on more experienced researchers. This fostering of early-career researchers is also critical to sustainability of the CTN because it ensures a constant flow of new members who can replace the key roles. Reputation of the CTN within the sector can be built by timely completion and publication of high-quality trials, advocacy and awareness, and through collaboration with other networks. Using the CTN name and logo on publications and presentations is an important branding tool that helps build the reputation of the CTN. Peer review to ensure research conducted was of high-quality and maintain the CTN reputation was seen as an important part of the sustainability, including a commitment to publish trial results at the end of the study. Some CTNs reported difficulties with the publication process, and control of timely publication by members remains an ongoing issue for many CTNs. Various approaches to writing committees were being taken, and the CTNs noted that establishing an authorship policy upfront is important.

A strong and effective pipeline of trials ensures an ongoing portfolio of studies that allows infrastructure to be maintained and retention of key experienced staff. Some CTNs had established processes for piloting clinical studies, and being able to support such studies to demonstrate feasibility was seen as an effective way to enhance success in a major project.

Lack of resources, especially financial ones, remains a significant barrier, with several CTNs noting the lack of appropriate remuneration for trial conduct impacts the workload of site trial coordinators, which in turn hampers engagement with CTNs. A potential solution would be to share trial coordinators between clinical areas at sites, which would ensure better resource utilisation, and justify long-term employment for the coordinator. However, there are limitations to this including study funding, prioritisation, requirements for specific knowledge/experience, and employment arrangements. Others felt that embedding trials into the healthcare system may provide a solution to this resourcing problem, and providing a career pathway for research staff would aid in retention.

“Trials shouldn’t be an add-on to good clinical environments and yet they still are; from the staffing that are at the trial sites, the way the hospitals engage in terms of governance processes, etcetera. It’s still very much an add-on. I’d like to see it become more of core business … with some cost recovery on the hospital side from their participation in trials as part of their core business.”

Funding remains a major issue for sustainability, with few CTNs reporting certain long-term funding. Some have been involved in fundraising efforts, but the investment for this was significant. Alignment with relevant professional societies and medical colleges provided some CTNs with financial support, but this was felt to be more challenging for CTNs with a multidisciplinary membership.

Lastly, the CTNs noted they must remain agile and adaptable, given the evolving clinical trial environment.

CTG prevalence survey respondents

Of the 38 CTNs invited to participate, 30 CTNs completed the survey (response rate 79%). The majority of CTNs were established prior to 2017 (20/30, 66%). “Facilitating CTNs” represented 60% (18/30) with “coordinating CTNs” 40% (12/30).

CTG prevalence survey results

Results of the prevalence survey are reported in Table 5. Typically, the CTNs had memberships that included a variety of disciplines, and most (87%) provided opportunities for consumer involvement and engagement. Emerging CTNs were less likely to describe their strategic plan as good or adequate compared to established CTNs (50% vs 86%), and coordinating CTNs were more likely to have a strategic plan compared to facilitating CTNs (100% vs 56%), and describe their plan as good or adequate (92% vs 50%). Succession planning was less established in emerging CTNs compared to established CTNs (40% vs 80%). While most CTNs employed an Executive Officer, the funding sources used were variable—the predominant source of funding was grant funding (27%), or grant funding supported by other sources (13%). Only 7% reported funding was via membership fees. Operational funding was via research grants (47%), government sources (40%), philanthropic funding (27%), or parent organisations (23%). Only 10% of CTNs received funding from industry or commercial sponsors. Almost half of respondents (47%) reported that they had no ongoing operational funding.

Table 5 CTN prevalence survey results

One in three CTNs reported having a written communication strategy, but there were differences between emerging vs established and coordinating vs facilitating CTN subgroups. In terms of establishing a reputable and recognised brand, the most commonly reported measures were a website and social media updates. Reputation was also supported by regular monitoring of CTN milestones. Importantly, there was no difference observed between established and emerging CTNs, although coordinating CTNs were much more likely to undertake monitoring of clinical milestones compared to facilitating CTNs (92% vs 39%).

Collaborations with other CTNs either internationally, or on a cross-disciplinary basis to contribute to the pipeline of clinical trials was common (90%), but was more common in established (100%) than emerging CTNs (60%) and in coordinating than facilitating CTNs (100% vs 75%). Emerging CTNs were also more likely to interact with sites regularly compared to established CTNs (89% vs 67%).

Most CTNs reported good or adequate structures for responding to change, with better adaptive capacity observed in more established or coordinating CTNs. Embedding trials into standard practice has the ability to simplify data collection, and this was reported by 13 CTNs, seven CTNs reported having partnerships with hospitals to embed trials, but only four CTNs had a documented strategy for embedding trials.

Trials are prioritised by informal discussions within the CTN (80%) or through concept development workshops (57%). Just under one-half of CTNs used an annual strategic prioritisation process (40%), while just under one-quarter (23%) reported using a Delphi process.

Discussion

Our study has identified themes of success for CTNs, the prevalence of these success factors, and the tools which could operationally support them. Following this research, ACTA has developed a toolkit that aims to support the work of CTNs (see https://clinicaltrialsalliance.org.au/resources/). Identifying and developing tools to support the critical operations for success is a key foundation for maximising efficiency and effectiveness in the sector.

In this study, we engaged with the CTNs to uncover the factors they thought were necessary for success. It remains to be seen whether these factors are those that are actually useful. Of the 13 key factors critical for success of CTNs, we found three that had particularly low prevalence: sustainable funding for operations, effective communication, and embedding trials. Sustainable funding remains problematic for CTNs, with most CTNs relying on research grants for the majority of their operations.

Clinical research underpins evidence-based medicine, which in turn leads to better healthcare. However, clinical trials may fail for a number of reasons outside of safety and efficacy [8]. Just over one-fifth of phase 3 studies fail due to lack of funding, and a significant number of trials are underfunded [8]. This lack of funding impacts not only on the ability to complete studies but also on ethics given patients may be recruited into a study that never completes. A key component of sustainable clinical trial funding is the inclusion of a budget to sufficiently resource for clinical trial development and multi-site infrastructure, such as CTNs. Alternative funding streams, such as direct hospital or government funding, are urgently needed to ensure the sustainability of the CTN sector in Australia. With a return shown to be 5.8:1, this funding would be a wise investment [5]. In the absence of this, CTNs are left to resort to fundraising or collaboration with international clinical trials or commercial clinical trials.

Whether an engaged membership is a result of or a cause of a successful CTN remains difficult to distinguish and perhaps may be a circular relationship. Despite this, engagement must be a focus at the outset as membership engagement is critical to the success of CTNs, with communication vital to ensuring their ongoing engagement. So, while effective communication is important for success, the effectiveness of both internal and external communication was reportedly poor. This may be due to the reliance on voluntary goodwill to produce the CTN communications often through newsletters and social media. Communication within a trial is part of the coordination of the trial itself. For coordinating CTNs, others have reported that communication in any trial setting is critical and is particularly so in studies that are spread across different states [9]. Support of clinical trial sites can be maintained through centralised training and weekly “frequently asked questions” sessions [9]. Relationship building between investigational sites and the network can improve the quality of work [10]. Clear and concise communication aids this relationship [10]. It is important that any messaging from the CTN to its trial sites or indeed any of its stakeholders is consistent, transparent, and organised [10].

Embedding trials into clinical practice also had poor prevalence, although this may be explained, at least in part, by a lack of understanding on the respondent’s part about how this is done operationally or because there was no documented procedure for how this was done. It was unclear whether this is a critical factor or an aspirational goal of CTNs and would certainly be reliant on policy, systems, services, management, and clinician buy-in beyond the CTN environment. However, given the focus on translational research, this would appear to be an appropriate strategic goal. Implementation of clinical trial results into practice at sites involved with the trial would also be more successful if trials are embedded, allowing sites and workforces to already be familiar and set up for practice change.

Notably, there were differences in the prevalence of key success factors between emerging and established CTNs. Consistently lacking in emerging CTNs were sustainable funding, an effective group of network sites, embedded trials, and effective communication. Our research has not yet identified how CTNs prioritise their activities; however, it appears there are translational issues that influence an individual CTN’s performance and, that over time, CTNs consolidate their systems and processes. Further, coordinating CTNs had high prevalence for seven of the 13 factors identified, while facilitating CTNs had high prevalence on only two of 13 factors (the presence of an executive officer and collaboration for a strong pipeline of trials—suggesting these two are possibly the most critical factors associated with success). Accordingly, the different roles of CTNs should be considered when interpreting differences in the prevalence of the critical success factors. Alternatively, it may reflect that facilitating CTNs tended to be younger and still establishing many of the critical factors. Those that had been established may simply be of higher priority.

There are a number of limitations in the prevalence survey. Firstly, the survey was designed to include factors that the involved CTNs thought were the most important factors and not what have been shown to be the most important, and as such, many of the factors may not actually be critical to success in some instances. It was a once-off survey, and in isolation, there are limits on the interpretation that may be drawn from a snapshot assessment. Repeating the survey annually or biennially may help identify more clearly the prevalence of success factors, assess whether those emerging CTNs show increased prevalence of success factors over time, and determine what success factors appear to be important at different stages of evolution of a CTN or for different types of CTNs. A limitation of the focus groups was that while the 13 key success factors were identified using a robust focus group approach, it would have been useful to explore whether the critical success factors differed between coordinating and facilitating CTNs. Given the majority of participants were from Melbourne, Australia, there may be limits to how generalisable the findings are for the CTNs across Australia and New Zealand.

CTNs play an important role in clinical trials research. By supporting both emerging and established CTNs to achieve critical success factors that influence success, we may be able improve the efficiency of CTNs. By continuing or expanding their clinical trial activities, which have already been established to be significant return on investment, we will improve evidence-based medicine and quality healthcare [5]. Particular focus needs to be on finding sustainable funding for CTNs and raising awareness of the critical role undertaken by CTNs to improve healthcare and patient outcomes. The Australian Clinical Trials Alliance is currently raising awareness and providing evidence for the cost-effectiveness of these networks to advanced evidence-based medicine.