Skip to main content

Development, acceptability, appropriateness and appeal of a cancer clinical trials implementation intervention for rural- and minority-serving urology practices



Few community urologists offer cancer patients the opportunity to participate in cancer clinical trials, despite national guidelines that recommend it, depriving an estimated 260,000 urological cancer patients of guideline-concordant care each year. Existing strategies to increase urologists’ offer of clinical trials are designed for resource-rich environments and are not feasible for many community urologists. We sought to design an implementation intervention for dissemination in under-resourced community urology practices and to compare its acceptability, appropriateness and adoption appeal among trial-naïve and trial-experienced urologists.


We used a design-for-dissemination approach, informed by the Theoretical Domains Framework and Behavior Change Wheel, to match determinants of the clinical trial offer to theoretically informed implementation strategies. We described the implementation intervention in evaluation workshops offered at urology professional society meetings. We surveyed participants to assess the implementation intervention’s acceptability and appropriateness using validated instruments. We also measured adoption appeal, intention to adopt and previous trial offer.


Our design process resulted in a multi-modal implementation intervention, comprised of multiple implementation strategies designed to address six domains from the Theoretical Domains Framework. Evaluation workshops delivered at four meetings, convened five separate professional societies. Sixty-one percent of those offered an opportunity to participate in the implementation intervention indicated intention to adopt. Average implementation intervention acceptability and appropriateness ratings were 4.4 and 4.4 (out of 5), respectively. Acceptability scores were statistically significantly higher among those offering trials compared to those not (p = 0.03). Appropriateness scores did not differ between those offering trials and those not (p = 0.24). After urologists ranked their top three innovation attributes, 43% of urologists included practice reputation in their top three reasons for offering clinical trials; 30% listed practice differentiation among their top three reasons. No statistically significant differences were found between those who offered trials and those who did not among any of the innovation attributes.


LEARN|INFORM|RECRUIT is a promising implementation intervention to address low accrual to clinical trials, poised for implementation and effectiveness testing. The implementation intervention is appealing to its target audience and may have equal uptake among trial-naïve and trial-experienced practices.

Peer Review reports


Participation rates in clinical trials to develop new cancer therapies have been directly linked to improvements in population-level outcomes for the subgroups of patients who participate in them [1]. However, only about 8% of cancer patients participate in trials, thwarting the development of new therapies [2]. A study conducted through the National Cancer Institute’s (NCI’s) National Clinical Trials Network found that 18% of cancer trials closed with low accrual or achieved less than 50% of their target enrollment at 3 years or more after the start of the trial [5]. In contrast, a large majority of cancer patients report willingness to participate [6,7,8], but may never be offered the opportunity. Many cancer clinical trials are conducted at National Cancer Institute-designated cancer centers, academic centers and select community oncology practices supported by the NCI [9]. Yet relatively few cancer patients are treated at these sites where clinical trials are available, resulting in less than half of all cancer patients having access to trials [2, 10, 11]. These structural limitations in how cancer care is delivered have recently been identified as the most important factor leading to low patient participation [2].

Structural constraints in cancer care delivery are particularly limiting for urological cancer patients’ access to clinical trials. Urological cancer accounts for one in every five newly diagnosed cancers in the US each year, involving approximately 330,000 patients [12]. Most of these cancers are diagnosed, and treatment paths established, by urologists, without input from medical oncologists to whom clinical trial infrastructure is most often directed [13]. Although multidisciplinary care is common in the treatment of some cancers [14, 15], multidisciplinary care in urological cancers is less well established, and particularly rare in community settings, where 80% of urological cancer patients are treated [1, 14, 16,17,18]. Consequently, relative to other cancer patients, approximately 260,000 urological cancer patients may be less likely to be offered clinical trials at the point of treatment decision-making and their treatment plans are less likely to include the systematic clinical trial consideration that is often part of multidisciplinary case review [19,20,21]. Despite interventions which reach into the community to increase access to cancer clinical trials [16, 22,23,24,25,26], community urologists report little awareness of the urological cancer trials available, even those available in their local community. They are less likely to report access to clinical trials than their academic counterparts [13], and few report offering clinical trials to patients [26]. To address the structural challenges which limit cancer patients’ access to clinical trials, thereby slowing the development of effective cancer therapies, the American Cancer Society in 2018 convened a national committee to make recommendations for overcoming these hurdles [2, 28]. That committee ranked development of new strategies targeting non-research sites to refer interested patients to trial opportunities as a top priority [28].

Unfortunately, little research exists to guide the development of implementation strategies aimed at research-naïve providers. To date most research on physician referral to clinical trials has focused on physicians at institutions already engaged in research [6, 29,30,31,32,33]. However, physicians with access to clinical trials may have needs that are different than those who do not [34,35,36]. For example, community urologists experienced in clinical trial referral describe perceived barriers among their less experienced peers that they know do not, in fact, exist in practice, suggesting that different strategies may be required at different points across the adoption to implementation continuum. For example, strategies needed to facilitate adoption may need to address perceptions of the potential impact on practice (e.g., the myth of “losing patients”), whereas those needed to facilitate implementation may need to address memory and attention through reminders and workflow integration. Thus, it is important to evaluate potential implementation strategies, distinguishing these two levels of experience.

Research-naïve urologists in community practice have reported willingness to try strategies to facilitate the offer of cancer clinical trials in their practice, but also report potential barriers in their ability to do so. Previous research based on the Theoretical Domains Framework (TDF) [37] has suggested that the offer of clinical trials may be influenced by constructs in select TDF domains: environmental resources; social influences; knowledge; memory, attention and decision processes; social/professional role and identity; and beliefs about consequences. Thus, an implementation intervention addressing these behavioral determinants may be effective in increasing the reach of clinical trials. Because the determinants were identified in rural-serving practices in a single state, the degree to which the implementation intervention would appeal to a broader group of urologists, particularly those in other rural-serving communities and minority-serving communities, was unknown. The present study describes the development and appeal of an implementation intervention addressing these behavioral determinants.


The objective of this study was to apply an implementation science approach to develop an implementation intervention to increase urology practices’ referral to cancer clinical trials and to compare the acceptability, appropriateness and appeal of the approach between urologists naïve to clinical trials and those experienced in offering clinical trials across a diverse group of urology practices in the South and Midwest United States.


We used a design-for-dissemination approach [38,39,40] informed by the Behavior Change Wheel [41] to develop an implementation intervention to address behavioral determinants of offering clinical trials. We then presented the implementation intervention to a diverse sample of urologists and surveyed them to evaluate the implementation intervention’s acceptability, appropriateness and appeal. The Institutional Review Board of a Midwestern university reviewed and approved the study.

Implementation intervention development

We assembled a transdisciplinary team of urologists, implementation scientists, education technologists and communications experts to increase the likelihood that the resulting approach would be adopted and used in practice [38,39,40]. We followed the six steps of the Behavioural Change Wheel (Fig. 1) to: (1) define the problem in behavioral terms; (2) select the target behavior; (3) specify the target behavior; (4) identify determinants of the target behavior; (5) identify intervention options; and (6) match behavioral-change techniques and mode of intervention delivery to the intended users’ context [42]. The Behaviour Change Wheel is derived from a distillation of 19 frameworks of behavior change through systematic literature review [42]. As others have noted, the process was iterative [43], resulting in our final intervention package and implementation strategies.

Fig. 1

Six steps of implementation intervention development

For Steps 1–3, we conducted narrative review of the available literature, relied on interviews with urologists about their decision-making processes in cancer care [44,45,46,47,48], and elicited clinical experience of study team members. Semi-structured qualitative interviews of 22 academic and community US urologists were conducted at the 2015 American Urological Association (AUA) annual meeting and by telephone, subsequent to the meeting. As part of the larger study on barriers to a particular treatment, interviews were transcribed and coded and the cancer care delivery process was mapped to understand the urology practice context.

Behavioral determinants to community urologists’ offer of clinical trials, needed for Step 4, were collected according to the TDF and reported elsewhere [26]. Briefly, we conducted semi-structured interviews of urologists and urology staff in rural-serving community urology practices in Kansas. Participation was limited to practices which had urological oncology trials open locally at the time of the interview. We analyzed the data, applying template analysis using a codebook of TDF constructs. The results were also used to understand the clinical trial referral process in behavioral terms (Step 1) and context surrounding the behavior (Step 6).

For each determinant identified in the previous study as salient, we selected intervention functions or policy categories from the Behavior Change Wheel’s list of nine functions and seven policy categories (Step 5). Finally, in Step 6 we reviewed behavioral-change techniques to identify those most relevant to each function or policy and simultaneously considered the best mode of delivery to arrive at theoretically supported implementation strategies to deliver the components. Selection was informed by feasibility and opportunities in the local communities in which urologists practiced. We interviewed cancer centers in the communities of urology practices participating in the determinants study about their willingness and experience engaging urology practices. We used a strategic planning approach and assessed their responses to identify strengths, weakness, threats and opportunities for supporting urology practice efforts. Thus, feasibility of potential approaches was used to inform the identification of optimal intervention options to address practice-identified determinants of referral and to select the most potentially effective behavioral-change techniques and modes of intervention delivery.

Implementation intervention evaluation

Evaluation approach

We created a workshop to present and test the appeal of the implementation intervention. The didactic workshop included evidence-based educational content about the value of clinical trials from a health care providers’ perspective, including the impact on the practice, the clinical workflow and the patient-provider relationship. Other content described the implementation intervention components. Educational content was delivered by community oncologists who were members of the local professional societies to which they were presenting and identified as influencers by organizers of the local meetings. Information about the implementation intervention was delivered by the implementation intervention developers. We organized the educational sessions in conjunction with local urology professional society meetings. Targeted urology professional societies served both state and regional professional societies. We chose multiple loco-regional venues over a single national venue as the local meetings are preferred by community urologists from smaller practices. The workshops were offered to all prospective meeting attendees at each of the meetings via letter to all urologists in the states served by state societies, and via announcement through the program booklet and meeting email communications sent to regional society members. Prospective attendees at the state meetings were offered US$50 gift cards for participation in a 1-h meeting. Prospective attendees at the regional meeting were offered CME credit for participation in a 1.5-h workshop. The evaluation workshop was delivered at four meetings, convening five separate professional societies. The societies represented urologists across 10 states in the Midwest and South. Workshop location and placement within the official meeting agenda varied.


Intervention appeal was evaluated via survey distributed to a purposive sample of all workshop attendees (Additional file 1). The survey instrument included three scales to assess the appeal of the implementation intervention and a single item assessing whether participants currently offer clinical trials. We used the four-item, validated scales, the Acceptability of Intervention Measure (AIM) [49] and the Intervention Appropriateness Measure (IAM) [49] to measure acceptability and appropriateness. We used a novel measure, the Attributes of Innovation Adoption [47] scale to assess the implementation intervention’s appeal. All items were rated on a Likert response scale ranging from 1 to 5 with higher scores indicating greater acceptability, appropriateness, or attribute appeal. Participants were asked to rank the three innovation attributes that were most important to them. We measured behavioral intent to adopt the implementation intervention by asking participants to provide contact information for follow-up, collected separately from the survey responses to protect confidentiality. Due to capacity limitations, behavioral intent was only measured when and where there was capacity to deliver the implementation intervention.


To calculate the acceptability and appropriateness scores, we summed the Likert ratings and averaged them across the four items. Each innovation attribute was scored individually. Ratings greater than 3.0 were considered salient to adoption. Student’s t test was used to assess differences by current offer of clinical trials. We combined the top ranked innovation attributes and calculated the frequency each was included in the top three among all participants.


The implementation intervention

Step 1. Define problem in behavioral terms

Narrative literature review suggested that patients do not participate in clinical trials due to a number of reasons. Often, they are asked too late in the process, after treatments have already been decided upon [50] or initiated [51]. Thus, ensuring that clinical trials be considered as a treatment option at the point of initial treatment decision-making is critical. Therefore, we articulated the clinical trial issue from a community urologists’ workflow (Fig. 2) to identify the ideal time to place clinical trials and provide scaffolding to understand lack of accrual from a behavioral perspective. In typical practice, potential cancer patients present for biopsy and return for treatment counseling, either in conjunction with result presentation or following telephoned results. Urologists devise a preliminary treatment recommendation upon reviewing the results, the timing of which varies by practice. This treatment recommendation is presented and discussed with the patient at a treatment counseling visit and a treatment is scheduled, sometimes with additional visits for discussion of treatment options.

Fig. 2

Process map to identify and select target behaviors (Step 1)

The target for behavior change was identified as the urologist. Urologists and staff agree that identifying and discussing treatment options falls within the professional responsibilities of the urologist [19]. However, physicians require motivation to include trials in their assessment and awareness of what trials are available [26, 52, 53]. To incorporate trials as a treatment option into treatment counseling, they need knowledge and skills to initiate the conversation, often lacking among most physicians who have little exposure to trials in their medical training.

A structural barrier to clinical trial accrual arises because most trial participation models require a practice to undergo an intensive process to obtain human subject credentialing, and make substantial additional investments in learning about trials, opening them in their practice, and conducting extensive data collection [54]. This may be an unrealistic expectation for many providers, but particularly for community-practicing urologists focused solely on clinical service provision. Based on the willingness to engage in the low-intensity efforts we observed in our qualitative study, we reconceptualized the target behavior as referral for eligibility screening, rather than eligibility screening itself. To execute the referral, urologists need trusted referral partners to whom they can refer. They also need a mechanism by which they can efficiently communicate with the referral partner to ensure that patient care—for which they are ethically and legally responsible—is delegated appropriately [26].

Step 2. Select target behavior

Having defined the problem in behavioral terms, we thus selected the target behaviors as: (1) considering clinical trials as a treatment option; (2) treatment counseling inclusive of clinical trials; (3) the act of referral; and (4) cancer program communication (Fig. 2).

Step 3. Specify target behavior

We specified each target behavior based on our interviews with community urologists and their staff, with particular attention paid to aligning the behaviors with the professional roles and identities perceived among both physicians and staff [26]. Table 1 illustrates for each target behavior who, what, when, where, how often, and with whom each behavior should occur. For simplicity, we recommend that urologists should consider, counsel and refer all newly diagnosed cancer patients for clinical trial eligibility screening. Consideration should occur at review of biopsy results using the broad eligibility criteria provided by the trial expert in the normal location of review. The referral should occur at the treatment counseling visit prior to patient decision on treatment. Notification of the referral to the cancer program should occur immediately following referral and could be delegated to clinic staff.

Table 1 Specification of the behavior

Step 4. Identify determinants of behavior

Previous work identified six of the 14 TDF domains as most salient for community urologists’ initial adoption of the four key clinical trial behaviors [19]. In particular, community urologists could be highly motivated by urology peers and professional societies to offer clinical trials. They believed that their trial involvement could accrue positive social consequences to their practice by differentiating their practice from others. They saw referral to a trial specialist as a natural alignment with the current practice of referring cancer patients to specialists in other treatment modalities (e.g., radiation oncologist). They acknowledged the need for concise trial information to consider trials as a treatment option, and brief skills training to counsel patients. They needed reminders of trial opportunities at the point of care and support for their patients, not only to prepare them for treatment counseling, but also to help them understand the referral process.

Step 5. Identify intervention options

We mapped each determinant to an empirically or theoretically supported intervention function or policy category from the Behavior Change Wheel. Table 2 lists the key determinants and each determinants’ COM-B category in columns 1 and 2, and the corresponding intervention functions and policies in columns 3 and 4.

Table 2 Mapping of behavioral determinants to intervention options and content and implementation options

Step 6. Identify behavior-change techniques and implementation options

For each intervention function or policy needed, our team brainstormed behavior-change techniques and potentially effective modes of delivery. These were iteratively shaped while reviewing opportunities available in the community or within national support structures and considering the contextual determinants in community practice. Columns 5 and 6 of Table 2 list the behavior-change techniques and specific mode of delivery.

Our design process resulted in a multi-modal implementation intervention comprised of multiple implementation strategies. Together, this set of strategies is called LEARN|INFORM|RECRUIT and includes continuing education workshops, newsletters, clinical trial reminders, point-of-care materials, referral tools, referral network building and patient-support materials. It is available to practices interested in adopting the program in controlled evaluation studies at Upon enrollment, LEARN|INFORM|RECRUIT is made available as an implementation intervention tailored to each practice through personalization of trial opportunities available in the local community, development of a dedicated referral pathway to the cancer center, and practice branding of patient-facing materials. It is externally facilitated by trained study staff through a mixture of in-person, telephone and electronic communications delivered in the practice setting and supported by a non-public website of video content. Other materials are printed for the practices’ convenience and mailed to the practice at program launch and when updates are necessary (e.g., opening of new clinical trials).

Implementation intervention evaluation


Characteristics of the evaluation workshops, audience size and participation are presented in Table 3. Across the four workshops 67 participants registered attendance (15% of all professional society meeting attendees). Fifty-four participants across the four meetings evaluating the implementation intervention provided evaluations with no missing data (81% response rate). Among them, 78% reported currently offering trials at their practice.

Table 3 Evaluation workshop meeting characteristics

Implementation intervention appeal

Among those attending the three workshops offering an opportunity to participate in the implementation intervention, 61% (33/54) asked to be contacted to participate in the program. Potential adopters represented urology practices from eight US states and two states in Mexico. Urologists from eight of 10 US states that were targeted as part of the evaluation workshop expressed interest in adopting the program. A practice from a state not targeted also expressed interest.

Intervention acceptability and appropriateness

Average implementation intervention acceptability and appropriateness ratings were high: 4.4 and 4.4 (out of 5), respectively (Table 4). Acceptability scores were statistically significantly higher among those offering trials compared to those not (p = 0.03). Appropriateness scores did not differ between those offering trials and those not (p = 0.24).

Table 4 Ratings of acceptability and appropriateness by prior offer of clinical trials

Top-rated innovation attributes (Fig. 3) were: (1) helping the urologist to match the right patient to right treatment; (2) increasing a practice’s reputation as offering cutting-edge treatment options; (3) helping to make care more patient-centered; (4) helping the urologist to adhere to practice guidelines; (5) differentiating the urologists’ practice from other practices; (6) lessening the patients’ risk of decisional regret; and (7) decreasing the need to refer. No statistically significant differences were found between those who offered trials and those who did not among any of the innovation attributes. After urologists ranked their top three innovation attributes, 43% of urologists included practice reputation in their top three reasons for offering clinical trials; 30% listed practice differentiation among their top three reasons.

Fig. 3

Average rating of innovation adoption attributes across the sample, n = 54. Higher scores indicate greater agreement


We developed an implementation intervention that is viewed as highly acceptable and appropriate to professionally engaged urologists, a majority of whom indicated an intention to adopt the program. The implementation intervention potentially has broad reach, as our participants represented both urban and rural, and community and academic practices across a sizable geographic portion of North America. Confirming our implementation intervention design, which underscored practice differentiation as a key motivator, participants highly valued innovations which would set them apart from other providers. Further, our design of the program to position the offer of clinical trials as an expansion of treatment options was validated by urologists’ high ratings of matching the right patient to the right treatment as an appealing innovation attribute. Because implementation intervention users have defined this idea of “precision medicine” as an expectation for an innovation that they are willing to adopt, any future implementation intervention effectiveness assessments should include the approach’s ability to deliver on this expectation.

The program we designed is unique in multiple ways among strategies to increase the offer of clinical trials. First, it focuses on the structural disjuncture in the care delivery environment that separates urological cancer patients from several guideline-based cancer services, including clinical trials. In this way, it is designed to increase the reach of clinical trials, a critical component of intervention effectiveness [55]. Increasing reach may be particularly important in the current era of cancer clinical trials, where precision medicine trials requiring higher levels of population screening to identify eligible participants are gaining dominance [56, 57]. Second, it leverages the existing professional identities of community-based urologists. Rather than requiring practices to provide the infrastructure to open trials in their own practice, such as study personnel and IRB regulatory requirements, it allows them to capitalize on natural referral behaviors and expands their networks to include trial experts. Other programs, such as the Society of Urological Oncology-Clinical Trials Consortium, cater to larger practices that can invest in research infrastructure [22]. While successful, not all practices can provide this level of skill and capacity to meet guideline recommendations. The National Cancer Institute’s Community Oncology Research Program, although effective in expanding trial options to community providers [1, 58], focuses on oncology practices [16] and the degree to which they effectively extend to non-oncology cancer providers is not well understood [59]. Other initiatives mainly offer educational and navigational support for patients [23, 24, 60,61,62] and rely on patient activation in a patient-physician dynamic which weighs heavily toward physician influence. Without recommendation of the physician, few patients will participate in trials [6, 63]. Finally, our intervention is unique because, to our knowledge, it is the first to apply principles of implementation science to the decades-old, intransigent problem of low accrual to clinical trials. Continued research to assess its effectiveness can further the developing field. Although the implementation intervention was highly rated, we observed some differentiation in its acceptability ratings between urologists experienced in offering clinical trials and research-naïve urologists. While both rated the implementation intervention as highly acceptable, those with experience rated it significantly higher. Although the instrument used to measure acceptability has been validated [64], this is the first study, to our knowledge, to provide scores for a population of potential adopters. Thus, scores have not been normalized among other interventions or adopters, providing little context in which to interpret either the ratings observed in this study or their differentiation among users. Future studies should compare ratings against those found in this study. Nonetheless, we observed no differentiation in ratings of appropriateness or individual attributes of innovation appeal, suggesting that, perhaps, in this sample of urologists, trial-naïve participants thought that they should offer clinical trials, but may still see components of the implementation intervention as not completely aligning with their practice ecology.

Moreover, there may be some motivational goals that we have not identified. Attributes included in the measure were identified from qualitative interviews with urologists around treatment decision making in prostate cancer and in the context of offering clinical trials [47]. The items remain to be validated in a larger sample of providers. However, participants provided no additional innovation attributes that promote adoption when afforded the opportunity.

Our implementation intervention is designed on some key assumptions. One is that community urologists are not currently engaging their patients about clinical trials. Although Ellis et al. reported recent qualitative data that supports this [19], the last nationally representative survey of urologists was conducted in 2005. By approaching urologists through state professional societies, we reached more providers who self-reported that they already offered clinical trials than those who self-identified as trial-naive. Thus, contemporary assessment of urologists’ involvement in clinical trials is needed. Changes in both the clinical trial landscape (i.e., growth of precision medicine trials, increased presence of National Cancer Institute Community Oncology Research Program (NCORP)) and urological cancer care delivery (e.g., continued consolidation of practices, decline in solo practice, shortages of common bladder cancer therapies currently in clinical trial) may have impacted urologists’ engagement in trials. One opportunity is to query the NCI’s Community Oncology Research Program to identify how many cancer components already include urologists.

Finally, we report potential adopters’ assessment of the acceptability, appropriateness and appeal of the implementation intervention, we did not test feasibility among urology practices, effectiveness of the approach in increasing referrals or accrual, or, importantly, the acceptability of the approach to cancer programs that are essential partners in the implementation approach. Academic physicians and cancer center administrators and staff will be critical partners supporting community physicians’ offer of trials. Thus, it is essential to assess these partners’ engagement, and willingness to participate, and to identify what successful strategies have already been tried [65]. Further, future research should address the degree to which cancer programs offering clinical trials would welcome additional eligibility screening responsibilities and measure the additional workload required.


Our study is not without limitations. We developed and tested the implementation intervention across the Midwest and select Southern states and found it to be appealing. However, results may not be generalizable. Whether the appeal of the implementation intervention extends to other practices awaits future opportunities to offer the program in other regions of the country. Efforts to extend the implementation intervention to other regions of the country are ongoing and should be rigorously tested with the appropriate study design. A cluster-randomized trial can assess whether an intervention informed by principles of implementation science can increase the rate of cancer patients’ participation. Secondly, our rigorous, theoretically based implementation development process should promote rapid adoption and implementation of the program. However, both whether the program will be adopted and the predictive value of acceptability and appropriateness ratings on subsequent adoption should be measured.

Finally, the application of the Behavior Change Wheel is subjective and resulted in a large number of intervention components. Although the framework is theoretically and empirically derived, mapping of behavior-change techniques to specific determinants of behavior is not yet an exact science. A great deal of heterogeneity in mapping the most appropriate technique to a specific determinant has been observed [66]. Thus, others may have categorized determinants, functions, or behavior-change techniques differently. Members of our team have experience with determinant coding and some have training in behavior-change technique coding, which should promote concordance. Nonetheless, the inability to empirically map the behavior-change techniques to determinants underscores the need for effectiveness testing. Additional implementation research can identify the core components of the implementation intervention and inform future efforts to optimize those implementation strategies.


Low-enrolling studies slow the uptake of innovations in cancer care [2, 58], and are costly to study sponsors and the institutions that conduct them [67]. We have developed a promising implementation intervention to address this problem in an understudied segment of cancer providers. The implementation intervention is appealing to its target audience. Implementation intervention effectiveness and the effect of its acceptability, appropriateness and appeal on the adoption of the offer of clinical trials should be further tested.

Availability of data and materials

The dataset created and analyzed during the current study is available from the corresponding author on reasonable request.



Acceptability of Intervention Measure


American Urological Association


Intervention Appropriateness Measure


National Cancer Institute


South Central Section (of the American Urological Association)


Theoretical Domains Framework


  1. 1.

    Unger JM, Cook E, Tai E, Bleyer A. The role of clinical trial participation in cancer research: barriers, evidence, and strategies. Am Soc Clin Oncol Educ Book. 2016;35:185–98.

    Article  Google Scholar 

  2. 2.

    American Cancer Society Cancer Action Network. Barriers to patient enrollment in therapeutic clinical trials for cancer: a landscape report. Washington, D.C.: American Cancer Society; 2018.

  3. 3.

    Korn EL, Freidlin B, Mooney M, Abrams JS. Accrual experience of National Cancer Institute Cooperative Group phase III trials activated from 2000 to 2007. J Clin Oncol. 2010;28(35):5197–201.

    Article  Google Scholar 

  4. 4.

    Schroen AT, Petroni GR, Wang H, Thielen MJ, Gray R, Benedetti J, et al. Achieving sufficient accrual to address the primary endpoint in phase III clinical trials from U.S. Cooperative Oncology Groups. Clin Cancer Res. 2012;18(1):256–62.

    Article  Google Scholar 

  5. 5.

    Bennette CS, Ramsey SD, McDermott CL, Carlson JJ, Basu A, Veenstra DL. Predicting low accrual in the National Cancer Institute’s Cooperative Group Clinical Trials. J Natl Cancer Inst. 2016;108(2):article number djv324.

    Article  Google Scholar 

  6. 6.

    Albrecht TL, Eggly SS, Gleason ME, Harper FW, Foster TS, Peterson AM, et al. Influence of clinical communication on patients’ decision making on participation in clinical trials. J Clin Oncol. 2008;26(16):2666–73.

    Article  Google Scholar 

  7. 7.

    Moorcraft SY, Marriott C, Peckitt C, Cunningham D, Chau I, Starling N, et al. Patients’ willingness to participate in clinical trials and their views on aspects of cancer research: results of a prospective patient survey. Trials. 2016;17:17.

    Article  Google Scholar 

  8. 8.

    Kaplan CP, Napoles AM, Narine S, Gregorich S, Livaudais-Toman J, Nguyen T, et al. Knowledge and attitudes regarding clinical trials and willingness to participate among prostate cancer patients. Contemp Clin Trials. 2015;45(Pt B):443–8.

    Article  Google Scholar 

  9. 9.

    Dimond EP, St Germain D, Nacpil LM, Zaren HA, Swanson SM, Minnick C, et al. Creating a “culture of research” in a community hospital: strategies and tools from the National Cancer Institute Community Cancer Centers Program. Clin Trials. 2015;12(3):246–56.

    Article  Google Scholar 

  10. 10.

    Institute of Medicine (US) Forum on Drug Discovery Development and Translation. Transforming Clinical Research in the United States: Challenges and Opportunities: Workshop Summary. The National Academies Collection: Reports funded by National Institutes of Health. Washington (DC): The National Academies of Sciences, Engineering, and Medicine; 2010.

  11. 11.

    American Society of Clinical Oncology. The state of cancer care in America, 2017: a report by the American Society of Clinical Oncology. J Oncol Pract. 2017;13(4):e353–e94.

    Article  Google Scholar 

  12. 12.

    U.S. Cancer Statistics Working Group. U.S. Cancer Statistics Data Visualizations Tool, based on November 2018 submission data (1999–2016). Atlanta: US. Department of Health and Human Services, Center for Disease Control and Prevention and National Cancer Institute; 2017.

  13. 13.

    Swanson GP, Carpenter WR, Thompson IM, Crawford ED. Urologists’ attitudes regarding cancer clinical research. Urology. 2007;70(1):19–24.

    Article  Google Scholar 

  14. 14.

    Smelser WW, Holzbeierlein JM. Multidisciplinary care in genitoruinary cancer treatment: an imperative for the care continuum. Alexandria: 2018 Gentiourinary Cancers Symposium; 2018.

  15. 15.

    Prades J, Remue E, van Hoof E, Borras JM. Is it worth reorganising cancer services on the basis of multidisciplinary teams (MDTs)? A systematic review of the objectives and organisation of MDTs and their impact on patient outcomes. Health Policy. 2015;119(4):464–74.

    Article  Google Scholar 

  16. 16.

    National Cancer Institute. National Cancer Institute Community Oncology Research Program (NCORP) 2018. Available from: Accessed 13 Nov 2018.

  17. 17.

    Williams SB, Kamat AM. The multidisciplinary approach to prostate cancer management: from diagnosis and beyond. Everyday Urol Oncol Insights. 2018;2(2):article number 98076.

  18. 18.

    Berger ER, Shore N. Our prostate cancer patients need true multidisciplinary care. Oncol Times. 2005;27(19):4.

    Article  Google Scholar 

  19. 19.

    Horvath LE, Yordan E, Malhotra D, Leyva I, Bortel K, Schalk D, et al. Multidisciplinary care in the oncology setting: historical perspective and data from lung and gynecology multidisciplinary clinics. J Oncol Pract. 2010;6(6):e21–6.

    Article  Google Scholar 

  20. 20.

    Onukwugha E, Petrelli NJ, Castro KM, Gardner JF, Jayasekera J, Goloubeva O, et al. ReCAP: impact of multidisciplinary care on processes of cancer care: a multi-institutional study. J Oncol Pract. 2016;12(2):155–6 e7-68.

    Article  Google Scholar 

  21. 21.

    McNair AG, Choh CT, Metcalfe C, Littlejohns D, Barham CP, Hollowood A, et al. Maximising recruitment into randomised controlled trials: the role of multidisciplinary cancer teams. Eur J Cancer. 2008;44(17):2623–6.

    CAS  Article  Google Scholar 

  22. 22.

    Shore ND, Dinney C, Uzzo R. The Society of Urologic Oncology Clinical Trials Consortium: a brief history and overview. Rev Urol. 2017;19(2):122–4.

    PubMed  PubMed Central  Google Scholar 

  23. 23.

    McCray AT, Ide NC. Design and implementation of a national clinical trials registry. J Am Med Inform Assoc. 2000;7(3):313–23.

    CAS  Article  Google Scholar 

  24. 24.

    McCray AT. Better access to information about clinical trials. Ann Intern Med. 2000;133(8):609–14.

    CAS  Article  Google Scholar 

  25. 25.

    Gafford JA, Gurley-Calvez T, Krebill H, Lai SM, Christiadi, Doolittle GC. Expanding local cancer clinical trial options: analysis of the economic impact of the Midwest Cancer Alliance in Kansas. Acad Med. 2017;92(9):1274–9.

    Article  Google Scholar 

  26. 26.

    Ellis SD, Geana M, Mackay CB, Moon DJ, Gills J, Zganjar A, et al. Science in the Heartland: exploring determinants of offering cancer clinical trials in rural-serving community urology practices. Urol Oncol. 2019.

  27. 27.

    Ellis SD, Geana M, Mackay C, Moon DJ, Gills J, Zganjar A, et al. Science in the Heartland: exploring determinants of offering cancer clinical trials in rural-serving community urology practices. Under review.

  28. 28.

    American Cancer Society Cancer Action Network. Overcoming barriers to patient enrollment in therapeutic clinical trials for cancer: recommendations. Washington, D.C.: American Cancer Society; 2018.

    Google Scholar 

  29. 29.

    Lara PN Jr, Higdon R, Lim N, Kwan K, Tanaka M, Lau DH, et al. Prospective evaluation of cancer clinical trial accrual patterns: identifying potential barriers to enrollment. J Clin Oncol. 2001;19(6):1728–33.

    Article  Google Scholar 

  30. 30.

    Minasian LM, O’Mara AM. Accrual to clinical trials: let’s look at the physicians. J Natl Cancer Inst. 2011;103(5):357–8.

    Article  Google Scholar 

  31. 31.

    Carpenter WR, Fortune-Greeley AK, Zullig LL, Lee SY, Weiner BJ. Sustainability and performance of the National Cancer Institute’s Community Clinical Oncology Program. Contemp Clin Trials. 2012;33(1):46–54.

    Article  Google Scholar 

  32. 32.

    Eggly S, Albrecht TL, Harper FW, Foster T, Franks MM, Ruckdeschel JC. Oncologists’ recommendations of clinical trial participation to patients. Patient Educ Couns. 2008;70(1):143–8.

    Article  Google Scholar 

  33. 33.

    Kaas R, Hart AA, Rutgers EJ. The impact of the physician on the accrual to randomized clinical trials in patients with primary operable breast cancer. Breast. 2005;14(4):310–6.

    CAS  Article  Google Scholar 

  34. 34.

    Klabunde CN, Keating NL, Potosky AL, Ambs A, He Y, Hornbrook MC, et al. A population-based assessment of specialty physician involvement in cancer clinical trials. J Natl Cancer Inst. 2011;103(5):384–97.

    Article  Google Scholar 

  35. 35.

    Baer AR, Michaels M, Good MJ, Schapira L. Engaging referring physicians in the clinical trial process. J Oncol Pract. 2012;8(1):e8–e10.

    Article  Google Scholar 

  36. 36.

    Kaplan CP, Napoles AM, Dohan D, Hwang ES, Melisko M, Nickleach D, et al. Clinical trial discussion, referral, and recruitment: physician, patient, and system factors. Cancer Cause Control. 2013;24(5):979–88.

    Article  Google Scholar 

  37. 37.

    Cane J, O'Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7:37.

    Article  Google Scholar 

  38. 38.

    Dearing JW, Kreuter MW. Designing for diffusion: how can we increase uptake of cancer communication innovations? Patient Educ Couns. 2010;81(Suppl):S100–10.

    Article  Google Scholar 

  39. 39.

    Brownson RC, Jacobs JA, Tabak RG, Hoehner CM, Stamatakis KA. Designing for dissemination among public health researchers: findings from a national survey in the United States. Am J Public Health. 2013;103(9):1693–9.

    Article  Google Scholar 

  40. 40.

    Medicine UoCDoF. Design for Dissemination Tool. Denver: University of Colorado; 2017.

    Google Scholar 

  41. 41.

    Michie S, van Stralen MM, West R. The Behaviour Change Wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6:42.

    Article  Google Scholar 

  42. 42.

    Michie S, Atkins L, West R. The Behaviour Change Wheel: a guide to designing interventions. Great Britain: Silverback Publishing; 2014.

    Google Scholar 

  43. 43.

    Gould GS, Bar-Zeev Y, Bovill M, Atkins L, Gruppetta M, Clarke MJ, et al. Designing an implementation intervention with the Behaviour Change Wheel for health provider smoking cessation care for Australian Indigenous pregnant women. Implement Sci. 2017;12(1):114.

    Article  Google Scholar 

  44. 44.

    Ellis SD, Thrasher JB, Jones EV, Kimminau KS. Treatment decision making in low-risk prostate cancer: recruitment and data collection feasibility. Sociey for Medical Decision Making Annual Meeting; October, 2015. St. Louis: Society for Annual Decision Making; 2015.

    Google Scholar 

  45. 45.

    Ellis SD, Kimminau K, Jones E, Thrasher JB. De-implementing curative therapy in low-risk prostate cancer: preliminary evidence of Urologists’ unlearning. Vancouver, British Columbia: Society for Medical Decision Making Annual Meeting; October 24–26, 2016; 2016.

    Google Scholar 

  46. 46.

    Ellis SD, Jones EV, Kimminau K, Petty L, Thrasher JB. Persistent barriers to adoption of active surveillance. Chicago: American Society for Clinical Oncology; 2016.

    Google Scholar 

  47. 47.

    Ellis SD, Geana M, Mackay C, Gills J, Griebling T, Zganjar A, et al. When an intervention is not designed for dissemination: developing a strategy to improve system-innovation fit. Arlington: 10th Annual Conference on the Science of Dissemination and Implementation; December 4–6, 2017; 2017.

    Google Scholar 

  48. 48.

    Brooks JV, Ellis SD, Morrow E, Kimminau KS, Thrasher JB. Patient factors that influence how physicians discuss active surveillance with low-risk prostate cancer patients: a qualitative study. Am J Mens Health. 2018;12(5):1719–27.

    Article  Google Scholar 

  49. 49.

    Weiner BJ, Lewis CL, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Under Review.

  50. 50.

    Logan JK, Tang C, Liao Z, Lee JJ, Heymach JV, Swisher SG, et al. Analysis of factors affecting successful clinical trial enrollment in the context of three prospective, randomized, controlled trials. Int J Radiat Oncol Biol Phys. 2017;97(4):770–7.

    Article  Google Scholar 

  51. 51.

    Geana M, Erba J. Accrual to Cancer Clinical Trials Project. KU Cancer Center Seminar. Kansas City: KU Cancer Center; 2016.

    Google Scholar 

  52. 52.

    Hamel LM, Penner LA, Albrecht TL, Heath E, Gwede CK, Eggly S. Barriers to clinical trial enrollment in racial and ethnic minority patients with cancer. Cancer Control. 2016;23(4):327–37.

    Article  Google Scholar 

  53. 53.

    Somkin CP, Ackerson L, Husson G, Gomez V, Kolevska T, Goldstein D, et al. Effect of medical oncologists’ attitudes on accrual to clinical trials in a community setting. J Oncol Pract. 2013;9(6):e275–83.

    Article  Google Scholar 

  54. 54.

    Leitch AM, Beitsch PD, McCall LM, Posther K, Newman LA, Herndon JE 2nd, et al. Patterns of participation and successful patient recruitment to American College of Surgeons Oncology Group Z0010, a phase II trial for patients with early-stage breast cancer. Am J Surg. 2005;190(4):539–42.

    Article  Google Scholar 

  55. 55.

    Glasgow RE, Harden SM, Gaglio B, Rabin B, Smith ML, Porter GC, et al. RE-AIM Planning and Evaluation Framework: adapting to new science and practice with a 20-year review. Front Public Health. 2019;7:64.

    Article  Google Scholar 

  56. 56.

    Lee SJC, Murphy CC, Geiger AM, Gerber DE, Cox JV, Nair R, et al. Conceptual model for accrual to cancer clinical trials. J Clin Oncol. 2019; JCO1900101.

  57. 57.

    Ersek JL, Black LJ, Thompson MA, Kim ES. Implementing precision medicine programs and clinical trials in the community-based oncology practice: barriers and best practices. Am Soc Clin Oncol Educ Book. 2018;38:188–96.

    Article  Google Scholar 

  58. 58.

    Unger JM, Moseley A, Symington B, Chavez-MacGregor M, Ramsey SD, Hershman DL. Geographic Distribution and Survival Outcomes for Rural Patients With Cancer Treated in Clinical Trials. JAMA Netw Open. 2018:1(4):e181235.

    Article  Google Scholar 

  59. 59.

    Carlos RC, Sicks JD, Chang GJ, Lyss AP, Stewart TL, Sung L, et al. Capacity for cancer care delivery research in National Cancer Institute Community Oncology Research Program Community Practices: availability of radiology and primary care research partners. J Am Coll Radiol. 2017;14(12):1530–7.

    Article  Google Scholar 

  60. 60.

    Urology Care Foundation. In: Association AU, editor. Clinical Trials Fact Sheet. Linthicum: American Urological Association; 2018.

    Google Scholar 

  61. 61.

    Blanke CD. In: Network SCR, editor. The Front Line. Seattle: SWOG Cancer Research Network; 2018.

  62. 62.

    Nipp RD, Hong K, Paskett ED. Overcoming barriers to clinical trial enrollment. Am Soc Clin Oncol Educ Book. 2019;39:105–14.

    Article  Google Scholar 

  63. 63.

    Geana M, Erba J, Krebill H, Doolittle G, Madhusudhana S, Qasem A, et al. Searching for cures: inner-city and rural patients’ awareness and perceptions of cancer clinical trials. Contemp Clin Trials Commun. 2017;5:72–9.

    Article  Google Scholar 

  64. 64.

    Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017;12(1):108.

    Article  Google Scholar 

  65. 65.

    Kaluzny AD, Lacey LM, Warnecke R, Morrissey JP, Sondik EJ, Ford L. Accrual of patients to randomized clinical trials. Factors affecting cancer prevention and control research. Int J Technol Assess Health Care. 1994;10(3):506–16.

    CAS  Article  Google Scholar 

  66. 66.

    Waltz TJ, Powell BJ, Fernandez ME, Abadie B, Damschroder LJ. Choosing implementation strategies to address contextual barriers: diversity in recommendations and future directions. Implement Sci. 2019;14(1):42.

    Article  Google Scholar 

  67. 67.

    Kitterman DR, Cheng SK, Dilts DM, Orwoll ES. The prevalence and economic impact of low-enrolling clinical studies at an academic medical center. Acad Med. 2011;86(11):1360–6.

    Article  Google Scholar 

Download references


The authors thank the urologists and staff who participated in training and evaluation workshops. We also thank the organizers of the urological professional society meetings who allowed us to present the workshops: Oklahoma Urological Association, Kansas Urological Society, Missouri Urological Association, South Central Section (SCS) of the AUA. We also thank the Gulf South NCI Community Oncology Research Program who hosted a meeting of urologists in the New Orleans, LA area.


Support for the study is provided through an unrestricted grant from the Brown Family Foundation and a fellowship from the National Cancer Institute Mentored Training in Dissemination and Implementation Research in Cancer (R25 CA171994). The article-processing charges related to the publication of this article were partially supported by The University of Kansas (KU) One University Open Access Author Fund sponsored jointly by the KU Provost, KU Vice Chancellor for Research and Graduate Studies, and KUMC Vice Chancellor for Research and managed jointly by the Libraries at the Medical Center and KU – Lawrence.

Author information




SE is responsible for the design, conduct, analysis and interpretation of the study and was a major contributor to manuscript development. MG analyzed and interpreted data. CM contributed to the design of the study and interpretation of results. AS assisted with data collection and study execution. JG, AZ, KS, CM and TG participated in study design and interpretation of results. BT contributed to design, conduct and interpretation of study results. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Shellie Ellis.

Ethics declarations

Ethics approval and consent to participate

The study protocol, including waiver of consent documentation, was approved by the University of Kansas Medical Center Institutional Review Board.

Consent for publication


Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:

LEARN|INFORM|RECRUIT Appeal Survey. (DOCX 17 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ellis, S., Geana, M., Griebling, T. et al. Development, acceptability, appropriateness and appeal of a cancer clinical trials implementation intervention for rural- and minority-serving urology practices. Trials 20, 578 (2019).

Download citation


  • Implementation science
  • Design for dissemination
  • Health care delivery
  • Clinical practice guidelines
  • Specialty care
  • Cancer clinical trials
  • Acceptability
  • Innovation adoption
  • Urology
  • Rural health disparities