Recruitment of men to a multi-centre diabetes prevention trial: an evaluation of traditional and online promotional strategies
Trials volume 20, Article number: 366 (2019)
Effective interventions are required to prevent the current rapid increase in the prevalence of Type 2 diabetes. Clinical trials of large-scale interventions to prevent Type 2 diabetes are essential but recruitment is challenging and expensive, and there are limited data regarding the most cost-effective and efficient approaches to recruitment. This paper aims to evaluate the cost and effectiveness of a range of promotional strategies used to recruit men to a large Type 2 diabetes prevention trial.
An observational study was conducted nested within the Testosterone for the Prevention of Type 2 Diabetes (T4DM) study, a large, multi-centre randomised controlled trial (RCT) of testosterone treatment for the prevention of Type 2 diabetes in men aged 50–74 years at high risk of developing diabetes. Study participation was promoted via mainstream media—television, newspaper and radio; direct marketing using mass mail-outs, publicly displayed posters and attendance at local events; digital platforms, including Facebook and Google; and online promotions by community organisations and businesses. For each strategy, the resulting number of participants and the direct cost involved were recorded. The staff effort required for each strategy was estimated based on feedback from staff.
Of 19,022 men screened for the study, 1007 (5%) were enrolled. The most effective recruitment strategies were targeted radio advertising (accounting for 42% of participants), television news coverage (20%) and mass mail-outs (17%). Other strategies, including radio news, publicly displayed posters, attendance at local events, newspaper advertising, online promotions and Google and Facebook advertising, each accounted for no more than 4% of enrolled participants. Recruitment promotions cost an average of AU$594 per randomised participant. The most cost-effective paid strategy was mass mail-outs by a government health agency (AU$745 per participant). Other paid strategies were more expensive: mail-out by general practitioners (GPs) (AU$1104 per participant), radio advertising (AU$1081) and newspaper advertising (AU$1941).
Radio advertising, television news coverage and mass mail-outs by a government health agency were the most effective recruitment strategies. Close monitoring of recruitment outcomes and ongoing enhancement of recruitment activities played a central role in recruitment to this RCT.
ANZCTR, ID: ACTRN12612000287831. Registered on 12 March 2012.
Worldwide, an estimated 1 in 11 adults has diabetes, and Type 2 diabetes accounts for 90% of these cases . Research to identify effective interventions to prevent diabetes is urgently needed to address this global problem. However, recruitment to disease prevention trials, including diabetes prevention trials, can be challenging. Firstly, since participants in disease prevention trials tend to be healthy and asymptomatic, clinicians may not be able to identify eligible patients through their clinics . Secondly, potential participants may not perceive benefit in participating in disease prevention research, particularly if they do not believe that they are at risk of the disease [3, 4]. Lack of perceived benefit may contribute to lower rates of consent, requiring larger numbers of people to be screened . Thirdly, screening numbers must be large in prevention trials if a modest effect size is hypothesised to ensure adequate power.
To overcome the challenge of recruiting sufficient numbers of participants, previous diabetes prevention trials have reported promoting study participation through: media coverage [6,7,8,9], advertising [7,8,9,10,11], mass mailings [7, 9, 12], referrals from physicians or clinics [6, 7, 9, 10, 12], community-based initiatives [6, 7, 10, 12, 13] and public screening events [7,8,9, 12]. Evaluations of these same recruitment strategies have also been reported in other research areas, including lifestyle improvement interventions , smoking cessation  and treatment of benign prostatic hyperplasia [16, 17]. While the existing literature provides useful recruitment guidance, papers have often lacked sufficient detail on how strategies were implemented and delivered, and how much they cost , making replication difficult . Evaluations of approaches to promote randomised controlled trial (RCT) participation to members of the public is an area of research need, identified as one of the top ten areas for recruitment methodology research in a recent priority-setting study .
Recently, online recruitment through Facebook and Google advertising has been reported to be both affordable and effective in recruiting participants to survey research  and trials of short duration involving web-based interventions [22, 23]. Online advertising has some advantages over more traditional promotional strategies as it is faster to implement, easier to monitor, has lower start-up costs and can potentially reach larger numbers of people quickly . However, to date, most evaluations of online strategies to recruit to RCTs have focussed on recruiting younger people [24,25,26]. Furthermore, evidence on the effectiveness of online strategies in the recruitment of men is mixed. Two studies found online promotions less effective in recruiting men compared to women [27, 28], but one found no significant difference in gender balance between online and traditional approaches . More evidence is needed to assess whether online recruitment strategies are effective in recruiting middle-aged and older men to RCTs .
The aim of this study was to describe and evaluate the strategies used to promote recruitment to the T4DM diabetes prevention RCT.
This observational study of recruitment strategies was set within the Testosterone for the Prevention of Type 2 Diabetes (T4DM) trial (ACTRN12612000287831). The design of the T4DM trial has been published separately . Briefly, T4DM is a large, multi-centre, phase-III, double-blind, placebo-controlled, 2-year trial of testosterone therapy combined with a lifestyle intervention (Weight Watchers®) compared to the lifestyle intervention alone for the prevention of Type 2 diabetes. The trial is running through six centres in Australian capital cities and recruitment occurred from January 2013 to February 2017. The trial enrolled men aged 50–74 years who were overweight or obese (≥ 95 cm waist circumference), had pre-diabetes or newly diagnosed Type 2 diabetes, and testosterone level ≤ 14.0 nmol/L. The trial is ongoing and follow-up is due to be completed in May 2019.
The T4DM trial design presented a number of recruitment challenges. Firstly, very few participants could be referred to the study by investigators at the participating centres since prospective participants were unlikely to be under the care of an endocrinologist. We therefore planned to seek prospective participants directly from the community, predicting that only one in four men screened in this non-targeted way would be eligible based on the entry criteria of elevated blood glucose and low serum testosterone. Secondly, we predicted that the placebo-controlled and injectable nature of the study treatment, as well as its 2-year duration, might limit the number of men willing to participate. Taking these two factors into account, we estimated that approximately 20,000 men would need to be screened in order to reach the final recruitment target of 1000 participants.
Screening and enrolment process
Men who heard about the study through the promotional strategies to be described in this paper were invited to complete a pre-screening questionnaire online (on the T4DM study website, www.diabetesprevention.org.au), or over the telephone (by calling the central coordinating centre). Men who were eligible on the pre-screening questionnaire were emailed or posted a pre-screening patient information and consent form, instructions and a request form to attend for screening blood tests to be conducted at one of a large number of contracted pathology collection centres from one national commercial pathology provider company. Participants who were eligible on the screening blood tests were then contacted by their preferred centre to arrange a screening clinic visit. Eligible and consenting participants were enrolled and randomised at the clinic.
Recruitment oversight and planning
During the recruitment phase, the Steering Committee met monthly by telephone to oversee the recruitment plan and monitor the ongoing performance of recruitment strategies. The Human Review Ethics Committees approved the study’s recruitment strategies and promotional material. Where promotional activities involved real-time communication with the public; for example, through Facebook posts, the approach to be taken with these communications, and the subject matter to be covered, received ethical review and approval.
Development of the recruitment plan involved four key considerations based on marketing principles : selection of the target audience, definition of the call to action, design of the promotional material and selection of promotional strategies to be used (Table 1).
Recruitment strategies were coordinated centrally by the study project manager (KB). In general, nationwide strategies were implemented by staff at the central coordinating centre, while local and community strategies were implemented by site study nurses and investigators. Each strategy is described in turn below. Strategy descriptions were guided by the Template for Intervention Description and Replication (TIDieR) Checklist, which lists the items to be reported when describing interventions to support replication .
Radio advertising involved 30-second paid advertisements on 20 different radio stations (nine talkback stations and 11 music stations). Three different scripts were recorded over the course of the study in order to keep the message fresh (see Additional file 1). Advertisements ran from January 2014 to July 2016, but were not run continuously on all stations over this period. Instead, advertisements ran in campaigns of 3–4 weeks’ duration, with stations running between one and seven campaigns over the course of study recruitment. In total 68 campaigns were run, 45 on talkback stations and 23 on music stations, and advertisements were played in a total of 7110 paid spots. In addition to paid spots, stations offered bonus filler spots free of charge. In some cases these were more frequent than the paid spots and so the total number of times that study radio advertisements were run is likely to be in the range of 10,000–15,000 times.
In Australia, radio stations are generally broadcast within a single state so advertisements were booked separately for each of the five states where the study sites were located. Radio stations were selected based on advertising costs and listener demographics. To inform selection of advertising times, we sourced listener demographics (including age and gender) by time of day for each radio station. Generally, men aged 50 to 74 years were most likely to listen in the early mornings and late afternoons on weekdays, although these were also the most expensive times to advertise. A single campaign was booked on selected stations and the number of participants screened and enrolled, as well as the cost per participant screened and enrolled, were measured. Campaigns on stations with a cost of less than AU$50 per participant screened were generally repeated. Modifications were made to the time of day that advertising was played based on the performance of previous campaigns and on advice from radio station advertising personnel.
Mass mail-outs involved posting a study invitation package to men on the Medicare database by the Australian Government Department of Human Services (DHS). The Medicare database, the infrastructure underpinning the national health scheme, includes Australian residents who are eligible for public healthcare, generally those who are Australian or New Zealand citizens, or have permanent Australian residency status. Mailings were conducted in July 2016 (40,000 invitations sent), September 2016 (60,000 invitations sent) and November 2016 (30,000 invitations sent), with 130,000 men in total being mailed once each. Mailing recipients were randomly selected from the Medicare database based on being male, aged 50–74 years and living within close proximity of one of the study sites (the initial mailing included a sample of men living within a 20-km radius of a study site and the subsequent mailings were further restricted to men living within 5–10 km of a study site). Men who had been prescribed testosterone or anti-diabetic therapies within the previous 12 months were excluded from the mailing list by linking to the Pharmaceutical Benefits Scheme database. The invitation package consisted of a cover letter from the DHS, an invitation letter from the study Chair and a study postcard (see Additional file 1). The mailing was conducted by a third party mailing house contracted by the DHS. The contact details of mailing recipients were kept confidential and were not shared with the study coordinating centre.
In addition, a one-time mail-out by a single network of general practices (GPs) to a targeted group of their patients was conducted from February to March 2013 in one city. Though the number of letters sent in this GP mail-out was not recorded it is estimated to be less than 500 letters.
It is possible but unlikely that men who received a letter in the GP mail-out in early 2013 also later received a letter from the DHS in 2016. The responses to the small GP mail-out and the later and much larger DHS mail-out were recorded and reported separately.
Television, radio and newspaper news coverage
In the period January 2013 to June 2016, approximately 15 press releases and approaches to media were made. The study chose not to engage a public relations firm due to cost concerns. Instead, press releases were facilitated by site investigators and prepared by University and Hospital media offices and were distributed to local, state and national television, radio and newspaper news organisations. Press releases highlighted newsworthy aspects of the study and provided quotes from study investigators and study participants. Media office contact details were provided so that journalists could arrange interviews with investigators and participants. Over the recruitment period, nine newspaper stories, eight television stories and seven radio news stories were broadcast.
Facebook promotions and advertising
The study Facebook page was set up by the central coordinating centre in May 2013. Over the period May 2013 to December 2016, 94 stories were posted to the study Facebook page. Stories covered a mixture of topics including information about the study and how to join, links to news stories about the study, men’s health information and general interest stories. In addition, 23 advertising campaigns and paid boosted posts were run intermittently in the period October 2013 to December 2016 (see Additional file 2 for definitions of common Facebook advertising terms). Advertisements focussed on inviting men to join the study. By contrast, boosted stories tended to promote news stories relating to the T4DM study. Advertisements and paid boosted stories targeted men aged 50 years or older living in a capital city with a T4DM study site. Examples of advertisements and posts can be found in Additional file 1. The Facebook Ads Manager application allowed advertising performance to be monitored in real time by reporting the number of impressions, number of clicks, cost per click and cost per 1000 impressions of each campaign. While paid campaigns were running, the coordinating centre reviewed performance statistics daily and increased or decreased the advertising spend according to the success of the advertisement. Comments from participants and from the public were used to refine Facebook page content over time.
Indirect Facebook promotions were also used. When participants completed the online screening questionnaire they were invited to share information about the study on their Facebook page. Local organisations and businesses, such as sporting clubs, social clubs and healthcare providers, were also approached to share information about the study on their own Facebook pages. We were unable to determine how many people and organisations shared information about the study on their Facebook pages.
Paid Google advertising was set up by the central coordinating centre using the Google AdWords application (see Additional file 2 for definitions of common Google advertising terms). The purpose of these advertisements was to display a link to the study website at the top of the Google search results screen when members of the public googled terms which indicated that they might be interested in joining the study. We identified four possible Google search themes: diabetes prevention, low testosterone, weight loss and nocturia (night-time urination). However, after further investigation the low-testosterone theme was rejected due to Google advertising rules and the weight-loss theme was rejected due to the high levels of competition and hence high cost. For the two remaining themes (diabetes prevention and nocturia), Google AdWords provided a list of the most commonly used related search terms, known as keywords, which we used to build our advertising campaigns (see Additional file 1). Advertising ran from July 2013 to October 2014 and October to December 2015. All advertisements were targeted to users within Australia only.
Later in the recruitment period, additional Google advertisements were run to ensure that people who searched for the T4DM study name were able to locate the website easily. Since the purpose of these advertisements was to facilitate screening of potential participants who already knew about the study rather than to promote the study to the public, these advertisements and their associated costs have not been included in this paper.
In December 2013, one paid advertisement was placed in a Sunday newspaper with a circulation of 250,000 in one capital city. If effective, we planned to roll out newspaper advertising to other cities.
Community outreach activities
Throughout the recruitment period, site staff, and to a lesser extent, central coordinating centre staff, conducted a range of community outreach activities. These included: (1) displaying posters in local businesses, organisations, libraries and hospitals; (2) attendance at local community and health service events and (3) approaching local businesses and organisations to promote the study to their customers, members and employees. Organisations who agreed to support the study included men’s community and recreational groups, private school old boys’ associations, private health insurance companies, trade unions, government workplaces, diabetes groups and Weight Watchers ®. These organisations supported the study through a variety of means including placing information about the study in their print newsletters, email newsletters, on their websites, on Facebook pages and on notice boards. The majority of community outreach activities occurred in the first year of study recruitment (2013), but continued sporadically throughout recruitment.
In addition to these unpaid promotions, one paid promotion through a professional football club based near one study site was trialled for 1 week in June 2016. The study was featured in the club’s weekly email newsletter to its 9300 members as well as in banner and gutter ads on the club’s website.
Healthcare provider referrals and promotions
Throughout the recruitment period we approached local general practitioners (GPs) and pathology collection centres to support study recruitment. The central coordinating centre wrote to 1024 GPs in the areas surrounding study sites to ask them to refer suitable patients to the study and to display a study poster in their waiting room areas. Site staff also attended local GP meetings to inform them about the study. We asked pathology companies to display posters in their waiting rooms and to print information about the study on the bottom of the reports of men who might be eligible for the study based on their blood test results.
Recruitment strategy monitoring and enhancement
The recruitment management process involved repeated cycles of strategy implementation, monitoring and enhancement. The number of participants enrolled as a result of each strategy, the direct costs and the staff effort involved were monitored in real time and reported to the Steering Committee on a monthly basis. The Committee identified the number of participants enrolled as the primary means for assessing strategy effectiveness but also considered cost-effectiveness, staff effort, potential to reach large numbers of men or to be targeted to men who were most likely to be eligible.
Outcomes and data analysis
Attributes which were thought to impact recruitment results were described for each strategy: content format (text, image, audio, audio-visual), content length (short, medium, long), approach to prospective participants (direct, indirect), level of targeting (ability to reach members of the public who were most likely to be eligible for the study in terms of age, location and health), potential reach (the number of people who would see the strategy), frequency of exposure, and whether or not the strategy included an online component.
Strategy exposure and contribution
Where possible, we recorded the number of people exposed to each strategy. In addition, all men completing the pre-screening questionnaire were asked to report how they heard about the study. This information was linked to the participant’s screening and enrolment status by a unique participant identifier to estimate the contribution of the strategy (the percentage of all screened and randomised participants contributed by each strategy) and to estimate how many participants heard about the study through online and traditional sources.
The direct cost of implementing each recruitment strategy was recorded. The direct cost did not include staffing costs or the cost of conducting screening and enrolment activities. Costs were recorded in Australian dollars and were adjusted for inflation using the Australian Consumer Price Index . All costs in this paper are expressed in June 2018 terms.
We determined that measuring the indirect cost of each strategy would not be feasible. Instead, we collected detailed feedback from recruitment staff (at the central coordinating centre and at study sites) in order to estimate the staff effort involved in implementing each strategy (categorised as low, moderate or high per participant enrolled).
Overall strategy appraisal
For each strategy, the number of participants randomised, the direct cost per participant, and the staff effort per participant were estimated and each scored 0 (lowest) to 3 (highest). Since the number of participants randomised was identified as the most important outcome, this was the primary means of assessing the effectiveness of each strategy (highly effective, effective, moderately effective, limited effectiveness, ineffective). However, the direct cost per enrolment, level of staff effort required per enrolment, and the strategy’s attributes were also considered to come to a final subjective appraisal of each strategy.
Data analysis was conducted in SAS v 9.4 (Cary, NC, USA). Simple descriptive statistics were used to describe the number of participants screened and randomised. Differences between groups in enrolment rates and in the proportion aged 60 years or older were tested using chi-square analyses with a significance level of 5%.
Overall study recruitment
During the recruitment period (January 2013 to February 2017) 19,022 men were screened and 1007 were randomised to the T4DM trial. The number of men screened per month fluctuated over the recruitment period (Fig. 1), most likely influenced by the mix of promotional activities occurring at the time. Spikes in the number of men screened were observed when the trial was featured in media news stories and when radio advertising campaigns and mass mail-outs were being conducted.
Evaluation of promotional activities
Number of participants recruited
Table 2 shows the number of men screened and randomised as a result of each promotional activity. Almost 80% of participants heard of the study through one of three methods; radio advertising (42% of participants), television news coverage (20%) or mass mail-outs (17%). No other single strategy contributed more than 4% of all enrolments. Excluding strategies that resulted in fewer than ten randomisations, the randomisation rate (percentage of screened participants who went on to be randomised) did not differ between strategies (p = 0.31). The average randomisation rate was 5%.
The response rate to the mass mail-out by the DHS was 2.5% (3211 men screened from 130,000 letters sent); 173 of these men went on to be randomised (0.1% of all men mailed). It was not possible to calculate the response rate for other recruitment strategies; for example, radio advertising and news stories, since the denominator number of men exposed to these strategies was unknown.
A total of AU$598,633 was spent on promotional activities at an average cost of AU$31 per participant screened and AU$594 per participant randomised. The total direct cost and cost per participant for each strategy are shown in Table 3. The cost of individual strategies ranged from no cost (free media news coverage and word of mouth) to AU$312 per screened participant for online promotion of the study by a football club. Of the paid strategies, mass mail-out by the DHS was the most cost-effective (AU$40 per screening and AU$745 per randomisation).
Staff time and effort
The task of organising, conducting and monitoring promotional activities took an estimated average of 20 person-hours per week over the 4-year recruitment period. The work was divided between seven study team members (one project manager at the central coordinating centre and six site-based study nurses) and fluctuated throughout the recruitment period. The staff effort required for each strategy, proportional to the number of participants enrolled, is estimated in Table 4. In general, paid strategies, such as advertising and mass mail-outs, required the least staff effort, while low-cost and unpaid strategies, such as community activities, required the most staff effort.
Overall strategy appraisal
Table 4 describes each strategy’s attributes, its ratings for the three key outcomes (number of participants enrolled, direct cost and staff effort), and a subjective appraisal of the advantages and disadvantages of the strategy. While radio advertising, television news coverage and mass mail-outs were identified as the most effective recruitment strategies, each of these strategies had at least one disadvantage in terms of cost, frequency or format.
Response to online recruitment strategies
Eight hundred and thirty-one people liked the study Facebook page, but engagement with content posted on the Facebook page was generally low. Unpaid posts usually received less than five likes (mostly from study staff) and few, if any, comments. Facebook-paid advertisements and boosted posts cost AU$10,029 and received 2,473,966 impressions, resulting in 21,477 clicks or other engagements. The average cost per click was AU$0.47 and the average cost per 1000 impressions was AU$4.05.
The results of the Google advertising campaigns are shown in Table 5. AU$1931 was spent on Google advertising to promote study participation, resulting in advertisements being displayed 57,202 times and clicked on 5939 times. The average click-through rate was 10% and the average cost per click was AU$0.33. We did not record how many of the people who clicked on an advertisement went on to be screened and randomised to the T4DM study.
In total, 1433 participants (8%) of participants reported hearing about the study online. However, this is likely to be an underestimation since some sources had online and offline components; for example, organisations promoted the study by publicly displaying posters and posting information on their websites, and it was, therefore, not always possible to determine whether a participant’s information source was online or offline. In younger participants (aged < 60 years) 7% reported using an online source compared to 8% in older participants (aged ≥ 60 years). The proportion of participants hearing about the study online did not differ by age (p < 0.28).
The three most effective recruitment strategies were: (1) repeated bursts of high-frequency, targeted radio advertising, (2) infrequent but high-reach television news reports (3) direct, mass-mailed invitations from a credible government health agency. Other promotional strategies, including newspaper and radio news coverage, newspaper advertising, publicly displayed posters, attendance at local community events, mentions in email and posted newsletters, and promoting the study online through Facebook, Google and other websites, collectively accounted for less than 20% of all randomisations. These findings are broadly consistent with those reported by other contemporary RCTs recruiting men aged over 50 years [16, 17, 34]. While previous studies reported that community outreach activities, such as displaying posters in the local community and attending community events, were ineffective [16, 17, 34], we achieved moderate success by expanding our community outreach activities to encompass online promotion through organisations’ email newsletters, websites and Facebook pages. While the numbers of resulting participants were small, the fact that these online approaches involved no direct cost and little staff effort made them a worthwhile component of the overall recruitment strategy mix. Unlike previous studies [16, 35, 36], we found that newspaper advertising was not an effective strategy and so this strategy was abandoned after the placement of only a single advertisement. This may be explained by the fall in print newspaper readership over recent years . However, this finding should be interpreted with caution as it was based on the response to a single newspaper advertisement in a single city. Overall, the average promotion cost per randomised participant was AU$594 with mass mailing the most cost-effective of the paid promotional strategies. This cost compared favourably with promotional costs in a large-scale RCT in diabetes prevention , and an RCT in testosterone supplementation in older men , which are estimated to have cost at least AU$2700 per participant in June 2018 terms. It should be noted that these RCTs had different target populations and eligibility criteria which may account in some part for their higher recruitment promotion costs.
Our efforts to recruit participants using Facebook and Google advertising achieved disappointing results. Like others [24, 28, 38], we found such online advertising fast and flexible to implement, and easy to monitor in real time. However, unlike studies recruiting predominantly younger people [21, 25] and women [27, 28], our Facebook advertising campaigns resulted in few enrolments. While large numbers of men aged over 50 years use Facebook , their engagement with the study content on Facebook was low. This was likely to have constrained the reach and impact of the study’s Facebook promotions. It was unclear whether this lack of engagement was due to a deficiency in the content we posted or due to men in this age group’s social media habits more generally . Future research could address this uncertainty by using the randomised split-testing capabilities built into the Facebook advertising interface to evaluate men’s responses to variations in content messaging and images [25, 28]. We also observed a disappointing response to our Google advertising campaign. We hypothesise that this failure was due to the nature of the study question focussing on diabetes prevention. Prospective participants may not have been aware that they were at an increased risk of developing diabetes. We presume they were, therefore, unlikely to search in Google for terms relating to diabetes prevention and pre-diabetes, limiting the reach of our Google advertisement. By contrast, studies that were able to define study-specific search terms, for example, relating to cessation of smokeless tobacco  or depression , reported that Google advertising was an effective and affordable recruitment strategy.
Limitations and areas for future research
Despite being rigorously conducted, the analyses presented in this paper are based on observational data. The results may be confounded by differences in the timing and target location of recruitment strategies, which we were not able control for in the analyses. Furthermore, the individual promotional strategies, by their nature, involved differences in the form, length and style of content. It is possible that differences in the observed responses to strategies were due, in some part, to these content differences rather than the promotional strategies themselves. The results observed in this study may not be generalisable to other RCTs due to the possible impact of differences in disease area, target population, study design and location.
Another limitation of this study was the difficulty in accurately measuring the contribution of each strategy to enrolment. This difficulty was two-fold. Firstly, some participants supplied insufficient information to pinpoint a specific recruitment strategy. For example, if the source of information was reported as ‘GP’ then it was unclear whether the participant was referred to the study by their GP, or whether they saw a study recruitment poster in the GP’s waiting room. Since we knew where and when particular strategies were being conducted, we cross-referenced the participant’s location and date of screening to resolve these uncertainties wherever possible. Secondly, for practical reasons we coded only a single recruitment source for each participant. However, it was evident from the optional, free-text responses provided by some participants, as well as from speaking directly to study participants, that some participants heard about the study through multiple sources. In such cases, the marginal contribution of these multiple strategies could not be measured, possibly influencing the estimates of effectiveness. To address these challenges, future studies could adapt digital marketing techniques, such as custom Universal Resource Locator (URL) tracking and Google Analytics goals, to more accurately track the sources of participant recruitment. Future researchers could also conduct participant interviews early in the recruitment phase to understand the media habits and preferences of the target population and the possible impact of multiple promotional sources. Such marketing activities may require specialised skills and additional resources. Trial recruitment managers face the challenge of straddling the divide between the methodological rigour of clinical trial research and the ‘move fast and break things’ culture of digital marketing . The best approach to combining these divergent paradigms is still to be determined, particularly given the ethical standards and oversight required in RCT recruitment .
A trial-and-error approach to optimising recruitment promotions is likely to increase recruitment costs and result in recruitment delays, yet, in the past, trial recruitment managers had little other choice due to the lack of reliable evidence. This observational evaluation presents an approach for selecting, implementing, monitoring and enhancing recruitment promotional activities. We hope that future trials can adapt and improve on this approach to meet their recruitment targets.
The most effective strategies to recruit men aged 50–74 years to the T4DM diabetes prevention RCT were repeated bursts of high-frequency radio advertisements supported by occasional television news coverage and mass mail-outs by a government health agency. Close monitoring of recruitment outcomes and ongoing enhancement of recruitment activities played an important role in overcoming the recruitment challenges in this RCT. This paper provides future researchers with estimates of the effectiveness of a range of traditional and online promotional strategies as well as presenting an approach to collecting and analysing promotional strategy recruitment metrics.
Availability of data and materials
The datasets used and analysed during the current study are available from the corresponding author on reasonable request.
Department of Human Services
General practitioner/general practice
Randomised controlled trial
Testosterone for the Prevention of Type 2 Diabetes study
Template for Intervention Description and Replication Checklist
Universal Resource Locator
International Diabetes Federation. IDF Diabetes Atlas. Brussels: International Diabetes Federation; 2017.
Hall MJ, Egleston B, Miller SM, Buzaglo JS, Millard J, Ridgway C, et al. Barriers to participation in cancer prevention clinical trials. Acta Oncol. 2010;49:757–66. https://doi.org/10.3109/0284186X.2010.485209.
Eborall H, Cunningham-burley S, Fowkes G. Gullible guinea pig or gift-giver: public attitudes towards participating in a randomised control trial for cardiovascular disease prevention. Psychol Health. 2004;19:47.
Veit CT. A single mathematical model predicts physicians’ recommendations and postmenopausal women’s decisions to participate in a clinical trial to prevent breast cancer or coronary heart disease. Med Decis Mak. 2004;24:330–50. https://doi.org/10.1177/0272989x04267007.
Cooper CL, Hind D, Duncan R, Walters S, Lartey A, Lee E, et al. A rapid review indicated higher recruitment rates in treatment trials than in prevention trials. J Clin Epidemiol. 2015;68:347–54. https://doi.org/10.1016/j.jclinepi.2014.10.007.
Douglas A, Bhopal RS, Bhopal R, Forbes JF, Gill JM, Lawton J, et al. Recruiting South Asians to a lifestyle intervention trial: experiences and lessons from PODOSA (Prevention of Diabetes & Obesity in South Asians). Trials. 2011;12:220. https://doi.org/10.1186/1745-6215-12-220.
Gerstein HC, Yusuf S, Holman R, Bosch J, Pogue J. Rationale, design and recruitment characteristics of a large, simple international trial of diabetes prevention: the DREAM trial. Diabetologia. 2004;47:1519–27. https://doi.org/10.1007/s00125-004-1485-5.
Ranjani H, Weber MB, Anjana RM, Lakshmi N, Narayan KM, Mohan V. Recruitment challenges in a diabetes prevention trial in a low- and middle-income setting. Diabetes Res Clin Pract. 2015;110:51–9. https://doi.org/10.1016/j.diabres.2015.07.013.
Rubin RR, Fujimoto WY, Marrero DG, Brenneman T, Charleston JB, Edelstein SL, et al. The Diabetes Prevention Program: recruitment methods and results. Control Clin Trials. 2002;23:157–71. https://doi.org/10.1016/s0197-2456(01)00184-2.
Hammersley MS, Meyer LC, Morris RJ, Manley SE, Turner RC, Holman RR. The Fasting Hyperglycaemia Study: I. Subject identification and recruitment for a non-insulin-dependent diabetes prevention trial. Metabolism. 1997;46:44–9.
Lindström J, Louheranta A, Mannelin M, Rastas M, Salminen V, Eriksson J, et al. The Finnish Diabetes Prevention Study (DPS): lifestyle intervention and 3-year results on diet and physical activity. Diabetes Care. 2003;26:3230–6 111111111://care.diabetesjournals.org/content/diacare/26/12/3230.full.pdf.
Blackwell CS, Foster KA, Isom S, Katula JA, Vitolins MZ, Rosenberger EL, et al. Healthy living partnerships to prevent diabetes: recruitment and baseline characteristics. Contemp Clin Trials. 2011;32:40–9. https://doi.org/10.1016/j.cct.2010.10.006.
Ramachandran A, Snehalatha C, Mary S, Mukesh B, Bhaskar AD, Vijay V. The Indian Diabetes Prevention Programme shows that lifestyle modification and metformin prevent type 2 diabetes in Asian Indian subjects with impaired glucose tolerance (IDPP-1). Diabetologia. 2006;49:289–97. https://doi.org/10.1007/s00125-005-0097-z.
Chatters R, Newbould L, Sprange K, Hind D, Mountain G, Shortland K, et al. Recruitment of older adults to three preventative lifestyle improvement studies. Trials. 2018;19:121 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5819193/pdf/13063_2018_Article_2482.pdf.
Galli L, Knight R, Robertson S, Hoile E, Oladapo O, Francis D, et al. Using marketing theory to inform strategies for recruitment: a recruitment optimisation model and the txt2stop experience. Trials. 2014;15:182. https://doi.org/10.1186/1745-6215-15-182.
Kusek JW, Ahrens A, Burrows PK, Clarke HS, Foster HE, Hanson K, et al. Recruitment for a clinical trial of drug treatment for benign prostatic hyperplasia. Urology. 2002;59:63–7 https://ac.els-cdn.com/S0090429501014546/1-s2.0-S0090429501014546-main.pdf?_tid=ac97c94a-3927-4346-8c10-a01df5a9caba&acdnat=1540249344_1e1c437e8e512d89295d6184d169cc74.
Lee JY, Foster HE Jr, McVary KT, Meleth S, Stavris K, Downey J, et al. Recruitment of participants to a clinical trial of botanical therapy for benign prostatic hyperplasia. J Altern Complement Med. 2011;17:469–72.
Gardner HR, Albarqouni L, Black P, Fernie G, Gillies K, Loudon K, et al. Strategies to improve participant recruitment to randomised controlled trials: a systematic review of non-randomised evaluations. https://journals.sagepub.com/doi/full/10.1177/1740774518790846. Accessed 16 June 2019.
Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: Template for Intervention Description and Replication (TIDieR) Checklist and guide. BMJ. 2014;348:g1687. https://doi.org/10.1136/bmj.g1687.
Healy P, Galvin S, Williamson PR, Treweek S, Whiting C, Maeso B, et al. Identifying trial recruitment uncertainties using a James Lind Alliance Priority Setting Partnership—the PRioRiTy (Prioritising Recruitment in Randomised Trials) study. Trials. 2018;19:147. https://doi.org/10.1186/s13063-018-2544-4.
Whitaker C, Stevelink S, Fear N. The use of Facebook in recruiting participants for health research purposes: a systematic review. J Med Internet Res. 2017;19:e290. https://doi.org/10.2196/jmir.7071.
Gordon JS, Akers L, Severson HH, Danaher BG, Boles SM. Successful participant recruitment strategies for an online smokeless tobacco cessation program. Nicotine Tob Res. 2006;8(Suppl 1):S35–41.
Morgan AJ, Jorm AF, Mackinnon AJ. Internet-based recruitment to a depression prevention intervention: lessons from the Mood Memos study. J Med Internet Res. 2013;15:e31. https://doi.org/10.2196/jmir.2262.
Frandsen M, Walters J, Ferguson SG. Exploring the viability of using online social media advertising as a recruitment method for smoking cessation clinical trials. Nicotine Tob Res. 2014;16:247–51. https://doi.org/10.1093/ntr/ntt157.
Ramo DE, Rodriguez TM, Chavez K, Sommer MJ, Prochaska JJ. Facebook recruitment of young adult smokers for a cessation trial: methods, metrics, and lessons learned. Internet Interv. 2014;1:58–64 https://ac.els-cdn.com/S2214782914000104/1-s2.0-S2214782914000104-main.pdf?_tid=81bd24f9-f77c-4977-9a61-fe13ccdf1813&acdnat=1540249371_c8c443dad93385022d85067963b1ad16.
Raviotta JM, Nowalk MP, Lin CJ, Huang HH, Zimmerman RK. Using FacebookTM to recruit college-age men for a human papillomavirus vaccine trial. Am J Mens Health. 2016;10:110–9.
Huesch MD, Mukherjee D, Saunders E. E-recruitment into a bipolar disorder trial using Facebook tailored advertising. Clin Trials. 2018;2018:1740774518784018. https://doi.org/10.1177/1740774518784018.
Juraschek SP, Plante TB, Charleston J, Miller ER, Yeh HC, Appel LJ, et al. Use of online recruitment strategies in a randomized trial of cancer survivors. Clin Trials. 2017:1740774517745829. https://doi.org/10.1177/1740774517745829.
Bracken K, Askie L, Keech AC, Hague W, Wittert G. Recruitment strategies in randomised controlled trials of men aged 50 years and older: a systematic review. BMJ Open. 2019;9:e025580. https://doi.org/10.1136/bmjopen-2018-025580.
Wittert G, Atlantis E, Allan C, Bracken K, Conway A, Daniel M, et al. Testosterone therapy to prevent type 2 diabetes mellitus in at-risk men (T4DM): design and implementation of a double-blind randomized controlled trial. Diabetes Obes Metab. 2019;21:772–80.
Percy L, Elliott RH. Strategic advertising management. 4th ed. Oxford: Oxford University Press; 2016.
Smith JA, Braunack-Mayer AJ, Wittert GA, Warin MJ. Qualities men value when communicating with general practitioners: implications for primary care settings. Med J Aust. 2008;189:618–21.
Consumer Price Index, Australia (June Quarter 2018) Australian Bureau of Statistics. Canberra, Australia 2018. Available from: http://www.abs.gov.au/ausstats/abs@.nsf/mf/6401.0. Accessed 27 July 2018.
Cauley JA, Fluharty L, Ellenberg SS, Gill TM, Ensrud KE, Barrett-Connor E, et al. Recruitment and Screening for the Testosterone Trials. J Gerontol A Biol Sci Med Sci. 2015;70:1105–11. https://doi.org/10.1093/gerona/glv031.
Butt DA, Lock M, Harvey BJ. Effective and cost-effective clinical trial recruitment strategies for postmenopausal women in a community-based, primary care setting. Contemp Clin Trials. 2010;31:447–56. https://doi.org/10.1016/j.cct.2010.06.003.
Garrett SK, Thomas AP, Cicuttini F, Silagy C, Taylor HR, McNeil JJ. Community-based recruitment strategies for a longitudinal interventional study: the VECAT experience. J Clin Epidemiol. 2000;53:541–8 https://ac.els-cdn.com/S0895435699001535/1-s2.0-S0895435699001535-main.pdf?_tid=52da7f59-faa9-4b4e-9a86-c26998e2141c&acdnat=1540249336_15ca2a90e24fa64c1d481caf3915ddab.
Samios Z. ABCs: half yearly circulation audit sees many newspapers suffer 10% declines while Sunday Telegraph falls below 400,000. https://mumbrella.com.au/abcs-sunday-telegraph-falls-below-400000-as-weekend-newspapers-decline-466270. Accessed 27 July 2018.
Arigo D, Pagoto S, Carter-Harris L, Lillie SE, Nebeker C. Using social media for health research: methodological and ethical considerations for recruitment and intervention delivery. Digital Health. 2018;4:2055207618771757. https://doi.org/10.1177/2055207618771757.
Jafrey I. Social Media Matters For Baby Boomers. https://www.forbes.com/sites/forbestechcouncil/2018/03/06/social-media-matters-for-baby-boomers/#1d91771e4425. Accessed 24 September 2018.
Mullin S. High velocity testing: should you really move fast and break things? https://conversionxl.com/blog/high-velocity-testing/. Accessed 19 September 2018.
The coordinating centre team: Caitlin Van Holst Pellekaan and Sandra Healey (NHMRC Clinical Trials Centre).
The T4DM study nurses: Glenda Fraser (ANZAC Research Institute and Concord Hospital), Jenny Healy (Austin Hospital), Helen Daniels and Chyn Soh (Fremantle Hospital and Fiona Stanley Hospital), Jody Sawyer (Princess Alexandra Hospital), Rosemary Cox and Fiona Cossey (The Queen Elizabeth Hospital), Lee Mahoney (The Keogh Institute for Medical Research).
Sherilyn Goldstone (NHMRC Clinical Trials Centre) for her assistance with the preparation of this manuscript.
The T4DM study participants.
National Health and Medical Research Council Project Grant, Bayer, Eli Lilly, University of Adelaide, University of Sydney.
Ethics approval and consent to participate
The following Human Review Ethics Committees approved the study’s recruitment strategies and also, where applicable, reviewed and approved promotional material: Sydney Local Health District HREC – CRGH, Central Adelaide Local Health Network Human Research Ethics Committee, South Metropolitan Health Service Human Research Ethics Committee, Bellberry Human Research Ethics Committee.
Consent for publication
GW has received research funding from Bayer, Lilly, Lawley Pharmaceuticals and Weight Watchers®, and speaker honoraria from Bayer, Lilly and Besins Healthcare.
CA has received honoraria from Besins Healthcare and is an advisory board member for Ferring.
MG has received research funding from Bayer, Novartis, Weight Watchers®, Lilly, and speaker’s honoraria from Besins Healthcare and Otsuka.
DJH has received institutional grants for investigator-initiated studies of testosterone pharmacology (Lawley, Besins Healthcare) but no personal income and has provided expert testimony to anti-doping and professional standards tribunals and testosterone litigation.
BBY has received speaker honoraria and conference support from Bayer, Lilly and Besins Healthcare, research support from Bayer, Lilly and Lawley Pharmaceuticals, and has been a member of advisory committees for Lilly and Besins Healthcare.
All other authors declare no relevant conflicts of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Bracken, K., Hague, W., Keech, A. et al. Recruitment of men to a multi-centre diabetes prevention trial: an evaluation of traditional and online promotional strategies. Trials 20, 366 (2019). https://doi.org/10.1186/s13063-019-3485-2