This article has Open Peer Review reports available.
A review of approaches to improve participation of culturally and linguistically diverse populations in clinical trials
© Hughson et al. 2016
Received: 30 October 2015
Accepted: 7 May 2016
Published: 26 May 2016
The under-representation of culturally and linguistically diverse participants in clinical trials is an ongoing concern for medical researchers and the community. The aim of this review is to examine the complex issue of recruiting culturally and linguistically diverse (CALD) older people to medical research and to examine responses to these issues. The review focuses on (1) trends in the existing literature on barriers to and strategies for recruiting CALD and older people to clinical research, (2) issues with informed consent for CALD populations, and (3) the efficacy of innovative approaches, including approaches incorporating multimedia in research and consent processes. The literature indicates that predominant barriers to greater involvement of CALD patients in clinical trials are communication, including literacy and health literacy considerations; English language competence; and cultural factors in the research setting such as mistrust of consent processes, as well as considerable practical and logistical barriers, including mobility considerations. Some evidence exists that incorporating multimedia resources into the informed consent process can improve patient understanding and is preferred by patients, yet these findings are inconclusive. A multi-methodological approach, including the use of culturally and linguistically sensitive multimedia tools, may help address the issue of low inclusion of CALD groups in clinical research. Researcher education needs to be taken into account to address preconceptions about CALD resistance to research participation and to raise awareness of cultural concerns in regard to research participation.
KeywordsCALD Informed consent Clinical research Older people Migrants Language Health literacy Recruitment Multimedia Low English proficiency
Elderly people of culturally and linguistically diverse (CALD)1 backgrounds are under-represented in clinical research [1–3]. Under-representation of CALD patients is problematic for several reasons, but the problems centre on the decreased generalisability of trial findings and equity for patients in accessing the benefits of participating in trials. First, the generalisability of research findings is compromised [2, 4, 5], and the potential effects of variation in the pathobiology of disease, diet and lifestyle, both positive and negative, may go undetected, as do factors such as race-related differences in drug responses . Some health problems are more prevalent among particular minority groups, and a disparity exists in health outcomes according to ethnicity  and between minorities and non-minorities . Another concern is that healthcare provision is not equal to all [7, 8]; people who participate in research may receive better care and have better health outcomes . The disadvantaged sectors of the population can be excluded from the potential benefits of trial participation , which can provide newer, higher-quality or specialised care and monitoring that is made available to participants in trials .
The necessity of including older people from minority groups in clinical research will intensify given that older people are a fast-growing segment of the population worldwide . In 2012, in Australia, people aged 65 years and over constituted 14 % of the population, but the proportion is projected to increase to 22 % in 2061, and to 25 % in 2101 . Currently, people from a CALD background make up a substantial proportion of Australia’s ageing population, with over 20 % of people older than 65 years of age having been born outside of Australia, and in just a few years’ time, by 2021, that figure will rise to 30 % .
Policy promoting equitable inclusion in clinical trials
(a)…the selection, exclusion and inclusion of research participants is fair…
(b) there is no unfair burden of participation in research on particular groups; …
(d) there is fair distribution of the benefits of participation in research .
In addition, one of the primary goals of the Australian government’s National Ageing and Aged Care Strategy for People from Culturally and Linguistically Diverse (CALD) Backgrounds is better practice through improving research and data collection mechanisms that are inclusive of the ageing CALD population .
Health literacy and ethnicity
A known correlation exists between the level of health literacy and ethnicity, with minority populations suffering from lower levels of health literacy than the general population . Furthermore, people with low English proficiency (LEP) have been shown to be disadvantaged in the informed consent process, scoring lower in measures of informed consent than fluent English speakers . Lower health literacy is strongly associated with less engagement with health care and, thus, poorer health outcomes . Available health education materials are often not literacy-level appropriate [27–29]. In Australia, only 33 % of the overseas-born population has ‘adequate or better’ health literacy compared to 43 % of the Australian-born population, and the figures are lower for more recent arrivals (27 %) and for people whose first language is not English (26 %) . Participating in a clinical trial is one health-related decision people from a CALD background may make in their lifetime, and evidence exists that the under-representation of CALD patients in clinical research is partly due to a limited awareness and understanding of research opportunities and processes .
Providing appropriate information to research participants is central to good clinical research practice [16, 31, 32] but presents challenges in approaching CALD patients. In cases where CALD patients do not speak English or have LEP or literacy, they are often not approached about participating. This occurs for two reasons: first, inadequate evidence is available on effective strategies for recruiting CALD patients, and second, research funding rarely covers the costs of developing CALD strategies.
While positive intent exists on the part of governments and the health sector to address the issue of low participation of CALD groups and while advances have been made in terms of identifying barriers and useful strategies associated with inclusion, minority groups are still not well represented in clinical trials [1, 9, 33]. This review examines current barriers to and strategies for involving CALD patients in clinical research, as well as the issues associated with informed-consent procedures in this group and the strategies that have been used to try to overcome these barriers. We also examine the outcomes of interventions trialling multimedia for the research-consent process in both English speakers and CALD patients in order to assess the feasibility of adapting such technologies for older CALD people.
Identifying barriers and developing strategies to overcome the lack of inclusion of CALD groups in medical research
Identifying barriers and developing strategies to overcome the lack of inclusion of older people in medical research
Improving the informed consent process
The use of multimedia aids in the informed consent process
We conducted a search in MEDLine in May 2015, and no results were returned for the defined topic of improving the informed consent process to improve CALD participation in clinical research utilising methods incorporating multimedia technology (e.g. computers, video, and tablets/iPads). The lack of studies trialling innovative methods of obtaining informed consent in populations where English language proficiency is low has been commented on in the past. For example, Hendrickson  found no documented use of videotape recruitment of non-English-speaking participants prior to her study. The dearth of existing reports examining the use of tablets in clinical trial research has also been noted .
In response to this finding, we then broke the research topics down into their components to be examined separately: research on the participation of CALD groups in medical research, research on the participation of older people in medical research (CALD and the elderly in general), studies on informed consent in a CALD context, and multimedia use in a research informed consent context. Combinations of the following terms were used in the search query: ‘informed consent’ AND/OR ‘migrants’, ‘minorities’, ‘low English proficiency’, ‘multimedia’, ‘elderly’, ‘iPad’, ‘video’, and ‘computer’.
Barriers to the inclusion of elderly CALD people in clinical trials
The major themes consistently identified by researchers as barriers to CALD participation in clinical research are: (1) mistrust; (2) communication barriers, including the complexity of written documents, language/literacy issues and lack of perceived benefit; (3) cultural barriers, including competing cultural beliefs/practices concerning health; (4) economic and time constraints; (5) mobility issues and health issues; and (6) opportunity barriers.
Mistrust is a commonly cited barrier to inclusion, manifesting in a number of ways: mistrust of mainstream society , of the scientific community or of research institutions , perhaps due to awareness of unethical practices on the part of the medical research community in the past [2, 6, 36], or a perception that academic institutions are elitist and not committed to the welfare of ethnic minority communities . Alternatively, the mistrust may be directed towards the general healthcare system or be culturally generated, as for example when the medical practices of the culture of origin differ substantially to those of mainstream Western physicians . Mistrust issues may be more pronounced among the elderly  who, because of increased frailty, can feel more vulnerable to potential exploitation .
Language is an evident communication barrier in CALD groups . Communication barriers also include issues associated with complex forms and informed consent procedures [38, 39], which are amplified for people with limited English language and/or low literacy . Achieving true informed consent, even amongst those for whom English is a first language, can be difficult [28, 39, 40]. Much debate has occurred on the comprehensibility of participant information forms, which are often notoriously lengthy and require a high (often tertiary) level of reading comprehension [17, 41–43]. Ensuring that a patient understands to what he/she is consenting becomes even more problematic in the presence of CALD barriers. While ensuring informed consent is paramount, the process of obtaining consent may end up having the undesired effect of creating barriers by arousing suspicion, promoting mistrust, and causing misunderstanding, which may ultimately discourage participation .
Failing to communicate research findings with communities is also a known barrier . Among the many reasons for low success in clinical trial recruitment in general is a ‘perceived lack of information about the purpose, procedures and value of clinical trials that [patients] have been asked to contribute to’ (p. 111) . This issue has been highlighted in studies on the elderly [37, 46].
Cultural barriers to research recruitment include inappropriate information provision , unsuitable data collection and assessment measures [47, 48], cultural stigmas associated with illness, and a lack of social presence in the communities . Specifically, for older CALD people, a lack of cultural diversity amongst research staff can act as a barrier to CALD participation; conversely ‘race matching’ (alignment of the race of the patient and researcher) may enhance participation [23, 49]. Beliefs about health and health care can vary significantly according to different cultures [8, 50] and may clash with the prevailing norms in the mainstream society in question, making individuals from CALD backgrounds less likely to explore all the healthcare options available to them.
Important cultural considerations have been identified in the consent process. Obtaining written informed consent from research participants remains standard practice . However, it may be inappropriate to seek written consent from participants of certain cultures . Killawi and colleagues  identified potential problems with obtaining signed, written consent, including an association with formal transactions, the arousal of suspicion or concern, the potential for threat for illiterate persons, or the implication of a lack of trust. Another factor undermining signed written consent is the possibility of the request for a signature being perceived as insulting if one’s word has already been given .
Economic and time constraints
CALD status and lower socioeconomic status are two factors that are often connected , and socioeconomic constraints have been cited by many researchers as barriers to recruitment of CALD participants in medical research [6, 10, 52, 53]. Time constraints, including those connected with family and job responsibilities, are also preventive factors in study participation [20, 53].
Mobility and health issues
Restrictions required by religious or cultural precepts may mean that women in some CALD groups may have compromised mobility resulting from their dependence on their husbands or another male relative for transportation . For the elderly in general, additional mobility barriers such as physical frailty, increased health issues, and illness can make it difficult for this group to participate in clinical trials . Common conditions such as visual impairment or decreased manual dexterity can inhibit participation; for example, written documents are more of a barrier [3, 54]. The elderly in general suffer greater degrees of social isolation, making them a less accessible group [2, 37].
Barriers are also created for CALD groups by the research process. These factors are important, since they determine whether or not a person is given the opportunity to participate in research at all. For example, people with LEP or low levels of literacy are routinely excluded from many clinical trials because they are viewed as unable to provide truly informed consent [6, 41]. Most research protocols do not engage with language proficiency at all . Furthermore, when clinicians believe the costs or burden of data collection are too great, they are less likely to refer or recruit patients to clinical trials . In a systematic review of studies on recruitment of underrepresented groups into clinical trials, 11 studies out of 18 found clinician attitudes were a barrier to enrolment, whereas only three of these studies found clinician attitudes to be an enabler . Provider attitudes that hindered patient recruitment related to patient age, comorbidities, disease stage, and/or a perception of patient mistrust of researchers.
In another review of studies examining barriers to recruitment of underrepresented populations in clinical trials, Ford et al. found that the most commonly reported barriers related to the opportunity to participate in trials . Potential participants were excluded from trials because of factors such as age, socioeconomic status, ethnic minority status, and comorbid conditions. This study also identified provider attitudes and eligibility criteria as the most frequently reported barriers to participation.
Overcoming the lack of inclusion of elderly CALD in clinical research
Summary of the barriers to the inclusion of elderly participants who are culturally and linguistically diverse (CALD) in clinical trials and the strategies that have been developed to overcome them
• Of society, the scientific community, and research institutions, and of the healthcare system
1. Community relationship building & outreach
• Enrol community gatekeepers in the research
• Build coalitions & partner with physicians
• Enrol trusted and prominent community members (community and religious leaders)
• Make use of other referral sources
• Get involved with the community (attend events, etc.)
• Provide support to the community outside of research in ways that are meaningful to the community
• Maintain connections with communities
• Involve the community in study design and implementation
• Low English proficiency
• Information presented is too complex
• Low health literacy
• Failure of researcher to communicate study rationale
• Failure of researcher to communicate research findings
2. Communication – initial & ongoing
• Provide education about health and medical research
• Provide training for key community representatives
• Provide community information sessions
• Explain potential benefits of participation
• Engage in follow up: provide feedback and study results to participants
• Provide translations of forms and materials
• Provide simple, appropriate, and systematic explanations of research
• Use innovative materials to enhance comprehension (e.g. video presentations, animations).
• Simplify language and decrease content in consent documentation
• Apply user-friendly formatting and presentation (large graphics, both visual and audio input)
• Culturally inappropriate assessment measures
• Potential clash of beliefs around health care
• Potential alienation of participants if staff lack cultural diversity
• Potential issues with requiring signed written consent
3. Cultural sensitivity
• Be informed about the background and cultural circumstances of the CALD group in question
• Use culturally appropriate language, symbolic gestures, and ideas. • Respect differences
• Make provision for possible adaptations based on cultural differences
• Use bilingual/bicultural staff and staff reflective of the community
• Ensure consent process is culturally compatible/acceptable (e.g. accept verbal consent if preferred by participant)
4. Economic & time constraints
• High likelihood of socioeconomic hardship
• Limited availability of participants
4. Facilitate access to research studies
• Take the research to the community
• Provide reimbursement for travel expenses
• Provide childcare
• Be flexible with scheduling
• Conduct research within the community/conduct home visits
• Provide transportation
• Mobility may be restricted for cultural or religious reasons.
• In elderly populations, mobility may be a factor because of increased frailty and/or impaired functionality.
5. Awareness raising amongst researchers and other stakeholders of barriers to CALD participation
• Revisit and adapt consent processes for CALD groups
• Make study designs less rigid
• People with low English proficiency or literacy levels are often excluded from trials.
• Clinicians attitudes are a strong barrier.
• If clinicians experience the costs or burden of data collection to be high, they are less likely to recruit participants.
Community relationship building and outreach
Establishing and building trust is paramount in any endeavour to enrol and retain members of minority groups in clinical research. A key strategy for doing this is through developing relationships within the community of the target group . This can be through establishing and maintaining a relationship with a community advisory board , so-called ‘gatekeepers’, such as family members or housing managers and enrolling them in the evaluation of the research opportunity, building coalitions, and partnering with physicians who provide care for the populations to be included in the trial [17, 46], thereby establishing a rapport with trusted and prominent members of the community such as community or religious leaders [6, 17, 58] and other potential referral sources [18, 57].
Becoming more involved with the community by attending events  and committing to support the community in ways that the community values are effective, even essential, strategies [6, 58] for building trust and demonstrating a genuine motivation to help the community. The maintenance of ongoing connections is also necessary , especially with community leaders . This is important as a trust-building exercise in the context of ongoing research studies. Involving the community in study design and implementation [15, 22], while utilising, for example, community-based participatory research frameworks [19, 59] recognises that communities contribute knowledge to the process, which in turn helps with the formulation of recruitment strategies and encourages the involvement of minority community groups .
Communication–initial and ongoing
Providing education about the purposes of medical research is an important strategy . Santoyo-Olsseen et al.  credit much of their success in recruiting lower socioeconomic and/or CALD adults to a diabetes risk-reduction program to their efforts to provide health education and raise awareness in the target community prior to study recruitment. Other educational strategies include providing training for key community representatives  or conducting community information sessions . Ensuring that participants are aware of potential benefits of participation also enhances the likelihood of participation [3, 17], and making sure participants agree with the benefits to themselves and their community and communicating research outcomes have been shown to create more favourable conditions for the participation of minority groups in medical research [6, 18]. Appealing to participants’ curiosity about their own health can be a helpful engagement strategy . It is important to ensure there is a process of follow-up and that participants are given feedback about or summaries of the study results [17, 18, 23, 57].
Making forms and materials available in the target group’s language(s) helps in overcoming language barriers –provided the translation protocol ensures cultural and literacy-level appropriateness . Explaining the research process in a simple, appropriate, and systematic way can help to alleviate concerns regarding low literacy and unfamiliarity with research . Also, using innovative materials and processes such as multimedia, descriptive videos or illustrations can greatly enhance the comprehension and retention of concepts [38, 47].
Specifically, with regard to the consent process, simplifying the language and content of forms and avoiding repetition and redundancy have been identified as effective ; succinct, comprehensible material is preferable to overloading a patient with detail, which may simply be confusing. Stunkel and colleagues  found that levels of comprehension did not differ for participants who were randomly assigned to read either a ‘shorter, more readable’ consent form compared to a longer, standard consent form. Tymchuk et al.  found that institutionalised elderly subjects benefited from simplified versions of forms more than from either storybook presentations or videotape. Another study showed that video education increased patients’ understanding of consent information compared to oral education .
Formatting that presents information clearly (bullet lists, adding section breaks, signposting definitions of terms, and increasing font size for visually impaired people) is recommended as a strategy to aid comprehensibility . McDougall and colleagues  found that their participants (older adults of ethnic and racial minorities) with visual and hearing deficits preferred PowerPoint presentations with large graphics, few words, and animation. People with hearing problems could more easily discern meanings if they could see as well as hear information. Taub and colleagues  confirmed the usefulness of a multi-step approach, using a comprehension test as part of the informed consent process with elderly participants.
It is important to have a clear understanding of the special circumstances of older CALD adults, which will differ according to the particular CALD group , including of their cultural-historical background  and socio-political conditions . Researchers must be mindful to use culturally appropriate language [3, 6], avoid language that may be stigmatising for some groups , or adopt symbolic gestures where appropriate , and they demonstrate an appropriate attitude of openness and respect for differences that is exclusive of stereotypes, prejudices, and biases , while also making provision for possible adaptations based on cultural differences . For example, some groups respond more favourably to personalised or direct, person-to-person recruitment strategies [19, 21, 23].
Employing researchers and staff reflective of the community, as well as employing bilingual, bicultural staff, enhances participation [6, 17, 57]. The process of gaining, building, and maintaining trust is easier with older CALD people when a researcher and respondent are from the same background . This was also a key strategy identified by Sheppard and colleagues  in their six studies on recruiting Latinos into cancer research in the United States. They reported very high success rates for recruitment (96 % on average). Other strategies employed by this team included the following: working in Latino places of social interactions, using high-profile community leaders to disseminate information (e.g. through media), simplifying and improving comprehension in the informed consent process, and emphasising culturally important ideas such as the significance of the family in Latino culture.
Improving the consent process for CALD participants has been the focus of several studies. Some CALD groups favour verbal information over written  and may see verbal consent as equivalent to written consent . A UK study found that obtaining informed consent from CALD persons using audio recordings was acceptable for the participants, and posed no difficulties . In a linked study conducted in Bangladesh, use of audio recording for the consent process encouraged individuals with low literacy levels to participate in research . Indeed, some institutions now recommend the use of audio for collecting data from minority ethnic communities . Hernandez and colleagues  urge researchers to consider employing a waiver of signed consent of participants when conducting research in immigrant communities. Avoiding written signed consent may be more appropriate for participants with limited literacy or a lack of education, or in cases where the preferred language modality is oral. Another approach for documenting consent, which has been suggested as potentially more acceptable in given communities, is witnessed consent .
Facilitate access to research studies
In minority groups, mobility and safety concerns influence whether or not a person feels comfortable, or even able, to participate in a research study. Conducting research within a community space or offering home visits has helped with recruiting and retaining research participants [1, 3, 19, 57, 58]. Other options include taking the study to the local neighbourhood of the target group and to places where the participants live and congregate (e.g. grocery stores and recreation centres) . Other strategies to improve access, including providing incentives and reimbursement (for travel/parking) [1, 6], providing transportation [19, 58], or even providing childcare , which can be very helpful strategies for communities where economic constraints are a barrier to research participation. However, introducing monetary incentives does raise ethical issues, as it may exert an unfair influence on some populations . Flexible scheduling is an important consideration to combat the time-constraint barrier  and has been successfully implemented to improve enrolment and retention rates .
Awareness raising amongst researchers and other stakeholders of barriers to CALD participation
Ford and colleagues concluded that ‘Because opportunity barriers largely reflect protocol design as well as the process of study implementation, investigators play a major role in determining the extent to which trials are accessible to underrepresented groups’ (p. 238) .
Doctors, researchers, and ethics committees have a crucial role to play in increasing the participation of CALD groups in clinical research, while still adequately protecting participants [25, 31]. Informed consent processes need to be revisited and adapted for CALD groups. It has been suggested that ‘less rigid study designs’ (p. 238) and systems that facilitate the participation of healthcare providers are ways to create conditions that will improve the recruitment of underrepresented populations to trials . Modified consent processes need to be developed to meet the needs of participants for comprehensibility and ease of use, while retaining ethical rigour. Nesting trials of innovative methodologies aimed at increasing recruitment within existing studies has been suggested as an effective way to test the robustness of new methods .
The use of multimedia to improve CALD participation in research
Researchers have attributed success in the recruitment of minority groups to a multifactorial methodology  and to the use of ‘an array of recruitment strategies’ . One problem is that engaging CALD communities requires significant resources, not the least of which is financial, that are rarely available in current research structures. Incorporating multimedia technology may be less costly in the long run and more sustainable than current practices that require interpreters. An electronic resource can be customisable to accommodate multiple languages and can be developed incorporating principles of cultural sensitivity. In addition, multimedia could be used by researchers who do not necessarily speak the language of the participant.
Furthermore, multimedia resources may have key roles to play in addressing health research literacy by explaining medical research, enabling researchers to assess comprehension through testing, and improving CALD patient comprehension of consent forms and procedures . In addition, technology offers possibilities to overcome lengthy and burdensome patient information sheets, a known source of patient dissatisfaction . The findings from trials of electronic resources to improve consent processes, even if only with non-CALD participants, can inform the development of resources for CALD groups.
Methods combining textual, audio, and graphic (static and/or dynamic) modes are increasingly being used in many settings. Simultaneously combining more than one mode (e.g. text and graphics) is associated with increased comprehension, learning, and retention than text alone, and splitting incoming information across more than one cognitive channel (e.g. audio and visual) is associated with significant learning gains .
Some reviews of clinical research using multimedia to enhance understanding of information and consent forms have reported inconsistent or minimal evidence of improvements using newer technologies [40, 45, 73], and proposed benefits to the willingness to participate and participant satisfaction have been either inconclusive or not apparent . Conclusions have been limited due to novelty, inconsistent methodologies, and definitions . In spite of these limitations, favourable conclusions concerning the effectiveness of multimedia in the informed consent process have also been drawn. Improved outcomes for participant comprehension [41, 74, 75], for both immediate information recall [76, 77] and delayed recall , participant satisfaction (particularly regarding understanding) [41, 79, 80], and reduced patient anxiety levels [78, 79] have all been attested in both systematic reviews and individual studies of multimedia approaches to consent procedures conducted with fluent English speakers. In addition, when consulted, participants have indicated a preference for incorporating multimedia in consent protocols .
In individual studies on consent procedures where the only difference was the medium used, patients generally preferred electronic over paper-based questionnaires in spite of the fact that the former took longer to complete [81, 82], perhaps because they had to be filled out completely in order for patients to finish them (which is in itself advantageous from a researcher perspective).
A few studies adopting interactive approaches, for example, where participants are required to demonstrate their understanding by answering a quiz after watching a video on an iPad or computer, show particular promise for improving the comprehension and retention of knowledge regarding research in participants [54, 83] and increasing participant interest in taking part in a study .
Acceptability of an electronic tablet resource to older CALD people
Some questions exist concerning the acceptability of using an electronic resource with older people. Nonetheless, one study comparing the use of a tablet-based screening procedure with an integrated voice response system for recruitment and screening of patients into pragmatic clinical trials concluded that the tablet was ‘highly accepted by older patients’ . Shneerson et al.  designed a questionnaire to elicit views from older cancer patients, their carers, friends, and families on the provision and design of multimedia resources used to deliver clinical trials information. Most respondents, whose average age was over 60, indicated that seeing a multimedia resource might encourage them to find out more about clinical trials, and more than half preferred a ‘more practical learning mode, as could be delivered via multimedia resources’. Preferred devices were mobile phones, laptops, and DVD players, although Shneerson concluded that ‘from a practical perspective tablet technologies would appear to be the most suitable delivery medium, being portable, cheap, less prone to operating system errors and simple to use’, and felt that the transition from PCs (with which they were familiar) to tablets would be relatively easy with suitable training and support.
A study evaluating user response to a talking touchscreen, introduced as a means to improve health literacy in underserved U.S. populations, found that most users, the majority of whom were computer-naïve, rated the design of the screen highly and found it easy to use . It is important to take into account, nonetheless, that in some cases, there may be a correlation between difficulty filling out a questionnaire on a tablet and CALD background, increased age, lower levels of education, and comorbid medical conditions . Furthermore, research into barriers to participation underlines the importance, for participants, of a human connection .
A main finding of this review is that addressing the obstacles to CALD participation in clinical research requires a complex, multi-methodological approach. A further aid to help eliminate barriers, we believe, lies in incorporating multimedia tools in this approach. To be effective, such tools need to form part of a culturally sensitive system, which we propose would include providing resources in the preferred languages of potential participants, and tailoring them to target communities’ language, literacy, and cultural needs. We envisage that it would always be preferable to have a researcher present with a participant, both to help with using the device, and also to be a tangible human presence.
Nonetheless, another important finding of this review has been to emphasize the fact that the ultimate success of any initiatives in the area of CALD inclusion in clinical research is contingent on the willingness of stakeholder groups, and notably ethics review boards, to participate in and co-operate with the creation of novel processes or the adaptation of existing processes, especially concerning consent. This would create the conditions needed to make the participation of CALD groups in clinical research a reality. We recommend that future work in the area focus on this key issue.
CALD, culturally and linguistically diverse; LEP, low English proficiency.
The term CALD is used in the current study. However, the term is used interchangeably with ‘migrant’ or ‘ethnic’ in the context of comparisons with other studies.
The authors wish to acknowledge funding from the Institute for a Broadband Enabled Society (IBES), now called Melbourne Networked Society Institute, University of Melbourne, Australia.
All authors made substantial contributions to the conception of the review, with DS and TP largely responsible for conception. JH was responsible for drafting the manuscript with ongoing input from RWK, AP, JH, and DS. AB, UK, and TP revised the manuscript critically. All authors read and approved the manuscript. All authors agree to be accountable for all aspects of the work and its quality.
JH has a PhD in linguistics, and is a research fellow in the School of Languages and Linguistics, The University of Melbourne (UoM).
RWK is an Associate Professor in healthcare communication in the Department of Medical Education, UoM. Her research includes intercultural healthcare communication.
AP holds a Master of Bioethics and is a research manager and researcher in the Anaesthesia, Perioperative and Pain Medicine Unit, the Melbourne Medical School, UoM
JH is Professor of Italian, School of Languages and Linguistics, and the Director of the Research Unit for Multilingualism and Cross-cultural Communication (RUMACCC), UoM.
AB is a PhD candidate in the Department of Italian, School of Languages and Linguistics, UoM.
UK is the Director of the Language Testing Research Centre, School of Languages and Linguistics, UoM.
TP is a staff specialist anaesthetist and Head of Quality and Safety in the Anaesthesia Department at St. Vincent’s Hospital Melbourne
DS is Professor and Chair of Anaesthesia, UoM and consultant anaesthetist at The Austin and other hospitals.
The authors declare that they have no competing interests.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- McDougall GJ, Simpson G, Friend ML. Strategies for research recruitment and retention of older adults of racial and ethnic minorities. J Gerontol Nurs. 2015;41(5):14–23. doi:10.3928/00989134-20150325-01. quiz 4-5.PubMedGoogle Scholar
- Stahl SM, Vasquez L. Approaches to improving recruitment and retention of minority elders participating in research: examples from selected research groups including the National Institute on Aging’s resource centers for minority aging research. J Aging Health. 2004;16(5 suppl):9S–17. doi:10.1177/0898264304268146.View ArticlePubMedGoogle Scholar
- Mody L, Miller DK, McGloin JM, Freeman M, Marcantonio ER, Magaziner J, et al. Recruitment and retention of older adults in aging research. J Am Geriatr Soc. 2008;56(1532-5415 (Electronic)):2340–8. doi:10.1111/j.1532-5415.2008.02015.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Giuliano AR, Mokuau N, Hughes C, Tortolero-Luna G, Risendal B, Ho RCS, et al. Participation of minorities in cancer research: the influence of structural, cultural, and linguistic factors. Ann Epidemiol. 2000;10(8, Supplement 1):S22–34. doi:10.1016/s1047-2797(00)00195-2.View ArticlePubMedGoogle Scholar
- King TE. Racial disparities in clinical trials. N Engl J Med. 2002;346(18):1400–2. doi:10.1056/nejm200205023461812.View ArticlePubMedGoogle Scholar
- Moreno-John G, Gachie A, Fleming CM, NÁpoles-Springer A, Mutran E, Manson SM, et al. Ethnic minority older adults participating in clinical research: developing trust. J Aging Health. 2004;16(5 suppl):93S–123. doi:10.1177/0898264304268151.View ArticlePubMedGoogle Scholar
- Glickman SW, Ndubuizu A, Weinfurt KP, Hamilton CD, Glickman LT, Schulman KA, et al. Perspective: the case for research justice: inclusion of patients with limited English proficiency in clinical research. Acad Med. 2011;86(3):389–93.View ArticlePubMedGoogle Scholar
- Henderson S, Kendall E. Culturally and linguistically diverse peoples’ knowledge of accessibility and utilisation of health services: exploring the need for improvement in health service delivery. Aust J Prim Health. 2011;17(2):195–201. doi:10.1071/PY10065.View ArticlePubMedGoogle Scholar
- Liu J, Davidson E, Sheikh A. Achieving ethnic diversity in trial recruitment. Pharm Med. 2011;25(4):215–22. doi:10.1007/bf03256863.View ArticleGoogle Scholar
- Ford JG, Howerton MW, Lai GY, Gary TL, Bolen S, Gibbons MC, et al. Barriers to recruiting underrepresented populations to cancer clinical trials: a systematic review. Cancer. 2008;112(2):228–42. doi:10.1002/cncr.23157.View ArticlePubMedGoogle Scholar
- George S, Moran E, Duran N, Jenders RA. Using animation as an information tool to advance health research literacy among minority participants. AMIA Annu Symp Proc. 2013;2013:475–84.PubMedPubMed CentralGoogle Scholar
- World Health Organization. WHO facts about ageing. www.who.int/ageing/about/facts/en. Accessed 1 Oct 2015.
- Australian Bureau of Statistics. 3222.0 Population Projections, Australia (2012 (base) to 2101) - 32220_2012 (base) to 2101.pdf. Commonwealth of Australia: Australian Bureau of Statistics; 2013. http://www.abs.gov.au/ausstats/abs@.nsf/mf/3222.0. Accessed 1 Oct 2015.Google Scholar
- Commonwealth of Australia. National Ageing and Aged Care Strategy for People from Culturally and Linguistically Diverse (CALD) Backgrounds. 2012. www.dss.gov.au/ageing-and-aged-care/older-people-their-families-and-carers/people-from-diverse-backgrounds/national-ageing-and-aged-care-strategy-for-people-from-culturally-and-linguistically-diverse-cald-backgrounds. Accessed 1 Oct 2015.
- Pirie A, Gute DM. Crossing the chasm of mistrust: collaborating with immigrant populations through community organizations and academic partners. Am J Public Health. 2013;103(12):2126–30.View ArticlePubMedPubMed CentralGoogle Scholar
- World Medical Association and others. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310(20):2191.View ArticleGoogle Scholar
- Curry L, Jackson J. The science of inclusion: recruiting and retaining racial and ethnic elders in health research. Washington, DC: Gerontological Society of America; 2003.Google Scholar
- Kauffman KS, DosReis S, Ross M, Barnet B, Onukwugha E, Mullins CD. Engaging hard-to-reach patients in patient-centered outcomes research. J Comp Eff Res. 2013;2(3):313–24. doi:10.2217/cer.13.11.View ArticlePubMedGoogle Scholar
- Santoyo-Olsson J, Cabrera J, Freyre R, Grossman M, Alvarez N, Mathur D, et al. An innovative multiphased strategy to recruit underserved adults into a randomized trial of a community-based diabetes risk reduction program. Gerontologist. 2011;51 suppl 1:S82–93. doi:10.1093/geront/gnr026.View ArticlePubMedPubMed CentralGoogle Scholar
- Ejiogu N, Norbeck JH, Mason MA, Cromwell BC, Zonderman AB, Evans MK. Recruitment and retention strategies for minority or poor clinical research participants: lessons from the Healthy Aging in Neighborhoods of Diversity Across the Life Span study. Gerontologist. 2011;51 suppl 1:S33–45. doi:10.1093/geront/gnr027.View ArticlePubMedPubMed CentralGoogle Scholar
- Arean PA, Alvidrez J, Nery R, Estes C, Linkins K. Recruitment and retention of older minorities in mental health services research. Gerontologist. 2003;43(1):36–44.View ArticlePubMedGoogle Scholar
- Hockenson Ryall AL, Abdulah DR, Rios DA, Hausdorff JM, Wei JY. Recruitment and retention of ethnically diverse elderly research subjects in an exercise intervention study. In: Curry L, Jackson AS, editors. The science of inclusion: recruitment and retaining racial and ethnic elders in health research. Washington, DC: Gerontological Society of America; 2003. p. 46-55Google Scholar
- Vesey GA. Recruitment and retention of minority elders in health-related research: A community-based approach. In: Curry L, Jackson JS, editors. The science of inclusion: recruitment and retaining racial and ethnic elders in health research. Washington, DC: Gerontological Society of America; 2003. p. 82–9.Google Scholar
- National Institutes of Health. National Institutes of Health Revitalization Act of 1993. Bethesda: NIH; 1993.Google Scholar
- National Health and Medical Research Council. National Statement on Ethical Conduct in Human Research. Australian Government. 2007. https://www.nhmrc.gov.au/guidelines-publications/e72. Accessed 1 Oct 2015.
- Hunt LM, de Voogd KB. Are good intentions good enough? Informed consent without trained interpreters. J Gen Intern Med. 2007;22(5):598–605.View ArticlePubMedPubMed CentralGoogle Scholar
- Shoemaker SJW, Michael S, Brach C. Development of the Patient Education Materials Assessment Tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information. Patient Educ Couns. 2014;96(3):395–403.View ArticlePubMedGoogle Scholar
- Paasche-Orlow MK, Taylor HA, Brancati FL. Readability standards for informed-consent forms as compared with actual readability. N Engl J Med. 2003;348(8):721–6. doi:10.1056/NEJMsa021212.View ArticlePubMedGoogle Scholar
- Michael J, Aylen T, Ogrin R. Development of a translation standard to support the improvement of health literacy and provide consistent high-quality information. Aust Health Rev. 2013;37(4):547–51.View ArticlePubMedGoogle Scholar
- Ethnic Communities’ Council of Victoria. An investment not an expense: enhancing health literacy in CALD communities. 2012. https://ehealthuq.wordpress.com/2012/10/17/an-investment-not-an-expense-enhancing-health-literacy-in-culturally-and-linguistically-diverse-communities-australian-policy-online/. Accessed 1 Oct 2015
- International Conference on Harmonisation Working Group. ICH harmonised tripartite guideline: guideline for good clinical practice E6 (R1)1996. Geneva: ICH; 1996.Google Scholar
- Regulation (EU) No 536/2014 of the European Parliament and of the Council of 16 April 2014 on clinical trials on medicinal products for human use, and repealing Directive 2001/20/EC, 536/2014. 2014. http://eur-lex.europa.eu/legal-content/en/TXT/?uri=CELEX:32014R0536. Accessed 23 May 2016.
- UyBico SJ, Pavel S, Gross CP. Recruiting vulnerable populations into research: a systematic review of recruitment interventions. J Gen Intern Med. 2007;22(6):852–63. doi:10.1007/s11606-007-0126-3.View ArticlePubMedPubMed CentralGoogle Scholar
- Hendrickson SG. Video recruitment of non-English-speaking participants. West J Nurs Res. 2007;29(2):232–42. doi:10.1177/0193945906292558.View ArticlePubMedGoogle Scholar
- Mudano AS, Gary LC, Oliveira AL, Melton M, Wright NC, Curtis JR, et al. Using tablet computers compared to interactive voice response to improve subject recruitment in osteoporosis pragmatic clinical trials: feasibility, satisfaction, and sample size. Patient Prefer Adherence. 2013;7:517–23. doi:10.2147/PPA.S44551.View ArticlePubMedPubMed CentralGoogle Scholar
- Johnson MRD. Hard to reach: easy to omit. Prim Care Respir J. 2011;20:229. doi:10.4104/pcrj.2011.00059.View ArticlePubMedGoogle Scholar
- Provencher V, Mortenson WB, Tanguay-Garneau L, Belanger K, Dagenais M. Challenges and strategies pertaining to recruitment and retention of frail elderly in research studies: a systematic review. Arch Gerontol Geriatr. 2014;59(1):18–24. doi:10.1016/j.archger.2014.03.006.View ArticlePubMedGoogle Scholar
- Tait AR, Voepel-Lewis T. Digital multimedia: a new approach for informed consent? JAMA. 2015;313(5):463–4.View ArticlePubMedGoogle Scholar
- Donovan-Kicken E, Mackert M, Guinn TD, Tollison AC, Breckinridge B. Sources of patient uncertainty when reviewing medical disclosure and consent documentation. Patient Educ Couns. 2013;90(2):254–60. doi:10.1016/j.pec.2012.10.007.View ArticlePubMedGoogle Scholar
- Flory J, Emanuel E. Interventions to improve research participants’ understanding in informed consent for research: a systematic review. JAMA. 2004;292(13):1593–601.View ArticlePubMedGoogle Scholar
- Sonne SC, Andrews JO, Gentilin SM, Oppenheimer S, Obeid J, Brady K, et al. Development and pilot testing of a video-assisted informed consent process. Contemp Clin Trials. 2013;36(1):25–31.View ArticlePubMedPubMed CentralGoogle Scholar
- Henry J, Palmer BW, Palinkas L, Glorioso DK, Caligiuri MP, Jeste DV. Reformed consent: adapting to new media and research participant preferences. IRB. 2009;31(2):1–8.PubMedPubMed CentralGoogle Scholar
- Jefford M, Moore R. Improvement of informed consent and the quality of consent documents. Lancet Oncol. 2008;9(5):485–93. doi:10.1016/S1470-2045(08)70128-1.View ArticlePubMedGoogle Scholar
- McCabe M, Morgan F, Curley H, Begay R, Gohdes DM. The informed consent process in a cross-cultural setting: is the process achieving the intended result? Ethn Dis. 2005;15(2):300–4. doi:D - kie: 129459 d - nrcbl: vf 18.3 oto - kie.Google Scholar
- Shneerson C, Windle R, Cox K. Innovating information-delivery for potential clinical trials participants: what do patients want from multi-media resources? Patient Educ Couns. 2013;90(1):111–7. doi:10.1016/j.pec.2012.06.031.View ArticlePubMedGoogle Scholar
- McDougall Jr G, Simpson G, Friend M. Strategies for research recruitment and retention of older adults of racial and ethnic minorities. J Gerontol Nurs. 2015;41:14–23. doi:10.3928/00989134-20150325-01. quiz 24-5.PubMedGoogle Scholar
- Bhutta ZA. Beyond informed consent. Bull World Health Organ. 2004;82(10):771–7.PubMedPubMed CentralGoogle Scholar
- Killawi A, Khidir A, Elnashar M, Abdelrahim H, Hammoud M, Elliott H, et al. Procedures of recruiting, obtaining informed consent, and compensating research participants in Qatar: findings from a qualitative investigation. BMC Med Ethics. 2014;15:9. doi:10.1186/1472-6939-15-9.View ArticlePubMedPubMed CentralGoogle Scholar
- Lipson J, Meleis AI. Methodological issues in research with immigrants. Med Anthropol. 1989;12(1):103–15.View ArticlePubMedGoogle Scholar
- Stone JR. Healthcare inequality, cross-cultural training, and bioethics: principles and applications. Camb Q Healthc Ethics. 2008;17(2):216–26.PubMedGoogle Scholar
- Mertens DM. Ethics and social justice in ethnocultural qualitative research. In: Nagata DK, Kohn-Wood L, Suzuki LA, editors. Qualitative strategies for ethnocultural research. Washington, DC: American Psychological Association; 2012. p. 61–84.View ArticleGoogle Scholar
- Brown DR, Fouad MN, Basen-Engquist K, Tortolero-Luna G. Recruitment and retention of minority women in cancer screening, prevention, and treatment trials. Annals of Epidemiology. 2000;10(8, Supplement 1):S13–21. doi:10.1016/s1047-2797(00)00197-6.
- Brown DR, Topcu M. Willingness to participate in clinical treatment research among older African Americans and Whites. Gerontologist. 2003;43(1):62–72.View ArticlePubMedGoogle Scholar
- Cornoiu A, Beischer AD, Donnan L, Graves S, de Steiger R. Multimedia patient education to assist the informed consent process for knee arthroscopy. ANZ J Surg. 2011;81(3):176–80. doi:10.1111/j.1445-2197.2010.05487.x.View ArticlePubMedGoogle Scholar
- Murray S, Buller AM. Exclusion on grounds of language ability--a reporting gap in health services research? J Health Serv Res Policy. 2007;12(4):205–8.View ArticlePubMedGoogle Scholar
- Howerton MW, Gibbons MC, Baffi CR, Gary TL, Lai GY, Bolen S, et al. Provider roles in the recruitment of underrepresented populations to cancer clinical trials. Cancer. 2007;109(3):465–76. doi:10.1002/cncr.22436.View ArticlePubMedGoogle Scholar
- Arean PA, Gallagher-Thompson D. Issues and recommendations for the recruitment and retention of older ethnic minority adults into clinical research. J Consult Clin Psychol. 1996;64(5):875–80.View ArticlePubMedGoogle Scholar
- Levkoff S, Sanchez H. Lessons learned about minority recruitment and retention from the Centers on Minority Aging and Health Promotion. Gerontologist. 2003;43(1):18–26.View ArticlePubMedGoogle Scholar
- Dilworth-Anderson P. Introduction to the science of recruitment and retention among ethnically diverse populations. Gerontologist. 2011;51(S1):S1–4.View ArticlePubMedPubMed CentralGoogle Scholar
- Stunkel L, Benson M, McLellan L, Sinaii N, Bedarida G, Emanuel E, et al. Comprehension and informed consent: assessing the effect of a short consent form. IRB. 2010;32(4):1–9.PubMedPubMed CentralGoogle Scholar
- Tymchuk AJ, Ouslander JG, Rader N. Informing the elderly. A comparison of four methods. J Am Geriatr Soc. 1986;34(11):818–22.View ArticlePubMedGoogle Scholar
- Armstrong AW, Alikhan A, Cheng LS, Schupp C, Kurlinkus C, Eisen DB. Portable video media for presenting informed consent and wound care instructions for skin biopsies: a randomized controlled trial. Br J Dermatol. 2010;163(5):1014–9. doi:10.1111/j.1365-2133.2010.10067.x.View ArticlePubMedGoogle Scholar
- Taub HA, Kline GE, Baker MT. The elderly and informed consent—effects of vocabulary level and corrected feedback. Exp Aging Res. 1981;7(2):137–46.View ArticlePubMedGoogle Scholar
- Barata PC, Gucciardi E, Ahmad F, Stewart DE. Cross-cultural perspectives on research participation and informed consent. Soc Sci Med. 2006;62(2):479–90. View ArticlePubMedGoogle Scholar
- Lakes KD, Vaughan E, Jones M, Burke W, Baker D, Swanson JM. Diverse perceptions of the informed consent process: implications for the recruitment and participation of diverse communities in the National Children’s Study. Am J Community Psychol. 2012;49(1-2):215–32. doi:10.1007/s10464-011-9450-1.View ArticlePubMedPubMed CentralGoogle Scholar
- Sheppard VB, Cox LS, Kanamori MJ, Cañar J, Rodríguez Y, Goodman M, et al. Brief report: if you build it, they will come: methods for recruiting Latinos into cancer research. J Gen Intern Med. 2005;20(5):444–7.View ArticlePubMedPubMed CentralGoogle Scholar
- Lloyd CE, Johnson MR, Mughal S, Sturt JA, Collins GS, Roy T, et al. Securing recruitment and obtaining informed consent in minority ethnic groups in the UK. BMC Health Serv Res. 2008;8:68.View ArticlePubMedPubMed CentralGoogle Scholar
- Roy T, Lloyd CE. Development of audio methods of data collection in Bangladesh. Divers Health Soc Care. 2008;5(3):187–98.Google Scholar
- Hernandez MG, Nguyen J, Casanova S, Suarez-Orozco C, Saetermoe CL. Doing no harm and getting it right: guidelines for ethical research with immigrant communities. New Dir Child Adolesc Dev. 2013;2013(141):43–60.View ArticlePubMedGoogle Scholar
- Brugge D, Kole A, Lu W, Must A. Susceptibility of elderly Asian immigrants to persuasion with respect to participation in research. J Immigr Health. 2005;7(2):93–101.View ArticlePubMedGoogle Scholar
- Bower P, Brueton V, Gamble C, Treweek S, Smith CT, Young B, et al. Interventions to improve recruitment and retention in clinical trials: a survey and workshop to assess current practice and future priorities. Trials. 2014;15. doi:10.1186/1745-6215-15-399.
- Clark RC, Mayer RE. E-learning and the science of instruction: proven guidelines for consumers and designers of multimedia learning. San Francisco: John Wiley & Sons; 2011.View ArticleGoogle Scholar
- Ryan RE, McLaughlin KJ PMF, Hill SJ MLKF, Hill SJ. Audio-visual presentation of information for informed consent for participation in clinical trials. Cochrane Database Syst Rev. 2014;5:CD003717. doi:10.1002/14651858.CD003717.pub3.Review.PubMedGoogle Scholar
- Nishimura A, Carey J, Erwin PJ, Tilburt JC, Murad MH, McCormick JB. Improving understanding in the research informed consent process: a systematic review of 54 interventions tested in randomized control trials. BMC Med Ethics. 2013;14(1):28.View ArticlePubMedPubMed CentralGoogle Scholar
- Palmer BW, Lanouette NM, Jeste DV. Effectiveness of multimedia aids to enhance comprehension of research consent information: a systematic review. IRB. 2012;34(6):1–15.PubMedGoogle Scholar
- Karunaratne AS, Korenman SG, Thomas SL, Myles PS, Komesaroff PA. Improving communication when seeking informed consent: a randomised controlled study of a computer-based method for providing information to prospective clinical trial participants. Med J Aust. 2010;192(7):388–92.PubMedGoogle Scholar
- Farrell EH, Whistance RN, Phillips K, Morgan B, Savage K, Lewis V, et al. Systematic review and meta-analysis of audio-visual information aids for informed consent for invasive healthcare procedures in clinical practice. Patient Educ Couns. 2014;94(1):20–32.View ArticlePubMedGoogle Scholar
- Nehme J, El-Khani U, Chow A, Hakky S, Ahmed AR, Purkayastha S. The use of multimedia consent programs for surgical procedures: a systematic review. Surg Innov. 2013;20(1):13–23.View ArticlePubMedGoogle Scholar
- Jimison HB, Sher PP, Appleyard R, LeVernois Y. The use of multimedia in the informed consent process. J Am Med Inform Assoc. 1998;5(3):245–56.View ArticlePubMedPubMed CentralGoogle Scholar
- Hoffner B, Bauer-Wu S, Hitchcock-Bryan S, Powell M, Wolanski A, Joffe S. ‘Entering a clinical trial: is it right for you?’: a randomized study of The Clinical Trials Video and its impact on the informed consent process. Cancer. 2012;118(7):1877–83. doi:10.1002/cncr.26438.View ArticlePubMedPubMed CentralGoogle Scholar
- Schlechtweg PM, Hammon M, Giese D, Heberlein C, Uder M, Schwab SA. iPad-based patient briefing for radiological examinations-a clinical trial. J Digit Imaging. 2014;27(4):479–85. doi:10.1007/s10278-014-9688-x.View ArticlePubMedPubMed CentralGoogle Scholar
- Schlechtweg PM, Hammon M, Heberlein C, Giese D, Uder M, Schwab SA. Can the documented patient briefing be carried out with an iPad app? J Digit Imaging. 2013;26(3):383–92. doi:10.1007/s10278-012-9560-9.View ArticlePubMedPubMed CentralGoogle Scholar
- Rowbotham MC, Astin J, Greene K, Cummings SR. Interactive informed consent: randomized comparison with paper consents. PLoS One. 2013;8:e58603. doi:10.1371/journal.pone.0058603.View ArticlePubMedPubMed CentralGoogle Scholar
- Yost KJ, Webster K, Baker DW, Jacobs EA, Anderson A, Hahn EA. Acceptability of the talking touchscreen for health literacy assessment. J Health Commun. 2010;15(S2):80–92.View ArticlePubMedPubMed CentralGoogle Scholar
- Hess R, Santucci A, McTigue K, Fischer G, Kapoor W. Patient difficulty using tablet computers to screen in primary care. J Gen Intern Med. 2008;23(4):476–80.View ArticlePubMedPubMed CentralGoogle Scholar