Skip to main content
Download PDF

Digital health interventions for all? Examining inclusivity across all stages of the digital health intervention research process

Trials volume 25, Article number: 98 (2024) Cite this article

Abstract

Digital interventions offer many possibilities for improving health, as remote interventions can enhance reach and access to underserved groups of society. However, research evaluating digital health interventions demonstrates that such technologies do not equally benefit all and that some in fact seem to reinforce a “digital health divide.” By better understanding these potential pitfalls, we may contribute to narrowing the digital divide in health promotion. The aim of this article is to highlight and reflect upon study design decisions that might unintentionally enhance inequities across key research stages—recruitment, enrollment, engagement, efficacy/effectiveness, and retention. To address the concerns highlighted, we propose strategies including (1) the standard definition of “effectiveness” should be revised to include a measure of inclusivity; (2) studies should report a broad range of potential inequity indicators of participants recruited, randomized, and retained and should conduct sensitivity analyses examining potential sociodemographic differences for both the effect and engagement of the digital interventions; (3) participants from historically marginalized groups should be involved in the design of study procedures, including those related to recruitment, consent, intervention implementation and engagement, assessment, and retention; (4) eligibility criteria should be minimized and carefully selected and the screening process should be streamlined; (5) preregistration of trials should include recruitment benchmarks for sample diversity and comprehensive lists of sociodemographic characteristics assessed; and (6) studies within trials should be embedded to systematically test recruitment and retention strategies to improve inclusivity. The implementation of these strategies would enhance the ability of digital health trials to recruit, randomize, engage, and retain a broader and more representative population in trials, ultimately minimizing the digital divide and broadly improving population health.

Peer Review reports

Inequalities in health are widely recognized; an individual’s health status and access to and use of health care services are affected by a host of factors, from individual lifestyle to social and community networks, living and working conditions, and general socio-economic, cultural, and environmental conditions [1]. Although population health aims to narrow these inequalities, they may actually be reinforced through current research practices. For instance, study factors such as how research study information is communicated (e.g., language used) and the requirements for taking part in a study (e.g., time and literacy required), along with individual factors such as perceived benefits and harms as well as trust of both healthcare professionals and the research process impact which populations can and want to take part in research [2, 3] and often limit participation from minoritized groups. Although certain groups (e.g., those with health conditions preventing them from safely engaging in an intervention) may be intentionally excluded, the exclusion of minoritized populations in research evaluating intervention efficacy/effectiveness is frequently unintended. Indeed, if researchers use the “usual” research methods without actively reflecting on the implications of this choice on the conclusions that can be drawn from data collected in this manner [4], this oversight can produce broader harm as evidence for effectiveness may be lacking for populations that may need support the most, such as populations with low socioeconomic status (SES), that consistently experience less favorable health outcomes and higher mortality rates than high SES populations [5, 6]. For example, one review demonstrated that individual-focused interventions such as dietary counseling can widen social inequalities by being less effective in minoritized populations, potentially because they require more individual agency (which minoritized populations tend to have less of due to a host of factors, including structural barriers) [7].

The advancement of digital technologies offers many possibilities for improving health, as remote interaction can enhance reach and access to underserved groups of society [8, 9]. However, research evaluating the reach, engagement, efficacy/effectiveness, and retention of digital health interventions demonstrates that such technologies do not equally benefit all and that some in fact seem to reinforce a digital health divide [10, 11]. This divide is partially related to issues of access to digital technologies, which may be less accessible to marginalized populations (e.g., people living in housing without Internet access or with lower Internet bandwidth, people living with disabilities, or racial and ethnic minorities; see [12] for a discussion). Beyond this, the broader research process can contribute to widening the digital health divide [13]. Thus, the aim of this article is to highlight and reflect upon study design decisions that might unintentionally enhance inequities across key research stages—recruitment (i.e., hearing about the study and initially indicating interest), enrollment (i.e., successfully making it through the screening procedures), engagement (i.e., actively participating in the intervention through behaviors such as attendance at sessions and self-monitoring health behaviors), efficacy or effectiveness (i.e., achieving the desired health improvement in ideal and broader settings), and retention (i.e., remaining involved in the study and participating in all phases of data collection) (Fig. 1). By better understanding these potential pitfalls, we may contribute to narrowing the digital divide in health promotion [13].

Fig. 1
figure 1

How homogeneous digital intervention study samples are made

Full size image

Documenting the digital divide at the different stages of the research process

Recruitment

In order to accept or refuse participation in research, one has to become aware of existing research studies and be provided with opportunities to take part [14], such as through different means of offline or online recruitment methods that tend to reach different audiences [15]. In addition, there is often an inadequate representation of racial and ethnic minoritized groups in clinical trials, attributed to distrust and fear of research and the medical profession or medical institutions [16] resulting from historic racial injustices [17, 18]. For instance, Pratap et al. compared the proportion of racial and ethnic minoritized participants recruited into large remote digital health trials conducted in the USA to United States Census data, indicating that the proportion of racial and ethnic minoritized participants recruited was substantially lower than would be expected given the states’ population averages [19]; on average, 9.2% fewer participants who identified as African American/Black and 8.1% fewer participants who identified as Hispanic/Latino were recruited into clinical trials compared to Census data.

Enrollment

Once initial interest to participate in a research study or trial is established, the screening stage follows, wherein specific criteria are applied to each participant to determine eligibility for enrollment into the study. Eligibility criteria might refer to age, gender, or medical history and importantly often include access, previous experience with, or habitual use of digital technologies. Access to, and use of, digital devices and the Internet is often lower in those who are of older age, have lower education status, have lower socioeconomic status, are living in rural areas, or are from minoritized ethnic groups [20, 21]. The perceived ubiquity of Internet-enabled mobile phone ownership is frequently regarded as an opportunity to improve patient engagement in clinical trials of digital health technologies [22]; however, access is not universal, and participation in digital health trials usually requires access to either a smartphone, computer, or tablet along with a reliable Internet connection. Moreover, a recent meta-regression across 80 trials found that 30% of trials involved behavioral “run-ins” (i.e., periods in which potential participants had to demonstrate adherence to specific behavioral tasks) and up to four pre-enrollment steps, likely further limiting participation and adversely impacting generalizability of study findings [23].

Engagement

Once eligible participants who have access to appropriate devices and an Internet connection enroll in a trial, digital intervention engagement can be suboptimal (i.e., when participants do not use intervention tools at all or as consistently as recommended [11]). This is critical as intervention engagement often directly influences treatment outcomes; for instance, more frequent self-monitoring is associated with greater health behavior change, but certain population sub-groups are less likely to self-monitor [11], which may result in reduced intervention effectiveness [24, 25]. Subpar engagement is often attributed to a low level of digital health literacy (i.e., the competencies, skills, and knowledge required to navigate such technologies [18]); however, digital literacy requirements also depend on the usability and design of digital health technologies [18]. Several strategies such as personalization, gamification, and reminders have been linked to increased engagement [26]. For example, in a teenage pregnancy prevention program, engagement was highest for text messages that included quizzes [27]. In another study, a gamified intervention that included self-chosen, immediate step goals was found to increase physical activity engagement among individuals living in low-income neighborhoods, while other types of step goals within the gamified intervention did not result in consistent increases in physical activity [28]. Yet, other studies have shown that users report some of these strategies to not be helpful or even that they might even put them off using the app altogether [29]. If linked to social inequality indicators, these engagement strategy preferences may risk widening instead of reducing inequalities.

Efficacy and effectiveness

The aforementioned differences in engagement can translate into differential effects for digital health interventions on desired outcomes, both in tightly controlled efficacy trials and broader, “real world” effectiveness studies. For example, a systematic review found that digital interventions increased physical activity in participants from high-SES backgrounds but no impact was observed in participants from low-SES backgrounds [10]. Similarly, age and gender differences in efficacy and effectiveness have been reported for mobile interventions for the management of weight-related behaviors including diet [11], potentially due to factors such as intervention tailoring. It is likely that the digital divide is not limited to interventions promoting physical health, but also affects the outcomes of mental health interventions. For instance, in a recent study, White women benefited from digital cognitive behavioral therapy to reduce insomnia, while Black women did not [30]. Despite evidence that some digital health interventions produce differential outcomes between populations, reporting of social inequality indicators in relation to the effectiveness of digital mental health interventions is scarce, which complicates drawing conclusions [31]. Aside from differences in engagement, there may be other factors that result in differences in efficacy and effectiveness. For instance, the relationship between intention and behavior is moderated by education [32]; potentially, highly educated individuals might be more aware of how they can reach the desired levels of behavior or are better able to put suggestions into action (e.g., engage in exercise, consume fruits and vegetables).

Retention

Digital health studies often suffer from high attrition rates [33], greatly affecting external validity and ability to generalize study results to the broader target population. Predictors of dropout include younger age [19, 34,35,36], lower education levels [34, 36], lower SES [37], lower confidence in the ability to make target health behavior changes [36], and poorer health status at baseline [34]. Previous research indicates that study retention may be overall improved by traditional means (e.g., offering financial incentives [38, 39] and providing reminders of assessment tasks), but there are additional considerations related specifically to digital health research. In particular, the use of live (i.e., “human”) versus automated reminders [34], along with the use of shorter, more frequent assessment tasks versus longer questionnaire batteries [35] and passive measurement (i.e., through smartphone applications or wearable sensors) [40] can reduce participant burden and increase retention in longer-term studies. Moreover, assessment completion can be incentivized by the return of individualized feedback reports based on the data collected [40].

Implications and recommendations

In this paper, we have summarized inequalities that may unintentionally occur across all phases of the research process (i.e., recruitment, randomization, engagement, efficacy/effectiveness, and retention), that may over-represent the subgroups that may need the digital health interventions the least. There is a clear need to make consistent and concerted efforts to ensure digital health research is more inclusive. For starters, we propose the standard definition of ‘effectiveness’ in digital intervention trials should be revised to include a measure of inclusivity. At a minimum, studies should measure sociodemographic characteristics at first contact to be able to report in detail the representativeness of the participants recruited, randomized, and retained relative to the demographic composition of the target population [41]. The following set of additional recommendations could enhance inclusivity across various stages of digital health intervention studies.

Recruitment

  1. 1.

    Researchers should conduct formative work to generate ideas for recruitment modalities that will reach diverse population sub-groups.

  2. 2.

    The implementation of recruitment techniques should be paired with continuous data-driven evaluation of the success of each recruitment modality (both overall and for specific minoritized groups). Conducting these evaluations in a systematic way within clinical trials (and reporting outcomes in the literature), similar to the Medical Research Council’s Systematic Techniques for Assisting Recruitment to Trials project [42], can more broadly enhance knowledge of successful recruitment strategies for various sub-groups.

Enrollment

  1. 3.

    Researchers should select eligibility criteria carefully to avoid selecting criteria that are unnecessarily narrow.

  2. 4.

    Every effort should be made to use flexible screening protocols that focus on decreasing participant burden.

  3. 5.

    When appropriate, researcher should provide necessary technology free of charge to those who do not have it [43].

Engagement

  1. 6.

    Evaluation of potential subgroup differences in engagement is critical; researchers should conduct sensitivity analyses examining sociodemographic differences in digital intervention engagement during formative, intervention-development periods (i.e., not waiting just to examine differences in efficacy and effectiveness trials).

  2. 7.

    Attempts to establish specific strategies for improving engagement should be made by engaging participants from historically under-represented groups in the design of the intervention [44, 45], using established patient and public involvement techniques to establish challenges and potential solutions (see [46] for a case study).

Efficacy/effectiveness

  1. 8.

    Similar to the sensitivity analyses suggested previously, researchers should conduct sensitivity analyses examining sociodemographic differences in digital intervention efficacy and effectiveness (preferably pre-planned and powered for in the study design phase, but at the very least conducted as exploratory analyses) to enable scrutiny of how likely the intervention is to benefit those individuals who likely need the support the most. As an example, we present a sensitivity analysis from the Moms Fit 2 Fight trial, which tested a behavioral gestational weight gain intervention with digital components among military personnel and their family members [47]. While the overall effect of this intervention was significant, sensitivity analyses revealed that the intervention effect on gestational weight gain was only significant for women who identified as White; the intervention did not significantly benefit women who identified as Black or with those from other racial groups [48], suggesting a need to adapt the intervention specifically for these populations. If these sensitivity analyses were not conducted, key decision-makers may have not been aware that further consideration of this intervention may be necessary for particular populations.

  2. 9.

    Better transparency of reporting on the inclusivity of key outcomes will facilitate richer evaluation of digital health technologies effectiveness, cost-effectiveness, acceptability, and safety, conforming to Health Technology Assessment frameworks that guide the adoption of new innovations. Indeed, these frameworks such as EUnetHTA core evaluation model [49] place emphasis on the ethical and social aspects of health technologies, and researchers should provide sufficient evidence to inform the wholistic judgement of digital health interventions.

Retention

  1. 10.

    Although it is common to report sample demographics of study participants at baseline (i.e., describing the sample of individuals recruited and ultimately enrolled), we also recommend that researchers examine sociodemographic characteristics related to retention to detect subgroups that may not be fully represented in the outcome analyses.

  2. 11.

    Studies embedded within larger clinical trials should also be employed to systematically evaluate different retention strategies that may be required for varying subgroups of participants, similar to the methods in the PROMETHEUS program [50].

Reporting

  1. 12.

    Preregistration of trial protocols should include (1) recruitment benchmarks for sample diversity, (2) comprehensive lists of sociodemographic characteristics assessed, and (3) analysis plans that proactively examine the representativeness of the sample in terms of recruitment, randomization, engagement, efficacy/effectiveness, and retention. Suggestions for an extension of study pre-registration forms have already been proposed for health research more broadly that may serve as a blueprint [51].

  2. 13.

    When reporting sociodemographic characteristics of study samples for digital intervention trials (again not only reporting on participants screened and recruited/randomized but also sociodemographic characteristics related to retention), researchers should consider a wide range of potential inequality indicators beyond the commonly reported age and gender variables. Cochrane’s PROGRESS plus framework provides an overview of socio-demographic factors that have been associated with health disparities [45]. Indeed, several factors listed in this framework have been associated with the digital divide, as evidenced in the meta-analysis of Western et al. which showed that low SES populations (based on education attainment, income, and deprivation) did not benefit from digital physical activity interventions that high SES populations did [10]. For many other indicators (e.g., sexual orientation and location), it is more difficult to draw conclusions since research is sparse, highlighting the need for future studies to report these metrics [11].

Case study: inclusiveness in a weight management trial

As one example of how inclusiveness can be examined at various stages of research, the recent Fit & Quit clinical trial conducted in the USA focused on reducing the weight gain that occurs upon quitting smoking by randomizing participants into one of three digital weight management interventions. This study included 305 participants, and 67.9% identified as women and 43.3% identified as Black/African American, with a mean age of 54.3 (standard deviation = 11.6) [52]. Researchers found that radio advertisements (i.e., on gospel, rhythm, and blues stations) were the best method for recruiting participants who identified as Black, and online recruitment strategies (e.g., Google, Facebook advertisements) were most effective for non-urban participants [15]. When examining the likelihood of proceeding to randomization, individuals identifying as Black (OR = 0.53, 95% CI = 0.33–0.84) had lower odds of proceeding from recruitment to randomization, and individuals residing in non-urban areas (OR = 8.76, 95% CI = 1.15–66.77) had higher odds of being randomized. In terms of engagement, participants identifying as White self-weighed more frequently than participants identifying with other racial groups (M (SD) = 3.0 (1.9) days/week vs. 2.3 (1.8), p = 0.002), with no differences observed by rurality or income. Similarly, individuals identifying as White had higher session attendance (47%) compared to those who identified with other racial groups (39% of sessions, p = 0.01), with no differences by rurality or income. In addition, while 12-month retention was overall high (89%), individuals who identified as Black (16.7% vs. 6.3% for White participants, p = 0.005) and participants with lower incomes (i.e., < $50,000) (15.7% vs. 2.4% of participants with higher incomes, p = 0.0002) had significantly higher attrition. Finally, all three weight management interventions were successful at reducing or eliminating post-cessation weight gain, and 46.9% of participants were successful at quitting smoking, with no race, rurality, or income-based differences (ps > 0.05) [52]. Thus, future applications of this research should pay particular attention to facilitating randomization, intervention engagement, and retention among participants who identify as Black as well as retention among those with lower incomes.

Conclusions

In summary, digital health interventions have the potential to address disparities by improving reach and engagement among individuals who would not otherwise have access to in-person programs (e.g., rural populations, those with caregiving responsibilities or transportation issues, those with non-traditional work schedules, individuals with disabilities, and those with lower SES). To appropriately fulfill the promise of digital health interventions, however, researchers need to modify common research practices to improve broader knowledge of for whom, where, and why inequalities arise. Tackling these challenges, and aiming to recruit, randomize, engage, and retain a broader and more representative swath of the population in digital health trials, is crucial to the future of our field and broader support of health equity. It is our duty as researchers not to be part of the problem, but to be part of the solution.

Availability of data and materials

Not applicable.

References

  1. Whitehead M, Dahlgren G. What can be done about inequalities in health? Lancet. 1991;338(8774):1059–63.

    Article  CAS  PubMed  Google Scholar 

  2. Bell JAH, Balneaves LG. Cancer patient decision making related to clinical trial participation: an integrative review with implications for patients’ relational autonomy. Support Care Cancer. 2015;23(4):1169–96.

    Article  PubMed  Google Scholar 

  3. Houghton C, Dowling M, Meskell P, Hunter A, Gardner H, Conway A, et al. Factors that impact on recruitment to randomised trials in health care: a qualitative evidence synthesis. Cochrane Database Syst Rev. 2020(10):MR000045.

  4. Wallace N, O’Keeffe S, Gardner H, Shiely F. Underrecording and underreporting of participant ethnicity in clinical trials is persistent and is a threat to inclusivity and generalizability. J Clin Epidemiol. 2023;162:81–9.

    Article  PubMed  Google Scholar 

  5. Gong G, Phillips SG, Hudson C, Curti D, Philips BU. Higher US rural mortality rates linked to socioeconomic status, physician shortages, and lack of health insurance. Health Aff. 2019;38(12):2003–10.

    Article  Google Scholar 

  6. Steptoe A, Zaninotto P. Lower socioeconomic status and the acceleration of aging: an outcome-wide analysis. Proc Natl Acad Sci. 2020;117(26):14911–7.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. McGill R, Anwar E, Orton L, Bromley H, Lloyd-Williams F, O’Flaherty M, et al. Are interventions to promote healthy eating equally effective for all? Systematic review of socioeconomic inequalities in impact. BMC Public Health. 2015;15(1):457.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Barbosa W, Zhou K, Waddell E, Myers T, Dorsey ER. Improving access to care: telemedicine across medical domains. Annu Rev Public Health. 2021;42:463–81.

    Article  PubMed  Google Scholar 

  9. Rush KL, Singh S, Seaton CL, Burton L, Li E, Jones C, et al. Telehealth use for enhancing the health of rural older adults: a systematic mixed studies review. Gerontologist. 2022;62(10):e564–77.

    Article  PubMed  Google Scholar 

  10. Western MJ, Armstrong MEG, Islam I, Morgan K, Jones UF, Kelson MJ. The effectiveness of digital interventions for increasing physical activity in individuals of low socioeconomic status: a systematic review and meta-analysis. Int J Behav Nutr Phys Act. 2021;18(1):148.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Szinay D, Forbes CC, Busse H, DeSmet A, Smit ES, König LM. Is the uptake, engagement, and effectiveness of exclusively mobile interventions for the promotion of weight-related behaviors equal for all? A systematic review. Obes Rev. 2023;24(3):e13542.

  12. Jahnel T, Dassow H-H, Gerhardus A, Schüz B. The digital rainbow: digital determinants of health inequities. Digital Health. 2022;8:20552076221129092.

    Article  PubMed  PubMed Central  Google Scholar 

  13. König LM, Krukowski RA, Kuntsche E, Busse H, Gumbert L, Gemesi K, Neter E, Firdous Mohamed N, Ross KM, John-Akinola YO, Cooper R, Allmeta A, Macedo Silva A, Forbes CC, Western MJ. Reducing intervention- and research-induced inequalities to tackle the digital divide in health promotion. Int J Equity Health. in press.

  14. Ford JG, Howerton MW, Lai GY, Gary TL, Bolen S, Gibbons MC, et al. Barriers to recruiting underrepresented populations to cancer clinical trials: a systematic review. Cancer Interdisciplinary Int J Am Cancer Soc. 2008;112(2):228–42.

    Google Scholar 

  15. Pérez-Muñoz A, Horn TL, Graber J, Chowdhury SMR, Bursac Z, Krukowski RA. Recruitment strategies for a post cessation weight management trial: a comparison of strategy cost-effectiveness and sample diversity. Contemp Clin Trials Commun. 2022;30:101037.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Rodríguez-Torres E, González-Pérez MM, Díaz-Pérez C. Barriers and facilitators to the participation of subjects in clinical trials: an overview of reviews. Contemp Clin Trials Commun. 2021;23:100829.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Scharff DP, Mathews KJ, Jackson P, Hoffsuemmer J, Martin E, Edwards D. More than Tuskegee: understanding mistrust about research participation. J Health Care Poor Underserved. 2010;21(3):879.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Adedinsewo D, Eberly L, Sokumbi O, Rodriguez JA, Patten CA, Brewer LC. Health disparities, clinical trials, and the digital divide thematic reviews on forward thinking on clinical trials in clinical practice. Mayo Clin Proc. 2023;98(12):1875–87.

  19. Pratap A, Neto EC, Snyder P, Stepnowsky C, Elhadad N, Grant D, et al. Indicators of retention in remote digital health studies: a cross-study evaluation of 100,000 participants. NPJ Digital Med. 2020;3(1):21.

    Article  Google Scholar 

  20. Anderson M. Mobile technology and home broadband. Pew Research Center. 2019.

    Google Scholar 

  21. Roessler P. The mobile phone revolution and digital inequality: scope, determinants and consequences. Prosperity Comm Backgr Pap Ser. 2018(15):1–39.

  22. Venter W, Coleman J, Chan VL, Shubber Z, Phatsoane M, Gorgens M, et al. Improving linkage to HIV care through mobile phone apps: randomized controlled trial. JMIR Mhealth Uhealth. 2018;6(7):e8376.

    Article  Google Scholar 

  23. McVay MA, Lavoie HA, Rajoria M, Leong MC, Lou X, McMahon LN, et al. Pre-enrollment steps and run-ins in weight loss trials: a meta-regression. Am J Prev Med. 2023;64(6):910–7.

  24. Krukowski RA, Harvey-Berino J, Bursac Z, Ashikaga T, West DS. Patterns of success: online self-monitoring in a web-based behavioral weight control program. Health Psychol. 2013;32(2):164.

    Article  PubMed  Google Scholar 

  25. Harvey J, Krukowski R, Priest J, West D. Log often, lose more: Electronic dietary self-monitoring for weight loss. Obesity. 2019;27(3):380–4.

    Article  PubMed  Google Scholar 

  26. Saleem M, Kühne L, De Santis KK, Christianson L, Brand T, Busse H. Understanding engagement strategies in digital interventions for mental health promotion: scoping review. JMIR mental health. 2021;8(12):e30000.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Devine S, Leeds C, Shlay JC, Leytem A, Beum R, Bull S. Methods to assess youth engagement in a text messaging supplement to an effective teen pregnancy program. J Biomed Inform. 2015;56:379–86.

    Article  PubMed  Google Scholar 

  28. Patel MS, Bachireddy C, Small DS, Harrison JD, Harrington TO, Oon AL, et al. Effect of goal-setting approaches within a gamification intervention to increase physical activity among economically disadvantaged adults at elevated risk for major adverse cardiovascular events: the ENGAGE randomized clinical trial. JAMA cardiology. 2021;6(12):1387–96.

    Article  PubMed  Google Scholar 

  29. König LM, Attig C, Franke T, Renner B. Barriers to and facilitators for using nutrition apps: systematic review and conceptual framework. JMIR Mhealth Uhealth. 2021;9(6):e20037.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Kalmbach DA, Cheng P, Reffi AN, Seymour GM, Ruprich MK, Bazan LF, et al. Racial disparities in treatment engagement and outcomes in digital cognitive behavioral therapy for insomnia among pregnant women. Sleep Health. 2023;9(1):18–25.

    Article  PubMed  Google Scholar 

  31. Lehtimaki S, Martic J, Wahl B, Foster KT, Schwalbe N. Evidence on digital mental health interventions for adolescents and young people: systematic overview. JMIR Mental Health. 2021;8(4):e25847.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Schüz B, Li AS-W, Hardinge A, McEachan RR, Conner M. Socioeconomic status as a moderator between social cognitions and physical activity: systematic review and meta-analysis based on the Theory of Planned Behavior. Psychol Sport Exerc. 2017;30:186–95.

    Article  Google Scholar 

  33. Eysenbach G. The Law of Attrition. J Med Internet Res. 2005;7(1): e11.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Geraghty AWA, Torres LD, Leykin Y, Pérez-Stable EJ, Muñoz RF. Understanding attrition from international internet health interventions: a step towards global eHealth. Health Promot Int. 2012;28(3):442–52.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Van der Mispel C, Poppe L, Crombez G, Verloigne M, De Bourdeaudhuij I. A self-regulation-based eHealth intervention to promote a healthy lifestyle: investigating user and website characteristics related to attrition. J Med Internet Res. 2017;19(7):e241.

    Article  PubMed  PubMed Central  Google Scholar 

  36. McKay HG, Danaher BG, Seeley JR, Lichtenstein E, Gau JM. Comparing two web-based smoking cessation programs: randomized controlled trial. J Med Internet Res. 2008;10(5):e40.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Cheng P, Luik AI, Fellman-Couture C, Peterson E, Joseph CL, Tallent G, et al. Efficacy of digital CBT for insomnia to reduce depression across demographic groups: a randomized trial. Psychol Med. 2019;49(3):491–500.

    Article  PubMed  Google Scholar 

  38. Griffith Fillipo IR, Pullmann MD, Hull TD, Zech J, Wu J, Litvin B, et al. Participant retention in a fully remote trial of digital psychotherapy: Comparison of incentive types. Front Digit Health. 2022;4:963741.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Khadjesari Z, Murray E, Kalaitzaki E, White IR, McCambridge J, Thompson SG, et al. Impact and costs of incentives to reduce attrition in online trials: two randomized controlled trials. J Med Internet Res. 2011;13(1):e26.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Kolovson S, Pratap A, Duffy J, Allred R, Munson SA, Areán PA. Understanding participant needs for engagement and attitudes towards passive sensing in remote digital health studies. Int Conf Pervasive Comput Technol Healthc. 2020;2020:347–62.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Withall J, Greaves CJ, Thompson JL, de Koning JL, Bollen JC, Moorlock SJ, et al. The tribulations of trials: lessons learnt recruiting 777 older adults into REtirement in ACTion (REACT), a trial of a community, group-based active aging intervention targeting mobility disability. J Gerontol A. 2020;75(12):2387–95.

    Article  Google Scholar 

  42. Rick J, Graffy J, Knapp P, Small N, Collier DJ, Eldridge S, et al. Systematic techniques for assisting recruitment to trials (START): study protocol for embedded, randomized controlled trials. Trials. 2014;15(1):1–11.

    Article  Google Scholar 

  43. Kim ES, Bruinooge SS, Roberts S, Ison G, Lin NU, Gore L, et al. Broadening eligibility criteria to make clinical trials more representative: American Society of Clinical Oncology and Friends of Cancer Research Joint Research Statement. J Clin Oncol. 2017;35(33):3737.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Brewer LC, Fortuna KL, Jones C, Walker R, Hayes SN, Patten CA, et al. Back to the future: achieving health equity through health informatics and digital health. JMIR Mhealth Uhealth. 2020;8(1):e14512.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Miller SJ, Sly JR, Alcaraz KI, Ashing K, Christy SM, Gonzalez B, et al. Equity and behavioral digital health interventions: strategies to improve benefit and reach. Transl Behav Med. 2023;13(6):400–5.

    Article  PubMed  Google Scholar 

  46. Skovlund PC, Nielsen BK, Thaysen HV, Schmidt H, Finset A, Hansen KA, et al. The impact of patient involvement in research: a case study of the planning, conduct and dissemination of a clinical, controlled trial. Res Involvement Engagement. 2020;6(1):1–16.

    Article  Google Scholar 

  47. Fahey MC, Talcott GW, Bauer CMC, Bursac Z, Gladney L, Hare ME, et al. Moms fit 2 fight: Rationale, design, and analysis plan of a behavioral weight management intervention for pregnant and postpartum women in the US military. Contemp Clin Trials. 2018;74:46–54.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Estevez Burns R, Hare ME, Andres A, Klesges RC, Talcott GW, LeRoy K, et al. An interim analysis of a gestational weight gain intervention in military personnel and other TRICARE beneficiaries. Obesity. 2022;30(10):1951–62.

    Article  PubMed  Google Scholar 

  49. Kristensen FB, Mäkelä M, Neikter SA, Rehnqvist N, Håheim LL, Mørland B, et al. European network for Health Technology Assessment, EUnetHTA: planning, development, and implementation of a sustainable European network for Health Technology Assessment. Int J Technol Assess Health Care. 2009;25(S2):107–16.

    Article  PubMed  Google Scholar 

  50. Doherty L, Parker A, Arundel C, Clark L, Coleman E, Hewitt C, et al. PROMoting the use of studies within a trial (PROMETHEUS): results and experiences from a large programme to evaluate the routine embedding of recruitment and retention strategies within randomised controlled trials routinely. Res Methods Med Health Sci. 2023;4(3):113–22.

    Google Scholar 

  51. Gültzow T, Neter E, Zimmermann H. Making Research Look Like the World Looks: Introducing the’Inclusivity & Diversity Add-On for Preregistration Forms’ Developed During an EHPS2022 Pre-Conference Workshop. 2023.

    Google Scholar 

  52. Pebley K, Bursac Z, Klesges RC, Ebbert JO, Womack CR, Graber J, et al. A randomized controlled trial to reduce post-cessation weight gain. Int J Obes. 2023;47:1–8.

Download references

Acknowledgements

Not applicable.

Funding

The drafting of this paper was facilitated by a workshop supported by the University of Bayreuth Centre of International Excellence “Alexander von Humboldt.”

Author information

Authors and Affiliations

  1. Department of Public Health Sciences, School of Medicine, University of Virginia, PO Box 800765, Charlottesville, VA, 22908-0765, USA

    Rebecca A. Krukowski

  2. Department of Clinical & Health Psychology, College of Public Health & Health Professions, University of Florida, PO Box 100165, Gainesville, FL, 32610-0165, USA

    Kathryn M. Ross

  3. Department for Health, University of Bath, Claverton Down, Bath, BA2 7AY, UK

    Max J. Western

  4. Nuffield Department of Primary Care Health Sciences, University of Oxford, Woodstock Road, Oxford, UK

    Rosie Cooper

  5. Leibniz Institute for Prevention Research and Epidemiology- BIPS, Achterstraße 30, 28359, Bremen, Germany

    Heide Busse

  6. Hull York Medical School, University of Hull, Allam Medical Building, Cottingham Road, Hull, UK

    Cynthia Forbes

  7. Centre for Alcohol Policy Research, La Trobe University, Plenty Road and Kingsbury Drive, Melbourne, 3086 VIC, Australia

    Emmanuel Kuntsche

  8. University of Bayreuth, Fritz-Hornschuch-Straße 13, 95326, Kulmbach, Germany

    Anila Allmeta

  9. Instituto de Estudos Em Saúde Coletiva IESC/ Universidade Federal Do Rio de Janeiro /Leibiniz Science Campus Digital Public Health/Ministério Público Do Estado Do Rio de Janeiro, Rua das Bauhineas 200, Bl B 1602, Península, Barra da Tijuca, Rio de Janeiro, 22776-090, Brazil

    Anabelle Macedo Silva

  10. Department of Health Promotion and Education, Faculty of Public Health, College of Medicine, University of Ibadan, College of Medicine, Queen Elizabeth Road, UCH Campus, Ibadan, Nigeria

    Yetunde O. John-Akinola

  11. University of Bayreuth, Faculty of Life Sciences: Food, Nutrition and Health University of Vienna, Faculty of Psychology, Wächtergasse 1, 1010, Vienna, Austria

    Laura M. König

Authors
  1. Rebecca A. Krukowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kathryn M. Ross
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Max J. Western
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rosie Cooper
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Heide Busse
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Cynthia Forbes
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Emmanuel Kuntsche
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Anila Allmeta
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Anabelle Macedo Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Yetunde O. John-Akinola
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Laura M. König
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors wrote sections of the manuscript, and they approved the final manuscript.

Corresponding author

Correspondence to Rebecca A. Krukowski.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Krukowski, R.A., Ross, K.M., Western, M.J. et al. Digital health interventions for all? Examining inclusivity across all stages of the digital health intervention research process. Trials 25, 98 (2024). https://doi.org/10.1186/s13063-024-07937-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13063-024-07937-w

Trials

ISSN: 1745-6215