Skip to main content
Download PDF

Generalizability and reach of a randomized controlled trial to improve oral health among home care recipients: comparing participants and nonparticipants at baseline and during follow-up

Trials volume 23, Article number: 560 (2022) Cite this article

Abstract

Background

The generalizability of randomized controlled trials (RCTs) with a low response can be limited by systematic differences between participants and nonparticipants. This participation bias, however, is rarely investigated because data on nonparticipants is usually not available. The purpose of this article is to compare all participants and nonparticipants of a RCT to improve oral health among home care recipients at baseline and during follow-up using claims data.

Methods

Seven German statutory health and long-term care insurance funds invited 9656 home care recipients to participate in the RCT MundPflege. Claims data for all participants (n = 527, 5.5% response) and nonparticipants (n = 9129) were analyzed. Associations between trial participation and sex, age, care dependency, number of Elixhauser diseases, and dementia, as well as nursing, medical, and dental care utilization at baseline, were investigated using multivariable logistic regression. Associations between trial participation and the probability of (a) moving into a nursing home, (b) being hospitalized, and (c) death during 1 year of follow-up were examined via Cox proportional hazards regressions, controlling for baseline variables.

Results

At baseline, trial participation was positively associated with male sex (odds ratio 1.29 [95% confidence interval 1.08–1.54]), high (vs. low 1.46 [1.15–1.86]) care dependency, receiving occasional in-kind benefits to relieve caring relatives (1.45 [1.15–1.84]), having a referral by a general practitioner to a medical specialist (1.62 [1.21–2.18]), and dental care utilization (2.02 [1.67–2.45]). It was negatively associated with being 75–84 (vs. < 60 0.67 [0.50–0.90]) and 85 + (0.50 [0.37–0.69]) years old. For morbidity, hospitalizations, and formal, respite, short-term, and day or night care, no associations were found. During follow-up, participants were less likely to move into a nursing home than nonparticipants (hazard ratio 0.50 [0.32–0.79]). For hospitalizations and mortality, no associations were found.

Conclusions

For half of the comparisons, differences between participants and nonparticipants were observed. The RCT’s generalizability is limited, but to a smaller extent than one would expect because of the low response. Routine data provide a valuable source for investigating potential differences between trial participants and nonparticipants, which might be used by future RCTs to evaluate the generalizability of their findings.

Trial registration

German Clinical Trials Register DRKS00013517. Retrospectively registered on June 11, 2018.

Peer Review reports

Background

Randomized controlled trials (RCTs) can ensure high internal validity (i.e., the observed effect of an intervention is likely to be correct within the study population) and are therefore the gold standard for the evaluation of an intervention’s effectiveness [1]. However, their generalizability (also termed as external validity, i.e., the degree to which the observed effect of an intervention is correct within the entire target population) is often queried. Especially a low response makes RCTs prone to systematic differences between participants and nonparticipants [2, 3]. This is particularly relevant for complex interventions and in populations with difficult recruitment conditions, such as home care recipients, a population group typically suffering from multimorbidity, frailty, cognitive impairments, and polypharmacy, which has been increasing in most countries due to demographic aging [4,5,6]. If systematic differences between participants and nonparticipants exist, the intervention under study may have a different degree of effectiveness when offered as usual care. This participation bias (also known as non-response bias), however, is rarely investigated because data on the group of nonparticipants is usually not available in RCTs [7, 8].

Routinely collected data is increasingly used for the recruitment and outcome assessment of RCTs [9,10,11,12], which resulted in the development of the Consolidated Standards of Reporting Trials (CONSORT) extension for RCTs conducted using cohorts and routinely collected data (CONSORT-ROUTINE) [13]. For examining participation bias, administrative databases, electronic health records, and registries provide a valuable data source because they typically comprise information for all participants and nonparticipants [14,15,16,17,18,19,20]. For example, when recruiting individuals from a population being insured with a health insurance fund, the data comprises detailed information on demographic characteristics, diagnoses, and medical care utilization for all participants and nonparticipants, both at baseline and during follow-up. This is a major advantage over RCTs using short questionnaires or personal interviews to assess at least some cross-sectional information for a selective group of nonparticipants agreeing to be surveyed [21,22,23,24,25,26,27]. A further benefit of non-response analyses based on routinely collected data is that they offer the opportunity to identify the potential for improving the recruitment of RCTs and reach of interventions [19, 28].

The purpose of this article is to investigate whether participation in a RCT of a complex intervention to improve oral health among home care recipients is associated with sex, age, care dependency, and morbidity, as well as nursing, medical, and dental care utilization at baseline using claims data. Associations between trial participation and the probability of (a) moving into a nursing home, (b) being hospitalized, and (c) death during follow-up are also examined.

Methods

Randomized controlled trial

The two-arm RCT MundPflege was approved by the University of Bremen Ethics Committee on 21 March 2018 (reference number “MundPflege”). The detailed methodology and the main findings of the trial have been published elsewhere [6]. In brief, seven German statutory health and long-term care (LTC) insurance funds, which have the same mandate to finance medical and nursing care and follow the same rules, invited all eligible home care recipients to participate. Individuals were eligible if they were (i) insured with one of the cooperating insurance funds, (ii) at least 18 years old, (iii) a home care recipient according to the German Social Code Book XI (i.e., a person whose LTC dependency was legally evaluated by the Medical Advisory Service of the statutory health insurance and who receives LTC benefits in the home care setting), and (iv) a resident in the Free Hanseatic City of Bremen or the federal state of Lower Saxony. Between the first and second quarters of 2018, the insurance funds identified all insured persons who met these inclusion criteria. The invitation letter and one reminder were sent out by the insurance funds in the second quarter of 2018 (hereinafter referred to as the baseline quarter). The letters informed about the importance of oral health among LTC dependents and the design of the RCT. The invited persons were asked to provide informed consent for trial participation. They were briefed that they will be randomized either into the treatment (50%) or control (50%) group, whereby only the former will receive an oral health intervention by dentists in addition to usual care at t0. The intervention comprised an oral health assessment, dental treatment recommendations, and oral health education in the patient’s domicile. The invited persons were also informed that both the treatment and control group participants will receive a blinded outcome assessment at t1 [6].

Of 3.2 million insured persons, 9656 met the inclusion criteria and were invited to participate (Fig. 1) [6]. Among the invited home care recipients, 527 (5.5% response, 259 treatment group and 268 control group participants) participated and 9129 were nonparticipants. In the treatment group, 164 participants received the intervention between May 2018 and November 2019 (t0). The oral health status of 112 treatment group (56.8% dropouts) and 137 control group (48.9% dropouts) participants was assessed between January 2019 and November 2020 (t1). The comparison of the baseline characteristics between the treatment group and control group participants whose outcomes were assessed is described elsewhere and indicated no differences [6]. After a mean follow-up time of more than 1 year, although not statistically significant, the intervention tended to improve objective oral health, which was the primary outcome. Regarding the secondary outcomes, the intervention significantly reduced the prevalence of any periodontal problems, while no effectiveness was observed regarding subjective oral health and the prevalence of periodontitis [6].

Fig. 1
figure 1

Flow diagram

Full size image

To guide reporting of this article, we followed CONSORT-ROUTINE [13] (Additional file 1).

Claims data

For the non-response analysis, health and LTC insurance claims data were available for all participants and nonparticipants of the RCT. The information which home care recipients were invited to participate in the RCT was provided by the insurance funds, and the information which individuals participated in the trial was obtained from the Competence Center for Clinical Trials of the University of Bremen, the trial’s trusted third party. These data were linked to the claims data via a unique person identifier.

The analyzed claims data covered the period from the third quarter of 2017 to the second quarter of 2019. The baseline period of the non-response analysis comprised the three quarters before the baseline quarter and the baseline quarter. The follow-up period of the non-response analysis included the four quarters following the baseline quarter.

Besides insurance periods, the claims data comprised information on sex, year of birth, LTC grades, outpatient diagnoses, nursing care utilization, and medical and dental care utilization. The data on the year of birth was used to define the age in the baseline quarter. The data on LTC grades in the baseline quarter differentiates into 5 LTC grades (grade 1 = low limitations, grade 2 = substantial limitations, grade 3 = severe limitations, grade 4 = very severe limitations without special challenges for nursing care, grade 5 = very severe limitations with special challenges for nursing care) [29]. All diagnoses were coded according to the International Classification of Diseases, 10th Revision (ICD-10). The outpatient diagnoses were used to obtain information on the 31 diseases of the Elixhauser comorbidity measure and dementia in the baseline period. The Elixhauser comorbidity measure comprises a broad range of diseases that can be identified based on administrative data and is commonly used in health services research. The 31 diseases were defined using the ICD-10 codes provided by Quan et al. [30]. Dementia, which is not included in the Elixhauser list, was defined based on the ICD-10 codes used by Hoffmann et al. [31]. The information on nursing care utilization comprised information on occasional in-kind benefits to relieve caring relatives, formal care in addition to informal care, respite care by a substitute when relatives are on holiday or sick, short-term care in an institution, and day or night care in an institution during the baseline period [29]. It also comprised information on moves to a nursing home during the baseline and follow-up period. The data on medical care utilization included information on referrals by general practitioners (GPs) to medical specialists during the baseline period which served as a proxy for the coordination of medical care by GPs. It also comprised information on hospital admissions during the baseline and follow-up period. The data on dental care utilization during the baseline period was coded according to the German uniform assessment standard for dental care (BEMA) parts 1 to 5 (part 1 = conservative and surgical treatment and X-ray examinations, part 2 = treatment of injuries of the viscerocranium [jaw fracture], jaw joint disorders [occlusal splints], part 3 = orthodontic treatment, part 4 = systemic treatment of periodontal diseases, part 5 = provision of dentures and crowns) [32, 33]. Information on mortality during the baseline and follow-up period was obtained from insurance periods ended due to death. Information on terminated insurance coverages during the baseline and follow-up period was also obtained from insurance periods.

Statistical analysis

In the first step, all baseline characteristics available in the claims data were compared between participants and nonparticipants using the chi-square test (expected frequency in all cells of a contingency table ≥ 5), Fisher’s exact test (expected frequency in at least one cell of a contingency table < 5), and nonparametric Wilcoxon-Mann–Whitney test. For all comparisons, the absolute difference between participants and nonparticipants and the corresponding 95% confidence interval (CI) was also calculated. The distributions of sex (female, male), age groups (< 60 [adults], 60–74 [youngest-old], 75–84 [middle-old], 85 + years [oldest-old]), and LTC grades (1/2 [less severe limitations], 3 [severe limitations], 4/5 [very severe limitations]) as well as the mean age were compared. With respect to morbidity, the prevalence of the 31 Elixhauser diseases (0–2 [quartile 1], 3–4 [quartile 2], 5–6 [quartile 3], 7 + [quartile 4] diseases) and dementia were compared. With regard to nursing care, the proportions of home care recipients who utilized the different types of nursing care were compared between participants and nonparticipants. Regarding medical care, the proportions of individuals having a referral by a GP to a medical specialist and being hospitalized were compared. With respect to dental care, the proportion of individuals who utilized the different dental treatments was compared.

Second, a multivariable logistic regression was applied. In this regression, participation in the RCT served as the dependent variable. The independent variables were sex, age group, LTC grade, the unweighted number of Elixhauser diseases, dementia, occasional in-kind benefits to relieve caring relatives, formal care, referral by a GP to a medical specialist, hospital admission, and dental treatment. Respite care by a substitute, short-term care in an institution, and day or night care in an institution were not considered as independent variables to avoid multicollinearity.

In the third step, the proportions of home care recipients who (a) moved into a nursing home, (b) were admitted to a hospital, and (c) died during follow-up were compared between participants and nonparticipants using Kaplan–Meier plots. The outcomes move to a nursing home and death were investigated because moves to a nursing home and deaths were reasons for the dropout of participants in the RCT MundPflege. The outcome hospital admission was examined because hospital admissions made it difficult to schedule appointments for the provision of the intervention and for the outcome assessment. The time scale in the Kaplan–Meier plots was time in days since 1 July 2018 (i.e., the first day after the baseline quarter) until the day of the respective outcome, of the end of insurance coverage, or of the end of the follow-up (i.e., 30 June 2019).

Finally, three Cox proportional hazards regressions were applied. In these regressions, move to a nursing home, hospital admission, and death were considered as the dependent variable. In each model, the explanatory variable of interest was trial participation. The time scale was the same as in the Kaplan–Meier plots (i.e., time in days since 1 July 2018 until the day of the respective outcome, of the end of insurance coverage, or of the end of the follow-up on 30 June 2019). Cox regressions were performed including all independent variables considered in the logistic regression as covariates.

The statistical analysis was conducted using SAS 9.4 (SAS Institute Inc., Cary, NC, USA).

Results

Descriptive findings on sex, age, care dependency, and morbidity, as well as nursing, medical, and dental care utilization at baseline

The descriptive comparisons at baseline considered all 527 participants and 9129 nonparticipants (Fig. 1). The proportion of men was higher and the mean age lower among participants than among nonparticipants (Table 1). The percentage distribution of LTC grades also differed showing a greater proportion of higher LTC grades in the group of participants compared to the group of nonparticipants. For the number of Elixhauser diseases, no significant differences were found. Regarding the 31 single Elixhauser diseases, only the prevalence of paralysis and the prevalence of other neurological disorders differed (Additional file 1). With respect to dementia, the prevalence was lower among participants compared to nonparticipants.

Table 1 Baseline characteristics of the invited home care recipients
Full size table

Regarding nursing care utilization, the proportion of home care recipients who received occasional in-kind benefits to relieve caring relatives at baseline was higher in the group of participants compared to the group of nonparticipants. No significant differences between both groups were observed with regard to the proportions of individuals who utilized formal care, respite care by a substitute, short-term care in an institution, and day or night care in an institution. With respect to medical care utilization, the proportion of individuals with a referral by a GP to a medical specialist was higher among participants than among nonparticipants, whereas no significant difference was observed for hospital admissions. Dental treatments were utilized more frequently by participants than nonparticipants. Regarding the five single dental treatment categories, significant differences were found for the three categories.

Logistic regression of trial participation at baseline

The adjusted analysis confirmed associations between trial participation and sex, age group, LTC grade, occasional in-kind benefits to relieve caring relatives, referrals by GPs to medical specialists, and dental care utilization, whereas the association between trial participation and dementia was no longer statistically significant (Table 2). Men were more likely to participate than women (odds ratio [OR] 1.29 [95% CI 1.08–1.54]). Moreover, homecare recipients at the age of 75–84 years and 85 + years were less likely to participate compared to those at the age of < 60 years (OR 0.67 [95% CI 0.50–0.90] and 0.50 [95% CI 0.37–0.69], respectively). With respect to LTC grades, individuals with a LTC grade of 4/5 had a higher chance to participate than those with a LTC grade of 1/2 (OR 1.46 [95% CI 1.15–1.86]). Occasional in-kind benefits to relieve caring relatives (OR 1.45 [95% CI 1.15–1.84]), referrals by GPs to medical specialists (OR 1.62 [95% CI 1.21–2.18]), and dental care utilization (OR 2.02 [95% CI 1.67–2.45]) were all positively associated with trial participation.

Table 2 Logistic regression analysis of trial participation at baseline (n = 9656)
Full size table

Descriptive findings on moves to a nursing home, hospital admissions, and deaths during follow-up

The analysis on moves to a nursing home during follow-up comprised 508 participants and 8429 nonparticipants (Fig. 1). In this analysis, 19 participants and 700 nonparticipants were not considered because they died (4 participants and 352 nonparticipants), terminated their insurance coverage (6 nonparticipants), or moved into a nursing home (15 participants and 342 nonparticipants) before the follow-up period started. During a mean follow-up time of 330.6 days (SD 84.3 days), 3.7% of the participants and 8.3% of the nonparticipants moved into a nursing home (log rank p = 0.0002) (Fig. 2A).

Fig. 2
figure 2

A Probability of not moving into a nursing home during follow-up among participants (n = 508) and nonparticipants (n = 8429). B Probability of not being hospitalized during follow-up among participants (n = 523) and nonparticipants (n = 8771). C Probability of survival during follow-up among participants (n = 523) and nonparticipants (n = 8771)

Full size image

The analyses on hospital admissions and deaths during follow-up comprised 523 participants and 8771 nonparticipants (Fig. 1). In these analyses, only the 362 individuals who died or whose insurance coverage ended before the follow-up period started were not considered. The proportion of home care recipients who were admitted to a hospital during a mean follow-up time of 264.7 days (SD 130.2 days) was 46.8% among participants and 41.9% among nonparticipants (log rank p = 0.0431) (Fig. 2B). With respect to mortality, 10.1% of the participants and 12.0% of the nonparticipants died during a mean follow-up time of 342.5 days (SD 68.3 days) (log rank p = 0.2129) (Fig. 2C).

Cox regressions of moves to a nursing home, hospital admissions, and deaths during follow-up

The cox regressions of moves to a nursing home confirmed that participants were less likely to move into a nursing home during follow-up compared to nonparticipants (Table 3). The adjusted hazard ratio (HR) for moving into a nursing home for participants vs. nonparticipants was 0.50 (95% CI 0.32–0.79). Regarding hospital admissions, the difference between participants and nonparticipants was no longer significant (HR 1.12 [95% CI 0.98–1.27]). With respect to mortality, again, no significant difference was observed (HR 0.92 [95% CI 0.70–1.21]).

Table 3 Cox regression analyses of moves to a nursing home (n = 8937), hospital admissions (n = 9294), and deaths (n = 9294) during follow-up
Full size table

Discussion

This work compared all participants and nonparticipants of a RCT to improve oral health among home care recipients at baseline and during follow-up. At baseline, differences were found for sex, age, LTC dependency, occasional in-kind benefits to relieve caring relatives, referral by a GP to a medical specialist, and dental care utilization. The largest difference was observed for dental care utilization, whereby participants were more likely to have utilized dental treatments than nonparticipants. No differences were observed for the number of Elixhauser diseases, dementia, and hospitalizations, as well as formal, respite, short-term, and day or night care. During 1 year of follow-up, trial participants were less likely to move into a nursing home than nonparticipants, whereas for hospitalizations and mortality no associations were found.

Regarding generalizability, our non-response analysis indicates that the findings of the RCT MundPflege are limited, but to a smaller extent than one would expect because of the low response, which might be explained by our comparatively less personal recruitment strategy [6, 34]. We revealed statistically significant differences between trial participants and nonparticipants for almost half of the comparisons. A relevant limitation of generalizability might result from the difference in dental care utilization at baseline. Assuming that individuals without dental care utilization are more likely to have unmet dental care needs than their counterparts, the intervention might have significantly improved the oral health status if the response among this population group had been greater. Therefore, the intervention could be more effective when offered as usual care.

The findings from our non-response analysis suggest that RCTs with a low response are not generally unsuitable for evaluating the effectiveness of complex interventions to improve health in populations with difficult recruitment conditions. Compared to RCTs which are the gold standard for the evaluation of interventions, non-randomized studies might achieve a higher response but, at the same time, tend to be more prone to selection bias potentially leading to incorrect estimates of effectiveness [35, 36]. In cases of RCTs with a low response, provided that statistical power is still sufficient, their methodological advantages can be exploited when data for all trial participants and nonparticipants is available (e.g., from administrative databases, electronic health records, or registries) which allows a systematic investigation of participation bias [14, 18].

With respect to the reach of the RCT, our non-response analysis shows that men, younger individuals, and those with high LTC dependency, who receive occasional in-kind benefits to relieve caring relatives, with a referral by a GP to a medical specialist, and with dental care utilization, were significantly more likely to participate than their counterparts. In line with a previous study [37], individuals who are not moving into a nursing home during follow-up were also more likely to participate. Because most of the differences between participants and nonparticipants were rather small and no differences for morbidity, formal care, respite care by a substitute, short-term care in an institution, day or night care in an institution, and hospitalizations at baseline as well as hospitalizations and mortality during follow-up were found, there is only little potential for improving the reach of the intervention among certain population groups. As a comparatively big difference between participants and nonparticipants was observed regarding dental care utilization at baseline, an opportunity for improving the reach of the RCT would offer the attempt to promote the participation of home care recipients currently not utilizing dental care. A potential reason for the lower dental care utilization among nonparticipants could be that their relatives are not organizing contacts to dentists although, in Germany, at least one annual contact to a dentist is advised. To increase the reach of the RCT among home care recipients currently not utilizing dental care, involving their GPs and formal caregivers (if applicable) in the recruitment strategy could be a promising approach [6, 34, 38, 39].

Ideally, RCTs have both a high response and representative study population. Barriers and facilitators to trial participation have been described [40,41,42,43,44], and a variety of tools and strategies have been proposed to improve the recruitment to RCTs [45,46,47,48,49]. Furthermore, it could be worthwhile to conduct a pilot trial to estimate the response expectable in the main trial and identify the potential for improvement [50]. This would be in line with the Framework for Developing and Evaluating Complex Interventions by the UK Medical Research Council and National Institute of Health Research, which strongly recommends to assess the feasibility of any parts of a study including recruitment prior to conducting a full-scale evaluation [51]. Moreover, if data for participants and nonparticipants of the pilot trial is available, a non-response analysis in addition to qualitative research could help to inform the recruitment strategy of the main trial. This approach may help to avoid the frequently occurring premature discontinuation or extension of RCTs due to not achieving response targets [52, 53].

Overall, routinely collected data provide a valuable opportunity to improve RCTs. Compared to traditional RCTs, those using routine data have been shown to generate additional insights [54]. However, getting access to data from administrative databases, electronic health records, and registries is often not straightforward [55]. If such data is accessible and considered in a RCT, characteristics should be reported not only for trial participants, but also for nonparticipants, as explicitly stated in CONSORT-ROUTINE item 15 [13]. Overall, the entire CONSORT-ROUTINE checklist should be considered to improve the currently limited reporting quality of RCTs using routinely collected data [9,10,11].

Strengths and limitations

The main strength of this work is that a large amount of data was available enabling us to systematically compare all participants and nonparticipants of a RCT at baseline and during follow-up. There are, however, some important limitations. First, the cooperating insurance funds identified all individuals eligible for participation between the first and second quarters of 2018. The invitation letters were, however, sent out in the second quarter of 2018. Due to this time gap of a few weeks, some eligible individuals moved into a nursing home, terminated their insurance coverage, or died before they received the invitation. All these individuals were considered as nonparticipants in our baseline analysis although they actually had no chance to participate. However, this should not have significantly affected our findings assuming that most of these individuals would not have participated in the RCT even if they had received the invitation earlier due to their upcoming move to a nursing home, termination of insurance coverage, or death. A second limitation is that claims data was available only for the first year of follow-up and not for the whole study period of the RCT which, due to the COVID-19 pandemic, ended 17 months later. It is, however, unlikely that the comparison of participants and nonparticipants after more than 1 year of follow-up would have led to other results than our comparison during the first year of follow-up. Third, t0 and t1 of the RCT were not considered in the present non-response analysis because information on t0 was available only for the treatment group participants who received the intervention at t0 (no information for treatment group dropouts, control group participants, and nonparticipants). Furthermore, information on t1 was available only for the treatment and control group participants who received the outcome assessment at t1 (no information for treatment group and control group dropouts, and nonparticipants). We therefore compared participants and nonparticipants only at baseline (i.e., when the invitation letters were sent out) and during a fix time period covering the first year of follow-up (i.e., the year after the invitation letters were sent out). Fourth, the claims data analyzed was originally assessed for billing purposes and includes no information on functional limitations, frailty, restricted cognitive abilities, and medical and dental care needs. LTC grades, however, served as a proxy for this missing information. Furthermore, we were unable to compare outpatient procedures such as vaccinations or other preventive services as well as drug prescriptions between participants and nonparticipants because this information was not provided. Fifth, while the validity of information on sex, age, LTC grades, nursing care utilization, medical and dental care utilization, and mortality is considered high due to their relevance for billing purposes, that of morbidity obtained from outpatient diagnoses might be lower. Sixth, in our descriptive analyses, we applied a large number of statistical tests, and therefore, some of our statistically significant findings might be explained by the problem of multiple testing. However, our adjusted analyses confirmed most of the descriptive findings. Finally, the RCT was conducted in two of 16 German federal states and recruited insured persons from seven of currently 97 statutory health and LTC insurance funds. The representativeness of the invited individuals for all statutorily (90% of the population) and privately (10% of the population) insured persons in Germany as well as for persons in other health care systems might therefore be limited. For example, the proportion of statutorily and privately insured persons differs between regions, and differences between German insurance funds regarding age, sex, educational level, and the prevalence of chronic diseases have been described [56, 57]. However, compared to other recruitment strategies such as recruiting via selected practices, recruiting trial participants from insurance funds can generally be expected to result in a higher representativeness because a larger proportion of the entire target population can be invited.

Conclusions

For half of the comparisons, differences between participants and nonparticipants of a RCT to improve oral health among home care recipients were observed at baseline and during follow-up. The generalizability of the findings of the RCT are therefore limited, but to a smaller extent than one would expect because of the low response. Routinely collected data provide a valuable data source for the investigation of potential differences between trial participants and nonparticipants, which might be used by future RCTs to evaluate the generalizability of their findings.

Availability of data and materials

The data that support the findings of this study are available from the insurance funds (i.e., BAHN-BKK, Betriebskrankenkasse der Deutschen Bank, Daimler BKK, energie-BKK, Novitas BKK, pronova BKK, SBK Siemens Betriebskrankenkasse) and Competence Center for Clinical Trials of the University of Bremen, but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Therefore, data sharing is only feasible upon reasonable request and in collaboration with the authors, insurance funds, and the Competence Center for Clinical Trials of the University of Bremen.

Abbreviations

BEMA:

German uniform assessment standard for dental care

CI:

Confidence interval

GP:

General practitioner

HR:

Hazard ratio

ICD-10:

International Classification of Diseases, 10th Revision

LTC:

Long-term care

OR:

Odds ratio

RCT:

Randomized controlled trial

SD:

Standard deviation

References

  1. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332.

    Article  PubMed Central  PubMed  Google Scholar 

  2. McDonald AM, Knight RC, Campbell MK, Entwistle VA, Grant AM, Cook JA, et al. What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies. Trials. 2006;7:9.

    Article  PubMed Central  PubMed  Google Scholar 

  3. Rothwell PM. External validity of randomised controlled trials: “to whom do the results of this trial apply?” Lancet. 2005;365(9453):82–93.

    Article  PubMed  Google Scholar 

  4. Mühlhauser I, Lenz M, Meyer G. Development, appraisal and synthesis of complex interventions - a methodological challenge. Z Evid Fortbild Qual Gesundhwes. 2011;105(10):751–61.

    Article  PubMed  Google Scholar 

  5. Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, et al. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000;321(7262):694–6.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  6. Czwikla J, Herzberg A, Kapp S, Kloep S, Rothgang H, Nitschke I, et al. Effectiveness of a dental intervention to improve oral health among home care recipients: a randomized controlled trial. Int J Environ Res Public Health. 2021;18(17):9339.

    Article  PubMed Central  PubMed  Google Scholar 

  7. Bayley A, Stahl D, Ashworth M, Cook DG, Whincup PH, Treasure J, et al. Response bias to a randomised controlled trial of a lifestyle intervention in people at high risk of cardiovascular disease: a cross-sectional analysis. BMC Public Health. 2018;18:1092.

    Article  PubMed Central  PubMed  Google Scholar 

  8. Hestbech MS, Siersma V, Dirksen A, Pedersen JH, Brodersen J. Participation bias in a randomised trial of screening for lung cancer. Lung Cancer. 2011;73(3):325–31.

    Article  PubMed  Google Scholar 

  9. Mc Cord KA, Imran M, Rice DB, McCall SJ, Kwakkenbos L, Sampson M, et al. Reporting transparency and completeness in Trials: paper 2 - reporting of randomised trials using registries was often inadequate and hindered the interpretation of results. J ClinEpidemiol. 2021. https://doi.org/10.1016/j.jclinepi.2021.09.012.

  10. Imran M, Mc Cord K, McCall SJ, Kwakkenbos L, Sampson M, Fröbert O, et al. Reporting transparency and completeness in Trials: paper 3 - trials conducted using administrative databases do not adequately report elements related to use of databases. J ClinEpidemiol. 2021. https://doi.org/10.1016/j.jclinepi.2021.09.010.

  11. McCall SJ, Imran M, Hemkens LG, Mc Cord K, Kwakkenbos L, Sampson M, et al. Reporting transparency and completeness in Trials: paper 4 - reporting of randomised controlled trials conducted using routinely collected electronic records - room for improvement. J ClinEpidemiol. 2021. https://doi.org/10.1016/j.jclinepi.2021.09.011.

  12. Fitzpatrick T, Perrier L, Shakik S, Cairncross Z, Tricco AC, Lix L, et al. Assessment of long-term follow-up of randomized trial participants by linkage to routinely collected data: a scoping review and analysis. JAMA Netw Open. 2018;1(8):e186019.

    Article  PubMed Central  PubMed  Google Scholar 

  13. Kwakkenbos L, Imran M, McCall SJ, McCord KA, Fröbert O, Hemkens LG, et al. CONSORT extension for the reporting of randomised controlled trials conducted using cohorts and routinely collected data (CONSORT-ROUTINE): checklist with explanation and elaboration. BMJ. 2021;373:n857.

    Article  PubMed Central  PubMed  Google Scholar 

  14. Hemkens LG, Juszczak E, Thombs BD. Reporting transparency and completeness in Trials: paper 1: introduction - better reporting for disruptive clinical trials using routinely collected data. J Clin Epidemiol. 2021. https://doi.org/10.1016/j.jclinepi.2021.09.017.

  15. Badenbroek IF, Nielen MMJ, Hollander M, Stol DM, de Wit NJ, Schellevis FG. Characteristics and motives of non-responders in a stepwise cardiometabolic disease prevention program in primary care. Eur J Public Health. 2021;31(5):991–6.

    Article  PubMed Central  PubMed  Google Scholar 

  16. Rogers JR, Liu C, Hripcsak G, Cheung YK, Weng C. Comparison of clinical characteristics between clinical trial participants and nonparticipants using electronic health record data. JAMA Netw Open. 2021;4(4):e214732.

    Article  PubMed Central  PubMed  Google Scholar 

  17. Laursen PN, Holmvang L, Lønborg J, Køber L, Høfsten DE, Helqvist S, et al. Unreported exclusion and sampling bias in interpretation of randomized controlled trials in patients with STEMI. Int J Cardiol. 2019;289:1–5.

    Article  PubMed  Google Scholar 

  18. Mc Cord KA, Al-Shahi Salman R, Treweek S, Gardner H, Strech D, Whiteley W, et al. Routinely collected data for randomized trials: promises, barriers, and implications. Trials. 2018;19:29.

    Article  PubMed Central  PubMed  Google Scholar 

  19. Simmons RG, Lee YC, Stroup AM, Edwards SL, Rogers A, Johnson C, et al. Examining the challenges of family recruitment to behavioral intervention trials: factors associated with participation and enrollment in a multi-state colonoscopy intervention trial. Trials. 2013;14:116.

    Article  PubMed Central  PubMed  Google Scholar 

  20. Vind AB, Andersen HE, Pedersen KD, Jørgensen T, Schwarz P. Baseline and follow-up characteristics of participants and nonparticipants in a randomized clinical trial of multifactorial fall prevention in Denmark. J Am Geriatr Soc. 2009;57(10):1844–9.

    Article  PubMed  Google Scholar 

  21. Badenbroek IF, Nielen MMJ, Hollander M, Stol DM, Drijkoningen AE, Kraaijenhagen RA, et al. Mapping non-response in a prevention program for cardiometabolic diseases in primary care: how to improve participation? Prev Med Rep. 2020;19:101092.

    Article  PubMed Central  PubMed  Google Scholar 

  22. Hughes-Morley A, Young B, Hempel RJ, Russell IT, Waheed W, Bower P. What can we learn from trial decliners about improving recruitment? Qualitative study Trials. 2016;17:494.

    PubMed  Google Scholar 

  23. Normansell R, Holmes R, Victor C, Cook DG, Kerry S, Iliffe S, et al. Exploring non-participation in primary care physical activity interventions: PACE-UP trial interview findings. Trials. 2016;17:178.

    Article  PubMed Central  PubMed  Google Scholar 

  24. Bleidorn J, Bucak S, Gágyor I, Hummers-Pradier E, Dierks ML. Why do - or don’t - patients with urinary tract infection participate in a clinical trial? A qualitative study in German family medicine. Ger Med Sci. 2015;13:Doc17.

    PubMed Central  PubMed  Google Scholar 

  25. Jenkins V, Farewell V, Farewell D, Darmanin J, Wagstaff J, Langridge C, et al. Drivers and barriers to patient participation in RCTs. Br J Cancer. 2013;108(7):1402–7.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  26. Barnes M, Wiles N, Morrison J, Kessler D, Williams C, Kuyken W, et al. Exploring patients’ reasons for declining contact in a cognitive behavioural therapy randomised controlled trial in primary care. Br J Gen Pract. 2012;62(598):e371–7.

    Article  PubMed Central  PubMed  Google Scholar 

  27. Sharp L, Cotton SC, Alexander L, Williams E, Gray NM, Reid JM. Reasons for participation and non-participation in a randomized controlled trial: postal questionnaire surveys of women eligible for TOMBOLA (Trial Of Management of Borderline and Other Low-Grade Abnormal smears). Clin Trials. 2006;3(5):431–42.

    Article  CAS  PubMed  Google Scholar 

  28. Kerry SM, Morgan KE, Limb E, Cook DG, Furness C, Carey I, et al. Interpreting population reach of a large, successful physical activity trial delivered through primary care. BMC Public Health. 2018;18:170.

    Article  PubMed Central  PubMed  Google Scholar 

  29. Blümel M, Spranger A, Achstetter K, Maresso A, Busse R. Germany: health system review. Health Syst Transit. 2020;22(6):1–272.

    PubMed  Google Scholar 

  30. Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi JC, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43(11):1130–9.

    Article  PubMed  Google Scholar 

  31. Hoffmann F, Kaduszkiewicz H, Glaeske G, van den Bussche H, Koller D. Prevalence of dementia in nursing home and community-dwelling older adults in Germany. Aging Clin Exp Res. 2014;26(5):555–9.

    Article  PubMed  Google Scholar 

  32. Kassenzahnärztliche Bundesvereinigung. BEMA. Einheitlicher Bewertungsmaßstab für zahnärztliche Leistungen gemäß § 87 Abs. 2 und 2h SGB V. Anlage A zum BMV-Z. Stand: 1. Januar 2022. https://www.kzbv.de/kzbv-bema-2022-01-01.download.e0968ae7bc6c38481fe778346850ec62.pdf. Accessed 14 Jan 2022.

  33. Rothgang H, Müller R, Mundhenk R, Unger R. BARMER GEK Pflegereport 2014: Schwerpunkt: Zahnärztliche Versorgung Pflegebedürftiger. Asgard: Siegburg; 2014.

    Google Scholar 

  34. Kapp S, Herzberg A, Czwikla J, Schmelzer C, Hoffmann F, Nitschke I, et al. Verbesserung der Mundgesundheit bei Pflegebedürftigen in häuslicher Pflege: Welche weiteren Schritte sind notwendig? G&S Gesundheits- und Sozialpolitik. 2020;74(3):33–8.

    Article  Google Scholar 

  35. Schünemann H. All evidence is real world evidence. https://blogs.bmj.com/bmj/2019/03/29/holger-j-schunemann-all-evidence-is-real-world-evidence. Accessed 19 Jan 2022.

  36. Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ. What is “quality of evidence” and why is it important to clinicians? BMJ. 2008;336(7651):995–8.

    Article  PubMed Central  PubMed  Google Scholar 

  37. Vass M, Avlund K, Hendriksen C. Randomized intervention trial on preventive home visits to older people: baseline and follow-up characteristics of participants and non-participants. Scand J Public Health. 2007;35(4):410–7.

    Article  PubMed  Google Scholar 

  38. Ho BV, Weijenberg RAF, van der Maarel-Wierink CD, Visscher CM, van der Putten GJ, Scherder EJA, et al. Effectiveness of the implementation project ‘Don’t forget the mouth!’ of community dwelling older people with dementia: a prospective longitudinal single-blind multicentre study protocol (DFTM!). BMC Oral Health. 2019;19:91.

    Article  PubMed Central  PubMed  Google Scholar 

  39. Lewis A, Kitson A, Harvey G. Improving oral health for older people in the home care setting: an exploratory implementation study. Australas J Ageing. 2016;35(4):273–80.

    Article  PubMed  Google Scholar 

  40. Rodríguez-Torres E, González-Pérez MM, Díaz-Pérez C. Barriers and facilitators to the participation of subjects in clinical trials: an overview of reviews. Contemp Clin Trials Commun. 2021;23:100829.

    Article  PubMed Central  PubMed  Google Scholar 

  41. Naidoo N, Nguyen VT, Ravaud P, Young B, Amiel P, Schanté D, et al. The research burden of randomized controlled trial participation: a systematic thematic synthesis of qualitative evidence. BMC Med. 2020;18:6.

    Article  PubMed Central  PubMed  Google Scholar 

  42. Sheridan R, Martin-Kerry J, Hudson J, Parker A, Bower P, Knapp P. Why do patients take part in research? An overview of systematic reviews of psychosocial barriers and facilitators. Trials. 2020;21:259.

    Article  PubMed Central  PubMed  Google Scholar 

  43. Walsh E, Sheridan A. Factors affecting patient participation in clinical trials in Ireland: a narrative review. Contemp Clin Trials Commun. 2016;3:23–31.

    Article  PubMed Central  PubMed  Google Scholar 

  44. Ross S, Grant A, Counsell C, Gillespie W, Russell I, Prescott R. Barriers to participation in randomised controlled trials: a systematic review. J Clin Epidemiol. 1999;52(12):1143–56.

    Article  CAS  PubMed  Google Scholar 

  45. Treweek S, Pitkethly M, Cook J, Fraser C, Mitchell E, Sullivan F, et al. Strategies to improve recruitment to randomised trials. Cochrane Database Syst Rev. 2018;2(2):Mr000013.

    PubMed  Google Scholar 

  46. Elliott D, Husbands S, Hamdy FC, Holmberg L, Donovan JL. Understanding and improving recruitment to randomised controlled trials: qualitative research approaches. Eur Urol. 2017;72(5):789–98.

    Article  PubMed  Google Scholar 

  47. Realpe A, Adams A, Wall P, Griffin D, Donovan JL. A new simple six-step model to promote recruitment to RCTs was developed and successfully implemented. J Clin Epidemiol. 2016;76:166–74.

    Article  PubMed Central  PubMed  Google Scholar 

  48. Paramasivan S, Strong S, Wilson C, Campbell B, Blazeby JM, Donovan JL. A simple technique to identify key recruitment issues in randomised controlled trials: Q-QAT - quanti-qualitative appointment timing. Trials. 2015;16:88.

    Article  PubMed Central  PubMed  Google Scholar 

  49. Caldwell PH, Hamilton S, Tan A, Craig JC. Strategies for increasing recruitment to randomised controlled trials: systematic review. PLoS Med. 2010;7(11):e1000368.

    Article  PubMed Central  PubMed  Google Scholar 

  50. Hubbard G, Campbell A, Davies Z, Munro J, Ireland AV, Leslie S, et al. Experiences of recruiting to a pilot trial of Cardiac Rehabilitation In patients with Bowel cancer (CRIB) with an embedded process evaluation: lessons learned to improve recruitment. Pilot Feasibility Stud. 2015;1:15.

    Article  PubMed Central  PubMed  Google Scholar 

  51. Skivington K, Matthews L, Simpson SA, Craig P, Baird J, Blazeby JM, et al. A new framework for developing and evaluating complex interventions: update of medical research council guidance. BMJ. 2021;374:n2061.

    Article  PubMed Central  PubMed  Google Scholar 

  52. Amstutz A, Schandelmaier S, Frei R, Surina J, Agarwal A, Olu KK, et al. Discontinuation and non-publication of randomised clinical trials supported by the main public funding body in Switzerland: a retrospective cohort study. BMJ Open. 2017;7(7):e016216.

    Article  PubMed Central  PubMed  Google Scholar 

  53. Sully BG, Julious SA, Nicholl J. A reinvestigation of recruitment to randomised, controlled, multicenter trials: a review of trials funded by two UK funding agencies. Trials. 2013;14:166.

    Article  PubMed Central  PubMed  Google Scholar 

  54. Mc Cord KA, Ewald H, Agarwal A, Glinz D, Aghlmandi S, Ioannidis JPA, et al. Treatment effects in randomised trials using routinely collected data for outcome assessment versus traditional trials: meta-research study. BMJ. 2021;372:n450.

    Article  PubMed Central  PubMed  Google Scholar 

  55. Macnair A, Love SB, Murray ML, Gilbert DC, Parmar MKB, Denwood T, et al. Accessing routinely collected health data to improve clinical trials: recent experience of access. Trials. 2021;22:340.

    Article  PubMed Central  PubMed  Google Scholar 

  56. Hoffmann F, Koller D. Different regions, differently insured populations? Sociodemographic and health-related differences between insurance funds. Gesundheitswesen. 2017;79(1):e1-9.

    PubMed  Google Scholar 

  57. Hoffmann F, Icks A. Structural differences between health insurance funds and their impact on health services research: results from the Bertelsmann Health-Care Monitor. Gesundheitswesen. 2012;74(5):291–7.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

Our special thanks go to the BKK Dachverband and the following insurance funds for the provision of claims data: BAHN-BKK, Betriebskrankenkasse der Deutschen Bank, Daimler BKK, energie-BKK, Novitas BKK, pronova BKK, and SBK Siemens Betriebskrankenkasse. Furthermore, we are grateful to the German Society for Gerodontology for their cooperation. Finally, we would like to thank Corinna Burfeindt for her valuable support.

Funding

Open Access funding enabled and organized by Projekt DEAL. The study was funded by the Innovation Committee of the Federal Joint Committee (grand number “01NVF17003”). The funding body was not involved in the design of the study; collection, analysis, and interpretation of the data; and writing of the manuscript.

Author information

Authors and Affiliations

  1. Department of Health Services Research, Carl von Ossietzky University of Oldenburg, Ammerländer Heerstraße 114-118, 26129, Oldenburg, Germany

    Jonas Czwikla & Falk Hoffmann

  2. Department of Health, Long-Term Care and Pensions, SOCIUM Research Center on Inequality and Social Policy, University of Bremen, Mary-Somerville-Straße 5, 28359, Bremen, Germany

    Jonas Czwikla, Alexandra Herzberg, Sonja Kapp & Heinz Rothgang

  3. High-Profile Area of Health Sciences, University of Bremen, Bibliothekstraße 1, 28359, Bremen, Germany

    Jonas Czwikla, Alexandra Herzberg, Sonja Kapp, Stephan Kloep & Heinz Rothgang

  4. Competence Center for Clinical Trials, University of Bremen, Linzer Straße 4, 28359, Bremen, Germany

    Stephan Kloep

  5. Division of Gerodontology, Clinic of Prosthetic Dentistry and Dental Materials Science, University Medical Center, Liebigstraße 10-14, 04103, Leipzig, Germany

    Ina Nitschke

  6. Clinic of General, Special Care and Geriatric Dentistry, Center of Dental Medicine, University of Zurich, Plattenstraße 11, CH-8032, Zurich, Switzerland

    Ina Nitschke

  7. Special Care- and Geriatric Dentistry, Städtisches Klinikum Harlaching München, Sanatoriumsplatz 2, 81545, Munich, Germany

    Cornelius Haffner

Authors
  1. Jonas Czwikla
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexandra Herzberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sonja Kapp
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stephan Kloep
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Heinz Rothgang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ina Nitschke
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Cornelius Haffner
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Falk Hoffmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JC, HR, and FH conceptualized and designed the study. AH, SKa, SKl, IN, and CH organized the acquisition of the data. JC and FH analyzed the data. All authors interpreted the results. JC and FH wrote the first draft of the manuscript which was critically revised by AH, SKa, SKl, HR, IN, and CH. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Jonas Czwikla.

Ethics declarations

Ethics approval and consent to participate

The study was approved by the University of Bremen Ethics Committee on 21 March 2018 (reference number “MundPflege”). Informed consent for participation in the RCT was obtained from all participants or, if applicable, their legal guardians. Since the non-response analysis was based only on pseudonymous claims data, informed consent for this part of the study was not required by law.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1:

Table S1. Baseline morbidity of the invited home care recipients. Table S2. Combined CONSORT 2010 and CONSORT-ROUTINE Checklist.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Czwikla, J., Herzberg, A., Kapp, S. et al. Generalizability and reach of a randomized controlled trial to improve oral health among home care recipients: comparing participants and nonparticipants at baseline and during follow-up. Trials 23, 560 (2022). https://doi.org/10.1186/s13063-022-06470-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13063-022-06470-y

Keywords

Trials

ISSN: 1745-6215