- Research
- Open access
- Published:
Factors associated with reporting of the Prevention of Falls Network Europe (ProFaNE) core outcome set domains in randomized trials on falls in older people: a citation analysis and correlational study
Trials volume 23, Article number: 710 (2022)
Abstract
Background
Core outcome sets are advocated as a means to standardize outcome reporting across randomized controlled trials (RCTs) and reduce selective outcome reporting. In 2005, the Prevention of Falls Network Europe (ProFaNE) published a core outcome set identifying five domains that should be measured and reported, at a minimum, in RCTs or meta-analysis on falls in older people. As reporting of all five domains of the ProFaNE core outcome set has been minimal, we set out to investigate factors associated with reporting of the ProFaNE core outcome set domains in a purposeful sample of RCTs on falls in older people.
Methods
We conducted a systematic citation analysis to identify all reports of RCTs focused on falls in older people that cited the ProFaNE core outcome set between October 2005 and July 2021. We abstracted author-level, study-level, and manuscript-level data and whether each domain of the ProFaNE core outcome set was reported. We used penalized LASSO regression to identify factors associated with the mean percentage of ProFaNE core outcome set domains reported.
Results
We identified 85 eligible reports of RCTs. Articles were published between 2007 and 2021, described 75 unique RCTs, and were authored by 76 unique corresponding authors. The percentage of ProFaNE core outcome set domains reported ranged from 0 to 100%, with a median of 40% and mean (standard deviation, SD) of 52.2% (25.1). RCTs funded by a non-industry source reported a higher mean percentage of domains than RCTs without a non-industry funding source (estimated mean difference = 17.5%; 95% confidence interval (CI) 1.8–33.2). RCTs examining exercise (15.4%; 95% CI 1.9–28.9) or multi-component/factorial (17.4%; 95% CI 4.7–30.1) interventions each reported a higher mean percentage of domains than RCTs examining other intervention types.
Conclusions
We found that RCTs funded by at least one non-industry source, examining exercise or multi-component/factorial interventions, reported the highest percentages of ProFaNE core outcome set domains. Findings may help inform strategies to increase the impact of the ProFaNE core outcome set. Ultimately, this may lead to enhanced knowledge of the effectiveness and safety of interventions to prevent and/or manage falls in older people.
Background
About 20–30% of community-dwelling people over the age of 65 fall annually [1,2,3]. Falls are the leading cause of injury-related hospitalizations and deaths in older people [4,5,6]. Serious injuries arising from falls include cuts or lacerations, soft tissue injuries, fractures, and traumatic brain injuries [7,8,9]. Even falls that do not result in physical harm may have debilitating effects on an older person’s health and quality of life, by contributing to fear of falling, activity restriction, reduced social participation, and depressive symptoms [10,11,12,13,14,15]. In addition to physical and psychological sequelae, the economic costs of falls are substantial. In Canada, falls cost an estimated $8 million annually [4]; in the United Kingdom (UK), the cost of falls in older people was estimated at £981 million in 1999 [16]; and in the United States (US), medical costs attributable to falls were estimated to be near $50 billion in 2015 [17].
The clinical effectiveness of interventions to prevent falls and their debilitating effects is best measured using systematic reviews and meta-analyses of quasi-randomized or randomized controlled trials (RCTs), as they yield the most precise effect estimates, optimize statistical power, and can measure and examine reasons for heterogeneity across studies [18]. Although there is strong evidence that exercise is effective at preventing falls in community-dwelling older adults (e.g. [19, 20]), we have limited understanding of the effects of many interventions on rates of serious injury and overall quality of life [20]. This is partly due to inconsistent outcome reporting across RCTs, as the body of evidence may be too small or cannot be synthesized when outcomes are not evaluated consistently. For example, in a recent systematic review and network meta-analysis examining fall-prevention interventions for adults aged 65 years or older in all settings, less than a quarter of eligible RCTs were included in network meta-analyses on injurious falls (54 of 283 RCTs, 19%), fractures (68 of 283 RCTs, 24%), and hip fractures (39 of 283 RCTs, 14%), while fewer than 1% of eligible RCTs were included in a network meta-analysis on quality of life (2 of 283 RCTs) [20]. Additionally, a high percentage of RCTs (183 of 283 RCTs, 65%) had an unclear risk of bias for selective outcome reporting [20], which threatens the validity of existing and future systematic reviews and meta-analysis, and can undermine our understanding of the true effectiveness and safety of interventions [21].
A core outcome set (COS) is an agreed standardized collection of outcomes that should be measured and reported, at a minimum, in a specific area of health [22, 23]. COS are advocated as means to standardize outcome reporting across trials, facilitate knowledge synthesis, minimize the risk of bias from selective outcome reporting, ensure important outcomes are measured and reported, and consequently reduce research waste and increase the impact and benefits of research [24,25,26,27]. COS often include outcome measurement instruments, which provide details on how to measure and report each outcome included in the COS [28].
In 2005, Lamb and colleagues, on behalf of the Prevention of Falls Network Europe (ProFaNE), published a COS identifying five core domains that should be measured and reported, at a minimum, in future randomized trials and meta-analysis focused on falls in community-dwelling older people: falls, injuries, psychological consequences of falling, generic health-related quality of life (HRQoL), and physical activity [29]. The ProFaNE group explained that “the desirable profile of an intervention is one that improves activity and confidence while reducing falling. HRQoL is likely to capture unanticipated effects of an intervention in terms of general, emotional, and social health.” (p.1619) [29].
Awareness of the ProFaNE COS is high, with more than 800 citations by researchers spanning multiple countries. Even so, reporting of all five domains of the ProFaNE COS has been minimal in RCTs examining the effectiveness or safety of interventions to prevent and/or manage falls in older people [30]. In the first decade after its publication, the ProFaNE COS was cited in 34 peer-reviewed articles reporting the results of RCTs focused on falls in older people [30]. Of these, one (3%) reported the effects of an intervention on all five domains of the ProFaNE COS [30]. Falls were most frequently reported (32 of 34, 94%), followed by physical injuries (16 of 34, 47%), while psychological consequences of falling (7 of 34, 21%), physical activity (8 of 34, 24%), and HRQoL (8 of 34, 24%) were each reported in less than a quarter of RCTs [30].
Several studies have been conducted on the impact of COS [31,32,33,34]. Most research on the implementation of COS has focused on understanding barriers and facilitators to their adoption (i.e. uptake), defined as the initial intention or decision to try to employ an innovation or evidence-based practice [35]. Hughes et al. surveyed chief investigators of funded and non-funded applications submitted to the United Kingdom (UK) National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Programme to investigate their reasons for deciding whether or not to include a COS in their proposals for a RCT [31]. Reasons cited by chief investigators for deciding to include a COS were the belief in the benefits of measuring a COS, having previous experience with the development of COS, and journal publication requirements [31]. Reasons cited for not including a COS in their proposals were being aware of conflicting guidance from granting agencies and patient and public partners, and a perceived need to measure and report the same outcomes as past trials, when the COS did not include those outcomes [31]. Wallace et al. surveyed trialists to understand their reasons for use and non-use of the Research Outcome Measurement in Aphasia (ROMA) COS, who identified belief in the benefits of using the ROMA COS as a facilitator, and lack of external incentives, collegial encouragement, and monitoring systems as barriers to using the ROMA COS [34]. Similarly, Fletcher et al. surveyed trialists to identify barriers to adoption of the hip fracture COS [33]. Key reasons for non-adoption included lack of awareness of the hip fracture COS during trial design and perceived inappropriateness of the COS for specific populations and intervention types [33].
Less attention has been paid to investigating factors influencing the fidelity of COS implementation, defined as the degree to which an intervention was implemented as it was prescribed in the original protocol or as it was intended by the programme developers [35]. Implementation fidelity has been conceptualized in terms of adherence to the programme protocol, dose or amount of the programme delivered, and quality of the programme delivered [35]. In other words, COS must first be adopted and then implemented to some degree of fidelity. A better understanding of factors influencing both the adoption (i.e. uptake) and fidelity of implementation (i.e. adherence) of COS is needed to develop and tailor strategies to realize their impact.
Despite this need, we have limited understanding of barriers and enablers to implementation of the ProFaNE COS. As we understand it, no research has investigated factors governing the extent to which the ProFaNE COS has been implemented with fidelity in RCTs focused on falls in older people. This knowledge may help inform strategies to increase the impact of the ProFaNE COS, which in turn should lead to more consistent outcome reporting across RCTs and, ultimately, better understanding of the effectiveness and safety of interventions to prevent and/or manage falls in older people. Thus, by way of a systematic citation analysis and correlational study in a purposeful sample of RCTs examining the safety or effectiveness of interventions to prevent and/or manage falls in older people, our objectives are twofold: (i) to identify factors associated with the percentage of ProFaNE COS domains reported and (ii) to identify factors associated with the likelihood of reporting each domain of the ProFaNE COS.
Methods
Study design
We conducted a systematic citation analysis of the ProFaNE COS and correlational study to identify factors associated with reporting of the ProFaNE COS in a purposeful sample of RCTs examining the safety or effectiveness of interventions to prevent and/or manage falls in older people.
Theoretical underpinning
Our investigation into factors associated with reporting of ProFaNE COS domains was informed by the Consolidated Framework for Implementation Research (CFIR) [36]. Some associations were hypothesis-driven; however, as little is known about factors influencing reporting of COS in RCTs, others were driven by exploration, but based on the assumption that characteristics of the inner setting, outer setting, intervention, and individuals involved are possible determinants of knowledge use [36].
Outcomes
Our primary outcome was the percentage of ProFaNE COS domains reported. Secondary outcomes included whether falls were reported (yes, no), injuries were reported (yes, no), psychological consequences of falling were reported (yes, no), HRQoL was reported (yes, no), and physical activity was reported (yes, no).
Explanatory variables and study hypotheses
We examined associations between author-level, study-level, and manuscript-level factors and primary and secondary outcomes.
Author-level variables
We included a single author-level variable—whether the geographic affiliation of the corresponding author was European vs. other. The ProFaNE COS was developed on behalf of a European network [29]. The CFIR states that the perception of stakeholders about whether the intervention or evidence-informed practice (here, the COS) is internally or externally developed can influence implementation success [36]. It is possible investigators within Europe may be more likely to perceive the ProFaNE COS was internally developed than investigators outside Europe. We hypothesized that articles authored by a corresponding author affiliated with one or more European institutions would report a greater percentage of ProFaNE COS domains than articles authored by a corresponding author based outside of Europe.
Study-level variables
Study-level variables included the setting (community vs. institution or combined), number of trial centres (single, multiple, not reported), number of trial arms (two vs. three or more), funding source(s) (at least one source of non-industry vs. no non-industry source(s) or not reported/unclear), intervention type (exercise, multi-component/factorial, other), mean age of the sample at baseline, percentage of the sample that was female at baseline, whether the population of interest had a specific disease diagnosis (yes, no), fall risk of participants at baseline (at risk, at high risk), sample size, and length of follow-up of the primary outcome variable (12 months or more vs. less than 12 months).
The CFIR states that the compatibility of the intervention or evidence-informed practice (the COS) with the local context, or inner setting, may influence the success of implementation [36]. Lamb et al. specified the selection of outcomes in the ProFaNE COS should focus on community-dwelling populations [29]. We hypothesized that articles reporting the results of RCTs conducted in community settings would report a greater percentage of domains than articles reporting the results of RCTs set in institutions, such as hospitals, long-term care or nursing homes, and assisted-living facilities.
Outcomes included in the ProFaNE COS are generally measured by self-report and were selected to accommodate “nearly all community-dwelling older people” [29]. Copsey et al. reported that some RCTs citing the ProFaNE COS restricted their population of interest to people with dementia or cognitive impairment, Parkinson’s disease, or stroke [30]. The measurement and validity of some or all of these outcomes in persons living with specific disease diagnoses, especially cognitive impairment or dementia [37], may require additional considerations when compared to the general population of community-dwelling older adults. Previously, in an international survey of trialists on their reasons for non-adoption of the hip fracture COS, more than half (54%) of respondents stated the COS was not appropriate for trials on people living with cognitive impairment and needed revision to accommodate this population [33]. We hypothesized that articles reporting the results of RCTs in populations with specific disease diagnoses, including but not limited to cognitive impairment or dementia, would report a smaller percentage of ProFaNE COS domains than articles reporting the results of RCTs in a general population of older adults.
Manuscript-level variables
Manuscript-level variables included the year of publication, the type of journal (specialist vs. general), and whether the manuscript cited the ProFaNE COS in the introduction (yes, no), methods (yes, no), and discussion (yes, no) sections. We hypothesized that articles citing the ProFaNE COS in the methods section would report a greater percentage of domains than articles that did not cite the ProFaNE COS in the methods section.
Population of interest and eligibility criteria
Our aim was to investigate factors governing the extent to which the ProFaNE COS was implemented with fidelity as intended by developers. Consistent with this aim, we made several decisions to restrict our population of interest. First, we only included articles citing the ProFaNE COS, as we interpreted citation as evidence of adoption (i.e. uptake). To be implemented with fidelity requires that an innovation or evidence-based practice must first be adopted by an individual, organization, or setting [35]. Second, we restricted our sample to completed RCTs, defined as prospective studies that assessed healthcare interventions in human participants who were randomly allocated to study groups [30]. As stated by developers, the ProFaNE COS was “intended to promote consistency in collection and reporting of essential elements (page 1619)” in future trials and meta-analyses [29]. We excluded non-completed RCTs, such as protocols, due to the prevalence of outcome modification after trial initiation and selective outcome reporting (e.g. [38]), making it impossible to explore implementation fidelity based on reporting of pre-specified outcomes. We also excluded early developmental trials, such as feasibility and pilot studies, where the focus was to examine whether the trial should be done, and if so, how. Third, we restricted our sample to RCTs that sampled older people, defined as whether the study excluded participants younger than 60 years of age, the mean age of included participants was 60 years of age or above, or the patient population was described as ‘older’, ‘elderly’, or ‘senior’ [30]. Fourth, we restricted our sample to RCTs examining the safety, effectiveness, or cost-effectiveness of interventions to prevent and/or manage falls. Fifth, we excluded articles if they were published in a language other than English. This was a practical consideration as time and funding were limited. Last, when more than one article was published on the results of a single RCT, we included all articles if they reported different outcomes at the same follow-up point analysed after trial completion, included the first article (determined by the date of publication) only when they reported the same outcomes at the same follow-up points analysed after trial completion, and included all articles if they reported the same or different outcomes at different follow-up points analysed after trial completion. This provided insight into temporal changes in barriers and enablers to the reporting of ProFaNE COS domains over the life course of a given trial.
Data sources
To identify eligible articles, we conducted a systematic citation analysis of the ProFaNE COS between 01 October 2005 and 12 July 2021. However, our search strategy differed for articles published prior to and after 16 January 2015. As Copsey et al. previously conducted a systematic citation analysis for all articles citing the ProFaNE COS between 01 October 2005 and 16 January 2015, we screened their list of included RCTs to identify articles published prior to 17 January 2015 [30]. As our eligibility criteria only differed from Copsey et al.’s with respect to the inclusion of multiple articles reporting data from the same RCT, we also screened Copsey et al.’s [30] list of RCTs excluded because they were secondary reports of already included RCTs. To identify articles citing the ProFaNE COS after 17 January 2015 and before 12 July 2021, we searched the Web of Science Citation Index, PubMed, and Scopus databases.
Article selection
To screen articles for eligibility, we adopted a two-step approach involving the title and abstract screening followed by full-text screening. To facilitate high-quality screening, reviewers were provided with protocols for screening. All difficulties, and their associated resolutions, were logged on screening tracking forms.
Our screening process differed for articles published prior to and after 17 January 2015. A single reviewer (AMBK) independently screened the titles and abstracts and subsequently the full text of articles published prior to 17 January 2015 for eligibility. We limited screening to a single reviewer as all articles published prior to 17 January 2015 had previously undergone screening by Copsey et al. [30]. Comparatively, at least two reviewers independently screened the titles and abstracts and subsequently the full text of articles published after 17 January 2015 for eligibility, as per the following. First, after removing duplicates, at least two reviewers independently screened the title and abstract of each article to determine the study type. Then, at least two reviewers independently screened the full text of the subset of articles with study types coded as RCT and unsure. When necessary, a third reviewer screened articles with conflicts between first and second screenings to resolve conflicts. If the third reviewer disagreed with both the first and second reviewer, two reviewers (AMBK and KMS) met in person to discuss and resolve conflicts.
Data abstraction
At least two reviewers independently abstracted data on whether each domain of the ProFaNE COS was reported, as well as author-level, study-level, and manuscript-level data from all articles meeting our eligibility criteria. To do this, a single reviewer (DS) from our research team first independently abstracted data from articles included in the analysis by Copsey et al., which also fulfilled our eligibility criteria [30]. Upon request, Copsey et al. shared the data they abstracted from these articles, which constituted the second abstraction, where possible [30]. For variables not abstracted by Copsey et al. [30], a second reviewer from our research team (AMBK) abstracted data from this subset of articles. Then, four reviewers, including one member of the research team (DS) and three research assistants, independently abstracted data from the remaining articles included in our sample. Reviewers were provided with a protocol for data abstraction and were instructed to log all difficulties and their associated resolutions on data abstraction tracking forms. A third, independent reviewer (AMBK or KMS) identified and resolved conflicts between the first and second abstractions.
Statistical analysis
Statistical significance in the final models was defined as P < 0.05. For all analyses, SAS (9.4) software, Cary, NC, US, was used.
Primary outcome: percentage of ProFaNE COS domains reported
We used the penalized regression method to select important predictors. This method allowed us to create a linear regression model that is penalized, for having too many variables in the model, by adding a constraint in the Eq. [39]. We used the LASSO (Least Absolute Shrinkage and Selection Operator) method [40] using the GLMSELECT procedure in SAS [41, 42], whereby a sequence of models is obtained using the LASSO algorithm, and in this sequence, the one yielding the smallest value of the Mallow’s C(p) statistic [43] is chosen as the final model. We produced diagnostic plots to select the most parsimonious model (Additional file 1). We used the split option for categorical variables to include all levels as dummy coded variables in the model. In the LASSO selection procedure, it is not possible to produce P values for the selected variables. In lieu, we ran a generalized linear model with these selected variables to estimate the mean difference and its 95% confidence intervals (95% CI) and P values to describe the variables that have significant relationships with the outcome, as that was our main interest.
Secondary outcomes: reporting of each domain of the ProFaNE COS
We used statistical models for classification to select important variables that are useful for predicting the outcome variable. We used cost-complexity pruning [44] and estimated the misclassification rate by tenfold cross-validation to prevent overfitting. For all models, the fitted model classified the response variable well (Additional file 2). The model is estimated using the HPSPLIT procedure in SAS [42]. In addition, we ran a generalized linear model with those selected variables to estimate the odds ratio (OR) and its 95% CI and P values to describe the variables that have significant relationships with each outcome, as that was our main interest.
Missing data
We were missing data for three explanatory variables and no outcome variables. We could not determine the number of trial centres in n = 13 (15.3%) articles, funding type in n = 4 (4.7%) articles, and length of follow-up for the primary outcome variable in n = 1 (1.2%) article. We used different approaches to handle missingness in our statistical analyses. First, we list-wise deleted the single article containing missing data for the length of follow-up of the primary outcome variable. Then, to treat missing data on the number of trial centres and funding type, we created missing data response options (“Not Reported”) for these variables. We entered these response options in statistical models either as their own category (number of trial centres) when frequencies permitted, or as part of an ‘Else’ category where missing cases were grouped with other response options (funding type). Thus, our final analytical sample comprised n = 84 articles.
Results
Data sources
By scanning the lists of RCTs included in and excluded from the analysis by Copsey et al. [30], we identified n = 44 unique articles that cited the ProFaNE COS and were published prior to 17 January 2015. Through database searching, we identified n = 761 unique articles that cited the ProFaNE COS and were published after 17 January 2015 and before 12 July 2021.
Article selection
We screened a total of n = 805 titles and abstracts and subsequently the full text of n = 216 articles (Fig. 1). Of the n = 216 articles that underwent full-text review, we included n = 85 articles (Additional file 3) and excluded n = 131 articles (Additional file 4).
Characteristics of included articles
Articles were most often authored by corresponding authors affiliated with institutions from Australia (18 of 85, 21%), followed by the UK (12 of 85, 14%), Germany (10 of 85, 12%), US (9 of 85, 11%), and Finland (6 of 85, 7%) (Table 1). The majority of articles (49 of 85, 58%) were authored by corresponding authors affiliated with one or more European institutions at the time of publication. Articles were authored by 76 unique corresponding authors.
Most (69 of 85, 81%) articles reported the results of RCTs based in community settings, 75 (of 85, 88%) reported at least one non-industry source of funding, and 24 (of 85, 28%) reported the results of RCTs in populations with specific disease diagnoses, including dementia or cognitive impairment (6 of 85, 7%), Parkinson’s and/or stroke (5 of 85, 6%), and others (13 of 85, 15%) (Table 1). Articles reported the results of 75 unique RCTs.
Articles were published between 2007 and 2021, with the largest proportion published after 2017 (33 of 85, 39%) (Table 1). The ProFaNE COS was cited in the introduction section by n = 12 (of 85, 14%) articles, including when providing the rationale for the article by justifying their selection, measurement and reporting of outcomes (n = 7), when providing a definition of a ‘fall’ (n = 3), and when synthesizing the evidence of the effectiveness of interventions to prevent falls or recommendations for clinical practice (n = 2). The ProFaNE COS was cited in the methods section by n = 72 (of 85, 85%) articles, namely when describing measurement and/or reporting of the falls domain (n = 69), including when providing a definition of a ‘fall’ (n = 52). Seven, three, two, and one articles cited the ProFaNE COS in the methods section when describing measurement and/or reporting of injuries, HRQoL, psychological consequences of falling, and physical activity, respectively. The ProFaNE COS was cited in the discussion section by n = 17 (of 85, 20%) articles, when discussing the strengths of the study afforded by following recommendations from the ProFaNE COS (n = 9), limitations of the study afforded by not following recommendations from the ProFaNE COS (n = 5), potential shortcomings of measuring falls via self-report (n = 1), issues related to the definition of falls provided in the COS (n = 1), and considerations made when selecting an instrument to measure HRQoL (n = 1).
Reporting of ProFaNE COS domains
The percentage of domains of the ProFaNE COS reported ranged from 0 to 100% across articles, with a median of 40% and a mean (SD) of 52.2% (25.1). Five domains were reported in n = 8 (of 85, 9%) articles, four domains in n = 12 (of 85, 14%) articles, three domains in n = 22 (of 85, 26%) articles, two domains in n = 27 (of 85, 32%) articles, one domain in n = 14 (of 85, 16%) articles, and zero domains in n = 2 (of 85, 2%) articles. Of the two articles that reported zero domains, one reported the effects of education on older adults’ knowledge of fall threats and fall-prevention behaviours [44] and the other reported the effects of community-based exercise programmes on the following risk factors for falls: balance, postural control, mobility, and leg strength [45]. The former purported to measure falls, but did not report any data on falls [44]. Of the ProFaNE COS domains, falls were reported most frequently (76 of 85, 89%), followed by injuries (47 of 85, 55%), psychological consequences of falling (39 of 85, 46%), and physical activity (31 of 85, 37%), while HRQoL was reported least often (29 of 85, 31%) (Table 2).
Factors associated with reporting of ProFaNE COS domains
Overall percentage of ProFaNE COS domains reported
Funding type and intervention type were associated with the percentage of ProFaNE COS domains reported (Table 3). In terms of funding type, articles that reported at least one non-industry source of funding reported a higher mean percentage of domains from the ProFaNE COS than articles that did not report at least one non-industry source of funding (estimated mean difference = 17.5%; 95% CI 1.2–33.2). In terms of intervention type, RCTs that tested exercise (estimated mean difference = 15.4%; 95% CI 1.9–28.9) or multi-component/factorial (estimated mean difference = 17.4%; 95% CI 4.7–30.1) interventions each reported a higher mean percentage of domains from the ProFaNE COS when compared to RCTs that examined other interventions.
Reporting of falls
The length of follow-up of the primary outcome variable and citation of the ProFaNE COS in the methods section were associated with the likelihood of reporting falls (Table 4.). The odds of reporting falls was higher in articles that reported primary outcomes after at least 12 months of follow-up compared to less than 12 months of follow-up (OR = 8.2; 95% CI 1.7–65.2). The odds of reporting falls was lower in articles that did not cite the ProFaNE COS in the methods section when compared to articles that did (OR = 0.02; 95% CI < 0.01–0.12).
Reporting of injuries
The length of follow-up of the primary outcome variable was associated with the likelihood of reporting injuries (Table 4). The odds of reporting injuries was higher in articles that reported primary outcomes after at least 12 months of follow-up when compared to less than 12 months of follow-up (OR = 9.8; 95% CI 3.1–35.7).
Reporting of psychological consequences of falls
Intervention type and citation of the ProFaNE COS in the discussion section were associated with the likelihood of reporting psychological consequences of falling (Table 4). The odds of reporting psychological consequences of falls was higher in RCTs that tested multi-component/factorial interventions when compared to other intervention types that did not involve exercise (OR = 8.8; 95% CI 1.9–51.4). The odds of reporting psychological consequences of falls was higher in articles that did not cite the ProFaNE COS in the discussion section when compared to articles that did (OR = 6.1; 95% CI 1.7–35.6).
Reporting of HRQoL
The number of experimental arms was associated with the likelihood of reporting HRQoL (Table 4). The odds of reporting HRQoL was lower in RCTs with multiple experimental arms compared to a single experimental arm (OR = 0.08; 95% CI < 0.01–0.54).
Reporting of physical activity
The intervention type was associated with the likelihood of reporting physical activity (Table 4). The odds of reporting physical activity was higher in RCTs that tested exercise (OR = 25.3; 95% CI 3.6–303.6) and multi-component/factorial (OR = 7.5; 95% CI 1.3–76.0) interventions when compared to other intervention types.
Discussion
By way of a systematic citation analysis and correlational study, we aimed to identify factors associated with reporting of the ProFaNE COS in a purposeful sample of articles disseminating the results of RCTs focused on falls in older people, which cited the ProFaNE COS.
Consistent with findings from Copsey et al. [30], we observed that the number of articles citing the ProFaNE COS has increased over time. Between January 2015 and July 2021, the ProFaNE COS was cited by 761 unique articles. Previously, Copsey et al. reported that between October 2005 and January 2015, the ProFaNE COS was cited by 464 unique articles [30]. Our findings lend support to Copsey et al.’s [30] claim that awareness of the ProFaNE COS has increased, and is likely to continue to increase, over time [30].
We also found that few of the included trials implemented the ProFaNE COS with fidelity. Just eight articles (of 85, 9%) reported all five domains of the ProFaNE COS, and most measured two (27 of 85, 32%). Additionally, as was observed by Copsey et al., not all domains of the ProFaNE COS were reported equally [30]. Falls were reported most often (76 of 85, 89%), followed by injuries (47 of 85, 55%), psychological consequences of falling (39 of 85, 46%), physical activity (31 of 85, 37%), and HRQoL (29 of 85, 31%).
The limited impact of COS has been reported previously. Smith et al. examined the impact of the hip fracture COS in RCTs registered in the ClinicalTrials.gov trial registry between 1997 and 2018 [32]. Of these, n = 96 RCTs fitted with the scope of the COS and were registered between 2015 and 2018, i.e. after dissemination of the hip fracture COS in 2014. Only 7 (7%) trials included all five domains of the hip fracture COS. Mortality, pain, activities of daily living, mobility, and HRQoL were included in 41% (39 of 96), 43% (41 of 96), 32% (31 of 96), 43% (41 of 96), and 25% (24 of 96) of registered trials, respectively [32].
Regarding our primary objective, we observed that funding type was associated with the percentage of ProFaNE COS domains reported. RCTs with at least one non-industry funding source reported a higher mean percentage of domains than RCTs without at least one non-industry funding source. This observation may, in part, reflect initiatives of non-industry trial funders across Europe (e.g. Horizons2020) and the US (e.g. Patient-Centered Outcomes Research Institute; PCORI) to endorse COS, often by requiring trialists to search for relevant COS, and to justify when relevant COS are not included in proposals for clinical trials. For example, in 2012, the UK NIHR HTA Programme directed applicants for all randomized trial and evidence synthesis funding streams to the Core Outcome Measures in Effectiveness Trials (COMET) database of planned, completed, and ongoing COS (www.comet-initiative.org), stating that “where established Core Outcomes exist they should be included amongst the list of outcomes unless there is good reason to do otherwise”. An evaluation of the extent to which applicants followed these guidelines and searched for a relevant COS between 2012 and 2015 concluded that this endorsement had a positive impact on COS uptake [31]. Findings may also be attributable to the nature of ProFaNE COS domains. Extant research suggests that for-profit-funded trials report favourable (positive) outcomes more often than trials funded by other sources for several reasons, including greater reliance on surrogate endpoints and insufficient follow-up [45]. No surrogate endpoints were included in the ProFaNE COS, and the recommended length of follow-up for each domain was a minimum of 12 months. In future research, it might be informative to explore whether differences in reporting of ProFaNE COS domains exist between RCTs funded by government grants vs. other forms of non-industry funding. Trials funded by government grants that have undergone multiple stages of peer/panel review, with external steering committees monitoring methodology along the various stages of the process, might be expected to produce higher quality trials that are the most likely to implement COS with fidelity.
We observed that intervention type was associated with the percentage of ProFaNE COS domains reported, suggesting the appropriateness of the ProFaNE COS may be affected by the characteristics of the intervention under study. Findings are consistent with the results of an international survey of trialists on their reasons for non-adoption of a universal hip fracture COS developed to cover a broad scope of interventions, whereby 35% (28 of 80), 34% (27 of 80), and 24% (19 of 80) of respondents believed the existing COS needed to be revised to accommodate rehabilitation, surgical, and anaesthetic interventions, respectively [33].
Specifically, RCTs examining exercise or multi-component/factorial interventions each reported a higher mean percentage of domains compared to RCTs examining other intervention types, such as advice, environmental modification or aides, medical interventions, and drugs or supplements. In secondary analyses, we found that odds of reporting physical activity was higher in RCTs examining exercise or multi-component/factorial interventions, while odds of reporting psychological consequences of falls was higher in RCTs testing multi-component/factorial interventions, when compared to other intervention types. The reasons why multi-component/factorial and exercise intervention trials may have been more compatible with the ProFaNE COS remain unclear. A better understanding of the mechanisms by which intervention type could affect the compatibility of ProFaNE COS would be useful to design and tailor implementation strategies to improve the appropriateness of the ProFaNE COS for a diversity of healthcare interventions. One avenue of future research may investigate whether differences in the shared knowledge, beliefs, self-efficacy, and/or motivation of trialists testing different intervention types to measure and report ProFaNE COS domains could explain our findings, as these are theorized to influence implementation success [36].
Best practices for COS development recommend the scope of healthcare interventions (e.g. surgery, drugs, medical devices) covered by the COS should be defined to reduce ambiguity and help users decide on the relevance of the COS to their work [46]. Although not explicitly defined, the ProFaNE COS could be interpreted as applying to a broad scope of interventions. Challenges in implementing COS with universal population and intervention coverage have been reported before (e.g. hip fracture COS [33]). Future research may seek to explore whether implementation fidelity of COS is impacted by the broadness of scope, defined in terms of the research or practice setting covered, health conditions covered, populations covered, and interventions covered [46]. COS developers should strike to achieve an optimal balance between the generalizability and specificity of a COS. If defined too broadly in scope, the COS may be perceived as needing adaptation to fit heterogenous trial designs. Conversely, if defined too narrowly in scope, developers risk the COS having limited potential for impact.
It is worth noting that since the original publication of the ProFaNE COS in 2005, minimum standards have been set for developing COS [46]. At a minimum, it is recommended that COS should be developed in partnership with (1) individuals (or groups) who will use the COS in research, including clinical trialists and industry; (2) healthcare professionals; and (3) people with lived experience of the health condition, including patients, family members, and carers. It is also recommended that care should be taken to avoid ambiguity in the language used in the list of outcomes [46]. People with lived experience of falls did not contribute to the development of the ProFaNE COS, and of the five ProFaNE COS domains, two (physical activity and HRQoL) were undefined, and one (physical activity) was not accompanied by a recommended measurement instrument set. It was stated that “further research is required before a measure of physical activity can be recommended for inclusion in the common data set (page 1621)” [29]. It is beyond the scope of this study to understand if and how these limitations of the ProFaNE COS could have impacted implementation fidelity. Nevertheless, the ProFaNE COS may need updating to reflect current standards in COS development.
There are diverse methodological approaches to assess barriers and enablers to knowledge use, and none is considered superior to others [47]. We elected to use a quantitative approach by examining associations between potential determinants of knowledge use and reporting of ProFaNE COS domains in a purposeful sample of RCTs. Our aim was to investigate factors governing implementation fidelity. However, implementation fidelity represents only one of several important implementation outcomes. Different methodological approaches are needed to explore other outcomes of successful implementation, including adoption (i.e. uptake) or penetration (i.e. reach) of the ProFaNE COS. These outcomes may be more appropriately assessed via a review of clinical trial registry data to explore what domains are purportedly included in all trials registered after 2005 that fit the scope of the ProFaNE COS [48].
Our study has several limitations. This study was not pre-registered, so we are unable to document whether the study methods were not changed after the study began. The observational nature of our research design means we have provided evidence of association, not causation. We have assumed that citation is evidence of adoption (i.e. uptake), such that clinical trialists who cited the ProFaNE COS had made an initial decision to try to employ it in their trial design. However, it is possible that some trialists became aware of the ProFaNE COS too late for it to have influenced their trial design. Our approach also failed to identify trialists who implemented the ProFaNE COS in their trial design but chose not to cite it when disseminating their study results. We were restricted to investigating the potential role of factors easily abstractable from peer-reviewed academic articles, even though the CFIR states that determinants of knowledge use are vast and span diverse levels of influence [36]. We intend to build on the findings of this study by interviewing corresponding authors of included articles to better understand barriers and enablers to implementation fidelity of the ProFaNE COS. Our sample size was not large enough to dependably handle a complex statistical model containing all of our explanatory variables. To overcome this limitation, we employed commonly used variable selection methods which we acknowledge can be problematic in terms of overfitting and the inclusion of nuisance variables in the models [49, 50]. To minimize this risk, we used variable selection methods that are penalized (e.g. LASSO) and cross-checked our results using multiple techniques to select the most parsimonious models.
Conclusion
By way of a systematic citation analysis of the ProFaNE COS and a correlational study, we found that amongst the subset of articles disseminating the results of RCTs in older people, the number (%) of ProFaNE COS domains reported ranged from 0 to 5 (0–100%), with a median of 2 (40%). We also observed that RCTs funded by at least one non-industry source, examining exercise or multi-component/factorial interventions, reported the highest mean percentages of ProFaNE COS domains. Findings may help inform strategies to increase the impact of the ProFaNE COS, which should lead to more consistent outcome reporting across RCTs and, ultimately, better understanding of the effectiveness and safety of interventions to prevent and/or manage falls in older people.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Abbreviations
- CFIR:
-
Consolidated Framework for Implementation Research
- CI:
-
Confidence interval
- COS:
-
Core outcome set
- COMET:
-
Core Outcome Measures in Effectiveness Trials
- HRQoL:
-
Health-related quality of life
- HTA:
-
Health Technology Assessment
- NIHR:
-
National Institute for Health Research
- OR:
-
Odds ratio
- PCORI:
-
Patient-Centered Outcomes Research Institute
- ProFaNE:
-
Prevention of Falls Network Europe
- RCT:
-
Randomized controlled trial
- SD:
-
Standard deviation
- UK:
-
United Kingdom
- US:
-
United States
References
Sibley KM, Voth J, Munce SE, Straus SE, Jaglal SB. Chronic disease and falls in community-dwelling Canadians over 65 years old: a population-based study exploring associations with number and pattern of chronic conditions. BMC Geriatr. 2014;14:22.
Lord SR, Sherrington C, Hicks C. Epidemiology of falls and fall-related injuries. In: Sherrington C, Lord SR, Naganathan V, editors. Falls in older people: risk factors, strategies for prevention and implications for practice. 3rd ed. Cambridge: Cambridge University Press; 2021. p. 3–22.
Bergen G, Stevens MR, Burns ER. Falls and fall injuries among adults aged >/=65 years - United States, 2014. MMWR Morb Mortal Wkly Rep. 2016;65(37):993–8.
Parachute. The Cost of Injury in Canada. Toronto: Parachute; 2015. Available from: https://parachute.ca/wp-content/uploads/2019/06/Cost_of_Injury-2015.pdf.
Kramarow E, Chen LH, Hedegaard H, Warner M. Deaths from unintentional injury among adults aged 65 and over: United States, 2000–2013. NCHS Data Brief. 2015;199:199.
Scott V, Wagar L, Elliott S. Falls & Related Injuries among Older Canadians: Fall‐related Hospitalizations & Intervention Initiatives. Prepared on behalf of the Public Health Agency of Canada, Division of Aging and Seniors. Victoria: Victoria Scott Consulting; 2010. Available from: https://baycrest.echoontario.ca/wp-content/uploads/2018/06/Falls-related-injuries-among-older-Canadians-Fall-related-hospitalizations-intervention-initiatives.pdf.
Rubenstein LZ. Falls in older people: epidemiology, risk factors and strategies for prevention. Age and Ageing. 2006;35(suppl 2):ii37–41.
Büchele G, Becker C, Cameron ID, König H-H, Robinovitch S, Rapp K. Predictors of serious consequences of falls in residential aged care: analysis of more than 70,000 falls from residents of Bavarian nursing homes. J Am Med Dir Assoc. 2014;15(8):559–63.
Harvey LA, Close JC. Traumatic brain injury in older adults: characteristics, causes and consequences. Injury. 2012;43(11):1821–6.
Fletcher PC, Hirdes JP. Restriction in activity associated with fear of falling among community-based seniors using home care services. Age Ageing. 2004;33(3):273–9.
Scheffer AC, Schuurmans MJ, van Dijk N, van der Hooft T, de Rooij SE. Fear of falling: measurement strategy, prevalence, risk factors and consequences among older persons. Age Ageing. 2008;37(1):19–24.
Friedman SM, Munoz B, West SK, Rubin GS, Fried LP. Falls and fear of falling: which comes first? a longitudinal prediction model suggests strategies for primary and secondary prevention. J Am Geriatr Soc. 2002;50(8):1329–35.
Choi NG, Bruce ML, DiNitto DM, Marti CN, Kunik ME. Fall worry restricts social engagement in older adults. J Aging Health. 2020;32(5–6):422–31.
Choi NG, Gell NM, DiNitto DM, Marti CN, Kunik ME. Depression and activity-limiting fall worry among older adults: longitudinal reciprocal relationships. Int Psychogeriatr. 2020;32(4):495–504.
Choi NG, Marti CN, DiNitto DM, Kunik ME. Longitudinal associations of falls and depressive symptoms in older adults. Gerontologist. 2019;59(6):1141–51.
Scuffham P, Chaplin S, Legood R. Incidence and costs of unintentional falls in older people in the United Kingdom. J Epidemiol Community Health. 2003;57(9):740–4.
Florence CS, Bergen G, Atherly A, Burns E, Stevens J, Drake C. Medical costs of fatal and nonfatal falls in older adults. J Am Geriatr Soc. 2018;66(4):693–8.
Haidich AB. Meta-analysis in medical research. Hippokratia. 2010;14(Suppl 1):29–37.
Sherrington C, Michaleff ZA, Fairhall N, Paul SS, Tiedemann A, Whitney J, et al. Exercise to prevent falls in older adults: an updated systematic review and meta-analysis. Br J Sports Med. 2017;51(24):1750–8.
Tricco AC, Thomas SM, Veroniki AA, Hamid JS, Cogo E, Strifler L, et al. Comparisons of interventions for preventing falls in older adults: a systematic review and meta-analysis. JAMA. 2017;318(17):1687–99.
Kirkham JJ, Altman DG, Chan AW, Gamble C, Dwan KM, Williamson PR. Outcome reporting bias in trials: a methodological approach for assessment and adjustment in systematic reviews. BMJ. 2018;362: k3802.
Williamson PR, de Avila OR, Clarke M, Gorst SL, Hughes K, Kirkham JJ, et al. Assessing the relevance and uptake of core outcome sets (an agreed minimum collection of outcomes to measure in research studies) in Cochrane systematic reviews: a review. BMJ Open. 2020;10(9):e036562.
Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST, et al. The COMET handbook: version 1.0. Trials. 2017;18(Suppl 3):280.
Williamson PR, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E, et al. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13:132.
Webbe J, Sinha I, Gale C. Core outcome sets. Arch Dis Child Educ Pract Ed. 2018;103(3):163–6.
Clarke M, Williamson PR. Core outcome sets and systematic reviews. Syst Rev. 2016;5:11.
Heneghan C, Goldacre B, Mahtani KR. Why clinical trial outcomes fail to translate into benefits for patients. Trials. 2017;18(1):122.
Prinsen CA, Vohra S, Rose MR, Boers M, Tugwell P, Clarke M, et al. How to select outcome measurement instruments for outcomes included in a “Core Outcome Set” - a practical guideline. Trials. 2016;17(1):449.
Lamb SE, Jørstad-Stein EC, Hauer K, Becker C. Development of a common outcome data set for fall injury prevention trials: the prevention of falls network Europe consensus. J Am Geriatr Soc. 2005;53(9):1618–22.
Copsey B, Hopewell S, Becker C, Cameron ID, Lamb SE. Appraising the uptake and use of recommendations for a common outcome data set for clinical trials: a case study in fall injury prevention. Trials. 2016;17(1):131.
Hughes KL, Kirkham JJ, Clarke M, Williamson PR. Assessing the impact of a research funder’s recommendation to consider core outcome sets. PLoS ONE. 2019;14(9):e0222418.
Smith TO, Collier T, Sheehan KJ, Sherrington C. The uptake of the hip fracture core outcome set: analysis of 20 years of hip fracture trials. Age Ageing. 2019;48(4):595–8.
Fletcher J, Sheehan KJ, Smith TO. Barriers to uptake of the hip fracture core outcome set: an international survey of 80 hip fracture trialists. Clin Trials. 2020;17(6):712–6.
Wallace SJ, Sullivan B, Rose TA, Worrall L, Le Dorze G, Shrubsole K. Core outcome set use in poststroke aphasia treatment research: examining barriers and facilitators to implementation using the theoretical domains framework. J Speech Lang Hear Res. 2021;64(10):3969–82.
Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76.
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.
Kolanowski A, Gilmore-Bykovskyi A, Hill N, Massimo L, Mogle J. Measurement challenges in research with individuals with cognitive impairment. Res Gerontol Nurs. 2019;12(1):7–15.
Lancee M, Lemmens CMC, Kahn RS, Vinkers CH, Luykx JJ. Outcome reporting bias in randomized-controlled trials investigating antipsychotic drugs. Transl Psychiatry. 2017;7(9):e1232.
James G, Witten D, Hastie T, Tibshirani R. An introduction to statistical learning: with applications in R. 2nd ed. New York, NY: Springer New York; 2021.
Tibshirani R. Regression shrinkage and selection via the Lasso. J Roy Stat Soc: Ser B (Methodol). 1996;58(1):267–88.
Cohen RA. Introducing the GLMSELECT procedure for model selection. In Proceedings of the Thirty-First Annual SAS Users Group International Conference 2006 Mar 26 (pp. 4770–4792).
SAS Institute Inc. 2015. SAS/STAT® 14.1 User’s guide. Cary, NC: SAS Institute Inc.
Mallows CL. Some comments on Cp. Technometrics. 2000;42(1):87–94.
Breiman L, Friedman JH, Olshen RA, Stone CJ. Classification and regression trees. Classification and regression trees (1st ed.). Routledge. https://doi.org/10.1201/9781315139470.
Chopra SS. MSJAMA: Industry funding of clinical trials: benefit or bias? JAMA. 2003;290(1):113–4.
Kirkham JJ, Davis K, Altman DG, Blazeby JM, Clarke M, Tunis S, et al. Core Outcome Set-STAndards for Development: the COS-STAD recommendations. PLoS Med. 2017;14(11):e1002447.
Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7:50.
Kirkham JJ, Clarke M, Williamson PR. A methodological approach for assessing the uptake of core outcome sets using ClinicalTrials.gov: findings from a review of randomised controlled trials of rheumatoid arthritis. BMJ. 2017;357:j2262.
Smith G. Step away from stepwise. Journal of Big Data. 2018;5(1):32.
Heinze G, Dunkler D. Five myths about variable selection. Transpl Int. 2017;30(1):6–10.
Acknowledgements
We are grateful to Dr. Bethan Copsey for sharing the data abstraction tools and datasets used and analysed in Copsey et al. (30), as well as for reviewing and providing comments on this manuscript prior to submission. We also thank Harrison Grogran, Olivia Tefft, Devashree Prabhu, and Josee Rochon for assisting with the data collection.
Funding
This work was supported by the Manitoba Medical Service Foundation (award number 08–2017-09). The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Author information
Authors and Affiliations
Contributions
AMBK contributed to the experimental design, data collection, data analysis, and preparation and review of the manuscript. DS contributed to the data collection and preparation and review of the manuscript. SEL and SRL contributed to the experimental design and review of the manuscript. RR contributed to the data analysis and review of the manuscript. KMS contributed to the experimental design, data collection, and review of the manuscript. The authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
SEL and SRL contributed to the development of the Prevention of Falls Network Europe (ProFaNE) COS. AMBK, DS, RR, and KMS declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Additional file 1.
Modelfit characteristics for our model on the percentage of ProFaNE COS domainsreported.
Additional file 2.
Modelfit characteristics for our models on likelihood of reporting each domain ofthe ProFaNE COS.
Additional file 3.
Listof included articles (n=85).
Additional file 4.
Listof articles excluded after full-text review (n=131).
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Korall, A.M.B., Steliga, D., Lamb, S.E. et al. Factors associated with reporting of the Prevention of Falls Network Europe (ProFaNE) core outcome set domains in randomized trials on falls in older people: a citation analysis and correlational study. Trials 23, 710 (2022). https://doi.org/10.1186/s13063-022-06642-w
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13063-022-06642-w