This article has Open Peer Review reports available.
Improving Nursing Home Care through Feedback On PerfoRMance Data (INFORM): Protocol for a cluster-randomized trial
© The Author(s). 2017
Received: 4 May 2016
Accepted: 8 December 2016
Published: 10 January 2017
Audit and feedback is effective in improving the quality of care. However, methods and results of international studies are heterogeneous, and studies have been criticized for a lack of systematic use of theory. In TREC (Translating Research in Elder Care), a longitudinal health services research program, we collect comprehensive data from care providers and residents in Canadian nursing homes to improve quality of care and life of residents, and quality of worklife of caregivers. The study aims are to a) systematically feed back TREC research data to nursing home care units, and b) compare the effectiveness of three different theory-based feedback strategies in improving performance within care units.
INFORM (Improving Nursing Home Care through Feedback On PerfoRMance Data) is a 3.5-year pragmatic, three-arm, parallel, cluster-randomized trial. We will randomize 67 Western Canadian nursing homes with 203 care units to the three study arms, a standard feedback strategy and two assisted and goal-directed feedback strategies. Interventions will target care unit managerial teams. They are based on theory and evidence related to audit and feedback, goal setting, complex adaptive systems, and empirical work on feeding back research results. The primary outcome is the increased number of formal interactions (e.g., resident rounds or family conferences) involving care aides – non-registered caregivers providing up to 80% of direct care. Secondary outcomes are a) other modifiable features of care unit context (improved feedback, social capital, slack time) b) care aides’ quality of worklife (improved psychological empowerment, job satisfaction), c) more use of best practices, and d) resident outcomes based on the Resident Assessment Instrument – Minimum Data Set 2.0. Outcomes will be assessed at baseline, immediately after the 12-month intervention period, and 18 months post intervention.
INFORM is the first study to systematically assess the effectiveness of different strategies to feed back research data to nursing home care units in order to improve their performance. Results of this study will enable development of a practical, sustainable, effective, and cost-effective feedback strategy for routine use by managers, policy makers and researchers. The results may also be generalizable to care settings other than nursing homes.
ClinicalTrials.gov Identifier: NCT02695836. Date of registration: 24 February 2016
KeywordsAudit and Feedback Nursing Homes Quality of Care Quality Improvement Performance Cluster Randomized Controlled Trial Clinical Microsystems
Importance of residential long term care
In Western countries, 3-8% of people aged 65 years or older live in nursing homes [1, 2] (e.g., 224 thousand in Canada , 1.3 million in the USA , and 2.9 million in Europe ), and demand for these services will substantially increase [1, 5, 6]. Between half and three quarters of nursing home residents have dementia (with a rising trend) [7–10], and these figures are likely underrated by at least 11% . Dementia progresses from mild impairment and difficulties organizing daily life to incontinence, unsteadiness, profound difficulties in communication and nutrition intake, confinement to bed, and finally death [8, 12, 13]. In 2015, 46.8 million people aged 60 years or older were living with dementia worldwide (4.8 million in North America, and 7.5 million in Western Europe) , and 96% of US Americans with Alzheimer’s (the most common type of dementia) are 65 years or older . Global numbers of older people with dementia will increase to 131.5 million by 2050. Currently, dementia has no prevention, cure, or effective treatment . Without dramatic breakthroughs in either prevention or treatment, as the number of frail elderly increases, so will their eventual need for nursing home care.
As older adults with dementia are able to remain at home longer with community care, future trends will be for transition to a nursing home later in the dementia trajectory [5, 15]. This will further increase levels and complexity of care in these settings. At the same time, absolute staffing levels and proportions of regulated staff are low in nursing homes, and up to 80% of direct care is provided by unregulated care providers (care aides) with little or almost no formal training [16–19].
Ongoing challenges to quality of care and use of best practices
Quality of care in nursing homes has been a challenge for decades [20–23] and evidence to support consistent quality improvement strategies is still lacking [21, 24]. Multiple international reports [23, 25–29] describe sub-optimal quality of care in nursing homes. For example, rates of adverse events (e.g., pressure ulcers) vary up to 10-fold across facilities , are much higher at particular times during a nursing home stay [30–34], and vary across nursing home ownership models [30, 35].
In acute and primary care, we know that persistent and deeply troubling research–practice gaps exist across countries, professions, and settings [36–39]. A reported 30–40% of patients do not receive evidence-based care and 25% receive unnecessary or potentially harmful care [40, 41]. In the nursing home sector these performance gaps result in deleterious resident outcomes (e.g., high modifiable symptom burden in the last year of life, unnecessary and inappropriate transfer to hospital especially in the last weeks of life) and deleterious workforce outcomes (e.g., high burnout, reduced job satisfaction). Almost no knowledge translation (KT) research has been undertaken in the nursing home sector [36, 42, 43] and only recently have there been calls for such research [44–46]. There is little work developing effective and efficient means by which to tailor and deliver ongoing performance data to foster intentional action to improve quality of care and worklife. There has been even less work rigorously evaluating such strategies. We located no work attempting to tailor organizational context data into performance data. Under these conditions, leaders in the nursing home sector are faced with daunting challenges in delivering acceptable standards of care. They have little guidance on how to feasibly identify actionable performance gaps over time or how to respond to them.
The Translating Research in Elder Care (TREC) research program
This project is a key element of our long term program of research (Translating Research in Elder Care – TREC) focused on advancing KT science [47, 48]. TREC’s mission is to improve quality of care and quality of life for older adults in nursing homes and work life for their care providers. At its most fundamental level, the goal of KT in clinical settings is to move research (e.g., performance data) to action (e.g., performance improvement). TREC is an ongoing research program and aims to create actionable performance data to improve performance at the clinical microsystem (care unit) level in nursing homes. The microsystem has rarely been targeted before in performance improvement trials and possesses unique features that make it key to leading care interventions and driving change in these settings [49, 50]. Our previous research demonstrated that members of the clinical microsystem are the best potential target to drive positive change in this field [49, 50]. Findings from this investigation will add new insights on how actionable research data on modifiable elements of organizational context (framed as performance data) can be used to improve performance in a complex adaptive system such as a nursing home . The next two sections describe the relevance of clinical microsystems and the microsystem work context for performance improvement.
Relevance of clinical microsystems
Focusing on clinical microsystems as the target for improvement activities is central to our work [52–54]. These “small group[s] of people who work together on a regular basis to provide care to discrete subpopulations of patients” (p. 474) are essential building blocks of organizations and the health system , and a critical target level for patient safety interventions . Care services are often organized at this microsystem level  and individual residents receive care on care units embedded in organizations . Targeting improvement strategies to the microsystem level can potentially transform health care systems . Emerging international evidence suggests that improvement efforts focusing on the microsystem level are successful [54, 59, 60]. We found that the microsystem level explained a significantly higher percentage of variance in work context outcomes (e.g., leadership, culture, evaluation) than did individual (resident/caregiver) or facility levels . Using longitudinal data on three risk-adjusted quality indicators (QIs) from the Resident Assessment Instrument – Minimum Data Set (RAI-MDS) 2.0 , we found that facility level reporting masks important inter-unit/intra-facility variation; thus quality improvement interventions should target both facility and microsystem levels . The intervention described here targets the leaders of these clinical microsystems. Using a complex adaptive system lens in the nursing home sector, Anderson [62, 63] suggested fostering manager development as the key to effective performance improvement. Similarly, McDaniel et al. [64–66] argue that health care organizations are complex adaptive systems.
Relevance of the care unit work context
Studies in nursing homes have increasingly focused on the broad contextual influences of organizations. Generally, context is theorized as culture [67–71], where specific elements (e.g., person-centeredness, staff engagement) are identified, measured, and associated with outcomes of interest (e.g., quality of life or care). Reports associate more positive cultures (more person-centered, less controlling, more relationship-based) with lower rates of feeding tube placement , lower restraint use , reduction in anti-psychotic prescribing , and better quality of care . The influence of context on successful adoption of innovation and on quality improvement success was suggested theoretically [75–85], examined empirically [57, 86–91], and has been the subject of several reviews [92–97]. The consensus is that context is an important influence on implementation success. Dopson and Fitzgerald  for example, synthesized 39 case studies probing context and found it an important mediator of innovation. They identified several contextual processes important to innovation adoption and, we argue, to using performance data to achieve performance goals. Among these processes were (1) sensing, interpreting, and integrating new evidence; (2) reinforcing or marginalizing new evidence; (3) relating new evidence to local context needs; and (4) discussing/debating new evidence with local stakeholders. Recent systematic reviews [96, 97, 99, 100] identified organizational features such as complexity, centralization, size, a research champion, organizational slack, and resources as important to innovation success. Positive influences include cultural context (culture, climate, openness to change, organizational innovativeness, leadership, evaluation, feedback), structural context (organizational structure, management and supervision, resources, time, staff development), physical context (organizational size), and social context (social influence, collaboration, relational capital, communication, participation in decision making). We have repeatedly demonstrated that organizational context as measured using the Alberta Context Tool (ACT) [77, 80, 101–105] has a positive association with better scores on staff outcomes (best practice use, burnout, job satisfaction) [57, 87, 88, 102, 103, 105] and recently have demonstrated this effect on resident outcomes .
Purpose and aims
To evaluate and compare three feedback strategies – a standard feedback strategy and two assisted and goal-directed feedback strategies
To assess possible longer term effects of each strategy
To refine, based on the outcomes of this evaluation, a practical assisted feedback strategy for use in the nursing home sector that targets the leaders of their clinical microsystems
Schedule of enrolment, interventions, and assessments (adapted SPIRIT flow diagram )
Goal setting workshop
Support workshop 1
Support workshop 2
Assessment of primary and secondary study outcomes (all facilities participating in TREC)
Unit profile survey
Facility profile survey
Enrolment (all TREC facilities in Alberta and British Columbia)
Recruitment and informed consent (BAF/EAF)
Standard Feedback (SF)
Basic Assisted Feedback (BAF)
Enhanced Assisted Feedback (EAF)
Ongoing phone/e-mail support (EAF)
Workshop evaluation surveys (at the end of each workshop)
Workshop fidelity checklist and debriefs (at the end of each workshop)
Detailed documentation of workshops (participant observations)
Focus groups with BAF/EAF unit managers
Semi-structured interviews with SF unit managers
Basic cost evaluation of the intervention
INFORM is set in urban nursing homes participating in our current phase of TREC – a stratified (region by size by operator) random sample of 91 urban sites. We recruited these sites recruited from three provinces: Alberta, British Columbia, Manitoba. These TREC facilities participate in a longitudinal observational study that generates a rich set of resident, staff, unit, and facility level measures and outcomes. Data from this ongoing observational study forms the baseline assessment, and data from future waves of this study will form the post-intervention and long-term follow-up assessments. We will invite all eligible sites in Alberta and British Columbia to participate in INFORM (sample information sheet and informed consent, Additional file 3). TREC Manitoba facilities are not eligible to INFORM, as they are participating in another TREC intervention study.
of the ten ACT concepts, FI best reflects if a care unit has a more or less favorable work context, suggesting that improving FI improves context in general;
FI scores are generally low in our study sample (mean = 1.32, possible maximum = 4), suggesting substantial room for improvement;
based on theory and care unit managers’ opinions, FI is highly modifiable;
systematically involving care aides in FIs (e.g., resident rounds) can provide care aides as well as the care team with crucial information about residents.
All TREC sites have received SF in October/November 2015, which included participation in a face-to-face Dissemination Workshop. During these half-day workshops, we presented feedback reports with a particular focus on the core set of actionable context targets (FI, evaluation, social capital, slack time). A trained facilitator ran these workshops, a senior researcher presented on the reports, and participants engaged in small group discussions to: (a) help with interpretation of results overall, (b) draw attention to elements of context that are modifiable, (c) encourage microsystem teams to improve more modifiable context elements. We did not set specific goals – we provided simple “do your best” instructions.
Goal Setting Workshops
Facilities in the BAF and EAF arm will participate in an additional face-to-face Goal Setting Workshop. In each of four regions (Edmonton, Calgary, Interior Health, Fraser Health) we will hold a separate workshop for BAF and EAF sites. We expect an average number of 10-20 participants at each workshop (i.e., approximately 0.5-1 person per care unit). Care managers (who are the primary target group of this intervention) often oversee more than one care unit in their facility at the same time. We will prepare a Goal Setting Workshop Package for each care unit, including (a) a feedback report on the care unit’s context data (FI, evaluation, social capital, slack time), and (b) a Goal Setting Workbook summarizing important details on the INFORM study, defining key concepts, and outlining the goal setting approach we are going to apply. We will send this Goal Setting Workshop Package to each participating unit one week ahead of the workshop. The workshops will take place at a conference venue (e.g., a hotel) located in the respective health region of the participating facilities. The workshops will involve small group activities that adhere to feedback and goal setting approaches including: (a) reflecting on context data, (b) performance goal setting, including establishing a series of proximal and/or learning goals that will provide teams with markers of progress and explicit strategies for attaining performance goals, respectively, and (c) identification of ways care unit managers can gather interim feedback to assess progress towards goal achievement. The same trained facilitator as in the Dissemination Workshops will lead the Goal Setting Workshops. A researcher and regional decision maker dyad will be present with expertise in performance data and the specific clinical care setting, respectively. The same researcher will attend all workshops; the regional decision maker will be specific to each respective region. Participants will generate an action plan and will receive instructions for tracking goal progress and reporting at the support workshops. Additionally, we will assist workshop participants to develop specific, measurable goals and tools to track goal achievement. For example, if a care unit managerial team defines the goal to include at least four care aides in their monthly family conferences, we will provide managers with a run chart template to track the number of care aides included in each of those meetings during the study.
Six months after the Goal Setting Workshop we will hold a 90-min. virtual support workshop (webinars) in the BAF arm and a 180-min. face-to-face support workshop in the EAF arm (using the same conference venues as for the Goal Setting Workshops). Groups will (a) report on their progress with proximal/learning goals and strategies used toward performance goals, (b) discuss challenges they encountered, and (c) receive support from the researcher–decision maker dyad in addressing these challenges. BAF participants will receive limited online peer-to-peer support, EAF participants will benefit from face-to-face support from peers and research team members. Six months later we will hold a second support workshop in the BAF and EAF arms, similar in content to the first. The second support workshops are designed to continue to help participants discuss progress and problem solve. To ensure consistency in intervention delivery, the same trained facilitator will lead all BAF and EAF workshops, and the same researcher–decision maker dyad as in the Goal Setting Workshops will be present. Participants will receive push emails three months before each support workshop to remind them of the dates and the tasks to prepare.
On-demand e-mail and phone support
EAF teams will also have access to on-demand e-mail and phone support from the facilitator throughout the intervention period to address questions they may have and help resolve challenges that arise as they work towards goal achievement.
Sample size calculation
Yijt is the ACT Formal Interactions (FI) score (primary outcome) of unit j in facility i at time t.
μ is the FI population mean.
A1ijt, A2ijt, and A3ijt are indicator variables for the interventions (SF, BAF, and EAF, respectively). The indicator variable is 1 if the unit has been exposed to the respective intervention and 0 otherwise.
β1, β2, and β3 are the treatment effects of the three interventions.
bi is a facility-level random effect (variability of units within the facility).
bij is a unit-level random effect (variability of time points within the unit).
εijt is a residual term (variability between units).
μ = 1.1 (FI mean was 1.32. Standard feedback here is similar to that provided in the previous TREC phase [112–114]. We assumed that this intervention will, at least, have a small effect (increase the FI score by 0.2) compared to no feedback.
Standard deviation of bi = 0.154 (ICC = 0.445) (FI variability of units, within facilities)
Standard deviation of bij = 0.104 (FI variability of units, across waves 1 and 2 in TREC 1.0)
Standard deviation of εijt = 0.192 (variability of the unit FI residual term)
Average cluster size = 3 units per facility
Inclusion and exclusion criteria
Inclusion and exclusion criteria for facilities and units
• Participation in the TREC observational study
• Located in one of four health regions in Alberta (Edmonton or Calgary) or British Columbia (Fraser Health or Interior Health)
• Have at least one care unit with ten or more care aide responses to our survey that we use to assess organizational context and staff outcomes at baseline (T-2)
• Does not participate in the TREC observational study
• Located in the Winnipeg Regional Health Authority
• Do not have at least one care unit with ten or more care aide responses to our survey that we use to assess organizational context and staff outcomes
• Surveys collected in this facility cannot be assigned to a clinical microsystem as defined by TREC within this facility
• Located within a facility participating in the TREC observational study in Alberta or British Columbia
• Ten or more care aide responses to our survey that we use to assess organizational context and staff outcomes are available
• A care manager can be identified who leads this unit
• Not located within a facility participating in the TREC observational study in Alberta or British Columbia
• Less than ten care aide responses to our survey that we use to assess organizational context and staff outcomes are available
• No care manager can be identified who leads this unit
Based on these criteria, 203 units in 67 facilities are potentially eligible.
Enrolment, randomization, allocation
An independent person, not otherwise involved in this study, randomly assigned the 67 facilities to each of the three study arms. Randomization was stratified by health region (Edmonton, Calgary, Fraser Health, Interior Health) to account for the different policies within those regions that might influence organizational context and quality of care, and to facilitate delivery of the feedback intervention. We maintained regional proportionality of the 67 facilities within each study arm. From the list of facilities in each of the four regions, we selected the required number of facilities to be assigned to each of the three study arms by assigning a computer-generated random number to each facility. To facilities assigned to BAF or EAF we will offer additional feedback. We will explain to managers the specific extra feedback (treatment) they will receive, but we will blind them to group allocation.
Prevention of contamination
To limit the possibility of contamination, we will perform a cluster randomization. While we cannot prevent managers from talking to each other at regional or provincial meetings, contamination is less likely when study units are physically separate, interventions are more complex, and/or aim at changing behavior [116, 117]. We will also enlist participants’ agreement a priori to not share workshop tools with other managers or facilities during the study. The INFORM intervention is a complex intervention involving professionally facilitated face-to-face and virtual interaction with guided goal setting. It is therefore unlikely that managers in SF sites would be able to implement such an intervention unaided  or to change practice by virtue of interaction with intervention site managers at professional and other similar meetings. While we cannot fully blind all participants to all aspects of the study, we have undertaken comprehensive efforts to blind different stakeholder groups involved in INFORM as best as possible (details Additional file 6).
Outcomes and instruments
Additional file 7 contains a summary table listing study outcomes, tools used to assess these outcomes, and their psychometric properties. Our outcome definition includes the name of the outcome, the time point each outcome will be assessed, the method of aggregation, the metric (e.g., change from baseline or group differences at a given time), the definition of the concept assessed, psychometric properties and an example item of each scale used.
Based on arguments outlined in Additional file 5, we selected the ACT FI score as the primary outcome. FI is defined as “formal exchanges that occur between individuals working within an organization (unit) through scheduled activities that can promote the transfer of knowledge” . The FI scale consists of four items asking care aides how often, in the last typical month, they participated in (a) team meetings about residents, (b) family conferences, (c) change-of-shift report, and (d) continuing education (conferences, courses) outside the nursing home (rated from 1 = never to 5 = almost always). The overall score is generated by recoding each item (1 and 2 to 0; 3 to 0.5; 4 and 5 to 1) and summing recoded values (possible range: 0–4).
We will assess three additional organizational context factors (evaluation, social capital, slack time) using the ACT, which is embedded within the TREC care aide survey, a suite of validated survey instruments completed by computer-assisted personal interview. We will also capture response to major near misses and managers’ organizational citizenship behavior, using data from our TREC unit survey, and performance reports and quality improvement activities, using data from our TREC facility survey. These instruments are described elsewhere [48, 49, 80, 102, 119].
We will capture instrumental and conceptual best practice use, psychological empowerment, job satisfaction, and a number of individual staff attributes, using the TREC care aide survey.
We will obtain resident data from the RAI-MDS 2.0, which is used internationally for comprehensive geriatric assessment of the health, physical, mental, and functional status of nursing home residents . In Canada its use is mandated in several provinces/territories, as well as by the Canadian Institute of Health Information for national reporting . Data for calculating QIs are captured in quarterly assessments completed on all residents . The two practice sensitive (i.e., modifiable by care staff) QIs  worsening pain and declining behavioural symptoms will form the focus at unit and facility levels.
Evaluation of the intervention sessions
We will adapt previously used questionnaires to the needs of this study. These surveys will include closed and open ended questions relating to participants’ intention to change, satisfaction with intervention components and with the intervention overall. These evaluation forms will be completed by participants at the end of the Goal Setting Workshops for both the BAF and the EAF arms, and at the end of the two support workshops; virtual for the BAF arm and face-to-face for the EAF arm.
Evaluation of intervention fidelity
To ensure that each intervention session is delivered as planned, consistently across arms and time, staff will work from study protocols. To evaluate protocol fidelity, study staff will observe, using a protocol checklist, adherence to workshop protocols during actual workshop delivery. There will also be space for staff to record field notes during each intervention session. We will track (a) how many of the participants invited to the workshops attended the workshops, (b) how many of the participants attended the workshop the full time, and (c) how many of the participants completed the workshop evaluations. For consistency of workshop delivery, the same facilitator and researcher will be present in all workshops. The regional lead will change for each region, but will be consistent across workshops within one region. For consistency of evaluation, the same staff member will complete field notes and fidelity checklist across workshops. This person will also run workshop debriefing sessions with the facilitator and researcher-regional decision maker dyad.
Evaluation of processes in the facilities
Detailed documentation of the first support workshop: We will document discussions and results in the first support workshop around the above named topics using detailed field notes, and we will analyze them using qualitative content analysis.
Focus Group 1: We will hold the first focus group one month following the second support workshop for BAF and EAF arms. The second support workshop is the final intervention component to which the participants are exposed. This will provide participants with the opportunity to reflect on the entire INFORM Intervention. We will ask participants in each region to sign up for a teleconference focus group session held at varying times to meet the needs of individual schedules (held separately for BAF and EAF groups).
Focus Group 2: We will hold the second focus group (organized the same way as the first focus group) one month before the long-term follow up data collection for both intervention arms. This will enable the research team to assess sustainability of the INFORM Interventions.
We will also conduct semi-structured interviews with SF care unit managers. We will ask questions about: people’s understanding of the ACT data, changes made on their unit as a result of feedback they received during the Fall 2015 TREC dissemination meetings, and barriers/facilitators to change. These interviews will allow us to compare the three study arms in terms of their ability to use the ACT data to improve context – the ultimate aim of INFORM. SF interviews will also provide information on additional quality improvement activities outside of INFORM, which may have contributed to unexpected “context” improvement during the intervention period.
We will assess costs for delivering the INFORM interventions. We will not include costs related to developing the interventions and to assessing the effectiveness of the interventions (i.e., research costs). Using these data, we will assess and compare intervention delivery costs for the three study arms. We will collect data on all direct salary (workshop facilitator) and non-salary operating and travel expenses from University of Alberta accounting software, itemized by subcategory. The facilitator and study staff will keep detailed records of time invested in preparing and delivering each intervention. The cost of time invested by regional investigators and decision makers will be included. We will ask care unit managers monthly how much additional time and costs they and their staff spent on INFORM activities (i.e., not related to regular care or quality improvement activities), using standardized questions.
To compare the effectiveness of the three feedback strategies in improving the FI score, we will use mixed-effects regression models: multiple, linear, multi-level regression models including random and fixed (or mixed) effects.
Controlling for potential biases
We will account for multiple measures within each unit and clustering of units within facilities. We will adjust all analyses for the three stratification variables of the TREC facility sample (region, owner-operator model and facility size). We will compare characteristics of units and facilities using descriptive statistics at baseline, and adjust where there are significant differences between treatment groups (as baseline difference can occur by chance despite appropriate randomization). Should data not meet the assumptions of this model (multivariate normality, linearity, normally distributed, uncorrelated residuals, random effects with mean zero) the model will be adjusted accordingly. Within TREC, there is an extensive program to monitor and assure data quality, and a comprehensive data cleaning processes . We will carry out an intention-to-treat analysis, as this best reflects the pragmatic nature of the study. We will compare these results to a per-protocol analysis, which better reflects adherence/non-adherence with the intervention. We will consider a care unit to be adherent with the intervention if at least one representative of this unit attends the Goal Setting Workshop and at least one of the two Support Workshops. The person attending the Goal Setting Workshop can be different from the person attending the Support Workshop(s). Units only attending the Goal Setting Workshop or not attending any of the workshops will be defined as non-adherent. We have registered the trial with ClinicalTrials.gov (NCT02695836) and we will use CONSORT guidelines  to report its findings.
We will monitor change of secondary outcomes over time in each study arm, and compare outcomes between three study arms using descriptive statistics, statistical process control methods and appropriate significance tests (t tests for normally distributed, linear, continuous outcomes; non-parametric tests for variables that do not meet these assumptions; chi-squared tests for categorical outcomes). We will longitudinally track two practice sensitive RAI QIs (worsening pain, declining behavioural symptoms), using statistical process control methods [125–128]. We will calculate risk-adjusted and unadjusted indicators and, using control charts, graphically display unit and facility performances. These data will enable us to assess possible effects of INFORM on resident care as measured by resident QIs at a time consistent with the intervention. We will assign a dichotomous variable (improved/not improved) to each unit in the intervention. Then, using logistic regression with improvement as the outcome we will investigate the effects of context (using ACT scales), best practice use, and staff characteristics on improvement. To this end we have developed a reliable classification system for individual control charts [49, 50].
INFORM is the first study to systematically assess effectiveness of different strategies to feed back research-based performance data to nursing home care units in order to improve their performance. In TREC we collect comprehensive longitudinal data on nursing home and care unit structural characteristics; modifiable features of nursing home and care unit work contexts; care providers’ use of best practices, health and quality of worklife; and various resident outcomes based on the RAI-MDS 2.0. Systematic feedback of these research data has been an important part of the long term TREC program. With INFORM we will be able to not only feed back data, but to systematically and comprehensively assess the most effective and cost-effective way to do so.
Our feedback interventions are based systematically on theory and evidence related to audit and feedback in the health care and organizational literature, goal setting, complex adaptive systems, and our own experiences with feeding back research results. This addresses Ivers and colleagues’ urgent call for systematic use of theory in future audit and feedback studies [129–131]. Furthermore, in order to determine which approaches to audit and feedback are most likely to change behaviours and improve performance, and to understand why these approaches work, head-to-head comparisons of different audit and feedback approaches, rather than comparison of audit and feedback with no intervention, are needed [129, 130].
We hypothesize that BAF and EAF will be significantly more effective than SF in improving formal interactions, context overall, and care provider as well as resident outcomes on nursing home care units. We furthermore expect that BAF and EAF will be similarly effective in doing so. However, we expect BAF to be more cost-effective than EAF. Results of this study will enable us to develop a practical, sustainable, effective, and cost-effective assisted feedback strategy that managers and policy makers can routinely use to improve performance of nursing home care units.
We have started recruitments of facilities and managerial teams on March 01, 2016, and recruitment is ongoing.
We would like to thank Fiona Clement for her support in developing the cost assessment component of this study. We would also like to thank Elizabeth Anderson, Anne-Marie Boström, Lisa Cranley, James Dearing, Jayna Holroyd-Leduc, Johan Thor, and Lori Weeks for their active participation in the INFORM committee meetings, and for their valuable contribution to developing and reviewing workshop contents and documents, as well as, process evaluation materials and processes.
INFORM is funded by a Canadian Institutes of Health Research (CIHR) Transitional Operational Grant (Application Number: 341532). The funder has not played and will not play any role in study design; collection, management, analysis, and interpretation of data; writing of the report; and the decision to submit the report for publication, and they have not had and will not have ultimate authority over any of these activities.
Availability of data and materials
TREC has established comprehensive data and intellectual property policies, formalizing in detail the accountability for the management of data resources (including definition, production, access and usage of data), roles and responsibilities of TREC team members, procedures to obtain permission to produce outputs based on TREC data, and procedures to request TREC data in order to generate these outputs. These policies are available upon request. TREC data are housed in the secure and confidential Health Research Data Repository (HRDR) in the Faculty of Nursing at the University of Alberta (https://www.ualberta.ca/nursing/research/supports-and-services/hrdr), in accordance with the health privacy legislation of participating TREC jurisdictions. These health privacy legislations as well as the ethics approval covering TREC data does not allow the removal of completely disaggregated Resident Assessment Instrument – Minimum Data Set (RAI-MDS) 2.0 data (i.e., resident-level records) from the HRDR – even if de-identified. Aggregated RAI-MDS 2.0 summary data and de-identified TREC survey data specific to this study can be requested through the TREC Data Management Committee (firstname.lastname@example.org) on the condition that researchers meet and comply with the TREC and HRDR data confidentiality policies.
MH, PGN, LRG, RAA, GGC, HJL, JES, DT, ASW, and CAE are investigators or knowledge users on the INFORM grant, have substantially contributed to developing the research proposal, and are actively involved in launching and carrying out the study. CE is the PI of TREC and INFORM. MH carried out the sample size simulations, prepared the first draft of this manuscript including all figures and tables, coordinated the review process within the authorship team, and incorporated revisions. MH, LRG, PGN, and CAE lead the development of the intervention, RAA, GGC, HJL, JES, DT, and ASW actively contributed to the intervention development. LRG in collaboration with MH lead the development of the process evaluation. All authors critically reviewed the manuscript in several rounds, suggested revisions and read and approved the final manuscript.
The authors declare to have no competing interests.
Consent for publication
Ethics approval and consent to participate
This study was approved by the Research Ethics Boards of the University of Alberta (Pro00059741), Covenant Health (1758), University of British Columbia (H15-03344), Fraser Health Authority (2016-026), and Interior Health Authority (2015-16-082-H). Operational approval will be obtained from all included facilities if required. Written informed consent will be completed with facilities and study participants.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- European Commission. Long-term care for the elderly: provisions and providers in 33 European countries. Luxembourg: Publications Office of the European Union; 2012.Google Scholar
- Hix C, McKeon L, Walters S. Clinical nurse leader impact on clinical microsystems outcomes. J Nurs Adm. 2009;39:71–6.PubMedView ArticleGoogle Scholar
- Statistics Canada. Living arrangements of seniors: Families, households and marital status. Structural type of dwelling and collectives, 2011 Census of Population. Ottawa: Statistics Canada; 2011.Google Scholar
- Harrington C, Carrillo H, Garfield R. Nursing facilities, staffing, residents and facility deficiencies, 2009 Through 2014. Menlo Park: The Henry J. Kaiser Family Foundation; 2015.Google Scholar
- Alzheimer Society of Canada. Rising tide: the impact of dementia on canadian society. Toronto: Alzheimer Society of Canada; 2010.Google Scholar
- Congress of the United States - Congressional Budget Office. Rising demand for long-term services and supports for elderly people. Washington, DC: CBO; 2013.Google Scholar
- Hirdes JP, Mitchell L, Maxwell CJ, White N. Beyond the ‘iron lungs of gerontology’: Using evidence to shape the future of nursing homes in Canada. Can J Aging. 2011;30:371–90.PubMedView ArticleGoogle Scholar
- Estabrooks CA, Poss JW, Squires JE, Teare GF, Morgan DG, Stewart N, Doupe MB, Cummings GG, Norton PG. A profile of residents in prairie nursing homes. Can J Aging. 2013;32:223–31.PubMedView ArticleGoogle Scholar
- Hoffmann F, Kaduszkiewicz H, Glaeske G, van den Bussche H, Koller D. Prevalence of dementia in nursing home and community-dwelling older adults in Germany. Aging Clin Exp Res. 2014;26:555–9.PubMedView ArticleGoogle Scholar
- Stewart R, Hotopf M, Dewey M, Ballard C, Bisla J, Calem M, Fahmy V, Hockley J, Kinley J, Pearce H, et al. Current prevalence of dementia, depression and behavioural problems in the older adult care home sector: the South East London Care Home Survey. Age Ageing. 2014;43:562–7.PubMedView ArticleGoogle Scholar
- Bartfay E, Bartfay WJ, Gorey KM. Prevalence and correlates of potentially undetected dementia among residents of institutional care facilities in Ontario, Canada, 2009-2011. Int J Geriatric Psychiatry. 2013;28:1086–94.View ArticleGoogle Scholar
- Alzheimer’s Association. Alzheimer’s disease facts and figures. Chicago: Alzheimer’s Association; 2014.Google Scholar
- World Health Organization. World Health Statistics 2014. Geneva, Switzerland; 2014.Google Scholar
- Alzheimer’s Disease International. World Alzheimer Report 2015: The global impact of dementia - an analysis of prevalence, incidence, cost and trends. London: ADI; 2015.Google Scholar
- Health Council of Canada. Seniors in need, caregivers in distress: What are the home care priorities for seniors in Canada? Toronto: Health Council of Canada; 2012.Google Scholar
- Canadian Healthcare Association. New directions for facility-based long term care. In: Series: Policy brief (CHA Press). Ottawa: CHA; 2007.Google Scholar
- Jansen I, Murphy J. Residential long-term care in Canada: our vision for better seniors’ care. St. Laurent. Ottawa: Canadian Union of Public Employees; 2009.Google Scholar
- McGregor MJ, Ronald LA. Residential Long-Term Care for Canadian Seniors: Nonprofit, For-Profit or Does It Matter? Montreal: Institute for Research on Public Policy (IRPP); 2011.Google Scholar
- Berta W, Laporte A, Deber R, Baumann A, Gamble B. The evolving role of health care aides in the long-term care and home and community care sectors in Canada. Hum Resour Health. 2013;11:25.PubMedPubMed CentralView ArticleGoogle Scholar
- Castle NG, Ferguson JC. What is nursing home quality and how is it measured? The Gerontol. 2010;50:426–42.View ArticleGoogle Scholar
- Rantz MJ, Zwygart-Stauffacher M, Flesner M, Hicks L, Mehr D, Russell T, Minner D. Challenges of using quality improvement methods in nursing homes that “need improvement”. J Am Med Dir Assoc. 2012;13:732–8.PubMedPubMed CentralView ArticleGoogle Scholar
- Temkin-Greener HP, Zheng N, Katz PMD, Zhao HS, Mukamel DBP. Measuring work environment and performance in nursing homes. Med Care. 2009;47:482–91.PubMedPubMed CentralView ArticleGoogle Scholar
- Tolson D, Rolland Y, Andrieu S, Aquino J-P, Beard J, Benetos A, Berrut G, Coll-Planas L, Dong B, Forette F, et al. International Association of Gerontology and Geriatrics: A Global agenda for clinical research and quality of care in nursing homes. J Am Med Dir Assoc. 2011;12:184–9.PubMedView ArticleGoogle Scholar
- Rantz MJ, Zwygart-Stauffacher M, Hicks L, Mehr D, Flesner M, Petroski GF, Madsen RW, Scott-Cawiezell J. Randomized multilevel intervention to improve outcomes of residents in nursing homes in need of improvement. J Am Med Dir Assoc. 2012;13:60–8.PubMedView ArticleGoogle Scholar
- Organisation for Economic Co-operation and Development (OECD): OECD health statistics 2015: Long-term care resources and utilisation -- long-term care recipients. http://stats.oecd.org/index.aspx?queryid=30143## (2015). Accessed 28 June 2016.
- National Advisory Council on Aging. Press Release. NACA demands improvement to Canada’s long term care institutions. Ottawa: National Advisory Council on Aging; 2005.Google Scholar
- Dunn F. Report of the Auditor General on Seniors Care and Programs. Edmonton: Auditor General; 2005Google Scholar
- British Columbia Office of the Ombudsperson. The Best of Care: Getting It Right for Seniors in British Columbia. Vancouver: Office of the Ombudsperson, British Columbia; 2010.Google Scholar
- Long-Term Care Task Force Ontario. Long-Term Care Task Force on Residential Care and Safety: An Action Plan to Address Abuse and Neglect in Long-Term Care Homes. Ottawa: Long-Term Care Task Force Ontario; 2012.Google Scholar
- Doupe M, Brownell M, Kozyrskyj A, Dik N, Burchill C, Dahl M, Chateau D, DeCoster C, Hing E, Bodnarchuk J. Using Administrative Data to Develop Indicators of Quality Care in Personal Care Homes. Winnipeg: Manitoba Centre for Health Policy, Department of Community Health Sciences, University of Manitoba; 2006Google Scholar
- Baumgarten M, Margolis D, Gruber-Baldini AL, Zimmerman S, German P, Hebel JR, Magaziner J. Pressure ulcers and the transition to long-term care. Adv Skin Wound Care. 2003;16:299–304.PubMedView ArticleGoogle Scholar
- Friedman SM, Williamson JD, Lee BH, Ankrom MA, Ryan SD, Denman SJ. Increased fall rates in nursing home residents after relocation to a new facility. J Am Geriatr Soc. 1995;43:1237–42.PubMedView ArticleGoogle Scholar
- Mitchell SL, Kiely DK, Hamel MB, Park PS, Morris JN, Fries BE. Estimating prognosis for nursing home residents with advanced dementia. JAMA. 2004;291:2734–40.PubMedView ArticleGoogle Scholar
- Rapp K, Lamb SE, Buchele G, Lall R, Lindemann U, Becker C. Prevention of falls in nursing homes: Subgroup analyses of a randomized fall prevention trial. J Am Geriatr Soc. 2008;56:1092–7.PubMedView ArticleGoogle Scholar
- McGregor MJ, Tate RB, McGrail KM, Ronald LA, Broemeling AM, Cohen M. Care outcomes in long-term care facilities in British Columbia, Canada. Does ownership matter? Med Care. 2006;44:929–35.PubMedView ArticleGoogle Scholar
- Boström A-M, Kajermo KN, Nordström G, Wallin L. Registered nurses’ use of research fndings in the care of older people. J Clin Nurs. 2008;17:1430–41.Google Scholar
- Estabrooks CA, Derksen L, Winther C, Lavis JN, Scott SD, Wallin L, Profetto-McGrath J. The intellectual structure and substance of the knowledge utilization field: a longitudinal author co-citation analysis, 1945 to 2004. Implement Sci. 2008;3:49.PubMedPubMed CentralView ArticleGoogle Scholar
- Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7:50.PubMedPubMed CentralView ArticleGoogle Scholar
- Grol R, Grimshaw JM. From best evidence to best practice: Effective implementation of change in patients’ care. Lancet. 2003;362:1225–30.PubMedView ArticleGoogle Scholar
- Schuster M, McGlynn E, Brook R. How good is the quality of health care in the United States? Milbank Q. 1998;76:517–63.PubMedPubMed CentralView ArticleGoogle Scholar
- Schuster MA, McGlynn EA, Brook RH. How good is the quality of health care in the United States? Milbank Q. 2005;83:843–95.PubMedPubMed CentralView ArticleGoogle Scholar
- Masso M, McCarthy G. Literature review to identify factors that support implementation of evidence-based practice in residential aged care. Int J Evid Based Healthc. 2009;7:145–56.PubMedView ArticleGoogle Scholar
- Rahman AN, Applebaum RA, Schnelle JF, Simmons SF. Translating research into practice in nursing homes: Can we close the gap? The Gerontologist. 2012;52:597–606.PubMedPubMed CentralView ArticleGoogle Scholar
- Morley JE, Caplan G, Cesari M, Dong B, Flaherty JH, Grossberg GT, Holmerova I, Katz PR, Koopmans R, Little MO, et al. International survey of nursing home research priorities. J Am Med Dir Assoc. 2014;15:309–12.PubMedView ArticleGoogle Scholar
- Boström A-M, Slaughter SE, Chojecki D, Estabrooks CA. What do we know about knowledge translation in the care of older adults? A scoping review. J Am Med Dir Assoc. 2012;13:210–9.PubMedView ArticleGoogle Scholar
- Morley JE. Certified nursing assistants: a key to resident quality of life. J Am Med Dir Assoc. 2014;15:610–2.PubMedView ArticleGoogle Scholar
- Estabrooks CA, Hutchinson AM, Squires JE, Birdsell J, Cummings GG, Degner L, Morgan D, Norton PG. Translating Research in Elder Care: An introduction to a study protocol series. Implement Sci. 2009;4:51.PubMedPubMed CentralView ArticleGoogle Scholar
- Estabrooks CA, Squires JE, Cummings GG, Teare GF, Norton PG. Study protocol for the Translating Research in Elder Care (TREC): Building context – an organizational monitoring program in long-term care project (project one). Implement Sci. 2009;4:52.PubMedPubMed CentralView ArticleGoogle Scholar
- Estabrooks CA, Morgan DG, Squires JE, Bostrom AM, Slaughter SE, Cummings GG, Norton PG. The care unit in nursing home research: evidence in support of a definition. BMC Med Res Methodol. 2011;11:46.PubMedPubMed CentralView ArticleGoogle Scholar
- Norton PG, Murray M, Doupe MB, Cummings GG, Poss JW, Squires JE, Teare GF, Estabrooks CA. Facility versus unit level reporting of quality indicators in nursing homes when performance monitoring is the goal. BMJ Open. 2014;4:e004488.PubMedPubMed CentralView ArticleGoogle Scholar
- Anderson RA, Issel LM, McDaniel RR. Nursing homes as complex adaptive systems: Relationship between management practice and resident outcomes. Nurs Res. 2003;52:12–21.PubMedPubMed CentralView ArticleGoogle Scholar
- Rincon TA. Integration of evidence-based knowledge management in microsystems: a tele-ICU experience. Crit Care Nurs Q. 2012;35:335–40.PubMedView ArticleGoogle Scholar
- Likosky DS. Clinical microsystems: a critical framework for crossing the quality chasm. J Extra-Corporeal Technol. 2014;46:33–7.Google Scholar
- Nelson EC, Batalden PB, Godfrey MM. Quality by design: a clinical microsystems approach. Somerset: Wiley; 2007.Google Scholar
- Nelson EC, Batalden PB, Huber TP, Mohr JJ, Godfrey MM, Headrick LA, Wasson JH. Microsystems in health care: Part 1. Learning from high-performing front-line clinical units. Jt Comm J Qual Improv. 2002;28:472–93.PubMedView ArticleGoogle Scholar
- Lanham HJ. A complexity science perspective of organizational behavior in clinical microsystems. In: Sturmberg JP, Martin CM, editors. Handbook of systems and complexity in health. Secaucus: Springer; 2013. p. 715–25.View ArticleGoogle Scholar
- Estabrooks CA, Scott S, Squires JE, Stevens B, O’Brien-Pallas L, Watt-Watson J, Profetto-Mcgrath J, McGilton K, Golden-Biddle K, Lander J, et al. Patterns of research utilization on patient care units. Implement Sci. 2008;3:31.PubMedPubMed CentralView ArticleGoogle Scholar
- Nelson EC, Godfrey MM, Batalden PB, Berry SA, Bothe AE, McKinley KE, Melin CN, Muething SE, Moore LG, Wasson JH, et al. Clinical microsystems, Part 1. The building blocks of health systems. Jt Comm J Qual Patient Saf. 2008;34:367–78.PubMedView ArticleGoogle Scholar
- Williams I, Dickinson H, Robinson S, Allen C. Clinical microsystems and the NHS: a sustainable method for improvement? J Health Organ Manag. 2009;23:119–32.PubMedView ArticleGoogle Scholar
- Kjøs B, Botten G, Gjevjon E, Romøren T. Quality work in long-term care: the role of first-line leaders. International J Qual Health Care. 2010;22:351–7.View ArticleGoogle Scholar
- Morris JN, Moore T, Jones R, Mor V, Angelelli J, Berg K, Hale C, Morris S, Murphy K, Rennison M. Validation of long-term and post-acute care quality indicators. Cambridge, Massachusetts: Abt Associates Inc, Brown University; 2003.Google Scholar
- Anderson RA, Corazzini K, Porter K, Daily K, McDaniel Jr RR, Colon-Emeric C. CONNECT for quality: protocol of a cluster randomized controlled trial to improve fall prevention in nursing homes. Implement Sci. 2012;7:11.PubMedPubMed CentralView ArticleGoogle Scholar
- Colon-Emeric CS, McConnell E, Pinheiro SO, Corazzini K, Porter K, Earp KM, Landerman L, Beales J, Lipscomb J, Hancock K, et al. CONNECT for better fall prevention in nursing homes: results from a pilot intervention study. J Am Geriatr Soc. 2013;61:2150–9.PubMedPubMed CentralView ArticleGoogle Scholar
- Lanham HJ, Leykum LK, Taylor BS, McCannon CJ, Lindberg C, Lester RT. How complexity science can inform scale-up and spread in health care: understanding the role of self-organization in variation across local contexts. Soc Sci Med. 2013;93:194–202.PubMedView ArticleGoogle Scholar
- McDaniel RR Jr, Driebe DJ. Complexity science and health care management. In: Advances in health care management, volume 3. Edited by Savage GT, Blair JD, Fottler MD. Bingley: Emerald Group Publishing; 2001. p 11-36.Google Scholar
- McDaniel Jr RR, Driebe DJ, Lanham HJ. Health care organizations as complex systems: new perspectives on design and management. Advanced Health Care Management. 2013;15:3–26.View ArticleGoogle Scholar
- Janes N, Sidani S, Cott C, Rappolt S. Figuring it out in the moment: A theory of unregulated care providers’ knowledge utilization in dementia care settings. Worldviews Evid-Based Nurs. 2008;5:13–24.PubMedView ArticleGoogle Scholar
- Hughes CM, Donnelly A, Moyes SA, Peri K, Scahill S, Chen C, McCormack B, Kerse N. “The way we do things around here”: An international comparison of treatment culture in nursing homes. J Am Med Dir Assoc. 2012;13:360–7.PubMedView ArticleGoogle Scholar
- Lopez R, Amella EJ, Strumpf NE, Teno JM, Mitchell SL. The influence of nursing home culture on the use of feeding tubes. Arch Intern Med. 2010;170:83–8.PubMedPubMed CentralView ArticleGoogle Scholar
- Nieboer AP, Strating MMH. Innovative culture in long-term care settings: The influence of organizational characteristics. Health Care Manag Rev. 2012;37:165–74.View ArticleGoogle Scholar
- Hughes CM, Lapane K, Watson MC, Davies HTO. Does organisational culture influence prescribing in care homes for older people?: A new direction for research. Drugs Aging. 2007;24:81–93.PubMedView ArticleGoogle Scholar
- Bonner AF, Castle NG, Men A, Handler SM. Certified nursing assistants’ perceptions of nursing home patient safety culture: Is there a relationship to clinical outcomes? J Am Med Dir Assoc. 2009;10:11–20.PubMedView ArticleGoogle Scholar
- Svarstad BL, Mount JK, Bigelow W. Variations in the treatment culture of nursing homes and responses to regulations to reduce drug use. Psychiatr Serv. 2001;52:666–72.PubMedView ArticleGoogle Scholar
- van Beek APA, Gerritsen DL. The relationship between organizational culture of nursing staff and quality of care for residents with dementia: Questionnaire surveys and systematic observations in nursing homes. Int J Nurs Stud. 2010;47:1274–82.PubMedView ArticleGoogle Scholar
- Harvey G, Loftus-Hills A, Rycroft-Malone J, Titchen A, Kitson A, McCormack B, Seers K. Getting evidence into practice: The role and function of facilitation. J Adv Nurs. 2002;37:577–88.PubMedView ArticleGoogle Scholar
- Welsh S, Edwards M, Hunter L. Caring for smiles--a new educational resource for oral health training in care homes. Gerodontology. 2012;29:e1161–2.PubMedView ArticleGoogle Scholar
- Berta W, Teare GF, Gilbart E, Ginsburg LS, Lemieux-Charles L, Davis D, Rappolt S. Spanning the know-do gap: understanding knowledge application and capacity in long-term care homes. Soc Sci Med. 2010;70:1326–34.PubMedView ArticleGoogle Scholar
- Grol RP, Bosch MC, Hulscher ME, Eccles MP, Wensing M. Planning and studying improvement in patient care: The use of theoretical perspectives. Milbank Q. 2007;85:93–138.PubMedPubMed CentralView ArticleGoogle Scholar
- Damschroder L, Aron D, Keith R, Kirsh S, Alexander J, Lowery J. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.PubMedPubMed CentralView ArticleGoogle Scholar
- Estabrooks CA, Squires JE, Cummings GG, Birdsell JM, Norton PG. Development and assessment of the Alberta Context Tool. BMC Health Serv Res. 2009;9:234.PubMedPubMed CentralView ArticleGoogle Scholar
- French B, Thomas L, Baker P, Burton C, Pennington L, Roddam H. What can management theories offer evidence-based practice? A comparative analysis of measurement tools for organisational context. Implementation Sci. 2009;4:28.View ArticleGoogle Scholar
- Kaplan HC, Provost LP, Froehle CM, Margolis PA. The Model for Understanding Success in Quality (MUSIQ): building a theory of context in healthcare quality improvement. BMJ Quality & Safety. 2012;21:13–20.View ArticleGoogle Scholar
- Kitson A, Harvey G, McCormack B. Enabling the implementation of evidence based practice: a conceptual framework. Qual Health Care. 1998;7:149–58.PubMedPubMed CentralView ArticleGoogle Scholar
- Kitson AL, Rycroft-Malone J, Harvey G, McCormack B, Seers K, Titchen A. Evaluating the successful implementation of evidence into practice using the PARiHS framework: Theoretical and practical challenges. Implement Sci. 2008;3:1.PubMedPubMed CentralView ArticleGoogle Scholar
- McCormack B, McCarthy G, Wright J, Coffey A. Development and testing of the Context Assessment Index (CAI). Worldviews Evid-Based Nurs. 2009;6:27–35.PubMedView ArticleGoogle Scholar
- Barnett J, Vasileiou K, Djemil F, Brooks L, Young T. Understanding innovators’ experiences of barriers and facilitators in implementation and diffusion of healthcare service innovations: a qualitative study. BMC Health Serv Res. 2011;11:342.PubMedPubMed CentralView ArticleGoogle Scholar
- Cummings GG, Estabrooks CA, Midodzi WK, Wallin L, Hayduk L. Influence of organizational characteristics and context on research utilization. Nurs Res. 2007;56:S24–39.PubMedView ArticleGoogle Scholar
- Estabrooks CA, Midodzi WK, Cummings GG, Wallin L. Predicting research use in nursing organizations: A multilevel analysis. Nurs Res. 2007;56:S7–23.PubMedView ArticleGoogle Scholar
- Janssen MAP, van Achterberg T, Adriaansen MJM, Kampshoff CS, Schalk DMJ, Mintjes-de GJ. Factors influencing the implementation of the guideline Triage in emergency departments: A qualitative study. J Clin Nurs. 2012;21:437–47.PubMedView ArticleGoogle Scholar
- Pepler CJ, Edgar L, Frisch S, Rennick J, Swidzinski M, White C, Brown TG, Gross J. Unit culture and research-based nursing practice in acute care. Can J Nurs Res. 2005;37:66–85.PubMedGoogle Scholar
- Scott SD, Estabrooks CA, Allen M, Pollock C. A context of uncertainty: How context shapes nurses’ research utilization behaviors. Qual Health Res. 2008;18:347–57.PubMedView ArticleGoogle Scholar
- Berta W, Teare GF, Gilbart E, Ginsburg LS, Lemieux-Charles L, Davis D, Rappolt S. The contingencies of organizational learning in long-term care: factors that affect innovation adoption. Health Care Manag Rev. 2005;30:282–92.View ArticleGoogle Scholar
- Estabrooks CA. Translating research into practice: implications for organizations and administrators. Can J Nurs Res. 2003;35:53–68.PubMedGoogle Scholar
- Fleuren M, Wiefferink K, Paulussen T. Determinants of innovation within health care organizations. Int J Qual Health Care. 2004;16:107–23.PubMedView ArticleGoogle Scholar
- Glisson C. Assessing and changing organizational culture and climate for effective services. Res Soc Work Pract. 2007;17:736–47.View ArticleGoogle Scholar
- Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Q. 2004;82:581–629.PubMedPubMed CentralView ArticleGoogle Scholar
- Kaplan HC, Brady PW, Dritz MC, Hooper DK, Linam WM, Froehle CM, Margolis P. The influence of context on quality improvement success in health care: A systematic review of the literature. Milbank Q. 2010;88:500–59.PubMedPubMed CentralView ArticleGoogle Scholar
- Dopson S, Fitzgerald L, editors. Knowledge to Action? Evidence-Based Health Care in Context. Oxford: Oxford University Press; 2005.Google Scholar
- Contandriopoulos D, Lemire M, Denis JL, Tremblay E. Knowledge exchange processes in organizations and policy arenas: a narrative systematic review of the literature. Milbank Q. 2010;88:444–83.PubMedPubMed CentralView ArticleGoogle Scholar
- Hutchinson AM, Mallidou AA, Toth F, Cummings GG, Schalm C, Estabrooks CA: Review and synthesis of literature examining characteristics of organizational context that influence knowledge translation in healthcare: Technical Report. In. (KUSP Report No. 10-01-TR). Edmonton: University of Alberta. (ISBN: 978-1-55195-269-7); 2010Google Scholar
- Estabrooks C, Squires J, Hutchinson A, Scott S, Cummings G, Kang S, Midodzi W, Stevens B. Assessment of variation in the alberta context tool: the contribution of unit level contextual factors and specialty in Canadian pediatric acute care settings. BMC Health Serv Res. 2011;11:251.PubMedPubMed CentralView ArticleGoogle Scholar
- Estabrooks CA, Squires JE, Cummings GG, Morgan D, Stewart N, Ginsburg L, Hayduk L, McGilton K, Kang SH, Norton P. The influence of organizational context on best practice use by care aides in residential long-term care settings. Journal of the American Medical Directors Association. in press.Google Scholar
- Squires J, Estabrooks C, Scott S, Cummings G, Hayduk L, Kang S, Stevens B. The influence of organizational context on the use of research by nurses in Canadian pediatric hospitals. BMC Health Serv Res. 2013;13:351.PubMedPubMed CentralView ArticleGoogle Scholar
- Estabrooks CA, Squires JE, Hayduk LA, Cummings GG, Norton PG. Advancing the argument for validity of the Alberta Context Tool with healthcare aides in residential long-term care. BMC Med Res Methodol. 2011;11:107.PubMedPubMed CentralView ArticleGoogle Scholar
- Cummings GG, Hutchinson AM, Scott S, Norton PG, Estabrooks CA. The relationship between characteristics of context and research utilization in a pediatric setting. BMC Health Serv Res. 2010;10:168.PubMedPubMed CentralView ArticleGoogle Scholar
- Estabrooks CA, Hoben M, Poss JW, Chamberlain SA, Thompson GN, Silvius JL, Norton PG. Dying in a nursing home: Treatable symptom burden and its link to modifiable features of work context. Journal of the American Medical Directors Association. in press.Google Scholar
- Chan AW, Tetzlaff JM, Gotzsche PC, Altman DG, Mann H, Berlin JA, Dickersin K, Hrobjartsson A, Schulz KF, Parulekar WR, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;346:e7586.PubMedPubMed CentralView ArticleGoogle Scholar
- Arnold BF, Hogan DR, Colford Jr JM, Hubbard AE. Simulation methods to estimate design power: an overview for applied research. BMC Med Res Methodol. 2011;11:94.PubMedPubMed CentralView ArticleGoogle Scholar
- Guo Y, Logan HL, Glueck DH, Muller KE. Selecting a sample size for studies with repeated measures. BMC Med Res Methodol. 2013;13:100.PubMedPubMed CentralView ArticleGoogle Scholar
- Campbell MJ, Walters SJ. How to design, analyse and report cluster randomised trials in medicine and health related research. Chinchester: Wiley; 2014.View ArticleGoogle Scholar
- CRAN: The Comprehensive R Archive Network. http://cran.r-project.org/ (2014). Accessed February 28, 2015.
- Estabrooks CA, Teare G, Norton PG. Should we feed back research results in the midst of a study? Implement Sci. 2012;7:87.PubMedPubMed CentralView ArticleGoogle Scholar
- Bostrom A-M, Cranley L, Hutchinson AM, Cummings GG, Norton P, Estabrooks CA. Nursing home administrators’ perspectives on a study feedback report: A cross sectional survey. Implement Sci. 2012;7:88.PubMedPubMed CentralView ArticleGoogle Scholar
- Hutchinson AM, Batra-Garga N, Cranley L, Bostrom A-M, Cummings GG, Norton P, Estabrooks CA. Feedback reporting of survey data to healthcare aides. Implement Sci. 2012;7:89.PubMedPubMed CentralView ArticleGoogle Scholar
- Translating Research in Elder Care (TREC). Minimum number of care aide responses needed per care unit to obtain stable, valid and reliable unit-aggregated ACT scores. (Internal report, available upon request). Edmonton: TREC; 2010.Google Scholar
- Howe A, Keogh-Brown M, Miles S, Bachmann M. Expert consensus on contamination in educational trials elicited by a Delphi exercise. Med Educ. 2007;41:196–204.PubMedView ArticleGoogle Scholar
- Keogh-Brown M, Bachmann M, Shepstone L, Hewitt C, Howe A, Ramsay C, Song F, Miles J, Torgerson D, Miles S, et al. Contamination in trials of educational interventions. Health Technol Assess. 2007;11.Google Scholar
- May C, Mair F, Dowrick C, Finch T. Process evaluation for complex interventions in primary care: understanding trials using the normalization process model. BMC Fam Pract. 2007;8:42.PubMedPubMed CentralView ArticleGoogle Scholar
- Katz PR, Karuza J, Intrator O, Zinn J, Mor V, Caprio T, Caprio A, Dauenhauer J, Lima J. Medical staff organization in nursing homes: Scale development and validation. J Am Med Dir Assoc. 2009;10:498–504.PubMedPubMed CentralView ArticleGoogle Scholar
- Canadian Institute for Health Information. Data quality documentation, continuing care reporting system, 2013-2014. Ottawa, ON: CIHI; 2015.Google Scholar
- Canadian Institute for Health Information. Continuing care reporting system RAI-MDS 2.0 output specifications, 2014–2015. Ottawa: CIHI; 2015.Google Scholar
- Estabrooks CA, Knopp-Sihota JA, Norton PG. Practice sensitive quality indicators in RAI-MDS 2.0 nursing home data. BMC Res Notes. 2013;6:460.PubMedPubMed CentralView ArticleGoogle Scholar
- Squires JE, Hutchinson AM, Bostrom AM, Deis K, Norton PG, Cummings GG, Estabrooks CA. A data quality control program for computer-assisted personal interviews. Nurs Res Pract. 2012;2012.Google Scholar
- Forjaz MJ, Rodriguez-Blazquez C, Ayala A, Rodriguez-Rodriguez V, de Pedro-Cuesta J, Garcia-Gutierrez S, Prados-Torres A. Chronic conditions, disability, and quality of life in older adults with multimorbidity in Spain. Eur J Intern Med. 2015;26:176–81.PubMedView ArticleGoogle Scholar
- Diaz M, Neuhauser D. Pasteur and parachutes: When statistical process control is better than a randomized controlled trial. Qual Saf Health Care. 2005;14:140–3.PubMedPubMed CentralView ArticleGoogle Scholar
- Montgomery DC. Introduction to Statistical Quality Control. 3rd ed. New York: John WIley and Sons; 1996.Google Scholar
- Thor J, Lundberg J, Ask J, Olsson J, Carli C, Härenstam KP, Brommels M. Application of statistical process control in healthcare improvement: systematic review. Qual Saf Health Care. 2007;16:387–99.PubMedPubMed CentralView ArticleGoogle Scholar
- Timmerman T, Verrall T, Clatney L, Klomp H, Teare G. Taking a closer look: Using statistical process control to identify patterns of improvement in a quality-improvement collaborative. Qual Saf Health Care. 2010;19:1–6.View ArticleGoogle Scholar
- Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O’Brien MA, Johansen M, Grimshaw J, Oxman AD. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;6:CD000259.Google Scholar
- Ivers NM, Sales A, Colquhoun H, Michie S, Foy R, Francis JJ, Grimshaw JM. No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implement Sci. 2014;9:14.PubMedPubMed CentralView ArticleGoogle Scholar
- Colquhoun H, Brehaut J, Sales A, Ivers N, Grimshaw J, Michie S, Carroll K, Chalifoux M, Eva K. A systematic review of the use of theory in randomized controlled trials of audit and feedback. Implement Sci. 2013;8:66.PubMedPubMed CentralView ArticleGoogle Scholar