Implementation, mechanisms of change and contextual factors of a complex intervention to improve interprofessional collaboration and the quality of medical care for nursing home residents: study protocol of the process evaluation of the interprof ACT intervention package
Trials volume 23, Article number: 561 (2022)
To improve interprofessional collaboration between registered nurses (RNs) and general practitioners (GPs) for nursing home residents (NHRs), the interprof ACT intervention package was developed. This complex intervention includes six components (e.g., shared goal setting, standardized procedures for GPs’ nursing home visits) that can be locally adapted. The cluster-randomized interprof ACT trial evaluates the effects of this intervention on the cumulative incidence of hospital admissions (primary outcome) and secondary outcomes (e.g., length of hospital stays, utilization of emergency care services, and quality of life) within 12 months. It also includes a process evaluation which is subject of this protocol. The objectives of this evaluation are to assess the implementation of the interprof ACT intervention package and downstream effects on nurse–physician collaboration as well as preconditions and prospects for successive implementation into routine care.
This study uses a mixed methods triangulation design involving all 34 participating nursing homes (clusters). The quantitative part comprises paper-based surveys among RNs, GPs, NHRs, and nursing home directors at baseline and 12 months. In the intervention group (17 clusters), data on the implementation of preplanned implementation strategies (training and supervision of nominated IPAVs, interprofessional kick-off meetings) and local implementation activities will be recorded. Major outcome domains are the dose, reach and fidelity of the implementation of the intervention package, changes in interprofessional collaboration, and contextual factors. The qualitative part will be conducted in a subsample of 8 nursing homes (4 per study group) and includes repeated non-participating observations and semistructured interviews on the interaction between involved health professionals and their work processes. Quantitative and qualitative data will be descriptively analyzed and then triangulated by means of joint displays and mixed methods informed regression models.
By integrating a variety of qualitative and quantitative data sources, this process evaluation will allow comprehensive assessment of the implementation of the interprof ACT intervention package, the changes induced in interprofessional collaboration, and the influence of contextual factors. These data will reveal expected and unexpected changes in the procedures of interprofessional care delivery and thus facilitate accurate conclusions for the further design of routine care services for NHRs.
ClinicalTrials.gov NCT03426475. Registered on 07/02/2018.
For nursing home residents (NHRs) in developed countries, relatively large rates of emergency and hospital admissions are reported, although the benefits of these admissions for the health and well-being of this population are uncertain . In these countries, the 12-month incidence of hospital admission for NHRs varies from 12 to over 50% [2, 3], with a mean number of 0.3 to 1.3 admissions per NHR per year [2,3,4]. Analyses suggest that up to half of admissions may be avoidable without posing additional risks to residents’ health and well-being [5,6,7]. Effective strategies to prevent unnecessary admissions are therefore highly required.
In Germany, as in other countries (e.g., Switzerland, UK, USA), medical care for NHRs is provided by general practitioners (GPs) and other medical specialists who run their own offices or are employed by larger offices or ambulatory medical care centers. Because NHRs are frequently not able to visit a GP’s office, GPs visit nursing homes for regular consultations and are accessible in urgent situations during usual working hours on weekdays. Outside these hours, the NHR and the nursing home staff must draw on out-of-hours GP services and other emergency services when they consider medical support to be necessary. The choice of the GP and other physicians is, by law, at the discretion of the individual NHR. Therefore, nursing homes usually collaborate with a variety of GPs and other medical specialists. Empirical data suggest that on average, ten GPs per nursing home are involved in medical care for NHRs in Germany .
A potentially effective strategy to avoid hospitalization is to improve interprofessional collaboration between nursing staff in nursing homes and GPs. Descriptive studies suggest that a continuous flow of information between registered nurses (RNs) and GPs can help prevent hospital admissions . Furthermore, a multicenter qualitative study involving NHRs, their relatives, RNs, and GPs of 18 nursing homes in Germany revealed a need for improvements in shared medical care by GPs and nursing home staff for NHRs, especially with regard to the mutual accessibility of GPs and RNs, person-centered allocation of tasks, scheduling and conduct of GPs’ nursing home visits, and the quality of interprofessional communication and flow of information . Based on these findings, a multicomponent strategy, the “interprof ACT intervention package,” was developed and piloted to improve the quality of RN-GP collaboration and thus the quality of medical care for NHRs in Germany . The core components of this intervention package are (1) name badges for GPs and RNs during GPs’ visits, (2) designating a contact person in the GP’s office and among the nursing home’s RNs, (3) a shared definition of mandatory rules for GPs’ availability, (4) standardized procedures for the scheduling, conduct and postprocessing of GPs’ visits in the nursing homes, (5) standardized forms for pro re nata medication prescriptions, and (6) shared goal setting involving the NHR and her/his relative(s), the GP and the RN in charge of the NHR. These components address key elements of interprofessional collaboration and communication in primary care for the elderly  and can be adjusted to local requirements and needs, such as pre-existing standards for GPs’ visits or forms for recording pro re nata medication prescriptions (Additional file 1).
The interprof ACT intervention package is currently being evaluated in a multicenter cluster-randomized controlled trial (CRT) (ClinicalTrials.gov NCT03426475). In this trial, 34 nursing homes across three study centers in Germany (Göttingen, Hamburg, and Lübeck) are randomly assigned to the implementation of the interprof ACT intervention package (intervention group) or usual care (control group). In the intervention group, implementation of the intervention package is facilitated by various strategies that aim to involve all local target groups of the interprof ACT intervention from the beginning onward. The main strategies are (1) the nomination and training of interprof ACT agents (“interprof ACT-Verantwortliche”, IPAVs) within the nursing homes, whose main tasks are to champion, coordinate, and monitor the implementation within the facility and maintain close contact with participating GPs and the local study team, (2) an in-house kick-off meeting involving all stakeholders, such as NHRs and their relatives, RNs, GPs, nursing home management, and home advisory boards, to discuss and agree on local adjustments to the interprof ACT intervention package, and (3) regular supervision of the IPAVs by the local study team (Fig. 1 and Additional file 1). The main goal of the interprof ACT trial is to evaluate the effects of the intervention package concerning the rate of hospital admissions (primary outcome) and other patient-important outcomes, such as quality of life, the utilization of emergency services, or prescriptions of potentially inappropriate medication among participating NHRs within 12 months. Trial outcome data are assessed at baseline (T0), 6 months (T1), and 12 months (T2) post randomization (Fig. 2). The full trial protocol has been published elsewhere .
Given the multiple intervention components and the allowed degree of tailoring, the interprof ACT intervention package should be viewed as a complex intervention [13, 14]. This complexity is enhanced by various contextual factors at the health system, organizational, and individual levels that may influence the reach, dose, and fidelity of the implementation, including attitudes of health professionals and NHRs as well as the existing local infrastructure and procedures of nurse–physician collaboration and communication . To capture the assumed heterogeneity of the implementation of the interprof ACT intervention package, the impact of this implementation on the quality of the RN-GP collaboration, and potential influences of contextual factors, a detailed process evaluation is needed in addition to the main trial [16, 17]. Therefore, a mixed methods process evaluation is embedded into the main trial to achieve systematic insights into the implementation of the interprof ACT intervention package (Fig. 2).
The process evaluation is based on a logic model  that illustrates the expected causal pathway of the interprof ACT intervention and potential moderating effects of relevant contextual factors . Within this model, it is assumed that the implementation strategies described above induce various activities by the IPAV, the nursing staff, and GPs to adopt and integrate the jointly chosen interprof ACT components into their daily routines. The degree and quality of this implementation, as measured by the dose and reach or the fidelity of implemented components, ultimately affect key elements of the RN-GP collaboration. These key elements are based on the conceptual model of interprofessional collaboration in primary elderly care  and are expected to mediate the effects of the interprof ACT intervention on the distal outcomes, i.e., the primary and secondary patient-important outcomes of the main trial (Fig. 3).
The overall aims are to assess the change in RN-GP collaboration induced by the implementation of the interprof ACT intervention package and to identify contextual determinants of implementation success and its downstream effects on intermediate outcomes. Following existing frameworks for process evaluations of complex interventions [16, 17], this overall aim is divided into several sub-objectives, each focusing on one specific part of the assumed causal pathway from group allocation to the impact of the interventions on the intermediate outcomes (i.e., the interprofessional collaboration). To capture the whole picture, the process evaluation comprises a quantitative part and a qualitative part. The main objective of the quantitative part is to estimate the degree of implementation of the interprof ACT components and its downstream effects on the quality of the RN-GP collaboration. Therefore, it addresses the following research questions:
What is the degree of implementation of the interprof ACT components with regard to dose and reach, fidelity, and local adaptations? To what degree does this implementation output vary between clusters?
What are the effects of the implementation of interprof ACT components on the quality of RN-GP collaboration?
Which contextual factors at the system, organization, intervention, staff, and NHR levels influence the implementation of the interprof ACT components as well as the quality of RN-GP collaboration?
The qualitative part mainly aims to assess the interprof ACT intervention with regard to feasibility and prospects for successive implementation into routine care. Therefore, it mainly addresses the following research questions:
How do relevant working processes change as a result of the implementation of interprof ACT intervention package?
How can effective processes be described and defined in a standardized form (process map)?
Altogether, the results of the process evaluation will facilitate the understanding of causal mechanisms behind the main trial findings and point to important facilitators and barriers as well as preferred strategies for large-scale implementation of the interprof ACT intervention package in routine care given that the trial results will be in favor of the intervention.
This study uses a mixed methods triangulation design  that combines the collection and analysis of quantitative data on the process, context, and output (nurse–physician collaboration, medical delivery) of the intervention implementation with concurrent qualitative in-depth inquiries of the implementation procedures and context-bound influences. Data for the quantitative part will be repeatedly collected alongside the trial in all trial clusters by means of questionnaires, standardized interviews, and minutes targeting all parties involved in the intervention package implementation and/or the medical care for NHRs. The qualitative data collection will be conducted in a subsample of eight nursing homes and mainly focuses on the interaction between involved health professionals and their working processes. Data will be collected by a combination of non-participating observations and semistructured interviews conducted at different measurement points throughout the trial (Table 1).
The protocol of this study follows existing recommendations for process evaluations of complex interventions [16, 17] and the reporting of complex interventions  and implementation studies (StaRI) . A detailed description of the interprof ACT intervention package and preplanned implementation strategies based on these reporting standards is shown in the supplementary data (Additional file 1).
Different trial partners lead the quantitative and qualitative parts. The quantitative part is mainly planned and overseen by the Nursing Research Unit, University of Lübeck, and all three study centers (Departments of General Practice and Primary Care in Göttingen and Hamburg, Nursing Research Unit in Lübeck) are involved in the data collection. The qualitative part is planned and will be conducted by the Chair of Organization and Corporate Development, Faculty of Economic Sciences, Georg-August-University Göttingen. However, both study parts were conceptualized in close collaboration with all trial partners, and the quantitative and qualitative data are assumed to complement each other and are triangulated at several stages of the research process. During the implementation of the trial, preliminary findings from the qualitative inquiries at the first measurement point (T0) will be taken into account in the choice of variables to be included in the quantitative measurements at T1 and T2. At the analysis stage, descriptive findings of both parts will be cross-mapped to inform statistical models to evaluate the downstream effects of the interprof ACT intervention on nurse–physician collaboration and the medical care provided to the NHRs.
Target populations and data sources
The quantitative part of the process evaluation will be conducted in all 34 participating nursing homes, whereas the qualitative part will be performed in a subsample of eight participating facilities (four per study group) with at least one nursing home per study group per study center. For the recruitment of this subsample, local study centers will invite participating nursing homes to take part in this qualitative inquiry until the subsample is completed. Additional criteria, such as the size and location of the nursing homes, will be considered in the recruitment to achieve a heterogeneous sample that allows for the identification of differences arising from these characteristics. Participating nursing homes will receive detailed information about the process evaluation and the quantitative and qualitative data collection methods. Participation in the process evaluation will be voluntary.
Table 1 provides an overview of the populations targeted by this process evaluation at the several measuring points, including information about the data sources used for the quantitative and qualitative parts. Both the quantitative and qualitative parts target RNs and GPs of participating nursing homes (all facilities or subsamples). The quantitative part additionally includes IPAVs, NHRs, nursing home directors, and study team members (Table 1). In the qualitative part, the latter three groups will only be involved when representatives of these groups take part in the kick-off meetings that will be held by nursing homes assigned to the intervention group to discuss the interprof ACT intervention and to reach consensus about which components should be implemented in this facility and which local adjustments are required to these components (Additional file 1). The detailed eligibility criteria for each target population and the recruitment strategies are outlined in Table 1. For NHRs, the same eligibility criteria will be applied as in the main trial , with the restriction that only residents who are themselves able to answer simple questions about satisfaction with medical care will be eligible.
Table 1 also provides information about the target sample size for each sub-population in the quantitative and qualitative parts of the process evaluation. For the quantitative part, no formal sample size estimation was carried out since this part of the study, like the qualitative part, is explorative by nature. However, the recruitment strategies in place are assumed to yield representative samples of each sub-population.
Data collection methods
Quantitative process evaluation
Table 2 provides an overview of the outcomes measured for the quantitative part of the process evaluation, including the populations targeted by these measurements. The outcomes were chosen based on the logic model underlying this study (Fig. 3). Therefore, data on five major outcome domains will be collected (highlighted in bold numbers in Fig. 3): (1) implementation strategies and activities, (2) implementation of the interprof ACT intervention package (as a whole and single components), (3) key elements of interprofessional collaboration, (4) further domains related to interprofessional collaboration and medical care, and (5) contextual factors. The major outcome domains (1) to (4) relate to the assumed change mechanisms (intermediate outcomes), while the major outcome domain (5) reflects potentially relevant moderators. Each of these major domains comprises multiple domains and, in some cases, additional subdomains, which are measured in terms of different dimensions such as attitudes, dose and reach, adaptations, current practice, or self-reported quality and satisfaction.
The major outcome domain “implementation strategies and activities” includes various dimensions (e.g., quality and satisfaction) of the implementation of the kick-off meetings and the IPAV training and the IPAV supervision. These dimensions will be assessed from the perspectives of the groups targeted by these implementation strategies (e.g., participants of the kick-off meetings, IPAVs) and those responsible for the application of these implementation strategies (e.g., study team members). Data will be gathered by means of semistructured minutes and questionnaires consisting of self-developed items. In addition, information will be collected on the activities undertaken by the IPAVs and the nursing staff to implement the interprof ACT interventions. This information will be collected from the IPAVs, RNs, and the study team members by means of standardized questionnaires consisting of self-developed items and items of the Normalization Measure Development (NoMAD) questionnaire [29, 30]. This instrument measures the core constructs of the Normalization Process Theory (NPT), i.e., coherence, cognitive participation, collective action, and reflexive monitoring, which describe and explain critical mechanisms of successful implementation of new complex interventions into the routines of healthcare practice [36, 37]. These constructs emphasize the continuous activities undertaken by involved persons both individually and collectively to make the new intervention meaningful, usable, and useful for their practice, to adapt novel and existing practices to each other and to evaluate and reflect on noticeable effects . The convergent and structural validity and the reliability of the NoMAD questionnaire have been confirmed in several surveys among health professionals [29, 30, 38]. For this process evaluation, items reflecting all four NPT core constructs were chosen from the German version of the NoMAD questionnaire  and adapted to the terminology of the interprof ACT intervention package (Additional file 2).
The major outcome domain “implementation of the interprof ACT intervention package” contains several domains and subdomains to measure the success of the implementation activities in terms of the following dimensions: dose and reach of the intervention package as a whole and of the individual interprof ACT interventions, adaptations made to the single interventions, and attitudes and subjectively experienced quality with regard to these interventions. Adaptations made to the interprof ACT interventions will be recorded by study team members by completing semistructured minutes of the kick-off meetings and the supervision encounters as well as by IPAVs (questionnaires at T1 and T2). Dose and reach are directly measured by repeated 10-step global judgments of the IPAVs on the degree of implementation (1 = not implemented, 10 = fully implemented). The judgments of IPAVs, RNs, nursing home directors, GPs, and NHRs on attitudes, current practices and/or subjectively experienced quality of/satisfaction regarding interprof ACT interventions will provide an indication of the fidelity and quality of the implementation. Respective judgments will be collected at T0 and T2 by means of standardized questionnaires comprising self-developed items and items of existing measurement instruments with established psychometric performance [26, 28,29,30,31] (Table 2 and Additional file 2).
Two other major domains, “key elements of interprofessional collaboration” and “further domains related to interprofessional collaboration and medical care,” address the key elements of interprofessional collaboration (e.g., interprofessional communication, involvement of NHRs, contribution of involved professions), additional aspects of collaboration and experiences of quality of care assumed to be influenced by the interprof ACT intervention package (Fig. 3). These outcomes will be assessed in terms of attitudes, current practice, and subjectively experienced quality or satisfaction from various perspectives (IPAVs, RNs, nursing home directors, GPs, and NHRs) at different measurement points (T0, T2). Standardized questionnaires consisting of self-developed items and items of established measurement tools [25,26,27,28, 31,32,33] will be used (Table 2 and Additional file 2).
The contextual factors (major outcome domain (5)) comprise domains and subdomains that relate to attributes of the health system, the involved organizations, the interprof ACT intervention package, the staff, and the NHRs, which are assumed to have moderating effects on implementation success, interprofessional collaboration, and the quality of medical care for NHRs (Fig. 3). Following the underlying evidence , they are classified as factors at the macro-, meso-, and microlevels. Data on these domains and subdomains will be collected from each target group that contains or is affected by the contextual factors of interest. To gather these data, standardized questionnaires, case report forms, and semistructured minutes of supervision encounters will be used. Whenever justified and possible, items of validated measurement tools [26, 29, 30, 34, 35] will be integrated into these data collection instruments, complemented by self-reported items (Table 2 and Additional file 2).
The questionnaires and minute templates were designed based on existing measurement tools used in previous process evaluation studies on complex interventions to improve the quality of care in long-term institutions [8, 40]. The questionnaires for the RNs and GPs consist of two parts: one general part (part A) containing items to be answered independently of collaboration with a certain GP (questionnaires for the RNs) or a certain nursing home (questionnaires for the GPs) and a specific part (part B) that contains items specifically referring to collaboration with a certain GP’s office (questionnaires for the RNs) or a certain nursing home (questionnaires for the GPs). The items of the general part A cover all major outcome domains but do not include items on current practice and the subjectively experienced quality of/satisfaction with interprofessional collaboration and medical care within the major outcome domains “implementation of interprof ACT intervention package” and “key elements of interprofessional collaboration.” Additionally, part A does not include items referring to contextual factors that are specific to one collaborating GP office or nursing home (e.g., staff competencies in this institution). Respective outcomes are assessed specifically by part B of the questionnaires for GPs and RNs. Here, GPs are asked to complete a part B questionnaire specifically for each nursing home in which they provide medical care for NHRs participating in this trial. This means that the questionnaires for the GPs include as many parts B as the number of participating nursing homes with which they are currently collaborating. In contrast, the questionnaires for the RNs include specific part B questionnaires only for a 30% random sample of the GPs’ offices currently responsible for medical care for participating NHRs in their nursing home. This approach is used to avoid a large number of part B questionnaires being completed by RNs in nursing homes collaborating with more than six GPs’ offices .
Questionnaires for the RNs and GPs were piloted before the T0 measurements by means of cognitive interviews with nursing staff (n = 8) of 3 nursing homes and 10 GPs not involved in the main study. Within this pilot study, the completeness and comprehensibility of the questionnaires were assessed, and the relevance of the items of the major outcome domain “key elements of interprofessional collaboration” was determined by means of the content validity index. The pilot study revealed only minor need for revision of the questionnaires. Based on these findings, single items were deleted, added, or rewritten.
All questionnaires are paper-based. The questionnaires for the RNs, IPAVs, and GPs will be completed by the target persons. Participating GPs will receive postal questionnaires together with prepaid opaque return envelopes. Questionnaires for RNs and IPAVs will be directly handed to them by study team members during scheduled follow-up visits. Anonymously completed questionnaires will be collected by the study team members (IPAV questionnaires) or NHD (RN questionnaires). Questionnaires for NHRs will be part of structured oral interviews conducted by study team members for the collection of patient-reported data for the main trial at follow-up visits T0 and T2 .
Qualitative process evaluation
Data collection for the qualitative part comprises non-participating observations and semistructured interviews. Table 3 presents an overview of the themes considered in interviews and observations.
For the observations, one or two researchers will visit each of the eight participating nursing homes at T0, T1, and T2 to observe GP-RN interaction and communication as well as the implementation of the interprof ACT intervention and associated work process changes in the intervention group. Short conversations with GPs and RNs will be held before or after the observations if possible. The participation of NHRs is not intended but may be considered in relevant situations of communication between GPs, RNs, and NHRs. In addition to these observations, one researcher will attend the kick-off meetings to observe the interaction and communication between GPs and RNs when choosing and tailoring interprof ACT intervention components. Observations will be carried out with semistructured observational guidelines . The observers will appear explicitly as scientists, and there will be no active participation of observers in the situation . Observers will take notes while observing. The notes will be pseudonymized and then stored in a word processor (e.g., Microsoft Word).
In addition to the non-participating observations, interviews will be conducted with GPs and RNs. These interviews complement the observation data for the description of the care process and the evaluation of process changes through the interprof ACT intervention package in the intervention group. Furthermore, they serve to clarify observations and assess respondents’ perceptions of the processes. Semistructured interview guidance will be developed by the researcher team consisting of open questions that allow an unbiased and flexible approach. Interviews will be conducted by one interviewer with one respondent at a time. Interviews will take place in the nursing homes or at any other place chosen by the interview partners. Interviews will be audio-recorded and subsequently transcribed and pseudonymized. If the interview partners prefer that the interview will not be recorded, the interviewer will take notes. During the data collection process, the research team will regularly discuss the initial findings and adjust the interview guidelines to account for emerging themes.
For observations, an overall time frame of 1 to 3 days at 1 to 3 h a day per nursing home per measurement point is expected. Approximately four interviews involving one or two GPs and two RNs are planned per nursing home per measuring point.
Study data in the process evaluation will be recorded both pseudonymized (Nursing or facility directors, NHRs) and anonymized (RNs, GPs, IPAVS, and study team members). For pseudonymized data, the personal data of the study participants will be kept separately from the study data. A retrospective correlation to a person is only possible with the help of a “key” which is maintained in the study center. The data will be entered into the electronic database according to the four-eye principle. The paper-based data is stored in the participating study centers and electronically in the database on a server. To ensure data quality, a plausibility check of the data will be carried out by an independent monitor. All original data will be stored for 10 years and destroyed afterwards. Generally, all data will be handled according to current data protection law.
Data will be analyzed in a stepwise manner. First, quantitative and qualitative data will be analyzed independently from each other, followed by iterative triangulation. All analyses will be explorative, framed by the underlying logical model (Fig. 3). Statistical analyses will be performed using IBM SPSS Statistics (version 20) and Microsoft Excel. Software packages of MAXQDA (VERBI GmbH) and SAP Signavio will be used to facilitate the management, analysis, and visualization of qualitative data.
To answer the three research questions of the quantitative process evaluation, data on outcome domains for each measurement point will be analyzed by means of descriptive statistics: frequencies (proportions) in the case of categorical variables and measures of location (mean and median) and dispersion (standard deviation (SD) and interquartile range (IQR)) in the case of ordinal or continuous variables. If psychometrically justified, items referring to the same outcome subdomain or dimension will be summarized into one sum score. Only items using the same answer scale will be considered for this aggregation, and the structural validity and reliability will be tested by means of exploratory factor analyses (principal component analysis) and Cronbach’s alpha. Newly created sum-scores will be maintained if each item loads with an eigenvalue of ≥0.25 on the one factor assumed a priori and/or the Cronbach’s alpha is above 0.4. These thresholds are specifically defined for the purpose of this study, taking statistical conventions for estimation and interpretation of these measures into account  and assuming that in many cases, not more than five items will be eligible for combination into one sum score.
Initially, all quantitative data will be analyzed and aggregated at the cluster (i.e., nursing home) level. For this aggregation, a two-step approach will be used. First, the direction and size of changes over time will be descriptively estimated for single items or subscales. For the assessment of pre-post changes (T0 versus T2), the difference between the two mean or median values will be estimated. Following the effect size Cohen’s d , a mean difference with a size of ≥20% of the standard deviation across study groups will be interpreted as a relevant change or difference. For median values, a median difference of 0.5 in case of ≤5-step scales or 1.0 in case of >5-step scales will be regarded as relevant. Change estimates will be classified as follows: relevant changes in the favorable direction reflect positive changes, relevant changes in the nonfavorable direction reflect negative changes, and nonrelevant changes are interpreted as indifferent effects (no changes). Some items will be assessed only at the T2 follow-up visit to directly measure the subjectively perceived degree of change in certain collaboration or care procedures. For these items, the size of the change will be classified based on the distance between the estimated mean value and the most neutral value of the scale. If the mean value is higher (positive values) or lower (negative values), the distance between 20% of the pooled standard deviation and the most scale-neutral value, a relevant change will be assumed, either positive or negative, depending on the direction. In other cases, the mean values will be regarded as being indifferent.
Second, the change estimates across items and subscales will be aggregated by means of vote counting, i.e., the number of positive and negative changes and indifferent findings will be counted and visually depicted in tables for single-outcome subdomains or dimensions. Vote counting findings from each perspective (NHRs, NHDs, RNs, IPAVs, GPs) will be summarized per nursing home in tables (intervention group only), and nursing homes will then be ranked with regard to their degree of implementation achieved for each interprof ACT component. Based on this ranking, nursing homes will be inspected for differences in contextual factors. These findings will then inform the triangulation of qualitative and quantitative data and the subsequent creation of a preliminary model to answer the second research question, i.e., the impact of the degree of implementation on the quality of interprofessional collaboration, taking moderating effects of potentially relevant contextual factors into account.
Furthermore, for the outcome domains “implementation of the interprof ACT intervention package,” “key elements of interprofessional collaboration,” and “further domains related to interprofessional collaboration and medical care,” the vote counting findings per nursing home will be visually compared between the intervention and control groups to explore between-group differences in the changes in attitudes and current practice and the quality of RN-GP collaboration.
Qualitative data analyses will serve to explore processes and identify intervention-induced process changes in the intervention group based on qualitative content analysis procedures . First, individual cases will be analyzed to describe the process patterns of time-ordered events (including involved actors and contextual factors) for each nursing home at different measurement points using process modeling techniques. Cases will then be compared to note similarities and differences across cases and to identify intervention-induced changes. Effective changes will be described and defined in a standardized form. To ensure reliable analysis of qualitative data, it will be conducted by two coders (i.e., two researchers) who independently assign codes, which will then be compared and discussed by the research team. In the course of this iterative process, the aim will be to identify relevant process changes and to reveal specific process patterns across cases that enable the development of general process descriptions by using process maps.
In addition to the separate analyses of quantitative and qualitative data, synthesis and joint analysis of quantitative and qualitative findings are planned to enhance the quality of the process evaluation and to evaluate and adapt the underlying logic model in accordance with the empirical findings. Concretely, triangulation will allow us to explore the impact of the degree and fidelity of the implementation of the interprof ACT intervention package on the quality of interprofessional collaboration, the interaction between contextual factors and the success of implementation, and the quality of interprofessional collaboration.
For this purpose, quantitative and qualitative findings will be integrated at different timepoints. First, after collecting initial T0 data sets, preliminary qualitative insights regarding the determinants of interprofessional collaboration and contextual factors will be exchanged and discussed against preliminary experiences derived from the supervision of IPAVs to allow for the adjustment of subsequent T1 and T2 questionnaires for emerging themes.
Second, after completion of the T2 data collection, the findings of the separate quantitative and qualitative analyses will be systematically cross-mapped in “joint displays” to compare and contrast insights. This procedure allows us to consider detailed variations within and across clusters (i.e., nursing homes) regarding implementation success, the quality of RN-GP collaboration, and potentially relevant contextual factors. In particular, this step enables a data-driven selection of variables for a model that, based on the a priori logic model, depicts the associations between these variables. To assess the direction and size of the impact of the implementation success on important variables of RN-GP collaboration, this model will be subject to generalized linear regression analyses. These analyses will both adjust for cluster-related effects and account for moderating effects of potentially relevant contextual factors. The exact regression model will be chosen based on the actual size and characteristics of available data. By ensuring that quantitative and qualitative analyses build upon each other , this triangulation facilitates the creation of a theoretically and empirically informed model for the final statistical exploration of core associations of assumed causal pathways of the interprof ACT intervention package.
This process evaluation integrates a variety of qualitative and quantitative data sources that span the perspectives of all stakeholders involved in the interprof ACT trial. It follows an evidence-based, theoretically underpinned logic model that illustrates the assumed causal pathway of the interprof ACT intervention package, including critical intermediate outcomes (mediators) and potentially relevant contextual factors (moderators). The basic assumption of the interprof ACT trial is that the consequential implementation of the interprof ACT intervention components will improve key elements of RN-GP collaboration, such as the quality of communication between these two professions (e.g., in the context of GPs’ nursing home visits) or shared goal setting, care planning, and coordination , particularly in view of likely or actually occurring deterioration of NHRs’ individual health conditions. It is further assumed that these improvements in RN-GP collaboration will lead to optimized procedures of medical care for NHRs and thus prevent unnecessary utilization of hospital or emergency care services. However, recent trials on the effectiveness of complex interventions pursuing similar goals as the interprof ACT trial (i.e., the prevention of unplanned hospital admissions or emergency care utilization through adaptations to interprofessional care pathways for NHRs) have shown mixed results [45, 46]. In secondary analyses of one of these trials, the INTERACT trial, a marked dose–response relation was observed, indicating that nursing homes that achieved higher degrees of implementation of the complex intervention reported lower rates of all-cause hospitalizations compared to nursing homes with poor implementation success . The same analyses also suggest that successful implementation of the INTERACT measures was largely driven by the motivation of nursing homes to implement quality improvement strategies rather than by the implementation strategies applied in this trial.
These findings underscore the limited predictability of the exact causal pathways and contextual factors that determine the downstream effects of complex interventions targeting the dynamic interplay of organizational conditions, professionals’ behavior, and NHRs’ preferences and needs that interact in shaping interprofessional care delivery to NHRs . The findings also highlight the need for a systematic and comprehensive process evaluation alongside the evaluation of the clinical effectiveness of such interventions. Therefore, we expect this mixed methods study to provide valid accounts of the implementation of the interprof ACT intervention package, the changes induced in interprofessional collaboration and the influence of contextual factors.
While the interprof ACT process evaluation triangulates quantitative and qualitative data from all parties involved in the delivery and reception of the interventions under evaluation, some potential limitations have to be considered. The qualitative part will be conducted only in a voluntary subsample of nursing homes in the intervention and control groups, thus increasing the risk or recruitment bias due to participation of highly motivated facilities. However, since recruitment strategies for the qualitative inquiries will strive for a heterogeneous sample in each study group and the risk of recruitment bias may be equally high across the intervention and the control groups, likely selective participation may affect the credibility of the process evaluation findings only to a minor degree. Furthermore, patient-reported data on the NHRs’ perception of the collaboration between the RNs and the GPs and their satisfaction with the quality of received medical care will be limited to those able to speak for themselves. Proxy assessment of these patient-reported experiences by nursing staff is considered as being not justified, and inclusion of family caregivers were regarded as being not feasible for logistical reasons. However, despite these limitations this process evaluation will cover a broad spectrum of perspectives and reveal expected and unexpected changes in the procedures of interprofessional care delivery. Thus, the findings of this process evaluation will facilitate accurate interpretation of the main trial findings and allow for robust conclusions concerning the further design of routine care services for NHRs. They will also improve our understanding of the mechanisms of interprofessional collaboration and their impact on the quality of care for vulnerable people such as NHRs.
Recruitment and data collection are finished, and data analysis is ongoing.
Availability of data and materials
Data are available upon reasonable request to the publication committee of the interprof ACT project.
Nursing home resident
Study team member
Nursing home director
Cluster-randomized controlled trial
Baseline data collection
Follow-up after 6 months
Follow-up after 12 months
Standards for Reporting Implementation Studies Statement
interprof ACT agent
Normalization Measure Development
Normalization Process Theory
Zufriedenheit in der Arztpraxis
Survey instruments of the research project “Optimization of the cooperation between general practitioners and home care services”
Measures of the European Project on Patient Evaluation of General Practice Care
Measures of interprofessional collaboration designed for the pre-post evaluation study “Interdisciplinary Implementation of Quality Instruments for the Care of residents with Dementia in Nursing Homes”
Partnership Self-Assessment Tool
Collaborative Practice Assessment Tool
Practice Environment Scale of the Nursing Work Index
Template for Intervention Description and Replication
Dwyer R, Gabbe B, Stoelwinder JU, Lowthian J. A systematic review of outcomes following emergency transfer to hospital for residents of aged care facilities. Age Ageing. 2014;43:759–66.
Hoffmann F, Schmiemann G. Influence of age and sex on hospitalization of nursing home residents: a cross-sectional study from Germany. BMC Health Serv Res. 2017;17:55.
Leutgeb R, Berger SJ, Szecsenyi J, Laux G. Potentially avoidable hospitalisations of German nursing home patients? A cross-sectional study on utilisation patterns and potential consequences for healthcare. BMJ Open. 2019;9:e025269.
Hsiao CJ, Hing E. Emergency department visits and resulting hospitalizations by elderly nursing home residents, 2001-2008. Res Aging. 2014;36:207–27.
Hogan DB, Amuah JE, Strain LA, Wodchis WP, Soo A, Eliasziw M, et al. High rates of hospital admission among older residents in assisted living facilities: opportunities for intervention and impact on acute care. Open Med. 2014;8:e33–45.
Quinn T. Emergency hospital admissions from care-homes: who, why and what happens? A cross-sectional study. Gerontology. 2011;57:115–20.
Harrison JK, McKay IK, Grant P, Hannah J, Quinn TJ. Appropriateness of unscheduled hospital admissions from care homes. Clin Med. 2016;16:103–8.
Richter C, Berg A, Langner H, Meyer G, Köpke S, Balzer K, et al. Effect of person-centred care on antipsychotic drug use in nursing homes (EPCentCare): a cluster-randomised controlled trial. Age Ageing. 2019;48:419–25.
Fassmer AM, Pulst A, Spreckelsen O, Hoffmann F. Perspectives of general practitioners and nursing staff on acute hospital transfers of nursing home residents in Germany: results of two cross-sectional studies. BMC Fam Pract. 2020;21:29.
Müller CA, Fleischmann N, Cavazzini C, Heim S, Seide S, Geister C, et al. Interprofessional collaboration in nursing homes (interprof): development and piloting of measures to improve interprofessional collaboration and communication: a qualitative multicentre study. BMC Fam Pract. 2018;19:14.
Tsakitzidis G, Timmermans O, Callewaert N, Verhoeven V, Lopez-Hartmann M, Truijen S, et al. Outcome indicators on interprofessional collaboration interventions for elderly. Int J Integr Care. 2016;16:5.
Müller C, Hesjedal-Streller B, Fleischmann N, Tetzlaff B, Mallon T, Scherer M, et al. Effects of strategies to improve general practitioner-nurse collaboration and communication in regard to hospital admissions of nursing home residents (interprof ACT): study protocol for a cluster randomised controlled trial. Trials. 2020;21:913.
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, et al. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655.
Lewin S, Hendry M, Chandler J, Oxman AD, Michie S, Shepperd S, et al. Assessing the complexity of interventions within systematic reviews: development, content and use of a new tool (iCAT_SR). BMC Med Res Methodol. 2017;17:76.
Threapleton DE, Chung RY, Wong SYS, Wong E, Chau P, Woo J, et al. Integrated care for older populations and its implementation facilitators and barriers: a rapid scoping review. International J Qual Health Care. 2017;29:327–34.
Grant A, Treweek S, Dreischulte T, Foy R, Guthrie B. Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials. 2013;14:15.
Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: medical Research Council guidance. BMJ. 2015;350:h1258.
Pfadenhauer LM, Gerhardus A, Mozygemba K, Lysdahl KB, Booth A, Hofmann B, et al. Making sense of complexity in context and implementation: the context and implementation of complex interventions (CICI) framework. Implement Sci. 2017;12:21.
Creswell JW, Clark V, Garrett A. Methological issues in conducting mixed methods research designs. In: Bergman M, editor. Advances in mixed methods research Theories and applications. London: Sage Publications; 2008. p. 66–83.
Frankenhauser-Mannuß J, Goetz K, Scheuer M, Szescenyi J, Leutgeb R. Ärztlicher Bereitschaftsdienst in Deutschland: befragung von Bereitschaftsdienstärzten zur gegenwärtigen Situation (Kritische Aussagen). [Out-of-hours primary care in Germany: general practitioners' views on the current situation]. Gesundheitswesen. 2014;76:428–33.
Kühne F, Behmann M, Bisson S, Walter U, Schneider N. Non-response in a survey of physicians on end-of-life care for the elderly. BMC Res Notes. 2011;4:367.
Steinhäuser J, Jäkel K, Szecsenyi J, Goetz K, Ledig T, Joos S. Prozeduren, die in der Allgemeinmedizin durchgeführt werden – eine Querschnittstudie. [Procedures performed in general practice - a cross-sectional study]. Gesundheitswesen. 2016;79:1004–11.
Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348:g1687.
Pinnock H, Sheikh A. Standards for reporting implementation studies (StaRI): enhancing reporting to improve care. NPJ Prim Care Respir Med. 2017;27:42.
Bitzer EM, Dierks ML, Dörning H, Schwartz FW. Zufriedenheit in der arztpraxis aus patientenperspektive – psychometrische prüfung eines standardisierten erhebungsinstrumentes. [Patient satisfaction with ambulatory care physicians - psychometric testing of a standardized questionnaire]. Z Gesundh Wiss. 1999;7:196–209.
van den Bussche H, Jahncke-Latteck ÄD, Ernst A, Tetzlaff B, Wiese B, Schramm U. Zufriedene hausärzte und kritische pflegende - probleme der interprofessionellen zusammenarbeit in der versorgung zu hause lebender menschen mit demenz. [Satisfied general practitioners and critical nursing staff - problems of interprofessional cooperation in the home care of dementia patients]. Gesundheitswesen. 2012;75:328–33.
Klingenberg A, Bahrs O, Szecsenyi J. Wie beurteilen Patienten Hausärzte und ihre Praxen? Deutsche Ergebnisse der europäischen Studie zur Bewertung hausärztlicher Versorgung durch Patienten (EUROPEP). [How do patients evaluate general practice? German results from the European project on patient evaluation of general practice care (EUROPEP)]. Z Arztl Fortbild Qualitatssich. 1999;93:437–45.
Holle D, Halek M, Mayer H, Bartholomeyczik S. Die Auswirkungen der Verstehenden Diagnostik auf das Belastungserleben Pflegender im Umgang mit Menschen mit Demenz in der stationären Altenhilfe. [The influence of understanding diagnostics on perceived stress of nurses caring for nursing home residents with dementia]. Pflege. 2011;24:303–16.
Rapley T, Girling M, Mair FS, Murray E, Treweek S, McColl E, et al. Improving the normalization of complex interventions: part 1 - development of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). BMC Med Res Methodol. 2018;18:133.
Finch TL, Girling M, May CR, Mair FS, Murray E, Treweek S, et al. Improving the normalization of complex interventions: part 2 - validation of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). BMC Med Res Methodol. 2018;18:135.
Center for the Advancement of Collaborative Strategies in Health. Partnership self-assessment tool questionnaire. 2002. https://atrium.lib.uoguelph.ca/xmlui/bitstream/handle/10214/3129/Partnership_Self-Assessment_Tool-Questionnaire_complete.pdf?sequence=1&isAllowed=y. Accessed 12 Dec 2021.
Lakeit S. Nurse-Physican Collaboration. A systematic literature review measuring nurse-physican collaboration, and the development and psychometric testing of one of these instruments for a German Target Group. Dissertation. Köln: Universität zu Köln (University of Cologne); 2015.
Schroder C, Medves J, Paterson M, Byrnes V, Chapman C, O'Riordan A, et al. Development and pilot testing of the collaborative practice assessment tool. J Interprof Care. 2010;25:189–95.
Aiken LH, Patrician PA. Measuring organizational traits of hospitals: the revised nursing work index. Nurs Res. 2000;49:146–53.
Lake ET. Development of the practice environment scale of the nursing work index. Res Nurs Health. 2002;25:176–88.
May C, Finch T. Implementing, embedding, and integrating practices: an outline of normalization process theory. Sociology. 2009;43:535–54.
May CR, Cummings A, Girling M, Bracher M, Mair FS, May CM, et al. Using normalization process theory in feasibility studies and process evaluations of complex healthcare interventions: a systematic review. Implement Sci. 2018;13:80.
Vis C, Ruwaard J, Finch T, Rapley T, de Beurs D, van Stel H, et al. Toward an objective assessment of implementation processes for innovations in health care: psychometric evaluation of the normalization measure development (NoMAD) questionnaire among mental health care professionals. J Med Internet Res. 2019;21:e12376.
ImpleMentAll Consortium. Normalization Measure development questionnaire (short NOMAD). German Version. 2018. https://www.implementall.eu/NoMAD_German.pdf. Accessed 25 Nov 2021.
Abraham J, Möhler R, Henkel A, Kupfer R, Icks A, Dintsios CM, et al. Implementation of a Multicomponent intervention to Prevent Physical Restraints In Nursing home residenTs (IMPRINT): study protocol for a cluster-randomised controlled trial. BMC Geriatr. 2015;15:86.
Emerson RM, Fretz RI, Shaw LL. Participant observation and fieldnotes. In: Atkinson P, Coffey A, Delamont S, Lofland J, Lofland L, editors. Handbook of ethnography. London: Sage; 2001. p. 352–68.
Miles M, Huberman A. Qualitative data analysis: an expanded sourcebook. Thousand Oaks: Sage Publications; 1994.
Field A. Discovering statistics using IBM SPSS statistics. London: Sage Publications Ltd; 2018.
Streiner D, Norman G, Cairney J. Health measurement scales: a practical guide to their development and use. Oxford: Oxford University Press; 2014.
Kane RL, Huckfeldt P, Tappen R, Engstrom G, Rojido C, Newman D, et al. Effects of an intervention to reduce hospitalizations from nursing homes: a randomized implementation trial of the INTERACT program. JAMA Intern Med. 2017;177:1257–64.
Sampson EL, Feast A, Blighe A, Froggatt K, Hunter R, Marston L, et al. Pilot cluster randomised trial of an evidence-based intervention to reduce avoidable hospital admissions in nursing home residents (Better health in residents of care homes with nursing-BHiRCH-NH study). BMJ Open. 2020;10:e040732.
Huckfeldt PJ, Kane RL, Yang Z, Engstrom G, Tappen R, Rojido C, et al. Degree of implementation of the interventions to reduce acute care transfers (INTERACT) quality improvement program associated with number of hospitalizations. J Am Geriatr Soc. 2018;66:1830–7.
Hughes G, Shaw SE, Greenhalgh T. Rethinking integrated care: a systematic hermeneutic review of the literature on integrated care strategies and concepts. Milbank Q. 2020;98:446–92.
The interprof ACT trial group would like to acknowledge support from Lars Magnus Alexander Alt in the literature reviews and cognitive interviews for the development of the questionnaires.
Open Access funding enabled and organized by Projekt DEAL. This research is funded by a public organism after competitive peer review (German Innovation Fund. Proposal ID VF1_2016-079). The funding institution will not interfere in any part of the study (https://innovationsfonds.g-ba.de/). Because the study is not subject to the German medical product act or medical devices act (AMG, MPG), no “real sponsor” is required. The quasi sponsor is the University Medical Center Göttingen (UMG), Robert-Koch-Str. 40, 37075 Göttingen. The internal registration number of the interprof ACT at the UMG Clinical Trials Unit is 01811. The sponsor is not directly involved in any study activities but has overall responsibility. All study activities are initiated and performed by either the Institute of General Practice (lead), the Institute of Medical Statistics, or the Clinical Trials Unit at the UMG.
Ethics approval and consent to participate
The interprof ACT trial received ethical approval from the ethical committee of the Medical Faculty, Georg-August-University, Göttingen (no. 31/7/17), the ethical committee of the University of Lübeck (no. 18-051), and the Ethics Committee of the Medical Association of Hamburg-Eppendorf (no. MC-304/17).
First, the facility director of the nursing home must sign written consent to participate in the interprof ACT trial after being informed orally and in written form about the study.
Second, NHRs or their legal guardians must give their written consent to participate in the interprof ACT trial. NHRs and/or their legal guardians receive oral and written information about the study and its procedure as well as about benefits for residents’ health and potential risks. The information is written in language that is easy to understand and contains information about the data to be collected from/about the NHR, either via data extraction from the patient chart (main trial only) or structured oral face-to-face interviews (main trial and process evaluation). NHRs and/or their legal guardians subsequently provide informed consent. Immediately before each face-to-face interview, a member of the research team will repeat this oral information, confirming the NHRs’ willingness to participate. Similar to the data extracted from the NHRs’ charts, the interview data will be collected in a pseudonymized manner based on a separate code list.
For all target populations, informed consent will be necessary for participation in any quantitative inquiry for the process evaluation if data will not be collected anonymously (RN, GP and IPAV questionnaires). Written information about scheduled surveys is included on the cover page of each questionnaire (except for the NHRs’ questionnaire), and completion of these questionnaires will be regarded as written informed consent. At each kick-off meeting, all attendees will be asked to confirm their participation in person on a list. These participant lists will be saved electronically at the local study center separately from all of the other trial documents and will be accessible only to the members of the local study team.
All target groups of the qualitative part of the process evaluation must provide written consent to participate after they receive detailed information about the qualitative process evaluation and data collection methods. Before each follow-up interview or observation, the study will again be explained, and participants will be given the opportunity to reconsider their consent.
Participation in the interprof ACT trial and in the integrated process evaluation is voluntary and can be withdrawn at any measurement point in the study without providing a reason as well as without consequences for medical/nursing care. If individuals withdraw their consent to participate, they will be asked to give consent to analyses of their collected data. If a person refuses to consent to the analyses, the collected data will be deleted immediately. An exception is data that were collected anonymously and that have already been aggregated and published.
Consent for publication
The authors declare they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Description of the intervention following the Template for Intervention Description and Replication (TiDier) criteria ; Characteristics of the interprof ACT intervention package (description based on the TIDieR checklist).
Overview of standardized instruments used for the quantitative part of the interprof ACT process evaluation; Instrument names, measured outcome domains and subdomains, and description of item and scaling formats.
About this article
Cite this article
Steyer, L., Kortkamp, C., Müller, C. et al. Implementation, mechanisms of change and contextual factors of a complex intervention to improve interprofessional collaboration and the quality of medical care for nursing home residents: study protocol of the process evaluation of the interprof ACT intervention package. Trials 23, 561 (2022). https://doi.org/10.1186/s13063-022-06476-6