Tailored Implementation for Chronic Diseases (TICD): a protocol for process evaluation in cluster randomized controlled trials in five European countries
© Jäger et al.; licensee BioMed Central Ltd. 2014
Received: 7 August 2013
Accepted: 20 December 2013
Published: 21 March 2014
In the ‘Tailored Implementation for Chronic Diseases (TICD)’ project, five tailored implementation programs to improve healthcare delivery in different chronic conditions have been developed. These programs will be evaluated in distinct cluster-randomized controlled trials. This protocol describes the process evaluation across these trials, which aims to identify determinants of change in chronic illness care, to examine the validity of the tailoring methods that were applied, and to analyze the association of implementation activities and the effectiveness of the program.
A multilevel approach was used to develop five tailored implementation interventions. In order to guide the process evaluation in five distinct trials, the study protocols for the cluster randomized trials and the related process evaluations were developed simultaneously and iteratively.
The process evaluation comprises three main components: a structured survey with health professionals in the trials, semi-structured interviews with a purposeful sample of this study population, and standardized documentation of organizational practice characteristics. Norway will only conduct the qualitative part of the analysis because the survey and documentation of practice characteristics are considered to be not feasible. The evaluation is guided by ‘logic models’ of the implementation programs: frameworks that specify the linkages between the strategies used, the determinants addressed by tailoring, and the anticipated outcomes. Standardization of measures across trials is sought to facilitate analysis of aggregated data from the trials.
This process evaluation will need to find a balance between standardization of methods across trials and the tailoring of measures to the specificities of each trial.
Keywordsprocess evaluation tailored implementation intervention chronic illness care
‘Tailored interventions’ are commonly defined as interventions that are designed to address previously identified determinants of practice (also called ‘barriers to and enablers of change’) . A range of different methods has been used for tailoring, but there is little insight into how tailoring is best conducted in order to optimize the effectiveness of interventions . The overall aim of the ‘Tailored Implementation for Chronic Disease (TICD)’ project is to provide insight into methods for tailoring implementation programs to determinants of implementation of evidence-based healthcare for patients with chronic illness . Research teams from five European countries participate in the TICD project, each focusing on a specific clinical condition: chronic obstructive pulmonary disease (COPD) in Poland, obesity in the United Kingdom, cardiovascular diseases in The Netherlands, depression in the elderly in Norway, and multimorbidity in Germany.
Addressing the identified determinants of practice in chronic illness care requires complex interventions, meaning interventions that have multiple components which interact with each other. In the TICD project, five tailored implementation interventions were developed, each of which will be evaluated in a cluster-randomized trial [4–8].
A frequent criticism of evaluations of complex interventions is their lack of detail in both description and analysis on what components of the intervention and which contextual factors contributed to the (lack of) effectiveness . We planned process evaluations in the TICD trials aiming to address this by exploring which factors are associated with change [10, 11]. This knowledge is essential for understanding the potential causal mechanisms underlying the effectiveness of interventions as well as for ensuring generalizability of intervention effectiveness across populations and settings . At the same time, this will provide insight into the validity of tailoring methods that were applied in earlier phases of the TICD project.
A factor of specific interest is the degree to which the offered interventions were applied as planned by the participants (depending on the context this has been labeled ‘fidelity’ or ‘integrity’ of the intervention ). This knowledge is important to draw valid conclusions about intervention effectiveness and the underlying change mechanisms. Potentially useful interventions may appear ineffective due to low or mistaken application as compared to the original plan . On the other hand, low fidelity or integrity may reflect the adaptation of interventions in the delivery phase to optimize their effectiveness.
To what extent did the practitioners or patients use the various components of the planned intervention programs?
What is the validity of the methods used to tailor the implementation program to determinants of practice?
Which factors are associated with effective implementation of evidence-based practice in chronic illness care, after use of a tailored implementation program?
How consistent are the findings regarding the previous questions across the five trials?
Tailoring concept within the Tailored Implementation for Chronic Diseases project
Development of a process evaluation protocol
The study protocols for the cluster randomized trials and the related process evaluations were developed simultaneously and iteratively, using email and telephone conversations among research teams to provide feedback and discuss controversial aspects. The project coordinator monitored and protected the coherence of the protocols and their fit with the overall aims of the TICD project.
Setting/target groups of the process evaluation
The intervention programs in the TICD project are situated in primary care settings and target physicians, nurses, and patients and their relatives. The study population in the process evaluation comprises the healthcare professionals of the intervention group, however implementation activities (whether planned or not) will be documented in the control arms as well.
The process evaluation has three main items: a survey with two components (A1 and A2), an interview study (B), and documentation of practice characteristics (C).
A1 - Survey on perceived change of determinants of practice
Scheme of a questionnaire to examine the validity of the methods used for tailoring (A1)
Standardized list of determinants (result of phase 2). Please list all determinants, no matter if they are intended to be addressed by the implementation program or not
Was this determinant in your opinion clearly addressed by the implementation program?
Free text field: Can you think of other factors, not yet listed above?
A2 - Survey to document implementation activities
Content - Was the content of the intervention delivered as planned?
Duration - Was the intervention fully implemented across the intended time period?
Frequency - Was the intervention conducted as frequent as planned?
Coverage - Was the intervention applied to all individuals in the targeted group?
Questionnaire to document implementation activities (A2)
Intervention component 1: Please describe the intervention component in sufficient detail (specify content, duration, frequency, coverage if applicable)
Did you use this item as described in terms of duration? (if applicable)
Did you use this item as described in terms of frequency? (if applicable)
Did you use this item as described in terms of coverage? (if applicable)
Did you find this item helpful for the implementation of the recommendation/to reach the targets?
Did you adapt the content or format of this item in any way? If yes, please specify below!
Intervention component 2: Please describe the intervention component in sufficient detail (specify content, duration, frequency, coverage if applicable)
Free text field: Can you think of other intervention components which might have been more helpful in order to improve the implementation of the recommendations/to reach the targets?
For each implementation program the core components will be specified. For each core component, content, duration, frequency and coverage will be recorded in a structured way (Table 2). The content of the items will be specific for the different trials. A free text field is added to identify strategies that have been missed in the tailoring process so far, allowing the evaluation of the methods used for tailoring.
B - Interviews with health professionals of the intervention group
What made you participate in the project in the first place? What where your reasons and what were your expectations?
- 2.Did the implementation program help you to adhere to the recommendations?
If yes, what components did you find helpful and why?
If no, why not and what strategies would have been more helpful?
Were there any other factors or developments which made it difficult for you or helped you to adhere to the recommendations? You can think of changes in regional or national healthcare, developments in the practice organization and team or events in your life and work.
Having experienced the program, what would you recommend for the future? You may think of further development, wider implementation or perhaps research.
Interim analysis after 5 to 10 interviews will be done to adapt the interview format and purposeful sampling scheme as required.
C - Practice characteristics and other contextual factors
Practice characteristics determined in all Tailored Interventions for Chronic Diseases (TICD) studies (C)
What is the yearly attending population (number of different patients that contact the practice in one year)?
What is the number of contacts your practice has per week?
How many physicians are working in your practice? (full-time equivalent)
How much non-physician staff is working in your practice? (full-time equivalent)?
How many inhabitants live in the city where your practice is located?
o >5,000 <20,000
o >20,000 <50,000
Do all physicians have access to medical guidelines in your practice?
o yes o no
Is a recall system for follow-up of chronically ill patients used in your practice?
o yes o no
Are the tasks in your practice clearly assigned to specific staff?
o yes o no
Does your practice have regular team meetings?
o yes o no
Did your practice set targets for quality improvement in the last 12 months?
o yes o no
Has the principal target (regarding quality improvement) been met?
o yes o no
Research question 1 - To what extent did the practitioners or patients use the various components of the planned intervention programs? and
Research question 2 - What is the validity of the methods used to tailor the implementation program to determinants of practice?In order to answer these questions a descriptive analysis of qualitative and quantitative data will be done separately in each country. The descriptive analysis of questionnaires will result in tables with frequency distributions. The qualitative interviews will be transcribed verbatim in the original languages. An iterative thematic analysis will be done, involving two or more researchers in order to identify key themes relevant for implementation.
Research question 3 - Which factors are associated with implementation of evidence-based practice in chronic illness care, after use of a tailored implementation program?
Research question 4 - How consistent are the findings across the trials?
A meta-analysis of quantitative and a meta-ethnography of qualitative data from all five studies will be conducted.
A qualitative meta-ethnography will be conducted using the data from the interviews. For this purpose a description of subcategories with significant quotations from all studies will be provided in English. This material will be analyzed according to the principles of Noblit and Hare’s reciprocal translational analysis, which is recommended to be used when analyzing studies with similar topics [17, 18]. The result of this analysis will be a set of key themes relevant in all studies. Interviews will be re-analyzed to explore how each single study contributes to the overall key themes. In a last step third order interpretation across the key themes will be considered if supported by a ‘line of argument’ across all studies.
A quantitative meta-analysis will be conducted, focused on the summary outcome measure in each trial, using the predictors that are identified as result from the analysis for question 3. While the items composing this aggregate outcome obviously differ among the trials, the assumption is that the resulting measures are reasonably comparable to allow pooling. Based on hypotheses derived from phase 1 and 2, individual data and/or clustered data from all trials will be included in a meta-regression analysis. This last step will efficiently make use of the total number of cases included across all studies and the shared instruments as described in this protocol.
Process evaluation is essential to understand the effects and to explore potential causal mechanisms of complex interventions. The planned process evaluation in the TICD project needs to find a balance between standardization of methods across trials and adaptation of measures to the specific characteristics of each trial. Despite obvious limitations, such as heterogeneity and suboptimal statistical power, we believe that the planned study will contribute to our knowledge on determinants of change in chronic illness care and methods for tailoring implementation programs to barriers and facilitators of change.
The trial is currently in the planning phase.
Tailored Implementation for Chronic Diseases.
The study is funded by the European Union Framework 7 program within the theme HEALTH.2013.3.1-1 under grant agreement no 258837.
- Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, Robertson N: Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2010, 3: CD005470-PubMed
- Bosch M, van der Weijden T, Wensing M, Grol R: Tailoring quality improvement interventions to identified barriers: a multiple case analysis. J Eval Clin Pract. 2007, 13: 161-168. 10.1111/j.1365-2753.2006.00660.x.View ArticlePubMed
- Wensing M, Oxman A, Baker R, Godycki-Cwirko M, Flottorp S, Szecsenyi J, Grimshaw J, Eccles M: Tailored implementation for chronic diseases (TICD): a project protocol. Implement Sci. 2011, 6: 103-10.1186/1748-5908-6-103.PubMed CentralView ArticlePubMed
- Krause J, Agarwal S, Bodicoat DH, Ring A, Shepherd D, Rogers S, Wensing M, Baker R: Evaluation of a tailored intervention to improve management of overweight and obesity in primary care; study protocol of a cluster randomised controlled trial. Trials. 2014, (submitted)
- Jäger C, Freund T, Steinhäuser J, Joos S, Wensing M, Szecsenyi J: A tailored implementation intervention to implement recommendations addressing polypharmacy in multimorbid patients: study protocol of a cluster randomized controlled trial. Trials. 2013, 14: 420-10.1186/1745-6215-14-420.PubMed CentralView ArticlePubMed
- Godycki-Cwirko M, Zakowska I, Kosiek K, Wensing M, Krawczyk J, Kowalczyk A: Evaluation of a tailored implementation strategy to improve the management of patients with chronic obstructive pulmonary disease in primary care – study protocol of a cluster randomized trial. Trials. 2014, (submitted)
- Aakhus E, Granlund I, Odgaard-Jensen J, Wensing M, Oxman AD, Flottorp SA: Tailored interventions to implement recommendations for elderly patients with depression in primary care: a study protocol for a pragmatic cluster randomised controlled trial. Trials. 2014, 15: 16-10.1186/1745-6215-15-16.PubMed CentralView ArticlePubMed
- Huntink E, Heijmans N, Wensing M, van Lieshout J: Effectiveness of a tailored intervention to improve cardiovascular risk management in primary care: study protocol for a randomised controlled trial. Trials. 2013, 14: 433-10.1186/1745-6215-14-433.PubMed CentralView ArticlePubMed
- Michie S, Fixsen D, Grimshaw JM, Eccles MP: Specifying and reporting complex behaviour change interventions: the need for a scientific method. Implement Sci. 2009, 4: 40-10.1186/1748-5908-4-40.PubMed CentralView ArticlePubMed
- Grant A, Treweek S, Dreischulte T, Foy R, Guthrie B: Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials. 2013, 14: 15-10.1186/1745-6215-14-15.PubMed CentralView ArticlePubMed
- Oakley A, Strange V, Bonell C, Allen E, Stephenson J: Process evaluation in randomised controlled trials of complex interventions. BMJ. 2006, 332: 413-416. 10.1136/bmj.332.7538.413.PubMed CentralView ArticlePubMed
- Dixon-Woods M, Bosk CL, Aveling EL, Goeschel CA, Pronovost PJ: Explaining Michigan: developing an ex post theory of a quality improvement program. Milbank Q. 2011, 89: 167-205. 10.1111/j.1468-0009.2011.00625.x.PubMed CentralView ArticlePubMed
- Cross WF, West JC: Examining implementer fidelity: Conceptualizing and measuring adherence and competence. J Child Serv. 2011, 6: 18-33. 10.5042/jcs.2011.0123.PubMed CentralView ArticlePubMed
- Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, Baker R, Eccles MP: A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013, 8: 35-10.1186/1748-5908-8-35.PubMed CentralView ArticlePubMed
- Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S: A conceptual framework for implementation fidelity. Implement Sci. 2007, 2: 40-10.1186/1748-5908-2-40.PubMed CentralView ArticlePubMed
- Grol R: Successes and failures in the implementation of evidence-based guidelines for clinical practice. Med Care. 2001, 39 (8 Suppl 2): II46-II54.PubMed
- Barnett-Page E, Thomas J: Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009, 9: 59-10.1186/1471-2288-9-59.PubMed CentralView ArticlePubMed
- Campbell R, Pound P, Morgan M, Daker-White G, Britten N, Pill R, Yardley L, Pope C, Donovan J: Evaluating meta-ethnography: systematic analysis and synthesis of qualitative research. Health Technol Assess. 2011, 15: 1-164.View ArticlePubMed
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.