Tailored educational intervention for primary care to improve the management of dementia: the EVIDEM-ED cluster randomized controlled trial

Abstract

Background

Early diagnosis of dementia is important because this allows those with dementia and their families to engage support and plan ahead. However, dementia remains underdetected and suboptimally managed in general practice. Our objective was to test the effect of a workplace-based tailored educational intervention developed for general practice on the clinical management of people with dementia.

Methods

The tailored educational intervention was tested in an unblinded cluster randomized controlled trial with a pre/post-intervention design, with two arms: usual/normal care control versus educational intervention. The primary outcome measure was an increase in the proportion of patients with dementia who received at least two documented dementia-specific management reviews per year. Case identification was a secondary outcome measure.

Results

23 practices in South-East England participated. A total of 1,072 patients with dementia (intervention: 512, control: 560) had information in their medical records showing the number of reviews within 12 months (or a proportion of) before intervention or randomization and within 12 months (or a proportion of) after. The mean total number of dementia management reviews after the educational intervention for people with dementia was 0.89 (SD 1.09; minimum 0; median 1; maximum 8) compared with 0.89 (SD 0.92; minimum 0; median 1; maximum 4) before intervention. In the control group prior to randomization the mean total number of dementia management reviews was 1.66 (SD 1.87; minimum 0; median 1; maximum 12) and in the period after randomization it was 1.56 (SD 1.79; minimum 0; median 1; maximum 11). Case detection rates were unaffected. The estimated incidence rate ratio for intervention versus control group was 1.03 (P = 0.927, 95% CI 0.57 to 1.86).

Conclusions

The trial was timely, coinciding with financial incentives for dementia management in general practice (through the Quality Outcomes Framework); legal imperatives (in the form of the Mental Capacity Act 2005); policy pressure (The National Dementia Strategy 2009); and new resources (such as dementia advisors) that increased the salience of dementia for general practitioners. Despite this the intervention did not alter the documentation of clinical management of patients with dementia in volunteer practices, nor did it increase case identification.

Trial registration

NCT00866099/Clinical Trials

Background

Timely diagnosis can relieve the significant psychological distress that people with dementia and their close supporters experience. It allows them to engage with medical and psychosocial support, acquire or strengthen positive coping strategies, fulfill short-term goals and plan for the future, all of which can maintain and/or improve functioning and morale [1–4].

Efforts to improve the identification and diagnosis of dementia should be targeted at primary care, as this is the first point of contact for most individuals and their caregivers. However, dementia presents many challenges for primary care [5–7] and it remains underdetected and suboptimally managed [8]. General practitioners in England consistently report limited skills and confidence in diagnosis and management of dementia [9], and a minority see this as a task only for specialists [10]. An educational intervention to improve the skills of practitioners in the recognition of, and response to, dementia syndromes would be expected to be beneficial. However, in the absence of organizational change in health services and incentivisation of high quality care, evidence suggests that education alone is unlikely to change practice [11].

Since 2000, a natural experiment has occurred in the English National Health Service (NHS) via the introduction of policies and financial incentives for dementia diagnosis and management that have made a trial of an educational intervention in primary care to improve clinical practice particularly timely. Evidence-based policy imperatives, including guidelines from the National Institute for Health and Care Excellence (NICE) [8], are driving NHS management to prioritize dementia, especially earlier diagnosis and better clinical management in the community (the English Dementia Strategy [12]). Since 2006 changes in the reimbursement of primary care physicians through the Quality and Outcomes Framework (QOF) have incentivised the diagnosis and management of dementia syndrome [13], rewarding construction of a dementia disease registry and completion of an annual review of each patient with dementia (see Table 1). These policy and financial incentives for dementia diagnosis and management have created an ideal environment for a trial of an educational intervention in primary care, designed to improve clinical practice.

Table 1 Dementia indicators for the quality and outcomes framework

Recognition of and response to dementia

This project builds on an earlier trial, which demonstrated that educational interventions can improve the recognition of dementia syndromes in general practice, but did not alter documented practice [15]. The educational intervention reported here was developed by a group of practitioners from relevant disciplines and advised by caregivers of people with dementia. The design process, which is described in detail elsewhere [16], was based on three premises:

• 1. An adult learning approach to solving real-world problems would be needed, emphasizing the educational benefits of conversations, understood as social acts of collaboration that generate and improvise ideas collectively [17].

• 2. Tailoring of learning to the needs of the learners can be achieved using processes of educational needs assessment [18] and educational prescription [19], both techniques developed by the evidence-based medicine movement [19]. Educational prescriptions are a useful tool for enabling follow-up of learning and can be used in any practice setting [20, 21].

• 3. Learning would take place in the workplace, and be open to all team members (healthcare practitioners and administrative staff). Through reciprocal learning [22] peers and colleagues can be the most effective educators [22, 23] and there is a particular benefit from learning from other disciplines [23, 24]. This group effect in learning meant that a cluster-randomized design, with randomization at the practice level, was appropriate to this trial.

Adopting these premises, we developed a flexible learning needs assessment tool that allows an educational intervention to be tailored to the group needs and skills of the existing practice team. After a feasibility study [25] this intervention was tested in the current EVIDEM-ED randomized controlled trial [26].

Methods

The effectiveness of the new educational intervention was tested in an unblinded cluster randomized, controlled trial with a pre/post design and with two arms; usual care versus educational intervention (see consort diagram in Figure 1).

Figure 1

The EVIDEM-ED flow diagram.

Standard significance tests assume that random sampling has taken place, and that the behavior or knowledge of any one individual is not affected by others in the sample. However, this study, like many health service evaluations, is based on clusters of individuals who may influence each other; in this case, colleagues in a practice who share information or similar views, or patients who receive similar treatment [27]. This 'clustering effect’ can mean that false conclusions are drawn about relationships in the data, an effect that we controlled for by using cluster membership (that is, respondents in each practice).

Study setting

The study took place in group practices within the geographical care covered by the North Thames Dementias and Neurodegenerative Diseases Research Network (NT DeNDRoN).

Educational intervention

The educational intervention consists of practice-based workshops designed to elicit and assess the educational needs for the team as a whole. Experienced general practitioners with a background in postgraduate education facilitated the small group workshops with the practice teams. This comprised of an initial needs assessment group discussion, which was guided by a set series of questions. The questions were designed to elicit information about what systems were already in place for the diagnosis of and care for dementia patients, but also to encourage all staff to reflect on what they were already doing, whether they were doing that well, what they still need to be doing, and how to fill the gaps in knowledge and service. The educational needs assessment questions were as follows:

• 1. How would you rate your current care for people with dementia and their carers (using a simple scale of good enough/satisfactory/needs substantial improvement)?

• 2. What grounds or criteria is your rating based on?

• 3. How do you arrive at your decision for diagnosis of dementia?

• 4. After diagnosis, what follow-up do you provide to people with dementia and their carers?

• 5. Are you using a shared care protocol for cholinesterase inhibitors? If 'yes’, then: (i) who was involved in producing the protocol; (ii) who is involved in its implementation (for example, hospital consultants, community psychiatric nurses (CPNs), Care of Older People team)?

• 6. How effective do you think cholinesterase inhibitors are and how effective have you found them in your practice?

• 7. What non-pharmacological alternatives do you have available to help your patients (and their carers)?

• 8. Based on your experience, what do you think are the important quality markers in caring for people with dementia? (What would you want for yourself?)

• 9. What would you like to improve in the care of patients with dementia, in your practice?

From this discussion, an educational prescription tailored to those needs was generated and supported with workshops and electronic information resources. The 'usual/normal’ care control practices were provided with a summary of the UK NICE/ Social Care Institute for Excellence (SCIE) dementia clinical guidelines [8] and were offered workshop training and electronic information resources at the end of the study. Practices were asked at the end of the study about any competing educational events.

Recruitment

Primary care practices

Group practices in the North Thames DeNDRoN area were identified in collaboration with the local Primary Care Research Networks. Practices were contacted by the Trial research team, by letter and by awareness-raising through primary care educational meetings and by regular newsletters. Inclusion criteria for the practices were willingness to participate as a group in an educational experiment, and routine use of electronic medical records to capture the content of clinical encounters.

Patients

All practices were asked to identify patients with dementia by using electronic searches of their clinical record system updated by manual checks of the resulting list by medical and nursing staff.

Sample size

The size of the sample was based upon the following assumptions: (1) we expected a difference between the intervention and control groups in the proportion of patients with two or more dementia reviews in the 12 months after the educational intervention (or randomization to normal/usual care) of 50% (that is, control 20% versus 70% intervention); and (2) 20 practices would be included with an effective sample size of 10 patients per practice. Thus, a total effective sample size of 200 was reached.

This sample size was calculated based on randomization at the cluster (practice) level and allowed the study to maintain 90% power to detect the difference postulated in the presence of an intraclass correlation (ICC) of 0.37 and less.

Outcome measurements

Primary outcome

Our hypothesis was that the proportion of patients receiving two dementia reviews per year (as recommended by guidelines as being clinically necessary [8]) would differ by 50%, (that is, 20% (usual care) versus 70% (intervention)) after the educational intervention. Electronic medical records were audited retrospectively for a period 12 months prior to intervention and again for the 12 months after intervention.

The primary outcome data was assessed by independent clinical research staff via an audit of the Dementia Indicators of the Quality Outcome Framework (see Table 1). Additional information collected was of the age, gender, date of diagnosis (pre or post trial randomization and/or intervention dates), diagnosis, and whether the person lived in their own home. (Individual patient consent was sought for a more detailed examination of the secondary outcome measures, reported elsewhere; see Figure 1). Clinicians independent of the practices classified management reviews for dementia in one of two ways; as planned, coded Dementia Annual Reviews eligible for inclusion in the Quality Outcomes Framework, and as opportunistic Dementia Reviews, which were documented reviews of any aspect of dementia during clinical encounters prompted by other problems. These clinical reviewers were trained in data extraction using a standardized data extraction form developed for the trial, and uncertainties about classification were discussed with the principal investigator.

Randomization and masking

Participating practices were randomized to intervention or usual care arms by an independent researcher using a computer randomization program [28]. Independent clinicians undertaking record reviews could have deduced the allocation of the practices and could have become deblinded.

Data collection

The trial lasted 36 months allowing for a follow-up period of 12 months to capture effects on clinical practice. The trial was conducted in accordance with Good Clinical Practice [29] (GCP) [26] and approval for the trial was received from Southampton & South West Hampshire Research Ethics Committee (A): reference 09/H0502/77.

Practices self-completed a data-recording form at randomization and at follow-up 12 months later. This comprised the following fields:

• Practice list size.

• Number of partners (full-time equivalent (FTE)).

• Number of practice nurses (FTE).

• Any other member of staff specific for caring for older people based within the practice? If so please state title.

• Do you currently look after residents in a nursing or care home? (if so how many patients have a dementia diagnosis? Do you include this number on your QOF reporting).

• Age/sex register:

  • Male 0 to 64 years

  • Female 0 to 64 years

  • Male 65+ years

  • Female 65+ years

• Practice deprivation score, (if known).

• Dem 1 QOF (no. of patients on register diagnosed with dementia).

• Dem 2 QOF (the percentage of patients diagnosed with dementia whose care has been reviewed in the previous 15 months).

The following anonymous audit data were collected by independent clinicians:

• Age at randomization and/or intervention.

• Registered with current practice pre or post randomization and/or intervention.

  • (If left) date left the practice

• Gender

• Living at home

• Diagnosis

• Date of diagnosis

• Number of dementia reviews within 12 months pre/post randomization and/or intervention

• Number of opportunistic dementia reviews within 12 months pre/post randomization and/or intervention

The data collection tools are available on request from the authors.

Statistical analyses

We assessed the effect of the interventions at the practice level and analyses were performed on an intention to treat basis. We analyzed differences in detection rates by using binary logistic regression to include the cluster effect. These were calculated before and after the intervention, excluding cases previously diagnosed in another practice. Analysis of quantitative data was undertaken using the SPSS software package [30].

Statistical modeling

Case identification: Multilevel Poisson regression modeling was used to compare the diagnosis rates between the intervention and control groups in the postintervention/randomization 12 months. The outcome used was the count of new diagnoses during this period (in those aged 65+) with the number of patients aged over 65 on the practice list used as the exposure variable. A two-level random coefficients model was fitted in order to take account of the cluster randomization. The model included a binary variable for group (0 = control, 1 = intervention). In order to take account of the differences in baseline rates the baseline counts in the 12 month preintervention/randomization period were included in the model.

Dementia management reviews: Multilevel logistic regression modeling was used to compare the proportion of patients who had two or more reviews in the postintervention or post-randomization 12 months, between the intervention and usual care groups. A separate model was fitted for each classification of reviews: planned; opportunistic; and total. Two-level random coefficient models were fitted in order to take account of the cluster randomization. The models included a binary variable for group (0 = control, 1 = intervention). In order to take account of the differences in baseline rates a cluster level variable was created indicating the proportion of patients within a cluster (practice) who had two or more reviews in the 12-month preintervention/randomization period.

Sensitivity Analysis: The above modeling was repeated with exclusion of those without the full 12 months follow-up in the postintervention/randomization period. Additionally, the cluster level variable indicating the proportion of patients within a practice with two or more reviews in the 12-month preintervention/randomization period was based only on those patients with the full 12 months of data (see Figure 1).

Results

Of 200 practices approached, 23 agreed to participate and provide the required level of access and data. The practices came from 12 different primary care organizations in urban, semirural and rural areas. Practice enrolment occurred in 2008 to 2010.

As a result of the field-testing the variety of learning materials used were broadened to include more reference material given during sessions and available online for instant access. The pacing of delivery for topics was also amended and the expert tutors became more knowledgeable and aware of areas of need that were consistent across individuals and groups.

The intervention practice teams were offered flexible delivery of the needs assessment of workshop training and advised that 1-h sessions be the optimal length of each session. Practices had a mean of three educational workshops, including the needs assessment workshop at the beginning of the trial (range 2 to 4). These were staggered across the practices and took place from 2009 to 2011. The educational needs assessment generated individual workshop content for each practice. The workshops were delivered by one or two general practitioner dementia experts each working to a tutor’s manual that included learning objectives and timings. For three of the workshops a single trainer with non-participant observer facilitated the workshops (SI (3); FL (3)) and in four practices both facilitators were present (SI and FL). A non-participant observer (JW) took notes at all meetings to ensure that all educational needs were met and to check for consistency in delivery of the intervention.

Each practice defined their own team, which comprised the core clinical staff but with an extension of the intervention to administrative and support staff, community nursing teams and other professionals linked with the practice. We found this approach to work well with consistent attendance throughout the sessions from the teams. In several practices, the support staff attended the sessions and one practice invited community nursing staff and one representatives from the local paramedic team.

Practice information

A total of 11 practices were randomized to the intervention arm and 12 to the usual care arm. The number of patients with dementia per practice ranged from 5 to 123 in the intervention arm and 6 to 108 in the usual care arm. The characteristics of the practices by arm of study are shown in Table 2. The differences between arms were: list size for the intervention arm; high deprivation score in the usual care arm group; number of patients resident in care homes in the intervention arm.

Table 2 Primary outcome analysis: practice characteristics by randomization group

Patient information

A total of 1,072 (intervention: 512, control: 560) patients had information available in their medical records showing the number of reviews (planned/opportunistic/total) within 12 months (or a proportion of) before intervention or randomization and/or within the 12 months (or a proportion of) after. The majority were female: intervention group 61% (N = 313), control group 70% (N = 382). The mean age for those people with dementia in the intervention group was 83 years (SD 8.7; minimum 33; maximum 104) and 83 years (SD 8.8; minimum 55; maximum 109) for those in the control group.

Dementia management reviews

The mean total number of dementia management reviews (planned and opportunistic) for people with dementia in the intervention group in the period before intervention was 0.89 (SD 0.92; minimum 0; median 1; maximum 4). For the period after intervention it was 0.89 (SD 1.09; minimum 0; median 1; maximum 8).

For those people with dementia in the control group prior to randomization the mean total number of dementia management reviews (planned and opportunistic) was 1.66 (SD 1.87; minimum 0; median 1; maximum 12). For the period after randomization it was 1.56 (SD 1.79; minimum 0; median 1; maximum 11).

Primary outcomes

The numbers of each type of review for each patient in the pre/post periods were dichotomized for the primary analyses. These were classified according to whether the individual had <2 or 2 ≥ dementia management reviews; summarized in Table 3 below.

Table 3 Percentage of patients, by group, with 2 ≥ reviews for each type of review in the pre/post periods

Estimated odds ratios (odds of having two or more reviews in the intervention versus the usual care group), along with the P value and 95% confidence intervals for those with a full and partial data period are presented in Table 4.

Table 4 Estimated odds, P value and 95% confidence intervals by classification of review

There was no significant difference in recording of dementia management reviews for patients diagnosed by the current practice before or after intervention.

Case detection

Preintervention period: In the preintervention/randomization 12 months period there were a total of 239 newly diagnosed cases of dementia. Of these, 11 (4.6%) were in people aged below 65. Of the 228 newly diagnosed cases in those aged 65 and over 99 were in the control practices and 129 in the intervention practices.

Postintervention period: In the postintervention/randomization 12 months period there were a total of 169 newly diagnosed cases of dementia. Six of these (3.7%) were in people aged below 65. Of the 163 newly diagnosed cases in those aged 65 and over, 78 were in the control practices and 85 in the intervention practices.

The primary analysis included diagnosis rates in those aged 65 and over. Each practice provided data for those aged 65 and over registered with the practice list and this was used as the denominator for the calculation of rates. For the control practices combined the total patient population was 15,699 versus 11,541 for the intervention practices.

Table 5 shows case detection rates in the pre/post-intervention/randomization periods. These are shown separately for the combined intervention practices and the combined usual care practices; also the minimum and maximum rates were calculated across the practices.

Table 5 Detection rates for new cases of dementia pre/post-intervention period by randomization arm

Case detection rates were unaffected by the intervention. The estimated incidence rate ratio (IRR) for the intervention versus the control group from this model was 1.03; the P value was 0.927 with 95% confidence intervals 0.57, 1.86.

Discussion

The English policy imperatives and financial incentives for dementia diagnosis and management have created a favorable environment for a trial of an educational intervention designed to improve clinical practice in primary care. The educational intervention was developed following the Medical Research Council’s recommendations for complex interventions [32], with strong elements of codesign modified by nominal groups to gain the insights and experiences of a range of practitioners [33]. Codesign is a technique adopted from product development, which has tangible benefits in developing or redesigning health services [34–37]. The educational needs assessment deployed in this trial is an example of a strategy aimed to improve quality of care by overcoming the translation block that obstructs the diffusion of clinical guidelines and knowledge into practice [38]. In this study, we found no significant improvement in case identification or documentation of dementia management reviews after an educational intervention tailored to practice educational needs, despite the financial incentives to identify and follow-up patients with dementia. There are several possible reasons for this.

The intervention may have been too weak to change practice. More workshops may have been needed, with reinforcement or mentoring of practitioners over longer periods of time. This level of educational input was not practicable in this trial, and we doubt that it would be feasible in real-world primary care organizations. Physicians have a limited ability to accurately self-assess their competence [39]. Although the educational needs assessment was designed as a group process to offset this tendency, more external assessment may have been needed to truly tailor the intervention to needs.

It is possible that the trial was underpowered for the 50% expected change. Other changes may be detectable. Professional knowledge, confidence and attitudes; Dementia management activity concordant with the NICE guidelines and carers’ satisfaction and unmet need were all measured pre and post intervention and will be reported elsewhere. It is possible that these or other unmeasured outcomes (such as patient satisfaction with care) may have had an impact as a result of the intervention.

Limitations of the study

It is possible that using medical record coded QOF management reviews as the primary outcome did not capture changes in dementia management. However, our creation of a category of 'opportunistic dementia review’ fitted with clinical practice and allowed a generous interpretation of clinical activity. Additionally, many patients with dementia joined or left during the pre/post periods, truncating the data collection time, so that length of follow-up may have been too short to capture a difference.

The study took place in the South East of England, with practices that were probably innovative early adopters, not typical general practices, and local educational programs developed to implement the National Dementia Strategy may have influenced practice activities, although we found no evidence of this. The volunteer practices were probably different from others, in that they wanted to take part in a pilot educational program about dementia. However, the results of this study have wider implications, particularly about the value of tailoring educational interventions. Distribution of newsletters and guidelines to normal care arm practices may have had an effect on their behavior.

The intervention was developed in ways consistent with current understanding of how effective interventions are made. Nevertheless, there may have been deficiencies in the development process. For example, the views of people with dementia and their family carers may not have had sufficient weight. Some professional perspectives may have been too powerful, resulting in an oversimplification of the educational needs assessment.

Conclusions

This study suggests that a tailored educational intervention aimed at general practitioners does not improve documentation of clinical management for people with dementia, or dementia case identification, even when policy pressure and consumer demand encourage changes in clinical practice and the reimbursement system rewards it. Educational interventions in settings where a coordinated system of dementia case management operated have shown positive effects [32], so the effective change may need to include the additional resource of case management as well as focused education.