Explaining high and low performers in complex intervention trials: a new model based on diffusion of innovations theory

Description of dataset and introduction to case examples

The final dataset for the process evaluation comprised 60 pages of field notes, 245 pages of interview transcripts and 70 pages of additional free-text documentation, plus quantitative data on the distribution of 11,000 rapid HIV tests across 20 intervention practices and 5193 serological (hospital laboratory) tests across 40 intervention and control practices, respectively.

Common findings: relative advantage and simplicity of the rapid test

Despite wide variation in uptake of rapid testing between practices, there were some findings common to all, particularly in relation to the intervention (Box 1). The front-line staff who delivered the intervention almost universally perceived a distinct relative advantage (and considered that patients also saw an advantage) in rapid, accessible and convenient testing in general practice compared with usual care (the serological test requiring venepuncture and at least a 2-day wait for results). The quick and actionable results would mean less waiting and administration and, many staff believed, fewer losses to follow-up. Staff reported that patients appreciated receiving their results instantly, and they themselves gained satisfaction in being able to provide this information quickly.

Lack of need for pre-test and post-test counselling and detailed sexual history testing, as well as location of the test in the context of a routine general practice encounter, effectively normalised and destigmatised the rapid HIV test and made it relatively easy for non-specialist staff to learn and deliver (and for patients to accept). However, HCAs in particular do not routinely test for what is considered a stigmatised and serious condition, so the test did require some change in their role and the way they related to patients—an issue that played out differently with different staff and in different practices (see case studies below).

Staff also commented that patients preferred rapid finger-prick testing to venous blood sampling. The test was technically simple, and phlebotomy skills were not needed. Even patients who disliked needles did not seem to mind the small lancet used quickly in the rapid test, a finding we demonstrated previously in a pilot study [35]. They also said that patients preferred the near-patient test, as they could visibly see that the result was their own, thereby increasing their trust in the test result.

In sum, the INSTI HIV-1/HIV-2 Rapid Antibody Test (the ‘hard core’ of the intervention; see Box 1) was perceived extremely positively by the staff charged with delivering it. Below, we present four contrasting case studies of practices where different individual and organisational factors combined to produce four very different contexts for assimilating, implementing and sustaining the intervention for the duration of the trial.

Practice A (high recruiter): high system antecedents, high system readiness

Practice A implemented the rapid testing intervention very successfully, offering more rapid tests than any other practiceand having a moderate decline rate (42 %), though only one case of HIV was detected via the New Patient Health Check. Our qualitative and quantitative data showed that effective implementation of the test was the result of key system antecedents for innovation, high system readiness for the rapid test and a smooth implementation process and strong adopter factors among front-line staff (see Fig. 1).

Junior practice staff were mentored by more senior staff, providing both pastoral support and opportunities for individual and team learning (the latter linked to the key construct of absorptive capacity; see Table 1). The practice was able to integrate new knowledge through regular practice meetings and feedback. Practice A showed interest in the monitoring of progress and the study’s overall performance, often asking how they rated in relation to other trial practices.

Perhaps partly for this reason, rapid testing was quickly incorporated into the New Patient Health Check and was viewed by staff as a good fit with that process (a construct described in the literature as ‘innovation–system fit’ [23]). Practice A was also one of the few practices that did not stress time constraints (linking with what in the model is called ‘slack resources’, defined in Table 1).

Early in the trial, a positive HIV diagnosis through rapid testing was made, demonstrating that the innovation ‘worked’ and achieved its objective, an attribute known as observability. This is likely to have reinforced the implementation process (see feedback arrows in Fig. 1).

In sum, practice A illustrated many of the key organisational preconditions for successful assimilation of innovation, including key elements of structure (large practice list size, maturity, slack resources, functional differentiation), absorptive capacity for new knowledge (high pre-existing knowledge and skills base and formal and informal processes for knowledge sharing among staff from different professional groups) and high readiness to change (leadership and vision, good managerial relations, risk-taking climate and high-quality data capture). It also showed high readiness for the particular innovation (innovation–system fit) because clinicians were already interested in HIV testing and keen to promote it further. Importantly, nobody in the practice appeared opposed to the innovation.

Practice B (high recruiter): moderate system antecedents, exceptional front-line staff, strong internal synergies

Practice B also assimilated rapid HIV testing very effectively as part of the New Patient Health Check. Despite being one of the small to medium-size practices and having a low turnover of patients (and hence fewer new registrants), this practice diagnosed twice as many patients through rapid HIV testing as any other practice in the study. The number of rapid tests offered (n = 870) was high for practice size, and the rate of tests declined was low (36 %). Yet, the serological testing rate prior to and throughout the trial was fairly low [fourth amongst the 40 (0.66/1000 serological testing rate during the trial period, and 2.07/1000 prior to the trial, respectively) participating practices prior to the trial], suggesting that the practice did not previously place significant emphasis on HIV testing. A number of factors at both the organisational and individual levels may help explain this success.

Although the practice had low serology rates prior to the trial, once testing was introduced and the staff were trained, the intervention was quickly adopted. Staff appeared engaged, seeing the relative advantage of the innovation.

An unusual feature of practice B was that a single individual (the practice nurse) undertook all New Patient Health Checks, for which she had a generous time allocation (30 min for each). She worked full-time and had her own dedicated consultation room. She had a professional and strongly patient-centred approach to her job, working largely autonomously and indicating general enjoyment of what she did.

Another adaptation in practice B was that GPs would refer patients to this nurse for rapid testing, regardless of whether they were booked for a new patient check. The nurse reported that some patients for whom the possibility of HIV infection was being considered were persuaded to have the rapid test when they may have declined the more invasive and less convenient serological testing.

As in practice A, a positive HIV diagnosis through rapid testing was made early in the trial, reinforcing staff confidence in the test.

Practice B is noteworthy, not merely for possessing many (though not all) key system antecedents and readiness factors for innovation and highly motivated front-line staff, but also in the way these elements were combined. The very professional and patient-centred practice nurse, for example, was able to give her very best to the study because the practice allocated plenty of time and allowed the nurse to work independently and adapt the innovation to suit her own working style and local microroutines. More subtly, the culture of the practice was to embrace innovations and support their embedding. Doctors recognised the nurse’s competence and interest in this innovation and began to send her additional patients for testing. In these and numerous other ways, the elements of innovativeness built on one another synergistically.

Practice C (low recruiter): low system antecedents, reluctant front-line staff

Practice C struggled to implement rapid testing. The practice was slow to offer the first test, and its rate of testing remained low throughout the study (in total, 72 rapid tests were offered, and 50 % of these were declined), despite multiple visits and ‘retraining’ from the research team. It had a low serology HIV testing rate prior to and throughout the trial. Low recruitment from this practice was explained by a combination of factors, both organisational and individual.

System antecedents were low in practice C. A small practice, it comprised three GPs, one nurse and one HCA (both of whom undertook New Patient Health Checks), one practice manager and two receptionists. Located within a large building housing multiple practices, the surgery; always seemed crowded and very busy.

Because of the understandable reluctance of busy front-line staff to accommodate the test, it never became routinised within the New Patient Health Check in practice C; it was not offered to most patients having these checks, and, unlike in practices A and B, it never came to be viewed by staff as part of that check. With such low numbers of tests being undertaken, it was not surprising that no cases of HIV were detected using rapid testing, so its observability was not evident in this practice.

The nurse raises an important point here—that the rapid HIV test was not merely a technical procedure but a professional interaction. Technically, it was simple and straightforward (albeit hard to accommodate if time was short), but because of its link to a stigmatising illness, it also required a professional, rather than merely transactional and task-oriented, relationship with the patient. Implicitly, the block to adoption may not have been the HCA’s views per se but the fact that her role—in this practice, at least—was not professionalised. HIV remains a stigmatised condition, and the line between a screening test and a diagnostic test can be fine, particularly in the case of the test used in the trial, which may be interpreted by patients as well as providers (two dots as a reactive result, one dot as a non-reactive result). It may have been that reluctance to offer rapid testing relates to the need to provide immediate feedback regarding test results. Whereas GPs are called upon to share test reactive results, HCAs and nurses expressed significant concern about managing reactive results and patient reactions as well as the interval between the test and calling upon the GP. This may have been a factor in the HCA’s reluctance to test. The nurse, though personally motivated and more professionally experienced, had only limited opportunity to offer rapid HIV testing, as most New Patient Health Checks were performed by the HCA.

It is also significant in the quotation above that the nurse took personal responsibility for trying to change the HCA’s attitude and behaviour in relation to rapid testing. Despite raising the issue with GPs and the practice manager, no action was apparently taken to explore or improve this staff member’s low performance on trial activities. In contrast to the subtle but important involvement of senior clinicians and managerial staff in practices A and B, the approach of similar staff in practice C was distinctly ‘hands off’.

It is noteworthy that the practice nurse made numerous efforts to implement the rapid test, but those efforts had very limited success in the context described above. For example, she showed creativity in ‘reinventing’ the finger-prick aspect of the test. (“As long as I get a decent drop of blood, just occasionally people don’t bleed terribly well. I don’t like the finger-pricker they give with it. I tend to use my ones…. They’re a bit more gentle.”) This motivation and creativity did not translate into tests actually performed, however, because most New Patient Health Checks were done by someone else, and the low absorptive capacity of the practice meant that the nurse’s improved method of testing was not effectively shared with the front-line staff member who had the most opportunity to actually do the test.

In sum, practice C was not an innovative practice, nor was it ready for the specific innovation of rapid HIV testing. The member of staff on which the intervention most depended was personally reluctant, and factors known to help the implementation phase (notably hands-on input from senior staff) were absent. In this environment, the presence of a single, keen and committed member of staff had only limited impact on the implementation of the intervention.

Practice D (low recruiter): keen doctors but low system antecedents and negative synergies

Practice D also struggled to implement rapid HIV testing as a part of the New Patient Health Check. The 557 rapid tests that were offered during the trial period (of which 43 % were declined) may appear relatively high, but the size of the practice and consistent registration of new patients demonstrated a number of missed opportunities for testing. The pattern of testing over time suggests that the innovation was never effectively routinised. Rather, periods with very low rapid testing were interspersed with periods in which a number of tests were performed within a short period of time.

On the surface, this low recruitment rate was surprising. Several of the GPs had a clinical interest in HIV; HIV serological testing rates were high both before the trial and during it (696 performed); and a high turnover of patients ensured high numbers of New Patient Health Checks.

As one of the largest and most diverse practices in the borough, practice D comprised 15 GPs, 9 nurses, 3 HCAs, 2 practice managers and more than 10 receptionists and administrators. Many staff worked part-time. There was time pressure on many activities, and the practice was constantly busy. The striking contrast between the very high HIV serology rates but very low rapid testing rates may be related to our finding that there were two distinct work cultures within the practice. Many of the GPs were highly qualified with some involved in community-based projects. Others had an interest in sexual health and regularly offered regularly offered opportunistic regularly offered opportunistic serology testing for HIV.

However, the nurses and HCAs appeared to have little or no involvement in these activities or protected time to become involved. Knowledge appeared to circulate well among the doctors, but to a much more limited extent between the doctors and the other practice staff, suggesting a problem with absorptive capacity (see the Discussion section). In general, non-medical staff did not have academic links. Many worked part-time and had a very task-oriented attitude toward their work (i.e., they came to work, completed what was expected of them and went home). Some staff described a lack of harmony in practice relationships as well as a sense of being personally overstretched. There appeared to be relational tensions between some staff in the practice that affected the implementation of the study protocol. For example, HCAs had asked reception staff to hand out leaflets about the study to patients at the reception desk, but this did not always happen. Unusually, the research team assisted in mediating this issue.

Although front-line staff expressed enthusiasm about providing testing and acknowledged the value of offering the test, they also viewed involvement in the trial as an additional task in their already high workload. The doctors in practice D viewed involvement in this trial as important both for them as professionals and for the practice population, but they did not appear to discuss with front-line staff how the innovation could successfully be incorporated into an already busy practice. As a result, opponents of the innovation (‘yet another task’) outnumbered supporters, and because it was nurses and HCAs who actually delivered the intervention, these individuals were more strategically placed to do so. Bursts of trial activity probably reflected periodic encouragement of front-line staff by GPs concerned to increase the practice’s performance statistics, but this is very different from routinising the innovation as business as usual (see the Discussion section).

Although decision making about offering the rapid test was largely devolved to front-line teams, this was complicated by poor communication and strained relationships, to the extent that front-line staff did not appear inclined to take responsibility for implementation. There was also a significant problem with time and resources because HCAs were often called upon to refocus their work for short periods to meet particular practice goals. There was little inter-practice feedback unless it was prompted by the study team, minimising opportunities for creating the kind of positive feedback loops that were evident in practices A and B.

Moreover, despite a number of HIV diagnoses made using serological testing, no diagnoses were made using rapid tests, indicating a lack of observability. It is telling that, whereas doctors in practice B altered their behaviour during the trial by sending patients to the nurse for rapid HIV testing, those in practice D continued to use serological testing when they suspected possible HIV in a patient. It appears that the rapid testing was seen as the province of a different group of staff, not something that was business as usual. GPs become involved in the rapid HIV testing algorithm in cases of reactive, indeterminate or invalid results, but because none occurred at practice D, this may have impacted their knowledge and involvement in trial activities.

In sum, despite much initial enthusiasm, practice D was impeded by a combination of structural, capacity-related and cultural factors (most crucially, limited slack resources), along with individual adopter traits and a weak process of implementation.