- Open Access
Explaining the mixed findings of a randomised controlled trial of telehealth with centralised remote support for heart failure: multi-site qualitative study using the NASSS framework
Trials volume 21, Article number: 891 (2020)
The SUPPORT-HF2 randomised controlled trial compared telehealth technology alone with the same technology combined with centralised remote support, in which a clinician responds promptly to biomarker changes. The intervention was implemented differently in different sites; no overall impact was found on the primary endpoint (proportion of patients on optimum treatment). We sought to explain the trial’s findings in a qualitative evaluation.
Fifty-one people (25 patients, 3 carers, 18 clinicians, 4 additional researchers) were interviewed and observed in 7 UK trial sites in 2016–2018. We collected 110 pages of documents. The analysis was informed by the NASSS framework, a multi-level theoretical lens which considers non-adoption and abandonment of technologies by individuals and challenges to scale-up, spread and sustainability. In particular, we used NASSS to tease out why a ‘standardised’ socio-technical intervention played out differently in different sites.
Patients’ experiences of the technology were largely positive, though influenced by the nature and severity of their illness. In each trial site, existing services, staffing levels, technical capacity and previous telehealth experiences influenced how the complex intervention of ‘telehealth technology plus centralised specialist remote support’ was interpreted and the extent to which it was adopted and used to its full potential. In some sites, the intervention was quickly mobilised to fill significant gaps in service provision. In others, it was seen as usefully extending the existing care model for selected patients. Elsewhere, the new model was actively resisted and the technology little used. In one site, centralised provision of specialist advice aligned awkwardly with an existing community-based heart failure support service.
Complex socio-technical interventions, even when implemented in a so-called standardised way with uniform inclusion and exclusion criteria, are inevitably implemented differently in different local settings because of how individual staff members interpret the technology and the trial protocol and because of the practical realities and path dependencies of local organisations. Site-specific iteration and embedding of a new technology-supported complex intervention may be required (in addition to co-design of the user interface) before such interventions are ready for testing in clinical trials.
BMC ISRCTN Registry 86212709. Retrospectively registered on 5 September 2014
As the number of patients with heart failure grows and services are increasingly overstretched, telehealth is often depicted as a partial solution [1, 2]. A key determinant of outcome in heart failure is the proportion of patients on maximum tolerated therapy , but most heart failure patients are on sub-optimal doses of medication [4, 5]. The efficacy of telehealth solutions for heart failure has been widely studied in randomised controlled trials (RCTs), some but not all of which have shown a small benefit over usual care, as measured (for example) by reduced hospital admissions [1, 2]. Where benefit has been demonstrated, it has generally been attributed to more timely and detailed provision of biomarker data (e.g. blood pressure, weight, oxygen saturation), which allows prompt adjustment of medication in response to indicators of decompensation . A new generation of trials has therefore focused on exploring and optimising the human component of the telehealth intervention [6, 7].
SUPPORT-HF2: a randomised trial of a complex socio-technical intervention
SUPPORT-HF2 was a RCT in 7 UK sites, which aimed to improve the use of recommended medical therapy in heart failure as defined by evidence-based guidelines [7, 8]. A total of 202 participants (mean age 71; mean left ventricular ejection fraction 33%) were selected for being at high risk of adverse outcomes or high potential to benefit from remote management; they were randomised to ‘supported medical management’ (intervention) or ‘enhanced self-management’ (control, so named to avoid participants feeling they were in a no-treatment arm). Those in both arms submitted daily symptom reports and measurements of weight, blood pressure and heart rate, alongside free-text comments. In the intervention arm, home monitoring was combined with a clinical decision support system that provided risk rankings and tailored alerts. These were processed by the central clinical management (CCM) team—a cardiologist and heart failure specialist nurses (HFSNs)—who acted on these system-generated alerts to provide treatment recommendations (where relevant) to patients and their health professionals.
In the control arm, patients’ measurements were recorded in raw format only, without any processing by the clinical decision support system or the CCM team. Feedback messages were generated automatically as soon as patients entered their data. If readings fell outside pre-defined ranges (according to evidence-based guidelines), automated messages encouraged patients to contact their usual health professional. Participants in both arms were also provided with educational self-management modules on the tablet devices.
We describe the hypothesis, inclusion criteria, blinding and outcome measures for the SUPPORT-HF2 trial in the Appendix. More details can be found in previous publications, which describe the trial design and baseline participant characteristics  and the quantitative findings . Despite being adequately powered, the trial showed no statistically significant difference between treatment arms in the primary outcome measure (proportion of patients on optimum medical treatment as measured by a ‘mean opportunity score’) or in various other endpoints including heart failure severity and disease-related quality of life .
Following the completion of the SUPPORT-HF2 trial, the technology is currently (February 2020) in development prior to commercial launch. The purpose of this paper is to describe a qualitative study into how the complex intervention of centralised remote support was interpreted by patients and staff and how, despite careful standardisation of the inclusion criteria, it played out differently in the different sites.
The technology and linked service model
The patient-facing component of the technology consisted of a tablet personal computer, blood pressure and heart rate monitor, and weighing scales. The tablet portal listed current medication, health data entered by the patient, educational modules on heart failure and a messaging link to SUPPORT-HF2 clinicians. Patients were asked to complete a daily symptom checker (e.g. questions on activity and breathlessness). They then measured their blood pressure and weight (using peripheral devices supplied as part of the trial) and entered the data, which were transmitted from the peripheral devices to the SUPPORT-HF app on the participant’s tablet via Bluetooth. Patients could view their data in graphical form along with an indication of the normal range.
Data entered onto the tablet were sent securely to the trial CCM team by 4G (4th generation mobile network technology) or high-speed internet access (Wi-Fi) connection. Clinicians (including usual care teams) had access to patient data in raw format ‘with no ranking or interpretation’ via a secure login. For patients in the intervention arm, the CCM team was supported by a clinical decision support system which colour-coded patient data depending on the estimated risk of deterioration or deviation from globally defined parameters. The CCM team also drew on pre-established management plans, blood test results and other information from electronic records and compared these with optimal therapy targets via a central dashboard .
When a patient’s medication could be improved, the CCM team sent a letter to the patient’s GP. The lead study nurse would confirm that the advice had been acted on, either by receiving a message from the patient or during 3-monthly scheduled reviews. Patients in the control arm received automated messages encouraging them to see their GP if their measurements fell outside pre-defined ranges, but no treatment or drug titration recommendations were provided. Patients in both arms were advised that SUPPORT-HF2 was an ‘add-on’ service that did not replace standard GP or hospital care.
‘Usual care’ for heart failure in the UK generally consists of routine management in general practice and acute or elective admissions as needed to hospital in general medical, elderly care or cardiological wards. It also includes follow-up after a hospital admission by either a secondary care team (consultant or heart failure specialist nurse-led clinics) or in primary care (general practice, sometimes supported by community-based specialist nurses offering home-based or community clinic care). Sometimes, patients are discharged from an acute heart failure admission solely to the care provided by general practitioners, which can vary considerably depending on the latter’s skills and workload pressures.
This qualitative study took place alongside the SUPPORT-HF2 trial. The evaluation questions were as follows:
How and why was the intervention (and the control intervention) implemented differently in different settings?
What was the patient experience of the remote monitoring technology and service as delivered in the intervention and control arms of the trial?
What was the staff experience of the technology and service as delivered in the intervention and control arms of the trial?
What were the material, technical and clinical challenges in delivering the technology and linked service model as defined for the trial?
These questions were refined slightly as the study progressed, for example, to take account of the differences between sites in existing heart failure services, which had a bearing on how the intervention was embedded locally.
Study design, governance and ethical approval
We conducted a multi-site qualitative evaluation with data collection from all 7 sites. The study was overseen by the Studies in Co-creating Assisted Living Solutions (SCALS)  steering group which had a lay chair and representation from NHS, social care, external academics and patients (some of whom had heart disease). Ethical approval for the qualitative evaluation was obtained from the Oxfordshire South Central Research Ethics Committee (REC no. 15/SC/0553) in September 2015 and subsequent amendments (which covered all sites visited).
Sampling and participants
Between May 2016 and September 2018, we interviewed and observed 51 people and collected various documents (summarised in Table 1).
Patient participants included people living with heart failure who were taking part in the SUPPORT-HF2 trial (both arms), plus, where relevant, their family members or carers. They were initially approached by heart failure nurses and trial managers. We sampled to obtain variety in clinical background and different experiences using the technology. Interviews were semi-structured and lasted 30–90 min; most were conducted in patients’ homes. One patient discussion group (comprising 6 patients with heart failure and one carer) took place in a side room off a cardiology ward and lasted 90 min. Topic prompts included living with heart failure, experience of using the remote monitoring technology, and experience of clinical and research encounters.
Staff participants included consultant cardiologists, heart failure specialist nurses (HFSNs), general practitioners (GPs) and SUPPORT-HF2 trial staff (those managing the study from the central hub site and those recruiting and supporting patients in the 6 other sites). Discussions covered clinicians’ experiences of managing patients with heart failure, the role of technology in supporting patients’ self-management and communication with health professionals, and their experiences of being part of the SUPPORT-HF2 trial.
Documents included national and international guidelines for acute and chronic heart failure management, the annual output from the National Heart Failure Audit, heart failure nurse operational procedures and standard clinical information collection templates for patients outside the SUPPORT-HF2 trial.
Data management and analysis
Forty-two of the 51 interviews were audio-recorded with consent and transcribed; the remainder were recorded as contemporaneous notes. Anonymised transcripts and other data sources were imported into NVivo 12. We analysed qualitative data thematically and produced summary narratives that we progressively refined over time, adding each new data item to an increasingly nuanced account of the overall patient, carer and staff experience . We discussed ongoing data collection and emerging findings in regular team meetings that helped us shape interim analysis and further data collection.
We drew on socio-technical theory—of which there are many interpretations and versions [9, 11]. Broadly speaking, socio-technical theories depict technologies as part of complex systems; they focus on how those technologies are perceived, interpreted and used by individuals and how the use (or non-use) of particular technologies affects and is affected by the wider system. A socio-technical approach to technologies in the home considers how and to what extent they are ‘domesticated’, both technically (e.g. do they work; are they dependable?) and symbolically (e.g. are they reassuring or threatening?) . A socio-technical approach to technologies in the workplace considers how they influence work practices and routines and how their use either enhances or challenges professional standards of quality, safety and equity of care [11, 13]. An early and important contribution to socio-technical theory was Albert Cherns’ work on the need for extensive reconfiguration of work practices to ensure that new technologies are smoothly embedded in processes and systems .
The specific socio-technical lens used for this analysis was NASSS, a multi-level theoretical framework which considers non-adoption and abandonment of technologies by users and challenges to organisational scale-up, spread and sustainability (Fig. 1) . NASSS has 7 domains, each of which may be characterised by complexity (that is, unpredictability, interdependence with other domains and unintended consequences): the illness or condition, the material features and functions of the technology, the value proposition (both financial and non-financial value, including negative value, associated with the technology), the intended users (staff, patients and carers), organisational characteristics (including general capacity to innovate, readiness for the technology, extent of change needed to revise existing routines and the work needed to plan, implement and monitor the change), wider system issues (including the technical context, such as availability of broadband, and the policy context, such as how national and regional policy decisions are playing out locally) and how all these domains interact and evolve over time. We analysed each domain (and their interactions) to consider how SUPPORT-HF2’s ‘standardised’ socio-technical intervention played out differently in different trial sites. The application of NASSS to a process evaluation is an example of what Mills et al. call a ‘type 4 logic model’—that is, a systematic way of exploring interdependencies in dynamic systems .
The study generated a rich and heterogeneous qualitative dataset comprising several hundred pages of interviews, ethnographic field notes, email exchanges and extracts from documents. Not all of these data were pertinent to this analysis; the NASSS framework guided us to relevant material. Below, we draw on the domains of the NASSS framework (Fig. 1) to explain how the trial was received and operationalised in the different settings, leading to different impacts on the adoption process and the care pathway.
Domain 1: The illness
Heart failure is a heterogeneous condition with a wide range of severity and different underlying causes and comorbidities; because of its links to smoking and body mass index, it disproportionately affects the poor and those with low health literacy . Patient participants in the SUPPORT-HF trial had a mean age of 71 (standard deviation 11) years. A high proportion had comorbidities (the commonest, in order of frequency, were atrial fibrillation, hypertension, ischaemic heart disease, diabetes, chronic kidney disease, chronic obstructive pulmonary disease and past stroke). The mean ejection fraction was 37% (SD 12%); 66% had heart failure with reduced ejection fraction. Their heart failure varied in severity from class 1 to class 4 on the New York Heart Association scale. All researchers were blinded to which arm the patients were in when they were assessed, so the data below reflect the experiences of patients in both intervention and control arms. As noted above, patients in both arms received the same equipment, though their health providers were involved in different ways.
The patients we interviewed (sometimes together with family members) had different experiences living with heart failure and other medical conditions; came from different socio-economic backgrounds, from rural and urban settings; and had varied views on the technology.
Heart failure patients typically described a range of symptoms alongside several comorbidities:
Had been generally suffering with breathlessness and tiredness, and pains in my chest, etc. Being a typical bloke, I did not really do anything about it. [Then] I fell down the stairs, top to bottom. And fractured a vertebrae. […] they have introduced and taken away [medicines] I’m quite light-headed most of the time. But I got kind of a double whammy on that, because I’m suffering from depression and anxiety as well […] I get migraines quite badly. I’ve got osteoporosis in the spine, as well […] memory’s not very clever either. Interview 9, heart failure patient, site E
Another patient described her effort to adjust to her new way of life, which required changes in the things she enjoyed and meant she had to rely on extra support:
And if I get up very slowly I’m alright, as the day goes on, I get better. I have got support rails. And I take a stick. But I try to get round the garden, to get a bit of exercise. Of course I cannot take [the dog] out now […] I do like to keep up standards and things as much as possible. I do have help in the house. Because some of the housework was rather difficult. Interview 7, heart failure patient, Site G
In some cases, the way patients experienced their conditions was clearly linked to how they engaged with the SUPPORT-HF technologies. Many trial participants suggested that the technologies captured well some of the key indicators for their heart condition. Others, however, found it more challenging to use the technology in the wider context of their health difficulties. A man in his early 50s with heart failure became withdrawn and abandoned the technologies when he unexpectedly found out he was not eligible for a heart transplant following a 6-year wait, due to a comorbid condition. Other patients with severe heart failure and/or comorbidities found the intervention extremely burdensome; some felt physically unable to use the weighing scales (one was an amputee; another had difficulty with balance). One man with severe fluid retention awaiting a heart transplant could not bear the anxiety engendered by a data trend indicating weight increase. Another lady explained how she was initially frightened by the diagnosis and felt the technologies made her even more anxious, but this subsided as she became more comfortable managing her condition. Others viewed the technology primary purpose as supporting their immediate health concerns (e.g. one believed, incorrectly, that stable weight indicated that a different comorbid condition was under control) and placed less value on monitoring stable indicators.
The staff gave examples of patients who, for various reasons including comorbidity, anxiety or wider life issues, declined to participate in the trial, withdrew after enrolment or expressed relief when the trial ended. Such cases were relatively rare, though withdrawal rates varied across sites.
Whilst the specific nature of the participants’ illness had a major influence on whether and to what extent they accepted and used the technology, because of the randomisation process, these differences were evenly distributed between the intervention and control arms of the trial. It is worth noting that of 363 patients assessed as eligible for the trial, 126 (35%) were considered ‘unsuitable’ by clinical staff who were participating in recruitment. Reasons for this are discussed in other sections below.
Domain 2: The technology
The tablet device used for SUPPORT-HF2 had been co-designed with patient input  and tested in a usability study (without the central support component) in which patients suggested some minor adjustments [17, 18] before the trial began. In an early qualitative study, patients were visited at home to document the variable and emergent ways in which they appropriated the technology, made sense of it and embedded it in their routines . Given this careful preliminary design phase, it is reassuring that most patient participants and their carers described the SUPPORT-HF2 technology as ‘easy to use’.
But this iPad is quite simple. You just switch it on. And it - You pick what you want to do. Weighing, or blood pressure. Just touch it, put it on there. Put the thing on, it’ll do it. The iPad will pick up the thing. It’d show you what it had read. Boom. Thank you for doing it. Done, that’s it - boom, finished. That was it. Interview 8, heart failure patient, Site G
The technology was not, however, ‘plug and play’. It was necessary to visit every participant at home, usually on two occasions, to set up the equipment and show the patient how to use it. This was a time-consuming process (typically 2 h for the first visit and 1 h for the second), though it is not clear how much of this time was spent setting up the equipment and how much on requirements specific to the clinical trial. Some patients needed extra support to feel confident with it.
Patient participants raised a number of practical issues with the home equipment. Sometimes, synchronisation of readings via Bluetooth between the monitor and the tablet was slow or there were problems with data transmission due to fluctuations in 4G signal; these problems usually resolved if a Wi-Fi connection could be established. These transmission issues sometimes led to patient confusion as they could see that the data had not been ‘sent’. Some felt they could not rely on the machine and assumed a responsibility for confirming their data had been received by the central support team and their medicines were listed correctly.
When technical difficulties occurred (quite commonly with WiFi or Bluetooth connection, but also due to malfunctioning equipment), some patients found solutions themselves, but in most cases, input from the trial team and local nurses was crucial for resolving them quickly. Research nurses used a workaround to help manage synchronisation or transmission issues. They gave participants a separate phone number to use if the messaging facility of the tablet was not working, and if they had not received any data or messages from a participant for a day or two, they would phone them to check that all was well. In most cases, such problems were due to patients not connecting properly to their own Wi-Fi.
Occasionally, the ‘easy-to-use’ tablet malfunctioned because of user error.
Another person had somehow changed the setting so it was now in a foreign language. And we couldn’t read it to correct it. Had to get [CCM research nurse] involved and she talked us through screen by screen. Research administrator, Site E [SUPPStaff15]
The trial design allowed for some iterative changes in the technology. Most of these were made to facilitate data collection for research, though the bioengineer’s input was sought to make adjustments to the dashboard with the aim of improving the clinical care of participants.
Domain 3: The value proposition
The SUPPORT-HF technology was part of a research trial, so (during the study period at least), there was no immediate requirement to demonstrate affordability or value for money, and the varying technology budgets of the different sites did not affect adoption decisions. It is worth noting, however, that the software changes described in the previous section were resource-intensive and costly, and this seems to have constrained the number of co-designed modifications. In the case of one patient, for example, the system did not flag up a clinically important weight increase because it did not occur rapidly enough (i.e. over a 3-day period—the conventional assessment time frame). The central support team worked with IT colleagues to adjust the system to change the alert algorithm in this case, but such changes were labour-intensive and hence expensive and unscalable. The CCM lead nurse commented that further refinement of the dashboard, enabling safety alert flags to be easily individualised, could improve the value of the technology since it would allow ‘personalisation’ of the alerts.
With any technology, there is a trade-off between advanced functionality and development cost. A basic product with ‘must-have’ but not ‘nice-to-have’ functions may prove more cost-effective. The SUPPORT-HF2 blood pressure monitors did not, for example, include the now widely available ‘Irregular Heart Beat Indicators’ (based on variability in the R-R interval on ECG). Since decompensation in heart failure is often triggered by new-onset atrial fibrillation, community staff suggested this might have been a useful inclusion, although potentially pushing up the price. The absence of rhythm detection meant that the possibility of undiagnosed atrial fibrillation affected confidence in remotely recorded blood pressure readings.
Domain 4: The intended adopters
Aside from issues raised by the nature of their health conditions (domain 1), most patient participants were interested to see their readings and described the technology as well-designed. They used the tablet and the peripheral devices without too much difficulty and saw great value in monitoring their condition, especially in terms of gaining reassurance and legitimising help-seeking when they needed clinical care. Those who continued to use the tablet until the end of their study participation regretted having to hand it back at the end of the trial. Others, however, described it as ‘intrusive’ and found it impossible to get into the routine of regular monitoring; a few were sceptical about its role in supporting clinical care:
Somebody is looking at a screen, without seeing the patient, saying what you should do. Without actually examining the patient. So, that was a bit - That wasn’t really - Yeah, I was [skeptical], about that. Yeah. That they don’t actually see the patient, they’re just based on figures. Interview 13, heart failure patient and wife, Site C
The trial also assumed that patient users would largely remain static in their homes where they would have good access to the devices on an everyday basis. This was not the case for more active patients (especially those who travelled away from home) who found this too restrictive, and one or two withdrew from the study for this reason.
Despite some challenges with implementing the intervention (see domain 2), most staff also expressed positive views on telehealth technology enabling remote support in heart failure care (a finding that should be interpreted in the context that these staff interviewees were almost all linked to the study in some way so had ‘skin in the game’). Many HFSNs felt that patients had benefited from both the technology and the day-to-day input from the central support team.
There were, however, tensions in the way different professional groups made sense of the potential of the technology in heart failure care. For example, some consultants thought that the reluctance of HFSNs to engage with the technology stemmed from a belief that the technology would replace their jobs. The nurses themselves had a more nuanced view. They expressed concern that if routine uptitration in relatively stable patients was carried out by a telehealth service, they would lose the valued reference point of the straightforward, treatment-responsive patient and be left with a case load of unrelentingly complex and unstable cases.
Another concern of nurses in particular (and also expressed by some patients) was that the structured and algorithmic element of the telehealth intervention would miss important aspects of quality care. Technology, HFSNs felt, provided narrow and decontextualised information and could not replace the nuanced and holistic assessments that experienced clinicians undertook on their patients—including, for example, home visits which gave them rich information about the patient’s environment and allowed them to observe how the patients approached activities of daily living and medication management. The cardiologist chief investigator, however, rejected the nurses’ characterisation of the SUPPORT-HF2 intervention as crudely algorithmic. On the contrary, he argued the intervention design recognised and accommodated the need for human input and judgement where necessary, whilst attempting to streamline redundancy and repetition.
The question of professional lines of responsibility for patients, especially in terms of nursing care, also caused concern amongst clinical participants. The SUPPORT-HF study team envisaged that the intervention would run alongside and complement usual care and not challenge the work of local heart failure teams, and the trial protocol assumed that data generated by the decision support dashboard and the advice given by the CCM would be uncontested and unproblematic. In reality, clinical disagreements sometimes led to conflicts between nurses from different teams and required time-consuming efforts to resolve differences and negotiate professional boundaries:
What we have all found is that co-managing doesn’t work. Because the numbers are saying one thing but we have visually seen the patients and we may have known those patients for a long time, know their complexities and when the SUPPORT-HF team give advice we sometimes disagree with some of the decisions. We’ve had to unpick that, it’s very complex. It meant a lot of sitting in a back room making calls Community HFSN, Site A [FACEHFSN3]
These conflicts appeared to stem from two things: an implicit knowledge hierarchy in which the technology’s decision support system appeared to override their personal knowledge of the patient and the fact that patients in the control arm were sent a generic alert (and a prompt to see their usual clinician) but no specific support. Neither the local trial staff nor the non-trial clinicians knew which arm the patients had been randomised to, so there was sometimes uncertainty about who should take action when an alert occurred.
Principal investigators in the study sites (who were all cardiologists) did not perceive role conflicts with the central remote support team, with the one exception of the cardiologist in site B, who vetted that team’s recommendations. He said he knew the patients personally and could take account of exceptions, and he felt the GPs (whose views on telehealth had been influenced by adverse past experiences) would have greater confidence in implementing recommendations from him than from the CCM.
Domain 5: The organisation
Study sites joined the trial with different existing service models, staffing levels, technical and clinical capabilities, and distribution of professional responsibilities. They had different local demographics, histories, cultures and past experiences with telehealth and other technology projects. Whilst recruitment criteria for SUPPORT-HF2 were the same in all sites, the organisational service context for each site (Table 2) led to variability in how the intervention was implemented and the extent to which it aligned or clashed with existing organisational and inter-organisational routines.
In site A (the main ‘hub’), some of the patients initially recruited by the CCM lead research nurse were also receiving support from a nurse-led community heart failure service. It proved very difficult to align the trial protocol with the existing community care routine, so the clinical teams eventually arrived at a policy of keeping trial participants separate from the community-based service and recruiting patients from areas with less intensive community-based services. Site D also had a nurse-led community heart failure service; these HFSNs appeared less interested in the trial and made few referrals (in a few cases, they actively resisted cooperation). But, the hospital-based team in site D actively recruited patients to SUPPORT-HF, partly because they saw remote monitoring as a solution to long geographical distances.
In site B, neither community HFSNs nor GPs recruited patients or participated actively in the study (probably because of a negative experience with a previous telehealth trial); the hospital research nurse and principal site investigator (a consultant cardiologist) took on responsibility for recruitment and prescribing. In site F, hospital-based HFSNs had a large number of complex or unstable patients which they deemed unsuitable for the trial (given the lack of active management in the control arm) and decided to refer only the more straightforward patients whom they would normally have referred to community HFSNs. In this site, the trial served as an alternative rather than an add-on to the usual care pathway, which may also explain why recruitment stalled part way through the trial.
In site C, there was no local cardiologist involvement in the trial; the community HFSN team actively recruited patients whom they were about to discharge to their GPs for follow-up, because they saw the intervention as a useful supplement to existing care. Although these community HFSNs would sometimes continue to see patients who were participants in the trial, no interprofessional conflicts were described with the CCM team (unlike in site A). In sites E and G, there was no specialist community nursing to augment GP management of heart failure; SUPPORT-HF2 was welcomed by local clinicians as it aligned well with pressing service needs. In site E, the principal site investigator (consultant cardiologist) deliberately sought to recruit patients in localities with substantial pressures on hospital outpatient appointments and lack of community HFSNs. In site G, the hospital outpatient clinic seized the opportunity to ‘hand patients over’ to the trial due to the lack of resources for specialist care post-discharge.
It fits in really nicely with the existing infrastructure. We need a way of surveilling all those being up-titrated. And to monitor the sicker ones such as those receiving home IV diuretics. So we can use it for disease management but also to see progression, and see when they’re falling off their perch. Consultant, PI, Site E [SUPPStaff11]
This is consistent with the SUPPORT-HF2 protocol, which hypothesised the value of digital health interventions to be higher ‘in contexts where quality of care is (on average) suboptimal with substantial unwarranted variability at the provider-level.’ (, p.62). The different experiences across sites, however, highlight the challenges of running an RCT alongside existing standard care that may be historically embedded and involve staff with varying degrees of buy-in to the research.
The different existing routines and practices (and the extent to which front-line staff were able and willing to adjust them) had a direct impact on the care pathway in the SUPPORT-HF2 trial. The rationale behind the trial included the importance of a tight feedback loop in deteriorating patients. This was sometimes achieved successfully, with local input from site staff and active intervention by the central support unit via the SUPPORT-HF2 dashboard (see Fig. 2).
Sometimes, however, the feedback loop did not work as intended, mainly due to challenges in communicating recommendations to GPs, establishing whether these had been actioned (either by GPs or by patients themselves) and confirming time frames for up-titration. Factors outside the control of the trial staff and local clinicians (for example, the unavoidable use of traditional ‘snail mail’ letters with GPs who at the time used neither email nor faxes) caused considerable delays in some sites.
Domain 6: The wider system
This telehealth trial tested the effectiveness of remote monitoring with centralised clinical support for heart failure patients but also sought to collect a dataset for machine learning and algorithm training to improve early warning prediction of deterioration. This was aligned with a wider orientation for automated, data-driven clinical care in the UK, supported by policy priorities for the development of artificial intelligence-enabled solutions [19, 20]. Despite being forward-looking (envisioning a future in which heart failure clinicians’ work would be supported and made more efficient by state-of-the-art technologies, thereby potentially overcoming workforce shortages and funding shortfalls), the trial encountered practical challenges as it relied on legacy infrastructures for data transmission and collection. For example, in site B, half of all potentially eligible patients could not be randomised due to the lack of broadband coverage in the area where they lived. The study also aimed to collect data on clinical investigations, events and medications directly from participants’ electronic health records to complete its dataset. However, in the majority of sites, it was not possible to achieve this level of integration with secondary and primary care systems in the duration of the trial. Instead, the trial team attempted to collect this information directly from patients, their clinicians or local research staff.
Domain 7: Evolution over time
There were three key areas where evolution over time affected the way the trial was operationalised. Firstly, over time, there were substantial changes in both staffing and priorities, which meant that the interpretation of the trial aims and the way it was embedded in the service changed as well, and this was reflected (for example) in fluctuating recruitment rates. Secondly, heart failure patients with complex comorbidities received care from multiple health professionals, and their health status changed during the trial, which required constant updating of patients’ medication and other clinical details on the SUPPORT-HF2 dashboard in the absence of data integration (updates were planned every 3 months for the control group and every 2 weeks for the intervention group). In the face of emergent challenges, the CCM nurse demonstrated significant adaptive capability, resolving technical difficulties and co-ordinating with patients and other health professionals to try and close the feedback loop. Thirdly, as mentioned earlier, adapting the technology to the level required for emerging trial needs proved impossible given funding restrictions.
Summary of main findings
This qualitative study of the SUPPORT-HF2 trial produced a number of key findings. First, heart failure patients and clinical staff saw potential in a technology that supports regular data monitoring but also identified challenges in mainstreaming its use. Trial participants experienced the technology as usable and useful; they were (mostly but not invariably) reassured by the monitoring and support and found that the data informed and legitimised their help-seeking behaviour. However, sicker patients and those with comorbidities found regular monitoring burdensome (something we have demonstrated previously in relation to self-monitoring more generally ) and described feelings of alienation or saw the technology as a constant reminder of their ill health. Clinical staff valued the technology, but they also identified challenges in reconciling what they saw as an algorithmically driven intervention with holistic, personalised care based on long knowledge of the patient. Co-management of patients by both the central trial team and community HFSNs produced tensions of professional responsibility and judgement.
The second key finding was that, as predicted by the NASSS framework, significant human effort was required to embed the intervention and make it ‘work’. It was often necessary for patients or staff to resolve issues with network or device connectivity to ensure that data were collected and transmitted reliably. Patients felt responsible for confirming that their data had been transmitted and providing context for them. Clinical staff had to engage in significant co-ordination work to communicate recommendations to other health professionals and/or to patients, establish whether these recommendations had been acted on, and confirm steps and time frames for medication changes. In contrast to the assumptions in the protocol, these were not neutral information exchange tasks but required professional negotiations and careful management of boundaries and expertise between clinicians.
The third key finding was that, again as predicted by the NASSS framework , implementation of this complex trial was highly variable across sites because of local contextual issues (although inclusion criteria were standardised). Existing service models, staffing levels and distribution of responsibilities, capacity at each site and previous experience with telehealth all had a strong influence on the extent to which the intervention was accepted and implemented. In some settings, technology-supported remote specialist input was seen as potentially useful in supplementing existing care and readily mobilised to fill specific gaps in service provision (including, in one site, using it as a safety net for sicker patients). Sites with relatively comprehensive existing services accommodated the SUPPORT-HF2 trial mainly by targeting it to patients considered stable enough not to need local HFSN input. In some sites, the trial was actively resisted for historical reasons. Recruitment and implementation were also influenced by concerns that patient care should not be compromised if they were allocated to the control arm.
Comparison with other studies
The use of telehealth in heart failure management remains controversial, with some clinicians and policymakers strongly enthusiastic [21,22,23,24] and others unconvinced or opposed [25,26,27]. In a recent review carried out by our team, we found that telehealth showed more benefits when ‘usual’ heart failure care was sub-optimal and when it targeted high-risk patients—its success improved with the number of variables monitored, the frequency of monitoring and the timeliness of human remote support . However, the literature on telehealth in heart failure (especially trials) appeared to suffer from publication bias, and poor recruitment was a common challenge [1, 28].
Factors shown to account for poor uptake of telehealth by heart failure patients include preference for face-to-face encounters, lack of perceived relative advantage of the technology over existing care, physical or mental impairments and lack of confidence that limit patients’ ability to use the technology . Consistent with our findings on the implementation of SUPPORT-HF2, several studies have also identified clinician non-acceptance as a major factor in the low uptake of telehealth [29,30,31,32,33]. It has been proposed that champions in telehealth service development could support acceptance as they work to enthusiastically cultivate relationships and promote and legitimise telehealth . Technical factors identified in this study, such as challenges with bandwidth availability (esp. in rural areas) and connectivity, and lack of interoperability with electronic patient records, have also been identified elsewhere .
Previous research has consisted of either outcome-focused RCTs or (less commonly) qualitative studies in a non-RCT setting. This study has extended the knowledge base by producing ‘behind-the-scenes’ qualitative insights into the socio-technical challenges of running a multi-site RCT of a highly complex care intervention. We have shown that however committed trial teams are to delivering a standardised intervention across multiple sites, in reality, the intervention will be shaped and constrained by historical path dependencies and local realities.
Clinical and practice considerations
In chronic heart failure, episodes of acute deterioration are common and if not promptly treated may lead to rapid decompensation. An assumption underlying SUPPORT-HF2 is that earlier detection of deterioration using the remote monitoring equipment (blood pressure, body weight, particular symptoms) could trigger rapid intervention that would prevent decompensation. But heart failure is a very complex condition with heterogeneous aetiology and frequent comorbidities, which means that many patients will be ‘exceptions’ to the standard algorithm and a high degree of personalisation of care is often required. For this reason, it will be hard to demonstrate in a trial that a standardised intervention significantly impacts on outcomes such as hospital admission rates or contacts with the health service.
Secondary care heart failure nurses emphasised safety concerns for the often very complex and unstable patients attending their clinics. Most took a strongly precautionary stance (possibly beyond that intended and expressed in the protocol’s exclusion criteria) that patients should not be recruited if there was a potential for harm. Such a restrictive recruitment strategy, whilst understandable from a professional perspective, not only slowed recruitment but may also have diluted the effect of the intervention (since patients likely to benefit from prompt efficient medication optimisation and enhanced monitoring rarely entered the trial at all). This conundrum is relevant to the design of any future trial of telehealth-supported care in heart failure.
Strengths and limitations
This qualitative study drew on a large volume of data to understand variability in individual adoption and organisational assimilation of central remote specialist support in a telehealth trial for heart failure. A multidisciplinary research team was involved in the analysis of the data, combining an understanding of clinical aspects and current service models in heart failure, with expertise on the qualitative evaluation of digital health solutions and the social study of technology. Whilst we sought to draw on a maximum variation sample and interview participants with different experiences with the technology, those who had negative views may be under-represented. We were unable to identify patients who had declined to participate in the trial (although we did interview patients who withdrew). As trial participants, our interviewees were not necessarily representative of heart failure patients in general. Community HFSNs who had refused to refer patients to the trial were also unavailable for the interview by our team. The slow recruitment of participants in the trial was due partly to the non-engagement of front-line nurses, but the reasons for their resistance could only be explored indirectly.
This paper has attempted, using socio-technical theory and the NASSS framework, to contextualise some of the SUPPORT-HF2 trial findings and explain the apparent lack of efficacy of a complex intervention involving both technology and human input. To summarise, in sites with insufficient specialist input to heart failure management in primary care, SUPPORT-HF2 was seen as a potentially useful component to patient care. At sites where care pathways in the community were relatively comprehensive, patient recruitment was highly selective and focused on patients who had had less to gain from the intervention. Our findings suggest, therefore, that the trial has not demonstrated definitively that the intervention is ‘ineffective’.
Complex socio-technical interventions, even when implemented in a so-called standardised way with uniform inclusion and exclusion criteria, are inevitably implemented differently in different local settings because of how individual staff members interpret the technology and the trial protocol and because of the practical realities and path dependencies of local organisations. Site-specific iteration and embedding of a new technology-supported complex intervention may be required (in addition to co-design of the user interface) before such interventions are ready for testing in clinical trials.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Central clinical management
Heart failure specialist nurse
Randomised controlled trial
Studies in Co-creating Assisted Living Technologies
Seamless User-centred Proactive Provision Of Risk-stratified Treatment for Heart Failure
Greenhalgh T, Shaw S, A’Court C. Understanding heart failure; explaining telehealth–a hermeneutic systematic review. BMC Cardiovasc Disord. 2017;17(1):156.
Dickinson MG, Allen LA, Albert NA, DiSalvo T, Ewald GA, Vest AR, Whellan DJ, Zile MR, Givertz MM. Remote monitoring of patients with heart failure: a white paper from the Heart Failure Society of America Scientific Statements Committee. J Card Fail. 2018;24(10):682–94.
Pocock SJ, Ariti CA, McMurray JJ, Maggioni A, Kober L, Squire IB, Swedberg K, Dobson J, Poppe KK, Whalley GA, et al. Predicting survival in heart failure: a risk score based on 39 372 patients from 30 studies. Eur Heart J. 2013;34(19):1404–13.
Callender T, Woodward M, Roth G, Farzadfar F, Lemarie JC, Gicquel S, Atherton J, Rahimzadeh S, Ghaziani M, Shaikh M, et al. Heart failure care in low- and middle-income countries: a systematic review and meta-analysis. PLoS Med. 2014;11(8):e1001699.
Conrad N, Judge A, Canoy D, Tran J, O’Donnell J, Nazarzadeh M, Salimi-Khorshidi G, Hobbs FDR, Cleland JG, McMurray JJV, et al. Diagnostic tests, drug prescriptions, and follow-up patterns after incident heart failure: a cohort study of 93,000 UK patients. PLoS Med. 2019;16(5):e1002805.
SUPPORT-HF 2 Investigators and Committees. Home monitoring with IT-supported specialist management versus home monitoring alone in patients with heart failure: design and baseline results of the SUPPORT-HF 2 randomized trial. Am Heart J. 2019;208:55–64.
Rahimi K, Nazarzadeh M, Pinho-Gomes A, Woodward M, Salimini-Korshidi G, Okhuma T, On behalf of SUPPORT-HF2 Study Group, et al. Home monitoring with technology-supported management in chronic heart failure: a randomised trial. Heart. 2020;106:1573–8.
Rahimi K, Woodward M, Korshidi R, Perez A, Fernandez C, Martin M, Mohseni H, Rodriguez R, Allison M, Conrad N. Home monitoring with IT-supported specialist management versus home monitoring alone in patients with heart failure: design and baseline results of the SUPPORT-HF 2 randomized trial. Am Heart J. 2018;208:55–64.
Greenhalgh T, Shaw S, Wherton J, Hughes G, Lynch J, A'Court C, Hinder S, Fahy N, Byrne E, Finlayson A, et al. SCALS: a fourth-generation study of assisted living technologies in their organisational, social, political and policy context. BMJ Open. 2016;6(2):e010208.
Glaser BG. The constant comparative method of qualitative analysis. Soc Probl. 1965;12(4):436–45.
Berg M. Patient care information systems and health care work: a sociotechnical approach. Int J Med Inform. 1999;55(2):87–101.
Greenhalgh T, Wherton J, Sugarhood P, Hinder S, Procter R, Stones R. What matters to older people with assisted living needs? A phenomenological analysis of the use and non-use of telehealth and telecare. Soc Sci Med (1982). 2013;93:86–94.
Cherns A. Principles of sociotechnical design revisted. Hum Relat. 1987;40(3):153–61.
Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, Hinder S, Fahy N, Procter R, Shaw S. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. 2017;19(11):e367.
Mills T, Lawton R, Sheard L. Advancing complexity science in healthcare research: the logic of logic models. BMC Med Res Methodol. 2019;19(1):1–11.
Triantafyllidis A, Velardo C, Chantler T, Shah SA, Paton C, Khorshidi R, Tarassenko L, Rahimi K. A personalised mobile-based home monitoring system for heart failure: the SUPPORT-HF Study. Int J Med Inform. 2015;84(10):743–53.
Rahimi K, Velardo C, Triantafyllidis A, Conrad N, Shah SA, Chantler T, Mohseni H, Stoppani E, Moore F, Paton C, et al. A user-centred home monitoring and self-management system for patients with heart failure: a multicentre cohort study. Eur Heart J Qual Care Clin Outcomes. 2015;1(2):66–71.
Chantler T, Paton C, Velardo C, Triantafyllidis A, Shah SA, Stoppani E, Conrad N, Fitzpatrick R, Tarassenko L, Rahimi K. Creating connections - the development of a mobile-health monitoring system for heart failure: qualitative findings from a usability cohort study. Digit Health. 2016;2:2055207616671461.
Joshi I, Morley J. Artificial intelligence: how to get it right. Putting policy into practice for safe data-driven innovation in health and care. London: NHS X; 2019.
O’Dowd A. Government pins hopes on £250m AI Centre for faster diagnosis and treatment. BMJ. 2019;366:l5106.
Stroetmann KA, Kubitschke L, Robinson S, Stroetmann V, Cullen K, McDaid D. How can telehealth help in the provision of integrated care? Copenhagen: World Health Organization Copenhagen; 2010.
Rojahn K, Laplante S, Sloand J, Main C, Ibrahim A, Wild J, Sturt N, Areteou T, Johnson KI. Remote monitoring of chronic diseases: a landscape assessment of policies in four European countries. PLoS One. 2016;11(5):e0155738.
Di Lenarda A, Casolo G, Gulizia MM, Aspromonte N, Scalvini S, Mortara A, Alunni G, Ricci RP, Mantovan R, Russo G, et al. ANMCO/SIC/SIT Consensus document: the future of telemedicine in heart failure. G Ital Cardiol. 2016;17(6):491–507.
European Commission. eHealth Action Plan 2012-2020 – innovative healthcare for the 21st Century. Communication from the Commission of the European Parliament, The Council, The European Economic and Social Committee, and the Committee of the Regions. Brussels, 6 December 2012. Brussels. http://ec.europa.eu/information_society/newsroom/cf/dae/document.cfm?doc_id=4188 (accessed 7 December 2016): European Commission; 2012.
Saner H, van der Velde E. eHealth in cardiovascular medicine: a clinical update. Eur J Prev Cardiol. 2016;23(2 suppl):5–12.
Gurne O, Conraads V, Missault L, Mullens W, Vachierys JL, Van Mieghem W, Droogne W, Pouleur AC, Troisfontaine P, Huez S, et al. A critical review on telemonitoring in heart failure. Acta Cardiol. 2012;67(4):439–44.
Dierckx R, Pellicori P, Cleland JG, Clark AL. Telemonitoring in heart failure: Big Brother watching over you. Heart Fail Rev. 2015;20(1):107–16.
Anker SD, Koehler F, Abraham WT. Telemedicine and remote management of patients with heart failure. Lancet. 2011;378(9792):731–9.
Sharma U, Clarke M. Nurses’ and community support workers’ experience of telehealth: a longitudinal case study. BMC Health Serv Res. 2014;14:164.
Taylor J, Coates E, Brewster L, Mountain G, Wessels B, Hawley MS. Examining the use of telehealth in community nursing: identifying the factors affecting frontline staff acceptance and telehealth adoption. J Adv Nurs. 2015;71(2):326–37.
Merrell RC, Doarn CR. Barriers or barricades? Telemedicine and e-Health. 2012;18(2):79–80.
Pols J. Care at a distance: on the closeness of technology. Amsterdam: Amsterdam University Press; 2012.
Wade VA, Eliott JA, Hiller JE. Clinician acceptance is the key factor for sustainable telehealth services. Qual Health Res. 2014;24(5):682–94.
Wade V, Eliott J. The role of the champion in telehealth service development: a qualitative analysis. J Telemed Telecare. 2012;18(8):490–2.
National Institute for Health and Clinical Excellence. Chronic heart failure in adults: management. Clinical Guidance 108. London: NICE; 2010.
Ponikowski P, Voors AA, Anker SD, Bueno H, Cleland JG, Coats AJ, Falk V, González-Juanatey JR, Harjola VP, Jankowska EA. 2016 ESC guidelines for the diagnosis and treatment of acute and chronic heart failure: the task force for the diagnosis and treatment of acute and chronic heart failure of the European Society of Cardiology (ESC). Developed with the special contribution of the Heart Failure Association (HFA) of the ESC. Eur J Heart Failure. 2016;18(8):891–975.
We thank the patients and their families who participated in this evaluation, as well as the SUPPORT-HF trial team and NHS organisations at the 7 trial sites. [to add thanks to reviewers].
Funding for this study was from a Senior Investigator Award to TG from the Wellcome Trust in its Society and Ethics Programme (WT104830MA). The funder had no role in the design of the study, the analysis of findings or the writing of this publication.
Ethics approval and consent to participate
Ethical approval was obtained in September 2015 from the Oxfordshire South Central Research Ethics Committee (REC no. 15/SC/0553) and subsequent amendments. Interviews were audio-recorded with participant consent.
Consent for publication
All participants quoted in this study have given written informed consent for publication.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Additional data on the SUPPORT-HF2 trial
The SUPPORT-HF2 study hypothesis was:
… in patients with heart failure, home monitoring with an integrated risk prediction and disease management service, which provided tailored alerts and advice to patients and clinical decision support to healthcare practitioners, is more effective in optimising medical therapy than home monitoring with the same monitoring equipment but without the use of the integrated data analysis and a centralised decision support service to advise general practitioners .
Adult patients with diagnosed heart failure were eligible for the SUPPORT-HF2 trial if they were assessed as having the potential to benefit from home monitoring and management (average self-assessed NYHA classes II to IV in the week before randomisation, or BNP > 100 pmol/L/NT-pro-BNP > 130 pmol/L within 30 days prior to randomisation, or at least two unmet treatment targets) and if they were at high risk of adverse outcomes (estimated one-year mortality risk > 10% or at least one hospital admission related to heart failure in the previous 12 months). Patients with heart failure with reduced ejection fraction (HFREF) and those with heart failure with preserved ejection fraction (HFPEF) were eligible. Optimal management of the former involves the use of up to 4 classes of drugs and in some cases device implantation, with symptomatic and prognostic benefit . Optimal management of the latter hinges around diuretics for symptom control, but to date, there are no therapies with proven prognostic benefit . Exclusion criteria included ‘any significant disease including critical, unstable or end-stage heart failure which, in the opinion of the investigator, may either put the participant at risk because of participation in the trial, or may influence the result of the trial, or the participant’s ability to participate in the trial’ (, p.56).
The CCM team in SUPPORT-HF2 was based in one of the sites (the ‘hub’) and supported 6 additional sites across the UK. As it would have been impossible to fully blind participants and clinicians to treatment options (home monitoring and home monitoring with remote specialist input), the study employed partial blinding, using an ‘attention control’ rather than a ‘usual care control’ arm. This meant participants were instead blinded to the study hypothesis so as to minimise potential ‘loser’ bias. To enable further comparison with usual care in the absence of facilitated self-monitoring, a third trial arm was established from patients who agreed to recruitment but were found ineligible in the run-in phase.
A total of 363 patients were assessed for eligibility, of whom 128 were excluded (96 declined; 22 did not meet criteria; 5 had other reasons; 5 died before screening). A total of 202 patients were randomised to the control or intervention arm. A further 25 of 33 patients who failed the run-in phase agreed to be followed up as part of the third trial arm to approximate ‘usual care’ .
The primary endpoint of SUPPORT-HF2 was used of recommended therapy defined as treatment consistent with the current guidelines for the management of patients with chronic heart failure. Additional predefined primary endpoint was improvement in patients’ physical well-being as assessed by Minnesota Living with Heart Failure Questionnaire and New York Heart Association (NYHA) scales and also by physiological measures including brain natriuretic peptide (BNP) level, heart rate and serum potassium in the ideal range. Secondary endpoints also included clinical safety as assessed by a composite of cardiovascular death, cardiovascular admissions (including renal failure and hypotensive episodes) and unscheduled outpatient visits.
About this article
Cite this article
Papoutsi, C., A’Court, C., Wherton, J. et al. Explaining the mixed findings of a randomised controlled trial of telehealth with centralised remote support for heart failure: multi-site qualitative study using the NASSS framework. Trials 21, 891 (2020). https://doi.org/10.1186/s13063-020-04817-x
- Heart failure
- Complex intervention
- Qualitative study
- Socio-technical theory
- NASSS framework