In this paper we describe a pilot randomised controlled trial of a Telehealth intervention for COPD. The results showed that it would not be feasible to continue the pilot trial to a full RCT. In conducting this research we have identified issues of critical importance for any subsequent study of this complex intervention, including the involvement of clinicians in recruitment and in the research process. The service commissioners’ expectation was that Telehealth-supported services might be clinically and cost-effective for people receiving time-limited support following hospital discharge. Although the pilot trial has not been able to give a robust indication of clinical or cost benefit due to its small sample size and incomplete data collection, it has achieved its objective of demonstrating the practicalities of answering these questions with a full scale RCT in this particular setting.
Compared to studies recruiting participants at multiple sites and working with several clinical teams, it could be presumed that working with one clinical team with high volume referral rates in one community would simplify the research process. The study was conducted within an existing clinical service, thus bringing increased external validity through its pragmatic design  and the ability to directly inform one region’s commissioning choices with regard to Telehealth. However, the process of undertaking this study reinforces findings from larger-scale studies (e.g., Hendy et al. ) regarding the difficulty of evaluating novel, complex interventions such as Telehealth, especially when trying to assess the utility of the intervention within a clinical service alongside undertaking rigorous research.
The study was dependent upon one small clinical team working against a backdrop of NHS restructuring. Even though initial difficulties with obtaining staff ‘buy in’ were identified and appeared to be resolved during the feasibility study, incremental demands created by the research combined with the loss of a key champion for the trial amongst the front line staff had a deleterious effect over time. Further staff attrition resulted in an eventual total loss of 60% of staff capacity within the frontline clinical team. Thus, the main challenge of involving frontline clinicians in the research and in data collection was that, understandably, clinical priorities always came first, and when resources were stretched there was little room for the rigorous research processes which were required for the pilot trial. PCT budgetary restrictions prevented return of the COPD team to its initial capacity. The combined effect of reduced staff capacity and some non-compliance with trial procedures (both staff and participants) resulted in incomplete data collection and slow recruitment to the trial. This also shows that GCP training of clinical staff is not enough to ensure adherence to trial procedures, thus research processes and data collection need to be rigorously monitored throughout the trial.
Despite the successful introduction of a Research Nurse in month 10 to consent participants and collect trial data, organisational changes within the PCT would not have allowed continuation of the pilot trial to a full RCT even if the limitations of the methodology were resolved. One example of an organisational change which impacted the project was the national reorganisation of the NHS, meaning that PCTs were disbanded in March 2013 and replaced by Clinical Commissioning Groups.
The consequences of both care pathways being relatively new within what was a recently introduced service cannot be underestimated. Fidelity to the pathways was difficult to achieve at the outset and was readily compromised. The waiting list for admission to the discharge service, which began to build in month 4 of the pilot trial due to the unanticipated 60% reduction in staff capacity, is a clear example of how the care pathways were changed due to service imperatives which were out of the control of the research team.
There were issues regarding clinician commitment to Telehealth and the work-based support they needed to deliver this new intervention efficiently and appropriately. The importance of training and on-going support to deliver Telehealth is now recognised , but was not available to the clinicians who participated in this study with the consequence that adherence easily eroded and confidence waned. Problems with device connectivity for some of the participants randomised to receive Telehealth despite application of the study’s inclusion/exclusion criteria reinforced the lack of clinician confidence.
Feedback from the feasibility study indicated that those receiving the clinician-delivered ‘standard service’ valued this service and benefitted from it, and that both groups expressed a preference for personalised face-to-face service. The question this poses is whether those receiving Telehealth were disadvantaged in this instance compared with those in receipt of a specialist face-to-face service, which was highly tailored to patient need but unlikely to be sustainable in the longer term. This preference for face-to-face care may be more influential than the perceived reassurance of daily monitoring provided through using Telehealth. Additionally, questions were raised by the study regarding which patients might gain most benefit from Telehealth. Study participants were recruited at the point of hospital discharge and it became evident that receptiveness to using Telehealth could be severely compromised by their illness. It is possible that asking a patient to use a piece of technology at this point, even one which is simple to use, could be a daunting commitment to take on in addition to recovering from their exacerbation. There are unanswered questions regarding when in the overall care pathway from acute to primary care can Telehealth technology be most effective and for how long, taking into account exacerbation severity. There are also questions around the representativeness of the sample, and when in the COPD disease pathway deployment of Telehealth would be of most benefit. Forty percent of patients referred to the COPD service failed to qualify for the trial on one or more of the eligibility criteria, indicating that trial participants may not have been fully representative of the COPD population as a whole and that our trial design may not have been as ‘pragmatic’ as had been intended. The definition of between one and three previous hospital admissions, which formed a key component of the eligibility criteria, was based on the local definition of early stage COPD. However, it could be argued that a person with three admissions in the previous 12 months may have more advanced disease compared to someone with one or no admissions. To summarise, commissioners and clinicians require greater guidance for deployment.
By definition, complex interventions are difficult to define, standardise, and measure, and enthusiasm to undertake research can underestimate this. The results obtained from this study question the viability of involving front line clinical staff in data collection for robust research evaluation. Whilst it is true that clinicians do not need to be involved in data collection, the observations of Bird et al.  provide further support regarding the importance of context and culture when conducting trials, yet this is not evidenced through on-going commissioning of trials of complex interventions.
Questions remain regarding how to most appropriately conduct local evaluations to inform commissioning decisions. Leykum et al.  suggest integration of participatory action research and randomised controlled trial methods to ensure that a complex intervention is adequately embedded within the setting. Arguably, this did occur within this study through the extensive involvement of commissioners and clinicians in study set up, design, and progress, and through regular meetings during the entire programme. As demonstrated by Hendy et al. , the drive to demonstrate population-based benefit through evaluation of complex interventions does not necessarily equate with the demands of implementing a complex intervention in situ.