Overview of The Healthlines Study
The Healthlines Study comprised two linked, parallel randomized controlled trials of patients with depression and raised risk of cardiovascular disease who were allocated to a telehealth intervention plus usual care or usual care alone [1]. The multicentre trials were conducted in and around Bristol, Sheffield and Southampton, UK. Delivered by NHS Direct Health Information Advisors, the intervention included telephone-based support and advice, the use of online tools and resources (e.g., computerized cognitive behavioural therapy), and motivational interviewing. In the depression trial, intervention participants received regular telephone coaching approximately every two weeks for a maximum of eight telephone sessions. The majority of telephone contact occurred in the first four months, but participants could remain in the trial for up to 12 months. The trials were approved by the National Research Ethics Service Committee South West – Frenchay (Reference 12/SW/0009), participants gave written informed consent, and the depression trial is registered with Current Controlled Trials (ISRCTN14172341; registered, 26 June 2012). Since the current interventions involve the depression group alone, we focus solely on this trial in this paper.
Recruitment to the depression trial of The Healthlines Study
Using an anonymized, code-based record search, participants were recruited to the Healthlines depression trial from 43 general practices between June 2012 and July 2013. Basic inclusion criteria were age ≥18 years, telephone and internet access, researcher-confirmed diagnosis of depression using the Clinical Interview Schedule – Revised [21], and PHQ-9 score ≥10 [2]. A number of exclusion criteria were applied, such as having a severe cognitive impairment, already receiving formal psychological therapy and substance dependency. The study invitation pack was posted by GP practice staff, with expressions of interest returned to the research team directly. After eligibility was confirmed over the telephone by researchers, interested participants completed a consent form and baseline questionnaire either online or via a postal copy, as they preferred. In total, 609 participants (307 intervention, 302 usual care) were randomized to the depression trial, of which 318, 233 and 58 were from the areas around Bristol, Sheffield and Southampton, respectively.
Standard follow-up procedures in the host trial
Participants in the depression trial were followed up with a questionnaire at three time points: 4, 8, and 12 months after randomization. All follow-up questionnaires included the same questions (e.g., depression (PHQ-9), anxiety, treatment satisfaction, health service use), and took approximately 40 minutes to complete. A few days before the 4-month follow-up was due, participants were automatically sent the questionnaire in the same format (online or postal) that they had chosen to complete at baseline.
For those who did not respond promptly, a standard procedure was followed in terms of sending out reminders for all follow-ups (see Fig. 1). This involved the following sequence of steps: sending an email reminder; phoning the participant; posting a questionnaire (a second copy was posted for those who were completing the paper version); posting just the primary outcome questions (the PHQ-9); finally, phoning participants to ask them to post back the PHQ-9 or offering to complete it over the phone. After this final phone reminder was completed, no further attempts were made to collect data for that follow-up time point. Note that in order to maximize the likelihood that the reminder email would be opened by the participant, the reminder email subject line (‘The Healthlines Study – Questionnaire reminder’) included both the study name and the purpose of the email. This helped to ensure that the reminder message was received by participants, even if they failed to open the email itself. Although the order of reminders was the same for all participants and follow-ups, the timing of the reminders varied slightly by completion method (see Fig. 1).
Overview of the embedded interventions
The following three studies are individually embedded within the Healthlines depression host trial, and separately compare an introduced response rate intervention to the standard follow-up questionnaire protocol method just described. Each of these studies seeks to investigate the impact on follow-up questionnaire response rates through pre-calling participants or not (Study 1), personalizing study materials by including a colour photograph of the research team within the questionnaire cover letter or email versus not including a photo (Study 2), and using an urgent action subject line (‘ACTION REQUIRED’) in the initial email reminder versus one that simply states it is a reminder (‘Questionnaire reminder’) (Study 3) (see Additional file 1).
Study 1 occurred at the 8-month follow-up time point. Based on an early and informal indication that there was a benefit from pre-calling participants (this group seemed to require fewer reminders), this procedure was adopted uniformly for all final, 12-month follow-ups. However, because participants were recruited over a period of several months, some participants were sent 12-month follow-up questionnaires before others had received their 8-month follow-up questionnaires, and so this decision was made before any analyses had been performed. Therefore, Studies 2 and 3 include the pre-call intervention from Study 1 as an adopted standard procedure, but examine the possibility of further boosting the 12-month follow-up response rate with two separate interventions in different study centres (see Additional file 2 for CONSORT checklist).
Study 1: advance notification through pre-calling study
The pre-calling intervention was devised in response to the lower than expected 8-month follow-up questionnaire completion rate in the first recruiting study centre (Bristol), and so was introduced six months after the first 8-month follow-up questionnaire was sent out. In ascending randomization date order, the remaining participants requiring the 8-month follow-up between September 2013 and March 2014 were alternately allocated to one of theo groups; they either received an advance notification telephone call (i.e., a pre-call) from a researcher one to three days ahead of being sent the questionnaire (intervention group) or they were simply emailed or posted the questionnaire without a telephone call (control group). The decision to email or post the questionnaire to control participants was based on how they had completed the questionnaire previously. During the intervention pre-call, participants were informed that the next study questionnaire was due, were asked whether they would like to complete it online or via post, and were encouraged to return the questionnaire within 10 days, and their contact details were verified (see Additional file 1).
Participants were often difficult to contact by telephone. Multiple attempts were made to contact intervention participants at different times of day over the course of the days leading up to when the 8-month follow-up was due. If direct telephone contact could not be made ahead of this due date, the questionnaire was sent to the participant based on their previous completion method (online or postal). In these instances, the research team left a voicemail message (or emailed or texted the same information, if voicemail was not possible) which communicated the same information that would have been relayed had direct communication been achieved, along with the researcher’s contact details.
All other procedures were identical for intervention and control participants. Questionnaire reminders, when required, followed the standard procedure outlined earlier (see Fig. 1).
Study 2: research team photo study
Of the participants with depression from the Bristol study centre who were due to complete the 12-month follow-up questionnaire between October 2013 and July 2014, half were randomized to receive a cover letter with a colour photo of the Bristol research team (intervention group) and half were randomized to receive the standard, black and white cover letter without a photo (control group) (see Additional file 1). In this embedded parallel group randomized controlled trial, the local researcher used simple randomization (1:1) by computing a random number in Excel for each participant, sorting the data by random number, and then allocating the first half of participants to the intervention group and the remaining half to the control group.
Participants approaching the 12-month follow-up received an advance notification telephone call from the local researcher one to three days ahead of the due date for this questionnaire. The phone conversation was similar to Study 1, except that participants were informed that they would receive an email thank you note with more details about the study upon completing the final questionnaire. If the researcher was unable to contact the participant, the same procedure as in Study 1 was followed.
We reasoned that a photo by itself was probably not very informative, but that if participants were able to link the name of the particular research team member or members that they had spoken to over the course of the study to the picture, this would increase the effect of the photo. Therefore, underneath the photo, the names of each of the five Bristol research team members were listed. To ensure maximum visibility, the photo was located in the upper right corner of the cover letter that was posted to each participant with either their questionnaire (paper-based) or with instructions around completing the online questionnaire. For participants who had opted to complete the online questionnaire, the same photo was also included at the end of the email that was sent with instructions on accessing the online survey (some email providers might automatically include the photo as an attachment, however). Since we wanted to ensure that the survey weblink and access details were not missed and the email was not deleted on seeing a photo of people the participant would not recognise, we placed the photo at the end of the email, close to the survey weblink. Finally, the same photo was incorporated into each reminder letter or email sent to intervention participants (Fig. 1).
Participants in the control group received the same study information as the intervention group, but without a photo. This was the standard cover letter or email that had been sent to all participants at the previous follow-up time points. As required, the reminder procedure as outlined in Fig. 1 was followed.
Study 3: action-oriented email reminder subject line study
In this embedded parallel group randomized controlled trial, approximately half of the participants with depression from the Sheffield and Southampton study sites who were due to complete the 12-month follow-up questionnaire between November 2013 and July 2014 were randomized 1:1 to receive either the intervention reminder email subject line (‘The Healthlines Study – ACTION REQUIRED’) or the standard reminder email subject line (‘The Healthlines Study – Questionnaire reminder’) (see Additional file 1). Study 3 used the same randomization procedure as Study 2.
As before, participants first received the advance notification call as described in Study 2, and were sent the questionnaire via their preferred completion method (online or postal). In line with the usual reminder protocol, if the completed questionnaire had not been returned within 5–7 days for online completers or 10 days for postal completers, participants were sent a reminder email from the local researcher using the randomly allocated subject line. Other than the email subject line intervention, the content of the reminder email was identical for both groups and the same as for previous follow-ups.
Since some participants at earlier follow-up time points said that they had overlooked the email in their inbox, the manipulated subject line included the new, action-oriented words in capitals, in order to increase the visual salience of the email. We expected that the use of directive, action-oriented language would increase the participant’s perception that reading the email and returning the completed questionnaire was of greater importance than the standard email reminders that they might have received with previous follow-ups. Given that the email subject line had changed for these participants, its novelty and distinctiveness should also be attention grabbing.
Sample size
Since these were embedded studies, in which there were practical constraints around the timing of implementing the interventions in different trial centres, we did not derive a formal sample size calculation a priori. Instead, sample size was determined by these constraints.
Primary and secondary outcomes
The primary outcome for the embedded studies was completion of the primary outcome (PHQ-9) questionnaire within 3 months. Secondary outcome measures included the total number of questionnaire reminders required (range: 0–5), as well as the time (in days) taken to complete and return the questionnaire, calculated as the difference between the date it was sent and date it was received.
Statistical analysis
First, appropriate descriptive statistics (mean, standard deviation (SD), frequencies) were used to summarize the baseline characteristics of participants. In each of the embedded studies, the primary intention-to-treat analysis investigated whether the particular response rate intervention had an effect on questionnaire completion. This was examined with unadjusted and adjusted logistic regression models (with participant sex, baseline age and PHQ-9 score, main Healthlines Study trial allocation arm and questionnaire method sent (paper or online) as covariates in the adjusted models). Since not all randomized participants received the response rate intervention in the photo and email reminder studies (see Fig. 2), either because they opted to complete the PHQ-9 questionnaire over the phone during the pre-call or because they responded before the first questionnaire reminder was due, sensitivity analyses were carried out on the primary outcome after excluding these participants. Next, secondary outcomes were analyzed using linear (number of questionnaire reminders received) and Cox regressions (days to questionnaire completion). Again, both unadjusted and adjusted models were tested, and sensitivity analyses were conducted where appropriate. Analyses were carried out using Stata13.1 (StataCorp).