Aim, design and setting of the SWAT
The main objective of this Study Within A Trial (SWAT; for the definition of a SWAT visit https://www.nihr.ac.uk/documents/studies-within-a-trial-swat/21512) was to establish whether, in parents of preterm babies, offering an unconditional incentive in advance (with the first mailing of a questionnaire) or promising a conditional incentive (in the first mailing) on completion of a questionnaire, improves response rate. This randomised controlled SWAT was nested within the Speed of Increasing milk Feeds Trial (SIFT), a multicentre RCT in neonatal units in the UK and Ireland caring for very preterm or very low birthweight (VLBW) infants. SIFT recruited 2804 infants, randomised 1:1 to either a slower (18 ml/kg) or faster (30 ml/kg) daily increase in milk volume until reaching full feeds, between 8 June 2013 and 30 June 2015. The primary outcome was survival without moderate or severe disability at 2 years of age (corrected for prematurity), assessed by a questionnaire sent directly to parents (principally mothers). Questionnaires were sent (to all parents where infant(s) vital status and address could be confirmed) by both post and as a link to an online submission form via email and text message where these contact details were available. The SWAT investigated whether an unconditional incentive (a monetary voucher given before completion of a questionnaire) was more effective than a conditional incentive (the promise of a monetary voucher on receipt of a completed questionnaire) and has been reported elsewhere . For ‘Before’ and ‘After’ letters and reminders see Additional file 1.
Eligible participants were parents of infants recruited to SIFT who were due to be sent a questionnaire at 2 years of age (corrected for prematurity) at the SWAT start date (1 Feb 2017). SIFT recruited in 55 centres in the UK and Republic of Ireland. Eligibility criteria for inclusion in SIFT have been described elsewhere . Participants were traced for survival and to confirm current residence. Where these details remained unknown, parents were not contacted. In addition, parents who had withdrawn consent to the 2-year follow-up were excluded.
Interventions and comparisons
Participants were allocated randomly to one of two groups:
A (After): the first letter posted to parents included a promise of an incentive (£15 gift voucher redeemable at high street shops) after receipt of a completed form.
B (Before): the first letter posted to parents enclosed the incentive (£15 gift voucher redeemable at high street shops) before the receipt of a completed form.
Participants were randomised with an allocation ratio 1:1 by permuted block randomisation (using variable block sizes of 2, 4, 6 and 8 allocated in proportion to elements of Pascal’s triangle i.e., a ratio of 1:3:3:1) and stratified by original SIFT allocation (faster/slower) and by singleton/multiple birth by the Trial Statistician at the NPEU CTU. The interventions were assigned to the participants immediately because they had all been enrolled to SIFT at that point, so there was no gap during which the sequence needed to be concealed. Infants from multiple births were allocated to the same incentive group. Vouchers were allocated per questionnaire, so parents of multiple births received a voucher for each infant. SIFT office staff at the NPEU Clinical Trials Unit were aware of allocation due to the nature of the interventions and the practicalities involved in sending out the letters and the vouchers.
The incentive was a £15 high street shop voucher (€15 for participants recruited in the Republic of Ireland), sent via post. Reminder letters in both groups mentioned the incentives. Letters to those in the group allocated ‘After’ reiterated the promise of an incentive; letters to those in the group allocated ‘Before’ tactfully mentioned the incentive sent with the first letter. Parents were also reminded to complete the questionnaire via text and/or email during the follow-up window. All parents were offered the option of completing the questionnaire online or, as a last resort, via telephone.
The incentives SWAT was implemented midway through the SIFT follow-up. Parents of SIFT participants who had returned their follow-up questionnaire prior to the incentives study being implemented were sent a £15 voucher in a spirit of fairness.
The primary outcome was the occurrence of questionnaire return, defined as receipt of a completed or partially completed questionnaire at the SIFT office. A questionnaire was considered completed or partially completed if the first three out of the five subsections of the questionnaire were completed (as these sections were required for the derivation of the primary outcome for the main SIFT trial).
Secondary outcomes included:
Total cost included postage, receipt of material via prepaid Freepost envelope, cost of envelopes, supplementary materials (for example, sticker sets sent with questionnaires for infants to play with), and value of gift vouchers. It did not include Freepost licence fee, printing, telephone calls, or trial staff time. All costs for participants were calculated in GBP. The cost of the €15 vouchers sent to participants in the Republic of Ireland was converted to GBP using the exchange rate (via xe.com) on 10 May 2017, the date of the invoice for these vouchers. Costs for these participants also included Air Mail postage fees.
SIFT completed recruitment on 30 June 2015, with 2804 infants randomised. Given an expected 5% mortality rate in the patient population, it was estimated that 1250 of these infants would survive until the projected start date of the SWAT (originally 1 December 2016). It was also estimated that about 10% of the parents of these infants would be lost to the follow-up phase, due to withdrawal of consent or to lack of information on contact details or infant survival status (always checked prior to engaging with parents). This would result in approximately 1100 infants eligible for the SWAT, giving 550 per group.
It was estimated that the response rate with no incentives would be approximately 66%, based on experience from the BOOST-II UK trial with a similar patient population and 2-year follow-up methods .
Based on the most applicable literature on studies investigating incentives, it was anticipated that the addition of an incentive would result in an absolute increase of 10% in the response rate [4, 8]. Five hundred and fifty infants per group would allow detection of an absolute difference in response rate of around 7% at 90% power and a two-sided 5% level of significance.
Infants (i.e. parent responses) were analysed in the groups to which they were randomly assigned, comparing the outcome of all infants allocated to ‘Before’ with all those allocated to the ‘After’ group, regardless of deviation from the protocol.
Baseline demographic information was summarised by randomised group using frequency counts and percentages for categorical data, means and standard deviations for normally distributed continuous data, or medians with interquartile ranges for other continuous data.
Comparative analysis entailed calculating the absolute difference in the proportion responding with corresponding 95% confidence interval (CI), and the difference in mean cost (plus 95% CI). In addition, the cost in £s per 1% increase in response rate was calculated, factoring in administration costs such as the number of reminder letters, as well as the monetary value of the incentive. For other outcomes relating to the method of completion and reminder letters, a similar strategy was used based on the distributions/type of data collected.
The principal comparison was the incentivised ‘Before’ group versus the incentivised ‘After’.
Pre-specified subgroup analysis examined the consistency of the effect of the timing of the incentive for the original allocation in SIFT (slower versus faster) and singleton versus multiple births, using the statistical test of interaction. An exploratory subgroup analysis exploring regional (England, Scotland, Wales, Northern Ireland, and Republic of Ireland) variation was also performed.
Pre-specified exploratory analysis examined the response rate in the period prior to the incentives study starting and during, overall (i.e. irrespective of incentive group allocation) with a 95% confidence interval.
No adjustment was planned for multiple testing since this SWAT involves a very small number of focused hypothesis tests.