Skip to main content

Assessing loss to follow-up in the MObile Technology for Improved Family Planning (MOTIF) randomised controlled trial

Abstract

Background

Loss to follow-up (LTFU) in clinical trials is an important source of bias that can affect statistical power and generalisability of findings. The aim of this paper is to assess factors associated with LTFU in the MObile Technology for Improved Family Planning (MOTIF) trial in Cambodia and compare how the result might have varied using different analytical methods.

Methods

Follow-up in the MOTIF trial was 86% at 4 months and 66% at 12 months. For the primary analysis, we undertook a complete case analysis, similar to the approach used in similar trials of interventions delivered by mobile phone to increase contraception use. We conducted an exploratory analysis and found that factors associated with LTFU were young age, lower socio-economic status, not planning to use post-abortion contraception, availability of phone credit and not providing additional contact numbers. We then undertook two analyses to estimate the effect of the intervention on the primary outcome at 4 and 12 months for comparison with the complete case analysis. First, we undertook multiple imputation, and second we conducted an analysis treating all participants’ LTFU as non-users of contraception.

Results

Using multiple imputation, we found that the risk ratio was slightly increased at 4 months and slightly decreased at 12 months compared with the complete case analysis. When counting all participants’ LTFU as non-users of contraception, we observed that, compared with the complete case analysis, the risk ratio was slightly decreased at 4 months and slightly increased at 12 months. Despite the changes in the risk ratio, use of the different analytical methods did not result in an effect using the complete case analysis becoming statistically significant or vice versa.

Conclusion

Future studies assessing contraception use might anticipate increased attrition amongst younger participants, those of lower socio-economic status or those who do not provide additional contact details. Attrition could be reduced by collecting as many contact details as possible, by providing incentives and possibly by enhanced counselling to groups at higher risk of LTFU on recruitment. Multiple imputation should be considered in addition to complete case analysis if LTFU not missing at random is expected or observed.

Trial registration

ClinicalTrials.gov, NCT01823861. Registered on 30 March 2013.

Peer Review reports

Background

Loss to follow-up (LTFU) in clinical trials is an important source of bias that can affect statistical power and generalisability of findings, and trials with large LTFU can be downgraded in systematic reviews. Systemic differences in LTFU between groups can result in attrition bias [1].

Successful follow-up requires participants to be found and also willing to participate with the data collection procedure [2]. LTFU can be classified as missing at random (MAR) or not missing at random (NMAR), such as increased attrition in younger people. In this situation a complete case analysis, restricted to participants with available follow-up data, may produce a biased estimation of the true effect. A common approach to dealing with missing outcome data is to impute outcomes and treat them as if they were real measurements, known as multiple imputation [3]. The aim of this paper is to assess factors associated with LTFU in the MObile Technology for Improved Family Planning (MOTIF) trial in Cambodia and compare how the result might have varied using different analysis methods.

Main text

For the MOTIF trial, the primary outcome—self-reported use of an effective contraceptive—was assessed by attempting to contact participants by phone at 4 and 12 months. We used evidence-based methods to try to reduce the chance of LTFU, such as collecting as many possible phone numbers on recruitment, including those of friends or family members, with consent of the participant [4, 5]. At least three attempts were made to contact participants by phone. If it was not a convenient time for the participant to talk, another time would be arranged. At the time of the trial, few participants used email, there was not a functioning postal system in rural areas in Cambodia, and we had limited resources with which to visit participants’ homes. At 4 months we obtained primary outcome data from 431 participants (86%). Amongst the 69 clients LTFU, 6 withdrew, and we were unable to contact 63 participants. At 12 months we obtained primary outcome data from 328 participants (66%). Amongst the 172 clients LTFU, 8 withdrew, and research assistants were unable to contact 164 participants.

For the primary analysis we undertook a complete case analysis, similar to the approach used in similar trials of interventions delivered by mobile phone to increase contraception use [6, 7]. Using a complete case analysis, we found that the intervention increased self-reported use of an effective contraception method at 4 months post-abortion (64% vs. 46%; risk ratio [RR] 1.39, 95% CI 1.17–1.66; p < 0.001) but not after 12 months (50% vs. 43%; RR 1.16, 95% CI 0.92–1.47; p = 0.208).

Investigating missing data

Reasons, as documented by the research assistants, for LTFU at 12 months amongst these 164 participants were as follows: 76 (46%) times the phone was switched off; 57 (35%) times there was an automated response that the number was not in use and not yet re-assigned; 16 (10%) times someone else answered the phone; in 10 (6%) cases where the participant had moved abroad; in 3 (2%) cases an automated response indicated that the phone could not receive incoming calls; and in 2 (1%) cases the participant did not want to talk.

We conducted an exploratory analysis to identify predictors of LTFU from the variables collected at trial recruitment to assess whether data were MAR or NMAR. To maximise statistical power, binary variables were created from some of the categorical variables. First we undertook univariable analysis to examine the crude association between each baseline variable and LTFU, expressed as an OR with a 95% CI. In the univariable analysis, age < 25 years, lower socio-economic status (SES; i.e., no access to motorised transport), not planning to use post-abortion contraception and not providing an additional friend/family contact number were associated with LTFU at 4 months (Table 1). Age < 25 years, lower SES and usually or sometimes having phone credit were associated with LTFU at 12 months (Table 2). There was no evidence of differential LTFU between those assigned to the intervention or control arm at 4 or 12 months.

Table 1 Predictors of missing data at 4 months
Table 2 Predictors of missing data at 12 months

We then undertook multivariable analysis to explore the association between variables and LTFU, controlling for confounding, using logistic regression analysis. Age and SES are associated with contraception use and may be associated with seeking LTFU [8, 9]. We therefore considered these variables as confounders in the analysis. In the multivariable analysis, age < 25 years, lower SES, not planning to use post-abortion contraception and not providing an additional friend/family contact number remained as predictors of LTFU at 4 months; LTFU was significantly increased amongst women aged < 25 years compared with those aged > 25 years (OR 2.31; p = 0.002), amongst women without access to motorised transport compared with women with access to motorised transport (adjusted OR 2.44; p = 0.008), amongst women not planning to use post-abortion contraception compared with women planning to use it (OR 2.34; p = 0.05), and amongst women who did not provide additional contact numbers compared with those who did (OR 2.40; p = 0.002) (Table 1).

At 12 months, age < 25 years, lower SES, and phone credit remained as predictors of LTFU; LTFU was significantly increased amongst women aged < 25 years compared with those aged > 25 years (OR 1.59; p = 0.023), amongst women without access to motorised transport compared with women with access to motorised transport (OR 2.04; p = 0.009), and amongst women who usually (OR 1.85; p = 0.014) or sometimes (OR 1.59; p = 0.046) had phone credit compared with women who always had phone credit (Table 2).

We then undertook two different analyses to compare with the complete case analysis (Table 3). First, we undertook multiple imputation to estimate the effect of the intervention on the primary outcome at 4 and 12 months. This involved replacing each missing primary outcome value with a plausible imputation based on the characteristics of missing values observed [10]. Because LTFU was NMAR, variables associated with LTFU were used in the multiple imputation model. Thus, logistic regression with the predictor variables age group, SES, post-abortion family planning intentions and whether an additional friend or family contact was provided was used for univariate imputation of the outcome at 4 months and the predictor variables age group, SES and phone credit for imputation of the outcome at 12 months. Multiple imputation was conducted separately for the outcomes at 4 and 12 months with 100 imputed datasets created for each outcome, respectively. Using multiple imputation, the intervention also showed an effect at 4 months (63% vs. 45%; RR 1.42, 95% CI 1.19–1.69; p = 0.001) but not at 12 months (48% vs. 43%; RR 1.12, 95% CI 0.89–1.40; p = 0.349). Second, we conducted sensitivity analyses treating all participants LTFU as non-users of contraception (extreme case scenario) to estimate the effect of the intervention on the primary outcome at 4 and 12 months [11]. In this analysis self-reported effective contraceptive use also remained significantly increased when we considered participants LTFU as non-users at 4 months (54% vs. 40%; RR 1.35, 95% CI 1.12–1.63; p = 0.002) but not at 12 months (34% vs. 27%; RR 1.25, 95% CI 0.95–1.63; p = 0.106).

Table 3 Effect measure variation according to analytical method used

Table 3 shows how the effect measure varies according to the analytical method used. Using multiple imputation, we found that the RR was slightly increased at 4 months and slightly decreased at 12 months compared with the complete case analysis. When counting all participants LTFU as non-users of contraception, we observed that, compared with the complete case analysis, the RR was slightly decreased at 4 months and slightly increased at 12 months. Despite the changes in the RR, use of the different analytical methods did not result in an effect becoming statistically significant or vice versa.

Conclusion

Factors associated with LTFU in the MOTIF trial were young age, lower SES, not planning to use post-abortion contraception, phone credit and not providing additional contact numbers. Reasons for LTFU amongst younger women might be that they changed phone numbers (could not be found) or perhaps the potential stigma of contraceptive use in young people (no longer wanting to participate in the trial). Younger participants were also found to be at increased risk of LTFU in a head injury trial [2]. LTFU amongst women of lower SES might be due to changing phone numbers or no longer having access to a phone. Women not planning to use contraception might have been less motivated to participate in the trial follow-up. The finding that LTFU was increased in those not providing multiple contacts is consistent with existing evidence [4]. Although the RR showed slight variation using multiple imputation and counting those LTFU as non-users, there was no significant change in the principal findings.

Whilst this study provides some insights into factors associated with LTFU in this trial, it may have limited generalisability to other studies. This study had a relatively small sample size and was an exploratory rather than a pre-specified analysis.

Future studies assessing contraception use might anticipate increased attrition amongst younger participants and those of lower SES or who do not provide additional contact details. Attrition could be reduced by collecting as many contact details as possible, providing incentives, and possibly providing enhanced counselling to groups at higher risk of LTFU on recruitment [4]. Multiple imputation should be considered in addition to complete case analysis if LTFU NMAR is expected or observed.

Abbreviations

LTFU:

Loss to follow-up

MAR:

Missing at random

MOTIF:

MObile Technology for Improved Family Planning

NMAR:

Not missing at random

PAFP:

Post-abortion family planning

RR:

Risk Ratio

SES:

Socio-economic status

References

  1. Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions. Version 5.1.0 [updated March 2011]. London: The Cochrane Collaboration. http://handbook-5-1.cochrane.org/.

  2. Edwards P, Fernandes J, Roberts I, Kuppermann N. Young men were at risk of becoming lost to follow-up in a cohort of head-injured adults. J Clin Epidemiol. 2007;60(4):417–24.

    Article  PubMed  Google Scholar 

  3. Carpenter J, Bartlett J. Missing data. London: London School of Hygiene & Tropical Medicine; 2016. http://www.missingdata.org.uk/. Accessed 1 July 2017.

    Google Scholar 

  4. Edwards PJ, Roberts I, Clarke MJ, DiGuiseppi C, Wentz R, Kwan I, et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev. 2009;3:MR000008.

    Google Scholar 

  5. Smith C, Vannak U, Sokhey L, Ngo TD, Gold J, Khut K, et al. MObile Technology for Improved Family Planning services (MOTIF): study protocol for a randomised controlled trial. Trials. 2013;14:427.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Hou M, Hurwitz S, Kavanagh E, Fortin J, Goldberg A. Using daily text-message reminders to improve adherence with oral contraceptives: a randomized controlled trial. Obstet Gynecol. 2010;116(3):633–40.

    Article  PubMed  Google Scholar 

  7. Castaño PM, Bynum JY, Andrés R, Lara M, Westhoff C. Effect of daily text messages on oral contraceptive continuation: a randomized controlled trial. Obstet Gynecol. 2012;119(1):14–20.

    Article  PubMed  Google Scholar 

  8. National Institute of Statistics, Directorate General for Health, and ICF International. Cambodia: demographic and health survey 2014. Phnom Penh, Cambodia, and Rockville, MD: Author; 2015. https://dhsprogram.com/pubs/pdf/FR312/FR312.pdf. Accessed 1 July 2017.

  9. Westoff CF. Unmet need for modern contraceptive methods. DHS Analytical Studies 28. Washington, DC: U.S. Agency for International Development; 2012.

    Google Scholar 

  10. Kenward M, Carpenter J. Multiple imputation: current perspectives. Stat Methods Med Res. 2007;16(3):199–218.

    Article  PubMed  Google Scholar 

  11. Smith C, Ngo TD, Edwards P, Free C. MObile Technology for Improved Family Planning: update to randomised controlled trial protocol. Trials. 2014;15:440.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank all clients and clinic staff who participated in the study.

Funding

The Marie Stopes International Innovation Fund funded the study for 15 months from October 2012 and had some influence on the study design but not on data collection or analysis. The UK Medical Research Council (MRC) funded the data analysis but had no influence on the analysis or reporting. CS is supported by an MRC Population Scientist Fellowship (grant reference MR/L012251/1).

Availability of data and materials

Data are from the MOTIF study, whose authors may be contacted at the London School of Hygiene and Tropical Medicine, Keppel Street, London, WC1E 7HT, UK (email: christopher.smith@lshtm.ac.uk).

Author information

Authors and Affiliations

Authors

Contributions

CS and CF designed the intervention and the trial. CS drafted the manuscript with input from CF and CJ. CJ advised on the statistical analysis and conducted the multiple imputation. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Chris Smith.

Ethics declarations

Ethics approval and consent to participate

Ethical approval for the MOTIF study was obtained from ethics committees at the London School of Hygiene and Tropical Medicine (reference 6378) and Marie Stopes International (reference 002-13-E) as well as from the Cambodia Human Research Ethics Committee (reference 0036 NECHR). Participants were provided with an information sheet to read, or it was read to them, and provided signed or thumb-printed consent or recorded verbal consent.

Consent for publication

As part of the consent form, participants were asked to agree with the statement, ‘I allow you to write about what I have said during our talk, and I understand that you won’t be using my real name’.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Smith, C., Jarvis, C. & Free, C. Assessing loss to follow-up in the MObile Technology for Improved Family Planning (MOTIF) randomised controlled trial. Trials 18, 577 (2017). https://doi.org/10.1186/s13063-017-2316-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13063-017-2316-6

Keywords