Skip to content

Advertisement

  • Review
  • Open Access
  • Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Post-trial follow-up methodology in large randomised controlled trials: a systematic review

Trials201819:298

https://doi.org/10.1186/s13063-018-2653-0

Received: 30 December 2017

Accepted: 13 April 2018

Published: 30 May 2018

Abstract

Background

Randomised controlled clinical trials typically have a relatively brief in-trial follow-up period which can underestimate safety signals and fail to detect long-term hazards, which may take years to appear. Extended follow-up after the scheduled closure of the trial allows detection of both persistent or enhanced beneficial effects following cessation of study treatment (i.e. a legacy effect) and the emergence of possible adverse effects (e.g. development of cancer).

Methods

A systematic review was conducted following PRISMA guidelines to qualitatively compare post-trial follow-up methods used in large randomised controlled trials. Five bibliographic databases, including Medline and the Cochrane Library, and one trial registry were searched. All large randomised controlled trials (more than 1000 adult participants) published from March 2006 to April 2017 were evaluated. Two reviewers screened and extracted data attaining > 95% concordance of papers checked. Assessment of bias in the trials was evaluated using the Cochrane Risk of Bias tool.

Results

Fifty-seven thousand three hundred and fifty-two papers were identified and 65 trials which had post-trial follow-up (PTFU) were included in the analysis. The majority of trials used more than one type of follow-up. There was no evidence of an association between the retention rates of participants in the PTFU period and the type of follow-up used. Costs of PTFU varied widely with data linkage being the most economical. It was not possible to assess associations between risk of bias during the in-trial period and proportions lost to follow-up during the PTFU period.

Discussion

Data captured during the post-trial follow-up period can add scientific value to a trial. However, there are logistical and financial barriers to overcome. Where available, data linkage via electronic registries and records is a cost-effective method which can provide data on a range of endpoints.

Systematic review registration

Not applicable for PROSPERO registration.

Keywords

MethodologyPost-trialRetentionRandomised controlled trialCostLong-termFollow-upEffective

Background

Randomised controlled trials (RCTs) are considered to be the ‘gold standard‘ for assessing the effects of a treatment. However, these trials usually report results following a relatively brief exposure to the intervention under investigation. Longer-term follow-up of trial participants is important as persistent effects may be detected years later after treatment cessation or even enhanced benefits observed decades later – a so-called ‘legacy effect‘ [1, 2]. Furthermore, delayed hazards may only emerge several years after exposure to certain treatments. Therefore, PTFU may add significant scientific value to the evaluation of many healthcare interventions.

We define post-trial follow-up (PTFU) as extended follow-up which starts after the end of the scheduled period of the trial. Such follow-up, regardless of the primary in-trial outcome, provides important information including safety of the intervention, identification of delayed hazards and long-term beneficial effects.

Retention of participants in PTFU is important since high rates of attrition may introduce bias if reasons for withdrawal are related to the intervention [3]. There are a variety of methods for PTFU, but little research has been done to evaluate which methods for PTFU leads to the best retention rates [4]. Choice of follow-up method is often determined by the funding for the trial and the local availability of relevant data. Telephone calls, postal questionnaires and face-to face interviews are the more traditional approach to follow-up. Web-based approaches and use of routine health records and electronic registries are becoming more popular due to advancing technology and options for accessing the information inexpensively [5, 6].

This systematic review compares methods used in approaches to PTFU and aims to inform the design of PTFU for a wide range of randomised trials. The main objective was to evaluate the retention rates (or levels of attrition) of the participants followed up during PTFU and to compare this to the type of methodology used. A secondary objective was to compare the costs of post-trial methodology as funding is often limited.

Methods

The methods used in this systematic review have been described in detail previously [7] and follow PRISMA guidelines Additional file 1.

Eligibility criteria

Briefly, all large (> 1000 adult participants) RCTs which investigated a healthcare intervention (i.e. medicine, surgery or psychiatric in nature) and involved PTFU were included. Only studies published between 2006 and 2017 were included. Alternative medicines (e.g. acupuncture) or holistic interventions including physical therapy were excluded from the review. Large RCTs were only included due to the reduced risk of random error in the outcomes.

PTFU was defined as passive follow-up which had occurred either after the scheduled closure of the trial or after the primary results had been published.

Search strategy

The search was conducted in five bibliographic databases on 13 April 2017, including Embase (OvidSP) (1 March 1974 to 12 April 2017), Medline (OvidSP) (1946 to present), PubMed, Cochrane Central Register of Controlled Trials (Cochrane Library, Wiley) (issue 3 of 12, March 2017) and Cochrane Methods Register (CENTRAL) (Cochrane Library, Wiley) (issue 3 of 4, July 2017). Searches were then restricted to articles published in English since 2006. Full details of strategies are provided in Additional file 2. In addition, a database search for completed and ongoing studies was conducted at ClinicalTrials.gov (https://clinicaltrials.gov/). Studies which were not yet published ‘grey literature’ were not included in the search strategy.

Data collection

Papers identified from the ClinicalTrials.gov registry were imported into a MS Excel spreadsheet. Duplicates and studies which had less than 1000 participants were removed using a filter option. The selection of eligible papers followed a concordance strategy between two reviewers (RLB and DE) which ensured that concordance was > 95% (Fig. 1) [7].
Figure 1
Fig. 1

PRISMA flow diagram detailing the process of study selection and data extraction. HCI healthcare intervention, PTFU post-trial follow-up, RCT randomised controlled trial

Medical interventions were defined as an intervention that was consumed orally, inhaled, or administered by intravenous or intramuscular injections including vaccines. A surgical intervention was defined as any intervention which was invasive (apart from those mentioned above and including blood transfusions). Potential studies were checked for eligibility by two reviewers who initially reviewed abstracts and then proceeded to full paper review in a step-wise process (Fig. 1).

In addition to those described in the protocol, some additional exclusions which were not originally listed were identified during the process of performing the systematic review in keeping with our definition of PTFU. This was required due to the heterogeneity of PTFUs. These include: (1) trials that were stopped before the scheduled closure of the trial; (2) cancer trials which had an open endpoint (e.g. survival as an endpoint with no clear scheduled plan of duration); (3) trials which continued with active intervention in the PTFU period with the primary outcome of safety and (4) trials eligible for inclusion but which did not contribute novel data as they only published additional subgroup or post-hoc analyses. A table of excluded trials is provided in Additional file 3.

Full papers deemed eligible for inclusion in the systematic review were extracted using a standardised Excel spreadsheet. Data was extracted by DE and RLB and concordance was checked. Primary outcome, healthcare intervention and attrition rates were tabulated for each study. Lead trialists were contacted via email to inform them of the systematic review and to clarify information where necessary. The papers included in the review were diverse with a range of interventions and different outcome measures. Due to the high level of clinical heterogeneity a meta-analysis was not possible.

Retention rates were calculated as the proportion of participants who were lost to follow-up compared to the overall total of those who started the PTFU period. Information about the cost of the PTFU was sought from study publications or via personal communication. Two attempts were made to contact the trialist via email and if there was no response or inadequate data, the trial was excluded from the cost analysis.

Assessment of risk of bias

Risk of bias was assessed for each included RCT on their primary results using the Cochrane Risk of Bias tool. Covdence.org was used to assess the levels of bias (low risk, high risk or unclear risk) in each methodological domain (sequence generation, allocation of sequence concealment, blinding, incomplete outcome data, selective reporting bias and other bias) and decisions checked by one of the senior authors [8]. The data recorded from Covidence.org was imported into Review Manager 5 (RevMan 5) for graphical representation [9].

Results

From 57,352 papers identified, 65 studies with PTFU were included in the systematic review (Fig. 1). Fifty trials involved medical interventions and 15 involved surgical interventions. There were no eligible psychiatric trials which had (all > 1000 participants). The duration of PTFU ranged from 1 to 20 years, with a median of 4.5 years of follow-up. The number of participants followed during the post-trial period ranged from 575 to 29 862.

Five methods of follow-up were identified: postal correspondence/questionnaire (19%); clinic appointments (35%); telephone interviews (26%); electronic data linkage (52%); and review of paper medical records (26%). In addition, in individual cases, specific follow-up was performed, e.g. endoscopy follow-up only [10]. Electronic data linkage and medical records review were used exclusively together in 11% of papers; either were used in combination with other methods in 74%. Overall, 48% of trials used more than one method to follow-up participants in the post-trial period (Tables 1 and 2). On average, two methods were used for each PTFU follow-up. Where data linkage was used, it was not always feasible to follow up all participants [11]. Some trials experienced difficulty accessing national electronic data in certain countries; for example, stricter regulations are apparent in Canada and for some North American participants (Medicare and Veteran Affairs) where a specific health ID number is required to access national data (Table 3). Trials experienced difficulty in accessing routinely collected health records in 3% of included papers and PTFU was restricted to those countries with robust and accessible centrally held records and registries (e.g. Sweden and Scotland) [12, 13].
Table 1

Post-trial follow-up (PTFU) in eligible medical trials. Note retention of participants expressed as % lost to follow-up

1st author, year

Primary outcome for PTFU

RCT name (PTFU name)

No. years PTFUa

Intervention

No. randomised in-trial

No. at the start of PTFU

% participants lost in PTFU

Type of PTFU for primary outcome

  

Post/Q

Clinic

Telephone

Data linkage

Paper records

Other

Alan, 2015

Mortality

ProHOSP

6

CAP antibiotics

1359

925

6

  

Y

 

Y

 

Arbel, 2016

Mortality

BIP

20

Bezafibrate

3090

3090

   

Y

  

Arber, 2011

Cancer, safety

PreSAP

2

Celecoxib

1561

1043

12

     

Y

Avenell, 2012

Mortality

RECORD

3

Vitamin D, Calcium

5292

4394

   

Y

  

Breitner, 2011

Alzheimer’s disease

ADAPT

2

Naproxen, Celecoxib

2528

2233

1

 

Y

    

Bulbulia, 2011

Mortality and morbidity

HPS

6

Simvastatin

20,536

17519

0

Y

  

Y

  

Cauley, 2013

Hip fractures, cancers, CVE and mortality

WHI

5

Calcium plus vitamin D

36,282

29862

1

    

Y

 

Cherry, 2014

Mortality, cancer

ESPIRIT

12

Oestrogen

1017

1017

   

Y

  

Chew, 2013

Progression of age-related macular degeneration

AREDS

5

Antioxidants

4757

3549

 

Y

  

Y

Y

Chowdhury, 2014

Diabetes mellitus, mortality, MACE

ANBP2

7

ACE inhibitor, Thiazide

6083

5678 (6083 linked to death registry)

Y

  

Y

  

Cushman, 2012

MACE, mortality

ALLHAT

13

Amlodipine, lisinopril

32,804

17,722 (CVD), 27,755 (mortality)

   

Y

  

Dienstag, 2011

Progression of Hep C

HALT-C

4

Peginterferon

1050

743

 

Y

    

Eastell, 2015

Bone mineral density

HORIZON-PFT

3

Zoledronic acid

7765

1223

 

Y

   

Y

Ebbing, 2010

Mortality

NORVIT, WENBIT

4

B vitamins

6845

6261

0

   

Y

  

Einstein, 2011

Safety, immunogenicity

2

HPV vaccine

1106

671

0

 

Y

   

Erdmann, 2014

Mortality, MI, stroke, MACE, (composite)

PROactive

3

Pioglitazone

5238

3599

9

 

Y

Y

 

Y

Y

Ezzedine, 2010

Skin cancer

SU.VI.MAX

5

Antioxidant vitamins

12,741

11054

2

Y

   

Y

 

Flossman, 2007

Colorectal cancer

UK-TIA

20

Aspirin

2449

2249

   

Y

Y

 

Colorectal cancer

BDAT

20

Aspirin

5139

5139

   

Y

Y

 

Ford, 2016

Mortality and morbidity

WOSCOPS

20

Pravastatin

6595

5778

   

Y

  

Gerstein, 2016

MACE, mortality (composite)

ACCORD (ACCORDIAN)

3

Intensive glucose control

10,251

8601

 

Y

Y

   

Gluud, 2008

Mortality

CLARICOR

3

Clarithromycin

4373

4029

1

   

Y

  

Gordon, 2012

Efficacy and safety

REVEAL

2

Adalimumab

1212

575

7

 

Y

    

Grau, 2009

Adenomas

AFPPS

4

Aspirin

1121

1007

14

Y

Y

    

Grubb, 2013

Cancer

REDUCE

2

Dutasteride

8231

2751

 

Y

Y

   

Hackshaw, 2011

Event-free survival

OVER 50S TRIAL

10

Tamoxifen

3449

3449

   

Y

  

Hague, 2016

Mortality, cancer

LIPID

10

Pravastatin

9014

7721

0

Y

Y

Y

Y

Y

 

Hayashino, 2009

Diabetes mellitus

PHI1

17

Aspirin

22,071

22,071

Y

    

Y

Hayward, 2015

MACE

VADT

5

Intensive glucose lowering vs standard therapy

1791

1791

22

Y

  

Y

  

Holman, 2008

Macrovascular outcomes

UKPDS

10

Intensive glycaemic control

3867

3277

20

Y

Y

 

Y

  

Hornslien, 2015

Stroke, MI, mortality

SCAST

3

Candesartan

2029

1286

2

   

Y

  

Investigators, 2011

Diabetes mellitus

DREAM (DREAM ON)

2

Rosiglitazone, ramipril

5269

1653

18

 

Y

    

Johnson, 2015

Vaccine efficacy

SPS (LTPS)

4

Vaccine

38,543

6867

6

 

Y

Y

   

Jones, 2015

Cancer, bone fractures

RECORD

4

Rosiglitazone

4447

2546

1

  

Y

Y

Y

 

Kostis, 2011

Mortality

SHEP

13

Chlorthalidone

4736

   

Y

  

Krane, 2016

MACE, mortality (composite)

4D

8

Atorvastatin

1255

637

3

Y

     

Lai, 2014

Mortality, liver cancer

ATBC

16

α-tocopherol,β-carotene

29,133

29105

   

Y

  

Laterre, 2007

Mortality

ADDRESS

1

Drotrecogin-α

2640

2621

9

Y

 

Y

 

Y

 

Leslie, 2011

Mortality

ENIGMA

4

Nitrous oxide

2050

2002

17

  

Y

 

Y

 

Leslie, 2015

MACE, mortality

ENIGMA-II

1

Nitrous oxide

7112

6651

12

  

Y

 

Y

 

Lewis, 2011

MACE

CAIFOS

5

Calcium

1510

1510

   

Y

  

Lloyd, 2013

MACE, cancers, mortality

PROSPER

3

Pravastatin

5804

5188

   

Y

  

Menne, 2014

Long-term micro, macrovascular benefit

ROADMAP (ROADMAP OFU)

3

Olmesartan medoxomil

4449

2198

0

 

Y

    

Ogihara, 2011

MACE, cancer, mortality

CASE-J (CASE-J Ex)

3

Candesartan, amlodipine

4728

2232

2

 

Y

    

Radford, 2014

Bone mineral density

Auckland Calcium Study

5

Calcium

1471

1408

17

  

Y

Y

  

Rothwell, 2010

Colorectal cancer

Thrombosis Prev Trial, Swedish Aspirin Low Dose Trial, Dutch TIA Aspirin Trial, UK-Tia Aspirin Trial, British Doctors Aspirin Trial

12, 13, 17, 18, 20

Aspirin

16,488

14033

 

Y

 

Y

Y

 

Tenkanen, 2006

MACE, cancer, mortality

Helsinki Heart Study

10

Gemfibrozil

4081

4081

0

Y

  

Y

  

Wang, 2015

Fracture incidence

NIT

16

Vitamins (14), minerals (12)

3318

3318

1

  

Y

 

Y

 

Weston, 2011

Persistence of antibodies

106316

3

Vaccine dip, pert, tetanus

2284

1505

 

Y

    

Whiteley, 2014

Disability

IST-3

1

Alteplase

3035

2348

2

   

Y

  

Zoungas, 2014

Mortality

ADVANCE (ADVANCE-ON)

6

Perindopril, indapamide

11,140

8494

 

Y

Y

   

where a is number of years (median/mean/max) published in the cited paper, years followed up to the nearest whole number, % participants lost to the nearest whole number,‘–’ no data available or not applicable where mortality records were sought, CVD cardiovascular disease, MACE major adverse cardiovascular events ± revascularisation, MI myocardial infarction. Where 0 participants have been lost to follow-up this has been confirmed either in the cited paper or directly with the corresponding trialist

Table 2

Post-trial follow-up (PTFU) in eligible surgical trials. Note, retention of participants is expressed as % lost to follow-up

1st author, year

Primary outcome for PTFU

RCT name (PTFU name)

No. years PTFU a

Intervention

No. participants randomised in trial

No. participants at the start of PTFU

% participants lost in PTFU

Method of PTFU for primary outcome

Post/Q

Clinic

Telephone

Data linkage

Paper records

Carson, 2015

Mortality

FOCUS

3

Blood transfusion

2016

2002

   

Y

 

Cho 2017

Mortality, MI, stroke, revascularisation

RISPO

4

RIPC, RIPostC

1328

1280

15

  

Y

Y

Y

Gada, 2013

Safety, efficacy, mortality

SPIRIT III

5

EES, PES

1002

 

Y

   

Gallagher, 2014

Mortality

RENAL (POST-RENAL)

4

Renal replacement therapy

1508

1464

   

Y

 

Halliday, 2010

Mortality, stroke

ACST-1

4

CEA or deferement

3120

3120

   

Y

 

Henderson, 2015

Mortality

RITA-3

5

PCI

1810

1810

0

   

Y

 

Hirsch, 2007

Mortality, MACE

ICTUS

4

PCI

1200

1124

3

  

Y

Y

Y

Hochman, 2011

Mortality, MACE

OAT

3

PCI

2201

1504

  

Y

Y

Y

Investigators, 2007

Mortality

BARI

5

PTCA

1829

1829

4

  

Y

Y

Y

Milojevic, 2016

Mortality

SYNTAX

5

PCI

1800

847

 

Y

Y

 

Y

Naunheim, 2006

Mortality

NETT

2

Lung-volume surgery

1218

70%

Y

  

Y

 

Patel, 2016

Mortality

EVAR-1

13

EVAR

1252

1252

2

 

Y

 

Y

Y

Powell, 2007

Mortality

UKSAT

12

Early AAA repair

1090

1090

0

   

Y

 

Sedlis, 2015

Mortality

COURAGE

6

PCI

2287

1211

   

Y

 

Wallentein, 2016

Mortality, MI (composite)

FRISC-II

15

PCI

2457

2421

1

   

Y

Y

where a is number of years (median/mean/max) published in the cited paper, years followed up to the nearest whole number, PCI percutaneous coronary intervention ± revascularisation, PTCA percutaneous transluminal coronary balloon angioplasty, EES everolimus-eluting stents, PES paclitaxel-eluting stents, EVAR endovascular aneurysm repair, CEA carotid endarterectomy, AAA abdominal aortic aneurysm, RIPC remote ischaemic preconditioning, RIPostC RIPC with postconditioning, MI myocardial infarction, MACE major adverse cardiovascular events ± revascularisation, Postal/Q postal communication or questionnaire, years followed up to the nearest whole number, % participants lost to the nearest whole number, 70% provided by trialist. Where 0 participants have been lost to follow-up this has been confirmed either in the cited paper or directly with the corresponding trialist

Table 3

Registries used for data linkage during post-trial follow-up (PTFU)

Country

Registry

Dataset

Website

USA

United States Renal Data System (USRDS)

Renal

www.usrds.org

Centres for Medicare and Medicaid Services (CMS ([formerly HCFA))a

Non-fatal events

www.cms.gov

National Death Index Plus Database

Cause- specific mortality

https://www.cdc.gov/nchs/ndi/

National Death Index and Social Security Administration

All-cause mortality

https://www.cdc.gov/nchs/nvss/deaths.htm

The Central Veterans Affairs Medical Information files

All-cause morbidity

https://www.va.gov/directory/guide/facility.asp?ID=5380

The Veterans Affairs Death Files

All-cause mortality

https://www.archives.gov/research/alic/reference/vital-records.html

Canada

Statistics Canada Mortality Database

All-cause mortality

http://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvey&SDDS=3233

England

NHS Digital (formerly HSCIC and Office of National Statistics)

Non-fatal events, all-cause mortality

https://digital.nhs.uk/

Scotland

Information and Statistical Division of the National Health Service for Scotland (Scottish Morbidity Record, General Register Office Death Record)

All-cause morbidity, mortality

http://www.isdscotland.org/

Israel

Ministry of Health from the Israeli Population Registry

All-cause mortality

https://www.health.gov.il/English/Pages/HomePage.aspx

Israel National Cancer Registry

Cancer

https://www.health.gov.il/English/MinistryUnits/HealthDivision/Icdc/Icr/Pages/default.aspx

Holland

Dutch Central Bureau of Statistics

All-cause mortality

http://www.iamexpat.nl/expat-page/official-issues/organisations/statistics-netherlands-cbs

Norway

Cardiovascular Disease in Norway (CVDNOR) project (for data < 2008)b

Cause-specific morbidity

https://cvdnor.b.uib.no/

Finland

Cause-of-Death Register (Statistics Finland)

All-cause mortality

http://tilastokeskus.fi/til/ksyyt/index_en.html

Population Register Centre c

Demographics

http://vrk.fi/en/frontpage

Finnish Cancer Registry

Cancer

http://www.cancer.fi/syoparekisteri/en/

Australia

Western Australia Data Linkage System (WADLS)

Non-fatal events, all-cause mortality

http://www.datalinkage-wa.org.au/

a Data only available for those with a valid Medicare or Social Security number (65% of all participants in the ALLHAT long-term follow-up), bRegistry linkage in Norway only available from 2008, c A personal identification number issued to each Finnish resident accesses demographic and medical records

Retention rates

Unfortunately, retention rates were often poorly reported in the PTFU, limiting the ability to assess the impact of methods used in relation to the proportion lost to follow-up.

All surgical trials investigated mortality as the primary outcome and, where data was available, the proportion of participants lost to follow-up in surgical trials ranged from 0.4 to 15.5%. However, data was not available for 53% of surgical trials. In medical trials, the primary outcome investigated varied more widely, although mortality as an endpoint was common and the proportion of participants lost to follow-up ranged from 0 to 22%. Data on loss to follow-up was not available in 44% of trials. Where mortality was the primary outcome, the number of participants lost to follow-up was not available in 32% of trials due to the use of mortality records where only notifications of deaths were fed back to the trialists.

Cost

Financial information was available for one third of the included trials. Consequently, it was not possible to provide a direct comparison between cost of PTFU and the different methodologies used due to the small sample size. The cost of PTFU ranged from £6000 to £14,600,000 (Table 4). Cost of PTFU per participant per year showed that IST-3 was the most economical costing £0.21 per participant per year using data linkage/medical records, closely followed by ‘Over 50s’ (£0.41) and RECORD trials (£0.46) which also used data linkage. LTPS was the most expensive PTFU per participant per year (US$531.53) using clinical appointments and telephone follow-up. ROADMAP which also followed up participants by clinic appointment only had a cost of €413.60 per participant per year.
Table 4

Comparing post-trial follow-up (PTFU) costs (where disclosed), by different follow-up methodologies

Type of follow-up, name of RCT or PTFU

Number of participants in PTFU

Duration of PTFU*

Incentive for participant follow-up

Cost of PTFU/grant received

Cost per participant per year

Clinical appointment only

 ROADMAP

2198

3.3

Travel reimbursement €20 per visit

€3,000,000

€413.60

Clinical appointment + telephone

 LTPS

6867

4

US$14,600,000

US$531.53

Data linkage/medical records only

 RECORD

4394

3

No

£6,000

£0.46

 FOCUS

2002

3

No

US$75,000

US$12.49

 NORWIT, WENBIT

6261

4

Letters sent to offer withdrawal from PTFU (registry follow-up)

NOK 16,000

NOK 0.64

 RENAL

1464

4

No

Undisclosed – original recruiting sites paid for finding and contacting participants

 

 CLARICOR

4029

3

£1,100,000

£91.01

 ‘Over 50s’

3449

10

no

£14,000

£0.41

 RITA-3

1810

5

£359,577

£39.73

 SCAST

1286

3

no

£7,000

€1.81

 CAIFOS

1510

4.5

no

AUD 848,206

AUD 124.83

 IST-3

2348

1

no

£500

£0.21

Telephone + data linkage/medical records

 ProHOSP

925

6

no

Negligible. Students conducted telephone follow-up as part of their training

 OAT

1504

3

no

US$100 administrative start-up, US$50 per call for each follow-up, US$30 per subject for re-consent payment, US$300 per event completing reporting

 ENIGMA

2002

3.5

no

AUD 53,807

AUD 7.68

 ENIGMA-II

6651

1

no

AUD 60,000

AUD 9.02

Postal correspondence + data linkage/medical records

 HPS

17,519

6

£250,000

£2.38

 ANBP2

6983

6.9

no

AUD 18,000

AUD 0.37

 ACST-1

3120

4

£120,000

£9.62

 VADT

1791

5

US$10 per survey gift card

US$10,00,000

US$111.67

Postal correspondence +telephone + medical records

 ADDRESS

2621

1

no

US$13,10,500

US$500

where a; median/max/range published in the cited paper, RCT randomised controlled trial, PTFU post-trial follow-up, NOK Norwegian Krone, AUD Australian dollar; ‘-’ no data available/ declined by corresponding trialist, ‘~‘ ,estimate; + RCT number as PTFU data not available. Results to 2 decimal places for cost per participant

Cochrane Risk of Bias

We hypothesised that those RCTs which had poor methodology or ‘high risk of bias’ might subsequently have a PTFU which was poorly organised and, therefore, have low retention rates (or a high proportion lost to PTFU). Of the 65 papers included in the systematic review, seven were excluded from the risk of bias assessment: these were PTFUs which followed-up an amalgamation of data from more than one trial or were part of a systematic review and, therefore, not suitable to be included in the analysis (the risks of bias from individual component trials could not be combined).

Of the 58 trials considered, the risk of bias could not be fully assessed in 11 trials due to lack of information in at least one domain. Low (or unclear) risk of bias in all domains was found in 43 (74%) of those assessed. Only seven trials (12%) had at least one domain which was high risk of bias, of which one had two domains at high risk (Fig. 2). Details of the individual risk of bias domains for each included study are provided in Additional file 4.
Figure 2
Fig. 2

Cochrane Risk of Bias graph. Review authors’ judgements about each risk of bias item presented as percentages across all included studies

Given the small number of trials found to have a high risk of bias in at least one domain and the highly variable retention rates observed during PTFU (Table 5), it is not possible to draw any clear conclusions with respect to possible associations between risk of bias and its potential impact on the proportion of participants that were lost to follow-up in the post-trial period.
Table 5

Comparison of randomised controlled trials (RCTs) which had high risk of bias compared to the proportion lost to follow-up during post-trial follow-up (PTFU). A summary of those RCTs with no risk of bias are also detailed

High-risk domain

Number of studies with high-risk domain

Proportion of participants lost to follow-up during PTFU (%)

Blinding participants and personnel

3

3.96–6.16

Incomplete outcome data

2

Other sources of bias

3

1.21–11.79

Selective outcome reporting

1

1.2

Low risk of bias in all domains

43 (no high/unclear risk of bias)

0–19.90

Discussion

This systematic review identified that PTFU methods varied and many trials used overlapping approaches which were more costly than needed. Data was limited on retention rates and so it was difficult to draw any firm conclusions on which method was best for PTFU.

Our main findings suggest that most PTFU published in the last 11 years does not appear to be designed in a cost-effective manner. Cost of PTFU was shown to vary widely and not many trials used incentives to retain participants. Despite only a third of trialists providing complete financial information for PTFU, follow-up by clinical appointment appeared to be the most expensive method, as might be expected given the resource implications. Postal or telephone correspondence in addition to data linkage did not appear to increase the cost per participant per year considerably. However, the effect of inflation over the 11 years included in this systematic review, makes quantitative comparison of cost differences difficult. Given the limited data available we have not attempted to adjust for inflation.

Data linkage or access to medical records is likely to be the most cost-effective method of following participants due to minimal staff required. However, a number of trialists highlighted the limitations of this approach, noting it to be time-consuming and frustrating with increasing regulatory costs and country-specific restrictions. In the UK, the process of accessing data electronically has become more stringent and costly, and markedly different to the processes which were encountered 10 years ago. There is also an issue of the data lagging behind by up to 2 years in some countries, which can impact on the completeness of results for a trial. Despite this, data linkages to national registries and electronic health records have been shown to be a valid and reliable method of PTFU [1215].

When designing this systematic review, we anticipated that papers published in the early half of the last 11 years would choose more traditional methods of PTFU, e.g. clinic- and telephone-based approaches, and more recent trials may increasingly use data linkage where available. However, this has not been the case. The majority of trials have used a variety of different methods to capture data for the same primary outcome. We were, therefore, unable to compare retention rates by each type of method used. In addition, sparsity of complete data in the review (typically poor reporting of the final number of participants at the end of the follow-up period) limited the ability to assess retention rates achieved with different PTFU methods.

We found limited evidence of high risk of bias in the methodology of the in-trial periods. A likely explanation for this is that the majority of the trials included in this review were well-designed, large RCTs in which results were published in high-impact journals. Furthermore, trials which employ poor methodology or have had negative results are more likely not to engage in PTFU due to lack of funding or interest.

Due to new guidelines (Consolidated Standards of Reporting Trials (CONSORT)) recommending increasing transparency in the reporting of RCTs, a more complete capture of data would be likely for any future study [16]. Research into appropriate methods in PTFU can only occur if there is transparency of the logistical and financial implications including number of participants lost to follow-up.

Conclusions

Post-trial follow-up of large RCTs can contribute significantly to the scientific value of a trial by determining the longer-term magnitude of the effects of an intervention. PTFU is valuable to ensure that there are no long-term hazards or beneficial effects which have been missed due to the common short in-trial periods for following up participants. However, it is not widely used as shown by the small number of eligible trials which had PTFU from the original search strategy.

Data linkage and the use of registries appear to be the most plausible and economical approach to PTFU. These methods also have the advantage of providing data for a wide range of endpoints. Improvement of electronic reporting and informatics could lead to better reporting and allow this type of method to be widely used.

Abbreviations

CENTRAL: 

Cochrane Central Register of Controlled Trials

EMBASE: 

Excerpta Medica database

PTFU: 

Post-trial follow-up

RCT: 

Randomised controlled trial

Declarations

Acknowledgements

Thank you to all corresponding trialists.

Funding

RLB has received funding from the Royal College of Surgeons of England Research Fellowship.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Authors’ contributions

RLB designed, carried out the systematic review including screening, data capture, data analysis, interpretation of results and wrote the paper. DE screened the papers from the search strategy and identified relevant papers including checking 10% of data from the extraction stage. NR assisted in the design of the search strategy and completed the search strategy. RB, LB assisted in the search strategy and in decisions relevant to the review. LB and RB assisted in drafting the review. AH assisted in the discussions of the review and drafting. All authors read and approved the final manuscript.

Ethics approval and consent to participate

Not applicable.

Consent for publication

RLB, DE, NR, AH, LB and RB consent for publication.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
MRC Population Health Research Unit, Clinical Trial Service Unit (CTSU), Nuffield Department of Population Health, University of Oxford, Oxford, UK
(2)
Bodleian Health Care Libraries, University of Oxford, Oxford, UK
(3)
Molecular and Clinical Sciences Research Institute, St George’s University of London, London, UK
(4)
Medical Research Council Population Health Research Unit, Nuffield Department of Population Health, University of Oxford, Oxford, UK
(5)
Clinical Trial Service Unit and Epidemiological Studies Unit, Nuffield Department of Population Health, University of Oxford, Oxford, UK

References

  1. Ford I., et al., Long-term safety and efficacy of lowering low-density lipoprotein cholesterol with statin therapy: 20-year follow-up of West of Scotland Coronary Prevention Study. (1524–4539 (Electronic)).Google Scholar
  2. Heart Protection Study Collaborative Group, et al. Effects on 11-year mortality and morbidity of lowering LDL cholesterol with simvastatin for about 5 years in 20,536 high-risk individuals: a randomised controlled trial. Lancet. 2011;378(9808):2013–20.View ArticlePubMedPubMed CentralGoogle Scholar
  3. Fergusson D, et al. Post-randomisation exclusions: the intention to treat principle and excluding patients from analysis. BMJ. 2002;325(7365):652–4.View ArticlePubMedPubMed CentralGoogle Scholar
  4. Brueton VC, et al., Strategies to improve retention in randomised trials: a Cochrane systematic review and meta-analysis. (2044–6055 (Linking)).Google Scholar
  5. Barton J, Young A, Lay M., Introduction of electronic data capture method using participant-completed online web-based follow up questionnaire in mail-based study achieves expected benefits and positive participant feedback. Trials, 2015. 16(2): p44.Google Scholar
  6. Scuffham P, Chaplin S, Legood R. Incidence and costs of unintentional falls in older people in the United Kingdom. J Epidemiol Community Health, 2003. 57(9): p. 740.Google Scholar
  7. Llewellyn-Bennett R, Bowman L, Bulbulia R Post-trial follow-up methodology in large randomized controlled trials: a systematic review protocol. Systematic Reviews, 2016. 5: p. 214.Google Scholar
  8. Covidence systematic review software, Veritas Health Innovation, Melbourne, Australia. Available at www.covidence.org.
  9. (RevMan), R.M., [Computer program] Version 5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration. 2014. http://community.cochrane.org/tools/review-production-tools/revman-5/about-revman-5. Accessed 30 Dec 2017.
  10. Arber N, et al. Five-year analysis of the prevention of colorectal sporadic adenomatous polyps trial. Am J Gastroenterol. 2011;106(6):1135–46.View ArticlePubMedGoogle Scholar
  11. Margolis KL, et al. Long-term follow-up of moderately hypercholesterolemic hypertensive patients following randomization to pravastatin vs usual care: the Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT-LLT). J Clin Hypertens (Greenwich). 2013;15(8):542–54. (1751–7176 (Electronic)).Google Scholar
  12. Hornslien AG, et al. Effects of candesartan in acute stroke on vascular events during long-term follow-up: results from the Scandinavian Candesartan Acute Stroke Trial (SCAST). Int J Stroke. 2015;10:830–5. https://doi.org/10.1111/ijs.12477.View ArticlePubMedGoogle Scholar
  13. Lloyd SM, et al. Long-term effects of statin treatment in elderly people: extended follow-up of the PROspective Study of Pravastatin in the Elderly at Risk (PROSPER). PLoS ONE [Electronic Resource]. 2013;8(9):e72642.View ArticleGoogle Scholar
  14. Kivimaki M, et al. Validity of cardiovascular disease event ascertainment using linkage to UK hospital records. Epidemiology. 2017;28(5):735–9.Google Scholar
  15. Herbert A, et al. Data Resource Profile: Hospital Episode Statistics Admitted Patient Care (HES APC). LID – https://doi.org/10.1093/ije/dyx015 [doi] FAU – Herbert, Annie. (1464–3685 (Electronic)).
  16. CONSORT Transparent Reporting of Trials. Available from: http://www.consort-statement.org/. Accessed 30 Dec 2017.

Copyright

© The Author(s). 2018

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate. Please note that comments may be removed without notice if they are flagged by another user or do not comply with our community guidelines.

Advertisement