A literature review on the representativeness of randomized controlled trial samples and implications for the external validity of trial results

Kennedy-Martin, Tessa; Curtis, Sarah; Faries, Douglas; Robinson, Susan; Johnston, Joseph

doi:10.1186/s13063-015-1023-4

Trials

Table 3 Key results and main author conclusions from Method B studies

From: A literature review on the representativeness of randomized controlled trial samples and implications for the external validity of trial results

Study	Real-world data source	% ineligibility^a	Key differences (ineligible versus eligible patients)	Main author conclusions^b
Cardiology
Bahit et al., 2003 [16]	MC-PR	33.6	Older, more females/previous MI, lower ASA use, longer LOS	Different
Bosch et al., 2008 [19]	SC-PR	41.2	Older, higher risk profile	Different
Collet et al., 2003 [53]	SC-PR	34.0	Older, more females, higher risk score, fewer in-hospital procedures	NE
Costantino et al., 2009^c [21]	SC-PR	66.2	ND	Different
Fortin et al., 2006 [55]	MC-PR	1.4–65.5	ND	NE
Koeth et al., 2009 [34]	MC-PR	46.4	Older, more females, more diabetes/hypertension, less guideline-recommended treatment	Different
Krumholz et al., 2003 [56]	MC-PR	84.5 (NRMI)	ND	Similar
Krumholz et al., 2003 [56]	MC-PR	90.6 (CCP)	ND	Similar
Lenzen et al., 2005 [35]	MC-PR	61.6	Older, more females, more co-morbid hypertension/ACS/renal impairment, less guideline-recommended treatment at baseline	Different
Masoudi et al., 2003 [36]	ID	67.0	ND	Different
Steg et al., 2007 [40]	MC-PR	33.6	Older, history of MI, diabetes, TIA, PAD, and CABG, less guideline-recommended treatment/procedures, high risk score	Different
Uijen et al., 2007^c [44]	MC-PR	53.0	ND	Different
Mental health
Blanco et al., 2008 [18]	GP	75.8	ND	Different
Goedhard et al., 2010 [26]	SC-PR	69.8	Older, more Axis II personality disorders	Different
Hoertel et al., 2013 [28]	GP	58.2 (bipolar)	ND	Different
Hoertel et al., 2013 [28]	GP	55.8 (mania)	ND	Different
Keitner et al., 2003 [32]	SC-PR	85.5	ND	Different
Khan et al., 2005 [33]	GP	98.2	ND	Different
Rabinowitz et al., 2003^c [59]	MC-PR	33.0	ND	Similar
Seemuller et al., 2010 [61]	MC-PR	69.0	Younger, trend to younger age at disease onset	Similar
Storosum et al., 2004 [41]	SC-PR	83.8^d	ND	Different
Surman et al., 2010^c [42]	SC-PR	61.0	More lifetime co-morbidity, lower overall functioning/SES	Different
Talamo et al., 2008 [63]	SC-PR	77.6	Few differences	Similar
van der Lem et al., 2011 [64]	SC-PR	75.5–81.2^e	ND	NE
Wisniewski et al., 2009 [47]	MC-PR	77.8	Older, less educated, more black/Hispanic, longer disease duration, history of suicide and substance abuse, more atypical features	Different
Zarin et al., 2005^c [49]	MC-PR	55.0 (bipolar) 38.0 (schizo-phrenia)	More co-morbidity, lower global functioning, greater use of antipsychotic medication	Different
Zetin and Hoepner, 2007 [50]	SC-PR	91.4	ND	Different
Zimmerman et al., 2004 [51]	SC-PR	65.8	ND	Different
Oncology
Clarey et al., 2012 [20]	SC-PR	31.0–76.0	ND	Different
Filion et al., 2012 [54]	SC-PR	–^f	ND	Similar
Fraser et al., 2011^c [25]	MC-PR	14.9	ND	Different
Mengis et al., 2003^c [38]	SC-PR	87.0	ND	Different
Mol et al., 2013 [58]	MC-PR	21.5	Worse performance status, higher alkaline phosphatase, less primary tumor resection	Similar
Somer et al., 2008 [39]	SC-PR	71.0	ND	Different
Terschüren et al., 2010 [43]	MC-PR	35.9 (HL)	ND	Different
Terschüren et al., 2010 [43]	MC-PR	70.4 (hgNHL)	ND	Different
Vardy et al., 2009 [46]	MC-PR	65.0–72.0	ND	Different

Please see Additional files 2 and 4 for more detailed results
^aPercentage of patients not eligible for RCT inclusion following the application of eligibility criteria. ^bDifferent: authors explicitly comment, in their opinion, that there were meaningful differences between populations that suggested they were not representative, that the data could not be extrapolated or were not applicable to real-world settings, and/or that external validity is impacted; NE: authors do not explicitly comment on external validity or do not comment on external validity despite demonstration of differences in baseline characteristics; Similar: authors comment that populations are similar and/or that RCT results are generalizable to the overall disease population. ^cStudies that employed Methods A and B; in these studies RCT samples and real-world populations were compared, the authors then used the eligibility criteria from the RCT of interest to determine how many patients would hypothetically have been eligible or ineligible for that trial. Results presented in this table are for Method B only (see Table 2 for Method A results). ^dPercentage of manic episodes not number of ineligible. ^e75.5 % based on application of stringent criteria using the Mittman regression equation to calculate HAM-D; 81.2 % based on application of stringent criteria using the Hawley or Zimmerman regression equation to calculate HAM-D. ^fInclusion/exclusion criteria were categorized to identify criteria that might impede RCT recruitment; if any individual category was not met by > 10 % of patients with breast cancer from a retrospective cohort, then the criterion was considered a barrier to recruitment. ACS acute coronary syndrome, ASA aspirin, CABG coronary artery bypass graft, CCP cooperative cardiovascular project, GP general population data, HL Hodgkin’s lymphoma, hgNHL high-grade non-Hodgkin’s lymphoma, ID insurance data, LOS length of stay, MC-PR patient records - multicenter (including multicenter registries and observational studies), MI myocardial infarction, ND not determined, NRMI National Registry of Myocardial Infarction, PAD peripheral arterial disease, SC-PR patient records - single center, SES socioeconomic status, TIA transient ischemic attack

Back to article page

ISSN: 1745-6215

Contact us

Submission enquiries: Access here and click Contact Us
General enquiries: info@biomedcentral.com