- Open Access
- Open Peer Review
Association of the FDA Amendment Act with trial registration, publication, and outcome reporting
Trialsvolume 18, Article number: 333 (2017)
Selective clinical trial publication and outcome reporting has the potential to bias the medical literature. The 2007 Food and Drug Administration (FDA) Amendment Act (FDAAA) mandated clinical trial registration and outcome reporting on ClinicalTrials.gov, a publicly accessible trial registry.
Using publicly available data from ClinicalTrials.gov, FDA documents, and PubMed, we determined registration, publication, and reporting of findings for all efficacy trials supporting FDA approval of new drugs for cardiovascular disease and diabetes between 2005 and 2014, before and after the FDAAA. For published trials, we compared the published interpretation of the findings (positive, equivocal, or negative) with the FDA reviewer’s interpretation.
Between 2005 and 2014, the FDA approved 30 drugs for 32 indications of cardiovascular disease (n = 17) and diabetes (n = 15) on the basis of 183 trials (median per indication 5.7 (IQR, 3–8)). Compared with pre FDAAA, post-FDAAA studies were more likely to be registered (78 of 78 (100%) vs 73 of 105 (70%); p < 0.001), to be published (76 of 78 (97%) vs 93 of 105 (89%); p = 0.03), and to present findings concordant with the FDA reviewer’s interpretation (74 of 76 (97%) vs 78 of 93 (84%); p = 0.004). Pre FDAAA, the FDA reviewer interpreted 80 (76%) trials as positive and 91 (98%) were published as positive. Post FDAAA, the FDA reviewer interpreted 71 (91%) trials as positive and 71 (93%) were published as positive.
FDAAA was associated with increased registration, publication, and FDA-concordant outcome reporting for trials supporting FDA approval of new drugs for cardiovascular disease and diabetes.
The US Food and Drug Administration (FDA) approves new drugs based on clinical evidence, requiring “adequate and well controlled investigations”, to demonstrate safety and efficacy . The FDA suggests that drug sponsors provide two or more “pivotal” efficacy trials — typically large, randomized, controlled trials—as well as “non-pivotal” trials that provide additional insights into drug efficacy and safety. These trials provide the earliest evidence to inform use of newly approved drugs, and as such it is critical that their findings are disseminated through the peer-reviewed medical literature to inform clinical care.
Among recently approved drugs, studies have shown that nearly 90% of pivotal clinical trials were published [3, 4]. However, older studies raised concerns about the completeness, and accuracy, of the published evidence . For instance, one study examining clinical trials supporting FDA approval of antidepressants between 1987 and 2004 demonstrated that only 70% of those submitted to the FDA had been published, and that trials with positive findings were more likely to be published . Furthermore, some trials interpreted by the FDA to have negative or neutral findings were published to convey positive findings. These practices, known as selective publication and selective outcome reporting, distort the published evidence. However, in 2007, the US FDA Amendment Act (FDAAA) was enacted, mandating clinical trial registration and results reporting on ClinicalTrials.gov, a publicly accessible clinical trial registry established by the National Institutes of Health, for all ongoing and forthcoming trials of FDA-regulated products . The extent to which the FDAAA mitigated selective registration, publication, and outcome reporting is not known.
Accordingly, our objective was to determine registration, publication, and reporting of findings for all efficacy trials supporting FDA approval of new drugs for cardiovascular disease and diabetes between 2005 and 2014, before and after the FDAAA. We focused our study on drugs in cardiovascular disease and diabetes because they are among the most well studied and widely prescribed drugs in the US market and represent a large proportion of all newly approved drugs . In addition, for published trials, we compared the published interpretation of the trial findings with the FDA reviewer’s interpretation.
We obtained information on FDA-approved drugs using the Drugs@FDA database, a publicly available data source that includes regulatory actions and labeling changes for all currently approved drugs . For each drug, there are hyperlinks to medical and statistical reviews prepared by the FDA that include all clinical data relevant to the new drug application. All Drugs@FDA reviews were downloaded as of August 2016.
We identified a sample of novel therapeutics (i.e., new molecular entities or novel biologic drugs) first approved by the FDA between January 1, 2005, and December 31, 2014, excluding generic drugs, reformulations, and combination therapies of non-novel therapeutic agents (Fig. 1). We limited our sample to drugs in the therapeutic area of cardiovascular disease and diabetes mellitus as defined by the World Health Organization’s Anatomic Therapeutic Classification system (http://www.whocc.no/atc_ddd_index/), contextualized for clinical relevance, as reported previously . All drugs were characterized as having received either priority or standard review by the FDA and by whether or not they were designated with orphan status.
Selection of clinical trials
The Drugs@FDA database contains documents for all regulatory actions related to new drug approvals, including medical reviews with clinical data relevant to the drug of interest, summarized by the FDA reviewer. The medical reviews include an overview of safety and efficacy, an outline of the sources of clinical data, integrated summaries of safety and efficacy, and, where relevant, review of individual clinical trials.
Within each document is a section titled “Sources of Clinical Data” which includes a “Table of Studies/Clinical Trials” subsection. From this table we identified all phase 2 and phase 3 clinical trials examining drug efficacy. We excluded phase 1 trials and trials missing primary outcome definitions or results, conducted to exclusively evaluate safety, ongoing at the time of approval, extensions of trials described previously in the review, and expanded-access trials.
Characterization of clinical trials
Trials supporting approvals were categorized as pre FDAAA if the primary completion date was before December 26, 2007 (date the policy took effect), as indicated on ClinicalTrials.gov, FDA documents, or peer-reviewed publications; all other trials were categorized as post FDAAA.
In addition, we categorized trials by whether they were designated as pivotal or nonpivotal, phase 2 or phase 3, and by several design characteristics: randomization (yes/no), blinding and allocation concealment (double blinded yes/no), comparator type (active yes/no), trial size (more than 500 patients yes/no), trial duration (longer than 12 weeks yes/no), and trial quality (Jadad score ≥ 3 yes/no)  We also extracted primary outcome definitions and results as well as the overall trial findings of the FDA reviewer (positive, equivocal, negative). Trials were classified as equivocal if the FDA reviewer judged the trial to lack significant findings on the primary outcome but have significant findings on several secondary outcomes, consistent with prior literature . For dose-ranging trials (trials for which patients were assigned to one of several doses or placebo), we used the overall decision of the FDA reviewer.
Identification of trials registered on ClinicalTrials.gov and published in the peer-reviewed literature
For each trial described in FDA documents, we conducted a comprehensive search of ClinicalTrials.gov and PubMed in August 2016 to identify any corresponding registered trial or publication, respectively. Our search strategy used a combination of drug name, sponsor company, dosage groups, sample size, active comparator (if used), and duration, an approach used in prior research [3, 10, 11].
Comparison of published trials and FDA-submitted trials
Data were abstracted from the FDA summaries and PubMed publications by one author (ATP) and validated by a second author (CXZ). Any differences in findings were reconciled by consensus. For each trial described in FDA documents for which a publication in the peer-reviewed literature was identified, we compared the reported primary outcome definition and results, and the overall study interpretation, between the two sources. The overall interpretation was categorized as positive, equivocal, or negative based on the FDA officer’s language in the medical review and the author’s language in the conclusion of the publication; the FDA and publication interpretation were then categorized as concordant or discordant. For example, the Dronedarone ADONIS study was characterized as equivocal in the FDA documents and positive in the peer-reviewed publication because the FDA reviewer wrote “Dronedarone … significantly delayed the first AF/AFL recurrence. It also delayed the symptomatic first recurrence. However, these reviewers disagree with the Sponsor that dronedarone significantly reduced ventricular rate at the time of first recurrence” , whereas the authors in the peer-reviewed publication concluded that “Dronedarone was significantly more effective than placebo in maintaining sinus rhythm and in reducing the ventricular rate during recurrence of arrhythmia” .
We determined the overall rate of ClinicalTrials.gov registration and publication, and then stratified studies by type, drug, and design characteristics, using chi-square and Fisher exact tests as appropriate to compare rates of registration and publication across groups. We then categorized trials by concordance between the FDA review and publication to compare rates of concordance among trials stratified by type, drug, and design characteristics, again using chi-square and Fisher exact tests as appropriate. Finally, we categorized trials by the FDA reviewer’s interpretation and determined the publication rate for studies deemed positive, negative, or equivocal. For each categorization, we stratified trials as pre or post FDAAA. Analyses were performed using Microsoft Excel 15.11.2 (Microsoft Corporation, Redmond, WA, USA) and EpiInfo 3.0 (Centers for Disease Control and Prevention, Atlanta, GA, USA). The PRISMA checklist is presented in Additional file 1.
Between 2005 and 2014, the FDA approved 30 novel therapeutics for 32 indications for the treatment of cardiovascular disease (17 (53%)) and diabetes (15 (47%); Table 1). Among these, three (9%) indication approvals were designated orphan status and six (19%) were designated priority review status.
We identified a total of 288 clinical trials (122 pivotal, 166 nonpivotal) supporting these 32 indication approvals, of which 183 (120 pivotal, 63 nonpivotal) met our inclusion criteria (Fig. 1), a median of 5.7 (IQR, 3–8) trials per approval. Of the 183 clinical trials identified, 78 (43%) were post FDAAA, 151 (83%) were phase 3, 176 (96%) were randomized, 160 (87%) were double blinded, 75 (41%) had an active comparator, 116 (63%) were longer than 12 weeks in duration, 93 (51%) had more than 500 patients, and 175 (96%) had Jadad score ≥ 3.
Trial registration and publication
Among the 183 trials, 151 (83%) were registered on ClinicalTrials.gov, 99 (54%) had posted results, and 169 (92%) were published in the peer-reviewed literature. Trials registered on ClinicalTrials.gov (145 of 151 (96%)) were more likely to be published than those not registered (24 of 32 (75%); p = 0.0005) (Table 2). Pre FDAAA, certain trial characteristics were associated with likelihood of registration and publication, including double-blind design, enrollment > 500 patients, and Jadad score ≥ 3 (Table 3). Post FDAAA, all trials were registered, and trial duration > 12 weeks was associated with likelihood of publication (Table 4). Additionally, post-FDAAA trials were more likely to be registered (78 of 78 (100%) vs 73 of 105 (70%); p < 0.001) and to post results (70 of 78 (90%) vs 30 of 106 (28%); p = 0.0001) on ClinicalTrials.gov. They were also more likely to be published (76 of 78 (97%) vs 93 of 105 (89%); p = 0.03).
FDA reviewer trial interpretation
Among the 105 pre-FDAAA trials, the FDA reviewer characterized 80 (76%) as positive, 13 (12%) as equivocal, and 12 (11%) as negative; whereas among the 78 post-FDAAA trials, the FDA reviewer characterized 71 (91%) as positive, 4 (5%) as equivocal, and 3 (4%) as negative (Fig. 2). Pre FDAAA, trials characterized as positive were more likely to be registered (63 of 80 (79%) vs 10 of 25 (40%); p = 0.0005) and published (76 of 80 (95%) vs 17 of 25 (68%); p = 0.001) than negative or equivocal trials. The higher publication rate was primarily driven by trials for drugs in diabetes, in which positive trials were more likely to be published (38 of 38 (100%) vs 8 of 15 (53%); p = 0.0001) than negative or equivocal trials; trials for drugs in cardiovascular disease had similar rates of publication for positive trials (38 of 42 (90%)) and negative or equivocal trials (9 of 10 (90%); p = 0.67). Post FDAAA, all trials were registered and there was no difference in publication rate between positive trials (69 of 71 (97%)) and negative or equivocal trials (7 of 7 (100%); p = 0.83; Fig. 3).
Concordance between publication and FDA reviewer interpretation
Among the 93 pre-FDAAA published trials, 91 (98%) conveyed positive and 2 (2%) negative findings. Of these, 78 (84%) were published in a manner concordant with the FDA reviewer’s interpretation and 15 (16%) were published in a manner that conveyed a more positive interpretation than that of the FDA reviewer. Among the 76 post-FDAAA published trials, 71 (93%) conveyed positive findings, 2 (3%) equivocal findings, and 3 (4%) negative findings. Of these, 74 (97%) were published in a manner concordant with the FDA reviewer’s interpretation and 2 (3%) were published in a manner than conveyed a more positive interpretation than that of the FDA reviewer (see Additional file 2 for details of all discordant interpretations). Overall, no trials were published in a manner that conveyed a more negative interpretation than that of the FDA reviewer.
Rates of concordant interpretation were higher for post-FDAAA trials (74 of 76 (97%)) than pre-FDAAA trials (78 of 93 (84%); p = 0.004). Pre FDAAA, rates of concordant interpretation were higher for phase 2 trials and for those with double-blinded design (Table 2). Additionally, positive trials were more likely to have concordant findings (76 of 76 (100%)) than negative or equivocal trials (2 of 17 (12%); p = 0.03). Post FDAAA, there were no significant differences in concordance based on drug or trial characteristics. Positive trials, however, were more likely to have concordant findings (69 of 69 (100%)) than negative or equivocal trials (5 of 7 (83%); p = 0.007). Overall, trials registered on ClinicalTrials.gov (136 of 145 (94%)) were more likely to have concordant findings than those not registered (15 of 24 (63%); p < 0.0001).
The FDAAA, US legislation that mandated clinical trial registration and outcome reporting on ClinicalTrials.gov, was associated with lower rates of selective registration, publication, and outcome reporting among trials supporting FDA approval of new drugs for cardiovascular disease and diabetes between 2005 and 2014. However, there is still room for improvement because 11% of pre-FDAAA trials remain unpublished and there was an inconsistency in outcome reporting between the FDA review and the publication for 16%, as the trials were published in a manner that conveyed a more positive interpretation than the FDA reviewer. Post FDAAA, only 3% of the trials were unpublished, and of those published only 3% were done so with a more positive interpretation than the FDA reviewer. Overall, our study suggests that while nearly one-third of pre-FDAAA trials were not represented completely and accurately in the published literature, only one in 15 post-FDAAA trials were not represented completely and accurately in the published literature.
Three drugs accounted for the vast majority of trials that were unpublished or published with a more positive interpretation than that of the FDA reviewer (all pre FDAAA): dronedarone, insulin detemir, and pramlintide. Dronedarone is approved to reduce the risk of hospitalization in patients with atrial fibrillation. Three trials, two published in the New England Journal of Medicine (EURIDIS and ADONIS) and one in the American Heart Journal (ERATO), had conflicting interpretations in the FDA review and the journal article. The FDA interpretations of the EURIDIS and ADONIS trials were both considered equivocal because each demonstrated that dronedarone delayed the time to first recurrence of atrial fibrillation, but the ventricular rate during recurrence was unchanged from placebo; in contrast, the interpretations in the publications were considered positive because both concluded that dronedarone reduced the ventricular rate during recurrence. The FDA interpretation of the ERATO trial was considered negative because it demonstrated lack of efficacy for a clinically important secondary endpoint, exercise tolerance, whereas the publication had a positive interpretation based on a reduction in ventricular rate during first recurrence, a surrogate marker of disease.
Insulin detemir is approved to improve glycemic control in patients with diabetes. The FDA interpretation of several trials was considered negative or equivocal because of the variable amount of short-acting insulin used by participants, making it difficult to determine the relative efficacy of insulin detemir vs other long-acting insulin formulations. In contrast, the interpretations in the publications were positive because they demonstrated better balance of glucose control and variability, and less weight gain. Pramlintide, similarly, is approved to improve glycemic control in patients with diabetes. In one trial, the FDA interpretation was negative because the trials failed to show efficacy for the prespecified primary endpoint of reduction in hemoglobin a1c at 52 weeks; in contrast, the journal article was positive because it demonstrated a reduction in hemoglobin a1c at 13 weeks. For busy clinicians reading only the abstract of a journal article without exposure to a deep analysis of the data, these altered interpretations can influence utilization of the medication.
Our findings show post-FDAAA improvement compared to Turner’s landmark paper describing selective publication and outcome reporting among studies supporting FDA approval of antidepressants between 1987 and 2004, which demonstrated that only 70% of clinical trials had been published, but that 94% of published trials conveyed positive findings, despite the fact that only 51% of the trials submitted to the FDA had been positive . Pre FDAAA, we found that three of four trials submitted to the FDA had positive results, 95% of which were published; and among those characterized by the FDA as equivocal or negative, 92% were either unpublished or published in a way that conveyed positive findings. The published literature gives the appearance that 98% of the trials are positive, despite the fact that only 76% of the trials submitted to the FDA were interpreted by the FDA reviewer as positive. Post FDAAA, we found that 91% of trials submitted to the FDA were positive, 97% of which were published; and among those characterized by the FDA as equivocal or negative, 29% were either unpublished or published in a way that conveyed positive findings. The published literature gives the appearance that 93% of the trials are positive, similar to the 91% interpreted as positive by the FDA reviewer.
Registration within ClinicalTrials.gov improved after implementation of the FDAAA, such that nearly all of the trials submitted to support FDA approval that were subject to the 2007 FDAAA regulations were registered on ClinicalTrials.gov, but only half have reported results thus far . These findings are slightly more encouraging than prior research, which demonstrated that only 41.5% of trials subject to the FDAAA reported results within 5 years of study completion . The improvement in reporting of results may be due to increasing familiarity with the process among clinical trialists, increasingly ubiquitous registration requirements for journal publication, and overall culture change regarding clinical trial transparency.
There are several limitations to our study. First, we only looked at trials supporting FDA approval of drugs used for the treatment of cardiovascular disease and diabetes. Our results may not be generalizable to all clinical areas, because prior research has shown that publication rates for pivotal trials supporting cardiovascular disease and diabetes approvals are lower than in other therapeutic areas , raising the possibility that there was a greater opportunity for improvement. Our findings should be confirmed for FDA approvals in other therapeutic areas. Second, our analysis was focused on trials providing efficacy data to support FDA approval and is not generalizable to trials focused on safety. Third, we split our observation period into two periods based on the adoption date of FDAAA, but there was likely a more gradual improvement than that implied by our analysis, and other factors, such as improved awareness and knowledge of clinical trial requirements over time among both trialists and journal editors, may have contributed to the improvement in registration, publication, and concordance rates. Fourth, we did not determine whether the results reported on ClinicalTrials.gov were concordant with the results presented in the FDA documents and published literature; this was outside the scope of this review. Fifth, for trials determined to be unregistered or unpublished we did not contact sponsor companies for confirmation. Finally, because our study was observational in nature, we are unable to draw causal conclusions about the impact of the FDAAA on trial registration, publication, and interpretation concordance.
The 2007 FDAAA was associated with higher rates of clinical trial registration on ClinicalTrials.gov, publication, and concordance between interpretation of findings in the FDA documents and published literature. However, many older trials remain unpublished or published with a more positive conclusion than that of the FDA review, suggesting a clear need to improve the historical integrity of the clinical research enterprise. Further efforts to improve accessibility of older and negative clinical trials, such as online FDA resources, within drug labels, and negative trial journals or repositories, are needed to ensure complete and timely dissemination of clinical research.
Food and Drug Administration
Food and Drug Administration Amendment Act
New Drugs: Grounds for Refusing Application; Approval of Application; “Substantial Evidence” Defined, 21 USC §355d. https://www.gpo.gov/fdsys/pkg/USCODE-2010-title21/html/USCODE-2010-title21-chap9-subchapV-partA-sec355.htm. Accessed Aug 2016.
Guidance for Industry: Providing Clinical Evidence of Effectiveness for Human Drug and Biological Products. http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/ucm078749.pdf. Accessed Aug 2016.
Smithy JW, Downing NS, Ross JS. Publication of pivotal efficacy trials for novel therapeutic agents approved between 2005 and 2011: a cross-sectional study. JAMA Intern Med. 2014;174(9):1518–20.
Miller JE, Korn D, Ross JS. Clinical trial registration, reporting, publication and FDAAA compliance: a cross-sectional analysis and ranking of new drugs approved by the FDA in 2012. BMJ Open. 2015;5:e009758. doi:10.1136/bmjopen-2015-009758.
Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358(3):252–60.
The Food and Drug Administration Amendments Act of 2007. In: Public Law 110-85. https://www.gpo.gov/fdsys/pkg/PLAW-110publ85/pdf/PLAW-110publ85.pdf.
Downing NS, Aminawung JA, Shah ND, Krumholz HM, Ross JS. Clinical trial evidence supporting FDA approval of novel therapeutic agents, 2005-2012. JAMA. 2014;311(4):368–77.
US Food and Drug Administration (FDA). Drugs@FDA: FDA Approved Drug Products. FDA website. https://www.accessdata.fda.gov/scripts/cder/drugsatfda. Accessed Aug 2016.
Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJM, Gavaghan DJ, McQuay HJ. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials. 1996;17(1):1–12.
Phillips AT, Rathi VK, Ross JS. Publication of Clinical Studies Supporting FDA Premarket Approval for High-Risk Cardiovascular Devices Between 2011 and 2013A Cross-sectional Study. JAMA Intern Med. 2016;176(4):551-2. doi:10.1001/jamainternmed.2015.8590.
Chang L, Dhruva SS, Chu J, Bero LA, Redberg RF. Selective reporting in trials of high risk cardiovascular devices: cross sectional comparison between premarket approval summaries and published reports. BMJ. 2015;350.
Moreschi G, Friedlin V. Dronedarone Clinical and Statistical Review NDA 22-425. Silver Spring: US Food and Drug Administration (FDA); 2009. p. 59.
Singh BN, Connolly SJ, Crijns HJGM, Roy D, Kowey PR, Capucci A, Radzik D, Aliot EM, Hohnloser SH. Dronedarone for maintenance of sinus rhythm in atrial fibrillation or flutter. N Engl J Med. 2007;357(10):987–99.
De Angelis C, Drazen JM, Frizelle FAP, Haug C, Hoey J, Horton R, Kotzin S, Laine C, Marusic A, Overbeke AJPM, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. N Engl J Med. 2004;351(12):1250–1.
Anderson ML, Chiswell K, Peterson ED, Tasneem A, Topping J, Califf RM. Compliance with results reporting at ClinicalTrials.gov. N Engl J Med. 2015;372(11):1031–9.
This project was not supported by any external grants or funds. The authors assume full responsibility for the accuracy and completeness of the ideas presented.
Availability of data and materials
The data generated or analyzed during this study are available from the corresponding author upon reasonable request.
Data access and responsibility
ATP and JSR had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Ethics approval and consent to participate
Because our examination of trial publications did not involve human subjects, ethics committee review was not required by the Yale University Human Research Protection Program. Consent to participate not applicable.
Consent for publication
NRD, HMK, and JSR receive support through Yale University from Johnson and Johnson to develop methods of clinical trial data sharing, from the Blue Cross Blue Shield Association to better understand medical technology evaluation, and from the Centers of Medicare and Medicaid Services (CMS) to develop and maintain performance measures that are used for public reporting. HMK and JSR receive support through Yale University from Medtronic, Inc. and the US FDA to develop methods for postmarket surveillance of medical devices. HMK, JEM, and JSR receive support through Yale University and New York University, respectively, from the Laura and John Arnold Foundation to better understand clinical research integrity and transparency. HMK reports that he chairs a scientific advisory board for UnitedHealthcare. ATP and CXZ declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
PRISMA checklist. (DOC 63 kb)
Presenting the clinical trials supporting FDA approvals of drugs in cardiovascular disease and diabetes that were published in the biomedical literature in a manner that was discordant with the FDA reviewer’s interpretation. (DOCX 32 kb)