Association of the FDA Amendment Act with trial registration, publication, and outcome reporting

Background Selective clinical trial publication and outcome reporting has the potential to bias the medical literature. The 2007 Food and Drug Administration (FDA) Amendment Act (FDAAA) mandated clinical trial registration and outcome reporting on ClinicalTrials.gov, a publicly accessible trial registry. Methods Using publicly available data from ClinicalTrials.gov, FDA documents, and PubMed, we determined registration, publication, and reporting of findings for all efficacy trials supporting FDA approval of new drugs for cardiovascular disease and diabetes between 2005 and 2014, before and after the FDAAA. For published trials, we compared the published interpretation of the findings (positive, equivocal, or negative) with the FDA reviewer’s interpretation. Results Between 2005 and 2014, the FDA approved 30 drugs for 32 indications of cardiovascular disease (n = 17) and diabetes (n = 15) on the basis of 183 trials (median per indication 5.7 (IQR, 3–8)). Compared with pre FDAAA, post-FDAAA studies were more likely to be registered (78 of 78 (100%) vs 73 of 105 (70%); p < 0.001), to be published (76 of 78 (97%) vs 93 of 105 (89%); p = 0.03), and to present findings concordant with the FDA reviewer’s interpretation (74 of 76 (97%) vs 78 of 93 (84%); p = 0.004). Pre FDAAA, the FDA reviewer interpreted 80 (76%) trials as positive and 91 (98%) were published as positive. Post FDAAA, the FDA reviewer interpreted 71 (91%) trials as positive and 71 (93%) were published as positive. Conclusions FDAAA was associated with increased registration, publication, and FDA-concordant outcome reporting for trials supporting FDA approval of new drugs for cardiovascular disease and diabetes. Electronic supplementary material The online version of this article (doi:10.1186/s13063-017-2068-3) contains supplementary material, which is available to authorized users.


Background
The US Food and Drug Administration (FDA) approves new drugs based on clinical evidence, requiring "adequate and well controlled investigations", to demonstrate safety and efficacy [1]. The FDA suggests that drug sponsors provide two or more "pivotal" efficacy trials [2]-typically large, randomized, controlled trials-as well as "non-pivotal" trials that provide additional insights into drug efficacy and safety. These trials provide the earliest evidence to inform use of newly approved drugs, and as such it is critical that their findings are disseminated through the peer-reviewed medical literature to inform clinical care.
Among recently approved drugs, studies have shown that nearly 90% of pivotal clinical trials were published [3,4]. However, older studies raised concerns about the completeness, and accuracy, of the published evidence [5]. For instance, one study examining clinical trials supporting FDA approval of antidepressants between 1987 and 2004 demonstrated that only 70% of those submitted to the FDA had been published, and that trials with positive findings were more likely to be published [5]. Furthermore, some trials interpreted by the FDA to have negative or neutral findings were published to convey positive findings. These practices, known as selective publication and selective outcome reporting, distort the published evidence. However, in 2007, the US FDA Amendment Act (FDAAA) was enacted, mandating clinical trial registration and results reporting on Clinical-Trials.gov, a publicly accessible clinical trial registry established by the National Institutes of Health, for all ongoing and forthcoming trials of FDA-regulated products [6]. The extent to which the FDAAA mitigated selective registration, publication, and outcome reporting is not known.
Accordingly, our objective was to determine registration, publication, and reporting of findings for all efficacy trials supporting FDA approval of new drugs for cardiovascular disease and diabetes between 2005 and 2014, before and after the FDAAA. We focused our study on drugs in cardiovascular disease and diabetes because they are among the most well studied and widely prescribed drugs in the US market and represent a large proportion of all newly approved drugs [7]. In addition, for published trials, we compared the published interpretation of the trial findings with the FDA reviewer's interpretation.

Data sources
We obtained information on FDA-approved drugs using the Drugs@FDA database, a publicly available data source that includes regulatory actions and labeling changes for all currently approved drugs [8]. For each drug, there are hyperlinks to medical and statistical reviews prepared by the FDA that include all clinical data relevant to the new drug application. All Drugs@FDA reviews were downloaded as of August 2016.

Drug sample
We identified a sample of novel therapeutics (i.e., new molecular entities or novel biologic drugs) first approved by the FDA between January 1, 2005, and December 31, 2014, excluding generic drugs, reformulations, and combination therapies of non-novel therapeutic agents (Fig. 1). We limited our sample to drugs in the therapeutic area of cardiovascular disease and diabetes mellitus as defined by the World Health Organization's Anatomic Therapeutic Classification system (http://www.whocc.no/atc_ddd_index/), contextualized for clinical relevance, as reported previously [7]. All drugs were characterized as having received either priority or standard review by the FDA and by whether or not they were designated with orphan status.

Selection of clinical trials
The Drugs@FDA database contains documents for all regulatory actions related to new drug approvals, including medical reviews with clinical data relevant to the drug of interest, summarized by the FDA reviewer. The medical reviews include an overview of safety and efficacy, an outline of the sources of clinical data, integrated summaries of safety and efficacy, and, where relevant, review of individual clinical trials.
Within each document is a section titled "Sources of Clinical Data" which includes a " Table of Studies/Clinical  Trials" subsection. From this table we identified all phase  2 and phase 3 clinical trials examining drug efficacy. We excluded phase 1 trials and trials missing primary outcome definitions or results, conducted to exclusively evaluate safety, ongoing at the time of approval, extensions of trials described previously in the review, and expanded-access trials.

Characterization of clinical trials
Trials supporting approvals were categorized as pre FDAAA if the primary completion date was before December 26, 2007 (date the policy took effect), as indicated on ClinicalTrials.gov, FDA documents, or peerreviewed publications; all other trials were categorized as post FDAAA.
In addition, we categorized trials by whether they were designated as pivotal or nonpivotal, phase 2 or phase 3, and by several design characteristics: randomization (yes/no), blinding and allocation concealment (double blinded yes/no), comparator type (active yes/no), trial size (more than 500 patients yes/ no), trial duration (longer than 12 weeks yes/no), and trial quality (Jadad score ≥ 3 yes/no) [9] We also extracted primary outcome definitions and results as well as the overall trial findings of the FDA reviewer (positive, equivocal, negative). Trials were classified as equivocal if the FDA reviewer judged the trial to lack significant findings on the primary outcome but have significant findings on several secondary outcomes, consistent with prior literature [5]. For dose-ranging trials (trials for which patients were assigned to one of several doses or placebo), we used the overall decision of the FDA reviewer.
Identification of trials registered on ClinicalTrials.gov and published in the peer-reviewed literature For each trial described in FDA documents, we conducted a comprehensive search of ClinicalTrials.gov and PubMed in August 2016 to identify any corresponding registered trial or publication, respectively. Our search strategy used a combination of drug name, sponsor company, dosage groups, sample size, active comparator (if used), and duration, an approach used in prior research [3,10,11].

Comparison of published trials and FDA-submitted trials
Data were abstracted from the FDA summaries and PubMed publications by one author (ATP) and validated by a second author (CXZ). Any differences in findings were reconciled by consensus. For each trial described in FDA documents for which a publication in the peerreviewed literature was identified, we compared the reported primary outcome definition and results, and the overall study interpretation, between the two sources. The overall interpretation was categorized as positive, equivocal, or negative based on the FDA officer's language in the medical review and the author's language in the conclusion of the publication; the FDA and publication interpretation were then categorized as concordant or discordant. For example, the Dronedarone ADONIS study was characterized as equivocal in the FDA documents and positive in the peer-reviewed publication because the FDA reviewer wrote "Dronedarone … significantly delayed the first AF/AFL recurrence. It also delayed the symptomatic first recurrence. However, these reviewers disagree with the Sponsor that dronedarone significantly reduced ventricular rate at the time of first recurrence" [12], whereas the authors in the peerreviewed publication concluded that "Dronedarone was significantly more effective than placebo in maintaining sinus rhythm and in reducing the ventricular rate during recurrence of arrhythmia" [13].

Statistical analysis
We determined the overall rate of ClinicalTrials.gov registration and publication, and then stratified studies by type, drug, and design characteristics, using chi-square and Fisher exact tests as appropriate to compare rates of registration and publication across groups. We then categorized trials by concordance between the FDA review and publication to compare rates of concordance among trials stratified by type, drug, and design characteristics, again using chisquare and Fisher exact tests as appropriate. Finally, we categorized trials by the FDA reviewer's interpretation and determined the publication rate for studies deemed positive, negative, or equivocal. For each categorization, we stratified trials as pre or post FDAAA. Analyses were performed using Microsoft Excel 15.11.2 (Microsoft Corporation, Redmond, WA, USA) and EpiInfo 3.0 (Centers for Disease Control and Prevention, Atlanta, GA, USA). The PRISMA checklist is presented in Additional file 1.

Concordance between publication and FDA reviewer interpretation
Among the 93 pre-FDAAA published trials, 91 (98%) conveyed positive and 2 (2%) negative findings. Of these, 78 (84%) were published in a manner concordant with the FDA reviewer's interpretation and 15 (16%) were published in a manner that conveyed a more positive interpretation than that of the FDA reviewer. Among the 76 post-FDAAA published trials, 71 (93%) conveyed positive findings, 2 (3%) equivocal findings, and 3 (4%) negative findings. Of these, 74 (97%) were published in a manner concordant with the FDA reviewer's interpretation and 2 (3%) were published in a manner than conveyed a more positive interpretation than that of the FDA reviewer (see Additional file 2 for details of all discordant interpretations). Overall, no trials were published in a manner that conveyed a more negative interpretation than that of the FDA reviewer.

Discussion
The FDAAA, US legislation that mandated clinical trial registration and outcome reporting on ClinicalTrials.gov, was associated with lower rates of selective registration, publication, and outcome reporting among trials supporting FDA approval of new drugs for cardiovascular   interpretation than the FDA reviewer. Post FDAAA, only 3% of the trials were unpublished, and of those published only 3% were done so with a more positive interpretation than the FDA reviewer. Overall, our study suggests that while nearly one-third of pre-FDAAA trials were not represented completely and accurately in the published literature, only one in 15 post-FDAAA trials were not represented completely and accurately in the published literature. Three drugs accounted for the vast majority of trials that were unpublished or published with a more positive interpretation than that of the FDA reviewer (all pre FDAAA): dronedarone, insulin detemir, and pramlintide. Dronedarone is approved to reduce the risk of hospitalization in patients with atrial fibrillation. Three trials, two published in the New England Journal of Medicine (EURIDIS and ADONIS) and one in the American Heart Journal (ERATO), had conflicting interpretations in the FDA review and the journal article. The FDA interpretations of the EURIDIS and ADONIS trials were both considered equivocal because each demonstrated that dronedarone delayed the time to first recurrence of atrial fibrillation, but the ventricular rate during recurrence was unchanged from placebo; in contrast, the interpretations in the publications were considered positive because both concluded that dronedarone reduced the ventricular rate during recurrence. The FDA interpretation of the ERATO trial was considered negative because it demonstrated lack of efficacy for a clinically important secondary endpoint, exercise tolerance, whereas the publication had a positive interpretation based on a reduction in ventricular rate during first recurrence, a surrogate marker of disease.
Insulin detemir is approved to improve glycemic control in patients with diabetes. The FDA interpretation of several trials was considered negative or equivocal because of the variable amount of short-acting insulin used by participants, making it difficult to determine the relative efficacy of insulin detemir vs other long-acting insulin formulations. In contrast, the interpretations in the publications were positive because they demonstrated better balance of glucose control and variability, and less weight gain. Pramlintide, similarly, is approved to improve glycemic control in patients with diabetes. In one trial, the FDA interpretation was negative because the trials failed to show efficacy for the prespecified primary endpoint of reduction in hemoglobin a1c at 52 weeks; in contrast, the journal article was positive because it demonstrated a reduction in hemoglobin a1c at 13 weeks. For busy clinicians reading only the abstract of a journal article without exposure to a deep analysis of the data, these altered interpretations can influence utilization of the medication.
Our findings show post-FDAAA improvement compared to Turner's landmark paper describing selective publication and outcome reporting among studies supporting FDA approval of antidepressants between 1987 and 2004, which demonstrated that only 70% of clinical trials had been published, but that 94% of published trials conveyed positive findings, despite the fact that only 51% of the trials submitted to the FDA had been positive [5]. Pre FDAAA, we found that three of four trials submitted to the FDA had positive results, 95% of which were published; and among those characterized by the FDA as equivocal or negative, 92% were either unpublished or published in a way that conveyed positive findings. The published literature gives the appearance that 98% of the trials are positive, despite the fact that only 76% of the trials submitted to the FDA were interpreted by the FDA reviewer as positive. Post FDAAA, we found that 91% of trials submitted to the FDA were positive, 97% of which were published; and among those characterized by the FDA as equivocal or negative, 29% were either unpublished or published in a way that conveyed positive findings. The published literature gives the appearance that 93% of the trials are positive, similar to the 91% interpreted as positive by the FDA reviewer.  Registration within ClinicalTrials.gov improved after implementation of the FDAAA, such that nearly all of the trials submitted to support FDA approval that were subject to the 2007 FDAAA regulations were registered on ClinicalTrials.gov, but only half have reported results thus far [14]. These findings are slightly more encouraging than prior research, which demonstrated that only 41.5% of trials subject to the FDAAA reported results within 5 years of study completion [15]. The improvement in reporting of results may be due to increasing familiarity with the process among clinical trialists, increasingly ubiquitous registration requirements for journal publication, and overall culture change regarding clinical trial transparency.
There are several limitations to our study. First, we only looked at trials supporting FDA approval of drugs used for the treatment of cardiovascular disease and diabetes. Our results may not be generalizable to all clinical areas, because prior research has shown that publication rates for pivotal trials supporting cardiovascular disease and diabetes approvals are lower than in other therapeutic areas [3], raising the possibility that there was a greater opportunity for improvement. Our findings should be confirmed for FDA approvals in other therapeutic areas. Second, our analysis was focused on trials providing efficacy data to support FDA approval and is not generalizable to trials focused on safety. Third, we split our observation period into two periods based on the adoption date of FDAAA, but there was likely a more gradual improvement than that implied by our analysis, and other factors, such as improved awareness and knowledge of clinical trial requirements over time among both trialists and journal editors, may have contributed to the improvement in registration, publication, and concordance rates. Fourth, we did not determine whether the results reported on ClinicalTrials.gov were concordant with the results presented in the FDA documents and published literature; this was outside the scope of this review. Fifth, for trials determined to be unregistered or unpublished we did not contact sponsor companies for confirmation. Finally, because our study was observational in nature, we are unable to draw causal conclusions about the impact of the FDAAA on trial registration, publication, and interpretation concordance.

Conclusions
The 2007 FDAAA was associated with higher rates of clinical trial registration on ClinicalTrials.gov, publication, and concordance between interpretation of findings in the FDA documents and published literature. However, many older trials remain unpublished or published with a more positive conclusion than that of the FDA review, suggesting a clear need to improve the historical integrity of the clinical research enterprise. Further efforts to improve accessibility of older and negative clinical trials, such as online FDA resources, within drug labels, and negative trial journals or repositories, are needed to ensure complete and timely dissemination of clinical research.

Additional files
Additional file 1: PRISMA checklist. (DOC 63 kb) Additional file 2: Table S1. Presenting the clinical trials supporting FDA approvals of drugs in cardiovascular disease and diabetes that were published in the biomedical literature in a manner that was discordant with the FDA reviewer's interpretation. (DOCX 32 kb) Abbreviations FDA: Food and Drug Administration; FDAAA: Food and Drug Administration Amendment Act John Arnold Foundation to better understand clinical research integrity and transparency. HMK reports that he chairs a scientific advisory board for UnitedHealthcare. ATP and CXZ declare that they have no competing interests.

Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Author details