Skip to main content
Download PDF

Registration, results reporting, and publication bias of clinical trials supporting FDA approval of neuropsychiatric drugs before and after FDAAA: a retrospective cohort study

Trials volume 19, Article number: 581 (2018) Cite this article

Abstract

Background

Mandatory trial registration, and later results reporting, were proposed to mitigate selective clinical trial publication and outcome reporting. The Food and Drug Administration (FDA) Amendments Act (FDAAA) was enacted by Congress on September 27, 2007, requiring the registration of all non-phase I clinical trials involving FDA-regulated medical interventions and results reporting for approved drugs. The association between FDAAA enactment and the registration, results reporting, and publication bias of neuropsychiatric trials has not been studied.

Methods

We conducted a retrospective cohort study of all efficacy trials supporting FDA new drug approvals between 2005 to 2014 for neuropsychiatric indications. Trials were categorized as pre- or post-FDAAA based on initiation and/or completion dates. The main outcomes were the proportions of trials registered and reporting results in ClinicalTrials.gov, and the degree of publication bias, estimated using the relative risks pre- and post-FDAAA of both the publication of positive vs non-positive trials, as well as of publication of positive vs non-positive trials without misleading interpretations. Registration and results reporting proportions were compared pre- and post-FDAAA using the two-tailed Fisher exact test, and the degrees of publication bias were compared by calculating the ratio of relative risks (RRR) for each period.

Results

The FDA approved 37 new drugs for neuropsychiatric indications between 2005 and 2014 on the basis of 142 efficacy trials, of which 101 were pre-FDAAA and 41 post-FDAAA. Post-FDAAA trials were significantly more likely to be registered (100% vs 64%; p < 0.001) and report results (100% vs 10%; p < 0.001) than pre-FDAAA trials. Pre-FDAAA, positive trials were more likely to be published (relative risk [RR] = 1.52; 95% confidence interval [CI] = 1.17–1.99; p = 0.002) and published without misleading interpretations (RR = 2.47; CI = 1.57–3.73; p < 0.001) than those with non-positive results. In contrast, post-FDAAA positive trials were equally likely to have been published (RR = 1; CI = 1–1, p = NA) and published without misleading interpretations (RR = 1.20; CI = 0.84–1.72; p = 0.30). The likelihood of publication bias pre-FDAAA vs post-FDAAA was greater for positive vs non-positive trials (RRR = 1.52; CI = 1.16–1.99; p = 0.002) and for publication without misleading interpretations (RRR = 2.06, CI = 1.17–3.61, p = 0.01).

Conclusions

The enactment of FDAAA was followed by significantly higher proportions of trials that were registered and reporting results on ClinicalTrials.gov and significantly lower degrees of publication bias among trials supporting recent FDA approval of drugs for neuropsychiatric indications.

Peer Review reports

Background

Research suggests that as many as 15–50% of completed clinical studies are never published [1,2,3,4,5,6,7,8,9,10] and that publication bias persists: trials with non-positive results are significantly more likely to remain unpublished than trials with positive results [3, 11,12,13,14,15]. Although under- or non-publication of clinical trials is a problem ubiquitous in medicine and science, many considered the problems to be particularly severe among clinical studies in neurology and psychiatry [16]. Clinical studies supporting approved drugs for neuropsychiatric indications, such as paroxetine (Paxil) [17], reboxetine (Edronax) [18], gabapentin (Neurontin) [19], and lamotrigine (Lamictal) [20], have been identified as being subject to underreporting. Data demonstrating these drugs to be potentially ineffective for approved indications or suggesting harm were not publicly disclosed until the pharmaceutical companies’ internal documents were reviewed during legal proceedings [16, 21, 22]. Similarly, among trials evaluating drugs indicated for depression [7], anxiety [23], and psychotic disorders [24] that were first approved by the US Food and Drug Administration (FDA) in recent decades, 80–90% of trials with negative or equivocal results were either not published or were published in a misleading manner to suggest a positive result, while nearly 100% of trials with positive results were published.

A publicly accessible, centralized trial registry can prevent clinical trials from remaining hidden by allowing interested individuals, including patients, clinicians, and regulators, to identify all trials relevant to a specific medical condition or to a specific medical product. In 1997, Congress passed the FDA Modernization Act (FDAMA), which mandated the first USA-based public registry ClinicalTrials.gov [25] in 2000, maintained by the National Institutes of Health (NIH) [26]. In 2005, the International Committee of Medical Journal Editors (ICMJE) issued a policy requiring trial registration as a condition of publication in member journals [27], which was followed by an increase in registered trials [28,29,30,31]. Nonetheless, the FDAMA only required registration of a small number of trials, while the ICMJE recommendation was only followed on a voluntary basis and still permitted publication of unregistered trials [32, 33].

In 2007, Congress passed the FDA Amendments Act (FDAAA) [34], applicable to essentially all non-phase I interventional studies involving FDA-regulated drugs, biological products, or devices. The FDAAA mandated that sponsors and investigators register all such trials in ClinicalTrials.gov prior to subject enrollment and report results to ClinicalTrials.gov within 30 days post approval of the indication being studied. The FDAAA is applicable to trials that began after September 27, 2007 and to earlier trials that were still ongoing as of December 26, 2007. Inappropriately delayed registration and results reporting, as well as reporting of false results, are punishable by fines of up to $10,000 per day and can lead to withholding of funding from studies receiving federal support.

It has now been ten years since the FDAAA was enacted. Its impact on clinical trial registration, results reporting, and publication bias has largely remained undetermined. One recent study demonstrated that the FDAAA was associated with increased registration and publication of clinical studies of new drugs approved to treat cardiovascular disease and diabetes [35]. However, no study has focused on trials involving drugs treating neurological and psychiatric conditions, an area in which concern for selective publication and outcome reporting remains.

The objective of our study was to compare the proportions of trials that were registered and reporting results on ClinicalTrials.gov, as well as the degree of publication bias, before and after the FDAAA, among clinical trials supporting FDA approval of drugs for neuropsychiatric indications. We designed a retrospective cohort study using a cohort of clinical trials that were submitted to and reviewed by the FDA for the approval of New Drug Applications (NDAs) between 2005 and 2014 for the treatment of neurologic and psychiatric conditions. For each trial, we determined their FDAAA eligibility based on trial initiation and/or completion dates [34], as well as their registration, results reporting, and publication status.

Methods

Data sources

Data were obtained from three sources: Drugs@FDA [36], ClinicalTrials.gov, and PubMed’s listing of MEDLINE-indexed journals. Drugs@FDA is a public database maintained by the FDA, providing access to regulatory actions and documents issued for each drug approved by the agency. ClinicalTrials.gov is a public clinical trial registry database maintained by the National Library of Medicine at the NIH [25]. PubMed’s list of MEDLINE-indexed journals includes more than 5500 biomedical journals.

Novel therapeutics approved for treating neurological and psychiatric disorders, 2005–2014

The Center of Drug Evaluation and Research, which is part of the FDA, provides annual reports summarizing all NDAs approved in each year [36, 37]. We downloaded the reports from 2005 to 2014, when available, and otherwise searched Drugs@FDA for those NDAs that were approved to treat neurologic and psychiatric disorders. Our study sample began with drugs approved in 2005 to align with our prior work [38] and because an earlier seminal study on the topic examined all antidepressants approved through 2004 [7]; we chose to exclude drugs approved after December 2014 to ensure that at least 24 months had passed between drug approval date and the date when we concluded the final search for the registration record, reported results, and publication, which was March 2017. For each NDA, we recorded its indication, orphan status, priority review status, accelerated approval status, sponsor, and approval date.

Efficacy trials supporting FDA new neuropsychiatric drug approval

As described in a comprehensive tutorial for how to use Drugs@FDA [39], we downloaded the relevant FDA files for each NDA from Drugs@FDA, including the approval letters, summary reviews, clinical reviews, and statistical reviews. Among these files, we searched for clinical trials evaluating the efficacy of the drugs under review. We included only trials for which the FDA discussed and characterized results, based on the assumption that these trials influenced the FDA’s decision to approve the study drug for the proposed indication. We excluded ongoing trials, phase I/safety-only trials, expanded access trials, terminated and withdrawn trials without enrollment, and trials evaluating indications different than that for which the drugs were originally approved. We also excluded failed trials. For each included trial, we recorded the following characteristics: pivotal status, phases, sponsors, study sites, trial length, randomization, blinding, types of control, description of the treatments, arms of the investigational drugs, enrollment numbers, and the primary efficacy endpoints. A pivotal study is defined by the FDA as “a definitive study in which evidence is gathered to support the safety and effectiveness evaluation of the medical product for its intended use” [40]. Pivotal status was frequently assigned prospectively by the FDA, occasionally assigned retrospectively by the FDA, or at times not assigned by the FDA and thus determined using a previously described method [38].

Determination of FDAAA status

The FDAAA, as enacted in 2007, clarified that new requirements would apply to trials that were initiated after September 27, 2007, as well as to trials initiated earlier but still ongoing as of December 26, 2007. Based on this, FDAAA applicable trials were categorized as post-FDAAA, while trials that were initiated or completed prior to the cut-off dates were categorized as pre-FDAAA.

Determination of registration and results reporting status on ClinicalTrials.gov

To determine whether trials were registered and reported results on ClinicalTrials.gov, one investigator (CXZ) performed the initial search using the following terms and their combinations: generic, or brand names of the study drugs, drug indications, trial IDs, trial acronyms, numbers of participants randomized, comparators, and study time frames. For trials that were not able to be matched with any registration record, a second investigator (JEB) independently performed a second round of searches. No new records were identified.

Determination of publication status

To determine whether trials were published, we searched PubMed for full-length publications using the same terms as we did for the registration record. Among identified publications, abstracts and conference reports were excluded. Publications reporting multiple trials, such as reviews and meta-analyses, were also excluded unless the results of each trial were analyzed and discussed individually in the level of detail as one would expect from a full-length publication. When the search terms returned too many similar entries in PubMed, we used Google Scholar to narrow the results. Google Scholar has the advantage that it can search among the full texts of publications hosted by a variety of online databases or platforms, while for many journals, especially those that require paid access, PubMed searches only among the titles and abstracts. We provide more detail on our search strategy in Additional file 1.

Interpretation of trial results: publication vs FDA

Trials were classified as positive, negative, or equivocal based on the FDA’s interpretation of the results as described in Additional file 1. The classification was based on whether the primary outcome(s) achieved statistical significance while taking into consideration the summary statements made by the FDA medical reviewers regarding whether or not the findings provided support for the efficacy claim of the study drugs. Published trial results were categorized similarly based on whether the primary outcomes achieved statistical significance according to the authors’ analysis while taking into consideration the authors’ conclusions in the abstract section. Trials with equivocal or negative results were grouped together as non-positive trials for purposes of calculating publication bias.

Validating the published interpretations

We validated the interpretations of the trial results made by the study investigators for each publication using the interpretations made by the FDA medical reviewers found in the FDA approval package as the gold standard. Both the conclusions in the abstract and the main text of the publications were validated. The two were considered in agreement if the interpretations were both categorized as positive, negative, or equivocal, and no major contradictions existed between the two statements. As an example of contradiction between two sources: the published interpretation of trial 02 of milnacipran (Savella) concluded that “both doses (100 and 200 mg/d) were associated with significant improvements in pain and other symptoms” [41]. This was considered different from the statement made by the FDA in the summary review documents, which stated that “[the] analysis of the ‘pain only’ responders does not indicate that there is a significant effect of MLN (Savella) on pain….(treatment effect) was driven by the patient global response outcome rather than the pain or function outcome…when studied in isolation, statistically significant treatment effects for pain and function were not demonstrated” [42]. Due to the interpretive nature of this comparison, two additional investigators (JEB and JSR) reviewed all instances where there was disagreement between the FDA’s and the publications interpretation.

Calculating the degree of publication bias

We calculated and compared two different measures of publication bias between pre- and post-FDAAA trials. First, we estimated the relative risk of publishing positive vs non-positive trials in each period. Second, we estimated the relative risk of publishing positive vs non-positive trials without misleading interpretations in each period. Thus, publication bias was calculated as the ratio of relative risks (RRR) pre-FDAAA vs post-FDAAA.

Data validation

Registration status, results reporting status, publication status, and publication-FDA interpretation agreement were validated as described previously. We performed quality control and data validation, having a second investiator (JEB) re-collect all data but not reported for purposes of this study. A second investigator (JEB) re-collected all data elements obtained for a random 10% sample of the included new drug approvals, using an online randomization tool [43] to randomly select 4 out of the 37 drugs. Among the 676 unique data elements collected by the two investigators, the rate of agreement was 99.6%, and disagreements were resolved through consensus.

Data analysis

We used descriptive statistics to characterize the proportions of trials that were registered and reporting results on ClinicalTrials.gov. We used two-tailed Fisher exact tests to compare the proportions among pre- and post-FDAAA trials. Analysis was performed using Epi Info Companion App for iOS version 3.1.1 (Centers for Disease Control and Prevention [CDC]; Atlanta, GA) [44], as well as MedCalc online statistical software [45], supplemented using an online program written by Hutchon [46] to calculate the RRRs to estimate both measures of publication bias.

Results

Characteristics of the neuropsychiatric drugs approved between 2005 and 2014

Between January 1, 2005 and December 31, 2014, 37 new drugs were approved by the FDA for the treatment of neuropsychiatric conditions, of which 23 (62%) were approved for neurological conditions and 14 (38%) for psychiatric disorders, which included 3 drugs for substance-use related conditions (Table 1). Among the 37 approved drugs, 34 (92%) were pharmacologic therapies, 3 (8%) were biologics; orphan status was granted for 9 (24%), priority review status for 6 (17%), and accelerated approval for 1 (3%) (see Additional file 2).

Table 1 New Drug Applications (NDAs) Approved by the FDA between 2005 and 2014 with indications for neurologic and psychiatric conditions
Full size table

Clinical trials supporting FDA approval

There were 142 trials that supported the approval of these 37 neuropsychiatric drugs (Fig. 1), of which 101 (71%) were categorized as pre-FDAAA and 41 (29%) as post-FDAAA. All 142 trials were funded by industry and 105 (74%) were phase III, 33 (23%) phase II, and 4 (3%) phase II/III. In addition, the results of 107 (75%) of the trials were interpreted by the FDA to be positive, 17 (12%) as equivocal, and 18 (13%) as negative (Table 2).

Fig. 1
figure 1

Identification of trials reviewed by the FDA for New Drug Applications with neuropsychiatric indications, 2005–2014. Flow chart depicting the process of selecting efficacy trials conducted to support new neuropsychiatric drugs and reviewed by the FDA for drug approval between 2005 and 2014. Trials can be excluded for satisfying one or more exclusion criteria

Full size image
Table 2 Characteristics of 142 efficacy trials supporting FDA approval of NDAs for neuropsychiatric conditions, 2005–2014
Full size table

Clinical trial registration and results reporting

The FDAAA was followed by significantly greater proportions of trial registration and results reporting. Pre-FDAAA, 64% (65 of 101) of clinical trials were registered on ClinicalTrials.gov, while 100% (41 of 41) of post-FDAAA trials were registered (p < 0.001; Fig. 2). Similarly, pre-FDAAA, 10% (10 of 101) of clinical trials reported results on ClinicalTrials.gov, while 100% (41 of 41) of post-FDAAA trials reported results (p < 0.001; Fig. 2); the results of 32 of 41 (78%) FDAAA trials were reported within 30 days of drug approval.

Fig. 2
figure 2

Registration and results reporting status of trials supporting FDA indications by FDAAA applicability, 2005–2014. Post-FDAAA trials were significantly more likely to be registered (100% vs 64%; p < 0.001) and report results (100% vs 10%; p < 0.001) than pre-FDAAA. Outcomes were compared by two-tailed Fisher exact tests

Full size image

Publication and published interpretations

Pre-FDAAA, among 72 positive trials, none were unpublished or published with misleading interpretation (Table 3). In contrast, among 29 non-positive trials, 10 were not published and 7 were published with misleading interpretations (Fig. 3). Post-FDAAA, among 35 positive trials, again none were unpublished and none were published with a misleading interpretation. In addition, among 6 non-positive trials, none were unpublished and 1 was published with a misleading interpretation. The publications of the following new drugs had misleading interpretations: droxidopa (Northera) of Chelsea Therapeutics, dalfampridine (Ampyra) of Acorda Therapeutics, iloperidone (Fanapt) of Vanda Pharmaceuticals, milnacipran hydrochloride (Savella) of Forest Research and Cypress Bioscience, and rulfinamide (Banzel) of Novartis (Additional file 3).

Table 3 Publication and publication-FDA agreement of trials supporting FDA approval of NDAs with neuropsychiatric indications with positive, equivocal, and negative results
Full size table
Fig. 3
figure 3

Publication status and publication-FDA agreement of neuropsychiatric Trials by FDAAA applicability and by trial results. Overall, more post-FDAAA trials were published (100% vs 90%; p = 0.06) and the publications were in agreement with the FDA’s interpretation (98% vs 93%; p = 0.28), but neither outcome was significant. When stratified by results, trials with positive results were all published during both pre-FDAAA (72 of 72) and post-FDAAA (35 of 35). When trials with negative results were examined in isolation, the publication rate was significantly higher after FDAAA as compared to before (5 of 5 vs 5 of 13; p = 0.04). There were not enough equivocal trials to draw comparison. All comparisons were based on two-tailed Fisher exact tests

Full size image

Publication bias

Pre-FDAAA, positive trials were more likely to be published (relative risk [RR] = 1.52; 95% confidence interval [CI] = 1.17–1.99; p = 0.002) and published without misleading interpretations (RR = 2.47; CI = 1.57–3.73; p < 0.001) than those with non-positive results. In contrast, post-FDAAA, positive trials were equally likely to have been published (RR = 1; CI = 1–1, p = NA) and published without misleading interpretations (RR = 1.20; CI = 0.84–1.72; p = 0.30). The likelihood of publication bias pre-FDAAA vs post-FDAAA was greater for publication of positive vs non-positive trials (RRR = 1.52; CI = 1.16–1.99; p = 0.002) and for publication without misleading interpretations (RRR = 2.06, CI = 1.17–3.61, p = 0.01).

Discussion

In this retrospective cohort study of 142 trials supporting the approval of 37 neuropsychiatric therapeutics approved by the FDA between 2005 and 2014, post-FDAAA trials were uniformly registered, reported results, published, and published without misleading interpretations. As compared to pre-FDAAA trials, proportions of trials that were registered and reporting results on ClinicalTrials.gov were significantly higher and the degree of publication bias was lower. Our results suggest that the FDAAA likely contributed to improving the registration, results reporting, and publication of clinical trials supporting FDA approval of new drugs used to treat neuropsychiatric indications, although other factors may also have been in play.

Prior work examining clinical trials supporting FDA approval of new drugs in 2012, many of which were completed before the FDAAA was enacted, found 57% were registered and 20% reported results on ClinicalTrials.gov [5]. Among trials supporting approval of neuropsychiatric drugs, we found similar rates among pre-FDAAA trials, 63% and 10%, but also showed that, among post-FDAAA trials, 100% were registered and reported results. A study involving trials supporting FDA approval of cardiovascular and diabetic drugs showed a similar association between the FDAAA and trial registration and results reporting [35]. Significantly improved clinical trial registration and results reporting after FDAAA enactment might have been anticipated, given the explicit requirement to require trial registration among all trials investigating FDA-regulated products and the alignment between this requirement and the ICMJE’s trial registration policy. Follow-up studies will be needed to determine whether these improvements persist over time. But over the near term, these findings suggest that the FDAAA contributed to improving selective registration and results reporting of clinical trials supporting FDA approvals.

As discussed previously, earlier studies have consistently demonstrated significant publication bias: positive trials are more likely to be published and published accurately or completely than non-positive trials. In our study, such publication bias was observed only among trials that were completed prior to FDAAA enactment, but not afterwards. In addition, overall rates of clinical trial publication were quite high, challenging the assumption that selective publication is worse among clinical trials of neuropsychiatric drugs than for other types of drugs. It is important to note that the “effective publication rate,” as perceived by clinicians and patients, may be lower because publications for some trials were more difficult to locate and were only found using specific trial data in multiple search engines. It is likely that these more-difficult-to-locate publications have a more limited impact on clinical practice, as practicing clinicians and patients have limited time, information, and training to conduct systemic and comprehensive literature searches.

Our findings have important implications for understanding the impact of the FDAAA and for developing future strategies to improve selective publication and outcome reporting more broadly. The FDAAA applies only to trials of medical products regulated by the FDA. But the practice of medicine includes not only the use of medical products to improve patient outcomes, but also behavioral, surgical, and other procedural interventions, as well as health system interventions. To ensure that the medical literature is as unbiased and representative as is possible, rules and regulations like the FDAAA, which mandate registration and results reporting in a publicly accessible database and that apply to all clinical research and health system studies, may be an effective strategy to promote comprehensive registration, results reporting, and publication.

Several factors should be considered in the interpretation of our findings. First, our study is cross-sectional and can only establish associations, not causality. Other reasons beyond the FDAAA, including academic advocacy for clinical trial data transparency, may have accounted for trial sponsors’ and investigators’ decisions to register, report results, and publish the findings of the clinical trials. More studies are needed to disentangle the impact of the FDAAA on clinical trial registration, results reporting, and publication from other factors that may have contributed to these trends, especially the ICMJE clinical trial registration policy—although it is not clear that this policy would affect either results reporting or publication. Second, we limited our search of trial registration to ClinicalTrials.gov. It is possible that trials conducted pre-FDAAA used other registers. Third, for trials determined to be unregistered or unpublished, we did not contact sponsor companies for confirmation. Fourth, our sample is limited to phase II and III trials supporting neuropsychiatric drugs, and the sample size of post-FDAAA trials, at 41, is relatively small. Fifth, our study was focused on pre-marketing phase II and III trials evaluating neuropsychiatric drugs successfully approved by the FDA. Our findings may not be generalized to phase I and phase IV post marketing trials, trials evaluating drugs that were not approved by the FDA, as well as trials for other types of drugs for which the registration and publication have yet to be characterized. Finally, our study was focused on the reporting and publication of trials’ primary results and did not examine reporting or publication of secondary and safety outcomes.

Conclusions

For clinical trials supporting the FDA approval of new drugs for neuropsychiatric indications, the proportions of trials that were registered and reporting results on ClinicalTrials.gov were significantly higher and publication bias was significantly lower after the passage of the FDAAA in 2007, suggesting that the FDAAA likely contributed to the reduction of selective registration and results reporting and to mitigating publication bias. These findings have important implications for understanding the potential impact of the FDAAA, along with other initiatives that may have improved research reporting, and for developing future strategies to improve selective publication and outcome reporting more broadly.

Abbreviations

FDA:

Food and Drug Administration

FDAAA:

Food and Drug Administration Amendments Act

FDAMA:

Food and Drug Administration Modernization Act

ICMJE:

International Committee of Medical Journal Editors

NDA:

New Drug Application

NIH:

National Institutes of Health

References

  1. Dickersin K, Chalmers I. Recognizing, investigating and dealing with incomplete and biased reporting of clinical research: from Francis Bacon to the WHO. JRSM. 2011;104:532–8.

    Article  Google Scholar 

  2. Ross JS, Tse T, Zarin DA, Xu H, Zhou L, Krumholz HM. Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis. BMJ. 2012;344:d7292.

    Article  Google Scholar 

  3. Decullier E, Huot L, Chapuis FR. Fate of protocols submitted to a French national funding scheme: a cohort study. PLOS One. 2014;9:e99561.

    Article  Google Scholar 

  4. Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010;303:2058–64.

    CAS  Article  Google Scholar 

  5. Miller JE, Korn D, Ross JS. Clinical trial registration, reporting, publication and FDAAA compliance: a cross-sectional analysis and ranking of new drugs approved by the FDA in 2012. BMJ Open. 2015;5:e009758.

    Article  Google Scholar 

  6. Jones CW, Handler L, Crowell KE, Keil LG, Weaver MA, Platts-Mills TF. Non-publication of large randomized clinical trials: cross sectional analysis. BMJ. 2013;347:f6104.

    Article  Google Scholar 

  7. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358:252–60.

    CAS  Article  Google Scholar 

  8. Smithy JW, Downing NS, Ross JS. Publication of pivotal efficacy trials for novel therapeutic agents approved between 2005 and 2011: a cross-sectional study. JAMA Intern Med. 2014;174:1518–20.

    Article  Google Scholar 

  9. Chen R, Desai NR, Ross JS, Zhang W, Chau KH, Wayda B, Murugiah K, Lu DY, Mittal A, Krumholz HM. Publication and reporting of clinical trial results: cross sectional analysis across academic medical centers. BMJ. 2016;352:i637.

    Article  Google Scholar 

  10. Lee K, Bacchetti P, Sim I. Publication of clinical trials supporting successful new drug applications: a literature analysis. PLOS Med. 2008;5:e191.

    Article  Google Scholar 

  11. Dickersin K, Min Y-I, Meinert CL. Factors influencing publication of research results: follow-up of applications submitted to two institutional review boards. JAMA. 1992;267:374–8.

    CAS  Article  Google Scholar 

  12. Suñé P, Suñé JM, Montoro JB. Positive outcomes influence the rate and time to publication, but not the impact factor of publications of clinical trial results. PLOS One. 2013;8:e54583.

    Article  Google Scholar 

  13. Ioannidis JPA. Effectiveness of antidepressants: an evidence myth constructed from a thousand randomized trials? Philos Ethics Humanit Med. 2008;3:14.

    Article  Google Scholar 

  14. Dwan K, Gamble C, Williamson PR, Kirkham JJ, Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias — an updated review. PLOS One. 2013;8:e66844.

    CAS  Article  Google Scholar 

  15. Song F, Parekh-Bhurke S, Hooper L, Loke YK, Ryder JJ, Sutton AJ, Hing CB, Harvey I. Extent of publication bias in different categories of research cohorts: a meta-analysis of empirical studies. BMC Med Res Methodol. 2009;9:79.

    Article  Google Scholar 

  16. Turner EH. Publication bias, with a focus on psychiatry: causes and solutions. CNS Drugs. 2013;27:457–68.

    Article  Google Scholar 

  17. Le Noury J, Nardo JM, Healy D, Jureidini J, Raven M, Tufanaru C, Abi-Jaoude E. Restoring Study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence. BMJ. 2015;351:h4320.

    Article  Google Scholar 

  18. Eyding D, Lelgemann M, Grouven U, Härter M, Kromp M, Kaiser T, Kerekes MF, Gerken M, Wieseler B. Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials. BMJ. 2010;341:c4737.

    Article  Google Scholar 

  19. Vedula SS, Bero L, Scherer RW, Dickersin K. Outcome reporting in industry-sponsored trials of gabapentin for off-label use. 2009;361:1963–71. https://doi.org/10.1056/NEJMsa0906126.

    CAS  Article  Google Scholar 

  20. Nassir Ghaemi S, Shirzadi AA, Filkowski M. Publication bias and the pharmaceutical industry: the case of lamotrigine in bipolar disorder. Medscape J Med. 2008;10:211.

    CAS  PubMed  PubMed Central  Google Scholar 

  21. Doshi P, Dickersin K, Healy D, Vedula SS, Jefferson T. Restoring invisible and abandoned trials: a call for people to publish the findings. BMJ. 2013;346:f2865.

    Article  Google Scholar 

  22. Anderson GM, Juurlink D, Detsky AS. Newly approved does not always mean new and improved. JAMA. 2008;299:1598–600.

    CAS  Article  Google Scholar 

  23. Roest AM, de Jonge P, Williams CD, de Vries YA, Schoevers RA, Turner EH. Reporting bias in clinical trials investigating the efficacy of second-generation antidepressants in the treatment of anxiety disorders: a report of 2 meta-analyses. JAMA Psychiatry. 2015;72:500–10.

    Article  Google Scholar 

  24. Turner EH, Knoepflmacher D, Shapley L. Publication bias in antipsychotic trials: an analysis of efficacy comparing the published literature to the US Food and Drug Administration database. PLOS Med. 2012;9:e1001189.

    Article  Google Scholar 

  25. ClinicalTrials.gov. www.clinicaltrials.gov. Accessed 7 Aug 2018.

  26. Food and Drug Administration Modernization Act of 1997 (FDAMA). In Information program on clincal trials for serious or life-threatening diseases: United States Code. www.revolvy.com/page/Food-and-Drug-Administration-Modernization-Act-of-1997. Accessed 16 Oct 2018.

  27. De Angelis CD, Drazen JM, Frizelle FAP, Haug C, Hoey J, Horton R, Kotzin S, Laine C, Marusic A, Overbeke AJPM, et al. Is this clinical trial fully registered? — A statement from the International Committee of Medical Journal Editors. 2009;352:2436–8. https://doi.org/10.1056/NEJMe058127.

    CAS  Article  Google Scholar 

  28. McGauran N, Wieseler B, Kreis J, Schüler Y-B, Kölsch H, Kaiser T. Reporting bias in medical research — a narrative review. Trials. 2010;11:37.

    Article  Google Scholar 

  29. Chan L, Heinemann AW. Clinical trial registration: the time has come. Am J Occup Ther. 2016;70:7001070010p1–2.

    Article  Google Scholar 

  30. Abaid LN, Grimes DA, Schulz KF. Reducing publication bias of prospective clinical trials through trial registration. Contraception. 2007;76:339–41.

    Article  Google Scholar 

  31. Weibel S, Elia N, Kranke P. The transparent clinical trial: Why we need complete and informative prospective trial registration. Eur J Anaesthesiol (EJA). 2016;33:72–4.

    Article  Google Scholar 

  32. Dal-Re R, Ross JS, Marusic A. Compliance with prospective trial registration guidance remained low in high-impact journals and has implications for primary end point reporting. J Clin Epidemiol. 2016;75:100–7.

    Article  Google Scholar 

  33. Mann E, Nguyen N, Fleischer S, Meyer G. Compliance with trial registration in five core journals of clinical geriatrics: a survey of original publications on randomised controlled trials from 2008 to 2012. Age Ageing. 2014;43:872–6.

    Article  Google Scholar 

  34. Food and Drug Administration Amendments Act (FDAAA) of 2007. In Expand clinical trial registry data bank United States Code. https://www.gpo.gov/fdsys/pkg/PLAW-110publ85/html/PLAW-110publ85.htm.

  35. Phillips AT, Desai NR, Krumholz HM, Zou CX, Miller JE, Ross JS. Association of the FDA Amendment Act with trial registration, publication, and outcome reporting. Trials. 2017;18:333.

    Article  Google Scholar 

  36. Drugs@FDA: FDA approved drug products. https://www.accessdata.fda.gov/scripts/cder/daf/. Accessed 7 Aug 2018.

  37. Drugs@FDA: FDA approved drug products: CDER’s new molecular entities and new therapeutic biological products. https://www.fda.gov/drugs/developmentapprovalprocess/druginnovation/default.htm. Accessed 7 Aug 2018.

  38. Downing NS, Aminawung JA, Shah ND, Krumholz HM, Ross JS. Clinical trial evidence supporting FDA approval of novel therapeutic agents, 2005-2012. JAMA. 2014;311:368–77.

    CAS  Article  Google Scholar 

  39. Turner EH. How to access and process FDA drug approval packages for use in research. BMJ. 2013;347:f5992.

    Article  Google Scholar 

  40. FDA. Design considerations for pivotal clinical investigations for medical devices. Guidance for industry, clinical investigators, institutional review boards and Food and Drug Administration staff. Silver Spring: FDA Center for Biologics Evaluation and Research; 2013.

    Google Scholar 

  41. Clauw DJ, Mease P, Palmer RH, Gendreau RM, Wang Y. Milnacipran for the treatment of fibromyalgia in adults: a 15-week, multicenter, randomized, double-blind, placebo-controlled, multiple-dose clinical trial. Clin Ther. 2008;30:1988–2004.

    CAS  Article  Google Scholar 

  42. Summary Review for Regulatory Action for Application Number 22-256. FDA Center for Drug Evaluation and Research Division of Anesthesia, Analgesia, and Rheumatology Products. pp. 11: FDA Center for Drug Evaluation and Research Division of Anesthesia, Analgesia, and Rheumatology Products. 2008;11. https://www.accessdata.fda.gov/drugsatfda_docs/summary_review/2009/022256s000SumR.pdf.

  43. Random.org. Randomness and Integrity Services Ltd. https://www.random.org/lists/. Accessed 7 Aug 2018.

  44. Centers for Disease Control and Prevention (CDC). Epi Info Companion App for iOS. 3.1.1 edition. Atlanta: Department of Health and Human Services-CDC; 2013.

    Google Scholar 

  45. MedCalc Statistical Software version 16.4.3. MedCalc Software bvba, Ostend, Belgium. 2016. https://www.medcalc.org.

  46. Calculations for comparing two estimated relative risks. http://www.hutchon.net/CompareRR.htm. Accessed 7 Aug 2018.

Download references

Funding

This project was not supported by any external grants or funds. The authors assume full responsibility for the accuracy and completeness of the ideas presented.

Availability of data and materials

The data are available by email; please contact the corresponding author to request additional information.

Author information

Authors and Affiliations

  1. Yale School of Medicine, New Haven, CT, USA

    Constance X. Zou & James M. Garritano

  2. Department of Psychiatry, Massachusetts General Hospital, Boston, MA, USA

    Jessica E. Becker

  3. McLean Hospital, Boston, MA, USA

    Jessica E. Becker

  4. Harvard Medical School, Boston, MA, USA

    Jessica E. Becker

  5. Division of Cardiovascular Medicine, Department of Internal Medicine, Beth Israel Deaconess Medical Center, Boston, MA, USA

    Adam T. Phillips

  6. Section of Cardiovascular Medicine, Department of Internal Medicine, Yale School of Medicine, New Haven, CT, USA

    Harlan M. Krumholz

  7. Department of Health Policy and Management, Yale School of Public Health, New Haven, CT, USA

    Harlan M. Krumholz & Joseph S. Ross

  8. Center for Outcomes Research and Evaluation, Yale-New Haven Hospital, New Haven, CT, USA

    Harlan M. Krumholz & Joseph S. Ross

  9. Section of General Medicine, Department of Internal Medicine, Yale School of Medicine, PO Box 208093, New Haven, CT, 06520, USA

    Jennifer E. Miller & Joseph S. Ross

Authors
  1. Constance X. Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jessica E. Becker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Adam T. Phillips
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. James M. Garritano
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Harlan M. Krumholz
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jennifer E. Miller
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Joseph S. Ross
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

CXZ, JEB, ATP, and JSR conceived and designed the study. CXZ, JEB, and JMG acquired the data. CXZ and JSR performed the statistical analysis. All authors analyzed and interpreted the data. CXZ and JSR drafted the manuscript. All authors critically revised the manuscript for important intellectual content. JSR performed supervision. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Joseph S. Ross.

Ethics declarations

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Competing interests

In the past 36 months, Ms. Zou received a fellowship through the Yale School of Medicine from the National Heart, Lung, and Blood Institute; Drs. Miller and Ross received research support through New York University from the Laura and John Arnold Foundation to support the Good Pharma Scorecard; Drs. Krumholz and Ross received research support through Yale University from Johnson & Johnson to develop methods of clinical trial data sharing, from Medtronic and the US Food and Drug Administration (FDA) to develop methods for postmarket surveillance of medical devices, and from the US Centers of Medicare and Medicaid Services (CMS) to develop and maintain performance measures that are used for public reporting; Dr. Krumholz also received compensation as a member of the Scientific Advisory Board for United Healthcare; and Dr. Ross also received research support through Yale University from the FDA to establish a Center for Excellence in Regulatory Science and Innovation (CERSI) at Yale University and the Mayo Clinic, from the Blue Cross Blue Shield Association to better understand medical technology evaluation, and from the Laura and John Arnold Foundation to support the Collaboration for Research Integrity and Transparency at Yale University. James Garritano is supported by the NIH Medical Scientist Training Program Training Grant T32GM007205. The remaining authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Additional information for the Methods section. Details on how the trial results were categorized as positive, equivocal, or negative and on the search strategy for publications. (DOCX 17 kb)

Additional file 2:

Characteristics of New Drug Applications approved by the FDA between 2005 and 2014 for the treatment of neurologic and psychiatric conditions (n = 37). (DOCX 15 kb)

Additional file 3:

List of trials supporting FDA NDA approval with neuropsychiatric indications published with interpretations not in agreement with the FDA’s. For each trial published with interpretations that disagree with those of the FDA, we provided the FDAAA status, trial funding source, FDA interpretation of trial results, quote of the FDA conclusion, publication interpretation, and quote of the published conclusion. (DOCX 35 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zou, C.X., Becker, J.E., Phillips, A.T. et al. Registration, results reporting, and publication bias of clinical trials supporting FDA approval of neuropsychiatric drugs before and after FDAAA: a retrospective cohort study. Trials 19, 581 (2018). https://doi.org/10.1186/s13063-018-2957-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13063-018-2957-0

Keywords

  • FDAAA
  • ClinicalTrials.gov
  • Clinical trial registration
  • Clinical trial results
  • Selective publication
  • Publication bias

Trials

ISSN: 1745-6215