Registration, results reporting, and publication bias of clinical trials supporting FDA approval of neuropsychiatric drugs before and after FDAAA: a retrospective cohort study

Research suggests that as many as 15–50% of completed clinical studies are never published [1–10] and that publication bias persists: trials with non-positive results are significantly more likely to remain unpublished than trials with positive results [3, 11–15]. Although under- or non-publication of clinical trials is a problem ubiquitous in medicine and science, many considered the problems to be particularly severe among clinical studies in neurology and psychiatry [16]. Clinical studies supporting approved drugs for neuropsychiatric indications, such as paroxetine (Paxil) [17], reboxetine (Edronax) [18], gabapentin (Neurontin) [19], and lamotrigine (Lamictal) [20], have been identified as being subject to underreporting. Data demonstrating these drugs to be potentially ineffective for approved indications or suggesting harm were not publicly disclosed until the pharmaceutical companies’ internal documents were reviewed during legal proceedings [16, 21, 22]. Similarly, among trials evaluating drugs indicated for depression [7], anxiety [23], and psychotic disorders [24] that were first approved by the US Food and Drug Administration (FDA) in recent decades, 80–90% of trials with negative or equivocal results were either not published or were published in a misleading manner to suggest a positive result, while nearly 100% of trials with positive results were published.

A publicly accessible, centralized trial registry can prevent clinical trials from remaining hidden by allowing interested individuals, including patients, clinicians, and regulators, to identify all trials relevant to a specific medical condition or to a specific medical product. In 1997, Congress passed the FDA Modernization Act (FDAMA), which mandated the first USA-based public registry ClinicalTrials.gov [25] in 2000, maintained by the National Institutes of Health (NIH) [26]. In 2005, the International Committee of Medical Journal Editors (ICMJE) issued a policy requiring trial registration as a condition of publication in member journals [27], which was followed by an increase in registered trials [28–31]. Nonetheless, the FDAMA only required registration of a small number of trials, while the ICMJE recommendation was only followed on a voluntary basis and still permitted publication of unregistered trials [32, 33].

In 2007, Congress passed the FDA Amendments Act (FDAAA) [34], applicable to essentially all non-phase I interventional studies involving FDA-regulated drugs, biological products, or devices. The FDAAA mandated that sponsors and investigators register all such trials in ClinicalTrials.gov prior to subject enrollment and report results to ClinicalTrials.gov within 30 days post approval of the indication being studied. The FDAAA is applicable to trials that began after September 27, 2007 and to earlier trials that were still ongoing as of December 26, 2007. Inappropriately delayed registration and results reporting, as well as reporting of false results, are punishable by fines of up to $10,000 per day and can lead to withholding of funding from studies receiving federal support.

It has now been ten years since the FDAAA was enacted. Its impact on clinical trial registration, results reporting, and publication bias has largely remained undetermined. One recent study demonstrated that the FDAAA was associated with increased registration and publication of clinical studies of new drugs approved to treat cardiovascular disease and diabetes [35]. However, no study has focused on trials involving drugs treating neurological and psychiatric conditions, an area in which concern for selective publication and outcome reporting remains.

The objective of our study was to compare the proportions of trials that were registered and reporting results on ClinicalTrials.gov, as well as the degree of publication bias, before and after the FDAAA, among clinical trials supporting FDA approval of drugs for neuropsychiatric indications. We designed a retrospective cohort study using a cohort of clinical trials that were submitted to and reviewed by the FDA for the approval of New Drug Applications (NDAs) between 2005 and 2014 for the treatment of neurologic and psychiatric conditions. For each trial, we determined their FDAAA eligibility based on trial initiation and/or completion dates [34], as well as their registration, results reporting, and publication status.

In this retrospective cohort study of 142 trials supporting the approval of 37 neuropsychiatric therapeutics approved by the FDA between 2005 and 2014, post-FDAAA trials were uniformly registered, reported results, published, and published without misleading interpretations. As compared to pre-FDAAA trials, proportions of trials that were registered and reporting results on ClinicalTrials.gov were significantly higher and the degree of publication bias was lower. Our results suggest that the FDAAA likely contributed to improving the registration, results reporting, and publication of clinical trials supporting FDA approval of new drugs used to treat neuropsychiatric indications, although other factors may also have been in play.

Prior work examining clinical trials supporting FDA approval of new drugs in 2012, many of which were completed before the FDAAA was enacted, found 57% were registered and 20% reported results on ClinicalTrials.gov [5]. Among trials supporting approval of neuropsychiatric drugs, we found similar rates among pre-FDAAA trials, 63% and 10%, but also showed that, among post-FDAAA trials, 100% were registered and reported results. A study involving trials supporting FDA approval of cardiovascular and diabetic drugs showed a similar association between the FDAAA and trial registration and results reporting [35]. Significantly improved clinical trial registration and results reporting after FDAAA enactment might have been anticipated, given the explicit requirement to require trial registration among all trials investigating FDA-regulated products and the alignment between this requirement and the ICMJE’s trial registration policy. Follow-up studies will be needed to determine whether these improvements persist over time. But over the near term, these findings suggest that the FDAAA contributed to improving selective registration and results reporting of clinical trials supporting FDA approvals.

As discussed previously, earlier studies have consistently demonstrated significant publication bias: positive trials are more likely to be published and published accurately or completely than non-positive trials. In our study, such publication bias was observed only among trials that were completed prior to FDAAA enactment, but not afterwards. In addition, overall rates of clinical trial publication were quite high, challenging the assumption that selective publication is worse among clinical trials of neuropsychiatric drugs than for other types of drugs. It is important to note that the “effective publication rate,” as perceived by clinicians and patients, may be lower because publications for some trials were more difficult to locate and were only found using specific trial data in multiple search engines. It is likely that these more-difficult-to-locate publications have a more limited impact on clinical practice, as practicing clinicians and patients have limited time, information, and training to conduct systemic and comprehensive literature searches.

Our findings have important implications for understanding the impact of the FDAAA and for developing future strategies to improve selective publication and outcome reporting more broadly. The FDAAA applies only to trials of medical products regulated by the FDA. But the practice of medicine includes not only the use of medical products to improve patient outcomes, but also behavioral, surgical, and other procedural interventions, as well as health system interventions. To ensure that the medical literature is as unbiased and representative as is possible, rules and regulations like the FDAAA, which mandate registration and results reporting in a publicly accessible database and that apply to all clinical research and health system studies, may be an effective strategy to promote comprehensive registration, results reporting, and publication.

Several factors should be considered in the interpretation of our findings. First, our study is cross-sectional and can only establish associations, not causality. Other reasons beyond the FDAAA, including academic advocacy for clinical trial data transparency, may have accounted for trial sponsors’ and investigators’ decisions to register, report results, and publish the findings of the clinical trials. More studies are needed to disentangle the impact of the FDAAA on clinical trial registration, results reporting, and publication from other factors that may have contributed to these trends, especially the ICMJE clinical trial registration policy—although it is not clear that this policy would affect either results reporting or publication. Second, we limited our search of trial registration to ClinicalTrials.gov. It is possible that trials conducted pre-FDAAA used other registers. Third, for trials determined to be unregistered or unpublished, we did not contact sponsor companies for confirmation. Fourth, our sample is limited to phase II and III trials supporting neuropsychiatric drugs, and the sample size of post-FDAAA trials, at 41, is relatively small. Fifth, our study was focused on pre-marketing phase II and III trials evaluating neuropsychiatric drugs successfully approved by the FDA. Our findings may not be generalized to phase I and phase IV post marketing trials, trials evaluating drugs that were not approved by the FDA, as well as trials for other types of drugs for which the registration and publication have yet to be characterized. Finally, our study was focused on the reporting and publication of trials’ primary results and did not examine reporting or publication of secondary and safety outcomes.

For clinical trials supporting the FDA approval of new drugs for neuropsychiatric indications, the proportions of trials that were registered and reporting results on ClinicalTrials.gov were significantly higher and publication bias was significantly lower after the passage of the FDAAA in 2007, suggesting that the FDAAA likely contributed to the reduction of selective registration and results reporting and to mitigating publication bias. These findings have important implications for understanding the potential impact of the FDAAA, along with other initiatives that may have improved research reporting, and for developing future strategies to improve selective publication and outcome reporting more broadly.