Table 6 Timeline of Annals’ responses to COMPare

October–December 2015: Following submission, all COMPare letters were accepted as online comments only. Reading these requires registration for an Annals user account.

14 December 2015: Annals editors published an 850-word critique of COMPare as an online comment on the CASCADE trial, which later appeared as a full page article in print in the March 1 edition and as a standalone online letter. This piece has no named authors. It contained various incorrect and internally inconsistent statements on outcome pre-specification and reporting, as documented in Table 3 and online [20]. Annals declined to publish a response from COMPare in print or below the standalone online letter (Trial 5, Annals, 14/12/15). In their critique, Annals stated that they prefer to use protocols over trial registries and that registries often contain “outdated, vague or erroneous entries” (Table 3). However, pre-trial protocols were not available for any of the Annals trials assessed by COMPare. From February to April 2016, the official Annals social media account claimed, incorrectly, to have fully published COMPare correspondence on four occasions [Annals tweets] after their non-publication of COMPare responses was reported elsewhere [21].

14–30 December 2015: Annals editors posted an identical comment beneath four out of five COMPare comments on misreported trials in Annals: “we caution readers and the research community against considering COMPare’s assessments as an accurate reflection of the quality of the conduct or reporting of clinical trials... we do not believe that COMPare’s comments on [trial] merit a response”. In our view, this conflicts with ICMJE guidelines: “The authors of articles discussed in correspondence ... have a responsibility to respond to substantial criticisms of their work using those same mechanisms and should be asked by editors to respond” [22]. Following this comment from Annals editors, no authors engaged on the issue of whether they had reported their pre-specified outcomes. In March 2016, Annals clarified their comments, asserting that their comment was not intended to dissuade authors from replying to COMPare comments. No trial authors have replied on the concerns raised about their outcome reporting since Annals’ initial comment.

1 March 2016: Two COMPare comments were published in print in Annals, with responses from an author and the editors (mentioned above). Both of these responses contained further errors and misunderstandings in relation to the outcomes published in the trial report; Annals declined to publish subsequent COMPare correspondence pointing out these issues.

18 March 2016: Following Annals editors stating that protocols should be used to assess outcome reporting fidelity in preference to registry entries, COMPare requested the protocol for Trial 45 (Everson et al.) from the lead author: this protocol was not published, but the “Reproducible Research Statement” in Annals stated that it was available on request. We received a reply from Gilead sciences, stating that the protocol is confidential [Everson emails, Annals]. COMPare raised concerns about this in a further online comment to Annals [Trial 45, Annals, 19/04/16]; Annals subsequently issued a correction to the Reproducible Research Statement.

19 April 2016: Annals changed its “Instructions to Authors” to require submission of protocol for subsequent publication alongside all trials in the future. Annals told journalists that this change was planned and predated COMPare’s concerns [23].