Table 4 Errors in New England Journal of Medicine responses on trial 22

“[The criticism by COMPare that] AEs leading to discontinuation [were] not correctly reported... is false. Protocol indicates safety and tolerability as second of 2 primary objectives, and registration lists incidence of AEs leading to discontinuation as 1 of 2 primary outcome measures. First line of Table 3 and first sentence of Safety section (p. 2604) reports that 1 of 624 patient treated with sofosbuvir­-velpatasvir discontinued due to AE” (NEJM first comments on trial 22 (1)).

NEJM are incorrect. The outcome in question was pre-specified as a primary outcome but incorrectly reported by NEJM as a secondary outcome. COMPare therefore coded it as reported, but incorrectly reported. This is clearly denoted in the COMPare assessment sheet for this trial, and the COMPare letter reads, “There were 2 pre­specified primary outcomes, of which one is reported in the paper; while one is incorrectly reported as a secondary outcome”.

“[The criticism by COMPare that] Secondary outcome SVR [was] not reported in publication... is false. This is reported in Table 2. The COMPARE reviewers may not appreciate that SVR4 (sustained virologic response week 4) is equivalent to HCV RNA <15 IU/mL at week 4, which is reported in Table 2. HCV RNA <15 IU/mL is the lower limit of detection of the assay, as indicated in the Table footnote” (NEJM first comments on trial 22 (2)).

This is invalid. COMPare correctly coded this outcome as missing. Table 2 does report HCV RNA <15 IU/mL at “week 4” but this was week 4 during treatment (which was 12 weeks long); SVR4 is sustained virologic response at week 4 post­treatment. Hence, we correctly concluded that HCV RNA <15 IU/mL at week 4 post-treatment (SVR4) was not reported in the publication. It seems that NEJM editors did not realise that SVR4 is 4 weeks post­treatment, rather than the 4th week of treatment, hence their misunderstanding and misreporting of this outcome in NEJM and their error in their review of the letter from COMPare.

“[The criticism by COMPare that] proportion with HCV RNA < LLOQ on treatment [was] not reported in publication… is false. The COMPARE reviewer may not appreciate that “HCV RNA < LLOQ” is equivalent to “HCV RNA < 15 IU/mL”. Table 2 reports HCV RNA < 15 IU/mL during treatment” (NEJM first comments on trial 22 (4)).

This is invalid. The time point for this outcome was given in the registry entry as “up to 8 weeks”, and results were reported in NEJM only for 2 and 4 weeks. We therefore concluded that the pre-specified outcome was not reported. The fact that this discrepancy relates only to the time point is made explicit in the letter submitted by COMPare to NEJM, which states that the outcome “is not reported at the pre-specified timepoint, but is reported at two novel time-points”. Because of variation in clinical presentation over time, and the attendant risk of selective reporting, under CONSORT each separate time point at which an outcome is measured is regarded as a separate outcome.

“[The criticism by COMPare that] HCV RNA change from baseline [was] not reported in publication… is false. The change in HCV RNA from baseline is conveyed by reporting the mean HCV RNA at baseline (Table 1) and the rates of HCV RNA < 15 IU/mL (Table 2). Table S4 [of the trial report manuscript] reports the HCV RNA levels for the 2 patients who virologic failure” (NEJM first comments on trial 22 (5)).

This is invalid and represents a concerning approach to reporting pre-specified outcomes. NEJM suggests that readers calculate the results for a pre-specified outcome themselves. In addition, “HCV RNA change from baseline” cannot be calculated from the numbers reported. Mean baseline HCV RNA is reported. Mean follow-up HCV RNA is not reported. Table 2 reports only the proportion of patients with HCV RNA < LLOQ (undetectably low).

“[The criticism by COMPare that] proportion with virologic failure [was] not reported in publication… is false. This is reported in Table 2 which reports virologic failure during treatment (0 patients) and virologic failure after treatment (1 patient)” (NEJM first comments on trial 22 (6)).

This is invalid. COMPare coded this outcome as “correctly reported”. This is clear on the assessment sheet.