Skip to main content

Empirical evidence for outcome reporting bias in randomized clinical trials of acupuncture: comparison of registered records and subsequent publications

Abstract

Background

Outcome reporting bias has received widespread recognition and been considered to pose two threats to the validity of clinical decision making because they overestimate the effect of treatments or distort the results of trials. However, the problem of outcome-reporting bias has not been systematically studied among randomized clinical trials of acupuncture. Our objectives were to evaluate the consistency between the registered records and subsequent publications with respect to outcomes and other data as well as to determine whether outcome-reporting bias favors significant primary outcomes.

Methods

A systematic search of 15 registries was conducted from their inception to January 2014 to identify randomized clinical trials on acupuncture for which the status was listed as ‘completed.’ The subsequent publications were retrieved by searching PubMed and three Chinese databases. Basic characteristics and the registration information were extracted from the registered records and publications. We performed comparisons regarding primary outcomes and other data between the registered records and subsequent publications to assess the consistency and selective outcome reporting.

Results

Eighty-eight trials on acupuncture with 96 published reports were identified. Only 19.3% (17/88) were registered before the start of the trial, suggesting prospective registration. The trial registration number was unavailable in 36 published reports (37.5%). A comparison of registered and published primary outcomes could be conducted in 71 publications (74.0%), and the inconsistency of the primary outcomes was identified in 45.1% (32 of 71); 71.4% (15 of 21) had a discrepancy that favored statistically significant primary outcomes, while 28.6% (6 of 21) favored nonsignificant primary outcomes. Furthermore, the other inconsistencies between the registry records and subsequent publications involved the inclusion criteria (54.7%), exclusion criteria (47.9%) and controls (22.9%).

Conclusions

We find that prospective registration for randomized clinical trials on acupuncture is insufficient, selective outcome reporting is prevalent, and the change of primary outcomes is intended to favor statistical significance. These discrepancies in outcome reporting may lead to biased and misleading results of randomized clinical trials on acupuncture. To ensure publication of reliable and unbiased results, further promotion and implementation of trial registration are still needed.

Peer Review reports

Background

Outcome reporting bias in a clinical study is defined as the selection of a subset of the original variables recorded on the basis of the significance and direction of the results [1]. It distorts the results of trials and leads to a reduction in the strength of evidence for conducting systematic reviews and developing clinical practice guidelines [2-5].

Prospectively trial registration in international clinical trial registries before participant enrollment is one of the main ways to potentially reduce selective outcome reporting and further outcome reporting bias [6,7]. The International Committee of Medical Journal Editors (ICMJE) announced a policy of requiring investigators to register clinical trials in a clinical trial registry before participant enrollment as a prerequisite for publication in 2004. This policy took effect in 2005 and is applicable to any clinical trial beginning enrollment after 1 July 2005 [8]. The World Health Organization formally established the International Clinical Trials Registry Platform (ICTRP; http://www.who.int/ictrp/en/) in 2004 to push for clinical trial registration. Furthermore, several peer-reviewed journals, such as The Lancet and Trials, publish study protocols to achieve transparency regarding performance and reporting of clinical trials.

Outcome reporting bias has received much attention. Many empirical studies evaluating the consistency concerning outcomes between the protocol or trial registry entry of randomized clinical trials (RCTs) and subsequent publications have demonstrated that outcome reporting bias is prevalent and favors statistically significant results [9-16]. A recent study focusing on RCTs of traditional Chinese medicine (TCM) showed discrepancies were identified in primary outcomes (29%) and safety reporting (28%) when comparing registered records with their subsequent publications [17]. However, the problem of outcome-reporting bias has not been systematically studied among RCTs of acupuncture.

Our objectives were to evaluate the consistency between the registered records and subsequent publications with respect to outcomes and other data as well as to determine whether outcome-reporting bias favors significant primary outcomes.

Methods

Inclusion criteria

We included RCTs on acupuncture alone or in combination with other interventions that were registered in trial registries. The current status was limited to ‘completed’ in the registered records. Acupuncture was defined as a collection of procedures involving penetration of the skin with needles manipulated by the hands or by electrical stimulation to stimulate certain points on the body [18]. We excluded phase 1 trials.

Search and selection of trials

A systematic search of 15 major international trial registries was conducted using the search terms “acupuncture” or “electroacupuncture” or “auriculotherapy” from their inception to January 2014. Trial registries included the Australian New Zealand Clinical Trials Registry (ANZCTR), Brazilian Clinical Trials Registry (ReBec), Chinese Clinical Trial Register (ChiCTR), Clinical Research Information Service (CRiS) public of Korea (KCT), ClinicalTrials.gov, Clinical Trials Registry-India (CTRI), Cuban Public Registry of Clinical Trials (RPCEC), EU Clinical Trials Register (EU-CTR), German Clinical Trials Register (DRKS), International Standard Randomized Controlled Trial Number Register (ISRCTN), Iranian Registry of Clinical Trials (IRCT), Japan Primary Registries Network (JPRN), Pan African Clinical Trial Registry (PACTR), the Netherlands National Trial Register (NTR), and Sri Lanka Clinical Trials Registry (SLCTR). If trials were listed as ‘completed’ in the registered records, the full texts of subsequently published articles were retrieved by searching PubMed without a language limit and three Chinese electronic bibliographic databases including China National Knowledge Infrastructure (http://www.cnki.net), Chinese Biomedical Database (http://sinomed.imicams.ac.cn/index.jsp), and Chinese VIP Information (http://vip.hbdlib.cn/index.asp) using the trial registration number, registered trial title, and investigator name. Identified articles were checked against the target registered records to determine whether they were a correct match. For each trial, all published articles reporting the final results were included. Abstracts and reports of preliminary results were excluded.

Data extraction

Two authors (Su CX, Han M, Ren J, or Li WY) independently extracted data using a standardized, piloted data extraction form. The form was modified based on the previous research [17]. We collected the following information reported in trial registries and published articles: trial design, sample size, participants and diseases/conditions, interventions, controls, primary and secondary outcomes along with the time frame of assessment, inclusion and exclusion criteria, and funding. For each registered trial, we also extracted the date registered, start and end dates of participant enrollment/the trial, sponsors, and country of origin. Publication year and language, journal name, registered number, and P values for all the outcomes were also extracted from each subsequently published report. The 2013 impact factor of journals where registered trials were published was sought. ‘Instructions for authors’ of the included journals was checked for whether the registration of the trial and reporting of the trial’s registration number were required in the published report. The “history of changes” in the registries was tracked to determine whether the registered information had been changed since the initial registration. Conditions were classified according to International Statistical Classification of Diseases and Related Health Problems, 10th Revision (ICD-10) (http://apps.who.int/classifications/icd10/browse/2010/en; last accessed on 27 January 2014).

Data analyses

Methodological quality of registered records and subsequent publications was assessed independently by two authors (Su CX, Ren J, or Li WY) using the Cochrane Collaboration risk of bias tool, which included generation of the allocation sequence, allocation concealment, blinding, incomplete outcome reporting, selective outcome reporting, and other risk of bias [19]. Other risk of bias was assessed based on the estimated sample size, explicit inclusion and exclusion criteria, and the risk of funding bias.

Two authors (Su CX, Ren J, or Li WY) evaluated the consistency and selective outcome reporting regarding the trial design, methodology, and outcomes from the registered records and subsequently published articles. The disagreements were resolved by consensus with a third party (Liu JP). We defined inconsistency as “when the items in the trial registry were not the same as those in the subsequent publications” [17]. If over 20% discrepancy was identified between the sample size in the registered trials and published reports, we considered it inconsistent [19]. Selective outcome reporting was defined as “when the primary outcome specified in the published articles was different, or changed from those defined in the trial registry, or the timing of assessment of the registered and published primary outcomes differed” [20,21].

A discrepancy was considered to favor statistically significant results (P < 0.05) if a new statistically significant primary outcome was described in the published articles or if a nonsignificant primary outcome was defined as a nonprimary outcome in the published articles [10,21].

Data were presented as counts, percentage and frequency for categorical variables. Proportions were compared by the χ2 test. P < 0.05 was considered statistically significant. SAS version 9.1 (SAS Institute Inc, Cary, NC) was applied for statistical analysis.

Results

Of 860 registered acupuncture trials retrieved, 399 indicated their current status as ‘completed’. After excluding 160 registered trials because of duplicates or being non-RCTs, non-acupuncture and phase I, we searched subsequent publications for 88 of the 239 registered trials (36.8%). The 88 trials produced 96 publications reporting study outcomes, among which all publications were in English (Figure 1).

Figure 1
figure 1

Flow diagram of study selection.

Of the 88 included trial protocols, 65 (73.9%; 72 articles) were from western countries; 23 (26.1%; 24 articles) were from eastern countries, of which only 7 were performed in mainland China (excluding Hong Kong and Taiwan). The included trial protocols were registered in eight registries and the majority was registered in either clinicaltrials.gov (43/88, 48.9%) or ISRCTN (36/88, 40.9%). The first trial was registered in 2001. Only 19.3% (17/88) were registered before the start of the trial, suggesting prospective registration. The remaining 71 trials were registered after the study began, even after the completion of the study. The proportion of retrospective registration was similar for trials from western countries (54/65, 83.1%) and eastern countries (17/23, 73.9%; P = 0.33). When comparing trials starting before 1 July 2005 (36/40, 90.0%), the proportion of retrospective registration was higher than that in trials starting after 1 July 2005 (35/48, 72.9%; P = 0.04).

Ninety-six articles were published in 49 journals from 1997 to 2014 (Figure 2). Seven (7.3%) were published in alternative and complementary journals. Sixteen (16.7%) were published in journals whose 2013 impact factor was high than 10 (Additional file 1: Table S1). Twenty-seven journals (48 articles; 50.0%) provided guidance on trial registration and reporting of the registration number in their instructions for the author. Of 36 articles (37.5%) without description of the registration number, 50.0% (18/36) were published in journals requiring the reporting of the registration number.

Figure 2
figure 2

Number of included randomized clinical trials on acupuncture in the trial registry and subsequent publication.

The characteristics of included articles are presented in Table 1. The majority of the included articles were parallel group (92, 95.8%), single center (55, 57.3%), two armed (59, 61.5%), and sponsored by a university, government or hospital (77, 80.2%). The five most frequent conditions based on ICD-10 were musculoskeletal (26.1%), nervous (19.3%), mental and behavioral disorders (17.0%), respiratory conditions (5.7%) and neoplasms (5.7%; Table 2). Manual acupuncture and electro-acupuncture were reported in 80 (83.3%) and 16 (16.7%) articles, respectively. Interventions in the treatment groups involved acupuncture alone in 70 (72.9%) articles or acupuncture combined with other interventions in 26 (27.1%) articles. There was considerable variation in the control group. Fifty-one articles (53.1%) applied sham/placebo acupuncture; other commonly used controls involved western medicine (26.0%), no intervention (9.4%) and non-pharmaceutical interventions (9.4%). A clear description of the primary outcome and secondary outcome was provided in 71 (74.0%) and 60 articles (62.5%), respectively. However, safety and health-economic outcomes were only reported in 38 (39.6%) and 5 articles (5.2%), respectively.

Table 1 Characteristics of the included articles
Table 2 Conditions based on the ICD-10 classification treated by acupuncture

Seventy-five subsequent publications (78.1%) provided detailed information on the generation of allocation sequences with the most use of a computer random number generator. Fifty-eight articles (60.4%) provided details on allocation concealment. Of 69 articles (71.9%) providing information on blinding, 37 applied single blinding, 28 double blinding, and 4 triple blinding. However, only 3.4% (3/88), 2.3% (2/88) and 61.4% (54/88) of the registered records provided information on the generation of an allocation sequence, allocation concealment and blinding, respectively. Incomplete outcome reporting occurred in 33 (34.4%) articles. The sample size of the 96 articles ranged from 10 (pilot trials) to 960, with the majority (88.6%) of sample sizes between 20 and 500 per trial (Additional file 2: Table S2). Only 34 articles (35.4%) carried out a sample size calculation. All registered records lacked information on sample estimation. The majority of the registered records and the subsequent publications clearly described the inclusion and exclusion criteria.

A comparison of registered and published primary outcomes was impossible in 25 articles (26.0%) because outcomes were not specified as primary outcome or secondary outcome in the subsequent publications. Of the remaining 71 articles, the primary outcomes of 32 articles (45.1%) were inconsistent with those specified in registered records (Table 3). The discrepancies involved a registered primary outcome omitted in the published article (22/71, 31.0%), an absent primary outcome in the registry defined as a primary outcome in the published article (9/71, 12.7%), a published primary outcome registered as a secondary outcome (1/71, 1.4%), a registered primary outcome defined as a secondary outcome in the published article (15/71, 21.1%) and different timing of assessment of the primary outcome in the published article and the registry (12/71, 16.9%). Fourteen articles had two reasons for the difference in primary outcomes, five articles had three reasons for the difference in primary outcomes, and one article had four reasons for the difference in primary outcomes. This inconsistency was also addressed in 13 articles published in journals requiring trial registration; the proportion of inconsistency was lower than those published in journals not requiring trial registration [13 of 38 (34.2%) vs. 19 of 33 (57.6%); P = 0.048]. Five of 13 articles were published in a high-impact-factor journal (IF > 10). When comparing trials from western countries with trials from eastern countries, the proportion of this discrepancy showed no significant difference [6 of 21 (28.6%) vs. 26 of 50 (52.0%); P = 0.07].

Table 3 Proportion of discrepancies in the specified primary outcomes between registered trials and published articles on acupuncture and discrepancies favoring statistically significant results

Furthermore, 81.7% (138/169) registered primary outcomes were clinically relevant outcomes (patient-centered outcomes). Among them, 22.5% (31/138) clinically relevant outcomes from the 15 registered trial records were missing in the final publications, and 9.4% (13/138) clinically relevant outcomes from 11 registered records had been changed from the original primary outcome to the secondary outcome in the publications.

Of the 32 articles with inconsistency in primary outcomes between the published article and registry, the influence of this inconsistency could only be evaluated in 65.6% (21 of 32). Among them, 15 of 21 (71.4%) had discrepancies that favored statistically significant primary outcomes, while 6 of 21 (28.6%) favored nonsignificant primary outcomes.

Comparing the 88 registered records with their 96 subsequent publications, the other inconsistencies were identified in the inclusion criteria (54.7%), exclusion criteria (47.9%) and controls (22.9%) (Table 4).

Table 4 Comparison of methodological components between registered records and subsequent publications of randomized clinical trials on acupuncture

The ‘history of changes’ was available in 44 registered trial records (44/88, 50.0%) from six registries. After the initial registration, a change of primary outcomes was identified in seven registered trial records (7/88, 8.0%), and 14 registered trial records (14/88, 15.9%) had the following changes: changes of study arms, sample size, blinding, controls and inclusion/exclusion criteria.

Discussion

To assess outcome reporting bias, we identified a sample of 96 articles of acupuncture trials. Among them, 33.3% (32/96) articles showed clear discrepancies between the registered primary outcomes and the published primary outcomes. The inconsistency was addressed in 13 articles published in those journals requiring trial registration, and 5 of 13 articles were published in high-impact-factor journals. Furthermore, the other inconsistencies between the registry records and subsequent publications involved the inclusion criteria (54.7%), exclusion criteria (47.9%) and control interventions (22.9%). Selective outcome reporting has received widespread recognition. In previous studies, inconsistencies of primary outcomes between the registered records and corresponding publications in Western medicine varied from 18% and 49% [22-24].

Prospective trial registration provides a good opportunity for the editors and peer reviewers to evaluate outcome reporting bias and other deviations between the registered records and subsequent publications. However, we found some evidence of selective outcome reporting in 45.1% of the articles and other discrepancies between the published articles and the registered records in terms of inclusion/exclusion criteria, blinding, sample size, etc. These results highlight that editors and peer reviewers may not cross-check the consistency between the submitted manuscript and the registered records to identify any discrepancies, and to further limit an outcome reporting bias, even though they had a chance to access the registered records. A previous qualitative study of editors’ views on trial registration and publication bias showed that 11 of 15 journals required trial registration as a requisite for publication, but most editors seldom check whether the trial is adequately registered, and some editors did not even enforce trial registration [25].

This study also demonstrated that the trial registration number was unavailable in 36 published reports. Our finding is consistent with previous research that also indicated poor reporting of the registration number in published reports [26]. This may hamper editors and peer reviewers in accessing the registration records and to further assess the selective outcome reporting [23]. Accordingly, an accurate trial registration number should be provided in the publications. Then, the editors and peer reviewers should systematically check the consistency between the submitted manuscript and registration records, especially regarding the adequate reporting of important items such as the primary outcome.

As highlighted in our results, which confirmed the earlier findings [20,21], changes in the primary outcomes, as specified in the publications, from those specified in the registration records favored statistically significant over nonsignificant results. Such changes pose a threat to the reliability of results and further preclude clinicians and policymakers from making correct clinical decisions. Possible reasons for the discrepancies include the following: a preference for primary outcomes with statistical significance to increase the opportunity of the trial for publication and to support the use of the treatment [23,27].

The main purpose of prospective registration of clinical trials is to enhance the transparency of the clinical trial and provide access to the trial protocol [6,8]. If the trial is retrospectively registered, the meaning of the trial registration is lost; if information in registries is insufficiently detailed, with some items not registered or inadequately registered, this information will be useless for comparing the consistency between the registered records and subsequent publications. Therefore, adequate registration is of great significance to safeguard against publication bias [21]. However, our study showed that the proportion of prospective registered trials was still considerably low even with the introduction of the ICJME initiative policy implementation; registered information differed widely among trials; essential information such as randomization and blinding was poorly registered in the registries. Hence, the principal investigators and sponsors have the responsibility to adequately register a clinical trial. Furthermore, greater efforts should be made to improve the quality control processes of trial registries [21].

Although more than half of journals have required trial registration following the ICMJE guidelines, some journals use ambiguous language (i.e., encouragement of the trial registration) in their instruction to authors. We found similar results in previous studies [22,28]. A lack of submission guideline clarity might contribute to explaining inadequate trial registration and poor reporting of registration numbers.

We acknowledge that our study has several potential limitations. First, only trials indicated as having a ‘completed’ status in registries were searched for publications. A recent study indicated that many authors seemed to forget to change the status of their study to being completed, even among published studies [29]. Therefore, those trials in the registries where the status was not listed as ‘completed’ but that had been completed may not have been searched for and included in this study, which may have caused a bias.

Second, only PubMed and three Chinese databases were searched for published reports on acupuncture; therefore, some trials that have been published and included in other databases may have been missed.

Conclusions

Although trial registration is now the rule, we find that prospective registration for RCTs on acupuncture is insufficient, selective outcome reporting is prevalent, and the change of primary outcomes is intended to favor statistical significance. These discrepancies in outcome reporting may lead to biased and misleading results of RCTs on acupuncture. To ensure publication of reliable and unbiased results, further promotion and implementation of trial registration are still needed.

Abbreviations

ICMJE:

International Committee of Medical Journal Editors

ICTRP:

International Clinical Trials Registry Platform

RCTs:

randomized clinical trials

TCM:

traditional Chinese medicine

ANZCTR:

Australian New Zealand Clinical Trials Registry

ReBec:

Brazilian Clinical Trials Registry

ChiCTR:

Chinese Clinical Trial Register

CRiS:

Clinical Research Information Service

CTRI:

Clinical Trials Registry-India

RPCEC:

Cuban Public Registry of Clinical Trials

EU-CTR:

EU Clinical Trials Register

DRKS:

German Clinical Trials Register

ISRCTN:

International Standard Randomized Controlled Trial Number Register

IRCT:

Iranian Registry of Clinical Trials

JPRN:

Japan Primary Registries Network

PACTR:

Pan African Clinical Trial Registry

NTR:

the Netherlands National Trial Register

SLCTR:

Sri Lanka Clinical Trials Registry

ICD-10:

International Statistical Classification of Diseases and Related Health Problems 10th Revision

References

  1. Hutton JL, Williamson PR. Bias in meta-analysis due to outcome variable selection within studies. Appl Stat. 2000;49:359–70.

    Google Scholar 

  2. Kirkham JJ, Dwan KM, Altman DA, Gamble C, Dodd S, Smyth R, et al. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ. 2010;340:c365.

    Article  PubMed  Google Scholar 

  3. Al-Marzouki S, Roberts I, Marshall T, Evans S. The effect of scientific misconduct on the results of clinical trials: a Delphi survey. Contemp Clin Trials. 2005;26:331–7.

    Article  PubMed  Google Scholar 

  4. Furukawa T, Watanabe N, Omori I, Montori V, Guyatt G. Association between unreported outcomes and effect size estimates in Cochrane meta-analyses. JAMA. 2007;297:468–70.

    Article  CAS  PubMed  Google Scholar 

  5. Pearson M, Peters J. Outcome reporting bias in evaluations of public health interventions: evidence of impact and the potential role of a study register. J Epidemiol Community Health. 2012;66:286–9.

    Article  PubMed  Google Scholar 

  6. Dickersin K, Rennie D. Registering clinical trials. JAMA. 2003;290:516–23.

    Article  PubMed  Google Scholar 

  7. Abaid LN, Grimes DA, Schulz KF. Reducing publication bias through trial registration. Obstet Gynecol. 2007;109:1434–7.

    Article  PubMed  Google Scholar 

  8. De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. N Engl J Med. 2004;351:1250–1.

    Article  PubMed  Google Scholar 

  9. Killeen S, Sourallous P, Hunter IA, Hartley JE, Grady HL. Registration rates, adequacy of registration, and a comparison of registered and published primary outcomes in randomized controlled trials published in surgery journals. Ann Surg. 2014;259:193–6.

    Article  PubMed  Google Scholar 

  10. Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291:2457–65.

    Article  CAS  PubMed  Google Scholar 

  11. Chan AW, Hróbjartsson A, Jørgensen KJ, Gøtzsche PC, Altman DG. Discrepancies in sample size calculations and data analyses reported in randomised trials: comparison of publications with protocols. BMJ. 2008;337:a2299.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Von Elm E, Rollin A, Blumle A, Huwiler K, Witschi M, Egger M. Publication and non-publication of clinical trials: longitudinal study of applications submitted to a research ethics committee. SwissMed Wkly. 2008;138:197–203.

    Google Scholar 

  13. Al-Marzouki S, Roberts I, Evans S, Marshall T. Selective reporting in clinical trials: analysis of trial protocols accepted by the Lancet. Lancet. 2008;372:201.

    Article  PubMed  Google Scholar 

  14. Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM. Trial publication after registration in ClinicalTrials.Gov: a cross-sectional analysis. PLoS Med. 2009;6:e1000144.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Hahn S, Williamson PR, Hutton JL. Investigation of within-study selective reporting in clinical research: follow-up of applications submitted to a local research ethics committee. J Eval Clin Pract. 2002;8:353–9.

    Article  CAS  PubMed  Google Scholar 

  16. Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, Cronin E, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One. 2008;3:e3081.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Liu JP, Han M, Li XX, Mu YJ, Lewith G, Wang YY, et al. Prospective registration, bias risk and outcome-reporting bias in randomised clinical trials of traditional Chinese medicine: an empirical methodological study. BMJ Open. 2013;3:e002968.

    Article  PubMed  PubMed Central  Google Scholar 

  18. National Center for Complementary and Alternative Medicine (NCCAM). Acupuncture: an introduction. [http://nccam.nih.gov/health/acupuncture/introduction.htm]

  19. Higgins Jpt, Green S, the Cochrane Collaboration. Cochrane handbook for systematic reviews of interventions. Version 5.1.0. 2011 [http://www.cochrane-handbook.org]

  20. Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ. 2005;330:753.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P. Comparison of registered and published primary outcomes in randomized controlled trials. JAMA. 2009;302:977–84.

    Article  CAS  PubMed  Google Scholar 

  22. Hannink G, Gooszen HG, Rovers MM. Comparison of registered and published primary outcomes in randomized clinical trials of surgical interventions. Ann Surg. 2013;257:818–23.

    Article  PubMed  Google Scholar 

  23. Chan AW, Krleza-Jeric K, Schmid I, Altman DG. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ. 2004;171:735–40.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Bourgeois FT, Murthy S, Mandl KD. Outcome reporting among drug trials registered in ClinicalTrials.gov. Ann Intern Med. 2010;153:158–66.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Wager E, Williams P, Project overcome failure to publish negative findings consortium. “Hardly worth the effort”? Medical journals’ policies and their editors’ and publishers’ views on trial registration and publication bias: quantitative and qualitative study. BMJ. 2013;347:f5248.

    Article  PubMed  PubMed Central  Google Scholar 

  26. van de Wetering FT, Scholten RJ, Haring T, Clarke M, Hooft L. Trial registration numbers are underreported in biomedical publications. PLoS One. 2012;7:e49599.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Smyth RM, Kirkham JJ, Jacoby A, Altman DG, Gamble C, Williamson PR. Frequency and reasons for outcome reporting bias in clinical trials: interviews with trialists. BMJ. 2011;342:c7153.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. You B, Gan HK, Pond G, Chen EX. Consistency in the analysis and reporting of primary end points in oncology randomized controlled trials from registration to publication: a systematic review. J Clin Oncol. 2012;30:210–6.

    Article  PubMed  Google Scholar 

  29. Korevaar DA, Ochodo EA, Bossuyt PM, Hooft L. Publication and Reporting of Test Accuracy Studies Registered in ClinicalTrials.gov. Clin Chem. 2014;60:651–9.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgments

We would like to thank Zhong Sun and Shang-Qian Gao for their contributions to the data extraction. This work is partly supported by the Project for Capacity Building of TCM Clinical Researchers (no. 201207007), Beijing Higher Education Young Elite Teacher Project (no. YETP0797) and Independent Research Project of Beijing University of Chinese Medicine (no. 522/010060514). LJP, SCX, HM, RJ and LWY are awardees by the Program for Innovative Research Team of Beijing University of Chinese Medicine (no. 2011-CXTD-09).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jian-Ping Liu.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

SCX designed the study, searched and selected studies, extracted and analyzed data, and drafted the manuscript. HM searched and selected studies, extracted data and revised the manuscript. RJ and LWY extracted data. YSJ helped with the design of tables and data analyses and made revisions to the manuscript. HYF provided method perspectives and revised the manuscript. LJP conceived and designed the study and made substantial revisions to the manuscript. All authors have read and approved the final submitted version.

Additional files

Additional file 1: Table S1.

Top ten journals publishing the included articles of acupuncture ranked by 2013 impact factor (IF) in descending order.

Additional file 2: Table S2.

Sample size of included articles.

Rights and permissions

Open Access  This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit https://creativecommons.org/licenses/by/4.0/.

The Creative Commons Public Domain Dedication waiver (https://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Su, CX., Han, M., Ren, J. et al. Empirical evidence for outcome reporting bias in randomized clinical trials of acupuncture: comparison of registered records and subsequent publications. Trials 16, 28 (2015). https://doi.org/10.1186/s13063-014-0545-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13063-014-0545-5

Keywords