Quality of reporting on randomized controlled trials on recurrent spontaneous abortion in China
Trials volume 16, Article number: 172 (2015)
Despite increasing numbers of RCTs done in China, detailed information on the quality of Chinese RCTs is still missing. The aim of this study was to assess the reporting quality of RSA RCTs and to identify significant predictors of reporting quality.
A literature review was conducted with the aim of identifying published RCTs on RSA conducted in China. In order to rate the report quality, we scored 1 for the item of CONSORT 2010 if it was reported and 0 if it was not stated or unclear. An overall quality score (OQS) with a range of 0–15 and a key methodological index score (MIS) with a range of 0–3 were calculated for each trial.
A total of 98 relevant RCTs were included in the final analysis. The median OQS was 7, with a minimum of 1 and maximum of 12. The general level of OQS was not high, especially among ‘sample size,’ ‘baseline data,’ ‘outcomes and estimation,’ and ‘ancillary analyses,’ all of which had a positive rate of less than 10%. The median MIS was 1 with a minimum of 0 and maximum of 1. ‘Allocation concealment,’ ‘blinding,’ and ‘intention-to-treat analysis’ were mentioned in 1 (1%), 1 (1%) and 69 (70%) of the studies, respectively. In univariate analysis, funding was the only factor associated with an increased OQS. Specifically, the mean OQS increased by approximately 1.52 for manuscripts supported by funding (95% CI: 0.12 – 2.92; p = 0.03). With regard to the MIS, no association was found for any variable.
RCTs of RSA conducted in China need improvement in order to meet the level of “reporting quality” required by the CONSORT statement.
Recurrent spontaneous abortion (RSA) is the occurrence of three or more consecutive pregnancies that end in miscarriage of the fetus before viability . The estimated frequencies of three or more and two or more consecutive pregnancy losses are 0.9% and 4.2%, respectively . RSA affects 1% to 5% of women of reproductive age in China  and is one of the most difficult forms of infertility affecting Chinese women, thus being a pressing issue . RSA can both physically and psychologically harm patients, and additionally bring heavy economic burdens upon families. Hence, research on the prevention and treatment of recurrent miscarriage is of significance both clinically and socially. Since the etiology and pathogenesis of RSA are largely unclear, various forms of intervention have been used in clinical practice, but the majority of them still lack sufficient evidence to show that they do prevent miscarriages in women with RSA .
To find the most efficient treatment, more and more randomized controlled trials (RCTs) in RSA have been appearing [6-8]. RCTs, as the ‘gold standard’ of evidence-based clinical practice, are generally considered to have the highest level of credibility in determining the efficacy of a new treatment . Moreover, the quality of reporting is essential for guiding journal peer-review decisions, for experts’ recommendations and for conducting unbiased meta analyses as it influences the interpretation of evidence. Understanding the importance of transparency in reporting clinical trials, an international team, including epidemiologists, statisticians and journal editors, developed the Consolidated Standards for Reporting Trials (CONSORT) Statement in 1996. Since then, more and more researchers have been using CONSORT to assess reporting quality [10-14].
Nevertheless, to the best of our knowledge, there has been no recent special report on the quality of RCTs of RSA conducted in China. The aim of this study is to assess the overall reporting quality of published articles of randomized trials of RSA with a special focus on the key methodological items that safeguard against biases, namely concealment of allocation, appropriate blinding, and analysis according to the intention-to-treat principle. Second, we also aimed to identify significant predictors of reporting quality.
A systematic and comprehensive literature review was conducted with the aim of identifying published RCTs of RSA conducted in China. We searched the following bibliographic databases: Embase (1980 to May 2014), Medline (1966 to May 2014), CINAHL (1982 to May 2014), China National Knowledge Infrastructure (CNKI, 1979 to May 2014), the Chinese Biological Medicine Database (CBM-disc, 1979 to May 2014), Wanfang databases (1982 to May 2014) and VIP database (1992 to May 2014). In addition, we searched the Cochrane Central Register of Controlled Trials (CENTRAL) (May 2014). Additionally, reference lists of eligible studies and previous systematic reviews were also reviewed to identify further eligible studies. The search terms used were in Chinese and English and included keywords such as spontaneous abortion, recurrent abortion, fetal loss, miscarriage, randomized trials, RCT and China.
Inclusion and exclusion criteria
Types of studies: We looked at randomized controlled trials that assessed the effect of a particular treatment in women with recurrent miscarriages. However, quasi-randomized trials, nonrandomized, cross-over RCTs, case reports and case-control studies were excluded. All RCTs had to be performed in China and primarily by Chinese researchers.
Participants’ criteria: We included pregnant women with a history of three or more consecutive unexplained miscarriages prior to 24 weeks of gestation. The target population of this review was women with miscarriages that remained unexplained after routine investigations.
Assessment of reporting quality
Given that we defined quality of reporting as the extent to which the rationale, method, conduct and results of the trial were reported, we adopted 15 relevant items from the revised CONSORT statement for our appraisal. CONSORT items were chosen because a lack of their reporting has been associated with estimates of increased levels of bias [15,16]. An overall quality score (OQS) with 15 items from the revised CONSORT statement was used as previously described [16-18]. We scored 1 for an item of CONSORT 2010 if it was reported, and 0 for an item if it was not stated or unclear. For the overall quality score (OQS), 15 items were scored and calculated with a range of 0–15 (Table 1) [17-19].
Key methodological items
Three key methodological categories of allocation concealment, blinding and intention-to-treat (ITT) analysis have been assessed separately because they relate to potential sources of bias [20-22]. We then developed eight “yes”/“no” items (Table 2) and wording that emphasized quality of reporting rather than adequacy of trial design. Each item was scored 1 if the method was appropriate and 0 if it was inappropriate or if the reporting was unclear.
Each article was reviewed by two independent investigators (Yunyun Hu and Yanxi Liu) who had received training in research methodology and statistics using modified CONSORT checklists. In addition the two investigators were blinded to each other’s ratings and completed the rating form independently. Information was extracted as the modified CONSORT checklist with 18 items (Tables 1 and 2). We judged the consistency of two assessors by calculating Cohen’s к-statistic. The consistency judgment criteria were as follows: if к = 1, agreement was judged as “perfect;” if к was between 0.8 and 1, agreement was “good;” if к was between 0.6 and 0.8, agreement was “substantial;” if к was between 0.4 and 0.6, agreement was “moderate;” if к was between 0.2 and 0.4, agreement was “fair;” if к was less than 0.2, agreement was “poor” [17,18].
The characteristics of the publications OQS and MIS were then described by descriptive analysis. To screen factors associated with OQS, it was used as the outcome variable and the characteristics of the publications as independent variables, which were modeled using linear regression. A multivariate regression model was performed only if variables in the univariate models were significant at p ≤ 0.10. In the final multivariate model, variables were significant predictors at p ≤ 0.05. As the outcome variable, MIS could be considered as a dichotomy, and the logistic regression model was used to select factors associated with MIS. Descriptive statistical analysis as well as linear and logistic regression analysis was performed using SPSS version 20.0 (SPSS, Chicago, IL, USA). Analysis of Cohen’s k-statistics was performed using SAS software, version 9.3 (SAS Institute, Inc., Cary, NC, USA). The database of RCTs of RSA conducted in China are provided in Additional file 1.
The RCTs selection process is outlined in Figure 1. The researchers applied the search method to find 311 reports related to the topic. After careful selection, a total of 98 relevant RCTs were included in the final analysis (see Figure 1).
Characteristics of included trials
There has been a spurt of growth based on the 76 (77.6%) from the years 2009–2014 compared with 22 (22.4%) from 1998–2008. Twenty-three articles (23.5%) were related to modern medicine, especially immunotherapy; 42 (42.9%) were related to Chinese medicine, and the remaining 33 (33.7%) combined both methods. As for source of finance, eight articles (8.2%) reported the funding sources, half of which were obtained from provincial sources, while the rest were from municipal sources. All cases were from the same hospital in each paper. Finally, in terms of choice of comparator interventions, the largest number of interventions comprised intramuscular injection of progesterone followed by an intramuscular injection of human chorionic gonadotropin (HCG).
Rater agreement: Substantial agreement was observed for items 2, 3, 4, 5, 6, 11, 13, 14, 15, 16, 17 and 18; good agreement was observed for items 7, 8 and 19; perfect agreement was observed for items 1, 9 and 12 (see Tables 1 and 2).
Quality of reporting
The ratings of overall reporting quality are listed in Table 1. The level of reporting was below average as the mean OQS was 7.10 with a standard deviation of 1.95. According to our research, part of the information was insufficient or inadequate in most studies. The general level of reporting was not high, especially for the items ‘sample size,’ ‘baseline data,’ ‘outcomes and estimation’ and ‘ancillary analyses,’ each with a positive rating of less than 10%. The lowest ratings were for ‘randomization’ and ‘harms,’ both with a positive rating of less than 50%. However, other items such as ‘background,’ ‘trial design,’ ‘interventions,’ ‘outcomes’ and ‘flow chart’ each received a high rating of over 80%.
Key methodological items
‘Allocation concealment,’ ‘blinding’ and ‘intention-to-treat analysis’ were mentioned in 1 (1%), 1 (1%) and 69 (70%) studies, respectively. The median MIS was 1 with quartile interval of 1. And 29 papers (29.6%) did even not mention either of them.
Exploratory analysis: factors associated with better reporting quality
Based on the multicollinearity diagnostics, there was no statistical correlation between different explanatory variables. In univariate analysis, funding was the only factor associated with an increased OQS. As only one variable was included in univariate analysis, the multivariate linear regression model was presented with only one explanatory variable. The multivariate linear regression model was built successfully in statistics (F = 4.64, p = 0.03). Specifically, the mean OQS increased by about 1.52 for manuscripts supported by funds (95% CI: 0.12–2.92; p = 0.03).
With regard to the MIS, in univariate logistic regression analyses, no variable could be included in this model, and the following multivariate logistic regression model was not performed Table 3.
This study has demonstrated that the qualities of reporting were not met from 1998 to 2014. This result suggests that RCTs of RSA conducted in China need to improve in order to meet the level of “reporting quality” required by the CONSORT statement. Low-scored items were related to the Methods and Results sections as follows: ‘ancillary analyses,’ ‘allocation concealment,” randomization,” sample size,’ ‘blinding,’ ’baseline data’ and ‘analysis by ITT.’ These key methodological categories, which are regarded as being highly correlated to avoid bias, emerged as the worst.
Well-designed and implemented randomized controlled trials (RCTs) have the strongest power to prove the efficiency of interventions. Randomization is now considered the best method to ensure that the baselines between groups are similar and to avoid removing selection bias between them. According to our study, only 14 articles (14.3%) reported the correct methods to obtain the allocation sequence by a random number table or computerized random number generator. Others created inaccuracies by dividing groups using birth date or admission number, which is regarded as not choosing individuals correctly because of the inability to conceal these allocation systems adequately.
Most articles included in our research did not adequately consider allocation concealment as the positive rate in our research was only 1% (1/89). This was below 30-50% in other evaluation reports related to trials in obstetrics and gynecology [23-25]. We recommend using an allocation concealment that can prevent foreknowledge of treatment and protect the enrolled participants from being influenced. By using this method, we could implement a generated allocation schedule . A lack of allocation concealment may compromise the unpredictable allocation sequence and cause selection bias, reducing the strength of conclusions. Compared with trials that reported allocation sequences adequately, the inadequately or unclearly concealed ones might yield larger estimates of treatment effects. Blinding is another important safeguard to ensure the quality of articles against performance and ascertainment bias, especially when assessing subjective outcomes . Although blinding is not always available to all trial participants, it is feasible to the patients and investigators in data collection and processing. Nonetheless, the majority of the 98 trials did not mention it: only one trial reported a description of blinding.
Sample size calculations need to strike a balance between statistical considerations and treatment effect difference compared to standard therapies. The author needs to show how the sample size was determined, as it not always easy to recruit enough participants within a set time besides those that withdraw. Approximately 66% failed to achieve their planned sample size based on a review of 41 RCTs . No trial had clearly described the process of sample size calculation, and more than 30% (34/98) had a sample size less than 50. This result was poor when compared to 37.8% in the spine area . We should pay more attention to sample size calculation as it indicates the difference between new interventions and controlled ones. Furthermore, very small-scale RCTs are often insufficient to evaluate benefits and more likely to cause bias risks.
For the reporting status of RCTs of RSA conducted in other countries, David et al.  found approximately 14% of RCTs reported items of random sequence or allocation concealment, and more than 50% of RCTs described the blinding method in the field of progestogen for preventing RSA. Morley et al.  found that 40% of RCTs reported items of random sequence or allocation concealment, and 60% described the blinding method in the field of human chorionic gonadotrophin (hCG) for preventing RSA. From the comparison between the status of RCTs of RSA conducted in China and other countries, it seems that the situation in China is worse according to these key methodology items.
Our analysis has revealed that research supported by funding was significantly associated with better OQS based on the CONSORT statement. Compared with the funding resource rate, which was 8% (8/98) in this study, it was below 56% for Hodgkin’s lymphoma in biomedical journals  and 66% for epidemiology . Since large-size trials cost more, some RCTs with smaller sample size than necessary were performed in case of lack of sponsorships, which typically led to negative conclusions.
Several suggestions could be made considering our results, which could enhance the reporting quality of future RCTs of RSA. First, several items, such as ‘ancillary analyses,’ ‘allocation concealment,’ ’ randomization,’ ‘sample size,’ ‘blinding,’ ‘baseline data’ and ‘analysis by ITT’ should be paid more attention when RCT reports are prepared. Clinical research training and consultation with statisticians and epidemiologists are also important in order to improve report quality. Second, funding applications should be encouraged as they can guarantee resources for RCT design, execution and evaluation. This study does have its limitations. First, each article enrolled was published in Chinese. There were no unpublished or currently in progress articles included, which may lead to publication bias. Second, to evaluate the quality of reporting in RCTs quantitatively, according to some rating methods published in previous studies [17-19], we extracted major items, not all items, from the CONSORT 2010 Statement. Third, satisfying some Consort recommendation in the report did not imply that the trial really fulfilled it. For example, a report may state that the analysis was by ITT, but in actuality several randomized patients were excluded from the report. Despite these limitations, we think our results have good internal validity as agreement was judged substantial, good or perfect in the evaluation process executed by two independent assessors.
Our findings show that the reporting quality of RCTs in recurrent miscarriage is unsatisfactory. Regarding the crucial methodological issues of allocation concealment, blinding and sample size calculation, our results stress the need to improve the reporting quality of RCTs of RSA conducted in China.
Randomized controlled trials
Recurrent spontaneous abortion
Overall quality score
Methodological index score
Consolidated Standards for Reporting Trials
China National Knowledge Infrastructure
Chinese Biological Medicine Database
Cochrane Central Register of Controlled Trials
Intention to treat
Human chorionic gonadotropin
Pandey MK, Rani R, Agrawal S. An update in recurrent spontaneous abortion. Arch Gynecol Obstet. 2005;272(2):95–108.
Sugiura-Ogasawara M, Suzuki S, Ozaki Y, Katano K, Suzumori N, Kitaori T. Frequency of recurrent spontaneous abortion and its influence on further marital relationship and illness: the Okazaki Cohort Study in Japan. J Obstet Gynaecol Res. 2013;39(1):126–31.
Lin QD, Qiu LH. Pathogenesis, diagnosis, and treatment of recurrent spontaneous abortion with immune type. Front Med China. 2010;4(3):275–9.
Wen DI, Bei HZ. Research on the pathogenesis, diagnosis and treatment of female reproductive disorders. J Shanghai Jiaotong Univ (Medical Science). 2012;32(9):1161–5.
Yang GY, Luo H, Liao X, Liu JP. Chinese herbal medicine for the treatment of recurrent miscarriage: a systematic review of randomized clinical trials. BMC Complement Altern Med. 2013;13:320.
Haas DM, Ramsey PS. Progestogen for preventing miscarriage. Cochrane Database Syst Rev. 2013;10:D3511.
Morley LC, Simpson N, Tang T. Human chorionic gonadotrophin (hCG) for preventing miscarriage. Cochrane Database Syst Rev. 2013;1:D8611.
Wong LF, Porter TF, Scott JR. Immunotherapy for recurrent miscarriage. Cochrane Database Syst Rev. 2014;10:D112.
Li JY, Zhang YF, Smith GS, Xue CJ, Luo YN, Chen WH, et al. Quality of reporting of randomized clinical trials in tai chi interventions-a systematic review. Evid Based Complement Alternat Med. 2011;2011:383245.
Pandis N, Shamseer L, Kokich VG, Fleming PS, Moher D. Active implementation strategy of CONSORT adherence by a dental specialty journal improved randomized clinical trial reporting. J Clin Epidemiol. 2014;67(9):1044–8.
Cui Q, Tian J, Song X, Yang K. Does the CONSORT checklist for abstracts improve the quality of reports of randomized controlled trials on clinical pathways? J Eval Clin Pract. 2014;20(6):827–33.
Xiao L, Hu J, Zhang L, Shang HC. Endorsement of CONSORT by Chinese medical journals: A survey of “instruction to authors”. Chin J Integr Med. 2014;20(7):510–5.
Mbuagbaw L, Thabane M, Vanniyasingam T, BorgDebono V, Kosa S, Zhang S, et al. Improvement in the quality of abstracts in major clinical journals since CONSORT extension for abstracts: a systematic review. Contemp Clin Trials. 2014;38(2):245–50.
Blair DA, Hughes PJ, Woolley TJ. Pharmacy journal abstracts published in PubMed that abide by the CONsolidated Standards Of Reporting Trials (CONSORT) guidelines. J Med Libr Assoc. 2014;102(2):110–4.
Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Clin Oral Investig. 2003;7(1):2–7.
Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet. 2001;357(9263):1191–4.
Lai R, Chu R, Fraumeni M, Thabane L. Quality of randomized controlled trials reporting in the primary treatment of brain tumors. J Clin Oncol. 2006;24(7):1136–44.
Toulmonde M, Bellera C, Mathoulin-Pelissier S, Debled M, Bui B, Ilaliano A. Quality of randomized controlled trials reporting in the treatment of sarcomas. J Clin Oncol. 2011;29(9):1204–9.
Rios LP, Odueyungbo A, Moitri MO, Rahman MO, Thabane L. Quality of reporting of randomized controlled trials in general endocrinology literature. J Clin Endocrinol Metab. 2008;93(10):3810–6.
Hollis S, Campbell F. What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ. 1999;319(7211):670–4.
Gluud LL, Thorlund K, Gluud C, Woods L, Harris R, Sterne JA. Correction: reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med. 2008;149(3):219.
Montori VM, Bhandari M, Devereaux PJ, Manns BJ, Ghali WA, Guyatt GH. In the dark: the reporting of blinding status in randomized controlled trials. J Clin Epidemiol. 2002;55(8):787–90.
Schulz KF, Chalmers I, Grimes DA, Altman DG. Assessing the quality of randomization from reports of controlled trials published in obstetrics and gynecology journals. JAMA. 1994;272(2):125–8.
Pildal J, Hrobjartsson A, Jorgensen KJ, Hilden J, Altman DG, Gotzsche PC. Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials. Int J Epidemiol. 2007;36(4):847–57.
Wood L, Egger M, Gluud LL, Schulz KF, Juni P, Altman DG, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ. 2008;336(7644):601–5.
Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. Int J Surg. 2012;10(1):28–55.
Mcdonald AM, Knight RC, Campbell MK, Entwistle VA, Grant AM, Cook JA, et al. What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies. Trials. 2006;7:9.
Chen X, Zhai X, Wang X, Su J, Li M. Methodological reporting quality of randomized controlled trials in three spine journals from 2010 to 2012. Eur Spine J. 2014;23(8):1606–11.
Kober T, Trelle S, Engert A. Reporting of randomized controlled trials in Hodgkin lymphoma in biomedical journals. J Natl Cancer Inst. 2006;98(9):620–5.
Chan AW, Altman DG. Epidemiology and reporting of randomised trials published in PubMed journals. Lancet. 2005;365(9465):1159–62.
This research was supported by the National Natural Science Foundation of China (grant nos. 81273795 and No. 81373672). The funder had no further role in the study design, data collection, analysis and interpretation of data, writing of the report or the decision to submit the paper for publication.
The authors declare that they have no competing interests.
JG, GPD and SPL planned and designed the study. JG, YYH, YXH, LML, DDH and GPD contributed to the acquisition of data. YDL, LZ, XJL and XJ analyzed the data and LML provided statistical support throughout the working process. JG, YYH, YXH, LML and SPL were primarily responsible for writing the paper. All authors were involved in the drafting and revision of the paper and approved the final version of the manuscript for submission.
Jie Gao and Gaopi Deng contributed equally to this work.
About this article
Cite this article
Gao, J., Deng, G., Hu, Y. et al. Quality of reporting on randomized controlled trials on recurrent spontaneous abortion in China. Trials 16, 172 (2015). https://doi.org/10.1186/s13063-015-0665-6
- CONSORT statement
- Recurrent spontaneous abortion
- Quality of reporting
- Randomized controlled trial