Global mapping of randomised trials related articles published in high-impact-factor medical journals: a cross-sectional analysis
Trials volume 21, Article number: 34 (2020)
Randomised controlled trials (RCTs) provide the most reliable information to inform clinical practice and patient care. We aimed to map global clinical research publication activity through RCT-related articles in high-impact-factor medical journals over the past five decades.
We conducted a cross-sectional analysis of articles published in the highest ranked medical journals with an impact factor > 10 (according to Journal Citation Reports published in 2017). We searched PubMed/MEDLINE (from inception to December 31, 2017) for all RCT-related articles (e.g. primary RCTs, secondary analyses and methodology papers) published in high-impact-factor medical journals. For each included article, raw metadata were abstracted from the Web of Science. A process of standardization was conducted to unify the different terms and grammatical variants and to remove typographical, transcription and/or indexing errors. Descriptive analyses were conducted (including the number of articles, citations, most prolific authors, countries, journals, funding sources and keywords). Network analyses of collaborations between countries and co-words are presented.
We included 39,305 articles (for the period 1965–2017) published in forty journals. The Lancet (n = 3593; 9.1%), the Journal of Clinical Oncology (n = 3343; 8.5%) and The New England Journal of Medicine (n = 3275 articles; 8.3%) published the largest number of RCTs. A total of 154 countries were involved in the production of articles. The global productivity ranking was led by the United States (n = 18,393 articles), followed by the United Kingdom (n = 8028 articles), Canada (n = 4548 articles) and Germany (n = 4415 articles). Seventeen authors who had published 100 or more articles were identified; the most prolific authors were affiliated with Duke University (United States), Harvard University (United States) and McMaster University (Canada). The main funding institutions were the National Institutes of Health (United States), Hoffmann-La Roche (Switzerland), Pfizer (United States), Merck Sharp & Dohme (United States) and Novartis (Switzerland). The 100 most cited RCTs were published in nine journals, led by The New England Journal of Medicine (n = 78 articles), The Lancet (n = 9 articles) and JAMA (n = 7 articles). These landmark contributions focused on novel methodological approaches (e.g. the “Bland-Altman method”) and trials on the management of chronic conditions (e.g. diabetes control, hormone replacement therapy in postmenopausal women, multiple therapies for diverse cancers, cardiovascular therapies such as lipid-lowering statins, antihypertensive medications, and antiplatelet and antithrombotic therapy).
Our analysis identified authors, countries, funding institutions, landmark contributions and high-impact-factor medical journals publishing RCTs. Over the last 50 years, publication production in leading medical journals has increased, with Western countries leading in research but with low- and middle-income countries showing very limited representation.
Randomised controlled trials (RCTs) are considered one of the simplest and most powerful tools for assessing the safety and effectiveness of treatment interventions [1,2,3]. When appropriately designed, conducted and reported, RCTs can produce an immediate impact on clinical practice and patient care .
The evolution of RCTs has been an enduring and continuing process [5,6,7,8,9,10,11,12,13,14,15]. Since the 1970s the publication landscape for RCTs has exhibited an exponential growth. For example, a 1965–2001 bibliometric analysis of the literature identified 369 articles published in 1970 compared to 11,159 published in 2000 . The development of clinical trial registries (such as clinicaltrials.gov) [9, 10], the exponential increase in journals publishing trial protocols, results and secondary studies, and growing support for data-sharing policies [11, 12] have created an open research environment of transparency and accountability. Furthermore, the publication of reporting guidelines (such as CONSORT and SPIRIT) [4, 13,14,15] have served to facilitate the transition between research and reporting to ensure standardisation and ease of readability.
RCTs published in major medical journals are highly cited and have an instrumental role in clinical practice and health policy decisions [5, 16, 17]. Previous studies have focused on the quality of the reporting of methods and results of RCTs [18,19,20,21,22] and publication practices [23,24,25,26,27,28] in selected samples of articles published in high-impact-factor (IF) medical journals. However, to the best of our knowledge, no mapping studies have been conducted on major medical journals to investigate the most common subjects, most productive scientists and countries, most prolific journals and “citation classics” across multiple specialties.
The objective of this study was to describe and characterise the global clinical research publication activity through RCT articles published in high-IF medical journals during the past decades.
This cross-sectional analysis investigated RCT-related articles (that is, primary RCTs, secondary analyses and methodology papers using clinical data) published in major medical journals. We excluded narrative reviews, systematic reviews, meta-analyses, pool-analyses, letters and newspaper articles. All RCT-related articles indexed in PubMed/MEDLINE had to be published in one of the major medical journals with an IF exceeding 10 (2016 IF according to the Journal Citation Reports [JCR] published in June 2017). These medical journals were chosen because they were identified as publishing clinical research with scientific merit and clinical relevance (see Table 1 for a list of the included medical journals).
On March 22, 2018, we systematically searched MEDLINE through PubMed (National Library of Medicine, Bethesda, MD, United States) for all RCT-related articles published in high-IF medical journals (from inception to December 31, 2017). A senior information specialist (AA-A) and a clinical epidemiologist (FC-L) designed an electronic literature search using a validated research methodology filter for RCTs (with 97% specificity and 93% sensitivity) . The search was peer reviewed by members of the study team, including a second (senior) information specialist (RA-B). The full search strategy is provided in Additional file 1. On May 7, 2018, we searched the Web of Science (WoS) (Clarivate Analytics, Philadelphia, Penn., United States) by using PubMed IDs (PMIDs) from the PubMed/MEDLINE searches. Merging MEDLINE with other citation indices such as the WoS combines the advantages of MEDLINE (e.g., Medical Subject Headings [MeSH], a comprehensive controlled vocabulary for indexing journal articles) with the relational capabilities and data of the WoS .
Data extraction and normalisation
For each included article, raw (meta) data on the journal and article titles, subject category, the year of publication, keywords, and the authors’ names, institutional affiliation(s), funding source, and country was downloaded online through the WoS by one researcher (A-AA). We also used the WoS to determine the extent to which each article had been cited in the scientific peer-review literature using the “times cited” number (that is, the number of times a publication has been cited by other publications). Two researchers (FC-L, RA-B) independently verified the data to minimise potential information errors. A process of normalisation was conducted by two researchers to bring together the different names of an author or country and the keywords (further details are available in Additional file 2). Specifically, one researcher (AA-A) checked the names by which an individual author appeared in two or more different forms (for example, “John McMurray” or “John J. McMurray” or “John J.V. McMurray”) using coincidence in that author’s place(s) of work as the basic criterion for normalisation (for example, University of Glasgow, Scotland, United Kingdom) , and a second researcher (FC-L or RA-B) verified the data. A threshold of 30 articles was applied to review 200 names by which an individual author appeared in two or more different forms.
We extracted both “author keywords” and “keyword plus,” which are automatically assigned by the WoS from the titles of the references of the articles, as topical (also called textural, linguistic or sematic) data . To ensure consistency in the data, one researcher (RA-B) corrected keywords by unifying grammatical variants and using only one keyword developed to name the same concept (for example, “randomized trial” or “randomized clinical trial” or “randomized controlled trial” or “randomised controlled trial”). In addition, the same researcher (RA-B) removed typographical, transcription and/or indexing errors, and a second researcher (FC-L) verified the data. All potential discrepancies were resolved via consensus amongst these investigators. All these data were collected and entered into a Microsoft Access® (Microsoft, Seattle, WA, United States) database between May 7, 2018, and January 9, 2019.
We analysed data for the number of articles, citations, signatures (or total number of authors included in all the articles of each author), collaboration index (that is the mean number of author’s signatures per article), countries, journals and keywords. Data were summarised as frequencies and percentages for the categorical items. The most prolific authors (>100 articles), countries (>100 articles), funding institutions (>100 articles), and the most cited papers (“top-100 citation classics”) were identified. Network plots were generated for intense scientific collaboration between countries (applying a threshold of 100 articles in collaboration).
We conducted an exploratory analyses of topical data using a set of unique keywords and their frequencies to examine the topic coverage, major topics (“word clouds” of keywords) and their interrelations (“co-words networks”) in RCT articles. The main goal in topical analyses is to understand the topical distribution of a dataset, i.e. what topics are covered and how much of each topic is covered in a scientific discipline . The most frequently used keywords were identified for the most prolific journals (with at least 1000 articles). Based on the most frequently used keywords (with at least 500 articles), a word cloud was created from text that the user provides and more emphasis was placed on words that appear with greater frequency in the source text. A “co-words network” was created to illustrate the co-occurrence of highly frequent words in the articles (applying a threshold of 100 articles in collaboration). The network analysis was carried out with the use of PAJEK (University of Ljubljana, Slovenia) , a software package for large network analysis that is free for non-commercial use to construct network graphs. The PRISMA checklist  (http://www.prisma-statement.org/) guided the reporting of the present analysis (and is available in Additional file 3).
A total of 39,329 records were identified by the PubMed/MEDLINE search (Fig. 1), and 39,305 articles met the study inclusion criteria (Additional file 4) after 24 records had been excluded (Additional file 5). Table 2 details the general characteristics of the articles.
The number of articles increased exponentially over the period 1965–2017 (Fig. 2). Approximately 60% (n = 23,635) of the articles have been published since 2000.
Journals and subject category
Forty journals published 39,305 articles, and 23.8% of them (n = 9355) were published by four journals with an IF > 30. The Lancet (9.1%; n = 3593), the Journal of Clinical Oncology (8.5%; n = 3343) and The New England Journal of Medicine (8.3%; n = 3275) published the largest number of articles, followed by The BMJ (6.4%; n = 2516) and Circulation (5.9%; n = 2331). Most articles were classified as “medicine, general & internal” (30.7%; n = 13,688); “cardiac & cardiovascular systems” (13.1%; n = 5828); or “oncology” (12.9%; n = 5760) according to the WoS journal categorisations (Table 2).
Authors, institutions and countries
Most articles (62.3%; n = 24,496) were written by seven or more authors, and only 11.4% (n = 4469) of the articles were written by three or fewer authors. The first authors of the articles were based most commonly in North America and Western Europe; first authors from the United States were responsible for 36.9% (n = 14,508) of the articles (Table 2). We identified 17 authors who published 100 or more articles (Table 3). All of the most productive authors were male. The most prolific authors were Robert M. Califf, with 239 articles (from Duke University, United States); Eugene Braunwald, with 218 (from Harvard University, United States); Salim Yusuf, with 217 (from McMaster University, Canada); Eric J. Topol, with 212 (from Scripps Translational Science Institute, United States); Harvey D. White, with 186 (from University of Auckland, New Zealand); Lars Wallentin, with 144 (Uppsala University, Sweden); and Christopher B. Granger, with 140 (from Duke University, United States).
Overall, 154 countries worldwide contributed to the analysed articles. The publication productivity ranking for countries (Table 4) was led by the United States (n = 18,393 articles, with 3.4 million citations), followed by the United Kingdom (n = 8028 articles, with 1.3 million citations), Canada (n = 4548 articles, with 1.0 million citations) and Germany (n = 4415 articles, with 0.9 million citations). A total of 37 countries had at least 100 articles in co-authorship. Figure 3 shows a visual representation of the most intense collaborative network between these 37 countries, in which we can see the relationships of some countries with respect to others and the position that each occupies in the network.
A total of 16,485 articles (41.9%) reported sources of funding. The 40 most frequent funding institutions (with 100 or more articles) are listed in Table 5. The main funders were the National Institutes of Health (NIH), with 7422 articles; Hoffmann-La Roche (n = 1188), Pfizer (n = 1139), Merck Sharp & Dohme (n = 1097) and Novartis (n = 1052).
Most cited articles
Overall, included articles received 5.9 million citations, of which 83.1% of the citations (n = 4,950,604) corresponded to 15,142 (38.5%) articles with more than 100 citations. In addition, 641 (1.63%) articles with more than 1000 citations accounted for 20.7% of the total citations (n = 1,234,462). The most cited articles by number of citations (“100 citation classics”) are listed in Table 6. All of the most cited papers were published in English. These most cited articles were published in nine journals, led by The New England Journal of Medicine, with 78 articles, followed by The Lancet (n = 9) and JAMA (n = 7). The list of most cited papers contained innovative research methodologies. For example, the most cited article was a method paper published in The Lancet (“Bland-Altman method”) . This seminal paper changed how method comparison studies are performed in clinical research. The list of the most cited papers also reflected important studies examining the health effects of pharmacological interventions on patients with chronic diseases. Common themes in major advances in health interventions included diabetes control [36,37,38,39,40,41]; the effects of hormone replacement therapy in postmenopausal women [42, 43]; therapies for diverse cancers such as glioblastoma, colorectal cancer, breast cancer, melanoma and hepatocellular carcinoma [44,45,46,47,48,49,50]; important interventional studies in the field of clinical cardiology, such as lipid-lowering statin therapy trials, antihypertensive trials, and antiplatelet and/or antithrombotic trials [51,52,53,54,55,56,57,58,59,60,61,62,63].
The most commonly used article keywords were “clinical trial” (16.1%; n = 6332 papers), followed by “therapy” (10.8%; n = 4267), “randomised controlled trial” (6.6%; n = 2587), “chemotherapy” (5.6%; n = 2224), “risk” (5.1%; n = 2026), “efficacy” (4.9%; n = 1933) and “double-blind” (4.9%; n = 1929). The most frequently used keywords in the most prolific journals are shown in Table 7. In addition, exploratory analyses of word clouds and networks based on keywords (co-words) showed the broad range of the topics covered (see Additional file 6).
In this cross-sectional analysis, we presented a global mapping of RCT-related articles published in high-IF medical journals for the period 1965–2017. We identified the most prolific scientists, institutions and countries, main funding sources, most common subjects and topics, “citation classics” and most prolific high-IF medical journals from multiple specialties over the last 50 years.
In general, we found a strong clustering of articles published in British and American medical journals (The Lancet, Journal of Clinical Oncology, The New England Journal of Medicine, The BMJ, Circulation, JAMA, JACC and Diabetes Care accounted for 53% of the RCT-related articles). Many of these journals have been developed by active medical associations, both nationally and internationally. We hypothesize that different publishing patterns between journals may potentially reflect editorial policies and/or preferences, with some general medicine journals (such as The Lancet and The New England Journal of Medicine) and specialty journals (such as Journal of Clinical Oncology and Circulation), being more interested in and/or promoting the publication of RCTs. In contrast, a substantial number of these articles are behind publication paywalls (very few of the medical journals in our study sample are Open Access), and thus, research results may not be accessible to a large fraction of the scientific community and society as a whole, including clinicians (and patients) who may want them to help inform their clinical practice.
The results of this study highlight the expanding collaborative networks between countries in multiple regions, revealing a discernible scientific community, with the most productive countries having an important number of collaborations. Publication activity efforts were global during the study period, with articles from scientists and institutions in more than 150 different countries. However, the scientific community is centred on a nucleus of scientists from Western countries, with the most intense global collaborations taking place among the United States, United Kingdom and Canada. The presence and influence that these countries have on biomedical research [64,65,66] may be due to their large multi-stakeholder research partnerships, greater financial investment in clinical research, and high population of active scientists and research centres compared to other countries.
Publication activity worldwide shows that low- and middle-income countries have low levels of articles in high-IF medical journals. Difficulties in healthcare, education and research systems, information access and communication, language barriers and economic and institutional instability all represent challenges (and clear disadvantages) for productivity in low- and middle-income regions. In addition, restrictions and difficulties in conducting clinical research in resource-poor situations result in the exclusion of many of these countries from the planning, conduct and publication of RCTs [67,68,69]. As might be expected, our results support previous findings that low- and middle-income countries [31, 70, 71] had minimal contributions in articles published in major medical journals. For example, a previous study  showed that most of the authors of original papers published in five high-impact general medical journals (including The New England Journal of Medicine, The Lancet, JAMA, The BMJ and Annals of Internal Medicine) were more frequently affiliated with institutions in the same country as the journal. To address some of these problems, scientists, institutions and funders should promote collaborations (beyond historical, cultural and political factors) to share knowledge, expertise and innovative methodologies for clinical research. This may involve partnerships with Western countries to support capacity and resource development and research training.
RCT-related articles were published most often in high IF medical journals devoted to general and internal medicine, cardiology and oncology (nearly 57% of all articles). Similarly, the lists of the most cited articles identified topics which reflect major advances in the management of chronic conditions (such diabetes, cardiovascular disorders and cancer). The large relative productivity in general internal medicine, cardiology and oncology may be explained by the important role of randomised evidence to novel treatments and preventive strategies for these chronic diseases. In line with previous research [72,73,74,75], most of these highly cited RCTs addressed interventions for burdensome conditions that are health priorities in Western countries [76, 77]. Funding of (international, collaborative) RCTs may come from varying sources including commercial and non-commercial sponsors. However, previous analyses of RCT-related articles published in high-IF journals have suggested that study sponsors may influence how RCTs are designed, conducted and reported, sometimes serving financial rather than public interests . Given that research funding is often restricted, the scientific community is responsible for using the available resources most efficiently when exploring research priorities to afford knowledge users and population health needs [76, 77, 79, 80].
Our findings suggest that women are vastly underrepresented in the group of most prolific scientists publishing in high-impact medical journals. This is in direct contrast to recent studies that have identified a gender gap in research publications [81,82,83,84]. For example, a previous study  showed that women in first authorship positions increased from 27% in 1994 to 37% in 2014 in leading medical journals (including Annals of Internal Medicine, JAMA Internal Medicine, The BMJ, JAMA, The Lancet and The New England Journal of Medicine), but progress has plateaued or declined since 2009. An urgent need exists to investigate the underlying causes of the potential gender gap to help identify publication practices and strategies to increase women’s influence [82, 84].
Several limitations exist in our study. First, we characterised the knowledge structures generated by a large number of articles published in major medical journals that are included in the WoS database. However, our results are limited to a subset of all clinical-trial-related articles published in 40 leading medical journals. We suspect that these articles represent those that have great implications for clinical practice and that are relevant to clinical practice guidelines and healthcare regulators. Although the publication production analysed has been drawn from an exhaustive analysis of the biomedical literature, possibly, the search missed some relevant articles (and journals). Some reports may be published in journals without being indexed as RCTs, making them difficult to identify. Second, as in many bibliometric analyses, the normalisation of the different names of an author, country and funding sources is fundamentally important to avoiding potential errors. We conducted a careful manual validation of the references and textual data to avoid typographical, transcription and/or indexing errors. However, we recognize this procedure does not assure complete certainty. Third, the affiliation addresses of authors do not necessarily reflect the country where the research was conducted or the research funding source. Fourth, topical analysis that extracts a set of unique keywords, word profiles and co-words may indicate intellectual organization in publication production, albeit with inherent limitations [85, 86]. Fifth, the use of citation analysis carries some problems [87,88,89,90,91]. A potential length time-effect bias exists, which puts the more recent articles at a disadvantage. In addition, the biomedical literature is rich in barriers and motivations for publication and citation preferences , including self-citation (bias towards one’s own work) , language bias (bias towards publishing and citing English articles), omission bias (bias whereby competitors are purposely not cited), and selective reporting and publication bias (bias in which “negative” results are withheld from publication and citation) [89,90,91,92]. In addition, citations are also treated as equal regardless of whether research is being cited for its positive contribution to the field or because it is being criticized. Finally, our methods represent only a mapping approach, which could be complemented further by more detailed analyses such as by examining the content (e.g. differences in journal or author characteristics between publicly funded and industry-funded studies, designs/methodology, etc.), the reporting and the reproducible research practices through research of research (“meta-research”) studies [92,93,94,95,96,97,98].
The global analysis presented in this study provides evidence of the scientific growth of RCT- related articles published in high-IF medical journals. Over the last 50 years, publication activity in leading medical journals has increased, with Western countries (most notably, the United States) leading but with low- and middle-income countries showing very limited representation. Our analysis contributes to a better conceptualization and understanding of RCT articles and identified the main areas of research, the most influential publication sources chosen for their scientific dissemination and the major scientific leaders. Given the dynamic nature of the field, whether the growth trend remains the same in the coming years and how the characteristics of the field change over time will be interesting to see.
Availability of data and materials
With the publication of this manuscript, the full dataset will be freely available online in the Open Science Framework (https://osf.io/r2vw5/), a secure online repository for research data.
Schulz KF, Grimes DA. The Lancet Handbook of essential concepts in clinical research. London: Elsevier Ltd.; 2006.
Gabriel SE, Normand SL. Getting the methods right--the foundation of patient-centered outcomes research. N Engl J Med. 2012;367(9):787–90.
Jadad AR. Randomised controlled trials: a user’s guide. London: BMJ Books; 1998.
Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA. 1996;276(8):637–9.
Tsay MY, Yang YH. Bibliometric analysis of the literature of randomized controlled trials. J Med Libr Assoc. 2005;93(4):450–8.
Yusuf S, Collins R, Peto R. Why do we need some large, simple randomized trials? Stat Med. 1984;3(4):409–22.
Roundtable on Value and Science-Driven Health Care, Forum on Drug Discovery, Development, and Translation, Board on Health Sciences Policy, Institute of Medicine. Large simple trials and knowledge generation in a learning health system: workshop summary. Washington (DC): National Academies Press; 2013.
Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996;312(7023):71–2.
Zarin DA, Tse T, Williams RJ, Carr S. Trial reporting in ClinicalTrials.gov - the final rule. N Engl J Med. 2016;375(20):1998–2004.
Zarin DA, Tse T, Ide NC. Trial registration at ClinicalTrials.gov between May and October 2005. N Engl J Med. 2005;353(26):2779–87.
Taichman DB, Backus J, Baethge C, Bauchner H, de Leeuw PW, Drazen JM, et al. Sharing clinical trial data: a proposal from the International Committee of Medical Journal Editors. PLoS Med. 2016;13(1):e1001950.
Naudet F, Sakarovitch C, Janiaud P, Cristea I, Fanelli D, Moher D, et al. Data sharing and reanalysis of randomized controlled trials in leading biomedical journals with a full data sharing policy: survey of studies published in The BMJ and PLOS Medicine. BMJ. 2018;360:k400.
Moher D, Schulz KF, Altman D. CONSORT Group (Consolidated Standards of Reporting Trials). The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA. 2001;285(15):1987–91.
Schulz KF, Altman DG, Moher D, CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. Trials. 2010;11:32.
Chan AW, Tetzlaff JM, Altman DG, Laupacis A, Gøtzsche PC, Krleža-Jerić K, et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med. 2013;158(3):200–7.
Thelwall M, Kousha K. Are citations from clinical trials evidence of higher impact research? An analysis of ClinicalTrials.gov. Scientometrics. 2016;109(2):1341–51.
Brueton VC, Vale CL, Choodari-Oskooei B, Jinks R, Tierney JF. Measuring the impact of methodological research: a framework and methods to identify evidence of impact. Trials. 2014;15:464.
Hays M, Andrews M, Wilson R, Callender D, O'Malley PG, Douglas K. Reporting quality of randomised controlled trial abstracts among high-impact general medical journals: a review and analysis. BMJ Open. 2016;6(7):e011082.
Ghimire S, Kyung E, Kang W, Kim E. Assessment of adherence to the CONSORT statement for quality of reports on randomized controlled trial abstracts from four high-impact general medical journals. Trials. 2012;13:77.
Charles P, Giraudeau B, Dechartres A, Baron G, Ravaud P. Reporting of sample size calculation in randomised controlled trials: review. BMJ. 2009;338:b1732.
Contopoulos-Ioannidis DG, Karvouni A, Kouri I, Ioannidis JP. Reporting and interpretation of SF-36 outcomes in randomised trials: systematic review. BMJ. 2009;338:a3006.
Pitrou I, Boutron I, Ahmad N, Ravaud P. Reporting of safety results in published reports of randomized controlled trials. Arch Intern Med. 2009;169(19):1756–61.
Ebrahim S, Montoya L, Kamal El Din M, Sohani ZN, Agarwal A, Bance S, et al. Randomized trials are frequently fragmented in multiple secondary publications. J Clin Epidemiol. 2016;79:130–9.
Alsheikh-Ali AA, Qureshi W, Al-Mallah MH, Ioannidis JP. Public availability of published research data in high-impact journals. PLoS One. 2011;6(9):e24357.
Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P. Comparison of registered and published primary outcomes in randomized controlled trials. JAMA. 2009;302(9):977–84.
Trikalinos NA, Evangelou E, Ioannidis JP. Falsified papers in high-impact journals were slow to retract and indistinguishable from nonfraudulent papers. J Clin Epidemiol. 2008;61(5):464–70.
Siontis KC, Evangelou E, Ioannidis JP. Magnitude of effects in clinical trials published in high-impact general medical journals. Int J Epidemiol. 2011;40(5):1280–91.
Ioannidis JP. Contradicted and initially stronger effects in highly cited clinical research. JAMA. 2005;294(2):218–28.
Haynes RB, McKibbon KA, Wilczynski NL, Walter SD, Werre SR, Hedges Team. Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytical survey. BMJ. 2005;330(7501):1179.
Sugimoto CR, Larivière V. Measuring research: what everyone needs to know®. Oxford: Oxford University Press; 2018.
Catalá-López F, Alonso-Arroyo A, Hutton B, Aleixandre-Benavent R, Moher D. Global collaborative networks on meta-analyses of randomized trials published in high impact factor medical journals: a social network analysis. BMC Med. 2014;12:15.
Börner K, Polley DE. Visual insights: a practical guide to making sense of data. Cambridge, Massachusetts: MIT Press; 2014.
Batagelj V, Mrvar A. Pajek 1.28. Program for large network analysis. Ljubljana: University of Ljubljana; 2010.
Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.
Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1(8476):307–10.
Diabetes Control and Complications Trial Research Group, Nathan DM, Genuth S, Lachin J, Cleary P, Crofford O, et al. The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. N Engl J Med. 1993;329(14):977–86.
UK Prospective Diabetes Study (UKPDS) Group. Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33). Lancet. 1998;352(9131):837–53.
Knowler WC, Barrett-Connor E, Fowler SE, Hamman RF, Lachin JM, Walker EA, Nathan DM, Diabetes Prevention Program Research Group. Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. N Engl J Med. 2002;346(6):393–403.
Tuomilehto J, Lindström J, Eriksson JG, Valle TT, Hämäläinen H, Ilanne-Parikka P, et al. Prevention of type 2 diabetes mellitus by changes in lifestyle among subjects with impaired glucose tolerance. N Engl J Med. 2001;344(18):1343–50.
UK Prospective Diabetes Study (UKPDS) Group. Effect of intensive blood-glucose control with metformin on complications in overweight patients with type 2 diabetes (UKPDS 34). Lancet. 1998;352(9131):854–65 PubMed PMID: 9742977.
Action to Control Cardiovascular Risk in Diabetes Study Group. Effects of intensive glucose lowering in type 2 diabetes. N Engl J Med. 2008;358(24):2545–59.
Rossouw JE, Anderson GL, Prentice RL, LaCroix AZ, Kooperberg C, Stefanick ML, et al. Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results From the Women's Health Initiative randomized controlled trial. JAMA. 2002;288(3):321–33.
Hulley S, Grady D, Bush T, Furberg C, Herrington D, Riggs B, et al. Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. Heart and Estrogen/progestin Replacement Study (HERS) Research Group. JAMA. 1998;280(7):605–13.
Stupp R, Mason WP, van den Bent MJ, Weller M, Fisher B, Taphoorn MJ, et al. Radiotherapy plus concomitant and adjuvant temozolomide for glioblastoma. N Engl J Med. 2005;352(10):987–96.
Hurwitz H, Fehrenbacher L, Novotny W, Cartwright T, Hainsworth J, Heim W, et al. Bevacizumab plus irinotecan, fluorouracil, and leucovorin for metastatic colorectal cancer. N Engl J Med. 2004;350(23):2335–42.
Slamon DJ, Leyland-Jones B, Shak S, Fuchs H, Paton V, Bajamonde A, et al. Use of chemotherapy plus a monoclonal antibody against HER2 for metastatic breast cancer that overexpresses HER2. N Engl J Med. 2001;344(11):783–92.
Fisher B, Costantino JP, Wickerham DL, Redmond CK, Kavanah M, Cronin WM, et al. Tamoxifen for prevention of breast cancer: report of the National Surgical Adjuvant Breast and Bowel Project P-1 Study. J Natl Cancer Inst. 1998;90(18):1371–88.
Hodi FS, O'Day SJ, McDermott DF, Weber RW, Sosman JA, Haanen JB, et al. Improved survival with ipilimumab in patients with metastatic melanoma. N Engl J Med. 2010;363(8):711–23.
Chapman PB, Hauschild A, Robert C, Haanen JB, Ascierto P, Larkin J, et al. Improved survival with vemurafenib in melanoma with BRAF V600E mutation. N Engl J Med. 2011;364(26):2507–16.
Llovet JM, Ricci S, Mazzaferro V, Hilgard P, Gane E, Blanc JF, et al. Sorafenib in advanced hepatocellular carcinoma. N Engl J Med. 2008;359(4):378–90. https://doi.org/10.1056/NEJMoa0708857.
Pedersen TR, Kjekshus J, Berg K, Haghfelt T, Faergeman O, Thorgeirsson G, et al. Randomised trial of cholesterol lowering in 4444 patients with coronary heart disease: the Scandinavian Simvastatin Survival Study (4S). Lancet. 1994;344(8934):1383–9.
Shepherd J, Cobbe SM, Ford I, Isles CG, Lorimer AR, MacFarlane PW, et al. Prevention of coronary heart disease with pravastatin in men with hypercholesterolemia. N Engl J Med. 1995;333(20):1301–7.
Sacks FM, Pfeffer MA, Moye LA, Rouleau JL, Rutherford JD, Cole TG, et al. The effect of pravastatin on coronary events after myocardial infarction in patients with average cholesterol levels. Cholesterol and Recurrent Events Trial investigators. N Engl J Med. 1996;335(14):1001–9.
Collins R, Armitage J, Parish S, Sleight P, Peto R. Heart Protection Study Collaborative Group. MRC/BHF Heart Protection Study of cholesterol lowering with simvastatin in 20.536 high-risk individuals: a randomised placebo-controlled trial. Lancet. 2002;360(9326):7–22.
Heart Outcomes Prevention Evaluation Study Investigators, Yusuf S, Sleight P, Pogue J, Bosch J, Davies R, Dagenais G. Effects of an angiotensin-converting-enzyme inhibitor, ramipril, on cardiovascular events in high-risk patients. N Engl J Med. 2000;342(3):145–53.
Pitt B, Zannad F, Remme WJ, Cody R, Castaigne A, Perez A, et al. The effect of spironolactone on morbidity and mortality in patients with severe heart failure. Randomized Aldactone Evaluation Study Investigators. N Engl J Med. 1999;341(10):709–17.
Pfeffer MA, Braunwald E, Moyé LA, Basta L, Brown EJ Jr, Cuddy TE, et al. Effect of captopril on mortality and morbidity in patients with left ventricular dysfunction after myocardial infarction. Results of the survival and ventricular enlargement trial. N Engl J Med. 1992;327(10):669–77.
Brenner BM, Cooper ME, de Zeeuw D, Keane WF, Mitch WE, Parving HH, et al. Effects of losartan on renal and cardiovascular outcomes in patients with type 2 diabetes and nephropathy. N Engl J Med. 2001;345(12):861–9.
Lewis EJ, Hunsicker LG, Bain RP, Rohde RD. The effect of angiotensin-converting-enzyme inhibition on diabetic nephropathy. The Collaborative Study Group. N Engl J Med. 1993;329(20):1456–62.
National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group. Tissue plasminogen activator for acute ischemic stroke. N Engl J Med. 1995;333(24):1581–7.
Connolly SJ, Ezekowitz MD, Yusuf S, Eikelboom J, Oldgren J, Parekh A, et al. Dabigatran versus warfarin in patients with atrial fibrillation. N Engl J Med. 2009;361(12):1139–51.
Yusuf S, Zhao F, Mehta SR, Chrolavicius S, Tognoni G, Fox KK. Clopidogrel in Unstable Angina to Prevent Recurrent Events Trial Investigators. Effects of clopidogrel in addition to aspirin in patients with acute coronary syndromes without ST-segment elevation. N Engl J Med. 2001;345(7):494–502.
CAPRIE Steering Committee, Gent M, Beaumont D, Blanchard J, Bousser MG, Coffman J, Easton JD, et al. A randomised. blinded. trial of clopidogrel versus aspirin in patients at risk of ischaemic events (CAPRIE). Lancet. 1996;348(9038):1329–39.
Røttingen JA, Regmi S, Eide M, Young AJ, Viergever RF, Ardal C, et al. Mapping of available health research and development data: what's there, what's missing, and what role is there for a global observatory? Lancet. 2013;382(9900):1286–307.
Moses H 3rd, Matheson DH, Cairns-Smith S, George BP, Palisch C, Dorsey ER. The anatomy of medical research: US and international comparisons. JAMA. 2015;313(2):174–89.
Sussex J, Feng Y, Mestre-Ferrandiz J, Pistollato M, Hafner M, Burridge P, et al. Quantifying the economic impact of government and charity funding of medical research on private research and development funding in the United Kingdom. BMC Med. 2016;14:32.
Shapiro HT, Meslin EM. Ethical issues in the design and conduct of clinical trials in developing countries. N Engl J Med. 2001;345(2):139–42.
Berkley JA. Randomised trials in developing countries. Arch Dis Child. 2014;99(7):607–8.
Knepper TC, McLeod HL. When will clinical trials finally reflect diversity? Nature. 2018;557(7704):157–9.
Sumathipala A, Siribaddana S, Patel V. Under-representation of developing countries in the research literature: ethical issues arising from a survey of five leading medical journals. BMC Med Ethics. 2004;5:E5.
Uthman OA, Okwundu CI, Wiysonge CS, Young T, Clarke A. Citation classics in systematic reviews and meta-analyses: who wrote the top 100 most cited articles? PLoS One. 2013;8(10):e78517.
Atal I, Trinquart L, Ravaud P, Porcher R. A mapping of 115,000 randomized trials revealed a mismatch between research effort and health needs in non-high-income regions. J Clin Epidemiol. 2018;98:123–32.
Emdin CA, Odutayo A, Hsiao AJ, Shakir M, Hopewell S, Rahimi K, Altman DG. Association between randomised trial evidence and global burden of disease: cross sectional study (Epidemiological Study of Randomized Trials--ESORT). BMJ. 2015;350:h117.
Catalá-López F, García-Altés A, Alvarez-Martín E, Gènova-Maleras R, Morant-Ginestar C. Does the development of new medicinal products in the European Union address global and regional health concerns? Popul Health Metr. 2010;8:34.
Isaakidis P, Swingler GH, Pienaar E, Volmink J, Ioannidis JP. Relation between burden of disease and randomised evidence in sub-Saharan Africa: survey ofresearch. BMJ. 2002;324(7339):702.
GBD 2017 Disease and Injury Incidence and Prevalence Collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet. 2018;392(10159):1789–858.
GBD 2017 DALYs and HALE Collaborators. Global, regional, and national disability-adjusted life-years (DALYs) for 359 diseases and injuries and healthy life expectancy (HALE) for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet. 2018;392(10159):1859–922.
Rasmussen K, Bero L, Redberg R, Gøtzsche PC, Lundh A. Collaboration between academics and industry in clinical trials: cross sectional study of publications and survey of lead academic authors. BMJ. 2018;363:k3654.
Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, et al. How to increase value and reduce waste when research priorities are set. Lancet. 2014;383(9912):156–65.
Moher D, Glasziou P, Chalmers I, Nasser M, Bossuyt PM, Korevaar DA, et al. Increasing value and reducing waste in biomedical research: who's listening? Lancet. 2016;387(10027):1573–86.
van den Besselaar P, Sandström U. Vicious circles of gender bias, lower positions, and lower performance: gender differences in scholarly productivity and impact. PLoS One. 2017;12(8):e0183301.
Lundine J, Bourgeault IL, Clark J, Heidari S, Balabanova D. The gendered system of academic publishing. Lancet. 2018;391(10132):1754–6.
Lundine J, Bourgeault IL, Clark J, Heidari S, Balabanova D. Gender bias in academia. Lancet. 2019;393(10173):741–3.
Filardo G, da Graca B, Sass DM, Pollock BD, Smith EB, Martinez MA. Trends and comparison of female first authorship in high impact medical journals: observational study (1994–2014). BMJ. 2016;352:i847.
Leydesdorff L. Words and co-words as indicators of intellectual organization. Research Policy. 1989;18(4):209–23.
Leydesdorff L. Why words and co-words cannot map the development of the sciences. J Am Soc Inf Sci. 1997;48(5):418–27.
Greenberg SA. How citation distortions create unfounded authority: analysis of a citation network. BMJ. 2009;339:b2680.
Mishra S, Fegley BD, Diesner J, Torvik VI. Self-citation is the hallmark of productive authors, of any gender. PLoS One. 2018;13(9):e0195773.
Dwan K, Gamble C, Williamson PR, Kirkham JJ, Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review. PLoS One. 2013;8(7):e66844.
Phillips AT, Desai NR, Krumholz HM, Zou CX, Miller JE, Ross JS. Association of the FDA Amendment Act with trial registration, publication, and outcome reporting. Trials. 2017;18(1):333.
Goldacre B, Drysdale H, Dale A, Milosevic I, Slade E, Hartley P, et al. COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time. Trials. 2019;20(1):118.
Goldacre B, Drysdale H, Marston C, Mahtani KR, Dale A, Milosevic I, et al. COMPare: Qualitative analysis of researchers' responses to critical correspondence on a cohort of 58 misreported trials. Trials. 2019;20(1):124.
Dechartres A, Bond EG, Scheer J, Riveros C, Atal I, Ravaud P. Reporting of statistically significant results at ClinicalTrials.gov for completed superiority randomized controlled trials. BMC Med. 2016;14(1):192.
Dechartres A, Trinquart L, Atal I, Moher D, Dickersin K, Boutron I, et al. Evolution of poor reporting and inadequate methods over time in 20 920 randomised controlled trials included in Cochrane reviews: research on research study. BMJ. 2017;357:j2490.
Becker JE, Krumholz HM, Ben-Josef G, Ross JS. Reporting of results in ClinicalTrials.gov and high-impact journals. JAMA. 2014;311(10):1063–5.
Bartlett VL, Dhruva SS, Shah ND, Ryan P, Ross JS. feasibility of using real-world data to replicate clinical trial evidence. JAMA Netw Open. 2019;2(10):e1912869.
Wallach JD, Boyack KW, Ioannidis JPA. Reproducible research practices, transparency, and open access data in the biomedical literature, 2015–2017. PLoS Biol. 2018;16(11):e2006930.
Ioannidis JP, Fanelli D, Dunne DD, Goodman SN. meta-research: evaluation and improvement of research methods and practices. PLoS Biol. 2015;13(10):e1002264.
Declaration of transparency
FC-L affirms that this manuscript is an honest, accurate and transparent account of the study being reported, that no important aspects of the study have been omitted, and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.
FC-L and RT-S are supported by the Institute of Health Carlos III/CIBERSAM. BH is supported by a New Investigator Award from the Canadian Institutes of Health Research and the Drug Safety and Effectiveness Network. DM is supported by a University Research Chair, University of Ottawa. The funders were not involved in the design of the study or the decision to submit the manuscript for publication nor were they involved in aspects of the study conduct. The views expressed in this manuscript are those of the authors and may not be understood or quoted as being made on behalf of, or as reflecting the position of, the funder(s) or any institution.
Ethics approval and consent to participate
This study outlines a cross-sectional analysis of secondary data and, hence, does not require ethical approval or consent to participate.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Full strategy in PubMed/MEDLINE.
Data extraction and normalisation processes.
List of PMID for included articles.
List of excluded articles.
Exploratory analysis of topical data.
About this article
Cite this article
Catalá-López, F., Aleixandre-Benavent, R., Caulley, L. et al. Global mapping of randomised trials related articles published in high-impact-factor medical journals: a cross-sectional analysis. Trials 21, 34 (2020). https://doi.org/10.1186/s13063-019-3944-9
- Evidence-based medicine
- Randomized controlled trial
- Scientific collaboration