Characterisation of trials where marketing purposes have been influential in study design: a descriptive study
© Barbour et al. 2016
Received: 6 July 2015
Accepted: 4 December 2015
Published: 21 January 2016
Analysis of trial documentation has revealed that some industry-funded trials may be done more for marketing purposes than scientific endeavour. We aimed to define characteristics of drug trials that appear to be influenced by marketing considerations and estimate their prevalence.
We examined reports of randomised controlled trials of drugs published in six general medical journals in 2011. Six investigators independently reviewed all publications, characterising them as YES/MAYBE/NO suspected marketing trials, and then met to reach consensus. Blinded researchers then extracted key trial characteristics. We used blinded cluster analysis to determine if key variables could characterise the categories of trials (YES/MAYBE/NO).
41/194 (21 %) trials were categorised as YES, 14 (7 %) as MAYBE, 139 (72 %) as NO. All YES and MAYBE trials were funded by the manufacturer, compared with 37 % of NO trials (p < 0.001). A higher proportion of YES trials had authors or contributors from the manufacturer involved in study design (83 % vs. 19 %), data analysis (85 % vs.15 %) and reporting (81 % vs. 15 %) than NO trials (p < 0.001). There was no significant difference between groups in the median number of participants screened (p = 0.49), but the median number of centres recruiting participants was higher for YES compared with NO trials (171 vs. 13, p < 0.001). YES trials were not more likely to use a surrogate (42 % vs. 30 %; p = 0.38) or composite primary outcome measure (34 % vs. 19 %; p = 0.14) than NO trials. YES trials were often better reported in terms of blinding, safety outcomes and adverse events than NO trials. YES trials more frequently included speculation that might encourage clinicians to use the intervention outside of the study population compared to NO trials (59 % vs.37 %, p = 0.03). Cluster analysis based on study characteristics did not identify a clear variable structure that accurately characterised YES/MAYBE/NO trials.
We reached consensus that a fifth of drug trials published in the highest impact general medical journals in 2011 had features that were suggestive of being designed for marketing purposes. Each of the marketing trials appeared to have a unique combination of features reported in the journal publications.
KeywordsTrials Marketing trials Publication General Medical Journals
Randomised clinical trials (RCTs) are done to reduce uncertainty over the efficacy and safety of an intervention and should be designed in such a way that they yield as much useful and unbiased information as possible from as few participants and clinical events as is necessary. However, analysis of trial documentation has revealed that some industry-funded drug trials may be done more for marketing purposes than science . Confidential internal company documents have revealed drug trials designed, or heavily influenced by, the marketing departments of pharmaceutical companies [1, 2]. These trials have often been referred to as marketing or seeding trials [3, 4].
Currently, there are few documented examples of marketing trials. Nonetheless, there are reasons why marketing trials should be of concern to patients and physicians. Notably, the true research objectives – to promote the use of a medical product – may not be clear to investigators and communicated to participants. The features that suggest a trial may be considered as marketing are, however, currently unclear. Vested interests, recruitment of investigators who are frequent prescribers of competing products; disproportionally high payments given to investigators; sponsorship by the company’s sales and marketing division; minimal requirements for data leading to poor data quality and recruitment of a large number of centres have all been suggested as features of a trial designed for marketing purposes [3, 4].
Pharmaceutical companies funding, designing and conducting drug trials have rational motives for establishing and advertising evidence of product efficacy and safety in order to increase prescribing of the drug during the window before patent expiration [1, 2]. Yet, the existence of marketing trials, and their provenance is difficult to prove in the absence of confidential internal company documents [1, 2]. Even less is known about their characteristics and how marketing considerations may intrude on trials of scientific and clinical value. Accordingly, the objective of this descriptive study was to define the characteristics of trials published in major medical journals that had features suggestive of marketing trials to inform efforts to better identify marketing considerations in trial design and reporting.
Phase 1: estimating the prevalence of suspected marketing-driven trials in leading general medical journals
Scoping of marketing trial features
In order to reach a broad consensus on the characteristics of a marketing trial, which have not previously been defined, the study team met to share 24 examples of published trials one or more of us suspected of being designed partly for purposes of marketing. Based on this discussion and features of confirmed marketing trials [1–4] six characteristics of marketing-influenced trials were proposed and agreed upon by raters: 1) a high level of involvement of the product manufacturer in study design, 2) data analysis, 3) and reporting of the study, 4) recruitment of small numbers of patients from numerous study sites for a common disease when they could have been recruited without difficulty from fewer sites, 5) misleading abstracts that do not report clinically relevant findings, and 6) conclusions that focus on secondary end-points and surrogate markers.
We generated a list of all trials evaluating one or more drug treatments published in 2011 in the top general medical journals, based on Impact Factor (New England Journal of Medicine (NEJM), The Journal of the American Medical Association (JAMA), Lancet, Annals of Internal Medicine, PLOS Medicine, and The BMJ). We included human drug and vaccine RCTs (excluding single patient and single arm trials). Papers describing subgroup analysis or just trial follow up data were excluded. Trials of devices were also excluded unless the device contained a drug e.g. drug eluting stents. We also excluded all studies that explicitly stated they were phase I or dosing escalation studies, but did not exclude phase II trials (or phase I studies that did not explicitly state their phase).
Independent rating of trials
Six members of the study team (VB, FG, RL, DB, CH, JR) independently rated included trials based on whether they were suspected to be marketing trials (YES, NO, or MAYBE). They did not use fixed criteria, but based their decisions on the extent to which they each felt the six characteristics of marketing-influenced trials described above influenced the research reported. Raters were not blinded to the journal, as successful blinding was not achievable with a team so familiar with the differences in journal formatting styles. One author [SS] then collated the independent ratings.. Trials with ≥4 independent YES ratings were categorised as suspected marketing trials (referred to as YES trials), whilst trials with ≥ 3 independent NO ratings were classified as non-marketing trials (NO trials). Trials with <3 NO and <4 Yes ratings were considered to be possible marketing trials and a consensus meeting was convened to discuss these in more depth. The initial independent ratings were collated and presented to the consensus group and then a consensus decision to assign a YES/MAYBE/NO for each of these trials was reached through discussion. VB and FG were excluded from rating trials that were published in their own journals (PLOS Medicine and The BMJ). For these trials, five ratings were collated, not six, and trials with ≥ 4 independent NO ratings were classified as non-marketing trials (NO trials).
Phase 2: characterisation of suspected marketing trials
In the second phase we sought to extract key variables, including basic information about the trials and independent assessments of their clinical relevance and quality to see if suspected marketing trials share characteristics that differentiate them from the other trials.
The study team developed a data extraction form (see Additional file 1) to capture variables that might categorise a trial as suspected marketing or not. Three members of the team (JR, VB, DB) and one independent data extractor piloted the form for ease of completion. A further eight independent data extractors were recruited to undertake further data extraction. They were blinded to our study objectives and the categorisation of the trials and were given instructions for gathering specific variables to encourage consistency. To assess clinical relevance and trial quality of each included study we extracted data from the McMaster PLUS (Premium LiteratUre Service) database of scientifically robust studies .
Correspondence with editors
To gather further information to characterise these studies, and the degree to which marketing interests might have influenced them, we wrote to the editors of the participating journals where we suspected trials were marketing (see Additional file 2). We asked editors if they considered that they might partly be marketing trials when they were being reviewed, and how their editorial teams approach the possibility of marketing as a driver in trials. To aid our understanding and inform our analysis we also asked for editors to share in confidence copies of the anonymised peer review comments relating to these articles.
Survey of authors
We emailed the corresponding author of all 194 trials to invite them to complete a survey (see Additional file 3) containing further questions about the design and conduct of the trials. We were transparent in explaining the purpose of our study and included a link to our published abstract presented at the Seventh International Congress on Peer Review and Biomedical Publication . Non-responders were sent one email reminder.
Initial data was collated in an Excel spreadsheet by SS and the analysis was performed using SPSS (version 18.0). We summarised data using proportions and used Chi-square and Kruskal Wallis tests to make comparisons across the three chosen categories (YES/MAYBE/NO). As the MAYBE trials were also suspected marketing trials we ran a sensitivity analysis to see if it made a difference to the results if the MAYBE trials were combined with the YES trials versus the NO trials. For this sensitivity analysis (two categories only = YES/NO) we used Chi-square and Mann Whitney U Tests.
To determine if the key variables extracted could correctly identify the three study categories (YES/MAYBE/NO) RP carried out a cluster analysis in R (version 3.1.0). Briefly, these methods use all categorical and numerical variables to determine a measure of ‘similarity’ between studies. Studies that have exactly the same values for all variables will have highest similarity (typically = 1) and those with completely different values the lowest (typically = 0). These similarities are then used to determine how close studies and clusters of studies are to each other hence defining membership to a selected number of clusters. The ‘internal structure’ of these clusters – tight groups within clusters and large differences across clusters – can be used to determine if the data show important differences between these studies – in our case MARKETING vs. NOT-MARKETING. We used a “gower” metric to account for a mixture of continuous and categorical data and “partitioning around medoids” (PAM - similar to k-means) to create the clusters . The number of clusters was set at 2, 3, 4 and 5 and silhouette graphs were used to test for internal structure with values of the average silhouette width above 0.5 required to determine that at least a reasonable structure has been identified . We also used dendrograms to explore how these studies group and in particular looking for a clear separation between marketing and not-marketing trials in relation to combinations of the variables. All cluster analyses were carried out blinded to the categorisation (YES/MAYBE/NO), which were only used to define consistency with membership allocation.
We did not seek ethical approval for our study. The main study did not involve humans and was mainly documentary analysis so did not require ethical approval. The only involvement of humans was the author survey. Authors did not give consent to take part; participation in the survey was considered an indication of consent. Authors were free to ignore our request to provide additional information about their studies.
Number (proportion) of trials categorised as suspected marketing trials by journal following the consensus meeting
Total number of eligible trials (%)
Annals of Internal Medicine
Authorship, writing, funding and manufacturer involvement characteristics by group
p value (YES vs. MAYBE vs. NO)
p value (YES+ MAYBE vs. NO)
p value (MAYBE + NO vs. YES)
n = 194
n = 41
n = 14
n = 139
Byline author from the product manufacturer?
Median (LQ, UQ) proportion of byline authors from product manufacturer
0 (0, 16)
22 (9, 38)
19 (0, 28)
0 (0, 0)
Byline author reporting a financial conflict of interest with the product manufacturer?c
Median (LQ, UQ) proportion of byline authors reporting COI with product manufacturerc
24 (0, 67)
82 (59, 100)
64 (49, 85)
6 (0, 40)
Group name on the byline
Is writing or editorial assistance in preparing the manuscript acknowledged?
Acknowledgements or main text
Professional writer is a byline author
No assistance is acknowledged
Study explicitly funded by product manufacturer?
Manufacturer involved in the design of the study?
Not explicitly described
Manufacturer involved in the data analysis?
Not explicitly described
Manufacturer involved in the reporting of the study?
Not explicitly described
Manufacturer control over the design of the study?
Not explicitly described
Manufacturer had control over the data analysis?
Not explicitly described
Manufacturer control over reporting of the study?
Not explicitly described
YES/MAYBE trials were more likely to have at least one author from the product manufacturer on the authorship byline (p < 0.001) and at least one author with a declared financial conflict of interest with the product manufacturer compared with the NO trials, p < 0.001 (Table 2). YES/MAYBE trials were also more likely to report editorial assistance with preparing the manuscript (e.g. the use of professional writers or an indication of help with manuscript preparation in the Acknowledgements section) than NO trials, p = 0.001. There was no significant difference between groups in the use of group authorship on the byline (p = 0.70).
A higher proportion of YES trials (32 %) reported the use of statistical imputation than NO trials (9 %) and MAYBE trials (7 %), p = 0.001 (see Additional file 4). YES trials were no more likely to use a surrogate primary outcome measure (p = 0.38) or a composite primary outcome measure (p = 0.14). However the Abstract’s conclusions of YES trials (39 %) were more likely to focus on surrogate outcomes (7 % MAYBE, 25 % NO; p = 0.05). YES trials included speculation or generalised phrasing 59 % of the time that might encourage clinicians to use the intervention outside of the study population compared to 29 % for MAYBE and 37 % for NO trials (p = 0.03). However, the last two results were affected if the MAYBE trials were reclassified as YES (see Additional file 4). Grouping the MAYBE trials with the YES trials and performing a two group comparison did not alter substantially the pattern of results for other outcomes.
A higher proportion of YES trials blinded clinicians (83 % vs. 43 % MAYBE and 53 % NO, p = 0.03), reported safety outcomes and adverse events in the Abstract (83 % vs. 54 %, p < 0.001) and the main text of the paper (100 % vs. 84 %, p = 0.006), than NO trials. There was no difference in terms of type of trial picked up in Evidence Updates (p = 0.36), nor in median highest clinical relevance (p = 0.34) and newsworthiness (p = 0.42) ratings.
Cluster membership by categories
Number of Clusters = 2
Number of Clusters = 3
Number of Clusters = 4
Number of Clusters = 5
Correspondence with editors and authors
We received a response from editors at all four journals where YES trials were published. Editors did not agree that marketing considerations had played a key part in any of their published studies and noted to us the underlying importance and novelty of the clinical research described by the manuscripts. For reasons of confidentiality, none of the editors were willing to share peer reviewers’ reports or any editorial notes on the decision making for these manuscripts.
From the 194 survey invitations to corresponding authors we received a delivery failure notice for 11 (6 %). Of the 183 authors 55 (30 %) completed at least some of the questionnaire. The response rate was higher for authors of MAYBE (38 %) and NO (33 %) trials compared with YES trials (18 %). Due to the low response rate we are unable to draw firm conclusions from the data.
Six clinical research experts and medical journal editors proposed that 21 % of drug trials published in 2011 in the leading general medical journals had characteristics consistent with the aim of marketing the product. Use of expert opinion is necessarily subjective and also open to bias. Therefore our methods aimed to reduce this bias, or at least to explore and make it explicit, and to better classify what seemed to us to be the characteristics of trials designed with marketing in mind. To achieve this we used independent data extractors to try and identify a consistent set of characteristics of marketing trials. The proposed set of marketing trials were more likely to be explicitly funded by the manufacturer, have authors or contributors from the manufacturer involved in or in control of the study design, data analysis and reporting of trials. In addition, suspected marketing trials contained speculation or generalised phrasing that might encourage clinicians to use the intervention outside of the study population.
There was an unavoidably circular aspect to our efforts, since we started with the feeling that marketing was a strong element of a number of trials and attempted to define the characteristics that contributed to that impression. Moreover, we could not define any groupings, based on the extracted variables, which could delineate trials as marketing. Furthermore, the reported methods of most of the proposed marketing trials were high quality. Our preconceptions, therefore, may have been biased by prior knowledge of marketing or seeding studies. In effect, the group of experts through consensus were able to identify a group of suspected marketing trials, but this grouping could not be validated, and remains open to experimenter’s bias: a subjective bias whereby the result is overtly influenced by the experimenters, in this case the six raters. We call upon other independent researchers to further investigate this important topic.
One characteristic, though, is worth mentioning. The proposed group of marketing trials did report recruiting large numbers of patients from multiple countries and study sites yielding a very small average number of patients per centre. Recruitment from multiple sites is more expensive and harder to manage. Legitimate reasons for recruiting from a large number of centres might include the need to get sufficient numbers of patients in a shorter period, targeted enrolment for regulatory approval purposes, and the need to ensure a diverse geographical sample. Nonetheless, compared with interventional trials conducted without industry support, it is notable that trials identified by the study team as marketing with a YES score reported on common conditions where, in the opinion of the raters, patients could easily have been recruited from a smaller number of centres in a similar time period, with less risk to data quality and at reduced cost. Involvement in the design of the study could include recruitment strategies. The justification for specific recruitment strategies are not typically reported in journal articles, protocols or trial registries so this information can only be gathered from the manufacturers or the authors of the study. We tried but failed to obtain this information from individual trial authors.
The question as to whether marketing trials exist remains controversial. At one end of the spectrum past editors such as Marcia Angell (NEJM) and Richard Smith (The BMJ) have claimed medical journals have become the advertising agencies of pharmaceutical companies [9, 10]. However, this view was not supported by the editors of the journals we contacted. To reach conclusive proof of marketing intent would require confirmatory evidence, such as access to key confidential company documentation, which we did not have.
An absence of marketing, though, may undermine implementation of beneficial interventions. As an example, tranexamic acid in trauma has been proven to be effective  but has not been widely used in practice; had the trial been designed with marketing in mind, recruiting smaller numbers of patients from a considerably larger number of centres, it might have been more influential (as well as more expensive). Nor is it even necessarily unethical to conduct a trial purely as a marketing exercise. The ISIS-2 trial examined the effect of streptokinase versus placebo in acute myocardial infarction , despite the fact that the authors were already persuaded on the basis of a meta-analysis that streptokinase worked . The placebo-controlled trial of streptokinase was done not to discover a novel truth but because it was the most effective way to change medical practice. Such confirmatory trials are an important component of the evidence-base to ensure generalisability and the ability to replicate the intervention.
This study has manifest limitations. Firstly, we recognise that we are a self-selected group and other readers may not agree with our interpretation. Whilst some of us work, or have worked, for journals included in the analysis we have attempted to reduce this influence: no one individual made a determination and editors did not assess their own journal’s papers. Secondly, our team does not include clinical experts in all the diseases reported. However, three of the six raters are medical practitioners who see or have seen patients with most of the conditions reported in the included trials. Thirdly, without access to key confidential internal documents, we cannot conclusively know by whom, and with what purpose, the trials were designed. Fourthly, we only focused on RCTs whereas marketing studies may have a range of study designs and therefore the estimate proposed in this study may vary substantially. Fifthly, we only studied the large general medical journals and the problem we are trying to characterise and quantify may be more or less common in the wider universe of medical journals where most RCTs are published. Sixthly, the study team categorised the trials as YES/MAYBE/NO using just the main trial report. It is possible that they may have categorised trials differently if they assessed all the supplementary documentation accompanying the publications. Data extractors did extract data from the online supplementary files. Seventhly, contacting authors and editors did not offer confirmatory evidence of any of our findings. Finally, there was a lot of unreported information (for example the proportion of the sample who dropped out) and this may have influenced the cluster analysis. We took a pragmatic approach of sampling all trials published in six general medical journals in a particular year and it is possible that the study did not have sufficient statistical power.
This study revealed the difficulties of trying to apply a quantitative approach to characterising something as nebulous as a marketing trial. Inevitably our conclusions are more suited to start a debate than to settle it. The main issue raised is whether the current system of designing and publishing interventional trials does serve the best interests of patients and health practitioners. We hope that this research will trigger further debate on trials and remind editors, peer reviewers and readers of the need to closely scrutinise clinical trials for their design (including recruitment strategies and the role of the product manufacturer in the design, analysis and reporting of the study) as well as their findings.
Manufacturers have to market their drugs but marketing imperatives should not unduly influence the design of the studies and the clinical question under investigation. In this study six raters reached a consensus that a fifth of drug trials published in the top six general medical journals were suggestive of trials designed to a large degree for marketing purposes. We were unable to create a taxonomy for marketing-influenced trials using statistical methods suggesting that individual trials have a unique combination of features reported in the journal publications. The pattern of features makes marketing-influenced studies difficult to identify by the average reader.
Further guidance is warranted to alert funders, ethics review boards, editors, peer reviewers and readers to warning signs of marketing-influenced trials. Marketing trials arise because manufacturers continue to have such a dominant role in the design, conduct and reporting of human testing of their own products. We support the idea that the design, analysis and reporting of clinical trials should only be done by truly independent investigators.
Trial categorised as a suspected marketing trial
Trial categorised as a possible marketing trial
Trial categorised as not a marketing trial
We thank our independent data extractors (Lindsay Kobayashi, Andreea Steriu, Ronnie Kasirye, Siri Steinmo, Peijue Huangfu, Janice Zhang, Rebecca Geary and Gemma Archer); and Andreea Steriu for additional research assistance.
Disclosures of funding support and explanation of the role of the funder/sponsor
We received a £5000 research grant from the Committee on Publication Ethics (COPE). The funder played no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. JR is supported by the National Institute on Aging (K08 AG032886) and by the American Federation for Aging Research through the Paul B. Beeson Career Development Award Program.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Krumholz SD, Egilman DS, Ross JS. Study of Neurontin: Titrate to Effect, Profile of Safety (STEPS) Trial. Arch Intern Med. 2011;171(12):1100–7.View ArticlePubMedPubMed CentralGoogle Scholar
- Hill KP, Ross JS, Egilman DS, Krumholz HM. The ADVANTAGE seeding trial: review of internal documents. Ann Intern Med. 2008;149:251–8.View ArticlePubMedGoogle Scholar
- Kessler DA, Rose JL, Temple RJ, Schapiro R, Griffin JP. Therapeutic-class wars – drug promotion in a competitive marketplace. N Engl J Med. 1994;331(20):1350–3.View ArticlePubMedGoogle Scholar
- Sox HC, Rennie D. Seeding trials: just say “no”. Ann Intern Med. 2008;149(4):279–80.View ArticlePubMedGoogle Scholar
- McMaster Plus. http://hiru.mcmaster.ca/hiru/HIRU_McMaster_PLUS_Projects.aspx Accessed 11 November 2015.
- Barbour V, Burch D, Godlee F, Heneghan C, Lehman R, Ross J, et al. Characterization of trials designed primarily for marketing purposes rather than addressing genuine clinical questions: a descriptive study [abstract].2013. http://www.peerreviewcongress.org/2013/Plenary-Session-Abstracts-9-10.pdf p31. Accessed 14 Jan 2015.
- Reynolds A, Richards G, de la Iglesia B, Rayward-Smith V. Clustering rules: a comparison of partitioning and hierarchical clustering algorithms. J Math Model Algorithms. 2006;5:475–504.View ArticleGoogle Scholar
- Rousseeuw PJ. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math. 1987;20:53–65.View ArticleGoogle Scholar
- Smith R. Medical journals are an extension of the marketing arm of pharmaceutical companies. PLoS Med. 2005;2(5):e138. doi:10.1371/journal.pmed.0020138.View ArticlePubMedPubMed CentralGoogle Scholar
- Angell M. The truth about drug companies: how they deceive us and what to do about it. New York: Random House; 2005.Google Scholar
- Roberts I, Shakur H, Coats T, Hunt B, Balogun E, Barnetson L, et al. The CRASH-2 trial: a randomised controlled trial and economic evaluation of the effects of tranexamic acid on death, vascular occlusive events and transfusion requirement in bleeding trauma patients. Health Technol Assess. 2013;17(10):1–79. doi:10.3310/hta17100.View ArticleGoogle Scholar
- Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. Lancet.1988;2(8607):349–60.
- Yusuf S, Collins R, Peto R, Furberg C, Stampfer MJ, Goldhaber SZ, et al. Intravenous and intracoronary fibrinolytic therapy in acute myocardial infarction: overview of results on mortality, reinfarction and side-effects from 33 randomized controlled trials. Eur Heart J. 1985;6(7):556–85.PubMedGoogle Scholar