Characterisation of trials where marketing purposes have been influential in study design: a descriptive study

Background Analysis of trial documentation has revealed that some industry-funded trials may be done more for marketing purposes than scientific endeavour. We aimed to define characteristics of drug trials that appear to be influenced by marketing considerations and estimate their prevalence. Methods We examined reports of randomised controlled trials of drugs published in six general medical journals in 2011. Six investigators independently reviewed all publications, characterising them as YES/MAYBE/NO suspected marketing trials, and then met to reach consensus. Blinded researchers then extracted key trial characteristics. We used blinded cluster analysis to determine if key variables could characterise the categories of trials (YES/MAYBE/NO). Results 41/194 (21 %) trials were categorised as YES, 14 (7 %) as MAYBE, 139 (72 %) as NO. All YES and MAYBE trials were funded by the manufacturer, compared with 37 % of NO trials (p < 0.001). A higher proportion of YES trials had authors or contributors from the manufacturer involved in study design (83 % vs. 19 %), data analysis (85 % vs.15 %) and reporting (81 % vs. 15 %) than NO trials (p < 0.001). There was no significant difference between groups in the median number of participants screened (p = 0.49), but the median number of centres recruiting participants was higher for YES compared with NO trials (171 vs. 13, p < 0.001). YES trials were not more likely to use a surrogate (42 % vs. 30 %; p = 0.38) or composite primary outcome measure (34 % vs. 19 %; p = 0.14) than NO trials. YES trials were often better reported in terms of blinding, safety outcomes and adverse events than NO trials. YES trials more frequently included speculation that might encourage clinicians to use the intervention outside of the study population compared to NO trials (59 % vs.37 %, p = 0.03). Cluster analysis based on study characteristics did not identify a clear variable structure that accurately characterised YES/MAYBE/NO trials. Conclusions We reached consensus that a fifth of drug trials published in the highest impact general medical journals in 2011 had features that were suggestive of being designed for marketing purposes. Each of the marketing trials appeared to have a unique combination of features reported in the journal publications. Electronic supplementary material The online version of this article (doi:10.1186/s13063-015-1107-1) contains supplementary material, which is available to authorized users.


Background
Randomised clinical trials (RCTs) are done to reduce uncertainty over the efficacy and safety of an intervention and should be designed in such a way that they yield as much useful and unbiased information as possible from as few participants and clinical events as is necessary. However, analysis of trial documentation has revealed that some industry-funded drug trials may be done more for marketing purposes than science [1]. Confidential internal company documents have revealed drug trials designed, or heavily influenced by, the marketing departments of pharmaceutical companies [1,2]. These trials have often been referred to as marketing or seeding trials [3,4].
Currently, there are few documented examples of marketing trials. Nonetheless, there are reasons why marketing trials should be of concern to patients and physicians. Notably, the true research objectivesto promote the use of a medical productmay not be clear to investigators and communicated to participants. The features that suggest a trial may be considered as marketing are, however, currently unclear. Vested interests, recruitment of investigators who are frequent prescribers of competing products; disproportionally high payments given to investigators; sponsorship by the company's sales and marketing division; minimal requirements for data leading to poor data quality and recruitment of a large number of centres have all been suggested as features of a trial designed for marketing purposes [3,4].
Pharmaceutical companies funding, designing and conducting drug trials have rational motives for establishing and advertising evidence of product efficacy and safety in order to increase prescribing of the drug during the window before patent expiration [1,2]. Yet, the existence of marketing trials, and their provenance is difficult to prove in the absence of confidential internal company documents [1,2]. Even less is known about their characteristics and how marketing considerations may intrude on trials of scientific and clinical value. Accordingly, the objective of this descriptive study was to define the characteristics of trials published in major medical journals that had features suggestive of marketing trials to inform efforts to better identify marketing considerations in trial design and reporting.

Methods
Phase 1: estimating the prevalence of suspected marketing-driven trials in leading general medical journals Scoping of marketing trial features In order to reach a broad consensus on the characteristics of a marketing trial, which have not previously been defined, the study team met to share 24 examples of published trials one or more of us suspected of being designed partly for purposes of marketing. Based on this discussion and features of confirmed marketing trials [1][2][3][4] six characteristics of marketing-influenced trials were proposed and agreed upon by raters: 1) a high level of involvement of the product manufacturer in study design, 2) data analysis, 3) and reporting of the study, 4) recruitment of small numbers of patients from numerous study sites for a common disease when they could have been recruited without difficulty from fewer sites, 5) misleading abstracts that do not report clinically relevant findings, and 6) conclusions that focus on secondary endpoints and surrogate markers.

Sample
We generated a list of all trials evaluating one or more drug treatments published in 2011 in the top general medical journals, based on Impact Factor (New England Journal of Medicine (NEJM), The Journal of the American Medical Association (JAMA), Lancet, Annals of Internal Medicine, PLOS Medicine, and The BMJ). We included human drug and vaccine RCTs (excluding single patient and single arm trials). Papers describing subgroup analysis or just trial follow up data were excluded. Trials of devices were also excluded unless the device contained a drug e.g. drug eluting stents. We also excluded all studies that explicitly stated they were phase I or dosing escalation studies, but did not exclude phase II trials (or phase I studies that did not explicitly state their phase).

Independent rating of trials
Six members of the study team (VB, FG, RL, DB, CH, JR) independently rated included trials based on whether they were suspected to be marketing trials (YES, NO, or MAYBE). They did not use fixed criteria, but based their decisions on the extent to which they each felt the six characteristics of marketing-influenced trials described above influenced the research reported. Raters were not blinded to the journal, as successful blinding was not achievable with a team so familiar with the differences in journal formatting styles. One author [SS] then collated the independent ratings.. Trials with ≥4 independent YES ratings were categorised as suspected marketing trials (referred to as YES trials), whilst trials with ≥ 3 independent NO ratings were classified as non-marketing trials (NO trials). Trials with <3 NO and <4 Yes ratings were considered to be possible marketing trials and a consensus meeting was convened to discuss these in more depth. The initial independent ratings were collated and presented to the consensus group and then a consensus decision to assign a YES/MAYBE/NO for each of these trials was reached through discussion. VB and FG were excluded from rating trials that were published in their own journals (PLOS Medicine and The BMJ). For these trials, five ratings were collated, not six, and trials with ≥ 4 independent NO ratings were classified as non-marketing trials (NO trials).

Phase 2: characterisation of suspected marketing trials
In the second phase we sought to extract key variables, including basic information about the trials and independent assessments of their clinical relevance and quality to see if suspected marketing trials share characteristics that differentiate them from the other trials.

Data extraction
The study team developed a data extraction form (see Additional file 1) to capture variables that might categorise a trial as suspected marketing or not. Three members of the team (JR, VB, DB) and one independent data extractor piloted the form for ease of completion. A further eight independent data extractors were recruited to undertake further data extraction. They were blinded to our study objectives and the categorisation of the trials and were given instructions for gathering specific variables to encourage consistency. To assess clinical relevance and trial quality of each included study we extracted data from the McMaster PLUS (Premium Litera-tUre Service) database of scientifically robust studies [5].

Correspondence with editors
To gather further information to characterise these studies, and the degree to which marketing interests might have influenced them, we wrote to the editors of the participating journals where we suspected trials were marketing (see Additional file 2). We asked editors if they considered that they might partly be marketing trials when they were being reviewed, and how their editorial teams approach the possibility of marketing as a driver in trials. To aid our understanding and inform our analysis we also asked for editors to share in confidence copies of the anonymised peer review comments relating to these articles.

Survey of authors
We emailed the corresponding author of all 194 trials to invite them to complete a survey (see Additional file 3) containing further questions about the design and conduct of the trials. We were transparent in explaining the purpose of our study and included a link to our published abstract presented at the Seventh International Congress on Peer Review and Biomedical Publication [6]. Non-responders were sent one email reminder.

Data analysis
Initial data was collated in an Excel spreadsheet by SS and the analysis was performed using SPSS (version 18.0). We summarised data using proportions and used Chi-square and Kruskal Wallis tests to make comparisons across the three chosen categories (YES/MAYBE/ NO). As the MAYBE trials were also suspected marketing trials we ran a sensitivity analysis to see if it made a difference to the results if the MAYBE trials were combined with the YES trials versus the NO trials. For this sensitivity analysis (two categories only = YES/NO) we used Chi-square and Mann Whitney U Tests.
To determine if the key variables extracted could correctly identify the three study categories (YES/MAYBE/ NO) RP carried out a cluster analysis in R (version 3.1.0). Briefly, these methods use all categorical and numerical variables to determine a measure of 'similarity' between studies. Studies that have exactly the same values for all variables will have highest similarity (typically = 1) and those with completely different values the lowest (typically = 0). These similarities are then used to determine how close studies and clusters of studies are to each other hence defining membership to a selected number of clusters. The 'internal structure' of these clusterstight groups within clusters and large differences across clusterscan be used to determine if the data show important differences between these studiesin our case MARKETING vs. NOT-MARKETING. We used a "gower" metric to account for a mixture of continuous and categorical data and "partitioning around medoids" (PAM -similar to k-means) to create the clusters [7]. The number of clusters was set at 2, 3, 4 and 5 and silhouette graphs were used to test for internal structure with values of the average silhouette width above 0.5 required to determine that at least a reasonable structure has been identified [8]. We also used dendrograms to explore how these studies group and in particular looking for a clear separation between marketing and not-marketing trials in relation to combinations of the variables. All cluster analyses were carried out blinded to the categorisation (YES/ MAYBE/NO), which were only used to define consistency with membership allocation.

Ethics
We did not seek ethical approval for our study. The main study did not involve humans and was mainly documentary analysis so did not require ethical approval. The only involvement of humans was the author survey. Authors did not give consent to take part; participation in the survey was considered an indication of consent. Authors were free to ignore our request to provide additional information about their studies. Figure 1 shows that we included 194 (referenced in Additional file 4) of the 263 potential trials identified from the search strategy. Of these, 150 (77 %) were from two journals: NEJM (92) Lancet (58).   Table 2). YES/ MAYBE trials were more likely to have authors or contributors from the manufacturer involved in the study design, data analysis and reporting of trials than NO trials (p < 0.001 for all three comparisons). In addition, for YES/MAYBE trials, the manufacturer was significantly (p < 0.001 for all three comparisons) more likely to have control of the design of the study, data analysis and reporting of the study.

Results
YES/MAYBE trials were more likely to have at least one author from the product manufacturer on the authorship byline (p < 0.001) and at least one author with a declared financial conflict of interest with the product manufacturer compared with the NO trials, p < 0.001 (Table 2). YES/MAYBE trials were also more likely to report editorial assistance with preparing the manuscript (e.g. the use of professional writers or an indication of help with manuscript preparation in the Acknowledgements section) than NO trials, p = 0.001. There was no significant difference between groups in the use of group authorship on the byline (p = 0.70).
There was no significant difference between groups in the median number of participants screened (p = 0.49), but the median number of centres recruiting participants was 171 for YES trials and 74 for MAYBE trials compared with 13 for NO trials ( Fig. 3 and Additional file 5), p < 0.001. The median number of patients screened per study site was 11 in YES trials, and 18 in MAYBE trials compared with 112 in NO trials, (p < 0.001). The median number randomised per centre was 9, 16 and 37 (p < 0.001), respectively. However, the median number of months to recruit patients was significantly less for YES trials (19 months) compared to MAYBE (24 months) and NO trials (32 months), p = 0.003. Of note, of the 194 trials several  were missing basic information about the numbers of patients screened (29 %), countries (12 %), and study sites (9 %). A higher proportion of YES trials (32 %) reported the use of statistical imputation than NO trials (9 %) and MAYBE trials (7 %), p = 0.001 (see Additional file 4). YES trials were no more likely to use a surrogate primary outcome measure (p = 0.38) or a composite primary outcome measure (p = 0.14). However the Abstract's conclusions of YES trials (39 %) were more likely to focus on surrogate outcomes (7 % MAYBE, 25 % NO; p = 0.05). YES trials included speculation or generalised phrasing 59 % of the time that might encourage clinicians to use the intervention outside of the study population compared to 29 % for MAYBE and 37 % for NO trials (p = 0.03). However, the last two results were affected if the MAYBE trials were reclassified as YES (see Additional file 4). Grouping the MAYBE trials with the YES trials and performing a two group comparison did not alter substantially the pattern of results for other outcomes.
A higher proportion of YES trials blinded clinicians (83 % vs. 43 % MAYBE and 53 % NO, p = 0.03), reported safety outcomes and adverse events in the Abstract (83 % vs. 54 %, p < 0.001) and the main text of the paper (100 % vs. 84 %, p = 0.006), than NO trials. There was no difference in terms of type of trial picked up in Evidence Updates (p = 0.36), nor in median highest clinical relevance (p = 0.34) and newsworthiness (p = 0.42) ratings.
In the cluster analysis we found no groupings that could identify trials as marketing versus non-marketing based on all the extracted variables (Figs. 4 and 5, Table 3). Table 3 presents how the different trials in the study were allocated assuming different number of groups (to account for possible subgroups within the YES/NO studies). Although when using only 2 or 3 groups most (36/41) of the YES trials were allocated to Group 2, this group also included a high number of NOs and MAYBEs. This is roughly equivalent to having high sensitivity (88 % of YES trials allocated there) with poor specificity. This performance did not improve by increasing the number of groups. The best average silhouette width (measure of adequate clustering) was obtained when selecting only two clusters, generating a similarity score of 0.28 suggesting that no substantial structure was identified.

Correspondence with editors and authors
We received a response from editors at all four journals where YES trials were published. Editors did not agree that marketing considerations had played a key part in any of their published studies and noted to us the underlying importance and novelty of the clinical research described by the manuscripts. For reasons of confidentiality, none of  the editors were willing to share peer reviewers' reports or any editorial notes on the decision making for these manuscripts.
From the 194 survey invitations to corresponding authors we received a delivery failure notice for 11 (6 %). Of the 183 authors 55 (30 %) completed at least some of the questionnaire. The response rate was higher for authors of MAYBE (38 %) and NO (33 %) trials compared with YES trials (18 %). Due to the low response rate we are unable to draw firm conclusions from the data.

Discussion
Six clinical research experts and medical journal editors proposed that 21 % of drug trials published in 2011 in the leading general medical journals had characteristics consistent with the aim of marketing the product. Use of expert opinion is necessarily subjective and also open to bias. Therefore our methods aimed to reduce this bias, or at least to explore and make it explicit, and to better classify what seemed to us to be the characteristics of trials designed with marketing in mind. To achieve this we used independent data extractors to try and identify a consistent set of characteristics of marketing trials. The proposed set of marketing trials were more likely to be explicitly funded by the manufacturer, have authors or contributors from the manufacturer involved in or in control of the study design, data analysis and reporting of trials. In addition, suspected marketing trials contained speculation or generalised phrasing that might encourage clinicians to use the intervention outside of the study population.
There was an unavoidably circular aspect to our efforts, since we started with the feeling that marketing was a strong element of a number of trials and attempted to define the characteristics that contributed to that impression. Moreover, we could not define any groupings, based on the extracted variables, which could delineate trials as marketing. Furthermore, the reported methods of most of the proposed marketing trials were high quality. Our preconceptions, therefore, may have been biased by prior knowledge of marketing or seeding studies. In effect, the group of experts through consensus were able to identify a group of suspected marketing trials, but this grouping could not be validated, and remains open to experimenter's bias: a subjective bias whereby the result is overtly influenced by the experimenters, in this case the six raters. We call upon other independent researchers to further investigate this important topic.
One characteristic, though, is worth mentioning. The proposed group of marketing trials did report recruiting large numbers of patients from multiple countries and study sites yielding a very small average number of patients per centre. Recruitment from multiple sites is more expensive and harder to manage. Legitimate reasons for recruiting from a large number of centres might include the need to get sufficient numbers of patients in a shorter period, targeted enrolment for regulatory approval purposes, and the need to ensure a diverse geographical sample. Nonetheless, compared with interventional trials conducted without industry support, it is notable that trials identified by the study team as marketing with a YES score reported on common conditions where, in the opinion of the raters, patients could easily have been recruited from a smaller number of centres in a similar time period, with less risk to data quality and at reduced cost. Involvement in the design of the study could include recruitment strategies. The justification for specific recruitment strategies are not typically reported in journal articles, protocols or trial registries so this information can only be gathered from the manufacturers or the authors of the study. We tried but failed to obtain this information from individual trial authors.