Use of randomisation in clinical trials: a survey of UK practice

Survey population

The survey was sent out at two time points: the primary survey was sent out in 2003, and a follow-up survey was sent out in 2011 to assess for any changes in opinions and practice over time. The same survey instrument was sent out at both time points (see Additional file1).

In 2003 the Directory of Academic Statisticians[7], which was the most comprehensive directory of statisticians and methodologists working in academic research in the UK at that time, was selected as the sampling frame for statisticians and researchers working in the field of clinical trials.

Within the Directory, people who identified themselves with the following categories of research interest were selected: health, healthcare, health service research, health statistics, health technique assessment, medical statistics, clinical trials and randomisation. The initial search yielded 527 people. The full text entry for each person was examined and this led to the removal of 158 people whose areas of expertise were subsequently determined to be not relevant – for example, decision support systems in the National Health Service; analysis of finance and health data; social statistician (crime, education, health, poverty, race) – resulting in a target survey population of 369. A further two statisticians who were known to have a specific interest in randomisation methods but who were not listed in the Directory were also included as part of the survey population. This gave a survey population of 371 that included known experts in the field and representatives from all major UK trials centres.

The questionnaire was sent out together with an accompanying letter explaining the purpose of the survey. The questionnaire was printed on yellow paper as there was some evidence that paper colour could enhance response rates[8]. A reference number was included on the questionnaire in order to track responses, but this was removed by two people in order to preserve anonymity. Recipients were requested to pass the questionnaire to another researcher if it was not relevant to themselves. No reminders were sent out as the questionnaire was intended to be anonymous.

In January 2011 the survey was sent out a second time to see whether practice had changed in the intervening period. By 2011 the structure of trials support had changed radically in the UK with the establishment of formal clinical trials units, which are required to be registered with the UK Clinical Research Collaboration. Each trials unit has a named information specialist lead (who will generally be the first point of contact for randomisation services). Reflecting this change, the survey was this time sent (by email) to the information specialist for each registered trials unit. Questionnaires were sent to 48 registered trials unit statisticians (70 members) and information specialists (62 members) on the UK Clinical Research Collaboration mailing lists. As before, no reminders were sent to nonresponders.

Data analysis

Data were analysed descriptively using numbers and percentages for binary variables. Responses to open-text boxes were categorised into common themes by the lead author (GCM). No ethical approval was required or sought.

Opinions about and use of different statistical randomisation methods

Of those respondents who indicated that assignment of patients to treatment groups should take account of prognostic factors to show balance at baseline (n=67), the most common reasons given were that it avoids imbalance (n = 18) and that it helps face-validity (n = 13). Other reasons given were that it achieves a gain in efficiency (n = 5), achieves more precision in treatment effect estimate (n = 4) or that it allows for simple analysis later (n = 3). Those who disagreed that assignment should take account of prognostic factors (n = 18) maintained that adjusted analysis sorts out most of the imbalance, it is not necessary for large trials and that it is best left to chance.

Opinions about and use of minimisation

Fifty-seven per cent of respondents had used the method of minimisation. Respondents indicated that the method seemed to work and most would use it again. The following methods were reported as being used to reduce the predictability of the next assignment: do not declare factors in the trial protocol; add in other irrelevant factors to provide noise; allocate the first block of about 20 patients (either some random number or 10% have been suggested) by permuted blocks or simple randomisation, and then use minimisation with a randomisation ratio of 0.75:0.25 (or 80/20); use blocking, and then restricted randomisation within each block; use centre as a minimisation factor in multicentre trials; use a probability between 0.5 and 1 for allocation; or alternate between simple randomisation/minimisation for small open trials.

When asked what was an appropriate level of randomness to use in minimisation, most respondents suggested a probability of from 0.66 to 0.95, with most saying that this level should be fixed throughout the life of the trial. Twenty-two respondents reported using values between 0.75 and 0.95, and seven had used values lower than 0.75. Seventeen respondents always used the method of Taves[10] with probability of assignment P= 1. Only one person always retained a random element in minimisation. One person had conducted a trial where the level of randomness was not fixed, but four others thought that it might be a good idea to vary randomness.

When asked about the number of factors that would sensibly be included as minimisation factors, most respondents felt that this was dependent on the size of the trial. However, suggested rules of thumb included that only known (evidence-based) prognostic variables should be used and more than 10 is generally seen to be too many, with more than five or six variables making it tricky to test the programming. Nine respondents would not set a limit on the number of factors, and 13 would set a limit of four variables or less.

Sixty-six per cent of those who had used minimisation adjusted for the minimisation factors using covariate adjustment as appropriate (40% always, 25% sometimes) through analysis of covariance (ANCOVA), multiple regression, logistic regression or Cox's regression. There was a suggestion to perform both unadjusted and unadjusted analyses.

The most common problem encountered when using minimisation was programming difficulty or errors, and a few respondents stressed the importance of finding appropriate purpose-written software. It was observed that overall treatment imbalance can result if stratifying by centre or clinician, and two respondents mentioned checking for balance.

Use of other forms of constrained randomisation

Respondents were asked about their use of other (less common) randomisation methods, including those of Atkinson[11], Titterington[12], Signorini and colleagues[13] and Klotz[14]. Forty-nine (42%) respondents had heard of Atkinson’s method but only one person had ever used it, saying the method did not work and they would not use it again. Although 25 (21%) respondents had heard of Titterington’s method and 13 (11%) had heard of Klotz’s method, no-one had ever used them. Of the 22 (19%) people who had heard of Signorini’s method, four had used the method and it had worked but two would not use the method again because they perceived it to be not as good as minimisation and not as easy to use.

Opinions on choosing a randomisation method

The most important factor when choosing a randomisation method was seen as the size of the trial. The general advice was to keep it simple and the most common problem was identification of prognostic factors and limiting their number. Other important factors included the practicality of the method, maintaining blindness, importance of prognostic factors, number of centres, size of trial, length of trial and reducing predictability.

Problems encountered when setting up randomisation schemes

Web-based randomisation

In 2003, 80% of respondents had never used Web-based randomisation systems although they would possibly use them in the future. In 2011 only one respondent had never used them and 11 (73%) had used them several times. The biggest problems were thought to be lack of expertise and security. Other problems to be overcome include access to the Web, speed and reliability of access, lack of willingness in centres, telephone methods being preferred and trust in those performing the randomisation.

Future directions

Most respondents thought that minimisation is likely to be the most popular randomisation method in the future, with the use of a random element to avoid the possibility of prediction of treatment allocation, followed by permuted blocks. However, this was not a consensus view as the stated views of experts within the survey population ranged from ‘do not be misled by the hype surrounding minimisation – it is most emphatically NOT the platinum standard’ to ‘I would favour minimisation because of its advantages not just for small trials but also for large trials in which analysis within subgroups would be based on better balance’.

Simple randomisation

The results of this survey have shown that trialists are currently using simple randomisation less often than permuted blocks, stratification and minimisation. The size of trial considered too small for simple randomisation ranged from <50 to <1,000, although some trialists surveyed believed that simple randomisation was suitable for any size of trial.

Within the wider literature, there is evidence that simple randomisation is suitable for larger trials[3, 15–17] but there is a high probability of possible imbalance between treatment groups in trials of up to 500. Simulations showed that imbalance can occur often in small trials with <100 participants (for example, >80% of occasions), but imbalance rarely occurs in trials with ≥1,000 participants[15]. Pocock and Simon also suggested that simple randomisation was only to be recommended for larger trials (say >200 participants), and cautioned that even in large trials problems may occur if one intends to analyse early results (say for data monitoring)[16].

Stratification

Forty-three per cent of trialists wrote that the number of strata to be used should depend on the size of trial. There was no general consensus on the number of strata that were considered too many, with suggested values between 1 and 100.

Pocock and Simon stated that it is seldom advisable to stratify on more than three or four variables and the size of the trial is the most important factor in deciding how many stratification variables are feasible[16]. Peto and colleagues dismiss stratification as a complication rendered unnecessary by the development of methods of analysis that adjust for covariates[18].

Pocock and Simon used simulations to show that increasing imbalance between treatment groups can arise due to increasing number of strata with incomplete randomised blocks[16]. Signorini and colleagues asserted that using a randomised block design presented a possibility of the overall trial having large differences in number of patients in each treatment, even though each stratum has more or less equal numbers on each treatment[13]. This leads to a loss in efficiency in the analysis. The same problem can arise when employing a randomisation scheme using minimisation with stratification by centre or clinician, although International Conference on Harmonisation (ICH) guidelines still recommend this[19].

Balancing prognostic factors at baseline

More than one-half of the trialists surveyed thought that groups should be balanced at baseline. The primary reasons given were that it helps face-validity and the analysis is simpler. The suggested limit on the number of prognostic factors ranged from 2 to 10, and some trialists would not set any limit.

There is controversy, especially among statisticians, as to whether prognostic factors should be used in assignment to treatment[20]. There are two main viewpoints: prognostic factors should be used to assure that patients assigned to the two arms show close balance in prognoses at baseline; or random assignment should be used without regard to prognostic factors, as fair comparison of treatment effect can be achieved through statistical adjustment of results.

Armitage and Gehan concluded that in a small to moderate-sized trial (≤100 patients) results might be invalid if prognostic factors are not used[21]. Comparisons between treatments should be made between groups that are comparable with respect to prognostic factors. Similarly, Rovers and colleagues suggested that investigators should always consider balanced allocation for a low number of patients[22]. They stated that, in a trial with 100 to 200 patients, substantial differences can occur in baseline characteristics if simple randomisation is used; if these differences can be measured, then they can be corrected to some degree in the analysis. If the number of patients decreases or the number of prognostic variables or categories increases, then the resulting imbalance could invalidate results.

The Committee for Proprietary Medicinal Products state that stratification for more than a few prognostic factors is not always possible[23]. They describe techniques of dynamic allocation such as minimisation that are often used to balance across several factors simultaneously as highly controversial even if deterministic schemes are avoided. They therefore strongly advise against the use of such methods. When deterministic schemes are used, the Committee for Proprietary Medicinal Products also suggest that the factors used in the allocation scheme should also be included as covariates in the analysis (although whether the analysis adequately reflects the randomisation scheme also remains controversial). They also state that if a multicentre trial is not stratified by centre then the reasons for not doing this should be explained and justified in the protocol.

Predictability

Nineteen per cent of trialists who had used minimisation were very concerned that the method was largely deterministic and 57% were only mildly concerned. Twenty-eight per cent were not concerned at all in 2003, but in 2011 this number had dropped to zero. Among the trialists surveyed, strategies for reducing the predictability of next assignment included not declaring the factors in the protocol, adding in other factors to produce noise, not stratifying by centre and alternating between minimisation and simple randomisation. When using a random element, the level of randomness chosen ranged from 0.66 to 0.95.

When choosing a balanced allocation method the probability of performing the prescribed allocation should be large enough to control imbalances in strata but small enough to prevent selection bias through prediction of next treatment[24]. The use of a random element (probability of assignment P<1) is supported by ICH E9[19], which also states that factors on which randomisation has been stratified should be accounted for later in the analysis. Gore recommends a value for probability of assignment of 0.75[25]. In a multicentre trial, predictability is not a problem unless stratifying by centre as recommended by ICH E9 guidelines, when all centres are essentially independent or they are not aware of the bands used[20].

The Food and Drug Administration also consider minimisation not to be random and recommend the incorporation of a random element[26]. However, a minimisation procedure to allocate treatment that does not allow investigators to predict the next treatment allocation would yield a properly randomised trial[16].

Randomised block design can also be highly predictable[13] since each block must contain equal numbers of patients on each treatment. The block size should be hidden from investigators but can sometimes be deduced from previous assignments if the block size is fixed. Therefore it is better to use random block sizes.

Analysis

In 2003 almost one-quarter of those using minimisation never adjusted for prognostic factors in the analysis, believing that if the minimisation has worked then there should be no need to do so. In 2011 this number had dropped to zero. Forty per cent indicated that they always adjusted for the minimisation factors in the analysis.

Taves maintains that clinical trials managed by minimisation should use ANCOVA for statistical comparisons[10]. Lachin and colleagues indicate for minimisation that the statistical analysis must incorporate adjustments for the covariates employed in the design in order to yield tests of proper size[17].

Forsythe and Stitt studied analysis of variance and ANCOVA in a simulation study of a small clinical trial and found some evidence that ANCOVA with minimisation was more powerful than ANCOVA with simple randomisation[27]. They showed that using minimisation without adjusting for covariates in the subsequent analysis distorted the significance level and the power of the test. However, they reported that other studies implied that the effects of stratifying by a covariate and then ignoring it in the analysis may not be severe. It may be that introducing randomness into a systematic design reduces such accidental bias, but further research is needed.

Vaughan Reed and Wickham discussed the validity of using an unadjusted analysis[28]. They maintained that the errors introduced by using an unadjusted analysis for a deterministic allocation method were likely to be less than the errors resulting from imbalance between treatment groups that might occur using simple randomisation. They maintained that serious imbalance between treatment groups undoubtedly causes problems with analysis and interpretation.

Strengths and limitations of the study