Red flags for randomisation

Introduction

Randomised controlled trials (RCTs) provide the best evidence for causality that can be gained from a single study. In most evidence hierarchies, only a systematic review and meta-analysis of RCTs is superior.

If we have observed that patients who received an intervention did better than patients who received a placebo or standard care, we want to be able to conclude that it was caused by the differential treatments they received. Ideally, there should be a low risk that there is some other explanation for what we’ve observed, such as bias or random chance, and this can be promoted through elements of study design such as proper randomisation, allocation concealment, triple blinding, and recruiting enough patients.

Randomisation is a key design feature that is used to minimise allocation bias and to control for known and unknown confounders. The purpose of blinding is to reduce psychological biases such as the placebo effect (patients), confirmation bias (investigators), and observer bias (assessors) (Sibbald and Roland 1998).

It is always possible to properly randomise and conceal group allocations

Sometimes blinding is unethical or not possible due to the nature of the intervention. However, it is always possible to produce a properly randomised allocation sequence and to ensure allocation concealment until the moment randomisation occurs. Unfortunately, errors in the conduct and reporting of randomisation are common (Schulz 1994, Vorland et al. 2021), most notably when a study is labelled as ‘randomised’ but the report describes an improper method or contains insufficient details. Failing to properly randomise and conceal allocations results in an automatic assessment of high overall risk of bias (Sterne et al. 2019).

The following red flags are commonly seen in the literature, indicating a widespread misunderstanding and misuse of randomisation.

Testing baseline group differences

Randomising the group allocations usually results in two groups of patients who are ‘similar’ overall at baseline. This does not mean that the groups will not differ on any one or more of the many demographic and clinical characteristics that could be compared. This is because group differences can arise due to random chance. For example, for every 20 characteristics tested at baseline we would expect one group difference to be statistically significant at the 5% level just by chance alone (even if the size of that difference was trivial). Therefore, performing and reporting tests for baseline group differences is not only unnecessary, but also potentially misleading (Moher et al. 2010). Simply put, there should never be p-values in table 1 for an RCT.

Too little or too much balance at baseline

Group imbalance on an important prognostic factor can affect the treatment effect estimate. If there are any prognostic factors for which balance is important (such as age, sex, or disease severity), stratification can be used. Essentially this means generating a separate allocation sequence for each stratum. This is especially useful if subgroup analyses are planned to estimate stratum-specific treatment effects. Otherwise, if imbalance occurs it should be commented upon, however, adjustments to the analysis that are not prespecified in the statistical analysis plan are cautioned against as they may introduce bias despite seeking to reduce it.

On the other hand, groups that are extremely similar on many baseline factors may indicate intentional data fabrication, since people wrongly assume that randomisation will produce more similarity than it actually does. Excessive similarity is over-represented in retracted trials, many of which have questionable data integrity (Carlisle 2017).

Equal sized groups

Typically, in a parallel 2-arm RCT, patients are individually randomised in a 1:1 ratio, which means each patient has a 50% chance of being allocated the intervention and a 50% chance of being allocated the control (hypothetically, via the flip of a fair coin). The statistical term for this method is simple randomisation, and the term ‘simple’ is not meant to be pejorative or suggest that a more complex method should be used.

A common misunderstanding of simple randomisation in a 1:1 ratio is that it will result in groups of equal size (as in a matched case-control study recruited in a 1:1 ratio). However, simple randomisation rarely produces equal sized groups, especially for small sample sizes. In fact, equal sized groups suggests that the randomisation process may have been improperly performed. Examples of incorrect methods are reported by Vorland et al. (Vorland et al. 2021), many of which would result in equal sized groups. Another incorrect method is described below.

Randomly ordering a list of allocations

Consider the following method of producing an allocation sequence for a trial seeking to recruit n = 100 patients. An investigator lists 50 rows of ‘A’ and 50 rows of ‘B’ in a column in spreadsheet and then randomly orders the list to produce the allocation sequence. Is this method proper? No. To see why, imagine that we have a bag of 50 ‘A’ tokens and 50 ‘B’ tokens, and each patient will be handed one randomly. Suppose that the first patient has been allocated to group A and now there are 49 ‘A’ tokens left and 50 ‘B’ tokens left. Then the second patient has a probability of 49/99 = 49.5% chance of being allocated to group A and a 50/99 = 50.5% chance of being allocated to group B (i.e. unequal chances). As allocation proceeds and the pool of tokens reduces, the allocation probabilities change (depending on how many tokens of each type have previously been given out). In fact, the last patient has 100% chance of being allocated to one group and 0% for the other since there will be only one token left. In other words, this method is analogous to selection without replacement in the language of probability theory.

Proper simple randomisation using tokens would proceed with replacement. In other words, we would begin with 50 ‘A’ tokens and 50 ‘B’ tokens and after each patient has drawn a token it is put back in the bag. In this way, each patient has 50 ‘A’ and 50 ‘B’ tokens to draw from and the allocation chances are always 50%/50%. (Equivalently, you could use just one token of each type, in which case it’s easy to see that this situation is the same as flipping a coin (Altman and Bland 1999).

Poor design of randomisation restrictions such as blocking

Designers of small-to-moderate sized trials need to be aware of the downside of simple randomisation since group size imbalance can reduce statistical power. One solution is a method called block randomisation (Broglio 2019). In this method, the total recruitment size is divided into smaller ‘blocks’, in each of which there are equal numbers of A’s and B’s listed in random order. The blocks themselves are randomly ordered and, ideally, varying block sizes are used. Further, this method provides an upper bound for the maximum possible imbalance, which is equal to half the (largest) block size.

The downsides of block randomisation are that it is restricted, i.e., not purely random, and may risk predictability of the allocation sequence if the block sizes are fixed and/or small especially if blinding is lax. The first method described in point 4 above could trivially be considered a form of block randomisation with a single block of size equal to the sample size – but this is far from ideal. However, the benefits of block randomisation outweigh the risks if it is designed carefully. For example, a trial seeking to recruit n = 100 patients could use block randomisation with randomly varying block sizes 4 and 6 to produce its allocation sequence. This would simultaneously keep the maximum imbalance small while still making it hard to predict the next allocation. As with simple randomisation, it should be noted that block randomisation doesn’t always produce equally sized groups.

Not reporting adequate details of the randomisation method

Reporting of randomisation methods is often poor in RCT reports (Schulz 1994). Merely stating that the allocation sequence was ‘computer-generated’ is not sufficient. Details of any use of block randomisation or stratification must be given, and if these are not applicable the term ‘simple randomisation’ should be stated if it was properly implemented. If a particular website was used it should be named and if an external randomisation service was used this should be stated. Details of allocation concealment should also be described not merely stated; for example, the use of ‘sealed envelopes’ is insufficient unless it is specified how they are tamper-proof and opaque enough to prevent discovery.

Conclusion

It is unethical to expose patients to the risks of trial participation if the study will not be able to produce a reliable contribution to science. The onus is on trial authors to convince the reader that their methods were sound, and this burden remains indefinitely, not just until the trial passes peer review. Since there is no method of analysis that can remedy insufficient randomisation or lack of allocation concealment, it is imperative for those involved in trial design and conduct to know and use best practice methods (International Conference on Harmonisation 1998) as well as how to properly report their use (Moher et al. 2010).

Disclosure statement

No conflict of interest was reported by the author.

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.