Design and statistical power
As depicted in Fig. 1a, the npRCT combines a traditional RCT, comparing intervention A versus B, with a precision RCT, comparing stratification to intervention A or B versus randomisation to A or B.
This RCT combination has a twofold purpose. First and as noted above, combining these two RCT types can provide an integrated platform that allows for sequential development and testing of a precision algorithm. Specifically, researchers can analyse collected data during the traditional RCT to see if any strong prescriptive variables are present that could guide treatment choice. For example, identification of pre-treatment prescriptive variables can be conducted using the PAI approach, which looks for treatment-moderating variables using different regression-based and machine learning techniques [6]. Ideally, any of such online analyses of traditional RCT data should be conducted by an independent trial collaborator/statistician that is not involved in day-to-day recruitment or treatment procedures, so as to prevent potential researcher biases. If prescriptive variables are identified that could optimise intervention allocation decisions, researchers can move on to the precision RCT stage, in which the algorithm is prospectively evaluated against random intervention allocation.
The second purpose of the npRCT is to offer a design that can save trial costs as compared to running traditional and precision RCTs independently. As both the traditional and the precision RCT include participants randomised to interventions of interest, the traditional RCT can be understood as nested within the combined npRCT design. As such, all individuals randomised to interventions of interest can be jointly analysed to determine the comparative intervention effectiveness of interventions A and B. This nesting increases statistical power by reducing the sample size of the traditional RCT by 1/2 times the sample size of the precision RCT.
Practical considerations
We want to highlight three major practical considerations that should be made when setting up an npRCT.
First, researchers should consider effect size and allocation ratio criteria a priori that determine if it is sensible to move on to the precision RCT stage or to maintain the traditional RCT. In terms of effect sizes, researchers need to specify minimum effect size benefits of the precision algorithm. If an effective algorithm can be identified based on this criterion, researchers can move on to the precision RCT stage. If an effective algorithm cannot be identified based on this criterion, researchers could maintain the traditional RCT design to focus on the comparative intervention effectiveness evaluation. Similarly, researchers should carefully consider how overall intervention main effects are weighed against treatment-moderating effects: Take a hypothetical example where intervention A is more effective than intervention B overall (i.e. a significant intervention main effect is present). Under this scenario, the algorithm may allocate a majority of participants to intervention A and only few participants to intervention B. If effectiveness differences are too large, a precision algorithm will lose its economic value as a precision medicine tool as it would make more sense to allocate all participants to intervention A irrespective of any prescriptive variables.
Second, researchers need to carefully consider potential blinding issues and biases arising from the npRCT, which are particularly likely if interventions cannot be concealed (e.g. through use of matched drug capsules). For example, researchers might learn how the precision algorithm allocates participants to interventions, which could lead to potential experimenter biases. Experimenter and expectation biases may also change from traditional to precision RCT stages. For example, participants might have a clear expectation that intervention A works best for them, so may assume (if allocated to intervention B) in the precision RCT stage that they were not assigned by the precision algorithm. To avoid expectation biases at the stratification stage, the aim should be that all individuals involved in the trial (i.e. participants, clinicians, and the research team) are blinded, so that knowledge about allocation group will not affect trial results.
Third, it will be important to set up a timeline that leaves sufficient time for online precision algorithm development. It is sensible to start precision algorithm development relatively late in the traditional RCT stage, so as to maximise sample sizes used for algorithm identification. However, the benefit of maximising sample sizes for algorithm identification should be weighed against practical time requirements to conduct analyses, especially if recruitment rate is high, to avoid potential delays between traditional and precision RCTs at the point of stratification.
Power calculation app
We have developed a free, open-source power calculation app for the npRCT that can be accessed via https://nprct.shinyapps.io/nprct/. This app includes power calculation strategies for (i) comparison of two groups on a continuous outcome (based on two-sample independent t tests), (ii) comparison of three or more groups on a continuous outcome (based on analysis of variance or ANOVA), and (iii) comparison of two or more groups on a binary or categorical outcome (based on χ2 tests).
This power calculation app shows savings with this design (in terms of participants per group) compared to running the traditional and the precision RCT independently (see Fig. 1b). The power calculation app also provides the transition point from traditional to precision RCT (cf. point of stratification); that is, the point of recruitment at which one needs to change towards the precision RCT design. Considering a continuous outcome and comparison of two interventions, for frequently reported small, medium, and large effect sizes, the npRCT would require 196 (Cohen’s d = 0.2), 32 (d = 0.5), and 13 (d = 0.8) participants fewer per group compared to two independent RCTs (assuming α = 0.05 and power = 0.8). These statistics emphasise recruitment savings gained by combining/nesting RCTs under the npRCT design.
In the power calculation app, we provide explanations on further practical and ethical considerations. We also provide a walk-through research example for the npRCT. Source code of the npRCT app and power computation functions are freely available via github under https://github.molgen.mpg.de/mpip/npRCT.app, which we encourage researchers to use and adapt for more complex npRCT design setups (e.g. with traditional RCT allocation ratios deviating from 1:1 allocation).