Skip to main content

Table 1 Should we do a further evaluation of the intervention in a SWAT?

From: Trial Forge Guidance 2: how to decide if a further Study Within A Trial (SWAT) is needed

The five proposed criteria for deciding whether the intervention needs another evaluation in a SWAT. The more criteria that are met, the more likely we are to conclude that further evaluation in a SWAT is appropriate.

 1. GRADE: the GRADE [22] certainty in the evidence for all key outcomes is lower than ‘high’.a

 2. Cumulated evidence: the cumulative meta-analysis shows that the effect estimate for each outcome essential to make an informed decision has not converged.b,c

 3. Context: the range of host trial contexts evaluated to date does not translate easily to the context of the proposed SWAT.d For the proposed SWAT consider PICOT [23]:

  • P – is the population in the host trial so different from those already included that the current evidence does not provide sufficient certainty?

  • I – are the health interventions in the host trial so different from those already included that the current evidence does not provide sufficient certainty?

  • C – is the comparator in the host trial so different from those already included that the current evidence does not provide sufficient certainty?

  • O – is the SWAT outcome(s) so different to those used in the existing evaluations that that the current evidence does not provide sufficient certainty?

  • T – in the time since the existing evaluations were done, have regulatory, technological or societal changes made those evaluations less relevant?

 4. Balance – participants: the balance of benefit and disadvantage to participants in the host trial and/or the SWAT is not clear.e

 5. Balance – host trial: the balance of benefit and disadvantage to the new host trial is not clear.f

  1. Notes
  2. a A GRADE assessment of ‘high’ means that we are confident that the true effect lies close to the estimate of effect coming from the cumulative meta-analysis [24]. In Cochrane’s deliberations as to when to close a Cochrane Review (https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.ED000107/full), the collaboration chose not to require ‘high’ GRADE certainty in the evidence because it was felt that this may not always be achievable. Although we recognise the pragmatic nature of this, we recommend ‘high’ in our criteria because SWATs are usually simple studies for which it should be possible to generate high certainty evidence. We will, however, keep this criterion under review to consider whether it needs relaxing.
  3. b This is a judgement that depends on the behaviour of the effect estimates and on whether the confidence intervals include the threshold for an important benefit (or disadvantage). For example, if there is drift in the effect estimates of a meta-analyses but the confidence intervals around the estimates are consistently above what you think is an important benefit (or below a relevant disadvantage) then the cumulative meta-analysis can be judged to have converged despite movement in the effect estimates. For more on GRADE see http://www.gradeworkinggroup.org.
  4. c This is a judgement that depends on the behaviour of the effect estimates and on whether the confidence intervals include the threshold for an important benefit (or disadvantage). For example, if there is drift in the effect estimates of a meta-analyses but the confidence intervals around the estimates are consistently above what you think is an important benefit (or below a relevant disadvantage) then the cumulative meta-analysis can be judged to have converged despite movement in the effect estimates. For more on GRADE see http://www.gradeworkinggroup.org.
  5. d This is to provide reassurance about the applicability of the result to different types of trials. Care is needed to avoid a default position of insisting on an evaluation in every conceivable context. In other words, is there any reason to believe that the intervention would not work in your context given the contexts already studied? It is possible that evidence from SWATs will eventually splinter off to focus specifically on certain contexts but, for now, we suggest pooling evaluations of the same intervention because there are so few SWAT evaluations of any intervention and this pooling will provide a basic foundation on which to build.
  6. e Where there may be no conceivable benefit or disadvantage for participants, they should be considered as balanced.
  7. f A benefit might be that the host trial recruits faster, or its data quality is improved. Examples of disadvantages might be that there are added costs to the host trial, or that a new task is introduced into the workload of trial managers.