Skip to main content
  • Oral presentation
  • Open access
  • Published:

A framework to assess the impact of applying formal criteria to check clinical relevance on top of statistical significance

Recently, the topic of addressing clinical relevance on the top of statistical significance in the analysis of RCTs has been paid increasing attention. Several formal criteria to serve this purpose have been published.

In this talk we present a framework which allows to investigate the impact of applying such formal criteria, independent of the scale of the outcome variable. The only assumption made is that we have one clinical trial correctly powered to achieve a 90% power. All treatment effects are then expressed as fractions of the assumed effect.

In a first step we quantify the actual risk of having accepted a treatment with an irrelevant effect in a successful RCT, and we find this risk to be often rather small. In a second step we investigate two popular criteria based on comparing the effect estimate or the lower bound of the confidence interval with a given threshold. We demonstrate their equivalence in large samples and quantify the impact of applying them in terms of decreasing the risk of accepting a treatment with an irrelevant effect and increasing the risk of failing to accept a treatment with a relevant effect.

Putting weights on these two types of errors, we can demonstrate that the formal criteria do often more harm than good. We discuss implications of these results on adapting sample sizes to demonstrate clinical relevance and on the application of the formal criteria if several studies are summarized in a meta-analysis.

Author information

Authors and Affiliations


Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Vach, W., Gladstone, B.P. A framework to assess the impact of applying formal criteria to check clinical relevance on top of statistical significance. Trials 14 (Suppl 1), O101 (2013).

Download citation

  • Published:

  • DOI: