Skip to content

Advertisement

  • Oral presentation
  • Open Access

A framework to assess the impact of applying formal criteria to check clinical relevance on top of statistical significance

  • 1 and
  • 1
Trials201314 (Suppl 1) :O101

https://doi.org/10.1186/1745-6215-14-S1-O101

  • Published:

Keywords

  • Public Health
  • Confidence Interval
  • Clinical Trial
  • Treatment Effect
  • Outcome Variable

Recently, the topic of addressing clinical relevance on the top of statistical significance in the analysis of RCTs has been paid increasing attention. Several formal criteria to serve this purpose have been published.

In this talk we present a framework which allows to investigate the impact of applying such formal criteria, independent of the scale of the outcome variable. The only assumption made is that we have one clinical trial correctly powered to achieve a 90% power. All treatment effects are then expressed as fractions of the assumed effect.

In a first step we quantify the actual risk of having accepted a treatment with an irrelevant effect in a successful RCT, and we find this risk to be often rather small. In a second step we investigate two popular criteria based on comparing the effect estimate or the lower bound of the confidence interval with a given threshold. We demonstrate their equivalence in large samples and quantify the impact of applying them in terms of decreasing the risk of accepting a treatment with an irrelevant effect and increasing the risk of failing to accept a treatment with a relevant effect.

Putting weights on these two types of errors, we can demonstrate that the formal criteria do often more harm than good. We discuss implications of these results on adapting sample sizes to demonstrate clinical relevance and on the application of the formal criteria if several studies are summarized in a meta-analysis.

Authors’ Affiliations

(1)
Clinical Epidemiology, Freiburg, Germany

Copyright

© Vach and Gladstone; licensee BioMed Central Ltd. 2013

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate. Please note that comments may be removed without notice if they are flagged by another user or do not comply with our community guidelines.

Advertisement