Patient Reported Outcomes: misinference from ordinal scales?
© Horton and Tennant; licensee BioMed Central Ltd. 2011
Published: 13 December 2011
Patient Reported Outcomes (PROs) are widely used in medical outcome studies, and usually take the form of administered of self-completed questionnaires. The data that these questionnaires produce is of the type known as ordinal scaling, where magnitudes of the attribute may be ascertained. At the same time, most outcome studies rely on the calculation of means, standard deviations, change scores, and concepts such as Minimally Important Difference (MID) or effect sizes. Yet, ordinal scales do not support the mathematical operations needed to calculate these type of statistic . Indeed when several items are measured on an ordinal scale it is far from certain that the sum of scores has even ordinal properties . Despite these constraints, these limitations are largely ignored, and thus statistics such as means and MID are widely reported for PROs. This runs the risk of drawing an incorrect inference from data based upon PROs .
This risk can be illustrated by considering the concepts of the ‘plateau’ and the calculation of the MID. Both are investigated by contrast of the ordinal raw score against the cardinal metric derived from fit of data to the Rasch measurement model . It can be shown that as the raw score from a scale moves towards the margins, then a smaller and smaller raw score change is associated with a standard metric unit of change. Thus patients may seem to be ‘slowing down’ in their improvement, or even ‘plateauing’, yet they are still moving the same metric distance. Likewise, when considering a magnitude of improvement such as an MID, the raw score distance associated with the MID can be shown to vary across the scale, depending upon the starting point. Thus for one patient the same MID may involve a change in the metric distance four times greater than that of another patient.
PROs provide ordinal estimates of the magnitude of a patient on the trait being measured. Appropriate non-parametric statistics should be used. Else, where possible, the data should be converted to the cardinal metric through use of the Rasch model, which is consistent with the requirements of the theory of Additive Conjoint Measurement .
- Forrest M, Andersen B: Ordinal scale and statistics in medical research. BMJ. 1986, 292: 537-538. 10.1136/bmj.292.6519.537.PubMed CentralView ArticlePubMedGoogle Scholar
- Luce RD, Tukey JW: Simultaneous conjoint measurement: a new type of fundamental measurement. Journal of Mathematical Psychology. 1964, 1: 1-27. 10.1016/0022-2496(64)90015-X.View ArticleGoogle Scholar
- Merbitz C, Morris J, Grip JC: Misinference from ordinal scales. Arch Phys Med Rehabil. 1989, 70: 308-312.PubMedGoogle Scholar
- Rasch G: Probabilistic models for some intelligence and attainment tests. 1960, Chicago: University of Chicago PressGoogle Scholar
- Stucki G, Daltroy L, Katz JN, Johannesson M, Liang MH: Interpretation of change scores in ordinal clinical scales and health status measures: the whole may not equal the sum of the parts. J Clin Epidemiol. 1996, 49: 711-717. 10.1016/0895-4356(96)00016-9.View ArticlePubMedGoogle Scholar
- Van Newby A, Conner GR, Bunderson CV: The Rasch model and additive conjoint measurement. Journal of Applied Measurement. 2009, 10: 348-354.PubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.