Debate: A subversive view of subsets - a dissident clinician's opinion

Clinical trialists and statisticians are very wary of subgroup analysis, for good reasons. Clinicians have to deal with situations in which subgroups of patients differ widely from one another in their prognosis and response to treatment. Few trials are large enough to demonstrate convincingly these differences in outcome, but often provide suggestive evidence. Should we ignore this and treat all patients as the same, or should we allow dubious statistical evidence to buttress biological plausibility in making clinical decisions?


When meta and mega agree
When a mega-trial comes up with the same findings as a meta-analysis, or vice versa, the evidence appears overwhelming. This was the case with the use of intravenous followed by oral beta-blockers in acute myocardial infarction in the International Study of Infarct Survival (ISIS)-1 study [1] and the subsequent systematic review [1]. It is not surprising, therefore, that perhaps the most prestigious of cardiological organisations [2] came up with following the Class 1 recommendation for deciding who should receive this therapy: "Patients without a contraindication to beta-adrenoceptor blocker therapy who can be treated within 12 hours of onset of infarction, irrespective of administration of concomitant thrombolytic therapy or performance of primary angioplasty".
There is no suggestion in this guideline that some patients conforming with those criteria might benefit or be harmed more than others. This may well have been because the ISIS-1 investigators were pioneers in highlighting the potential dangers of subgroup analysis. This they did most vividly by pointing out how the results varied with a patient's astrological sign.

Ignorance, inefficiency or incredulity?
In many countries, notably the UK, the contemporary use of intravenous followed by oral beta-blockers is very low. It was reported that the British investigators used this therapy in only 2% of the patients in ISIS-4. Were they ignorant or inefficient? It is unlikely that it was ignorance because many of the hospitals participating in ISIS-4 also participated in ISIS-1. Was it inefficiency? This is unlikely because the use of aspirin and thrombolysis (studied in http://cvm.controlled-trials.com/content/1/1/028 commentary review reports primary research ISIS-2) was very high, by international standards. So it seems probable that the physicians concerned were not wholly convinced by ISIS-1.

How much weight should we put on biological plausibility?
There is a tendency to believe that somehow the designers of trials have been able to choose a basically homogeneous group of patients and, equally, that they have sound reasons for their exclusion criteria. But this is far from the case. Beta-blockers are known to lower blood pressure, sometimes dangerously, in patients already hypotensive or in acute heart failure. It was entirely appropriate, therefore, that low blood pressure was an exclusion criterion in ISIS-1. The investigators chose the figure of 100 mmHg, below which one should not give the drug [1]. Why 100? There is no reason for choosing this number except for its roundness. Might it not be equally appropriate to choose a higher number? An unexpected finding in ISIS-1 was that the main reason for the lower mortality in the treated group was the prevention of cardiac rupture on the first day. Hypertension is one of the factors in its genesis. As shown in Table 1, there does seem to be a relationship between blood pressure and outcome.

Would it be very wrong to use this information in helping to decide whom to treat?
Although not conventionally significant, the Metoprolol in Acute Myocardial Infarction (MIAMI) trial [3] leant support to the ISIS-1 trial and was the most important other contributor to the meta-analysis. Risk categories were predefined in this study, as shown in Table 2.
Would it be right to treat the more than 50% of patients with 0-2 risk factors with metoprolol? Yet that is what the pundits imply we should do. On the other hand, it would be brave to dismiss the findings in the high-risk subgroups.
A proper statistical analysis (as shown with the figures presented in the tables) shows some, but not overwhelming, support for the arguments advanced.

Can we trust meta-analyses?
The first meta-analysis of 27 536 patients [1], undertaken in 1986, was significant at the 0.02 level, and demonstrated that intravenous followed by oral beta-blockade would save six lives in every 1000 patients treated. The second metaanalysis [4], of 29 260 patients, undertaken in 1999, showed no significant benefit (95% confidence interval 0.85-1.08). Breslow-Day test: χ 2 = 5.269, DF = 2, P = 0.072. Breslow-Day (low systolic blood pressure versus the other two blood pressure ranges): χ 2 = 3.412, DF = 1, P = 0.065. Testing for interaction of blood pressure and study, P = 0.015. The first led authorities to recommend intravenous followed by oral beta-blockers for all patients. By the same logic, one should now treat no patients with this regimen. Clearly, this is ridiculous. So do we recommend it then for subgroups? Of course, it is desirable for the subgroups to be defined in advance and for them to be few in number. It is also customary to say that subgroup findings should be hypothesis-generating, but these hypotheses are seldom tested in adequately sized trials.
It seems to me reasonable to look for evidence of benefit or harm in certain biologically plausible subgroups, even though the statistical basis for this evidence is not compelling. It is ludicrous and economically mad that we have to treat 1000 patients to benefit six because our trial methodology cannot cope. An intelligent review of subgroups would allow this ratio to be much more reasonable.

Conclusion
There are sound reasons for not drawing conclusions from subgroup analysis. Certainly, it is quite wrong use a computer to look for statistically significant subgroups, as was done in the case of the astrological signs. But cardiologists have to make decisions on the basis of the very imperfect evidence which clinical trials usually provide. Let them use their knowledge of the biology of the condition to interpret which subgroups are clinically relevant and look for (albeit imperfect) statistical support.