Table 1 Key features and limitations of different approaches to assessing adverse effects of healthcare interventions
From: Drug safety assessment in clinical trials: methodological challenges and opportunities
| Study design | Key features | Limitations |
|---|---|---|
| Spontaneous or voluntary reporting systems, including journal-published case reports | Captures very wide range of events | No denominator or control group, difficult to quantify risk |
| Particularly useful for detecting signals of rare (low background incidence in treated population) and/or unexpected events (e.g., new unrecognized pathology) | Format and type of information differs substantially among regulators and journals | |
| Sophisticated statistical techniques have been developed for signal detection | Clinical details may be incomplete, causality uncertain | |
| Selective reporting or under-reporting of cases | ||
| Randomized clinical trials | Randomization reduces possibility of confounding at baseline | Rigid recruitment criteria may lead to exclusion of patients who are at risk of adverse effects |
| Certain adverse effects can be prospectively specified for detailed monitoring | Powered for detection of significant difference between groups for beneficial effect, estimates for adverse effects may lack precisions | |
| Intervention is typically well defined | ||
| Non-randomized studies | ‘Real-world’ use with more generalizable data and longer follow-up | Monitoring for rare or unexpected events may be less rigorous, and the trials may not be of sufficient duration to detect long-term problems |
| Potentially able to specify and assess rare events as primary outcomes in case control designs | Non-randomized nature is susceptible to confounding | |
| May be able to explore relationship to dose, duration and patient susceptibility factors | Drug exposures are often based on computerized records rather than dispensing or actual use | |
| Meta-analysis of controlled observational studies and/or trials | Pooled analysis has greater power to detect significant differences, even with rare events | Reliant on quality of primary data |
| Missing or unreported data on adverse events is a major problem, as are the statistical techniques of pooling sparse data | ||
| Aims to summarize complete data set and can evaluate consistency of findings among studies |
Susceptible to selective outcome reporting of primary studies Heterogeneity within the pooled analysis |