PRACTICAL SCENARIO

Investigators conducted a systematic review (SR) study that included eight randomized controlled trials (RCTs) comparing drug A vs. drug B for the treatment of condition Y. The outcome of interest was 30-day all-cause mortality. Each study reported an effect estimate (OR) to compare the two drugs. Investigators then generated a pooled estimate to summarize the overall effect across the studies. How is that achieved in a meta-analysis?

WHAT IS A META-ANALYSIS?

A meta-analysis is a statistical approach that combines results from individual studies identified in an SR and calculates a pooled estimate of the magnitude and direction of treatment effects.(1) Consequently, the overall sample size and the precision of the estimate increase, and the width of confidence intervals decreases. The combined treatment effect is estimated by calculating a weighted average across individual study estimates. The weight assigned to each study result is related to the precision of each estimate, which in turn is related to the sample size of the study. Therefore, larger studies have a greater influence on the final pooled estimate.

Frequently, a meta-analysis follows an SR of individual RCTs or observational studies. Depending on the nature of the research question, a meta-analysis can be used to answer questions about intervention effectiveness, diagnostic/prognostic test accuracy, and disease burden (prevalence and incidence).

SRs often include studies with distinct features that lead to clinical, methodological, and statistical heterogeneity. Clinical heterogeneity arises from differences in study participants, interventions, or outcome definitions. Methodological heterogeneity arises, for example, when some of the RCTs included are blinded, and others are not. In a meta-analysis, statistical heterogeneity is formally assessed by calculating the I2 statistic, which ranges from 0% to 100%. An I2 > 50% indicates high heterogeneity, which should raise the question of whether it is reasonable to perform a meta-analysis or not and to prompt the search of potential underlying reasons for heterogeneity.

FOREST PLOTS: A VISUAL SUMMARY OF META-ANALYSIS RESULTS

A forest plot(2) is the key graphical representation of the major findings of an SR and meta-analysis. In our example (Figure 1), each row represents one of the 8 RCTs included in the SR with their respective effect estimates (OR and 95% CI). The bottom row represents the pooled estimate of the effect, that is, the result of the meta-analysis. Each individual study has a different relative weight; for example, study 7 has the largest weight, which is likely associated with a high precision of the estimate (smaller CI). Notably, specific estimates of most individual studies are not statistically significant (95% CI includes the value of 1), whereas the pooled estimate shows a statistically significant beneficial effect of drug A vs. drug B. The heterogeneity of the study was 45%, estimated by the I2 statistic.

 




SRs combined with meta-analyses are often considered as one of the highest levels of analysis in evidence-based medicine because they combine the results of various RCTs/observational studies and offer a more precise estimate of the effect size of a given intervention. They can be very useful for clinical decision making, although their results are only as good as the studies included in the analysis.

REFERENCES

1. Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al., editors. Cochrane Handbook for Systematic Reviews Interventions version 6.0 (updated July 2019). Chichester (UK): Wiley & Sons; 2019.
2. University of Oxford. Centre for Evidence-Based Intervention [homepage on the Internet]. Oxford: University of Oxford [cited 2016 Jun 1]. How to analyze the forest plot: assess the heterogeneity amongst the studies. Available from: https://www.spi.ox.ac.uk/how-to-interpret-the-sample-forest-plot