Meta-analysis: what is it all for?
Meta-analysis has grown increasingly complex in recent years, with a growing range of alternative, and sometimes conflicting, methods for combining studies, weighting studies, estimating heterogeneity, calculating confidence intervals and so on. It is increasingly difficult to decide what methods to use, and difficult for stakeholders to understand the methods and be confident of their validity.
To challenge the growing complexity of meta-analysis by seeking to return to the first principles for meta-analysis, and considering why we perform meta-analyses at all.
It will be argued that there are two key purposes for meta-analysis:
1. To summarise the evidence in identified studies of an intervention.
2. To investigate a deeper “scientific truth” about the intervention.
The talk will make the case that the growing complexity in meta-analysis stems from a confusion of the two stated purposes of meta-analysis: by making the flawed assumption that a simple summary of evidence extracted from publications can tell us about the truth underlying the included studies.
The talk will demonstrate that a simple “assumption-free” weighted average approach to meta-analysis is always valid if we are aiming only to summarise the available evidence. This also applies when considering heterogeneity. A simple approach has limitations, particularly with few studies or data, but shifting to more complex methods does not solve the problems.
Conversely, identifying the “truth” about an intervention requires strong assumptions about the studies, that are likely to be flawed. Alternatively, it requires a more complex analysis approach, with better data, such as an individual participant data meta-analysis.
We need to be clearer what the purpose of any meta-analysis is, and honest about its limitations. Simple, conventional approaches may be best when we are restricted to summarising evidence in publications.
Greater clarity will ensure that patients and stakeholders have a better understanding of what any meta-analysis is actually telling them, and what it can’t tell them, without unhelpful methodological complications. This should support greater understanding of the strengths, and limitations, of meta-analyses.
Patient, public and/or healthcare consumer involvement: No involvement