Meta-analysis: what is it all for?

Date & Time
Tuesday, September 5, 2023, 11:05 AM - 11:25 AM
Location Name
Session Type
Oral presentation
Statistical methods
Oral session
Statistical methods and meta-analysis
Simmonds M1
1Centre for Reviews and Dissemination, University of York, UK

Background Meta-analysis has grown increasingly complex in recent years, with a growing range of alternative and sometimes conflicting methods for combining studies, weighting studies, estimating heterogeneity, calculating confidence intervals, and so on. It is increasingly difficult to decide what methods to use, and it is difficult for stakeholders to understand the methods and be confident of their validity. Objectives To challenge the growing complexity of meta-analysis by seeking to return to the first principles for meta-analysis and considering why we perform meta-analyses at all. Concepts It will be argued that there are two key purposes for meta-analysis: 1. To summarise the evidence in identified studies of an intervention. 2. To investigate a deeper “scientific truth” about the intervention. The talk will make the case that the growing complexity in meta-analysis stems from a confusion of the two stated purposes of meta-analysis: by making the flawed assumption that a simple summary of evidence extracted from publications can tell us about the truth underlying the included studies. The talk will demonstrate that a simple “assumption-free” weighted average approach to meta-analysis is always valid if we are aiming only to summarise the available evidence. This also applies when considering heterogeneity. A simple approach has limitations, particularly with few studies or data, but shifting to more complex methods does not solve the problems. Conversely, identifying the “truth” about an intervention requires strong assumptions about the studies that are likely to be flawed. Alternatively, it requires a more complex analysis approach, with better data, such as an individual participant data meta-analysis. Conclusions We need to be clearer on what the purpose of any meta-analysis is and be honest about its limitations. Simple, conventional approaches may be best when we are restricted to summarising evidence in publications. Greater clarity will ensure that patients and stakeholders have a better understanding of what any meta-analysis is actually telling them and what it can’t tell them, without unhelpful methodological complications. This should support greater understanding of the strengths and limitations of meta-analyses. Patient, public, and/or healthcare consumer involvement: No involvement.