Background: Random effects meta-analysis is widely used for synthesizing the studies of a systematic reviews under the assumption that the underlying study-specific treatment effects come from a common normal distribution. However, this assumption is sometimes not justifiable, such as in presence of substantial heterogeneity between the studies or highly skewed data. Few alternative options have been suggested in the literature, but they are not used in practice.
Objectives: To identify and compare alternative between-study distributions for random effects meta-analysis.
Methods: We conducted a methodological systematic review to identify articles that proposed and explored random-effects meta-analysis models relaxing the between-study normality assumption. Subsequently, we performed a simulation study to compare the identified models. We considered 90 scenarios, varying the amount of heterogeneity between studies, the skewness of the data, and the number of included studies.
Results: We identified 1022 articles in PubMed, of which 1015 were excluded after screening the title/abstract and according to our eligibility criteria. We further added relevant articles through hand-searching in Google Scholar and other related journals, and we concluded with 13 eligible articles suggesting 20 alternative random effects models. Our simulation study is still ongoing, but preliminary results reveal that in the presence of skewed or high heterogeneous data, the normal between-study assumption can lead to highly biased summary effects.
Conclusions: We conclude that the plausibility of the normality assumption should be assessed more thoroughly when conducting a meta-analysis, and alternative methods should be used when normality is not deemed plausible. Patient, public, and/or healthcare consumer involvement: None.