Measuring the impact of zero-cases studies in evidence synthesis practice using the harms index and benefits index (Hi-Bi)

Session Type
Evidence synthesis innovations and technology
Xu C1, Luis F2, Lin L3, Zorzela L4, Yu T5, Vohra S4
1Key Laboratory of Population Health Across Life Cycle, Ministry of Education of the People's Republic of China (Anhui Medical University), China
2School of Public Health, Faculty of Medicine, University of Queensland, Australia
3Department of Epidemiology and Biostatistics, University of Arizona, USA
4Department of Pediatrics, Faculty of Medicine & Dentistry, University of Alberta, Canada
5Research Center of Epidemiology and Statistics (CRESS-U1153), Université Paris Cité, France

Background: In evidence synthesis practice, dealing with studies with no cases in both arms has been a tough problem, for which there is no consensus in the research community.
Objectives: In this study, we propose a method to measure the potential impact of studies with no cases for meta-analysis results which we define as harms index (Hi) and benefits index (Bi) as an alternative solution for deciding how to deal with such studies.
Methods: Hi and Bi are defined by the minimal number of cases added to the treatment arm (Hi) or control arm (Bi) of studies with no cases in a meta-analysis that lead to a change of the direction of the estimates or its statistical significance. Both exact and approximating methods are available to calculate Hi and Bi. We developed the “hibi” module in Stata so that researchers can easily implement the method. A real-world investigation of meta-analyses from Cochrane reviews was employed to evaluate the proposed method.
Results: Based on Hi and Bi, our results suggested that 21.53% (Hi) to 26.55% (Bi) of Cochrane meta-analyses may be potentially impacted by studies with no cases, for which studies with no cases could not be excluded from the synthesis. The approximating method shows excellent specificity (100%) for both Hi and Bi, moderate sensitivity (68.25%) for Bi, and high sensitivity (80.61%) for Hi compared to the exact method.
Conclusions: The proposed method is practical and useful for systematic reviewers to measure whether studies with no cases impact the results of meta-analyses and may act as an alternative solution for review authors to decide whether to include studies with no events for the synthesis or not.
Patient, public and/or healthcare consumer involvement: None

Figure 1.pdf
Figure 2.pdf