Background: In 2019, Cochrane released a revised Risk of Bias (RoB2) tool to address limitations of the previous tool. Since then, difficulties with implementation, adherence, and time taken to complete assessments have been reported. For large Individual Participant Data (IPD) Meta Analyses (MAs), applying RoB tools leads to additional challenges and opportunities.
Objectives: To develop a modified RoB tool combining elements of the original Risk of Bias (RoB1) with RoB2, incorporating checks assessing RoB using IPD, and implementing this in a meta-analysis (MA), including IPD from 59 trials and aggregate data (AD) from 33 trials.
Methods: We collated data from all included trials, and reviewed data availability for all outcomes. Content and methods experts were consulted to develop a consensus on domains and response options. Assessments were completed by two trained team members, and disagreements resolved in consensus meetings including a third team member. Remaining items were resolved by an advisory group, including members with extensive clinical, biostatistical, systematic reviews, and consumer experience.
Results: Instead of assessing RoB for each outcome, as done in RoB2, or for each study, as done in RoB1, the developed checklist (Figure) pre-specified RoB to be assessed per outcome category. These included primary outcome mortality, delivery room outcomes, beyond delivery room outcomes, and long-term outcomes. We judged RoB to be nearly identical for each category as most commonly either all or no follow-up outcomes were blinded. Additional checks utilising IPD were completed, assessing random allocation and incomplete outcome data. Overall RoB for each study, per outcome category, was judged as high, some concerns or low, matching RoB2. The modified RoB was implemented for all 92 trials. Availability of IPD and direct contact with investigators enabled higher rates of certainty for each domain, and strong consensus across raters compared to AD trials.
Conclusions: RoB1 and RoB2 have shortcomings for application in large IPD-MAs. A modified RoB tool for application in a large IPD meta-analysis was successfully developed and implemented within our study. Patient, public, and/or healthcare consumer involvement: It is essential to assess RoB of trials included in IPD-MAs, as these frequently inform patient care guidelines.

Cochrane_Abstract_RoB_1Mar2023_figure.pdf