Background: In 2008, Cochrane released the Cochrane risk-of-bias (RoB) tool. A revised version for randomized trials (RoB 2) that seeks to address different concerns is being tested since 2019.
Objectives: To compare the performance of RoB vs RoB 2 tools when applied to the same Cochrane review, analyzing differences in the assessment results in both individual domains and overall risk of bias.
Methods: An ongoing Cochrane review entitled ‘Blood pressure targets for hypertension in people with chronic renal disease’, whose protocol was published in 2019 (CD008564), was included for the RoB 2 tool pilot. Once finished with the main assessment, the risk of bias was again rated using the RoB tool. Any discrepancy on randomization, deviations-from-intended-interventions/blinding, missing/incomplete outcome data, reported result and overall bias, was noted and interpreted. The ‘Outcome Measurement’ domain was only estimated with the RoB 2 tool.
Results: Six randomized trials (AASK, ACCORD, HOT, MDRD, SPRINT, SPS3) were included in the review. According to the RoB 2 tool, the bias assessment by domain was judged as follows: ‘Randomization’ [Low risk, 6 studies]; ‘Deviations from intended interventions’ because of blinding concerns [Some concerns, 6 studies]; ‘Missing/incomplete outcome data’ [Low risk, 6 studies]; ‘Outcome measurement’ [Low risk, 4 studies; Some concerns: 1 study; High risk: 1 study]; ‘Reported result’ [Low risk, 6 studies]; and ‘Overall bias’ [Some concerns, 5 studies; High risk, 1 study] (Table 1). When RoB was used, a similar judgement was reached on Randomization, missing/incomplete outcome data, and reported result. On the contrary, blinding of participants/clinicians was judged as high risk for all studies, which subsequently led to high risk for all trials in the ‘Overall bias’ assessment (Table 2).
Conclusions: In the Cochrane review under consideration, RoB 2 led to a downplayed overall risk of bias compared to the RoB tool, mainly due to the fact that less emphasis is placed on blinding in the RoB 2 tool. Further research is needed on the practical implications of moving to another risk-of-bias tool, which is closely linked to how the certainty of evidence will be graded in the future. Patient, public, and/or healthcare consumer involvement: None.