Background: Randomised controlled trials (RCTs) imply random allocation of participants to an intervention or a control group. When assessing risk of bias according to the Cochrane tool, the randomisation domain is crucial and may separate RCTs into low and high risk of bias.
Objectives: To compare meta-analyses of RCTs with low and high risk of bias in the randomisation domain.
Methods: A systematic review was performed within a health technology assessment (HTA) evaluating the effects of progesterone (Intervention) versus placebo (Comparison) in singleton pregnancies (Participants) regarding the Outcomes preterm birth (<37 gestational weeks) and neonatal mortality (<28 days). Systematic literature searches were performed in PubMed, Embase, Medline and the Cochrane Library, and 27 RCTs fulfilled this PICO. The RCTs were assessed using the Cochrane Risk of Bias tool and pooled in meta-analyses using random effects models. The meta-analyses were performed separately for RCTs with low and high risk of bias in the randomisation domain, the former defined as low risk of bias regarding both randomisation sequence generation and allocation concealment and the latter as (unclear or) high risk of bias in at least one of these.
Results: Meta-analyses of 14 RCTs with low risk of bias, compared with 12 RCTs with high risk of bias, showed a smaller reduction in the risk ratio of preterm birth: 0.82 (95% CI 0.71; 0.95) versus 0.56 (0.47; 0.65). The pooled risk differences were -6.6 (-11; -2.3) versus -20 (-29; -10) percentage points. Regarding neonatal mortality, the pooled risk ratios in low- and high-risk-of-bias RCTs (n=12 and n=4, respectively) were 0.60 (0.39; 0.92) versus 0.28 (0.07; 1.20). The pooled risk differences were -0.7 (-1.7; 0.4) versus -11 (-25; 3.0) percentage points.
Conclusions: Meta-analyses of RCTs with low and high risk of bias in the randomisation domain yielded substantially different results. The two included aspects, randomisation sequence generation and allocation concealment, may bias the results considerably. RCTs with high risk of bias in the randomisation domain seem to overestimate efficacy and thus may not be appropriate for drawing conclusions in systematic reviews. Patient involvement: A patient representative was involved in choosing outcomes.