Pooled risk differences - a comparison between estimates from randomised controlled trials and estimates from propensity score-matched non-RCTs
2HTA-Centrum and Dept of Obstetrics and Gynecology, Sahlgrenska University Hospital; Dept of Obstetrics and Gynecology, Sahlgrenska Academy, University of Gothenburg, Sweden
3HTA-Centrum, Sahlgrenska University Hospital; Dept of Health and Rehabilitation, Sahlgrenska Academy, University of Gothenburg, Sweden
4HTA-Centrum, Sahlgrenska University Hospital, Sweden
Background: Estimating number needed to treat (NNT) from pooled risk differences may provide useful information for decision making. Traditionally, such information is obtained from randomised controlled trials (RCTs). With increasing amounts of observational data available, as well as published comparisons using propensity score (PS)-matching to address confounding by indication, it may be appealing to rely on pooled non-RCTs for this purpose, particularly when RCTs are not available.
Objectives: To compare estimates of NNT based on pooled absolute risk differences from RCTs versus PS-matched non-RCTs.
Methods: The basis for these analyses was RCTs and non-RCTs identified in a health technology assessment (HTA) comparing clopidogrel versus ticagrelor as part of dual antiplatelet treatment after acute coronary syndrome (HTA-centrum/Sahlgrenska University Hospital, Sweden, 2021:123). The present comparison between pooled RCTs and pooled non-RCTs focused on three outcomes: all-cause mortality, myocardial infarction (MI) and major bleeding. For each outcome, pooled absolute risk differences with 95% confidence intervals (CI) were estimated using random-effects meta-analyses for RCTs and PS-matched non-RCTs separately, and NNT were calculated for statistically significant results.
Results: In all, 21 RCTs (29,314 patients) and 13 non-RCTs (103,363 PS-matched patients; 11%‒79% of the studied populations) were included in the meta-analyses. For all-cause mortality, the pooled risk differences from RCTs and non-RCTs were 0.5 (95% CI: -0.3; 1.4) and 1.5 (0.3; 2.6) percentage points, respectively, i.e. passing the line of unity for RCTs whilst favouring ticagrelor for non-RCTs with an NNT of 67 to avoid one death. For MI, the pooled risk differences from RCTs and non-RCTs were 0.8 (0.03; 1.5) and 0.5 (-0.2; 1.2) percentage points, respectively, i.e. passing the line of unity for non-RCTs whilst favouring ticagrelor for RCTs with an NNT of 125 to avoid one MI. For major bleeding, the pooled risk differences from RCTs and non-RCTs were -0.8% (-1.5; -0.03) and -0.6 (-1.2; -0.1) percentage points, respectively, both favouring clopidogrel with an NNT of 125 versus 167 to avoid one major bleeding.
Conclusions: Statistical significance/NNT of pooled risk difference diverged between RCTs and PS-matched non-RCTs, a finding of importance for certainty of evidence assessments when RCTs are not available for conclusions.