Pooled risk differences - a comparison between estimates from randomised controlled trials and estimates from propensity score-matched non-RCTs
2HTA-Centrum and Dept of Obstetrics and Gynecology, Sahlgrenska University Hospital; Dept of Obstetrics and Gynecology, Sahlgrenska Academy, University of Gothenburg, Sweden
3HTA-Centrum, Sahlgrenska University Hospital; Dept of Health and Rehabilitation, Sahlgrenska Academy, University of Gothenburg, Sweden
4HTA-Centrum, Sahlgrenska University Hospital, Sweden
Background: Estimating number needed to treat (NNT) from pooled risk differences may provide useful information for decision-making. Traditionally, such information is obtained from randomised controlled trials (RCTs). With increasing amounts of observational data available, as well as published comparisons using propensity score (PS)-matching to address confounding by indication, it may be appealing to rely on pooled non-RCTs for this purpose, particularly when RCTs are not available.
Objectives: To compare estimates of NNT based on pooled absolute risk differences from RCTs versus PS-matched non-RCTs.
Methods: The basis for these analyses was RCTs and non-RCTs identified in a health technology assessment (HTA) comparing clopidogrel versus ticagrelor as part of dual antiplatelet treatment after acute coronary syndrome (HTA-centrum/Sahlgrenska University Hospital, Sweden, 2021:123). The present comparison between pooled RCTs and pooled non-RCTs focused on the following three outcomes: all-cause mortality, myocardial infarction (MI), and major bleeding. For each outcome, pooled absolute risk differences with 95% confidence intervals (CI) were estimated using random-effects meta-analyses for RCTs and PS-matched non-RCTs separately, and NNT were calculated for statistically significant results.
Results: In all, 21 RCTs (29,314 patients) and 13 non-RCTs (103,363 PS-matched patients; 11%-79% of the studied populations) were included in the meta-analyses. For all-cause mortality, the pooled risk differences from RCTs and non-RCTs were 0.5 (95% CI: -0.3; 1.4) and 1.5 (0.3; 2.6) percentage points, respectively, i.e., passing the line of unity for RCTs whilst favouring ticagrelor for non-RCTs with an NNT of 67 to avoid one death. For MI, the pooled risk differences from RCTs and non-RCTs were 0.8 (0.03; 1.5) and 0.5 (-0.2; 1.2) percentage points, respectively, i.e., passing the line of unity for non-RCTs whilst favouring ticagrelor for RCTs with an NNT of 125 to avoid one MI. For major bleeding, the pooled risk differences from RCTs and non-RCTs were -0.8% (-1.5; -0.03) and -0.6 (-1.2; -0.1) percentage points, respectively, both favouring clopidogrel with an NNT of 125 versus 167 to avoid one major bleeding.
Conclusions: Statistical significance/NNT of pooled risk difference diverged between RCTs and PS-matched non-RCTs, a finding of importance for certainty of evidence assessments when RCTs are not available for conclusions