Impact of treatment duration on assessing medication-related harms in evidence synthesis of randomized trials: An empirical investigation
2Research Center of Epidemiology and Statistics (CRESS-U1153), Université Paris Cité, France
3School of Public Health, The University of Queensland, Australia
4Department of Epidemiology and Biostatistics, University of Arizona, USA
5Department of Pediatrics, Faculty of Medicine & Dentistry, University of Alberta, Canada
6Norwich Medical School, University of East Anglia, UK
Background: In clinical trials, the intervention duration may differ between study arms, which could modify treatment effects.
Objectives: In this study, we investigated the extent and impact of treatment duration on the effects of harms assessment in evidence synthesis practice.
Methods: Systematic reviews of randomized controlled trials assessing the safety of medications with at least one pairwise meta-analysis with binary outcome published between 1st January 2015 and 1st January 2020 were searched and identified through PubMed. Data from reviews that reported 2 by 2 table data for each trial in the meta-analyses with 5 or more studies were collected. Information on treatment duration of the included trials of each arm were also collected. A re-analysis of these data was conducted using a naïve method (i.e., Mantel-Haenszel) that did not address heterogeneous treatment duration as well as with four methods (e.g., mixed effect Poisson models) that addressed treatment duration. The effect size, P value, statistical power, and heterogeneity were compared for these methods. A simulation study was further carried out to verify the findings.
Results: A total of 151 systematic reviews with 588 meta-analyses were identified. Based on 328 (55.78%) meta-analyses with heterogeneous treatment duration, failing to address heterogeneous treatment duration resulted in 9.82% to 11.63% of the meta-analyses changing the direction of the effect estimates and 19.04% to 28.62% changing the significance of the P value, with higher statistical power and smaller heterogeneity observed. The simulation study confirmed accuracy and precision improved when treatment duration was addressed.
Conclusions: Heterogeneous treatment duration frequently occurs in evidence synthesis practice, which could result in spurious conclusions about adverse events and misleading healthcare recommendations. Researchers should routinely account for heterogeneous treatment duration in evidence synthesis whenever possible.
Patient, public and/or healthcare consumer involvement: None