Comparison between randomized clinical trials designating an outcome as primary and those designating the same outcome as secondary

Date & Time
Wednesday, September 6, 2023, 11:35 AM - 11:45 AM
Location Name
Victoria
Session Type
Oral presentation
Category
Bias
Oral session
Bias
Speakers
Yi-Wen Jiang, Shenzhen Institute of Advanced Technology, Chinese Acadamy of Sciences
Yuanxi Jia, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences
Authors
Jiang Y1, Jia Y1, Yang Z1, Robinson KA2, Tang J1
1Shenzhen Institute of Advanced Technology, Chinese Acadamy of Sciences, China
2Department of Medicine, School of Medicine, Johns Hopkins University, Baltimore, USA, USA
Description

Background: An outcome may be selected as a primary or a secondary outcome in a randomized clinical trial (RCT). Researchers may expect a larger effect size from this outcome when it is selected as a primary outcome than when it is selected as a secondary outcome, leading to expectation bias.
Objectives: To compare the effect estimates between RCTs selecting an outcome as a primary outcome (RCT-POs) and those selecting the same outcome as a secondary outcome (RCT-SOs) assessing the same clinical question.
Methods: This was a retrospective cohort study on RCTs. We screened six high-impact journals (the Annals of Internal Medicine, The BMJ, JAMA, JAMA Internal Medicine, The Lancet, and PLoS Medicine) published between 2017 and 2022 for meta-analyses that assessed the efficacy of clinical interventions and produced statistically significant estimates. In each meta-analysis, RCT-POs were included in the exposure group, whereas RCTs-SOs were in the control group. The ratio of risk ratio (RRR), hazard ratio (RHR), or odds ratio (ROR, with OR transformed from SMD) between RCTs-PO and RCTs-SO were pooled to form a single estimate by random-effects meta-analyses. We planned to include 100 meta-analyses.
Results: This was an interim analysis. A total of 59 meta-analyses were identified, including 27 assessing medical products and 32 assessing other interventions. Six hundred six RCTs were included, comprising 259 RCT-POs and 347 RCT-SOs. On average, RCT-POs produced an effect estimate 1.18 (95% CI: 1.08-1.30, I2=44.0%) times greater than RCT-SOs. RCT-POs assessing medical products produced an effect estimate 1.24 (95% CI: 1.12-1.38; I2=44.3%) times greater than RCT-SOs, whereas RCT-POs assessing other interventions produced an effect estimate 1.09 (95%: 0.93-1.29; I2=44.7%) times greater than RCT-SOs.
Conclusions: RCT-POs generally produced an effect estimate 18% greater than RCT-SOs, implying that the expectations bias could significantly distort the results of RCTs, especially those assessing the efficacy of medical products. Patient, public, and/or healthcare consumer involvement: