Comparison between randomized clinical trials designating an outcome as primary and those designating the same outcome as secondary
2Department of Medicine, School of Medicine, Johns Hopkins University, Baltimore, USA, USA
Background: An outcome may be selected as a primary or a secondary outcome in a randomized clinical trial (RCT). Researchers may expect a larger effect size from this outcome when it is selected as a primary outcome than when it is selected as a secondary outcome, leading to expectation bias.
Objectives: To compare the effect estimates between RCTs selecting an outcome as a primary outcome (RCT-POs) and those selecting the same outcome as a secondary outcome (RCT-SOs) assessing the same clinical question.
Methods: This was a retrospective cohort study on RCTs. We screened six high-impact journals (the Annals of Internal Medicine, The BMJ, JAMA, JAMA Internal Medicine, The Lancet, and PLoS Medicine) published between 2017 and 2022 for meta-analyses that assessed the efficacy of clinical interventions and produced statistically significant estimates. In each meta-analysis, RCT-POs were included in the exposure group, while RCTs-SOs were in the control group. The ratio of risk ratio (RRR), hazard ratio (RHR), or odds ratio (ROR, with OR transformed from SMD) between RCTs-PO and RCTs-SO were pooled to form a single estimate by random-effects meta-analyses. We planned to include 100 meta-analyses.
Results: This was an interim analysis. A total of 59 meta-analyses were identified, including 27 assessing medical products and 32 assessing other interventions. 606 RCTs were included, comprising 259 RCT-POs and 347 RCT-SOs. On Average, RCT-POs produced an effect estimate 1.18 (95% CI: 1.08-1.30, I2=44.0%) times greater than RCT-SOs. RCT-POs assessing medical products produced an effect estimate 1.24 (95% CI: 1.12, 1.38; I2=44.3%) times greater than RCT-SOs, while RCT-POs assessing other interventions produced an effect estimate 1.09 (95%: 0.93, 1.29; I2=44.7%) times greater than RCT-SOs.
Conclusions: RCT-POs generally produced an effect estimate 18% greater than RCT-SOs, implying that the expectations bias could significantly distort the results of RCTs, especially those assessing the efficacy of medical products.
Patient, public and/or healthcare consumer involvement: