Background: Evidence on the value of pilot trials for subsequent trial’s quality is scarce.
Objectives: The current study aims to assess whether conducting a pilot trial is associated with improved quality, feasibility, and likelihood of observing statistically significant efficacy in the corresponding full-scale trial.
Methods: PubMed was searched on February 19th, 2022, for pilot trials published from 2005 to 2018 and their subsequent full trials. Using the meta-analysis of the full trial, we located other full-scale trials on the same or similar research topic but without a pilot trial. Trial feasibility was measured by parameters including recruitment rate and attrition. Proxies of trial quality included publication outcomes and Cochrane Risk of Bias Assessment results dichotomized as low versus high risk of bias. Random-effects logit models with a clustered sandwich estimator of variance were used to explore the associations adjusted for study-level confounders.
Results: Fifty-eight full-scale trials with a preceding pilot trial and 151 full-scale trials without a pilot trial were identified from 47 meta-analyses. Most trials examined behavioral interventions (31.9%). Among full-scale trials with or without a pilot trial, 30.0% and 40.1%, respectively, found a statistically significant difference in the primary efficacy endpoint (P>0.05); they had comparable recruitment rates (median: 18.8 vs 16.0 participants per month) and dropout percentages (mean: 12.6% vs 13.5%). Trials with a pilot trial were on average published 0.9 years earlier (1.7±1.0 vs 2.6±2.0 years, P=0.006) and were published in peer-reviewed journals that had a higher impact factor (60.9±75.0 vs 24.8±50.3, P<0.001). The presence of a pilot trial was-associated with an increased likelihood of having a low risk of bias in the full-scale trial’s allocation concealment (OR=3.67, 95% CI: 1.47 - 9.18) and masking of participants and researchers (OR=4.63, 95% CI: 1.64 – 13.07). No significant association was seen for the domains of random sequence generation, blinding of outcome data, incomplete outcome data, and selective reporting.
Conclusions: Pilot trials may be associated with less bias, but not with increased intervention efficacy or trial feasibility of the full-scale trials.
Patient, public and/or healthcare consumer involvement: None