Research waste caused by symptom-based eligibility: a systematic review of Cochrane reviews and their component trials
2Department of Obstetrics and Gynaecology, Tameside & Glossop Acute Services NHS Trust, Ashton-Under-Lyne, UK, UK
Background: Systematic reviews of randomised trials are the gold standard of evidence for interventions. Therefore, it is imperative that trials are open to all relevant patients and that reviews include all appropriate trials. In gynaecology it is common that patients with the same diagnosis experience differing symptoms. Where an intervention may benefit the underlying condition, how do trialists define eligibility and select outcomes in the likely scenario that potential participants have no symptoms in common? Similarly, how often do reviewers exclude trials because outcomes are relevant to other symptoms?
Objectives: To identify inefficiencies and sources of research waste in the design of recent gynaecology trials and Cochrane reviews.
Methods: We searched for Cochrane reviews in polycystic ovarian syndrome (PCOS) and endometriosis as exemplar conditions. Reviews were included if the intervention was intended to treat all condition-specific symptoms. We restricted to trials published since 2012 to consider ‘current’ approaches. For each trial we recorded the number of potentially eligible participants excluded because of their symptoms. Similarly, for each review we recorded the numbers of trials and participants excluded unnecessarily.
Results: There were 89 distinct PCOS trials in 13 reviews, and 13 Endometriosis trials in 11 reviews. Most trials restricted eligibility to participants with specific symptoms (55% PCOS, 46% endometriosis). Reviews excluded 27% of available trials (n=27 in PCOS, n=1 in endometriosis) based on pre-specified outcomes not relevant to the symptoms of randomised participants. These excluded trials could have added 1914 participants (17%) and 44 participants (3%) to the evidence base for PCOS and endometriosis respectively.
Conclusions: Cochrane reviews excluded a quarter of available trials. Excluding trials because pre-specified review outcomes are not relevant to randomised participants greatly reduces the evidence base. The issue of outcome reporting bias is now well-recognised. We present a subtly different issue of outcome reporting that may lead to imprecision, rather than bias, in the context where an underlying condition presents diverse symptoms. Further work is ongoing to propose more efficient strategies in this context for both trialists and reviewers. Patient involvement: Involvement since idea inception, patients continue to help guide project progress through regular meetings.