The problems with systematic reviews: working towards the solutions

Date & Time
Tuesday, September 5, 2023, 4:05 PM - 4:25 PM
Location Name
St James
Session Type
Oral presentation
Priority setting
Oral session
Methodological quality and evidence synthesis innovation
Uttley L1, Falzon L1, Carroll C1, Quintana D2, Montgomery P3, Page M4, Moher D5
1University of Sheffield, UK
2University of Oslo, Norway
3University of Birmingham, UK
4Monash University, Australia
5University of Ottawa, Canada

Background: Systematic reviews are the gold-standard of evidence syntheses and underpin decision making which affects outcomes for patients and health consumers. Our living systematic review has found 485 articles documenting 67 discrete problems relating to the conduct and reporting of systematic reviews which can potentially jeopardise their reliability or validity.[1] A dedicated website has been created ( to thematically categorise and elaborate these issues around four overarching domains relating to systematic reviews being 1) comprehensive, 2) rigorous, 3) transparent, and 4) objective. The next phase of this research is to engage researchers working in systematic review to discuss the challenges around the trustworthiness of systematic reviews and explore solutions to the problems identified. (PROSPERO: CRD42020181371; Open Science Framework (OSF):
Objectives: To perform a prioritisation exercise of the problems with systematic reviews identified by consulting with researchers internationally and across disciplines to improve future systematic reviews.
Methods: Corresponding study authors of the 485 included papers will be invited via email to take part in a prioritisation exercise. Systematic review problems will be ranked based on relative importance, the severity or risk posed, and the ease of solution for the 67 problems identified thus far. Study methods for this work will be prespecified and registered with the OSF.
Results: Results of the prioritisation exercise with researchers will be twofold: 1) The project website will be expanded to incorporate the results of the engagement with researchers and to increase the value of as a research resource in systematic reviews and 2) A corresponding journal article will be used to further promote the research findings.
Conclusions: Systematic reviews have a pivotal role in medical decision-making due to apparent comprehensive, rigorous, replicable, and objective processes. In addition to outlying the many problems that systematic reviews are prone across these domains we will work with the research community to ensure that outputs which inform patient care represent trusted evidence. References: Uttley, L., Quintana, D. S., Montgomery, P., Carroll, C., Page, M. J., Falzon, L., ... & Moher, D. (2023). The Problems with Systematic Reviews: A Living Systematic Review. Journal of Clinical Epidemiology.