Background: Systematic reviews of public health interventions can be methodologically challenging because they are more likely to incorporate complexity in populations, interventions, contexts, outcome measurements and study designs. When a systematic review or protocol manuscript is submitted for publication to Cochrane Public Health (CPH), Methods Editors conduct an initial triage review and complete methods review.
Objectives: To identify common methodological errors requiring amendment in manuscripts and increase awareness of the available guidance to reduce their occurrence in future reviews.
Methods: We conducted a retrospective audit of written feedback provided by CPH Methods Editors to authors of a protocol or review manuscript submitted for publication between July 2019 and June 2022. A data extraction form was used to document methodological issues identified in written feedback documents. Four Methods Editors piloted the data extraction form on two manuscripts, after which one Methods Editor extracted data for the remaining documents.
Results: Forty-six feedback documents relating to twenty-seven reviews were identified. Examples of the errors identified related to the following: application of risk of bias domains, inconsistency between the methods described and used in the analysis, application of synthesis methods other than meta-analysis, inconsistent reporting of results in different sections of the review and incomplete reporting of results. Some of these errors had important implications for the conclusions of the review. In addition, it was common for manuscripts to lack details necessary to allow readers to understand and appraise the methods used. Some errors required authors to review and edit their manuscripts for consistency or to refer to existing guidance, e.g., on risk of bias assessment or synthesis other than meta-analysis. Data extraction for this project is in progress, and complete results will be reported in the presentation.
Conclusions: These results present an opportunity for authors to plan for and appropriately implement methods guidance to avoid common errors in future reviews.
Patient, public and/or healthcare consumer involvement: The public were not involved in this study. Enhancing the methodological strength of published systematic reviews ensures reliable support for the public and other decision makers.