Background: The COVID-19 pandemic presented policymakers with time-sensitive decision problems within an ever-changing environment and rapidly increasing volume of research. The Wales COVID-19 Evidence Centre (WCEC), a collaboration of systematic review and economic evaluation expert groups, was established to shape the Welsh government’s response. WCEC developed a flexible evidence review process to respond to decision-makers’ needs within days, weeks, or months. The approach used, as well as the methodological shortcuts applied, was tailored depending on decision-maker needs, timeframe, and volume and type of evidence. The phased approach generated three products: Rapid Evidence Summary (RES), Rapid Evidence Map (REM), and Rapid Review (RR). Decision-maker, stakeholder, and public/patient engagement was integral from the onset. A core management team worked closely with partner groups, undertaking evidence reviews. Three key rapid review guidance publications informed a Best Practice Framework to support reviewers in their work.
Objectives: To reflect on our experience of implementing the WCEC review process and identify key learning points. To ascertain whether our reviews, mainly completed within 2 months, comply with best-practice guidance, and methods were aligned across review groups.
Methods: We compared methods used in our RRs and REMs with the Best Practice Framework recommendations. Findings, and key learning points, were discussed at a meeting of the core management team and collaborating review group members.
Results: In 2021-2022, 26 RRs and 5 REMs were completed. Our methods aligned with or exceeded most recommendations, particularly developing review questions, preliminary scoping work, conducting searches, critical appraisal, narrative synthesis, and report production. However, our review protocols were not registered; study selection, data extraction, and quality appraisal were generally conducted by a single reviewer; and the assessment of confidence in the evidence was generally subjective. Key learning points were identified, which will be presented. Notably, study selection, quality appraisal, and data extraction should include a verification sample of >20%. The strong stakeholder involvement was considered a particular strength, as was sharing knowledge between reviewers.
Conclusions: Our review process enabled us to successfully address a high volume of review questions swiftly, using a transparent and tailored approach. The findings will inform future development.
Patient, public and/or healthcare consumer involvement: None.