Wales COVID-19 Evidence Centre: a bespoke evidence review process engaging stakeholders for supporting time-sensitive policy and clinical decision-making
2Health and Care Research Wales Evidence Centre, PRIME Centre Wales, Division of Population Medicine, School of Medicine, Cardiff University, UK
3Health Technology Wales (HTW), UK
4Specialist Unit for Review Evidence (SURE), Cardiff University, UK
5Bangor Institute for Health & Medical Research (BIHMR), School of Medical and Health Sciences, Bangor University, UK
6Wales Centre for Evidence Based Care (WCEBC), School of Healthcare Sciences, Cardiff University, UK
7Public Health Wales, UK
8Ser Cymru, Welsh Government, UK
Background: The COVID-19 pandemic presented policymakers with time-sensitive decision problems within an ever-changing environment and rapidly increasing volume of research. The Wales COVID-19 Evidence Centre (WCEC), a collaboration of systematic review and economic evaluation expert groups, was established to shape Welsh Government's response.
WCEC developed a flexible evidence review process to respond to decision-makers’ needs within days, weeks, or months. The approach used, and methodological shortcuts applied, were tailored depending on decision-maker needs, timeframe, and volume and type of evidence. The phased approach generated three products: Rapid Evidence Summary (RES), Rapid Evidence Map (REM), and Rapid Review (RR). Decision-maker, stakeholder and public/patient engagement was integral from the onset. A core management team worked closely with partner groups, undertaking evidence reviews. Three key rapid review guidance publications informed a Best Practice Framework to support reviewers in their work.
Objectives: To reflect on our experience of implementing the WCEC review process and identify key learning points. To ascertain whether our reviews, mainly completed within 2months, comply with best-practice guidance, and methods were aligned across review groups.
Methods: We compared methods used in our RRs and REMs with the Best Practice Framework recommendations. Findings, and key learning points, were discussed at a meeting of core management team and collaborating review group members.
Results: In 2021-2022, 26 RRs and 5 REMs were completed. Our methods aligned with or exceeded most recommendations, particularly: developing review question, preliminary scoping work, conducting searches, critical appraisal, narrative synthesis, and report production. However, our review protocols were not registered, study selection, data extraction and quality appraisal were generally conducted by a single reviewer, and the assessment of confidence in the evidence was generally subjective. Key learning points were identified, which will be presented. Notably, study selection, quality appraisal, and data extraction should include a verification sample of >20%. The strong stakeholder involvement was considered a particular strength, as was sharing knowledge between reviewers.
Conclusions: Our review process enabled us to successfully address a high volume of review questions swiftly, using a transparent and tailored approach. The findings will inform future development.
Patient, public and/or healthcare consumer involvement: none