Background: The COVID-19 pandemic instigated unprecedented challenges for clinical and policy decisions. In response to these challenges, the pandemic also resulted in much-awaited innovations. Objective: To perform a scoping review describing challenges and innovations in evidence synthesis and evaluation during the COVID-19 pandemic.
Methods: We searched medical research databases, up to July 2022, for articles describing challenges or innovations in evidence synthesis and evaluation during the COVID-19 pandemic. We used a reflexive thematic analysis to generate themes that summarize and describe the data.
Results: We identified 91 references, reporting on 21 and 20 unique challenges and innovations, respectively. Salient challenges included the volume and rapidity of published COVID-19 research (often of poor quality), which precluded efficient identification of practice-changing research. To help evidence users navigate the volume of research, several initiatives produced living databases and catalogues of COVID-19 studies. During the pandemic, the scientific community very enthusiastically embraced preprint servers to allow for the rapid dissemination of potentially lifesaving research. The medical community, however, expressed concerns that preprints may lead to the dissemination of research that contains errors or fraudulent or fabricated data. Most of the evidence guiding the treatment of patients with COVID-19 came from platform trials—adaptive trials designed with the option to add and remove arms based on accumulative evidence. Although there are many trials addressing pharmacologic interventions, there were few trials addressing nonpharmacologic interventions to curb transmission, despite their widespread global implementation. The pandemic also resulted in a breakthrough for living systematic reviews and guidelines, which, unlike traditional reviews and guidelines that are only up to date at a single point in time, are updated as new evidence emerges.
Conclusions: The COVID-19 pandemic posed unprecedented challenges to evidence synthesis and evaluation but also spurred critical innovations. As the pandemic shifts from an acute threat to a long-term public health issue, now is a critical time to evaluate and learn from our response to inform future preparedness efforts.
Patient, public and/or healthcare consumer involvement: While we did not include consumers as part of this review, several publications included in our review describe challenges and innovations related to consumer involvement.