IMPACT OF A SINGLE REVIEWER ON CERTAINTY OF EVIDENCE ASSESSMENT IN NETWORK META-ANALYSIS
2Departamento de medicina interna, Hospital Alemán de Buenos Aires, Argentina
3Department of Health Research Methods, Evidence, and Impact, McMaster University, Canada
4Department of Health Research Methods, Evidence, and Impact, McMaster University, Chile
Background: COVID-19 has driven the need for timely evidence to inform decision-making. However, summarizing the rapidly growing evidence has been a challenge. Systematic reviews with network meta-analyses (NMA) provide a complete, broad, and updated view of the evidence, making it the best type of evidence synthesis to inform the development of practice recommendations. The rating of certainty of evidence (CoE) is a crucial step that determines the conclusions from the review, but it can be very time-consuming in the context of NMA. To our knowledge, there is no empirical evidence regarding the impact of having a single reviewer conducting CoE assessments in the context of NMA.
Objectives: To evaluate the impact of CoE assessments by a single reviewer versus two reviewers on the evidence summaries of NMA.
Methods: We carried out a retrospective methodological study using data from all outcomes included in the body of evidence that informs recommendations of the World Health Organization (WHO) COVID-19 therapeutics living guideline. We used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) assessments performed in the context of the COVID-19 LNMA that informs the recommendations. Pairs of reviewers independently, following training and calibration exercises, conducted these assessments following GRADE guidance, and solved disagreements through consensus. We defined an important difference in the ratings as an increase from low or very low to high or moderate or a decrease from high or moderate to low or very low. We will calculate the proportion of outcomes with important differences comparing the CoE rating from a single versus two reviewers. We will use a generalized linear mixed model to account for the outcomes of the same comparison.
Results: This study is ongoing, and results will be presented at the Colloquium as available.
Conclusions: This study is ongoing, and results will be presented at the Colloquium as available.
Patient, public, and/or healthcare consumer involvement: Not applicable,