Background: COVID-19 has driven the need for timely evidence to inform decision-making. However, summarizing the rapidly growing evidence has been a challenge. Systematic reviews with network meta-analyses (NMA) provide a complete, broad, and updated view of the evidence making it the best type of evidence synthesis to inform the development of practice recommendations. The rating of certainty of evidence (CoE) is a crucial step that determines the conclusions from the review, but it can be very time-consuming in the context of NMA. To our knowledge, there is no empirical evidence regarding the impact of having a single reviewer conduct CoE assessments in the context of NMA.
Objectives: To evaluate the impact of CoE assessments by a single reviewer versus two reviewers on the evidence summaries of NMA
Methods: We carried out a retrospective methodological study using data from all outcomes included in the body of evidence that informs recommendations of the WHO COVID-19 therapeutics living guideline. We used the GRADE assessments performed in the context of the COVID-19 LNMA that informs the recommendations. Pairs of reviewers independently, following training and calibration exercises, conducted these assessments following GRADE guidance, and solved disagreements through consensus. We defined an important difference in the ratings as an increase from low or very low to high or moderate or a decrease from high or moderate to low or very low. We will calculate the proportion of outcomes with important differences comparing the CoE rating from a single versus two reviewers. We will use a generalized linear mixed model to account for the outcomes of the same comparison.
Results: This study is ongoing, and results will be presented at Colloquium as available.
Conclusions: This study is ongoing, and results will be presented at Colloquium as available.
Patient, public and/or healthcare consumer involvement: Not applicable