Background: The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach assesses certainty of evidence (CoE) in a systematic review (SR) and facilitates drawing appropriate conclusions that consider limitations of the evidence. However, GRADE is not widely used in dentistry SRs.
Objectives: To evaluate the impact of using GRADE on conclusions in dentistry SRs.
Methods: From a sample of 200 dentistry SRs of randomized controlled trials included in a larger methodological study, we selected a subsample based on the most frequently reported outcomes. We will use the odds ratio and its 95% confidence interval to determine whether SRs not using GRADE are more likely to formulate inappropriate conclusions (i.e, conclusions relying on statistical significance, not considering limitations or formulating recommendations). For SRs not using GRADE, we will compare conclusions made by authors to conclusions we made based on our own GRADE assessments using the same effect thresholds as SR authors. We will calculate the number of conclusions that changed with respect to the described certainty (i.e., the extent to which authors recognized uncertainty, analogous to high, moderate, low, or very low CoE in GRADE) and magnitude of effect (i.e., whether the conclusions state there was a presence or absence of effect, negligible or important effect) after utilizing GRADE.
Results: We present the results of a subsample of 19 SRs reporting the outcome of pain. Six (32%) used GRADE, and 13 (68%) did not. Two SRs formulated inappropriate conclusions; both did not use GRADE. For SRs without GRADE, our GRADE assessments changed 62% of conclusions with respect to the described CoE. Our GRADE assessments changed the described magnitude in 25% of conclusions. We will present results for additional outcomes currently under analysis.
Conclusions: After completing all analyses, our conclusions will focus on the influence of GRADE on formulating appropriate conclusions which will provide insight into potential limitations that may arise when SRs formulate conclusions without GRADE assessments. Developing more transparent SR conclusions is relevant to patients as SRs are a valuable source of evidence for decision-makers. Patient, public, and/or healthcare consumer involvement: Patients were not involved.