Impact of Grading of Recommendations, Assessment, Development and Evaluation (GRADE) on conclusions of dentistry systematic reviews

Date & Time
Monday, September 4, 2023, 11:25 AM - 11:35 AM
Location Name
Session Type
Oral presentation
Assessment of the certainty of evidence
Oral session
Assessing evidence certainty
Azab M1, Ibrahim S2, Chu J3, Li A3, Modi K4, Wang C4, Hong K5, Abumustafa Y6, Yan Y4, Park S7, Miroshnychenko A2, Brignardello-Petersen R2
1Temerty Faculty of Medicine, University of Toronto, Canada
2Department of Health Research Methods, Evidence, and Impact, McMaster University, Canada
3Faculty of Science, McMaster University, Canada
4Faculty of Health Sciences, McMaster University, Canada
5Department of Kinesiology, Faculty of Science, McMaster University, Canada
6Faculty of Medicine, University of Ottawa, Canada
7School of Pharmacy, University of Waterloo, Canada

Background: The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach assesses certainty of evidence (CoE) in a systematic review (SR) and facilitates drawing appropriate conclusions that consider limitations of the evidence. However, GRADE is not widely used in dentistry SRs.
Objectives: To evaluate the impact of using GRADE on conclusions in dentistry SRs.
Methods: From a sample of 200 dentistry SRs of randomized controlled trials included in a larger methodological study, we selected a subsample based on the most frequently reported outcomes. We will use the odds ratio and its 95% confidence interval to determine whether SRs not using GRADE are more likely to formulate inappropriate conclusions (i.e, conclusions relying on statistical significance, not considering limitations or formulating recommendations). For SRs not using GRADE, we will compare conclusions made by authors to conclusions we made based on our own GRADE assessments using the same effect thresholds as SR authors. We will calculate the number of conclusions that changed with respect to the described certainty (i.e., the extent to which authors recognized uncertainty, analogous to high, moderate, low, or very low CoE in GRADE) and magnitude of effect (i.e., whether the conclusions state there was a presence or absence of effect, negligible or important effect) after utilizing GRADE.
Results: We present the results of a subsample of 19 SRs reporting the outcome of pain. Six (32%) used GRADE, and 13 (68%) did not. Two SRs formulated inappropriate conclusions; both did not use GRADE. For SRs without GRADE, our GRADE assessments changed 62% of conclusions with respect to the described CoE. Our GRADE assessments changed the described magnitude in 25% of conclusions. We will present results for additional outcomes currently under analysis.
Conclusions: After completing all analyses, our conclusions will focus on the influence of GRADE on formulating appropriate conclusions which will provide insight into potential limitations that may arise when SRs formulate conclusions without GRADE assessments. Developing more transparent SR conclusions is relevant to patients as SRs are a valuable source of evidence for decision-makers. Patient, public, and/or healthcare consumer involvement: Patients were not involved.