Background: Despite the increasing popularity of systematic reviews (SRs), it is crucial to assess their certainty of evidence (CoE) because the design itself does not guarantee high CoE.
Objectives: To evaluate whether CoE was reported in SRs in infertility journals and whether they used appropriate wording to describe that CoE.
Methods: We selected five fertility journals with the highest impact factors. We performed a search on PubMed, identifying potential SRs with meta-analysis. We screened the studies by title and abstract and analyzed whether they classified the evidence in the full text and in the abstract. When CoE was not evaluated, we analyzed it with Grading of Recommendations Assessment, Development and Evaluation (GRADE). We described how often the authors used a tool for CoE and the level of the CoE published in the selected SRs. We also analyzed whether using a tool for CoE was associated with the P-value and whether the P-value was associated with the level of the evidence. Finally, we analyzed whether the study authors made any effort to adapt the wording used in the abstract to the CoE and the magnitude of the described estimated effect.
Results: CoE was reported in 21.4% of the SRs and in less than 10% of the abstracts. Although we did not find important differences in the reports of CoE of those that showed statistically significant differences and those that did not, P-value was associated with the wording chosen by authors. In general, magnitude of the effect was not expressed with consistent wording in 54.8% (23/42) of the SRs, wheras level of CoE was not expressed with consistent wording in 92.9% (39/42). Whereas magnitude of the effect was more consistently expressed in studies with statistically significant findings, CoE was better expressed when the P-value was greater than 0.05.
Conclusions: In major infertility journals, less than 25% of authors reported the overall CoE in SRs. Authors still focus more on the discussion if the found difference was by chance or not and less on limitations in the study design, imprecision, indirectness, inconsistency, and publication bias. Authors should make efforts to interpret results in the context of those evaluations. Patient, public, and/or healthcare consumer involvement: None.