Risk of bias tools: a systematic review of usability

Date & Time
Tuesday, September 5, 2023, 2:45 PM - 2:55 PM
Location Name
Session Type
Oral presentation
Oral session
Bias and certainty of evidence
Tomlinson E1, Rutjes AW2, Leeflang M3, Davenport C4, Mallett S5, Whiting P6
1Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK, United Kingdom
2Department of Medical and Surgical Sciences for Children and Adults (SMECHIMAI), University of Modena and Reggio Emilia, Modena, Italy, Italy
3Amsterdam University Medical Centers, University of Amsterdam, the Netherlands
4Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham Birmingham, B15 2TT, UK; NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, United Kingdom
5Centre for Medical Imaging, University College London, United Kingdom
6Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK, United Kingdom

Background: Inappropriate design, conduct and analysis of studies can lead to biased estimates of outcomes. Therefore, risk of bias (RoB) assessment of included studies is a crucial step in systematic reviews. Several tools exist to facilitate RoB assessments. Despite their widespread use, their usability is not well summarised.
Objectives: To review published findings of studies evaluating commonly used RoB tools.
Methods: We included RoB tools listed in the Library of Assessment Tools and InsTruments Used to assess Data validity in Evidence Synthesis (LATITUDES) Network. We conducted a forward citation search from the primary report for each RoB tool. Two review authors independently screened titles and abstracts and potentially relevant full texts. We included studies with the objective of evaluating the design and/or usability of an included tool. We summarise the structure and features of the different included tools, as well as similarities and differences between the tools. For each tool, we outline included study characteristics and methodology. Employing a deductive approach, one reviewer will use NViVo software to extract information meeting prespecified codes developed by the team (e.g., comments on usability of the tool, specific challenging items, and recommendations for future development). If other relevant information arises, we will move to an inductive approach and add new codes (which will be discussed within the team).
Results: This review is in progress. Results will be presented at the Colloquium.
Conclusions: Risk of bias assessment is an important part of the systematic review process. The findings of this review are expected to inform the development and updating of RoB tools, which in turn may positively affect the robustness of evidence production.
Patient, public and/or healthcare consumer involvement: None.