Introduction to a new risk of bias tool for network meta-analysis (RoB NMA tool)
2Institute of Health Policy Management and Evaluation, University of Toronto, Canada
3Ottawa Hospital Research Institute, canada
4MRC Clinical Trials Unit at UCL, United Kingdom
5Population Health Sciences, Bristol Medical School, University of Bristol, United Kingdom
6Cochrane Hypertension Review Group, University of British Columbia, Canada
7Centre for Reviews and Dissemination, University of York, United Kingdom
8Bristol TAG, Population Health Sciences, Bristol Medical School, University of Bristol, Canada
9Dalla Lana School of Public Health & Institute of Health Policy, Management, and Evaluation, University of Toronto, Canada
Background: Network meta-analyses (NMAs) provide comparative evidence about the effects of all healthcare interventions for a given condition. However, NMA results can be undermined by flaws in design, conduct, analyses and reporting. Several tools are available for the critical appraisal of an NMA. However, none aim specifically to assess the risk of bias (RoB) in an NMA.
Objectives: To describe a new RoB NMA tool to assess the risk of bias in an NMA, developed using rigorous methodology.
Methods: As an international steering committee, we developed a protocol and made conceptual decisions about the tool’s structure. We developed the tool using a multistep approach: definition of the scope of the tool; conduct a methodological review to compile a preliminary list of concepts related to bias in NMAs [1]; web-based Delphi survey to collate expert opinion regarding the quality of NMAs [2]; an online survey to solicit decision-makers’ needs [2]; and refinement through piloting.
Results: The RoB NMA tool includes three domains: bias arising from the network characteristics/geometry, bias arising from potential effect modifiers, and bias arising from statistical synthesis. The three domains comprise a total of 16 statements which can identify potential limitations of the NMA, leading to a judgement about the risk of bias within each domain (low, high or unclear risk of bias). The tool should be used to identify potential limitations in the way an NMA was conducted that may lead to bias in its results or conclusions. The RoB NMA tool should be conducted as a supplement to the assessment of the risk of bias in the underlying evidence review, for example, using the ROBIS tool. Piloting of the RoB NMA tool is ongoing.
Conclusions: The development of the RoB NMA tool was evidence based and included piloting and refinement. We believe that the RoB NMA tool will allow objective rating of the RoB in an NMA.
Patient, public and/or healthcare consumer involvement: None.