Development of a health-system guidance implementation tool using a modified Delphi method and a formative evaluation approach

Date & Time
Tuesday, September 5, 2023, 11:05 AM - 11:25 AM
Location Name
St James
Session Type
Oral presentation
Oral session
Evidence synthesis and clinical guidelines: tools and methods
Wang Q1, Chen Y2, Yang K3, Schünemann HJ4, Wilson MG1, Lavis JN1
1Department of Health Research Methods, Evidence and Impact, Faculty of Health Sciences, and McMaster Health Forum, McMaster University, Canada
2WHO Collaborating Centre for Guideline Implementation and Knowledge Translation, Lanzhou University, China
3Evidence-Based Medicine Center, and Evidence Based Social Science Research Center, Lanzhou University, China
4Department of Health Research Methods, Evidence and Impact, Michael G DeGroote Cochrane Canada and GRADE Centres, McMaster University, Canada

Background: Compared with clinical practice guidelines, the implementation context for health-system guidance (HSG) is more multi-layered and complicated because of the complexity of health systems, the diversity of evidence, and the highly context-sensitive and multifactorial policymaking process. Current guideline implementation frameworks and tools might not be suitable for relatively complex HSG implementation. Therefore, a comprehensive and well-organized theoretical framework to support the HSG implementation was developed.
Objectives: Building on a theoretical framework we developed, this project aims to develop an operationalized tool for supporting global HSG implementation at national or subnational levels.
Methods: First, a modified Delphi approach with two steps was applied, including the following: (1) generating items based on the theoretical framework and (2) using a modified Delphi method to select and refine items to be incorporated in the tool. Second, a formative evaluation approach with four steps was applied, including the following: (1) operationalizing the HSG implementation tool based on the Delphi-derived components (i.e., domains and individual items in each domain); (2) theoretically ‘testing’ the application of the tool with several HSG examples; (3) examining the face validity and usability through semi-structured interviews; and (4) finalizing the HSG implementation tool based on the above findings.
Results: The main output of this study is an operationalized HSG implementation tool that can support HSG implementation at the national and/or subnational level. Also, we will learn about whether, how, and why the HSG implementers will use this tool and what could be done to make this tool more usable. The results will be presented at the conference.
Conclusions: The operationalized tool will help HSG users identify potential factors that facilitate or hinder the HSG implementation process, and strategies can leverage facilitators and address barriers during their preparation for HSG implementation; on the other hand, the tool will help HSG developers integrate the implementation considerations during the development process of HSG. Patient, public, and/or healthcare consumer involvement: As the target population of HSG recommendations, the healthcare consumer representative will be invited to participate in the modified Delphi and formative evaluation processes to provide the related insights that contribute to the tool development.