Assessing the quality and completeness of health system guidelines and recommendations for pandemics using the AGREE-HS tool

Date & Time
Wednesday, September 6, 2023, 12:30 PM - 2:00 PM
Location Name
Session Type
Understanding and using evidence
Ursić L1, Žuljević MF2, Roje R3, Vuković M1, Mijatović A1, Matas J1, Duplančić D2, Sapunar D4, Marušić A1
1Department of Research in Biomedicine and Health, University of Split School of Medicine, Croatia
2Department of Medical Humanities, University of Split School of Medicine, Split, Croatia, Croatia
3Scientific Department, University Hospital of Split, Split, Croatia, Croatia
4Department of Histology and Embryology, University of Split School of Medicine, Split, Croatia, Croatia

Background: The World Health Organization (WHO) declared two global disease outbreaks in the 21st century as pandemics: H1N1 in 2009 and COVID-19 in 2019. Although numerous studies assessed the effectiveness of healthcare-system level strategies and interventions for pandemics, there is a lack of studies on the quality and completeness of guidelines and recommendations for implementing such strategies and interventions, as most have focused on clinical treatments rather than health-system strategies.
Objectives: We aim to assess the completeness and quality of healthcare system guidelines and recommendations for the H1N1 and COVID-19 pandemics issued by the WHO, European Centres for Disease Control and Prevention (ECDC), and U.S. Centers for Disease Control and Prevention (CDC).
Methods: We retrieved guidelines and recommendations from the WHO, ECDC, and CDC repositories, their website sections dedicated to the H1N1 pandemic, and the Morbidity and Mortality Weekly Report and WHO Guidelines Approved by the Guidelines Review Committee. Following deduplication, we conducted a title/abstract screening, followed by a full-text screening (Figure 1). The inclusion/exclusion criteria were informed by the AGREE-HS tool (Table 1). The main study outcome will be each document’s overall score (range = 20-140) calculated by summing scores from four assessors on the AGREE-HS tool. Secondary outcomes will be the final scores of each item (calculated by summing up each appraiser’s scores for each item) and word clusters (topics) extracted using an automated qualitative analysis (topic modeling) from the assessors’ comments on the items. Preliminary results: After deduplication, title/abstract screening, and full-text screening, we included 197 documents in the analysis (Table 2). Fewer health systems guidance documents were issued during the H1N1 pandemic (n = 34) than the COVID-19 pandemic (n = 163). We are currently piloting the AGREE-HS assessment and will present the results at the Colloquium.
Conclusions: We expect that our study will improve our understanding of health system guidance development processes in three globally relevant organizations and give insight into possible improvements. Relevance for patients: The study will give patients a better insight into health system guidance development processes, enhancing their understanding of how evidence is implemented in decision-making, which affects them as end users.