“Info without side effects” – Development and application of a checklist for lay users for evaluating online health information

Date & Time
Tuesday, September 5, 2023, 5:15 PM - 5:25 PM
Location Name
Session Type
Oral presentation
Understanding and using evidence
Oral session
Teaching the public to understand and use evidence
Griebler U1, Kien C1, Klerings I1, Lutz B2, Krczal E3, Ledinger D1, Mair I1, Emprechtinger R4, Keser Aschenberger F5, Kerschner B1
1Department for Evidence-based Medicine and Evaluation, University of Krems, Krems a.d. Donau, Austria
2Department for Knowledge and Communication Management, University of Krems, Krems a.d. Donau, Austria
3Department for Economy and Health, University of Krems, Krems a.d. Donau, Austria
4Department for Evidence-based Medicine and Evaluation and Faculty of Health and Medicine, University of Krems, Krems a.d. Donau, Austria
5Department for Continuing Education Research and Educational Technologies, University of Krems, Krems a.d. Donau, Austria

Objectives: The internet is an important resource for health information (HI). However, many users have difficulties in deciding whether the information they found online is trustworthy and objective. We aimed to develop and validate a checklist for helping laypersons with evaluating the trustworthiness of online HI.
Methods: We searched for existing checklists and conceptual research for HI assessment and extracted all items/indicators. After we categorized, deduplicated, and translated items into German, six HI experts identified the most pertinent items in a modified Delphi process (“item-shortlist”). The item-shortlist was tested by laypersons for comprehensibility and applicability (cognitive tests, n=19) and usability (assessment of 15 selected HI, n=20). The research team applied the item-shortlist to 100 HI, and two researchers independently assessed the objective certainty of evidence for each HI. These assessments were used to determine the predictive validity of the items.
Results: We extracted a total of 1,740 items from 73 documents. After all reduction and modification steps, we tested a shortlist of 23 items. The claimed strength of evidence of the HI relative to the objective certainty of evidence for the specific health topic was used as outcome measure to test the predictive validity regarding the correctness of the HI of each item as well as a set of items. To create the final checklist, we considered the laypersons’ qualitative results, the effect estimates of each item, the inter-reliability measures among laypersons, among experts, and between laypersons and experts. The final checklist comprises the following 6 items: absence of advertisement, balanced presentation of information, citation of sources, limited use of technical terms, presence of a publication date, and a medical disclaimer. It will be published on the project webpage https://www.infos-ohne-nebenwirkung.at/.
Conclusions: The creation of a checklist for the assessment of online HI that can be understood and used by laypersons is a great challenge. The final checklist has to be tested with further HI test sets to strengthen its validity. Patient, public, and/or healthcare consumer involvement: We involved potential users of the checklist in two stages of the development process of the checklist to ensure the comprehensibility and applicability of the checklist.