Objectives: The internet is an important resource for health information (HI). However, many users have difficulties in deciding whether the information they found online is trustworthy and objective. We aimed to develop and validate a checklist for helping laypersons with evaluating the trustworthiness of online HI.
Methods: We searched for existing checklists and conceptual research for HI assessment and extracted all items/indicators. After we categorized, deduplicated, and translated items into German, six HI experts identified the most pertinent items in a modified Delphi process (“item-shortlist”). The item-shortlist was tested by laypersons for comprehensibility and applicability (cognitive tests, n=19) and usability (assessment of 15 selected HI, n=20). The research team applied the item-shortlist to 100 HI, and two researchers independently assessed the objective certainty of evidence for each HI. These assessments were used to determine the predictive validity of the items.
Results: We extracted a total of 1,740 items from 73 documents. After all reduction and modification steps, we tested a shortlist of 23 items. The claimed strength of evidence of the HI relative to the objective certainty of evidence for the specific health topic was used as outcome measure to test the predictive validity regarding the correctness of the HI of each item as well as a set of items. To create the final checklist, we considered the laypersons’ qualitative results, the effect estimates of each item, the inter-reliability measures among laypersons, among experts, and between laypersons and experts. The final checklist comprises the following 6 items: absence of advertisement, balanced presentation of information, citation of sources, limited use of technical terms, presence of a publication date, and a medical disclaimer. It will be published on the project webpage https://www.infos-ohne-nebenwirkung.at/.
Conclusions: The creation of a checklist for the assessment of online HI that can be understood and used by laypersons is a great challenge. The final checklist has to be tested with further HI test sets to strengthen its validity. Patient, public, and/or healthcare consumer involvement: We involved potential users of the checklist in two stages of the development process of the checklist to ensure the comprehensibility and applicability of the checklist.