Objectives: The aim of this project was to evaluate if the CADTH narrow economic search filter (CADTH narrow) could produce a similar sensitivity to the NHS EED search filter (NHS EED) while achieving a lower number needed to read (NNR) when searching for economic evaluations in the Ovid MEDLINE database.
Methods: We identified references of economic evaluations from a sample of 13 systematic reviews conducted by the Swedish Agency for Health Technology Assessment and Assessment of Social Services (SBU). The references that met the inclusion criteria within each review were used to generate the gold standard set for Ovid MEDLINE (n=126). We calculated the sensitivity of the CADTH narrow and NHS EED filters, respectively, against the gold standard set. We also created a subset of the gold standard which only consisted of references that were included in the final analyses of the systematic reviews after appraisals of quality and transferability to the Swedish setting (n=46). NNR was then calculated in 4 out of the 13 systematic reviews using existing search strategies in combination with the evaluated filters.
Results: The overall sensitivity for the entire gold standard was 79% for CADTH narrow as opposed to 98% for NHS EED. When the gold standard was limited to the subset of articles included in the systematic reviews after appraisals of quality and transferability, the sensitivity for CADTH narrow increased to 91% while remaining at 98% for NHS EED. CADTH narrow had a lower NNR than NHS EED in all 4 systematic reviews examined with an average of 16 as opposed to 73 for NHS EED.
Conclusions: The sensitivity of CADTH narrow differs depending on the choice of reference gold standard. Within this project, CADTH narrow generated a sensitivity that was almost on par with NHS EED when the gold standard set was limited to economic evaluations appraised as having sufficient methodological quality and transferability to the Swedish setting. NNR was substantially lower with CADTH narrow than with NHS EED and the reviewer screening burden was reduced by a factor of 5.
Patient, public and/or healthcare consumer involvement: No.