Background: The removal of duplicate references from extensive systematic searches in different literature sources is a time-consuming and laborious process for authors of evidence syntheses. Different deduplication approaches are practiced by author teams, e.g., manual, semi-automated or fully automated using specialized software. These approaches vary in time-to-be-invested, completeness and accuracy of identified duplicates. Commonly used tools for a multi-step detection of duplicates are reference management programmes (e.g., EndNote, Citavi) and built-in deduplication features of systematic review software (e.g., Covidence, Rayyan). However, deduplication processes are not made transparent in all tools and are sometimes error-prone. Recently, developed deduplication tools such as Deduklick (fully automated) and the Systematic Review Accelerator (SRA) Deduplicator (automated, but needing manual control) use artificial intelligence-based algorithms, including natural language normalisation and sets-of-rules created by information specialists.
Objectives: We aimed to evaluate and compare the performance, transparency, time efficiency and manageability of five frequently used manual, semi-automated and fully automated deduplication tools: EndNote, Covidence, Rayyan, Deduklick and SRA Deduplicator.
Methods: We compiled 10 heterogenous datasets from different bibliographic databases and trials registers, covering various healthcare topics and varying in size between 100 to 1200 records. We tested the five deduplication tools on each dataset and compared their ability to correctly identify duplicates, their transparency in the process, time efficiency and manageability.
Results: The preliminary findings of this ongoing project showed that Deduklick and SRA Deduplicator achieved the best performance in detecting duplicates among all tools tested and compared to manual deduplication in EndNote by an experienced information specialist. The two tools delivered deduplication results in a transparent, downloadable report. Deduklick proved superior in time efficiency and manageability. Comparing Rayyan and Covidence, the latter achieved higher accuracy in identifying duplicates than Rayyan. Both tools provide reports of duplicates, but these were not downloadable in Covidence. In Rayyan a control of duplicate results is strongly recommended. At the conference, we will present further findings, advantages and disadvantages of the five deduplication tools.
Conclusions: Our project’s findings will help inform a more efficient conduct of evidence syntheses.
Patient, public and/or healthcare consumer involvement: Because we present a methodological evaluation, patients or healthcare consumers were not involved.