Background: There is an increased number of systematic reviews of research priority setting exercises published. These reviews can be critical in informing future priority setting exercises and reduce unnecessary duplication of studies. However, these reviews have a range of qualities, and there is a need for an appropriate tool to appraise these studies.
Objectives: To adapt the AMSTAR tool for appraisal of systematic reviews of research priority setting studies and pilot them for a group of systematic reviews identified as part of a previous publication (Tan 2022).
Methods: From previous studies on evaluation of priority setting exercises (Nasser 2021), we identified key elements that might affect the AMSTAR tool and developed either adapted questions or adapted guidance to answer the questions. Afterwards, the tool was used to pilot the reviews of Tan 2022 review.
Results: Although the original 16 items were maintained, the questions were adapted so that the research priority setting method, as opposed to the intervention, was the focus. Thirty-one studies were identified in Tan 2022 review that were evaluated using the AMSTAR tool. Only two reviews were assessed as low quality, and the remaining 29 reviews were assessed to be of critically low quality overall.
Conclusions: Our adapted AMSTAR tool can be used to appraise systematic reviews of research priority setting exercises and differentiate their quality. Patient, public, and/or healthcare consumer involvement: There was no direct involvement; however, the work builds on methodological research focusing on patient involvement in research amongst other research.