Background: PROBAST (Prediction model Risk Of Bias Assessment Tool) has been launched in January 2019. Since then, there has been much progress and literature on the methodology for prediction modelling in general and on the use of artificial intelligence (AI) and machine learning (ML) techniques in this field in particular. Hence, it is timely to develop PROBAST+AI that applies to studies on developing and evaluating (validating) multivariable diagnostic and prognostic prediction models using any data analytical (statistical) AI or ML technique.
Objectives: To develop and test PROBAST+AI, a quality and risk of bias assessment tool that applies to studies on developing and evaluating (validating) multivariable diagnostic and prognostic prediction models using any AI/ML data analytical technique.
Methods: Using a Delphi process (at least 3 survey rounds) among a diverse and large (>200) group of key stakeholders and experts on prediction model and AI/ML, we identified the relevant items for PROBAST+AI. Participants gave their opinion on a large series of quality, risk of bias, and applicability domains and signalling questions, using a 5-Point Likert scale. Participants were also asked to add new suggestions using free-text boxes.
Results: Currently, two Delphi rounds have been conducted, and a third is planned in the spring of 2023, followed, if deemed necessary, by a final consensus expert meeting. The results and PROBAST+AI will be presented at the meeting. Conclusion: PROBAST+AI will provide key stakeholders in diagnostic and prognostic prediction models (including primary study authors, systematic reviewers, guideline developers, healthcare providers, prediction model developers, and patients), guidance on the relevant methodological aspects of diagnostic, and prognostic prediction model studies using any AI/ML data analytical technique. Patient or healthcare consumer involvement: Part of the PROBAST+AI Delphi and working group. Our project has methodological implications for systematic reviews.