Programme

The theme of this year’s Colloquium is ‘Forward together for trusted evidence’; exploring the challenges for the future around trustworthiness of healthcare data and information whilst also celebrating 30 years of producing trusted evidence.

We’re delighted to bring you a varied programme of workshops, oral presentations, posters, Special Sessions and meetings. Content falls into four programme streams: 1) producing trusted evidence; 2) advocating for trusted evidence; 3) informing health and care decisions; 4) co-production and working together.

On this page, you can filter by session type and/or by subject category. You can also search by key phrases (to clear a search, click on the x by the right-hand side of the search bar).

Please note, the content below is subject to change.

 

Plenaries: These sessions bring together inspiring speakers offering different perspectives on key themes, through linked presentations and a panel discussion.

Special sessions: These sessions have been chosen for their innovative content and relevance to our theme “Forward together for trusted evidence”. Some are linked to plenaries, giving you an opportunity to explore the topics in greater depth.

Oral sessions: Sessions that group multiple oral presentations with a similar topic.

Skills lab: Sessions to help you develop skills and support your personal and professional development.

 

9:00 AM - 5:30 PMEditorial Board Meeting (restricted)Meeting 
9:00 AM - 5:30 PMCouncil Meeting (restricted)Meeting 
12:30 PM - 5:30 PMGeographical Group Directors' MeetingMeeting

Please find the agenda for the meeting here.

3:00 PM - 4:00 PMConsumer MeetingMeeting

Are you a healthcare consumer (patient, carer, member of the public)? Come and join members of the Consumer Network Executive before the colloquium. Meet others like yourself, find out what support there is for you, discover interesting and relevant sessions, and ask any questions that you have.

7:30 AM - 8:30 AMCochrane Evidence Synthesis and Methods Editorial Board meeting - By Invitation OnlyMeeting

A meeting of the Editorial Board of Cochrane Evidence Synthesis and Methods to discuss the journal and future developments.

7:30 AM - 8:45 AMCochrane Scholars Workshop for American Academy of Otolaryngology HNS - By Invitation OnlyMeeting

This session, moderated by Martin Burton and Richard Rosenfeld, will orient Cochrane Scholars and staff from the American Academy of Otolaryngology - Head and Neck Surgery to principles of systematic review and meta-analysis, and will prepare attendees to get the most out of the Colloquium.

7:45 AM - 8:45 AMCochrane Thematic Groups - By Invitation OnlyMeeting

This meeting will provide an opportunity for Thematic Groups leaders and members to discuss shared interests, opportunities for cross working and consider where collaboration around common interests could benefit all Thematic Groups.

8:00 AM - 8:45 AMThomas Chalmers Award Committee - By Invitation OnlyMeeting

Committee Members to meet and discuss onsite process.

9:00 AM - 10:30 AMGlobal health, equity and trustPlenary

An important goal of global health is to advance health equity for all people worldwide. Trust in global health research, practice and policy is crucial for achieving this goal. In this plenary we start by examining the historical roots of mistrust in global health, citing examples of research abuses and barriers to the delivery of care in marginalised communities. This is followed by a discussion of challenges for evidence generation, synthesis and use encountered during the COVID-19 pandemic. Next, we shift our focus to the impact of the profit motive on trust and equity in health and offer guidance on the conduct of systematic reviews focussing on the commercial determinants of health. We conclude by providing recommendations for improving research integrity and building trust in global health research.  

Keynotes:

  • How COVID broke the evidence pipeline (Helen Pearson)
  • Building trust in (global) health research (Gowri Gopalakrishna)
  • Why trust is an important issue in global health (Jimmy Volmink)
  • Commercial determinants of health: influence of private sector activities on evidence and equity (Mark Petticrew)

The session was chaired by Cochrane's Editor-in-Chief, Karla Soares-Weiser and Eva Madrid from Cochrane Chile.

 

10:30 AM - 11:00 AMBreakCoffee break 
11:00 AM - 12:30 PMResearch integrity, transparency and fraudOral session
11:00 Opening remarks and introduction
11:05 Assessment of trustworthiness has a significant impact on conclusions of Cochrane reviews
11:15 Prospective trial registration in the Research Integrity Assessment (RIA) of randomized controlled trials (RCTs)
11:25 The implementation of policy to manage potentially problematic studies in Cochrane review updates – a meta-epidemiological study
11:35 Breaking Ground or Breaking Bad? Examining the Fate of Preprints in Prevention Research with a Mixed Methods Study
11:45 Comparison of effect estimates between preprints and peer-reviewed publications: a meta-epidemiological study of COVID-19 trials
11:55 How should we handle predatory journals in evidence synthesis?
12:05 Impact of including conference abstracts in systematic reviews: analysis from a living systematic review and network meta-analysis of COVID-19
12:25 Closing remarks
11:00 AM - 12:30 PMAssessing evidence certaintyOral session
11:00 Opening remarks and introduction
11:05 Can we use GRADE to create new diagnostic criteria for a condition? An application of GRADE principles for establishing diagnostic criteria for a disease
11:25 Impact of Grading of Recommendations, Assessment, Development and Evaluation (GRADE) on conclusions of dentistry systematic reviews
11:35 Challenges and potential solutions for reporting findings from multicomponent meta-regression models in GRADE summary of findings tables
11:45 India covid guidelines- a living synthesis and guideline process
11:55 Using GRADE-CERQual to assess confidence in findings from qualitative evidence syntheses: how well are review authors applying the approach?
12:05 Applying GRADE-CERQual to interpretive review findings: Reflections from a Cochrane meta-ethnography on childhood vaccination acceptance
12:15 Closing remarks
11:00 AM - 12:30 PMGlobal health, equity, diversity and inclusionOral session
11:00 Opening remarks and introduction
11:05 Operationalising decolonisation in systematic reviews: can current tools help to mitigate for bias?
11:25 Missing and masked: equity in a systematic review of remote interventions for substance misuse.
11:35 How well do we consider equity in efficiency analysis studies of vaccines? A systematic review of equity-informative economic evaluations of vaccines
11:45 Methods used in systematic reviews to conceptualise dimensions of health equity impacts of public health interventions: umbrella review
11:55 Operationalizing the GRADE-Equity criterion to inform guideline recommendations: Application to a Medical Cannabis guideline
12:05 Improving equity, diversity, and inclusion in Journals
12:15 Incorporating Equity, Diversity, and Inclusion into Cochrane Systematic Reviews with AI
12:25 Closing remarks
11:00 AM - 12:30 PMNetwork Meta-analysisOral session
11:00 Opening remarks and introduction
11:05 Unravelling active ingredients of task-shifting interventions in low-resource settings for common mental disorders: developing a taxonomy of intervention components and ranking their efficacy.
11:25 An assessment of the design-by-treatment interaction model for network meta-analysis inconsistency
11:35 Using arm-based network meta-analysis for binary outcomes for generalizability of findings across baseline risk values
11:45 A novel modeling approach for producing treatment hierarchies in network meta-analysis
11:55 Inconsistency identification in Network Meta-Analysis via Stochastic Search Variable Selection
12:05 Evaluation and development of a novel interactive Summary of Findings table for network meta-analysis. A qualitative user-testing study with clinicians
12:15 Closing remarks
11:00 AM - 12:30 PMMapping evidenceOral session
11:00 Opening remarks and introduction
11:05 Evidence gap maps: a visual tool for promoting evidence and monitoring gaps in research
11:25 Mapping the Maps: Methods and Uses of Evidence and Gap Maps.
11:35 Mapping Reviews, Scoping Reviews and Evidence and Gap Maps (EGMs) – Same but Different. The ‘Big Picture’ Review Family
11:45 A digital map of systematic reviews on non-pharmacological interventions to inform policy making in infectious disease control
11:55 Novel methods used when conducting an evidence gap map surrounding interventions for treating obstetric fistula
12:05 Working with policy makers to maximise the utility of EGMs: experiences of Exeter Policy and Research Programme Evidence Review Facility
12:15 The concept of “evidence relevant to” in the rehabilitation field: post COVID-19 condition mapping for the World Health Organization Guidance
12:25 Closing remarks
11:00 AM - 12:30 PMBeyond the intervention question: three new Cochrane Handbooks as the pillars of methodological standards for producing high-quality systematic reviews with different types of evidenceSpecial Session

Cochrane's vision is a world of better health for all people where decisions about health and care are informed by high-quality evidence. Since its inception, Cochrane has pioneered and developed systematic review methodology with highly structured, transparent and reproducible standards. Historically, Cochrane has concentrated on systematic reviews of interventions, with the Cochrane Handbook for Systematic Reviews of Interventions being the leading guide for preparing and maintaining systematic reviews on the effects of healthcare interventions within Cochrane and globally. However, the health landscape has changed and given rise to complex challenges and a need for trusted health evidence to guide decision-making in areas beyond intervention research.

To continue meeting the needs of the users of our research, it is essential for Cochrane to increase its capacity to address different research questions. To provide guidance on the methodological standards for producing high-quality systematic reviews with other types of evidence, Cochrane has expanded its Handbook collection to include the Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy (complete and available to buy), the Cochrane-Campbell Handbook for Qualitative Evidence Synthesis (complete and available in draft), and the Cochrane Handbook for Systematic Reviews of Prognosis (in development with some chapters available in draft). These three new Cochrane Handbooks are led by the Cochrane Screening and Diagnostic Tests, Qualitative and Implementation, and Prognosis Methods Groups, respectively, and draw on the expertise of hundreds of contributing methodologists, researchers and editors worldwide.

In this session, the Editors for each of the three new Cochrane Handbooks will introduce their Handbook and discuss methodological challenges that arise from these types of systematic reviews. In addition, they will highlight new developments in research methodologies and what they hope to achieve with the addition of these Handbooks to the collection of Cochrane guides that are available for authors, editors and the wider community.

Target audience: Authors, editors, methodologists and anyone interested in different types of systematic reviews.

Format: Lecture, panel discussion

11:00 AM - 12:30 PMForward together: new ways to participate in CochraneWorkshop - discussion

Background: Cochrane has developed many new ways to get involved, which go beyond authoring reviews. These include Cochrane Crowd, Cochrane Engage, translations, and consumer engagement. These initiatives are part of Cochrane’s Membership Project that seeks to broaden our community by giving a wider range of people the opportunity to be part of Cochrane. In addition, developments and changes within Cochrane might open up lots of other new opportunities for people to participate. We seek to have an inclusive and diverse community, and so in this workshop we will discuss the impact and limitations of these existing options and consider additional ways in which the community can be involved.
Objectives: The objective of this session is to learn about opportunities to get involved in Cochrane’s ecosystem and identify new ideas for improving the current offering.
Description: The session will be divided into two parts with greater emphasis given to the future-focused second part. The first part will look back at what we have achieved so far through brief presentations on topics, such as volunteer best practice or case studies regarding Cochrane Crowd, Cochrane Engage, translations, and consumer engagement. This will then be followed by discussion. The second part of the session will be future focused, asking participants how they think Cochrane can build on this work to further develop our diverse and global community through new ideas or through addressing the limitations of the existing options. This will take the form of small group discussions. The session will close with an open discussion about priorities for the future. Relevance and importance to patients: Patients are benefiting from these initiatives, for example, by gaining access to plain language summaries translated into their languages and improved evidence production. Patients are able to share their lived experience as healthcare consumers within Cochrane to improve evidence.

11:00 AM - 12:30 PMResearch priority setting that inform or use systematic reviewsWorkshop - discussion

Background: The Cochrane priority setting methods group has been working on developing guidelines and methods in this area and how it relates to the Cochrane Collaboration. There are a few areas that would be relevant to this conversation. These include the following: - conducting research priority setting exercises that inform prioritisation of topics for conducting or updating systematic reviews; - using systematic reviews on primary research on the topic as a source of information for stakeholders who participate in a research priority setting process; and - using systematic reviews of other research priority setting exercises to either replace a research priority setting exercise or inform one.
Objectives: Our workshop will provide tools and guidance on how to use systematic reviews in the research priority setting process in any of the categories defined above. This includes how the methods of the research priority setting need to be adapted or the approach to conduct systematic reviews needs to be changed or adapted.
Description: The workshop will start with a few presentations, followed by small group activities, and finally result in a final discussion and summing-up. Presentation: There will be two presentations: (a) the methods of setting priorities for research—this includes guidance on how to make decisions and what methodology to choose based on the focus of the exercise and the stakeholders involved. This will be accompanied by a diagram on how to engage stakeholders in (b) the methods to conduct systematic reviews of research priority setting exercises and appraising them. The latter uses an adapted version of AMSTAR to conduct the evaluation. Exercise: The groups will be given different examples of research priority-setting exercises and systematic reviews of research priority-setting exercises along with appraisal forms to evaluate. Discussion and summing-up: The workshop will finish with reports from individual groups along with discussions on how these methods can be used in their organisations.

11:00 AM - 12:30 PMDo's and Don'ts in the Rapid Review search: Find information faster without losing confidence in the resultsWorkshop - discussion

Background: Evidence syntheses are key tools to support reliable, unbiased, and reproducible healthcare decisions. However, identifying, appraising, and synthesizing new evidence is often resource intensive. The Cochrane Rapid Review Methods Group (RRMG) investigates ways to meet growing demands within resource constraints by using rapid review (RR) evidence synthesis methods. RR information retrieval methods aim to assist in accelerating the review process while still being systematic, transparent, and reproducible. Search processes may be abbreviated by 1) reducing time spent on conducting searches and 2) reducing the size of the search result. There are many ways to make searches more efficient, ranging from searching fewer sources to the application of limits, filters/hedges, and restrictions (e.g., publication dates, language, and precision-focused search strategies). But how do we decide which approach is appropriate for a particular topic or review goal?
Objectives: The aim of the workshop is to present and discuss potential abbreviated approaches in RR information retrieval and their appropriate application to different research questions or goals (e.g., focused clinical questions, broader public health topics, and horizon scanning).
Description: The workshop comprises four sections, combining short expert presentations with large and small group discussions. 1. Short presentation of the topic, definition of terms, and recently published guidance focusing on literature searching for RRs from the RRMG.[1] 2. Large group discussion: Collection of “shortcut strategies” and their pros/cons, as used by the participants. 3. Brief case studies from the presenters of how their organizations use RR search methods depending on the research question or review goals. 4. Small group discussion based on a case scenario: Participants will discuss possible search approaches for a specific research question, weighing pros and cons of frequently used shortcuts. Workshop prerequisites: Participants should be familiar with the fundamentals of systematic searching. Familiarity with RR methodology is welcome but not necessary. Patients or healthcare consumers are not involved in this workshop. However, the application of appropriate RR search methods is important for providing robust evidence rapidly in healthcare settings. [1] Klerings I, Robalino S, et al. Rapid Reviews Methods Series: Guidance on Literature search. BMJ Evidence-Based Medicine. 2023(in-press).

11:00 AM - 12:30 PMBetter data extraction with Covidence and RevMan WebWorkshop - training

Background: Cochrane review authors can now import completed data extractions and quality assessments (Risk of Bias) from Covidence into RevMan Web. When completed successfully, all the data collected in Covidence are added to the RevMan Web review, and authors are ready to proceed with setting up the analyses. This new workflow saves time and reduces the risk of error. Careful preparation and formatting of the data will ensure successful import into RevMan Web. This workshop will explain the requirements of this workflow in detail; demonstrate the process; and help reviewers to troubleshoot common problems.
Objectives: At the end of this session, workshop attendees will be able to describe the steps of the workflow in detail; map study arms from Covidence to interventions in RevMan Web; and identify the fields in Covidence and RevMan Web that must match exactly to ensure successful transfer of data.
Description: The new workflow requires use of the Extraction 1 tool in Covidence and the study-centric data feature in RevMan Web. Review authors download their data from Covidence in csv files which are then imported to RevMan Web. This workshop will provide slides and a demonstration of the new workflow. Example export files will be shown and modified to illustrate the process. Workshop attendees will also have the opportunity to ask questions, share their own experiences and ideas and give feedback. Healthcare consumers have not been involved in the planning of this workshop. Note for organisers: This session can be repeated as needed to enable as many people as possible to attend over the course of the 3 days.

11:00 AM - 12:30 PMConsumer Involvement 101: producing Cochrane evidence with consumersWorkshop - training

Background: This workshop is for researchers and consumers who are interested in involving people in producing systematic reviews, and it aims to address the questions that people have, share the different possible methods, and explore the resources and support that are available. By the end of the session, participants will understand where to start their involvement journeys. Cochrane is committed to the production of Cochrane Evidence with consumers in its new Consumer Engagement and Involvement Framework and by signing the Putting People First pledge to involve patients in health research. Currently, approximately 1 in 10 reviews have some form of consumer involvement in the authoring of reviews. Whilst most researchers and consumers support the principles of consumer involvement, there is still much uncertainty about methods for involving people in reviews.
Objectives: Introduce and discuss the concept and principles of coproduction; share examples and experiences of involvement and engagement from both researchers’ and consumers’ perspectives; introduce the ACTIVE framework for stakeholder engagement and involvement; identify barriers and enablers to engagement and involvement; describe the range of resources available to support coproduction; consider the implications of the increasing priority given to involvement and engagement; and introduce the principle of coproduction and diversity and inclusion in our work.
Description: The workshop will be an interactive and practical session with opportunities for group work and discussion. The first part of the session will be a discussion about the key principles of involvement, engagement and coproduction. Presenters will be invited to share a number of short case studies of examples of involvement and coproduction from both researchers’ and consumers’ perspectives. These will be followed by a Q&A session. Participants will be introduced to the ACTIVE framework including concepts of power sharing, involvement in the lifecycle of review production, and the extent and nature of involvement. In small groups, participants will explore the practicalities of involvement and coproduction using the framework and case studies as the basis for discussion. The session will conclude with signposting to resources to support involvement, a discussion about the future of coproduction, and identifying barriers and increasing diversity and inclusion in this work.

11:00 AM - 12:30 PMPrepare for success! How to lead a review team and complete your review (or update) on time.Workshop - training

Background: An essential feature of performing Cochrane reviews is working well in a collaborative and interdisciplinary author team. Beyond the methodological aspects, logistical and management issues in the systematic review process can be rather challenging. Identifying and managing the different tasks required to complete a systematic review often falls on the lead author. Careful planning and effective communication within the team can reduce delays and ensure timely completion of tasks.
Objectives: To discuss logistical and management challenges while planning and undertaking a systematic review; to identify possible solutions to managing a systematic review team and ensuring tasks are completed timely; and to weigh the pros and cons of these solutions and learn from the experiences of other teams.
Description: This workshop will appeal to anyone interested in conducting a systematic review for the first time, those leading a review team, or those interested in learning about practical issues concerning team management. We will give a short presentation at the beginning of the workshop (15 minutes) to outline the scope of the session, introduce key issues concerning the effective management of a systematic review as well as common reasons for delay, and explain group activities. Attendees will participate in small groups to discuss challenges relating to logistical and management issues and complete a number of small tasks. These tasks involve brainstorming key issues around managing a review team (e.g., communication, responsibilities, setting targets, and keeping on track) and identifying the tasks involved in a review (from protocol to publication), methods for screening/data extraction (e.g., templates), and document/file management. We will try to ensure each group is formed by people with different backgrounds and expertise to bring a variety of views to the discussion (40 minutes). The groups will be asked to report the most critical aspects identified (5 minutes per group). We will stimulate debate on practical issues and provide tips that can assist review teams in undertaking project management from our own experience. Time will be allowed for discussion of the main areas of concern, and a short report on the workshop will be provided to all participants the week after the Colloquium.

11:00 AM - 12:30 PMIntroduction to analysis and meta-analysis of interrupted time series studiesWorkshop - training

Background: Interrupted Time Series (ITS) studies are commonly used to evaluate public health and policy interventions when randomisation is impractical or infeasible; for example, examining the effects of mass media campaigns on the use of methamphetamine among young adults. In an ITS study, measurements on a group of individuals (e.g., community) are taken repeatedly both before and after the intervention. The key benefit of the ITS design is that any secular trend in the period before the intervention can be accounted for when estimating the impact of the intervention. Several effect measures can be used to characterise both short- and long-term effects of the intervention (e.g., immediate level-change and long-term level-change). Meta-analysis of these effect estimates can usefully inform decision-making.
Objectives: In this workshop, we aim to equip review authors with the knowledge and tools to incorporate ITS in their reviews by: i) demonstrating how to digitally extract data from ITS graphs; ii) how to analyse ITS studies and meta-analyse their results; and iii) highlight design features to consider when assessing the risk of bias. This workshop will require access and basic competency in Stata or R and will assume knowledge of meta-analysis.
Description: We will use a combination of presentations and computer practicals. For the analysis of the ITS studies, we will focus on fitting segmented linear regression models and demonstrate how to set up the data for analysis and undertake the analysis. We will discuss the complexities that arise when analysing time series data (e.g., autocorrelation). We will then demonstrate how to meta-analyse the resulting effect estimates. Finally, via example, we will use the ROBINS-I framework to discuss features of ITS designs that may bias effect estimates.

11:00 AM - 12:30 PMImpacts of climate change on health and health systems: Producing evidence syntheses to support decision-makingWorkshop - training

Background The climate emergency is a pressing threat to human health and health systems. Members of the Cochrane community must bring our evidence synthesis expertise to the work of supporting effective decision making to adapt to or mitigate its impacts. Because of the multi-level and systemic challenge of this topic, relevant evidence is complex and heterogeneous. Synthesis of this evidence requires appropriate methods that incorporate interdisciplinary approaches. Objective To introduce participants to the knowledge and skills needed to conduct comprehensive and rigorous evidence syntheses on climate-health topics. Participants will gain an understanding of available relevant evidence synthesis methods and will be introduced to the skills needed to conduct their own syntheses. They will also have the opportunity to network with other researchers and practitioners working in the field. Description Overview of key concepts and terminology - Introduction to climate change and human health linkages - Adaptation and mitigation initiatives related to health and health systems Decisions related to conducting a review: - Framing the PICO question - Search strategies and selecting databases - Incorporating logic models/conceptual frameworks to link health and climate variables - Decisions about scope (lumping versus splitting) and levels of analysis - Addressing equity considerations - Establishing a review advisory board There will be facilitated small-group work sessions addressing specific issues for designing a protocol on a climate-health question. Participants can bring their own topics or work from sample topics provided by the facilitators. The workshop will conclude with a brief discussion of future methods needs for climate-health syntheses.

11:00 AM - 12:30 PMROB-ME: a tool for assessing risk of non-reporting biases in systematic reviews with or without meta-analysisWorkshop - training

Background: Researchers’ decisions about whether, when, how, or where to report studies or results are often influenced by the P value, magnitude, or direction of the study results (‘non-reporting biases’). A consequence is bias in systematic reviews because the available evidence differs systematically from the missing evidence. Existing tools for assessing the risk of non-reporting biases are limited in terms of their scope, guidance for reaching risk of bias judgements, and measurement properties.
Objectives: Introduce ROB-ME, a comprehensive new tool for assessing the risk of non-reporting biases in systematic reviews with or without meta-analysis and provide participants with the opportunity to apply ROB-ME.
Description: The workshop will be split into two parts. 1. Introduction to ROB-ME: We will provide a brief overview of the key components of ROB-ME. These include:
•Specifying which syntheses will be assessed for risk of bias.
•Determining which studies meeting the inclusion criteria for the review have missing results.
•Considering the potential for missing studies across the review.
•Answering signalling questions to inform risk of bias judgements. These questions ask users to consider the extent of missing results in the studies identified, as well as the risk that a synthesis is biased because additional studies or results, beyond those already assessed, are missing systematically. 2. Applying ROB-ME: Participants will apply ROB-ME to an example systematic review within small groups. Each group will assess the example review with regard to a particular component of ROB-ME. In a plenary session, we will discuss the results of each group’s assessment and issues that arose during the assessment process. The workshop will conclude with a facilitated, structured discussion focusing on the implications of using the tool alongside other risk of bias tools (e.g., RoB 2, TACIT) and further development needs for guidance and software.

12:30 PM - 2:00 PMLunch break and meetingsLunch break 
12:30 PM - 2:00 PMPoster session 1Poster session

During this session the following posters will be presented:

12:45 PM - 2:00 PMCochrane Hypertension - By Invitation OnlyMeeting

Cochrane Hypertension Group staff, editors, authors (Vancouver base and Pamplona satellite)

12:45 PM - 2:00 PMCochrane US Network Meeting 1 - By Invitation OnlyMeeting

The first in-person meeting of the US Network - 1 of 2 

Please find the agenda for this meeting below.

Meeting Agenda

1:00 PM - 1:45 PMScandinavian GRADE NetworkMeeting

The aims of this meeting are as follows:
- To meet other GRADE users in the Scandi countries and to get involved in the Scandi GRADE Network
- To receive an update on activities from the Network Coordinators
- To discuss ideas for future activities of the Network.
All GRADE users in the Scandinavian countries are most welcome!

1:00 PM - 1:45 PMCochrane Climate-Health Working GroupMeeting

The Cochrane Climate-Health Working Group was established in 2020. Members are committed to applying the tools of evidence synthesis and knowledge translation to supporting decision-makers in addressing the impacts of climate change on human health and health systems. The meeting will provide an overview of our current funded and unfunded projects. New group members are always welcome.

1:00 PM - 1:45 PMPrognosis Methods Group / Cochrane Handbook for Prognosis Reviews - By Invitation OnlyMeeting 
1:00 PM - 1:45 PMAdverse Effects Methods GroupMeeting

Opportunity to meet 3 of the 4 co-convenors of the group and discuss the role and future directions of the group. We are keen to hear from anyone with an interest in harms.

1:00 PM - 1:45 PMSexual & Reproductive Health - Steering group - By Invitation OnlyMeeting

A meeting for the steering group of this newly established thematic group 

1:00 PM - 1:45 PMCochrane Africa contributors meetingMeeting

 

This will be an open meeting of the Cochrane Africa Network which aims to: 1. raise awareness about Cochrane Africa, 2. share the updated strategy for Cochrane Africa 3. gather ideas from the community for how contributors can collaborate with, and contribute to the network. A brief presentation about Cochrane Africa's new strategy will be followed by an open discussion about engaging more contributors in the Network's work.

1:00 PM - 1:45 PMPRISMA-QES Development meeting - By Invitation OnlyMeeting

A meeting to further develop PRISMA-QES reporting guideline project involving members of Cochrane Qualitative Implementation and Methods Group.

1:00 PM - 2:00 PMICTRP: trial results display and search functionalityMeeting

This meeting is to discuss potential exciting new enhancements to the ICTRP meta register. The changes relate to displaying information regarding results availability for completed trials, and changes to the search interface to improve trial discovery.

1:00 PM - 2:00 PMMeet the CEO and Editor-in-ChiefNetworking session

Cochrane's CEO Catherine Spencer and Editor-in-Chief Karla Soares-Weiser will be available for an informal opportunity to meet and answer your questions.

Meet Catherine and Karla at the Cochrane Community Booth.
 

2:00 PM - 3:30 PMRapid reviews and other rapid evidence products 1Oral session
14:00 Opening remarks and introduction
14:05 How to rapidly review the literature when planning a new clinical trial – a practical guide
14:25 Identifying high priority methodological questions for conducting rapid systematic reviews: Preliminary results from an eDelphi study
14:45 Risk factors for abstracts falsely excluded during single-reviewer screening – a methods study
14:55 Machine-learning assisted screening increases efficiency of systematic review
15:05 Rapid reviews, how much do they comply with Cochrane recommendations in their methodology to provide reliable evidence?
15:15 Evaluation of the Interim Cochrane Rapid Review Methods guidance – a mixed-methods study on the understanding of and adherence to the guidance
15:25 Closing remarks
2:00 PM - 3:30 PMCapacity building in evidence synthesisOral session
14:00 Opening remarks and introduction
14:05 Maximizing collaboration between university students and Cochrane
14:25 Creating a systematic review infrastructure: Implementing Cochrane tools for students, teachers, researchers and clinicians in a university setting
14:45 Building capacity in producing trusted evidence – Evidence Synthesis Ireland and Cochrane Ireland Fellowships
15:05 Evidence Based Research Training School
15:25 Closing remarks
2:00 PM - 3:30 PMStatistical methodsOral session
14:00 Opening remarks and introduction
14:05 Beyond Statistical Significance: Investigating How Systematic Review Authors Communicate Meaningful Differences of Nonsignificant Results
14:15 Is the Freeman‐Tukey double arcsine transformation a reliable approach? for proportion meta-analysis
14:25 Characteristics, reporting, and methods of trials included in time-to-event meta-analyses of systematic reviews: A meta-epidemiological review
14:35 Comparison of statistical methods used to meta-analyse results from interrupted time series studies: an empirical study
14:45 Effect estimates can be accurately calculated with data digitally extracted from interrupted time series graphs
14:55 Is a new approach for rating the quality evidence of effect estimates derived from matched-adjusted indirect comparisons (MAIC) needed?
15:05 Using simple microsimulation to estimate risk difference from a meta-analysis
15:15 Less Ethical Challenges, More Trial Compliance: Progress and Methodological Elements of Zelen's Design
15:25 Closing remarks
2:00 PM - 3:30 PMEngaging stakeholders and building partnershipsOral session
14:00 Opening remarks and introduction
14:05 The Theory of Everything in Health Decision-Making: Step 2
14:25 CEOsys – An ecosystem for COVID-19 evidence in Germany: challenges and lessons learned as a guide for future networks
14:45 The role of collaborative evidence networks in promoting and supporting evidence-based health care (EBHC) globally
14:55 Knowledge mobilisation of rapid reviews to inform health and care policy and practice: lessons from the Wales COVID-19 Evidence Centre
15:05 Providing evidence to the WHO, the experience of Cochrane Rehabilitation regarding the Rehabilitation 2030 initiative and the COVID-19 pandemic
15:15 Boosting global and local partnerships to promote equitable access of COVID-19 guideline recommendations: case study in China
15:25 Closing remarks
2:00 PM - 3:30 PMHow Cochrane responded to the need for timely, unbiased, informative and accurate evidence on new diagnostic tests during the Covid-19 pandemicSpecial Session

In mid-March 2020, leaders in Cochrane put out a call for help to provide evidence to assist decision-making during the emerging pandemic. On 26th March 2020 we formed the Cochrane Covid Diagnostic Test Accuracy Group – an international group of methodologists, test accuracy specialists, statisticians, epidemiologists and clinicians committed to deliver a portfolio of reviews to provide and maintain a reliable evidence base on which test policies could be based.

In the following days, Cochrane published two Cochrane Diagnostic Test Accuracy Protocols (on days 29 and 68), and five Cochrane Diagnostic Test Accuracy systematic reviews (on days 91, 103, 153, 189 and 239) reporting on the accuracy of antibody tests, signs and symptoms, rapid antigen and molecular tests, imaging tests, and routine laboratory tests. Since then, Cochrane has published eight updates of these reviews, and a further four reviews are close to being completed. To date, these five reviews have been cited 3375 times, and used in 22 international guidelines.

Completing these reviews has involved: recruiting over 100 researchers on a voluntary basis from more than 14 countries across six continents; developing new methods to deal with the magnitude of papers, use of pre-prints and new data sources; adapting protocols as knowledge and understanding developed; identifying and engaging with stakeholders (including the World Health Organization (WHO), the Foundation for Innovative New Diagnostics (FIND) and the Cochrane Infectious Diseases group) to ensure the reviews were informative and addressed relevant questions; and developing working arrangements across Cochrane to enable timely publication, including rapid peer review, editorial support and fixing software challenges. Our approach exemplifies Cochrane's principles of collaboration, enthusiasm, avoiding duplication of effort, and minimizing bias whilst striving for clinical relevance, quality, open access and avoiding all conflicts of interest.

This session will introduce the story of the team's approach and work. We will highlight the findings of the reviews, discuss the challenges and solutions we found in working in a new and moving area of health and technology, and discuss how we tried to make sure our findings had impact. We are keen to discuss our experience with the audience and the lessons we have learned.

Target audience: Reviewers, methodologists, policymakers, funders, media, the public: all who are interested in seeing how an international team collaborates effectively and efficiently

Format: Multiple short presentations followed by a panel discussion

2:00 PM - 3:30 PMThe Commercial Determinants of Health and Evidence Synthesis (CODES): methodological guidance for systematic reviewsSpecial Session

The field of the Commercial Determinants of Health (CDOH) is growing rapidly, as evidenced by a new World Health Organization (WHO) programme on CDOH and increased researcher and funder interest. Evidence synthesis will be a crucial tool in the evolution of CDOH. While CDOH reviews can draw on existing methodological guidance, there are areas where the methods will differ, and there is no overarching guidance on the conduct of CDOH-focussed systematic reviews, nor on the specific methodological and conceptual challenges.

CODES provides guidance on CDOH-focussed systematic reviews, from shaping the review question, through to disseminating the review. Existing methods guidance was used to identify key stages and provide a structure for the CODES guidance. The main objective of the session is to (i) present the guidance and initiate a discussion on its potential use and value to reviewers working in the field of CDOH, and (ii) to identify next steps for the development and implementation of CODES guidance.

The session will be primarily focussed on discussion of the main steps of the CODES guidance: on the face validity of the guidance, on its likely acceptability to reviewers and how this can be enhanced. It will start with a short presentation on CDOH, and then a short structured exercise to identify participants’ views of where CDOH-related biases may enter the review process. The findings of this exercise will be considered in the context of the current version of the CODES guidance, to identify key topics for further discussion. The session will end with a short presentation on equity considerations and how these might be incorporated into CODES.

This session will highlight the special methodological and other considerations for CDOH reviews, including equity considerations, and provide pointers to areas for future methodological and guideline development. More generally, CODES will contribute to the reliability and utility of CDOH reviews (in line with the Colloquium theme of “Producing Trusted Evidence") and will help stimulate the production of reviews which are less affected by commercial biases, and more likely to improve the health and wellbeing of patients and the wider public. A member of the public will help design the session.

Target audience: Review authors

Format: Discussion

2:00 PM - 3:30 PMPutting evidence at the centre of everyday lifeWorkshop - discussion

Background: Citizens make many decisions each and every day. Some are personal decisions for themselves or their families, whereas others are decisions related to their community, their country, or the world. In January 2022, the Global Commission on Evidence to Address Societal Challenges released a report.(1) Two recommendations in particular speak directly to the impact of evidence on citizens: 1) help citizens use evidence to inform both personal decisions and decisions related to their community, their country, and the world; and 2) address the spread of false, inaccurate, and misleading information (also called ‘misinformation’), which can lead to bad decisions and harmful consequences. These recommendations are in perfect alignment with Cochrane’s goals of producing trusted evidence, advocating for evidence, and informing healthcare decisions.
Objectives: This interactive workshop aims to support a discussion about ‘putting evidence at the centre of everyday life,’ including:
• the challenges in using evidence in everyday life;
• possible solutions to address these challenges; and
• potential barriers and facilitators to move forward with these solutions. The workshop will also be an opportunity to explore how to promote, contribute to, and/or lead efforts to put evidence at the centre of everyday life (keeping in mind the importance of a global lens and the challenges/opportunities present in different contexts).
Description: The workshop will consist of a short presentation to prompt discussion, followed by small-group and plenary discussions. Workshop participants will have the opportunity to share their insights about how the challenges can be experienced locally and globally. Each small group will be asked to tackle a potential strategy to put evidence at the centre of everyday life, discussing whether and how it is relevant to their context and how it could be operationalized. These may include strategies to: - help citizens judge what others are claiming or more generally find (and receive) reliable information; - make evidence available to citizens when they are making choices; - engage citizens in asking questions and answering them (with new research or with existing evidence); and - make evidence-based choices the default or easy option.

2:00 PM - 3:30 PMAssessing risk of bias in non-randomized studies of interventions: introduction to the ROBINS-I toolWorkshop - training

Background: Non-randomized studies of interventions (NRSI) can provide information about effects of interventions that is not available from randomized trials, but their results may be affected by confounding, selection and information biases. Assessing the risk of bias in NRSIs included in systematic reviews is essential to ensure the robustness of review findings. The ROBINS-I (Risk of Bias in Non-randomized Studies – of Interventions) tool (BMJ 2016;355:i4919), which provides a structured approach to such assessments, has been updated since its publication.
Objectives: 1) To describe the ROBINS-I tool to assess risk of bias in NRSI. 2) To describe recent updates and improvements to ROBINS-I. 3) To provide participants with hands-on experience in ROBINS-I assessments.
Description: This workshop will describe key features of the ROBINS-I tool and improvements to the version published in 2016. - Specification of a hypothetical pragmatic randomized trial that is free of bias, as a basis for risk of bias assessments. - Preliminary considerations at review protocol stage. - Specification of the effect of interest (either the effect of assignment to intervention or the effect of adhering to intervention). - Assessments of risk of bias within seven domains (confounding; selection of participants into the study; classification of interventions; deviations from intended intervention; missing data; measurement of outcomes; selection of the reported result). - Signalling questions to inform judgements on risk of bias. - Use of algorithms to map answers to the signalling questions to suggested risk of bias judgements. - Overall risk of bias in the result. The workshop will include a guided practical session. Working in small groups, participants will complete parts of a risk of bias assessment for a selected non-randomized study. There will be opportunities for discussion of the tool and its role in systematic reviews. Participants are encouraged to bring their own portable computing device to access detailed guidance from the internet (www.riskofbias.info). The facilitators are unable to provide printed copies of the guidance document.

2:00 PM - 3:30 PMHow to author, publish, and dynamically update digital and trustworthy living evidence summaries, guidelines, and decision aids using MAGICappWorkshop - training

Background: There is an increasing emphasis on using living evidence to inform decision-making; however, creating living guidelines and decision aids is challenging. MAGICapp (www.magicapp.org) is an open-access software for evidence users, synthesizers, guideline developers, and proponents of shared decision-making to facilitate the creation, dissemination, and updating of trustworthy, digitally structured living evidence and clinical decision support tools. MAGICapp was designed with multiple stakeholders (healthcare providers, consumers, methodologists) while adhering to Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) guidance, with features that are continuously updated through research and innovation.
Objectives: To learn about and get practical experience with authoring, publishing, and dynamic updating of digitally structured living guidelines and evidence summaries with summary of findings (SoF) tables, recommendations, and decision aids using MAGICapp.
Description: The workshop will first introduce examples and features of rigorous practice evidence summaries and guidelines, starting from structured clinical questions (patient/population, intervention, comparison and outcomes [PICO] format) to making SoF tables and recommendations and how MAGICapp facilitates a living format through dynamic updating. Participants will split into groups, simulating panels responsible for updating a guideline for a clinical issue where there is a newly published Cochrane systematic review. Each group will receive a Cochrane review and access to an example brief guideline in MAGICapp that includes a recommendation with a linked SoF table and decision aids (semi-automatically generated from the SoF). Participants will collaborate to author a modified guideline based on the latest review evidence using MAGICapp’s features for updating and adapting content. Workshop facilitators will assist groups through “guideline panel” discussions and using MAGICapp. In a wrap-up session, participants will share experiences of creating and publishing living evidence and guidelines in MAGICapp and propose how to improve the software.

2:00 PM - 3:30 PMIntroduction to meta-analysis 1: meta-analysis of binary and continuous outcomesWorkshop - training

Background: A core component of many systematic reviews is meta-analysis, which is a method for statistically combining results across studies. Meta-analysis results often underpin healthcare decision-making. Most commonly, meta-analysis of binary or continuous outcomes are undertaken. Many issues need to be considered when meta-analysing binary or continuous outcomes ranging from data extraction through to the selected meta-analysis method.
Objectives: To provide review authors with knowledge to undertake meta-analysis of binary and continuous outcomes. This workshop is part of a series of workshops delivered by the Cochrane Statistical Methods Group.
Description: We will use a combination of presentations and practicals. We will begin with a brief recap of meta-analysis models. We will then cover issues specific to meta-analysis of binary and continuous outcomes, including: data extraction (e.g., for binary outcomes, extraction of event frequencies and/or effect estimates; and for continuous outcomes, extraction of standard deviations, or calculation from standard errors, confidence intervals, test statistics and P-values); effect measures (e.g., risk ratio, odds ratio, standardised mean difference), with rationale for choosing between them; and different meta-analysis methods (e.g., inverse-variance, Mantel-Haenszel).

2:00 PM - 3:30 PMFirst do no harm: how can systematic reviewers do justice to harms? Deciding which harms to search for and howWorkshop - training

Background: Any intervention that can have an effect has the potential to have an adverse effect. All systematic reviews of interventions should at least consider the harms of that intervention. Harms are important to patients, impacting on morbidity and mortality. There are many differences (as well as similarities) in how researchers should approach a review question on harms, as opposed to a question on benefit. Formulating the question on harms is a challenging but key step in the review process. Review teams need to decide whether to focus on specific harms (and if so which ones) or whether to have a broader objective. Balancing the quest for an answerable, useful question with limited resources is difficult. Searching for included studies on harms can also be problematic and often requires a different approach to searching for effectiveness studies. Obstacles include the lack of harm information in journal abstracts, inconsistent terminology and indexing, and the need to search a range of study designs beyond randomised controlled trials (RCTs).
Objectives: To give guidance to review authors on formulating the question, signposting the available data sources on harms, and retrieving data for incorporation into a systematic review. This will include the development of optimal search techniques for different interventions and discussion of the impact of CONSORT-harms.
Description: We will facilitate discussion on the advantages and disadvantages of a range of approaches to question formulation in systematic reviews incorporating harms. Next, participants will receive a number of scenarios and will work together in small groups to plan a search strategy for a comprehensive evaluation of harms. The scenarios will be drawn from real-life situations to cover a wide range of potential harms. At the end, the groups will give feedback on their search protocols and any points raised will be discussed further. Examples from existing reviews will be provided by the facilitator, as well as tips and tricks to solve specific issues, and examples of sources of support and help available.

2:00 PM - 3:30 PMMapping Reviews and Evidence Gap Maps: Evidence syntheses for broader health questionsWorkshop - training

Background: Demand for evidence syntheses to inform health practice, policy, and research agendas has grown exponentially with methods evolving to address the increasingly diverse types of questions that patients, practitioners, and policymakers pose. Mapping reviews and evidence gap maps (EGMs) have emerged over recent years to answer broader research questions. They aim to identify gaps in existing evidence and guide future research. Typically, mapping reviews and EGMs have a visual output that can be interpreted by a wide range of key stakeholders, including patients. We will draw on our expertise undertaking EGMs across a diverse range of topics (including preterm birth, medicines repurposing and elder abuse) to illustrate their utility and method. Learning Outcomes: Workshop attendees will gain an awareness of mapping review and EGM method including when suitable for use; gain an awareness of stakeholder involvement and engagement in mapping reviews and EGMs; be able to create a framework for an EGM; and be able to create an EGM using EPPI-Mapper. Embedded signposting within handout materials will enable self-directed learning to a greater depth.
Description: · 5 minutes: Introduction to workshop facilitators and how the session will run. · 15 minutes: Overview of mapping review/EGM purpose using examples from health and social care research. Use of an interactive e-learning tool to reinforce understanding. · 10 minutes: Small group work designing and assessing the suitability of research questions for EGMs. · 15 minutes: Introduction to methods for undertaking an EGM, including stakeholder involvement and engagement, particularly focused on EGM framework development. · 30 mins: Small group work to create an EGM framework for a pre-defined question using physical materials and a “walk the wall” facilitation approach. · 10 mins: Demonstration of EPPI-Mapper software for EGM creation. · 20 mins: Attendees use EPPI-Mapper to create their own EGM. · 5 mins: Round-up of learning outcomes and close. · Post-workshop: An interactive online tool (Padlet) will allow participants to reflect on their learning and indicate if they would like to know more about anything. We will collate responses and use them to create an FAQ with signposted resources.

2:00 PM - 3:30 PMHealth Equity: Implications for Systematic ReviewsWorkshop - training

Background: This is a core training session from a Cochrane Methods Group. The Cochrane Handbook for Systematic Reviews of Interventions includes a chapter on equity and specific populations. A Cochrane Interactive Learning module has been developed to accompany this chapter. This workshop aims to introduce all systematic reviewers to equity methods and provide tips for including equity considerations in all reviews. Average results may obscure differences in outcomes across specific populations, who may experience health inequity. Systematic reviews can explore the robustness of findings across specific populations, who may experience health inequity. Equity is defined as the absence of avoidable differences in health outcomes.
Objectives: Participants will understand how to 1) incorporate equity considerations into their systematic reviews and 2) report equity considerations completely and transparently.
Description: Participants will be introduced to the mnemonic “PROGRESS-Plus” and how it can be used to formulate questions and design methods to consider health equity (PROGRESS-Plus: Place of residence, Race/ethnicity/language/culture/language, Occupation, Gender/sex, Religion, Education, Socioeconomic status, Social capital). We will discuss how to consider specific populations such as older adults, migrants, and those who are socioeconomically disadvantaged. We will introduce participants to guidance for reporting equity-focused reviews (PRISMA Equity 2012) and the GRADE Working Group guidance on considering health equity in guideline development. We will discuss the importance of stakeholder engagement in systematic reviews. Participants will work in small groups to discuss hot topics in the consideration of health equity in systematic reviews and how to implement equity assessments in all Cochrane reviews.

2:00 PM - 3:30 PMDemystifying R Part 1: How to use Shiny apps in information retrievalWorkshop - training

Background: Information specialists increasingly rely on software tools for information retrieval. Tools created with the coding language R are of particular interest because R is both free and open source. These tools can be easily customized and extended. In addition, it is relatively easy to create graphical user interfaces (GUI) for R packages (so-called Shiny apps), which are particularly useful for people without coding experience. This is the first part of a two-part workshop. Part 2, submitted by Sarah Young, will present practical hands-on activities using R Coding and the RStudio coding environment.
Objectives: The workshop aims to help participants incorporate R into their workflow and to demystify coding jargon. In addition, they will gain hands-on experience with individual Shiny apps that can help improve efficiency in information retrieval.
Description: The workshop has a high level of interaction. It will cover the following topics: - Short introduction to R, with a focus on Shiny apps. - Hands-on activities with various Shiny apps including: 1) litsearchr (https://elizagrames.shinyapps.io/litsearchr/), development of search strategies using text mining and keyword co-occurrence networks; 2) citationchaser (https://estech.shinyapps.io/citationchaser/), citation chasing using Lens.org; and 3) citesource (https://github.com/ESHackathon/CiteSource), comparison of source contributions at different stages of the review process and generation of search summary tables. - Group discussion about the benefits and challenges of R tools for evidence synthesis, including identifying opportunities for collaboration and learning Our workshop is aimed at information specialists, librarians and other researchers who would like to learn the basics of R and try out different Shiny apps in a hands-on environment. If possible, please bring a laptop. No previous programming experience is required. The workshop is relevant to patient care because the use of software tools can lead to more efficient production of evidence syntheses, making them available in a more timely manner. In addition, the tools used in the workshop will help to create more robust searches, which are the basis for the accuracy and reliability of evidence syntheses, and ultimately help to improve the quality of the evidence used to inform decisions about patient care.

2:00 PM - 3:30 PMAssessing the certainty of the evidence from network-meta analysis using the GRADE approachWorkshop - training

Background: Assessing the certainty of the evidence is among the Methodological Expectations of Cochrane Intervention Reviews (MECIR) standards, and it is also acknowledged as a key step in non-Cochrane reviews. Appropriate interpretation of the results from network meta-analysis (NMA) requires considering the certainty of the evidence.
Objectives: For individuals considering conducting an NMA, to gain familiarity with and begin to gain facility in applying the GRADE Working Group’s approach to rating the certainty of evidence in comparisons within an NMA.
Description: The GRADE Working Group has developed an approach to rating the certainty of evidence (also known as confidence in evidence or quality of evidence) in each comparison within a network meta-analysis. The approach involves three steps: i) rate the certainty of the direct estimates, ii) rate the certainty of the indirect estimates, and iii) rate the certainty of the network estimates. The workshop will begin with an interactive lecture providing details of the approach and then review a step-by-step template for applying the approach. Workshop participants will then break into groups of five or six to work through an example of a network meta-analysis, guided by facilitators when needed. Before the end of the session, the facilitators will lead the large group in a discussion of the results and details of the assessments.

3:30 PM - 4:00 PMBreakCoffee break 
4:00 PM - 5:30 PMEditorial processes and supporting review authorsOral session
16:00 Opening remarks and introduction
16:05 The Central Editorial Service: who we are, what we do, and what happens to your draft after submission
16:25 The Cochrane Evidence Pipeline: transforming the way we identify evidence
16:45 Unlocking innovations for sharing and using Cochrane content with study centric data management and a focused review article
16:55 Updating the web-based "Right Review" tool: an international Delphi process
17:05 The Systematic Review Toolbox: an updated resource to support evidence synthesis
17:15 Challenges of coordinating large-scale systematic reviews
17:25 Closing remarks
4:00 PM - 5:30 PMEngaging stakeholders, building capacity, developing partnershipsOral session
16:00 Opening remarks and introduction
16:05 Developing capacity for the production and use of Living Evidence in decision making: The LE to Inform Health decisions project.
16:25 Enhancing capacity for network meta-analysis in Sub-Saharan Africa
16:35 Partnerships for practice improvement: JBI's collaborative approach to developing evidence-based point-of-care resources
16:45 Driving the evidence-based healthcare agenda forward: a qualitative reflexive thematic analysis of global health partnerships
16:55 Opioid use disorder treatment in Canada: knowledge sharing between regionalized networks
17:05 Global evidence, local adaptation (GELA): Enhancing evidence-informed guideline recommendations for newborn and young child health in three countries in sub-Saharan Africa
17:15 Closing remarks
4:00 PM - 5:30 PMMethodological and reporting qualityOral session
16:00 Opening remarks and introduction
16:05 TRIPOD-SRMA: Reporting guideline for transparent reporting of systematic reviews and meta-analyses of prediction model studies
16:25 Do Cochrane and non-Cochrane editors and authors prefer reporting statements based on statistically significant differences or do they prefer non-binary options?
16:45 Methodological and reporting quality of systematic reviews and meta-analyses in dermatology: a cross-sectional study
16:55 Methodological quality of systematic reviews on Chinese herbal medicine: a methodological survey
17:05 Frequency of use and reporting adequacy of Cochrane RoB 2 tool in non-Cochrane systematic reviews published in 2020: meta-research study
17:15 Closing remarks
4:00 PM - 5:30 PMKnowledge translationOral session
16:00 Opening remarks and introduction
16:05 Misinformation in COVID-19: nimble innovation within Cochrane to appraise the source
16:25 Message Lab – a platform for research and improvement of public health messages
16:35 Highlights and challenges of co-creating evidence-based knowledge translation tools for parents about the COVID-19 pandemic
16:45 Stakeholder involvement for investigating communication-based interventions to increase COVID-19 vaccine uptake
16:55 Bringing Cochrane reviews to Polish professionals and consumers
17:05 Abstracts of Cochrane reviews are getting longer, but this has no large impact on the reporting quality.
17:15 Closing remarks
4:00 PM - 5:30 PMNon-randomised studies and mixed methodsOral session
16:00 Opening remarks and introduction
16:05 Incorporating data from atypical experimental study designs in systematic reviews: experiences from Cochrane Infectious Diseases
16:25 Non-randomised studies of interventions in systematic reviews – Limitations and opportunities illustrated with an exemplary review on COVID-19 vaccination in children
16:45 Supplementing Systematic Review Evidence with Health System Data: A Target Trial Emulation of Triptan in Migraine Patients with Increased Cardiovascular Risk
16:55 Use and methodology of framework synthesis in mixed-methods literature reviews
17:05 Interpreting the results of observational studies in the context of variation expected due to analytic flexibility
17:15 A method for the rapid assessment of high-cost cancer drug indications in a hospital setting
17:25 Closing remarks
4:00 PM - 5:30 PMLiving evidence – from concept to realitySpecial Session

We need trusted, up-to-date guidelines to stimulate clinicians to change their practice and provide patients with the best care. This is vital in a world where mistrust has become the norm, from anything to the news, body image or fake science.  Credible guidelines must be independent, based on reliable information and – critically and demonstrably – in line with the latest research. 

Turning this vision into reality is not straightforward.  Timelines for identifying new research, evaluating data, reviewing and amending guideline recommendations typically stretch from months into years.  A new collaboration, the Global Alliance of Living Evidence (GALE), is attempting to demonstrate how this can be done, drawing on recent successes during the pandemic.

This Special Session will provide an opportunity to hear from leaders in the field and to use live voting to track audience opinions.  The session will explore the potential for new technologies such as artificial intelligence to support the process, and to consider what this means for future Cochrane Reviews.

Target audience: Anyone with an interest in living systematic reviews, in particular those keen to collaborate with others to develop the approach and share ideas. 

Format: There will be some initial presentations followed by a group discussion to explore challenges and identify potential solutions.

4:00 PM - 5:30 PMMind the gap! Building the evidence base for co-producing evidence synthesisSpecial Session

Cochrane is committed to the co-production of its evidence. The proposed Co-production Methods Group will support this by spearheading methods research to establish best practice; supporting the development and sharing of methods of co-production; sign-posting Cochrane Review authors to resources aimed at supporting best practice; maintaining and developing learning resources of learning resources; supporting learning and dissemination activities; and updating co-production guidance in Cochrane Handbooks.

This session is an opportunity for stakeholders to come together to examine the current state of evidence, listen to one another, identify gaps, evidence, information and practice and to determine the Cochrane Community’s priorities for future work to extend and improve the practice of co-production.

Objectives:

  • Bring together stakeholders in co-production and build a community of people with an interest in improving methods
  • Engage in a lively multi-stakeholder dialogue to understand and consider various perspectives on co-production
  • Describe the current state of co-production in evidence production in Cochrane
  • Explore the evidence base for co-production of evidence synthesis
  • Identify future priorities for the Co-production Methods Group

This interactive Special Session will bring together researchers, consumers and other stakeholders from the Cochrane Community and other Methods Groups, and partner organizations. The aims and objectives of the proposed Co-production Methods Group will be described and discussed. Participants will explore definitions of co-production and who the stakeholders are of systematic reviews and other forms of research. Researchers and consumers will share examples of methods research including the ACTIVE project, involvement in Living Systematic Reviews and rapid evidence production, and resources to support co-production. Presenters will describe current research, including the MuSE project. Participants will discuss organizational policy, and the nature and extent of co-production within Cochrane and the wider community, including the Cochrane Consumer Network, and consumer perspectives on current practice. Drawing on all of the preceding information and discussion, participants will identify and rank future priorities for the Co-production Methods Group.

Target audience: People with an interest in the methodology of stakeholder engagement and involvement

Format: Discussion

4:00 PM - 5:30 PMIntroduction to meta-analysis 2: dealing with heterogeneityWorkshop - training

Background: The studies in a meta-analysis may vary in their included populations, how the intervention of interest was used, how outcomes were assessed, and in study design and conduct. This variability in study properties may lead to variability in the outcomes across studies. This variability in outcomes in a meta-analysis is called heterogeneity. Determining whether heterogeneity across studies is present in a meta-analysis and identifying its possible causes are critical components of any meta-analysis.
Objectives: To provide review authors with the knowledge to understand and investigate heterogeneity across studies in a meta-analysis and to recognise the limitations of the methods available. This workshop is part of a series of workshops delivered by the Cochrane Statistical Methods Group.
Description: We will address approaches to dealing with heterogeneity across studies in a meta-analysis. We first discuss potential sources of across-study variability and provide an overview of methods for identifying whether heterogeneity is present in a meta-analysis. We then focus on issues related to dealing with heterogeneity once it has been identified. In particular, we discuss whether or not to combine results; how heterogeneity is handled in the choice between fixed-effect and random-effects analyses; and the use of subgroup analyses (with a brief mention of meta-regression). Discussion will be supplemented with practical examples from the Cochrane Database of Systematic Reviews.

4:00 PM - 5:30 PMSynthesizing and presenting results when meta-analysis is not possibleWorkshop - training

Background: In reviews in which meta-analysis is not used, authors commonly report results study-by-study or draw conclusions without reporting how findings were interpreted across studies. These approaches may leave decision-makers to make sense of the findings themselves and undermine confidence in the evidence. There are many circumstances that may preclude the use of meta-analysis of effect estimates. For example, when there is incomplete information reported about the intervention effect estimates (e.g., missing standard errors) or inconsistency in the reported effect metrics across studies. Planning for circumstances that may preclude meta-analysis can ensure that reviewers make the best use of available data and produce more useful syntheses for decision-makers. This workshop will cover structured summary, synthesis methods, and visual display methods that can be used in these circumstances (sometimes referred to as “narrative synthesis”).
Objectives:
• To illustrate the importance of planning for scenarios in which meta-analysis is not possible through application in a practical example.
• To demonstrate the application of structured summary, other synthesis, and visual display methods.
Description: Based on guidance in Chapter 12 of version 6 of the Cochrane Handbook for Systematic Reviews of Interventions, we will use a combination of presentations, interactive exercises, and group work to cover the following topics:
• Scenarios that may preclude meta-analysis
• Other synthesis and presentation methods, along with their advantages and disadvantages, and guidance on when to use which approach
• Exploring extracted data to determine what other synthesis methods might be used if meta-analysis is not possible
• Suggestions for what to write in the protocol, the methods section of the review, and how to describe the results of the synthesis Participants will work through examples that illustrate different approaches to structured summary and statistical synthesis of data.

4:00 PM - 5:30 PMPractical advice for accessing patient-level data from a data sharing platform for evidence synthesisWorkshop - training

Background: Since 2014, opportunities to request and access the patient level datasets collected in a clinical trial have become more commonplace. Vivli is an independent, non-profit organisation that has developed a global data-sharing and analytics platform. Patient-level data is available from 7,000 clinical trials that are provided by academic funders, pharmaceutical companies, and charitable funders.
Objectives: This workshop will provide an: ● Introduction to the Vivli platform, including an overview of data available and hands-on group work on how to submit a request for data ● Overview of first-hand experience of using patient level data using the data sharing platform, Vivli, as a case study ● Discussion of some of the opportunities and challenges (e.g., when data needed to answer the hypothesis is not available via a single data sharing platform) associated with using patient level data
Description: This session will be a mixture of hands-on practice of exploring the clinical trial data on the Vivli platform, the submission process, and what the resultant data and document package looks like. Alongside this, there will be a chance to hear from researchers who have used the platform and the perspectives of patient advocates and data providers, as well as an opportunity to ask questions about the data access process. Session Plan : Welcome and Introduction to IPD Meta-analysis and Data Sharing Platforms (10 minutes). Case study: Perspectives from a data requester, patient advocate, and data provider on the process for sharing data (20 minutes). Hands-on demonstration and activity: Using Vivli to submit a data request (10 minutes). How to formulate a data sharing request (20 minutes). --Account Creation --Overview of standard information in a data request form --Governance process. Criteria used by the Independent Review Panel (IRP) to review a request Group work: Practice filling out a data request using a Cochrane Protocol (15 minutes). Report Back: What was easy about this process? What was unclear? Discussion and Questions (15 minutes) Proposers: Catrin Tudur Smith (University of Liverpool), Rebecca Sudlow (Roche), Julie Wood (Vivli), and Alan Chant (Patient Advocate and Independent Review Panel member)

4:00 PM - 5:30 PMSystematic reviews of prognosis studies I: Introduction, design and protocol of systematic reviews of prognosis studiesWorkshop - training

Background: Prognosis studies are abundant in medical literature. Hence, systematic reviews of these studies are increasingly required and conducted to identify and critically appraise the existing evidence. A Cochrane handbook for prognosis reviews is currently being prepared describing guidance for conducting a systematic review of prognosis studies. The CHARMS tool is developed to provide guidance for design and conduct of the first steps of systematic reviews of prognosis studies. CHARMS can assist reviewers of prognosis studies in defining the review objectives and design of the review and creating the data extraction list to enhance critical appraisal of the primary studies.
Objectives: This workshop will introduce participants to the Cochrane Handbook for Reviews of Prognosis Studies, the types of prognosis research, and explain the differences between prognosis and intervention studies. We will provide guidance on how to define a proper review question and how to design a protocol and data extraction form to enhance subsequent critical appraisal.
Description: The workshop will start with an introduction to the different types of prognosis studies and to systematic reviews of prognosis studies. We subsequently discuss the key items important for framing a review question, the essentials of a protocol, and data extraction using the CHARMS checklist. The workshop will consist of interactive lectures and small group exercises.

4:00 PM - 5:30 PMIntegrating the findings of a qualitative evidence synthesis (QES) with the findings of a review of intervention effectsWorkshop - training

QMIG Methods Workshop
Background: Methods for qualitative evidence synthesis (QES)—a systematic review method for bringing together qualitative studies—are now well established, and decision-makers are able to access rigorous QES on issues such as patient experiences and contextual factors influencing intervention implementation. QES often generates theories and explanations about why and how interventions work. By integrating these theories with evidence on intervention effects, review teams can offer vital information to support the implementation of review findings in practice. However, the diversity of approaches for integrating QES and effectiveness syntheses can make it challenging for reviewers to understand how best to integrate. The Cochrane Qualitative and Implementation Methods Group (CQIMG) develops and publishes guidance on conducting QES and integrating QES findings with evidence on intervention effects. In this CQIMG methods workshop, participants will learn to distinguish between two key integration approaches. Comparison of syntheses involves systematically comparing the QES and effectiveness synthesis findings; for example, the findings from each synthesis may be juxtaposed in a matrix, to illustrate where they are concordant and discordant and where there are gaps. Connection of syntheses involves using the findings of one synthesis (QES or effectiveness) to inform the data extraction and synthesis of a second synthesis; for example, the QES findings may be used to structure a subgroup analysis of the effectiveness data. Participants will draw on real integration examples and critically examine the diversity of methods and tools.
Objectives: By the end of this workshop, participants will be able to - recognise and appraise approaches and tools for integrations; - describe challenges, and potential opportunities, for integration; and - apply lessons learnt to their own review activity.
Description: This interactive workshop presents options for integrating QES and intervention effects data (10 minutes) using Cochrane, Campbell and other reviews as examples. Group exercises, supported by experienced facilitators, illustrate different integration methods and tools (50 minutes). Groups are encouraged to think reflectively (10 minutes) on strengths and limitations of diverse approaches. The session concludes with an open forum to address issues relating to participants’ review activity (10 minutes). Timings allow 10 minutes for flexibility.

4:00 PM - 5:30 PMDemystifying R Part 2: An introduction to coding in R and RStudioWorkshop - training

Background: Tools like R and Python are becoming increasingly useful in the conduct of systematic reviews and evidence synthesis. In R specifically, many tools have been developed to facilitate the systematic review process. Some of these tools provide vignettes and examples to help novice coders make use of the tools in a coding environment like RStudio, and others have graphical user interfaces that make them accessible to users without coding experience.
Objectives: Building on Demystifying R Part 1 (workshop proposal submitted separately by Elke Hausner), this workshop will introduce learners to the basics of the R coding language and the RStudio coding environment, in the context of litsearchr, an R package that supports term harvesting, Boolean search construction, and search strategy testing, among other steps.
Description: We will draw from recently developed Library Carpentry curriculum (https://carpentries-incubator.github.io/lc-litsearchr/) to provide a live coding, fully hands-on session. The workshop will cover the basics of RStudio, creating objects and variables, working with packages and libraries, and best practices for code documentation. This workshop is geared towards participants with no coding experience. Taking Part 1 of this series is optional but recommended. Participants will learn basic concepts of working with files, folders, and objects in R and RStudio, will understand different data structures, and will be introduced to litsearchr as a tool that can be incorporated into their systematic review workflows. Using open-source tools like R for evidence synthesis allows for more reproducible methods, expediting the evidence synthesis process, and improving transparency. Thus, this workshop will contribute to improved patient care through the facilitation of more efficient and transparent evidence synthesis products.

4:00 PM - 5:30 PMMetaDTA and MetaBayesDTA: Interactive web applications to conduct meta-analysis of diagnostic test accuracy studiesWorkshop - training

Background Diagnostic tests form an essential part of current medical practices aiming to distinguish between patients with the disease and healthy individuals. They are used across a diverse range of healthcare settings and are often a pre-requisite to identifying treatment options and enabling access to services. Recommended statistical methods for meta-analysis of diagnostic test accuracy (DTA) studies require the fitting of complex non-standard statistical models, which can be a barrier to their application. MetaDTA (https://crsu.shinyapps.io/dta_ma/) is a free interactive online application which, for meta-analyses DTA studies using the bivariate model, plots the summary receiver operating characteristic curve, encourages sensitivity analysis and incorporates quality assessment results from the QUADAS-2 tool. MetaBayesDTA (https://crsu.shinyapps.io/MetaBayesDTA/) is a version that extends the functionality to allow for imperfect gold standards, subgroup analysis, meta-regression and comparative test accuracy evaluation. Both apps produce visualisations which facilitate the communication of results to all stakeholders, including patients and healthcare professionals. Due to the rich feature-set and user-friendly “point and click” interface of these apps, this workshop should appeal to a wide audience, including non-statistical experts. Objectives To illustrate how MetaDTA and MetaBayesDTA can be used to conduct DTA meta-analyses (using methods recommended in Version 2 of Cochrane Handbook) and for participants to get hands-on experience of using the apps in a structured and supportive environment. Description This workshop will provide an introduction to MetaDTA and MetaBayesDTA and offer participants experience using these web-based applications. During this workshop, we will: i. Demonstrate how MetaDTA and MetaBayesDTA can be used to conduct a Cochrane DTA meta-analysis using an example from a Cochrane reviews; ii. Demonstrate the wide array of interactive functions available within MetaDTA and MetaBayesDTA; iii. Provide participants with a dataset and worksheet designed to guide them, at their own pace, through the stages of conducting an analysis in MetaDTA and MetaBayesDTA; and iv. Provide time for questions, discussion and feedback. Topics covered will include loading data, customising summary receiver operating characteristics (SROC) plots, obtaining statistics, visualising study quality results and covariate effects, conducting sensitivity analyses and exporting tables and figures. All participants must bring a laptop or tablet or may share with another participant.

4:00 PM - 5:30 PMPerforming and disseminating review findings through art and designWorkshop - training

Background: Interest in using multimodal, arts-based methods in the context of health-related research has been increasing recently. Its value lies not only in its ability to unearth complex, rich and nuanced data but also in how the process/processes can simultaneously facilitate the accessibility of evidence (MacGregor et al. 2022; West et al. 2022; Archibald & Blines 2021; Ball et al. 2021; Boydel 2019; Fraser & Al Sayah 2011). From a new materialist perspective, knowledge is not merely an independent, tightly bound entity to be used or consumed but is rather a dynamic construct that can adapt and change as it is applied and used in everyday contexts.
Objectives: The objectives of this workshop include engaging in collaborative arts-based techniques to make sense of a specific case of systematically reviewed evidence, i.e., Factors that influence parents’ and informal caregivers’ views and practices regarding routine childhood vaccination: A qualitative evidence synthesis (Cooper et al. Cochrane Database of Systematic Reviews 2021, Issue 10. CD013265. DOI: 10.1002/14651858.CD013265.pub2), and to demonstrate how this process can facilitate the production of embodied knowledge through active engagement with the evidence in tangible, material ways.
Description: (1) Workshop participants will be introduced to the workshop and divided into subgroups (5 minutes); (2) each group will engage in a performative exercise based on an aspect of the reviewed evidence (20 minutes); (3) each individual will respond to their performative experience through a mark-making exercise using charcoal and/or ink on paper (20 minutes); (4) each group will be prompted through a process of cutting up their group’s drawings and reassembling parts of it through collage to re-represent the evidence in graphic form on A3 paper (25 minutes); and (5) each group will present their graphic translation of evidence to the larger group (20 minutes). What will result could be regarded as an example of how multimodal arts-based engagement with systematically reviewed content in the context of healthcare can make the evidence more accessible—and hence also applicable—to those engaging with it, as well as how creative representation of evidence can possibly enrich the communicative potential of empirical data.

Workshop set-up.jpg

4:00 PM - 5:30 PMComplying with Cochrane's Conflict of Interest policyWorkshop - training

Background: All Cochrane Library content must comply with Cochrane’s conflict of interest (CoI) policy, which applies to all individuals involved in creating Cochrane Library content. The Research Integrity team and Cochrane’s policy team are proposing this workshop to support authors, editors, and others interested in understanding the application of Cochrane’s CoI policy and its related editorial policy on authorship. Authors bear primary responsibility for ensuring compliance with both the CoI and authorship policies prior to submission of a completed article, and Cochrane managing editors check for policy adherence when receiving a completed submission. At this stage, it can be difficult to correct breaches of the CoI policy without creating new breaches of Cochrane’s authorship policy, such as ghost authorship (see “Conflict of interest and authorship” at https://community.cochrane.org/news/conflicts-interests-and-authorship-lessons-revised-policy). Noncompliance discovered at submission or later may result in an article’s rejection, representing a major loss of effort for the author team and the loss of a timely article for the Cochrane Library.
Objectives: Engage participants in - analyzing case studies of author CoIs to determine whether they comply with the CoI policy; - discussion about interpreting and applying the CoI policy; and - Q&A about the cases and related CoI/authorship scenarios.
Description: Introduction (ca 15 minutes): Facilitators will introduce the framework of Cochrane’s CoI policy (2020), its connection to Cochrane’s editorial policy on authorship, and the most salient points for authors and editors. Case studies (three rounds, ca 20/25 minutes each): Participants will consult handouts showing the CoIs of a fictional author team, the topic of the fictional protocol or review, the text of Cochrane’s CoI and authorship policies, and CoI support documents. In small group discussions, participants will decide a) whether the authors comply with the CoI and authorship policies and b) whether and how compliance could be achieved, if there is a breach. Organizers will then seek participants’ decisions through an interactive on-screen survey and lead a whole-group discussion/Q&A for each case study. Conclusions (ca 5 minutes): Organizers will summarize support resources.

5:45 PM - 7:15 PMAnnual General Meeting (AGM)Special Session

All Cochrane Members and supporters are cordially invited to attend Cochrane's 2023 Annual General Meeting!

The aim of the Annual General Meeting is for Cochrane's Trustees - its Governing Board - and senior officers to explain their management of the charity to you, the Members. It also provides you with an opportunity to ask questions and decide on business and current issues affecting the organization, which are put to the vote as 'Resolutions'.

Voting on AGM Resolutions:

All Cochrane Members as defined by the Membership Terms & Conditions are entitled to vote on AGM Resolutions. You are entitled to vote even if you are not attending the meeting. If you are not sure whether you are a Member, or you are having trouble voting, please contact support@cochrane.org.

You may choose to vote online in advance of the AGM regardless of whether you are planning to attend in person. If you are attending, you can choose to wait until the AGM to vote. In all circumstances at the time of voting you must have access to an internet-connected device.

To find out more about voting, visit: https://agm.cochrane.org/agm-2023

Watch a recording of the 2023 AGM on YouTube.

7:30 AM - 8:30 AMNutrition & Physical Activity TG - By Invitation OnlyMeeting

Opportunity for TG members and participants to meet and to discuss the workplan of the TG and brainstorm future directions.

7:30 AM - 8:45 AMIberoamerican Network MeetingMeeting

Meeting for all the members of the Iberoamerican Cochrane Network.

7:30 AM - 8:45 AMStudy within a review (SWAR) network meetingMeeting

 

A study within a review (SWAR)is a research study that can help provide evidence to inform decisions about how we plan, do and share the findings of future reviews. We’d like to bring together people who are interested in SWARs, and start a SWAR Network. We envision this network would explore SWAR methods, keep everyone up-to-date, and potentially explore challenges and opportunities around SWARs. This meeting is hosted by Evidence Synthesis Ireland (https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.evidencesynthesisireland.ie%2F&data=05%7C01%7Cskhamissa%40cochrane.org%7Ccf60af1acc584846cb2c08db29569453%7Cb6c2e21e4db74533916398c1451c1caa%7C0%7C0%7C638149224049203720%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=cpHq2XY7BB3I%2FynrxDbiu%2BWNSrUECp9wmuvRnKgNlkg%3D&reserved=0) and Cochrane Ireland, who recently launched their inaugural SWAR Award Scheme.

7:30 AM - 8:45 AMCochrane Francophone Network - By Invitation OnlyMeeting

Cochrane France has created a Cochrane Francophone Network in 2020. Members of Cochrane Belgium, Cochrane Canada - Francophone, Cochrane Cameroon and Cochrane Switzerland had participated in several online meetings. For this occasion, we would also like to involve the francophone members of Cochrane Skin, Cochrane PEC, Cochrane Cochrane lung cancer and Cochrane Musculoskeletal. The purpose of the meeting is to share information about training activities and projects that each group is developing in French and discuss about the ways we can work together to disseminate and participate in those activities. In this way, Cochrane evidence will be available in the French speaking countries. We have also proposed to develop this network, particularly to enhance Cochrane representation in Francophone Africa where evidence based activities are very limited.

7:30 AM - 8:45 AMInformation Retrieval Methods Group Annual meetingMeeting

An annual meeting of IRMG members and others with an interest in information retrieval methodology. A chance for members to catch up and view presentations on group members recent activity.

7:45 AM - 8:45 AMCochrane Statistical Methods GroupMeeting

This is the administrative and scientific meeting of the Statistical Methods Group (SMG). A brief update will be provided on SMG activities. This will be followed by presentations and discussion. Guido Skipka will talk on Methods for evidence synthesis in the case of very few studies.

9:00 AM - 10:30 AMEnsuring integrity in biomedical researchPlenary

We all need data we can trust. Cochrane policy requires that studies with serious research integrity problems, including fraudulent data, be excluded from Cochrane Reviews.  This plenary explored the scope and root causes of the problem of fraudulent and problematic research.  Solutions to identifying and preventing the publication of research with serious research integrity problems were discussed.  The international panel of speakers offered perspectives from a variety of disciplines on paper mills, tools to identify fraudulent studies, and what systematic reviewers and journals can do to improve research integrity. There was ample time for audience participation in the discussion of proposed solutions.

Keynotes:

  • Quantity over quality: a primer on research paper mills (Jenny Byrne)
  • Journals identifying and eliminating problem studies (John Carlisle)
  • This is a global issue: The African Research Integrity Network (ARIN) (Limbanazo Matandika)
  • How to reduce structural and academic incentives that promote fraud (Cyril Labbe)
  • Protecting the integrity of systematic reviews (Lisa Bero)

Related sessions:

This session was chaired by Richard Van Noorden, from Nature.

10:30 AM - 11:00 AMBreakCoffee break 
11:00 AM - 12:30 PMGlobal health, equity and partnershipsOral session
11:00 Opening remarks and introduction
11:05 Archie Cochrane's Second E - the creation of Value based Healthcare
11:15 Strengthening the global evidence base on public health and social measures through a WHO research initiative
11:25 Defining Racial Health Equity (RHE): A Landscape Review of Definitions, Terminology and Related Concepts
11:35 Engaging racially and ethnically diverse stakeholders in evidence syntheses: A review of reviews and guidance documents
11:45 Equity considerations in the conduct of evidence syntheses in partnership with patients and other stakeholders
11:55 Implications of centering racial health equity in systematic reviews: results of qualitative interviews with stakeholders
12:05 From Agenda to Action: Collaborative Evidence Networks and the United Nations Sustainable Development Goals
12:15 Global Evidence, Local Adaptation: Integrated Knowledge Translation strategies to enhance evidence-informed newborn and child health guidelines in three African countries
12:15-12:30 Closing remarks
11:00 AM - 12:30 PMEvidence synthesis innovations and technologyOral session
11:00 Opening remarks and introduction
11:05 Integrating Machine Learning into a Systematic Review Workflow: Testing the Cochrane RCT Classifier in a Research Consultancy Setting
11:25 The improved Systematic Review Data Repository Plus (SRDR+): A free, “FHIR-ed up” tool for screening, data extraction, and data sharing
11:45 Building acceptance for machine learning in study selection within a systematic review institution: Experiences from the Norwegian Institute of Public Health
12:05 ChatGPT and large language models for systematic review tasks: What are the opportunities for improvement?
12:15 What is the level of expertise of ChatGPT in the domain of systematic reviews and meta-analysis?
12:25 Closing remarks
11:00 AM - 12:30 PMEvidence synthesis and clinical guidelines: tools and methodsOral session
11:00 Opening remarks and introduction
11:05 Development of a health-system guidance implementation tool using a modified Delphi method and a formative evaluation approach
11:25 Scope, quality and reporting of clinical practice guidelines for newborn and child health in South Africa, Nigeria, and Malawi.
11:35 An Evidence Ecosystem Evaluation for the Prevention and Control of Healthcare-Acquired Infections in China
11:45 The development and testing of the Scientific, Transparent and Applicable Rankings tool (STAR) for clinical practice guidelines
11:55 A real-world example demonstrating the application of Cochrane guidance to ensure transparent synthesis of evidence addressing a broad policy question
12:05 Tools and methods for assessing the transferability of health technology assessment results across jurisdictions: a systematic review
12:15 The regulatory process for drug approval needs independent meta-analytical evidence: is Cochrane a possible candidate? The case of psychotropic medicines
12:25 Closing remarks
11:00 AM - 12:30 PMPatient or healthcare consumer involvementOral session
11:00 Opening remarks and introduction
11:05 Involvement of children and young people in development of an evidence synthesis framework for what interventions best prevent childhood obesity
11:15 Bringing a patient perspective to understanding interventions that aim to reduce length of stay in hospital: a mixed methods synthesis
11:25 Stakeholder involvement to enhance the relevance and accessibility of a Cochrane review of physical rehabilitation after stroke
11:35 Development and co-design of a behavioral activation intervention targeting depression among people with dementia for the Swedish context
11:45 Co-designing outcomes for a review on self-harm and suicide prevention interventions: Are we failing our young people?
11:55 Involving people in determining outcomes for a proposed Health Technology Assessment surrounding pelvic organ prolapse
12:05 Importance of consensus methods and involving end users to develop evidence-based materials when evidence is scarce: a Red Cross example
12:15 Analysis of the proportion, role, function, and reporting quality of patient and public involvement (PPI) in systematic reviews and meta-analyses: A cross-sectional study
12:25 Closing remarks
11:00 AM - 12:30 PMStatistical methods and meta-analysisOral session
11:00 Opening remarks and introduction
11:05 Meta-analysis: what is it all for?
11:25 Reliably estimating interactions and subgroup effects in aggregate data meta-analysis
11:45 Likelihood Ratio Meta-Analysis
12:05 Towards More Scientific Meta-Analyses
12:15 Development of a free tool for data estimation and conversion for meta-analysis (DECoMA)
12:25 Closing remarks
11:00 AM - 12:30 PMCommunicating your workSkills Lab

Communications experts from Cochrane invite you to a special session about communicating your research, aimed at authors and researchers who want to share the impact of their work with wider audiences. In this fast-paced 90-minute session we will go through some basics of science communication, ask thought-provoking questions, offer small group activities, and share case studies to enhance your skill in engaging others through confident storytelling.

This interactive session will cover:

  • Principles of science communication 
  • Guidance on maximising the impact of your communications, including advice on working with your institution’s own press office or communications team
  • Examples of good science communication and practical tips, for example, on using plain language and avoiding jargon 
  • Advice on how to tell a good story

Using storytelling tactics you’ve heard in the session, you’ll also have the opportunity to explore how you could communicate your own research. 

Presenters: Harry Dayantis, Georg Ruschemeyer, Wanjiru Mwangi

Moderators/facilitators: Katie Abbotts, Muriah Umoquit

11:00 AM - 12:30 PMAn Agile Scientific Strategy for Global Health: Cochrane’s Future DirectionSpecial Session

Cochrane, globally recognized as a trusted health evidence producer, is paving the way for the future by embracing transformative changes aimed at enhancing its global health impact. This Special Session at the Cochrane Colloquium will shed light on these transformative changes and demonstrate how they establish a pathway towards a pioneering scientific strategy.

Underlining the need for adaptability, Cochrane is recalibrating its processes and methods to enhance experiences for both authors and users of Cochrane evidence. These transformational shifts, although substantial, are integral to maintaining the organization’s relevance and influence in the dynamic global health landscape.

At the core of these transformations is a strategic emphasis on 20 thematic areas that align with the health-related objectives of the United Nations Sustainable Development Goals (SDGs). Collaborating with a range of stakeholders, Cochrane aspires to generate trusted evidence syntheses and champion their worldwide application, aiming for improved health outcomes globally and promoting equity, diversity, and inclusion.

This session will outline the critical steps towards developing this scientific strategy, including a detailed overview of the transformational changes in progress. It will also offer an opportunity to delve into the focus on the Sustainable Development Goals (SDGs) and initiate a broader consultation process to select key themes. The session represents an invitation to engage with Cochrane’s ambitious vision for global health and its commitment to fostering a culture of evidence-informed health and care worldwide.

Target audience: Everyone

11:00 AM - 12:30 PMMaximizing the potential of data associated with Cochrane reviews: Opportunities and future directions for the new review data packageWorkshop - discussion

Background: Sharing the data associated with Cochrane reviews beyond the analyses data opens up an array of benefits and opportunities, including: facilitating sharing and use of data; increasing opportunities for re-use and further analysis; increased potential to impact policy; increasing research visibility, discovery, impact and recognition; facilitating research validity through replication and verification; decreasing the risk of research fraud through transparency; facilitating collaborative research and reducing redundancy and research waste across siloed groups; use of real research in educational materials; enabling public understanding; and promotion of citizen science. Cochrane recently introduced a new data package available to download on published Cochrane reviews, which include included studies data, included study arms data and results data, risk of bias assessments and the support for judgements, analyses data and the review’s references. This data package is organized in different data formats to facilitate re-use, including via RevMan or other tools such as Microsoft Excel and various statistical packages. These same data formats also underpin the route that enables authors to transfer data between Covidence and RevMan.
Objectives: To gather feedback and ideas from the participants to determine new ways the data files could be used, as well as potential changes to the formats that will improve their overall usefulness or unlock new opportunities for use or re-use of review data.
Description: We will review the different datasets contained within the data package and have a semi-structured discussion on each dataset, as well as brainstorm and prioritize additional datasets that could be included in future.

11:00 AM - 12:30 PMIssues in Using, Interpreting, and Presenting Patient-Reported Outcomes in Cochrane ReviewsWorkshop - training

Background: Systematic reviews and meta-analyses of randomized trials that include patient-reported outcomes measures (PROMs) provide crucial information for patients and clinicians facing challenging healthcare decisions. Based on emerging methods, guidance on combining PROMs in meta-analyses and interpreting results will likely enhance their usefulness for decision-makers.
Objectives: In the context of GRADE summary of findings tables in Cochrane reviews, participants will be able to 1. identify the issues in retrieving, analyzing, and interpreting PRO results in clinical trials; 2. discuss preferred alternatives with an emphasis on the minimal important difference (MID) for aggregating across different PRO instruments measuring the same construct; 3. describe how to retrieve MIDs and introduce the concept of judging the credibility of MID estimates; and 4. present PRO results in GRADE summary of findings tables.
Description: This workshop will use interactive lectures with questions to participants and small group discussions to address the challenges in including and interpreting PROs in Cochrane Reviews. Participants and facilitators will explore methods in analyzing and making PROs readily understandable to users and consumers of Cochrane reviews. These challenges involve making decisions on interpreting the magnitude of an effect (e.g., whether differences are trivial, small but important, or large). We will introduce approaches to pooling across different instruments measuring the same construct with emphasis on the concept of the MID (the smallest difference that would motivate a patient to use an intervention), allowing the dichotomization of continuous outcomes (e.g., the proportion of patients who achieve a MID). We will present methods for identifying and evaluating the credibility of MID estimates. During the workshop, issues will arise for discussion in breakout groups. At the end of these breakout sessions, each group will report back with the group’s conclusions. This workshop is also recommended for participants attending a second workshop that will focus on evaluating MID credibility and the application of presentation approaches that rely on the MID in systematic reviews and clinical practice guidelines.

11:00 AM - 12:30 PMConducting a Cochrane Methods Peer Review – Good Practice and Common ChallengesWorkshop - training

Background: Cochrane is moving towards a central editorial service that clearly separates author and editorial roles in the evidence synthesis production model. As part of this process, the Cochrane Central Editorial Service sends every Cochrane review for consumer, clinical, search and methods peer review prior to publication. This robust process ensures that evidence published is accurate, reliable and useable. To ensure that this process remains efficient and useful for authors and results in high-quality systematic reviews, the peer reviewer reports should be of the highest standard, clear, consistent, and appropriately worded.
Objectives: The objectives of this workshop are to highlight good practice in the creation of Cochrane Methods Peer Review reports; to provide practical, hands-on guidance to help editors identify and address common challenges during the peer review process; and to discuss the current opportunities available for getting involved in conducting editorial peer review.
Description: The workshop will begin with a brief PowerPoint presentation, introducing the methods peer review process conducted by the Central Editorial Service, highlighting the key features of a good practice peer review report, and providing an overview of the most common challenges currently facing editors during the peer review process. The issues discussed will include (i) how to differentiate between when authors have made an objective error that requires ‘correction’ and a subjective decision that requires clearer ‘justification’, (ii) appropriate and consistent wording of the comments, and (iii) how to deliver consistent and appropriate recommendations on whether each article should be rejected, accepted or revised. Following this, the attendees will work in small groups with the facilitators to identify challenges in some real-life examples and discuss the best way to rectify and ultimately avoid the issues. The workshop will conclude with an open discussion regarding the current opportunities available to authors and editors who are interested in getting more involved in the editorial peer review process.

11:00 AM - 12:30 PMCochrane-Wikipedia Initiative: Hands-on workshop to help improve health content that people are accessing onlineWorkshop - training

Background: Improving the quality and reliability of online health articles that are accessed regularly across the internet is a significant challenge. Millions of people access medical articles on Wikipedia each day across 286 different languages. On English language Wikipedia alone, 40,000 human health and medicine articles are viewed billions of times a year, often surpassing the viewership of more reliable sources such as the World Health Organization and Centers for Disease Control and Prevention for topics including information about COVID-19. Cochrane partnered with Wikipedia in 2014 with the joint goal of improving the reliability of health information accessed on Wikipedia. As of 2023, 3,400 Cochrane Systematic Reviews published in the past 10 years help inform evidence in medical articles shared on English Wikipedia. There are additional ongoing initiatives in other languages including Spanish, Dutch, French, Malay, and German Wikipedias. Many Cochrane community members and volunteers participate in this initiative, collaborating with the Wikipedia editing community and using high-quality and trusted evidence to correct errors of omission, expand medical articles, and remove unreliable or outdated content shared on Wikipedia.
Objectives: The first objective of this workshop is to encourage more people with a background in interpreting and sharing evidence to consider Wikipedia in their knowledge translation and dissemination projects. The second objective is to enable these individuals with the skills that they need to move forward together and improve this commonly accessed resource using trusted evidence.
Description: This workshop is a hands-on session that will introduce the project and provide an opportunity for participants to edit Wikipedia with in-person support from Cochrane’s Wikipedian in Residence. Participants will have the following learning objectives: 1) describe how health-related evidence is shared on Wikipedia; 2) understand how to create a personal Wikipedia editing account; and 3) be able to add evidence from at least one trusted source to a health-related Wikipedia article.

11:00 AM - 12:30 PMA hands-on introduction to NMAstudio: a web-application to produce and visualize network meta-analysesWorkshop - training

Background: Evaluation of network meta-analysis (NMA) evidence can be challenging and time-consuming, especially when dealing with large networks of interventions. Despite the fact that NMAs are increasingly adopted to inform decision-making, the lack of user-friendly software is still limiting their usage to a restricted community of users.
Objectives: To present NMAstudio, a novel fully interactive web application that simplifies the NMA appraisal, while facilitating prompt production, interpretation and sharing of key NMA outputs.
Description: This workshop offers the opportunity for the first hands-on practice with NMAstudio. A dataset will be provided to the participants and will be used to demonstrate the available features of our tool. The participants will upload the data and interact directly with an NMA network diagram. By clicking one or more nodes-treatments or edges-comparisons, we will show how NMAstudio can assist the users in each of the fundamental steps of NMA. In particular, we will show how to produce customized outputs such as boxplots of effect modifiers to check transitivity; pairwise, NMA and bi-dimensional forest plots; league tables coloured by risk of bias or confidence ratings from the CINeMA framework; incoherence tests; ranking plots and more. Importantly, we will also illustrate how to generate an access token that can be used to save the project and share it publicly. Overall, this workshop is intended to demonstrate how NMAstudio can help investigators interested in conducting a network meta-analysis, by enhancing the NMA findings while assisting with the interpretation of results and evaluation of the modelling assumptions. Note: Participants should preferably bring their own laptop to the session and/or can share with other participants.

11:00 AM - 12:30 PMSearching for studies for inclusion in Cochrane Reviews: a core Cochrane methods introductory workshop for Cochrane Review authors and othersWorkshop - training

Background: Searching for studies forms the basis of any systematic review, and the effectiveness of the searching in identifying the relevant studies for inclusion in the review has an impact on the reliability of the results of the review.
Objectives: To provide Cochrane Review authors with guidance on searching for studies for inclusion in a Cochrane Review. Participants will gain an understanding of some of the issues and challenges involved, as well as practical advice on successfully conducting searches and working with librarians and information specialists to undertake searches.
Description: Questions to be considered will include the following: where to search for studies, how to search efficiently and what to do with the search results. This session is primarily aimed at Cochrane Review authors and will focus on Cochrane policy on searching for studies as described in the Cochrane Handbook for Systematic Reviews of Interventions (Searching for studies) and the Methodological Expectations of Cochrane Intervention Reviews (MECIR) standards (methods.cochrane.org/mecir). The workshop will highlight each overall area addressed in the above chapter in the Cochrane Handbook for Systematic Reviews of Interventions, including: - developing a search strategy using patient/population, intervention, comparison and outcomes (PICO); - sensitivity versus precision; - Boolean operators and controlled vocabulary; - going beyond MEDLINE/PubMed (identification and comparison of resources); - organizing and managing the search results and documenting and reporting the search process. The facilitator is the founding Co-Convenor of the Cochrane Information Retrieval Methods Group (IRMG) and lead author of the Cochrane Handbook for Systematic Reviews of Interventions chapter on searching for studies, and they also co-led the searching for studies element of the MECIR initiative. The session will include presentations, demonstrations, group discussion and hands-on exercises. An interactive approach will be adopted, and questions and comments will be encouraged throughout. Participants should bring their own laptops and be prepared to share with those who are unable to do so.

11:00 AM - 12:30 PMConducting Rapid Qualitative Evidence Synthesis: Why, when and how?Workshop - training

QMIG Methods workshop:
Background: As we learned from the COVID-19 pandemic, sometimes questions need to be answered quickly in order to be useful for policy and practice. This applies equally to questions answerable by qualitative evidence synthesis. However, providing evidence quickly may carry implications for how systematic and rigorous the synthesis is and/or confidence in the review findings.
Objectives: We aim to help attendees at this workshop to: Consider why and when a rapid approach to qualitative evidence synthesis (QES) could be useful; Replicate important decisions to minimise the impact of speed on review quality; and Consider the differences between a rapid review, QES and a rapid QES.
Description: The presenters, co-convenors of the Cochrane Qualitative and Implementation Methods Group, will draw on working examples of a rapid QES published in Cochrane to illustrate key decisions that are needed when conducting rapid QES. We will use problem-solving activities with feedback to help attendees prepare for conducting a rapid QES. This workshop is intended for those with some experience in systematic review or QES who are interested in learning more about developing a flexible toolkit of ways to conduct evidence synthesis that optimise timeliness and rigour.

11:00 AM - 12:30 PMSystematic reviews of diagnostic test accuracy studies: Introduction to meta-analysisWorkshop - training

Background: For making informed decisions about medical tests, healthcare organisations and policymakers typically rely on information from studies that assess diagnostic accuracy, i.e., how well a test gets the diagnosis right in people who have and those who do not have the target condition. Therefore, it is essential that valid methods are used to produce high-quality evidence reviews that will be used to inform recommendations for patient care. This core workshop organised by the Cochrane Screening and Diagnostic Tests Methods Group is designed for those who have a keen interest in gaining insight into test accuracy meta-analysis and how to use Review Manager (RevMan) to produce graphical summaries.
Objectives: To introduce the concepts underlying test accuracy meta-analysis and recommended statistical methods and to demonstrate graphical capabilities within RevMan.
Description: Using published examples, participants will learn of key challenges, best practice for conducting test accuracy meta-analysis and how to present results. Specifically, the workshop is based on the guidelines formulated in the new Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy. The workshop will be delivered through an interactive presentation, discussions and practical demonstrations using RevMan Web. Participants are expected to bring a laptop. Those who do not have a laptop will still benefit from the onscreen demo and discussions. Prior to the workshop, participants will be sent the RevMan file that will be used during the workshop.

11:00 AM - 12:30 PMSystematic reviews of prognosis studies II: Risk of bias assessment in systematic reviews of prognosis studiesWorkshop - training

Background: Prognosis is a description of the probable course of individuals with a health condition. Review and synthesis of overall prognosis, prognostic factor and prediction model studies is a relatively new and evolving area. Critical appraisal of prognosis studies is challenging but essential to assess and identify biases sufficiently large to distort study results. The Quality in Prognostic Studies (QUIPS) and Prediction model Risk Of Bias ASsessment Tool (PROBAST) tools are useful and reliable to assess risk of bias in studies of prognostic factors and prediction models, and the RoB-OPS tool is being developed for assessment of overall prognosis studies. The QUIPS tool can be used to assess risk of bias in studies of prognostic factors. It contains six important areas to evaluate the validity of prognostic factor studies and includes prompting items related to these six areas with suggestions for operationalization and grading. PROBAST is the tool for assessing the quality and risk of bias of prediction model studies. It consists of four domains and 20 signaling questions.
Objectives: This workshop will familiarize participants with sources of bias in overall prognosis, prognostic factor and prognostic model studies and introduce participants to the QUIPS, PROBAST and RoB-OPS tools.
Description: This interactive workshop will inform and train participants on systematic reviews of overall prognosis, prognostic factors and prognostic models. The workshop will be split into two sessions. The first session will give an overview of the different critical appraisal tools and guidance on how to use them. In the second part of the workshop, participants will have the opportunity to practice with these tools and interpret the results.

12:30 PM - 1:15 PMRapid Reviews Methods Group (First half) - By Invitation OnlyMeeting

In this meeting we would like to discuss the ongoing work of the Cochrane Rapid Reviews Methods Group and use the chance to finally meet in person again

12:30 PM - 1:30 PMMeet the EditorsNetworking session

Grab your lunch and take a moment to meet the Editors of the Cochrane Handbooks. Simply join them at the designated table for an informal chat and ask them any questions you may have.

The Editors will be available at:

  • 12:30 – 13:00: Editors of the Cochrane Handbook for Systematic Reviews of Interventions and the Cochrane Handbook for Qualitative Evidence Synthesis
  • 13:00 – 13:30: Editors of the Cochrane Handbook for Systematic Reviews of Prognosis Studies
12:30 PM - 1:45 PMSpeakers of the special session - Research and Publication Integrity - By Invitation OnlyMeeting

This is a closed meeting to debrief following the plenary session.

12:30 PM - 2:00 PMLunch break and meetingsLunch break 
12:30 PM - 2:00 PMConsumer Executive Meeting - By Invitation OnlyMeeting

Meeting of the Cochrane Consumer Network Executive 

12:30 PM - 2:00 PMPoster session 2Poster session

During this session the following posters will be presented:

12:30 PM - 2:15 PMCochrane FieldsMeeting

Exchange with all Field members and others interested in Field activities.

1:00 PM - 1:45 PMNAVIGATE Meeting - By Invitation OnlyMeeting

This meeting is with collaborators from both JBI and Campbell organisations. It is a research meeting to plan future work.

1:00 PM - 1:45 PMQualitative and Implementation Methods Group Meeting - By Invitation OnlyMeeting

Strategy Meeting

1:00 PM - 1:45 PMMultimodality methods meetingMeeting

The multimodality methods group seeks to inform and guide the Cochrane systematic review community in combining multiple types of evidence, displayed in a variety of different forms and using multiple dissemination channels to make reviews more comprehensive and inclusive. We would like to extend the range of formats used in the production and dissemination of reviews. This means including different forms of evidence (numerical, narrative and sensory data), vary with form in presenting insights or using different formats to communicate with the public, this may be inclusive of creative methods such as performance and art. We initiate this methods group based on our collective interest in the value of information carriers beyond the printed page that represent evidence and what they might contribute to the inclusivity agenda from Cochrane. It is our aim to respond to information and presentation needs from different populations currently involved or who might become involved in Cochrane. We invite you to hop on board!

1:00 PM - 1:45 PMTranslation project managers - By Invitation OnlyMeeting

Closed meeting with Cochrane's translation teams.

1:00 PM - 1:45 PMCentral Editorial Service Sign-off Editors MeetingMeeting 
1:00 PM - 1:45 PMBias Methods GroupMeeting

 

The meeting will involve a short summary of the groups work in 2022 and scientific presentations and discussions with focus on ROBINS-E and the revised ROBINS-I.

1:00 PM - 1:45 PMInformation Specialists - By Invitation OnlyMeeting

We will discuss issues relevant to Cochrane Information Specialists and other information specialists working on Cochrane reviews.

1:00 PM - 1:45 PMGELA - By Invitation OnlyMeeting 
1:00 PM - 2:00 PMCochrane Skin Meeting - By Invitation OnlyMeeting

Meeting of the group to present future orientations and organisation

1:00 PM - 2:00 PMEvidence Synthesis Taxonomy InitiativeMeeting

The JBI Evidence Synthesis Taxonomy Initiative aims to develop a comprehensive evidence synthesis taxonomy to encourage and facilitate appropriate research synthesis to inform policy and practice. This meeting will discuss progress, next steps and a chance to collaborate further.

1:00 PM - 2:00 PMMeet the Thematic GroupsNetworking session

Grab your lunch and take a moment to meet representatives of the Thematic Groups. Simply join them at the designated table for an informal chat and ask them any questions you may have.


The following Thematic Groups will be available from 13:00 to 14:00:

  • Global Ageing
  • Health Equity
  • Heart, Stroke and Circulation and Hypertension
  • Nutrition and Physical Activity
  • Person Centred Care, Health Systems and Public Health
  • Sexual and Reproductive Health
  • Work, Health and Social Security 
1:00 PM - 2:00 PMMeet the Author Journey Lead: Find out how to propose a new or updated reviewNetworking session

Cochrane welcomes proposals for new and updated Cochrane Reviews. Find out more about how to propose a new or updated review, and what the topic experts that consider proposals are looking for. All questions very welcome.

1:15 PM - 2:00 PMRapid Reviews Methods Group (Second half)Meeting

In this meeting we would like to discuss the ongoing work of the Cochrane Rapid Reviews Methods Group and use the chance to finally meet in person again.

2:00 PM - 3:30 PMCo-production and co-designOral session
14:00 Opening remarks and introduction
14:05 Using a co-design approach to increase the awareness, use and engagement of a digital map of COVID-19 recommendations
14:15 Lessons learned from developing an evidence-based guideline for psychosocial support by Red Cross volunteers: Involving end users throughout the process
14:25 Systematic reviews to inform health and social care policy: Incorporating patient and public perspectives
14:35 Co-Producing a resource for intervention development: The Nature on Prescription Handbook.
14:45 The power of storyboarding as an analytical tool in QES: From review to fieldwork to dissemination
14:55 Method and tools for implementing comparative effectiveness research: a Dutch example
15:15 Producing Cochrane reviews with multi-stakeholder synergic involvement. An experience with treatments for multiple sclerosis
15:25 Closing remarks
2:00 PM - 3:30 PMPriority setting and partnershipsOral session
14:00 Opening remarks and introduction
14:05 Prioritising topics, formulating questions, and selecting outcomes for a World Health Organization nutrition guideline
14:25 Identifying guideline needs for newborn and child health in South Africa, Malawi and Nigeria: a priority-setting exercise
14:35 Prioritising Cochrane reviews on the prevention and treatment of obesity: a mapping and stakeholder consultation project
14:45 Working together to prioritise and synthesise evidence about promising alternative models of healthcare delivery for a more sustainable health system
14:55 Health and Care Research Wales Evidence Centre: Mapping policy evidence-needs to support evidence-based decision-making
15:05 AHRQ Evidence-based Practice Center and NIH Pathways to Prevention programs: A decade-long partnership in closing evidence gaps
15:15 Closing remarks
2:00 PM - 3:30 PMOverviews of reviews, scoping reviews and network meta-analysisOral session
14:00 Opening remarks and introduction
14:05 NMAstudio: a fully interactive web-application for producing and visualizing network meta-analyses
14:15 Rank‐Heat Plot: An R Shiny tool for presenting multiple network meta-analysis outcomes
14:25 How to extract, analyse and present results in scoping reviews
14:45 Scoping reviews for WHO guidelines: using a two-tiered search strategy to balance efficiency and comprehensiveness
14:55 Preferred Reporting Items for Overviews of Reviews: Development and Dissemination of the PRIOR Statement
15:15 Mismatch between overall and pairwise overlap analysis in a sample of overviews: A methodological review
15:25 Closing remarks
2:00 PM - 3:30 PMBias and certainty of evidenceOral session
14:00 Opening remarks and introduction
14:05 Introduction to a new risk of bias tool for network meta-analysis (RoB NMA tool)
14:25 Ten Top Tips for using the Risk of Bias 2 (RoB2) tool
14:35 The revised JBI critical appraisal tool for the assessment of risk of bias for randomized controlled trials
14:45 Risk of bias tools: a systematic review of usability
14:55 Challenges in using RoB2: worked example from a systematic review of remote therapies for alcohol and drug misuse.
15:05 Introducing LATITUDES, a library of assessment tools and instruments used to assess data validity in evidence syntheses
15:15 Is Quality Appraisal Enough? Confidence in Heterogeneous Interventions’ Effectiveness
15:25 Closing remarks
2:00 PM - 3:30 PM30 Years of the Cochrane Collaboration: why and how did it get started? A conversation with some of those who were there at the beginningSpecial Session

As we celebrate 30 years since the founding of the Cochrane Collaboration, Karla Soares-Weiser and Jimmy Volmink discuss with some of the founders of the Collaboration what motivated them and others who co-founded the organization. Karla and Jimmy are joined by Iain Chalmers, Muir Gray and Jini Hetherington among others in this look back at the formation of the Collaboration in 1993. Answers to a range of questions explore the collaborative ethos that inspired the creation of a global network to prepare and maintain systematic reviews of research to inform decisions in health care.

Target audience: Anyone interested in starting a revolution is welcome

Format: An informal panel conversation

2:00 PM - 3:30 PMResearch and Publication IntegritySpecial Session

Identifying biomedical and clinical research manuscripts and publications with serious integrity problems helps to generate reliable evidence from the literature. This session will continue discussions from the earlier keynote session on research integrity, which will offer perspectives on paper mills, tools to identify fraudulent studies, and how systematic reviewers and journals can improve research and publication integrity. This Special Session aims to elicit further views and discussion from the audience on these and other challenges, to inform efforts to find and implement workable solutions. The session will conclude by describing how attendees can join efforts to improve the reliability of published biomedical and clinical research. 

Target audience: The Special Session will be open to all delegates. Attendees will not need to have joined the keynote session on research integrity to attend this session.

Format: The session will commence with brief opening comments (5-10 minutes) to introduce the topic to attendees who did not attend the earlier keynote session. We then wish to maximize audience participation and discussions whilst retaining some structure so that we can capture audience opinions and ideas (60 minutes). The session will conclude with a brief summary of the major points raised and describe how attendees can help to improve the reliability of published biomedical and clinical research (10 minutes).

Speakers: Jennifer Byrne (Co-chair), John Carlisle (Co-chair), Lisa Bero, Cyril Labbé, Limbanazo Matandika, Richard Van Noorden and Gowri Gopalakrishna.

2:00 PM - 3:30 PMConsidering scope in the planning and completion of systematic reviewsWorkshop - training

Background: A burgeoning issue in evidence-based medicine is the scope and scale of how research questions are formulated. The scope of the review question and formulation of objectives have major implications on several stages of the systematic review process. This workshop will inform authors on how to consider scope when planning and conducting a systematic review.
Objectives: (1) To evaluate how the scope of a research question and objectives need to be accounted for in the inclusion criteria of a systematic review. (2) To formulate analysis plans that target the objectives within a review and how analyses can be tailored based on the scope of a review question. (3) To consider how scope can impact the structure of a review in terms of writing up the results and summary versions of a manuscript.
Description: This workshop will begin with a 5-minute introduction to the topic of scope and various examples of scope impacting review development. There will then be an activity during which participants will work in groups to plan a review on a generic topic. Ideally, if there were 50 participants, there would be 10 groups of 5, with half the groups working on a review topic in which the scope is too large and the other half working on a review topic in which the scope is appropriate. The activity will be divided into three 20-minute sections: 1. The first section will be on assessing an existing patient/population, intervention, comparison and outcomes (PICO) inclusion criterion for the review question and objectives. Participants will be asked whether they think the inclusion criteria is appropriate and what they would change. 2. The second section will deal with analysis plans. Participants will be asked what comparisons they would consider for the review, types of analyses, and whether all outcomes included in the review need to be analysed across all comparisons. 3. The final section will require participants to evaluate the results section and summary versions of the review. The final 25 minutes will be devoted to feedback from the activity and lessons learned from existing reviews in which scope has implicated review development.

2:00 PM - 3:30 PMMetaInsight: An interactive user-friendly “point and click” web application to conduct network meta-analysisWorkshop - training

Background Network meta-analysis (NMA) allows the comparison of multiple competing interventions for a condition. Such analyses have become very popular due to their relevance for clinical decision-making. MetaInsight is a freely available user-friendly web-app with a “point and click” interface, including novel graphical displays to aid understanding and facilitate effective communication of the (often complex) results of an NMA to a range of stakeholders, including patients and decision-makers. It is actively used worldwide, with approximately 600 user hours per month, and is continuously being developed according to user needs. Objectives i. To provide an interactive demonstration of the functionality of MetaInsight v4 including user-interactive features and novel graphical visualisations; ii. To provide a user’s experience of using MetaInsight for conducting network meta-analyses for National Institute for Health and Care Research (NIHR) and Cochrane reviews; iii. For participants to gain hands-on experience of MetaInsight by completing a worksheet which will guide them through the analysis of an NMA dataset using the app. Description This workshop will provide an introduction to MetaInsight for conducting NMA, including the perspectives of a current user of the app, and demonstrate the wide array of functions available within the app as well as ‘tips & tricks’ for running an analysis. Participants will be provided with ‘hands-on’ experience of using the app. They will be given an example dataset and worksheet designed to guide them, at their own pace, through the stages of conducting an analysis in MetaInsight. Topics covered will include the following: loading data; obtaining statistics using both Frequentist and Bayesian analyses; visualising the NMA results, including ranking statistics, using novel visual graphical displays; conducting sensitivity analyses; and a new feature for exporting a report of the analysis including all data inputs, results tables and figures, facilitating analysis transparency. There will be plenty of time provided for questions, discussion and feedback. Finally, there will be a brief discussion of the current limitations of the app and future planned developments. Participants will be required to bring a laptop or tablet or may share with another participant.

2:00 PM - 3:30 PMHow to do a rapid review – updated methods guidanceWorkshop - training

Background: Having quick access to systematically produced evidence syntheses is crucial in addressing urgent health questions. The Cochrane Rapid Review Methods Group (RRMG) has published interim guidance to facilitate the conduct of Rapid Reviews (RR) both within Cochrane and beyond (1). This guidance aims to provide support and direction for conducting effective and efficient RRs. Three years later, the RRMG updated the guidance integrating new knowledge on the conduct of RRs and feedback from users of the interim guidance.
Objectives: In this workshop, members of the RRMG will present the updated RR methods guidance.
Description: In a mix of short lectures, hands-on work, and examples of good practice and common pitfalls, participants will learn how to plan and conduct an RR according to the updated guideline. Learning outcomes
• Workshop participants know the recommendations from the updated Cochrane RR Methods Guidance.
• Workshop participants can use the guidance in practice. Relevance and importance to patients: The updated guidance from the Cochrane RRMG on RRs plays a crucial role in ensuring the high methodological standards of RRs, thereby impacting health care decisions and patient outcomes. Facilitators: Chantelle Garritty, Candyce Hamel, Declan Devane, Ursula Griebler, Gerald Gartlehner, and Barbara Nussbaumer-Streit will co-facilitate this workshop. Reference: (1) Garritty C, Gartlehner G, Nussbaumer-Streit B, King VJ, Hamel C, Kamel C, et al. Cochrane Rapid Reviews Methods Group offers evidence-informed guidance to conduct rapid reviews. J Clin Epidemiol 2021;130:13–22.

2:00 PM - 3:30 PMApplying the Instrument to assess the Credibility of Effect Modification Analyses (ICEMAN) in a meta-analysisWorkshop - training

BACKGROUND: Subgroup analyses are relatively easy to perform but can be difficult to interpret. Credibility is often low, whereas the risk of over-interpretation is high. ICEMAN is a structured approach generated through expert consensus and provides eight items for judging the credibility of subgroup effects identified in a meta-analysis. OBJECTIVES: Learn how to apply ICEMAN to an apparent subgroup effect and, if credible, how to design a corresponding GRADE summary of findings table. DESCRIPTION: Participants will apply ICEMAN to examples from the literature. We will discuss, for each item, conceptual and practical challenges. The workshop does not include statistical exercises; the focus is on the interpretation and reporting of subgroup effects. Participants should bring their laptop, if available, and are welcome to discuss their own examples. Key references: (1) Schandelmaier et al. CMAJ, 2020: Doi: 10.1503/cmaj.200077 (2) Guyatt et al. J Clin Epidemiol, 2023: Title: “GRADE Guidance 36: Updates to GRADE’s approach to addressing inconsistency” (in press)

2:00 PM - 3:30 PMLearning by doing: Introducing Cochrane ClassmateWorkshop - training

Background: Built on the premise that people often learn best by doing, Classmate was built to enable teachers of evidence synthesis to use the tasks available on Cochrane Crowd in their teaching environments. Classmate now offers trainers and teachers a range of tasks and learning activities that relate to producing and understanding high quality health evidence. The process is simple: a teacher logs into Classmate and selects the task or tasks they want their students to do. They set the timeframe in which the task is to be done and then invite their students to the task. They can put their students into groups or teams, and they can monitor their students’ progress in the task. Classmate enables teachers to enhance or supplement their teaching; it provides students with the opportunity to contribute to real and needed tasks; and it helps Cochrane keep up to date with the ever-increasing amounts of primary research being produced.
Objectives: Workshop participants will be introduced to the latest version of Classmate. This will include understanding what the platform does and how it works, as well as an introduction to the microtasking and microlearning concepts that are fundamental to both Classmate and Cochrane Crowd. Participants will also have the opportunity to set up and take part in a Classmate learning activity (and win some prizes!).
Description: The workshop will begin with a series of short presentations introducing both Cochrane Crowd and Classmate. These presentations will cover what each platform does and the range of possible tasks available from study design identification tasks to PICO (population, intervention, comparison, outcome) extraction tasks. For the remaining time, the workshop will be highly interactive. In teams, participants will be put into teams and join a live task, which we will set up together in the workshop using Classmate (crowd.cochrane.org/classmate). The workshop is open to all but likely to be especially appealing to consumers, trainers, teachers, and students.

2:00 PM - 3:30 PMIdentifying who benefits most from treatments: How to analyse, present and interpret interactions and subgroup effects in meta-analysisWorkshop - training

Background: Researchers often wish to identify which individuals benefit more (or less) from interventions; this idea underpins the concept of stratified medicine. As single studies are typically underpowered for exploring whether participant characteristics determine an individual’s response to treatment, meta-analysis can provide a solution. Whilst individual participant data provide the most power and analytical flexibility to investigate interactions between such characteristics and the intervention effect, aggregate data (AD) can also often be used. However, approaches to the analysis, presentation and interpretation of interactions vary widely.
Objectives: In this workshop we aim to demystify interactions in meta-analysis and show how they can be explored using AD. Participants will (i) understand the purpose of subgroup and interaction analysis in trials and meta-analysis, as well as its strengths and limitations; (ii) explore the concept of aggregation bias and its consequences for interaction testing; (iii) extract and calculate a simple “within-trial” interaction effect using AD; (iv) understand how to use AD trial data to calculate subgroup effects compatible with “within-trial” interaction effects; and (v) explore how to present these interactions and subgroup effects clearly using novel implementations of forest plots. Material and examples are taken from Fisher et al., BMJ 2017 doi:10.1136/bmj.j573, and Godolphin et al., Research Synthesis Methods 2023 doi:10.1002/jrsm.1590.
Description: The 90-minute workshop will consist of short slide presentations, group discussion and practical activity. We will begin by considering subgroups and interactions within a single randomised trial. Using real examples, participants will discuss interpretation of results and what can and cannot be concluded from the given data. Participants will learn how interactions from multiple trials may be pooled using meta-analysis. We demonstrate how this can be done with AD, using examples from the literature. Working in small groups, participants will perform a simple AD interaction analysis and discuss results and interpretation. For clinical decision-making, subgroup effects are key; therefore, we will also show how to use the entirety of available trial data to estimate subgroup effects consistent with the ‘within-trial’ interaction. Finally, participants will explore how to present interactions and subgroup effects on forest plots and learn how to perform these analyses using Stata.

2:00 PM - 3:30 PMSystematic reviews of prognosis studies III: Meta-analytical approaches in systematic reviews of prognosis studiesWorkshop - training

Background: Prediction models are commonly developed and validated for predicting the presence (diagnostic) or future occurrence (prognostic) of a particular outcome. Prediction models have become abundant in the literature. Many models have been validated in numerous different studies. Also, numerous studies investigate the (added) value of a prognostic factor/predictor/biomarker to existing predictors. In both situations, aggregating such data is important for making inferences on the predictive performance of a specific model or predictor/marker. Meta-analytical approaches for both situations have recently been developed.
Objectives: This workshop introduces participants to statistical methods for meta-analysis in systematic reviews of prognosis studies. We will mainly focus on meta-analysis of the performance of a prognostic model. We discuss opportunities/challenges of the statistical methods and of common software packages.
Description: In this workshop, we illustrate these statistical approaches and how to combine, quantitatively, results from published studies on the predictive accuracy of a prognostic model. We illustrate this with various empirical examples. In the second part, participants will work in small groups to interpret the findings of a published meta-analysis.

2:00 PM - 3:30 PMVaccine efficacy/effectiveness: Calculation, visualization, and interpretation based on an exemplary systematic review on COVID-19 vaccination in childrenWorkshop - training

Background: Vaccine efficacy/effectiveness (VE) is commonly used to express the effect of vaccines to prevent diseases, e.g., COVID-19. Although the general formula for VE calculation is straight forward on the first glance [VE=(1-VE_ratio)*100], one has to take into account that several different VE-ratio measures are used in individual studies, which are potentially included in systematic reviews and meta-analyses. These include, for example, risk ratios, odds ratios, hazard ratios, or incidence rate ratios. This cannot only lead to questions in VE calculation but also in issues regarding interpretation of a pooled estimate of different VE ratios. Moreover, visualization of VE estimates in forest plots is not yet possible in RevMan Web resulting in the need for alternative approaches, e.g., using the software R.
Objectives: To explain, discuss, and interpret VE calculation, VE meta-analysis, and VE visualization in forest plots based on an exemplary systematic review on COVID-19 vaccination in children.
Description: In this workshop, we would like to address all participants interested in deeper insights in VE methodology and meta-analyses of VE outcomes (e.g., SARS-CoV-2 infection, hospitalisation). Previous knowledge on VE and statistics is advantageous, but not required. We will start with a short introduction on VE calculation. In group exercises, participants will calculate, interpret, present, and discuss VE for different outcomes and study designs. We will illustrate the clinical relevance by comparing VE with a calculated corresponding absolute effect for selected outcomes. Further, we will present different ways of VE visualization based on forest plots from different published examples, which we will discuss interactively with the workshop participants. We will demonstrate how to calculate and produce a VE forest plot in R, including subgroup and sensitivity analyses using simple, user-friendly R code from package ‘meta’, which has been implemented recently. The code will be made available to all participants after the workshop. After completing the workshop, participants will have a solid understanding of VE calculation, visualization, and interpretation, which will be useful for an enhanced understanding and for conducting systematic reviews on vaccination effects.

2:00 PM - 3:30 PMFinding, critically appraising, and using a core outcome set (COS) to inform your systematic reviewWorkshop - training

Background: A core outcome set (COS) is an agreed standardised set of outcomes that should be measured and reported, as a minimum, in all clinical trials in a specific health condition. This would allow research to be compared and combined, ensuring all studies contribute usable information for the core outcomes. The involvement of relevant stakeholders, especially patients and health professionals, in COS development helps ensure the outcomes important to those groups are included. Many organisations, including Cochrane, now actively endorse COS use in systematic reviews to help ensure outcomes important to patients and other key stakeholders are considered. The Cochrane Handbook for Systematic Reviews of Interventions acknowledges the relevance of COS to defining review questions and planning the review, helping ensure the inclusion of patient-centred outcomes. By maintaining the free, searchable Core Outcome Measures for Effectiveness Trials (COMET) database, the COMET Initiative can help systematic reviewers to identify and use COS. Among a sample of Cochrane reviews from 2019, 7% referred to a COS when choosing outcomes, yet 35% of the remaining reviews could have considered a relevant existing COS. Most Cochrane Review Group editors (86%) thought that COS should definitely or possibly be used to inform the selection of review outcomes.
Objectives: i. Describe the rationale for using COS in systematic reviews and demonstrate how the COMET database helps facilitate this. ii. Identify issues to consider when deciding whether a COS is applicable to a specific systematic review and whether a COS has been developed using reasonable methods.
Description: This interactive workshop will be co-facilitated by Paula Williamson (Chair, COMET Management Group), Ian Saldanha (Department of Epidemiology, John Hopkins), Maureen Smith (Chair, Cochrane Consumer Executive), and Heather Barrington (COMET Patient and Public Involvement Coordinator). Workshop participants will discuss how they choose outcomes for their review. Searching the COMET database will be demonstrated. Participants will explore COS examples, assess their relevance to particular systematic review topics, and discuss how they might assess COS relevance to their own work. The COS minimum standards will be presented, and participants will be invited to assess the example COS against these standards.

2:00 PM - 3:30 PMAssessing risk of bias in comparative diagnostic accuracy studies using QUADAS-CWorkshop - training

Background: Diagnostic Test Accuracy reviews addressing comparative questions may include studies directly comparing index tests in the same study group. Because these comparative accuracy studies aim to estimate the difference in accuracy between tests (rather than solely the accuracy of each test), we need to consider additional sources of bias specific to test comparisons. The QUADAS-C tool, an extension to QUADAS-2, considers these additional sources of bias and is endorsed by Cochrane for assessing risk of bias in comparative accuracy studies.
Objectives: To learn how to assess risk of bias in comparative accuracy studies using QUADAS-C.
Description: First, we will briefly discuss possible study designs for comparing the accuracy of two tests (10 min). We will then brainstorm, in small groups, which shortcomings could be present across the domains Patient Selection, Index Test, Reference Standard and Flow and Timing (20 min). Subsequently, the structure of and items in QUADAS-C will be presented (10 min). Lastly, we will evaluate a comparative accuracy study in small groups using QUADAS-C (30 min), followed by a plenary discussion of any discrepancies between assessors (20 min). Importance to patients: Although this concerns a methodological topic and does not contribute directly to patient health, using biased research may lead to misleading recommendations about diagnostic tests.

3:30 PM - 4:00 PMBreakCoffee break 
4:00 PM - 5:30 PMLiving evidence: innovations and technologyOral session
16:00 Opening remarks and introduction
16:05 Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) extension for living systematic reviews (LSRs)
16:25 Conceptualizing the reporting of living systematic reviews
16:45 Interactive website for communicating the results from a living systematic review and network meta-analysis for COVID-19
17:05 A living network meta-analysis of treatments for rheumatoid arthritis: Novelty at the frontier of living evidence
17:15 Integration of Machine Learning in a living systematic review of baseline risks of Venous Thromboembolism complications in hospitalized patients with COVID-19
17:25 Closing remarks
4:00 PM - 5:30 PMMethodological quality and evidence synthesis innovationOral session
16:00 Opening remarks and introduction
16:05 The problems with systematic reviews: working towards the solutions
16:25 Common methodological errors in submitted manuscripts for systematic reviews and protocols in public health
16:45 Specifying the ‘PICO’ for each synthesis: current practice and development of the InSynQ (Intervention Synthesis Questions) checklist and guide
16:55 Challenges and innovations in evidence synthesis and evaluation during the COVID-19 pandemic
17:05 Introduction to the WISEST (Which Systematic Evidence Synthesis is best) Project: Developing an automated clinical decision-support algorithm to choose amongst systematic review(s) on the same topic
17:15 The numerous typologies and taxonomies of evidence synthesis
17:25 Closing remarks
4:00 PM - 5:30 PMIndividual patient data meta-analysisOral session
16:00 Opening remarks and introduction
16:05 Maximizing the value of IPD when meta-analysing randomised trials
16:25 Power calculations to help plan individual participant data meta-analysis projects
16:35 Data sharing: experiences accessing individual participant data for a collaborative meta-analysis on umbilical cord management strategies at preterm birth.
16:45 Should aggregate data be included where individual participant data are unavailable? Lessons from a large individual participant data meta-analysis
16:55 Strategies for maximizing IPD retrieval in IPDMA: A mixed method study
17:05 Consolidating tools for assessment of risk of bias for individual participant data meta-analysis: a case study.
17:15 Data sharing policies across the health research lifecycle: a cross-sectional analysis of global funders, ethics committees, trial registries, journals and data repositories
17:25 Closing remarks
4:00 PM - 5:30 PMTeaching the public to understand and use evidenceOral session
16:00 Opening remarks and introduction
16:05 Developing digital resources to teach secondary school students to think critically about health information and choices: human-centred design
16:25 Teaching school children about evidence-based health care: Cochrane UK’s outreach programme
16:45 Piloting the Informed Health Choices resources in Barcelona primary schools: A mixed methods study
16:55 The People’s Review: a protocol for a systematic review powered by the public.
17:05 Level of scientific evidence needed to make an informed health decision among researchers, healthcare workers and consumers: a cross-sectional study
17:15 “Info without side effects” – Development and application of a checklist for lay users for evaluating online health information
17:25 Closing remarks
4:00 PM - 5:30 PMThe Art of Getting By: Leadership and Team Working StylesSkills Lab

The session will give you some insights into a generic leadership model that is applicable to running a country, a healthcare system, a university department, a research programme, a sports team, a family, even your own life. The team working style session looks at the four basic ways that members of a team interact. You will explore these and find out your own preferred working style and how to work better with different working styles, so you can get on with people better.

This is linked to a second session - The Art of Getting By: Influencing skills: Communication and Negotiation on Wednesday 6th September at 11.00-12.30, but can also be attended as a stand-alone session. Both sessions will be employing simple "heuristics" – ways of understanding complex situations and making better choices. They are so simple you can keep them in your head. The skills are generic, and you can employ them in your private life as well as your working life. You will even return home with a couple of heuristics to try out the same evening!

4:00 PM - 5:30 PMDiversity and Inclusion in CochraneSpecial Session

In the last few years, there has been an increasing debate on how scientific journals represent sex, gender, ethnicity and other aspects of human diversity. Moreover, the hegemony of the English language in scientific publications lays the foundation for structural inequalities for native speakers in other languages. Cochrane has conducted a listening and learning exercise to identify the challenges and opportunities in the area of Equality, Diversity and Inclusion (EDI). Now, we must work as an organization to promote inclusivity and minimize structural biases in our Collaboration. We selected some topics and recommendations to discuss in this special session.

Please read the following report from Cochrane: Cochrane launches report summarizing what we have heard from our diversity and inclusion listening and learning exercise.

Section one: lightening presentations (25 minutes)
Introduction: Juan and Vivian (5 minutes)
Theme 1: Engaging a diverse community of authors (5 minutes)
Theme 2: Minimizing biases in the editorial process (5 minutes)
Theme 3: Equity and priority in review topics (5 minutes)
Theme 4: Engaging a wide range of stakeholders (5 minutes)

Section two: World Café table discussions. 12 minutes per table plus 2 minutes moving between tables (based on the themes)

Section three: Feedback and wrap-up (10 minutes)
Moderators have a maximum of 60 seconds each to call out some key messages.
Juan and Vivian to close the session.

Target audience: Anyone with an interest in supporting diversity and inclusion

Format: A combination of presentations and discussion

4:00 PM - 5:30 PMMaintaining our place as a trusted health evidence producer: the Future of Evidence Synthesis programmeSpecial Session

The global health, research and publishing landscape has changed beyond recognition since our organization began in the early 1990s – giving rise to both complex challenges and new opportunities for Cochrane. To maintain our hard-earned place as one of the world’s most trusted health evidence producers, we need to adapt and directly address a diverse range of critical issues – from the loss of traditional funding sources and the move to a fully Open Access model, to the increasing demand for rapid evidence in new formats.

To ensure Cochrane’s future we are working to deliver a new model for producing Cochrane evidence syntheses through our Future of Evidence Synthesis programme.  This transformational programme of work will roll out over the next three to four years and is made up of ten targeted projects, encompassing changes to our organizational structure, software, review development and editorial processes, policies and the format of Cochrane Reviews.

This Special Session is an opportunity for people to understand better how key projects will impact on the Cochrane evidence synthesis ecosystem and what these changes will mean for people across the Cochrane Community. There will be an opportunity to ask questions about the programme and provide ideas that will contribute towards successful implementation. 

Target audience: Anyone interested in how Cochrane produces high-quality evidence synthesis in the future.

Format: Presentations plus discussion / Questions & Answers

4:00 PM - 5:30 PMProducing Trusted Evidence: Identifying the Challenges and Finding the SolutionsWorkshop - discussion

Background: Having mapped the nature, scope, and quality of evidence syntheses on COVID-19 to explore the relationship between review quality and the extent of researcher, policy, and media interest, we found low quality reviews being published at pace, often with short publication turnarounds. Poorly conducted systematic reviews can lead to inaccurate representations of the evidence, misleading conclusions, and reduced applicability, limiting their usefulness and ultimately contributing to research waste. Everyone in the research community (researchers, peer-reviewers, journal editors, funders, decision makers, clinicians, journalists, and the public) can help facilitate the conduct of robust systematic reviews that are published and communicated in a timely manner, reducing research waste and increasing the transparency and accessibility of all systematic reviews.
Objectives: - To highlight some of the key issues around the quality and volume of evidence synthesis research published. - To discuss potential solutions to these issues. - To prepare action plans to make changes to personal or organisational ways of working to implement solutions/tackle the key issues moving forward in practice.
Description: This 90-minute workshop will consist of: - A brief (10 min) presentation to outline the key issues identified from the research. - The audience will be split into small groups (we will aim for each group to include a mix of perspectives represented e.g., researcher, peer reviewer, publishers, and members of the public where possible). Each group will have a different issue and potential solutions to focus on and discuss. The discussion will be based around what changes individuals can make to help improve/resolve the issues highlighted and noting down achievable steps as part of a personal plan (20-30 mins). - The audience will then be asked to regroup into groups representing similar perspectives and continue to discuss the issues and solutions that could be made at an organisational level (relevant to their background/perspective) and again noting down potentially achievable steps (20-30 mins). - The final 20-30 mins will be used to share individual and organisational plans and encourage people to make an action plan at a personal and/or organisational level to facilitate change.

4:00 PM - 5:30 PMLessons from four years of Cochrane’s Methods Support Unit: what are we asked and how do we answer?Workshop - training

Background: Established in 2019, the Methods Support Unit (MSU) provides methods advice to editors and authors preparing Cochrane Reviews. We have helped people from 49 Cochrane Review Groups, dealing with more than 1,100 requests. These requests range from advice on a single methodological/statistical issue to a full review of a draft protocol or review. In addition, MSU has provided many hours of interactive training and authored guidance on key methods issues. As a result, MSU is in a unique position to understand the methodological and statistical concerns of Cochrane authors and editors and to provide accessible training of immediate usefulness.
Objectives: Our workshop will highlight the ‘Top Five’ issues encountered by authors and editors working on Cochrane reviews. These are the key issues that generate questions for the MSU team and that we encounter when reviewing manuscripts. We will explain how to handle these issues through a series of interactive exercises and discussions. As a result, this workshop will help to strengthen evidence production for the future by tackling ‘common errors’ and ensure Cochrane continues to produce robust and trusted evidence.
Description: We will use the most commonly asked questions as the basis for our workshop which will be delivered by experienced methodologists and statisticians. We will prepare materials and deliver a series of five 15-minute interactive sessions to teach attendees how to handle these issues. There will be plenty of time for questions and discussion. We will cover the following:
• Issues arising from including cluster and crossover randomised controlled trials.
• Use and misuse of the standardised mean difference effect measure.
• Risk of Bias 2.0: Bias due to deviations from the intended interventions: how is this different from the ‘Blinding of participants and personnel’ domain in Risk of Bias 1? What counts as a deviation from the intended interventions?
• How should the results of a network meta-analysis be summarised?
• Assessing imprecision in GRADE: what is an optimum information size? We did not involve patients or healthcare consumers in preparing this workshop, but we will pitch the session to be accessible to the widest possible audience.

4:00 PM - 5:30 PMStoryboarding as a multimodal analytical technique in a qualitative evidence synthesisWorkshop - training

Background: Multimodality is centralized around the fundamental question of how to use and combine modes of expression to organize, analyze, synthesize, interpret, and share research evidence. The use of multiple, engaging modalities, for example words in combination with images, establishes new audiences for research findings and promotes health equity by communicating research in an accessible, comprehensible way. Storyboarding is defined as the process of putting together images to arrange or illustrate a story in a specific sequence. This can be considered as a “visual text of images” (Hart, 2008; Naicker, et al. 2020). Visual storyboarding techniques can present themselves as a legitimate and rigorous method in a systematic review project (Hendricks et al. 2022). They can capture meaning, while at the same time embracing creativity, extending rigor, and increasing the review’s appeal for a broader public. This analytic technique is useful for working with studies which include photos and other types of data. See published QES: Hendricks LA, Young T, Van Wyk SS, Matheï C, Hannes K. (2022): https://doi.org/10.3390/ijerph191811317
Objectives: This workshop aims to familiarize participants with the storyboarding as an analytical technique in QES, with the practical exposure of working with multimodal review data using art materials and open access software.
Description: Participants will be introduced to types of multimodal data and the concept of storyboarding (5 min). Participants will be invited to write a short paragraph on a single experience (5 min). Following this, we will provide guided facilitation on using art for synthesis and break into small groups (5 min). The group will be tasked to create 6 block storyboard (templates provided) (30 min), with half of the small group using digital mediums and the other half using art materials, which they will then merge into a single storyboard (15 min). Following this, a group representative will be posted at the storyboard as the groups move around and listen to other stories (15 min). Participants will experience first-hand how storyboarding and art could be used to make sense of data in reviews.

4:00 PM - 5:30 PMInterpreting systematic review findingsWorkshop - training

Background: Systematic reviews are considered the gold standard of evidence for health professionals and are increasingly used to support decision-making in health care. Understanding review results is of paramount importance in translating research evidence into clinical practice. Critical thinking needs to be applied to read and/or use a systematic review, as interpretation of findings can be quite challenging when the available evidence is inconclusive or uncertain.
Objectives: 1) To introduce the concept of clinical significance; 2) to explain the rationale behind the certainty of the evidence; 3) to give a brief overview of the “Summary of findings” table; and 4) to reflect on translating the results of systematic reviews into clinical practice.
Description: This workshop will appeal most directly to new review authors and users of systematic reviews, including patients and their caregivers. We will give a short presentation at the beginning of the workshop (15 minutes) to outline the scope of the session and introduce key concepts. Then, participants will work in small groups under the guidance of the facilitators. We will try to ensure each group is formed by people with different backgrounds and expertise to bring different views and perspectives to the discussion. Based on specific clinical scenarios, we will use practical examples extracted from Cochrane reviews and Cochrane Clinical Answers. The groups will have time to read, think about the interpretation of the results, and discuss the implications for practice and/or further research. Each group will then share the work done with the participants. We will stimulate the debate on key issues, such as the relevance of outcome selection, basic interpretation of continuous and binary outcome measures, magnitude of the effects, importance of measuring heterogeneity, and impact of the risk of bias of included studies. One laptop per working group is needed.

4:00 PM - 5:30 PMAdvanced meta-analysis 1: Random-effects methods to be implemented in RevManWorkshop - training

Background: Meta-analysis is typically used to estimate the mean effect size of an outcome of interest. However, also of interest is estimation of the variability in the effect sizes (heterogeneity), because this allows assessment of the consistency of effects across studies and estimation of the likely effect of the intervention in an individual setting. The DerSimonian and Laird random-effects method is widely used (and is currently the only method available in RevMan) to estimate heterogeneity, however, its performance has long been challenged.
Objectives: To provide review authors with knowledge of new random-effects methods to be implemented in RevMan Web. This workshop is part of a series of workshops delivered by the Cochrane Statistical Methods Group and requires knowledge of the material presented at the workshop ‘Introduction to meta-analysis 2: dealing with heterogeneity’.
Description: Several alternative methods to the DerSimonian and Laird random-effects method have been proposed that have better statistical properties in specific scenarios. We will provide an overview of the new random effects methods to be implemented on RevMan Web. Specifically, we will introduce new methods to estimate the heterogeneity variance (and its confidence interval); calculate a confidence interval for the meta-analysis effect; and calculate a prediction interval. We will cover reasons for selecting a particular method; how the estimates should be interpreted; what the impact might be when updating Cochrane Reviews using the new methods; and what to write in a Cochrane protocol. We will use a combination of presentations, interactive exercises, and group work.

4:00 PM - 5:30 PMQuality Assessment of Diagnostic Accuracy Studies using QUADAS-2Workshop - training

Background: One of the key characteristics of a systematic review is the assessment of the quality of the included studies. If these individual studies are flawed, then the results of a meta-analysis and review in general are also flawed. To assess the quality of diagnostic test accuracy (DTA) studies, we will practise the use of the QUADAS-2 tool.
Objectives:
• Understand mechanisms that may lead to high risk of bias in DTA studies;
• understand the concept of applicability of DTA studies;
• understand additional sources of bias for comparative DTA studies;
• assess the quality of a DTA study using QUADAS-2; and
• interpret the findings of quality assessment.
Description: This workshop organised by the Cochrane Screening and Diagnostic Tests Methods Group is designed for individuals who are planning or currently undertaking a Cochrane DTA review, those involved in an editorial role or those who are using test accuracy evidence. The facilitators can tailor the workshop to accommodate the needs of individuals attending as far as possible but plan to address the following four topics:
• design and general aspects of diagnostic test accuracy studies;
• sources of bias and variability in diagnostic test accuracy studies;
• application of QUADAS-2 to a DTA study;
• a brief introduction to QUADAS-C; and
• reporting of the quality assessment results. The workshop will draw on the experience of the DTA Editorial Team as well as facilitators’ own experience of undertaking DTA reviews across a breadth of topic areas. The workshop will be delivered through a mixture of interactive presentations, discussions and small group exercises.

4:00 PM - 5:30 PMSystematic reviews of prognosis studies IV: The use of GRADE for assessing the certainty of evidence for questions of overall prognosis and prognostic factorsWorkshop - training

Background The GRADE approach is widely used for assessing certainty in inferences drawn from evidence. The GRADE approach has been adopted to assess certainty in evidence on overall prognosis and prognostic factors. Objective Present participants with an interactive session designed to understand and apply GRADE to evidence on overall prognosis and prognostic factors. Description We will engage participants on 10 topics related to the application of the GRADE for questions of prognosis. We will provide participants with examples from real systematic reviews. Participants will review the reviews and apply the GRADE guidance to determine the certainty. Study design: The participants will be engaged in discussions over the ideal study design when assessing evidence related to prognosis. Risk of bias: The domain will be introduced (high vs low risk of bias studies) with focus on considerations necessary for evidence related to prognostic factors (adjusted vs unadjusted associations). Inconsistency: We review the concept of inconsistency (and its relation to statistical heterogeneity). We will assess the impact of aberrant studies and their impact on the overall conclusion of the meta-analysis. Imprecision: The participants will review the different frameworks for assessment of imprecision, along with the difference between relative compared with absolute effect estimates. The workshop will provide an online calculator for determining absolute risks necessary for the assessment of imprecision under the partially contextualized framework. Indirectness: In this segment, the attendees will compare the research question of a systematic review to the characteristics of studies addressing the research question. We will provide guidance for circumstances when it is necessary to rate down our certainty for indirectness. Publication bias: Example of a systematic review will be provided in which the assessment of publication bias based on visual inspection (funnel plot) is different compared to the statistical test. Domains for rating up and special considerations with use of GRADE: Discussion on the three domains for increasing certainty, and challenges with making judgment with domains of GRADE that are highly interrelated.

4:00 PM - 5:30 PMHow to incorporate information on funding and conflicts of interest in trials when conducting a Cochrane Review: the TACIT toolWorkshop - training

Background: Industry funding and trial researchers’ financial conflicts of interest are common in clinical trials and are associated with favourable trial conclusions and seemingly greater treatment effects. Information on funding and conflicts of interest is therefore considered important when interpreting trial findings. The Cochrane Handbook includes a section on conflicts of interest based on the concepts behind a new Tool for Addressing Conflicts of Interest in Trials (TACIT) to be used in the context of systematic reviews, e.g., Cochrane Reviews. TACIT provides a framework for collecting funding and conflicts of interest information for a trial in order to reach a judgement on whether there is notable concern about conflicts of interest affecting the design, conduct, analysis or reporting of the trial, and the sufficiency of information this assessment is based on.
Objectives: To introduce participants to the concepts behind TACIT and provide practical experience with the tool.
Description: The workshop will consist of two parts.1) Introduction: The empirical basis and the background behind the development of TACIT will be briefly presented, and we will introduce the tool framework and key concepts. 2) Using TACIT: Through a practical exercise in small groups the participants will be introduced to the tool. Participants will be guided in the usage of TACIT from retrieving funding and conflicts of interest information to judging whether a trial has notable concern about conflicts of interest and the sufficiency of information available for this assessment. In plenary, we will discuss the assessments by the individual groups and how information from TACIT may further inform Cochrane Review data analysis and interpretation of results. Participants may bring a laptop to assist them in the exercise, but it is not necessary for participation. See www.tacit.one for further information about the tool.

4:00 PM - 5:30 PMHow to use and evaluate OpenAlex tools for efficient automated updating of systematic reviews and maps in EPPI-ReviewerWorkshop - training

Background: Keeping on top of new evidence is a perennial challenge. Systematic reviews can quickly become out of date, and maintaining surveillance of new evidence can be resource intensive and costly. For example, identifying new eligible studies for updating systematic reviews and maps of research—including those that have been transitioned to using a ‘living reviews’ approach—conventionally involves running updated Boolean searches across multiple electronic databases and manual screening of all retrieved unique records. However, new tools and technologies are beginning to address this challenge. First, the OpenAlex dataset—an open access, continually updated dataset and knowledge graph comprising more than 250 million records of research articles from across science—can be considered a ‘one stop shop’ for published research in many topic areas, reducing the need to search many, or sometimes any, subscription databases. Second, the scope of a review can be ‘learned’ by machine learning algorithms, facilitating efficient identification of eligible studies from within large datasets. OpenAlex tools in EPPI-Reviewer make use of these important enablers to support efficient updating of systematic reviews and maps, using precise automated searches of this potentially comprehensive single source with machine learning.
Objectives: To (i) introduce OpenAlex tools in EPPI-Reviewer; (ii) learn how to use these tools to set up automated study identification workflows for updating systematic reviews or maps of research; and (iii) discuss how to evaluate the ability of these workflows to efficiently identify new research.
Description: Participants will need a laptop with wireless connectivity in order to gain practical ‘hands-on’ experience of using OpenAlex tools in EPPI-Reviewer. The workshop will cover how to use these tools to set up an example Cochrane intervention review for prospective updating via automated searches of the OpenAlex dataset. We will also demonstrate options for managing and screening records retrieved by these searches. Time will be allocated to a discussion of how these tools and methods should be evaluated. This session will be of interest to information specialists, review authors, and others interested in learning about new approaches to automate and optimise the updating process.

7:30 PM - 11:30 PMSocial event at the Natural History MuseumSocial event

We’re delighted to be hosting this year’s social event at the Natural History Museum on Tuesday 5th September, starting at 7:30pm until late. 

After the museum closes its doors to the public, Cochrane Colloquium attendees will be granted exclusive access to the main entrance area, Hintze Hall. This hall has dramatic Romanesque arches and a majestic staircase. You’ll have the opportunity to explore the surrounding museum specimens in the Hall and dance beneath the suspended 25-meter blue whale skeleton named ‘Hope’.

Dinner, which will include bowl food and canapés, will be included and there will be an open bar (including selected drinks) for three hours, with the opportunity to buy additional drinks too.

In the spirit of giving back, the Colloquium organizers have requested a £15 donation during registration, to be given to St. Mungo's, a national charity based in London that works to prevent homelessness and aid individuals in their journey to recover from it. By encouraging donations, we hope to minimize non-attendance and food waste, while positively impacting the community.

Find out more.

Dress code: There is no dress code for the event. Please remember to bring your conference badge.

Time: Tuesday 5th September, 7:30pm until late. 

Guests: If you would like to bring guests, you’re welcome to buy additional tickets at £75 each (including VAT). To do so, please email registration@cochrane.org and let them know how many additional tickets you would like.

Address: Natural History Museum, Cromwell Rd, South Kensington, London SW7 5BD, UK

7:30 AM - 8:45 AMAnnual Methods Groups Convenors and Staff meeting - By Invitation OnlyMeeting

Each Cochrane Methods Group is dedicated to a specific method or review type relevant to Cochrane Reviews. Methods Groups are usually led by a small group of senior and experienced methodologies (Convenors and Co-convenors) with a wider membership of experienced and/or interested researchers. They are involved in methods research, defining best practices and supporting implementation in Cochrane. This meeting will help in the strategic planning of the Methods Groups activities for 2023-2024.

9:00 AM - 10:30 AMBuilding trust through co-creation: re-imagining evidencePlenary

Generating evidence through the combination of scientific and lived experience has the potential to enable mutual trust between research and society. The aim of this session is to spark a lively discussion about addressing the impact of experiential knowledge and through that, building deeper understandings and relationships between researchers, research participants and people involved.  In this session, we will explore why information and evidence as we know it may not be enough to support decision making, and reflect on examples that demonstrate ways to build trust and co-create evidence, with a discussion of differences as well as mutual learnings in the global south and global north.

Keynotes: 

  • Building Trust: Challenging division (Peter Beresford)
  • Co-creating Engagement and Evidence, Globally (Bella Starling)
  • From TB to COVID and Back Again: Community Engagement for Essential Evidence (Tasha Koch)
  • Using Creative and Participatory Methods to Research Prioritisation in Primary Care (Shoba Dawson)

This session will be chaired by Richard Morley, Cochrane's Consumer Engagement Officer and Maureen Smith, Chair of Cochrane Consumer Network Executive.

10:30 AM - 11:00 AMBreakCoffee break 
11:00 AM - 12:30 PMBuilding partnerships and implementing evidenceOral session
11:00 Opening remarks and introduction
11:05 From Evidence to Action: Implementing Cochrane Systematic Reviews Into Clinical Practice
11:25 Forward Together: An Evidence-Based Practice Partnership
11:35 Powering transformative change through collaborative action: The Healthcare Information for All Strategy
11:45 Learning effects of an evidence-based shared decision-making curriculum: the co-design and co-produce with clinical teachers, patients, and undergraduate medical students
11:55 Collective impact approach for more effective knowledge translation
12:05 Cochrane Rehabilitation Blogshot translation: a partnership model of evidence dissemination
12:15 A call to support Cochrane's social responsibility of improving health equity - the Equity Group
12:25 Closing remarks
11:00 AM - 12:30 PMCommunicating evidenceOral session
11:00 Opening remarks and introduction
11:05 Presenting the results of a systematic review to non-experts: the Dissemination Checklist
11:25 Developing infographics to communicate COVID-19 rapid review conclusions in South Africa
11:45 Trusting what you see: the importance of images shared with evidence for health decision-making and how to get it right
12:05 Plain language summaries of published health research articles: how well are we doing?
12:15 Comparison of linguistic characteristics of plain language summaries of systematic reviews between medical and non-medical sciences: a cross-sectional study
12:25 Closing remarks
11:00 AM - 12:30 PMCore outcomes and patient reported outcomesOral session
11:00 Opening remarks and introduction
11:05 Core outcome sets and Cochrane systematic reviews: Promising findings and opportunities for progress
11:25 Did randomized trials evaluating pharmacological treatments for non-severe COVID-19 prioritize patient-important outcomes in their results?
11:35 Mechanical ventilation core outcome set uptake in Cochrane systematic reviews. A cross-sectional study
11:45 Capturing the primary outcome for Cochrane reviews of deprescribing specific medications: an overview of Cochrane reviews
11:55 Patient organisations and the core outcome set revolution
12:05 Research waste caused by symptom-based eligibility: a systematic review of Cochrane reviews and their component trials
12:15 Discharged patients' PRO-AEs provide credible evidence of in-hospital adverse events
12:25 Closing remarks
11:00 AM - 12:30 PMInformation retrievalOral session
11:00 Opening remarks and introduction
11:05 Clinical trial register searches for systematic reviews of randomized clinical trials involving interventions
11:15 Searching clinical trials registers: guidance and recommendations for systematic reviewers
11:25 A comparison of different forward citation chasing tools for complementary searches for Cochrane systematic reviews
11:35 Understanding the health impacts of climate change: search filter development for exposure pathways
11:45 Novel Population Search Filter Development Methodology - the Black Persons Living in the United States Project
11:55 Can we reduce the workload of systematic reviews without compromising quality? A new platform to accelerate search and study identification
12:05 An open competition involving thousands of competitors failed to construct useful search filters for new diagnostic test accuracy systematic reviews
12:15 Information specialists - barriers and facilitators to career progression and methodology research
12:25 Closing remarks
11:00 AM - 12:30 PMBiasOral session
11:00 Opening remarks and introduction
11:05 Risk of bias and reporting quality of randomised controlled trials in paediatric pain: a cross-sectional study
11:15 Interventions during study design and conduct to reduce the risk of bias in interventional studies: a scoping review
11:25 Effects of allocation concealment and blinding in trials addressing treatments for COVID-19: A methods study
11:35 Comparison between randomized clinical trials designating an outcome as primary and those designating the same outcome as secondary
11:45 Trial Characteristics and Treatment Effect Estimates in Randomized Controlled Trials of Chinese Herbal Medicine: A Meta-Epidemiological Study
11:55 Impact of active placebo controls on estimated drug effects in randomised trials
12:05 A critical appraisal tool for systematic reviews of patient-reported outcome measures
12:05 Does type of funding affect reporting in network meta-analysis?
12:25 Closing remarks
11:00 AM - 12:30 PMGoing farther, faster together: Implementing the recommendations of the Evidence Commission reportSpecial Session

One year has passed since the publication of the report of the Global Commission on Evidence to Address Societal Challenges (henceforth the Evidence Commission). We see reasons for optimism, as well as reasons to double down on efforts to implement the report’s recommendations. While government policymakers in some countries (like those newly elected in some Latin American countries) are open to new approaches to decision-making and evidence use, many policymakers, organizational leaders and professionals have largely returned to pre-pandemic approaches. While some funders and donors and some impact-oriented evidence producers have piloted coordination mechanisms, many evidence producers continue to operate without coordination and to generate significant research waste. While many citizens have become more aware of the potential value of evidence, many others have become more distrustful of decision-makers and evidence.

This Special Session has two overarching objectives:

  • to provide an overview of the Evidence Commission’s three implementation priorities
    • formalizing and strengthening domestic evidence-support systems
    • enhancing and leveraging the global evidence architecture
    • putting evidence at the centre of everyday life
  • to explore ways to promote, contribute to and/or lead efforts to implement the Evidence Commission’s recommendations and implementation priorities.

The Evidence Commission Secretariat and its partners in 12 countries are conducting Rapid Evidence-Support System Assessments (RESSAs) and sharing lessons learned through the RESSA Country Team Leads Group. The goal in each country is to:

  1. identify what is going well that needs to be systematized and scaled up;
  2. identify gaps that should be prioritized to address; and
  3. work with decision-makers, organizational leaders, professionals and citizens to push for improvements.

The Evidence Commission Secretariat and its partners are exploring ways to develop one or more viable models and to seek funding and the support of evidence producers to pilot the model and then to scale it up based on lessons learned.

Efforts are also underway to partner with citizen-serving non-governmental organizations and citizen leaders to identify what works to:

  1. help citizens judge what others are claiming or more generally find (and receive) reliable information;
  2. make evidence available to citizens when they are making choices;
  3. engage citizens in asking questions and answering them (with new research or with existing evidence); and
  4. make evidence-based choices the default or easy option.

Target audience: Evidence producers, evidence intermediaries and Cochrane Consumers.

Format: This is a Special Session with brief presentations followed by interactive discussions with the audience to explore ways to promote, contribute to and/or lead efforts to implement the Evidence Commission’s recommendations.

11:00 AM - 12:30 PMHave your say on Cochrane’s future: Defining Cochrane’s next strategySpecial Session

Cochrane’s vision of a world of better health for all people, where decisions about health and care are informed by high-quality evidence, is more important than ever, but how do we decide where to focus our efforts when working towards this ambitious vision? The answer is Cochrane’s new strategy that will allow us to prioritize our efforts to make sure we are having the greatest impact possible.

We can’t do this without you! Cochrane is in the process of developing a new strategy to help us achieve our vision and mission, and to guide us through the opportunities and challenges of the next five years. Cochrane has always been a collaboration made up of brilliant people from all over the globe, so join us in this Special Session to have your say, which will help us produce a strategy that is meaningful to everyone in Cochrane.

This session will give you the chance to hear first-hand from the Chief Executive Officer Catherine Spencer, about the plans for a new organizational strategy and you will get the chance to engage in this critical process, allowing you to have your say on Cochrane’s future priorities and direction. There will be a mixture of presentations, questions and answer sessions, and small group conversations, to allow the greatest possible engagement from everyone present.

We want the strategy to reflect Cochrane's global and diverse nature, so would particularly welcome delegates from low- and middle-income countries, many different languages and many different roles in the organization.

Please join us to shape Cochrane’s future!

Target audience: Authors, group members, Cochrane Community. We should particularly welcome delegates from low- and middle-income countries, many different languages and many different roles in the organization, to ensure our strategy is diverse and inclusive.

Format: Presentation then focus group discussions

Additional facilitators: Susan Evans, Jordi Pardo Pardo, Flavia Marcocci, Fiona O'Brien.

11:00 AM - 12:30 PMTransdisciplinary Approaches for Setting Research Priorities on Climate Change and Health: Engaging Decision Makers, Researchers, and CommunitiesWorkshop - discussion

Background: Climate change is having a profound impact on human, animal and planetary health. These impacts will also trigger social, economic and political changes, indicating the need for a wider understanding of the evidence landscape for identifying policy and practical solutions. There is a growing recognition of the importance of involving field workers, communities and indigenous people in setting research priorities to ensure they are contextually relevant, and the process is inclusive, where people most familiar with problems and issues in their regions are involved in such decisions. In this workshop, we will explore how decision-makers, funders and researchers can work together to set a research agenda for prioritizing evidence syntheses in this field.
Objectives: Our workshop will provide an overview of how to involve decision-makers, researchers and indigenous people in setting research priorities on climate change and health.
Description: - The workshop will begin with an overview of how climate change affects humans, animals and planetary health, including transdisciplinary approaches of One Health and Planetary Health. - This will be followed by a presentation identifying policy and practice needs. - Next, we will present how to undertake stakeholder mapping and create a decision-making framework. This will demonstrate how to untangle some of the complexities in this area. - Exercise: We will provide individuals with a report on future research needs with local decision-makers on response to disasters due to climate change that affected the health of the local community in the island of Palawan in the Philippines. The participants will analyse and appraise those results and how to translate them into questions for systematic reviews. - Discussion: We will open the session for a wider conversation on climate change and public health, how the issues raised will be applicable and generalisable to different sectors, and how an evidence ecosystem, such as Cochrane, fits into this. This workshop will provide participants with the knowledge and tools to engage in effective and inclusive priority-setting exercises that address the complex and wide-ranging issues related to climate change and health.

11:00 AM - 12:30 PMIf, when and how to sample primary studies for inclusion in a qualitative evidence synthesisWorkshop - training

Background: Too much data can threaten the quality of a data synthesis in a qualitative evidence synthesis (QES), and too little can limit transferability across multiple contexts. As a QES seeks to provide an interpretation of the evidence, it is not essential to identify and include every available, relevant study but rather only studies that adequately represent relevant data from diverse participants and contexts. Sampling can be considered within the study selection process to enable the review team to include a manageable number of studies in their review. Sampling methods are, however, not well articulated, and selecting an inappropriate sample introduces threats to QES rigour.
Objectives: The objective of this workshop will be to guide participants through new Qualitative and Implementation Methods Group guidance on sampling using practical exercises to facilitate an understanding of the following: - the difference between comprehensive and theoretical/purposeful screening approaches; - what to do if the search identifies too few or too many studies; and - how to sample studies for inclusion in a synthesis. Furthermore, we will challenge participants to engage in problem solving and creative thinking which may lead to new methodological ideas in the field of sampling for study inclusion in a QES.
Description: This workshop will begin with a brief presentation of the key terms and concepts participants need to understand to participate in the groupwork. Participants will then be broken into small groups. Each group will receive a case study of a QES to form the basis of their groupwork and discussion. They will work together to develop a sampling strategy for the QES they have been assigned. Throughout the workshop, we will shift between group and plenary discussion to resolve the case studies using participants’ experiences and creative thinking. As sampling for study inclusion can take diverse forms, participants will be challenged to find solutions appropriate to the QES they are working with. QIMG Core methods workshop.

11:00 AM - 12:30 PMIntroducing INSPECT-SR: a tool for detecting problematic randomised controlled trials in health systematic reviewsWorkshop - training

Background: There is a growing awareness about the problem posed by untrustworthy randomized controlled trials (RCTs) in systematic reviews. It appears that some RCTs have been partially or entirely fabricated. We call trials subject to serious research integrity issues “problematic studies”, and recent examples can be found in reviews of ivermectin for the treatment of COVID-19. The Cochrane policy Managing Potentially Problematic Studies states that studies should not be included until serious concerns about trustworthiness have been resolved. However, there is no consensus around how to identify problematic studies. The INSPECT-SR (INvestigating ProblEmatic Clinical Trials in Systematic Reviews) project is developing a tool for identifying problematic RCTs in systematic reviews by combining empirical evidence with consensus methodology to develop a draft tool, which will then be refined through user testing. To date, the project has had input from over 70 people with expertise and experience in research integrity.
Objectives: 1. To introduce the draft INSPECT-SR tool to workshop participants. 2. To provide training in the draft tool by way of application to real examples. 3. To gather feedback from participants which will be used to improve the tool.
Description: This workshop will begin with a short (10 min) introduction to the draft INSPECT-SR tool and the workshop activity. Participants will work in groups to apply the tool to studies in Cochrane Reviews. The tool includes signaling questions in four domains. Illustrative examples are Inspecting results in the paper: Are the results substantially divergent from others in the meta-analysis? and Inspecting conduct, governance and transparency: Is the recruitment of participants plausible within the stated time frame for the research? The activity will be divided into 15-minute segments, corresponding to each domain. In each, participants will work in small groups to apply the signaling questions from one domain with support from facilitators (10 min), followed by facilitator-led discussion amongst all participants (5 min). The final 20 minutes will be devoted to overall discussion of participants’ impression of the tool, including capture of user feedback via questionnaire. Consumer involvement The project has an expert advisory panel representing a variety of stakeholders, including two lay members.

11:00 AM - 12:30 PMScoping reviews: breaking down the what, why, and howWorkshop - training

Background: Evidence synthesis provides the best available evidence to decision-makers and is pivotal to well-functioning, rapid-learning health systems. Scoping reviews have grown in popularity within the evidence synthesis community. Scoping reviews are distinct from but related to systematic reviews and are suited to answering different research questions usually beyond the scope of a systematic review. Whilst systematic reviews are able to answer questions on the feasibility, appropriateness, meaningfulness and experiences of a particular intervention, scoping reviews identify and map evidence on a topic and identify main concepts, theories, sources and knowledge gaps. The conduct and reporting of scoping reviews is often inconsistent, and this workshop hopes to support further understanding in this area.
Objectives: Our objective is to provide participants with information on the role of scoping reviews, the procedures for conducting and reporting scoping reviews and the PRISMA extension for scoping reviews (PRISMA-ScR) reporting guidelines, introduction to new JBI methodology on the role of knowledge users and the methods of capturing, analysing, and presenting results of scoping reviews.
Description: Following a short presentation, participants will be divided into small, facilitator-led working groups to discuss the provided scoping review exemplars to 1) determine whether the examples are correct using the JBI scoping review methodology and/or the PRISMA-ScR and 2) discuss what implications the exemplar scoping review has for other researchers, decision-makers, and patients. The workshop will then continue with a brief presentation and discussion on knowledge users’ role in scoping reviews. The workshop will conclude with an expert panel where participants can ask questions about scoping reviews and their relevance to guideline development. Participants will have access to resources in the session (methodological guidance articles, infographics and decision-making trees) via Dropbox.

11:00 AM - 12:30 PMAdvanced meta-analysis 2: Performing meta-analysis in the presence of rare eventsWorkshop - training

Background: Rare events are a common problem in meta-analysis, particularly for secondary and safety outcomes. When the events are rare, standard meta-analytical approaches have important limitations and may lead to biased and imprecise results. Alternative methods have been proposed in the literature that outperform in terms of bias and precision compared with the standard inverse-variance (IV) method. This workshop is part of a series of workshops delivered by the Cochrane Statistical Methods Group.
Objectives: The objective of this workshop is to provide guidance on handling rare events in meta-analysis. We aim to review the problems associated with the IV method and to describe several alternative methods that can be used instead.
Description: In this workshop, we will review the properties of the IV approach and explain the reasons that render this method problematic when the studied endpoints are rare. We will explain the advantages of one-stage meta-analysis models over two-stage models in such cases and go through the assumptions and the properties of some of these models. The workshop will also include a practical part in which participants will apply different models in R using data from a clinical example involving rare events. By the end of this workshop, participants will have a good understanding of the problems related to meta-analysis of rare events and will be able to fit meta-analysis models appropriate for such datasets.

11:00 AM - 12:30 PMPreparing Cochrane Reviews for submission, editing and publication: common errors and how to solve themWorkshop - training

Background: All Cochrane Reviews and protocols are copy-edited by Cochrane’s Production Service (CPS) before publication to improve clarity, consistency, accuracy and completeness. Any improvements that can be made to reviews earlier in the process will: - improve overall author, editor and copy editor experience; - reduce handling time; - reduce time between submission and publication; - improve the final quality of published reviews; - make reviews more accessible to readers, including patients and healthcare consumers; and thus - increase their use and implementation. The CPS team has identified a number of errors and issues that appear regularly in traditional reviews that will inform our workshop.
Objectives: To equip attendees to identify and correct, and so avoid, common errors in reviews and protocols prior to submission and copy-editing.
Description: The workshop will cover several common errors, covering elements of structure, formatting, style and consistency. Attendees will be talked through a variety of issues and then work either individually, or in small groups, on real-life exercises to help them to understand and master each point. We will discuss the answers and options for each within the workshop. We will discuss the appropriate use of resources, such as the Cochrane Style Manual, MECIR (Methodological Expectations of Cochrane Intervention Reviews)/PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidance, and the new Plain Language Summary requirements (introduced in 2021), alongside copy-editing checklists and other resources. Before the workshop, participants will be asked to submit examples of review text that have caused them concern during the course of preparing Cochrane Reviews for submission or copy-editing, and a selection of these will be used as practical, hands-on examples. During the workshop, we will identify areas where further guidance or additional resources would be helpful, which will inform future developments. Come and share your top tips for producing good content with people who care about making reviews the best that they can be before submission and publication!

11:00 AM - 12:30 PMHow to plan and implement synthesis questions (part 1): using the InSynQ checklist and guide for question developmentWorkshop - training

Background: This is the first of a two-part workshop. This session introduces the InSynQ (Intervention Synthesis Questions) checklist and guide which aims to facilitate the development and reporting of the questions addressed in systematic reviews. The concept of defining a review ‘question’ (the objective) and developing criteria for including studies in the review using the patient/population, intervention, comparison and outcomes (PICO) framework is familiar to most authors. However, without changing the review eligibility criteria, the synthesis can be structured to address different questions (e.g., broader or narrower) simply by grouping the interventions, outcomes or populations differently. The results of a systematic review are ultimately determined by these synthesis questions and the decisions authors take in deciding which studies are eligible to answer each question. Version 6 of the Cochrane Handbook for Systematic Reviews of Interventions introduced the concept of ‘PICO for each synthesis’ to bring greater focus on the need to plan and report details of the synthesis questions addressed in a systematic review. InSynQ (InSynQ.info) provides a practical tool to help implement this guidance. Objectives
• To demonstrate the importance and process of planning the PICO for each synthesis (i.e., the grouping and comparisons) through application in a practical example; and
• To introduce the InSynQ tool and show how it can be used to develop and report synthesis questions in a protocol and review.
Description: Based on the guidance in Chapters 2, 3 and 9 of the Cochrane Handbook, we will use a combination of group work, (fun) interactive exercises and short presentations to cover the following topics.
• Deciding how interventions and outcomes will be grouped (small-group work and discussion to plan a hypothetical synthesis)
• Using InSynQ to define your synthesis questions and PICO criteria for each comparison (interactive example) Workshop participants will gain an appreciation for the need to plan and fully specify their synthesis questions and an understanding of how InSynQ can be used for this purpose.

11:00 AM - 12:30 PMDrawing conclusions from network meta-analysisWorkshop - training

Background: Although the ability of ranking treatments is usually listed as one of the advantages of network meta-analysis (NMA), rankings fail to consider other pieces of relevant information and may lead to misleading conclusions. The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) Working Group has developed two frameworks for drawing conclusions from NMA: a minimally contextualized framework and a partially contextualized framework. The frameworks establish that drawing appropriate conclusions from NMA requires explicit consideration of estimates of effect for each pairwise comparison, their certainty of evidence, and the rankings.
Objectives: For individuals considering conducting an NMA, to gain familiarity on how to draw conclusions from network meta-analysis appropriately considering the estimates of effect, the quality of the evidence, and the rankings.
Description: This workshop will focus on the minimally contextualized framework for a single outcome. The framework allows for the classification of interventions in groups, from the most effective to the least effective (or from the most harmful to the least harmful, depending on the outcome). The workshop will begin with an interactive lecture providing details of the framework and then review a step-by-step template for applying it. Workshop participants will then break into groups of five or six to work through an example of a network meta-analysis, guided by facilitators when needed. Finally, the large group will discuss the results and other details of the use of the framework.

11:00 AM - 12:30 PMQuestion formulation and Interpretation of Diagnostic Test Accuracy Reviews: Accuracy and beyondWorkshop - training

Background: Question formulation is a fundamental first step in the review process and guides decisions about eligibility criteria, definitions of test comparisons to be evaluated, assessment of applicability, planning of statistical analysis and the interpretation of results. Using test accuracy statistics to describe the practical implications of using a test is a recognised challenge for the interpretation of test accuracy reviews. The workshop will offer guidance, as formulated in the Handbook for Diagnostic Test Accuracy Reviews, for formulating a diagnostic test accuracy (DTA) review question, interpretation and presentation of results.
Objectives:
•Understand what is meant by a testing pathway and representation of test comparisons
•Understand the importance of question formulation for judgments about applicability and for the interpretation of results
•Understand the importance of prevalence
•Understand how estimates of test accuracy can be used to describe the practical implications of using a test
Description: This workshop, organised by the Cochrane Screening and Diagnostic Tests Methods Group, is designed for individuals who are planning or currently undertaking a Cochrane DTA review, those involved in an Editorial role or those who are using test accuracy evidence. The facilitators can tailor the workshop to accommodate the needs of individuals attending as far as possible but plan to address the following four topics:
•The importance of setting and test comparisons as part of question formulation
•Anticipating and capturing sources of variability
•Common measures used to express test accuracy
•How estimates of test accuracy can be used to describe the practical implications of using a test
•Constructing Summary of Findings tables The workshop will draw on the experience of the DTA Editorial Team, as well as facilitators’ own experience of undertaking DTA reviews across a breadth of topic areas. The workshop will be delivered through a mixture of interactive presentations, discussions and small-group exercises.

12:30 PM - 1:30 PMAcute and Emergency Care - By Invitation OnlyMeeting

Discuss priority setting of conducting reviews related to medical treatment and management of patients who are experiencing life-threatening medical conditions or injuries. Answer questions relevant to meta-analysis and systematic reviews relevant to pre-hospital and in-hospital care, critical care and anaesthesia.

12:30 PM - 2:00 PMLunch break and meetingsLunch break 
12:30 PM - 2:00 PMPoster session 3Poster session

During this session the following posters will be presented:

12:45 PM - 2:00 PMCochrane US Network Meeting 2Meeting

In-person Meeting #2. This meeting is open to anyone from the United States.

Please find the agenda for the meeting below.

Meeting Agenda

1:00 PM - 1:45 PMCIS Exec meeting - By invitation only - By Invitation OnlyMeeting

Meeting of the CIS Exec

1:00 PM - 1:45 PMSystematic Review Proposal Vetting for AAO-HNS Cochrane Scholars - By Invitation OnlyMeeting

This session, moderated by Martin Burton and Richard Rosenfeld, will allow Cochrane Scholars from the American Academy of Otolaryngology - Head and Neck Surgery to vet their proposals for systematic reviews, and receive feedback on optimizing the scope, methods, and objectives.

1:00 PM - 1:45 PMThomas Chalmers Award Committee - By Invitation OnlyMeeting

Committee Members to meet and discuss onsite process.

1:00 PM - 1:45 PMCochrane Person-Centred Care, Health Systems and Public Health Thematic GroupMeeting

The Person Centred Care, Health Systems and Public Health Thematic Group brings together the expertise currently housed in Cochrane Effective Practice and Organisation of Care (EPOC), Cochrane Consumers and Communication (CCC) and Cochrane Public Health (CPH). Combining the expertise and networks of these three Cochrane review groups provides a sound base to generate more integrated evidence that considers health systems governance, financing and delivery arrangements, and population-level public health interventions, and how health systems and public health can be more person-centred.

1:00 PM - 1:45 PMConsumers at the ColloquiumMeeting

This is a meeting for consumers (patients, carers and public) attending the Colloquium. It is an opportunity for consumers to meet together, with members of the Cochrane Consumer Network Executive and other members of Cochrane. The meeting will hear about developments in Cochrane that affect consumers. It will also be an opportunity for people to raise issues, ask questions and discuss issues of mutual interest to our consumer community.

1:00 PM - 1:45 PMCochrane Methods Executive meeting - By Invitation OnlyMeeting

The Methods Executive (ME) provides a leadership forum within Cochrane and ensures all key methodological roles can meet and liaise. It provides an advisory role to the Editor in Chief and other Cochrane governance committees and supports the Methods Groups and wider methods community in Cochrane. This meeting will help in the strategic planning of the ME priorities for 2023-2024.

1:00 PM - 2:00 PMLibrary of PeopleNetworking session

The #CochraneLondon Library of People is an innovative addition to this year’s event, offering attendees a unique opportunity to engage in conversations with human ‘books’ who have stories and life experiences to share. 

The Library of People works just like a library with books - except the stories are told by people and you can interact with them! Members of The Cochrane Book Club will be your librarians. You will be able to listen to your book’s story in small groups; have an open conversation with them and ask questions. Each book will have a list of prompt questions you can ask to help you get started. 

The Library of People will feature a diverse group of ‘books’ covering various nationalities, stages of career development, activities and roles at Cochrane, and personal interests. 

The Library of People is a powerful tool for promoting empathy, understanding, connection, and learning. Don’t miss this exciting opportunity to engage with a living book and gain new perspectives at the event!

1:30 PM - 2:00 PMEmergency and Critical Care: Save lives Meeting

We hope to catch up with the latest news and discuss how to approach the future of Cochrane.

2:00 PM - 3:30 PMRapid reviews and other rapid evidence products 2Oral session
14:00 Opening remarks and introduction
14:05 Rapid evidence synthesis for COVID-19 international guidance: a World Health Organization commissioned Cochrane rapid scoping review
14:15 REH-COVER (Rehabilitation – COVID-19 Evidence-based Response) action: a “rapid” and “living” systematic review methodology
14:25 Wales COVID-19 Evidence Centre: a bespoke evidence review process engaging stakeholders for supporting time-sensitive policy and clinical decision-making
14:35 Evolving rapid review methods to meet the needs of evidence-informed decision-making: National treatment guidelines for COVID-19 in South Africa
14:45 Supporting COVID-19 Decision Making through Rapid Evidence Syntheses and Products
14:55 Finding best available evidence in a time of crisis
15:05 Stakeholder engagement in a rapid review to determine the effectiveness of interventions to attract, recruit and retain social care workers
15:15 Engaging Citizen Partners within a Rapid Review Process
15:25 Closing remarks
2:00 PM - 3:30 PMDiagnostic Test Accuracy and prognostic evidenceOral session
14:00 Opening remarks and introduction
14:05 PROBAST+AI: assessing quality, risk of bias and applicability of diagnostic and prognostic prediction models based on AI or ML techniques
14:15 The Application of PROBAST and Prevalence of Unfavorable Risk of Bias in Systematic Reviews of Prediction Models
14:25 Risk of bias and applicability assessments for overall prognosis studies (RoB-OPS): Current development status
14:35 Landscape of methodological quality assessment tools for diagnosis and prognosis research
14:45 Diagnostic test accuracy network meta-analysis methods: A scoping review and empirical assessment
14:55 Pooling biological specimens in primary studies compromises evidence generation in test accuracy reviews: a case study
15:05 Closing remarks
2:00 PM - 3:30 PMCommunicating evidence, misinformation and research transparencyOral session
  Opening remarks and introduction
14:05 Adherence to PRISMA 2020 statement in non-Cochrane systematic reviews of interventions. A meta-epidemiological study
14:15 Publication bias - a cross-sectional study of randomised trials in Sub-saharan Africa: ongoing challenges of research waste
14:25 Time to publication for results of clinical trials: The definitive systematic review
14:35 Are there missing randomized trials in Cochrane systematic reviews and what is their impact on the results?: A methodological study
14:45 Reasons for missing evidence in rehabilitation meta-analyses: a cross-sectional meta-research study
15:05 Identifying Candidate Harms for a Systematic Review
15:25 Closing remarks
2:00 PM - 3:30 PMLiving evidence and prospective meta-analysesOral session
14:00 Opening remarks and introduction
14:05 Methods and guidance on conducting, reporting, publishing and appraising living systematic reviews: a scoping review
14:15 Lessons learnt from conducting a living systematic review and network meta-analysis for covid-19: maximizing efficiency and access of results
14:25 Living Reviews: Practical Considerations for Adapting Scope and Communicating Evolving Evidence
14:35 The Australian National COVID-19 Clinical Evidence Taskforce – the impact of living guidelines in a critical area of clinical need
14:45 How often should “living” systematic reviews be updated? A cross-sectional study
14:55 Living Evidence to Inform Health Decisions Framework (LE-IHD): A practical interactive framework based tool to guide the incorporation of Living Evidence in the development of knowledge transfer products
15:05 How can a framework for prospective, adaptive meta-analysis (FAME) improve the quality of Cochrane reviews?
15:25 Closing remarks
2:00 PM - 3:30 PMThe Art of Getting By: Influencing skills - Communication and NegotiationSkills Lab

This session on influencing skills explores a model of communication that allows you to handle criticism, including harassment and bullying language, managing yourself in an interview, in an exam, writing an academic paper, and in everyday communication. The session on negotiation looks at how to get better outcomes when negotiating.

This is linked to the session The Art of Getting By: Leadership and Team Working Styles on Tuesday 5th September at 16.00-17.30, but can also be attended as a stand-alone session. Both sessions will be employing simple "heuristics" - ways of understanding complex situations and making better choices. They are so simple you can keep them in your head. The skills are generic, and you can employ them in your private life as well as your working life. You will even return home with a couple of heuristics to try out the same evening!

2:00 PM - 3:30 PMMoving the Cochrane-WHO partnership forward: current and future activities and how to get involved Special Session

Cochrane has been a ‘non-state actor in official relations’ with the World Health Organization (WHO) since 2011. The partnership is one of Cochrane’s largest and most impactful: around 76% of new guidelines issued by WHO in 2021 were informed by evidence from Cochrane Reviews. Discussions for a new programme of work will commence in the third quarter of 2023, so it is timely to revisit and renew the relationship between the two organizations.

This session intends to update participants on the status of Cochrane’s existing collaboration with WHO and on the organization’s current and future priorities. It is also a chance for participants to hear from WHO representatives about opportunities to engage directly in this partnership, which ultimately aims to improve global health outcomes by ensuring health and care decisions are informed by trusted and timely evidence.

After a brief introduction to the Cochrane-WHO collaboration and a short keynote from a WHO representative on current organizational priorities, the session will highlight key topics for discussion including: the use of rapid, living and curated evidence (and associated methods and innovative technologies) to inform WHO global guidance and ensure it remains up to date; expanding guidance and infrastructure to support better clinical trials and the reporting of their results; support for expanding areas of WHO’s work, such as evidence relating to traditional medicine; and how to build or strengthen relationships with specific WHO departments, as well as regional and country offices. An open discussion will then follow.

WHO guidance has a direct impact on the lives of billions of people worldwide. The primary activity of the Cochrane-WHO partnership is to ensure that WHO is able to draw on rigorously assessed, trusted and timely evidence for use in this guidance. Patients and consumers are encouraged to join the session.

Target audience: Anyone interested in the Cochrane-WHO partnership

Format: Presentation, panel discussion, open discussion with audience

2:00 PM - 3:30 PMUndertaking overview of systematic reviews – methods, challenges and the way forward (as identified by a group of clinical academics and methodologists who recently figured it all out)Workshop - discussion

Background: Overviews of systematic reviews (hereinafter referred to as ‘overviews’) are increasingly being used for the synthesis of evidence where the subject areas are broad and the evidence is rapidly expanding. Despite methodological advances and development of methodological guidelines, experience in their adoption is limited and many challenges remain. This proposed workshop will provide a forum for participants to learn more about methodological development in undertaking overviews and to share practical experiences.
Objectives: (1) To raise awareness of tools and guidelines currently available in relation to the conduct of overviews; (2) to provide a forum for review authors to share their practical experiences and highlight potential challenges in undertaking overviews; and (3) to discuss potential areas for refinement of current tools and guidance and further methodological research.
Description: This workshop will be a highly interactive event that offers opportunities for participants to learn more about methods for conducting overviews and for authors who have conducted overviews to share their experiences. The structure of the workshop will be aligned to key steps (and corresponding methods and challenges) for the preparation of an overview: (1) defining the research question and scope; (2) planning literature search; (3) selecting and mapping studies; (4) assessing the quality of systematic reviews; (5) extracting data; (6) synthesising and presenting evidence—quantitatively and narratively; and (7) drawing conclusions and formulating recommendations. For each key step, we will invite participants to share tools and methods that they have used and issues that they faced, then supplement the discussion (where necessary) by drawing on the facilitators’ own experience, having recently completed a highly complex overview of systematic reviews on perioperative oxygen therapy funded by the UK National Institute for Health and Care Research. We will ensure discussions cover both practical issues (e.g., how to deal with overlapping evidence between systematic reviews and primary studies not covered by existing reviews) and theoretical underpinning (e.g., lumping versus splitting evidence in quantitative synthesis). We will utilise a mobile/online platform for real-time audience interaction to ensure that the workshop is truly participatory.

Cochrane Colloquium 2023 Overview of systematic review workshop plan v5.docx

2:00 PM - 3:30 PMPreparing for submitting your manuscript to the Cochrane’s Central Editorial Service for Peer Review – observations from the Central Editorial Service Quality Assurance TeamWorkshop - training

Background: Cochrane has recently been moving towards a Central Editorial Service that clearly separates review development roles (e.g., authoring/author support) from the editorial roles in the evidence synthesis production model. As part of this process, the Cochrane Central Editorial Service sends every Cochrane review for consumer, clinical, search, and methods peer review prior to publication. This robust process ensures that Cochrane reviews are accurate, reliable, and usable. However, the editorial process is often delayed, or the manuscript is rejected because methodological and reporting issues are identified during peer review. Authors and editors should be aware of the most common errors noted during the methods peer review process to help them identify, rectify, and ultimately avoid making these errors.
Objectives: The objectives of this workshop are to highlight common methodological and reporting errors made in Cochrane Systematic Reviews; to provide practical, hands on guidance to help authors and editors address these errors; and to discuss the current opportunities available for getting involved in conducting editorial peer review.
Description: The workshop will begin with a brief PowerPoint presentation, introducing the methods peer review process conducted by the Central Editorial Service and providing an overview of common errors identified during this process. The errors discussed will include (i) inappropriate prioritisation of comparisons and outcomes across the different sections of the review, (ii) inconsistent consideration of all 5 GRADE domains during the certainty of evidence assessment, and (iii) overinterpretation of the results leading to misleading and inappropriate conclusions. Following this, the attendees will work in small groups with the facilitators to identify these errors in some real-life examples and discuss the best way to rectify or avoid the issues. The workshop will conclude with an open discussion regarding the current opportunities available to authors and editors who are interested in getting more involved in the editorial peer review process.

2:00 PM - 3:30 PMIntroduction to the RoB 2 tool for assessing risk of bias in a randomized trialWorkshop - training

Background: Randomized trials provide evidence about the effects of healthcare interventions. However, trial results can be undermined by flaws in design, conduct, analyses and selective reporting. Therefore, assessments of risk of bias in results of the included randomized trials are mandatory in Cochrane systematic reviews. The revised risk of bias tool (RoB 2), published in 2019, is the recommended method for doing this.
Objectives: To explain the key concepts that underpin RoB 2 and gain experience in using it to assess risk of bias in randomized trials.
Description: We will present, implement and discuss the key features of RoB 2:
• specification of the result to be assessed and sources of information used;
• specification of the effect of interest (that of assignment to intervention, or adhering to intervention);
• five consolidated bias domains;
• signalling questions that lead to algorithm-guided risk of bias judgements within bias domains; and
• derivation of an overall risk of bias judgement for the assessed result. We will explore the five bias domains included within RoB 2: bias arising from the randomization process, bias due to deviations from intended interventions, bias due to missing outcome data, bias in the measurement of the outcome and bias in the selection of the reported result. A worked example will be used to provide hands-on training and facilitate discussion. Participants will have the option to complete their assessments in a semi-automated Excel template and are encouraged to bring their own portable device to access detailed guidance that is available at www.riskofbias.info. Please note that the facilitators are unable to provide printed copies of the materials.

2:00 PM - 3:30 PMEPPI-Reviewer: review-production software that adapts to your needsWorkshop - training

Background: EPPI-Reviewer has been part of the Cochrane ecosystem of tools since 2015 and is available free of charge for Cochrane reviews; its role is to facilitate the review production of complex reviews. Unlike other software supporting review production phases, EPPI-Reviewer is designed with flexibility in mind and includes a wide range of cutting-edge machine learning and data services. EPPI-Reviewer is also a methodological research tool: its flexibility allows and facilitates methodological research and innovation; our current research focuses on enabling automation on the search and screening phases, integrating them both in ways that are specifically designed to support workflows for “living” systematic reviews. The same underlying technologies are expected to also drive significant innovations regarding data reuse and discoverability.
Objectives: Introduce the core concepts behind the EPPI-Reviewer architecture and tools. Provide a general overview of how to use it according to different needs (meta-analyses, mixed methods, “living” systematic reviews, meta-ethnographies, etc.). Provide an introduction to our latest innovations and future plans.
Description: The workshop will focus on the key elements of EPPI-Reviewer, concentrating on the features that are designed explicitly to support flexibility. Participants will be guided through the overall structure of the software, including highlights on the latest developments (living reviews, producing maps, and visualisations) and the ideas that inform our plans for the future. Practical exercises will focus on a few of the showcased features giving each participant the chance to try out the methodologies and approaches that best suit their use-case. This session is suitable for reviewers who are just starting to use EPPI-Reviewer, those who are ready to learn more, and those who are considering it for their future work. The ideal participant would have some experience in conducting systematic reviews (although beginners are welcome). Participants should bring their own laptop to the session, working in small groups will be encouraged.

2:00 PM - 3:30 PMWhat does good co-production in evidence synthesis look and feel like?Workshop - training

Background: There is increasing interest in incorporating co-production in systematic reviews. However, there has been a lack of a unifying definition of co-production and the absence of a single definition can lead to ‘conceptual stretching’ and even misuse of the term. Rather than focus on a single definition, in this workshop, we consider the values that underpin co-production, including being human, inclusive, transparent and challenging, and how they can be enacted within evidence synthesis.
Objectives: Participants will:
• Learn about the values that underpin co-production.
• Consider how co-production can be integrated within the development of a systematic review.
• Consider how the values that underpin co-production can be operationalised within their own evidence syntheses and systematic reviews.
Description: Co-production requires a substantial shift in the way in which systematic reviews are produced, from conceptualisation to dissemination. This workshop will introduce participants to the values of co-production and how these can be used to inform the design and implementation of co-production within different types of systematic review (15 min). In groups, participants will have an opportunity to consider how the values of co-production can be implemented through considering different scenarios and case studies (30 min and 15 min feedback). In a final group discussion, participants will be invited to consider how the values of co-production can be enacted in their own evidence synthesis projects (20 min and 10 min summary). Throughout, the facilitators will also reflect on their own learning from implementing co-production in evidence synthesis, the emotions and ‘feels’ experienced as part of the process and the value that co-production brought, as well as sharing insight of what they would do differently in future. Relevance and importance to patients and the public: Whereas co-producing evidence presents a challenge to conventional ways of producing evidence in academic environments, it can also lead to evidence that is responsive to the needs of the public, patients and policymakers. Co-production can deliver evidence that actually matters to people and builds capacity and empowers individuals and communities whose voices have historically been marginalised.

2:00 PM - 3:30 PMHow to plan and implement synthesis questions (part 2): Implementing PICO in RevMan to streamline data extraction and analysesWorkshop - training

Participants will need to bring their own laptop.
Background: This is the second of a two-part workshop. Defining your review ‘question’ (the objective) and developing criteria for including studies in the review using the PICO framework is a fundamental step in a systematic review. As well as this review-level PICO, the concept of ‘PICO for each synthesis’ was introduced in Version 6 of the Cochrane Handbook for Systematic Reviews of Interventions to bring greater focus on the need to plan and report each synthesis question addressed in a systematic review. The first session introduces the InSynQ (Intervention Synthesis Questions) checklist and guide for question development and reporting and demonstrates how to plan the PICOs for hypothetical syntheses. Once a review author has defined their PICO criteria for their review and each synthesis, early investment in setting this up in RevMan can streamline the review process going forward.
Objectives: In this second session, participants will:
• Set up the review and syntheses PICOs, with associated groupings, in RevMan using the study centric data structure
• Gain an understanding of how this can streamline data extraction and facilitate completing your analyses
Description: This workshop will: 1. Summarize the concept of PICO for each synthesis and the InSynQ guide (introduced in part 1). 2. Complete a practical exercise in RevMan covering: a. How to set up your predefined PICO criteria for your review and each synthesis. b. How to use the review and synthesis criteria in RevMan to inform your data extraction forms. c. How to import your extracted data from included studies into RevMan. d. How to set up and complete your analyses with just a few clicks (including how RevMan automatically transforms the results data from your included studies into the review’s analyses). 3. Provide further information about the benefits of study centric data management in RevMan and how it makes systematic review production more efficient. Attendees are strongly encouraged to attend both sessions.

2:00 PM - 3:30 PMGRADE target of certainty rating and implications for judgements regarding imprecisionWorkshop - training

Objectives: This workshop will introduce participants to the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) guidance on choosing targets of certainty of evidence and its implication on rating imprecision of evidence.
Description: We will give a presentation to start the workshop, providing an overview of some basic concepts regarding GRADE certainty of evidence (target of certainty of evidence, possible thresholds/ranges of interest), practical principles on how to determine targets of certainty ratings, and GRADE updated guidance on imprecision ratings (primary approach for imprecision ratings in both systematic reviews and guidelines, situations in which one should consider rating done more than one levels for imprecision). Next, participants will receive examples from existing reviews. We will facilitate participants to discuss in these examples their judgements regarding the target of certainty ratings and how many level(s) to rate down for imprecision. After discussions, we will provide tips for each example. At the end, we will encourage participants to share their perspectives and opinions on the updated GRADE guidance presented.

2:00 PM - 3:30 PMComparing multiple interventions with network meta-analysisWorkshop - training

Background: Standard meta-analysis methods for clinical trials focus on comparisons of two interventions, such as a drug versus placebo or a new intervention versus standard practice. In clinical practice, there are rarely only two interventions under consideration. Extensions of meta-analysis to address three or more treatments have been the subject of much methodological research in recent years and are increasingly being applied. Most simply, indirect comparisons can be performed in ways that respect the randomization within each clinical trial. More complex forms are the so-called network meta-analyses, also known as multiple treatments meta-analyses or mixed treatment comparison meta-analyses. These allow the simultaneous analysis of clinical trials involving different treatments.
Objectives: To introduce the concepts and methods of indirect comparison and network meta-analysis in the context of a Cochrane systematic review, following the Handbook Chapter drafted by the Cochrane Comparing Multiple Interventions Methods Group (CMIMG).
Description: This workshop is aimed at statisticians, epidemiologists, and other quantitatively minded researchers who want to understand state-of-the-art statistical syntheses of clinical trials involving multiple interventions. The workshop will provide insights into network meta-analysis models that can be used to derive estimates for the relative effects of all treatments of interest. By the end of this workshop, participants will have an understanding of the role and potential of indirect comparisons and network meta-analysis in the evaluation of healthcare interventions: the principles, steps, and statistical methods involved and the biases that can distort indirect comparisons and network meta-analysis.

3:30 PM - 4:00 PMBreakCoffee break 
4:00 PM - 5:30 PMCochrane Lecture / Closing PlenaryPlenary

Forward together for trusted evidence

Given the ever-growing potential for misinformation and disinformation in an interconnected world, the need for trusted evidence has never been greater. As we navigate a post-COVID-19 pandemic world, evolving data and technological opportunities, and the reality of change, ensuring that the evidence that informs health and care decisions is timely, trusted and relevant will be an ongoing challenge. This Cochrane Lecture explores a framework for sustaining and promoting trusted evidence that not only withstands scrutiny but actively guides informed decision-making by addressing emerging needs through a continuous feedback loop between evidence producers and users. The lecture will emphasise the importance of the interlinked pillars of relevance, equity, integrity, transparency, and rigour underpinned by collaboration between stakeholders and researchers as we journey forward together embracing diverse perspectives, experience and expertise to co-produce and deliver evidence that is not only trusted but makes a difference for patients and the public globally.

Cochrane's annual prizes and awards will be presented at the end of the session. 

 

11:05 AM - 11:15 AMAssessment of trustworthiness has a significant impact on conclusions of Cochrane reviewsResearch integrity, transparency and fraud

Background: There is increasing concern that a significant proportion of randomised trials included in Cochrane reviews may not be trustworthy. Applying a Trustworthiness Screening Tool (TST) has already had a clinically important effect on several reviews published by the Cochrane Pregnancy and Childbirth Group.
Objectives: We wanted to assess the impact of removing untrustworthy randomised trials from already published Cochrane reviews on a defined clinical area (nutritional interventions during antenatal and postnatal period).
Methods: We applied the Cochrane Pregnancy and Childbirth TST to 375 randomised trials included in 18 Cochrane reviews. The TST has five domains (is the research governance trustworthy; are the baseline characteristics trustworthy; is the study feasible; are the results plausible; and is all relevant information available?). When additional information was needed, authors were contacted using a standard template. At least two attempts were made to contact the authors. At the end of the evaluation process, each study was classified as i) included (YES to all domains); ii) excluded (retracted study); or iii) awaiting classification (any NO to the TST questions).
Results: 95/375 studies (25%) were removed, affecting 14/18 (78%) reviews. 13/18 reviews (72%) showed a difference in the Summary of Findings tables (direction and size of effects and/or GRADE ratings). 6/18 Cochrane reviews (33%) were judged to require updating because of important differences in their conclusions, implication for practice, and/or implication for research.
Conclusions: Formal assessment of trustworthiness and inclusion only of studies that satisfy prespecified criteria for trustworthiness affects conclusions in a relatively large number of Cochrane reviews, with potentially important clinical implications for practice and research. The lack of consensus regarding the best tool(s) for assessing trustworthiness cannot be an excuse for ignoring this issue in future Cochrane reviews.
Patient, public and/or healthcare consumer involvement: N/A. Statement on relevance and importance to patients: Our work found that removal of studies that did not fulfil prespecified trustworthiness criteria affected the conclusions of a third of the Cochrane reviews that we assessed.

11:05 AM - 11:25 AMCan we use GRADE to create new diagnostic criteria for a condition? An application of GRADE principles for establishing diagnostic criteria for a diseaseAssessing evidence certainty

Background: Although it is critical to consider downstream consequences of changing disease definitions and to avoid the perils of overdiagnosis, disease definitions and diagnostic criteria are sometimes required to be developed, modified, or updated. In 2021, a group in Australia was tasked with updating the guideline for the diagnosis and assessment of Fetal Alcohol Spectrum Disorder. However, there is currently no universally agreed approach to developing diagnostic criteria or assessment guidelines.
Objectives: Our aim was to ensure that any new criteria or definitions proposed by our guideline group were as transparent as possible and supported by a rigorous development process. In line with best practice in guideline development, we aimed to use GRADE to assist with our guideline. However, there was no specific GRADE guidance currently for this type of guideline.
Methods: We used a novel application of the GRADE approach to inform the development of our guideline. Firstly, to determine potential diagnostic criteria for FASD, we performed a range of systematic reviews using the GRADE for establishing certainty in prognostic factors to determine the association between particular exposures and outcomes with FASD. We then used adapted evidence to decision frameworks (EtDF) to make a recommendation regarding whether a particular diagnostic criteria should be considered in our final set of criteria. Following this, we created an overarching EtDF for our proposed diagnostic criteria for FASD. Results and Conclusion: Our proposed revised definition, criteria, and assessment guidelines for FASD are informed by rigorous systematic reviews and a transparent decision-making process. We believe these methods may be suitable for adoption or adaptation for other groups creating diagnostic criteria and revising disease definitions.
Patient, public and/or healthcare consumer involvement: Extensive engagement with people with FASD and their caregivers occurred during this project.

11:05 AM - 11:25 AMOperationalising decolonisation in systematic reviews: can current tools help to mitigate for bias?Global health, equity, diversity and inclusion

Background: Decolonisation of research methods refers to the undoing of colonial ideologies within academic thinking and broadening critical and theoretical positions within methodological approaches to address power imbalances and inherent biases. Trust in systematic reviews is underpinned by transparency, rigour, and objectivity at all stages of the review process. Frameworks, such as GRADE (Grading of Recommendations, Assessment, Development, and Evaluations) and GRADE-CERQual (Confidence in the Evidence from Reviews of Qualitative research), aim to improve transparency surrounding certainty or confidence in the evidence. A key element of these tools is to assess the overall directness or relevance of the synthesis findings to examine whether they offer a ‘restricted’ answer to the review question because of the populations, interventions, comparators, or outcomes studied. Application of GRADE and CerQual could therefore be used to identify underrepresentation of important ethnic groups or marginalised populations in research.
Objectives: To explore how GRADE and GRADE-CERQual could be employed to support decolonisation of systematic reviews.
Methods: An exploration of how systematic reviews on issues known to disproportionately affect certain ethnic groups could use GRADE and CERQual to support interpretation of the evidence and decolonisation of research. We use case examples of reviews on conditions known to disproportionately affect people of African, African-Caribbean, and South-Asian descent, such as diabetes, to illustrate how reviewers might consider the directness of the evidence to those groups.
Conclusions: Systematic review teams should routinely consider the potential for underrepresentation of important or marginalised ethnic groups as a restricting factor when applying the ‘directness’ dimension of GRADE or the ‘relevance’ dimension of CERQual. Review teams should consider providing equity statements in their summary of findings table to mitigate issues of colonial bias in research and to reduce the risk of perpetuating underrepresentation of marginalised populations. Patient, public, and/or healthcare consumer involvement: No specific patient or healthcare consumer involvement was sought for this abstract because of the theoretical nature of the work. However, the authors both value and advocate for diverse stakeholder involvement in the systematic review process.

11:05 AM - 11:25 AMEvidence gap maps: a visual tool for promoting evidence and monitoring gaps in researchMapping evidence

Background: Evidence Gap Maps (EGMs) are visual tools that present the available evidence on different thematic areas and highlight gaps for future research. EGMs, constructed by international Non-governmental Organisation Sightsavers, summarize, appraise, and present evidence from systematic or literature reviews across different eye health conditions, including cataract, refractive error, glaucoma, diabetic retinopathy, and trachoma. Updating an EGM periodically allows us to monitor changes to the extent and quality of the evidence base and to refine the focus for future research. This presentation will focus on the methods used to develop and update the EGMs and discuss their potential in promoting rigorous, high-quality research and countering misinformation.
Methods: After conducting a comprehensive literature search, we sift and extract data from all relevant reviews. Critical appraisal is conducted independently using the Supported Use of Research Evidence checklist, allowing us to attribute a confidence level of low, medium, or high to the review. We assessed the strength of evidence of the effect of each review. If the review reaches a conclusive answer to their research question using the available evidence, the evidence is classified as strong. If they were unable to answer the question because of insufficient evidence, the evidence is classified as weak. Otherwise, the evidence is inconclusive. Each review is displayed in a matrix where the columns show thematic areas relevant to the eye health condition, labelled as sectors and subsectors, and the rows show the strength of evidence. Updates are conducted in the same way as the original EGMs. Changes to the evidence base were assessed by comparing the numbers and percentages of included reviews across a number of areas, including the thematic focus, geographic representation, strength of evidence of effect, and methodological quality. Conclusion and patient and healthcare consumer involvement: EGMs allow policymakers and clinicians to plan their interventions based on the most up-to-date knowledge and high-quality research, which helps ensure that patients receive the best possible care. Updating the EGMs periodically allows researchers to monitor changes in the extent and quality of systematic review evidence and to prioritise future work to improve the evidence base and to close evidence gaps.

Refractive error evidence gap map details tab.png

11:05 AM - 11:25 AMUnravelling active ingredients of task-shifting interventions in low-resource settings for common mental disorders: developing a taxonomy of intervention components and ranking their efficacy.Network Meta-analysis

Background: The global burden associated with common mental disorders is high, especially for people living in low resource settings. Although psychosocial interventions delivered by locally available lay or community health workers are effective, mechanisms of intervention response are poorly understood. One of the greatest barriers is that psychosocial interventions are administered as complex, multi-component “packages of care”.
Objectives: Our aim is to systematically review all the randomized controlled trials (RCTs) that have tested the efficacy of psychosocial interventions delivered through the task shifting modality to treat people suffering from common mental disorders (depression, anxiety, and related somatic complaints) in low resource settings, dismantle the intervention protocols creating a taxonomy of active intervention components, and reevaluate their efficacy.
Methods: We will use the component network meta-analysis (cNMA) methodology. The major benefit of cNMA is the possibility to disentangle intervention components and explore their effectiveness separately or in various combinations (even in disconnected networks). cNMA increases statistical power by combining direct and indirect comparisons while fully respecting the randomized structure of the evidence. According to the additive cNMA model which we will implement, adding a component “c” to a composite intervention “X” will lead to an increase (or decrease) of the effect size by an amount only dependent on “c”, and not on “X”. We will denote the corresponding component specific incremental standard mean difference (iSMD) so that iSMDc = SMD(X+c) v. (X). Combining these component-specific iSMDs will allow the estimation of SMD between any two composite interventions.
Results: A network of comparisons and a hierarchy that includes all intervention components expressed as iSMD, indicating the added benefit of adding a component to an intervention, will be presented. By selecting the most effective components it will be possible to outline a novel task shifting psychosocial intervention to be tested in future RCTs.
Conclusions: These findings will set the basis for further investigations in the field of precision medicine. This project is funded by the European Union’s HORIZON EUROPE research programme under grant agreement No 101061648 and is prioritized by Cochrane Global Mental Health. Patient, public, and/or healthcare consumer involvement: not applicable.

Abstract Cochrane_componentNMA.pdf

11:15 AM - 11:25 AMProspective trial registration in the Research Integrity Assessment (RIA) of randomized controlled trials (RCTs)Research integrity, transparency and fraud

Background: The WHO and the Declaration of Helsinki require that every clinical study must be prospectively registered in a publicly accessible study registry (i.e., registration before enrolling the first patient). However, most evidence syntheses ignore violations of this fundamental moral and ethical principle. The Research Integrity Assessment (RIA) (doi:10.1002/jrsm.1599), a tool to assess adherence of randomized-controlled trials (RCTs) to the principles of Good Clinical Practice, and key elements of research integrity in clinical trials, assesses prospective registration and may help evidence synthesis authors to deal with studies which do not comply with the requirements. RIA was piloted in the updated Cochrane review “Ivermectin for preventing and treating COVID-19” revealing that lack of prospective registration was the most frequent reason for the exclusion of studies according to RIA.
Objectives: To evaluate trial registration in RCTs included in the study pool of COVID-19 systematic reviews (SRs) on interventional drugs and to analyse the impact with the application of RIA.
Methods: We searched for SRs investigating at least 1 of 13 different interventions for treatment or prevention of COVID-19 in Medline/PubMed until 09 June 2022. We selected Cochrane and non-Cochrane SRs with the largest RCT pool, extracted all included RCTs, evaluated trial registration data, and applied the RIA domain 2 (prospective trial registration) to all RCTs. RIA was independently applied by two reviewers; conflicts were resolved by a third reviewer.
Results: We identified 188 RCTs for RIA domain 2 assessment. Twelve RCTs were not registered and 176 RCTs were registered in at least 1 of 10 clinical trial registries. Using RIA, we assessed 109 prospectively registered RCTs as ‘no concernʼ, 51 non- or retrospectively registered RCTs as ‘excludeʼ, and 28 RCTs with inconsistent information between publication and registry record, missing information, inaccessible registry record, or no publication as ‘awaiting classificationʼ (Table 1). Conclusion: Almost one in three studies on interventional treatments for COVID-19 has not complied with the international standard of prospective trial registration. Authors of evidence synthesis can use the RIA tool as a transparent mechanism to manage studies that do not adhere to the international standard. Patient and/or public involvement: none.

Table -1_2023-03-06.pdf

11:25 AM - 11:35 AMImpact of Grading of Recommendations, Assessment, Development and Evaluation (GRADE) on conclusions of dentistry systematic reviewsAssessing evidence certainty

Background: The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach assesses certainty of evidence (CoE) in a systematic review (SR) and facilitates drawing appropriate conclusions that consider limitations of the evidence. However, GRADE is not widely used in dentistry SRs.
Objectives: To evaluate the impact of using GRADE on conclusions in dentistry SRs.
Methods: From a sample of 200 dentistry SRs of randomized controlled trials included in a larger methodological study, we selected a subsample based on the most frequently reported outcomes. We will use the odds ratio and its 95% confidence interval to determine whether SRs not using GRADE are more likely to formulate inappropriate conclusions (i.e, conclusions relying on statistical significance, not considering limitations or formulating recommendations). For SRs not using GRADE, we will compare conclusions made by authors to conclusions we made based on our own GRADE assessments using the same effect thresholds as SR authors. We will calculate the number of conclusions that changed with respect to the described certainty (i.e., the extent to which authors recognized uncertainty, analogous to high, moderate, low, or very low CoE in GRADE) and magnitude of effect (i.e., whether the conclusions state there was a presence or absence of effect, negligible or important effect) after utilizing GRADE.
Results: We present the results of a subsample of 19 SRs reporting the outcome of pain. Six (32%) used GRADE, and 13 (68%) did not. Two SRs formulated inappropriate conclusions; both did not use GRADE. For SRs without GRADE, our GRADE assessments changed 62% of conclusions with respect to the described CoE. Our GRADE assessments changed the described magnitude in 25% of conclusions. We will present results for additional outcomes currently under analysis.
Conclusions: After completing all analyses, our conclusions will focus on the influence of GRADE on formulating appropriate conclusions which will provide insight into potential limitations that may arise when SRs formulate conclusions without GRADE assessments. Developing more transparent SR conclusions is relevant to patients as SRs are a valuable source of evidence for decision-makers. Patient, public, and/or healthcare consumer involvement: Patients were not involved.

11:25 AM - 11:35 AMMissing and masked: equity in a systematic review of remote interventions for substance misuse.Global health, equity, diversity and inclusion

Background: To reduce inequity, knowing whether interventions ‘work’ is not enough; we also need to know whether vulnerable subgroups experience them differently. However, vulnerable groups may not have equal opportunity to participate in research and, hence, may be underrepresented or missing entirely. Unless study findings are reported by subgroup, any differences among those included will be masked.
Objectives: As part of a systematic review (SR) exploring the effectiveness of remote therapies for drug/alcohol misuse, we undertook equity-related work with the objectives of understanding (1) who was excluded from the research, (2) who was represented in the study populations and (3) whether sub-groups experienced outcomes differently.
Methods: For each included primary study we assessed (1) the exclusion criteria; 2) the sociodemographic characteristics reported using the PROGRESS-plus framework; and (3) whether substance use outcomes were reported by equity-relevant characteristics and, if so, whether there was any variation.
Results: A total of 52 studies were included in the review, of which 47 had eligibility criteria with the potential to result in the exclusion of vulnerable subgroups, most notably those with mental health conditions (MHCs) and those without access to digital technology. Few studies justified the exclusions or reported how many people were excluded based on each criterion. Sociodemographic characteristics of the study populations were reported for all studies, although what the characteristics were and how they were measured varied. Most commonly and consistently reported were gender, age and ethnicity but it was rare that the study outcomes were reported by these or other factors.
Conclusions: The exclusion of certain vulnerable subgroups from the research process and the lack of reporting of outcomes by potential axes of inequalities have important implications for the interpretation and generalizability of findings of SRs. Particularly troubling in our review was the frequency with which those with MHCs were excluded. Understanding how different groups experience interventions is key to allow practitioners to be able to make informed decisions about their adoption in specific vulnerable groups and to ensure that interventions do not inadvertently produce, or exacerbate, inequities.
Patient, public and/or healthcare consumer involvement: An advisory group from policy and practice inputted to all aspects of the SR.

11:25 AM - 11:35 AMMapping the Maps: Methods and Uses of Evidence and Gap Maps.Mapping evidence

Background: Evidence and Gap Maps (EGMs) sit within a family of evidence synthesis methods that seek to address broader research questions. EGMs are a valuable tool in which a wider understanding is needed of existing research in a topic area and locating evidence gaps. They do not synthesise existing evidence, but by locating, categorizing, coding, and presenting the evidence in an interactive web based tool, with links to the primary research, they offer a valuable visualisation of existing evidence. Their popularity is growing, and they are particularly valued by policy makers.
Objectives: We undertook a scoping review to address the following questions: 1) What methods are currently used in their development and updating; 2) to what extent do they adhere to recommended guidance; 3) what procedures are in place for updating and maintaining EGMs; and 4) what methods are in place to evaluate the use of web based EGMs.
Methods: We developed a search strategy that included websites as well as database searches. We contacted hosting organisations to locate published EGMs. We used a piloted data extraction table to gather data. We only included EGMs with public facing online maps. We also retrieved any linked or supporting documentation.
Results: We included 96 EGMs addressing a wide range of topics, though they were predominantly in the field of international development, social care, and public health. Their potential in health care is currently less well realised. Approximately 43% did not undertake critical appraisal of the evidence. More than half of the EGMs were not linked to a description of their methodology or information that described the search dates or plans to update the map.
Conclusions: EGMs are an increasingly adopted approach in evidence synthesis and valued by policy makers. They are valuable tools for visualising evidence and highlighting gaps. Yet the guidance for their development is limited, and there is little standardisation in the processes of linking maps to the methods underpinning them. EGM processes must be more transparent and rigorous. Patient, public, and/or healthcare consumer involvement: Extensive consultation with the public on views on EGMs

11:25 AM - 11:35 AMAn assessment of the design-by-treatment interaction model for network meta-analysis inconsistencyNetwork Meta-analysis

Background: Network meta-analysis (NMA) is a powerful method that simultaneously synthesizes evidence from studies addressing the same clinical question comparing multiple interventions. The method allows inferences based on direct and indirect comparisons in a network. However, NMA results are reliable only when the prerequisite assumptions are met. Of interest, the consistency assumption requires that direct and indirect evidence in a network is in agreement. The design-by-treatment (DBT) interaction model is considered the best method to date; however, its statistical properties have not been well studied for complex networks.
Objectives: To assess the Type I error and Power of the inconsistency estimator from the DBT interaction model in triangular networks with arms denoted A, B, and C.
Methods: A simulation study in which, over 10,000 repeated iterations, we will simulate network meta-analysis data over a range of scenarios, fit frequentist, random-effect network meta-analysis models, and estimate the inconsistency of the network using the DBT model. From the 10,000 iterations, we estimate the estimator’s Power and Type I error based on the observed p-values. We consider varying values for the true odds ratio for the AB- (i.e., 0.65, 1.2, and 1.4) and AC-comparison (0.75, 1, and 1.4), inconsistency factor (0, 0.3, and 1), and number of studies between comparisons (1, 2, 5, and 10). Preliminary
Results: The power of the inconsistency test ranged from approximately 0.5 to 0.75, depending on the simulation scenario. Furthermore, the Type I error of the test ranged from approximately 0.4 to 0.45. Preliminary results indicate that the main driver of Power and Type I error is the assumed inconsistency factor in the data-generating mechanism.
Conclusions: Preliminary results indicate that the DBT inconsistency estimator suffers from a high Type I error and lacks sufficient Power to reliably detect inconsistency in a network. This suggests that further methodological work in assessing network inconsistency may be necessary so that NMAs used in informing decision-making are trustworthy. We intend to expand the simulations in our study to reflect other types of networks observed in practice (e.g., different geometries).
Patient, public and/or healthcare consumer involvement: Patients were not involved at this stage of the project.

11:25 AM - 11:35 AMThe implementation of policy to manage potentially problematic studies in Cochrane review updates – a meta-epidemiological studyResearch integrity, transparency and fraud

Background: Cochrane reviews are regularly updated to reflect the most up-to-date evidence on health interventions. In 2021, Cochrane introduced a new policy to manage potentially problematic studies that are eligible for inclusion in a Cochrane review. This includes the management of studies with retractions, expressions of concern, and concerns about the trustworthiness of the data. It remains unclear how well such a policy is implemented for published Cochrane review updates.
Objectives: To provide an overview of the frequency of problematic studies included in Cochrane reviews and whether they have been excluded in Cochrane review updates according to Cochrane’s new policy.
Methods: We searched the Cochrane Database of Systematic Reviews to identify intervention review updates published in 2022 and their previous version. We compared these two versions to identify studies that were included in the previous version and excluded in the 2022 update. We also identified retracted studies that were included in the previous version by searching the Retraction Watch Database and assessed whether these were excluded from the updated version. Additionally, we recorded whether and how Cochrane’s new policy was implemented in the updates.
Results: We identified 123 pairs of Cochrane reviews from 37 Cochrane groups. The previous version of these reviews was published between 1996 and 2021, with a median of six included studies and 607 participants. There were 16 (13%) reviews with at least one previously included study excluded in the 2022 updates, including 4 (3%) excluding at least one (median 4, range 1-54) potentially problematic study. The previous version of two (2%) reviews included at least one retracted study, and one (1%) review did not exclude them in the 2022 update. Sixteen (13%) updates clearly specified the methods to assess and/or handle problematic studies, including 4 (3%) using a trustworthiness assessment tool and 12 (10%) mentioning the handling of retracted studies.
Conclusions: To facilitate the production of trustworthy evidence, it is imperative to identify and appropriately manage problematic studies eligible for Cochrane reviews. This study reflects the suboptimal implementation of Cochrane’s new research integrity policy in review updates, suggesting further guidance is needed.

11:35 AM - 11:45 AMChallenges and potential solutions for reporting findings from multicomponent meta-regression models in GRADE summary of findings tablesAssessing evidence certainty

Background: We recently completed a Cochrane review of diabetes quality improvement (QI) trials that aimed to identify promising QI strategies (or combinations of strategies) to deploy in practice or examine in future research. We coded interventions according to the presence of absence of 12 QI strategies and assessed the association between intervention components and postintervention outcomes with Bayesian meta-regressions. This is atypical for a Cochrane review—our analysis and output do not match the usual Cochrane processes—and thus necessitated adjustments in our reporting of results to align with Cochrane procedures.
Methods: We will describe the misalignment between the goals and outputs of our analysis with the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) approach for producing Cochrane Summary of Findings (SoF) tables and our operationalizations and solutions.
Results: The standard approach for producing an SoF table is to have a table for each intervention of interest. This was not feasible in our review, as studied interventions were diverse and rarely replicated and did not capture the full range of possible interventions that could comprise combinations of the 12 QI strategies. Most importantly, the standard SoF table was inconsistent with the goal of our analyses, which was learn across (diverse combinations of QI strategy) interventions to develop a theory of which QI strategies (or combinations) of strategies may be most promising for future practice or research. We therefore adjusted GRADE tables to report findings as they pertained to each QI strategy for each outcome. Additionally, we parsed data to inform GRADE assessments for each component (e.g., sample size, risk of bias, precision, consistency and directness) to rate certainty of evidence. Discussion: GRADE is a transparent tool for communicating review findings. Although the goal of our analyses did not match the standard setup of GRADE SoF tables, we found it feasible to adapt SoF tables to better align with our synthesis objective. Further GRADE guidance may be required to help guide future and ongoing similar reviews aimed at teasing apart active ingredients of interventions, rather than interventions as a whole.

11:35 AM - 11:45 AMHow well do we consider equity in efficiency analysis studies of vaccines? A systematic review of equity-informative economic evaluations of vaccinesGlobal health, equity, diversity and inclusion

Background: The Immunization Agenda 2030 prioritizes the populations without access to vaccines. Health equity has been increasingly incorporated into economic evaluations of vaccines to foster equitable access. Robust and standardized methods are needed to evaluate health equity impact of vaccination programs to ensure monitoring and effective addressing of inequities. However, methods currently vary and potentially affect the application of findings to inform policy decision-making.
Objectives: To identify and summarize economic evaluations of the health equity impact of vaccines and immunization programs, focusing on the methods and applications.
Methods: We searched PubMed, Embase, Econlit, and CEA Registry up to December 15, 2022. We included articles that met the following eligibility criteria: full-text articles of economic evaluations estimating costs, outcomes, and health equity impact of vaccines across equity-relevant subgroups in any context. We summarized how health equity was incorporated and evaluated in the selected studies. We performed reporting quality assessment using the Consolidated Health Economic Reporting Standard (CHEERS) 2022 statement.
Results: Twenty-one studies were included that performed health equity impact analysis to estimate the distributional impact of vaccines, such as deaths averted and financial risk protection, across equity-relevant subgroups. Eleven studies performed only health equity impact analysis as part of cost-effectiveness analyses to estimate the distributional impact and subpopulation incremental cost-effectiveness ratios of vaccines. Nine studies are Extended Cost-Effectiveness Analyses (ECEAs) that performed health equity impact analysis of vaccines with an estimation of the distributional financial risk protection. One study is a Distributional Cost-Effectiveness Analysis (DCEA) that performed a health equity impact analysis of vaccines incorporating equity-weighting and opportunity costs as the money was displaced to be spent on vaccines instead of other health services. These studies showed similar findings that introducing vaccines or improving vaccination coverage resulted in fewer deaths and higher financial risk benefits in subpopulations with higher disease burdens and lower vaccination coverage, particularly poorer income groups and those living in rural areas.
Conclusions: Methods to incorporate equity have been evolving progressively. Vaccination programs can enhance equity if their design and implementation address existing inequities in order to provide equitable vaccination coverage and achieve health equity. Patient, public, and/or healthcare consumer involvement: None.

11:35 AM - 11:45 AMMapping Reviews, Scoping Reviews and Evidence and Gap Maps (EGMs) – Same but Different. The ‘Big Picture’ Review FamilyMapping evidence

Background: Scoping reviews, mapping reviews and evidence and gap maps (EGMs) are evidence synthesis methodologies that address broad research questions, aiming to describe a bigger picture rather than address a specific question about intervention effectiveness. They are being increasingly used to support a range of purposes including guiding research priorities and decision-making. There is, however, a confusing array of terminology used to describe these different approaches.
Objectives: In this commentary we aim to describe where there are differences in terminology and where this equates to differences in meaning. We demonstrate the different theoretical routes that underpin these differences. We suggest ways in which the approaches of scoping and mapping reviews may differ in order to guide consistency in reporting and method.
Methods: We undertook a critical analysis of the methods and cited methods used in published scoping, mapping and EGM reviews and guidance.
Results: Scoping reviews, mapping reviews and evidence maps are terms that are not used consistently within the literature, with different terms used to describe similar approaches and review objectives and sometimes the same term is used to describe different approaches. We show the scientific traditions that have shaped these approaches. Scoping reviews tend to have a more focused question and data extraction and analysis is richer than in mapping reviews and EGMs. EGMs have a unique place within the range of evidence synthesis approaches for greater transparency in identifying gaps in the evidence. EGMs are visual and interactive web-based tools. We propose that mapping and scoping reviews and EGMs have similarities that unite them as a group but with their unique differences. The accompanying graphic summarises these differences (Figure 1).
Conclusions: Understanding these similarities and differences is important for informing the development of methods used to undertake and report these types of evidence synthesis. We hope this work will help to improve consistency in description and reporting.
Patient, public and/or healthcare consumer involvement: None.

Figure 1.pdf

11:35 AM - 11:45 AMUsing arm-based network meta-analysis for binary outcomes for generalizability of findings across baseline risk valuesNetwork Meta-analysis

Background: Differences in baseline risk (BR) across studies may be a source of heterogeneity in meta-analysis (1; see attachment for a full list of references); thus, generalizability of findings across subgroups of patients requires constancy of treatment effects across BRs (2). In the case of risk ratio (RR), its independence from BR cannot hold due to its range limitations (2). Odds ratio (OR), instead, is theoretically independent from BR (3), but this property may not hold in practice (4). Moreover, in the specific case of using OR in network meta-analysis (NMA), even factors affecting baseline risk but not conditional effect may lead to inconsistency (5). Thus, regardless of the outcome measure adopted, in the case of binary outcomes, treatment effects should be considered as likely dependent from BR.
Objectives: To suggest a method making results from NMA for binary outcomes more reliable and generalizable, by expressing their dependence on BR, to equip clinicians and patients with better predictions of potential outcomes for each treatment option.
Methods: The arm-based (AB) model from (6) estimates log-odds of risks for each treatment arm by allowing both for dependence between baseline risk and treatment effect and for random effects (guaranteeing evaluation of heterogeneity and inconsistency). After estimating the logit of the probability of baseline risk and treatment effect, marginal treatment and BRs can be derived and marginal estimates of the desired effect size calculated (4).
Results: We will present an example of use of the AB model from (6) with a dichotomous outcome, showing pitfalls of the comparison-based approach using both OR and RR as outcome measures.
Conclusions: The never-ending debate about whether OR or RR should be used as an effect measure seems to be focused lately on ‘portability’, i.e., on their possible independence from BR (7). Owing to the general implausibility of such assumption in practice for both outcomes, we argue that NMA for binary events should adopt an AB approach, in order to express how results depend on BR, regardless of the outcome measure adopted.
Patient, public and/or healthcare consumer involvement: No patient, public or healthcare consumer has been involved.

references_abstract_Federico_Tedeschi_Colloquium_2023.pdf

11:35 AM - 11:45 AMBreaking Ground or Breaking Bad? Examining the Fate of Preprints in Prevention Research with a Mixed Methods StudyResearch integrity, transparency and fraud

Background: The availability of unsound or scientifically invalid work is a concern associated with preprint articles. The popularity of preprints in prevention research has increased due to the COVID-19 pandemic.
Objectives: The objectives of this study were to assess the consistency of results and conclusions of preprints in prevention research compared to peer-reviewed articles and to explore the perception of key stakeholders regarding the growing number of published preprints.
Methods: The study employed a mixed-methods approach that involved the development of a Python-based Web crawler to search MedRxiv for prevention preprints from January 1 to September 30, 2020. We ran an update search one year later to identify which preprints were published. We dually screened all results for prevention articles and developed a scheme to classify changes in effect sizes and conclusions. We also conducted and analyzed 19 qualitative online interviews with stakeholders who have expertise in prevention.
Results: The WebCrawler retrieved a total of 2,238 preprints, of which 594 were prevention research studies and 329 were epidemiological studies. Nearly half of these studies (48.9%) were published in peer-reviewed journals within one year, with a median time from upload to publication of 5.3 months (range: -0.1 to 11.3 months). Among published preprints, 16.8% of articles showed a change in effect size, which was a major change in 4.4% articles (i.e., a change greater than 25% of the original effect size). The conclusions changed in 43% of the studies, mostly in terms of style or wording (39%). Stakeholders perceive that preprints have advantages over peer-reviewed articles, including free access, fast submission, transparency, and feedback from colleagues. However, they also feel that preprints are not compatible with current quality standards and feedback culture and may have a detrimental effect on publication and career. Conclusion: A few prevention research articles experience changes in effect estimates and conclusions when they are published. Although these changes are small in number, they affect one in every 23 articles. Although preprints may seem like an attractive publishing option, there is still skepticism about their quality and potential consequences. We therefore warrant caution of using preprints of prevention research in decision-making.

11:45 AM - 11:55 AMIndia covid guidelines- a living synthesis and guideline processAssessing evidence certainty

Background: COVID-19 has had an unprecedented impact worldwide on health, healthcare, societies, and economies. Evidence for interventions emerged rapidly but was difficult to examine systematically in a short span of time. Few guidelines used formal evidence synthesis and GRADE approaches, particularly when tailoring to low- and middle-income countries (LMICs). To address this gap, Christian Medical College, Vellore, India, partnered with the Clinical Infectious Diseases Society of India (CIDS), Cochrane Infectious Diseases Group, Prof BV Moses Center for evidence-informed healthcare, and experts from 16 institutions in India and globally to form The Covid Management Guidelines India Group.
Objectives: To produce guidelines for the management of patients with acute COVID-19 in secondary and tertiary care settings in India.
Methods: The process incorporated GRADE and Cochrane Rapid Review approaches to identify and prioritise questions in areas of equipoise, screen and extract data from available studies, synthesise and determine certainty in the evidence, and enable expert working groups to produce recommendations or best practice statements tailored for India and other LMICs using the GRADE Evidence to Decision framework, which was all disseminated on a widely accessible platform: https://indiacovidguidelines.org.
Results: In the course of the project, clinicians across the country were trained in the development of guidelines using formal evidence synthesis and the GRADE approach. The guideline group comprising core, steering, methodology, evidence synthesis, dissemination, and intervention expert working groups with clearly defined roles and responsibilities synthesised guidelines pertaining to the use of interventions used in the treatment of COVID-19, including antivirals, anti-inflammatory, anticoagulation, antibodies, respiratory support, and other supportive management. Although acceptability and applicability of the guidelines is being investigated formally through a survey, one example of impact is that one state (Kerala, India) adopted our recommendation on anticoagulation in the state’s COVID-19 guidelines. An external advisory panel ensured scientific challenge and applicability to settings outside of those in which the experts practice.
Conclusions: This unique partnership applied a transparent GRADE approach to ensuring scientific rigor and an evidence-based approach taking into account local contextual factors in making clear timely guidelines relevant to India.

11:45 AM - 11:55 AMMethods used in systematic reviews to conceptualise dimensions of health equity impacts of public health interventions: umbrella reviewGlobal health, equity, diversity and inclusion

Background: Systematic reviews of interventions often fail to adequately consider equity. This impairs their ability to inform policy and practice decisions that aim to account for differential impacts. Central to this issue is how the dimensions of equity impacts (e.g., socioeconomic status) are conceptualised, as this shapes how equity is treated in the conduct and reporting of reviews.
Objectives: To describe the methods and frameworks (e.g., PROGRESS-Plus) used in systematic reviews to conceptualise the dimensions of equity impacts of public health interventions and issues encountered in their application. This complements a Cochrane review (Welch 2022) that surveyed the descriptive and analytic methods used to investigate health equity via PROGRESS-Plus.
Methods: Umbrella review of systematic reviews that include a focus on equity impacts of public health interventions. We used electronic database searches supplemented with automated searches of the OpenAlex dataset. An active learning algorithm was used to prioritise title-abstract records for manual screening. We extracted data from a purposively selected sample of reviews.
Results: We manually screened 2,060 prioritised title-abstract records, from which 322 full-text reports were assessed. We included 120 reports of systematic reviews. PROGRESS-Plus was the only formal method used for conceptualising dimensions of equity impacts, although most reviews that did not use it used equivalent dimensions (68/75 (91%)). Where intended methods were unable to be applied fully, this was usually because primary research studies did not report the necessary information (55/68 (81%)). Most reviews (87/120 (73%)) included an explicit rationale for focusing on equity impacts in general, but only 7% (8/120) justified their focus on (or exclusion of) specific dimensions. Authors highlighted challenges with applying these methods, including investigating constructs, such as socioeconomic status, that lack standardised operationalisation and measurement.
Conclusions: PROGRESS-Plus is the predominant method for conceptualising dimensions of equity impacts of public health interventions and mostly appears to be sufficiently comprehensive to encompass the scope of such investigations. However, significant practical and conceptual challenges need to be addressed if reviews assessing these impacts are to more meaningfully contribute to cumulative and useful evidence bases. Patient, public, and/or healthcare consumer involvement: None, but can contribute to more robust evidence production.

11:45 AM - 11:55 AMA digital map of systematic reviews on non-pharmacological interventions to inform policy making in infectious disease controlMapping evidence

Background: The COVID-19 pandemic accentuated the need for comprehensive evidence synthesis to guide decisions regarding infection disease control and preparedness. In Sweden, an investigator was tasked by the government with reviewing the Infection Control Act and analyzing the need for new regulations for future pandemics, with special considerations to measures that were effective during the COVID-19 pandemic. Against this background, SBU was requested to provide support to the investigator.
Objectives: To facilitate policymaking by conducting an overview of systematic reviews of nonpharmacological interventions (NPIs) and present the results in an interactive, user-friendly format.
Methods: A PICO was formulated comprising 10 NPIs used to prevent the spread of infections during outbreaks of COVID-19, SARS, MERS, Influenza and Ebola. A structured literature search for systematic reviews was conducted in Medline and Scopus. Eligible reviews were assessed for risk of bias using the AMSTAR checklist. The reviews were mapped according to risk of bias level and type of interventions and infections targeted and incorporated in a digital map.
Results: Seventy-eight systematic reviews were identified and included in the map. Of those, 43 were assessed as having high risk of bias and 35 as having low to moderate risk of bias. Most reviews focused on the effectiveness of various physical distance interventions or the effectiveness of facemasks. The vast majority of reviews and primary studies were conducted during the COVID-19 pandemic. The digital map facilitated navigation among the reviews and was used to support the legislative process.
Conclusions: Digital maps may be a useful tool to present and visualize available evidence and evidence gaps in an intuitive and interactive format that facilitates navigation. They can aid decision-making in complex fields such as public health measures, where it is challenging but important to identify and take into account the best available evidence of potential benefits and harms.
Patient, public and/or healthcare consumer involvement: Not applicable.

11:45 AM - 11:55 AMA novel modeling approach for producing treatment hierarchies in network meta-analysisNetwork Meta-analysis

Background: Network meta-analysis (NMA) allows synthesising the evidence simultaneously on multiple treatments. A key output of NMA is the relative ranking of the treatments; nevertheless, it has attracted a lot of criticism. This is mainly because ranking is a very influential output and, thus, prone to over-interpretations even when relative effects imply small differences between the alternative treatments. To date, common ranking methods rely on score metrics which are calculated based on the summary effects. Such metrics lack a straightforward interpretation, although it is still unclear how to measure their uncertainty.
Objectives: To introduce a new modelling approach for networks of interventions that produces treatment hierarchies accounting for the clinical importance of the study-specific relative effects as well as for their uncertainty.
Methods: We adapt methodology previously suggested for ranking in sports tournaments into the context of NMA. We use extensions of the so called ‘Bradley-Terry’ models, which are a family of probabilistic models that aim to predict the outcome of pairwise comparisons. We first translate the study-specific relative effects and their confidence intervals into wins, losses, and ties based on predefined minimally clinically important differences between treatments. Then, based on the number of wins, our model estimates the ‘worth’ of each treatment which is used as an intuitive measure to rank the treatments. This approach naturally captures the uncertainty of ranking because the estimates of treatment worth are accompanied by confidence intervals. The model also allows to consider simultaneously multiple outcomes in ranking by implementing a vector of several contrasts for each outcome. Finally, study precision and other important characteristics, such as risk of bias, can be incorporated by an additional parameter that represents the ‘importance’ of each study. Results and
Conclusions: We illustrate our model using a Cochrane NMA comparing 7 treatment classes for chronic plaque psoriasis. Our model is able to clearly indicate the two classes with the largest worth but also classes with similar worth. The latter is a major strength of our approach as it preserves from exaggerating unimportant differences between treatments and drawing spurious conclusions. Patient, public, and/or healthcare consumer involvement: None.

11:45 AM - 11:55 AMComparison of effect estimates between preprints and peer-reviewed publications: a meta-epidemiological study of COVID-19 trialsResearch integrity, transparency and fraud

Background: Preprints have emerged as a major source of research communication during the COVID-19 pandemic. However, questions were raised concerning the reliability of their results.
Objectives: To evaluate whether effect estimates differ between preprint and peer-reviewed journal randomised controlled trials (RCTs).
Methods: Data were derived from the COVID-NMA (covid-nma.com) initiative, a living systematic review of RCTs evaluating preventive interventions, treatments and vaccines for COVID-19. Meta-analyses with at least one preprint and one peer-reviewed journal article evaluating pharmacological treatments vs. standard of care/placebo were included up to July 20, 2022. Predefined COVID-NMA ‘critical outcomes’ at 28 days were considered. A meta-epidemiological analysis estimated the difference in effect estimates [expressed as the ratio of odds ratio (ROR)] between preprint and peer-reviewed journal RCTs. An ROR of <1 indicated that preprint trials yielded larger effect estimates.
Results: Thirty-seven meta-analyses (114 RCTs—44 preprints, 70 peer-reviewed journal articles) were selected. Twenty-four meta-analyses assessed hospitalized patients (81 RCTs), and 13 assessed outpatients (33 RCTs). The median number of RCTs per meta-analysis was 2 (IQR, 2-4; maximum, 11). The median sample size of RCTs was 199 (IQR, 99-478), 68% were prospectively registered, 67% received industry or mixed funding, 79% were multicentric trials, and 75% had some concerns of overall risk of bias. Overall, there was no significant difference in effect estimates between preprint and peer-reviewed journal trials (ROR, 0.88; 95% CI, 0.71-1.09; I2 = 17.8%; τ2= 0.06) (Figure 1). Post-hoc sensitivity analyses of meta-analyses with only two trials (ROR, 0.86; 95% CI, 0.51-1.45; I2 = 22.2%; τ2= 0.19) and at least three trials (ROR, 0.98; 95% CI, 0.84-1.14; I2 = 0.0%; τ2= 0.00) found consistent results.
Conclusions: No important difference in the treatment effect between preprints and peer-reviewed publications was found, particularly in meta-analyses with at least three trials. Systematic reviewers and meta-analysts should assess preprint inclusion individually, accounting for risk of bias and completeness of reporting.
Patient, public and/or healthcare consumer involvement: Peer review is a lengthy process that, during a pandemic, can inadvertently cost lives. Because of the demand for faster access to scientific knowledge, preprints offer a solution, particularly to patients.

Figure1.jpeg

11:55 AM - 12:05 PMUsing GRADE-CERQual to assess confidence in findings from qualitative evidence syntheses: how well are review authors applying the approach?Assessing evidence certainty

Background: Assessing how much confidence users can place in qualitative evidence synthesis (QES) findings is important for producing trusted qualitative evidence on patients’ experiences and perspectives of health issues and interventions. Applying the GRADE-CERQual approach to assess confidence in the evidence is a required step in a Cochrane QES. Use of GRADE-CERQual has increased rapidly globally.
Objectives: We evaluated how GRADE-CERQual has been used in the literature by, firstly, describing GRADE-CERQual’s uptake in QES; and, secondly, developing and applying reporting and fidelity criteria to identify good examples and areas for improvement.
Methods: We undertook citation searches in six databases for 18 key publications on the GRADE-CERQual approach and also used keyword searches. Two researchers screened the output, and publications identified as evidence syntheses that used GRADE-CERQual went on to full-text coding and charting. Reporting and fidelity criteria were developed and then applied using NVivo12 software. We used qualitative content analysis and descriptive statistical approaches to analyse the data.
Results: Our searches identified 1,312 records. Two hundred thirty-three of these were reviews claiming to have applied GRADE-CERQual. Approximately 41.6% (97 studies) had seriously misapplied the GRADE-CERQual approach and were excluded from further fidelity and reporting assessment. Serious misuses were applying GRADE-CERQual in quantitative evidence synthesis, interpreting it to be a critical appraisal tool, and applying it to review findings as a whole, instead of individual review findings. One hundred thirty-six studies applied it as intended to individual review findings. We categorised the most common reporting issues into three broad areas: labelling, terminology, and completeness. The most common fidelity issues centred on the conceptualisation and application of the four GRADE-CERQual components. Fewer fidelity concerns were identified in syntheses citing the most recent guidance compared to those citing earlier publications. Conceptual challenges were identified around distinguishing between quality and confidence, between themes and review findings, and between making assessments at the study level versus the review finding level.
Conclusions: Findings from this evaluation help to inform new agendas for the field of QES, identify topics for further GRADE-CERQual guidance development, and support review authors to avoid common pitfalls and improve reporting and fidelity.

11:55 AM - 12:05 PMOperationalizing the GRADE-Equity criterion to inform guideline recommendations: Application to a Medical Cannabis guidelineGlobal health, equity, diversity and inclusion

Background: Equity is at the forefront of clinical and public health practice and policy. The incorporation of equity in guideline recommendation requires additional considerations for development methodology. The GRADE working group proposed the consideration of equity in several stages of the guideline development process, such as applying an equity lens in the conduct of the evidence syntheses. However, there is no pragmatic guidance on how to collect evidence from systematic reviews to inform equity considerations in guideline recommendations.
Objectives: We operationalized the GRADE-Equity criterion for collecting and appraising evidence from primary studies of systematic reviews to inform guideline recommendations. We demonstrated the application of this plan in a clinical practice guideline on prescribing medical cannabis for patients with chronic pain.
Methods: We developed the operationalization plan through three steps. First, we reviewed guidance related to guideline development, including the GRADE-Equity series, GRADE evidence to decision (EtD) frameworks. Second, we drafted the operationalization plan using the four signalling questions for the “impact on health equity” criterion of the GRADE EtD and refined the draft based on analyses of reported equity considerations in primary studies. Third, we presented the plan to the steering committee of the medical cannabis guideline and developed an evaluation framework to pilot our approach using primary studies identified from systematic reviews of benefits and harms, values, and preferences.
Results: We propose the following seven-step operationalization plan: 1) identify populations experiencing inequities, 2) examine available data for specific populations, 3) evaluate population baseline risk for primary outcomes, 4) assess representation of these populations in primary studies, 5) appraise analyses, 6) identify barriers to implementation of effective interventions for these populations, and 7) suggest supportive strategies to facilitate implementation of effective interventions. The evaluation of each study ranged from 10 to 30 minutes, depending on the amount of detail provided in the studies. We present examples describing its application to the cannabis guideline.
Conclusions: We present a pragmatic approach to inform equity considerations of systematic reviews informing guideline recommendations. The application of this approach in different topics is needed to test its reliability and feasibility.
Patient, public and/or healthcare consumer involvement: no.

11:55 AM - 12:05 PMNovel methods used when conducting an evidence gap map surrounding interventions for treating obstetric fistulaMapping evidence

Background: Evidence gap maps (EGMs) are a novel approach to establishing the breadth and depth of evidence on health. Their visual nature promotes knowledge translation to key stakeholders, including patients, practitioners and policymakers. The methods used to produce EGMs are systematic and rigorous but are open to challenges owing to their emerging nature in the canon of evidence synthesis. These challenges resulted in novel methods used within the production of an EGM surrounding interventions for obstetric fistula.
Objectives: To describe two novel methods used in conducting an EGM of interventions for treating obstetric fistula.
Methods: Although development of a framework is mandatory for EGMs, how stakeholders should be involved in establishing the framework is currently unclear. As such, a survey of key stakeholders (including patients and practitioners) was produced using Qualtrics to establish their perspectives on the most important interventions and outcomes to prioritise within the framework. The EGM was then produced in line with recommendations from the Campbell Collaboration and followed a prepublished protocol. Data synthesis was conducted in EPPI-Mapper. Sensitivity analyses using filters applied to the EGM were performed to determine where sufficient homogeneity to perform future evidence syntheses may be possible; this is not currently part of EGM guidance.
Results: Involving key stakeholders in the development of the EGM formed the structure of the evidence map by arranging the framework by what was considered most important to them. However, only 39 responses were received; focus groups or individual interviews may have garnered further insight, particularly if carried out in areas of high fistula prevalence. The reporting of the sensitivity analyses highlighted deficiencies in the evidence base; only 7 of 28 included studies [a combination of randomised controlled trials (RCTs) and non-RCTs] remained in the EGM and further illuminated the type of new primary data needed.
Conclusions: Further guidance on stakeholder involvement in EGMs, as well as development in the technology and methodologies supporting the production of EGMs, may help to address some of the current challenges faced in their production.
Patient, public and/or healthcare consumer involvement: Patients and the public were involved in answering the survey that determined the framework.

11:55 AM - 12:05 PMInconsistency identification in Network Meta-Analysis via Stochastic Search Variable SelectionNetwork Meta-analysis

Background: The reliability of the results of a network meta-analysis (NMA) lies in the plausibility of the key assumption of transitivity, which implies that the effect modifiers’ distribution is similar across treatment comparisons. Transitivity is statistically manifested through the consistency assumption, which suggests that direct and indirect evidence are in agreement. Several methods have been suggested to evaluate consistency. A common approach for testing network consistency suggests adding inconsistency factors to the NMA model.
Objectives: To evaluate the consistency assumption of NMA in a Bayesian framework.
Methods: We describe each inconsistency factor with a candidate covariate whose inclusion on the model relies on variable selection techniques. Our proposed method, Stochastic Search Inconsistency Factor Selection (SSIFS), evaluates the consistency assumption both locally and globally by applying the stochastic search variable selection method to determine whether the inconsistency factors should be included in the model. The posterior inclusion probability of each inconsistency factor quantifies how likely a specific comparison is to be inconsistent. We use posterior model odds or the median probability model to decide on the importance of inconsistency factors.
Results: Differences between direct and indirect evidence can be incorporated into the inconsistency detection process. A key point of our proposed approach is the construction of a reasonable “informative” prior concerning network consistency. The prior is based on the elicitation of information-derived historical data from 201 published network meta-analyses. The performance of our proposed method is evaluated in two published network meta-analyses.
Conclusions: SSIFS is a novel Bayesian method that evaluates the consistency assumption both globally and locally. The proposed method is publicly available in an R package called ssifs, developed and maintained by the authors of this work. Patient, public, and/or healthcare consumer involvement: -

11:55 AM - 12:05 PMHow should we handle predatory journals in evidence synthesis?Research integrity, transparency and fraud

Background: Synthesizers of evidence are increasingly likely to encounter studies published in predatory journals during the evidence synthesis process. Predatory journals and the studies published within them have caused significant disruption across the scientific landscape and present unique concerns within academia. This is due to the lack of transparent editorial oversight they employ, which may increase the risk that the studies published within these journals are erroneous or fraudulent.
Objectives: Under the broader aim to develop methodological guidance for the use of studies published within predatory journals in evidence syntheses, the objective of this research was to explore the attitudes, opinions and experiences of experts in the synthesis of evidence regarding predatory journals.
Methods: A global, descriptive survey-based cross-sectional study was carried out between April 1 2021 and June 1 2021. The survey was sent to prominent bodies in the field of evidence-based healthcare and systematic reviews, including JBI, Cochrane, Guidelines International Network (GIN), Campbell Collaboration and GRADE. The study utilized a self-administered questionnaire that was coded and disseminated online through Survey Monkey for data collection from the target participant. Because the target audience was those who had experience in evidence synthesis, survey logic was used to remove responses from respondents who lacked this experience.
Results: Two hundred and sixty-four evidence synthesis experts responded to this survey. Most respondents agreed with the definition of a predatory journal (86%); however, several (19%) responded that this definition was difficult to apply practically. Many respondents believed that studies published in predatory journals are still eligible for inclusion into an evidence synthesis project. However, this was only after the study had been determined to be ‘high-quality’ (39%) or if the results were validated (13%). Only 32% of the respondents previously used a checklist or tool to identify a predatory journal.
Conclusions: The results of this project identify that there is a need for more consensus-based guidance regarding the inclusion of studies published within predatory journals into an evidence synthesis project. While critical appraisal of these studies is an expected quality control method, evidence synthesis authors are urged to consider additional steps in their future evidence synthesis projects.

12:05 PM - 12:15 PMApplying GRADE-CERQual to interpretive review findings: Reflections from a Cochrane meta-ethnography on childhood vaccination acceptanceAssessing evidence certainty

Background: GRADE-CERQual (Confidence in the Evidence from Reviews of Qualitative research) was developed to support the increasing use of review findings from qualitative evidence syntheses within policy and decision-making. To date, the approach has mainly been applied to aggregative synthesis methodologies and descriptive review findings. Current GRADE-CERQual guidance suggests there is a need for testing with more diverse review methods and outputs, which will provide important insights for the ongoing evolution of the approach.
Objectives: This presentation aims to contribute to the evolution of GRADE-CERQual by reflecting on our experiences of applying the approach to the findings that emerged from a Cochrane meta-ethnography on childhood vaccination acceptance. Specifically, we describe the similarities and differences, challenges, and dilemmas we experienced when applying the approach to the more interpretive versus more descriptive findings that emerged from our review.
Results: We found that we were able to apply the core criteria and principles of GRADE-CERQual in ways that were congruent with the methodologies and epistemologies of a meta-ethnographic approach and associated interpretive findings. We also found that the practical application processes were similar for the more descriptive versus interpretive findings. The main differences we experienced related to the level of demand placed on the evidence supporting the review finding and the level of complexity involved with the decisions. Compared with the more descriptive findings, the more interpretive findings required supporting data that were richer, thicker, more contextually situated, and methodologically stronger for us to have the same level of confidence in them. Making the assessments for the more interpretive findings also involved greater dilemmas and more complicated forms of judgement, particularly for the component of coherence. We provide practical examples to illustrate these complexities and how we approached them.
Conclusions: Our experiences of applying GRADE-CERQual to more interpretive review findings generated various challenges and quandaries which we highlight and hope will offer a platform for further engagement. This could enhance the usability of GRADE-CERQual and in turn the kinds of knowledge that count within health decision-making, with ultimate positive effect on patients and other evidence users. Patient, public, and/or healthcare consumer involvement: None.

12:05 PM - 12:15 PMImproving equity, diversity, and inclusion in JournalsGlobal health, equity, diversity and inclusion

Background: Journal editors are the gatekeepers of our research. With increasing evidence that Equity, Diversity, and Inclusion (EDI) are beneficial to research and academia, some editors have expressed their desire to improve EDI of their journals. The Royal Society of Chemistry (RSC) established a minimum set of requirements aimed at improving EDI in scholarly publishing through a joint commitment action plan with more than 56 publishing organizations. There are several commitments in place to improve EDI in journals, however, their effectiveness is yet to be determined.
Objectives: We provide practical approaches for editors, journal publishers, and peer reviewers to improve EDI in academic journals based on the six minimum standards set by the RSC.
Methods: We searched publisher websites, empirical studies, and relevant resources that evaluated the impact of any approach to improve EDI in journals.
Results: We suggest the following six approaches that journal editors can adopt to meet the RSC minimum criteria: (1) adopt a journal EDI statement with clear, actionable steps to achieve it; (2) promote the use of inclusive and bias-free language; (3) appoint a journal’s EDI director or lead; (4) establish an EDI mentoring approach; (5) monitor adherence to EDI principles; and (6) publish reports on EDI actions and achievements. Peer reviewers need to support the author’s needs whether they were language needs by directing them to language services or their clinical conditions that require accommodations. We also provide examples of journals that have implemented some of these strategies to improve EDI.
Conclusions: EDI issues in academia are tightly intertwined with systemic oppression that is integrated in policies and regulations of academic progression. Publishers and journal editors need to test the feasibility of implementing these approaches and assess their impact on improving EDI. Patient, public, and/or healthcare consumer involvement: no.

12:05 PM - 12:15 PMWorking with policy makers to maximise the utility of EGMs: experiences of Exeter Policy and Research Programme Evidence Review FacilityMapping evidence

Background: Evidence and gap maps (EGMs) are used to summarize the quantity, quality, and main characteristics of an evidence base and are an interactive resource for evidence users to identify research that meets their specific interests and requirements. EGMs are of great potential utility for health and social care policy makers who wish to gain an overview of a broad evidence base to inform decision making. However, differences in expectations and understanding of EGMs and systemic review processes between researchers and policy makers means production of EGMs and ensuring their future utility is not always straightforward.
Objectives: To discuss the challenges experienced by the Exeter Policy and Research Programme Evidence Review Facility whilst working alongside policy makers to produce EGMs to inform health and social care policy.
Methods: To establish if an EGM is appropriate to meet the needs of policy makers, we work closely with them to understand the size and main features of the existing literature base through extensive scoping. We provide examples of previous EGMs to support policy makers’ understanding of their functionality and consider how they may use the EGM. Policy makers are consulted on designing the framework and are given the opportunity to revise its format prior to publication of the final report.
Results: Working with policy makers to create EGMs that are relevant and useful can be a protracted process, especially when a focused research question has not been identified. It is not always possible to structure the EGM according to the needs of policy makers or to identify the features of the evidence of interest because of limitations in reporting, which may only become fully apparent after commencing the mapping process.
Conclusions: A better understanding of how EGMs are used to inform policy making would enable researchers to determine the potential strengths and limitations of this approach in different contexts.
Patient, public and/or healthcare consumer involvement: During development of the map, feedback from patients and members of the public is sought to inform the scope and format and ascertain the accessibility of the EGM.

12:05 PM - 12:15 PMEvaluation and development of a novel interactive Summary of Findings table for network meta-analysis. A qualitative user-testing study with cliniciansNetwork Meta-analysis

Background: Clinicians often need to evaluate comparative effectiveness of relevant treatment options for their patients. Network meta-analyses (NMAs) provide an important resource for clinicians to guide decision-making; however, these analyses are complex, and interpretation may prove challenging for users.
Objectives: To evaluate and develop a novel interactive Summary of Findings table (the MATCH-IT tool) for network meta-analysis results, to support clinicians in interpreting and applying comparative effectiveness evidence into decision-making.
Methods: We performed qualitative user-testing with a convenience sample of practicing physicians working in hospitals and general practice in Norway, Belgium, and Canada. User testing entailed a brief introduction with a clinical scenario, which was a think aloud session with participant-tool-interaction followed by a semi-structured interview. The tests were recorded, transcribed, and analysed using directed content analysis. The results informed the iterative development process of the MATCH-IT tool.
Results: Five rounds of user-testing, with 26 participants in total, have resulted in four iterations with updates in the tutorial, layout, navigation functionality, and interphase, as well as the practical-issues-section. We have also added an FAQ section. Clinicians who participated in user-testing had been in clinical practice from 0 to 30 years (median 6, IQR 3-11). Most had little or no prior experience with interpretation of NMA results and sparse knowledge about GRADE methodology. Most clinicians perceived MATCH-IT as easy to interpret and navigate and appreciated its ability to provide overview of the evidence, although some of the interactive features proved to be difficult to discover intuitively. They perceived pictograms and inclusion of practical-issues-information as potentially useful features when interacting with patients. Categorization of results with colour coding to indicate both certainty of evidence and relative effectiveness of interventions was appreciated, and filtering functionality allowed participants to efficiently browse and focus in on the interventions believed to be most relevant for a given clinical scenario.
Conclusions: MATCH-IT shows promise in supporting decision-making for clinicians when presented with results from NMAs. The tool has already been published within Cochrane systematic reviews and clinical practice guidelines. Patient, public, and/or healthcare consumer involvement: Clinicians were the subject for user testing, ensuring user friendliness for this key consumer group.

MatchIt for Clinicians.png

12:05 PM - 12:25 PMImpact of including conference abstracts in systematic reviews: analysis from a living systematic review and network meta-analysis of COVID-19Research integrity, transparency and fraud

Background: Summarizing all available evidence while decreasing bias raises important methodological challenges for systematic reviews (SRs). Grey literature in SRs is important for minimizing the risk of publication bias but is often inadequately reported, which may reduce the credibility of the evidence synthesis.
Objectives: To evaluate the extent of consistency between conference abstracts (CAs) and their final publications and the impact of including results from CAs on meta-analytic effect estimates and the certainty of the evidence (CoE) in our living systematic review and network meta-analysis (SRNMA) for COVID-19.
Methods: We will conduct a retrospective review of all CAs included in our living SRNMA of drug treatments for COVID-19 until January 2023. For all CAs included in our living SRNMA with a subsequent final publication, two reviewers, working independently, will compare the results and risk of bias assessments of the CA and its final publication. To assess the impact of the inclusion of CAs on effect estimates and the CoE, we will include the outcomes most frequently reported in the CAs reporting data for interventions with existing recommendations according to the January 2023 World Health Organization guidelines. We will conduct two analyses, one including the CAs and one excluding them. We will compare the results of these analyses to evaluate the magnitude of the change in the pooled estimate and identify any changes to the CoE.
Results: This work is in progress. We will describe the nature and frequency of discrepancies between CAs and their final publications. We will present how effect estimates and the CoE changed with and without the CAs. We will consider the direction of effect to be different, whether the effect changes from beneficial to harmful or vice versa, considering the thresholds of effectiveness established in the SRNMA. Conclusion: The results of this analysis will highlight the impact of including CAs in a living SRNMA, which can inform eligibility decisions for future evidence syntheses. Understanding the influence of including CAs in network meta-analyses can assist decision-makers with interpreting the results of evidence syntheses with CAs.
Patient, public and/or healthcare consumer involvement: Patient partners were not involved in this study.

12:15 PM - 12:25 PMIncorporating Equity, Diversity, and Inclusion into Cochrane Systematic Reviews with AIGlobal health, equity, diversity and inclusion

Background: Cochrane systematic reviews are a valuable source of evidence-based information for healthcare decision-making. However, it is important to ensure that the analysis of these reviews considers equity, diversity, and inclusion (EDI) to represent diverse populations better and address disparities in health outcomes.
Objectives: This study aimed to assess the feasibility and effectiveness of using artificial intelligence (AI) to incorporate EDI considerations into the analysis of Cochrane systematic reviews.
Methods: A sample of Cochrane systematic reviews was analyzed using AI algorithms to identify EDI-related issues, such as disparities in study populations, missing data on diverse populations, and the under-representation of marginalized groups in the evidence base. The algorithms used machine learning techniques to analyze the data and provide insights into these disparities.
Results: The study found that AI algorithms effectively identified EDI-related issues in the analysis of Cochrane systematic reviews. The AI-powered analysis provided insights into disparities in study populations and the under-representation of marginalized groups in the evidence base. The algorithms also provided recommendations for improving EDI considerations in future studies and systematic reviews.
Conclusions: The study highlights the importance of considering EDI in analyzing Cochrane systematic reviews. Using AI to incorporate EDI considerations is a feasible and effective way to ensure that the information is representative of diverse populations and addresses disparities in health outcomes. The study demonstrates the potential of AI to support EDI in the analysis of evidence-based information and to improve the quality and representativeness of Cochrane systematic reviews. In this study, patient and public involvement was incorporated through consultation with stakeholders and subject matter experts to understand the barriers and facilitators to incorporating EDI considerations in the analysis of Cochrane systematic reviews. This engagement helped to ensure that the AI algorithms developed in this study effectively addressed the needs and perspectives of diverse populations and that the recommendations provided were relevant and meaningful to patients, the public, and healthcare consumers.

12:15 PM - 12:25 PMThe concept of “evidence relevant to” in the rehabilitation field: post COVID-19 condition mapping for the World Health Organization GuidanceMapping evidence

Background: In the current absence of direct “evidence on” rehabilitation interventions for people with symptoms of post-COVID-19 condition (PCC), we can search and synthesize the indirect “evidence relevant to” coming from interventions effective for these symptoms in other health conditions. The World Health Organization (WHO) required this information to inform expert teams and provide specific recommendations in their Guidelines. So we defined “evidence relevant to” as the synthesis of evidence focusing on the rehabilitative management of impairments, activity limitations and participation restrictions, informed by different health conditions presenting with similar symptoms.
Objectives: We were asked by the WHO to synthesize the Cochrane evidence relevant to rehabilitation for PCC-related symptoms: fatigue, postexertional malaise, orthostatic intolerance dyspnea, arthralgia, dysphagia, dysphonia, olfactory disease, cognitive impairment, anxiety and depression.
Methods: We searched the last 5 years’ Cochrane Systematic Reviews (CSRs) using specific terms for each symptom, “rehabilitation” and their synonyms in the Cochrane Library. We extracted the available evidence and summarized it using maps. We grouped the included CSRs for health conditions and interventions, indicating the effect and the certainty of evidence.
Results: Out of 5,734 CSRs, we found 32 for fatigue, 4 for exercise intolerance, 10 for arthralgia, 15 for dyspnea, 1 for dysphagia, 17 for cognitive impairment and 37 for anxiety and depression, published between 2016 and 2021. They provided data from several health conditions, such as cancer, chronic respiratory diseases, osteoarthritis and neurological disorders. Effective interventions for fatigue, exercise intolerance, dyspnea and arthralgia included exercise training and physical activities, telerehabilitation, multicomponent and educational interventions. Dysphagia was mainly treated with swallowing therapy, while cognitive aspects were managed with exercise, cognitive training and educational programs. The overall certainty of evidence was low to very low and moderate/high in a few cases. We did not identify CSRs that specifically addressed postexertional malaise, orthostatic intolerance, dysphonia and olfactory disease.
Conclusions: The current findings served as the basis for the recommendations on treatments for PCC symptoms published in the current WHO Guidelines for clinical practice. These results are the first step of indirect evidence able to generate helpful hypotheses for clinical practice and future research. Patient involvement: NA.

2:05 PM - 2:15 PMBeyond Statistical Significance: Investigating How Systematic Review Authors Communicate Meaningful Differences of Nonsignificant ResultsStatistical methods

Background: For decades, statisticians and methodologists have severely criticized the undue reliance on strict p-value thresholds (usually 0.05) and the misinterpretation of statistical significance in medical research. Despite legitimate criticism, primary research and systematic reviews continue to rely heavily on statistical significance without taking a nuanced approach when interpreting nonsignificant effect estimates.
Objectives: We aimed to evaluate the language employed by Cochrane authors to emphasize differences in nonsignificant treatment effects they considered relevant. We also sought to measure the magnitude of these differences and determine whether they differed from statistically nonsignificant effect estimates that authors interpreted as similar or not different.
Methods: We screened all Cochrane reviews of interventions published between November 2017 and 2022 for statistically nonsignificant effect estimates that authors presented as meaningful differences. We classified interpretations qualitatively and assessed them quantitatively by calculating two areas under the curve (AUC): one for the confidence interval (CI) exceeding the null, indicating a greater effect of one intervention, and another for the CI exceeding a minimal important difference. We compared the AUC of nonsignificant effect estimates that authors interpreted as meaningful with those interpreted as nonmeaningful.
Results: We screened 2,337 Cochrane reviews, finding 139 cases that emphasized meaningful differences of nonsignificant results. Most commonly, authors used qualifying words (e.g., may/might/could) to express uncertainty (47.5%) and sometimes emphasized lack of significance (19.4%). In 27% of cases, authors made absolute claims about the greater efficacy or harm of one intervention without acknowledging statistical uncertainty, which could be misleading. The AUC analysis of the CIs is ongoing and will be available at the Colloquium.
Conclusions: Nuanced interpretations of statistically nonsignificant results were rare in Cochrane reviews. Authors should consider these subtleties when interpreting nonsignificant effect estimates to ensure that meaningful differences are not overlooked and that conclusions drawn are as accurate and informative as possible. Future research should explore the most effective language for distinguishing and interpreting effect estimates so that knowledge users interpret this information correctly.
Patient, public and/or healthcare consumer involvement: The choice of how to present statistically nonsignificant results can substantially impact the interpretation of results made by decision-makers.

2:05 PM - 2:25 PMMaximizing collaboration between university students and CochraneCapacity building in evidence synthesis

Background: Conducting Cochrane reviews can be challenging. These reviews are often carried out by healthcare professionals and experienced researchers because of their complexity and associated workload. University students must complete a research project to obtain an undergraduate or postgraduate degree. Thus, there is a vast potential for collaboration between students and Cochrane. Cochrane is present in universities and hospitals, so one potential partnership may be the realization of end-of-degree research projects and master/PhD theses based on Cochrane activities.
Objectives: (1) To explore how to incorporate Cochrane activities into the undergraduate and postgraduate curricula, (2) to share the experiences of centres in promoting the participation of university students in Cochrane activities and (3) to promote networking among Cochrane trainers.
Description: The workshop is directed at Cochrane trainers, coordinators and undergraduate and postgraduate curricula teachers. The session will highlight new opportunities to link Cochrane activities and university students. The workshop will include two short presentations (10 minutes each) and one interactive session (40 minutes). The group will reconvene for the final 30 minutes to gather feedback about synergies between students and Cochrane. The first presentation (Cochrane Sweden and Cochrane Austria) will focus on experiences with the Cochrane International Mobility scheme (see also https://www.cochrane.org/news/cochrane-international-mobility) and how this could be integrated into the university curriculum. The second presentation will show the experience of different Cochrane Centres (Cochrane Madrid, Iberoamerican Cochrane Centre-Barcelona and Cochrane Complementary Medicine-Switzerland) in integrating Cochrane activities and evidence synthesis methods into the medical degree curriculum. The main section of the workshop will be interactive. Participants will conduct a SWOT analysis to identify strengths, weaknesses, opportunities and threats related to the previously exposed proposals. Relevance to diversity: The integration of Cochrane activities in the university represents a unique opportunity to involve different stakeholders to support informed decisions in healthcare.

2:05 PM - 2:25 PMThe Theory of Everything in Health Decision-Making: Step 2Engaging stakeholders and building partnerships

Background: Clinicians, the public (including patients), policymakers, payers, regulators, and science communities invest considerable amounts of resources in making decisions at various levels using systematic reviews, health technology assessments (HTA), guideline recommendations, coverage decisions, selection of essential medicines and diagnostics, quality improvement, and policy and evidence briefs. With the World Health Organization (WHO), we have developed a conceptual approach that we call “Theory of Everything (ToE) in health decision-making” to connect the actors in this ecosystem. Our work, focusing on high-income countries, shows that criteria and methods that these actors use differ but that they frequently overlap and exist together (Schünemann et al., Lancet PH 2022). The widely used Evidence to Decision (EtD) frameworks form the connecting framework for our conceptual approach (Figure 1). This work has received broad attention, in particular by regulators, and in the guideline, HTA, and health policy community. These actors need to be linked together, and capacity is required to create interaction and decision consensus in and across jurisdictions. However, particularly in emerging economies and low and middle income countries (LMIC), the risk of sparse, redundant or conflicting decisions across the decision ecosystem may be costly. In particular, without coordination, actions by different stakeholders might contribute to decision instability and volatile national health policies.
Objectives: We are evaluating the flexibility, gradualism, and completeness of the ToE through consideration of new perspectives and exploration of its applicability at LMIC level to understand barriers-enablers and necessary prerequisites.
Methods: We are using a mixed-methods approach including surveys of existing and new stakeholders who play a role in the ecosystem, including regulators and industry, in high-income countries and LMIC, a Delphi approach, and actual work with countries to gain further insights regarding the applicability of the approach. Anticipated results and conclusions: At the Cochrane Colloquium we will report on 1) the update and expansion of the ToE; 2) case examples for countries of connecting the decision-making actors, and 3) a draft blueprint and roadmap for operationalization of the ToE in various settings.

ToE slide original.pdf

2:05 PM - 2:25 PMHow to rapidly review the literature when planning a new clinical trial – a practical guideRapid reviews and other rapid evidence products 1

Objectives: In preparation of a clinical trial, the existing evidence on the topic needs to be systematically reviewed to avoid research waste and to provide a rationale for the planned trial. If a current systematic review (SR) is not available, a new one should be conducted. However, such an evidence-based research approach is rarely used, and no guidance for clinical researchers exists. Therefore, we aimed 1) to develop a practical guide for clinical researchers to rapidly review the literature before starting a new trial and 2) to conduct formal usability testing of the guidance.
Methods: We developed the guidance for clinical researchers in several steps: 1) literature search for existing rapid review (RR) methods guidance, guidance on conducting literature reviews specifically for clinical research, and relevant methods research; 2) literature selection including a relevancy rating; 3) data extraction; 4) consultation with our stakeholder panel on the first draft; 5) usability testing of the first interim version; and 6) final version of the guidance.
Results: We screened 602 documents and included 24 existing guidances and 29 methods studies. We extracted and synthesized recommendations for each literature review domain (i.e., research question, eligibility criteria, screening, etc.). Our interim guidance consists of two documents—the practical guide listing essential and optional tasks, explanations, and recommended tools, and a template with signaling questions, which allows users to efficiently complete and document the work process. The suggested approach consists of two parts: Part A “Evidence from existing SRs and ongoing trials” and Part B “Evidence from primary studies.” Decision points for “How to proceed from here” with several possible scenarios are presented.
Conclusions: To make clinical research more evidence based, we developed a systematic approach for clinical researchers to rapidly review the literature when planning a new clinical trial.
Patient, public and/or healthcare consumer involvement: The interim guidance and template will undergo formal usability testing, where the main target group of the guide—researchers in the planning phase of a clinical trial—will apply the guide, use the template, and provide feedback thereafter.

2:15 PM - 2:25 PMIs the Freeman‐Tukey double arcsine transformation a reliable approach? for proportion meta-analysisStatistical methods

Background: Proportion meta-analyses are frequently used in epidemiology to estimate the burden of disease. They are usually based on transformed proportions using Freeman‐Tukey double arcsine transformations (FTT). Schwarzer et al. generated some controversy because they considered that this method produces seriously misleading results and proposed the generalized linear mixed models (GLMM) as a more elaborate approach.[1] However, Suhail et al., using the same set of studies, reanalyzed the data and concluded that the FTT is the most reliable approach and remains the preferred transformation in proportion meta-analysis.[2]
Objectives: To compare the reliability and robustness of FTT with GLMM in a large set of proportion meta-analyses.
Results: Results will be shown at the colloquium.
Methods: We will conduct GLMM[1] and FTT over a large set of proportions from a living systematic review and meta-analysis about safety, immunogenicity, and effectiveness of COVID-19 vaccines for pregnant people (https://safeinpregnancy.org/lsr/) applying recommended safeguards[2]: a) avoiding the use of the average of the double arcsine and its variance for synthesis; b) using the inverse of the variance of the pooled FTT proportion; and c) modifying the confidence intervals to prevent numerical inaccuracies.
Conclusions: It is important to verify the best approach to undertake proportions meta-analysis, which is critical for estimations in epidemiology and decision-making. References 1. Schwarzer G, Chemaitelly H, Abu-Raddad LJ, Rücker G. Seriously misleading results using inverse of Freeman-Tukey double arcsine transformation in meta-analysis of single proportions. Res Synth Methods. 2019;10: 476–483. 2. Doi SA, Xu C. The Freeman-Tukey double arcsine transformation for the meta-analysis of proportions: Recent criticisms were seriously misleading. J Evid Based Med. 2021;14: 259–261.

2:25 PM - 2:35 PMCharacteristics, reporting, and methods of trials included in time-to-event meta-analyses of systematic reviews: A meta-epidemiological reviewStatistical methods

Background: Previous studies demonstrated limitations in the reporting of time-to-event (TTE) analyses in trial publications. The challenges that systematic review authors face in trials they include for TTE meta-analysis remain unexplored.
Objectives: To explore the characteristics, reporting and methods of trials included in aggregate data TTE meta-analyses of Cochrane and non-Cochrane reviews.
Methods: We extracted data from trials included in pairwise, hazard ratio (HR)–based meta-analyses of primary outcomes and overall survival/all-cause mortality of 50 systematic reviews. Reviews were identified from the Cochrane Database and Core Clinical Journals through systematic searches (02/2020; MEDLINE). For reviews, trials and their outcomes, we extracted, in duplicate, data on review/trial characteristics, outcome definitions, general and TTE analyses and specific characteristics relevant to TTE analysis.
Results: Reviews included 235 trials in eligible meta-analyses, resulting in 315 individual trial analyses. Most prominently assessed in trials (91%; 214/235) and reviews (88%; 44/50) was all-cause mortality/overall survival. Outcome definitions (61%; 132/315), censoring reasons (41%; 130/ 315) and follow-up specifications (56%; 175/ 315) for trial outcomes were frequently missing. Available TTE outcome data per trial were most frequently an HR combined with a log-rank p-value, survival curves and either time-point specific (16%; 50/315) or median survival times (15%; 47/315). The most prominently used individual measures were survival curves (83%; 263/315). HRs were reported for 76% (240/315) of trial outcomes. Review authors most frequently used reported HRs or recalculated TTE data from p-values (each 5%; 15/ 315), although TTE data sources were specified sporadically for individual trial outcomes. Reviews included most frequently intention-to-treat (64%; 202/315) and unadjusted analyses (25%; 80/315) but often did not specify either. Except for missing outcome data, for which numerical data were available in 57% (134/235), TTE relevant study characteristics, e.g., informative censoring and proportional hazards, were sporadically addressed in trial publications. Direct comparison to the reporting of their including reviews indicated that limitations and variability of trial reporting translates to the review level as well.
Conclusions: Reporting of trials included in TTE meta-analyses of recent systematic reviews appears variable and limited, similar to their including reviews.
Patient, public and/or healthcare consumer involvement: Not applicable.

2:25 PM - 2:45 PMCreating a systematic review infrastructure: Implementing Cochrane tools for students, teachers, researchers and clinicians in a university settingCapacity building in evidence synthesis

Background: In 2017, Cochrane Sweden was launched, and a collaboration between the Medical Faculty, the Medical Degree (MD) Program and the Faculty Library was initiated based on a need for guidance and relevant learning activities for students, PhD students and researchers in systematic review methodology. The MD Program had defined scientific scholarship and evidence-based medicine (EBM) as core qualifications. The library and MD Program already worked together, and involving Cochrane Sweden became essential.
Objectives: To collaboratively develop relevant learning activities, support and guidance for carrying out systematic reviews using Cochrane tools and methodology for our students, PhD students’ teachers, researchers and clinicians.
Methods: Relevant areas were identified where Cochrane tools could be implemented as stand-alone tools or integrated with other learning activities for our user groups. Identifying methods for support and methodological guidance were key elements in the implementation. Cochrane Interactive Learning (CIL), Covidence and RevMan Web (RMW) were consecutively implemented to meet the objectives. [Table 1- Overview of Cochrane tools implemented at Lund University, Faculty of Medicine]
Results: The process of implementing, evaluating and improving the use of Cochrane tools for our user groups extends over many years. Cochrane tools have been implemented in the PhD program, MD program and the Master programs in Medical Science, Psychology and Public Health. The usage statistics and feedback on Cochrane tools show that they are appreciated and help users’ understanding of review method and workflow. The number of Cochrane Reviews, other systematic reviews and reviews where students and researchers collaborate are increasing. The library’s systematic review service has been expanded to include tools and methodological guidance in the review process. The “Lund model” has also become an inspiring example for other MD programs.
Conclusions: Our collaboration and integrative approach have resulted in an infrastructure of Cochrane tools, people and competencies supporting systematic reviews and EBM. The synergies between the collaborating parts made the implementation of Cochrane tools possible.
Patient, public and/or healthcare consumer involvement: Our students, teachers, clinicians and researchers are given a solid foundation in EBM methodology, which has a strong potential to have a positive impact on patient care.

Table 1 Overview Cochrane Tools Lund University.pdf

2:25 PM - 2:45 PMCEOsys – An ecosystem for COVID-19 evidence in Germany: challenges and lessons learned as a guide for future networksEngaging stakeholders and building partnerships

Background: The COVID-19 pandemic presented a major disruption worldwide. Given the extensive, rapidly evolving evidence, high-quality evidence syntheses that provide context-sensitive and up-to-date data were urgently needed as a basis for evidence-informed clinical and public health decision-making. CEOsys (‘COVID-19 evidence ecosystem’) was established as a national German network for living systematic reviews (LSRs) and living guidelines.
Objectives: To describe the infrastructure, processes and impact of CEOsys and to discuss challenges faced and lessons learned when organizing and maintaining the network, which may provide guidance for future networks in view of pandemics and health-related crises.
Methods: We present an experience report from representatives of the former CEOsys network (09/2020-12/2021; full list of collaborators: www.covid-evidenz.de), a project within the German Network of University Medicine (NUM). After describing its development, we outline CEOsys work processes and present our outputs before highlighting challenges encountered, adaptations made and lessons learned.
Results: CEOsys gathered 18 university hospitals and 8 (non-)university partners. Processes included i.a. prioritization, conducting LSRs, supporting evidence-based guidelines and knowledge translation, backed by setting up a technical infrastructure and capacity-building. We published 12 COVID-19-related Cochrane reviews, supported three living guidelines (inpatient/outpatient treatment, school-based infection prevention) and developed 10 methodological approaches and 10 dissemination formats. Challenges included CEOsys’ late initiation, prioritization, the continuously evolving knowledge on COVID-19 and revision of outcomes for LSRs or establishing a ‘living’ technical infrastructure. Based on lessons learned, the cooperation between clinicians and methodologists within CEOsys and the network’s international collaborative partners (e.g., Cochrane International) represented major advantages. Mediated by the Institute for Medical Knowledge Management, CEOsys greatly benefitted from collaborations with guideline groups. Future networks should, for example, involve key stakeholders early on, aim for (inter-)national collaborations, balance traditional publishing of results and living documents and evaluate their impact on decision-makers and the public.
Conclusions: CEOsys achieved rapid, high-quality and up-to-date evidence syntheses, collaboratively produced across multiple sites and on a broad range of issues. Our experiences are now taken up by the follow-up project PREPARED (‘PREparedness and PAndemic REsponse in Germany’), which uses the infrastructure and processes established to design a blueprint for a more sustainable evidence ecosystem.

2:25 PM - 2:45 PMIdentifying high priority methodological questions for conducting rapid systematic reviews: Preliminary results from an eDelphi studyRapid reviews and other rapid evidence products 1

Background: Rapid systematic reviews (RRs) are a useful tool to provide healthcare decision-makers with the most up-to-date scientific evidence. Yet there remains much uncertainty about the most effective and efficient methods for carrying out RRs. In fact, there is even uncertainty about which are the main methodological concerns for RRs.
Objectives: To elicit a consensus from RR researchers on what they consider the most important methodological questions when conducting RRs in time-efficient ways.
Methods: People with experience in conducting and using evidence syntheses (researchers, knowledge users, policymakers, and healthcare providers) were invited to participate in an eDelphi study. A core group of experts generated an initial list of questions based on the available literature. Participants were then asked to rate and rank the importance of the methodological questions. Three survey rounds were performed to achieve a consensus. Items deemed of low importance were removed at each round. Items rated to be of high importance by ≥75% of participants were included in the final list.
Results: Each survey round elicited feedback from 52 to 70 experts. Of participants, 78% self-identified as researchers, 13% as healthcare practitioners, and 9% as policymakers. More than 30% of participants reported 15+ years of experience in evidence synthesis. The first survey round started with 29 items, and new items were suggested. In the second round, 26 items remained on the list. In the final round, only 7 Items were believed to be of high importance by ≥75% of participants. Three items concerned the search strategy, two focused on study selection, and two focused on quality/bias assessment. Conclusion: Seven items were consistently identified as highly important methodological concerns related to RRs. A consensus meeting will be held to discuss additional items and to generate a summary document containing the final priority list. The results will be used to inform further research in this area, contributing to the development of reliable and rigorous RRs. Patient, public or healthcare consumer involvement: Given the emphasis on the methodological aspects of RRs, patients were not included. The eDelphi does include interested parties, such as guideline and policy developers and end-users.

2:35 PM - 2:45 PMComparison of statistical methods used to meta-analyse results from interrupted time series studies: an empirical studyStatistical methods

Background: The interrupted time series (ITS) design is commonly used to evaluate large-scale policy change or public health interventions when randomisation is infeasible. In ITS studies, measurements are collected at regular intervals before and after an interruption. The pre-interruption period is used to estimate an underlying time trend that, when projected into the post-interruption period, creates a counterfactual for what would have occurred without the interruption. The impact of the interruption can then be quantified using a variety of metrics such as immediate and long-term effects. Several statistical methods are available for the analysis and meta-analysis of ITS studies. However, there has been no empirical evaluation of the impact of using different statistical methods to analyse ITS studies and meta-analyse their results.
Objectives: To empirically compare meta-analysis results obtained from different meta-analysis and ITS analysis methods when applied to real-world ITS data.
Methods: ITS datasets were sourced from published meta-analyses and reanalysed using two ITS estimation methods. The level- and slope-change effect estimates were calculated and combined using fixed-effect and four random-effects meta-analysis methods. We compared the meta-analytic effect estimates, 95% confidence intervals, p-values and estimates of heterogeneity across the statistical method combinations.
Results: Of an eligible 40 reviews, data from 17 meta-analyses (including 283 ITS studies) were obtained and analysed. We found that the meta-analysis method choice did not systematically impact the meta-analytic effect estimates, standard errors and between-study variances, irrespective of the ITS analysis method. However, the meta-analytic confidence intervals and p-values were impacted by the meta-analysis confidence interval method, and the ITS analysis method used may modify this impact.
Conclusions: The effect estimates, standard errors and between-study variance estimates were minimally impacted by ITS analysis and meta-analysis method choice. However, confidence intervals and p-values varied depending on the statistical method used, which may impact the interpretation of a meta-analysis. In conjunction with evidence from numerical simulation, this study provides insights into which methods to use in different scenarios, which may assist researchers in evidence synthesis of public health or policy interventions.
Patient, public and/or healthcare consumer involvement: No patients/consumers were involved in the design/reporting of this study.

2:45 PM - 2:55 PMThe role of collaborative evidence networks in promoting and supporting evidence-based health care (EBHC) globallyEngaging stakeholders and building partnerships

Background: The success of the global evidence-based healthcare (EBHC) agenda depends on individuals and organisations working together within a functioning evidence ecosystem. Collaborative evidence networks like JBI, Cochrane, Campbell and others, although differing in form and function, were all established with a common goal: to collaboratively produce, summarise and disseminate evidence to inform policy and practice to improve lives. Over the past 30 years, with the rise of digital technology, increasing recognition of the importance of evidence-based decision-making and the evolution of network structures, these networks have become central to the way evidence is synthesised, translated and implemented, yet little has been written about their strategic development or management.
Objectives: To explore the strategic functions and form of the JBI Collaboration to understand the role of a collaborative evidence network in promoting and supporting EBHC globally.
Methods: The network functions approach (NFA) was employed to map the form (governance, management and administrative structures) and core functions of the JBI Collaboration.
Results: The results of the NFA mapping established the ways the JBI Collaboration enables development, exchange and dissemination of knowledge; the purposeful building of social capital; mobilisation of resources; and amplification and advocacy of members’ work to increase the capacity and effectiveness of members in achieving their unified purpose. It demonstrated how participation in the network enables members to contribute to and draw on collective expertise; provides access to platforms and processes that broaden the dissemination and impact of members’ work; provides access to scientific resources, training and opportunities for individual/institutional capacity building; and provides practical knowledge of how to solve complex health problems using JBI’s model of EBHC.
Conclusions: Collaborative evidence networks play a crucial role in bringing together diverse stakeholders from across the evidence ecosystem to drive the EBHC agenda forward. These networks combine formal and informal structures to facilitate collaboration to address complex global health issues. There would be value in cognate networks using the NFA to identify the areas of overlap, working together to systematically reduce duplication of effort and strengthen our understanding of how we can collaborate across networks to achieve impact in EBHC globally.

2:45 PM - 2:55 PMRisk factors for abstracts falsely excluded during single-reviewer screening – a methods studyRapid reviews and other rapid evidence products 1

Background: Because of the growing need to provide evidence syntheses under time constraints, recent research has explored rapid review methods, which often employ single-reviewer literature screening. However, the single-reviewer screening process is error-prone; on average, 13% of relevant studies are missed. To date, it is unclear whether certain types of studies or publications have a higher risk of being falsely excluded than others.
Objectives: The aim of our methods study with an observational design was to identify risk factors on a study-, abstract-, journal-, and reviewer-level for eligible studies to be falsely excluded during single-reviewer abstract screening.
Methods: We used a database of 1,000 inclusion and exclusion decisions pertaining to 80 eligible references from a crowd-based randomized controlled trial assessing the accuracy of single-reviewer abstract screening. We gathered a list of potential risk factors for studies falsely excluded during single-reviewer abstract screening. One investigator collected potential risk factor variables independently, and a second investigator checked the data. We built a random forest model using R version 4.2.2 to identify variables that could predict the risk of false exclusion of studies. We split up the dataset into a training and a validation set. We trained the random forest classifier with the training set and used it to predict classes of “single false exclusion” for the validation set. We calculated Mean Decrease Accuracy and Mean Decrease Gini to rank the importance of the collected variables.
Results: The random forest model ranked study design and the Scimago Journal & Country Rank (SJR) of the publishing journal as the variables with the highest importance for false exclusions. Cross-sectional studies, controlled-before-after studies, and publications in journals with an SJR ≤ 2.29 had a higher risk of being falsely excluded. Other variables, such as the screener being a native speaker, self-rated topic expertise, and previous review screening experience, were ranked as the least important. The model achieved a good area under the curve at 0.71.
Conclusions: Our results indicate that study design and a low SJR of the journal may be important risk factors for studies to be falsely excluded during single-reviewer abstract screening. Investigators need to be cognizant of these risk factors when conducting reviews that include non-randomized evidence. Patient or healthcare consumer involvement: Rapid reviews generally involve patients and healthcare providers in the review process in order to focus on patient-relevant health outcomes.

2:45 PM - 2:55 PMEffect estimates can be accurately calculated with data digitally extracted from interrupted time series graphsStatistical methods

Background: Interrupted time series (ITS) studies are frequently used to examine the impact of population-level interventions. Systematic reviews with meta-analyses including ITS designs may inform public health and policy decision-making. Reanalysis of ITS may be required for inclusion in meta-analysis. Although publications of ITS rarely provide raw data for reanalysis, graphs are often included, from which time series data can be digitally extracted. However, the accuracy of effect estimates calculated from data digitally extracted from ITS graphs is currently unknown.
Objectives: To assess the accuracy of effect estimates calculated from digitally extracted data from ITS graphs.
Methods: Forty-three ITS with available datasets and time series graphs were included. Time series data from each graph were extracted independently by four researchers using digital data extraction software. Data extraction errors were analysed. Segmented linear regression models were fitted to the extracted and provided datasets, from which estimates of immediate level and slope change (and associated statistics) were calculated and compared across the datasets. These estimates were then standardised for comparison across the datasets. For a particular dataset, standardisation was achieved by dividing the level and slope change estimates by the range of the outcome of the provided time series data (i.e., the maximum observed value of the outcome minus the minimum observed value); this yielded standardised effect estimates that could range from 0 to 1.
Results: Although there were some data extraction errors of time points, primarily due to complications in the original graphs, they did not translate into important differences in the standardised level and slope change effect estimates (and associated statistics) (Figures 1 and 2).
Conclusions: Using digital data extraction to obtain data from ITS graphs should be considered in reviews including ITS. Including these studies in meta-analyses, even with slight inaccuracy, is likely to outweigh the loss of information from noninclusion.
Patient, public and/or healthcare consumer involvement: There was no involvement with patient, public and/or healthcare consumers.

Figure_1_box_plot_level.png
Figure_2_box_plot_slope.png

2:45 PM - 3:05 PMBuilding capacity in producing trusted evidence – Evidence Synthesis Ireland and Cochrane Ireland FellowshipsCapacity building in evidence synthesis

Background: Evidence Synthesis Ireland (ESI), which includes Cochrane Ireland, aims to build evidence synthesis knowledge, awareness and capacity among the public, healthcare institutions, policymakers, clinicians and researchers on the island of Ireland. One way we do this is through the ESI Fellowship Scheme, launched in January 2019.
Objectives: The scheme builds capacity in Fellows through developing the practical skills of how to plan, design, conduct and report an evidence synthesis. Fellows receive mentoring from evidence synthesis experts and contribute to reviews that have a strong policy and/or practice impact. Process: The scheme places Fellows virtually with experienced review teams nationally and internationally to gain hands-on evidence synthesis experience in a high-quality environment with one-to-one mentorship. Fellowships range from 6 months to 2 years and are unpaid, allowing those working full-time to apply. Benefits include free attendance at ESI workshops, co-authorship, €1000 travel bursary, access to Covidence, and networking/collaboration opportunities. Eligible applicants are clinicians/trainees, researchers, academics and/or postgraduate residents of Ireland/Northern Ireland. Fellows are selected through a competitive process and are expected to commit to 8 hours/week to the review in addition to theoretical training.
Results: Fellows are placed on a range of review types: 50% on Cochrane reviews (reviews of effectiveness, qualitative evidence synthesis, rapid reviews, living reviews and diagnostic test accuracy) and 50% on other reviews (umbrella, overviews, scoping reviews, prognostic reviews, evidence and gap maps and mixed methods). To date, 68 Fellows have been placed. Fellows are mostly female (64%), over half have never completed a systematic review and 35% are early career. Fellows are academics (30%), healthcare professionals (35%), postdoctoral researchers (24%), clinician-academics (6%), policymakers (6%) and others (9%). Fellows report capacity built-in evidence synthesis competencies and illustrate this capacity through leading further reviews, collaborating with mentors and teaching evidence synthesis.
Conclusions: ESI provides a unique opportunity for researchers to develop their evidence synthesis skills and contribute to evidence-based healthcare in Ireland. This model could be replicated by others to strengthen capacity in producing robust evidence. Patient relevance: The scheme is relevant to patients as it contributes to strengthening the capacity of researchers/clinicians to produce robust evidence.

2:55 PM - 3:05 PMKnowledge mobilisation of rapid reviews to inform health and care policy and practice: lessons from the Wales COVID-19 Evidence CentreEngaging stakeholders and building partnerships

Background: Policy and practice decisions must be evidence-based, but ensuring that research evidence is readily available and used by decision-makers is challenging. This was exacerbated during the COVID-19 pandemic, when knowledge mobilisation strategies to bridge the gap between evidence synthesis and informing decisions were critical. The Wales COVID-19 Evidence Centre (WCEC) was set up to ensure that health and social care policy and practice decision-makers had timely access to relevant COVID-19 research evidence.
Objectives: To describe WCEC knowledge mobilisation methods for rapid evidence reviews, reflect on the impact, and share learning.
Methods: Knowledge mobilisation methods were flexible to meet decision-makers’ (our stakeholders’) needs. We collaborated with stakeholders throughout to ensure we addressed their requirements and identified policy and practice implications, evidence impact and develop knowledge mobilisation plans. We identified barriers and facilitators to improving our methods over time and ensure all outputs are widely accessible.
Results: Coproduction with stakeholders (priority-setting, review conduct, planning pathway-to-impact and knowledge mobilisation) built trust in our methods and evidence reliability. Reviews informed Welsh health and care decision-making during the pandemic and recovery periods, addressing current health and care pressures. From April 2021 to December 2022, 20 of our reviews informed Welsh Government guidance, moving to COVID alert level 0, schools reopening, a Public Health Wales campaign to raise vaccine uptake, and led to Welsh Government commissioning of carbon dioxide monitors in schools. A review of strategies to support learning and wellbeing among 16- to 19-year-old learners was a key piece of evidence included in the Welsh Government’s ‘Renew and Reform: Post-16 and Transitions Plan’, and reviews on inequalities that were exacerbated during the pandemic (LGBTQ+, women and girls, people living with disabilities) informed a Welsh Government action plan. Recently, a review of the effectiveness of diagnostics centres is guiding the design and implementation of new community diagnostic centres in Wales (January 2023).
Conclusions: Coproduction throughout supported knowledge mobilisation and led to the use of rapid reviews by scientific advisors and policy and practice decision-makers. Patient, public, and/or healthcare consumer involvement: The WCEC had a public partnership group of 8 members who contributed to all our processes.

2:55 PM - 3:05 PMMachine-learning assisted screening increases efficiency of systematic reviewRapid reviews and other rapid evidence products 1

Background: Conventional systematic review (SR) methods are time-consuming and highly resource intensive. Artificial intelligence (AI) algorithms such as machine learning and deep learning can help reviewers complete these tasks in less time and with fewer resources. PICO Portal (PP) is an AI-assisted SR platform that prioritizes articles for screening using several algorithms including both decision tree and deep learning models.
Objectives: To assess the AI-assisted screening in PICO Portal.
Methods: Our data set comprised eight completed SRs, each using two independent screeners, with a total of 56,728 records (range: 4,204 to 14,185) on a range of topics from social to biomedical sciences. For each SR, we simulated the screening using batches of 100 articles to train and build predictions for eligibility, re-ranking successive articles, and comparing the predicted eligibility with the actual results from the SRs. We plotted the proportions of title/abstract and full-text included records that were captured by the AI screening at the title and abstract level for each project and calculated a weighted average of this efficiency (by project size). We meta-analyzed the sensitivity and specificity of the predictions versus the reviewers’ final decisions using Stata ‘metadta’.
Results: We estimate that if the active learning AI predictions had been used, reviewers would have needed to screen only 20-50% of title/abstracts to capture 95% of eligible records (Figure 1). After screening 10%, 25%, 50%, and 70% of title/abstract records, the average project would have captured approximately 60%, 85%, 95%, and 99% of the records included in the title/abstract stage (Figure 2). Sensitivity was better than specificity (95% vs. 68%) (Figures 3 and 4).
Conclusions: Based on our analysis, we estimate that 40-60% of screening effort can be saved using PICO Portal, an AI-assisted, web-based, SR platform. Future research should examine the impact of missing the final 5% of records on review conclusions and assess the resource-benefit ratio. Patient relevance and involvement: Our findings and future recommendations are from the researcher and funder perspective. Our conclusions directly impact the amount of time reviewers need to complete an SR. This work did not involve any stakeholders, patients, or consumers.

Figure 1a & 1b.png
Figure 2.png
FIgure 3.png
Figure 4.png

2:55 PM - 3:05 PMIs a new approach for rating the quality evidence of effect estimates derived from matched-adjusted indirect comparisons (MAIC) needed?Statistical methods

Background: In health technology assessments (HTAs), matching-adjusted indirect comparisons (MAICs) are used when head-to-head randomised studies comparing a drug (therapy) in question and a comparator, e.g., standard care in the treatment of a disease, are not available. MAICs use individual patient data (IPD) from trials of one treatment to match baseline summary statistics reported from trials of another treatment. MAICs use an approach similar to propensity score weighting, whereby treatment outcomes are compared across balanced trial populations. Although the Grading of Recommendations Assessment, Development and Evaluation (GRADE) and Confidence in Network Meta-Analysis (CINeMA) approaches to rate the quality of treatment effect estimates from network meta-analysis (NMA) have been suggested, it seems that an approach for MAICs is missing.
Objectives: a) To evaluate the prevalence of MAIC use in submissions to the National Institute for Health and Care Excellence (NICE); and b) to explore how to rate the quality/certainty of the evidence in MAIC using the currently available approaches.
Methods: Scoping searches of NICE website (without time restrictions) were conducted, and these will be supplemented with searches in Medline, Embase and Central. Prevalence data will be synthesised quantitatively. The existing GRADE and CINeMA approaches will be compared accounting for similarities and differences in the MAIC and Bucher methods.
Results: Preliminary findings suggest that MAICs are predominantly being used for reimbursement decisions in oncology. Worryingly, a large proportion of submissions to NICE rely on unanchored comparisons whereby a common comparator arm is missing, and these types of MAICs make much stronger assumptions and are widely regarded as unfeasible. The work is still ongoing; we will have more findings by September 2023 to present at the Colloquium.
Conclusions: Currently there is no guidance on how to rate the certainty of evidence of effect estimates obtained from MAICs. Although MAIC and Bucher methods share certain similarities, there are still distinct differences between the two. We believe a new approach is warranted, and determining certainty of evidence in MAICs may help decision-makers in making informed recommendations.

3:05 PM - 3:15 PMProviding evidence to the WHO, the experience of Cochrane Rehabilitation regarding the Rehabilitation 2030 initiative and the COVID-19 pandemicEngaging stakeholders and building partnerships

Background: Shortly after the launch of Cochrane Rehabilitation in December 2016, the World Health Organization (WHO) launched “Rehabilitation 2030: a call for action” with the aim to strengthen and upscale rehabilitation services in health systems worldwide in response to the growing needs. Cochrane Rehabilitation collaborated with the WHO Rehabilitation Programme in several projects on rehabilitation interventions, in the framework of Rehabilitation 2030 and the COVID-19 pandemic.
Objectives: The aim was to provide methodological support and Cochrane evidence at the request of the WHO, to inform clinical recommendations and guidance on rehabilitation management.
Methods: Different methods have been used according to the needs of the specific projects. For the Package of Interventions for Rehabilitation (PIR), developed for 20 selected health conditions, we performed an overview of all the CSRs according to the inclusion criteria defined with WHO. The CSRs identified during the screening process were summarized using an evidence map. We grouped outcomes and comparisons of included CSRs indicating the effect and the quality of evidence to provide a comprehensive view of what is known. Regarding the COVID-19 pandemic, we created an interactive living evidence map, based on priorities for research defined in collaboration with the WHO rehabilitation programme: limitations of functioning (disability) of rehabilitation interest, the phases of the disease, and research questions at the rehabilitation micro, meso and macro level. Furthermore, indirect evidence on symptoms of post COVID-19 condition was extracted and synthesized from CSRs on the same symptoms in other health conditions.
Results: Table 1 reports the results on the three main projects (PIR, ‘Evidence on’ COVID-19 and ‘Evidence relevant to’ symptoms of post COVID-19 condition) in collaboration between the WHO and Cochrane Rehabilitation.
Conclusions: In its collaboration with the WHO, Cochrane Rehabilitation has developed different strategies and methodologies to respond rapidly to emerging questions regarding rehabilitation interventions. The aim is to inform clinicians as well as policymakers to allow evidence-based decision-making. These approaches could benefit other similar fields in healthcare in the future and inform scientists on further research needs.
Patient, public and/or healthcare consumer involvement: Not applicable.

CR WHO Collaboration Table 1.pdf

3:05 PM - 3:15 PMRapid reviews, how much do they comply with Cochrane recommendations in their methodology to provide reliable evidence?Rapid reviews and other rapid evidence products 1

Background: During the COVID-19 pandemic, a large increase in rapid reviews (RRs) was identified. This is mainly because the design appears as an alternative to provide timely and rapid responses to emerging issues. RRs follow the method of systematic reviews, but they simplify or omit some steps to shorten the length of time in which the review is conducted.
Objectives: To identify and describe which steps were performed and which were omitted in published RRs, according to the list of recommendations suggested by the Cochrane Methods Group.
Methods: We searched databases (MEDLINE, Embase, Epistemonikos, and CINAHL) to identify RRs of interventions for COVID-19. Two investigators independently screened and selected reviews that self-reported as RRs and that at least delivered a meta-analysis. We will extract the data related to the methodology used by each RR and compare them with the recommended steps by classifying them as “reported” or “omitted.” We will analyze compliance for each study and for the totality of the included studies. To chart compliance, we will classify it as “very high” when compliance ranges from 76% to 100% (100% corresponds to reporting for the eight recommendations), “moderate” when between 51% and 75%, “low” when between 26% and 50%, and “very low” when between 0% and 25%.
Results: This is a work in progress. The results will describe the performance of the method in a certain group of RRs. We will present a summary table of the Cochrane Methods recommendations for RRs, identifying whether there was reporting or omission of the step for each recommendation.
Conclusions: RRs are being used for decision-making both for everyday health and for public health policies, so it is essential that they are performed with a clear and specific method. We hope that the results of this analysis can inform whether an adequate method is being followed when carrying out RRs.
Patient, public and/or healthcare consumer involvement: This study does not involve direct participation of patients or consumers.

3:05 PM - 3:15 PMUsing simple microsimulation to estimate risk difference from a meta-analysisStatistical methods

Background: Absolute risk reduction or risk difference (RD) is a key effect measure required for decision-making and its confidence interval (CI) is the basis for imprecision judgments. Many methodology groups (e.g., Cochrane and GRADE) recommend obtaining RD from linear transformation of a risk ratio (RR) that is usually derived from a meta-analysis. This transformation uses an assumed baseline risk (BR) and follows the equation RD= RR X (RR-1). The 95% CI of RD is derived from the same equation using the 95% CI of RR.
Objectives: In this proposal, we demonstrate several limitations to this traditional approach using a simulated case study and offer an alternative approach.
Methods: We simulated a case study using a published systematic review [beta blockers vs. placebo on all-cause mortality in symptomatic heart failure with preserved ejection fraction, RR, 0.79 (0.66-0.96)]. The first analysis was based on the traditional linear transformation of RR into RD using BR that we simulated from 0% to 100%. The second analysis is based on the proposed new approach which is a microsimulation in which RR is drawn from a Lognormal distribution with mean and standard error of the identified RR and BR is drawn from a Beta distribution which shape parameters were derived from a population-based study [Olmsted County, Minnesota, Mortality at 1 year= 0.29 (0.27-0.31)]. 10,000 simulations (draws) generated RD histogram with median, 2.5 and 97.5 percentiles (Open-source code in R).
Results: The traditional approach (Figure 1) demonstrates these limitations: 1) RD CI does not incorporate uncertainty in BR and derives all its uncertainty from the treatment effect, 2) CI widens linearly as BR increases, making RD estimates imprecise in higher-risk populations (counterintuitive), and very precise in low-risk populations (potentially misleadingly precise as it ignores uncertainty in BR). The proposed approach (Figure 2), produces a joint distribution of RD that incorporates uncertainty in BR to resolve the limitations of the first approach.
Conclusions: Simple transformation of RR into RD has many limitations and does not incorporate uncertainty in BR. An alternative proposed approach incorporates uncertainty in BR that was derived from a population-based study. Patient/public/consumer involvement: None.

cochrane RD figures.pdf

3:05 PM - 3:25 PMEvidence Based Research Training SchoolCapacity building in evidence synthesis

Background and objectives: The overall aim of the training school (TS) is to introduce to the participants the concept of Evidence Based Research (EBR) and support their ability to use existing evidence synthesis to justify, design and place results of the new study in the context of existing knowledge in order to avoid redundant research.
Methods: The TS is targeted at Early Career Investigators and senior Clinical Health Researchers who are interested in how to perform EBR. Following several working meetings of the group that has knowledge of existing courses and training in Systematic reviews, Evidence-Based Medicine/Practice/Healthcare, etc., from various organisations such as Cochrane, JBI, GRADE, etc., and by analysis of the content of their curricula and by an iterative discussion in the group, pilot EBR TS were designed, which were tested in Tartu in 2019 and then evolved to synchronous on-line teaching and in the end were developed as an asynchronous on-line course on the Moodle and Canvas platforms. The evaluation form for participants was designed and applied, and feedback from teachers and facilitators of TS plus feedback from EVidence Based RESearch - EVBRES COST Action Programme Management Group and Core Group were received and analyzed with every new version of TS.
Results: The curriculum and programme of TS were designed and tested. We taught the TS seven times to over 150 successful participants who finished the whole TS. TS is divided into 14 modules. Each module has its own specific interactive content. The overall rating of TS by our participants was 4.34 on a scale one (poor) to five (Excellent).
Conclusions: The development of the EBR TS is a significant step in increasing awareness around EBR among researchers, raising the importance of the development of trustworthy and important evidence synthesis and primary research in the context of existing evidence and thus preventing research waste. Consumer involvement: Several colleagues originally inexperienced in EBR or Evidence-Based Practice education were involved in the design of the TS, so the TS development was co-informed by their needs in EBR knowledge and skills.

3:15 PM - 3:25 PMBoosting global and local partnerships to promote equitable access of COVID-19 guideline recommendations: case study in ChinaEngaging stakeholders and building partnerships

Background: The eCOVID19 Recommendations Map and Gateway to Contextualization (RecMap) is a living platform presenting 7280 guideline recommendations. Funded by the Canadian Institute of Health Research (CIHR), RecMap involves the collaboration of over 40 partner organisations globally and has launched a knowledge mobilisation (KM) project to benefit wider stakeholders in different countries.
Objectives: To increase the awareness, use, and engagement of the RecMap among guideline developers in China and to identify barriers encountered in the KM activities.
Methods: We used a co-design framework to guide our activities, where the RecMap central team, China team leads, voice of local stakeholders and public, and the context of China COVID-19 policies were considered in planning and iterating the activities. These activities include group discussion, local stakeholder consultation, guideline adolopment (contextualise and adopt recommendations from existing guidelines), plain language recommendation (PLR) translation, webinars, and social media release. We ensured an inclusive participation of age and gender, guideline developers from both clinical and research background, and public involvement.
Results: We collaborated with China GRADE Centres, Cochrane China Network Affiliates, and Cochrane Campbell Global Ageing Partnership. We conducted 10 discussions and consultations with experts and held webinars with 1.4 million views. We found that local policies and language are key barriers for introducing and implementing RecMap recommendations in China. The restricted and unpredictable China COVID-19 policies in the past months have hindered our guideline adolopment progress, so it is important to test the feasibility of KM activity with local stakeholders at an early stage to avoid waste of time and resources. To encourage more Chinese to visit the English RecMap website, we prioritised translation of 40 PLRs into Chinese. After identifying mistakes in PLRs translated by volunteers, we established a workflow with professional translator, methodology editor, clinical editor, and citizen editor to co-produce Chinese PLRs to ensure quality and readability.
Conclusions: The rapid global health challenges require us to establish a collaborative platform and streamline the collaboration process for each stakeholder spending less time and fewer resources on negotiating the mechanisms to achieve more efficient, effective, and equitable partnership outcomes. Public involvement: Citizen editor was involved in PLR development.

3:15 PM - 3:25 PMEvaluation of the Interim Cochrane Rapid Review Methods guidance – a mixed-methods study on the understanding of and adherence to the guidanceRapid reviews and other rapid evidence products 1

Objectives: The Cochrane Rapid Review Methods Group (RRMG) developed interim guidance to support authors in conducting rapid reviews (RRs). The objective of this mixed-methods study was to assess the adherence to and usability of this guidance. We also explored why some Cochrane authors of COVID-19–related reviews preferred doing a full systematic review (SR) over an RR (see published protocol: https://osf.io/3a8zb).
Methods: We identified all reviews citing the Interim Cochrane RRMG guidance up to February 17, 2022, and performed an exploratory adherence analysis. We interviewed 20 RR authors to assess the comprehensibility of recommendations and reasons for any deviations. Further, we surveyed nine authors of COVID-19–related full SR.
Results: We analyzed 128 RRs (111 non-Cochrane, 17 Cochrane) citing the Interim Cochrane RRMG guidance. Several recommendations were not followed by a large proportion of RR authors, whereas in some cases, full SR methodology was used instead of recommended abbreviations. The recommendations that were not followed by the most-analyzed RRs were the stepwise approach to study design inclusion (97%), limiting the number of outcomes (88%), and peer review of at least one search strategy (88%). The rationale for using a standardized title and abstract form was not obvious to some RR authors, and 39%/45% preferred dual independent screening of abstracts/full texts over screening only excludes dually. The most reported reasons for deviating from the guidance were time constraints, unclarities in the recommended approach, or inapplicability to the specific RR. Overall, the guidance was viewed as user-friendly. However, without pre-existing experience of SR conduct, applying the guidance was perceived as difficult. The main reasons for conducting a full COVID-19–related SR over an RR despite the time pressure during the pandemic were late availability of the guidance, preset mandate to conduct an SR, lack of methodological clarity, and inapplicability to the evidence base.
Conclusions: Clarifications are warranted throughout the Interim Cochrane RRMG guidance to ensure that users with various experience levels can understand and apply its recommendations accordingly.
Patient, public and/or healthcare consumer involvement: We interviewed RRMG guidance users for feedback on its comprehensibility and usability to improve the interim guidance.

3:15 PM - 3:25 PMLess Ethical Challenges, More Trial Compliance: Progress and Methodological Elements of Zelen's DesignStatistical methods

Background: Classical randomized controlled trials often meet difficulties in enrollment and compliance due to subjects’ preference for the randomization scheme. Zelen’s design can largely resolve these issues through randomizing subjects before informing stage.
Objectives: This study systematically reviewed the progress of Zelen’s design and its modifications in clinical research and clarified its methodological elements, advantages, and limitations.
Methods: A systematic literature search was conducted for Zelen’s design from the inception of databases (PubMed, Cochrane library, CNKI, WanFang, and Vip database) to February 2023. The data were extracted, including sample size, population characters, length of follow-up, Zelen’s model, group shifting, and statistical methods.
Results: One hundred twelve trials were included. The range of sample size was from 15 to 23,024. The longest follow-up length was 17.2 years. The dominant disease in this design was mental disorders (21, 18.75%), followed by osteoarthrosis diseases (16, 14.29%), cancer (11, 9.82%), cardiovascular diseases (10, 8.93%), and others. Regarding types of consent, more than half of the trials used a double-consent (57, 50.89%), 32 (28.57%) used a single-consent, and others did not report the consent procedure. Twelve trials (10.71%) used a modified Zelen’s design nested within an observational study. This design used a two-stage informed consent. Stage 1, patients were invited to participate in a cohort study; Stage 2, patients randomized in the experimental group were informed of the allocation result and inquired whether they would like to follow the treatment. Five trials used the McNulty-Zelen design, which could be applied in cluster randomized controlled trials and overcome the potential bias of the Hawthorne effect, e.g., trials involving educational interventions. Informed consent was obtained from a third party on behalf of the participants. Intention-to-treat analysis was the main population used in Zelen’s design.
Conclusions: Zelen’s design has a broad application foreground in clinical trials. It also could adapt to research needs by combining with various observational studies. Patient or healthcare consumer involvement: No patients or consumers participated in research activities.

4:05 PM - 4:25 PMThe Central Editorial Service: who we are, what we do, and what happens to your draft after submissionEditorial processes and supporting review authors

Background: Cochrane has embarked on an important change programme to transform the way that we produce high-quality, independent and timely evidence. By 31 March 2024, the editorial process for all submitted protocols, reviews and updates will be managed centrally by the Central Editorial Service.
Objectives: To explain the rationale for and workings of the Central Editorial Service for both prospective authors and members of Cochrane entities.
Description: The Central Editorial Service currently handles the editorial process for most drafts submitted for publication in the Cochrane Database of Systematic Reviews (CDSR). This will progress to all content from 31 March 2024. The separation of the process of producing Cochrane Reviews from the editorial process to assess their suitability for publication in the Cochrane Library represents a major change for Cochrane. This talk will explain the rationale for the Central Editorial Service, and the difference between the Central Editorial Service and a Cochrane Review Group, which remains an understandable source of confusion for many Cochrane contributors. The presentation will explain the editorial process from submission, editorial assessment and peer review to the final editorial decision We will also signpost to other Colloquium workshops that authors may find helpful in preparing their Cochrane Reviews.

Colloquium 2023 Abstract - Central Editorial Service Oral Presentation.pdf

4:05 PM - 4:25 PMDeveloping capacity for the production and use of Living Evidence in decision making: The LE to Inform Health decisions project.Engaging stakeholders, building capacity, developing partnerships

Background: It is necessary to invest in strategies that allow groups and organizations in charge of developing evidence synthesis to inform clinical or health policy decisions and effectively incorporate new emergent evidence in the development of knowledge transfer (KT) products such as clinical practice guidelines (CPG), health technology assessment (HTA), and structured evidence summaries.
Objectives: To develop and evaluate a capacity-building strategy for health sector organizations to incorporate the living evidence (LE) model in the development of KT products.
Methods: The project was developed in three phases: P1. Definition of a guidance framework; P2. Training in LE strategy and supporting tools; P3. Applying the framework and acquiring knowledge in the development of real-life LE synthesis following the learning-by-doing methodology. Study population: Technical staff working on CPG, developing organizations, HTA agencies, and tertiary hospitals running institutional HTA. The Epistemonikos-L.OVE platform and tools were used as the technological enablers for the LE process. Each component of the capacity-building strategy was evaluated through online anonymous surveys. A final set of in-depth interviews were conducted to evaluate the whole capacity-building strategy used.
Results: Phase 1: Based on methodological articles and brainstorming meetings, we developed a preliminary framework. After its testing among potential users and expert consultation, the final version of the framework was used in the next phases of the project. For phases 2 and 3, we enrolled 21 team members from a total of seven organizations. Six training workshops in the LE methodology and Epistemonikos tools were carried out. Participating organizations’ teams developed a total of seven living evidence syntheses following the framework and using Epistemonikos tools. They maintained the evidence surveillance and related monitoring tasks for at least 8 months, which were reported on a monthly basis and made available at the project website (https://livingevidenceframework.com/en/platform/). Eighty-five percent of participants took part of the online survey and 68% of in-depth interviews to evaluate the capacity-building strategy.
Conclusions: We addressed three basic needs to build a sustainable capacity among groups and organizations for the production and use of LE synthesis: the need for having guidance; for training; and for being supported during the process.
Patient, public and/or healthcare consumer involvement:

4:05 PM - 4:25 PMMisinformation in COVID-19: nimble innovation within Cochrane to appraise the sourceKnowledge translation

Background: During COVID-19, the public was put in danger by erroneous beliefs derived from poor-quality research amplified through social media. In the Cochrane Infectious Diseases Group (CIDG), we needed to be innovative with methods to address COVID-19 misinformation. Here, we will present this evolving response, leading to an approach to appraise research that caused unsubstantiated hypotheses. In this process, we adapted Cochrane methods to appraise comparative serological studies.
Objectives: 1. To present the CIDG strategy to address the COVID-19 misinformation pandemic. 2. To outline novel methods for critically appraising comparative serological studies.
Methods: The CIDG identified important areas of misinformation where our methods could provide an authoritative voice. This started with Cochrane Reviews of trials to help end the use of unapproved drugs for COVID-19 (chloroquine and ivermectin). Later, we identified a potentially misinformed hypothesis derived from laboratory research (“microclots” cause the post-COVID-19 syndrome [PCS]) leading to public demand for the treatment plasmapheresis. Because there were no randomized controlled trials, we rigorously appraised these laboratory studies using Cochrane principles in collaboration with laboratory scientists and thrombotic disorder specialists.
Results: Cochrane Reviews of trials of chloroquine and ivermectin challenged fake and poorly conducted science in the public narrative; some trials were excluded from the ivermectin review based on a failure to fulfill the expected ethical and scientific criteria. These reviews were highly cited and used as the gold standard to state there was no evidence these treatments worked. Conducting a Cochrane Review of plasmapheresis for PCS was problematic, as trials had not been conducted and would be expensive. So we used novel methods to appraise the comparative serological studies that generated the hypothesis that microclots were associated with PCS. We identified bias across three domains: collection and handling of samples; experimental methodology; and reporting and interpretation of results.
Conclusions: Sometimes policy needs systematic reviews that evaluate the rationale for a treatment to help justify whether a trial is worth doing. These can be laboratory studies with sera or animal studies. Cochrane principles and methods can be adapted to meet this need, but flexibility within Cochrane formats and editorial policies could facilitate this.

4:05 PM - 4:25 PMTRIPOD-SRMA: Reporting guideline for transparent reporting of systematic reviews and meta-analyses of prediction model studiesMethodological and reporting quality

Background: Systematic reviews (SRs) and meta-analyses (MAs) of prediction model studies range in breadth and can aim to identify, appraise and summarise the evidence about existing models and their predictive performance. They differ in many important ways from SRs of intervention studies. Emphasis is on predictive performance of models obtained from internal or external validation, rather than effect sizes. Although TRIPOD provides guidance for reporting primary prediction model studies and PRISMA 2020 is available for SRs of interventions, there is currently no tailored guideline for reporting SRs of prediction model studies.
Objectives: To develop a tailored checklist for the transparent reporting of SRs and MAs of diagnostic and prognostic prediction model studies.
Methods: We formed an executive committee responsible for developing TRIPOD-SRMA. Reporting items from existing guidelines such as TRIPOD, PRISMA 2020 and others were used to produce the initial checklist. A modified Delphi approach was then used to elicit views from a wider group of experts with experience of SRs and primary prediction model studies. In the first round, 86 individuals were invited to participate, of whom 43 participants responded and formed the Delphi panel. Online surveys were used to inform the consensus process on which items to include and gather feedback to refine the checklist. After two Delphi rounds and consideration of all feedback, TRIPOD-SRMA was finalised and approved by all members of the executive committee.
Results: The new TRIPOD-SRMA checklist will be presented. It consists of 26 reporting items within six sections. Some reporting items remain unchanged or minimally modified from PRISMA 2020 as there are common elements to all SRs. However, 11 items have been tailored or included specifically to address SRs of prediction model studies. A checklist for abstracts is also included.
Conclusions: TRIPOD-SRMA and the accompanying checklist for abstracts provide the first reporting guideline for SRs and MAs of prediction model studies. Use of TRIPOD-SRMA by journals and authors will help to improve transparency of SRs of prediction model studies and ensure readers can more easily evaluate the evidence for existing prediction models within a clinical field.
Patient, public and/or healthcare consumer involvement: None.

4:05 PM - 4:25 PMIncorporating data from atypical experimental study designs in systematic reviews: experiences from Cochrane Infectious DiseasesNon-randomised studies and mixed methods

Background: For ethical, logistical and cost-related reasons, studies examining interventions to prevent occurrence and transmission of infectious diseases often use atypical experimental designs. Analysis methods for these study designs may be established, but inclusion of data from these studies in systematic reviews often raises unique challenges, which methodological experts at Cochrane Infectious Diseases must overcome by determining appropriate analysis approaches.
Objectives: To present examples of atypical experimental study designs and outline how Cochrane Infectious Diseases authors have included data from these study designs in systematic reviews.
Methods: We qualitatively reviewed examples of the inclusion of atypical experimental study designs in Cochrane Infectious Diseases reviews in recent years. We extracted information on the key features of these study designs and the analysis approach used by review authors to synthesise data from these studies. A descriptive account of the main issues that were encountered, and lessons learnt are presented.
Results: We identified three key examples of inclusion of data from atypical experimental study designs in Cochrane Infectious Diseases reviews: namely, the inclusion of 1) interrupted time series studies in a review of space spraying for preventing malaria transmission; 2) experimental hut studies in a review of piperonyl butoxide combined with pyrethroids in insecticide‐treated nets to prevent malaria; and 3) a cluster-randomised, test-negative study in a review of Wolbachia-carrying Aedes mosquitoes for preventing dengue infection. Review authors used various approaches to synthesise data from these studies.
Conclusions: Even when studies with atypical experimental designs have been carefully planned, conducted and analysed, it is not always the case that study authors report effect estimates and standard errors that can be entered directly into RevMan. Review authors must occasionally calculate effect estimates that take special features of the data (for example, clustering or seasonality) into consideration. Where effect estimates and standard errors are reported by the study authors, literature presenting the development of the atypical experimental study design and accompanying analysis methods should be reviewed carefully. Collaboration between Cochrane statisticians is key to ensuring robustness of methods and results reported from reviews that include studies with atypical experimental designs.
Patient, public and/or healthcare consumer involvement: None.

4:25 PM - 4:35 PMEnhancing capacity for network meta-analysis in Sub-Saharan AfricaEngaging stakeholders, building capacity, developing partnerships

Background: The use of network meta-analysis (NMA) in systematic reviews (SRs) of effects is increasing. Despite skilled researchers and biostatisticians in Sub-Saharan Africa (SSA), with access to formal training in evidence synthesis, there is limited training in NMA.
Objectives: To describe our experience with enhancing capacity for NMA in SSA.
Methods: We developed two NMA online courses. The 8-week Primer in NMA SRs helped participants to find, appraise, interpret, and consider the use of NMA while the 5-week Global NMA Masterclass facilitated conduct of NMA using Stata and R. A multidisciplinary team collaborated to develop the content and format with text-based lessons, self-assessments, embedded lectures, asynchronous discussions, and weekly synchronous Q&A sessions. We integrated HIV/TB and COVID clinical examples. Participants provided feedback and facilitators reflected on implementation.
Results: Forty-three participants (post-graduate students, clinicians, and decision-makers from various SSA countries) attended. They reported that the training was very relevant to their work. The self-directed, online learning approach enabled participants to learn at their convenience. However, some participants lacked motivation and commitment. As NMA is complex, requiring dedicated learning schedules, participants reported that they needed more time, and some would have preferred in-person learning. Using examples relevant to SSA and everyday problems encountered by clinicians made the learning more authentic.
Conclusions: These were the first courses related to NMA offered in SSA. Harnessing the benefits of online learning while being cognizant of the challenges and striving to improve on the offering is key for future training. There is great potential to conduct NMA across SSA. Patient, public, and/or healthcare consumer involvement: No direct involvement in the development and offering of our courses. However, given the significant interest in the examples used in the NMA workshop (HIV/TB and COVID), and their impact on patients’ lives and wellbeing, we would welcome public partnership in drawing attention to our findings.

4:25 PM - 4:35 PMMessage Lab – a platform for research and improvement of public health messagesKnowledge translation

Relevance and importance to patients: The aim of the Message Lab is to help improve how evidence-based public health messages are communicated, so that the messages and underlying research are informative and correctly understood by patients and the public.
Background: Well-designed public health messages can make people aware of what they can do (or not do) for their health and help them to make informed choices. On the other hand, poorly designed messages can confuse people, lead to poorly informed decisions, and diminish trust in health authorities and research.
Objectives: To set up a platform to facilitate the design and evaluation of public health messages.
Methods: Our starting point is evidence-based guidance for effective health communication, including the use of simple and familiar wording, clear visual design, and careful testing in target audiences. We will prioritise questions together with an international advisory group, focusing on important uncertainties about how best to communicate information about the effects of health interventions. These include, for example, how to communicate uncertainty about effects, the use of positive versus negative framing for different types of decisions, and the use of visual displays of intervention effects. We identified recruitment platforms that can be used for online randomized trials and platforms for user testing. We assessed these based on several criteria. We then tested selected platforms in a randomized trial comparing three ways of communicating the overall uncertainty of effects and comparing including the margin of error (confidence interval) with not including it.
Results: In this presentation, we will discuss the advantages and disadvantages of different user testing and recruitment platforms for online randomized trials, and we will present the findings of our trial of different ways of communicating the uncertainty of effects.
Conclusions: The Message Lab offers an efficient way for researchers to test different methods of communicating public health messages and for health authorities to design public health messages that are effectively communicated.
Patient, public and/or healthcare consumer involvement:

Message Lab Cochrane final.pdf

4:25 PM - 4:45 PMThe Cochrane Evidence Pipeline: transforming the way we identify evidenceEditorial processes and supporting review authors

Background: The Evidence Pipeline seeks to transform study identification for Cochrane and other evidence synthesis producers (see Figure). Research enters the Evidence Pipeline and goes through tailored workflows involving crowdsourcing and machine learning, working together to produce accurate, reliable metadata about studies. The Evidence Pipeline has been applied at both the review level and at the repository level. At the review level, a workflow called Screen4Me helps to reduce author screening burden through assessment of title-abstract records; at the repository level, studies relevant for Cochrane’s Central Register of Controlled Trials (CENTRAL) or the Cochrane COVID-19 Study Register (CCSR) are identified using Crowd and machine, helping to keep these important resources up to date.
Methods: This presentation will provide a detailed look at the main component parts of the Evidence Pipeline applied at the specific review level (via S4M) and at the data repository level (CENTRAL and CCSR). It will also describe our evaluation work on each component part and on the system as a whole in comparison to traditional methods.
Results: At the time of writing, the Evidence Pipeline has been used (via the Screen4Me workflow) in over 100 Cochrane reviews. The mean reduction in screening workload for author teams is 72% with a mean time to screening task completion of ten days per review. At the repository level, 99% of RCTs in CENTRAL have been identified by the Evidence Pipeline.
Conclusions: The Cochrane Evidence Pipeline aims to expedite the study identification process for reviews and other evidence synthesis outputs, offering significant workload reduction for author teams within the current review production paradigm, but is also enabling a new paradigm to emerge—one that leverages economies of effort and scale to help create comprehensive, curated repos of studies for the EBHC communities who need it. Patient, public, and/or healthcare consumer involvement: The Evidence Pipeline uses Cochrane Crowd which is Cochrane’s citizen science platform. Anyone with an interest in health can join Cochrane Crowd. This collective effort has played a significant role in enabling the Cochrane Evidence Pipeline.

Picture 1.jpg

4:25 PM - 4:45 PMDo Cochrane and non-Cochrane editors and authors prefer reporting statements based on statistically significant differences or do they prefer non-binary options?Methodological and reporting quality

Background:
Background: For decades the p value-based interpretation and reporting of results dominated the publications. Nowadays, the scientific community agrees that this binary approach is not enough. A systemic reform is necessary for moving from this binary approach to some other non-binary options. Some of them, that were discussed in the last Cochrane Handbook version (2019), are the point estimate, the confidence interval, and the minimal important difference, suggesting some narrative statements. However, it is not clear how Cochrane and non-Cochrane editors and authors agree with this approach and, even more important, if all retrieve the same conclusions when they look at some specific results.
Objectives: To evaluate how Cochrane and non-Cochrane editors and authors interpret results and which preferred reporting style is for them.
Methods: We are conducting an online survey among healthcare professionals and consumers. These stakeholders have to choose the binary or non-binary option that better expresses the results for the following scenario: After exhaustive literature searches, a systematic review identified only two pivotal randomized controlled trials (RCTs) that evaluated the effectiveness of drug X versus placebo (P) in patients with a rare genetic disease. The risk of bias for all domains was low in both RCTs (assessed using the Cochrane RoB-2 tool), and there was no methodological, clinical, or statistical heterogeneity between studies. The meta-analysis yielded the following results: Mortality risk: X 26%(10/39) and P 45%(18/40) Risk difference: X 19% lower (95% CI 40% lower to 1% higher) Risk Ratio: 0.57 (95% CI 0.30 a 1.08) P = 0.0721 *95% CI: 95% Confidence Interval
Results: will be shown at the colloquium.
Conclusions: It will be interesting to see which preferred reporting statements are for the Cochrane and non-Cochrane communities, and to compare the preferences between authors and editors. Besides, we will know the level of agreement or disagreement in each group, which is critical for the efficiency of the editorial process.
Patient, public and/or healthcare consumer involvement: Proper interpretation of study results is critical for the healthcare process

4:25 PM - 4:45 PMNon-randomised studies of interventions in systematic reviews – Limitations and opportunities illustrated with an exemplary review on COVID-19 vaccination in childrenNon-randomised studies and mixed methods

Background: Systematic reviews (SRs) of randomised controlled trials (RCTs) often provide the highest certainty evidence for decision-making. However, if RCTs are not available to inform specific health questions, other study designs should be considered. This includes nonrandomised studies of interventions (NRSIs), which are frequently used in the field of vaccination.
Objectives: The key objectives are to illustrate the limitations and opportunities of including NRSIs in SRs using an exemplary review on vaccination against COVID-19 in children.
Methods: The exemplary review (https://doi.org/10.1016/S2352-4642(23)00078-0) followed Cochrane methodological standards. Drawing on the experience gained in the review process, practical challenges from study selection to interpretation of results will be illustrated. Further, limitations and opportunities of using NRSIs for informed decision making will be explored.
Results: The following key challenges in conducting our review with NRSIs were observed: (1) diversity of study designs (e.g., traditional cohort, ecological or test-negative case-control studies) in the field of vaccination, introducing clinical and methodological heterogeneity. (2) Additional risks of bias beyond the ones known from RCTs that originate from several factors pre-, at-, and postintervention level, particularly relating to methods used for selecting the study population, assignment of intervention groups and implementation of the intervention. (3) NRSIs often present different measures of adjusted effects (e.g., risk ratios, odds ratios or incidence risk ratios), challenging the synthesis of vaccine effectiveness estimates. (4) The evidence derived from NRSIs in a fast-changing pandemic situation requires very careful interpretation in relation to applicability and transferability (e.g., baseline immunity of study population and general population or investigated virus variants).
Conclusions: NRSIs are usually at a higher risk of bias and more heterogeneous in design than RCTs. Especially in fast-changing epidemiological landscapes, as observed during the COVID-19 pandemic, available RCT data were quickly outdated, whereas some NRSIs timely addressed changes in epidemiology (e.g., shift of variants). When considering concomitant limitations, our experience showed that NRSIs in this exemplary SR could be successfully used for timely information of health policy and practice. Consumer involvement: The paediatric working group of the Standing Committee on Immunisation (STIKO) in Germany was involved from prioritisation of the review question to interpretation of the findings.

4:35 PM - 4:45 PMPartnerships for practice improvement: JBI's collaborative approach to developing evidence-based point-of-care resourcesEngaging stakeholders, building capacity, developing partnerships

Background: Developing clinically relevant, trustworthy, evidence-based resources to connect research to practice and assist with informing clinical decision-making is central to the improvement of healthcare outcomes. Development of such resources requires a multifaceted, coordinated, collaborative partnership approach. JBI has established a unique framework to engage and collaborate with key stakeholders across 32 specialty field areas to develop rigorous, up-to-date, clinically relevant evidence-based resources to inform clinical practice. Objective: To describe and discuss the JBI framework for collaborative partnership for practice improvement within the core activities of the JBI Transfer Science program.
Results: Underpinned by a rigorous methodological approach, more than 5,000 evidence-based resources across 32 specialty fields have been developed using a collaborative partnership framework. The core components and key characteristics of the partnership approach (including a scientific methodology group, expert reference groups, field collaborators and scientific writers) will be presented alongside discussion of the practicalities, enablers and barriers of implementing this framework within specialty field groups. The individual and shared role and function of the group will be explored with supportive examples demonstrating impact on practice. Conclusion: Incorporating a strategic framework that focuses on collaborative partnerships, in which a shared common goal between the groups is to improve evidence-based practice, can assist with reducing the research-practice gap and improving care delivery.

4:35 PM - 4:45 PMHighlights and challenges of co-creating evidence-based knowledge translation tools for parents about the COVID-19 pandemicKnowledge translation

Background: The COVID-19 pandemic presented an unprecedented need for trusted evidence. We have an established program to co-create knowledge translation (KT) tools for parents and families that merge the best available research evidence with parent experiences to support decision-making for their children’s health. We received funding from the Canadian Institutes of Health Research to conduct qualitative research to identify parents’ experiences and information needs related to the pandemic and to co-create six KT tools.
Objectives: Describe the process of co-creating evidence-based KT tools for parents and discuss highlights and challenges.
Methods: We conducted qualitative interviews and focus groups with parents to understand their experiences and information needs related to caring for their children during the pandemic. We involved our established parent advisory group throughout the process to guide methods, pilot test processes, assist with recruitment, review KT tool prototypes, and participate in usability testing.
Results: The following themes were identified among parents who had managed a child with COVID-19: dealing with uncertainty, anxiety, stress and stigma, and getting through it. Themes related to the pandemic more generally were the following: loss of skill (children), changes in physical health, children’s and parents’ mental health, and accessing trustworthy information. After discussing themes with the parent advisory group, three topic areas were selected for KT tool development: managing a child with COVID-19, socialization during and beyond the pandemic, and COVID-19 vaccination. Six tools were developed: animated video and interactive infographic for each topic (https://www.echokt.ca/covid/).
Conclusions: A highlight and strength was working directly with parents. We involved our established parent advisory group as a partner on the funding application and throughout the project, and we believe this was critical to our success. Challenges included the number of KT tools (2 tools on each of 3 topics) developed using multiple methods (interviews, focus groups, living evidence synthesis) within a relatively short timeframe; changing evidence, information and protocols over time and across jurisdictions; reliance on external sources for evidence and information; appropriate messaging (informative vs. directive); and acknowledging and affirming parents’ varied experiences. Patient, public, and/or healthcare consumer involvement: Parents contributed to methods, pilot testing, recruitment, KT tool development, and usability testing.

4:45 PM - 4:55 PMUnlocking innovations for sharing and using Cochrane content with study centric data management and a focused review articleEditorial processes and supporting review authors

Background: Cochrane was founded in 1993 and has grown into a global network producing gold standard evidence syntheses. Over those years we have been doing things in the same way and during that time the field has evolved. To maintain our place as one of the world’s leaders in producing trusted evidence, we are making some big changes in 2023. Reviews will be shorter. Users of evidence are best served by a more focused article. This also makes writing and updating reviews simpler as you do not need to make changes in multiple places. The main body of the review will contain sections that are most useful to users of Cochrane evidence, with a clearer distinction between the main article and supplementary materials. With a shorter review, we are moving the structure to where it is needed most: the data. This change is powered by moving to study centric data management in RevMan, which comes with a host of benefits for authors. By improving the structure of data associated with Cochrane reviews, we can increase their impact by making it easier for policy makers, guideline developers, and others to reuse our evidence.
Objectives: This presentation will provide an update on changes to the review format and data management for Cochrane reviews. It will cover: 1. Why has Cochrane changed the review format and how data are managed in RevMan? 2. What does it mean for review authors, editors, peer reviewers, and users of Cochrane evidence? 3. How the changes will unlock the potential for innovations in how we tailor our evidence for key stakeholders.
Conclusions: The standardized review article with more structured data will result in a better experience for authors, with the reassurance that fewer words do not equate to lower standards. The process of producing Cochrane reviews will be more efficient, facilitating living reviews and unlocking the potential for innovations in how we tailor our evidence for key stakeholders. Patient, public, and/or healthcare consumer involvement: User experience testing and consultations informed developments.

4:45 PM - 4:55 PMDriving the evidence-based healthcare agenda forward: a qualitative reflexive thematic analysis of global health partnershipsEngaging stakeholders, building capacity, developing partnerships

Background: The global evidence community has long recognised that collaboration is key to producing trustworthy, pragmatic evidence. The COVID-19 pandemic highlighted the need for partnerships in developing rapid evidence-informed responses, streamlining global efforts, reducing research waste and ensuring the best-available evidence is accessible, transparent and understood. There is, however, growing concern about making partnerships equitable for—and beneficial to—all partners. Although collaboration has increased, vested interests, bureaucracy and inability to adapt remain limiting factors. The campaign for World Evidence-Based Healthcare (EBHC) Day 2022 focused on ‘partnerships for purpose’ within and across the global evidence ecosystem, calling on the global evidence community to share their experiences through blog submissions around the formation, development, maintenance, evaluation and outcomes of partnerships in EBHC.
Objectives: To explore the challenges, facilitators and outcomes of global health partnerships promoting and supporting evidence-based healthcare.
Methods: This is an inductive qualitative study of the 41 blogs published for the World EBHC Day 2022 Campaign. Analysis was conducted using reflexive thematic analysis according to Braun and Clarke.
Results: Numerous themes were identified across three research foci, including shared challenges experienced across partnerships relating to the lack of equity, diversity and inclusion policies, practice or understanding; navigating different operational systems and working practices; and programmatic misalignment. Results indicated that key facilitators for successful partnerships were dedicated resources for partnership management; commitment to information sharing, transparency and open communication; a focus on equity, diversity and inclusion; ensuring a common purpose and aligned values; and working to actively build trust, respect, relationships and social capital. Reported outcomes from partnerships included increased efficiencies and reduced duplication of effort; leveraging shared resources and expertise; addressing knowledge needs/gaps; increased funding opportunities; professional development; and expanded networks, increased reach and global impact.
Conclusions: Since COVID-19, we have had to come to terms with a tectonic shift in how our partnerships and relationships will work in its aftermath. To this end, the findings from this study and the 41 blogs published for World EBHC Day provide valuable lessons for navigating new ways of working together. Patient, public, health consumer involvement: Several blogs were written by patient advocates.

4:45 PM - 4:55 PMStakeholder involvement for investigating communication-based interventions to increase COVID-19 vaccine uptakeKnowledge translation

Background: Long-awaited COVID-19 vaccines are effective in preventing severe COVID-19 disease. However, vaccine hesitancy, identified as one of the top 10 threats to global health by the WHO, has proven to be an important issue in the COVID-19 pandemic.
Objectives: Involve important stakeholder in the review process and investigate the effectiveness of communication strategies to increase COVID-19 vaccine intention.
Methods: Within this project funded by the Germany Research Foundation, we invited stakeholders to an online meeting to develop a relevant research question. The discussion was based on the findings of our prior scoping review, which maps current evidence on interventions to increase COVID-19 vaccine intention. Together with relevant stakeholders from policy and research, we prioritised types of interventions and target populations and developed the PICO for a subsequent systematic review. We will discuss the implementation of the intervention strategies, identified in the systematic review, in another stakeholder meeting by the end of the project.
Results: As the initial stakeholder meeting was so successful, most stakeholders agreed to further participate in the project and be involved in the upcoming systematic review. Thus, we had biweekly meetings until the end of the project with experts from the Robert Koch Institute and the NRW Center for Health. We conducted the planned systematic review of randomised controlled trials and included studies investigating communication-based interventions to increase COVID-19 vaccine uptake, for any population. The policymakers were particularly interested in communication strategies, as these are easy to implement. Outcome measures, considered most relevant by the stakeholder, were COVID-19 vaccine uptake, vaccine confidence and intention. We performed meta-analyses using Review Manager Desktop version 5.4. Our findings included 27 studies. Evidence from our meta-analyses shows that for COVID-19 vaccine uptake there may be a slight increase in vaccine uptake when education and information strategies or social norm strategies are applied, compared to no intervention.
Conclusions: Our findings will have implications for policy decision-making in prospective vaccination promotion, not only regarding COVID-19. Patient, public, and/or healthcare consumer involvement: Involving stakeholders in the review process ensured that relevant research questions are answered and facilitated translation and dissemination of the results.

4:45 PM - 4:55 PMMethodological and reporting quality of systematic reviews and meta-analyses in dermatology: a cross-sectional studyMethodological and reporting quality

Background: There has been an increasing number of systematic reviews and meta-analyses in dermatology over the past decade, however the quality of these studies is unknown.
Objectives: To determine the change in the methodological and reporting quality of systematic reviews and meta-analyses in dermatology over the past decade.
Methods: This was a cross-sectional study of systematic reviews and meta-analyses in dermatology published in the 10 highest ranked dermatology journals in 2010 and 2019. Included studies were identified by searching MEDLINE, Embase, CDSR, ACP Journal Club, DARE, CCA, CCRCT, CMR, HTA, and NHS EED. Methodological quality was assessed in duplicate by two independent investigators with the ROBIS tool and, for studies of interventions, the AMSTAR-2 tools. Reporting quality was assessed with the PRISMA 2009 and PRISMA-A 2013 statements.
Results: We identified 27 systematic reviews and meta-analyses published in 2010 and 127 published in 2019. There was no evidence of a difference in the proportion of systematic reviews and meta-analyses at high/unclear risk of bias with ROBIS (Fisher’s exact test = 1.00) or with critically low methodological quality using AMSTAR-2 (Fisher’s exact test = 0.456), between 2010 and 2019. The difference in proportion of systematic reviews and meta-analyses at high/unclear overall risk of bias with ROBIS was -1.2% (95% CI -17.3%-14.9%) in 2010 (81.4%) than 2019 (82.6%) (Figures 1-2). There was evidence of a difference in proportion of PRISMA (t(26)=2.7, p=0.01), and very strong evidence of a difference in proportion of PRISMA-A (t(26)=4.2, p<0.001) checklist items adequately reported between 2010 and 2019. The difference in mean proportion of PRISMA checklist items adequately reported was 3.6 items more (95% CI 1.8 – 5.4 items more) in 2019 (mean=10.7 items, SD=2.4 items) than 2010 (mean=7.1 items, SD=2.9 items), and of PRISMA-A checklist items adequately reported was 1.1 items more (95% CI 0.2–2.0 items more) in 2019 (mean=5.6 items, SD=1.5 items) than 2010 (mean=4.4 items, SD=1.7 items)
Conclusions: There was no improvement in the methodological quality of systematic reviews and meta-analyses in dermatology between 2010 and 2019, but strong evidence of an improvement in the reporting quality.
Patient, public and/or healthcare consumer involvement: None.

4:45 PM - 4:55 PMSupplementing Systematic Review Evidence with Health System Data: A Target Trial Emulation of Triptan in Migraine Patients with Increased Cardiovascular RiskNon-randomised studies and mixed methods

Background: A recent framework from the US Evidence-based Practice Centers advises that when evidence from systematic reviews is deemed insufficient for decision-making, electronic health record (EHR) data from health systems should be considered to fill the knowledge gap. A 2021 systematic review demonstrated that triptans improved pain and function for acute migraine attacks. However, the systematic review identified insufficient evidence about patients with increased risk of cardiovascular events who were often excluded from clinical trials.
Objectives: To evaluate the safety of triptan treatments in migraine patients with cardiovascular disease or multiple cardiovascular risk factors using EHR data from a large tri-state health system.
Methods: This target trial emulated a randomized clinical trial that hypothetically assigned patients to triptans or no triptan treatments. Eligible patients were adults (aged ≥18 years) with at least a 1-year history of migraines, confirmed cardiovascular or cerebrovascular disease, or with at least two cardiovascular risk factors. Patients in the triptan group were matched with those in the no-triptan group in a 1:1 ratio. The primary outcome was major adverse cardiovascular events (MACE) at 60 days of starting treatments.
Results: A total of 3,719 patients who received triptan were matched to 3,719 patients who did not receive triptans (Table 1). Figure 1 shows the cumulative incidence curves for MACE. The absolute risk of MACE outcomes was low (Table 2). At 60 days, 154 patients (4.14%) in the triptan group reported a MACE outcome compared to 37 patients (0.99%) in the no triptan group (RR: 4.16; 95% CI: 2.96 – 5.84). Patients treated with triptans also had significantly higher risk of nonfatal myocardial infarction, nonfatal stroke, transient cerebral ischemia, and heart failure. Five patients (0.13%) in the triptan group died versus seven (0.19%) in the non-triptan group. Certainty in the evidence was judged to be moderate.
Conclusions: This target trial emulation showed that triptans were associated with increased risk of MACE outcomes, raising the certainty of evidence in this outcome from insufficient, as was judged in a systematic review, to moderate. Patient, public, and/or healthcare consumer involvement: None.

table 1.pdf
Table 2.pdf
Figure1.pdf

4:55 PM - 5:05 PMUpdating the web-based "Right Review" tool: an international Delphi processEditorial processes and supporting review authors

Background: A diverse range of evidence synthesis approaches have been developed to meet end users’ (such as policy makers) knowledge needs. Examples include systematic reviews, rapid reviews, scoping reviews, realist reviews, network meta-analysis, meta-narrative reviews, and meta-synthesis. For researchers and those commissioning reviews, choosing the most appropriate method may be confusing. “Right Review” is a web-based decision support tool that guides users through a series of simple questions for recommending evidence synthesis methods suitable for their research question. Currently, the tool separates quantitative reviews and qualitative evidence synthesis.
Objectives: To update the Right Review tool to establish a common set of questions for the synthesis of both quantitative and qualitative studies.
Methods: A two-round modified online international Delphi was conducted (2021) with researchers, policy makers, health care providers, patients, and decision makers. Along with demographic questions, participants were asked to rate the importance and or clarity of the tool guiding questions, definitions of evidence synthesis types, and the tool output structure. A consensus definition of at least 70% agreement for each item was decided a priori. Any items not reaching consensus after round 2 were discussed by the international project steering committee (PSC).
Results: Twenty-four experts from 9 countries completed round 1 and 12 round 2. A total of 46 items were presented in round 1, with 21 reaching consensus. Twenty-seven items were presented in round 2, with eight reaching consensus. Items not reaching consensus, including 8 tool guiding questions, 9 review definitions (predominantly related to qualitative evidence), and 2 tool output items, were brought forward for discussion by the PSC. Three items were removed entirely, and the remainder revised and edited and/or combined with existing items.
Conclusions: The Delphi process informs the update of this tool which is free, practical decision support tool that helps reviewers choose an appropriate method.

4:55 PM - 5:05 PMOpioid use disorder treatment in Canada: knowledge sharing between regionalized networksEngaging stakeholders, building capacity, developing partnerships

Background: Opioid agonist therapy (OAT), typically involving buprenorphine or methadone as pharmacotherapy, is an effective treatment for opioid use disorder (OUD) and is a key response to growing opioid-related harms in Canada. OAT delivery in Canada varies between provinces/territories, including differences in prescribing requirements and primary care involvement. For provinces/territories that aim to improve OAT delivery, knowledge of other jurisdictions’ policies, expected outcomes, and how they have been influenced by political, sociocultural, and health system factors, can inform clinical and policy decision-making.
Objectives: To compare OAT prescribing policies across Canadian provinces/territories and understand how and why they occur.
Methods: We conducted a comparative policy analysis, collecting data from policies, guidelines, reports, education/training materials, and published research articles on provincial health systems and drug policies. To describe and compare OAT across all Canadian provinces, we constructed timelines of OAT policies for each jurisdiction from 2014. Experts in each province (clinicians; policymakers) reviewed these to ensure accuracy and relevance. We classified these policies based on provider involvement; financial, personnel, and educational resources for providers; treatment settings; and costs to patients. We also examined any cross-provincial interactions and resource sharing.
Results: In May 2018, Health Canada removed a federal requirement to obtain an exemption to prescribe methadone, which could increase patients’ access to this treatment. After examining provincial policy changes before and after this decision, we identified patterns in OAT prescribing regulations: western jurisdictions (e.g., British Columbia, Alberta, Saskatchewan) tended towards more restrictive prescribing requirements and oversight than their eastern counterparts (e.g., Quebec, New Brunswick, Nova Scotia). We also observed similar geographical patterns with recommendations for providers to use training/guidelines from either British Columbia (west) or Ontario (east).
Conclusions: Our findings suggest that OUD treatment policies may be influenced by geographical, political, and/or professional networks. What is considered evidence or knowledge for decision-making may be shaped by these factors and local policymaking needs. Patient involvement: By engaging people with lived/living experience and decision-makers, further exploration of these factors and their effect on OUD treatment policies can better inform the development of contextually relevant and appropriate policies to improve OUD care.

4:55 PM - 5:05 PMBringing Cochrane reviews to Polish professionals and consumersKnowledge translation

Background: Improving health literacy and increasing the use of reliable health information may reduce vulnerability to unreliable health-related information. Health professionals may help in translating the knowledge to nonprofessionals, but they often find it challenging.
Objectives: To show our experiences and pathway of knowledge translation (KT) actions for consumers.
Methods: We applied for funding from the Social Responsibility of Science. Project started in the November 2022 and will last for 2 years. The project is based on Cochrane systematic reviews (SRs) and plain language summaries (PLSs). We started collaboration with health professionals to prioritise SR’s topics which are deemed important to healthcare practice (e.g., physiotherapists) and to patients’ groups with whom we already cooperate (e.g., cystic fibrosis patients). Prioritization was based on subjective assessments on a scale from one to ten of the utility of the topic for healthcare practice completed by two healthcare professionals independently. SRs which got the highest assessments and were deemed to be current were grouped into themes by health condition. Results from SRs will be shared using plain language summaries and additionally expressed in the form of infographics and animations. All necessary professional terms will be explained with comic drawings or short (1-3 minute) webinars. The aim for all is to make an easily readable and understandable “information pill” from each theme. All our experiences and expertise, gathered from our work as Cochrane Poland, will be structured to prepare the guide for Polish professionals to help in everyday KT.
Results: Project is connected with translations of 120 PLSs per year and preparation of additional products (infographics, animations). We expect to have at least three topic themes expressed in plain language and described in graphic forms for presentation during the Cochrane Colloquium.
Conclusions: Promotion of SRs among health professionals in Poland still needs effort, but translation of studies results’ into easily accessible language for patients and lay audience is even more challenging and emerging.
Patient, public and/or healthcare consumer involvement: Our project mainly aims to bring professional knowledge from Cochrane SRs close to patients’ everyday lives.

4:55 PM - 5:05 PMMethodological quality of systematic reviews on Chinese herbal medicine: a methodological surveyMethodological and reporting quality

Background: Systematic reviews (SRs) synthesise the best evidence of effectiveness and safety on Chinese herbal medicine (CHM). Decision-making should be supported by the high-quality evidence of prudently conducted SRs, but the trustworthiness of conclusions may be limited by poor methodological rigour.
Methods: This survey aimed to examine the methodological quality of a representative sample of SRs on CHM published during January 2018 to March 2020. We conducted literature search in the Cochrane Database of Systematic Reviews, MEDLINE via Ovid, and EMBASE via Ovid. Eligible SRs must be in Chinese or English with at least one meta-analysis on the treatment effect of any CHM documented in the 2015 Chinese Pharmacopoeia. Two reviewers extracted the bibliographical characteristics of SRs and appraised their methodological quality using AMSTAR 2 (Assessing the Methodological Quality of Systematic Reviews 2). The associations between bibliographical characteristics and methodological quality were investigated using Kruskal-Wallis tests and Spearman’s rank correlation coefficients.
Results: We sampled and appraised 148 SRs. Overall, one (0.7%) was of high methodological quality; zero (0%), four (2.7%), and one-hundred forty-three (96.6%) SRs were of moderate, low, and critically low quality. Only 13 SRs (8.8%) provided a pre-defined protocol; none (0%) provided justifications for including particular primary study designs; six (4.1%) conducted a comprehensive literature search; two (1.4%) provided a list of excluded studies; nine (6.1%) undertook meta-analysis with appropriate methods; and seven (4.7%) reported funding sources of included primary studies. Cochrane reviews had higher overall quality than non-Cochrane reviews (P ˂ 0.001). SRs with European funding support were less likely to have critically low quality when compared with their counterparts (P = 0.020). SRs conducted by more authors (rs = 0.23; P = 0.006) and published in higher impact factor journals (rs = 0.20; P = 0.044) were associated with higher methodological quality.
Conclusions: Our results indicated that the methodological quality of SRs on CHM is low. Future authors should enhance the methodological quality through registering a priori protocols, justifying selection of study designs, conducting comprehensive literature search, providing a list of excluded studies with rationales, using appropriate method for meta-analyses, and reporting funding sources among primary studies.

4:55 PM - 5:05 PMUse and methodology of framework synthesis in mixed-methods literature reviewsNon-randomised studies and mixed methods

Background: Framework synthesis (FS) is a method for synthesis of qualitative research in systematic reviews (SRs), which is being increasingly utilised to synthesise data from diverse sources to explore complex healthcare issues. While published guidance on conducting mixed-methods SRs and qualitative FS is available, there is little consensus on conducting mixed-methods FS.
Objectives: To conduct a targeted literature review (TLR) to evaluate the current use and methodology of FS in mixed-methods literature reviews.
Methods: This TLR was conducted according to a prespecified protocol. In February 2023, we searched MEDLINE, Embase and Cochrane Database of Systematic Reviews (via Ovid) for mixed-methods reviews using FS. Records were reviewed by a single individual at the title/abstract and full-text review stages. Information was extracted on rationale for FS use, framework development, extraction, synthesis and integration of quantitative and qualitative data, quality assessment (QA), outputs and reported limitations of FS.
Results: We initially identified 240 unique records, of which 123 were excluded at title/abstract review and 68 excluded at full-text review, leaving 49 peer-reviewed publications for analysis. All but four reviews were SRs, with two rapid reviews, one scoping review and one SR of SRs. Primary quantitative, qualitative or mixed-methods studies were eligible for inclusion; secondary research or opinion were included in less than 25% of reviews. Two reviews only extracted author summary or interpretation of quantitative findings. The level of detail on how FS was conducted varied, with some detailing the process step by step and others only citing the method. Very few described how quantitative and qualitative data were integrated. Where reported, methods included convergent synthesis, whereby quantitative data are transformed into qualitative statements through narrative interpretation (“qualitising”), and the Pillar Integration Process, whereby data are listed by type and then matched to corresponding qualitative or quantitative data before further synthesis. Most reviews used QA tools designed to appraise multiple study designs; however, some did not perform formal QA owing to heterogeneity. Few reviews cited qualitative reporting guidelines.
Conclusions: The volume of mixed-methods literature reviews using FS being published is increasing. With limited consensus on best practice, further guidance and improved transparency in reporting is required.

5:05 PM - 5:15 PMThe Systematic Review Toolbox: an updated resource to support evidence synthesisEditorial processes and supporting review authors

Background: In 2014, the Systematic Review (SR) Toolbox was developed to gather tools that could support researchers undertaking an SR. As the breadth of evidence synthesis methodologies have expanded greatly since this time, we updated the SR Toolbox in 2022 to reflect these developments. We also analysed the software tools and guidance within the SR Toolbox at the time to identify gaps that could potentially be filled by new tools and guidance in future.
Objectives: To describe the process and present the results of updating the SR Toolbox and analysing its content.
Methods: In February 2022, we manually extracted all guidance documents and software tools within the SR Toolbox to that point on a Microsoft Excel spreadsheet. The spreadsheet categorised records by types of evidence syntheses records as described by Sutton et al. (2019) and by the stage of process in undertaking evidence syntheses (e.g., searching, screening, and data extraction). A single reviewer extracted and analysed these records, with a second reviewer checking a proportion for accuracy. By using this spreadsheet and Microsoft Access, we updated the SR Toolbox and performed a simple quantitative analysis on the included records.
Results: On 13 May 2022, we launched the updated SR Toolbox. At the time, 235 software tools and 112 guidance documents were included. Most guidance documents (N = 78) and software tools (N = 223) were applicable to systematic reviews of interventions. However, there were fewer included tools and documents that related to reviews of reviews (N = 66 software tools and N = 22 guidance documents) and qualitative evidence syntheses (N = 19). Most documents provided guidance on critical appraisal and quality assessment (N = 70), with software mainly focusing on searching (N = 84) and data synthesis (N = 82). However, there were very few tools or guidance documents that related specifically to stakeholder engagement (N = 5).
Conclusions: The SR Toolbox can support those looking to undertake different types of evidence syntheses. However, there are currently gaps in the guidance and software tools available, which may warrant further research.
Patient, public and/or healthcare consumer involvement: None.

5:05 PM - 5:15 PMGlobal evidence, local adaptation (GELA): Enhancing evidence-informed guideline recommendations for newborn and young child health in three countries in sub-Saharan AfricaEngaging stakeholders, building capacity, developing partnerships

Background: Despite progress in the health of newborns and children, most countries in sub-Saharan Africa have not met the Sustainable Development Goals for under-five mortality. Consequences of COVID-19 are being felt on health systems and services, further hampering healthcare delivery to children alongside growing poverty, hunger and inequity. Clinical practice guidelines (CPGs) are needed to bridge the gap between research evidence and practice directly impacting patient care, health system funding and access to health services.
Objectives: The Global Evidence, Local Adaptation (GELA) project is working with ministries of health in South Africa, Nigeria and Malawi to maximise the impact of research through increasing decision-makers’ and researchers’ capacity to use global research to develop locally relevant CPGs for newborn and child health.
Methods: From 2023 to 2025, we are implementing a multi-faceted, multidisciplinary research and capacity strengthening programme, adaptation methodology and digital platforms to support authoring delivery for contextually rich CPGs. This is delivered through partnerships with national CPG groups, including policy makers, epidemiologists and civil society representatives.
Results: A 3-year project was successfully initiated in all countries. Steering Group members and Guideline Development Panelists were identified and are participating in various project-related activities. Recently, the teams completed a landscape analysis of CPGs in newborn and child health and concluded the priority-setting process identifying three topics for new CPGs for newborn and child health per country. Monitoring and evaluation activities are using a variety of approaches including integrated knowledge translation, observation of CPG groups and evaluation of capacity needs for decision-makers.
Conclusions: Overall, the project is on course to achieve its targets with priority topics identified and CPG groups being convened. Our success is enabled through a project team of African and international leaders in the field of evidence-based healthcare and CPG methods who have partnered with national ministries in Malawi, Nigeria and South Africa, the World Health Organization headquarters (WHO HQ) and the Afro regional office and civil society groups. Ongoing evaluation will help us learn which aspects work well to reduce waste and save resources for our settings and may be scalable to other countries like ours.
Patient, public and/or healthcare consumer involvement: Yes.

5:05 PM - 5:15 PMAbstracts of Cochrane reviews are getting longer, but this has no large impact on the reporting quality.Knowledge translation

Background: In our previous study on the reporting quality of abstracts according to PRISMA-A, we found that abstracts of Cochrane reviews (CRs) were better reported than non-Cochrane reviews, differed significantly in length (median length: 507 words vs. 249 words), and appeared to be getting longer in recent years. However, the considered sample of CRs was too small to draw robust conclusions. Overall, there is a general lack of studies that address the quality of reporting or trends in the length of the abstracts of CRs.
Objectives: To evaluate the reporting of abstracts of CRs according to PRISMA-A and to investigate the possible influence of the abstract´s length.
Methods: This was a retrospective, observational study based on all CRs indexed in Medline (via PubMed) until November 18th, 2022. Trend analyses were performed on the number of publications and length of abstracts over the years for all 15,188 abstracts. Then, PRISMA-A adherence was assessed by two independent reviewers for a random sample of a planned sample size of 440 abstracts. Results were stratified by the number of words and year of publication.
Results: Overall, the median number of words in the abstract was 469 (IQR: 389-686 words), steadily increasing from 353 words in 2000 to 807 words in 2021, with less than 1% of the abstracts being less than or equal to 300 words (in 2000: 30.7%). Analyses on PRISMA-A adherence show a mean score of 6.1 fully reported items. Stratified by year, PRISMA-A adherence increases with increasing word count in 2000-2010 and 2011-2015, whereas there is no difference in PRISMA-A adherence by the abstract length in 2016-2022.
Conclusions: Over the years, abstracts of CRs have become longer and longer, running up to 1,000 words. This conflicts with the Cochrane Handbook, which recommends a maximum length of 400 words before being adapted to MECIR in 2019, which has recommended a length of less than 700 words since 2012 but allows a length of up to 1,000 words. It is questionable whether such long abstracts meet the goal of presenting all essential contents of the article in an informative, accurate, appealing, and concise form. Patient, public, and/or healthcare consumer involvement: No involvement.

5:05 PM - 5:15 PMFrequency of use and reporting adequacy of Cochrane RoB 2 tool in non-Cochrane systematic reviews published in 2020: meta-research studyMethodological and reporting quality

Background: Risk of bias (RoB) assessment is an integral part of the systematic review methodology. New version of the Cochrane RoB tool for randomized trials (RoB 2) was published in 2019. It has been reported that the Cochrane RoB tool from 2011 was used inadequately in Cochrane and non-Cochrane reviews. There are insufficient data about the use of RoB 2 tool.
Objectives: This study aimed to analyze the frequency of usage of RoB 2 tool and the adequacy of reporting RoB 2 assessments in non-Cochrane reviews published in 2020.
Methods: We conducted a meta-research study. We included non-Cochrane systematic reviews of interventions published in 2020. For the reviews that have used RoB 2 tool, we analyzed how the findings of the RoB 2 assessment were reported.
Results: Among 3880 analyzed reviews, the authors most frequently reported use of the Cochrane 2011 RoB tool (N=2228; 57.4%), followed by Cochrane RoB 2 tool (N=267; 6.9%). From 267 reviews that reported using RoB 2 tool, 213 (79.8%) actually used RoB 2 tool. Among 213 reviews that used RoB 2 tool, 180 (85.4%) used all domains specified by the RoB 2 tool. In 26 (12.2%) reviews, erroneous statements were used to indicate RoB 2 assessment. Only 20 (9.4%) reviews presented a complete RoB 2 assessment through a detailed table with answers to all signaling questions. The judgment of risk of bias through RoB 2 tool was not justified by a comment in 158 (74.2%) of reviews. Only in 33 (14.5%) of reviews, the judgment in all domains was justified in the accompanying comment. Conclusion: The majority of non-Cochrane reviews published in 2020 still used RoB 2011 tool and many authors reported the results of RoB 2 assessment inadequately. Interventions for improving RoB 2 tool use in non-Cochrane reviews are warranted. Relevance and importance to patients: Efforts to improve research methodology and how studies are conducted can lead to better conducted and written studies, and better evidence that can be used for managing patients.

5:05 PM - 5:15 PMInterpreting the results of observational studies in the context of variation expected due to analytic flexibilityNon-randomised studies and mixed methods

Background: When investigators analyze data from observational studies, they make numerous potentially justifiable, but still subjective, analytic decisions on which direction, magnitude, and statistical significance of findings can be contingent. This allows investigators to test many alternative analytic specifications and selectively report results for the specification that yields the most statistically significant or interesting results. Objective: We present a novel approach to interpret the results of observational studies in the context of the variation expected because of analytic flexibility. We apply this new method to the effect of unprocessed red meat on all-cause mortality.
Methods: We reviewed all observational studies addressing the effect of red meat and all-cause mortality from a recently published systematic review and documented variations in analytic choices across studies. We used data from the National Health and Nutrition Examination Survey (NHANES) 2007 to 2014 linked with National Death Index and applied specification curve analysis—a novel analytic method that involves defining and implementing all plausible and justifiable analytic specifications—to investigate the effect of unprocessed red meat on all-cause mortality. Our choice of analytic specifications was informed by analytic methods used in published studies addressing the same or similar questions.
Results: We applied specification curve analysis to NHANES, including 10,661 participants. In total, we performed 1,208 unique analyses, 48 (3.97%) of which produced statistically significant results, 40 (83.33%) of which indicated red meat reduced all-cause mortality, and 8 (16.67%) of which indicated red meat to increase mortality. The specification curve analysis produced a median hazard ratio of 0.94 [IQR: 0.83 to 1.05].
Conclusions: Our results suggest that inconsistency in the reported results in the literature may be explained by differences in analytic methods. We encourage evidence users to interpret the results of observational studies in the context of variation expected due to analytic flexibility. Patient, public involvement: We did not involve consumers in this investigation.

VoE Figure.pdf

5:15 PM - 5:25 PMChallenges of coordinating large-scale systematic reviewsEditorial processes and supporting review authors

Background: Large-scale systematic reviews are becoming more common owing to the rapid growth in publications on certain topics. They raise unique challenges for managing and coordinating the review process for which there is limited guidance. We will present the challenges we encountered and how we addressed them during two large Cochrane systematic reviews: Quality improvement strategies for diabetes care (n=553 trials) and Audit and feedback (n=290 trials).
Objectives: To address the logistical challenges that are common in the coordination of large systematic reviews.
Methods: We will compare the approaches to coordinating two large systematic reviews. We will focus on common logistical challenges such as recruiting and maintaining a team with enough availability to complete the project in a timely manner; delegating work and ensuring consistency between reviewers on a large team; and managing and housing a very large dataset. We will also compare tools for managing large reviews such as the systematic review programs DistillerSR and Covidence for usability and customisability as well as online repositories for external data storage.
Results: The main challenge in coordinating large systematic reviews is dedicating enough resources to ensure the review can be completed before another update is required. Having a large team can be useful for keeping momentum going in screening and extraction; however, it is vital to ensure that reviewers are being consistent by using clear and specific instructions and having a training period for each reviewer. Smaller teams are better for ensuring consistency in reporting, but screening and extraction takes longer. The use of technology such as DistillerSR and Covidence facilitates the organisation of screening a large set of studies. DistillerSR offers more customisability but may be cost-prohibitive. Covidence links nicely with RevMan Web but may not be as user-friendly. An online data repository such as Zenodo is useful for streamlining the main review and preventing any issues with the Cochrane CENTRAL system when it comes time for publication; however, there is always a risk of repositories no longer being in service in the future.

5:15 PM - 5:25 PMA method for the rapid assessment of high-cost cancer drug indications in a hospital settingNon-randomised studies and mixed methods

Background: Hospitals are usually the entry point of new technologies. Facing the challenge posed by the growing costs of cancer drugs, there is a need for healthcare providers to articulate systematic and timely formulary decision-making processes considering not only the efficacy and safety of these treatments, but also their economic impact and efficiency.
Objectives: To develop a method for the rapid assessment of high-cost cancer drug indications (HCCDI) for formulary decision-making in a hospital setting.
Methods: We identified and structured decision criteria from previous formulary decisions regarding HCCDI, and a literature review of decision frameworks in oncology and the Chilean health reimbursement system. We generated a criterion/indication combination score for two samples of HCCDI (solid tumors and hematologic oncology) by measuring their performance and translating it to a common 0-100 scale. From the consensus among clinicians, managers, pharmacists, and health economics researchers of the institution, we assigned weights to each criterion to calculate the aggregate scores. Finally, we validated each step of the process with the decision-makers.
Results: The identified criteria were (1) absolute and relative difference in overall survival and (2) progression-free survival of the area under the Kaplan-Meier curves; (3) difference in the proportion of patients with severe adverse events; (4) improvement of quality of life; (5) incremental expected costs; and (6) efficiency. The inputs for the assessment of each HCCDI were high-quality studies identified through rapid reviews of electronic databases and health technology assessment reports. Aggregation of criteria 1-4 was named “benefit index”. The quotient of the benefit index and the HCCDI costs was called “efficiency index”. The costs were then used to calculate the indication’s budget impact.
Conclusions: We developed a formulary decision-making tool for the rapid assessment of HCCDI validated by decision-makers. The method has allowed us to assess 79 indications for solid tumors and 21 for hematologic oncology since 2019, facilitating the reproducibility of the decision criteria. Improvements in the principles regarding HCCDI formulary decisions contribute to a more egalitarian use of resources and patients’ access to healthcare. Patient, public, and/or healthcare consumer involvement: No patients, public, or healthcare consumers were involved in this project.

11:05 AM - 11:15 AMArchie Cochrane's Second E - the creation of Value based HealthcareGlobal health, equity and partnerships

Every country on earth is now facing a resource crisis that will not be resolved simply by focusing more on effectiveness; a new paradigm is required - value based healthcare (VBHC). The EU definition of VBHC is now adopted globally, except in the USA which is not committed to universal healthcare. Their the term ‘value’ is the relationship between outcomes and costs for the patients treated which would be classified as efficiency in all other countries, but Archie did not use the term Efficiency in that narrow way. Archie wrote that he was concerned about ‘the optimum use of equipment and personnel ...to cover all these activities I have used the word efficiency’. He was clearly thinking about value, the first person to do so in modern healthcare. The EU definition has four dimensions - personal, allocative, technical and social, and all four are excellently introduced by Archie. Personal value resonates through his description of care in the prisoner of war camp. Allocative value is highlighted in his text on ‘inequalities between diseases... the division of the medical budget amongst all the medical activities’ (p76). Technical value is described in emphasising the need for the ‘optimum use’ of resources for not only problems of treatment but also ‘..screening, diagnosis, place of treatment and length of stay, and, if necessary, rehabilitation’ (p2) and social value in describing the ‘local pride’ that people had for their ‘cottage hospital’ and how this influenced the flow of resources (p34). Effectiveness and Efficiency therefore introduced VBHC fifty years ago! As health services around the world come to understand that they cannot close the gap between demand and resources by ensuring that only effective interventions are offered, there is a huge opportunity for the Collaboration to lead the VBHC paradigm as it led the Evidence Based Healthcare Paradigm. This does not mean doing cost-benefit analyses. It means using the experience and judgement in all the groups to promote stewardship and provide advice on the best use of resources, on value, not just on effectiveness.

11:05 AM - 11:15 AMInvolvement of children and young people in development of an evidence synthesis framework for what interventions best prevent childhood obesityPatient or healthcare consumer involvement

Background: We conducted systematic reviews and meta-analyses of over 250 randomized trials of interventions to prevent obesity in children aged 5 to 18 years. Although we found that interventions targeting diet, activity, and both combined may have small beneficial average effects on BMI, considerable heterogeneity arose from different participant and intervention characteristics. To explore how intervention characteristics, and their synergistic effect, impacted on the intervention effectiveness in preventing obesity, we aimed to reanalyse the results of the studies. To achieve this, we developed a novel analytic framework that addresses key intervention characteristics that are likely to explain differential effects. As children are the direct target of most of the interventions in our review, we decided to ask them what features they think an intervention should have to work best at achieving its goal.
Objectives: To describe the beneficial role that children and young people played in the development and implementation of our analytic framework.
Methods: A literature review informed our preliminary analytic framework, which we discussed with our project advisory group (also including two young people aged 15+). We then organised two workshops with children and young people (in one instance also with their parents) in which we discussed our framework aims and asked the participants for advice on what intervention characteristics may impact its effectiveness. The coding of the trials according to the finalised framework is being undertaken in part by young people themselves.
Results: By asking our panels of children and young people the simple question “What should we do to prevent childhood obesity?”, it emerged that interventions they thought would be best received were fun, casual, enjoyable, interesting, and interactive. Additionally, our panel suggested that the credibility of the person delivering the intervention and the opportunity to choose activities (e.g., their favourite sport) are also important features.
Conclusions: Working with children and young people in both the planning and conduct of our work has been a scientifically useful and personally stimulating experience. Patient, public, and/or healthcare consumer involvement: Children and young people were involved in the development of the analytic framework.

11:05 AM - 11:25 AMDevelopment of a health-system guidance implementation tool using a modified Delphi method and a formative evaluation approachEvidence synthesis and clinical guidelines: tools and methods

Background: Compared with clinical practice guidelines, the implementation context for health-system guidance (HSG) is more multi-layered and complicated because of the complexity of health systems, the diversity of evidence, and the highly context-sensitive and multifactorial policymaking process. Current guideline implementation frameworks and tools might not be suitable for relatively complex HSG implementation. Therefore, a comprehensive and well-organized theoretical framework to support the HSG implementation was developed.
Objectives: Building on a theoretical framework we developed, this project aims to develop an operationalized tool for supporting global HSG implementation at national or subnational levels.
Methods: First, a modified Delphi approach with two steps was applied, including the following: (1) generating items based on the theoretical framework and (2) using a modified Delphi method to select and refine items to be incorporated in the tool. Second, a formative evaluation approach with four steps was applied, including the following: (1) operationalizing the HSG implementation tool based on the Delphi-derived components (i.e., domains and individual items in each domain); (2) theoretically ‘testing’ the application of the tool with several HSG examples; (3) examining the face validity and usability through semi-structured interviews; and (4) finalizing the HSG implementation tool based on the above findings.
Results: The main output of this study is an operationalized HSG implementation tool that can support HSG implementation at the national and/or subnational level. Also, we will learn about whether, how, and why the HSG implementers will use this tool and what could be done to make this tool more usable. The results will be presented at the conference.
Conclusions: The operationalized tool will help HSG users identify potential factors that facilitate or hinder the HSG implementation process, and strategies can leverage facilitators and address barriers during their preparation for HSG implementation; on the other hand, the tool will help HSG developers integrate the implementation considerations during the development process of HSG. Patient, public, and/or healthcare consumer involvement: As the target population of HSG recommendations, the healthcare consumer representative will be invited to participate in the modified Delphi and formative evaluation processes to provide the related insights that contribute to the tool development.

11:05 AM - 11:25 AMIntegrating Machine Learning into a Systematic Review Workflow: Testing the Cochrane RCT Classifier in a Research Consultancy SettingEvidence synthesis innovations and technology

Background: There is strong evidence that machine learning can substantially reduce the burden of manual systematic review (SR) screening. However, outside of Cochrane and some academic groups, the adoption and acceptability of tools is still weak.
Objectives: To assess the accuracy of the Cochrane Randomised Controlled Trial (RCT) Classifier to accelerate systematic review screening in a research consultancy setting.
Methods: Our Review and Evidence Synthesis (RES) team are developing a semiautomated workflow (“RESbot”), which pieces together compatible tools to accelerate SR production. We have tested the Cochrane RCT Classifier as a potential candidate to include in the workflow to support screening. From December 2022 to February 2023, the classifier was tested on three SRs of RCTs covering interventions for renal denervation (Review A), postpartum depression (Review B), and schizophrenia (Review C). For each review, the search results were screened manually by two independent reviewers. The same results were run through the classifier, using both the “sensitive” and “precision” version. Classifier results were cross-checked against reviewer decisions.
Results: For Review A, the search retrieved 2,795 records. Manual screening found 24 eligible trials. Loading the search results through the classifier reduced the volume to 1,504 (sensitive) and 701 (precision). The precision set contained 23 of the included trials and the sensitive set contained all 24. For Review B, 2,153 records were retrieved by the search. Manual screening found 22 eligible trials. The classifier reduced the volume to 1,594 (sensitive) and 1,265 (precision). Both sets contained all 22 trials. For Review C, 1,172 records were retrieved by the search. Manual screening found 20 eligible trials. The classifier reduced the volume to 823 (sensitive) and 548 (precision). Both sets contained all 20 trials. The reduction in screening burden across the tests ranged from 25.9% to 46.1% using the sensitive and 41.2% to 74.8% using the precision setting.
Conclusions: The Cochrane RCT Classifier performed well in our tests, with no trials missed across the three reviews using the sensitivity setting and only one trial missed using the precision setting (in Review A). Our findings support the wider adoption of this classifier to accelerate review production.

11:05 AM - 11:25 AMMeta-analysis: what is it all for?Statistical methods and meta-analysis

Background Meta-analysis has grown increasingly complex in recent years, with a growing range of alternative and sometimes conflicting methods for combining studies, weighting studies, estimating heterogeneity, calculating confidence intervals, and so on. It is increasingly difficult to decide what methods to use, and it is difficult for stakeholders to understand the methods and be confident of their validity. Objectives To challenge the growing complexity of meta-analysis by seeking to return to the first principles for meta-analysis and considering why we perform meta-analyses at all. Concepts It will be argued that there are two key purposes for meta-analysis: 1. To summarise the evidence in identified studies of an intervention. 2. To investigate a deeper “scientific truth” about the intervention. The talk will make the case that the growing complexity in meta-analysis stems from a confusion of the two stated purposes of meta-analysis: by making the flawed assumption that a simple summary of evidence extracted from publications can tell us about the truth underlying the included studies. The talk will demonstrate that a simple “assumption-free” weighted average approach to meta-analysis is always valid if we are aiming only to summarise the available evidence. This also applies when considering heterogeneity. A simple approach has limitations, particularly with few studies or data, but shifting to more complex methods does not solve the problems. Conversely, identifying the “truth” about an intervention requires strong assumptions about the studies that are likely to be flawed. Alternatively, it requires a more complex analysis approach, with better data, such as an individual participant data meta-analysis. Conclusions We need to be clearer on what the purpose of any meta-analysis is and be honest about its limitations. Simple, conventional approaches may be best when we are restricted to summarising evidence in publications. Greater clarity will ensure that patients and stakeholders have a better understanding of what any meta-analysis is actually telling them and what it can’t tell them, without unhelpful methodological complications. This should support greater understanding of the strengths and limitations of meta-analyses. Patient, public, and/or healthcare consumer involvement: No involvement.

11:15 AM - 11:25 AMStrengthening the global evidence base on public health and social measures through a WHO research initiativeGlobal health, equity and partnerships

Background: Public health and social measures (PHSM) refer to nonpharmaceutical interventions implemented by individuals, communities and governments at all levels. During the COVID-19 pandemic, PHSM have been applied globally at an unprecedented scale and time span. PHSM have protected both lives and livelihoods, but they have also had negative health, social and economic consequences for individuals and societies, particularly for those living in vulnerable conditions. Producing robust evidence on the effectiveness and negative impacts of PHSM and understanding factors influencing adherence to measures is crucial to equitable and context-specific decisions and strengthening emergency preparedness and response. However, the complexity of PHSM and ethical, legal and feasibility limitations related to PHSM research in an emergency context, especially randomized controlled trials, have hampered systematic evaluation of PHSM.
Objectives: The World Health Organization (WHO) launched a global initiative in 2021 to build robust data and research evidence to better understand the effectiveness of PHSM and improve precision in future PHSM decisions and policies.
Methods: The initiative is taking a comprehensive approach to strengthen the evidence base on PHSM spanning from conceptual work to promoting primary research production and evidence use in policy.
Results: The initiative’s research pillar promotes the production of trustworthy and actionable evidence on PHSM through (i) a conceptual framework on PHSM fostering a common and comprehensive understanding of PHSM and their impacts, (ii) a series of global evidence syntheses on the effectiveness and impact of PHSM, factors influencing the adherence to measures as well as social protection policies for protecting livelihoods, (iii) a global research agenda for setting priorities for research production for the next decade, (iv) a suite of study protocols to collect high-quality, comparable and timely data on the effectiveness of PHSM in various settings during an emergency, and (v) a research database serving as a one-stop repository of primary studies and living systematic reviews on PHSM.
Conclusions: The WHO PHSM initiative aims to produce and catalyse evidence that is comparable, high quality, equity focused and policy relevant to support decision-makers to make evidence-informed decisions about PHSM to better protect communities during future emergencies.

11:15 AM - 11:25 AMBringing a patient perspective to understanding interventions that aim to reduce length of stay in hospital: a mixed methods synthesisPatient or healthcare consumer involvement

Background: Hospital-led, multicomponent organisational strategies that aim to enhance recovery after surgery are associated with improved clinical outcomes. Less is known about aspects of interventions or their delivery which can lead to better patient outcomes.
Objectives: To establish which aspects of multicomponent interventions to enhance recovery after surgery and reduce length of stay in hospital are associated with better outcomes for older adults admitted to hospital for planned procedures.
Methods: Working in partnership with expert clinical advisors, older adults, and carers with experience of hospital stays for planned procedures, we undertook a mixed methods synthesis. We first conducted a quantitative review (of comparative studies reporting length of stay and patient reported outcome or experience measures) and a qualitative review (of the experiences of patients, carers and staff of interventions received/delivered). We then developed a logic model based on patient perspectives represented in the qualitative review and discussions with clinical stakeholders and our patient and carer groups. The logic model was used as the focus of a Qualitative Comparative Analysis (QCA) to integrate the findings of the quantitative and qualitative reviews.
Results: A complex balance of intervention components trigger successful interventions: these represent approaches that allow patients to understand their treatment (e.g., through providing information in different formats), to ask questions about their treatment (through spending additional time with patients), and to build supportive relationships (through having a consistent point of care) and strategies that facilitate patients to monitor their own progress (goal-focused) and challenge themselves in recovery (through early mobilisation). Limitations: We did not consider the efficacy of different clinical procedures (largely unseen or unknown to most patients) that may also influence outcomes; evidence for some of the components came from weak studies; most studies screened patients with more complex needs out of the intervention; poor or absent reporting meant that we had to discard or combine some elements identified within the logic model.
Conclusions: Grounding our synthesis in the experiences of patients, carers, and clinicians enabled us to generate meaningful implications for policy and practice that go beyond measuring success in terms of reducing time spent in hospital.

11:25 AM - 11:35 AMScope, quality and reporting of clinical practice guidelines for newborn and child health in South Africa, Nigeria, and Malawi.Evidence synthesis and clinical guidelines: tools and methods

Background: Low- and middle-income countries are disproportionately affected by high rates of under-5-year-old mortality, often caused by preventable conditions. High-quality, evidence-based clinical practice guidelines (CPGs) may play a key role in improving clinical management and quality of care and, thus, in reducing child mortality. However, limited availability or poor accessibility, inadequate reporting and low-quality methodologies of available CPGs may undermine their utility and expected impact in improving quality of care and outcomes.
Objectives: To conduct a scoping review to identify CPGs for newborn and child health published in South Africa, Nigeria, and Malawi and to assess their quality.
Methods: We searched key websites (June-July 2022) for published national and subnational de novo or adapted CPGs addressing newborn and child health in the three countries. Pairs of reviewers screened for eligibility, extracted information from included CPGs (scope, condition or topic, target population and end users, developers, methods), and appraised their quality using the Appraisal of Guidelines for Research & Evaluation (AGREE II) instrument. Data were analysed descriptively in Microsoft Excel and STATA-17.
Results: We included 40 CPGs. Most provided guidance on communicable diseases (19/40), whilst eight on non-communicable diseases. The majority of CPGs did not report on methods for assessing the certainty of evidence (7/40), of which one CPG reported using GRADE. Overall, CPGs scored well on clarity of presentation (median 81%, IQR 67-94), and scored poorly on AGREE II domains rigour of development (median 11%, IQR 4-32) and editorial independence (median 6%, IQR 0-27). Similar trends in AGREE II domain scores were observed across all countries.
Conclusions: We highlight gaps in methodological and reporting quality of CPGs for newborn and child health across the three countries, which may impact our credibility of recommendations. Gaps in guidance were identified as the topics of CPGs often did not align with country-specific disease burdens. CPGs specifically addressing malnutrition, neonatal disorders, and trauma related-mental health disorders need to be considered. We further highlight a lack of regional and national CPG repositories which may impact on access to and use of CPGs. Patient, public, and/or healthcare consumer involvement: No direct patient involvement in the conduct of our research.

11:25 AM - 11:35 AMDefining Racial Health Equity (RHE): A Landscape Review of Definitions, Terminology and Related ConceptsGlobal health, equity and partnerships

Background: Vast gaps in healthcare access exist within the United States, particularly across racial and ethnic groups. Promoting health equity has become a priority of the US government and major health organizations; however, a national consensus on shared definitions of these terms does not yet exist. In this study, we will compile, evaluate, and synthesize “racial health equity” (RHE) definitions via a landscape review. Inclusion criteria will encompass systematic reviews, primary literature, narrative reviews, and healthcare guides that include RHE definitions. Defining RHE is a first step to centering and advancing racial health equity.
Objectives: To evaluate RHE terminology and definitions from key health organizations, primary articles, narrative reviews, and systematic reviews and to compile and synthesize findings to evaluate research gaps and make recommendations for future research.
Methods: We developed a three-part search strategy to encompass RHE definitions. First, definitions of RHE were compiled from major health organizations with relevance to high-income countries. Second, for systematic reviews, we used a comprehensive search strategy in MEDLINE, Psychinfo, the Cochrane Library, and Campbell systematic reviews since 2020. Third, for primary research articles and narrative reviews, we searched MEDLINE, CINAHL, Psychinfo, and the Campbell Collaboration library, with no date restriction. Both systematic search strategies were developed with a research librarian. Study title/abstracts and full texts will be screened by two reviewers and disputes will be resolved by consensus.
Results: Health equity definitions largely differed across healthcare organizations, although some definitions were shared and cited the same original source. Few major healthcare organizations reported explicit definitions of RHE, although it was commonly listed as a priority. Preliminary search results of RHE across primary research articles yielded ~2,500 articles. Search results of systematic reviews identified >7,500 citations.
Conclusions: Defining racial health equity is a first step toward establishing actionable interventions to further research and reduce health inequities. This landscape review is part of a larger research project, funded by the Robert Wood Johnson Foundation, investigating racial health equity in systematic reviews.
Patient, public and/or healthcare consumer involvement: This study and the parent project were reviewed by multiple diverse stakeholders in healthcare.

11:25 AM - 11:35 AMStakeholder involvement to enhance the relevance and accessibility of a Cochrane review of physical rehabilitation after strokePatient or healthcare consumer involvement

Background: In 2013-2014, our Cochrane systematic review relating to physical rehabilitation after stroke was produced with involvement of stroke survivors, carers and health professionals (“stakeholders”) who made key decisions about the review scope, structure and presentation of results. In 2022-2023, we conducted a major update of this review, again involving key stakeholders to ensure continued relevance and impact of this review. Objective: To describe and reflect on the stakeholder involvement in this Cochrane review of physical rehabilitation after stroke. We will:
• Describe involvement using the ACTIVE framework (who, how, when, level of involvement) and any impact/changes;
• Reflect on what did/did not work well.
Methods: We recruited a group of stakeholders, comprising people with lived experience of receiving or providing physical rehabilitation after stroke in the UK, aiming for a group with a range of different experiences. We held a series of pre-planned meetings, each with a specific goal in relation to informing the review. We held two international webinars to gain international perspectives. Stakeholders provided reflections and feedback.
Results: The stakeholder group comprised three stroke survivors, four carers and seven physiotherapists. To date, we have held four online meetings, supplemented with additional sub-group meetings, recorded presentations and email communication. Two further meetings are planned. A total of 124 people (including patients, carers, professionals and reviewers), from three continents, attended the international webinars. A complex new framework to describe physical rehabilitation following stroke was co-produced. This framework informed the grouping of studies within the review analyses. The group highlighted the need for additional comparisons and subgroup analyses to make the results meaningful and relevant. Wording of key results and implications for practice was co-produced. Barriers and facilitators to successful involvement were identified. For example, online meetings created challenges to equitable participation; use of recorded presentations were an efficient and successful way of keeping stakeholders up-to-date and informed.
Conclusions: Stakeholder involvement influenced the review, ensuring analyses focused on questions that were meaningful and relevant and that results were clearly communicated to end users of research. This example contributes to a growing body of evidence about methods of involving stakeholders in Cochrane reviews.

11:25 AM - 11:45 AMThe improved Systematic Review Data Repository Plus (SRDR+): A free, “FHIR-ed up” tool for screening, data extraction, and data sharingEvidence synthesis innovations and technology

Data Sharing Making data gathered during systematic reviews (SRs) publicly available could reduce unnecessary duplication, support analyses for secondary research questions, and facilitate methodological research. However, only a third of recent SRs have data availability statements, and among those, only 13% have downloadable data from a website/repository. What Is SRDR+? The Systematic Review Data Repository Plus (SRDR+) is a free, online platform for screening and extracting, archiving, and sharing data gathered during SRs. Since inception, it has had 10,083 user accounts from 117 countries. Data have been made publicly available (open access) for 225 SRs containing 19,898 studies on numerous topics. The US Agency for Healthcare Research and Quality (AHRQ) funds SRDR+. Key Features of SRDR+ SRDR+ is an open-source relational database that allows machine learning–informed screening, flexible data extraction and risk of bias assessment, data adjudication, and data sharing. Newly enhanced SRDR+ features include (1) a data comparison tool to help adjudicate duplicate independently extracted data and (2) a single location for handling citations for abstract screening through full-text screening, data extraction, and data export. SRDR+ can also share data that were not initially extracted into SRDR+ (i.e., data can be imported from other platforms). “FHIR-ed up” Data Sharing Fast Healthcare Interoperability Resources (FHIR) is a standard for electronic exchange of health data. The Evidence-Based Medicine on FHIR (EBMonFHIR) initiative is developing standards for sharing SR data, such as citations, study design, population/intervention/exposure/outcome definitions, results, risk of bias, and certainty of evidence. We are enabling SRDR+ to fully comply with FHIR to facilitate data sharing and allow interoperability of SR data with other platforms, including for guideline development (e.g., MAGICapp) and study registries (e.g., ClinicalTrials.gov). SRDR+ FHIR-enabled resources will be available via a representational state transfer (RESTful) application programming interface (API), which facilitates backwards compatibility, and will be human readable through the Fast Evidence Interoperability Resources (FEvIR) platform. Plan for the Colloquium Presentation We will provide an overview of SRDR+ and demonstrate progress in making SRDR+ fully FHIR enabled. Patient Involvement Although no patients were involved in this abstract, various SRDR+ projects have involved patient collaborators and contain data for patient-important outcomes.

11:25 AM - 11:45 AMReliably estimating interactions and subgroup effects in aggregate data meta-analysisStatistical methods and meta-analysis

Background: A key question for meta-analysis is to reliably assess whether treatment effects vary across different participant subgroups. Traditionally, these interactions have been estimated using approaches known to induce aggregation bias, so we previously recommended a within-trial approach to provide unbiased estimates for binary or ordered-categorical patient-level treatment-covariate interactions. However, patients, clinicians, and policymakers need reliable estimates of treatment effects within specific covariate subgroups, on relative and absolute scales, in order to target treatments appropriately, which estimation of an interaction effect itself does not provide. Objective and
Methods: Our objective is to describe further developments to the “within-trial” framework by providing practical methods to (1) estimate within-trial interactions across two or more subgroups; (2) estimate subgroup-specific (“floating”) treatment effects that are compatible with the within-trial interactions and make maximum use of available data; and (3) clearly present these data using novel implementation of forest plots. We describe the steps involved and show how the methods can be applied using detailed aggregate (“summary”) data either extracted from trial publications or obtained directly from trial authors. We demonstrate the within-trial framework by applying it to two examples taken from previously published meta-analyses in which detailed aggregate data were available.
Results: In our first example, a meta-analysis of the effects of interleukin-6 antagonists on outcomes for patients hospitalised with COVID-19, we show how the method can be used for a binary covariate. Our second example, a meta-analysis of the effects of postoperative radiotherapy on survival of patients with non-small cell lung cancer, allows us to demonstrate the framework for a categorical covariate with three levels. We demonstrate how to implement these methods using a newly developed command (metafloat) in Stata.
Conclusions: Our within-trial framework allows straightforward estimation of a range of within-trial treatment-covariate interactions, compatible subgroup-specific treatment effects, and corresponding forest plots to clearly and effectively demonstrate how treatment effects differ across patient subgroups. Patient, public, and/or healthcare consumer involvement: One patient and one member of the public are involved in the dissemination of the results of this study, including advising on how best to present the methodology to various audiences.

11:35 AM - 11:45 AMAn Evidence Ecosystem Evaluation for the Prevention and Control of Healthcare-Acquired Infections in ChinaEvidence synthesis and clinical guidelines: tools and methods

Background: The COVID-19 pandemic has created significant challenges for healthcare, especially the increase in health-acquired infections (HAIs), which has put significant pressure on the development of social and economic policy in China. At present, the number of HAIs-related research and policy reports is gradually increasing and gaining attention from different stakeholders. At the same time, the current scope of HAIs research, translation, and implementation in China is unclear and cannot further effectively support evidence-informed decision-making.
Objectives: To evaluate the current evidence ecosystem of HAIs prevention and control in China and identify key issues and challenges to improve it in the future.
Methods: We established a national and international multidisciplinary working group. There were two stages for this evaluation. First, we systematically searched seven main databases (PubMed, Web of Science [WoS], Embase, Cochrane Library, China National Knowledge Infrastructure [CNKI], Virtual Interactive Platform [VIP], and Chinese Biomedical Database [CBM]) for all HAIs-relevant documents such as clinical trials, models, reviews, guidelines, policies, and implementation systems from their individual inception to Feb 20, 2023. Reviewers working in pairs and independently selected documents according to the inclusion criteria and extract information. For included documents, according to their type, their evidence quality was assessed. Second, referring to the MAGIC (Making Grade the Irresistible Choice) evidence ecosystem, reviewers evaluated the completeness of the HAIs prevention and control ecosystem in China through the processes of evidence production, synthesis, dissemination, implementation, and evaluation.
Results: Our search identified 28,126 documents. The detailed results will be shown at the conference.
Conclusions: Relevant research and documents on HAIs are growing rapidly. However, repeated research in the same field leads to serious research waste and an incomplete evidence ecosystem. It is suggested that policymakers, stakeholders, and researchers should refer to the current evidence ecosystem before making decisions to fill the research gap.
Patient, public and/or healthcare consumer involvement: None.

11:35 AM - 11:45 AMEngaging racially and ethnically diverse stakeholders in evidence syntheses: A review of reviews and guidance documentsGlobal health, equity and partnerships

Background: The conduct of systematic reviews and other evidence syntheses often involves narrowly focused questions addressing proximal and individual behavioral causes for health outcomes, without considering that distal social determinants of health deeply influence health behaviors and health outcomes. Addressing pervasive racial health inequities, often driven by social determinants and structural racism, requires disrupting research norms by centering marginalized groups. Evidence synthesis guidance does not routinely require incorporating racial health equity across review processes, and a national consensus on how to center racial health equity in evidence syntheses is currently lacking. A key aspect of rethinking methods in the context of racial health equity is revisiting the purpose, nature, and outcomes of engaging stakeholders—including patients, clinicians, community organizations, purchasers, payers, health systems, policy makers, and training institutions—in reviews.
Objectives: (1) To systematically review and characterize current practices in engaging stakeholders in evidence syntheses of interventions to improve racial health equity and (2) to compile and synthesize extant guidance for engaging diverse stakeholders in evidence syntheses.
Methods: We searched MEDLINE and the Cochrane and Campbell Collaboration databases, relevant synthesis and stakeholder organization web sites, and reference lists of salient papers. We included evidence syntheses addressing effects of interventions that aimed to improve racial health equity or methods/guidance documents that addressed approaches for engaging diverse stakeholders in syntheses. Two reviewers independently screened citations. One reviewer will conduct data extraction and quality assessment, with verification by a second reviewer. We will conduct descriptive analysis of key characteristics related to engagement.
Results: Our database and additional searches identified more than 7,500 citations. Screening and extraction are ongoing, but preliminary signals suggest that few reviews of racial health equity interventions explicitly report involving stakeholders. Guidance relevant to stakeholder engagement typically does not directly address engaging racially or ethnically diverse stakeholders but does speak to general equity and ethics considerations.
Conclusions: This review of stakeholder engagement specific to racial health equity, part of a larger Robert Wood Johnson Foundation-funded project, will help advance our understanding of current and needed guidance for involving racially/ethnically diverse stakeholders. Patient, public, and/or healthcare consumer involvement: Protocol review by technical experts.

11:35 AM - 11:45 AMDevelopment and co-design of a behavioral activation intervention targeting depression among people with dementia for the Swedish contextPatient or healthcare consumer involvement

Background: Approximately 40% of people with dementia experience depression. Whilst psychological interventions are effective for the treatment of depression in dementia, access remains low. A behavioural activation self-help intervention, with support to the person with dementia to use the intervention provided by an informal caregiver, who themselves receives guidance from an occupational therapist (or other trained healthcare professional), may represent a solution. Objective: To develop a behavioral activation intervention targeting depression among people with dementia for the Swedish context together with key stakeholders.
Methods: Semistructured interviews and focus groups were held with people with dementia (n=8), informal caregivers (n=19), healthcare professionals (n=18), and nongovernmental organisations (n=7) and analysed using manifest content analysis. A Public Advisory Group, consisting of informal caregivers of people with dementia (n=4), worked alongside the research team to support the interpretation and sense-making of research findings and codesign of the intervention.
Results: An overarching theme, ‘Tailoring and flexibility’, resulted from interviews and focus groups. Stakeholders expressed a need to adapt intervention material to increase relevancy and representativeness by (1) adding multiple case stories to illustrate different life situations, age groups, and ethnic backgrounds and to increase relevance to Swedish society and culture; (2) designing new illustrations, as proposed illustrations were perceived as old-fashioned and reinforced ageing and dementia stereotypes; and (3) reducing text to minimize treatment burden. Stakeholders also expressed a need for flexibility concerning intervention delivery and expressed a need for choice concerning (1) location of guidance sessions to enable face-to-face sessions to be delivered in a familiar, safe and convenient environment (e.g., home or familiar community setting); (2) mode of guidance (e.g., face to face, telephone, online); and (3) amount of guidance (e.g., frequency and number of guidance sessions).
Conclusions: Results informed the development of a tailored intervention, optimised to improve acceptability, feasibility and relevancy for people with dementia and depression. A planned feasibility study will further examine feasibility and acceptability of the intervention. Public involvement: Our Public Advisory Group has worked closely together with the research team to support the interpretation and sense-making of research findings and codesign the intervention to increase acceptability and relevancy of the intervention.

11:45 AM - 11:55 AMThe development and testing of the Scientific, Transparent and Applicable Rankings tool (STAR) for clinical practice guidelinesEvidence synthesis and clinical guidelines: tools and methods

Background: Clinical practice guidelines are a critical tool for guiding physicians in clinical practice. Guidelines have been evaluated from different perspectives using various tools. However, the existing evaluation tools have several limitations. First, these tools do not address some key elements of guideline quality, such as guideline applicability, transparency of the processes and methods used for development, and prospective registration. Second, some of the evaluation tools have not been adequately assessed for reliability and validity. Third, most evaluation tools have a limited scope such as methodological quality, reporting quality, or implementation. Thus, a comprehensive evaluation of a guideline is time-consuming because it requires the use of multiple tools encompassing different dimensions. In addition, there may be an overlap of items across different evaluation tools. Fourth, interpreting the results of multiple tools in combination and comparing them across different guidelines is challenging.
Objectives: To overcome these barriers and to improve the quality of Chinese guidelines, we formed a working group to develop a unified, comprehensive, and practical evaluation tool named STAR. Methods and
Results: A scoping review was developed to formulate the initial items related to the three dimensions of scientificity, transparency, and applicability; two rounds of Delphi expert survey resulted in a total of 39 items grouped into in 11 domains. Based on a hierarchical analysis of the results of the importance survey, each domain and each item were given weights that reflect their relative importance. Finally, a consensus on the final STAR rating tool was reached in an expert consensus meeting. The tool was tested and found to be reliable, valid, and easy to use.
Conclusions: STAR may be the first tool that uses scientific approach to assign different weights to the domains or items of guideline evaluation, and it is applicable to registered guidelines that have voluntarily applied to the research center for ranking and providing relevant supporting materials.

The STAR Tool.pdf

11:45 AM - 11:55 AMEquity considerations in the conduct of evidence syntheses in partnership with patients and other stakeholdersGlobal health, equity and partnerships

Background: There is an increased recognition of the value of coproduction and working in partnership with stakeholders to produce trusted and equity-sensitive evidence syntheses. Stakeholders are individuals or groups who are impacted by health research including patients, caregivers, the public, policymakers, healthcare providers, and program managers.
Objectives: (1) To summarise the equity considerations included in existing frameworks and guidance for conducting evidence syntheses in partnership with stakeholders; (2) To map evidence on the impact of stakeholder engagement in conducting evidence syntheses across equity characteristics; and (3) To develop a logic model for equitable partnerships with stakeholders when conducting evidence syntheses.
Methods: This study was conducted as part of the ongoing work of the Multi-Stakeholder Engagement (MuSE) Consortium, an international group of over 120 members representing stakeholders, such as patients, healthcare providers, policymakers, research funders, and guideline developers from various organizations (e.g., WHO, GRADE, AHRQ, RAND, JBI). We conducted a scoping review and systematic evidence map. We searched six bibliographic databases (2010-2022) and grey literature for publications describing guidance or evaluations of stakeholder engagement during an evidence synthesis. We extracted data in duplicate and identified equity considerations using the PROGRESS+ equity framework. We engaged stakeholders in the development of our logic model and preliminary recommendations.
Results: This presentation will include preliminary recommendations for equitable partnerships with stakeholders when conducting evidence syntheses, including: (1) Identification and recruitment of diverse stakeholders, (2) Power-sharing and equitable decision-making during review conduct, (3) When and how to engage stakeholders to promote equity-sensitive syntheses, and (4) Potential impacts and outcomes of stakeholder engagement in evidence syntheses on health equity. Conclusion: The findings from this project will inform the upcoming MuSE Consortium guidance for stakeholder engagement in evidence syntheses and will be integrated into training delivered by the Cochrane Equity Thematic Group. The results will be particularly relevant and important for patients, as the guidance will be coproduced with their leadership and will promote their engagement in conducting reviews to ensure that evidence used for health decision-making includes equity considerations. Patient, public, and/or healthcare consumer involvement: The MuSE Consortium includes stakeholder coleads representing patients, the public and consumers.

11:45 AM - 11:55 AMCo-designing outcomes for a review on self-harm and suicide prevention interventions: Are we failing our young people?Patient or healthcare consumer involvement

Background: Involvement of end users in the production of reviews has mostly been limited to consultative review of researcher-authored protocols with involvement in outcome identification far less common. To address this gap, for a recent Cochrane review on suicide and self-harm prevention interventions in education settings, we collaborated with young people (YP) to codesign review outcomes that were relevant to them.
Objectives: The key objectives were to understand how our codesigned outcomes compared with outcomes that are usually included in self-harm intervention trials and the extent to which YP have been involved in the design and conduct of the trials included in the review.
Methods: Five participatory codesign workshops were completed with 32 YP to codesign outcomes and reflect on how these outcomes compared with those measured in self-harm intervention and prevention interventions. The codesigned outcomes included in the review were better for more coping skills and safer environments (i.e., more acceptance and understanding at home/school). We did not further refine these into more narrowly defined outcomes through the researcher/clinician lens but took a scoping approach and included all constructs relevant to them.
Results: YP offered notably different views about what the most important review outcomes should be, emphasizing the broad diversity of experience and recovery. Importantly, the outcomes that they codesigned highlighted significant idiosyncrasy in what YP perceive as helpful and pointed to important targets of intervention. They also highlighted the importance of moving away from clinical, deficit-based and individually focused outcomes to more holistic and strengths-based outcomes. In the 51 trials included in the review, a large range of constructs relevant to the codesigned outcomes were used. However, very few trials measured the impact of improvements in these constructs on self-harm. Involvement of YP in designing these trials varied significantly but was very limited in most studies. Conclusion: The findings clearly elucidate the lack of alignment between what recovery from self-harm means (in terms of outcomes) for young people between YPs and researchers. Thus, involvement of YP in the ‘upstream’ stage of research is essential to ensure that review outcomes measure what is actually relevant to them.

11:45 AM - 12:05 PMBuilding acceptance for machine learning in study selection within a systematic review institution: Experiences from the Norwegian Institute of Public HealthEvidence synthesis innovations and technology

Background: The Norwegian Institute of Public Health’s (NIPH) machine learning (ML) implementation team within the Cluster for Reviews and Health Technology Assessments sprung from the need to produce more health technology assessments and systematic reviews, faster, during the COVID-19 pandemic. We introduced and scaled up ML among nonspecialists through using off-the-shelf software, beginning with a 6-month team whose mandate was to explore potential benefits of ML. We then continued with a team to build acceptance and use of the tools through teaching and peer support.
Objectives: To present our experiences with starting up and implementing an ML team for evidence synthesis. More specifically, we will present key elements of our team’s strategy, including: - Embedding process and performance evaluations into existing commissioned reviews (i.e., how we “tested” ML functions within ongoing work) - Creating tailored training materials and providing peer support - Onboarding new team members and keeping up to date - Managing workflow changes - Fostering acceptance and implementation of ML
Conclusions: We will conclude by discussing lessons learned and the role of a nonspecialist, implementation-focused ML network or working group in providing training to evidence synthesis organizations.

11:45 AM - 12:05 PMLikelihood Ratio Meta-AnalysisStatistical methods and meta-analysis

Background: A CI in an updated meta-analysis may not have the expected coverage if the investigator does not account for whether the earlier meta-analysis failed to reject the null hypothesis.
Objectives: To discuss the method of likelihood ratio meta-analysis (LRMA) in relation to Cochrane’s usual method of meta-analysis.
Methods: A likelihood ratio–based approach is used in a meta-analysis to pool data from separate studies to quantitatively assess where the total evidence points. A log-likelihood ratio function is used for the measure of association in each study. Those functions are summed to obtain a combined function from which a total effect estimate, and ‘intrinsic’ confidence interval, can be retrieved. Results can be presented in a familiar forest plot format. An LRMA is demonstrated using an example of the influence of a concomitant conventional synthetic disease-modifying antirheumatic drug (csDMARD) on adherence to biologic DMARDs (bDMARD) in rheumatoid arthritis. The example includes population-based cohort studies of adult patients with rheumatoid arthritis between January 2007 and March 2014 in five Canadian provinces, and the US IBM MarketScan database. The outcome of discontinuation of bDMARD therapy is compared between patients who are concomitant versus non-concomitant users of csDMARDs.
Results: The study population comprises 20 221 new users of bDMARDs: those using adalimumab (7609), etanercept (9809), abatacept (1024) or infliximab (1779). Overall, concomitant use of csDMARD therapy is not statistically significantly associated with reduced discontinuation of bDMARD treatment in a random effects LRMA in these patients (hazard ratio=0.90, 95% intrinsic CI=0.79-1.02). The association is statistically significant when using usual random effects meta-analysis (95% CI=0.81-0.99). In the hypothetical scenario where the IBM MarketScan data were added after the original analysis, the 95% intrinsic CI remains unchanged at 0.79-1.02, but the 95% CI becomes uninterpretable.
Conclusions: LRMA yields the same point estimate as a usual meta-analysis but with a 95% intrinsic CI that is wider than the traditional 95% CI. The intrinsic CI is more readily interpretable. Further, with LRMA, there is no need to account for previous statistical significance in an updated analysis.
Patient, public and/or healthcare consumer involvement: None.

11:55 AM - 12:05 PMA real-world example demonstrating the application of Cochrane guidance to ensure transparent synthesis of evidence addressing a broad policy questionEvidence synthesis and clinical guidelines: tools and methods

Background: Specifying questions and criteria at the level of each synthesis can enhance the transparency of reviews and help ensure that reviews address questions of importance to decision-makers. Version 6 of the Cochrane Handbook for Systematic Reviews of Interventions introduced the concept of ‘PICO for each synthesis’ to bring greater focus on the need to plan and report details of the synthesis questions addressed within a systematic review. The results of a systematic review are ultimately determined by these synthesis questions and the decisions authors take in deciding which studies are eligible to answer each question. Without changing the review eligibility criteria, a synthesis can be structured to address different questions (e.g., broader or narrower) simply by grouping interventions, outcomes, or populations differently. Yet this level of specification is uncommon and can be challenging in broad systematic reviews such as those commissioned to address policy questions.
Objectives: To describe our approach to question specification in a broad systematic review commissioned to inform government decisions about health insurance rebates, including:
• Use of an analytic framework and staged consultation to pre-specify a proposed structure for synthesis, agree on the final PICO for each synthesis, and triage studies for analysis.
• The benefits of this approach for decision-making, workflow, and relevance of the synthesis.
Methods: This was 1 of 16 reviews each examining the effects of a natural therapy in any population (condition, injury, risk factor) on any health outcome. Following guidance in Chapter 3 of the Cochrane handbook, we prespecified a framework with proposed population and outcome groupings for synthesis (https://doi.org/10.1186/s13643-022-02015-1). We compiled an aggregate list of populations and outcomes from included studies organised by the framework. Populations and outcomes were prioritised by the commissioners, enabling us to finalise the synthesis PICOs and triage studies for meta-analysis.
Results: The synthesis examined effects on six outcomes (pain, fatigue, sleep, mental health, quality of life, physical function) from 234 studies categorised into 20 population groups. For patients, the approach underpins transparent synthesis of evidence commissioned to inform decisions that affect their health. Patient, public, and/or healthcare consumer involvement: through the commissioner (www.nhmrc.gov.au/health-advice/all-topics/complementary-medicines/natural-therapies-review).

11:55 AM - 12:05 PMImplications of centering racial health equity in systematic reviews: results of qualitative interviews with stakeholdersGlobal health, equity and partnerships

Background: Evidence synthesis guidance does not routinely require incorporating racial health equity across synthesis processes, and a national consensus on how to center racial health equity in evidence syntheses is currently lacking. Diverse stakeholder voices—from patients, clinicians, community organizations, researchers including systematic reviewers, purchasers, payers, research funders, health systems, policymakers, and training institutions—can inform new guidance on the processes, methods, analyses, and dissemination of systematic reviews that center racial health equity.
Objectives: To obtain diverse stakeholder perspectives on (1) definitions of racial health equity; (2) optimal scope, team composition, methods, interventions, dissemination, and translation approaches that facilitate centering racial health equity in systematic reviews; and (3) measures of success and priority areas for centering racial health equity in systematic reviews.
Methods: We will conduct 25 to 35 one-on-one interviews with stakeholders representing varied perspectives. We will use both deductive and inductive approaches to qualitative analysis linking codes to our research questions and formulating cross-cutting themes across multiple interviews and stakeholder groups.
Results: Preliminary results point to key nuances in how racial health equity is defined. Stakeholders support centering racial health equity in systematic reviews, particularly when wide racial disparities are documented. While reviews should continue to focus on effectiveness of interventions, stakeholders also point to the importance of systematically assessing the context and implications of disparities in etiology, access to care, and uptake of interventions. Stakeholders suggest broadening evidence inclusion to qualitative data to have a mixed synthesis review process that provides additional context to racial health inequities. Stakeholders also suggested expanding systematic review methodology to require analytic frameworks to frame questions in ways that consider racial and ethnic centricity. In terms of stakeholder involvement, stakeholders emphasized including those with lived experience in review teams from the beginning of a review and compensating them for their contributions.
Conclusions: This qualitative exploration of stakeholder perspectives, funded by the Robert Wood Johnson Foundation, will help contextualize our understanding of current practices, gaps in methodological guidance, and future research directions for centering racial health equity.
Patient, public and/or healthcare consumer involvement: Included among interviewed stakeholders.

11:55 AM - 12:05 PMInvolving people in determining outcomes for a proposed Health Technology Assessment surrounding pelvic organ prolapsePatient or healthcare consumer involvement

Background: Patient and public involvement and engagement (PPIE) is an expectation when conducting research, including Health Technology Assessments (HTA), and is increasingly expected as a key component of evidence syntheses. When determining outcome measures for a proposed HTA evaluating nonsurgical interventions for treating pelvic organ prolapse (POP) in women, we wanted to ensure our research was of most relevance to women with the condition. Here, we describe the methods we used to collaborate with women with POP.
Objectives: To describe the PPIE methods that informed the scope of our evidence synthesis HTA proposal.
Methods: In November 2021, we held a workshop with women with POP to discuss our research question, its relevance to women with the condition, and what interventions and outcome measures they would like to see included in our evidence synthesis. Following this, we designed a survey using Qualtrics, which was distributed via PPIE networks and patient groups in the UK in May 2022. From this, we derived an initial ranking of outcomes and time-points. We brought these results to an online workshop with women with POP hosted on Zoom in December 2022. Here, we gave a background to the proposed project and used Mentimeter to derive a final ranking of outcome measures and follow-up times of most importance to attendees.
Results: Following our process, four outcome measures of most relevance to women with POP were chosen for the proposed project. Long-term follow-up (at 12 months or more) was identified as the most important time-point at which to assess change in these outcomes. Our discussions within the workshops also identified other areas through which the proposed project could be made more relevant to women with POP.
Conclusions: The involvement of women with POP in our proposal has greatly enhanced the relevancy of our research to patients and the public. People undertaking Cochrane reviews may wish to consider adopting or adapting this approach to help ensure their research is of most relevance to their population of interest. Patient, public, and/or healthcare consumer involvement: PPIE was conducted throughout our work.

12:05 PM - 12:15 PMTools and methods for assessing the transferability of health technology assessment results across jurisdictions: a systematic reviewEvidence synthesis and clinical guidelines: tools and methods

Background: There has been a growing interest in using Health Technology Assessment (HTA) as a tool for supporting decision making in health policies. However, the complexity of HTA methods and a lack of local expertise have limited its usage in many countries. The World Health Organization has taken measures in recent years to encourage countries to conduct and use HTA, including through resolutions from the Eastern Mediterranean (EM) Region Regional Committee, which in 2019 called for increased investment in strengthening national institutional capacities for evidence-informed policy making through the use of HTAs. Given the limitations in national technical capacities, there is a focus on adapting HTA results from other settings to the national context of countries.
Objectives: To systematically review the tools developed for HTA transferability and evaluate their strengths and limitations.
Methods: The systematic review was conducted between 1995 and 2021 and included studies that introduced tools, methods, and frameworks for transferability of HTA results across jurisdictions. Databases, such as MEDLINE, EMBASE, Cochrane Library, Epistemonikos, Web of Sciences, EconLit, Economic Working Papers Database, Health Economic Evaluation Database, the NHS Economic Evaluation Database, Scopus, and Google Scholar, were searched, along with relevant bibliographies. The data were extracted and assessed by at least two reviewers and synthesized using both tabulation and narrative approaches.
Results: A total of 10,375 documents were evaluated, resulting in 17 studies that met the inclusion criteria. These 17 studies consisted of 13 newly developed tools (methods) that were appraised based on 12 critical elements, including ease of use, rapid screening criteria, and factors affecting transferability. The majority of the models were checklists, with only a few deemed suitable for full HTA. Three models have been validated through published studies, but there is no evidence of utilization in the countries of the EM region.
Conclusions: Although the existing tools provide valuable resources for evaluating transferability, there remains a need for a more comprehensive tool to support decision-makers in low-resource settings considering country context and capacity. Patient, public, and/or healthcare consumer involvement: not applicable.

Abstract on sys rev on HTA transferability tool WHO EMRO. docx.pdf

12:05 PM - 12:15 PMChatGPT and large language models for systematic review tasks: What are the opportunities for improvement?Evidence synthesis innovations and technology

Background: The advancement of Artificial Intelligence (AI) assisted technologies leave many wondering about the current capabilities, limitations, and opportunities for integration into scientific endeavors. Large language models (LLM) – such as ChatGPT, designed by OpenAI – have recently gained widespread attention with their ability to respond to various prompts in a natural-sounding way. Systematic reviews (SRs) utilize secondary data and often require many months and tens to hundreds of thousands of dollars to complete, making them attractive grounds for developing AI-assistive technologies.
Objectives: To evaluate the responses of ChatGPT to SR tasks to assess applicability, correctness, and potential usefulness to reviewers.
Methods: We gave ChatGPT a set of SR tasks during a live webinar attended by over 400 people and evaluated the responses, considering our own expertise and attendees’ comments.
Results: When tasked with developing an SR question and eligibility criteria, ChatGPT appropriately formulated a structured question based on the requested PICO elements and produced criteria related to PICO, study design, language, and publication dates. ChatGPT failed to write a PubMed search strategy for a given SR question: while the output matched the structural components of a search, it was unusable due to fabricated pseudo-MeSH terms and inappropriate and incorrect filters. When tasked with screening 27 titles for a given question, ChatGPT correctly identified 8/10 eligible articles while also including one ineligible article. When tasked with summarizing multiple abstracts, ChatGPT accurately identified relevant results from 2/3, but misidentified background information from the third as results. In all tasks, ChatGPT’s response required interpretation and expertise to assess validity. Many attendees expressed hesitancy and distrust of ChatGPT for completing SR tasks.
Conclusions: While ChatGPT and LLMs show some promise for aiding in SR tasks, the technology is in its infancy and needs development for such applications. In addition, great caution should be taken by all, particularly non-content experts, to critically evaluate any content produced by LLMs due to much of the output appearing, at a high level, to be valid, while some was erroneous and in need of active vetting. Consumer involvement: The webinar was public; anyone could attend and comment.

12:05 PM - 12:15 PMFrom Agenda to Action: Collaborative Evidence Networks and the United Nations Sustainable Development GoalsGlobal health, equity and partnerships

Background: The United Nations Sustainable Development Goals were adopted in 2015 to replace the Millennium Development Goals. As we approach the midway point towards the 2030 target date, it is timely to reflect on the contribution of those in the evidence synthesis community to achieving these important global objectives. The Global Commission on Evidence concluded that both coverage and quality of evidence syntheses was uneven, that many of the reviews were out of date, and many did not include studies from low- or middle-income countries.
Objectives: This study sought to determine alignment of published JBI systematic and scoping reviews to the targets of Sustainable Development Goal 3 (Global Health and Well-being) and to understand author awareness of the goals and their relationship to their work.
Methods: A desktop audit of systematic and scoping reviews published in the journal JBI Evidence Synthesis was undertaken followed by a short author survey.
Results: The results of this pilot study indicate that the SDGs are not currently a significant consideration for most authors, but there is a desire to learn more and integrate them into their prioritisation processes. Fundamental to maximising the potential of evidence syntheses and the achievement of the targets set out in the SDGs (particularly SDG-3) are issues relating to policy, partnership, and prioritisation. The results of this pilot clearly highlight the need for the global evidence community to work together in partnership to ensure that policy and practice relevant reviews are being prioritised.
Conclusions: Although the SDGs are not the only driver to influence the prioritisation of evidence syntheses, they are an important consideration and part of our collective social responsibility. Cooperation and coordination of effort to prioritise evidence syntheses would assist in avoiding research waste, duplication of effort, and the conduct of irrelevant reviews. It is time to think about resetting and recommitting our mutual agendas to transform the evidence ecosystem, maximise the limited resources available to us, and truly make a global impact on health outcomes. Patient, public, and/or healthcare consumer involvement: N/A.

12:05 PM - 12:15 PMImportance of consensus methods and involving end users to develop evidence-based materials when evidence is scarce: a Red Cross examplePatient or healthcare consumer involvement

Background: Up to 20% of children and adolescents worldwide experience mental health problems. Because young people find it easier to talk to their peers than to adult family members, teachers, or professionals, training them in how they can support each other is very important.
Objectives: The project aimed to develop materials to train children and youth in how to support their peers in the context of mental health problems. Because (very) low certainty evidence about the association between peer support and mental health problems was anticipated, the secondary objective was to develop a formal consensus method to collect good practice input from a broad group of stakeholders.
Methods: We systematically collected evidence about mental health risk or protective factors related to how children/adolescents interact with their peers (Q1), and about educational programs aimed at social skills and mental health (problems) (Q2). Next, a modified Delphi consensus method was developed and statements describing the potential content and characteristics of interventions, based on the identified evidence, were sent out to 23 stakeholders, including content experts and mental health peer workers. All statements for which consensus could not be reached were discussed in two face-to-face meetings.
Results: For Q1, we included 160 observational studies with very low certainty evidence about factors, such as peer support, peer influence, peer rejection/acceptance, friendship quality/intensity, and bullying. For Q2, we included 9 systematic reviews resulting in evidence conclusions of moderate to low certainty on the effectiveness of peer support programs. This resulted in 146 Delphi statements; for 77% of the statements, consensus was reached after a first Delphi round, the face-to-face discussions, and a second Delphi round. Forty-one statements were not withheld. The accepted statements were used to formulate the content and characteristics of peer support interventions.
Conclusions: Because the evidence about the content for our peer support program was of very low certainty, the input of various stakeholders, from academic experts to peer workers, was indispensable. To receive input from all profiles (including end users, to assure relevance for practice) and to obtain formal consensus, we successfully used a modified Delphi consensus method.

12:05 PM - 12:15 PMTowards More Scientific Meta-AnalysesStatistical methods and meta-analysis

Background: Meta-analysis typically estimates a quantity that differs from the implicitly intended estimand; typically, standard approaches estimate the average effect of a treatment for a population of imperfect studies, rather than the true scientific effect that would be measured in a population of hypothetical perfect studies. We advocate for an alternative approach, called response-surface meta-analysis, which models the relationship between study design quality and effect size in order to estimate the effect in the hypothetical ideal study.
Objectives: We demonstrate how we can extend meta-regression to perform response-surface meta-analysis, as well as how resulting estimates can differ from estimates obtained by traditional methods.
Methods: We perform a response-surface meta-analysis by performing a meta-regression with design quality as a covariate and reporting the predicted mean and confidence interval at the ideal design quality. We apply this approach to a simulation study with a known true effect, as well as an empirical example published in the Cochrane Library. Using a univariate measure of design quality in the synthetic example and risk of bias as a proxy for quality in the empirical example, we compare estimates from response-surface meta-analysis with those of traditional approaches.
Results: In the simulation study, response-surface meta-analysis usually covers the true effect within one standard error. In contrast, we obtain overly biased results from traditional fixed-effects and random effects meta-analyses. In the empirical study, although the original analysis favors the intervention, response-surface meta-analysis does not provide strong evidence for the superiority of the intervention.
Conclusions: Response-surface meta-analysis reframes meta-analysis as an endeavor to estimate the true scientific effect that would be measured under a perfect study, rather than to summarize the existing population of imperfect studies. More work is needed to standardize the response-surface approach, such as determining a rating system for design quality. We hope this work will help to encourage clarity on the causal estimand of interest in meta-analysis. Patient, public, and/or healthcare consumer involvement: N/A.

12:15 PM - 12:25 PMThe regulatory process for drug approval needs independent meta-analytical evidence: is Cochrane a possible candidate? The case of psychotropic medicinesEvidence synthesis and clinical guidelines: tools and methods

Background: The U.S. Food and Drug Administration (FDA) and the European Medicines Agency (EMA) are responsible for scientific evaluation and approval of medicines. Approval of new drugs is based on the evaluation of individual clinical studies conducted by manufacturers who produce new compounds. Concerns have been raised regarding the rules governing approval of medicines, quality and transparency of evaluations.
Objectives: To critically review regulations for drug approvals in Europe and the U.S. and to ascertain whether the use of meta-analytical approaches may inform regulatory decisions. Mental disorders are used as a paradigmatic example in view of several medicines already available for these conditions.
Methods: We reviewed EMA and FDA regulations for approval of new drugs for mental disorders. Additionally, as case examples, regulatory trials for three recently approved drugs for mental disorders (esketamine, paliperidone 3-months long-acting and cariprazine) will be meta-analyzed to summarize evidence available at the time of approval.
Results: FDA and EMA require at least two “positive” trials showing superiority over placebo, or noninferiority over active comparators, to grant marketing authorization for new psychotropic medicines. The use of meta-analytical techniques is not a requirement. We will present the results of meta-analyses of regulatory trials of esketamine, paliperidone 3-months long-acting and cariprazine and of network meta-analyses including these medicines. Such examples will show that pooling available evidence for efficacy and tolerability outcomes may offer an added dimension to the evaluation of new medicines, especially for comparing new medicines with those already on the market.
Conclusions: Evaluation of new medicines should be complemented by regulatory meta-analyses of all relevant clinical studies performed at the time of approval, including network meta-analysis if feasible, and individual participant data (IPD). This would be especially relevant in areas where several medicines already exist, as it would inform regulatory authorities on comparative efficacy and acceptability of new drugs compared with those already available. Regulatory meta-analyses should ideally be conducted by independent organizations such as Cochrane. Cochrane may work in consultation with regulatory authorities in developing transparent and high-quality overviews of existing evidence to complement data from clinical studies during the regulatory approval process. Patient/public/healthcare-consumer involvement: Results will improve quality of care.

12:15 PM - 12:25 PMWhat is the level of expertise of ChatGPT in the domain of systematic reviews and meta-analysis?Evidence synthesis innovations and technology

Background: ChatGPT is an artificial intelligence chatbot developed by OpenAI and launched in November 2022. Although it has demonstrated a strong understanding and cognitive ability in numerous fields, its comprehension of topics related to systematic review and meta-analysis is currently not well-understood.
Objectives: To determine ChatGPT’s level of understanding of knowledge related to systematic review and meta-analysis.
Methods: We learned the methods for intervention systematic reviews and meta-analyses systematically from the Cochrane Handbook. Then, the core research team members discussed and identified 10 questions related to systematic reviews and meta-analyses and obtained relevant answers by asking ChatGPT. Three investigators independently assessed the accuracy of ChatGPT’s responses and rated them on a scale of 1-10, with a score of 10 indicating a perfect answer. Finally, the average accuracy of the three researchers for each question was calculated. The primary outcome measure was the total score of the 10 questions, which was calculated as (the mean score of question 1 + the mean score of question 2 + ... + the mean score of question 10) / 100 x 100%.
Results: Ten questions and answers related to systematic review and meta-analysis were generated (Figure). The three researchers scored the 10 questions 91, 86, and 86, respectively. The average score was 87.7. The highest average score was 9.7 for Q4, indicating that ChatGPT provided the most accurate response to that question. The lowest average score was 7.3 for Q9, indicating that ChatGPT’s response to that question seemed to deviate from the truth to a greater extent.
Conclusions: ChatGPT has a relatively high level of understanding of systematic reviews and meta-analysis knowledge, which is limited by the level of knowledge of the respondents, and a study with a large sample is needed to further validate the results. Patient, public, and/or healthcare consumer involvement: None.

Chatgpt Q & A.pdf

12:15 PM - 12:25 PMAnalysis of the proportion, role, function, and reporting quality of patient and public involvement (PPI) in systematic reviews and meta-analyses: A cross-sectional studyPatient or healthcare consumer involvement

Background: Patient and public involvement (PPI) is essential to guarantee that systematic reviews are pertinent and significant to those impacted by a health condition and to those using systematic reviews to guide health policy or practice. Therefore, the BMJ issued a new policy requiring that all submitted manuscripts must include a PPI declaration in their Methods section under a subheading called Patient Involvement in 2014.
Objectives: This study aimed to systematically analyze the proportion, role, function, and reporting quality of PPI in systematic reviews and meta-analyses.
Methods: We will conduct a hand search for all systematic reviews and meta-analyses published in BMJ journals from 2015 to 2022. Two investigators will independently extract data from the “Patient Involvement” section of the systematic reviews. Our analysis will include the proportion, quantity, and role of PPI in systematic reviews. We will also assess, based on the 12 stages proposed by the Cochrane Involving People learning resource, the specific function of PPI in developing questions, planning methods, writing and publishing protocols, running searches, selecting studies, collecting data, assessing risk of bias, analyzing data, interpreting findings, writing reviews and publishing, and disseminating them. Lastly, using the ACTIVE framework and GRIPP2 checklist, we will assess the reporting quality of the PPI in