Andrew Gabriel
Pronouns:
Research Mentor(s): Deanna Marriott
Research Mentor School/College/Department: / Nursing
Program:
Authors: Andrew Gabriel, Deanna Marriott
Session: Session 5: 2:40 pm – 3:30 pm
Poster: 4
Abstract
Background: Performing meta-analysis of published scientific literature is a long and time consuming process. This project has the goal of assessing how AI performs when taking on such tasks, and to determine where it went wrong and how it can be improved. Methods: Papers were selected based on the following criterion: they had a sample size of at least 20 individuals with diabetes melitus, who underwent bariatric surgery (either Sleeve Gastrectomy or Roux-en-Y Gastric Bypass), and the papers had to be published after 2005. Data extraction by team members was then compared to the same data extracted by Microsoft co-Pilot AI. We extracted 14 items focused on diabetes remission as a primary outcome. Team members provided information for each question about how the AI performed along with any notes or references. Results: Microsoft co-Pilot AI had a fair level of success when answering questions related to descriptions of the study such as what type of study it is (e.g., Retrospective Cohort, Prospective Cohort, Clinical Cohort), primary outcomes (e.g.Is Diabetes remission a primary or secondary outcome?), and geographic location of the study. It was inconsistent and unreliable in producing quantitative data such as producing tables for demographics and remission/relapse data. Conclusions: AI is not yet ready to be considered a reliable tool for performing meta analysis.
