Alexander Chumak
Research Mentor: Bing Ye
Mentor Department: Life Sciences Institute, University of Michigan, Medicine
Author(s): Alexander Chumak, Bobby Tomlinson, Bing Ye
Session: Session 1 (9:00 AM – 9:50 AM)
Presentation Type: Poster 132
Abstract
Machine learning models are increasingly utilized to categorize and analyze complex animal behaviors; however, evaluating the performance of these models often requires significant technical literacy. The behavior analysis software LabGym computes detailed evaluation metrics to test models following training. These include per-class precision, recall, F1-scores, and confusion matrices. Currently, these metrics are output as raw terminal text. This makes it difficult for researchers to visually identify underperforming behavior classes, detect systematic misclassifications, and investigate model weaknesses without tedious manual inspection. To enhance transparency in behavior analysis model inferences, we developed an Automated Diagnostics module in the LabGym GUI as outlined below. To overcome these interpretation barriers, an effective diagnostic tool must present evaluation metrics structurally and connect them directly to the underlying visual evidence. To achieve this, we developed a fully interactive evaluation dialog featuring a sortable per-class performance table and a visual confusion matrix with a normalized/raw data toggle. Within the Python-based codebase, this was implemented by creating a cross-platform file-tracking architecture that maps backend evaluation arrays directly to the original test dataset. This allows the confusion matrix cells to be fully interactive; clicking a specific misclassification cell immediately loads the exact ground-truth video files responsible for the error. The system automatically highlights behavior inaccuracies falling below an established F1-score threshold. This level of specificity and interactivity provides non-technical users with clear and intuitive pathways to interpreting and improving performance to their trained deep neural network models. By eliminating the friction between statistical evaluation and visual confirmation, this interactive diagnostic tool empowers non-technical researchers to rapidly audit model performance and identify whether errors stem from data imbalance, labeling inaccuracies, or algorithmic blind spots. Future work will expand on this foundation by integrating Natural Language Processing (NLP) to generate text-based summaries of the lowest-performing classes, as well as Interactive Ethograms that allow users to jump to specific timestamped errors directly from the behavior timeline.
