Brandon Shaw
Pronouns: He/Him/His
Research Mentor(s): Stefan Larson
Co-Presenter: Maslia, Daniel
Research Mentor School/College/Department: SkySync / NonUM
Presentation Date: April 20
Presentation Type: Poster
Session: Session 5 – 3:40pm – 4:30 pm
Room: League Ballroom
Authors: Brandon Shaw, Daniel Maslia, Stefan Larson
Presenter: 79
Abstract
Machine learning systems now have the ability to quickly categorize terabytes of images, which is valuable for organizations who wish to categorize their data or find sensitive “needle-in- the-haystack†data. While machine learning model performance may be high on in-distribution data, models tend to perform worse on unknown inputs or on data that is out-of-distribution. This project analyzes a trained object detection model on a large (1 million images) test dataset in order to benchmark the model’s false positive rate. We find that the model is prone to making false-positive predictions, sometimes with very high confidence. Due to this we determined the overall precision of the model to be quite low. We evaluate the model mis-classifications qualitatively, and find that the model often makes errors on object categories that appear visually similar to the target object categories. Knowing which objects fail the most, and when, gives us insight into how we can update the model by re-training it with previously mis-classified images.
Engineering
