Jisang Ahn
Pronouns: he/him
Research Mentor(s): Alanson Sample
Research Mentor School/College/Department: Computer Science and Engineering / Engineering
Program:
Authors: Jisang Ahn, April Feng, Alanson Sample, Yasha Iravantchi
Session: Session 1: 9:00 am – 9:50 am
Poster: 25
Abstract
High-bandwidth acoustic systems have a wide range of applications across various fields due to their ability to capture inaudible sounds beyond human hearing, which offers more robust and accurate human activity recognition. However, implementing user-friendly voice-activated command recognition is particularly challenging on low-power systems (e.g., Raspberry/Orange Pi devices) when paired with high-bandwidth systems because currently available open-source libraries for wakeword detection (e.g., “Hey Siri”) and voice transcription have been trained exclusively on low-bandwidth data while also being computationally expensive. As such, we have developed a custom methodology for voice-activated command recognition tailored to high-bandwidth, low-power devices. In particular, we describe in this study how we implemented real-time wake word detection and voice transcription mechanisms for PUFFIN, a high-bandwidth microphone system built to monitor daily activities in the homes of people with Multiple Sclerosis (MS) using privacy-preserving feature extraction and embedded machine learning techniques. Leveraging a condensed version of OpenAI’s Whisper model, the system successfully detects designated wakewords by continuously scanning a portion of the high-bandwidth acoustic signal. Ultimately, this process allows users to interact with PUFFIN through spoken commands, facilitating the creation of a dataset for training PUFFIN’s ML model to ultimately detect daily patient activities and forecast health conditions.
