April Feng
Pronouns: she/her
Research Mentor(s): Alanson Sample
Research Mentor School/College/Department: Computer Science and Engineering / Engineering
Program:
Authors: Yasha Iravantchi, April Feng, Jisang Ahn, Alanson Sample
Session: Session 1: 9:00 am – 9:50 am
Poster: 25
Abstract
High frequency audio recording systems have a wide range of applications across various fields due to their ability to capture sound at frequencies beyond human hearing, such as in medical diagnosis or patient care. However, implementing user-friendly voice-activated command recognition is particularly challenging in low compute-power systems (such as Raspberry or Orange Pi devices) equipped with high-frequency microphones, because most presently implemented open source libraries for wakeword detection and voice transcription have been trained exclusively on audible-frequency data. As such, we have developed a custom methodology for voice-activated command recognition tailored to high-frequency, low compute-power devices. In particular, we describe in this study how we implemented wake word detection and voice transcription mechanisms for PUFFIN, a high frequency microphone system built to monitor daily activities in the homes of Multiple Sclerosis (MS) patients using privacy-preserving feature extraction and embedded machine learning techniques. Leveraging OpenAI’s Whisper model, the system successfully detects and designated wakeword by continuously and periodically scanning and searching fixed intervals of audio. Ultimately, this process allows users to interact with PUFFIN through spoken commands, facilitating the creation of a dataset for training PUFFIN’s ML model to eventually automatically detect daily patient activities.
