Robust Multimodal Cough Detection with Optimized Out-of-Distribution Detection for Wearables

#machine #learning #cough #sensing #multimodal #IMU

Longitudinal and continuous monitoring of cough is crucial for early and accurate diagnosis of respiratory diseases. Recent developments in wearables provides at-home remote symptom monitoring with respect to more accurate and less frequent assessment in the clinics, but face practical challenges such as speech privacy, poor audio quality and background noise in uncontrolled real-world settings. Our work addresses these challenges by developing and optimizing a multimodal cough detection system, enhanced with an Out-of-Distribution (OOD) detection algorithm. The cough sensing modalities include audio and Inertial Measurement Unit (IMU) signals. The system is optimized through training with an enhanced dataset and a weighted multi-loss approach for in-distribution classification, while OOD detection is improved by reconstructing training data components. Experiments demonstrate robustness across window sizes from 1–5 seconds and effectiveness at low audio sampling rates, where privacy is preserved. The optimized system achieves 90.1% accuracy at 16 kHz and 87.3% at 750 Hz, even with half the inference data being OOD. Most misclassifications arise from nonverbal sounds (e.g., sneezes, groans). Overall, the proposed Audio-IMU multimodal model with OOD detection significantly improves cough detection performance and offers a practical solution for real-world wearable applications. Wearable devices with on-board neural acceleration capabilities have been developed to enable fusion of air and bone microphones, and inertial measurements together with real-time processing.

11:50 am to 12:00 pm: Social

12:00 pm to 12:45 pm: Talk

12:45 pm to 1:00 pm: Q/A