The pursuit of smooth, hands-free interaction in the rapidly developing field of wearable technology has produced ground-breaking discoveries. TongueTap, a technology that synchronizes multiple data streams to enable tongue gesture recognition for controlling head-worn devices, is a promising development. This method allows users to interact silently, without using their hands or eyes, and without needing specially made interfaces that are typically placed inside or close to the mouth.
In collaboration with Microsoft Research, Redmond, Washington, USA, researchers at Georgia Institute of Technology have created a tongue gesture interface (TongueTap) by combining sensors in two commercial of-the-shelf headsets. Both headsets contained IMUs and photoplethysmography (PPG) sensors. One of the headsets includes EEG (electroencephalography), eye tracking, and head tracking sensors. The data from the two headsets, Muse 2 and Reverb G2 OE devices, was synchronized using the Lab Streaming Layer (LSL), a system for time synchronization commonly used for multimodal brain-computer interfaces.
The team pre-processed the pipeline using a 128Hz low-pass filter using SciPy and Independent Component Analysis (ICA) on the EEG signals while applying Principal Component Analysis (PCA) to the other sensors, each sensor separately from the others. For gesture recognition, they used a Support Vector Machine (SVM) in Scikit-Learn using a radial basis function (RBF) kernel with hyperparameters C=100 and gamma=1 to do binary classification and determine whether a moving window of data contained a gesture or if it was a non-gesture.
They collected a large dataset for evaluating tongue gesture recognition with the help of 16 participants. The most interesting result from the study was which sensors were most effective at classifying tongue gestures. The IMU on the Muse was the most effective sensor, achieving 80% alone. Multimodal combinations, including the Muse IMU, were even more efficient, with a variety of PPG sensors achieving 94% accuracy.
Based on the sensors with the best accuracy, It was observed that the IMU behind the ear is a low-cost method of detecting tongue gestures with a position allowing it to be combined with past mouth-sensing approaches. Another critical step for making tongue gestures viable for products is a reliable, user-independent classification model. A more ecologically valid study design with multiple sessions and mobility between environments is necessary for the gestures to translate to more realistic environments.
A big step forward in the direction of smooth and intuitive wearable device interaction is represented by TongueTap. Its capacity to identify and categorize tongue gestures using commercially available technology paves the way for a time when discrete, accurate, and user-friendly head-worn device control is conceivable. The most promising application for tongue interactions is in controlling AR interfaces. The Researchers plan to study this multi-organ interaction further by experimenting with its use in AR headsets and comparing it to other gaze-based interactions.
Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to join our 33k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.
If you like our work, you will love our newsletter..
Arshad is an intern at MarktechPost. He is currently pursuing his Int. MSc Physics from the Indian Institute of Technology Kharagpur. Understanding things to the fundamental level leads to new discoveries which lead to advancement in technology. He is passionate about understanding the nature fundamentally with the help of tools like mathematical models, ML models and AI.
Credit: Source link
