Image credit: Peter Schreiber Media for iStock
When a person has a neurological condition and can no longer speak, brain-machine interfaces implanted via surgery may help convert brain signals or neural activity into produced speech. But inserting these brain-machine interfaces under the scalp is invasive. Imagine if non-invasive, topical sensors which capture brain signals from the scalp’s surface such as with electroencephalography (EEG), could translate thoughts into fully produced speech. This could help individuals who have lost the ability to produce speech due to paralysis or other disease or disorder. This is basis for a new paper being presented at the 2025 International Conference of Acoustics Speech and Signal Processing (ICASSP) in Hyderabad, India.
The paper, “Enhancing Listened Speech Decoding from EEG via Parallel Phoneme Sequence Prediction,” whose first author is USC Ming Hsieh Department of Electrical and Computer Engineering PhD student Jihwan Lee, demonstrates a method that improves the performance of models that decipher what you are listening to from EEG brain signals. This new and improved tool to remove noise or errors that could distort the accuracy of the model, is 2x as effective as a previous version he worked on within the USC Signal Analysis and Interpretation Lab (USC SAIL) last year.
“The incredible collective and converging technological advances in biosensing, signal processing and AI are enabling new insights into the brain-body-behavior connections underlying human speech and language processing and generation including critically when some of these systems are affected by injury, disease or disorder. This, in turn, opens up new avenues for creating technologies that can support and enhance human communication and interaction,” says Shrikanth Narayanan, USC University Professor, Nikias Chair in Engineering, and Director of SAIL, as well as the senior author on the study.
Lee notes that scientists are far from creating a tool that directly converts EEG signals into produced speech but he explains that this step is akin to the beginnings of language development in children. Language development in children happens in steps, with various building blocks being acquired and honed, as the ability to hold rich spoken language interaction emerges.
Lee describes the tool he created as a combination of the underlying type of speech technologies used in a device like Siri or Alexa or Google home to interpret or understand your spoken speech with a newly designed algorithm to understand how your brain comes to understand/comprehend speech.
Why is this algorithm necessary? The “recorded signal [from the brain] is very noisy,” says Lee. This means that is hard to get a clear read of what the signal represents.
The next step for this project is to tackle produced speech, indicates Lee, who says he is “excited that this non-invasive approach can be applied to a much wider range of people who have communications disorders, while the previous invasive approaches are very limited to individuals who are willing to undergo invasive brain surgery.”
Narayanan and several colleagues from USC in collaboration with researchers in India also hosted a workshop on related theme of “Mapping Brain-Body-Behavior Signal Dynamics in Human Speech Production and Interaction” in Hyderabad as part of ICASSP 2025.
Published on April 11th, 2025
Last updated on April 15th, 2025
