First Real-Time Brain-Controlled Hearing Device

Researchers at Columbia demonstrated the first real-time brain-controlled hearing system that can identify which speaker a listener is focusing on in a noisy environment and automatically amplify that voice while suppressing others. “This breakthrough addresses the ‘cocktail party effect,’ a major limitation of conventional hearing aids, which often struggle to distinguish between overlapping conversations in noisy settings,” reports Neuroscience News. In the new study, Columbia researchers teamed up with surgeons and their epilepsy patients who were undergoing brain surgery to better pinpoint the sources of their seizures. The hospital patients, who volunteered to be part of this study, already had electrodes implanted in their brains. [senior author Nima Mesgarani’s] system used the electrodes to measure the brain activity of the patients as they focused on one of two overlapping conversations played simultaneously. The system then automatically detected which conversation a patient was paying attention to and adjusted the volume in real time, turning up that conversation while quieting the other. For one volunteer, the experience of controlling the system with her brain was literally unbelievable. She accused the researchers of secretly adjusting the volumes. Others told stories about friends and family with hearing impairments who could benefit from such a technology. One person said: “It seems like science fiction.”

[…] The scientists developed real-time machine-learning algorithms that could examine the brainwaves and identify which conversation the patients were paying attention to. Once deployed, their system could rapidly deduce which conversation each listener was paying attention to and make it easier for them to hear it. This happened both when the researchers guided the subjects toward a particular conversation, and when the subjects chose freely, as would be necessary in a real-world conversation. “For this to work in real time, the system has to be very fast, accurate and stable for the experience to feel pleasant for the listener,” Dr. Mesgarani said. The scientists found their new system correctly identified which conversation the volunteers paid attention to. This dramatically improved the intelligibility of the speech the volunteers focused on, reduced listening effort, and was consistently preferred by the volunteers when compared to conversations the system did not provide assistance with. One volunteer recalled her uncle, who had hearing problems. “Can you imagine if this technology existed in a world [where] … he could access it? He might actually live a much more peaceful… life.” The research has been published in Nature Neuroscience.