(Neuralink)
Neuralink’s brain chip has already shown it can help people with disabilities remotely control a mouse cursor or a robot arm. Now it’s reached a new milestone by translating brain signals into audible words.
On Tuesday, the company shared the results by featuring the Neuralink user, Kenneth Shock, who underwent surgery to receive the brain chip back in January. Shock has ALS, a neurodegenerative disease known to rob people’s ability to walk and talk, which can give them a serious speech impediment.
However, Neuralink demonstrated in a video that its N1 brain chip can help Shock more easily communicate, without him even moving his mouth. The implant can interpret his brain signals into text, which can then be spoken out loud by a computer program.
Neuralink gave a brief overview about how the approach works. It looks like the implant isn’t exactly translating his inner monologue like telepathy, but reading specific brain signals and matching them to words he wants to form. “There are certain areas of the brain that become activated and generate signals that are routed to the muscles of the mouth, tongue, and voice box,” the company says of the clinical trial.
(Neuralink)The video also shows the implant and Neuralink's software detecting Shock's brain signals and matching them to “phonemes,” or the smallest unit of linguistic sound.
In setting up the speech de-coding process, Neuralink’s machine learning engineer Skyler Granatir explained: “We will guide him for some sentences to attempt to say, and we’ll use that data to try to map neural intent to actual words.” In the first stage, Shock vocally spoke out the sample sentences. In the second stage, he merely mouthed the sentences, silently.
“The goal is we want him to be able to simply just intend to move his mouth and for our BCI (Brain-Computer Interface) to decode his speech,” Granatir adds.
During the third stage of testing, the video then shows Neuralink’s software detecting Shock’s speech, without any mouth movement at all. The implant is able to match the brain signals to each phoneme, assemble them together, then convert them into words, forming a full sentence that the program can speak out loud using Shock’s original voice.
“I’m talking to you with my mind,” he says at one point through the implant.
(Neuralink)The test is part of Neuralink’s ongoing “VOICE clinical trial,” so it may take some years before the technology becomes more widely available. The video also suggests the de-coding process still needs work. For example, it can take a few moments for the software to read his brain signals and convert them into spoken words. Still, Neuralink is aiming to drastically improve the technology over time.
“We’re going to continue advancing the quality of the sensors, the number of the sensors,” Granatir says. “We want to build the system that goes directly from the brain to voice in real-time.”