Neural Prosthetic Translates Human Brain Signals into Speech

Loss of communication is not uncommon in patients suffering from debilitating neurodegenerative diseases like ALS and clinical locked-in syndromes. One solution to restore communication involves decoding signals directly from the brain. However, coarse neural recordings inadequately capture the rich structure of human brain signals. Now, a new speech prosthetic developed by a team of Duke neuroscientists, neurosurgeons, and engineers can translate a person’s brain signals into speech. The new technology might one day help people unable to talk regain the ability to communicate through a brain-computer interface.

This work is published in Nature Communications, in the paper, “High-Resolution Neural Recordings Improve the Accuracy of Speech Decoding.”

“There are many patients who suffer from debilitating motor disorders, like ALS (amyotrophic lateral sclerosis) or locked-in syndrome, that can impair their ability to speak,” said Gregory Cogan, PhD, a professor of neurology at Duke University’s School of Medicine. “But the current tools available to allow them to communicate are generally very slow and cumbersome.”

The best speech decoding rate currently available measures roughly 78 words per minute. People, however, speak much more quickly—roughly 150 words per minute. The lag between spoken and decoded speech rates is partially due to the relatively few brain activity sensors that can be fused onto a paper-thin piece of material that lays atop the surface of the brain. Fewer sensors provide less decipherable information to decode.

For this project, researchers packed 256 microscopic brain sensors onto a postage stamp-sized piece of flexible, medical-grade plastic.

After fabricating the new implant, the team recruited four patients to test the implants. The experiment required the researchers to place the device temporarily in patients who were undergoing brain surgery for some other condition, such as treating Parkinson’s disease or having a tumor removed.

Then, the authors performed “high-resolution, micro-electrocorticographic (µECoG) neural recordings during intra-operative speech production.”

Participants heard a series of nonsense words, like “ava,” “kug,” or “vip,” and then spoke each one aloud. The device recorded activity from each patient’s speech motor cortex as it coordinated nearly 100 muscles that move the lips, tongue, jaw, and larynx.

The neural and speech data from the surgery suite was fed into a machine learning algorithm to see how accurately it could predict what sound was being made, based only on the brain activity recordings.

The authors report obtaining neural signals with 57X higher spatial resolution and 48% higher signal-to-noise ratio compared to macro-ECoG and SEEG. This increased signal quality improved decoding by 35% compared to standard intracranial signals.

The team hopes to create a cordless version of the device. “You’d be able to move around, and you wouldn’t have to be tied to an electrical outlet, which is really exciting,” explained Cogan.