Researchers have developed a brain-computer interface (BCI) capable of translating both speech and gestures in real time. The device, described in *Nature Neuroscience*, uses artificial intelligence to convert brain activity into text and animate an avatar, offering a potential communication pathway for individuals with severe paralysis.
Samantha Brosler, a PhD student in bioengineering at the University of California, San Francisco, explains that current BCI technology often focuses on restoring only one function at a time. This new system aims to capture a wider range of expressivity in communication. “This gives us hope for neural prosthesis that will enter day-to-day usage,” says Christian Herff, a computational neuroscientist at Maastricht University in the Netherlands.
The device overcomes the challenge of overlapping neural signals by utilizing a 253-electrode array placed on the surface of the brain’s cortex. This allows it to capture signals related to both speech and body movements. Researchers tested the device with two participants: one who experienced impaired speech and movement after a brainstem stroke, and another with amyotrophic lateral sclerosis (ALS) causing progressive speech paralysis and motor impairments.
The first participant was asked to silently speak five phrases and attempt gestures like waving and clapping. The second participant vocalized ten phrases and imagined ten gestures – such as a shrug and thumbs up – while limiting movement to facial muscles. The BCI recorded the participants’ brain activity during these tasks, demonstrating its ability to decode intended communication.
Read the original coverage
💬 Comments
📜 Comment Policy