Researchers have developed a new signal processing and automatic speech recognition methods for the first time, that it’s possible to turn a person’s thoughts into a legible phrase, using what they’re calling a “brain-to-text” interface.
They presented their “Brain-to-text: decoding spoken phrases from phone representations in the brain” in the scientific journal Frontiers in Neuroscience.
Speech is produced in the human cerebral cortex. Brain waves associated with speech processes can be directly recorded with electrodes located on the surface of the cortex. It has now been shown for the first time that is possible to reconstruct basic units, words, and complete sentences of continuous speech from these brain waves and to generate the corresponding text.
”It has long been speculated whether humans may communicate with machines via brain activity alone”. ”As a major step in this direction, these recent results indicate that both single units in terms of speech sounds as well as continuously spoken sentences can be recognized from brain activity.“
”In addition to the decoding of speech from brain activity, their models allow for a detailed analysis of the brain areas involved in speech processes and their interaction”.
The present work is the first that decodes continuously spoken speech and transforms it into a textual representation. For this purpose, cortical information is combined with linguistic knowledge and machine learning algorithms to extract the most likely word sequence. Currently, Brain-to-Text is based on audible speech. However, the results are an important first step for recognizing speech from thought alone.
The brain activity was recorded in the USA from 7 epileptic patients, who participated voluntarily in the study during their clinical treatments. An electrode array was placed on the surface of the cerebral cortex (electrocorticography (ECoG)) for their neurological treatment. While patients read aloud sample texts, the ECoG signals were recorded with high resolution in time and space. Later on, the researchers in Karlsruhe analyzed the data to develop Brain-to-Text. In addition to basic science and a better understanding of the highly complex speech processes in the brain, Brain-to-Text might be a building block to develop a means of speech communication for locked-in patients in the future.
The researcher Peter Brunner says the research was limited by his time with the patients and was also limited by their conditions: Each patient had the electrodes placed on different regions of their brain, depending on the part expected to be causing seizures. With more time to “train” the interface and more targeted electrode placement (he believes that the superior temporal gyrus, in the temporal lobe of the brain, would be ideal), brain-to-text interfaces could become much better. He also hopes to make interfaces that don’t need to be placed directly on the brain.
“This could be relevant for people who suffer from ALS—their muscles don’t work but there’s no effect on the brain,” he said. “You would probably want to have a person train the interface before they’re fully locked in.”
Brunner’s device wasn’t connected to the internet, but he said there’s no reason why it couldn’t be, which would allow real-time brain-connected communication.
