U.S. researchers have used a WWII-type code-breaking technique to predict from neural activity how a rhesus monkey will move its arm. The approach, developed by scientists at the University of Pennsylvania, the Georgia Institute of Technology, and Northwestern University, is not dissimilar to that of Alan Turing’s Bletchley Park team, which applied the statistics of how letters and words construct sentences, to crack Germany’s WWII Enigma code. Led by Konrad Körding, Ph.D., a Penn Integrates Knowledge Professor, and Eva Dyer, Ph.D., currently an assistant professor in the department of biomedical engineering at the Georgia Institute of Technology and Emory University, the modern-day neural code-breakers hope that their approach could one day be used to develop brain–computer interfaces that interpret thoughts to control prosthetic limbs, or even to create speech in paralyzed patients.
They report on their technology in Nature Biomedical Engineering, in a paper entitled “A Cryptography-Based Approach for Movement Decoding.”
“Brain decoding” can be used to predict what someone is viewing or their movements. The technology uses neural recordings to infer activity or intent, the authors explain. However, “to train a decoder, one generally needs to infer the measured variables of interest (covariates) from simultaneously measured neural activity.” In effect, the training data are collected by simultaneously recording the brain activity and the predicted outcomes.
However, it’s not always possible to obtain all the required supervised data. Unlike current approaches, the technology developed by Dr. Körding’s team, working with physiology professor Lee Miller, Ph.D., and his group at Northwestern University, doesn’t require simultaneously measured neural activity and motor outputs. Instead, the technology uses “the statistics of movement” to predict what motor output will result from certain neural activities.
It’s a similar approach to that used by cryptographers to decode a cypher. Codebreakers leverage what they already know about the distribution of individual characters and their joint statistics, the authors continue. “For example, the probability of observing a written ‘E’ is much higher than the probability of observing a ‘Z.’ Using such information, Alan Turing and his Bletchley Park team cracked the World War II German Enigma code by exploiting the distribution of words known to appear in encrypted messages from their enemies.” The key concept is to learn a mapping from the encrypted to decrypted text, which produces a distribution that has a similar structure to what is expected based on prior knowledge.
The team used a similar approach to try and decode neural activity, through a series of studies with three rhesus macaques. The animals carried out a task that involved reaching a target placed at different points around a central start, and as they completed the tasks, indwelling electrodes recorded the electrical activity of several hundred neurons associated with the animals’ arm movements.
Brain–computer interfaces can already use this sort of data to move robotic prostheses. It’s an approach known as supervised learning, which trains the interface to recognize how patterns of neuronal activity relate to specific movements and then reconstruct movements in response to neuronal signals recorded from the monkey’s brain.
In contrast, the cryptographic approach doesn’t use supervised learning, Körding points out. “In cryptography, 'supervised learning' would be called a 'known plaintext attack.' That is, we have both the encrypted and unencrypted message and just need to figure out the rules that turn one into the other. What we wanted to do in this study was to be able to decode the brain, using a movement model, from the encrypted message alone.”
Körding’s team’s approach identified a way of mapping neuronal firing patterns to the monkeys’ actual arm movements.They call the movement decoding approach distribution-alignment decoding (DAD). “Essentially, the algorithm tries a range of possible decoders until we get something where the output looks like typical movements,” Körding comments. “There are issues scaling this up—it's a hard computer science problem—but this is a proof of concept that cryptanalysis can work in the context of neural activity.”
The cryptoanaytical approach also does away with the lengthy training and calibration that is associated with supervised learning. “These training periods can be long and annoying, and in some circumstances thay are truly problematic—say, if your arm is missing rather than paralyzed,” Körding adds. “Willing movement is different from imagining moving a nonexistent arm.”
Encouragingly, the DAD method performed as well as supervised decoders that have access to both neural and behavioral training data, according to the authors. DAD could also use data from one animal, to predict movement in another. “DAD can be used in an across-subject setting by using one subject’s movement data to decode the neural activity from a different subject.”
A cryptoanalysis approach to decoding neural activity could make it possible to develop brain–computer interfaces that literally mind-read to control prosthetics or generate speech. “I think we should be able to do this within the next decade,” Körding claims. “You could ask a 'locked-in' patient to generate neural patterns associated with specific words, but the corpus of language is very large. Rather than having them generate a pattern that is associated with every word they want to say, we could build a decoder that transforms those patterns until it looks like language.”
Working toward thought-driven prosthetics or artificial speech will require continued improvement in the brain-recording technology, including the development of electrode arrays that can simultaneously record from a million neurons. And while it’s early days yet, the researchers are optimistic about the potential for their approach. “The Germans were actively working against decryption, and modern ciphertexts are basically impossible to break,” Körding said. “We have it easier. The brain ended up with this encryption system through natural selection, so it's essentially making the same kind of 'mistakes' that allowed us to crack Enigma in the first place.”