Moving a Paralyzed Hand—A Biomedical Big Data Success Story
Ian Burkhart was paralyzed during a vacation to the Outer Banks of North Carolina in June of 2010 when he collided with a sandbar while diving into incoming waves and suffered a spinal cord injury. Ian’s injury severed the communication links between his brain and his body from the chest down. Since then, he has been unable to move his legs or his hands, which limits his ability to live independently. However, over the past two years, Ian has taken part in a study of technology that could help him regain the ability to move his paralyzed right hand with the assistance of our neural bypass system.
The goal of this technology is to reconnect the link from the brain of someone like Ian to his body, bypassing the damaged spinal cord and connecting brain activity directly to muscle activation. Ian’s success with the neural bypass system is just one of many promising technological advances to improve the lives of those living with spinal cord injuries. Hardly any of these technologies would be possible without careful data analysis and statistical algorithms to help translate electrical brain activity into action.
The neural bypass technology that Ian uses is a result of a large collaboration requiring expertise from many diverse disciplines, including statistics. Recently, in this very magazine, Radman, et al., put forth “a call to action—for statisticians to spread beyond the mathematical roots of statistics to a budding computational future where Big Data challenges are tackled side by side with biomedical and behavioral scientists.”
This call was not without context. It highlighted two recent large government funding initiatives, the Big Data to Knowledge (BD2K) and the Brain Research through Advancing Innovative Neurotechnologies (BRAIN), as opportunities ripe for statisticians to make significant contributions. Radman, et al., also discussed several barriers that statisticians face when attempting to tackle challenges associated with biomedical science. These include a lack of sufficient biological background and the limited amount of openly accessible data sources for exploration. In light of these barriers, the authors discuss ways that statisticians and biomedical scientists can work together to tackle urgent biomedical problems.
Here we highlight a recent success by our joint team of statisticians and engineers at Battelle and medical professionals at the Ohio State University that we believe embodies the ideas put forth by Radman, et al.: developing a neural bypass system to allow individuals with paralysis to regain mobility. This project required the synthesis of several disciplines and a strong collaboration between doctors, engineers, and statisticians. It is our hope that reading about this work will inspire other statisticians to engage in such complex collaborative projects with the ultimate goal of improving the lives of people like Ian Burkhart.
Translating Neural Signals into Movement: Brain-Computer Interfaces
Brain-computer interface (BCI) systems convert neural signals from the brain into intended actions that can be used, for example, to guide prosthetic devices or control a computer cursor. BCI systems typically consist of sensors to record electrical activity in the brain, a method to extract features from the raw signals, a decoding method that translates these signals into intended action commands, and a device such as a robotic arm that physically executes these commands. (For a useful overview of BCI work, see the 2014 review article from Kao, et al.)
While there are a few options for the type of brain signal that a BCI system can use, much recent work has focused on signals recorded directly from the motor cortex via surgically implanted microelectrode arrays (MEAs). BCIs built using these recordings display better performance than BCIs that use less-invasive means to collect brain data. Over the years, intracortical BCIs have been shown to allow non-human primates and paralyzed humans to control computers, electronic wheelchairs, and robotic arms through imagined movements.
In a recent article for Nature, “Restoring Cortical Control of Functional Movement in a Human with Quadriplegia,” we demonstrate that these same intracortically recorded signals can be used to allow a person with quadriplegia to regain functional control of a paralyzed limb using an external muscle stimulation cuff.
While all of these studies provide compelling evidence for the viability of BCI systems in improving the lives of individuals with paralysis, barriers still remain that must be overcome before a clinically deployable BCI system can be achieved. Several of these barriers include challenging Big Data problems that would benefit from the involvement of statisticians.
Statistical Challenges Associated with Building BCI Systems
Among the statistical challenges associated with constructing BCI systems for human usage are data availability, intelligent feature engineering, and predicting the user’s intentions.
The first problem is a scarcity of data. It is difficult to find the right kind of patients who are willing, as Ian was, to volunteer for elective surgery to implant the MEA. Combine that with the myriad challenges involved in getting regulatory approval, along with assembling and funding the large, diverse team necessary to support such a study, and it is clear why there have been very few intracortical-BCI studies involving human subjects to date.
Traditionally, the data associated with those studies have remained with the groups who performed them. This lack of exploratory data inherently limits the advancement of human BCI research, since only those researchers with access to the right kind of data can work on improving these systems. This is beginning to change, however, with several brain data-related competitions being run by Kaggle and the announcement that McGill University’s Montreal Neurological Institute will be making data and results freely available to the community after they are published.
Processing the raw electrical activity recorded from the brain is also a challenge. First, the data must be processed in real time to be of use in the BCI system. If there is a significant lag between Ian thinking of a movement and his hand moving, the system will be clunky and unintuitive.
For our study, we used a 96-channel Utah microelectrode array with 1.5 mm-long electrodes (Blackrock Microsystems, Inc., Salt Lake City, UT) to acquire raw voltage data from the brain. Each of the 96 channels of the microelectrode array records at a rate of 30,000 samples per second. Thus, 2.88 million samples have to be processed every second of the recording session, or about 1.7 x 108 samples per minute.
After data acquisition, relevant features have to be extracted as inputs into the decoding algorithm. The purpose of the feature extraction step is to both reduce the dimensionality (typically, the temporal dimension) of the data and filter it to increase the signal-to-noise ratio as much as possible.
A complication in feature extraction is the fact that the quality of recorded neural signals is known to decay over time. More precisely, biological responses, material changes, and engineering failures occurring at the electrode recording site post-implantation are known to cause degradation in recorded signal quality.
Intelligent feature extraction is a method by which we and other researchers have attempted to combat this signal degradation. For example, one common type of feature derived from the raw data is the neural firing rate. There are several competing methods to calculate firing rates. One simple method is by counting the number of times the voltage crosses a preset threshold within a certain time window. The firing rate is generally considered to be a good indicator of how “active” a neuron or group of neurons are. That is, neural firing rates tend to increase in brain regions that are actively engaged during a specific task.
A third challenge for constructing reliable BCI systems for human use is the development of decoding algorithms that can translate extracted neural features into intended movement commands accurately and quickly. Although this may seem ripe for the application of standard statistical learning techniques, the fact that neural activities are known to be non-stationary creates additional complexity. In this setting, non-stationarity refers to unexpected changes in the statistical characteristics of the neural signals over time.
These instabilities have myriad causes, including changes in motor and cognitive states or micro movements of the electrode array on the surface of the brain. Changes in motor state, such as the position of the shoulder and arm, can significantly change neural activity related to hand movements. Cognitive states can also influence neural activity. Using the system requires Ian to concentrate for extended periods of time. Variables like sleep, caffeine, and medication can affect his ability to concentrate, which manifests in his neural activity.
Lastly, each electrode measures activity at a very small region, so very slight movements of the array on the surface of the brain can lead to large changes in the signals received. Regardless of the source of the non-stationarities, they can cause problems for decoding algorithms that assume a fixed relationship between neural signal and intended movement.
Many standard decoding methods, such as the Kalman filter, assume a fixed linear relationship between neural activity and intended movement. Although attractive for their conceptual simplicity and ease of implementation, linear methods may not sufficiently capture the complex relationship between neural activity in the motor cortex and intended movement.
Neural Bypass Technology
Quite a few researchers focus on solving these and other challenges related to BCIs. In our recent Nature paper, we demonstrated that intracortically recorded signals can be linked in real time to muscle activation to restore hand movement in a human being. We used a chronically implanted intracortical MEA to record activity from Ian’s motor cortex and applied novel feature extraction, coupled with non-linear support vector machine (SVM) decoding algorithms, to translate the neuronal activity into control signals. A custom-built high-resolution neuromuscular electrical stimulation system then passed these signals to Ian’s forearm muscles. A flow chart of the system can be seen in Figure 1.
Figure 1. Overview of our neural bypass system. We record electrical activity on each of the 96 electrode channels. We then apply wavelet decomposition to the data from each channel and normalize and average the coefficients for select scales to create one mean wavelet power (MWP) value for each channel for each time point. We feed these values into decoders for each movement and use the output to drive stimulation and evoke the desired hand movement.
Figure credit: Mingming Zhang.
The system provided isolated finger, hand, and wrist movements. In one experiment, Ian achieved continuous cortical control of six different wrist and hand motions. He was also able to complete functional tasks relevant to daily living, such as lifting a mug and swiping a credit card. He was even able to play a guitar video game!
Here is how we addressed some of the common BCI statistical issues to build a system that Ian could use.
To cope with the degradation in signal recording quality, we applied wavelet decomposition to the signals recorded from the 96-channel MEA. Wavelet decomposition is a technique for decomposing a temporal signal that provides both frequency and temporal information. In a previous study published in Bioelectronic Medicine, our group showed that the coefficients of the wavelet transformation preserve frequency information about the raw signal, and are an effective way to process neural signals for BCI-related applications.
Wavelets are also appealing because they can be computed quickly and automatically. In our study, we applied wavelet decomposition to each of the raw voltage signals recorded from the 96 channels in 100 millisecond segments. For each channel, we obtained a feature termed MWP by averaging standardized wavelet coefficients over the time segments, as well as over four of the wavelet scales, corresponding to a range of frequencies where we expect to detect neural activity. We updated these 96 MWP values every 100 milliseconds and used them as input to our decoding algorithm. Thus, we were able to take the 288,000 data points collected every 10th of a second and summarize it with 96 values in real time.
We used MWP because we have found that it does not suffer from as severe degradation over long periods of time compared to other commonly used features, such as firing rates.
We then processed the MWP features by multiple simultaneous neural decoders, one for each movement, using a nonlinear kernel method with a non-smooth SVM created by one of our collaborators (see Further Reading for more details). Because of the complex non-stationarities of neural signals, we trained decoders in blocks. Each block consisted of randomized repetitions of desired motions, with short rest periods between them. After each block, we adapted (created or updated) the decoders using data from the most recent three blocks.
During the first block, before a decoder had been built, the system provided “scripted” feedback (stimulation corresponding to the cue) while Ian imagined the cued movements. This allowed us to capture data with the stimulation system on, including any artifacts or effects induced by the movements. After the first block, we trained a decoder and Ian received muscle stimulation in real time corresponding to the output of the decoding algorithm that was monitoring his brain activity.
Ian usually does five to seven blocks of training. Then we fix the decoder and he uses that decoder for the rest of the experiment. Experimental variables such as the number of motions, randomized repetitions, and blocks vary depending on the experiment being performed. For instance, as discussed in the Bouton, et al., paper, Ian was able to control six motions simultaneously—the largest number of movements demonstrated in the literature for someone with Ian’s type of injury.
To play the guitar video game required three distinct motions, corresponding to the three buttons on the guitar. The number of repetitions within a block has been inversely related to the number of movements. When Ian had been controlling two or three movements, he was asked to repeat the movement four to six times per block, but when he was controlling six movements, the number of repetitions was reduced to three or four. This was to ensure that the blocks do not last for too long.
A typical block is designed to last 3 to 5 minutes and be followed by a short rest break. This helps Ian maintain the focus and concentration he needs to control the system.
While the experimental setup may have changed depending on the task, the data collection and processing stayed the same. Every 100 ms, the system recorded 3,000 data points on each of the 96 electrodes. The data was transformed to MWP features and then classified using the decoders. Based on the decoders, stimulation parameters were updated to evoke the motion the decoder predicted. The result was that Ian was able to use the system to bypass his damaged spinal cord and control his hand using his thoughts. An example of decoder output can be seen in Figure 2.
Figure 2. Decoder output for an experiment with six discrete movements. The blue line is where Ian is cued to perform that movement via a hand on the computer monitor and zero otherwise. The green trace in each plot corresponds to the output of the decoder for that movement. Each decoder operates independently and outputs a value between -1 and 1, with values closer to 1 predicting movement and values closer to -1 predicting non-movement. If none of the decoders outputs a value above zero, no stimulation is administered. If any of the decoders output above zero, the movement corresponding to the decoder with the highest output is stimulated.
For each individual decoder, accuracy is calculated as the percentage of time points where the decoder agreed with the cue, sensitivity is the percentage of time points during the movement cue that were correctly classified, and specificity is the percentage of time points when the movement was not cued that were correctly classified. Decoder outputs below -0.5 were set to -0.5 for visual clarity.
As a demonstration, we tested the system on blocks consisting of five trials of each of six hand, wrist, and finger movements presented in a random order. The number of different movements that can be successfully distinguished by decoders is an important measure of any BCI system and one of the reasons we used an invasive MEA rather than a non-invasive EEG cap. We showed that Ian was able to achieve an overall accuracy of around 70%, indicating the potential use of such a system in allowing other individuals to regain functional use of a paralyzed limb.
Figure 3 shows some of the tasks that Ian has been able to complete with the system. Currently, he is only able to use the system in the laboratory. More innovation is needed to develop robust, portable systems that could be used on a continuous, daily basis.
Figure 3. Ian performing complex dexterous tasks. 1) Mimicking the virtual hand on the monitor to generate a wrist flexion for the individual movement task described in the Bouton, et al., paper. The colored dots on Ian’s fingers, as well as the additional “sixth finger,” are used to track the location of his fingers in three-dimensional space using a stereo camera positioned above his hand. 2) Swiping a credit card through a card reader. This is a task that Ian suggested would contribute to his ability to live independently. 3) Ian playing a guitar video game. 4) Ian pouring the contents of a mug into a glass. We use dice instead of liquid because of the proximity to sensitive electronics.
The Future for Statisticians in BCI Research
Despite demonstrated success of these recent advances in neural bypass technology, much more work remains before BCI systems can be made suitable for everyday use. One big challenge is creating a long-term, stable BCI system that does not require frequent retraining of the decoding algorithm. Currently, daily supervised retraining of the decoder copes with signal variances and maintain accuracy over time. However, daily retraining can be burdensome and time-consuming, and the need for it will undoubtedly hinder the widespread adoption of any BCI-controlled neuroprosthetic devices.
A relatively new line of research in intracortical-BCI systems is the development of decoders that adapt autonomously to changes in the neural signal that occur during BCI operation. These decoders, generally referred to as adaptive, update themselves regularly during normal BCI use without requiring knowledge of a user’s true intended action, thus eliminating the need for daily retraining. This is certainly one area where statisticians can play a large role in advancing the state of BCI systems.
Another area for statisticians is the development of feature extraction techniques. Although we have found that MWP provides a stable feature for BCI system use, there may be other methods that could perform as well, if not better.
The next generation of sensors will acquire significantly more data, which will still have to be processed in real time. In fact, DARPA has put out a call for a device that records from 1 million channels simultaneously! At that scale, statistical signal processing methods will be needed not just for decoding, but also for compressing the signals so they can be transmitted efficiently from the brain to the controller.
Collaboration is Essential
To conclude, we agree with Radman, et al., that advances in BCI systems cannot be made without collaborations between statisticians, engineers, physicians, and neuroscientists. Future advancements in the field of BCIs will depend on access to more data, more cross-disciplinary collaborations, and novel technological and engineering advances. We encourage statisticians to move into the biomedical fields to help not only in neuroengineering, but also in other areas that will undoubtedly see greater advances with the aid of statisticians.
Many biomedical fields have recently seen an explosion in data collection capabilities. This has led to not only exciting opportunities for statisticians to analyze data, but also the opportunity to positively affect people’s lives.
Further Reading
Bouton, C.E., A. Shaikhouni, N.V. Annetta, M.A. Bockbrader, D.A. Friedenberg, D.M. Nielson, G. Sharma, P.B. Sederberg, B.C. Glenn, W.J. Mysiw, A. G. Morgan, M. Deogaonkar, and A.R. Rezai. 2016. Restoring Cortical Control of Functional Movement in a Human with Quadriplegia. Nature 533-7602: 247–250.
Humber, Cary, Kazufumi Ito, and Chad Bouton. 2010. Nonsmooth Formulation of the Support Vector Machine for a Neural Decoding Problem. arXiv preprint arXiv:1012.0958.
Kao, Jonathan C., Sergey D. Stavisky, David Sussillo, Paul Nuyujukian, and Krishna V. Shenoy. 2014. Information Systems Opportunities in Brain—Machine Interface Decoders (PDF download). Proceedings of the IEEE 102-5: 666–682.
Mallat, Stéphane. 1999. A Wavelet Tour of Signal Processing. The Sparse Way. Burlington, MA: Elsevier/Academic Press.
Radman, Tom, Erica Rosemond, and Michelle Dunn. 2015. Calling All Statisticians for the Next Wave of Biomedical Big Data Discoveries. CHANCE 28-2: 19–22.
Sharma, Gaurav, Nicholas Annetta, David Friedenberg, Tony Blanco, Daphne Vasconcelos, Ammar Shaikhouni, Ali R. Rezai, and Chad Bouton. 2015. Time Stability and Coherence Analysis of Multiunit, Single-Unit and Local Field Potential Neuronal Signals in Chronically Implanted Brain Electrodes. Bioelectronic Medicine 2:63–71.
The U.S. Food and Drug Administration (Investigational Device Exemption) and Ohio State University Medical Center Institutional Review Board (Columbus, OH) provided approval for this study. The study met institutional requirements for conduct with human subjects and was registered at the www.ClinicalTrials.gov website (identifier NCT01997125). The participant referenced in this work provided permission for photographs, videos, and medical information and completed an informed consent process before the study commenced.
About the Authors
David Friedenberg is a principal research statistician at Battelle and has been the algorithm and data analysis lead for the NeuroLife™ neural bypass collaboration between Battelle and the Ohio State University for the past three years. He earned his PhD in Statistics from Carnegie Mellon in 2010 and a BS in Mathematics and Statistics with a minor in Computer Science from Miami University in 2004.
Michael Schwemmer is a research statistician at Battelle and has been working on the data analysis and algorithm design for the NeuroLife™ project for the past several months. He has a PhD in Applied Mathematics from the University of California, Davis and has held postdoctoral research positions at Princeton University and the Mathematical Biosciences Institute at the Ohio State University.
