Brain–Computer Interfaces: Why Not Better?

Introduction

Beginning in the early 2000s, a series of human pilot trials has demonstrated “proof of concept” of the potential for implanted brain–computer interface (iBCI) systems: People with severe paralysis have been able to control computer cursors and physical devices, using only the activity of a small ensemble of neurons in the cerebral cortex ( ). The advent of iBCIs was built upon a vast base of prior accomplishments. First, fundamental neuroscience in animals provided core scientific knowledge needed to understand where and how movement is represented in the brain. Second, BCI research emerged as a distinct neurotechnology field, built upon fundamental and translational discoveries leading to the design, implementation, and translation of BCIs for human application. Third, computational neuroscience developments provided a mathematical and algorithmic framework to decode neural signals into commands. Fourth, technology advances provided both (1) implantable sensors that could detect useful signals over long times and (2) computational devices in powerful, affordable architectures able to perform decoding in real time.

It has become apparent, beginning with scalp-based EEG demonstrations, that even the simplest BCIs could provide clinical benefit for people, especially those who are severely paralyzed and unable to communicate or care for themselves ( ). Severe paralysis, including the inability to move the body and limbs for useful actions, can result from large strokes (mainly in the brainstem), upper level spinal cord injury (SCI), or neurodegenerative diseases like amyotrophic lateral sclerosis (ALS). BCIs capturing neural signals at high resolution, which requires implanted sensors, have the potential to extend clinical benefits to those with less severe paralysis (e.g., hemiplegia) and to offer a range of control options, including reanimation of paralyzed limbs. In the years leading up to iBCI human proof of concept, several fundamental neuroscience discoveries were particularly important in laying the foundation for eventual clinical translation. ¹

1 Regrettably, no single review yet has successfully captured the complex history of essential knowledge and engineering advances that produced the current iBCI field, in part because the success of BCIs draws upon advances in so many fields. Attribution to any one person for key advances is always open to debate, so one must consult multiple sources for a comprehensive perspective of BCI history.

In my view, the work of Ed Evarts, a brilliant neurophysiologist at the National Institute of Mental Health who passed away prematurely, ²

2 http://www.nasonline.org/search.html?q=evarts&submit.x=0&submit.y=0 .

created one of the most important foundational elements for iBCIs to develop. He showed that the spike rate of single motor cortex neurons in behaving primates coded movement variables (a number in a time window) ( ). The results also reinforced the idea that the motor cortex initiated behavior because activity commenced before movement. Subsequent work that adopted the Evarts method of single-neuron recording in conscious monkeys showed that more complex motor behavior could be decoded from neuron population activity using a simple mathematical framework to combine the activity of small populations of motor cortex neurons. In particular, Apostolos Georgopoulos and his students, and others inspired by the population coding concept, showed that motor cortex population collective activity provided reasonable estimates of the direction of reach or other motor variables ( ). A collection of additional nonhuman primate studies in the early 2000s, probably an inflection point in iBCI development, provided a series of advances that led to: (1) an implantable multielectrode array (MEA) technology suitable for human use that could record ensembles of ∼100 neurons in humans ( ), (2) the demonstration that arm actions could be decoded from roughly randomly selected ensembles of fewer than 100 neurons simultaneously recorded (∼dozens of neurons) from a small patch of cortex ( ), and (3) the BCI proof of concept that able-bodied monkeys could use decoded representations of neural population activity in real time to control a computer cursor ( ) or a robotic arm ( ) to perform behavioral tasks they first learned using their arm. Of the many other contributors to the fundamental background necessary to reach this point, two of particular historical note are (1) the demonstration already in the late 1960s by Eberhard Fetz that monkeys could learn to control a single motor cortex neuron ( ) and (2) the call issued by William Heetderks’ National Institutes of Health (NIH) Neural Prosthesis Program in 1990 ( ). This NIH program requested proposals for researchers to develop a BCI to restore hand function in animals that could eventually help people with SCI regain control ( Fig. 25.1 ). This program and its leaders, and a succession of BCI programs mounted by the Defense Advanced Research Projects Agency ( ), arguably established a framework for, inspired, and set the time frame for the considerable number of researchers who went on to realize this vision in monkeys, and then in humans, and to create the currently flourishing BCI field.

Across the following decade iBCI studies, now involving more than a dozen humans, repeatedly confirmed the ability of people with paralysis stemming from many different origins to control devices. Although not formally consolidated in a single report, none of these studies involving many years of implant evaluation report safety concerns or infection. For iBCIs, control emerges when desired actions are imagined—no learning is required. Thus, neural activity emerging when a person imagines moving his or her arm to a set of spatial locations can be mapped directly to commands to move a robot arm or computer cursor to those locations. People paralyzed from neurodegenerative disorders, brainstem stroke, or SCI have been able to use a few dozen signal channels of spiking-level data from one or sometimes two microelectrode arrays to control cursors, to operate a computer for communication (e.g., typing interface), or to carry out useful everyday reach-and-grasp actions such as drinking or eating, using a multijoint robotic arm ( ). Importantly, the study provided one example in which a sensor implanted 5 years earlier provided useful neural signals, indicating that long-lasting interfaces could be feasible. In the same person, reliable control occurred across each of 5 days around the 1000th day after this implant ( ), affirming the potential for useful control years after an interface has been implanted.

The BCI field has somewhat divided into two areas, one developing communication and the other emphasizing movement applications. These applications require different implementations of assistive technology, even though both rely on having useful control signals. Fig. 25.1 provides a schematic of the range of BCI applications, users, and technology to convey the spectrum of applications and the potential value of many different types of BCIs. Both communication and movement applications of BCIs have made progress. People with severe paralysis who have limited ability to communicate, because they cannot speak or cannot move their hands or arms well enough to type or write, are able to use spelling and other communication devices through a computer ( ). Communication rates have continued to improve so that iBCIs now far exceed rates obtained by EEG signals ( ), achieving around 34 characters/min. While this typing speed is still about one-sixth the rate of an able-bodied typist, it is at least five times the rate achieved by noninvasive BCI systems. A second BCI direction is to re-create arm functions, by using either a robotic arm or the person’s own arm. Perhaps the most ambitious vision, as foreshadowed in the 1990 National Institute of Neurological Disorders and Stroke call ( Fig. 25.1 ), is to reanimate the arm with a physical bridge from the brain to the arm. This vision aims to restore brain-to-body control via a physical connection. This replacement neural system requires not only a BCI that can collect neural signals complex enough to provide flexible control of the enormously complex set of arm muscles ( ), but also functional electrical stimulation (FES) system technology.

The FES system provides coordinated neuromuscular stimulation so that neural signals can produce natural arm movements ( ). This vision is feasible. In 2017 a person with SCI was able to use a neural interface to an implanted FES system to produce multijoint reach-and-grasp actions with that person’s own limb ( ). Thus both rapid communication and reanimation of the body have achieved proof of concept in humans, but not the kind of control achieved by intact biological systems.

Despite the past years of iBCI communication and movement control proof of concept in humans, a commercial product is not available as of this writing, although there have been prior attempts. The BCI field had one early commercial launch attempt with two startup companies: Neural Signals and Cyberkinetics. Cyberkinetics, founded in the early 2000s by the author and colleagues, was a venture capital–funded effort to produce a BCI that could restore movement and control. Although it successfully translated preclinical work from the author’s laboratory to human early-stage (FDA investigational device exemption or IDE) trials of the BrainGate BCI system for people with severe paralysis, it ceased operations in 2009 owing to a lack of funding. The clinical trial was transferred back to the academic group, which now, as part of a consortium, continues to develop BrainGate under an IDE. As will be described below, Cyberkinetics and their academic collaborators provided an important proof of concept in the early 2000s that signals recorded through chronically implanted electrode arrays in paralyzed humans could be used as control signals ( ). The principles and technology established in this trial are used in all ongoing human iBCI trials today. Further, this large step to translate basic research to humans would have taken many additional years without the translational team and the large capital investment that this company was able to attract. Neural Signals was founded in the late 1980s by Philip Kennedy. Although Neural Signals remains in operation, it does not have active clinical trials of an iBCI as of this writing. Neural Signals was a pioneer in showing that neural interfaces could be placed in humans ( ) and also was important in guiding the translation of BCIs to humans. Interestingly, there appears to be a second wave of commercial interest in very early stage iBCI-related companies, Kernel and Neuralink. Both are backed by successful entrepreneurs (Bryan Johnson and Elon Musk) who appear to be aiming to advance BCI development. Even with substantial capital, which is essential to translate such complex medical devices to humans, effective iBCIs still face considerable development challenges that will be characterized in this chapter.

Looking at the present state of the field, existing iBCI systems face complex biological and technological issues: control is slower, more variable, less dexterous, and less accurate than can be achieved by an able-bodied person; the technology used is bulky, cumbersome, and not portable; the implant requires a percutaneous connector that must be attached to a cable by a trained technician; and the implanted MEA sensor, while able to provide useful signals for years, is generally considered to have variable reliability and stability. Further, the design of this MEA may not provide a sufficient number or form of needed signals over many years. Across this set the interface and decoding problems are paramount.

These ( ) shortcomings are not surprising. Creating BCIs to restore movement or communication is an exceptionally complex task that requires the integration of fundamental neuroscience, computational theory and algorithms, and technology, and must also include clinical needs, regulatory science and health care delivery system policy, and consideration of their ethical, legal, and social implications ( Fig. 25.2 ). After this initial wave of proof-of-concept translation to humans it is now particularly important to assess these accomplishments and identify and act on the key basic neuroscience, computational, and technology advances to transform current lab-based BCIs into available medical devices.

The intention of this chapter is to address key aspects of the question: Why are BCIs not better? I will give a broad outline of the state of the field and then try to distinguish major limits to progress toward a commercially viable and clinically meaningful device. The aim is to provide a stimulus to accelerate the development of BCIs from a proof of concept to a real-world medical device available to a wide range of people with various forms of paralysis (from severe to mild) whose lives could be significantly enhanced through this technology. The chapter will focus on four challenge areas:

• goals for BCIs

• neuroscience knowledge

• computational processing abilities

• technology for a clinically useful product

iBCIs, whether to restore locomotion, communication (from typing to speech), or reach-and-grasp arm actions, share many of the same challenges. This chapter will largely deal with challenges for iBCIs that re-create functions achieved by the arm (spanning reaching and grasping of the user’s own arm to operating a computer, including communication as if the user were employing a keyboard and mouse with his or her hands), where most of the research and development effort has been directed. Nearly all of the issues described are readily applied to other body systems; these all have substantial value and could benefit by being considered in the same framework presented here for the arm. It is noteworthy here that significant progress in lower limb control by iBCIs is being made, including technology that could create a brain–spinal interface for body reanimation ( ). My goal is to point out some of what I feel are the most important challenges to tackle now. Other aspects of the very large literature on BCIs are covered in the following reviews: . While useful for simpler applications, by and large external BCIs, which lack access to information-rich neural signals (described under Signals: What to Record? section), cannot provide sufficient information needed to approach natural, complex multidimensional arm and hand control or communication speeds—a potentially achievable goal for iBCIs.

Brain–Computer Interface Clinical Goals

Setting a well-defined goal is a key first step for BCIs to shape and focus critical evaluation of shortcomings as well as guiding progress. In hindsight, those in the field have sometimes debated, for example, performance goals for BCIs with different intended uses. For example, bit rate has been argued across BCIs that have different levels of automated control and disparate long-term visions. BCIs with different intended clinical applications can be valuable for different purposes but require different success metrics. From the user application perspective, BCI goals can range broadly from providing a single-state switch to the full restoration of complex limb movement, as schematized in Fig. 25.3 . To a person with severe paralysis, unable to move at all or to speak, a reliable switch (one trustworthy bit) is powerfully enabling—it can allow communication that is otherwise impossible. Devices, like that based on a P300 speller with a wearable EEG cap, have already reached the commercial stage ( www.intendix.com ). As such, they are an important step to demonstrate the translational possibilities for BCI technology in general. By contrast, for a person limited only in his or her ability to make useful hand grasps a switch is not worthwhile because such actions are already achievable in many other ways. A BCI to restore dexterous grasp, especially if it required surgical implantation in the brain, would, however, need to deliver performance closer to the reliability, speed, and flexibility of natural control to be acceptable. The goal axis presented in Fig. 25.3 can be seen as a way to distinguish different BCI types and applications as well as a tool to identify common scientific challenges and differences. It should be seen as a “value” axis to judge the potential worth of the technology to the user.

Without specifying different goals it is difficult to agree upon acceptable performance criteria or a particular implementation. Setting goals is also critical to establish clinical paths and assess risk. If a goal is to achieve reliable single-state switch control, the success of external BCIs in that realm would argue that a complex device surgically placed in the brain is not warranted. Nevertheless some might elect to receive a surgically implanted device if it were unobtrusive (inside the body) and available all the time, rather than requiring frequent reapplication by a caregiver, as is required for a common EEG cap. However, if the ultimate BCI goal is full restoration of limb movement, and one (reasonably) agrees that the intracranial signals are the only source of such commands, then a more elaborate system is needed. When comparing the value of BCI technologies it is important to consider that a succession of technologies that may be worse in many ways at earlier stages than an alternative, such as current iBCIs, may make sense, and be essential, as a rational stepping-stone toward a more complex goal. Thus, Fig. 25.3 presents a schema to evoke better goal setting. Establishing clear sets of goals, and accepting that there can be a range of goals for different purposes, can help lead to better BCI systems and accelerate progress, leading to better products for BCI users.

Basic Elements of a Brain–Computer Interface

There is general agreement on common system features for BCI systems of all types. BCIs require a neural interface (a “probe,” electrode, or signal detector) for signal acquisition (or to provide feedback in the case of stimulation probes), signal processing hardware that can amplify and condition (e.g., filter, digitize) neural signals, communication devices to transmit signals to computational devices, a decoder (algorithms and associated computational and other hardware) to transform neural activity patterns into a command, and any of a wide range of controlled devices, generally known as assistive technologies (ATs). Various versions of the block diagram for BCI system design of more or less complexity have appeared beginning about 1973 ( ). BCI systems include connections to physical surrogates for the arm, such as a robotic limb or an exoskeleton that supports arm movement, or a computer (for communication); these form the main AT types. A prosthetic hand can also be considered a BCI end effector because it is another physical surrogate controller. In addition, an FES system intended to bridge the BCI device to the arm could also be considered an AT in the service of generating natural arm movements based on BCI commands. The design of a BCI system is unusually multifaceted: it requires integration of knowledge and expertise from neuroscience, computation, and technology ( Fig. 25.1 ). In addition, clinical, commercial, regulatory, and health care considerations as well as ethical, legal, and social implications (ELSI) are critical to the implementation of a device that can eventually be made available and has value to users ( ). Some of the most timely challenges to iBCI progression are considered in the next sections, but of all the challenges as of this writing, the most formidable are the neural interface and the decoder.