A central feature of voluntary movement is the sense of volition, but when this sense arises in the course of movement formulation and execution is not clear. Many studies have explored how the brain might be actively preparing movement before the sense of volition; however, because the timing of the sense of volition has depended on subjective and retrospective judgments, these findings are still regarded with a degree of scepticism. EEG events such as beta event-related desynchronization and movement-related cortical potentials are associated with the brain's programming of movement. Using an optimized EEG signal derived from multiple variables, we were able to make real-time predictions of movements in advance of their occurrence with a low false-positive rate. We asked participants what they were thinking at the time of prediction: Sometimes they were thinking about movement, and other times they were not. Our results indicate that the brain can be preparing to make voluntary movements while participants are thinking about something else.
The awareness of the intention to move has been considered by both philosophers (Hume, 1748) and physiologists (Kristeva-Feige et al., 1995; Libet, Gleason, Wright, & Pearl, 1983; Kornhuber & Deecke, 1965). In trying to connect the subjective experiences of intention with the physiology of movement initiation using EEG and EMG phenomena, researchers have had difficulty because their experiments have depended on a retrospective subjective report for timing (Libet et al., 1983). Even with the more robust results of advanced technologies such as fMRI finding cortical activity even further in advance of movement, the reliance on subjective reports of timing remains (Soon, Brass, Heinze, & Haynes, 2008). In addition, despite the mounting evidence in favor of Libet's conclusions about the nature of will, action, and the brain, the limitation of the retrospective and subjective data has continued to draw criticisms (Klemm, 2010) and have left the debate unresolved (Lavazza & De Caro, 2010; Heisenberg, 2009). Evidence from the study of mind wandering appears to show that persons can make actions while having interfering thoughts (Stawarczyk, Majerus, Maquet, & D'Argembeau, 2011). A number of investigators still insist, however, that movements are not initiated unconsciously (Trevena & Miller, 2010).
As highlighted by several authors (Danquah, Farrell, & O'Boyle, 2008; Haggard & Cole, 2007; Lau, Rogers, & Passingham, 2007), the perception of intention is, at least in part, a reconstructed phenomenon; some evidence would even suggest that it is fully reconstructed (Sirigu et al., 2004). Despite the attempts of many researchers to minimize subjectivity from their paradigms (Lau et al., 2007; Spence, Shore, & Klein, 2001), there remains the need for new types of analysis. In this regard, we have approached a more objective probing of the premovement conscious experience by employing temporally random questioning of participants (Matsuhashi & Hallett, 2008). Although with different timing from the Libet et al. experiment, this work does indicate that the statistically defined time of movement intention generally precedes the onset of measurable brain activity.
Taken together, the literature suggests that one becomes aware of their will to perform a movement only after the brain has started to prepare for this act (for a review, see Hallett, 2007). This suggests that a movement might be programmed while our mind is engaged in thinking about something else. There are some types of movements that people make where there is little sense of volition, including automatic movements and associated movement. In general, people would say that these movements are certainly voluntary, but their attention was directed elsewhere. Our consideration here is movements that are overtly voluntary and that are associated with a strong sense of volition. It is the timing of this sense that we wish to elucidate.
We have recently developed a method using EEG to predict when a voluntary movement was about to occur (Bai et al., 2011). Other investigators have reported such methods also (Fried, Mukamel, & Kreiman, 2011; Anderson & Fuglevand, 2008; Wessberg et al., 2000). In our work, participants made spontaneous movements, and by analyzing the EEG, we could predict movement before its occurrence. Our model performs a multivariate analysis of an array of 29 EEG leads covering the whole head. The selection of the best set of features and computational methods was chosen according to prior work on the optimal classification of movement intention (Bai et al., 2007). Event-related desynchronization in the beta oscillatory band (approximately 15–30 Hz) over the C3 region usually contributes most to the model in most cases, consistent with prior studies (Neuper, Wortz, & Pfurtscheller, 2006; Bai, Mari, Vorbach, & Hallett, 2005; Figure 1). Our model was created to ensure that we could predict movements with no more than 10% false positives, thus yielding a high degree of certainty that when movement is predicted, it will occur. Our prediction ability is less than that of Fried et al. (2011), but their method used invasive techniques.
Our methods allow for real-time, noninvasive investigation of conscious involvement during movement preparation of movements with a strong sense of volition. People's minds may wander, and this method should allow on-line decoding of thought and its relationship to behavior. This approach, presumably less subject to retrospective construction than earlier studies, will allow us to identify how often people are thinking about the impending movement and how often about something else.
Twenty-seven, right-handed, naive participants (10 men, age = 27.7 ± 4.6 years) were included. All were right-handed according to the Edinburgh inventory (Oldfield, 1971). The protocol was approved by the institutional review board of the National Institutes Health. All participants gave their informed oral and written consent before the experiments in accordance with the Code of Ethics of the World Medical Association (Declaration of Helsinki) and the National Institutes Health guidelines.
EEG was recorded with a 29 tin electrode cap (Electro-Cap International, Inc., Eaton, OH), according to the international 10–20 system. Fz was used as the ground electrode, and recordings were referenced to the right ear lobe. Right wrist extensor EMG and EOG were collected using surface tin electrodes. The amplified analogue signal was sent to a Hewlett Packard PC workstation and converted to digital signal from an analogue–digital converter (National Instruments Corp., Austin, TX). The digital signal was on-line processed in a homemade MATLAB (MathWorks, Natick, MA) Toolbox: “BCI to virtual reality” (BCI2VR). Signals from all channels were amplified (Neuroscan, Inc., El Paso, TX), filtered (0.1–100 Hz), and digitized (sampling rate, 250 Hz).
Participants were seated in a chair with the forearms supported by pillows. They were asked to perform a self-paced, voluntary movement task of right wrist extension while observing a rotating clock hand that had a full rotation in 2.56 sec. They were asked to perform the movement as spontaneously as possible, within the constraints of the experiment, when they experience the will to move, as in the classic Libet paradigm. Participants were asked to minimize blinking and were given the following instructions: (1) Do not move during the first rotation of the clock, as the computer is collecting baseline information, thereafter move spontaneously. (2) Do not use any cues, such as numbers on the clock or counting. In addition, there was a movement indicator light that would turn green upon detection of the wrist movement by EMG. We recognize that such movements are not fully spontaneous in such a setting, yet this is one of the only ways to collect such data, and the movements do have generally have a strong sense of volition and that is what we want to study.
Data were collected over no more than four calibration sessions, each of which contained 60 trials. Participants performed self-paced, voluntary right wrist extensions according to the aforementioned instructions. EMG and EOG activities were continuously monitored on-line. Once EMG activity above 50 μV was detected by the computer program, the clock hand would stop rotating and the green light would turn on, and an event marker was made in the EEG records. The EMG signals were bipolarly derived, high-passed at 5 Hz, and rectified before being sent to a threshold detector.
The data from the calibration sessions were analyzed according to the methods developed by Bai et al. (2007) in the creation of a predictive model. One of the calibration sessions was used to develop a model based on the various spatial filtering, temporal filtering, and pattern classification analyses chosen by a genetic algorithm to optimize our feature selection strategy. A parametric model for the prediction was constructed using the baseline data set from the first, movement-free rotation of the clock and the intention data set from the 1.5 sec preceding a movement. Once the model had been constructed from these baseline and intention data sets, it was validated by performing a pseudo on-line run on another calibration data set. Trials in which predictions were made within the 1.5 sec before a movement were classified as true positives (TP); trials in which predictions were made before the 1.5 sec preceding a movement were classified as false positives (FP); all other trials in which there were no predictions were classified as false negatives (FN). Different feature selections allowed for different true positive-to-false positive ratios and were displayed in a relative operating characteristic (ROC) curve. To ensure the robustness of our concretely constructed model, we performed an off-line validation on a collected data set and chose a threshold that ensured <10% of trials had predictions not followed by a movement—a false positive (FP)—and >45% of trials had predictions followed by a movement—a true positive (TP; Figure 2). The remainder of the trials, 45%, were instances where our model failed to predict a movement—false negatives (FN). As our objective was to eliminate uncertainty in our predictions regardless of the participant's awareness of his or her intention to move or the presence of movement on EMG, minimization of the FP rate was our first priority. If such a robust model could not be derived from the data set, the experiment was aborted. It should be noted that the false positive rate considering only the predicted trials was 10% of 55% or 18%.
The model was then used for subsequent on-line, real-time prediction sessions, each consisting of 60 trials. To accumulate a data set of at least 50 predictions, we performed no greater than four on-line predictions sessions, during which the model would attempt to predict the movements of the participants in real time. Data about movements without prediction were not collected or analyzed. During those “predictions” sessions, participants were given the same instructions as the calibration sessions. Additionally, they were told that the clock might stop by itself from time to time. In case this would happen, they were told to stop any moving process and answer the question that would appear at the bottom of the screen. Indeed, trials in which a prediction occurred introduced this novel element to the task: at the time of prediction, the clock was stopped and the green light was turned on as if a movement had occurred, and participants were given 10 sec to respond to the questions, (1) “Did you feel the intention to move?” and (2) “If not, what were you thinking about?” All other trials—in which no predictions were made—were no different from those in the preceding calibration sessions and comprised the false negative (FN) trials (i.e., movement without prediction). After at least 50 predicted trials, one final calibration session was performed to confirm that the model had remained robust throughout experimentation.
Descriptive Data Classification
Movement predictions occurred sometimes too close to the movement onset for the movement to be aborted. Thus, participant responses collected from the on-line prediction sessions were organized into four prediction type categories based on the presence of EMG activity (verified off line) and/or the subjective experience of intention (based on the response to Question 1, “Did you feel the intention to move?”): (1) movement with intention, (2) movement without intention, (3) no movement with intention, and (4) no movement without intention. The feedback recorded in response to Question 2 (“If not, what were you thinking about?”) was then subcategorized by the researcher into 1 of 13 response type subcategories: inhibition, interesting, metacognition, movement-related, [the act of] moving, null, the clock, the light, the movement [itself], the question, the task, timing, and other (Figure 4). Despite the conditional component—“if not”—of the second question, participants occasionally chose to respond to both questions even if they felt the intention to move (a “yes” response to Question 1), these data were reported and included in the descriptive analysis.
Fourteen participants' data were withdrawn due to poor model formulation, in some circumstances caused by participant nonadherence to the experimental protocol or excessive noise in the signal. This left 13 participants for analysis; in each, the model assured a maximum false positive rate of 18%.
In the on-line prediction phase, predictions fell into four categories (Figure 3): movements with intention (43%), movements without intention (13%), no movement with intention (12%), and no movement without intention (32%). The 82% of true positives afforded by our model comprised of the 55% of instances of intention, regardless of movement, 13% of instances of movement without intention, and a portion (14%) of the instances of no intention or movement, where the brain was still preparing to make a movement. The presence of intention with a prediction verified the legitimacy of the model and comprised 55% of total predictions. In 43% of the trials, the prediction was made “late” in the course of movement formulation and execution so that the participant actually moved despite the light turning on in advance of the movement. The remaining 12% of predictions associated with the intention to move occurred in the absence of movement. The reported intention to move by subjects served to verify the predictions in subsequent analyses.
Of particular interest are the instances of predictions without intention. Such predictions comprised 45% of total predictions, of which 13% had an associated movement, further validating the robustness of our paradigm. The 55% of trials with intention plus these 13% of trials with movement, even without intention, make up 68% of the true positives. The 32% of predictions without movement or intent contain all the 18% false positives and another 14% of true positives. This 14% of true positives indicate predictions for movements that presumably were going to be made, but the participant cancelled the brain process of movement generation when the light was turned on before any subjective volition. We cannot tell the difference between the false positives and the true positives, nor can the participants. However, in these 14% of the predictions without movement or intent, the brain was preparing to make a movement. These predictions as well as the 13% of predictions when there was movement but no intention indicate that the participant's awareness of the intention to move is not necessary for the brain's preparation, and even the execution, of movement.
In those instances where participants did not report the intention to move (45% of total predictions), we asked them what they were thinking about. Thoughts ranged through a wide variety of topics (Figure 4), which were subcategorized as part of the post hoc descriptive analysis to characterize what the participants were thinking about during the trials. Often thoughts were far afield from the experimental setting, such as lunch or skiing, although sometimes they had something to do with the experimental setting, such as features of the clock.
Many participants, even when intending to move, spontaneously reported thoughts that they had in parallel to the intention to move, comprising 15% of total predictions and a quarter of all responses. When intending to move, thoughts were far more likely to relate to moving or to the experimental setting. Despite the vast majority of thoughts falling into these response subcategories, the participants were found to be thinking about topics unrelated to their movement in 31% of the reports.
In the presence of intention when our prediction occurred without movement or before the movement, participants sometimes reported an emotional dissonance. They used terms such as “perturbed,” “physically shocked,” “frustrated,” and “miserable failure.” Some participants expressed a sense of surprise at the early prediction, one going so far as to jerk back in the seat in response to the first few predictions. Many participants attempted to “beat” the clock to prevent being anticipated.
Our results verify our earlier result, in a group of new participants, that human voluntary movement can be predicted with reasonable accuracy in real time on a single-trial basis, instead of requiring averaging over multiple trials in a post hoc analysis. We do not know exactly how far in advance each prediction occurs, but from the model, we know that the predictions occur in the 1.5 sec before movement. In 56% of trials, participants moved, though we anticipated their movement, and in a further 12%, participants said that they were intending to move, but did not, presumably because the intention was vetoed (though we did not tell them to abort the movement if the light turned on before them doing so). It is this capability to veto movement after an intention arises that Libet felt was the embodiment of free will. Hence for 68% of predictions, there was either movement or intention or both. This indicates that the algorithm works well and that in many circumstances participants are indeed thinking about the intention to move before the move, further validating the robustness of the model. As the frequency histogram indicates that most predictions are late in the premovement interval, it is not surprising that the sense of intention would be present in a majority of predictions. When a movement is made following a prediction, it is possible that the “sense of intention before the movement” may be retrospectively generated. Hence, these movements are difficult to interpret in terms of the actual timing of the intent.
In 13% of predictions, though the participants moved, they said that were not intending to move at the time of prediction, and this might seem counterintuitive. There are several explanations for this situation. Some of the movements might have been automatic, movements that occur without thinking about them. Although the experimental setup was focused on movement, it was relatively simple and may not have required much thought, and this would allow movements to be automatically carried out while thinking about something else. Another possibility is that intention and subsequent movement occurred after our prediction. Because we were asking the thoughts at the time of prediction, this might have been premature. No participant mentioned this to us, and we did not ask explicitly about this possibility; however, participants did have the freedom to describe any of their thoughts. In addition, participants might have forgotten what they were thinking at the time of the probe. This is a general limitation of our paradigm; although they did not have to remember for more than a few seconds.
Another possibility for movement despite lack of intention is that the intention to move is sometimes retrospectively constructed and that construction did not occur because of the prediction, as has been noted in instances of post-movement interference paradigms (Lau et al., 2007). Further insight into what might have been happening in this circumstance can come from a careful look at what thoughts the participants reported (Figure 4). There is particular interest in thinking about “inhibition,” which occurred about 22% of the time. This might have indicated that the person had been thinking about movement, then about inhibition, but though thinking about inhibition did not actually implement the inhibition.
In 32% of the movements, there was no movement and no intention. Removing the 18% false positives (when the brain was not preparing to move) leaves 14% of the movements where the brain was preparing to move, but the participants were thinking about something else. We presume that these are the early predictions, but we do not know that for sure. As noted above, it is also possible that the 13% of movements without intention indicate that the brain was preparing to move before the sense of intention developed. These results are in accord with our hypothesis stemming from earlier subjective or more indirect evidence that the brain begins the preparation to move before the conscious appreciation that this is happening, and the sense of volition gradually develops in conscious awareness thereafter (Matsuhashi & Hallett, 2008). It is even reasonable to assume that some of these predictions are happening far in advance of the 1.5 sec preceding a movement, as was seen in prior studies (Soon et al., 2008), thereby disrupting the motor program formulation or execution. Because the >82% validity of our model ensures that the 14% of the nonintentioned, nonmovement trials are times when the brain is formulating a movement, our heuristic is presumed to be capturing signals of movement formulation that are present in a period in which disruption may prevent the movement or the sense of intention from actualizing (Figure 5).
When prediction occurred without overt intention, sometime thoughts were related to the experimental situation. Thus, it could be conjectured the “train of thought” might be considered to be approaching the idea of making a movement. Often, however, the thoughts seemed random, and we could not create a more logical classification as has been done by others in similar experimental situations (Kvavilashvili & Fisher, 2007).
An alternate explanation in the situation of prediction without volition is that there might have actually been volition, but that it was backwardly masked by the light and the question. This seems unlikely as these stimuli are related to the movement and would have more likely facilitated thoughts of volition than suppressed them.
The dissonance experienced by participants with predictions is likely due to a disruption of expected temporal associations. As pointed out by Wegner (Wegner & Wheatley, 1999), the normal sense of causality requires cause before effect at an appropriate time interval. In our calibration trials that occurred at the beginning, movement caused the light to turn green giving rise to the sense of agency. Agency in this experiment is not for the movement; it is for the light. In most of the movements, the movement and light are “entangled,” but when we make the predictions, they are disentangled and the light gives a time marker for which the participant has to describe his or her thoughts. When the effect occurs without cause (volitional intent) or when the effect occurs earlier than expected, the sense of agency is distorted.
Multivariate, whole-brain analysis has provided a means of capturing activity relating to movement preparation (Bai et al., 2007). Modifying our model to take into account more variables, such as signal evolution over time, might provide greater sensitivity and specificity in signal analysis (Libet, 1993). Despite repeated validation of the isolated movement paradigm in our experiment and by others (Neuper et al., 2006; Bai et al., 2005; Kristeva-Feige et al., 1995; Libet et al., 1983), removal of the clock, light, and any other cues along with eliminating instruction sets to track purely spontaneous movement will more realistically emulate truly volition activity. With refinement of our model and a larger data set, a more definitive timeline will potentially be developed.
Recent developments have begun to integrate mind and machine in much more functional ways (McFarland, Krusienski, Sarnacki, & Wolpaw, 2008; Moritz, Perlmutter, & Fetz, 2008; Velliste, Perel, Spalding, Whitford, & Schwartz, 2008; Chapin, 2004; Carmena et al., 2003; Fetz, 1999), emphasizing the ability to interface with the brain in real time. To effectively manage the “efferent” limb of a brain computer interface, whole-brain analysis of the sort we have done here offers a robust method to identify the variety of intentional actions as well as delve more deeply into the nature of volition and agency.
The work was supported by the National Institute of Neurological Disorders Intramural Research Program. L. Schneider was funded by the Howard Hughes Medical Institute-National Institutes of Health Medical Research Scholars Fellowship, Chevy Chase, MD. E. Houdayer was funded by the Fyssen Foundation.
Reprint requests should be sent to Mark Hallett, Chief, Human Motor Control Section, National Institute of Neurological Disorders, National Institutes Health, Building 10, Room 7D37, 10 Center Drive, MSC 1428, Bethesda, MD 20892-1428, or via e-mail: email@example.com.