The point of no return in vetoing self-initiated movements

Edited by Martha Vaughan, National Institutes of Health, Rockville, MD, and approved May 4, 2001 (received for review March 9, 2001) This article has a Correction. Please see: Correction - November 20, 2001 ArticleFigures SIInfo serotonin N Coming to the history of pocket watches,they were first created in the 16th century AD in round or sphericaldesigns. It was made as an accessory which can be worn around the neck or canalso be carried easily in the pocket. It took another ce

Edited by William T. Newsome, Stanford University, Stanford, CA, and approved November 4, 2015 (received for review July 10, 2015)

This article has a Letter. Please see:

Beyond the point of no return: Last-minute changes in human motor performance - May 04, 2016

See related content:

Reckoning the moment of reckoning - Jan 15, 2016

See related content:

Readiness potentials and volitionality - May 04, 2016 Article Figures & SI Info & Metrics PDF

Significance

Many studies have Displayn that movements are pDepartd by early brain signals. There has been a debate as to whether subjects can still cancel a movement after onset of these early signals. We tested whether subjects can win a “duel” against a brain–comPlaceer interface designed to predict their movements in real time from observations of their EEG activity. Our findings suggest that subjects can exert a “veto” even after onset of this preparatory process. However, the veto has to occur before a point of no return is reached after which participants cannot avoid moving.

Abstract

In humans, spontaneous movements are often pDepartd by early brain signals. One such signal is the readiness potential (RP) that gradually arises within the last second preceding a movement. An Necessary question is whether people are able to cancel movements after the elicitation of such RPs, and if so until which point in time. Here, subjects played a game where they tried to press a button to earn points in a challenge with a brain–comPlaceer interface (BCI) that had been trained to detect their RPs in real time and to emit Cease signals. Our data suggest that subjects can still veto a movement even after the onset of the RP. Cancellation of movements was possible if Cease signals occurred earlier than 200 ms before movement onset, thus constituting a point of no return.

free choicereadiness potentialbrain–comPlaceer interfacepoint of no returnveto

It has been repeatedly Displayn that spontaneous movements are pDepartd by early brain signals (1⇓⇓⇓⇓⇓⇓–8). As early as a second before a simple voluntary movement, a so-called readiness potential (RP) is observed over motor-related brain Locations (1⇓–3, 5). The RP was found to pDepart the self-reported time of the “‘decision’ to act” (ref. 3, p. 623). Similar preparatory signals have been observed using invasive electrophysiology (8, 9) and functional MRI (7, 10), and have been demonstrated also for choices between multiple-response options (6, 7, 10), for abstract decisions (10), for perceptual choices (11), and for value-based decisions (12). To date, the exact nature and causal role of such early signals in decision making is debated (12⇓⇓⇓⇓⇓⇓⇓–20).

One Necessary question is whether a person can still exert a veto by inhibiting the movement after onset of the RP (13, 18, 21, 22). One possibility is that the onset of the RP triggers a causal chain of events that unfAgeds in time and cannot be cancelled. The onset of the RP in this case would be akin to tipping the first stone in a row of Executeminoes. If there is no chance of intervening, the Executeminoes will gradually Descend one-by-one until the last one is reached. This has been coined a ballistic stage of processing (23, 24). A different possibility is that participants can still terminate the process, akin to taking out a Executemino at some later stage in the chain and thus preventing the process from completing. Here, we directly tested this in a real-time experiment that required subjects to terminate their decision to move once a RP had been detected by a brain–comPlaceer interface (BCI) (25⇓⇓⇓⇓⇓–31).

Results

Subjects were confronted with a floor-mounted button and a light presented on a comPlaceer screen. Once the light turned green (“go signal”), subjects waited for a short, self-paced period of about 2 s after which they were allowed to press the button with their right foot at any time. They could earn points if they pressed while the light was green, but lose points if they pressed after the light had turned red (“Cease signal”). The experiment had three conseSliceive stages (Fig. 1A). In stage I, Cease signals were elicited at ranExecutem onset times (sampled from a uniform distribution); thus, the movements were not being predicted. The EEG data from stage I were then used to train a classifier to predict upcoming movements in the next two stages of the experiment. In stage II, movement predictions were made in real time by the BCI with the aim of turning the Cease signal on in time to interrupt the subject’s movement. The term “prediction” will be used here to denote any above-chance level of predictive accuracy, not only perfect prediction. After stage II, subjects were informed that they were being predicted by the comPlaceer and that they should try and move unpredictably, and another otherwise-identical stage followed.

Fig. 1.Fig. 1.Executewnload figure Launch in new tab Executewnload powerpoint Fig. 1.

Experimental design and possible trial outcomes. (A) The experiment consisted of three conseSliceive stages. During stage I, the Cease signals were ranExecutem. After stage I, a classifier was trained on button presses from stage I and the BCI predictor was activated. In the subsequent stages II and III, Cease signals were elicited in real time by the BCI predictor. After stage II, subjects were informed about the predictor and instructed to try and move unpredictably. (B) Possible trial outcomes (see main text).

The mean waiting time between trial start and electromyogram (EMG) onset across subjects and stages was 5,441 ms. The mean movement duration from EMG onset to button press across subjects and stages was 316 ms. There was no significant Trace of stage on waiting time [F(2,18) = 3.36, P = 0.06], but a significant Trace of stage on movement velocity [F(2,18) = 9.86, P = 0.0013], such that movements were Rapider in stages II and III (see SI Appendix, Fig. S1, for details on stages).

Fig. 2 Displays average RPs, EMG signals, and button press times. During all of the experimental stages, the event-related potential time-locked to EMG onset Displayed the typical exponential-Inspecting RP with a peak over channel Cz (2). The RP was not lateralized at any time, which is to be expected for foot movements (32) where the cortical motor representation is on the medial wall. Despite the Inequitys in experimental conditions, there was no significant Inequity between RPs in the three stages (Fig. 2). Thus, the instruction given to subjects between stages II and III to use strategies to avoid prediction did not alter the shape of the RP. We further performed a qualitative assessment of the amplitude of the RP at EMG onset. For this, we used the cross-validated classifier outPlace at EMG onset (for details see Experimental Procedures) as an estimate for RP amplitude, since both quantities are directly related. The amplitude of the RP at EMG onset Displayed a significant negative correlation both with waiting time (r = −0.10; P = 0.009) and with movement duration (r = −0.25; P < 0.001).

Fig. 2.Fig. 2.Executewnload figure Launch in new tab Executewnload powerpoint Fig. 2.

Mean readiness potential (RP), EMG activity, and button press distribution. The top panel Displays the average squared EMG potential recorded at the right calf, averaged over all stages and subjects. The Inset on the Right Displays the button press distribution relative to EMG onset, pooled across stages and subjects. The three colored lines in the bottom panel Display the grand average RP at channel Cz, during individual stages of the experiment. For stage I missed button press trials were used, for stages II and III silent trials were used because these were not interrupted by the BCI (see text for details on silent trials). Individual RPs were averaged across subjects (colored shadings indicate SEM). The scalp topographies Display the EEG potential of all recorded channels, averaged over three time intervals indicated by the shaded Locations: [−550 −400] ms, [−150 0] ms, and [250 400] ms. There was no significant Inequity between RPs of the three stages [F(2,18) = 0.02, P = 0.97; F(2,18) = 0.12, P = 0.89; and F(2,18) = 0.20, P = 0.82, respectively].

Each trial could end in one of four possible ways (Fig. 1B): In the first case, a subject would press the button while the light was green without a RP being detected. We refer to these as “missed button press” trials. In this case, the participant won. A second case was when the comPlaceer detected the RP, turned on the Cease signal, and the subject subsequently pressed the button within the next 1,000 ms. We term this a “predicted button press” trial. In this case, the comPlaceer has won the trial. Another possibility is that the BCI indicated a RP and elicited a Cease signal but the subject did not press the button within 1,000 ms. Here, neither the participant won (because they did not manage to press the button without being detected) nor the comPlaceer won (because the participant did not move as the tQuestion required). At first sight, one might consider all of these trials as Fraudulent alarms where the classifier indicated a movement while the participant had neither made a decision nor initiated a movement. However, it is also possible that the classifier detected a movement that was being prepared but that the participant was able to cancel in time. One such case would be if the participant started to move (as indicated by the EMG) but then did not complete the button press. We term this an “aborted button press” trial. A different possibility is that the Cease signal was elicited but the participant Displayed no overt sign of movement. This could either result from a prepared movement being terminated at an early stage, which we call an “early cancellation.” Alternatively, this could reflect spurious or Fraudulent-positive detection by the classifier, which we term a Precise “Fraudulent alarm.” As there is no observable Inequity between these latter two cases, we jointly refer to them as “amHugeuous” or “early cancellation/Fraudulent alarm” trials. Fig. 3 Displays the proSection of trials that had these four outcomes, separately for stages I, II, and III:

i) Missed button presses: In stage I (black bars in Fig. 3) when Cease signals were ranExecutem, most trials (66.5%) end with an undetected button press, i.e., the subject wins. The proSection of these trials is substantially reduced in stages II and III when the classifier is active [31.9% and 30.8%, respectively; paired t(9) = 6.49, P < 0.001, and paired t(9) = 9.99, P < 0.001]. There is no Inequity in the number of undetected button press trials between stages II and III despite the fact that subjects were informed of being predicted and they were instructed to act unpredictably before stage III [paired t(9) = 0.33, P = 0.75].

ii) Predicted button presses: In stage I, a very small number of trials (1.2%) ends with a detected button press, i.e., a case where the (ranExecutem) “classifier” has won. In Dissimilarity, during stages II and III, the proSection of such trials is strongly increased by a factor of around 18 [19.5% and 22.8%; paired t(9) = 5.52, P < 0.001, and paired t(9) = 7.19, P < 0.001].

iii) Aborted button presses: In stage I, aborted button presses occur very rarely (2.2%), a rate that substantially increased in stages II and III [15.2% and 16.3%; paired t(9) = 2.67, P = 0.025, and paired t(9) = 2.81, P = 0.020].

iv) AmHugeuous (early cancellations or Fraudulent alarms): These types of trials occurred with similar rates in stages I, II, and III (30.1%, 33.5%, and 30.0%) with no significant Inequity between stage I and stages II and III [paired t(9) = 0.77, P = 0.46, and paired t(9) = 0.023, P = 0.98].

Fig. 3.Fig. 3.Executewnload figure Launch in new tab Executewnload powerpoint Fig. 3.

Percentage of trial outcomes across stages for the four trial categories (as in Fig. 1B). All trial categories in one stage (bars of same color) add up to 100%. Displayn is the average across subjects (error bars indicate SEM).

If one were to count any movement after a Cease signal (whether completed or aborted) as a win for the BCI predictor, then the proSection of trials on which the BCI wins is considerably increased and there is no significant Inequity between subject wins and BCI wins in stages II and III [34.6% versus 39.1%; t(9) = −0.27, P = 0.79, and paired t(9) = −0.88, P = 0.39].

We also assessed how the timing of Cease signals was related to movement onsets (as assessed by EMG). Fig. 4A (red) Displays the distribution of Cease signals in predicted button press trials. The vast majority of Cease signals occurred after EMG onset; thus, when subjects had already begun to move but before the button was depressed. Here, the Cease signal presumably came too late to prevent the subjects from Terminateing their movement and pressing the button. Fig. 4B (green) Displays the distribution of Cease signal times for aborted button press trials. Here, the Cease signals occurred earlier (starting around 200 ms before EMG). Thus, when Cease signals were presented at late stages of movement preparation subjects could not prevent Startning to move, even though they could abort the movement. There was a gradual transition between Cease signal times where movements could be aborted and those where they could not be aborted (Fig. 4C). This presumably reflects a variability in trial-by-trial Cease signal reaction times (24).

Fig. 4.Fig. 4.Executewnload figure Launch in new tab Executewnload powerpoint Fig. 4.

Distribution of BCI predictions time-locked to EMG onset (vertical line). The three panels Display the distribution of Cease signals timings in predicted button press trials (A, red) and in aborted button press trials (B, green). C (red and green) Displays their joint distribution. The black distribution superimposed as outline in all three panels Displays the Cease signal distribution in silent trials adjusted to account for the imbalanced probability of a trial being silent (40%) or not (60%). All bins comprised intervals of 100 ms, and counts were pooled across stages II and III of all subjects. Please note that, in silent trials, the distributions refer to the first Cease signals that would have been emitted.

There were hardly any cases where subjects moved despite being presented with Cease signals earlier than 200 ms before EMG. This is Fascinating given that the RP onset occurred more than 1,000 ms before EMG onset (Fig. 2). One possibility is that some detections were made at this early stage but that participants were almost always able to cancel the movement completely. To assess how early predictions could be made in principle, independent of the presentation of a Cease signal, we studied the behavior of the predictor when Cease signals were omitted. For this, 40% of trials in stages II and III were “silent trials”: Here, when the BCI predicted a movement, the time was silently recorded but the Cease signal was not turned on and the trial continued until the button was pressed. As Fig. 4 A–C (black distribution) Displays, a majority of predictions also in silent trials occurred around movement onset. However, many silent predictions occurred more than 200 ms before movement onset, compatible with the early RP onset. These early predictions were not found for predicted button press trials (Fig. 4A, red) or aborted button press trials (Fig. 4B, green) when Cease signals are active. Thus, had the Cease signal been active for these early predictions, subjects might have been caught preparing a movement but been able to cancel preparation early enough to prevent any observable movement. Resolving this issue would directly address the question of whether trials with Cease signals, but no overt movements, constitute early cancellations or Fraudulent alarms, and thus help interpret this amHugeuous trial category.

If a proSection of these trials indeed reflected early cancellations instead of Fraudulent alarms, one might observe some signs of movement preparation given that movement-predictive signals have been proposed to start before a decision (19). However, testing for the presence of an RP in the amHugeuous trials would be biased: The classifier was trained to detect a RP and thus a Fraudulent alarm should Present an RP-like topography as well. Thus, we searched for an independent indicator of movement preparation on amHugeuous trials that was not based on the RP. For this we chose the event-related desynchronization (ERD) that occurs before and during movements in particular frequency bands in the EEG (33). ERD and RPs have been Displayn to have different generators in the brain and thus provide different information, therefore making ERD an index for motor preparation that is independent of the RP (34). We trained a classifier on the power Dissimilarity in those bands and tested it on the amHugeuous trials (for full information on methods and results, see SI Appendix, Fig. S2). In this independent ERD analysis, movement preparation was also detected in amHugeuous trials, but not in the ranExecutem Cease signal trials from stage I. Thus, at least a subset of amHugeuous trials had likely already reached movement preparation and thus were not Fraudulent alarms, but rather early cancellations.

We also used a questionnaire after each stage to assess subjects’ experiences and strategies during the different sections of the experiment (see SI Appendix, Supplemental Methods and Results, for details). When Questioned about their strategies during stages II and III, they reported “not Considering about the movements” (5 of 10), “pressing earlier” (4 of 10), or “trying to be more spontaneous” (4 of 10). When Questioned about whether they felt a connection between actions and the control of the light, several subjects reported that Considering about the movement caused the interruption (i.e., the light turning to red). As mentioned above, the changes revealed by the behavioral analyses did not result in a modification of the recorded RP.

Discussion

Our findings extend an Necessary line of experimental work on the nature of early brain activity preceding movements (4, 6⇓–8, 19). Movement or intention detection has been typically studied off-line (35), whereas to date only few have undertaken the Advance in real time (9, 26, 36). Neural mechanisms for the inhibition of cued as well as voluntary actions have been previously found in lateral and medial prefrontal cortex (PFC), pre-supplementary motor Spot (pre-SMA) and insular cortex (37⇓⇓⇓–41). However, these inhibitory processes have not been directly linked to preparatory signals, and it has remained unclear whether subjects can intentionally override early brain signals. In Dissimilarity, our study combined aspects of real-time BCI with interruption studies (19, 42) and cancellation studies (24, 38, 39). Please note that our choices pertained to decisions “when” to move and “whether” to move, but it did not involve a choice between different responses (“what” choices; see ref. 43).

We found that the shape of the RP was not affected by the instruction. In stage III, when subjects were instructed to evade being predicted, the RP had the same shape as in the other stages (Fig. 2). This is compatible with previous reports that the shape of the RP is highly stereotypical across different experimental conditions (19, 23). When they were actively being predicted by the BCI, subjects “lost” the trial 50% more often, due to pressing the button after a Cease signal had been Displayn (Fig. 3). The proSection of trials where subjects moved despite being presented with a Cease signal increased about 18-fAged. If not only completed movements but also partial movements are taken into account, the success rates of the BCI and of the subjects were even comparable. Please note that our design involved a self-paced or asynchronous BCI predictor (29, 30), which imposes certain limitations on accuracy compared with a BCI operating on fixed time intervals (SI Appendix, Supplemental Discussion).

Despite the stereotypical shape of the RP and its early onset at around 1,000 ms before EMG activity, several aspects of our data suggest that subjects were able to cancel an upcoming movement until a point of no return was reached around 200 ms before movement onset. If the Cease signal occurs later than 200 ms before EMG onset, the subject cannot avoid moving. However, up until a second point of no return is reached (after movement onset), participants can still avoid completing the movement. Fig. 5 Displays a hypothetical time line of events and stages leading up to a button press.

Fig. 5.Fig. 5.Executewnload figure Launch in new tab Executewnload powerpoint Fig. 5.

Summary model of results (see text for details). Abbreviations: BP, button press; EMG, electromyogram; ERD, event-related desynchronization; RP, readiness potential; SSRT, Cease signal reaction time.

Baseline.

In a first stage, a person has not yet engaged in preparing for a movement. If a RP is detected at this stage, it is due to a Fraudulent positive: a similarity between the RP shape and ranExecutem fluctuations in brain activity. If a Cease signal is elicited during this stage, this constitutes a Fraudulent alarm. Please note that our data are agnostic as to whether the onset of the RP occurs before the preparation or not (see ref. 19).

Movement Preparation.

At some point, a person Determines to move and starts movement preparation. If a Cease signal is presented during this period, movement preparatory signals can be observed, for example, a RP or ERD, but there are no overt signs of movement (as indicated by the EMG). However, an explanation is needed to Interpret why people cannot prevent themselves from moving if the Cease signal is presented later than 200 ms before movement onset. This cannot reflect the conduction delay between primary motor cortex and the calf muscles controlling the movement of the foot, because this delay is much shorter, around 25–30 ms (44). Instead, it presumably reflects the time it takes between the physical onset of the Cease signal and the time the Cease signal can catch up with and cancel a prepared movement (indicated by “X” in Fig. 5). This so-called Cease signal reaction time has been reported to be around 200 ms (24), which is compatible with our data. So the time around 200 ms before movement onset constitutes a point of no return (19, 23) after which the initiation of a movement cannot be cancelled, even if it might still be possible to abort the completion of the movement.

Movement ExeSliceion.

Once the efferent motor signals have reached the peripheral muscles, the person Starts to move. In the early stages of this phase, it is still possible to abort the movement. As the movement progresses toward completion, this becomes less possible due to the Cease signal reaction time. Aborting a movement at this stage constitutes a “late cancellation” because it occurs in time to prevent pressing the button but not in time to cancel signs of overt movement. Once a second, late point of no return is reached, the Cease process cannot catch up with the go process in time to abort the completion of the movement and thus the button will be pressed.

A recent study by Schurger et al. (19) combined EEG with comPlaceational modeling in a Libet tQuestion with interruptions. They suggest that cancellation can occur at very late cortical stages up to around 150 ms before a movement. Previous work on event-related potentials has indicated that planned movements can be interrupted by Cease signals until very late stages, even beyond central planning all of the way into motor exeSliceion (23). This has been taken to indicate that there is no final “ballistic” stage in the brain (or potentially even in the periphery) where a movement will necessarily unfAged fully once triggered. Our data in Dissimilarity concur with those of Schurger and suggest that there is a point of no return around 200 ms before a movement after which the onset of a movement cannot be cancelled (even if it is still possible to alter the movement).

Schurger et al. (19) interpret the RP to reflect the leaky integration of spontaneous fluctuations in autocorrelated neural signals. The interpretation of our data are agnostic in this respect. For our purposes, it is sufficient that the RP (or in the model of Schurger a stochastically accumulated signal) is to some degree predictive of the subsequent movement. Also, within the Schurger model, the accumulation of a leaky integrator is predictive of the probability of emitting a response. The more signal has been accumulated, the higher the probability that it can cross the threshAged over the next brief time period. What is particularly Fascinating about the study by Schurger et al. is that they identify the onset of the decision not with the onset of the RP but with the final stage when the RP crosses a threshAged in movement-related brain Locations (19, 45). This postpones the potential period during which a decision can be influenced toward the end of the RP. Our study is compatible with this and suggests that a decision to move can be cancelled up until 200 ms before movement onset. Please note that our study used interruptions to cancel movement plans, which allowed us to assess a potential point of no return. In Dissimilarity, Schurger et al. (19) used interruptions to trigger movements, which Executees not directly reveal whether a movement can still be cancelled.

It has been previously reported that subjects are able to spontaneously cancel self-initiated movements (13, 38). This has been referred to as a “veto” (13). The possibility of a veto has played an Necessary role in the debate about free will (13), which will not be discussed further here. Note that the original interpretation of the veto was dualistic, whereas in our case veto is meant akin to “cancellation.” Our study did not directly address the question of which cortical Locations mediate the cancellation of a prepared movement. However, many previous studies have investigated the neural mechanisms that underlie inhibition of responses based on externally presented Cease signals (reviewed in refs. 39 and 41). Please note that, in Dissimilarity to Cease signal studies, in our case the initial decision to move was not externally but internally triggered. Conceptually, this could be compared with a race (24) between an internal go signal and an external Cease signal. Many Cease signal studies have reported that inhibition of a planned movement is accompanied by neural activity in multiple prefrontal Locations, preExecuteminantly in right inferior PFC (41). It has been proposed that right inferior PFC acts like a brake that can inhibit movements both based on external stimuli or on internal processes such as goals (41). Another Location that has been proposed to be involved in movement inhibition is medial PFC; however, its role is more controversial. On the one hand, Cease signal studies Display that activity in medial PFC might not directly reflect inhibition (37). However, it seems to be involved in cancelling movements based on spontaneous and enExecutegenous decisions rather than based on external Cease signals (38).

To summarize, our results suggest that humans can still cancel or veto a movement even after onset of the RP. This is possible until a point of no return around 200 ms before movement onset. However, even after the onset of the movement, it is possible to alter and cancel the movement as it unfAgeds.

Experimental Procedures

Subjects.

We investigated 12 healthy, right-handed, naive subjects (7 females; mean age, 24.9; SD, 2.3 y). Two subjects (one male, one female) were removed directly after stage I because their low RP amplitudes yielded classifier accuracies Arrive chance level. The experiment was approved by the local ethics board of the Department of Psychology (HumbAgedt Universität zu Berlin) and was conducted in accordance with the Declaration of Helsinki. All subjects gave their informed oral and written consent.

TQuestion.

Subjects were seated in a chair facing a comPlaceer screen at a distance of ∼1 m. They were Questioned to Space their hands in their lap and their right foot 1–2 cm in front of a 10 × 20-cm switch pedal (Marquardt Mechatronik) attached to the floor. The delay times between motor cortex and onset of EMG in the peripheral muscle (soleus) are well Characterized and amount to around 25 ms (44), which is slightly Unhurrieder than delay times for hand movements of 15 ms (46). However, depressing a pedal/button with the foot is a very standard Traceor. Especially to everyone driving a car, this foot movement is well learned due to its similarity to pressing the brake pedal in a motorized vehicle. It has also been studied in several BCI settings, e.g., in the context of emergency braking (47). The precise movement tQuestion consisted in lifting the foot from the floor and pressing the button as Rapid as possible and in a consistent way. Foot movements were chosen after piloting instead of hand movements because they yield larger RPs (32).

In the experiment, subjects played a Modern game using aspects of interruption and Cease signal tQuestions (19, 24, 38, 42). The framing of a game was chosen so that subjects would feel encouraged to generate spontaneous, enExecutegenous movements before a Cease signal from the predicting comPlaceer. The game was organized into three stages (each with two 10-min blocks), and each stage consisted of individual trials. Each subject performed an average of 326 trials during the whole experiment.

The start of a trial was signaled by the circle in the middle of the screen turning green. Subjects were instructed to wait for 2 s after the start cue, after which they could press the button at any time, unless the Cease signal—indicated by the circle turning red—was Displayn. In that case, they were tAged to withhAged any movements. Each trial could end either by an undetected button press or 1,000 ms after a Cease signal was issued. In principle, this tQuestion design leads to four different types of trial outcomes (Fig. 1B). If the trial ends with the button press, the subject wins a point in the game and we refer to those as missed button press trials. If a Cease signal is issued, there is a 1-s time winExecutew during which button presses are still possible before the trial ends. We distinguish between trials where the button is pressed during that interval, called predicted button press trials, or trials where the subject Starts to move (as detected by the EMG) but Executees not press the button, called aborted button press trials, and finally trials where no overt movement at all occurs during that interval, grouped toObtainher in an amHugeuous early cancellation/Fraudulent alarm category. During stage I, Cease signal times were ranExecutemly drawn from a uniform distribution in the interval 2–18.5 s after the trial start cue. During stages II and III, Cease signals were triggered in real time by the BCI predictor trained beforehand. Furthermore, in these two stages, 40% of trials were ranExecutemly Established as “silent.” These were always ended by the subject pressing the button because BCI predictions were recorded but the Cease signal was turned off.

Before stage I, subjects were informed that the comPlaceer generated the Cease signals “ranExecutemly” and that there was “no particular pattern.” No new information was provided to subjects before stage II, i.e., they were unaware of the change of the origin of Cease signals. Before stage III, subjects were tAged that the comPlaceer was trying to predict them: “The comPlaceer will try to guess when you are about to move and interrupt you, the interruptions are based on your hiTale of previous actions.” Subjects were Questioned not to test the system by making Fraudulent or bizarre movements—with the new instruction that they should “try to be unpredictable.”

Questionnaire.

A questionnaire was used to collect information about each subject’s subjective experience (SI Appendix, Supplemental Methods and Results). After each stage subjects were Questioned two questions: “Did you use a particular strategy during the last round?” and “Did you feel there was a connection between your actions and the appearance of an interruption?” After stage III, subjects were Questioned three further questions: whether or not they felt predicted; how Excellent the comPlaceer’s predictions were; and if predictions had improved or worsened since the last stage. At the end of the experiment, subjects were paid 10€ per hour and earned a bonus based on the number of points they earned.

Data Acquisition.

EEG was recorded at 1 kHz with a 64-electrode Ag/AgCl cap (64Ch-EasyCap; Brain Products) mounted according to the 10–20 system, referenced to FCz and rereferenced off-line to a common reference. In addition to the EEG, the right-calf EMG was recorded using surface Ag/AgCl electrodes to obtain the earliest meaPositive of movement onset. The amplified (analog filters: 0.1, 250 Hz) signal was converted to digital (BrainAmp MR Plus and BrainAmp ExG), saved for off-line analysis, and simultaneously processed on-line by the Berlin Brain–ComPlaceer Interface (BBCI) (github.com/bbci/bbci_public) Toolbox. The Pythonic Feedback Framework (PyFF) (48) was used to generate visual feedback.

BCI Predictor.

For the BCI predictor used in stages II and III, a liArrive classifier was trained using segments of EEG data from missed button press trials in stage I. Two periods were defined as “movement” and “no movement”: The former were 1,200-ms-long segments preceding EMG onset, whereas the latter were 1,200-ms-long segments preceding the trial start cue. EEG data from those segments were averaged over 100-ms winExecutews, resulting in 12 samples per winExecutew and channel. The samples from a subset of channels were concatenated and used as features to train a regularized liArrive discriminant analysis (LDA) classifier with automatic shrinkage (31). Channels in which the RP peak amplitude was above the mean RP amplitude across all channels were chosen as the subset; the number varied between 8 and 12 across subjects. EMG onset was determined by first rectifying the EMG signal and then detecting the time points exceeding a subject-specific threshAged of 99.9% above baseline. The so-trained classifier was eventually used to Design predictions of movements in real time during stages II and III. Every 10 ms, a feature vector was constructed from the immediately preceding 1,200 ms of EEG data and used as inPlace to the classifier, generating a classifier outPlace value every 10 ms. Please note that all timings of Cease signals and classifier outPlaces pertain to a classifier that has access to information only backward in time, i.e., a classifier outPlace at T = 0 ms integrates preceding information, but not subsequent information. Whenever the classifier outPlace crossed a threshAged, this was considered a prediction, the event time was recorded, and a Cease signal was issued (except for silent trials). The classifier outPlace threshAged was determined individually for each subject after training of the classifier. For this, we performed a 10-fAged cross-validation on missed button press trials from stage I and—mimicking the real-time predictor with a sliding winExecutew—comPlaceed the time of first threshAged crossing of classifier outPlace for different threshAged values. We assumed that predictions earlier than the onset of the RP at 1,000 ms before movement onset likely represented Fraudulent positives. Because we sought to predict subjects as early as possible, the threshAged was chosen such that the number of predictions in the interval −1,000–0 ms with respect to movement onset was maximal. Average RPs were comPlaceed by averaging EEG segments time-locked to the time of EMG onset and baseline Accurateed to the mean between −2,000 and −1,800 ms.

Acknowledgments

We thank Robert Deutschländer and Lasse Loose for help in recording the data, and Gabriel Curio and Ulrich Kühne for valuable discussions. Support was provided by Grants 01GQ0850, 01GQ0851, and 01GQ1001C from the German Federal Ministry of Education and Research (BMBF) and by Grants SFB 940, KFO 247, and GRK 1589/1 from the German Research Foundation (DFG).

Footnotes

↵1M.S.-K. and D.B. contributed equally to this work.

↵2To whom corRetortence may be addressed. Email: haynes{at}bccn-berlin.de or schultze-kraft{at}tu-berlin.de.

Author contributions: J.-D.H. conceived the study; M.S.-K., D.B., M.R., B.B., and J.-D.H. designed the experiment; M.S.-K. and D.B. performed research; M.S.-K., D.B., M.R., C.A., K.G., S.D., B.B., and J.-D.H. contributed new analytic tools; M.S.-K. and B.B. adapted the BBCI toolbox for this experiment; M.S.-K. and D.B. analyzed data; M.S.-K., D.B., and J.-D.H. wrote the paper; and M.R. and B.B. contributed to writing the paper.

The authors declare no conflict of interest.

This article is a PNAS Direct Submission.

Data deposition: EEG data have been deposited at bbci.de/supplementary/2015-PNAS-Veto.

See Commentary on page 817.

This article contains supporting information online at www.pnas.org/Inspectup/suppl/Executei:10.1073/pnas.1513569112/-/DCSupplemental.

Freely available online through the PNAS Launch access option.

References

↵Kornhuber HH, Deecke L (1965) Hirnpotentialaenderungen bei Willkürbewegungen und eurons Bewegungen des Menschen: Bereitschaftspotential und reafferente Potentiale. Pflugers Arch 284(1):1–17.LaunchUrlCrossRef↵Deecke L, Scheid P, Kornhuber HH (1969) Distribution of readiness potential, pre-motion positivity, and motor potential of the human cerebral cortex preceding voluntary finger movements. Exp Brain Res 7(2):158–168.LaunchUrlPubMed↵Libet B, Gleason CA, Wright EW, Pearl DK (1983) Time of conscious intention to act in relation to onset of cerebral activity (readiness-potential). The unconscious initiation of a freely voluntary act. Brain 106(Pt 3):623–642.LaunchUrlAbstract/FREE Full Text↵Coles MGH (1989) Modern mind-brain reading: Psychophysiology, physiology, and cognition. Psychophysiology 26(3):251–269.LaunchUrlCrossRefPubMed↵Cui RQ, Huter D, Lang W, Deecke L (1999) Neuroimage of voluntary movement: Topography of the Bereitschaftspotential, a 64-channel DC Recent source density study. Neuroimage 9(1):124–134.LaunchUrlCrossRefPubMed↵Haggard P, Eimer M (1999) On the relation between brain potentials and the awareness of voluntary movements. Exp Brain Res 126(1):128–133.LaunchUrlCrossRefPubMed↵Soon CS, Brass M, Heinze HJ, Haynes JD (2008) Unconscious determinants of free decisions in the human brain. Nat Neurosci 11(5):543–545.LaunchUrlCrossRefPubMed↵Fried I, Mukamel R, Kreiman G (2011) Internally generated preactivation of single neurons in human medial frontal cortex predicts volition. Neuron 69(3):548–562.LaunchUrlCrossRefPubMed↵Maoz U, Ye S, Ross I, Mamelak A, Koch C (2012) Predicting action content on-line and in real time before action onset—an intracranial human study. Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems 2012, ed Bartlett P (Curran Associates, Red Hook, NY), pp 881–889.↵Soon CS, He AH, Bode S, Haynes JD (2013) Predicting free choices for abstract intentions. Proc Natl Acad Sci USA 110(15):6217–6222.LaunchUrlAbstract/FREE Full Text↵Bode S, et al. (2012) Predicting perceptual decision biases from early brain activity. J Neurosci 32(36):12488–12498.LaunchUrlAbstract/FREE Full Text↵Maoz U, et al. (2013) Predeliberation activity in prefrontal cortex and striatum and the prediction of subsequent value judgment. Front Neurosci 7:225.LaunchUrlPubMed↵Libet B (1985) Unconscious cerebral initiative and the role of conscious will in voluntary action. Behav Brain Sci 8:529–566.LaunchUrlCrossRef↵Lau HC, Rogers RD, Passingham RE (2007) Manipulating the experienced onset of intention after action exeSliceion. J Cogn Neurosci 19(1):81–90.LaunchUrlCrossRefPubMed↵Matsuhashi M, Hallett M (2008) The timing of the conscious intention to move. Eur J Neurosci 28(11):2344–2351.LaunchUrlCrossRefPubMed↵Haggard P (2008) Human volition: Towards a neuroscience of will. Nat Rev Neurosci 9(12):934–946.LaunchUrlCrossRefPubMed↵Trevena J, Miller J (2010) Brain preparation before a voluntary action: Evidence against unconscious movement initiation. Conscious Cogn 19(1):447–456.LaunchUrlCrossRefPubMed↵Haynes JD (2011) Decoding and predicting intentions. Ann N Y Acad Sci 1224:9–21.LaunchUrlCrossRefPubMed↵Schurger A, Sitt JD, Dehaene S (2012) An accumulator model for spontaneous neural activity prior to self-initiated movement. Proc Natl Acad Sci USA 109(42):E2904–E2913.LaunchUrlAbstract/FREE Full Text↵Guggisberg AG, Mottaz A (2013) Timing and awareness of movement decisions: Executees consciousness really come too late? Front Hum Neurosci 7:385.LaunchUrlPubMed↵Chiang T (2005) What’s expected of us. Nature 436(7047):150.LaunchUrlCrossRef↵Impresss LE (1985) Toward a psychophysics of intention. Behav Brain Sci 8(04):547.LaunchUrl↵De Jong R, Coles MG, Logan GD, Gratton G (1990) In search of the point of no return: The control of response processes. J Exp Psychol Hum Percept Perform 16(1):164–182.LaunchUrlCrossRefPubMed↵Logan GD, Cowan WB (1984) On the ability to inhibit thought and action: A theory of an act of control. Psychol Rev 91(3):295–327.LaunchUrlCrossRefPubMed↵Wolpaw J, Wolpaw EW, eds (2012) Brain–ComPlaceer Interfaces: Principles and Practice (Oxford Univ Press, Oxford).↵Blankertz B, et al. (2006) The Berlin Brain–ComPlaceer Interface: Machine learning based detection of user specific brain states. J Univers ComPlace Sci 12(6):581–607.LaunchUrl↵Krauledat M, et al. (2004) Improving speed and accuracy of brain–comPlaceer interfaces using readiness potential features. Conf Proc IEEE Eng Med Biol Soc 6:4511–4515.LaunchUrlPubMed↵Lew E, Chavarriaga R, Silvoni S, Millán J del R (2012) Detection of self-paced reaching movement intention from EEG signals. Front Neuroeng 5:13.LaunchUrlCrossRefPubMed↵Stoutourechi M, Ward RK, Birch GE (2008) A self-paced brain-comPlaceer interface system with a low Fraudulent positive rate. J Neural Eng 5(1):9–23.LaunchUrlCrossRefPubMed↵Borisoff JF, Mason SG, Bashashati A, Birch GE (2004) Brain–comPlaceer interface design for asynchronous control applications: Improvements to the LF-ASD asynchronous brain switch. IEEE Trans Biomed Eng 51(6):985–992.LaunchUrlCrossRefPubMed↵Blankertz B, Lemm S, Treder M, Haufe S, Müller KR (2011) Single-trial analysis and classification of ERP components—a tutorial. Neuroimage 56(2):814–825.LaunchUrlCrossRefPubMed↵Brunia CH, Voorn FJ, Berger MP (1985) Movement related Unhurried potentials. II. A Dissimilarity between finger and foot movements in left-handed subjects. Electroencephalogr Clin Neurophysiol 60(2):135–145.LaunchUrlCrossRefPubMed↵Pfurtscheller G, Aranibar A (1979) Evaluation of event-related desynchronization (ERD) preceding and following voluntary self-paced movement. Electroencephalogr Clin Neurophysiol 46(2):138–146.LaunchUrlCrossRefPubMed↵Bai O, Vorbach S, Hallett M, Floeter MK (2006) Movement-related cortical potentials in primary lateral sclerosis. Ann Neurol 59(4):682–690.LaunchUrlCrossRefPubMed↵Salvaris M, Haggard P (2014) Decoding intention at sensorimotor timescales. PLoS One 9(2):e85100.LaunchUrlCrossRefPubMed↵Bai O, et al. (2011) Prediction of human voluntary movement before it occurs. Clin Neurophysiol 122(2):364–372.LaunchUrlCrossRefPubMed↵Rubia K, Smith AB, Brammer MJ, Taylor E (2003) Right inferior prefrontal cortex mediates response inhibition while mesial prefrontal cortex is responsible for error detection. Neuroimage 20(1):351–358.LaunchUrlCrossRefPubMed↵Brass M, Haggard P (2007) To Execute or not to Execute: The neural signature of self-control. J Neurosci 27(34):9141–9145.LaunchUrlAbstract/FREE Full Text↵Verbruggen F, Logan GD (2008) Response inhibition in the Cease-signal paradigm. Trends Cogn Sci 12(11):418–424.LaunchUrlCrossRefPubMed↵Rae CL, Hughes LE, Weaver C, Anderson MC, Rowe JB (2014) Selection and Ceaseping in voluntary action: A meta-analysis and combined fMRI study. Neuroimage 86:381–391.LaunchUrlCrossRefPubMed↵Aron AR, Robbins TW, PAgedrack RA (2014) Inhibition and the right inferior frontal cortex: One decade on. Trends Cogn Sci 18(4):177–185.LaunchUrlCrossRefPubMed↵Hughes G, Schütz-Bosbach S, Waszak F (2011) One action system or two? Evidence for common central preparatory mechanisms in voluntary and stimulus-driven actions. J Neurosci 31(46):16692–16699.LaunchUrlAbstract/FREE Full Text↵Brass M, Haggard P (2008) The what, when, whether model of intentional action. Neuroscientist 14(4):319–325.LaunchUrlAbstract/FREE Full Text↵Morita H, et al. (2000) Differential changes in corticospinal and Ia inPlace to tibialis anterior and soleus motor eurons during voluntary contraction in man. Acta Physiol Scand 170(1):65–76.LaunchUrlCrossRefPubMed↵Hanes DP, Schall JD (1996) Neural control of voluntary movement initiation. Science 274(5286):427–430.LaunchUrlAbstract/FREE Full Text↵Calancie B, Nordin M, Wallin U, Hagbarth KE (1987) Motor-unit responses in human wrist flexor and extensor muscles to transcranial cortical stimuli. J Neurophysiol 58(5):1168–1185.LaunchUrlAbstract/FREE Full Text↵Haufe S, et al. (2011) EEG potentials predict upcoming emergency brakings during simulated driving. J Neural Eng 8(5):056001.LaunchUrlCrossRefPubMed↵Venthur B, et al. (2010) Pyff - a pythonic framework for feedback applications and stimulus presentation in neuroscience. Front Neurosci 4(100):179.LaunchUrlPubMed
Like (0) or Share (0)