Rapid and Accumulated Modulation of Action-Effects on Action

Actions are accompanied by sensory feedback. For example, we hear our own voices when we speak. The importance of auditory feedback in regulating motor control has been recognized since the 1950s using the delayed auditory feedback paradigm (van Vugt & Tillmann, 2015; Sasisekaran, 2012; Pfordresher & Dalla Bella, 2011; Chase, Harvey, Standfast, Rapin, & Sutton, 1959; Kalmus, Denes, & Fry, 1955; Black, 1950; Lee, 1950). In most natural situations, the auditory feedback from actions is usually instant (a counterexample is when we speak in a large room; see Black, 1950). In the delayed auditory feedback paradigm, the latency of auditory feedback is experimentally manipulated so that the auditory feedback follows actions with a small amount of delay. A typical finding is that delayed auditory feedback has adverse effects on the performance of actions in various systems. For example, when people hear their own voice with a 200-msec delay, they speak louder and slower.

In a manual tapping task, delayed auditory feedback leads to increased tapping peak force (e.g., Chase et al., 1959). In some recent demonstrations, participants were asked to make keypresses, and they pressed more lightly when a sound followed the press without any delay as compared to when the sound was delayed or when no sound was present (Cao, Steinborn, Kunde, & Haendel, 2020; Neszmélyi & Horváth, 2018; see also Chase, Rapin, Gilden, Sutton, & Guilfoyle, 1961). Although it is clear that finger tapping behavior can be modulated by auditory feedback, the underlying mechanism is not clear. A motor response starts with a motor program and can be modified by on-line feedback (Shadmehr, Smith, & Krakauer, 2010; Todorov, 2004; Wolpert, Miall, & Kawato, 1998). Two questions are relevant here. The first question is how quick the auditory feedback can modulate an ongoing keypress, or stated as whether an on-line modulation is possible for a keypress. A keypress is a very quick movement, which lasts about 300 msec. Thus, it may be programmed to be rather ballistic, and an on-line modulation is not possible. The answer to the question cannot be derived from existing studies as the feedback delay in a series of keypresses is constant. When a series of keypresses is followed by auditory feedback of the same delay, that is, the feedback delay is predictable, the prediction of the auditory feedback may change the motor output (Shadmehr et al., 2010; Flanagan, Vetter, Johansson, & Wolpert, 2003). Therefore, it is not known if a keypress peak force increase because of a delayed auditory feedback results from the auditory feedback of the current keypress or learning from previous trials. However, if an unpredictable auditory feedback delay also leads to a modulation of keypress peak force (e.g., the force is higher for delay vs. no delay or for long delay vs. short delay), an on-line modulation process can be assumed to be at work.

The second question is closely related to the first question, that is, whether a motor-prediction-related signal can be detected in the keypress behavior when the auditory feedback is constantly delayed. Motor prediction refers to the process of predicting the sensory feedback from own actions (reafferent inputs), and it is deemed as a part of motor program (Wolpert & Flanagan, 2001). In the time course of a keypress, the dependent variable of keypress peak force used in existing studies occurs too late to be a motor-prediction-related signal if it receives on-line modulation. For motor-prediction-related signals, one needs to focus on the very early phase of motor output in which an influence of sensory feedback is physiologically impossible (because of sensory feedback delay). Therefore, a successful detection of a modulation in the early phase of keypress force trajectory because of feedback predictability reflects a change in motor program and the related motor prediction. This may open up a possibility to study motor control processes within keypressing behavior, a very common response in cognitive studies, in a very fine-grained fashion.

To answer the above two questions, a keypress experiment was performed with a manipulation of the predictability of auditory feedback delay. The keypress force was recorded as the dependent variable. We show that the keypress peak force is modulated by the auditory feedback delay when the delay is predictable, and most importantly, also when the delay is unpredictable (evidence for on-line modulation). We also show that the early phase of keypress force trajectory is modulated by the predictability of the auditory feedback delay (evidence for a change in motor program).

Forty-nine healthy participants (28 women; mean age = 28.7; SD = 8.8) volunteered to participate in the study. Of all the participants, 23 were colleagues and friends, and the rest were recruited from a local participant pool. Only external participants received monetary compensation for participation. Forty-one participants were included in the peak force analysis and 39 in the force trajectory analysis (see Data Exclusion section). Informed consent was obtained from all participants. The study was approved by the local ethics committee (Institute of Psychology, Faculty for Human Sciences, University of Würzburg; Project No. GZ 2018-27) and conducted in accordance with the Declaration of Helsinki and the General Data Protection Regulation in Europe.

Participants sat in front of a laptop and completed a self-paced keypress task. The keypress, in our case, was made against a flat surface similarly to making a keypress on a touch screen without any movement of the key itself (Figure 1A). The auditory feedback after a keypress was manipulated (Figure 1B). There were two predictable auditory feedback delay conditions, Fix60 and Fix160. In the Fix60 condition (100 keypresses), a keypress was always followed by a 60-msec delayed sound. In the Fix160 condition (100 keypresses), a keypress was always followed by a 160-msec delayed sound. Fix60 and Fix160 conditions were presented in separate testing blocks so that the same delay was presented in a testing block. There were two unpredictable auditory feedback delay conditions, Mix60 and Mix160. In the Mix60 condition (100 keypresses), a keypress was always followed by a 60-msec delayed sound. In the Mix160 condition (100 keypresses), a keypress was always followed by a 160-msec delayed sound. The Mix60 and Mix160 conditions were presented in the same testing block (‘Mix’ in Figure 1B). To make sure that the feedback delay was unpredictable, the feedback delays in this testing block were randomized using the randperm function in MATLAB (The MathWorks). The presentation order of predictable and unpredictable auditory feedback conditions was counterbalanced among participants. In addition, a NoSound condition (100 keypresses) was included in separate testing blocks, in which no auditory feedback was provided after a keypress. The NoSound condition served as a baseline condition in which the general effect of auditory feedback on keypress can be compared (Neszmélyi & Horváth, 2018).

For each participant, 500 keypresses (or trials) were collected in blocks of 50 keypresses (10 blocks in total), and a self-paced break was given between blocks. During each testing block, participants were asked to fixate the screen center, where the block number was constantly presented. Participants were told to make a keypress after about every 2 sec. Of the 41 participants included in the final analysis (see Data Exclusion section), the average interkeypress interval was 2.54 (SD = 0.98) sec in NoSound condition, 2.34 (SD = 0.99) in mix condition, 2.36 (SD = 1.07) in Fix60 condition, and 2.52 (SD = 1.03) in Fix160 condition. No significant difference was found between conditions in the interkeypress interval with a within-subject one-way ANOVA (F(3, 120) = 1.47, p = .24, η_p² = 0.04).

The auditory feedback was a 1000-Hz tone (50 msec in duration; 5-msec rise/fall ramp), which was presented via headphones (Vic Firth SIH1) at a comfortable volume level (same volume for all participants). Participants wore the headphones throughout the experiment (i.e., including the NoSound condition). The force information of a keypress was collected by a force sensing resistor (FSR; Model 402, Interlink Electronics Inc.). The FSR, which is also the key being pressed, has a circular active area with a diameter of ∼1.3 cm. A 10-kΩ resistor, from which the voltage information was collected, was connected in series with the FSR with an input voltage of 5V. With increased force applied on the active area of the FSR, the FSR would reduce its resistance, which would result in increased current flow in the circuit and increased voltage on the 10-kΩ resistor. Analog to digital conversion was achieved using a microcontroller ATMega16 (10-bit precision; Microchip Technology Inc.). Each digit in the recorded data represents ∼5 mV (5V/1023). The force was sampled at 500 Hz. The relationship between applied force and recorded voltage (i.e., the raw FSR force value) is monotonic but not linear (Figure 1A, right). During the testing, participants put the index finger of their dominant hand directly on the FSR (softly in contact). The index finger was always in contact with the FSR, resting very softly on the sensor before and after each keypress so that no force was detected by the FSR, even during the break between blocks. This contact with the FSR was most often below the FSR force detection threshold (around 0.3 N). It brought two advantages with the finger always being in contact with the device: (a) No auditory feedback from the keypress itself (i.e., physical vibrations) could be heard by participants, and (b) the relative position between the finger and the FSR did not change throughout the experiment, which made the FSR output comparable among all keypresses. In conditions with auditory feedback, the timing of auditory feedback was controlled by the FSR output. A sound was played after the FSR output reached a threshold value of 200 (∼0.87 N), which was referred to as the keypress start point. A keypress was made by briskly depressing the index finger. The measured delay between the keypress start and physical sound wave output was 60.6 msec (SD = 6.0; 100 sound measurements) and 161.3 msec (SD = 6.8; 100 sound measurements) for the intended 60- and 160-msec delay, respectively.

Participants were given a few practice keypresses (between four and eight keypresses) to get familiarized with the key (FSR) before data collection. During the practice, participants were instructed on the correct way to make a keypress. Participants always had to keep the finger in contact with the key, briskly depress the key, and then wait for about 2 sec before the next keypress. Each successful keypress was followed by a letter “k” on the screen immediately after the keypress start point (no visual feedback was given in the formal testing). At the end of the formal testing, participants were asked to make three additional strong keypresses. These three keypresses gave larger peak FSR output values than keypresses from any testing condition, indicating that the keypress force collected during the testing was in the working range of FSR. After the data collection, participants were asked if they realized anything wrong with respect to the sound. No one mentioned the delay of the sound.

For each keypress, the associated force trajectory and the peak force were extracted (see Figure 1A for an example). In our experimental setup, the data stream of FSR output was not available for controlling the auditory feedback until 1 sec after the start of each trial. Therefore, in conditions with auditory feedback, if a keypress was made immediately after the preceding keypress (within 1 sec), the intended timing manipulation of auditory feedback based on the FSR output cannot be achieved. These trials were excluded. Trials that had double peaks in the keypress force trajectory with at least 40 msec of raw force value lower than 200 between the two peaks were also excluded as there was a pause during the keypress, and the manipulation of auditory feedback timing may also not be accurate. For each participant, the number of excluded trials was calculated for each testing block. A testing block was excluded if more than 20% of trials (i.e., 10 trials) were excluded. A participant was excluded if no blocks remained after the block exclusion procedure for one or more auditory feedback conditions (Mix60, Mix160, Fix160, Fix60). In total, eight participants were excluded after this step, that is, 41 participants were included in the peak force analysis. Among the remaining participants, 13 blocks (3.2% of all blocks) were excluded from seven participants. On average, 96.3 (SD = 12.0) trials were left in the NoSound condition, 93.0 (SD = 18.1) trials in Mix60, 91.8 (SD = 16.6) trials in Mix160, 93.8 (SD = 15.4) trials in Fix60, and 97.3 (SD = 3.3) trials in Fix160. No significant difference in the number of trials was found between conditions (one-way within-subject ANOVA: F(4, 160) = 1.93, p = .14, η_p² = 0.05).

In the force trajectory analysis, the force trajectory of each trial was aligned to the force start point and averaged across trials. For each keypress included in the peak force analysis, the force start point was defined as the last time point with a force value of 0 within the 400-msec time window ending at the peak force point. If no points with a force value of 0 were found, the trial was excluded. Two more participants were excluded from further analysis as no more than 15 trials were left in at least one condition. The average number of trials included in the force trajectory analysis was 86.5 (SD = 22.9) in the NoSound condition, 88.1 (SD = 17.7) in Mix60, 84.8 (SD = 17.9) in Mix160, 85.7 (SD = 20.6) in Fix60, and 81.8 (SD = 23.8) in Fix160. There was no significant difference in the number of trials between conditions in the force trajectory analysis (one-way within-subject ANOVA: F(4, 152) = 0.83, p = .47, η_p² = 0.02).

All the results were reported with force values converted to Newton in the main text. However, qualitatively similar results were obtained from the data analysis with raw FSR force values. Because the keypress peak force was not normally distributed, robust statistical measurements based on randomization were used, and individual data were always presented for clarity. The data analysis was performed with MATLAB (The MathWorks Inc.). The p value of .05 was taken as the statistical significance cutoff, and corrections for multiple comparisons were made and mentioned where relevant.

To answer Question 1, the critical test was to compare the keypress peak force between Mix60 and Mix160. If a difference in the auditory feedback delay can lead to a difference in the peak force when the delay was not predictable, the difference in the peak force could only be because of an on-line modulation. We performed a two-level nonparametric test. First, a randomization test was made to test whether Mix160 had a higher peak force than Mix60 for each participant so that each participant had a p value (referred to as “p_rand” later). One-tailed test was used here because it has been shown that a long auditory feedback delay led to a higher keypress force than a short keypress delay (Neszmélyi & Horváth, 2018; Ruhm & Cooper, 1962). The rationale of the randomization test is that, if there are no differences between two conditions, values from the two conditions should be exchangeable. Therefore, an individual p_rand value was obtained in the following four steps: 1. Compare Mix160 and Mix60 using an unpaired t test, and keep the t value as the original t. 2. Take all the force values from Mix160 and Mix60, and randomize the order of those force values before assigning them back to the two conditions. Compare the two conditions after randomization, and get a t value as a randomized t. 3. Repeat Step 2 for 1,000 times so that there are 1,000 randomized t values. 4. Check the proportion of randomized t values that are bigger than the original t, which is then the individual p_rand value. We report all p_rand values as p_rand < 0.001 when p_rand value is 0 after 1,000 randomizations. Note that, in this case, a small p_rand value indicates that Mix160 is higher than Mix60, and a large p_rand value indicates the opposite. Second, the group level p_rand value for Mix160 having higher peak force than Mix60 was obtained by comparing the individual p_rand values to 0.5 (chance level) with the randomization test. The procedure was similar to the procedure of getting individual p_rand values except that the randomization only took place within participants, that is, the condition relabeling was performed for each participant separately so that each participant always had the same pair of data points. The same randomization test was also used to make comparisons between, for example, Fix160 and Fix60 (see the Results section).

To answer Question 2, the average keypress force trajectory was analyzed. This analysis allows us to examine the early phase of motor output that is free from the influence of sensory feedback. The force trajectory of each keypress was selected from 200 msec before the force start point to 800 msec after. Within-subject two-tailed t tests were made on the average force trajectory at each time point (500 time points in total) of the force trajectory between Mix60 and Mix160, between Fix60 and Fix160, between the average of Mix60 and Mix160 (“Mix” in Figure 3A legend) and the average of Fix60 and Fix160 (“Fix” in Figure 3A legend), and between the NoSound condition and the average of the four conditions with auditory feedback (“WithSound” in Figure 3A legend). Multiple comparisons over time points were corrected for each between-condition comparison using a cluster based permutation method with a cutoff p value at .05 (Maris & Oostenveld, 2007).

The original data and MATLAB analysis code are freely available on-line at: doi.org/10.6084/m9.figshare.12146271.v1.

The manipulation of the auditory feedback from keypress had a significant influence on the keypress (Figure 2, left). Consistent with previous studies on the effect of delayed auditory feedback, all the four conditions in which an auditory feedback followed the keypress had significantly weaker peak force than the condition in which no auditory feedback was provided (all the four between-condition comparisons had p_rand < 0.001 with one-tailed randomization test; NoSound: mean = 5.56 N, SD = 6.21 N; Mix60: mean = 2.07 N, SD = 3.05 N; Mix160: mean = 2.19 N, SD = 3.34 N; Fix60: mean = 1.49 N, SD = 1.17 N; Fix160: mean = 1.79 N, SD = 1.05 N). A long fixed delay (Fix160) led to higher keypress peak force than a short fixed delay (Fix60; p_rand < 0.001, one-tailed randomization test; Figure 2, middle).

To test the effect of an on-line modulation of auditory feedback on the keypress action, a comparison of peak force was made between the 60-msec delayed sound and the 160-msec delayed sound when both sounds were presented in a random order (i.e., unpredictable). Indeed, a 160-msec delayed sound still led to a stronger keypress force than a 60-msec delayed sound in this case (Mix160 > Mix60, p_rand < 0.001, one-tailed randomization test; Figure 2, right), which showed that auditory feedback could influence the triggering action on the fly. Excluding the two participants showing a large difference between Mix160 and Mix60 (Figure 2, right) had no influence on the statistical results. Similar results were obtained with the data analysis on raw force values. However, the on-line modulation from auditory feedback can only occur if the auditory feedback arrived at a time point earlier than the peak force. Therefore, a sanity check was made. The mean duration between the keypress start and the peak force point (Figure 1A) was 95.53 msec (Mix60) and 108.29 msec (Mix160; 176.45 msec in NoSound, 83.21 msec in Fix60, and 121.25 msec in Fix160). The 95.53-msec delay of peak force in the Mix60 condition may seem a bit short to be influenced by a 60-msec delayed sound (35.53-msec reaction latency from sound onset to motor response), but it was impossible for a 160-msec delayed sound to have a direct influence on the peak force at 108.29 msec in the Mix160 condition. This peak force delay was an average of all the 41 participants included in the data analysis, and not everyone showed an on-line modulation effect. If the participants showing a strong on-line adjustment effect were considered (p_rand value smaller than 0.05; n = 7), the average peak force delay was 133.02 msec in the Mix60 and 183.01 msec in the Mix160. If participants were grouped based on the p_rand value, half of the participants with small p_rand values (p_rand < 0.3; n = 20) had an average peak force delay of 112.84 msec in the Mix60 and 133.28 msec in the Mix160; the other half with large p_rand values (p_rand > 0.3; n = 21) had an average peak force delay of 79.03 msec in the Mix60 and 84.49 msec in the Mix160. Therefore, there is enough time for a 60-msec delayed sound to exert its on-line modulation effect on motor output. The individual difference in the on-line modulation effect strongly depends on the idiosyncratic keypress behavior.

The modulation effect of delayed auditory feedback on peak force was stronger when the delay was predictable (Fix160-Fix60; mean = 0.30 N; SD = 0.46 N) than when the delay was not predictable (Mix160-Mix60; mean = 0.13 N; SD = 0.38 N, p_rand = 0.04, two-tailed randomization test). This suggests that there are other sources of modulation in the Fix conditions besides on-line modulation. We tested the idea that the extra modulation effect might even be detectable in the early phase of the keypress force trajectory beyond the influence of any sensory feedback. To do this, single trial keypress force trajectories were aligned to the force start point (Figure 1A, middle), averaged like an event-related-potential study in electroencephalogram, and compared between conditions (Figure 3A, B). Interestingly, significant differences in the average force trajectory were found between the Fix60 and the Fix160 conditions in two clusters. In the first cluster (starting at 20 msec after the force start point), the Fix60 condition had a higher force than the Fix160 condition (see Figure 3B for a difference plot). However, individual results suggested that this pattern was not stable as more than 45% of participants showed the effect in the other direction (Figure 3C). Furthermore, there was no such cluster when the data analysis was performed with the raw force values. The second cluster (starting at 72 msec) showed a reliable pattern that the Fix160 condition had a higher force than the Fix60 condition (38 of 39 participants showed results in this direction; Figure 3D). Because the starting point of a cluster cannot be taken as the point where statistical difference emerges (see Sassenhagen & Draschkow, 2019), the Fix60 and Fix160 conditions were compared with a within-subject t test at each time point in the entire time window. This was followed by a false discovery rate adjustment. The first significant point from this analysis was at 80 msec, which was much earlier than the delivery of the 60-msec delayed auditory feedback. The average delivery time of the 60-msec delayed sound calculated from the force start point was at about 117 msec (the average sound delivery time was marked with triangles in Figure 3A, B). Therefore, the difference in force trajectory between the Fix60 and Fix160 conditions started much before the sound delivery and cannot be explained by auditory feedback. The early difference can only be because of different motor commands.

The same comparison between the Mix60 and Mix160 conditions led to a significant cluster in the time window between 182 and 520 msec. The earliest difference point as identified through a whole time window comparison followed by a false discovery rate adjustment was at 192 msec, that is, after the delivery of the 60-msec delayed sound. This difference can be interpreted as a feedback modulation, with a short delayed sound leading to a weaker force as compared to a long delayed sound. More strikingly, when the average force trajectory of the Mix conditions (Mix60, Mix160) was compared to the average force trajectory of the Fix conditions (Fix60, Fix160), a significant cluster was found between 4 and 114 msec, that is, in the very early stage of the keypress action. As can be seen on the right of Figure 3A, force trajectories from the conditions with auditory feedback formed two bundles in the early phase. The first bundle, with a shallow slope, was formed by the keypresses with predictable feedback timing (Fix60, Fix160). The second bundle, with a steep slope, was formed by the keypresses with unpredictable feedback timing (Mix60, Mix160). Similar results were obtained with the data analysis on raw force values. Results from the force speed (slope of force value over time) and force acceleration (slope of force speed over time) analysis also showed a similar pattern, that the Fix conditions had lower speed and acceleration than the Mix conditions in the very beginning from the keypress force start (results not shown). When there was no auditory feedback (NoSound condition), the keypress force was higher than when the auditory feedback was provided, across the entire analysis time window.

The human motor system generates actions, which, within given task constraints, are optimized with respect to certain variables such as effort, comfort, or variability (Todorov, 2004). When interacting with force sensitive devices, where only the exerted force matters, it seems natural that the motor system aims to reduce unnecessary forces and, thus, minimize metabolic effort. It has been shown that the latency and the availability of auditory feedback plays an important role in regulating the motor output during a keypress (Cao et al., 2020; Neszmélyi & Horváth, 2018; van Vugt & Tillmann, 2015; Pfordresher & Dalla Bella, 2011; Chase et al., 1959, 1961; Kalmus et al., 1955). In the current study, we showed that auditory feedback can exert an on-line modulation on keypress behavior before the keypress is complete. We also showed that predictable delays in auditory feedback can lead to a change in motor program.

On-line control suggests that force output is increased until extra sensory feedback, which signals that enough force has been exerted, comes in. This process was clearly confirmed by demonstrating that the keypress peak force was reduced with early (60 msec) compared to late (160 msec) auditory feedback when both were equally unpredictable. In motor control studies, there is an ongoing debate on how quick sensory feedback can influence the motor output. For example, human-goal-directed movements (e.g., pointing to an object with the finger: Elliott, Helsen, & Chua, 2001; Woodworth, 1899) probably require a latency of at least 100 msec before a visual signal can influence the motor output (Elliott et al., 2001). In the current study, the average latency from the onset of auditory feedback to the time point of keypress peak force, where a significant modulation from auditory feedback was found, was 73 msec (133–60) in the Mix60 condition for the seven participants showing a strong on-line modulation effect. This suggests that the latency for an on-line modulation of motor output from sensory feedback may be overestimated, at least in the auditory domain. Interestingly, our results are consistent with reports of fast orienting response to visual stimulus. Neck muscles in monkeys (Corneil & Munoz, 2014; Corneil, Olivier, & Munoz, 2004) and arm muscles in humans (Pruszynski et al., 2010) show a very short latency response to visual stimuli (55–95 msec in monkeys). A very recent study in monkeys also showed that at least for the initiation of smooth pursuit eye movements, a latency as short as 50 msec was enough for a visual signal to show an impact on the oculomotor output (Buonocore, Skinner, & Hafed, 2019). The current study, to the best of our knowledge, is the first to demonstrate that a self-generated auditory feedback can have an almost comparably fast influence on motor output (finger movement).

The motor program change was observed in the early phase of keypress force trajectory. A long predictable auditory feedback delay (160 msec) led to a steeper increase of keypress force than a short predictable auditory feedback delay (60 msec) in a time window far earlier than the arrival of auditory feedback. Therefore, the early force difference cannot be explained by sensory feedback and can only be explained by a change in the motor program. The motor program change is related to the predictability of the auditory feedback delay. Supporting this, the early force difference was not observed when the auditory feedback delay was not predictable (Mix60 vs. Mix160). There may be at least two explanations for the motor program change. One explanation is that the effect from on-line modulation accumulates, and the motor program change is a passive adaptation to the auditory feedback delay. According to this explanation, the order of the steepness of early force trajectory should be (from high to low): Fix160, Mix160, Mix60, Fix60. This is because a long feedback delay leads to a higher force than a short feedback delay, and a higher force should be associated with a steeper slope (see the force trajectory of NoSound condition; Figure 3A). Fix160 had a 160-msec feedback delay in all trials and should have the steepest slope. Mix160 also received a 160-msec feedback delay in all trials but was interrupted by some 60-msec feedback trials in between, so it should have the second steepest slope. Following the same rationale, it may not be difficult to figure out that Mix60 and Fix60 should have the third and the fourth steepest slope, respectively. However, the results clearly showed that the force trajectories of both the Mix160 and Mix60 conditions were above the Fix160 and Fix60 conditions (Figure 3A, right). Furthermore, the average of the Mix160 and Mix60 conditions had a statistically significant steeper slope than the average of the Fix160 and Fix60 conditions in the time window between 4 and 114 msec from the keypress force start. Therefore, an explanation of passive motor adaptation is highly unlikely for the early keypress force slope change (note that we do not exclude the possibility that a passive motor adaptation may exist). The other explanation is that the motor program change is a result of an active motor adaptation, which involves the prediction of the auditory feedback. This explanation is consistent with neurophysiological findings that the brain can learn the auditory feedback delay (Elijah, Le Pelley, & Whitford, 2018; Cao, Veniero, Thut, & Gross, 2017) and that the prediction of sensory feedback is crucial to motor control (Kunde, Koch, & Hoffmann, 2004; Flanagan et al., 2003; Wolpert & Flanagan, 2001). Therefore, the change in the early phase of keypress force trajectory may well be a sign of an updated motor prediction.

The approach of averaging force trajectories over trials is not without caveats. For example, the difference between the Mix60 and Mix160 conditions in the force trajectory only showed up from 182 msec, which was after the peak of the average force trajectory. It was already shown that the peak force was different between the two conditions from previous analyses. Thus, trial averaging smears the timing information of between-condition differences because of an intertrial variation in the force trajectory. However, the early component of the force trajectory can be considered as a realization of the motor command behind, and trial-averaging should work to reveal the signal in a way similar to the ERP technique in human neuroimaging studies. There are surely other approaches to the analysis of force profiles (e.g., looking at the skewness and kurtosis of each keypress, see Ulrich, Rinkenauer, & Miller, 1998). The trial-averaging approach, as we discussed, is theoretically sound for the purpose of the current study. We conclude that the force profile difference between the Fix60 and Fix160 conditions, between the average of the Mix conditions and the average of the Fix conditions, in the very early time window clearly indicates an open-loop component of motor control, whereas the force profile difference between the Mix60 and Mix160 conditions should be interpreted as a sign of a closed-loop control (cf. Wing, 1977).

Overall, our findings suggest that the prediction of the auditory feedback changes the motor program, which indicates that the prediction of sensory feedback is crucial to motor control. At the same time, unpredictable auditory feedback can lead to an on-line modulation of motor output at a very low latency (∼70 msec).

We thank Normann Mangold for technical support, Lisa von Boros for assistance with data collection, Michael Steinborn for helpful comments on an earlier draft, and Victoria Nicholls for English editing. L. C. and B. H. were supported by the European Research Council (grant 677819, awarded to B. H.).

Reprint requests should be sent to Liyu Cao, Department of Psychology (III), Julius-Maximilians-Universität Würzburg, Röntgenring 11, 97070, Würzburg, Germany, or via e-mail: [email protected].