## Abstract

According to the neuromuscular model of virtual trajectory control, the postures and movements of limbs are performed by shifting the equilibrium positions determined by agonist and antagonist muscle activities. In this study, we develop virtual trajectory control for the reaching movements of a multi-joint arm, introducing a proportional-derivative feedback control scheme. In virtual trajectory control, it is crucial to design a suitable virtual trajectory such that the desired trajectory can be realized. To this end, we propose an algorithm for updating virtual trajectories in repetitive control, which can be regarded as a Newton-like method in a function space. In our repetitive control, the virtual trajectory is corrected without explicit calculation of the arm dynamics, and the actual trajectory converges to the desired trajectory. Using computer simulations, we assessed the proposed repetitive control for the trajectory tracking of a two-link arm. Our results confirmed that when the feedback gains were reasonably high and the sampling time was sufficiently small, the virtual trajectory was adequately updated, and the desired trajectory was almost achieved within approximately 10 iterative trials. We also propose a method for modifying the virtual trajectory to ensure that the formation of the actual trajectory is identical even when the feedback gains are changed. This modification method makes it possible to execute flexible control, in which the feedback gains are effectively altered according to motion tasks.

## 1 Introduction

In the neural control field of limb movements, there is a certain concept that focuses on the viscoelasticity of muscles, suggesting that the central nervous system adjusts the activity of motoneurons to specify the equilibrium point at which the extensor and flexor muscles of a joint are balanced. In other words, muscles have properties similar to springs, and the viscoelasticity and equilibrium point are determined by motoneuron activities, which makes it possible to realize various postures and movements.

Polit and Bizzi (1979) conducted experiments on the elbow joint movements of rhesus monkeys and inferred that a monkey's central nervous system only specifies the final equilibrium position of the movement by adjusting the viscoelasticity of the flexor and extensor muscles to form a trajectory toward a target. In addition, Bizzi, Accornero, Chapple, and Hogan (1984) executed experiments on the elbow joint movements of deafferented monkeys and suggested that the activation pattern of motoneurons defines not only the final equilibrium position but also the time series of equilibrium positions over the entire movement. This time series of equilibrium positions has been called, for example, an *equilibrium trajectory*, *equilibrium-point trajectory*, and *virtual trajectory*. Hogan (1984) presented a mathematical model of virtual trajectory control, in which driving torque is generated by muscular contraction corresponding to the difference between the arm position and the equilibrium position at every moment. According to this control model, the virtual trajectory is specified by the neural commands, and then the limb moves while tracking the equilibrium positions owing to the viscoelasticity of the musculoskeletal system; therefore, it is not necessary to calculate the complex dynamics of the limb. Flash (1987) applied the equilibrium trajectory control to multi-joint arm movements and discussed the trajectory formation of human reaching movements. Flash then concluded that the hand trajectories observed in human reaching movements could be reproduced from the straight equilibrium trajectories defined by the minimum jerk criterion (Flash & Hogan, 1985). Gomi and Kawato (1996) experimentally measured the perturbation for external forces during human reaching movements and estimated the arm stiffness whose value was considerably lower than that in Flash's (1987) simulation. Furthermore, they demonstrated that the equilibrium-point trajectories calculated for the estimated low stiffness were complex and far from the actual straight trajectories. Their experimental results suggest that the smooth trajectory of a human hand cannot be formed by simply specifying the equilibrium-point trajectory as a copy of the desired trajectory.

The virtual trajectory needs to be designed in such a way that the actual trajectory formed by tracking the sequence of equilibrium positions matches the desired trajectory. How can such a suitable virtual trajectory be obtained? Shadmehr (1990) and Katayama and Kawato (1993) proposed learning schemes to obtain such suitable virtual trajectories. However, their learning schemes require an enormous amount of training, and it might be difficult to effectively respond to the change of the stiffness that is characterized by the spring-like behavior of agonist and antagonist muscles. In particular, if the stiffness is relatively low, it is more delicate to achieve the desired movement because the required virtual trajectory becomes more complex. A more significant problem is that the relation between the virtual trajectory and the actual trajectory depends on the dynamics of the arm, which means that the procedure to get the suitable virtual trajectory includes the calculation of the arm dynamics. It should be noted that the original scheme of the equilibrium trajectory control has an advantage of operating without calculating the arm dynamics.

The traditional model of the equilibrium trajectory control for musculoskeletal systems is considered to be similar to feedback control, where the feedback gains are determined by the muscle stiffness. In this study, we develop the virtual trajectory control from the viewpoint of control engineering rather than the control model of living things. We aim at improving control performance and the expansion of mechanical application. In this development, a more practical way for obtaining proper virtual trajectories is required to succeed in trajectory tracking. We propose a repetitive control method in which the virtual trajectory is corrected at every trial according to the information of the trajectory error. This correction is advisable to be executed without explicit calculation of the arm dynamics. It is expected that in the repetitive control, the virtual trajectory derives the desired trajectory after several corrections. The effectiveness of the proposed repetitive control is examined for the trajectory tracking of multijoint arm movements using computer simulations.

The actual trajectory that is guided by a virtual trajectory depends on the feedback gains. For example, if low gains are set, the actual trajectory tends to be far from the virtual trajectory. Naturally, the virtual trajectory required to derive a desired trajectory must be adjusted according to the values of the feedback gains. We investigate how the virtual trajectory should be modified so that the identical actual trajectory is formed even when the feedback gains are changed.

## 2 Virtual Trajectory Control

*reference trajectory*in control engineering, is given for a $k$-joint arm. In virtual trajectory control, once a virtual trajectory is specified, an actual trajectory is formed according to the arm dynamics. Therefore, the critical point of this control scheme is how to find a virtual trajectory $xv*$ required to form an actual trajectory $x$ that coincides with the desired trajectory $x*$. If the arm dynamics $f(\theta ,\theta \u02d9,\theta \xa8)$ is already known, the required virtual trajectory $xv*$ can be calculated as follows. Consider that the arm movement starts at $t0$ and ends at $tN$, and variables are discretized by infinitesimal time $\Delta t=ti-ti-1$ ($i=1,2,\u2026,N$). Then the hand position at $ti$ is denoted by $x(ti)$, and its velocity $x\u02d9(ti)$ is defined by the backward difference of the position, such that

Consider that the arm dynamics and kinematics are known, and the time series ${x*(t0),x*(t1),\u2026,x*(tN)}$ of the desired hand trajectory is specified. If the initial value $xv*(t0)$ of the virtual trajectory is defined, the complete required virtual trajectory is obtained as follows: $xv*(t1)$ is calculated from $xv*(t0)$, and $xv*(t2),\u2026,xv*(tN-1)$, and $xv*(tN)$ are sequentially calculated in accordance with equation 2.4. Therefore, the required virtual trajectory ${xv*(t):t0\u2264t\u2264tN}$ that derives a desired trajectory ${x*(t):t0\u2264t\u2264tN}$ depends on the choice of initial values. It should be noted that there exist multiple virtual trajectories deriving the identical actual trajectory.

Although the hand virtual trajectory is considered in our description, the virtual trajectory control for joint angles can be formalized in a similar manner; this formalization is easier because the calculation concerning Jacobian is unnecessary.

## 3 Repetitive Control for the Correction of Virtual Trajectories

If the dynamics of a controlled object is known, it is possible to calculate the required virtual trajectory $xv*$ that realizes the desired trajectory $x*$ according to equation 2.4. However, the details of the arm dynamics and the exact values of its physical parameters are unknown in many cases. In this section, we propose a repetitive control method to achieve the desired trajectory by correcting a virtual trajectory at every motion trial without calculating the arm dynamics.

^{1}Equation 3.1 expresses the error of the actual trajectory. In order to find $xv*$ satisfying $\Psi (xv*)=0$, Newton's method is applied in the function space, where its recurrence formula is generally expressed by

^{2}of equation 3.4. Furthermore, if the virtual trajectory $xv(n)$ is close to the required virtual trajectory $xv*$, the difference between ${JT(\theta *(ti))}#f(\theta *(ti),\theta \u02d9*(ti),\theta \xa8*(ti))$ and ${JT(\theta (n)(ti))}#f(\theta (n)(ti),\theta \u02d9(n)(ti),\theta \xa8(n)(ti))$ will be small. Based on the above three assumptions, the third term of the right-hand side of equation 3.4 becomes relatively small and can be neglected. Thus, equation 3.4 can be approximated by

This theoretical derivation is appropriate in the range of the neighborhood of the exact solution $xv*(t)$. Therefore, the repetitive control requires that the first virtual trajectory $xv(1)(t)$ is near the required virtual trajcectory $xv*(t)$ to avoid divergence. From this perspective, the desired trajectory $x*(t)$ is selected as the first virtual trajectory $xv(1)(t)$. Namely, the control in the first trial of the repetitive control is the same as the conventional feedback control in which general trajectory tracking is performed.

^{3}The proposed update rule that is obtained by neglecting the term on the arm dynamics is extremely concise. However, it should be noted that this simplification is applicable under the three conditions of high gains, fine sampling, and neighborhood of the solution $xv*(t)$. Figure 1 shows the schematic of the virtual trajectory correction by the proposed repetitive control.

Incidentally, it is easy to derive the algorithm for updating virtual trajectories for simple P-control (see equation 2.2), which is given by equation 3.9, as well as PD-control. Although the update rules are the same, the performance of iterative control is not the same between for P-control and PD-control because the required virtual trajectories are significantly different.

## 4 Virtual Trajectory Modification for Gain Change

When the feedback gains are changed, the actual trajectory naturally changes in the virtual trajectory control. In this section, we investigate how the virtual trajectory should be modified for the gain change in order to obtain the same actual trajectory as that before the gain change.

From the above consideration, when the feedback gains are changed by equation 4.1 for any constant $\alpha >0$, the corresponding virtual trajectory can be obtained by simple calculation (see equation 4.5), and the same actual trajectory can be achieved as before the change. Accordingly, once a virtual trajectory realizing the desired trajectory for a set of gains is found, it is easy to adjust the virtual trajectory so that the identical desired trajectory is achieved for constant multiplied gains corresponding to a motion task.

## 5 Numerical Experiments

Parameters . | Link 1 . | Link 2 . |
---|---|---|

Mass: $mi$ (kg) | 1.680 | 1.644 |

Length: $li$ (m) | 0.325 | 0.367 |

COM: $si$ (m) | 0.1417 | 0.2503 |

Inertia: $Ii$ (kg$\xb7m2$) | 0.0522 | 0.1475 |

Viscosity: $bi$ (kg$\xb7m2$/s) | 0.2 | 0.2 |

Parameters . | Link 1 . | Link 2 . |
---|---|---|

Mass: $mi$ (kg) | 1.680 | 1.644 |

Length: $li$ (m) | 0.325 | 0.367 |

COM: $si$ (m) | 0.1417 | 0.2503 |

Inertia: $Ii$ (kg$\xb7m2$) | 0.0522 | 0.1475 |

Viscosity: $bi$ (kg$\xb7m2$/s) | 0.2 | 0.2 |

### 5.1 Redundancy of the Required Virtual Trajectory

As described in section 2, when the dynamics of the two-link arm (see equation 5.1) and the physical parameters (see Table 1) are known, it is possible to calculate a suitable virtual trajectory that leads the hand to the desired minimum jerk trajectory.

### 5.2 Repetitive Virtual Trajectory Control

Assuming that the dynamics of the two-link arm was unknown, we simulated the repetitive control for two cases of different feedback gains: low gains ($Kp=30E,Kd=10E$) and high gains ($Kp=150E,Kd=50E$), where $E=diag(1,1)$. Virtual trajectories were corrected 10 times for the cases of respective gains, according to the update algorithm in equation 3.9 with the reduction coefficient $\u025b=0.3$.

As we have described, we performed numerical experiments on the repetitive control for a point-to-point reaching movement with moderate speed. Additional experiments were tried for three types of movements.

#### 5.2.1 Reaching Movements with Different Speed

#### 5.2.2 A Wider Range of Reaching Movements

#### 5.2.3 A Complex Curved Movement

Thus, we found some typical performance in the numerical experiments on the proposed repetitive control for reaching and drawing movements of the two-link arm. In the first trial, the virtual trajectory was the same as the desired trajectory, while the actual trajectory was somewhat different from the desired trajectory. Thereafter, as the virtual trajectory control was repeated, the virtual trajectory deviated from the desired trajectory by degrees, while the actual trajectory gradually got closer to the desired trajectory.

### 5.3 Gain Change and Virtual Trajectory Modification

Incidentally, the virtual trajectory (see the red solid lines in the upper row of Figure 17B) obtained by the combination of the repetitive control and the gain change was somewhat different from the required virtual trajectory (see the red dotted lines in the upper row of Figure 3A), which was obtained by the calculation in section 5.1. It was indicated that even if the virtual trajectories have some differences, they derive almost the same actual trajectories.

## 6 Conclusion

Gaining insights from the equilibrium position control model for living things, we have proposed the repetitive control method based on virtual trajectories and examined the performance of trajectory tracking for a two-link arm. It was confirmed by computer simulations that the virtual trajectory was adequately corrected and the actual trajectory got closer to the desired trajectory, according to the proposed algorithm of the repetitive control. Our repetitive control is applicable under the three conditions of high gains, small sampling time, and neighborhood of a solution. When the high gains are set, the required virtual trajectory can be near the desired trajectory, and the actual trajectory formed in the first trial is relatively close to the desired trajectory. Accordingly, the control trials are repeated around the neighborhood of the solution. That is, the high gains often ensure that the iterative virtual trajectories are close to the required virtual trajectory. In our computer simulations, when the feedback gains were reasonably high, the actual trajectory almost agreed with the desired trajectories within approximately 10 iterative trials. In addition, we have proposed the method of virtual trajectory modification and confirmed that when the feedback gains were changed, the virtual trajectory could be modified by simple calculation such that the actual trajectory was not changed.

Virtual trajectory control was originally proposed in the 1980s as equilibrium position control, which can be regarded as P-control. Katayama and Kawato (1993) indicated from their computer simulation of a muscular skeletal model of a human arm that virtual trajectories based on equilibrium-point control were much more complex than actual trajectories. For our virtual trajectory control based on a P-control scheme, if relatively low gains are set, a rather complicated virtual trajectory is required to derive a straight actual trajectory, as shown in Figure 4. Therefore, it is not so easy in a P-control scheme to find the required virtual trajectory even for ordinary arm movements; the trajectories of its repetitive control sometimes diverge, as shown in Figures 9 and 10. The basic form of the original equilibrium-point control is adequate for relatively slow movements. McIntyre and Bizzi (1993) modified the basic equilibrium-point model for applying the model to faster movements. Considering a servomechanism of the musculoskeletal system for a single joint, they proposed an equilibrium-point control model with reflex feedback. In their model, the virtual trajectory is modified by reflex feedback, which is related to the use of velocity and position signals. Accordingly, the model modified by McIntyre and Bizzi (1993) can be regarded as PD-control. In our letter, we also adopted PD-control, but we focused on virtual trajectories for multi-joint arm movements. We have developed virtual trajectory control based on a PD-control scheme and observed that the required virtual trajectory did not largely deviate from the actual trajectory. It is conjectured that if such a PD-control scheme is applied to the trajectory formation in the simulation of Katayama and Kawato (1993), the required virtual trajectory might be relatively smooth and controllable.

The feedback gain and the virtual trajectory are complementary to each other in virtual trajectory control. This complementary relation provides useful strategies for motion control. When the feedback gains have to be set relatively low owing to some conditions, it is sometimes difficult to achieve the desired trajectory by more than 10 trials in the repetitive control. In such a case, the following two-step control technique is useful. First, setting high gains by prescribing a constant multiplication of the low gains, the required virtual trajectory that derives the desired trajectory is obtained by several iterations of the repetitive control. Then, the high gains are returned to the previous low gains and the virtual trajectory modification (see equation 4.5) is applied. We confirmed in section 5.3 (see Figure 17) that the virtual trajectory control was able to achieve the desired trajectory by using the above way even when the gains were considerably low. A similar strategy might be adopted in human arm movements. That is, for inexperienced motion tasks, human movements are firmly performed with the high stiffness of limbs by the coactivation of agonist and antagonist muscles in the beginning. However, such high stiffness brings an increase in the consumption of metabolic energy and occasionally causes unstable control by an excessive feedback signal. After the skill of the motion task is acquired by some training, the movements are comfortably performed with the lower stiffness by the moderate activation of muscles. For example, recording the electromyography of shoulder and elbow muscles in human pointing tasks, Gribble, Mullin, Cothros, and Mattar (2003) observed that the co-contraction levels of muscles decreased over time, which means that limb stiffness was reduced after sufficient practice.

To date, there have been no research reports suggesting that the proposed repetitive control and gain change in this letter have been used in living things. Assuming that the proposed control model is implemented in the brain, it is conjectured that the mechanism of the repetitive virtual trajectory control is different from that of the original equilibrium-point control hypothesis. Including the spring-like muscle properties and peripheral neural feedback loops, multiple neural feedback loops going around the brain stem, cerebellum, cerebral cortex, and so on should be investigated in order to elucidate the mechanism of virtual trajectory control with repetitive control. The virtual trajectory control that we have developed in this letter deviates from the concept of the original equilibrium trajectory control. From the viewpoint of control, the term *guidance trajectory* might be more appropriate rather than the term *virtual trajectory* for trajectory tracking.

Although our control model assumes that the desired trajectory is given in advance, some computational approaches do not need the desired trajectory. For example, Todorov and Jordan (2002) proposed a stochastic framework of optimal feedback control, in which the movement trajectory is produced through the state estimation by a Kalman filter and the calculation of control signals by the state feedback. It is an advanced problem to develop virtual trajectory control for trajectory formation without a reference trajectory. This development might be possible by representing the virtual trajectory in terms of several via-points and by repeating the correction of these via-points to reduce the target error according to some criteria.

Future work will focus on more theoretical advances and control experiments. First, improving the repetitive control method based on a Newton-like method is an important subject of study. We obtained the update rule 3.9 of virtual trajectories by replacing the Fréchet derivative in Newton's method with an identity mapping. However, this simplification occasionally causes worse convergence of iteration. Replacing the Fréchet derivative with a computable linear operator that can properly reflect the arm dynamics is desirable. Second, that the proposed control method is applied to more complicated motion tasks with force control is promising. For example, in order to move a hand pushing a wall with specified force, it could be effective to shift the virtual trajectory in the direction of the force. The adaptation of the control model to force-field perturbation is also an attractive research subject. Developing an iterative learning scheme based on adaptive dynamic programming, Jiang and Jiang (2014) reproduced trial-by-trial trajectories, as observed in experiments of human reaching movements in viscous force fields. Similarly, the repetitive virtual trajectory control might be applied to multi-joint arm movements in various force fields without the knowledge of environments. In addition, the proposed virtual trajectory control can benefit the applications of control engineering (e.g., robot control and vehicle control) because the proposed methods provide simple and useful control.

## Notes

^{1}

Here $x$ and $xv$ express trajectories (functions), not positions.

^{2}

The coefficient of the first term is the identity matrix and the coefficient of the second term is roughly the identity matrix, if the sampling time is sufficiently small.

^{3}

An iteration method derived from Newton's method by replacing the gradient of a function for variation with a computable simple linear operator is called a Newton-like method.

## Acknowledgments

We thank the editor and anonymous reviewers for their constructive comments that were helpful for improving the presentation of our letter.