Emergence of diverse behaviors from interactions between nonlinear oscillator complex networks and a musculoskeletal system

To understand the relationship between brain structure and behavior in the general movements of fetuses and infants from a complex systems perspective, we investigated how behaviors emerge from interactions between complex networks of nonlinear oscillators and musculoskeletal bodies. We prepared a snake-like robot and some network structures in a physical simulator. The various conditions imposed on the networks were (a) no connection among oscillators, (b) scalefree network, (c) one-dimensional lattice, (d) small-world network, and (e) random network. In the experiments, the robot exhibited multiple crawling and bending behaviors. By estimating the numbers of behavioral attractors, we revealed a qualitative difference between the scale-free network and other complex networks. Introduction Animals and humans exhibit various adaptive behaviors. These behaviors likely emerge from complex interactions among environment, body dynamics, and brain activities, rather than from any single factor, posing several questions. How do the behaviors emerge? What underlying structure shapes such a complex and adaptive interaction? Do diverse interactive behaviors among systems emerge under any specific condition? To answer these questions, we must understand the relationship between structure and functionality in biological entities from a complex systems perspective. Kuniyoshi and Suzuki (2004) proposed a model in which adaptive behaviors emerge through body constraint as a chaotic coupled field. In this model, on the basis of coupled map lattices and globally coupled maps (Kaneko and Tsuda (2003)), transitional adaptive behaviors should emerge as chaotic itinerancy among behavioral attractors. The model connects each chaotic element (logistic map) to a single linear actuator (muscle); that is, the actuator receives the chaotic elements output as a motor command and sends the outputs of its length sensors back to the chaotic element. The body structure behaves as a coupled field of deterministic chaotic activities. However, when the model is executed, the number of emerged behaviors is less than expected, indicating that chaotic itinerancy was suppressed. Sensor


Introduction
Animals and humans exhibit various adaptive behaviors.These behaviors likely emerge from complex interactions among environment, body dynamics, and brain activities, rather than from any single factor, posing several questions.How do the behaviors emerge?What underlying structure shapes such a complex and adaptive interaction?Do diverse interactive behaviors among systems emerge under any specific condition?To answer these questions, we must understand the relationship between structure and functionality in biological entities from a complex systems perspective.Kuniyoshi and Suzuki (2004) proposed a model in which adaptive behaviors emerge through body constraint as a chaotic coupled field.In this model, on the basis of coupled map lattices and globally coupled maps (Kaneko and Tsuda (2003)), transitional adaptive behaviors should emerge as chaotic itinerancy among behavioral attractors.The model connects each chaotic element (logistic map) to a single linear actuator (muscle); that is, the actuator receives the chaotic elements output as a motor command and sends the outputs of its length sensors back to the chaotic element.The body structure behaves as a coupled field of deterministic chaotic activities.However, when the model is executed, the number of emerged behaviors is less than expected, indicating that chaotic itinerancy was suppressed.Complex network structures in the human brain are regarded as either small-world networks (Watts and Strogatz (1998)) or Scale-free networks (Barabasi and Albert (1999)).We are interested in the functional role of such brain structures.This paper proposes a first step for investigating the emergence of organisms versatile behaviors by a constructive approach.To this end, we constructed a simulator built from complex nonlinear oscillator networks and a snake-like musculoskeletal body model.We analyzed and compared the emergent behaviors from the coupled dynamics of the musculoskeletal body and complex network structures.

General movements of fetuses and infants
General movements, defined as purposeless whole-body movements, characterize the movements of fetuses and infants.The movements begin at about eight weeks gestation, and continue to evolve in healthy infants after birth.Within the first two months of birth, infants develop writhing movements, characterized by small-to-moderate amplitude and slow-to-moderate speed Prechtl (2001), although the arms may move rapidly and largely at this stage.After two months, fidgety movements appear, which are characterized by simultaneous variable acceleration of all moving parts of the body with other gross movements.After about 20 weeks, infants gradually develop voluntary movements.
Such movements are considered to be generated by the medulla oblongata in the brain stem.Anencephalic infants, who lack the neocortex, can move their whole body roughly and rapidly.Additionally, since the quality of general movements is related to white matter, the neocortex might be responsible for movement control Spittle et al. (2008).

Brain structure and complex network theory
What kind of structure exists in the brain?This question has been answered by diffusion tensor imaging (DTI), which visualizes the nervous fiber connectivity in white matter.The brain structure is a variant of small-world networks, known as a rich-club network (van den Hauvel and Sporns (2011)).
A small-world network is one in which any two nodes are connected through very few interceding nodes, compared to the network size.Mathematically, in a small-world network, the average minimum distance d ji between two nodes i and j is proportional to the logarithm of the total number of nodes N in the network.Thus, the small-world network satisfies (Watts and Strogatz (1998)) (1) Scale-free networks are frequently encountered in complex network theory.In this type of network, connections are concentrated among several nodes, and the majority of nodes have only a few connections.Scale-free networks are characterized by the power law where k is the number of edges from a node, P (k) is the probability of k, and γ is a constant.These networks are considered to be controlled by a small number of hub nodes.

Brain rhythms and nonlinear oscillators
Brain activities manifest as brain waves of certain frequencies, which are synchronized or desynchronized depending on internal and external situations (Buzsaki and Watson (2012)).This synchronicity is called functional connectivity, while the anatomy of the nervous fiber tracts observed by DTI is called structural connectivity (Sporns (2012)).One of our interests is to elucidate how anatomical structures induce functional structures.Oscillatory activities are observed not only in the neocortex but also in the brain stem.As mentioned above, such oscillatory activities are often modeled by nonlinear oscillator equations.As is well known, coupled nonlinear oscillators show deterministic chaotic activities such as synchronization, desynchronization, and great chaos, which is an exploratory behavior (Tsuda et al. (2004); Asai et al. (2000)).
Chaotic itinerancy by chaotic activities coupled through a body structure Kaneko and Tsuda (2003) proposed two coupled chaos models -coupled map lattice (CML) and globally coupled map (GCM) -which are suitable for general investigation of complex systems.They found that the models generate both ordered and disordered patterns such as deterministic chaotic itinerancy.Inspired by these models, Kuniyoshi and Suzuki (2004) proposed that adaptive and exploratory behaviors such as chaotic itinerancy could be induced by coupling chaotic elements through a robotic body as a physical constraint.As mentioned above, this model shows adaptive behaviors in varying environments, but exploratory behaviors are fewer than expected.
A model based on interactions between the body and complex networks.

Complex network
Several real-world networks are considered to be scale-free or small-world.Such a network structure applied to chaotic elements generates synchronized clusters among chaotic elements (Jalan et al. (2005)).Small-world network and random network Watts and Strogatz (1998) proposed that small-world networks can be generated by reconnecting a coupled map lattice as follows: 1. Start from a ring lattice of n nodes coupled to neighboring nodes up to a specified distance m.
2. Rewire each edge randomly with probability p.
z The behaviors of these networks are varied by reconnecting each connection in CML with probability p.The network is purely CML and completely random at p = 0 and p = 1, respectively.Typically, small-world properties emerge between p = 0.01 and p = 1.

Scale-free network
In this study, we implemented a scalefree network using the following algorithm, known as the Barabasi-Albert (BA) model (Barabasi and Albert (1999)).
1.The algorithm begins with a single node.
2. Count k i , where k i represents the degree of node i 3. Calculate P (k i ) by dividing k i of each node by the sum of the degree of all nodes (Equation ( 3)).
4. Add a new node with m connections to existing nodes, on the basis of the probability of coupling P (k i ).In the BA model, the number of connections to a single node in the network obeys the power law.Many real-world networks, such as the world wide web and human societal relationships, are known to be scale-free, and have been actively researched in recent years.Scale-free networks are also a type of small-world networks because of their structural characteristics.We have confirmed that the BA algorithm yields short average distance between two nodes, relative to the network size.Therefore, in a broad sense, we regard scale-free networks as small-world.

Nonlinear oscillators
Our model adopts two types of nodes (nonlinear oscillators): output nodes, which directly connect to the body, and hidden nodes, which connect to each other with no direct connection to the body.Output nodes activate muscle fibers and receive feedback from length sensors of muscles, while hidden nodes affect the behavior of the model through the network.
Each oscillator is represented as a boundary value problem (BVP) (Equation (??)) that behaves like the action potential of neurons.Multiple coupled BVPs are known to bifurcate (Equation ( 6)) and consequently display complex behaviors (Asai et al. (2000)).
There are two types of nodes (nonlinear oscillators) in our model.One of the types are output nodes, which have connections to the body directly, and another type are hidden nodes, which connected each other and have no direct connection to the body.Output nodes activate muscle fibers and receive feedback from length sensors.Hidden nodes affect the behavior of the model through the network.Each oscillator is represented by a BVP equation ((Equation ( 4)) and (Equation ( 5))) that behaves like the action potential of neurons.It is known that bifurcation occurs in multiple coupled BVPs (Equation ( 6)), which induce complex behaviors (Asai et al. (2000)).
) (OutputN ode) In the above expressions, xi[i] and yi[i] denote the action potential and inhibitory potential of node i, respectively, and z is the tonic input.Sin [i] is the sensor feedback from the muscle fibers, while g is the gain of sensor values.a, b, c, ,δ are constants.k[i] is the degree of node i. Cij is a weight matrix that represents the coupling state in the network.The tonic input controls the stability of oscillations; the higher the tonic input, the more chaotic the oscillators.
The sensory gain controls the strength of connection between the musculoskeletal and oscillator systems.

Network structures of nonlinear oscillators
Experiments were conducted on the following nonlinear oscillator networks.

Experiment (a)
No connections among the nonlinear oscillators.
Each oscillator connects to its corresponding muscle.Experiment (b.2) Scale-free network, modeled by BA with 500 hidden nodes.
To examine the effect of network size on the emergence of behaviors, we conducted experiments without hidden nodes and with 500 hidden nodes on all network models.Each connection weight was  randomly drawn from a uniform random distribution.The connection weights to each node were normalized to a norm of 1.Although the connections between nodes were bidirectional, the connection weights were determined independently for each direction.
The degree distribution in networks with 500 hidden nodes is displayed in Figure 2. Table 1 shows the average distance between nodes in the networks.

Snake-like body
The muscles of the snake-like body in our model are polyarticular, enabling more synchronous body movements (Niiyama et al. (2007)).Table 2 and 3 provide an overview of the snake-like body.Simulations were performed using Open Dynamics Engine (ODE) Smith (2001).Our model consists of a set of nonlinear oscillators and the snake-like body.The nonlinear oscillators are separated into two categories: One comprises output nodes, which generate motor commands sent to muscles and receive the length sensor outputs from the muscles.Another comprises hidden nodes, which are not directly connected to the muscles.

Experimental settings
Experiments were conducted on the abovementioned networks.To examine the effects of the system parameters, we varied the tonic input and sensor gain parameter.For each parameter set, 100 experiments were performed with different randomized weights.

Number of periodic motion patterns of the body
We estimated the number of periodic behaviors in the system by analyzing the time series of the robots joint angles, as described below.
1. Fourier-transform the series of joint angles over a shifting time window.In this study, the width of the time window is 5 [sec], shifting with a time step of 0.1 [sec].After Fourier transformation, the frequency of maximum amplitude identifies the main component of the oscillatory behaviors in each joint.Accordingly, we refer to this wave of maximum amplitude as the representative wave.
2. Calculate the phases of the representative waves from the reference joint angle (Figure 7).Each point in the multidimensional phase space represents a single periodic movement.
3. For clustering, reduce the number of dimensions by principal component analysis.We selected five principle components, thereby retaining 90 % of the variance.
4. Estimate the number of periodic behaviors by applying meanshift clustering (Comaniciu and Meer (2002)) to the data analyzed by principle component analysis.

Count the clusters.
To reduce trivial behaviors, omit clusters containing fewer than 10 % of the data points.The robot mimics the motions of a natural snake.Throughout the experiments, the behaviors can be classified into a maximum of four movements: forward crawling (forward traveling waves), backward crawling (backward traveling waves), bending (stationary waves), and phase-shifted bending.
The average estimated behavioral attractors in Experiment (a) (no connections among the oscillators; one-to-one correspondence between oscillators and muscles) are shown in Figure 8 for different tonic inputs and sensor gains.The number of attractors is one for almost all parameter values.The results of Experiments (b.1) and (b.2) are shown in Figure 9 and 10, while those of Experiments (c.1) and (c.2) are shown in Figure 11 and 12, respectively.Distinct peaks appear in the landscapes of the estimated number of attractors in the networks of the parameter space.These results indicate that there are appropriate and inappropriate connectivity between networks and the body for versatile behaviors.
The effect of the sensor gain parameter was investigated at fixed tonic input 0.55.The results for networks without hidden nodes (Experiments (a), (b.1)-(e.1))and containing 500 hidden nodes (Experiments (a), (b.2)-(e.2))are shown in Figure 13 and Figure 14, respectively.Peak locations in networks generated by our algorithm and the Watts algorithm are identical, but differences exist between the networks themselves.Significant differences are 1 2 3 4 5 6 7 8 9  ).We consider that the scale-free nature of the network, rather than the average shortest distance between nodes, influences the movement behavior.

Discussion
We examined several emergent behaviors resulting from interactions between different network structures, using nonlinear oscillators coupled to a musculoskeletal body.The behaviors of the models were diversified by the presence of complex network structures with optimal sensor gains.Because the number of periodic behaviors peaks around a certain sensor gain, we suggest that the interactions will be important to induce the most diverse behaviors.Since bodily constraints induce stable periodic behaviors in Experiment (a), we consider the musculoskeletal body actuated by polyarticular muscles as an attractor of consistent behaviors.According to Jalan et al. (2005), oscillator complex networks dynamically generate diverse clusters.Therefore, for diverse behaviors to emerge, the combined dynamics of complex network and body must be well-matched.This consideration might be relevant to studies of the autistic brain and general movements.Taga et al. (1999) identified a U shape development in healthy general movements.Using nonlinear prediction analysis, they re- ported that movement complexity is high immediately following birth, decreases at around 2 months, and increases thereafter.The simulation results in this paper suggest that behavior becomes simpler when the dynamics are poorly matched, even if the nervous system is correctly characterized by a complex network.Developmental behavioral changes in human infants may manifest from matching of dynamics among the nervous system, the body and the ambient environment.
A recent MRI study found that the white matter in the brains of children with autistic spectrum disorder (ASD) is structurally different from that of their typically developing peers (Wolff et al. (2012); Courchesne et al. (2007)).Specifically, the brain tissue of ASD children displayed fewer long-range connectivities and stronger local connectivities.Motor development in premature infants who are later diagnosed as ASD is also atypical (Karmel et al. (2010)).Hadders-Algra (2008) proposed that the structural differences in the cerebral cortex of individuals with cerebral palsy and ASD cause reduced variability in motor behavior.
Can such structural differences explain the behavioral character-istics of ASD, such as repetitive behaviors?From a complex systems perspective, repetitive behaviors in ASD may be regarded as a kind of stable state pulled into strong attractors within the dynamics of brain structure, body, and surrounding environment.Namely, an apparent relationship exists between complex cerebral networks and behavioral characteristics.Although we did not provide a direct treatment of this problem in this paper, we believe that our approach will yield theoretical insights into developmental disorders such as ASD.

Conclusion
We modeled complex networks by nonlinear oscillators connected to a musculoskeletal body model, and conducted simulations in a range of scenarios.Diverse behaviors emerged in the combined network/body system under certain network structures and sensor gains.The results suggest that the network structure of human brains plays an important role in the emergence of diverse behaviors (such as general movements) in early human development.Future work will reveal fine differences between cerebral network structures and the whole body musculoskeletal system of fetuses and infants (Kuniyoshi and Sangawa (2006); Mori and Kuniyoshi (2010)).Such studies will significantly advance our understanding of human behavioral development.

Figure 1 :
Figure 1: Emergence of behaviors modeled as interactions among nonlinear oscillators and bodies.(a) The model of behavioral emergence based on chaotic coupled field (Kuniyoshi and Suzuki model).(b) Model proposed in this paper.The emergence model interacts with nonlinear oscillators possessing a certain complex network structure.

Figure 2 :
Figure 2: Distribution of degree of connection to one node in each network

Figure 3 :
Figure 3: Conceptual diagram of the model

Figure 4 :
Figure 4: Appearance of our snake-like robot with the muscles

Figure 5 :
Figure 5: Example of crawling movement by our snake-like body.Time series runs from right to left, and from top to bottom.

Figure 6 :
Figure 6: An example of bending movement by our snake-like body.Time series runs from right to left, and from top to bottom.
Experiment (b.1) Scale-free network, modeled by BA with no hidden nodes.Experiment (c.1) One-dimensional lattice.Nonlinear oscillator network, modeled by the algorithm of Watts et al. with p = 0. Experiment (d.1) Small-world network.Nonlinear oscillator network, modeled by the algorithm of Watts et al. with p = 0.01.Experiment (e.1) Random network with no hidden nodes.Nonlinear oscillator network, modeled by the algorithm of Watts et al. with p = 1.

Figure 7 :
Figure 7: Left: Conceptual diagram of the joint angle data at the time of behavioral transition.Right: An example of clustering of a trajectory in two principle components of the phase space, obtained by mean-shift clustering method.Each color represents a cluster.Finally, trivial clusters involving less than10 % of the data points are eliminated from the estimated behavioral attractors.

Figure 8 :
Figure 8: Number of behavioral attractors without any network structures

Figure 9 :Figure 10 :
Figure 9: Number of behavioral attractors with a scale-free network (hidden node:0)

Figure 14 :
Figure 13: The number of behavioral attractors without a hidden network.Tonic input: 0.55

Table 1 :
Average shortest distance between two nodes in each network with 500 hidden nodes