This paper introduces Alter3, a humanoid robot that demonstrates spontaneous motion generation through the integration of GPT-4, Large Language Model (LLM). This overcomes challenges in applying language models to direct robot control. By translating linguistic descriptions into actions, Alter3 can autonomously perform various tasks. The key aspect of humanoid robots is their ability to mimic human movement and emotions, allowing them to leverage human knowledge from language models. This raises the question of whether Alter3+GPT-4 can develop a “minimal self” with a sense of agency and ownership. This paper introduces mirror self-recognition and rubber hand illusion tests to assess Alter3’s potential for a sense of self. The research suggests that even disembodied language models can develop agency when coupled with a physical robotic platform.

We investigated the effect of time delay on hand motion and subjective time perception in a virtual environment. Our results indicated that time delays correlated with decreased hand motion speed and altered time perception. These findings suggest that body movements act as critical reference points for constructing subjective time, and alterations in these references can influence time perception.

In this study, we introduce a novel system whereby a humanoid robot, named Alter3, employs a selective combination of three strategies - Mimicking, Imitation, and Dream - to replicate human behavior observed through its camera-based eyes. This work builds upon previous research [Masumori et al. (2021); Ikegami et al. (2021)]. In Mimicking mode, Alter3 recreates “how” a human moves by calculating joint angles. In Imitation mode, it identifies and reproduces symbolic poses through a pre-trained Variational AutoEncoder (VAE), essentially replicating “what” the human did. When imitation proves unsuccessful, Alter3 engages its Dream mode, where it recalls altered memories through selection and mutation processes, allowing it to generate movements based on experience. Moreover, in the absence of a human subject, Alter3, with its eyes closed, retrieves and performs movements from memory. Our findings reveal that the concurrent use of the three strategies (Mimicking, Imitation, Dreaming) stabilizes the latent space state and optimizes the range of identifiable poses. Furthermore, the behavior that Alter3 generates through Dream mode evolves from symbolic movements via the Imitation pathway. These findings suggest that new movements can be created from concept-based motions by selectively employing both methodical (Mimicking) and symbolic (Imitation) motions.

In this study, we report the investigations conducted on the mimetic behavior of a new humanoid robot called Alter3. Alter3 autonomously imitates the motions of a human in front of him and stores the motion sequences in its memory. Alter3 also contains a self-simulator that simulates its own motions before executing them and generates a self-image. We investigate how this mimetic behavior evolves with human interaction, by analyzing memory dynamics and information flow between Alter3 and humans. One important observation from this study is that when Alter3 fails to imitate human motion, humans tend to imitate Alter3 instead. This tendency is quantified by the alternation of the direction of information flow. At the conference we will also report on the experiments we carried out recently, in which two Alters imitated each other, and in which we let people possess and imitate Alter.

Acoustic ecologist Bernie Krause hypothesized that rich soundscapes in mature ecosystems are generated by sound communication between different species with differentiating acoustic niches. This hypothesis, called the acoustic niche hypothesis, proposes that in a mature ecosystem, the singing of a species occupies a unique bandwidth in frequency and shifts in time to avoid competition, thus making the communication efficient. We hypothesize that selective pressure on communication complexity is required for differentiating and filling acoustic niches by a limited number of species, in addition to selective pressures on communication efficiency. To test this hypothesis, we built an evolutionary model where agents can emit complex sounds. Our simulations with the model demonstrate that selective pressure on communication efficiency and complexity leads to an evolution in spectral differentiation with a limited number of species filling the acoustic niche. This is the first demonstration of acoustic niche differentiation using an artificial life model with complex-sounding agents. We also propose multi-timescale complexity measurement, extending the Jensen–Shannon complexity using multi-scale permutation entropy. We analyze the evolved soundscape in the simulations using this measure. The result shows that multi-timescale complexity in soundscape evolved, suggesting that evolving niche differentiation leads to ecological complexity. We implement the extended model in real space and demonstrate that the system can adaptively generate sounds, differentiating acoustic niches with environmental sounds.

Our previous study showed that embodied cultured neural networks and spiking neural networks with spike-timing dependent plasticity (STDP) can learn a behavior as they avoid stimulation from outside. In a sense, the embodied neural network can autonomously change their activity to avoid external stimuli. In this paper, as a result of our experiments using cultured neurons, we find that there is also a second property allowing the network to avoid stimulation: if the network cannot learn to avoid the external stimuli, it tends to decrease the stimulus-evoked spikes as if to ignore the input neurons. We also show such a behavior is reproduced by spiking neural networks with asymmetric-STDP. We consider that these properties can be regarded as autonomous regulation of self and non-self for the network.

Based on the principle of artificial life, we developed an upper-body android called "Alter." Alter is human-like in appearance, receives sensory information from the outside via an autonomous sensor system located around the android, and moves spontaneously using two autonomous systems of internal dynamics. Its body and arms contain a central pattern generator with seven degrees of freedom and hundreds of plastic artificial neurons. We investigated Alter’s environmental adaptability and the spontaneity of is behavioral patterns. In addition, we discuss the conditions under which a robot can become lifelike.

Spiking neural networks with spike-timing dependent plasticity (STDP) can learn to avoid the external stimulations spontaneously. This principle is called "Learning by Stimulation Avoidance" (LSA) and can be used to reproduce learning experiments on cultured biological neural networks. LSA has promising potential, but its application and limitations have not be studied extensively. This paper focuses on the scalability of LSA for large networks and shows that LSA works well in small networks (100 neurons) and can be scaled to networks up to approximately 3,000 neurons.