We present a robotic embodiment experiment based on real-time functional magnetic resonance imaging (rt-fMRI). In this study, fMRI is used as an input device to identify a subject's intentions and convert them into actions performed by a humanoid robot. The process, based on motor imagery, has allowed four subjects located in Israel to control a HOAP3 humanoid robot in France, in a relatively natural manner, experiencing the whole experiment through the eyes of the robot. Motor imagery or movement of the left hand, the right hand, or the legs were used to control the robotic motions of left, right, or walk forward, respectively.

Issue Section:
Articles
This content is only available as a PDF.
© 2014 by the Massachusetts Institute of Technology
2014

Supplementary data

Robot Commentary

This video accompanies the article “fMRI-based Robotic Embodiment: Controlling a Humanoid Robot by Thought Using Real-time fMRI” by Ori Cohen, Sébastien Druon, Sébastien Lengagne, Avi Mendelsohn, Rafael Malach, Abderrahmane Kheddar, and Doron Friedman.

In the first part of the video, the subject sees an avatar standing in the center of a three-door room. The subject is given pseudo random instructions. 'Right' and 'left' commands result in the avatar turning toward the right or left door correspondingly and a 'forward' command results in the avatar moving toward the top door. After each action the subject is instructed to rest and during that time the avatar executes a pre-determined command that corresponds with the instruction.

In the second part of the video the subjects in Israel viewed a live video feed through a camera located at the eyes of the robot. The robot was located in a lab in France. The subjects saw a technician who instructed them to move left, right or forward using his hand gestures. This also allows to assess that the robot motion truly reflects the instructions initiated by the subject. The objective given to the subjects in this experiment was to walk around two obstacles in an “eight”-shaped course. Each action performed by the subject is mapped to a pre-computed robotic motion. Turning left, right, or walking forward corresponds to left-hand, right-hand, or legs imagery, respectively. On the right side, brain activity from the respected region of interest is displayed. (© Ori Cohen)

- mp4 file
You do not currently have access to this content.