Shin'ichiro Nakaoka (Researcher) and others of Humanoid Research Group (Leader: Kazuhito Yokoi), Intelligent Systems Research Institute (Director: Hirohisa Hirukawa), the National Institute of Advanced Industrial Science and Technology (AIST; President: Tamotsu Nomakuchi), have developed integrated software, "Choreonoid," to directly generate and edit the whole-body motion of a humanoid robot.

This software automatically and instantaneously converts a sequence of humanoid poses (key poses) input by the user to a dynamically stable movements, enabling a user without knowledge of robotics to generate various humanoid movements easily in a similar manner to the generation of a computer graphic character's motions. This technology is expected to contribute to the creation of new content industries using humanoid robots.

The dancing of "Cybernetic Human HRP-4C" produced by Masaru Ishikawa (Special Researcher, Information and Robot Technology Research Initiative, the University of Tokyo) and SAM (choreographer/dancer) and generated using Choreonoid, will be demonstrated on October 16 and 17, 2010 at DIGITAL CONTENT EXPO 2010 held at the National Museum of Emerging Science and Innovation (Miraikan) in Koto Ward, Tokyo. Details of this technology will be presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems to be held on October 18–22, 2010, in Taipei.

Figure 1 Example of motion editing using Choreonoid, the developed integrated software

In recent years, the research and development of humanoid robots has been actively pursued. Although the application of humanoid robots has been limited to academic areas, a new market, focusing on content areas, is expected to emerge.

To promote the use of humanoid robots in content areas it is necessary to separate the roles of producers, namely engineers produce hardware and content creators make contents (software), as in such areas as TV, PCs, games, etc. However, content creators without expertise in robotics have difficulty in creating contents that make full use of the capabilities of humanoid robots.

To create new content industries combining robotic technology and media technology, AIST has developed a humanoid robot, Cybernetic Human HRP-4C, with a human-like appearance and human-like motion. In addition, AIST is conducting the development of integrated software for easy creation of contents that make full use of the capabilities of HRP-4C, and we have made various attempts to apply the software to content areas.

Development of the motion pattern design tool, which forms part of this project, is supported by the "Intelligent RT Software Project (FY2007 to FY2011)" commissioned by the New Energy and Industrial Technology Development Organization.

We have devised a method to directly generate and edit the whole-body motion of a humanoid robot and developed integrated software, Choreonoid. ("Choreonoid" is a compound of the words "choreograph" and "humanoid".) Figure 2 is an example of the use of the software. Each pose is called a key pose and is provided by the user. A sequence of movements is formed by connecting these key poses. A variety of movements can be expressed by changing the key poses.

Figure 2 Key pose input and edit screen of Choreonoid (top) and example of humanoid motion (bottom)

In many cases, a bipedal walking robot falls over because simply connecting the key poses does not take into consideration a dynamic balance between the robot and the floor and the impact on the leg when landing. This issue has significant influences on human-sized robots with small feet, such as HRP-4C, making it impossible to achieve a stable motion by simply connecting key poses. To solve this problem, we have developed a technology that automatically corrects and connects key poses to realize a stable motion every time the user inputs or modifies a key pose. This technology enables the generation of the motion that can readily be performed by a bipedal walking robot (Fig. 2, bottom). Figure 3 shows the concept of this technology, and Fig. 4 shows its details.

Figure 3 Concept of the correction and interpolation function for stable motion

Figure 4 Detailed flow of the correction and interpolation of key poses

The user-input key pose is first checked for the condition of contact between the foot sole and the floor and is then corrected to fit the sole to the surface of the floor, so that it neither sinks into the floor nor is held above it. Then, the transition of the Zero Moment Point (ZMP) and the transition between interpolation spaces (a joint space and a Cartesian space) are determined for smooth transition of the contact condition. If the provided key poses are not enough to smoothly move the ZMP, a supplementary key pose (as shown in B' in Fig. 4) is automatically inserted. Also, a supplementary key pose is added to smoothly lift the sole from the floor and land it on the floor. A stable motion trajectory can be generated by integrated processing of the transition of the ZMP, the transition between the interpolation spaces, and a sequence of key poses. In addition, the results of this process are fed back into the user-input key poses and the waist position of each key pose is automatically corrected as needed. The above process is performed every time the user inputs or modifies a key pose, allowing the user to generate a motion while checking to see if the motion is as desired.

These technologies enable users without knowledge of robot control programming to generate motions of a robot, making full use of the robot's capabilities. The developed software is convenient because it can be used on an ordinary PC and no other software is required to generate motions.

In the next step, we will make Choreonoid available to the public and will create new contents for humanoid robots in collaboration with content creators.