Summary of Research Project Results under JSPS FY2003
"Research for the Future Program"

1.Research Institution The University of Tokyo
2.Research Area Physical and Engineering Sciences
3.Research Field Intuitive Human Interface
4.Term of Project FY 1999 - FY 2003
5.Project Number 99P01402
6.Title of Project Multimodal Anthropomorphic Interface and the Foundations of its Intuitive and Affective Functions

7.Project Leader
Name Institution,Department Title of Position
Mitsuru , Ishizuka The University of Tokyo, Graduate School of Information Science and Technology Professor

8.Core Members

Names Institution,Department Title of Position
Shuji,Hashimoto Waseda University,Faculty of Science and Engineering Professor
Shigeo,Morishima Seikei University,Faculty of Engineering Professor

9.Summary of Research Results

   Toward a new important style of human interfaces beyond current GUI (Graphical User Interface), we have carried out our research and development on anthropomorphic interfaces, where multimodal lifelike agents with face, body and voice communication capability play the role of presenter, guide, communication partner, etc. As a key factor of making these agents be accepted by many people as attractive and friendly partners, we have also worked on their affective functions mainly related to emotion. Multimodal contents with character agents are considered important in our research as well as multimodal interfaces.
   A description language called MPML (Multimodal Presentation Markup Language) has been developed as a core system technology of integrating many functions. It allows many ordinary people to write attractive multimodal contents with ease, and is expected to play a role corresponding to that of HTML for Web contents. While the main target of MPML is mutimodal Web contents, some other versions of MPML have been developed, such as MPML-VR for 3D virtual reality space (VRML space) and MPML-Mobile for mobile phones. Through international presentations and discussions, MPML is now well-known internationally and has contributed toward the international standardization of a mutimodal-contents description language.
   The research in Japan on anthropomorphic interface agents with texture-mapped realistic faces is unique in the world. We have propelled this research as they can provide human-like natural impression different from character agents. Precise 3D facial models are developed for providing advanced functionalities for the anthropomorphic agents. New applications, such as replacing a face and voice of one person in a video stream with those of another person, have been demonstrated. A technology of creating cartoon-like portrait characters from real portrait images has been developed. A new technology called Thermo-key has been invented for producing video avatars using infrared-camera image.
   Humanoid robots are also expected to play the role of interface agents in near future. We have exploited multimodal interfaces including facial expressions, gestures as well as voice communications for the humanoid robots. This work conducted at the Humanoid Robotics Institute (Waseda Univ.), an internationally well-known research group in humanoid robotics, has achieved advanced functionalities being considered a frontier in this area.

10.Key Words
(1)Multimodal Interface   (2)Anthroporphic Interface   (3)Lifelike Character Agent
(4)Multimodal Contents   (5)Affective Interface   (6)Video Avatar
(7)Humanoid Robot Interface   (8)Cartoon-like Portrait Agent   (9)Contents Description Language