Tamio Tanikawa (Senior Research Scientist) and others of RT-Synthesis Research Group (Leader: Tetsuo Kotoku), the Intelligent Systems Research Institute (Director: Hirohisa Hirukawa) of the National Institute of Advanced Industrial Science and Technology (AIST; President: Tamotsu Nomakuchi), in collaboration with the Information Technology Research Institute (Director: Satoshi Sekiguchi) of AIST, Collaborative Research Team for Communication and Control of Smart Grid (Leader: Tetsuya Higuchi) of AIST, Research Institute (Director: Seishi Kato) of the National Rehabilitation Center for Persons with Disabilities (NRCPD), and Misawa Homes Institute of Research and Development Co., Ltd. (MHIRD) (President: Shoichi Ueno), have developed a system design technology that allows people with disabilities to combine various disability aids and use them in an integrated manner. Home environment models to address the needs of individuals with disabilities have also been proposed and demonstrated.

In the demonstrated system, input and output devices that use different control methods are networked by using a common network platform technology for robot modules, "RT (Robot Technology) Middleware." This allows devices to be easily added to or removed from the network and enables cooperation of the devices on the network; thereby, it allows the user to combine devices according to his/her needs.

The system is equipped with high-performance interfaces for equipment control, namely a stereovision-based, fast-response gesture interface that can be customized according to the level of mobility of a person with disabilities and a speech-recognition interface that is robust to noise in the home environment and can recognize unclear speech. In addition, active casters for easy movement of home equipment are installed in the home environment to provide physical assistance.

AIST worked with NRCPD to evaluate various disability aids and with MHIRD to introduce disability aids into a home environment. The home environment models based on these technologies are expected to help people with disabilities to live independent lives, facilitate their participation in social activities, and reduce the load on their helpers.

Figure 1 Networking of various disability aids via RT Middleware (top), and the home environment models with the aids installed to facilitate independent living of people with disabilities (bottom).

Currently available disability aids have been developed as a system with functions customized for the condition of each person with disabilities. This has resulted in high equipment prices and is an issue that prevents improvement in quality of life for such individuals.

In recent years, with advances in network technology, ordinary integrated devices are now made with modularized functions interconnected by a built-in network. Devices with this type of configuration make it easy to develop new products and have advantages such as shorter development cycles and reduced cost. Introduction of this system concept into the development of disability aides is expected to make it possible to provide disability aids customized for each person at affordable prices.

With the aim of helping to improve quality of life through robotics research and development, AIST has been developing various robots to assist human beings. One of such activities is the development of a robotized home environment, or a spatial robot, based on the concept of ubiquitous robotics. We are developing RT Middleware to modularize various robot components (RT components) and facilitate the creation of a system integrating these components. In the area of welfare equipment, we are conducting research and development of human interfaces including a stereo camera-based gesture input device and a speech recognition system into which those with speech impairment due to cerebral palsy can input speech. In order for many people with disabilities to use these advanced technologies, a system with functions that can be freely combined according to the needs of the user is required. Aiming to support those having difficulty in living independent and self-supporting lives, we have been conducting research and development to enable the cooperation of various disability aids being developed at AIST via a common network platform, such as RT Middleware. We also aim to introduce the system suitable for the characteristics of each individual's disability into a home environment. The research and development is conducted as part of "Research and Development Project to Support the Independent Living of People with Disabilities (FY2009)" of the Ministry of Health, Labour and Welfare.

In order to help people with disabilities live independent lives, we have developed human interface technologies: a gesture input device and a speech input device.

The gesture input device (Fig. 2), which detects the movement of a person by a stereo camera, calculates the changing shape of the person three-dimensionally at high speed and has the advanced function to identify gestures in a non-contact, non-constraint manner. Therefore, even those whose physical disability prevents them from using a remote control or equipment can control the equipment by moving the hand, shoulder, or head. Thus, a device-operating environment suitable for the user can be provided. With modification of the software, the device can be adjusted to changes in the movement of a person with a progressive illness. To demonstrate the applicability of the high-speed system to entertainment and rehabilitation activities, as well as the control of home appliances, experiments were conducted in which persons played a tennis video game by controlling the machine by hand and shoulder gestures. The experiments showed that an environment where persons with and without disabilities enjoy a game together can be realized.

Figure 2 Stereo camera used for the gesture input device (left) and results of gesture recognition (right).

A speech recognition technology that is robust to ambient noise and can recognize unclear speech was introduced to the speech input device (Fig. 3) as a tool to control home equipment and a function of cooperating with other input devices was realized. This speech recognition technology is robust to ambient noise and can recognize unclear speech. The technology was developed in a research project (FY2004 to FY2006) on an electric-powered wheelchair that can be maneuvered by those with mobility and speech impairments.

Figure 3 Speech input device capable of recognizing unclear speech.

An active caster with a built-in motor (Fig. 4) has been developed as an output device to physically assist people with disabilities. The object to be moved can be maneuvered easily and remotely by the casters attached to it. Depending on the weight and size of the object, more than one caster may have to be attached. The casters need not to be attached in exact position, because the position is automatically calculated and the object can be remotely maneuvered. The caster can be attached to various objects to meet the needs of the user.

Figure 4 Attaching the active caster (left) and remotely controlling a food delivery cart (right).

In addition, a system has been developed to have devices, such as the gesture input device and the active caster, cooperate with each other through RT Middleware. As a result, the disability aids that have been developed can be customized for the user by changing their combination according to the required function. If disability aids are distributed in a network, the home environment as a whole can be considered as one system of disability aids, with functions distributed across a room (Fig. 5).

Figure 5 System configuration in which devices cooperate with each other through RT Middleware.

The following three home environment models have been developed to show that systems based on this technology can meet the needs of people with different disabilities:
(1) A home environment model to help people with mobility impairment (home equipment control assist system)
(2) A home environment model to help the cooking training of people with brain dysfunction from stroke (cooking assist system)
(3) A home environment model to help people with visual impairment (an environment management system to detect changes in the home environment)

Issues in the home environments of wheelchair-bound persons with mobility impairment include furniture getting in the way of the wheelchair, difficulty opening and closing doors, and difficulty controlling home appliances. Figure 6 shows a home environment model in which a user can use human interfaces such as the gesture input device and the speech input device to control home equipment including remote controls of appliances (e.g. TVs, video players, and air-conditioning units) and automated furniture and doors with the active casters. Active control of the home environment, such as autonomous movement of furniture in the way to ensure wheelchair access, can be realized by the cooperative operation of these disability aids.

Figure 6 Home equipment control assist system.

This model is for those who have no physical impairment but suffer higher brain dysfunction as the result of stroke. A survey report (Summary of the 2008 Patient Survey) by the Ministry of Health, Labour and Welfare says that the total number of patients in Japan with cerebrovascular diseases such as stroke is 1.339 million; this is comparable to the total number of those with malignant neoplasm (cancer) (1.518 million). Cognitive impairment due to cerebrovascular disease or traumatic brain damage is called higher brain dysfunction. Its symptoms include attention deficit disorder, memory dysfunction, and executive dysfunction. Cooking training is commonly used as a rehabilitation activity for these patients. Cooking assistance is also important for them to live independent lives. Figure 7 shows a cooking assist system consisting of an information assist tool to display cooking processes and sensors installed in the kitchen. The sensors cooperate with the information assist tool while monitoring the cooking processes. The system provides an environment for people with disabilities to cook for themselves. It can also be used to train people with brain dysfunction. The information assist tool (Fig. 8) always provides instructions on the cooking process to the user using the PC in the kitchen. The user checks the process, using the touch panel, and can cook as instructed. The sensors installed in the kitchen check that the cooking process is safely performed and cooperate with the cooking process display tool.

Figure 7 Cooking assist system for people with brain dysfunction from stroke.

Figure 8 Screenshot of the cooking process display.

With increasing length of the period of impairment, the visually impaired can live a lifestyle increasingly close to that of the non-visually impaired in their home environment. This is because the accuracy of the home environment model in the resident’s mind improves as he or she spends more time in that environment. However, a change brought about in the home environment, e.g. by visitors, leads to a difference from the model, disturbing the lifestyle of the visually impaired person. Figure 9 shows a system that solves this problem by regularly measuring the positions of items composing the home environment, such as furniture and doors, with web cameras and wireless tags, and shows any difference from the normal environment model of the visually impaired person. The web cameras determine the position of each piece of furniture by recognizing a marker attached to it, and the wireless tags use field intensity to measure the position of furniture outside the range of the cameras. Also, a system with buzzers installed in conventional communication tags has been introduced to help find small items, such as keys and remote controls.

Figure 9 Web camera-based environment change detection system

Development and standardization of modularization technology will be promoted to provide systems at affordable prices that can improve the quality of life of people with disabilities, as well as the elderly and people without disabilities, and can be designed at the user level. Namely, disability aids, including those that have been commercialized, will be modularized so that they can be used on a common network. In order to provide these new technologies to the user as soon as possible, a place is needed where the users, the developers, and the clinicians can collaborate with each other. A research and development system that allows the users to give feedback to the developers will be developed under the leadership of NRCPD, a clinical organization. Next-generation home models, such as RT Homes, that can provide a home environment customized for people with disabilities by using sensors and actuators and actively changing room layouts, will be developed as a business model of the future by MHIRD in collaboration with other participants.