Team members: Michelle Li, AJ Libunao & David Lu
Concept
The project objective is to create a sociable robot that interacts with a human based on the biometric feedback from the human being. The heart plays a vital role in affecting emotional, cognitive, psychological, and physiological changes within the human body and as such, we will focus on the heart rate variability as the main input for the robot. The robot can respond and react to the user’s emotional state through speech, movement, and facial expressions. Through these interactions, we believe that using a sociable robot in this particular interaction will help people better manage their physical and emotional states than with a specific monitoring device.
The heart plays an important role in affecting the emotional and cognitive state of human beings through physiological changes in the body. Therefore, theories that we draw upon for our robot creation are from the spheres of psychology, child development and social psychology.
The HCI field of affective computing by which systems detect user’s physiological states is still relatively new. Many research works deal only with the user’s physiology and responding with environmental changes, for example, but not necessarily focusing responding to or changing the person’s emotions. By creating a robot that can develop a relationship based on emotional communication, verbal or non-verbal, a meaningful interaction can be achieved.
With the limited resources and knowledge about biometric feedback and robotics, we can only build a very simple experimental project using the biometric feedback concept. This concept is quite interesting because it is very difficult to lie through our physiological and emotional states. By illuminating the body as the interface in this interaction, human and robot can have a more direct and valid interaction.
Further applications of this idea could enhance the quality of the interaction for educational and health care robots, such as Autom, the weight loss coach robot. With biometric feedback, a robot can automatically detect a person’s physiological/emotional state and use that extra information to further analyze health status, track progress, and suggest further lifestyle improvements.
Interactions
The interaction is rather simple. Emotabot engages in interaction when it detects a person in close proximity by raising its head and giving a verbal greeting. It then reads the heart rate variability of the user via the emWave device to infer the user’s emotional state and then responds accordingly to create the illusion of understanding using visual facial cues and speech. When the user is in a calm, mental state, that is, he has a coherent and stable heart rate, the robot will smile, open its ears, and provide positive comments. On the other hand, when the user is in a stressed state, that is, he exhibits an inconsistent heart rate, the robot will display a sad expression and tell jokes to encourage the user to relax. Conversation is facilitated between robot and human through turn-taking tactics. When Emotabot senses that the user is leaving, it says ‘goodbye’ and drops its head.
Technical Details
Emotabot was built using LEGO Mindstorm NXT for the physical form and actuating motors and uses emWave as an interface between the robot and human.
The emWave is a commercialized personal stress reliever. Stress creates incoherence in our heart rhythms. When we are in a state of high heart rhythm coherence the nervous system, heart, hormonal and immune systems are working efficiently and we feel good emotionally. The emWave has two biofeedback sensors that measure users’ heart rate variability (heart rhythm). A figure sensor is built into the device that only requires users to place the index finger on it and an external ear sensor that requires users to clip it on the ears. Different LED colors represent different states of the users. We then use the LED color display to link up with the LEGO Mindstorm NXT through the Mindstorm light sensor.
The essential technology we used in the NXT is the light sensor to connect the robot mechanism with the emWave device. By having the light sensor detect color output from the emWave, we can determine what state the user is in (green and blue corresponds to a calm state and “in-between” state, respectively, while red denotes a stressed state). An ultrasonic sensor detects the user’s proximity in relation to the robot. In this case, if the user is within 24 inches of Emotabot, the robot will raise its head to greet the use. Once the user leaves the field of view of the ultrasonic sensor, the robot will lower its head and bid the user farewell.