Face Recognition for Social Robots

Overview

I'm developing a face-learning method for social robots. With this method, the robot teaches itself to recognize users' faces.

A user initiates the learning process by showing the robot one example image of his or her face and marking where the eyes and nose are. The robot learns an initial representation of the face from these inputs. This initial representation is good enough for the robot to recognize that user fairly often and to avoid false detections most of the time. The initial model doesn't need the reliability of a security system. It only needs to work well enough to keep the user interested and interacting with the robot.

While the user continues to interact with it, the robot searches for the user's face in its video stream. When it detects that face, it tracks it with a face tracker. During this process, the robot captures and stores short video segments of this user.

Later, perhaps while the robot is sleeping at its charging station, it analyzes these video segments, teaching itself to recognize that user better. From the video clips, it can gather examples of different facial expressions and head positions that are typical for this user. Using these examples, it learns a better representation of that user's appearance.

Links to More

The workshop paper we presented at CVPR 2006 on this project is here.