Phantom faces for face analysis

In this project a face bunch graph is employed to generate phantom faces and to determine facial attributes. When a bunch graph is matched onto an image, a best fitting jet is selected at each node independently of the other nodes. We call these jets local experts. What can we tell about the face in the image if we disregard the image but keep the matching information, i.e. the identity of the selected local experts and the locations of the nodes? We can, for example, generate a phantom face simply by taking grey value patches from the model images from which the local experts were selected, and connecting them with smooth transitions. Though such a phantom face is usually composed from more than fifteen different models, it looks natural and similar to the original, see Figure 1.

labeled bunch graph (20 kB)

Figure 1: Face bunch graph and phantom face. A face bunch graph (middle) is matched to an image (left). Reconstructing the image from the local experts yields a phantom face (right). The models in the bunch graph are labeled with some attributes, which can be used to infer attributes also from the face in the image

Since the phantom face looks similar to the original, one can also expect that some attributes, such as sex, can be transferred from the models to the image. Let us assume that the models are labeled as being male/female or bearded/unbearded, or having glasses/no glasses. If we list the labels of the local expert models, we get a matrix of votes for a certain combination of these attributes. If we simply count, whether more local experts come from male or female models, we get a fairly reliable vote for the attribute of the image face, see Figure 2. This also holds for the other two attributes, if one considers only the upper half for glasses and the lower half for a beard.

analyzed face (22 kB)

Figure 2: An example of a face analysis. The left face is the original and the right one the phantom face; the male/female votes are listed in the left matrix; the glasses votes are listed in the upper rows of the right matrix; the bearded votes are listed in the lower rows. In this case, all local experts vote for male, most of the upper ones vote for glasses, and most of the lower ones vote for a beard. The situation is not always so clear but high correct classification rates can be achieved.

This method is simple and not specialized to faces in any way. I expect the system being able to classify domestic animals or other classes of objects as well, given the same level of standardization for the images.

The Institut für Neuroinformatik (INI) is a central research unit of the Ruhr-Universität Bochum. We aim to understand the fundamental principles through which organisms generate behavior and cognition while linked to their environments through sensory systems and while acting in those environments through effector systems. Inspired by our insights into such natural cognitive systems, we seek new solutions to problems of information processing in artificial cognitive systems. We draw from a variety of disciplines that include experimental approaches from psychology and neurophysiology as well as theoretical approaches from physics, mathematics, electrical engineering and applied computer science, in particular machine learning, artificial intelligence, and computer vision.

Universitätsstr. 150, Building NB, Room 3/32
D-44801 Bochum, Germany

Tel: (+49) 234 32-28967
Fax: (+49) 234 32-14210