In this project a face bunch graph is employed to generate phantom faces and to determine facial attributes. When a bunch graph is matched onto an image, a best fitting jet is selected at each node independently of the other nodes. We call these jets local experts. What can we tell about the face in the image if we disregard the image but keep the matching information, i.e. the identity of the selected local experts and the locations of the nodes? We can, for example, generate a phantom face simply by taking grey value patches from the model images from which the local experts were selected, and connecting them with smooth transitions. Though such a phantom face is usually composed from more than fifteen different models, it looks natural and similar to the original, see Figure 1.
Figure 1: Face bunch graph and phantom face. A face bunch graph (middle) is matched to an image (left). Reconstructing the image from the local experts yields a phantom face (right). The models in the bunch graph are labeled with some attributes, which can be used to infer attributes also from the face in the image
Since the phantom face looks similar to the original, one can also expect that some attributes, such as sex, can be transferred from the models to the image. Let us assume that the models are labeled as being male/female or bearded/unbearded, or having glasses/no glasses. If we list the labels of the local expert models, we get a matrix of votes for a certain combination of these attributes. If we simply count, whether more local experts come from male or female models, we get a fairly reliable vote for the attribute of the image face, see Figure 2. This also holds for the other two attributes, if one considers only the upper half for glasses and the lower half for a beard.
Figure 2: An example of a face analysis. The left face is the original and the right one the phantom face; the male/female votes are listed in the left matrix; the glasses votes are listed in the upper rows of the right matrix; the bearded votes are listed in the lower rows. In this case, all local experts vote for male, most of the upper ones vote for glasses, and most of the lower ones vote for a beard. The situation is not always so clear but high correct classification rates can be achieved.
This method is simple and not specialized to faces in any way. I expect the system being able to classify domestic animals or other classes of objects as well, given the same level of standardization for the images.