Research Project (1993-1995)

Phantom faces for face analysis

Laurenz Wiskott

In this project a face bunch graph is employed to generate phantom faces and to determine facial attributes. When a bunch graph is matched onto an image, a best fitting jet is selected at each node independently of the other nodes. We call these jets local experts. What can we tell about the face in the image if we disregard the image but keep the matching information, i.e. the identity of the selected local experts and the locations of the nodes? We can, for example, generate a phantom face simply by taking grey value patches from the model images from which the local experts were selected, and connecting them with smooth transitions. Though such a phantom face is usually composed from more than fifteen different models, it looks natural and similar to the original, see Figure 1.

labeled bunch graph (20 kB)

Figure 1: Face bunch graph and phantom face. A face bunch graph (middle) is matched to an image (left). Reconstructing the image from the local experts yields a phantom face (right). The models in the bunch graph are labeled with some attributes, which can be used to infer attributes also from the face in the image

Since the phantom face looks similar to the original, one can also expect that some attributes, such as sex, can be transferred from the models to the image. Let us assume that the models are labeled as being male/female or bearded/unbearded, or having glasses/no glasses. If we list the labels of the local expert models, we get a matrix of votes for a certain combination of these attributes. If we simply count, whether more local experts come from male or female models, we get a fairly reliable vote for the attribute of the image face, see Figure 2. This also holds for the other two attributes, if one considers only the upper half for glasses and the lower half for a beard.

analyzed face (22 kB)

Figure 2: An example of a face analysis. The left face is the original and the right one the phantom face; the male/female votes are listed in the left matrix; the glasses votes are listed in the upper rows of the right matrix; the bearded votes are listed in the lower rows. In this case, all local experts vote for male, most of the upper ones vote for glasses, and most of the lower ones vote for a beard. The situation is not always so clear but high correct classification rates can be achieved.

This method is simple and not specialized to faces in any way. I expect the system being able to classify domestic animals or other classes of objects as well, given the same level of standardization for the images.

Relevant Publications:

Black colored reference are the principal ones. Gray colored references are listed for the sake of completeness only. They contain little additional information. .ps-files are optimized for printing; .pdf-files are optimized for viewing at the computer.

  1. Wiskott, L. (26. October 1997).
    Phantom faces for face analysis.
    Proc. IEEE Int'l Conf. on Image Processing, ICIP'97, Santa Barbara, Oct. 26-29, vol. III pp. 308-311.
    (bibtex, abstract.html)

  2. Wiskott, L. (10. September 1997).
    Phantom faces for face analysis.
    Proc. 7th Int'l Conf. on Computer Analysis of Images and Patterns, CAIP'97, Kiel, Sep. 10-12, eds. G. Sommer, K. Daniilidis, and J. Pauli, number 1296 in series Lecture Notes in Computer Science, publ. Springer-Verlag, Heidelberg, pp. 480-487.
    (bibtex, abstract.html)

  3. Wiskott, L. (1997).
    Phantom faces for face analysis.
    Pattern Recognition, 30(6):837-846.
    (bibtex, abstract.html, paper.pdf,

  4. Wiskott, L. (1. June 1996).
    Phantom faces for face analysis.
    Proc. 3rd Joint Symposium on Neural Computation, JSNC'96, Pasadena, June 1, publ. University of California, San Diego, pp. 46-52.
    (bibtex, abstract.html)

  5. Wiskott, L. (March 1996).
    Phantom faces for face analysis.
    Internal report, IR-INI 96-06, Institut für Neuroinformatik, Ruhr-Universität Bochum, 44780 Bochum, Germany, 12 pages.
    (bibtex, abstract.html)

  6. Wiskott, L. (1995).
    Labeled graphs and dynamic link matching for face recognition and scene analysis.
    Verlag Harri Deutsch, Thun - Frankfurt am Main, Reihe Physik 53 (PhD thesis), ISBN 3-8171-1481-8, 110 pages.
    (bibtex, abstract.html, paper.pdf,

  7. Wiskott, L., Fellous, J.M., Krüger, N., and von der Malsburg, C. (1995).
    Face recognition and gender determination.
    Proc. Int'l Workshop on Automatic Face- and Gesture- Recognition, IWAFGR'95, Zurich, June 26-28, ed. Martin Bichsel, publ. MultiMedia Laboratory, University of Zurich, pp. 92-97.
    (bibtex, abstract.html,,, paper)

Related Project:

setup April, 1996; updated December 14, 1998
Laurenz Wiskott,