Proc. The Mathematical, Computational and Biological Study of Vision, Oberwolfach, November 4-10, eds. D. Mumford, J.-M. Morel, and C. von der Malsburg, publ. Mathematisches Forschungsinstitut Oberwolfach, Report 49/2001, pp. 21-22, (abstract) (2001-11-04) (bibtex)

Unsupervised learning of invariances in a simple model of the visual system.

Laurenz Wiskott

Abstract: A new algorithm for unsupervised learning of invariances is presented. The basic idea is to learn a nonlinear input-output function which extracts slowly varying aspects from the input signal by minimizing the temporal variation of the output signal. This is a known approach. The algorithm, however, differs from existing learning rules. Firstly, it computes the solution in a closed form (like PCA) and is guaranteed to find the optimum within the considered function class. Secondly, not only one but many uncorrelated output signal components can be generated easily, which is important for hierarchical networks.
The algorithm is then applied to a simple model of the visual system with a one-dimensional retina. Depending on what stimuli are used for training, the network can learn translation-, scale-, rotation- (cyclic shift), contrast-, or illumination-invariance. Relatively few stimulus patterns are needed for training to achieve good generalization to new patterns. The representation generated is suitable for pattern recognition. Overall the model suggests that it may be plausible that our visual system learns invariances based on fairly limited visual experience.

Relevant Project:

July 12, 2002, Laurenz Wiskott,