Abstract: A new algorithm for learning invariance manifolds is introduced that allows a neuron to learn a non-linear transfer function to extract invariant or rather slowly varying features from a vectorial input sequence. This is generalized to a group of neurons, referred to as a Gibson-clique, to learn slowly varying features that are uncorrelated. Since the transfer functions are non-linear, this technique can be applied iteratively to learn more and more complex and invariant features in a hierarchical architecture. Two simple examples demonstrate the general properties of the learning algorithm.