Is predictive coding enough to learn sufficiently rich representations? Theory of Neural Systems


According to the predictive coding theory in neuroscience, the brain continuously predicts incoming sensory stimuli [2]. It accomplishes learning and inference by minimizing the discrepancy between predicted and actual stimuli. A fundamental assumption in the predictive coding theory is that an unsupervised learning signal, such as a prediction error, is enough to learn sufficiently rich representations of sensory stimuli which are helpful for various tasks,e.g., the classification of visual stimuli.

Transferred to artificial neural networks and phrasing the prediction task as a reconstruction of input stimuli, the network’s layers must simultaneously carry classification- and reconstruction-driven information. Our group has demonstrated in a non-predictive coding deep learning context that merging these information types into shared layers presents a trade-off effect. Additionally, we observed that class information is not necessarily available in reconstruction-driven representations. Given that computational predictive coding models face challenges performing reconstruction and classification simultaneously, similar effects may also occur in these models.

This project seeks to explore these effects within predictive coding models. The candidate will implement and train a predictive coding model following [1], analyzing learned representations. Furthermore, predictive coding literature proposes solutions for models struggling with simultaneous classification and reconstruction [3, 4] and we aim to assess how these affect the balance between classification and reconstruction information.

Required skill:
- Interest in machine learning and computational neuroscience
- Programming experience with Python and preferably a deep learning framework, e.g., PyTorch

Jan Rathjens and Prof. Dr. Laurenz Wiskott

[1] Beren Millidge. Implementing Predictive Processing and Active Inference: Preliminary Steps and Results. en-us. Mar. 2019. doi: 10.31234/ url:

[2] Beren Millidge, Anil Seth, and Christopher L. Buckley. Predictive Coding: a Theoretical and Experimental Review. arXiv:2107.12979 [cs, q-bio]. July 2022. doi:10.48550/arXiv.2107.12979. url: 

[3] Jeff Orchard and Wei Sun. Making Predictive Coding Networks Generative. en. Oct.2019. url:

[4] Alexander Tschantz et al. Hybrid Predictive Coding: Inferring, Fast and Slow. arXiv:2204.02169[cs, q-bio]. Apr. 2022. doi: 10.48550/arXiv.2204.02169. url:

The Institut für Neuroinformatik (INI) is a central research unit of the Ruhr-Universität Bochum. We aim to understand the fundamental principles through which organisms generate behavior and cognition while linked to their environments through sensory systems and while acting in those environments through effector systems. Inspired by our insights into such natural cognitive systems, we seek new solutions to problems of information processing in artificial cognitive systems. We draw from a variety of disciplines that include experimental approaches from psychology and neurophysiology as well as theoretical approaches from physics, mathematics, electrical engineering and applied computer science, in particular machine learning, artificial intelligence, and computer vision.

Universitätsstr. 150, Building NB, Room 3/32
D-44801 Bochum, Germany

Tel: (+49) 234 32-28967
Fax: (+49) 234 32-14210