Modeling context-dependent learning using deep reinforcement learning Computational Neuroscience


Fifteen percent of Germans suffer from anxiety disorders. Current treatments make intense use of exposure therapy to extinguish fearful behavior in patients. Unfortunately, the seemingly extinguished fear might renew, either spontaneously or due to changes in the environmental context, when the patient leaves the doctor’s office. Therefore, studying the neural mechanisms of context-dependent fear extinction and renewal is important for understanding anxiety disorders and their treatment in human patients. While the hippocampus is known to play a major role in fear renewal, the neural basis of this phenomenon, and especially its context-dependency, are still not fully understood. To address this issue, we developed a computational model of spatial learning in rodents based on modern machine learning techniques, including reinforcement learning with experience replay (keras-rl), and deep neural networks (keras, TensorFlow). This model controls the behavior of artificial agents that explore complex virtual environments (Blender, Unity). Fear learning and extinction can be easily induced in our model, and modification of the virtual environments results in visual context changes that cause the agents to show fear renewal. Furthermore, we analyze the context-dependency of renewal by scrutinizing the deep neural networks trained during the simulations, comparing the results to the neural correlates of extinction learning in rodents. Within this larger endeavor, projects at the Bachelors and Masters level will focus on developing new modules in the computational framework and/or studying specific scientific questions about spatial navigation, spatial learning, extinction learning and renewal.


The Institut für Neuroinformatik (INI) is a central research unit of the Ruhr-Universität Bochum. We aim to understand the fundamental principles through which organisms generate behavior and cognition while linked to their environments through sensory systems and while acting in those environments through effector systems. Inspired by our insights into such natural cognitive systems, we seek new solutions to problems of information processing in artificial cognitive systems. We draw from a variety of disciplines that include experimental approaches from psychology and neurophysiology as well as theoretical approaches from physics, mathematics, electrical engineering and applied computer science, in particular machine learning, artificial intelligence, and computer vision.

Universitätsstr. 150, Building NB, Room 3/32
D-44801 Bochum, Germany

Tel: (+49) 234 32-28967
Fax: (+49) 234 32-14210