Examining the role of spatial representations in navigation learning Computational Neuroscience

Description

Recent advances in reinforcement learning have demonstrated the ability of machine learning models to learn complex planning and navigation tasks. For example, it was recently demonstrated that large-scale deep learning models can be trained on multiple different tasks, like playing video games or controlling a robot. When exposed to a new, previously unseen task, these networks can master this task without further learning [1]. This is a first step towards reaching the ability to generalize between multiple different tasks that was previously exclusive to humans and some animals. At the same time, modern brain research is uncovering the mechanisms and neural representations that underlie animal behavior in similar tasks. Most prominently the discovery of place cells [2,3] has demonstrated the existence of a spatial map inside the brain, that allows animals to navigate complex environments. More recent studies report brain-wide representations that encode memory items [4]. Whether the same representations that are found in the brain also enable deep learning agents to solve complex behavioral tasks is currently unknown. Most modern machine learning studies treat deep networks as 'black boxes' not further investigating the emerging internal representations. In this project we close this gap by investigating the role of neural representations that emerge in a behaving simulated reinforcement learning agent that interacts with a biologically inspired task. Different mechanisms that are known from machine learning, such as working memory are studied and compared to their biological counterpart.

This project can be scaled to fit a bachelor or master level thesis, and gives the opportunity to work with both, state-of-the-art machine learning tools and integrating them with insights from modern neuroscience experiments. The project will be conducted in the Python programming language and Tensorflow.

Require skills:

  • Good programming skills and preferable experience with Python.
  • Knowledge in machine learning and experiences in Tensorflow are strongly preferable.
  • Interests in neuroscience is a plus.

References:

[1] Reed, Scott, et al. "A Generalist Agent." arXiv preprint arXiv:2205.06175 (2022). https://arxiv.org/pdf/2205.06175.pdf see also: https://www.deepmind.com/publications/a-generalist-agent

[2] O'keefe, John, and Lynn Nadel. "The hippocampus as a cognitive map." Oxford university press, 1978.

[3] Burgess, Neil, Eleanor A. Maguire, and John O'Keefe. "The human hippocampus and spatial and episodic memory." Neuron 35.4 (2002): 625-641.

[4] Roy, Dheeraj S., et al. "Brain-wide mapping reveals that engrams for a single memory are distributed across multiple brain regions." Nature Communications, 13, 1 (2022). https://www.nature.com/articles/s41467-022-29384-4

Supervisors:

David Kappel and Prof. Sen Cheng

The Institut für Neuroinformatik (INI) is a central research unit of the Ruhr-Universität Bochum. We aim to understand the fundamental principles through which organisms generate behavior and cognition while linked to their environments through sensory systems and while acting in those environments through effector systems. Inspired by our insights into such natural cognitive systems, we seek new solutions to problems of information processing in artificial cognitive systems. We draw from a variety of disciplines that include experimental approaches from psychology and neurophysiology as well as theoretical approaches from physics, mathematics, electrical engineering and applied computer science, in particular machine learning, artificial intelligence, and computer vision.

Universitätsstr. 150, Building NB, Room 3/32
D-44801 Bochum, Germany

Tel: (+49) 234 32-28967
Fax: (+49) 234 32-14210