Description
Recent advances in reinforcement learning have demonstrated the ability of machine learning models to learn complex planning and navigation tasks. For example, it was recently demonstrated that large-scale deep learning models can be trained on multiple different tasks, like playing video games or controlling a robot. When exposed to a new, previously unseen task, these networks can master this task without further learning [1]. This is a first step towards reaching the ability to generalize between multiple different tasks that was previously exclusive to humans and some animals. At the same time, modern brain research is uncovering the mechanisms and neural representations that underlie animal behavior in similar tasks. Most prominently the discovery of place cells [2,3] has demonstrated the existence of a spatial map inside the brain, that allows animals to navigate complex environments. More recent studies report brain-wide representations that encode memory items [4]. Whether the same representations that are found in the brain also enable deep learning agents to solve complex behavioral tasks is currently unknown. Most modern machine learning studies treat deep networks as 'black boxes' not further investigating the emerging internal representations. In this project we close this gap by investigating the role of neural representations that emerge in a behaving simulated reinforcement learning agent that interacts with a biologically inspired task. Different mechanisms that are known from machine learning, such as working memory are studied and compared to their biological counterpart.
This project can be scaled to fit a bachelor or master level thesis, and gives the opportunity to work with both, state-of-the-art machine learning tools and integrating them with insights from modern neuroscience experiments. The project will be conducted in the Python programming language and Tensorflow.
Require skills:
- Good programming skills and preferable experience with Python.
- Knowledge in machine learning and experiences in Tensorflow are strongly preferable.
- Interests in neuroscience is a plus.
References:
[1] Reed, Scott, et al. "A Generalist Agent." arXiv preprint arXiv:2205.06175 (2022). https://arxiv.org/pdf/2205.06175.pdf see also: https://www.deepmind.com/publications/a-generalist-agent
[2] O'keefe, John, and Lynn Nadel. "The hippocampus as a cognitive map." Oxford university press, 1978.
[3] Burgess, Neil, Eleanor A. Maguire, and John O'Keefe. "The human hippocampus and spatial and episodic memory." Neuron 35.4 (2002): 625-641.
[4] Roy, Dheeraj S., et al. "Brain-wide mapping reveals that engrams for a single memory are distributed across multiple brain regions." Nature Communications, 13, 1 (2022). https://www.nature.com/articles/s41467-022-29384-4
Supervisors: