What we do
We are an interdisciplinary research group focusing on principles of self-organization in neural systems, ranging from artificial neural networks to the hippocampus/memory system. By bringing together machine learning and computational neuroscience we explore ways of extracting representations from data that are useful for goal-directed learning.
On the machine learning side our work is centered around reinforcement learning, where an agent learns to interact with its environment. In this context we investigate learning of representations for different kinds of data, such as visual data and graphs, by means of deep learning as well as classical unsupervised methods. Additionally, we research model-based agents that can remember their environment and are capable of planning ahead. On all these frontiers, we do not only seek to improve algorithmic performance but also to develop new ways of building more interpretable, explainable and human-friendly AI.
On the neuroscience side, our work focuses on computational modeling of brain functions concerned with encoding, storage and recall of memories. Through this we aim to understand how information is learned, represented within different types of memory and finally reconstructed from memory.
Representation Learning for Reinforcement Learning
Reinforcement learning has become an increasingly powerful tool with the advent of neural networks, which provide a powerful all-in-one solution to solve complex tasks. A major downside of this approach, however, is its lack of transparency and transferability - which are important qualities if these systems are to be applied in real-world applications. This is especially important for the many cases where safety or fairness play an important role. By separating the learning of useful data representations from the process of solving a task, we aim to improve explainability as well as flexibility in reinforcement learning to address these issues; With a good representation of input data, the model required to solve a task can be simpler and thus easier to understand. At the same time, it becomes more feasible to transfer already established, transparent solutions to similar tasks.
Our group has developed the highly cited Slow Feature Analysis (SFA), which is able to extract slowly moving features from data. Over recent years, we have extended SFA and applied it to various problem settings. Nowadays, we are also working with more recent deep representation learning methods - especially different flavors of variational autoencoders (VAEs).
Memory-based Reinforcement Learning
Besides the representation learning aspect outlined above, we are interested in model based deep reinforcement learning algorithms for complex domains. In model based reinforcement learning, the goal is to first learn the environmental transition and reward dynamics of a given problem and then use the learned dynamics to solve the problem more data-efficiently. Since stochasticity or partial observability tend to play an important role in complex and high-dimensional settings, memory techniques are commonly used to aggregate information over the course of multiple (possibly many) time steps. Furthermore, to increase robustness and model performance, sampling models have become an increasingly popular choice over to the more common expectation models in recent years.
Our main research focus in this setting lies on the improvement of the planning performance of learned environment models. Thereby, we are specifically interested in designing model architectures that qualify for efficient and precise planning by construction instead of working on the planning procedure itself. To achieve this, we combine the above mentioned sampling models and memory techniques with principles from hierarchical reinforcement learning.
System Level Modeling of the Hippocampus
Remembering what we have done and experienced in the past is essential for defining what we are and deciding what to do in the future. However, our so-called episodic memory is far less reliable than one might think. We neglect, change, and even add things to our memories, often in ways that makes them more in line with what we would generally expect or like. Thus, episodic memory seems to be largely a generative process, where incomplete memory traces are enriched and modified by general so-called semantic information and expectations about the world.
In the neuroscience side of our group we use advanced machine learning techniques and develop a model of the interaction between the episodic and semantic memory system, mainly during storage and retrieval of episodic memories, but also for learning semantics. Our model will describe the interplay between hippocampus and neocortex. We hypothesize that the hippocampus stores and retrieves selected aspects of an episode, which are necessarily incomplete, and the neocortex reasonably fills in the missing information based on general semantic information. For modeling we have used many generative models ranging from restricted boltzmann machines (RBM) to variational autoencoders (VAE) and vector quantized variational autoencoders (VQVAE) and also PixelCNN.
This research is part of an interdisciplinary DFG funded research group "Constructing scenarios of the past: A new framework in episodic memory". We have close interaction with partners from psychology collaborators that study the neural mechanisms via fMRI and behavioral experiments and philosophy partners that address fundamental questions that arise within and about our framework.