A Dashboard for University Student Guidance
We develope an interactive dashboard to support academic advising through personalized, data-driven insights.
- RUB
- Computer Science
- INI
- Research Groups
- Theory of Neural Systems
Theory of Neural Systems
What we do
We are an interdisciplinary research group focusing on principles of self-organization in neural systems, ranging from artificial neural networks to the hippocampus/memory system. By bringing together machine learning and computational neuroscience we explore ways of extracting representations from data that are useful for goal-directed learning.
On the machine learning side our work is centered around reinforcement learning, where an agent learns to interact with its environment. In this context we investigate learning of representations for different kinds of data, such as visual data and graphs, by means of deep learning as well as classical unsupervised methods. Additionally, we research model-based agents that can remember their environment and are capable of planning ahead. On all these frontiers, we do not only seek to improve algorithmic performance but also to develop new ways of building more interpretable, explainable and human-friendly AI.
On the neuroscience side, our work focuses on computational modeling of brain functions concerned with encoding, storage and recall of memories. Through this we aim to understand how information is learned, represented within different types of memory and finally reconstructed from memory.
Representation Learning for Reinforcement Learning
Reinforcement learning has become an increasingly powerful tool with the advent of neural networks, which provide a powerful all-in-one solution to solve complex tasks. A major downside of this approach, however, is its lack of transparency and transferability - which are important qualities if these systems are to be applied in real-world applications. This is especially important for the many cases where safety or fairness play an important role. By separating the learning of useful data representations from the process of solving a task, we aim to improve explainability as well as flexibility in reinforcement learning to address these issues; With a good representation of input data, the model required to solve a task can be simpler and thus easier to understand. At the same time, it becomes more feasible to transfer already established, transparent solutions to similar tasks.
Our group has developed the highly cited Slow Feature Analysis (SFA), which is able to extract slowly moving features from data. Over recent years, we have extended SFA and applied it to various problem settings. Nowadays, we are also working with more recent deep representation learning methods - especially different flavors of variational autoencoders (VAEs).
Memory-based Reinforcement Learning
Besides the representation learning aspect outlined above, we are interested in model based deep reinforcement learning algorithms for complex domains. In model based reinforcement learning, the goal is to first learn the environmental transition and reward dynamics of a given problem and then use the learned dynamics to solve the problem more data-efficiently. Since stochasticity or partial observability tend to play an important role in complex and high-dimensional settings, memory techniques are commonly used to aggregate information over the course of multiple (possibly many) time steps. Furthermore, to increase robustness and model performance, sampling models have become an increasingly popular choice over to the more common expectation models in recent years.
Our main research focus in this setting lies on the improvement of the planning performance of learned environment models. Thereby, we are specifically interested in designing model architectures that qualify for efficient and precise planning by construction instead of working on the planning procedure itself. To achieve this, we combine the above mentioned sampling models and memory techniques with principles from hierarchical reinforcement learning.
System Level Modeling of the Hippocampus
Remembering what we have done and experienced in the past is essential for defining what we are and deciding what to do in the future. However, our so-called episodic memory is far less reliable than one might think. We neglect, change, and even add things to our memories, often in ways that makes them more in line with what we would generally expect or like. Thus, episodic memory seems to be largely a generative process, where incomplete memory traces are enriched and modified by general so-called semantic information and expectations about the world.
In the neuroscience side of our group we use advanced machine learning techniques and develop a model of the interaction between the episodic and semantic memory system, mainly during storage and retrieval of episodic memories, but also for learning semantics. Our model will describe the interplay between hippocampus and neocortex. We hypothesize that the hippocampus stores and retrieves selected aspects of an episode, which are necessarily incomplete, and the neocortex reasonably fills in the missing information based on general semantic information. For modeling we have used many generative models ranging from restricted boltzmann machines (RBM) to variational autoencoders (VAE) and vector quantized variational autoencoders (VQVAE) and also PixelCNN.
This research is part of an interdisciplinary DFG funded research group "Constructing scenarios of the past: A new framework in episodic memory". We have close interaction with partners from psychology collaborators that study the neural mechanisms via fMRI and behavioral experiments and philosophy partners that address fundamental questions that arise within and about our framework.
Group Leader
Members
Former Members
Dr.-Ing. Alberto N Escalante B
Former Student Members
External Homepage
Our Mission Statement
Career: Help young scientists realize their potential in science and society.
Research: Discover principles of self-organization of intelligent systems.
Education: Teach creativity through mathematical intuition.
Society: Contribute to an informed discussion about artificial intelligence.
Technology: Support technological progress by sharing our expertise.
Insights into Course Difficulty Variations
We analyze how course difficulty varies across student groups and over time using various methods from Curriculum Analytics. It aims to uncover hidden inequities and support fairer, data-driven curriculum decisions.
Exploring the relationship between Slow Feature Analysis (SFA) and the Successor Representation (SR)
SFA and SR are two methods for constructing representations that stem from different areas of machine learning, and which are based upon different principles. Despite this, SFA and SR share a number of key properties, both in terms of their mathematics and the types of information they are sensitive to. This work studies their connection along these two axes.
Probabilistic Perspectives on Slow Feature Analysis
This ongoing project aims to investigate and extend Slow Feature Analysis (SFA) through a probabilistic lens to gather insights on the dependency of optimal features on the process statistics of Markov Decision Processes or reinrepret SFA as variational inference problem.
Simulating Changes in Prerequisite-Free Curricula
In higher education, curricula define the structure of study programs. Changing them can have a major impact on the students following these programs. This project develops a simulation framework to estimate the impact of potential curriculum changes, such as the recommended workload and exam scheduling.
A tutorial on the spectral theory of Markov chains
This project provides an in-depth introduction to Markov chains and explores their connection to graphs and random walks. Tools from linear algebra and graph theory are used to describe the transition matrices of different types of Markov chains, with a particular focus on exploring properties of the eigenvalues and eigenvectors corresponding to these matrices.
Exploration of Deep Double Descent through Ensemble Formation
We empirically decompose the generalization loss of deep neural networks into bias and variance components on an image classification task by constructing ensembles using geometric-mean averaging of the sub-net outputs and we isolate double descent at the variance component of the loss. Our results show that small models afford ensembles that outperform single large models while requiring considerably fewer parameters and computational steps. We also find that deep double descent that depends on the existence of label noise can be mitigated by using ensembles of models subject to identical label noise almost as thoroughly as by ensembles of networks each trained subject to i.i.d. noise.
Predicting Latent Space Representations for Planning
Pairing the EfferenceNet with a good but generic feature map allows us to perform an accurate search in the latent space of manipulating unseen objects. This remarkably simple method, inspired by the neurology of the cerebellum, reveals a promising line of future work. We validate our method by on a viewpoint-matching task derived from the NORB data set.
Combat Task Interference in Multi-Task Model-Based Reinforcement Learning by Using Separate Dynamics Models
In model-free multi-task reinforcement learning (RL), abundant work shows that a shared policy network can improve performance across the different tasks. The rationale behind this is that an agent can learn similarities that all tasks have in common and thus effectively enrich the sample count for all tasks at hand. In model-based multi-task RL however, we found evidence suggesting that a dynamics model can suffer from task confusion or catastrophic interference if it is trained on multiple tasks at once.
Learning gradient-based ICA by neurally estimating mutual information
Several methods of estimating the mutual information of random variables have been developed in recent years. They can prove valuable for novel approaches to learning statistically independent features. In this paper, we use one of these methods, a mutual information neural estimation (MINE) network, to present a proof-of-concept of how a neural network can perform linear ICA. We minimize the mutual information,as estimated by a MINE network, between the output units of a differentiable encoder network. This is done by simple alternate optimization of the two networks. The method is shown to get a qualitatively equal solution to FastICA on blind-source-separation of noisy sources.
Measuring the Data Efficiency of Deep Learning Methods
We propose a new experimental protocol and use it to benchmark the data efficiency — performance as a function of training set size — of two deep learning algorithms, convolutional neural networks (CNNs) and hierarchical information-preserving graph-based slow feature analysis (HiGSFA), for tasks in classification and transfer learning scenarios.
Ongoing project by Stefan Richthofer: Predictable Feature Analysis
To apply Slow Feature Analysis (SFA) to interactive scenarios it needs to deal with a control signal. Predictability is a crucial property of features involving control and this project deals with Predictable Feature Analysis (PFA): an SFA-inspired approach to extract predictable features and leverage them to solve continuous control tasks.
-
Inverting Transformer-based Vision Models