Probing Length Generalization in Mamba via Image Reconstruction
We explore Mamba's length generalization capabilities through a toy vision task in which Mamba is trained to reconstruct images from patch sequences.
Jan Rathjens, M.Sc.
Theory of Neural Systems
Theory of Neural Systems
Ruhr-Universität Bochum
Institut für Neuroinformatik
Universitätsstraße 150
Building NB, Room NB 3/70
Universitätsstraße 150
Building NB, Room NB 3/70
D-44801 Bochum, Germany
-
Probing Length Generalization in Mamba via Image ReconstructionRathjens, J., Schiewer, R., Wiskott, L., & Subramoney, A.In L. Pasa, Lintas, A., Tetko, I. V., Micheli, A., Navarin, N., Villa, A. E. P., et al. (Eds.), Artificial Neural Networks and Machine Learning – ICANN 2026 (pp. 235–246) Cham: Springer Nature Switzerland
2026
Understanding Transformer-Based Vision Models via Modular Feature InversionRathjens, J., Reyhanian, S., Kappel, D., & Wiskott, L.Transactions on Machine Learning ResearchLearning What to Remember: Semantic Attention in Generative Episodic Memory ModelFayyaz, Z., Rathjens, J., Cheng, S., & Wiskott, L.In Proceedings of the International Joint Conference on Neural Networks (IJCNN)Probing Length Generalization in Mamba via Image Reconstruction2025
Understanding Transformer-based Vision Models through InversionRathjens, J., Reyhanian, S., Kappel, D., & Wiskott, L.arXiv e-print2024
Antagonism between Classification and Reconstruction Processes in Deep Predictive Coding NetworksRathjens, J., & Wiskott, L.In ESANN 2024 proceesdings (pp. 161–166) Bruges (Belgium) and online: Ciaco - i6doc.comClassification and Reconstruction Processes in Deep Predictive Coding Networks: Antagonists or Allies?Summer Term 2026
Lab courses An Introduction to Python for Data Analysis Summer Term 2025
Lab courses An Introduction to Python for Data Analysis Summer Term 2024
Lab courses An Introduction to Python for Data Analysis Summer Term 2022
Lectures Database Systems
2025 
2024 Understanding Transformer-based Vision Models through Inversion
In this study, we revisit feature inversion, introducing a novel, modular variation that enables significantly more efficient application of the technique. We demonstrate how our method can be systematically applied to the large-scale transformer-based vision models, Detection Transformer and Vision Transformer, and how reconstructed images can be qualitatively interpreted in a meaningful way.
2023 Interplay between Classification- and Reconstruction-driven Processes
We analyze the interaction between classification- and reconstruction-driven processes in deep neural networks, finding an antagonistic rather than a synergistic relationship.
The Institut für Neuroinformatik (INI) is a research unit of the Faculties of Computer Science and Medicine at the Ruhr-Universität Bochum. Its scientific goal is to understand the fundamental principles through which organisms generate behavior and cognition while linked to their environments through sensory and effector systems. Inspired by our insights into such natural cognitive systems, we seek new solutions to problems of information processing in artificial cognitive systems. We draw from a variety of disciplines that include experimental psychology and neurophysiology as well as machine learning, neural artificial intelligence, computer vision, and robotics.
Universitätsstr. 150, Building NB, Room 3/32
D-44801 Bochum, GermanyTel: (+49) 234 32-28967
Fax: (+49) 234 32-14210