Memory-augmented Neural Network in spatial navigation Computational Neuroscience

Description

Deep learning has gained enormous success in various fields in the last decade. In majority of the models, a deep artificial neural network (DNN) alone functions as the classifier or controller in the task, and the corresponding research usually focuses on searching for better network architectures or training algorithms to improve the performance of the models. However, one shortage of these network-alone algorithms is their struggle of capturing long temporal relationships in the data. Recurrent neural network, especially long-shot term memory (LSTM) have been proposed to tackle this problem but the temporal distance they can deal with is still limited. Recently, a new kind of architecture has emerged where a DNN makes inference coupled with an external memory and is referred to as Memory-augmented Neural Network (MANN) [1, 2, 3]. Unlike the memory used in traditional algorithm such as Deep Q learning [4], where the stored data is replayed to speed up the training of the DNN and gets discarded once the training is done, the memory in MANN works as a functional unit together with the DNN and participates in the inference process. In these cases, the DNN needs to learn to interact not only with the external data, but also with the memory module. This architecture endows the model further power to solve tasks which require even longer temporal relationships for their solutions because the model can store information from a remote past and retrieve it whenever needed.

For this master thesis, the student needs to implement a MANN model based on [3] and use it to solve spatial navigation problems. Potential research questions such as “what kind of spatial leaning task is most suitable for the MANN model to solve?”, “What is the policy of encoding and retrieving memory that the model comes up with after training?”, “What are the contents of the stored memories?”, etc. will be selectively addressed. The student can also formulate their own research questions. There has been a Pytorch implementation of the model from the Internet, but the student needs to implement a Keras/Tensorflow version to make the algorithm compatible with the framework which has been developed within the research group for studying spatial navigation.

Require skills:

  • Good programming skills and experiences with Python is a must.

  • Knowledge in machine learning and experiences in Tensorflow are strongly preferable.

  • Interests in neuroscience is a plus.

References:

[1] Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka GrabskaBarwinska, Sergio Gómez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, et al. Hybrid computing using a neural network with dynamic external memory. Nature, 538(7626):471, 2016

[2] Arbaaz Khan, Clark Zhang, Nikolay Atanasov, Konstantinos Karydis, Vijay Kumar, and Daniel D Lee. Memory augmented control networks. arXiv preprint arXiv:1709.05706, 2017.

[3] Tsendsuren Munkhdalai, Alessandro Sordoni, Tong Wang, Adam Trischler. Metalearned Neural Memory. arXiv preprint arXiv:1907.09720, 2019.

[4] Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529–533, 2015

Supervisors:

Xiangshuai Zeng and Prof. Dr. Sen Cheng

The Institut für Neuroinformatik (INI) is a central research unit of the Ruhr-Universität Bochum. We aim to understand the fundamental principles through which organisms generate behavior and cognition while linked to their environments through sensory systems and while acting in those environments through effector systems. Inspired by our insights into such natural cognitive systems, we seek new solutions to problems of information processing in artificial cognitive systems. We draw from a variety of disciplines that include experimental approaches from psychology and neurophysiology as well as theoretical approaches from physics, mathematics, electrical engineering and applied computer science, in particular machine learning, artificial intelligence, and computer vision.

Universitätsstr. 150, Building NB, Room 3/32
D-44801 Bochum, Germany

Tel: (+49) 234 32-28967
Fax: (+49) 234 32-14210