Emergent behavior and neural representations in spatial learning

Prof. Dr. Sen Cheng
PI

Sandhiya Vijayabaskaran, M.Sc.
researcher

Dr.-Ing. Thomas Walther
inactive

Computational Neuroscience

Collaborator: Edison Pignaton de Freitas, Paulo Rogerio de Almeida Ribeiro

Funding:

SFB 1280

Animals must forage for food and water, find mates, avoid predators and return to their resting location in order to survive. Spatial navigation and learning are thus vital to the success of many species, and a variety of navigation behaviors and strategies have been observed. Place cells, grid cells, and other diverse cell types discovered in the hippocampus and adjoining areas have been thought to be neural representations that support spatial navigation and learning. However, the mechanisms that lead to the emergence of the multitude of cell types involved in navigation as well as the wide variety of observed navigation strategies are still unclear. We study spatial navigation using deep reinforcement learning to understand how experimentally observed behaviors may emerge in an artificial agent in a virtual environment. To this end, simple standard navigation tasks, such as the Morris Water Maze, as well as more complex paradigms such as extinction learning are used. Once the spatial behavior is learned, we can study the spatial representations that emerged in the network and that allow the artificial agent to navigate. These representations can then be compared to neural codes for space in the hippocampus.

Publications

2023

A Multisession SLAM Approach for RatSLAM
Menezes, M., Muñoz, M., de Freitas, E. P., Cheng, S., de Almeida Neto, A., Ribeiro, P., & Oliveira, A.
Journal of Intelligent & Robotic Systems, 108(4)

A Multisession SLAM Approach for RatSLAM

@article{MenezesMuñozde FreitasEtAl2023, author = {Menezes, Matheus and Muñoz, Mauro and de Freitas, Edison Pignaton and Cheng, Sen and de Almeida Neto, Areolino and Ribeiro, Paulo and Oliveira, Alexandre}, title = {A Multisession SLAM Approach for RatSLAM}, journal = {Journal of Intelligent & Robotic Systems}, volume = {108}, number = {4}, month = {July }, year = {2023}, doi = {10.1007/s10846-023-01816-3}, }

Menezes, M., Muñoz, M., de Freitas, E. P., Cheng, S., de Almeida Neto, A., Ribeiro, P., & Oliveira, A. (2023). A Multisession SLAM Approach for RatSLAM. Journal of Intelligent & Robotic Systems, 108(4). http://doi.org/10.1007/s10846-023-01816-3
CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning
Diekmann, N., Vijayabaskaran, S., Zeng, X., Kappel, D., Menezes, M. C., & Cheng, S.
Frontiers in Neuroinformatics, 17

CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning

@article{DiekmannVijayabaskaranZengEtAl2023, author = {Diekmann, Nicolas and Vijayabaskaran, Sandhiya and Zeng, Xiangshuai and Kappel, David and Menezes, Matheus Chaves and Cheng, Sen}, title = {CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning}, journal = {Frontiers in Neuroinformatics}, volume = {17}, month = {March}, year = {2023}, doi = {10.3389/fninf.2023.1134405}, }

Diekmann, N., Vijayabaskaran, S., Zeng, X., Kappel, D., Menezes, M. C., & Cheng, S.. (2023). CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning. Frontiers in Neuroinformatics, 17. http://doi.org/10.3389/fninf.2023.1134405

2022

xRatSLAM: An Extensible RatSLAM Computational Framework

de Souza Muñoz, M. E., Menezes, M. C., de Freitas, E. P., Cheng, S., de Almeida Ribeiro, P. R., de Almeida Neto, A., & de Oliveira, A. C. M.

Sensors, 22(21), 8305

@article{de Souza MuñozMenezesde FreitasEtAl2022,
	author		=	{de Souza Muñoz, Mauro Enrique and Menezes, Matheus Chaves and de Freitas, Edison Pignaton and Cheng, Sen and de Almeida Ribeiro, Paulo Rogério and de Almeida Neto, Areolino and de Oliveira, Alexandre César Muniz},
	title		=	{xRatSLAM: An Extensible RatSLAM Computational Framework},
	journal		=	{Sensors},
	volume		=	{22},
	number		=	{21},
	pages		=	{8305},
	month		=	{October},
	year		=	{2022},
	doi		=	{10.3390/s22218305},
}

Navigation task and action space drive the emergence of egocentric and allocentric spatial representations
Vijayabaskaran, S., & Cheng, S.
PLOS Computational Biology, 18(10), e1010320

@article{VijayabaskaranCheng2022, author = {Vijayabaskaran, Sandhiya and Cheng, Sen}, title = {Navigation task and action space drive the emergence of egocentric and allocentric spatial representations}, journal = {PLOS Computational Biology}, volume = {18}, number = {10}, pages = {e1010320}, month = {October}, year = {2022}, doi = {10.1371/journal.pcbi.1010320}, }

Vijayabaskaran, S., & Cheng, S.. (2022). Navigation task and action space drive the emergence of egocentric and allocentric spatial representations. PLOS Computational Biology, 18(10), e1010320. http://doi.org/10.1371/journal.pcbi.1010320
Learning Cognitive Map Representations for Navigation by Sensory–Motor Integration
Zhao, D., Zhang, Z., Lu, H., Cheng, S., Si, B., & Feng, X.
IEEE Transactions on Cybernetics, 52(1), 508–521

@article{ZhaoZhangLuEtAl2022, author = {Zhao, Dongye and Zhang, Zheng and Lu, Hong and Cheng, Sen and Si, Bailu and Feng, Xisheng}, title = {Learning Cognitive Map Representations for Navigation by Sensory–Motor Integration}, journal = {IEEE Transactions on Cybernetics}, volume = {52}, number = {1}, pages = {508–521}, month = {January}, year = {2022}, doi = {10.1109/tcyb.2020.2977999}, }

Zhao, D., Zhang, Z., Lu, H., Cheng, S., Si, B., & Feng, X. (2022). Learning Cognitive Map Representations for Navigation by Sensory–Motor Integration. IEEE Transactions on Cybernetics, 52(1), 508–521. http://doi.org/10.1109/tcyb.2020.2977999

2021

Context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach
Walther, T., Diekmann, N., Vijayabaskaran, S., Donoso, J. R., Manahan-Vaughan, D., Wiskott, L., & Cheng, S.
Scientific Reports, 11(1)

@article{WaltherDiekmannVijayabaskaranEtAl2021, author = {Walther, Thomas and Diekmann, Nicolas and Vijayabaskaran, Sandhiya and Donoso, José R. and Manahan-Vaughan, Denise and Wiskott, Laurenz and Cheng, Sen}, title = {Context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach}, journal = {Scientific Reports}, volume = {11}, number = {1}, month = {February}, year = {2021}, doi = {10.1038/s41598-021-81157-z}, }

Walther, T., Diekmann, N., Vijayabaskaran, S., Donoso, J. R., Manahan-Vaughan, D., Wiskott, L., & Cheng, S.. (2021). Context-dependent extinction learning emerging from raw sensory inputs: a reinforcement learning approach. Scientific Reports, 11(1). http://doi.org/10.1038/s41598-021-81157-z

2020

Automatic Tuning of RatSLAM′s Parameters by Irace and Iterative Closest Point
Menezes, M. C., Muñoz, M. E. S., Freitas, E. P., Cheng, S., Walther, T., Neto, A. A., et al.
In IECON 2020 The 46th Annual Conference of the IEEE Industrial Electronics Society (pp. 562–568)

@inproceedings{MenezesMuñozFreitasEtAl2020, author = {Menezes, M. C. and Muñoz, M. E. S. and Freitas, E. P. and Cheng, S. and Walther, T. and Neto, A. A. and Ribeiro, P. R. A. and Oliveira, A. C. M.}, title = {Automatic Tuning of RatSLAM′s Parameters by Irace and Iterative Closest Point}, booktitle = {IECON 2020 The 46th Annual Conference of the IEEE Industrial Electronics Society}, pages = {562–568}, month = {October}, year = {2020}, doi = {10.1109/IECON43393.2020.9254718}, }

Menezes, M. C., Muñoz, M. E. S., Freitas, E. P., Cheng, S., Walther, T., Neto, A. A., et al. (2020). Automatic Tuning of RatSLAM′s Parameters by Irace and Iterative Closest Point. In IECON 2020 The 46th Annual Conference of the IEEE Industrial Electronics Society (pp. 562–568). http://doi.org/10.1109/IECON43393.2020.9254718

2019

A Parallel RatSlam C++ Library Implementation
de Souza Muñoz, M. E., Menezes, M. C., de Freitas, E. P., Cheng, S., de Almeida Neto, A., de Oliveira, A. C. M., & de Almeida Ribeiro, P. R.
In Communications in Computer and Information Science (pp. 173–183) Springer International Publishing

@incollection{de Souza MuñozMenezesde FreitasEtAl2019, author = {de Souza Muñoz, Mauro Enrique and Menezes, Matheus Chaves and de Freitas, Edison Pignaton and Cheng, Sen and de Almeida Neto, Areolino and de Oliveira, Alexandre César Muniz and de Almeida Ribeiro, Paulo Rogério}, title = {A Parallel RatSlam C++ Library Implementation}, booktitle = {Communications in Computer and Information Science}, pages = {173–183}, publisher = {Springer International Publishing}, year = {2019}, doi = {10.1007/978-3-030-36636-0_13}, }

de Souza Muñoz, M. E., Menezes, M. C., de Freitas, E. P., Cheng, S., de Almeida Neto, A., de Oliveira, A. C. M., & de Almeida Ribeiro, P. R. (2019). A Parallel RatSlam C++ Library Implementation. In Communications in Computer and Information Science (pp. 173–183). Springer International Publishing. http://doi.org/10.1007/978-3-030-36636-0_13
Deep reinforcement learning in a spatial navigation task: Multiple contexts and their representation
Diekmann, N., Walther, T., Vijayabaskaran, S., & Cheng, S.
In 2019 Conference on Cognitive Computational Neuroscience Berlin, Germany: Cognitive Computational Neuroscience

@inproceedings{DiekmannWaltherVijayabaskaranEtAl2019, author = {Diekmann, Nicolas and Walther, Thomas and Vijayabaskaran, Sandhiya and Cheng, Sen}, title = {Deep reinforcement learning in a spatial navigation task: Multiple contexts and their representation}, booktitle = {2019 Conference on Cognitive Computational Neuroscience}, publisher = {Cognitive Computational Neuroscience}, address = {Berlin, Germany}, year = {2019}, doi = {10.32470/CCN.2019.1151-0}, }

Diekmann, N., Walther, T., Vijayabaskaran, S., & Cheng, S.. (2019). Deep reinforcement learning in a spatial navigation task: Multiple contexts and their representation. In 2019 Conference on Cognitive Computational Neuroscience. Berlin, Germany: Cognitive Computational Neuroscience. http://doi.org/10.32470/CCN.2019.1151-0

2018

A Neuro-Inspired Approach to Solve a Simultaneous Location and Mapping Task Using Shared Information in Multiple Robots Systems
Menezes, M. C., de Freitas, E. P., Cheng, S., de Oliveira, A. C. M., & de Almeida Ribeiro, P. R.
In 2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV) IEEE

@inproceedings{Menezesde FreitasChengEtAl2018, author = {Menezes, Matheus Chaves and de Freitas, Edison Pignaton and Cheng, Sen and de Oliveira, Alexandre Cesar Muniz and de Almeida Ribeiro, Paulo Rogerio}, title = {A Neuro-Inspired Approach to Solve a Simultaneous Location and Mapping Task Using Shared Information in Multiple Robots Systems}, booktitle = {2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV)}, publisher = {IEEE}, month = {November}, year = {2018}, doi = {10.1109/icarcv.2018.8581270}, }

Menezes, M. C., de Freitas, E. P., Cheng, S., de Oliveira, A. C. M., & de Almeida Ribeiro, P. R. (2018). A Neuro-Inspired Approach to Solve a Simultaneous Location and Mapping Task Using Shared Information in Multiple Robots Systems. In 2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV). IEEE. http://doi.org/10.1109/icarcv.2018.8581270
Autonomous Exploration Guided by Optimisation Metaheuristic
Santos, R. G., de Freitas, E. P., Cheng, S., de Almeida Ribeiro, P. R., & de Oliveira, A. C. M.
In 2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV) IEEE

@inproceedings{Santosde FreitasChengEtAl2018, author = {Santos, Raphael Gomes and de Freitas, Edison Pignaton and Cheng, Sen and de Almeida Ribeiro, Paulo Rogerio and de Oliveira, Alexandre Cesar Muniz}, title = {Autonomous Exploration Guided by Optimisation Metaheuristic}, booktitle = {2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV)}, publisher = {IEEE}, month = {November}, year = {2018}, doi = {10.1109/icarcv.2018.8581136}, }

Santos, R. G., de Freitas, E. P., Cheng, S., de Almeida Ribeiro, P. R., & de Oliveira, A. C. M. (2018). Autonomous Exploration Guided by Optimisation Metaheuristic. In 2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV). IEEE. http://doi.org/10.1109/icarcv.2018.8581136