Avatar
jonas.lins@ini.rub.de

Dr. Jonas Lins

Theory of Cognitive Systems

Theory of Embodied Cognition

jonas.lins@ini.rub.de

 

 

Research Interests

We use language to talk about our environment and we use our senses and effectors to interact with the world directly. Strangely, language conveys information in a radically different format than that used by our perceptual and motor systems. Language is composed of clearly delimited units, and the perceptual form of these units is in  a way arbitrary, that is, not structurally related to the world in any obvious manner. In contrast, the things and actions which these units or their combinations refer to are represented by metrically organized distributions of activation over neural populations -- which often bear a structural resemblance to their counterparts in the world (a visual shape, a movement trajectory, etc.).

I am interested in the neural mechanisms which map between these two representational formats. This mapping requires to scrutinize the continuous neural activation landscapes that arise from the sensory and motor surfaces in order to extract those aspects that are described by an utterance or that we want to talk about. This in turn requires controlled mechanisms of detection and selection that go beyond the mere "representation" of the environment.

I have approached this topic using the test case of spatial language about relations, such as "The red cup to the left of the blue cup.", exploring how the components of such a sentence may be linked to their visual counterparts in a scene. Spatial relations provide an interesting test bed for the overarching topic of language grounding since they are concepts which need to be actively constructed by neural processes and for which we do not possess dedicated sensory organs.

In collaboration with Mathis Richter, I have developed an operational neural process model that implements a neural mechanism for the linkage between spatial language and visual scenes. It is based on the theoretical framework of Dynamic Field Theory. This framework allows to model the evolution of activation distributions in neural populations in a biologically plausible manner. It is based in neurophysiological research about how neurons close to the sensorimotor surfaces interact to endow motor and sensory decisions with stability, which we believe is a hallmark of all cognition.

In a second step, I have complemented this theoretical research with behavioral experiments that aim to measure the selection processes within sensorimotor representations that, according to the model, are associated with language understanding. To infer these processes, I leverage the close link and overlap between sensory and motor representations, showing deviations in response trajectories people produce when they select the visual object described by a sentence such as  "The red item above the green one.". These deviations typically consist of biases into the direction of non-target objects, such as a distractor object that shares the target's color, or the reference object of the phrase (the green one in the example). That these items evoke attraction effects even though they are not behavioral targets suggests that the attraction stems from the implication of these items in the cascade of neural processes that links the linguistic input to the visual scene.

Selected Publications

    2019

  • Computer mouse tracking reveals motor signatures in a cognitive task of spatial language grounding
    Lins, J., & Schöner, G.
    Attention, Perception, & Psychophysics
    pdf

    • 2017

  • A neural dynamic model generates descriptions of object-oriented actions
    Richter, M., Lins, J., & Schöner, G.
    Topics in Cognitive Science, 9(1), 35–47
    A neural dynamic model generates descriptions of object-oriented actions (PDF)

    • 2014

  • Neural Fields
    Lins, J., & Schöner, G.
    In S. Coombes, beim Graben, P., Potthast, R., & , J. W. (Eds.) (pp. 319–339) Springer Berlin Heidelberg
    pdf

    2021

  • A Neural Dynamic Model of the Perceptual Grounding of Spatial and Movement Relations
    Richter, M., Lins, J., & Schöner, G.
    Cognitive Science, 45(10), e13045
    Paper (open publication) Link to Journal

    • 2020

  • Scene memory and spatial inhibition in visual search: A neural dynamic process model and new experimental evidence
    Grieben, R., Tekülve, J., Zibner, S. K. U., Lins, J., Schneegans, S., & Schöner, G.
    Attention, Perception, & Psychophysics
    SpringerLink (Open Access)
  • Grounding Spatial Language in Perception by Combining Concepts in a Neural Dynamic Architecture
    Sabinasz, D., Richter, M., Lins, J., & Schöner, G.
    In S. Denison, Mack, M., Xu, Y., & Armstrong, B. C. (Eds.), Proceedings of the 42nd Annual Conference of the Cognitive Science Society (pp. 620–626) Cognitive Science Society
    pdf

    • 2019

  • Computer mouse tracking reveals motor signatures in a cognitive task of spatial language grounding
    Lins, J., & Schöner, G.
    Attention, Perception, & Psychophysics
    pdf

    • 2017

  • Mouse Tracking Shows Attraction to Alternative Targets While Grounding Spatial Relations
    Lins, J., & Schöner, G.
    In Proceedings of the 39th Annual Conference of the Cognitive Science Society Austin, TX: Cognitive Science Society
  • A neural dynamic model generates descriptions of object-oriented actions
    Richter, M., Lins, J., & Schöner, G.
    Topics in Cognitive Science, 9(1), 35–47
    A neural dynamic model generates descriptions of object-oriented actions (PDF)

    • 2016

  • A neural dynamic model parses object-oriented actions
    Richter, M., Lins, J., & Schöner, G.
    In A. Papafragou, Grodner, D., Mirman, D., & Trueswell, J. C. (Eds.), Proceedings of the 38th Annual Conference of the Cognitive Science Society (pp. 1931–1936) Austin, TX: Cognitive Science Society
    pdf

    • 2015

  • The Dynamics of Neural Activation Variables
    Reimann, H., Lins, J., & Schöner, G.
    Paladyn, Journal of Behavioral Robotics, 6(1), 57–70
    pdf

    • 2014

  • Neural Fields
    Lins, J., & Schöner, G.
    In S. Coombes, beim Graben, P., Potthast, R., & , J. W. (Eds.) (pp. 319–339) Springer Berlin Heidelberg
    pdf
  • Autonomous Neural Dynamics to Test Hypotheses in a Model of Spatial Language
    Richter, M., Lins, J., Schneegans, S., Sandamirskaya, Y., & Schöner, G.
    In P. Bello, Guarini, M., McShane, M., & Scassellati, B. (Eds.), Proceedings of the 36th Annual Conference of the Cognitive Science Society (pp. 2847–2852) Austin, TX: Cognitive Science Society
    pdf
  • A neural dynamic architecture resolves phrases about spatial relations in visual scenes
    Richter, M., Lins, J., Schneegans, S., & Schöner, G.
    In 24th International Conference on Artificial Neural Networks (ICANN) (pp. 201–208) Heidelberg, Germany: Springer
    pdf

    • 2012

  • The Function and Fallibility of Visual Feature Integration: A Dynamic Neural Field Model of Illusory Conjunctions
    Lins, J., Schneegans, S., Spencer, J., & Schöner, G.
    Frontiers in Computational Neuroscience, (128)
    pdf

About the INI

The Institut für Neuroinformatik (INI) is a research unit of the Faculty of Computer Science at the Ruhr-Universität Bochum. Its scientific goal is to understand the fundamental principles through which organisms generate behavior and cognition while linked to their environments through sensory and effector systems. Inspired by our insights into such natural cognitive systems, we seek new solutions to problems of information processing in artificial cognitive systems. We draw from a variety of disciplines that include experimental psychology and neurophysiology as well as machine learning, neural artificial intelligence, computer vision, and robotics.

Contact

Universitätsstr. 150, Building NB, Room 3/32
D-44801 Bochum, Germany

Tel: (+49) 234 32-28967
Fax: (+49) 234 32-14210

sekretariat@ini.rub.de
www.ini.rub.de