The dentate gyrus in the hippocampus is special in that it generates new neurons even in adult animals. In this project we investigate what the functional role of this adult neurogenesis might be. Our basic hypothesis is that it is a way to resolve the stability-plasticity dilemma the dentate gyrus might be faced with.
Since neural networks store patterns in a distributed fashion in the synaptic weights, the traces of old patterns degrade as new patterns are stored, a phenomenon known as catastrophic interference; see Fig. 1. The stability-plasticity dilemma arises from the fact that one can either chose a low learning rate, to make stored patterns stable, or a high learning rate, to make the network plastic and enable it to learn new patterns. However, one cannot have both, stability of old patterns and the plasticity to learn new patterns. One common way to overcome this dilemma is to train the network in an interleaved fashion, meaning that all patterns are presented repeatedly, either in some alternating order or randomly; see Fig. 2. In that case all patterns are learned simultaneously. An alternative we hypothesize for the dentate gyrus is to engage new neurons for qualitatively new patterns; see Fig. 3.
Figure 1: A network is successively trained for pattern associations A-A', B-B', and C-C'. If each pattern pair is presented only once, the learning rate must be high and while the weights adapt to a new pattern pair previously learned associations are forgotten, indicated by the colors of the connections. This is known as catastrophic interference. (Reducing the learning rate and presenting each pattern repeatedly would not help as long as one doesn't interleave the presentations; see Fig. 2.)
Figure 2: One way to avoid catastrophic interference is to train the network repeatedly on all patterns in an interleaved fashion with a small learning rate, in which case the different pattern associations are learned simultaneously and the weights combine the information for the different patterns, indicated by brown connections.
Figure 3: Another solution is to freeze the learned connections and add new neurons for new patterns. Only the new neurons have plastic connections and can be trained with a high learning rate.
It is commonly assumed that the hippocampus helps in resolving the stability-plasticity dilemma for the cortex. According to this theory, the hippocampus stores patterns quickly, and thus suffers from catastrophic interference to some extent (a mild form of Fig. 1), and then replays the stored patterns to the cortex where they are basically learned in an interleaved fashion. The hippocampus has three stages, the dentage gyrus (DG), CA3, and CA1 (plus subiculum). CA3 is assumed to be the actual storage while DG performs an encoding and CA1 a decoding of the patterns to permit an efficient storage of patterns in CA3. We believe that the dentate gyrus also has a potential stability-plasticity dilemma, since it has to learn new efficient encoding schemes, e.g. if an animal is transferred to a qualitatively new environment, while maintaining the old ones. Our hypothesis is that is does so in the fashion illustrated in Figure 3 by keeping old connections fixed and adding new neurons. Figure 4 shows a schematic overview of the cortico-hippocampal system. The colors of the connections indicate that CA3 actually suffers from catastrophic interference while the cortex and dentate gyrus avoid it by interleaved training and neurogenesis, respectively.
Figure 4: An overview over the cortico-hippocampal system. Cortex and CA3 serve as associative memories and have strong recurrent connections. Dentate Gyrus (DG) encodes and CA1 decodes information stored in CA3. Red connections in CA3 indicate that it suffers (mildly) from catastrophic interference; cf. Fig. 1. Brown connections in the cortical memory indicate interleaved learning; cf. Fig. 2. Colored connections in the dentate gyrus indicate learning with neurogenesis, cf. Fig. 3.