Projects in continual lifelong deep learning
Scalable Machine Learning
Description
Various projects in continual learning are available: in the context of both recurrent and feedforward networks, and in supervised and reinforcement learning.
Some examples of possible thesis topics:
- Learning to continually learn, similar to (Shawn et al. 2020) but using recurrent networks and more principled (but also biologically plausible) approaches for meta-learning to continually learn.
- Understanding catastrophic forgetting in deep networks from a loss landscape view using methods like (Li et al. 2018)
Beaulieu, Shawn, et al. ‘Learning to Continually Learn’. ArXiv:2002.09571 [Cs, Stat], Mar. 2020. arXiv.org, http://arxiv.org/abs/2002.09571.
Li, Hao, et al. ‘Visualizing the Loss Landscape of Neural Nets’. Advances in Neural Information Processing Systems 31, 2018, pp. 6389–99. Neural Information Processing Systems, http://papers.nips.cc/paper/7875-visualizing-the-loss-landscape-of-neural-nets.pdf.