Natural language processing, robotics, video processing, stock market forecasting and other similar tasks require models that can deal with sequence data and understand temporal dependencies. Two major classes of models that have been designed to deal with sequence data are recurrent neural networks (RNNs/LSTMs) and transformer architectures. Designing and understanding these models is a very active and diverse area of research. Applications of these models are also widespread. The recent explosion of interest in topics such as language modelling and machine translation is based on advances in these models which includes GPT-3, DALL-E, etc.
In this course you’ll first understand the fundamentals of recurrent neural networks and transformers that led to these breakthroughs. Then we’ll go through and discuss both seminal and recent research papers on these topics to throw light on algorithms and challenges in this field.
The allocation of the limited seminar places (20) is done via the Moodle course https://moodle.ruhr-uni-bochum.de/enrol/index.php?id=47628 for the corresponding semester. Registration for the winter semester is possible until 10.10.2022. Please enter your degree program in the comment field.
- Course type
- Winter Term 2022/2023
moodle course available
every week on Wednesday from 14:00 to 16:00 in room online.
First appointment is on 19.10.2022
Last appointment is on 08.02.2023
We expect a solid level of mathematics as taught in the Applied Computer Science Bachelor‘s. Tools commonly used in machine learning are
- basic probability theory/statistics (expectations, variance, foundational distributions and densities, markov chains)
- linear algebra (matrices, vectors, eigenvalues/eigenvectors)
- calculus (functions, derivatives/gradients, simple integrals)
The course material is in English, the course language will be English.