Jun.-Prof. Dr. Tobias Glasmachers

Institut für Neuroinformatik
Ruhr-Universität Bochum
Universitätsstraße 150
Building NB, Room NB 3/27
D-44801 Bochum
Germany
Theory of Machine Learning

Groups

Tobias Glasmachers

About Me

I am a junior professor for theory of machine learning at the Institut für Neuroinformatik, Ruhr-Universität Bochum, Germany. My research interests are (supervised) machine learning and optimization.

Short CV

  • 2004-2008: Ph.D. in Christian Igel's group at the Institut für Neuroinformatik in Bochum. I received my Ph.D. in 2008 from the Faculty of Mathematics, Ruhr-Universität Bochum, Germany.
  • 2008-2009: Post-doc in the same group.
  • 2009-2011: Post-doc in Jürgen Schmidhuber's group at IDSIA, Lugano, Switzerland.
  • since 2012: Junior professor for theory of machine learning at the Institut für Neuroinformatik, Ruhr-Universität Bochum, Germany. I am the head of the optimization of adaptive systems group.

SVM model selectionResearch

My research is located in the area of machine learning, a modern branch of artificial intelligence research. This is an interdisciplinary research topic in between computer science, statistics, and optimization, with connections to the neurosciences and applications in robotics, engineering, medicine, economics, and many more disciplines. Within this wide area I am focusing on two aspects: supervised learning (including modern deep learning), and optimization with simple gradient-based methods and evolutionary algorithms.

Machine Learning

Supervised learning is a learning paradigm with endless (mostly technical) applications. A learning machine (algorithm) builds a predictive model from data provided in the form of input/output pairs. This allows for the automated solution of classification and regression problems. A primary example is classification of objects in images, a classic computer vision task. I have recently started to reach out to reinforcement learning problems in 3D environments for fully autonomous behavior learning of robots or computer game agents (bots). My research activities include both theoretical and practical aspects.

Optimization

Gradient-based optimization, particularly relatively simple first order methods like (stochastic) gradient descent and coordinate descent, are at the heart of many modern training procedures for learning machines, in particular for (possibly regularized) empirical risk minimization. This includes backpropagation based training of (deep) neural networks, as well as convex (primal or dual) optimization, e.g., for support vector machine training.

Evolutionary Algorithms (EAs) are a class of nature-inspired algorithms that mimic the process of Darwinian evolution. This process is resolved into the components inheritance, variation, and selection. It has been widely recognized that EAs are useful for search and optimization, in particular when derivatives are not available. Formally they can be understood as randomized direct search heuristics. They are suitable for tackling black-box optimization problems. I focus on evolution strategies, a class of optimization algorithms for continuous variables, and on multi-objective optimization.

Shark

I am an active developer of the Shark Machine Learning Library. Shark is an open-source, modular, and fast C++ library. Check it out!

Asynchronous ES

An asynchronous natural evolution strategy.

Adaptive Coordinate Frequencies Coordinate Descent

Coordinate descent with online adaptation of coordinate frequencies for fast training of linear models.

LASSO-code, modified liblinear.

Hypervolume Maximization

Maximization of dominated hypervolume for multi-objective benchmark problems.

xCMA-ES

CMA-ES with multiplicative covariance update.

Pareto Archive

An efficient archiving algorithm for non-dominated solutions in multi-objective optimization.

Stochastic Gradient Optimization

Comparison of SGD, SAG, SVRG, and ADAM for training kernel machines on a budget.

Limits of End-to-end Learning

Code for reproducing the experiments in the paper.

Duales Training nichtlinearer Support-Vektor-Maschinen mit Budget

This DFG funded research project has started in October 2016.

The Black-box Optimization Competition (BBComp)

The Black-box Optimization Competition (BBComp) is an online competition for black-box optimization in the continuous domain. It is the first competition of its kind where problems are truly black-boxes to participants. This competition allows for a fair and unbiased (as unbiased as possible) comparison of black box optimization methods. The large problem suite and the black-box interface avoid over-fitting to narrow suites of benchmark problems.

Support-Vektor-Maschinen für extrem große Datenmengen

This research project had started in November 2013 and ended in February 2016. It was conducted in cooperation with the chair Computergestützte Statistik at the Technical University of Dortmund. It was funded by the Mercator Research Center Ruhr (MERCUR). The official project homepage is found here.

Glasmachers, T. (2017). A Fast Incremental BSP Tree Archive for Non-dominated Points. In Evolutionary Multi-Criterion Optimization (EMO). Springer.
Glasmachers, T. (2017). Limits of End-to-End Learning. In Proceedings of the 9th Asian Conference on Machine Learning (ACML).
Glasmachers, T. (2017). Global Convergence of the (1+1) Evolution Strategy (No. arxiv:1706.02887). arxiv.org.
Irmer, T., Glasmachers, T., & Maji, S. (2017). Texture attribute synthesis and transfer using feed-forward CNNs. In Winter Conference on Applications of Computer Vision (WACV). IEEE.
Krause, O., Glasmachers, T., & Igel, C. (2017). Qualitative and Quantitative Assessment of Step Size Adaptation Rules. In Conference on Foundations of Genetic Algorithms (FOGA). ACM.
Demircioğlu, A., Horn, D., Glasmachers, T., Bischl, B., & Weihs, C. (2016). Fast model selection by limiting SVM training times (No. arxiv:1302.1602.03368v1). arxiv.org.
Doğan, Ü., Glasmachers, T., & Igel, C. (2016). A Unified View on Multi-class Support Vector Classification. Journal of Machine Learning Research, 17(45), 1–32.
Glasmachers, T. (2016). Finite Sum Acceleration vs. Adaptive Learning Rates for the Training of Kernel Machines on a Budget. In NIPS workshop on Optimization for Machine Learning.
Glasmachers, T. (2016). Small Stochastic Average Gradient Steps. In NIPS workshop on Optimizing the Optimizers.
Horn, D., Demirciğlu, A., Bischl, B., Glasmachers, T., & Weihs, C. (2016). A Comparative Study on Large Scale Kernelized Support Vector Machines. Advances in Data Analysis and Classification (ADAC), 1–17.
Krause, O., Glasmachers, T., Hansen, N., & Igel, C. (2016). Unbounded Population MO-CMA-ES for the Bi-Objective BBOB Test Suite. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO).
Krause, O., Glasmachers, T., & Igel, C. (2016). Multi-objective Optimization with Unbounded Solution Sets. In NIPS workshop on Bayesian Optimization.
Loshchilov, I., & Glasmachers, T.. (2016). Anytime Bi-Objective Optimization with a Hybrid Multi-Objective CMA-ES (HMO-CMA-ES). In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO).
Weihs, C., & Glasmachers, T.. (2016). Supervised Classification. In C. Weihs, Jannach, D., Vatolkin, I., & Rudolph, G. (Eds.), Music Data Analysis: Foundations and Applications.
Krause, O., & Glasmachers, T.. (2015). A CMA-ES with Multiplicative Covariance Matrix Updates. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO).
Danafar, S., Rancoita, P. M. V., Glasmachers, T., Whittingstall, K., & Schmidhuber, J. (2014). Testing Hypotheses by Regularized Maximum Mean Discrepancy. International Journal of Computer and Information Technology (IJCIT), 3(2).
Glasmachers, T. (2014). Handling Sharp Ridges with Local Supremum Transformations. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO).
Glasmachers, T. (2014). Optimized Approximation Sets for Low-dimensional Benchmark Pareto Fronts. In Parallel Problem Solving from Nature (PPSN). Springer.
Glasmachers, T., Naujoks, B., & Rudolph, G. (2014). Start Small, Grow Big - Saving Multiobjective Function Evaluations. In Parallel Problem Solving from Nature (PPSN). Springer.
Wierstra, D., Schaul, T., Glasmachers, T., Sun, Y., Peters, J., & Schmidhuber, J. (2014). Natural Evolution Strategies. Journal of Machine Learning Research, 15, 949–980.
Glasmachers, T. (2013). A Natural Evolution Strategy with Asynchronous Strategy Updates. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO).
Glasmachers, T. (2013). The Planning-ahead SMO Algorithm (No. arxiv:1305.0423v1). arxiv.org.
Glasmachers, T., & Doğan, Ü. (2013). Accelerated Coordinate Descent with Adaptive Coordinate Frequencies. In Proceedings of the fifth Asian Conference on Machine Learning (ACML).
Krause, O., Fischer, A., Glasmachers, T., & Igel, C. (2013). Approximation properties of DBNs with binary hidden units and real-valued visible units. In Proceedings of the International Conference on Machine Learning (ICML).
Doğan, Ü., Glasmachers, T., & Igel, C. (2012). Turning Binary Large-margin Bounds into Multi-class Bounds. In ICML workshop on RKHS and kernel-based methods.
Doğan, Ü., Glasmachers, T., & Igel, C. (2012). A Note on Extending Generalization Bounds for Binary Large-margin Classifiers to Multiple Classes. In Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases (ECML-PKDD).
Glasmachers, T. (2012). Convergence of the IGO-Flow of Isotropic Gaussian Distributions on Convex Quadratic Problems. In C. C. Coello, Cutello, V., Deb, K., Forrest, S., Nicosia, G., & Pavone, M. (Eds.), Parallel Problem Solving from Nature (PPSN). Springer.
Glasmachers, T., Koutník, J., & Schmidhuber, J. (2012). Kernel Representations for Evolving Continuous Functions. Journal of Evolutionary Intelligence, 5(3), 171–187. http://doi.org/10.1007/s12065-012-0070-y
Cuccu, G., Gomez, F., & Glasmachers, T.. (2011). Novelty Restarts for Evolution Strategies. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC). IEEE.
Glasmachers, T., & Schmidhuber, J. (2011). Optimal Direct Policy Search. In Proceedings of the 4th Conference on Artificial General Intelligence (AGI).
Graziano, V., Glasmachers, T., Schaul, T., Pape, L., Cuccu, G., Leitner, J., & Schmidhuber, J. (2011). Artificial Curiosity for Autonomous Space Exploration. ACTA FUTURA.
Schaul, T., Glasmachers, T., & Schmidhuber, J. (2011). High Dimensions and Heavy Tails for Natural Evolution Strategies. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO).
Schaul, T., Pape, L., Glasmachers, T., Graziano, V., & Schmidhuber, J. (2011). Coherence Progress: A Measure of Interestingness Based on Fixed Compressors. In Proceedings of the 4th Conference on Artificial General Intelligence (AGI).
Glasmachers, T. (2010). Universal Consistency of Multi-Class Support Vector Classification. In Advances in Neural Information Processing Systems (NIPS).
Glasmachers, T., & Igel, C. (2010). Maximum Likelihood Model Selection for 1-Norm Soft Margin SVMs with Multiple Parameters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(8), 1522–1528.
Glasmachers, T., Schaul, T., & Schmidhuber, J. (2010). A Natural Evolution Strategy for Multi-Objective Optimization. In Parallel Problem Solving from Nature (PPSN). Springer.
Glasmachers, T., Schaul, T., Sun, Y., Wierstra, D., & Schmidhuber, J. (2010). Exponential Natural Evolution Strategies. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO).
Sun, Y., Glasmachers, T., Schaul, T., & Schmidhuber, J. (2010). Frontier Search. In Proceedings of the 3rd Conference on Artificial General Intelligence (AGI).
Glasmachers, T. (2008). On related violating pairs for working set selection in SMO algorithms. In M. Verleysen (Ed.), Proceedings of the 16th European Symposium on Artificial Neural Networks (ESANN). d-side publications.
Glasmachers, T. (2008). Gradient Based Optimization of Support Vector Machines. Doctoral thesis, Fakultät für Mathematik, Ruhr-Universität Bochum, Germany.
Glasmachers, T., & Igel, C. (2008). Second-Order SMO Improves SVM Online and Active Learning. Neural Computation, 20(2), 374–382.
Glasmachers, T., & Igel, C. (2008). Uncertainty Handling in Model Selection for Support Vector Machines. In G. Rudolph, Jansen, T., Lucas, S., Poloni, C., & Beume, N. (Eds.), Parallel Problem Solving from Nature (PPSN) (pp. 185–194). Springer.
Igel, C., Heidrich-Meisner, V., & Glasmachers, T.. (2008). Shark. Journal of Machine Learning Research, 9, 993–996.
Igel, C., Glasmachers, T., Mersch, B., Pfeifer, N., & Meinicke, P. (2007). Gradient-Based Optimization of Kernel-Target Alignment for Sequence Kernels Applied to Bacterial Gene Start Detection. IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB), 4(2), 216–226.
Mersch, B., Glasmachers, T., Meinicke, P., & Igel, C. (2007). Evolutionary Optimization of Sequence Kernels for Detection of Bacterial Gene Starts. International Journal of Neural Systems, 17(5), 369–381.
Glasmachers, T. (2006). Degeneracy in Model Selection for SVMs with Radial Gaussian Kernel. In M. Verleysen (Ed.), Proceedings of the 14th European Symposium on Artificial Neural Networks (ESANN). d-side publications.
Glasmachers, T., & Igel, C. (2006). Maximum-Gain Working Set Selection for Support Vector Machines. Journal of Machine Learning Research, 7, 1437–1466.
Mersch, B., Glasmachers, T., Meinicke, P., & Igel, C. (2006). Evolutionary Optimization of Sequence Kernels for Detection of Bacterial Gene Starts. In Proceedings of the 16th International Conference on Artificial Neural Networks (ICANN). Springer-Verlag.
Glasmachers, T., & Igel, C. (2005). Gradient-based Adaptation of General Gaussian Kernels. Neural Computation, 17(10), 2099–2105.

I am offering Master theses in the areas of machine learning and optimization. Prerequisites:

  • completed at least one of my lectures
  • programming skills and/or solid mathematical background

Please contact me for details and for currently open topics.