Reduction of Variance-related Error through Ensembling: Deep Double Descent and Out-of-Distribution Generalization
Rath-Manakidis, P.,
Hlynsson, H. D., &
Wiskott, L.In
Proceedings of the 11th International Conference on Pattern Recognition Applications and Methods - ICPRAM, (pp. 31–40) SciTePress
@inproceedings{Rath-ManakidisHlynssonWiskott2022,
author = {Rath-Manakidis, Pavlos and Hlynsson, Hlynur Davíð and Wiskott, Laurenz},
title = {Reduction of Variance-related Error through Ensembling: Deep Double Descent and Out-of-Distribution Generalization},
booktitle = {Proceedings of the 11th International Conference on Pattern Recognition Applications and Methods - ICPRAM,},
pages = {31–40},
organization = {INSTICC},
publisher = {SciTePress},
year = {2022},
doi = {10.5220/0010821300003122},
}
Summer Term 2023
Winter Term 2022/2023
Summer Term 2022
Our research explores explainable object detection by utilizing prototypal features. This approach enables the development of transparent AI models where the decision-making process is visualized and can be intuitively understood, enhancing user trust and allowing for interaction.
tachAId is an interactive guide offering concrete recommendations for developing human-centered AI across its lifecycle. It helps stakeholders build user-focused AI by addressing key challenges like fairness and transparency, bridging ethical principles with technical solutions.
We empirically decompose the generalization loss of deep neural networks into bias and variance components on an image classification task by constructing ensembles using geometric-mean averaging of the sub-net outputs and we isolate double descent at the variance component of the loss.
Our results show that small models afford ensembles that outperform single large models while requiring considerably fewer parameters and computational steps. We also find that deep double descent that depends on the existence of label noise can be mitigated by using ensembles of models subject to identical label noise almost as thoroughly as by ensembles of networks each trained subject to i.i.d. noise.