Скачать или смотреть Sparse Training in Supervised and Unsupervised Deep Learning

The talk starts with a quick overview of Prof. Mocanu's research line. Next, he will introduce along this line one of the many challenges (the dense connectivity) which prevents them from having scalable artificial neural networks at both levels, cloud and edge computing. An emerging state-of-the-art possible solution is presented, i.e., sparse-to-sparse training with dynamic sparsity, which reduces the number of connections up to quadratically while persistently improving the performance.

The discussion starts from the first sparse training works on complex Boltzmann machines [Mocanu et al., Machine Learning 2016] and sparse evolutionary training [Mocanu et al., Nature Communications 2018] in the typical single task (un)supervised learning. Further, it gradually introduces newer approaches in the more challenging contexts of continual or federated learning. Besides the fundamental theoretical novelty, some practical aspects, such as truly sparse implementations and deep learning energy efficiency are also considered and open for discussion.

This seminar was given by Decebal Mocanu who is Associate Professor in Machine Learning within the Department of Computer Science at the University of Luxembourg; a Guest Faculty Member within the M&CS department at the Eindhoven University of Technology (TU/e); and an alumni member of TU/e Young Academy of Engineering. In 2017, Decebal received his PhD degree from TU/e. During his doctoral studies and after that, Decebal undertook four research visits at the University of Pennsylvania (2014), Julius Maximilians University of Wurzburg (2015), the University of Texas at Austin (2016), and the University of Alberta (2022).

In the long term, Decebal is interested in studying the synergy between artificial intelligence, neuroscience, and network science for the benefits of science and society.