42. Rahim Entezari - TU-Graz & CSH - Improving generalization in parameter and data space

# Summary Did you ever had the experience that you where training a network, investing a lot of time in finding the right hyper parameters and testing different initializations to push that validation accuracy over certain threshold? Only to then find out when putting the model into production, that it significantly underperforms? If you did, then you experienced one common problem with deep neural networks. The performance gap between in and out-of distribution generalization. Today on the show PhD Rahim Entezari is giving us a wonderful tour through his PhD journey investigating ways to understand and improve generalization performance of deep neural networks. Rahim will explain how one can improve generalization by different methods in data or in parameter space. We will discuss how using different forms of sparsity, or the efficient creation of deep ensemble networks by permutation of network configurations can improve generalization from a parameter space perspective. Or, from a data perspective where we discuss how data quality and data diversity effects the generalization performance of modern deep neural networks. I hope you enjoy this interview, full of interesting concepts and ideas from deep learning theory. # TOC 00:00:00 Introduction 00:02:18 Background Knowledge 00:06:56 Guest Introduction 00:12:35 Generalization from a Data or Parameter Perspective 00:16:21 In and out of distribution Generalization 00:20:30 Structured and Unstructured Sparsity 00:29:55 Generalization in Parameter space 00:46:56 Generalization in Data space # Sponsors Quantics: Supply Chain Planning for the new normal - the never normal - https://quantics.io/ Belichberg GmbH: We do digital transformations as your innovation partner - https://belichberg.com/ # References Rahim Entezari - https://www.linkedin.com/in/rahimentezari/

There are no comments yet.

Please log in to write the first comment.