L11.6 Xavier Glorot and Kaiming He Initialization

Sebastian's books: https://sebastianraschka.com/books/I

MPORTANT NOTE: In the video, I talk about the number of input units in the denominator ("fan in"), but to be correct, it should have been number of input units for both the current and the next layer ("fan in" + "fan out").

Slides: https://sebastianraschka.com/pdf/lect...

Papers:

Xavier Glorot and Yoshua Bengio. "Understanding the difficulty of training deep feedforward neural networks." Proceedings of the thirteenth international conference on artificial intelligence and statistics. 2010. http://proceedings.mlr.press/v9/gloro...]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification." In Proceedings of the IEEE international conference on computer vision, pp. 1026-1034. 2015. https://arxiv.org/abs/1502.01852

-------

This video is part of my Introduction of Deep Learning course.

Next video:    • L11.7 Weight Initialization in PyTorc...  

The complete playlist:    • Intro to Deep Learning and Generative...  

A handy overview page with links to the materials: https://sebastianraschka.com/blog/202...

-------

If you want to be notified about future videos, please consider subscribing to my channel:    / sebastianraschka