Deriving Matrix Equations for Backpropagation on a Linear Layer

Описание к видео Deriving Matrix Equations for Backpropagation on a Linear Layer

Doing the index tracking to figure out the matrix form of backpropagation is one of the more tedious aspects of working with neural networks but still quite useful to go through in detail every now and then. I can't claim you'll find this video entertaining or particularly interesting, but I hope some of you will find it useful.

Note that at 1:53 I made a mistake. It should be that b ∈ R^N. The batch dimension B was already accounted for when I wrote the bias matrix as repeated rows of b.

Sections:
0:00 - Setting up notation
6:50 - ∂L / ∂W
20:10 = ∂L / ∂b
23:30 = ∂L / ∂x

Комментарии

Информация по комментариям в разработке

Похожие видео