Скачать или смотреть [MXDL-15-02] Hebbian learning (2) - Weight decay

In this video, we'll look at a Hebbian learning model with weight decay, and use it to perform the pattern recognition we did in the last video, and see what's different. Finally, let's use this to classify MNIST images.

A simple Hebb's rule is not a convergent learning rule since the weights continue to grow with each update step. To solve this problem, Stephen Grossberg proposed the Hebbian learning rule with weight decay in the 1960s.

Weight decay acts as a natural regularizer, which helps to prevent the weights from becoming too large.

There are two main types of weight decay: One is the instar learning rule, and the other is the outstar learning rule. Both learning rules were proposed by Grossberg.

#HebbianLearning #Hebb'sRule #WeightDecay #InstarLeariningRule #OutstarLeariningRule