Jensen's Inequality

The machine learning consultancy: https://truetheta.io
Join my email list to get educational and useful articles (and nothing else!): https://mailchi.mp/truetheta/true-the...
Want to work together? See here: https://truetheta.io/about/#want-to-w...

Article on the topic: https://truetheta.io/concepts/machine...

Jensen's Inequality appears multiple times in any rigorous machine learning textbook. It's essential for the key principles and foundational algorithms that make this field so productive. In this video, I state what it is, explain why it's important and show why it's true.

SOCIAL MEDIA

LinkedIn :   / dj-rich-90b91753  
Twitter :   / duanejrich  

Enjoy learning this way? Want me to make more videos? Consider supporting me on Patreon:   / mutualinformation  

Sources and Learning More

To see Jensen's Inequality used in the justification for the EM algorithm, see section 11.4.7 of [1]. For its use in Information Theory, see section 2.6 of [2].

[1] Murphy, K. P. (2012). Machine Learning: a Probabilistic Perspective. MIT Press, Cambridge, MA, USA.

[2] Cover, T. M. & Thomas, J. A. (2006), Elements of Information Theory 2nd Edition, Wiley-Interscience, NY USA

Contents
00:00 - Why Jensen's Inequality is important
02:01 - Stating the Inequality
03:30 - Showing the Inequality
06:36 - Outro