The Most Important (and Surprising) Result from Information Theory

The machine learning consultancy: https://truetheta.io
Join my email list to get educational and useful articles (and nothing else!): https://mailchi.mp/truetheta/true-the...
Want to work together? See here: https://truetheta.io/about/#want-to-w...

Information Theory contains one idea in particular that has had an incredibly impact on our society.

David MacKay's lecture:    • Information Theory, Pattern Recogniti...  

SOCIAL MEDIA

LinkedIn :   / dj-rich-90b91753  
Twitter :   / duanejrich  
Github: https://github.com/Duane321

Enjoy learning this way? Want me to make more videos? Consider supporting me on Patreon:   / mutualinformation  

SOURCES

Chapters 1, 9 and 10 from [1] were the primary source. Chapter 7 of [3] was a secondary source. [2] is inclusive of Shannon's 1948 original paper and provided useful historical context.

[1] D. MacKay. Information Theory, Inference and Learning Algorithms. Cambridge University Press. 2003.

[2] C. E. Shannon and W. Weaver. The Mathematical Theory of Communication. University of Illinois Press. 1964

[3] T. M. Cover and J. A. Thomas. The Elements of Information Theory, 2nd Edition. Wiley-Interscience. 2006.

TIMESTAMPS
0:00 Problem Statement and the R3 Coding Strategy
2:06 Bit Error Probability and Rate
3:30 The Trillion Dollar Question
4:12 Claude Shannon Proves Something Remarkable
6:11 Sidebar on other Educational Content
6:49 The Trick
7:38 Check out David Mackay's Textbook and Lectures, plus Thank You