Factor Analysis and Probabilistic PCA

The machine learning consultancy: https://truetheta.io
Join my email list to get educational and useful articles (and nothing else!): https://mailchi.mp/truetheta/true-the...
Want to work together? See here: https://truetheta.io/about/#want-to-w...

Factor Analysis and Probabilistic PCA are classic methods to capture how observations 'move together'.

SOCIAL MEDIA

LinkedIn :   / dj-rich-90b91753  
Twitter :   / duanejrich  

Enjoy learning this way? Want me to make more videos? Consider supporting me on Patreon:   / mutualinformation  

SOURCES

[1] was my primary source since it provides the algorithm used in the Scikit Learn's Factor Analysis software (which is what I use). Since it walks through the derivation of the fitting procedure, it is quite technical. Ultimately, that level of detail came in handy for this video.

[2] and [4] were my go-to for Probabilistic PCA. A primary reason is Christopher Bishop is one of the originators of PPCA. That came with a lot of thoughtful motivation for the approach. The discussion there includes a lot advantages of PPCA over PCA.

[3] was my refresher on this subject when I first decided to this video. Like many of us, I'm a fan of Andrew Ng, so I was curious how he'd explain the subject. He emphasized that this model is particularly useful in high dimension-low data environments - something I forward in this video.

[5] is an excellent overview of FA and PPCA (as long as you're comfortable with linear algebra and probability). In fact, Kevin Murphy's entire book is like that for every subject and that's why it's my absolute favorite text.

---------------------------

[1] D. Barber, Bayesian Reasoning and Machine Learning, Cambridge University Press, 2012

[2] C. Bishop, Pattern Recognition and Machine Learning, Springer, 2006

[3] A. Ng, "Lecture 15 - EM Algorithm & Factor Analysis | Stanford CS229: Machine Learning (Autumn 2018)",    • Lecture 15 - EM Algorithm & Factor An...  

[4] M. Tipping and C. Bishop, "Mixtures of Probabilistic Principal Component Analysers", MIT Press, 1999

[5] K. P. Murphy. Probabilistic Machine Learning (Second Edition), MIT Press, 2021

CONTENTS
0:00 Intro
0:21 The Problem Factor Analysis Solves
2:27 Factor Analysis Visually
5:52 The Factor Analysis Model
10:56 Fitting a Factor Analysis Model
14:13 Probabilistic PCA
15:43 Why is it Probabilistic "PCA"?
16:59 The Optimal Noise Variance

TOOLS

If you'd like to apply Factor Analysis, I'd recommend Scikit-Learn: https://scikit-learn.org/stable/modul...

Social Media
Twitter :   / duanejrich  
Patreon :   / mutualinformation