MADE: Masked Autoencoder for Distribution Estimation

This is an in-depth explanation of a technique for doing density estimation with the help of masked autoencoders.

Why should you master it?
Masked Autoencoder for Distribution Estimation is now being used as a building block in modern Normalizing Flows algorithms such as Inverse Autoregressive Normalizing Flows & Masked Autoregressive Normalizing Flows.

To learn more about Normalizing Flows check out my very comprehensive tutorial (   • Normalizing Flows - Motivations, The ...  )

Sections:
00:00 Introduction
01:27 Goals
03:39 But what is an Autoencoder? (Essential Background)
10:16 Autoencoder for Density Estimation - Formulation & Associated Challenges!
19:12 The Big Idea to address the challenges
21:54 How it works (Step by Step algorithm explanation)
26:56 Summary of Key Insights
28:43 Experiments & Results
31:04 Conclusions

Link to the paper:
Masked Autoencoder for Density Estimation - https://arxiv.org/abs/1502.03509

Links to the key papers that make use of MADE
1) Improving Variational Inference with Inverse Autoregressive Flow (https://arxiv.org/abs/1606.04934)
2) Masked Autoregressive Flow for Density Estimation (https://arxiv.org/abs/1705.07057)

Multiple implementations are available for this paper:
https://paperswithcode.com/paper/mask...

#densityestimation
#autoencoders
#normalzingflows