Optimal Transport and Information Geometry for Machine Learning and Data Science

Optimal transport and information geometry provide two distinct frameworks for studying the distance between probability measures. Although these are separate theories, there are many connections between them and they both have applications to data science and machine learning.

This video is adapted from a talk at the SIAM Conference on Mathematics of Data Science (MDS22).


At several points in the recording the main microphone cut out so I had to use back-up audio. Sorry for those audio hiccups.

0:00 Introduction
0:30 Introduction to Optimal Transport
7:08 Introduction to Information Geometry
12:21 Natural Gradients
13:12 Entropy Regularized Optimal Transport
16:38 Conclusion and Further Reading

References (in order of their appearance):

Khan, Gabriel, and Jun Zhang. "When optimal transport meets information geometry." Information Geometry (2022): 1-32.
https://arxiv.org/abs/2206.14791

Kantorovich, Leonid V. "On the translocation of masses." Journal of mathematical sciences 133, no. 4 (2006): 1381-1382.

Brenier, Yann. "Polar factorization and monotone rearrangement of vector‐valued functions." Communications on pure and applied mathematics 44, no. 4 (1991): 375-417.

Gangbo, Wilfrid, and Robert J. McCann. "The geometry of optimal transportation." Acta Mathematica 177, no. 2 (1996): 113-161.

Peyré, Gabriel, and Marco Cuturi. "Computational optimal transport: With applications to data science." Foundations and Trends® in Machine Learning 11, no. 5-6 (2019): 355-607.
https://arxiv.org/abs/1803.00567

Smith, Lewis. “A gentle introduction to information geometry” https://www.robots.ox.ac.uk/~lsgs/pos...

Nielsen, Frank. "An elementary introduction to information geometry." Entropy 22, no. 10 (2020): 1100.
https://arxiv.org/abs/1808.08271

Amari, Shun-ichi, and Hiroshi Nagaoka. Methods of information geometry. Vol. 191. American Mathematical Soc., 2000.

Santambrogio, Filippo. "Optimal transport for applied mathematicians." Birkäuser, NY 55, no. 58-63 (2015): 94.

Villani, Cédric. Optimal transport: old and new. Vol. 338. Berlin: springer, 2009.
https://cedricvillani.org/sites/dev/f...

If you are interested in the fugue excerpt at the start of the video, you can download a rough version of the sheet music here.
https://differentialgeometri.files.wo...

#MachineLearning #DataScience #InformationGeometry #OptimalTransport