So You Think You Know How to Take Derivatives? | Steven Johnson | ASE60

So you think you know how to take derivatives?

Derivatives are seen as the "easy" part of learning calculus: a few simple rules, and every function's derivatives are at your fingertips! But these basic techniques can turn bewildering if you are faced with much more complicated functions like a matrix determinant (what is a derivative "with respect to a matrix" anyway?), the solution of a differential equation, or a huge engineering calculation like a fluid simulation or a neural-network model. And needing such derivatives is increasingly common thanks to the growing prevalence of machine learning, large-scale optimization, and many other problems demanding sensitivity analysis of complex calculations. Although many techniques for generalizing and applying derivatives are known, that knowledge is currently scattered across a diverse literature, and requires students to put aside their memorized rules and re-learn what a derivative really is: linearization. In 2022 and 2023, Alan and I put together a one-month, 16-hour "Matrix Calculus" course at MIT that refocuses differential calculus on the linear algebra at its heart, and we hope to remind you that derivatives are not a subject that is "done" after your second semester of calculus.

Contents
00:00 Welcome and introduction
02:15 Derivatives: the “easy” part of calculus?
06:41 Why matrix calculus matters
11:24 Automatic differentiation: from calculus to compilers
14:44 Derivatives as linearization
19:20 Gradients as linear operators
24:41 The chain rule and forward vs reverse mode
27:49 “Adjoint method” for engineering optimization
29:05 Final comments and questions

S/O to https://github.com/agchesebro for the video timestamps!

Want to help add timestamps to our YouTube videos to help with discoverability? Find out more here: https://github.com/JuliaCommunity/You...

Interested in improving the auto generated captions? Get involved here: https://github.com/JuliaCommunity/You...