Emin Martinian - Statistical Profiling (and other fun with the sys module) - PyCon 2019

"Speaker: Emin Martinian

Profiling involves computing a set of data about how often and how
long various parts of your program are executed. Profiling is useful
to understand what makes your program slow and how you can improve
it. After a quick review of deterministic profiling tools and
techniques, I will describe how you can do statistical profiling with
existing packages or write your own from scratch.

Statistical profiling involves occasionally sampling what your program
is doing instead of watching each line or function. A key feature of
statistical profiling is that by using a moderate sampling frequency,
you can profile your production code with almost no overhead. This lets
you find the actual bottlenecks in real use cases.

The core technical focus of the talk is python's sys module and how it
lets you easily examine a running program. I also describe some tricks
to be aware of related to threading, context switches, locks, and so on.
At the conclusion of the talk, you will hopefully understand how to use
an existing statistical profiler or write a customized version yourself.


Slides can be found at: https://speakerdeck.com/pycon2019 and https://github.com/PyCon/2019-slides"