Stephen Whitworth - Building robust machine learning systems

Filmed at PyData London 2017

Description
With the growth of AI, ever growing parts of products we build are changing from the deterministic to the probabilistic. The accuracy of machine learning applications can deteriorate in the wild without strategies for testing, monitoring and introspection. You'll leave this talk knowing how to combine the best of software engineering and machine learning to build robust machine learning products.

Abstract
As machine learning becomes more prevalent, ever growing parts of the systems we build are changing from the deterministic to the probabilistic. The accuracy of machine learning applications can quickly deteriorate in the wild without strategies for testing models, instrumenting their behaviour and the ability to introspect and debug incorrect predictions.

This session will take an applied view from my experience of building production machine learning infrastructure at Ravelin. You’ll learn useful practices and tips to help ensure your machine learning systems are robust. We’ll go into:

Labels and Data - can you trust it? Can you infer them?
Testing - how do you ensure your model is doing the basics, up to the more complicated examples?
Auditing and versioning - what's the provenance of your model? What data was it trained on? With which hyper parameters? Can you reproduce it?
Debugging and introspection when deployed - when you make an awful prediction, can you figure out why that happened and prevent it happening again?
And more, with the aim of helping you sleep a little better at night knowing your model is out there in the wild.

www.pydata.org

PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R.

We aim to be an accessible, community-driven conference, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases. 00:00 Welcome!
00:10 Help us add time stamps or captions to this video! See the description for details.

Want to help add timestamps to our YouTube videos to help with discoverability? Find out more here: https://github.com/numfocus/YouTubeVi...