Скачать или смотреть Defending Against Decision Degradation with Full Spectrum Model Monitoring Case Study and AMA

Speaker:
Mihir Mathur, Machine Learning Product Lead, Lyft
Mihir Mathur is the lead Product Manager for Machine Learning at Lyft, where he works on building ML and AI tools that power automated intelligent decisions across realtime pricing, ETAs, fraud detection, safety classification etc. In the past Mihir has worked on building delightful products for millions of users at Quora, Houzz, and Thomson Reuters. Mihir graduated magna cum laude from UCLA with a Bachelor’s and Master’s in Computer Science.


Abstract:
ML models at Lyft make millions of high stakes decisions per day including decisions for real-time pricing, physical safety classification, fraud detection, and much more. Preventing models from degrading and making ineffective decisions is therefore critical. Over the past two years, we’ve invested in building a full-spectrum model monitoring solution to catch and prevent model degradation. In this talk, we’ll discuss our suite of approaches for model monitoring including real-time feature validation, performance drift detection, anomaly detection, and model score monitoring as well as the cultural change needed to get ML practitioners to effectively monitor their models. We’ll also discuss the impact our monitoring system delivered by catching problems.