Скачать или смотреть MLOps: A/B Testing with MLflow, Kafka, and DuckLake

Learn how to implement an end-to-end machine learning workflow, from data ingestion, to A/B testing and monitoring, using MLflow, Kafka and DuckLake. We'll discuss model training and tracking with MLflow, real-world deployment approaches, how to track inference results and feedback in your data lakehouse using Kafka and DuckLake, and how to compute monitoring statistics such as prediction drift, feature drift, or estimated performance, for unlabeled data, as well as taking advantage of user feedback to estimate model error. This should give you the tools you need to build and maintain your own ML lifecycle.

🔗 Resources:
💻 GitHub repo: https://github.com/DataLabTechTV/data...
📝 Blog post: https://datalabtechtv.com/posts/mlops...
📚 MLflow: https://mlflow.org/
📚 DuckLake: https://ducklake.select/
📚 Apache Kafka: https://kafka.apache.org/

📺 Watch all the MLOps videos here ►    • MLOps End-to-End  

🔔 Subscribe for more deep dives data science, data engineering, and all things data!
☁️ Follow the channel on Bluesky: https://bsky.app/profile/datalabtecht...
🎧 Join our community's Discord:   / discord  
👽 Join our subreddit:   / datalabtechtv  
💻 Check out the channel's GitHub: https://github.com/DataLabTechTV
🌐 Visit our website for blog posts: https://datalabtechtv.com

#mlops #mlflow #ducklake #kafka #monitoring #abtesting

00:00:00 Intro
00:00:59 Data Roles
00:02:40 End-to-End ML Workflow
00:12:45 Orchestration with a Justfile
00:14:11 DataLab ML Package
00:14:48 Model Training
00:15:27 MLflow Logging
00:18:39 MLflow UI
00:20:46 Datasets and ETL
00:23:57 Inference Serving and Logging
00:28:13 Simulating User Feedback
00:33:13 Model Monitoring
00:33:36 Prediction Drift
00:34:28 Feature Drift
00:36:35 Estimated Performance
00:38:17 User Evaluation
00:39:34 Data Quality
00:39:56 Explainability
00:40:26 A/B Testing Results
00:44:49 Rambling
00:47:06 Outro