Trel for Data Scientists - Building ML Pipelines

Follow along as we build a Customer Lifetime Value (CLV) pipeline using Trel's innovative, self-serve abstractions. This tutorial demonstrates how Trel simplifies complex data operations, allowing you to manage your workflow without extensive DevOps knowledge.
We'll cover four key stages:

- Setting Up Trel: Configure repositories and set up your data catalog.
- Data Ingestion: Use automated sensors and manual techniques.
- Data Preparation: Leverage immutable datasets and automated quality checks.
- Modeling & Prediction: Create, verify, and apply your CLV model.

Watch as we demonstrate how Databricks notebooks can be directly put into production, streamlining the transition from experimentation to deployment. This hands-on guide showcases Trel's ability to ensure data integrity and reproducibility throughout your pipeline.
Whether you're new to DataOps or looking to optimize existing workflows, this tutorial offers valuable insights into leveraging Trel for your data science projects.

Tags:
#DataScience #Trel #CLVPipeline #DataIngestion #FeatureEngineering #Modeling #DataPreparation #MachineLearning #DataOps #Tutorial #SelfServe #NotebookProduction

0:00 Preparing for data ingestion
2:00 Principles of Data Ingestion
5:00 Manual data insertion
6:20 Principles of Formula-based automation
9:06 clv.features Databricks notebook
13:01 Registering, executing and automating clv.features job
20:50 clv.model Databricks notebook
24:44 Registering and executing remaining jobs through automation
26:56 Enabling data ingestion to demonstrate full automation