Скачать или смотреть Using SQL Scripts to Build in DataForge - Full developer walkthrough

What is DataForge | Legacy data platform | Building a Fact Table in Dataforge from SQL Scripts - Case Study

#dataengineering #data #datapipelines

► This video is sponsored by DataForge 🙌🏼, the declarative data platform
► For more info, check it out here: www.dataforgelabs.com
► Slack community: https://bit.ly/dataforge-slack
► Free academy for data engineering: learn.dataforgelabs.com

Introduction to DataForge Framework:    • Introduction to the DataForge Column-Pure ...  

In this video we build a new data platform from scratch using DataForge by analyzing existing SQL scripts and translating them into DataForge objects. We walk through how to analyze existing pipeline SQL scripts structured in stages, rebuild the logic in DataForge column-by-column, and then review some of the long-term advantages of this approach.

▬▬▬▬▬▬ T I M E S T A M P S ⏰ ▬▬▬▬▬▬
00:00 - Intro and Overview
01:28 - Legacy Codebase Introduction
06:08 - Expired Premium Lineage Analysis
10:42 - Creating Sources from Connections in DataForge
15:24 - Building Type Conversion Rules
23:15 - Complex Transformation Legacy Code Analysis
26:34 - Building Relations
30:57 - Window Function Rules
34:38 - Self-Relations
36:53 - Alternative Type Conversion Rule Syntax
37:58 - Breaking Down SQL into Smaller Rules
42:10 - Using Self-Relation Traversal
43:38 - Rules vs. SQL Script Staging
46:27 - All Rules Solution Overview
50:24 - Expired Premium Rules Lineage
51:37 - Building an Output Introduction
55:45 - Adding a Channel to an Output
58:03 - Publishing Data to Snowflake
1:01:48 - Wrap-up and Final Thoughts