Microsoft Fabric: Processing Bronze to Silver using Fabric Notebooks

Notebooks in Fabric are a lot like notebook experiences in other tools you're probably already familiar with. They allow us to write code in a variety of languages and create a commentary alongside our code using interactive cells. Fabric notebooks also have built in integration with Lakehouses, and provide a built-in filesystem that can be used to store arbitrary files somehow associated with the notebook.

Notebooks are our go-to tool for wrangling Bronze data into Silver, which tends to follow standard processing patterns. In this video we'll do a whirlwind tour of notebooks in Microsoft Fabric, and take a look at how our "Process to Silver" notebook takes shape.

00:00 Intro
00:10 Creating a blank notebook
01:04 Mounting a Lakehouse in a Fabric Notebook
02:05 Querying files in a Lakehouse from a Fabric Notebook
03:28 Query PySpark DataFrames directly with SQL
04:12 Built-in notebook filesystem
05:14 Walkthrough of "Process to Silver" notebook - reading data and transforming
08:18 Walkthrough of "Process to Silver" notebook - merging and writing data
10:03 Walkthrough of "Process to Silver" notebook - reading back written data
10:15 Roundup & Outro

Useful links:

📖 Microsoft Fabric Notebooks: https://learn.microsoft.com/en-us/fab...
📖 Connect Lakehouses and Fabric Notebooks: https://learn.microsoft.com/en-us/fab...
📖 Built-in Notebook filesystem: https://learn.microsoft.com/en-us/fab...
📖 Delta Lake Merge: https://delta.io/blog/2023-02-14-delt...

Series contents:

📺 Part 1 - Lakehouse & Medallion Architecture -    • Microsoft Fabric: Lakehouse & Medalli...  
📺 Part 2 - Planning and Architecting a Data Project -    • Microsoft Fabric: Inspecting 28 MILLI...  
📺 Part 3 - Ingest Data -    • Microsoft Fabric: Ingesting 5GB into ...  
📺 Part 4 - Creating a shortcut to ADLS Gen2 in Fabric -    • Microsoft Fabric: Creating a OneLake ...  
📺 Part 5 - Navigating OneLake data locally -    • Microsoft Fabric: Local OneLake Tools  
📺 Part 6 - Role of the Silver layer in the Medallion Architecture -    • Microsoft Fabric: Role of the Silver ...  
📺 Part 7 - Processing Bronze to Silver - THIS VIDEO!
📺 Part 8 - Good Notebook Development Practices -    • Microsoft Fabric: Good Notebook Devel...  

If you want to learn more about Fabric, take a look at some of our other content:

🤖 [Course] Microsoft Fabric: from Descriptive to Predictive Analytics:    • Microsoft Fabric: Machine Learning Tu...  
👉 Perspectives on #MicrosoftFabric: https://endjin.com/what-we-think/talk...
👉 A Tour Around #MicrosoftFabric: https://endjin.com/what-we-think/talk...
👉 Introduction to #MicrosoftFabric: https://endjin.com/blog/2023/05/intro...

and find all the rest of our content here: https://endjin.com/blog/2023/05/micro...

#Microsoft #PowerBI #MicrosoftFabric #lakehouse #datalake #onelake #data #ai #analytics #medallion #bronze #silver #gold #notebooks