Spark Tutorial in Microsoft Fabric (3.5 HOURS!)

FREE 40-minute Fabric fundamentals course: https://www.skool.com/microsoft-fabri...

In September, I recorded a series of video to help you Learn Apache Spark in Microsoft Fabric. I merged all videos in the series into this 3.5 hour Spark tutorial, so that it’s easier for you to watch and share will friends and colleagues who might also be interested in learning Spark in Fabric.

New to Microsoft Fabric? Watch this:    • What is Microsoft Fabric? The GAMECHA...  

Spark is the engine behind both the Data Engineering AND the Data Science experiences in Microsoft Fabric, so in this 3.5 HOUR tutorial, I walk you through Apache Spark: what it is, why you should learn it, how to use it, and how it integrates into Microsoft Fabric.

Link to the GitHub for this series, including all links to reference material mentioned through this tutorial: https://github.com/LearnMicrosoftFabr...

No previous Spark knowledge is required, some basic Python would be useful!

#pyspark #microsoftfabric #apachespark

00:00:00 Welcome
00:03:58 Why Spark?
00:13:47 Components of Spark
00:17:58 Spark DataFrame
00:23:54 Read Files into DataFrame
00:36:32 Read/Write to Lakehouse Table
00:43:07 Basic DataFrame Operations
00:52:20 DataFrame Filtering
01:03:43 GroupBy and Aggregate Functions
01:12:21 Handling missing values
01:24:44 Joining DataFrames
01:31:13 Time-series data
01:37:52 Spark SQL
01:46:03 Spark Machine Learning
02:07:30 Configuring Spark
02:24:19 Autotuning Spark Configuration
02:30:37 Library Management
02:35:30 High-concurrency mode
02:41:28 Spark Scala
02:47:55 Fabric MSSparkUtils
02:57:23 Monitoring Spark
03:03:45 Questions and Answers

-BROWSE MY OTHER FABRIC PLAYLISTS-
DATA ENGINEERING    • Data engineering (Microsoft Fabric)  
END-TO-END FABRIC PROJECT    • Плейлист  
INTRO TO MICROSOFT FABRIC    • Intro to Microsoft Fabric  
DATA FACTORY    • Data Factory (Microsoft Fabric)  

-LINKEDIN-
Not following the LinkedIn page yet? Here's the link:   / learnmicrosoftfabric  

-ABOUT WILL-
Hi, I'm Will! I'm hugely passionate about data and using it to create a better world. I currently work as a Consultant, focusing on Data Strategy, Data Engineering and Business Intelligence (within the Microsoft/Azure/Fabric environment). I have previously worked as a Data Scientist. I started Learn Microsoft Fabric to share my learnings on how Microsoft Fabric works and help you build your career and build meaningful things in Fabric.

-SUBSCRIBE-
Not subscribed yet? You should! There are lots of new videos in the pipeline covering all aspects of Microsoft Fabric.
https://youtube.com/@learnmicrosoftfa...