Extract and Load from External API to Lakehouse using Data Pipelines (Microsoft Fabric)

FREE 40-minute Fabric fundamentals course: https://www.skool.com/microsoft-fabri...

#datapipeline #microsoftfabric #lakehouse
In this video, I walk through how to query an external API using a Data Pipeline, extracting some JSON data and loading it into a Lakehouse in Microsoft Fabric.

This video is the first in an End-To-End Microsoft Fabric Project, you can follow the series via this Playlist:    • Плейлист  

In my experience, this is one of THE most important and impactful things you can learn as a data engineer.

Next parts of the series include:
- Transforming data in the Lakehouse using Data Engineering Notebooks.
- Visualising the data in Power BI.

-LINK TO OPENWEATHERMAP-
Here's the API used in this tutorial: https://openweathermap.org/api

-OTHER VIDEOS YOU MIGHT LIKE-
LAKEHOUSE -    • Microsoft Fabric Lakehouse Tutorial  
Data warehouse -    • Microsoft Fabric Data Warehouse Expla...  
Data pipelines -    • Three data pipeline use cases to make...  
End-to-end data flows project -    • Dataflows end-to-end project (Microso...  
OneLake -    • OneLake - the FIRST thing you need to...  

-TIMELINE-
0:00 Intro
1:27 Exploring the API
4:10 Looking at the Lakehouse Destination
4:56 Creating the data pipeline
6:43 Source configuration
12:56 Destination configuration
15:28 Running the pipeline
16:00 Summary
6:14 Creating tables
8:09 Saving & sharing queries
10:03 Insert data
10:23 Selecting (viewing) data
10:58 Caution
11:36 Joining data
12:36 Creating views
14:42 Cross data warehouse querying
17:09 Summary


-LINKEDIN-
Not following the LinkedIn page yet? Here's the link:   / learnmicrosoftfabric  

-ABOUT WILL-
Hi, I'm Will! I'm hugely passionate about data and using it to create a better world. I currently work as a Consultant, focusing on Data Strategy, Data Engineering and Business Intelligence (within the Microsoft/Azure/Fabric environment). I have previously worked as a Data Scientist. I started Learn Microsoft Fabric to share my learnings on how Microsoft Fabric works and help you build your career and build meaningful things in Fabric.

-SUBSCRIBE-
Not subscribed yet? You should! There are lots of new videos in the pipeline covering all aspects of Microsoft Fabric.