Quick Azure SQL Server Data Migration to Databricks with only one line of SQL

Описание к видео Quick Azure SQL Server Data Migration to Databricks with only one line of SQL

Demonstrates how to easily import tables from a SQL Server into Databricks lakehouse storage without using any Python or Scala. I only use one SQL statement in Databricks to create the database(catalog). Everything I do is only point and click on menu options. The result is a Power BI dashboard that accesses all the data in the new Databricks database.
(   / greg-c-coopman-b07b54103   )

I hope that Relational SQL database (non Databricks) DBAs and Developers watching will get a "Aha" moment as they watch this video on how they can instantly start to leverage the SQL side of Datababricks.

Although the demo data is not huge, it does compose of several hundred thousand rows.

I go through, in detail how to:
1. Create the CSV files from Azure SqL database tables (via CSVs)
2. Understanding the Databricks SQL Endpoint (Cluster) that is required to be running
3. Create a new database in the Databricks metastore
4. Use Databricks Import Data wizard to import CSV files to the Databricks database
5. Use Databricks Partner Connect to setup a connection token for Power BI access to the Databricks metastore/database/tables
6. Import tables from Databricks to Power BI and Create dashboard

Although, it is definitely not required, I do have a video about the database tables used above in another video,    • PowerApp - General Technical WalkThro...   , that explains a PowerApp application I created.

Quick Access
-----------------------
00:00 Data Flow Diagram Walkthrough
06:49 Understanding the data to be imported
09:53 Creating CSV files from Azure SQL database tables (Datagrip)
13:55 -- Recap --
14:50 Databricks SQL Endpoint and metastore/database creation
20:21 -- Recap --
21:11 Create Databricks database and Import Wizard to populate Databricks Lakehouse
32:09 -- Recap --
33:22 Databricks Partner Connect to Power BI Setup/Power BI report creation
39:24 Power BI Report review
42:24 Final Thoughts
43:48 Thank you and please Subscribe

Комментарии

Информация по комментариям в разработке