PySpark Full Course | Basic to Advanced Optimization with Spark UI PySpark Training | Spark Tutorial

PySpark Tutorial | Apache Spark Full Course | Spark Tutorial for beginners | PySpark Training Full Course

Only training that covers Basic to Advanced Spark with Spark UI and with live examples. Here is what it covers in length in next 6 hrs 45 min:

Chapters:
00:00:00 - What we are going to Cover?
00:00:25 - Introduction
00:01:10 - What is Spark?
00:02:29 - How Spark Works - Driver & Executors
00:06:04 - Spark Transformations & Actions
00:10:31 - Spark DataFrames & Execution Plans
00:13:33 - Understand Spark Session
00:21:28 - Write Spark DataFrame Schema
00:32:13 - Cast Column | Add Column | Static Column Value |Rename
00:42:16 - Working with Strings, Dates and Null
00:55:38 - Sorting data, Union and Aggregation in Spark
01:03:18 - Window Functions, Unique Data & Databricks Community Cloud
01:12:33 - Data Repartitioning & PySpark Joins | Coalesce vs Repartition
01:23:20 - Understand Spark UI, Read CSV Files and Read Modes
01:38:28 - Read Complex File Formats | Parquet | ORC
01:47:44 - Read, Parse or Flatten JSON data
02:03:40 - How Spark Writes data | Write modes in Spark
02:17:20 - Understand Spark Execution on Cluster
02:29:27 - User Defined Function (UDF)
02:38:45 - Understand DAG, Explain Plans & Spark Shuffle with Tasks
02:55:18 - Understand and Optimize Shuffle in Spark
03:10:20 - Data Caching in Spark | Cache vs Persist
03:23:23 - Broadcast Variable and Accumulators in Spark
03:35:43 - Optimize Joins in Spark & Understand Bucketing for Faster joins
04:03:35 - Static vs Dynamic Resource Allocation in Spark
04:13:48 - Fix Skewness and Spillage with Salting in Spark
04:34:51 - AQE aka Adaptive Query Execution in Spark
04:46:12 - Spark SQL, Hints, Spark Catalog and Metastore
05:05:20 - Read and Write from Azure Cosmos DB using Spark
05:26:17 - Get Started with Delta Lake using Databricks
06:06:06 - Optimize Data Scanning with Partitioning in Spark
06:13:17 - Data Skipping and Z-Ordering in Delta Lake Tables
06:31:45 - Delta Tables - Deletion Vectors and Liquid Clustering

Original Playlist has more that 250k views {   • PySpark - Zero to Hero | PySpark Tuto...  )
Other popular playlist}

Github link with all notebooks: https://github.com/subhamkharwal/pysp...

Other Popular playlist on our channel Ease with Data:
Databricks Zero to Hero: {   • Databricks - Zero to Hero| Databricks...  }
Spark Streaming with PySpark: {   • Spark Streaming with PySpark | Struct...  }

Follow me on LinkedIn:   / subhamkharwal  
Follow Ease With Data YouTube Channel: ‪@easewithdata‬

Make sure to Like and Subscribe 💓

#pyspark #apachespark #spark #dataengineering