Скачать или смотреть Introduction to PySpark using AWS & Databricks

--------------------------- Setup ---------------------------
0:00 - 1:11 : Introduction | Roadmap for video
1:11 - 3:15 : What is big data and why do we need a big data framework?
3:15 - 4:32 : What is Spark? Why do we use it?
4:32 - 5:45 : Why AWS & Databricks?
5:45 - 6:23 : AWS ElasticMapeReduce (EMR)
6:23 - 7:34 : AWS Management Console, AWS Regions & Availability Zones
7:35 - 8:15 : Databricks Console, Cloud Resources & Workspaces
7:35 - 14:13 : Creating Credential Configuration, AWS IAM (Cross Account IAM Roles), External ID's, JSON Policy Permissions, AWS ARN (Amazon Resource Name)
14:13 - 18:51 : Creating a Storage Configuration, S3, Buckets and Objects
18:51 - 22:23 : Creating a Workspace, Defining clusters, Master/Executor Architecture, Cluster Managers, DBFS (Databricks File System)
22:23 - 25:09 : Creating the cluster, defining node types & Auto scaling
25:09 - 26:00 : AWS EC2 (Elastic Cloud Compute)
26:00 - 29:25 : Upload data to DBFS, Create a new notebook

--------------------------- Coding ---------------------------
29:25 - 33:15 : pyspark, SparkSession, RDD's (Resilient Distributed Datasets)
33:16 - 35:30 : Import data using SQL, showcase data and display the schema of the DataFrame
35:30 - 40:39 : Change column format, select specific columns, or perform manipulations on columns
40:39 - 43:07 : Use SQL language by creating a Hive table abstraction through your DataFrame
43:08 - 45:10 : Using filter() for condition(s) on DataFrame
45:10 - 49:50 : Grouping by variables and aggregating a single or multiple columns together
49:50 - 53:15 : Working with null values

--------------------------- Resources ---------------------------
AWS Free Tier account: https://aws.amazon.com/free/?all-free...
Databricks Standard Free Trial Account: https://databricks.com/try-databricks
Access to free datasets: www.Kaggle.com