Скачать или смотреть Upload practice dataset on GCP Spark cluster | Dataproc | CCA 175 Certification

Data Setup On #Spark #Cluster #Google #Cloud #Platform #Dataproc #CCA175
We will learn how to setup practice dataset in Google Cloud Platform in below steps:

⏰ TIMESTAMPS:
00:00 Steps to setup practice dataset in Google Cloud Platform ( GCP )
00:23 Create bucket in GCP (Google Cloud Platform)
01:11 Upload dataset files to the bucket
01:47 Mount the GCP bucket as a File System
02:41 Put files from GCP bucket to HDFS

✔️ DOWNLOAD PRACTICE DATASET
https://github.com/proedu-organisatio...

✔️ CCA 175 COMPLETE COURSE
https://www.udemy.com/course/cca-175-...

✔️ Gcsfuse Installation Steps:
1. In the Cloud Console, go to the VM instances page.
2. In the list of virtual machine instances, click SSH in the row of the instance that you want to connect to.
3. Execute below commands in the SSH console.
export GCSFUSE_REPO=gcsfuse-`lsb_release -c -s`

echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" | sudo tee /etc/apt/sources.list.d/gcsfuse.list

curl https://packages.cloud.google.com/apt... | sudo apt-key add -

sudo apt-get update

sudo apt-get install gcsfuse

✔️ Mount Bucket:
1. Execute below commands in the SSH console.
mkdir retail_db_dataset
gcsfuse --implicit-dirs spark-cluster-1 ~/retail_db_dataset

✔️Put Data To HDFS:
1. Execute below commands in the SSH console.
hdfs dfs -mkdir /user/spark/dataset/
hdfs dfs -put retail_db_dataset/* /user/spark/dataset/

🔵 COMPLETE APACHE SPARK TUTORIAL PLAYLIST 🔵
✔️    • What is Apache Spark  


🔵 WORKING WITH STRUCTURED DATA IN APACHE SPARK 🔵
✔️ How to read write CSV file    • How to read write CSV file/data in Apache ...  \
✔️ How to read/write AVRO file    • How to read/write AVRO file/data in Apache...  
✔️ How to read write Parquet file data    • How to read write Parquet file data in Apa...  
✔️ How to read/write JSON file    • How to read/write JSON file/data in Apache...  
✔️ How to read/write Hive Metastore    • How to read/write Hive Metastore table in ...  

🔵 WORKING WITH DATE COLUMNS IN APACHE SPARK 🔵
✔️ Manipulate Date in Spark dataframe    • How to manipulate Date in Spark dataframe|...  
✔️ Work with/manipulate String columns    • How to work with/manipulate String  column...  


🔵 WORKING WITH WINDOWING, AGGREGATE FUNCTIONS IN APACHE SPARK 🔵
✔️ Ranking Functions in Apache Spark    • How to use Ranking Functions in Apache Spa...  
✔️ Windowing Functions in Apache Spark    • How to use Windowing Functions in Apache S...  
✔️ Analytical Functions in Spark    • Analytical Functions in Spark | GROUP BY |...