Скачать или смотреть aws s3 bucket | parquet file | Connect AWS S3 Bucket in Databricks

In This Video we are going to learn,
Convert SQL Server Result to Json file
Upload Json in S3 bucket
Read Json file from AWS S3 bucket using (Databricks - pyspark)
convert Json to parquet file
and write Parquet file into AWS S3 bucket.
Connect your AWS S3 Bucket in Databricks

Code:
access_key = ''
secret_key = ''
sc._jsc.hadoopConfiguration().set("fs.s3a.access.key", access_key)
sc._jsc.hadoopConfiguration().set("fs.s3a.secret.key", secret_key)

aws_region = "ap-south-1"
sc._jsc.hadoopConfiguration().set("fs.s3a.endpoint", "s3." + aws_region + ".amazonaws.com")

df = spark.read.format("json")\
.option("multiline","True")\
.option("inferschema","True")\
.option("header","True")\
.load("s3://mytestbucketsagar/patient_1million.json")

df.write.format("parquet")\
.option("Header","true")\
.option("mode","overwrite")\
.option("path","s3://mytestbucketsagar/onemillion")\
.save()

your queries:
parquet
pyspark
spark
databricks
write parquet file aws s3 bucket
read json file pyspark
read json file spark

Notes / SQL Query link:
https://learnsqlwithsagar.blogspot.com

My key Links-
🚀 YouTube Videos-    / @mssqlexpert  
🌍 My website / blog - https://learnsqlwithsagar.blogspot.com/
📸 Instagram -   / learnsqlwithsagar  
🚀 YouTube Shorts - https://www.youtube.com/@mssqlexperts...

Who am I :
Hey guys, I am Sagar, welcome to this channel - #mssqlexpert. this channel is all about learning SQL Server. my goal is to help you in SQL server learning.
and Learn complete SQL from basics to Expert Level with an Industrial Knowledge.
This SQL course is for beginners who having zero knowledge in SQL. Even those having SQL knowledge they will also get deep information about SQL Language & Microsoft SQL Server (RDBMS). This course will cover how to write SQL Query in MS SQL Server. We will go through a Database, Database Objects (Table, View, Stored Procedure, Function, Trigger, Cursor), Different SQL Queries, Joining, Temp Table, Table Variable, CTE, Indexing, Stats, Query Execution plan, Query Optimization and much more.

Please Like, Share, Subscribe, & Comment.

Featured playlist
   • SQL Course | SQL Full Course 2024 | sql fu...  
   • Database Knowledge  
   • SQL project  
   • MySQL  

#mssqlexpert #sql #sql course #data #database #sql server #sqlserver #mssqlserver #RDBMS #sql tutorial #sql full course #sql tutorial for beginners
#sql for data analysis #sql for beginners