Скачать или смотреть Weather Data Pipeline Engineering Project | Databricks & Tableau End-to-End Data Engineering

Welcome to my Weather Pipeline Engineering project showcase! I'm Lok Sundar Ganthi, a Data Engineer pursuing my Master’s in Business Analytics at The University of Texas at Dallas.

In this video, I demonstrate an advanced, end-to-end data engineering pipeline using Databricks and Tableau. The pipeline integrates multiple real-time weather data APIs into a unified dataset, performs automated data ingestion, cleansing, and aggregation through Delta Lake tables, and provides insightful visual analytics through Tableau dashboards.

Project Highlights:

Automated data ingestion from multiple weather APIs into Bronze Delta tables.
Robust data cleaning and integration at the Silver data layer.
Creation of enriched, aggregated Gold layer tables for actionable insights.
Interactive and visually engaging Tableau dashboards for effective analytics.
Automated orchestration and scheduling of workflows using Databricks jobs.
This project demonstrates comprehensive data engineering best practices clearly—from raw data management through powerful analytics visualization.

🌟 Skills Showcased:

Databricks Delta Lake & Spark (PySpark)
Data pipeline orchestration and automation
Tableau Visualization
Cloud Data Engineering and ETL processes
🛠️ Tech Stack: Databricks, Delta Lake, Tableau, PySpark, API Integration.

📁 Portfolio & Code Repository: https://loksundar.com

🔗 Connect with me:

LinkedIn:   / ganthi-lok-sundar  
Thanks for watching! Please like, subscribe, and comment below if you enjoyed the project or have any questions.