Скачать или смотреть #37 Spark In Depth | Objects, Garbage Collection, On-Heap vs Off-Heap | Arun Kumar | ForumDE

Previous Video :- Spark In Depth | What is Java Virtual Machine (JVM) | Arun Kumar | ForumDE
Video link:-    • #36 Spark In Depth | What is Java Virtual ...  

Visit us at: https://forumde.in/
Contact us :- +91 70429 28331 | +91 9801830173

Welcome to another session of the Spark In Depth series, where we dive deep into the internals of Apache Spark to help you build a strong foundational understanding of big data processing.

In this video, we explore some of the most crucial concepts related to memory management in Spark – Objects, Garbage Collection, On-Heap Memory, and Off-Heap Memory.

Spark applications deal with large volumes of data, and efficient memory handling plays a major role in performance and stability. You’ll learn what exactly objects are in the JVM and how Spark creates and manages them while processing RDDs or DataFrames.

We explain how Garbage Collection (GC) works in JVM, why it becomes a bottleneck in large-scale Spark jobs, and how tuning GC behavior is essential to optimize performance.

The video also clearly differentiates between On-Heap Memory and Off-Heap Memory. You'll understand the default behavior of Spark using JVM heap memory, and how off-heap memory (via Tungsten project) reduces GC overhead by storing serialized binary data outside of the heap.

This session is especially helpful for developers and data engineers who want to optimize Spark jobs or understand why memory issues occur in production environments.

GITHUB LINK:-
https://github.com/forumde-arun/arun_...


Contact Us :-
Instagram link:
  / forum_de_team  

Microsoft form-
https://forms.office.com/r/JiALLp84GN

Telegram-
https://t.me/+yvWJpw3nRrNhMmQ9

LinkedIn link:
  / arun-kumar-19283775  

Facebook page link-
  / forum.de.team  

WhatsApp Channel link:-https://chat.whatsapp.com/K6Pih0QwytZ...


#Spark #reduceByKey #groupByKey #BigData #DataEngineering #SparkTransformations #SparkActions