Part 1: Spark Lazy Evaluation - Introduction to Transformations and Actions

Описание к видео Part 1: Spark Lazy Evaluation - Introduction to Transformations and Actions

In the next few set of videos we will be discussing about the Pyspark Transformations and Actions.

What is Lazy Evaluation?
In Spark, RDD Transformations are lazy operations meaning none of the transformations get executed until you call an action on PySpark RDD. Since RDD’s are immutable, any transformations on it result in a new RDD leaving the current one unchanged.

Spark has certain operations which can be performed on RDD.
An operation is a method, which can be applied on a RDD to accomplish certain task. RDD supports two types of operations, which are Action and Transformation. An operation can be something as simple as sorting, filtering and summarizing data.

Комментарии

Информация по комментариям в разработке