Apache Spark : Commonly used Transformations : Map, Filter, Flatmap Transformations
Official Website: http://bigdataelearning.com
Learning Objectives :: In this module, you will learn some of the commonly used transformations. You will learn some of the basic RDD transformations like Map, Filter, and Flatmap transformations.
This video also shows how to apply the Map, Filter, and Flatmap transformation using Scala & python.
Topics :: Commonly used Transformations, Basic RDD Transformations, Map, Filter, Flatmap Transformations
Commonly used Transformations
=============================
Basic RDD Transformations,
=========================
map, filter, and flatmap are some of the basic RDD transformations
Map
====
Map is a transformation , that takes a function and applies the function to each elements of the input RDD.
The result of the function , will become the value of each element , in the resultant RDD.
say, If inputRDD contains values 1 to 4, then map transformation to square the values will return {1,4,9,16} as the resultant RDD
Here the square function is applied to each elements of the inputRDD
Filter
====
Filter is a transformation , that returns a new RDD , with only the elements that passes the filter condition.
say, if inputRDD contains values 1 to 3, then applying transformation to filter elements that are not '1' , will return only '2' and '3' as the resultant RDD.
Flatmap
=======
Flatmap() is the transformation that takes a function , and applies the function to each elements of the RDD as in map() function.
The difference is that flatmap will return multiple values for each element in the source RDD.
say, if inputRDD contains the values {"hello world" and "how are you"} then applying split function, to flatmap transformation, will
return an array of words like {"hello","world","how","are","you"}. Since the flatmap transformation returns multiple values for each element, there are 5 elements in the resultantRDD, where as inputRDD has only 2 elements.
To recollect , if we apply the split function to Map transformation instead of Flatmap transformation, then we will get the values as the highlighted ones.
Here the words will be splitted into multiple words, however the words belonging to an element of source RDD is still a single element in the resultant RDD.
In questa pagina del sito puoi guardare il video online Apache Spark : Commonly used Transformations : Map, Filter, Flatmap Transformations della durata di ore minuti seconda in buona qualità , che l'utente ha caricato BigDataElearning 10 luglio 2017, condividi il link con amici e conoscenti, su youtube questo video è già stato visto 33,060 volte e gli è piaciuto 445 spettatori. Buona visione!
