Flatmap vs map in Apache Spark

Image for post
Image for post
Photo by Vadim Sadovski on Unsplash
object LocalPi {

val conf: SparkConf = new SparkConf().setMaster("local[*]").setAppName("test")

def main(args: Array[String]): Unit = {
val sc = new SparkContext(conf)
val input = sc.parallelize(List("akka netflix", "production", "ata", "abc"))
val result = input.map(x => x)
println(result.collect().mkString(","))
}

}
akka netflix,production,ata,abc
object LocalPi {

val conf: SparkConf = new SparkConf().setMaster("local[*]").setAppName("test")

def main(args: Array[String]): Unit = {
val sc = new SparkContext(conf)
val input = sc.parallelize(List("akka netflix", "production", "ata", "abc"))
val result = input.flatMap(x => x)
println(result.collect().mkString(","))
}

}
a,k,k,a, ,n,e,t,f,l,i,x,p,r,o,d,u,c,t,i,o,n,a,t,a,a,b,c

References

Learning Spark Lightning-Fast Big Data Analysis

Written by

A passionate automation engineer who strongly believes in “A man can do anything he wants if he puts in the work”.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store