scala编程学习:map,filter,flatmap
(2015-11-11 09:40:15)
标签:
mapfliterflatmao |
分类: scala学习 |
var a = 1 to 10000
val rdd1=sc.parallelize(a)
filter vs map
val rdd4=rdd1.filter(a=>a>500)
val rdd5=rdd1.map(a=>a>500)
rdd4.count
res0: Long = 9500
rdd4.first
res3: Int = 501
rdd5.count
res0: Long = 10000
rdd5.first
res2: Boolean = false
可见,filter滤掉了小于5的部分。而map只是添加了布尔值。
map vs flatMap(flatMap 是flatten和map的结合。)
val rdd=sc.parallelize(List("I am a girl","He is a boy"))
val fm=rdd.flatMap(_.split(" "))
fm.collect
结果:Array[String] = Array(I, am, a, girl, He, is, a, boy)
val rdd=sc.parallelize(List("I am a girl","He is a boy"))
val fm=rdd.map(_.split(" "))
fm.collect
结果:Array[Array[String]] = Array(Array(I, am, a, girl), Array(He, is, a, boy))
flatten:非集合扁平化操作(捋直了)
val l=List(List("c","d"),List("dd","ss"))
l.flatten
res23: List[String] = List(c, d, dd, ss)
val rdd1=sc.parallelize(a)
filter vs
val rdd4=rdd1.filter(a=>a>500)
val rdd5=rdd1.map(a=>a>500)
rdd4.count
res0: Long = 9500
rdd4.first
res3: Int = 501
rdd5.count
res0: Long = 10000
rdd5.first
res2: Boolean = false
可见,filter滤掉了小于5的部分。而map只是添加了布尔值。
map vs flatMap(flatMap 是flatten和map的结合。)
val rdd=sc.parallelize(List("I am a girl","He is a boy"))
val fm=rdd.flatMap(_.split(" "))
fm.collect
结果:Array[String] = Array(I, am, a, girl, He, is, a, boy)
val rdd=sc.parallelize(List("I am a girl","He is a boy"))
val fm=rdd.map(_.split(" "))
fm.collect
结果:Array[Array[String]] = Array(Array(I, am, a, girl), Array(He, is, a, boy))
flatten:非集合扁平化操作(捋直了)
val l=List(List("c","d"),List("dd","ss"))
l.flatten
res23: List[String] = List(c, d, dd, ss)
后一篇:[转载]数据融合技术原理与方法