2015/12/9 spark meetup december講演資料

Copyright 2015 CREATIONLINE, INC. All Rights Reservedⓒ

Spark Meetup December

2015.12.9

1


Agenda

• Spark Wrap up : 2014-2015• Upcoming: Spark 1.6

2 2


Spark Wrapup: 2014-2015

3


2014-2015 overview

[Feb/2] 0.9.0

[Apr/4] 0.9.1

[May/30] 1.0.0

[Jul/1] 1.0.1

[Aug/5] 1.0.2

[Sep/11] 1.1.0

[Nov/26] 1.1.1

[Dec/18] 1.2.0

[Feb/9] 1.2.1

[Mar/13] 1.3.0

[Apr/13] 1.2.2/1.3.1

[Jun/11] 1.4.0

[Jul/15] 1.4.1

[Sep/9] 1.5.0

[Oct/2] 1.5.1

[Nov/9] 1.5.2

[Feb/27] Apache TLP

[Jun/30-Jul/2] Spark Summit 2014

[Mar/18-19] Spark Summit East 2015

[Nov/5] wins Daytona Gray Sort

100TB Benchmark

[Oct/27-29] Spark Summit Europe 2015

[Jun/15-17] Spark Summit 2015

2014 2015

4


Major Distribution Support

2014/042014/052014/062014/072014/082014/092014/102014/112014/12

2015/012015/022015/032015/042015/052015/062015/072015/082015/092015/102015/112015/12

0.9.11.0.01.0.1

1.0.21.1.0

1.1.11.2.0

1.2.11.3.0

1.2.2/1.3.1

1.4.01.4.1

1.5.01.5.11.5.2

CDP 5.1.0(Spark 1.0)





MapR 4.0.0(Spark 0.9)

MapR 4.0.1(Spark -.-)

MapR 4.0.2(Spark -.-)

MapR 4.0.2(Spark 1.4.1/1.3.1/1.2.1)

MapR 5.0.0(Spark 1.4.1/1.3.1)

HDP 2.2.4(Spark 1.2.1)

HDP 2.3.0(Spark 1.3.0)

HDP 2.2.8/2.3.2(Spark 1.4.1/1.3.1)

5


Community Growth

600Last 12 months

300Last 12-24 months

6


Language Share

7


Commiters Share

144

4

32

14OTHERS

8


Japanese Spark Guru goes to Commiter

9


Major Confs

10


Spark Survey 2015

https://databricks.com/blog/2015/09/24/spark-survey-results-2015-are-now-available.html

11


Upcoming: Spark 1.6

12


Upcoming: Spark 1.6

• Dataset API: typed interface over Dataframes / Tungsten

• DataFrame integration with GraphX and Streaming

• More Tungsten features: faster in-memory cache, SSD storage, better code generation

• Data sources for Streaming

13

2015/12/9 spark meetup december講演資料

Software