2012/8/1 夏合宿 発表資料

夏合宿 発表２００１２／８／１

In 山代温泉

前田 経一

What is “The BIG Switch”

MapReduce and Functional Programming

Topics

Nicholas Carr “The BIG SWITCH : Rewiring the World, from Edison to Google”

Buzzword “Cloud” spread by the book.

Electricity From Edison : private power generation To Utility : central electricity generating

systemComputing as well From Private computing To Utility computing

From Edison to Google

Why “Private to Utility”

Urbanization

Independent vs Dependent

Do It Yourself vs Utility

Generalist vs Specialist

Country vs City

Google

MapReduce

Hadoop

Neumann architecture Program & Data

Data → Program Program → Data

Mobile Agent

Data Locality

“The problem is simple: while the storage capacities of hard drives have increased massively over the years, access speeds—the rate at which data can be read from drives— have not kept up. “

Random Access > Sequential Access

Data access speed

map(String key, String value): // key: document name // value: document contents for each word w in value: EmitIntermediate(w, "1");

Map Function

reduce(String key, Iterator values): // key: a word // values: a list of counts int result = 0; for each v in values: result += ParseInt(v); Emit(AsString(result));

Reduce Function

Map and Reduce concepts came from Functional Programming Language.

Reduce Function must have associativity becouse of non-deterministic.

DAG + Operator

MapReduce and Functional Programming