7/30/2019 Write Your 1st Map-Reduce Program

1/17

Write your first Map-ReduceProgram

Agenda

Install Eclipse on Ubuntu.Create a Java project in eclipse.Set Java build path.Add external Hadoop jars.Write WordCount program.Mapper for WordCount program.Reducer for WordCount program.Main method for WordCount program.Input file for WordCount program.

Run your WordCount program.Provide input/output arguments.Verify the output.

Install Eclipse from given site

https://apps.ubuntu.com/cat/applications/eclipse-platform/

7/30/2019 Write Your 1st Map-Reduce Program

2/17

Install Eclipse using software center

Verify successful installation

7/30/2019 Write Your 1st Map-Reduce Program

3/17

Launch Eclipse from Dash home

Choose your eclipse workspace

7/30/2019 Write Your 1st Map-Reduce Program

4/17

Create a Java project in eclipse

7/30/2019 Write Your 1st Map-Reduce Program

5/17

Create a package in the project

Create a class in the package

7/30/2019 Write Your 1st Map-Reduce Program

6/17

Set Java build path

7/30/2019 Write Your 1st Map-Reduce Program

7/17

Set Java build path

7/30/2019 Write Your 1st Map-Reduce Program

8/17

Set Java build path

Write your Map-Reduce program

7/30/2019 Write Your 1st Map-Reduce Program

9/17

Mapper for WordCount

public static class WordCountMapper extendsMapper

{private final static IntWritable one = new IntWritable(1);private Text word = new Text();public void map(Object key, Text value, Context context) throwsIOException, InterruptedException{StringTokenizer itr = new StringTokenizer(value.toString());while (itr.hasMoreTokens()){word.set(itr.nextToken());context.write(word, one);

}}}

Reducer for WordCount

public static class WordCountReducer

7/30/2019 Write Your 1st Map-Reduce Program

10/17

extends Reducer{private IntWritable result = new IntWritable();public void reduce(Text key, Iterable values,Context context) throws IOException, InterruptedException

{int sum = 0;for (IntWritable val : values){sum += val.get();}result.set(sum);context.write(key, result);}}

Main Method

public static void main(String[] args) throws Exception{Configuration conf = new Configuration();if(args.length != 2){System.err.println("Usage: WordCount ");System.exit(2);

}Job job = new Job(conf, "Word Count");job.setJarByClass(WordCount.class);job.setMapperClass(WordCountMapper.class);job.setReducerClass(WordCountReducer.class);job.setOutputKeyClass(Text.class);job.setOutputValueClass(IntWritable.class);FileInputFormat.addInputPath(job, new Path(args[0]));FileOutputFormat.setOutputPath(job, new Path(args[1]));System.exit(job.waitForCompletion(true) ? 0 : 1);}

Input for WordCount program

7/30/2019 Write Your 1st Map-Reduce Program

11/17

Run your WordCount program

7/30/2019 Write Your 1st Map-Reduce Program

12/17

Provide input/output arguments

7/30/2019 Write Your 1st Map-Reduce Program

13/17

Output of WordCount program

7/30/2019 Write Your 1st Map-Reduce Program

14/17

Create JAR for your program

7/30/2019 Write Your 1st Map-Reduce Program

15/17

Create JAR for your program

7/30/2019 Write Your 1st Map-Reduce Program

16/17

Create JAR for your program

7/30/2019 Write Your 1st Map-Reduce Program

17/17

Thanks

Contact Point :-

www.bispsolutions.com