apache hadoop – a course for undergraduates eclipse...

Copyright © 2013 Cloudera, Inc. All rights reserved. Not to be reproduced without prior written consent.

1

Apache Hadoop – A course for undergraduates

Eclipse Reference for Labs


2

Table of Contents

Using Eclipse for MapReduce Development ................................................................... 3

Starting Eclipse ......................................................................................................................... 4 Locating Exercise Source Code ............................................................................................ 5

Working with Files .................................................................................................................. 6 Working with Java in Eclipse ............................................................................................... 7

Assembling JARs .................................................................................................................... 11

Learning more about Eclipse and Hadoop ................................................................... 14


3

Using Eclipse for MapReduce Development The Virtual Machine (VM) used for homework labs includes Eclipse pre-‐installed and configured.

There are Eclipse projects (directories) for each of the homework labs in this course that involve Java coding.

This document is intended to help students who are new to Eclipse.


4

Starting Eclipse Start Eclipse by double-‐clicking the Eclipse icon on the VM desktop.


5

Locating Exercise Source Code 1. In the Package Explorer pane on the left hand side of the Eclipse window,

open the project folder associated with the exercise you are working on. (Click the triangle icon or double-‐click the name to open a folder.) The exercise instructions will tell you which project to work on.

2. Within the project, open the src folder. Within the src folder, choose the stubs package folder, which contains partially completed source files.

A Note on Packages

It is considered good form in Java development to organize Java classes into

qualified packages to minimize the chance of conflicting package names (e.g.

“com.mycompany.foo”). In this course, to keep things simple, all code is under

unqualified simple package names.


6

Working with Files 1. Double click a source file in the Package Explorer to open it in a new tab in

the editor panel on the right.

2. You can open multiple files and switch between them using tabs. Close a file by clicking the X on the tab.

3. Save changes to a file by clicking the Save icon ( ), typing Ctrl-‐S or using the File>Save menu option.


7

Working with Java in Eclipse Eclipse has many features to make working with Java much easier. Here are just a few you may find helpful in class. Refer to Eclipse help or many available useful Eclipse references on the Internet for more information.

Context-sensitive Auto-complete

Eclipse will attempt to provide you with valid choices depending on the current cursor location. For example, in the image below, locating the cursor next to job. pops up a list of available methods on the job object.

Type Ctrl-‐Space to request context-‐sensitive help at any time. In the example below, placing the cursor next to the for keyword and typing Ctrl-‐Space brings up a list of suggestions. Choosing one (e.g. “iterate over array”) inserts a template for the selected usage.


8

Auto-compile

The auto-‐compile feature in Eclipse is enabled, which means that after you make a change, Eclipse will automatically attempt to compile your Java code into a .class file. If there are errors preventing compilation, it will mark them in your source code, as well on the file and folder in the Package Explorer.

Error checking

When Eclipse detects a problem in your code, it will display an error or warning icon along the left edge (known as the “gutter”). If you hover over the icon, a description of the problem pops up.

In the example below, we’ve passed an incorrect parameter type to the Job constructor.

Access to Hadoop API Javadocs

Hover your mouse over a method or variable to view the Javadocs. (This feature only works if you have Internet access.)


9

Access to Hadoop Source Code

The training VM has been provisioned with the Hadoop source code (Java). Select a Hadoop element and type F3 to open the element’s source code.

In the example below, selecting the Mapper class name and typing F3 opens the source file for that class.


10

The source code location is configured per library in the Java build path. Source code might not be available for all libraries.


11

Assembling JARs In order to test your code, you must assemble a JAR file containing the compiled class files in the project using the following steps.

1. Verify that your Java code does not have any compiler errors or warnings.

2. In the Package Explorer, open the Eclipse project for the current exercise (e.g. averagewordlength). Right click on the package you want assemble (e.g. stubs) and select Export.

A red X indicates a compiler error


12

3. Select Java > JAR File from the Export dialog box, then click Next.

4. Specify a location for the JAR file. You can place your JAR files wherever you like, e.g.:


13

5. You can then use your JAR file as you normally would. For example, to use it to submit a MapReduce job, open terminal window, change to the directory where you placed your JAR file, and run the hadoop jar command. Remember that you will need to reference the class using its package-‐qualified name, e.g. stubs.AverageWordLength.


14

Learning more about Eclipse and Hadoop For more information about using Eclipse with Hadoop, please see a blog post on Cloudera’s website: http://blog.cloudera.com/blog/2013/08/how-to-use-eclipse-with-mapreduce-in-clouderas-quickstart-vm/ For more information about Eclipse in general, there are numerous excellent tutorials and full documentation available on the Web.

apache hadoop – a course for undergraduates eclipse...

Documents