How to setup Hadoop for Development

If you enthusiastic or interested in the code of Apache Hadoop then perform the following steps to build and view the code either in Netbeans or Eclipse IDE

Requirements:

1. Linux box


2. subversion


3. Java 1.6


4. Apache Maven


5. Apache Ant

Checkout Code:

You need to check out the code from trunk of apache hadoop

svn co http://svn.apache.org/repos/asf/hadoop/common/trunk/ Hadoop

Protocol Buffers:

Hadoop uses google's protocol buffers for serialization. You need to install it

1. Download the protocol buffers from google code (search it for the link). I have downloaded protocolbuf-2.4.1.tar.gz


2. Extract the downloaded file and navigate to the extracted folder through command line and run following steps

./configure
make
sudo make install

Note:  Make sure that every step passes before proceeding to next step

Note: Step 3 make sure you run it as super user or sudo

3. Test your protocol buffer installation by running protoc in a command line and you should get something like “Missing input file”

Setup:

Navigate to the folder where you have checked out Hadoop and run the following commands. It is just basic maven installation

[sourcecode lanaguage="shell"]
mvn clean
mvn install -DskipTests


*Note: Build might fail if so then rerun the 2 steps. It should work for the second time.

At this point you are good to go.

*If you use Netbeans then it is piece of cake to load maven projects. Navigate to the Hadoop folder and Netbeans should automatically recognize the maven project. Click Open to open the project

*If you use eclipse then run additional command after step 2 in Setup section

[sourcecode lanaguage="shell"]
mvn eclipse:eclipse


If you are interested in sources and javadocs

[sourcecode lanaguage="shell"]
mvn eclipse:eclipse -DdownloadSources=true -DdownloadJavadocs=true


Now open the eclipse and import > Existing projects into workspace > Next, navigate to the Hadoop folder and click Finish.

Please go to Apache Hadoop's site for more details.