HADOOP SINGLE NODE INSTALLATION PROCEDURE (UBUNTU) (src:brinda.com)

Hello EveryOne

Well Come to new technology

Hadoop

The above you can the see the architecture of the Hadoop, mainly consisting of 4 components i.e., Jobtracker, Tasktracker, Namenode and Datanode.

Let us discuss the 4 components of the Hadoop and later will discuss in depth about various components of the Hadoop

HADOOP SINGLE NODE INSTALLATION PROCEDURE (UBUNTU) (src:brinda.com)

1. Disabling IPV6

Step 1: Open the sysctl file using the following command

$ sudo gedit /etc/sysctl.conf

(Add the following lines end of the file) 
# disable ipv6
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1

Step 2: Test ipv6 is disabled or not.. Run below command

$ cat /proc/sys/net/ipv6/conf/all/disable_ipv6

Reboot the machine in order to make the changes take effect
A return value of 0 means IPv6 is enabled, a value of 1 means disabled.

2. Setting JAVA PATH

Step 1: Following command checks which version of java is currently being used by system.

$ java –version


Step 2: Following command is used to know the java path

$ sudo update-alternatives –config java 

Step 3: To set the java path, "export" command is used as following

$ export JAVA_HOME=/usr/lib/jvm/java-7-oracle 

Step 4: The Following command conforms the java path is set

$ echo "$JAVA_HOME" 

Step 5: Open ~/.bashrc file using gedit and copy the Java Home path in the ~/.bashrc file 

3. Setting Hadoop

Step 1: Download Hadoop from its official site
$ wget http://apache.mesi.com.ar/hadoop/common/hadoop-1.2.1/hadoop-0.20.0.tar.gz

Step 2: Unzip and move the Hadoop folder into opt folder

$ tar –xyz hadoop-0.20.0.tar.gz
$ sudo mv /home/vce/Download/hadoop /opt/

Step 3: Give the permission to hadoop folder by using the following command

$ sudo chmod 777 –R /opt/hadoop/ 

Step 4: To set the hadoop path, "export" command is used as following

$ export HADOOP_HOME=/opt/hadoop 

Step 5: Open ~/.bashrc file using gedit and copy the Hadoop Home path in the ~/.bashrc file

4. Configuring Hadoop

Step 1: Change directory to hadoop folder

$ cd /opt/hadoop

Step 2: Edit the following hadoop configuration files and make following changes.

$ sudo conf/core-site.xml

Step 3: Edit the following hadoop configuration files and make following changes.

$ sudo conf/hdfs-site.xml

Step 4: Edit the following hadoop configuration files and make following changes.

$ sudo conf/mapred-site.xml

Step 5: Edit the following hadoop configuration files and Set JAVA_HOME path as per your system configuration for java.

$ sudo conf/hadoop-env.sh

Step 6: Format Name Node

$ bin/hadoop namenode –format

5. Start Hadoop Services

Step 1: The following command start all hadoop services.
$ bin/start-all.sh
6. Test and Access Hadoop Services

Step 1: ‘jps” command checks if all services are started well.

$ jps

7. Configuring Hadoop with Eclipse

Step 1: Download Eclipse Europa from its official site

$ wget http://eclipse.org/downloads/packages/release/europa/eclipse-europa-linux-gtx.tar.gz

Step 2: Unzip and give the +x permission to eclipse folder by following command.

$ sudo chmod +x /opt/eclipse/

Step 3: Copy eclipse-plugin from hadoop folder to eclipse folder using following command.

$ sudo cp /opt/hadoop/contrib/eclipse-plugin/hadoop-0.20.2-eclipse-plugin.jar /opt/eclipse/
plugins/