How to Configure Hadoop Cluster using Ansible Playbook

Task Descriptionđź“„

đź”° 11.1 Configure Hadoop and start cluster services using Ansible Playbook

Today I am going to show you how to configure Namenode and Datanode and start the services of both using an ansible-playbook.

Here, I am creating two playbooks. One is for Namenode and the Second is for Datanode.

First, we are going to configure Namenode

a) First Copy the software (Hadoop and Java) using the copy module.

b) Install both the software using the shell module.

c) Creating and copying the content of core-site.xml and hdfs-site.xml file using the template module.

d) Creating a Namenode Directory using the File module.

e) Format the Namenode

F) At last, Start the Namenode.

See the below image to configure Namenode.

Below are the files of core-site.txt and hdfs-site.txt for Namenode to be copied on the core-site.xml and hdfs-site.xml

hdfs-site.xml
core-site.xml

Then, Run the playbook, by ansible-playbook conf-namenode.yml

The output will be shown below:

Secondly, we are going to configure Datanode

a) First Copy the software (Hadoop and Java) using the copy module.

b) Install both the software using the shell module.

c) Creating and copying the content of core-site.xml and hdfs-site.xml file using the template module.

d) Creating a Datanode Directory using the File module.

e) At last, Start the Datanode.

See the below image to configure Datanode.

Below are the files of core-site.txt and hdfs-site.txt for Datanode to be copied on the core-site.xml and hdfs-site.xml

core-site.xml
hdfs-site.xml

Then, Run the playbook, by ansible-playbook conf-datanode.yml

The output will be shown below:

Now, the Cluster has been set up and configured.

Run hadoop hdfsadmin — report command to check the connectivity of Hadoop cluster

See the output below.

Conclusion

We have successfully configured and started the services of the Hadoop cluster using ansible-playbook.

I am a blogger. Check my blog → technicalej.in