: This topic is part of the Using Hadoop with OneFS - Isilon Info Hub
This article continues where Cloudera and Isilon Implementation Part 1
To deploy a Hadoop Cluster with Isilon, select Custom Services install. This option allows you to select the required components.
Initially no services are selected.
Select the Hadoop services that you want to deploy;
DO NOT SELECT HDFS. You are deploying Isilon as the storage & effective NN. Therefore, you do not need the Cloudera HDFS Service.
MapReduce is also not recommended, because Yarn includes MapReduce2. if your application is legacy and not written for Yarn, then you can enable and use MRv1.
After selecting the services, continue and assign roles. Because this is a single host deployment, all roles are deployed on the same host. Consult the Cloudera documentation for best practices regarding role assignments.
Leave the Isilon role as default. There is no reason to change it.
Because this is test host, select all defaults. Consult the Cloudera documentation for best practices regarding the database.
Test the Database Connections.
On the Cluster Setup page, assign the Isilon to the following two parameters. Notice that the ports are different.
- default_fs_name hdfs://smartconnectzonename:8020
- webhdfs_url http://smartconnectzonename:8082/webhdfs/v1
Assign the two Isilon parameters.
You can leave all other settings as defaults.
Cluster Setup starts.
Review the setup as it runs.
Setup completes. You can see additional details by opening the specific services.
Continue. You are finished with the Hadoop cluster deployment.
Click Finish and return to the main Cloudera Manager dashboard to review the status.
It is common to see alarms and service down issues on the dashboard. Review the alarms and services, and triage them as needed. Some services may just need restarting. Follow standard protocols in starting these services.
- Start the service.
- Monitor and review logs as needed.
- Review the Isilon /var/log/hdfs.log (Remember that Isilon is a clustered system, so all node logs need reviewing.)
It is also common to see Configuration issues. Make the required changes as needed to resolve each issue.
Restart services to resolve Alarms following configuration changes.
The Hadoop cluster and Services are now fully operational and can be tested.
Basic functionality can be tested.
Browse the Isilon HDFS root:
hadoop fs -ls /
Write to the Isilon HDFS root:
Run some basic smoketest jobs. For example, PI or teragen/teravalidate/terasort to test mapreduce.
With Cloudera 5.7, you may notice that Impala service is not started fully. Additional configuration changes are needed to get this Service started.
The steps to get Impala running are described here ---- > Get Cloudera 5.7 Impala starting with Isilon
You are finished installing and configuring Cloudera CDH against Isilon OneFS.