Cloudera and Isilon Implementation Part 2


Cloudera and Isilon Implementation Part 2




Note: This topic is part of the Using Hadoop with OneFS - Isilon Info Hub.

This article continues where Cloudera and Isilon Implementation Part 1 stopped.

To deploy a Hadoop Cluster with Isilon, select Custom Services install. This option allows you to select the required components.

Image

Initially no services are selected.

Image

Select the Hadoop services that you want to deploy;

DO NOT SELECT HDFS. You are deploying Isilon as the storage & effective NN. Therefore, you do not need the Cloudera HDFS Service.

MapReduce is also not recommended, because Yarn includes MapReduce2. if your application is legacy and not written for Yarn, then you can enable and use MRv1.


Image

After selecting the services, continue and assign roles. Because this is a single host deployment, all roles are deployed on the same host. Consult the Cloudera documentation for best practices regarding role assignments.


Image
Image
Image

Leave the Isilon role as default. There is no reason to change it.
Image

Continue.

Image

Because this is test host, select all defaults. Consult the Cloudera documentation for best practices regarding the database.

Test the Database Connections.

Image

Continue.

On the Cluster Setup page, assign the Isilon to the following two parameters. Notice that the ports are different.
  • default_fs_name hdfs://smartconnectzonename:8020
  • webhdfs_url http://smartconnectzonename:8082/webhdfs/v1
Image
Image
Image
Image
Image

Assign the two Isilon parameters.

Image

You can leave all other settings as defaults.
Continue.
Cluster Setup starts.

Image
Review the setup as it runs.

Image

Image

Setup completes. You can see additional details by opening the specific services.
Continue. You are finished with the Hadoop cluster deployment.

Image

Click Finish and return to the main Cloudera Manager dashboard to review the status.

Image
Image

It is common to see alarms and service down issues on the dashboard. Review the alarms and services, and triage them as needed. Some services may just need restarting. Follow standard protocols in starting these services.
  • Start the service.
  • Monitor and review logs as needed.
  • Review the Isilon /var/log/hdfs.log (Remember that Isilon is a clustered system, so all node logs need reviewing.)
Image

It is also common to see Configuration issues. Make the required changes as needed to resolve each issue.

Image

Restart services to resolve Alarms following configuration changes.

Image

The Hadoop cluster and Services are now fully operational and can be tested.

Image

Basic functionality can be tested.

Browse the Isilon HDFS root:
hadoop fs -ls /

Image

Write to the Isilon HDFS root:

Image

Run some basic smoketest jobs. For example, PI or teragen/teravalidate/terasort to test mapreduce.

Image

With Cloudera 5.7, you may notice that Impala service is not started fully. Additional configuration changes are needed to get this Service started.

Image

The steps to get Impala running are described here ---- > Get Cloudera 5.7 Impala starting with Isilon.

You are finished installing and configuring Cloudera CDH against Isilon OneFS.


Article ID: SLN319500

Last Date Modified: 03/27/2020 05:14 PM

Rate this article

Accurate
Useful
Easy to understand
Was this article helpful?
0/3000 characters
Please provide ratings (1-5 stars).
Please provide ratings (1-5 stars).
Please provide ratings (1-5 stars).
Please select whether the article was helpful or not.
Comments cannot contain these special characters: <>()\
characters left.