: This topic is part of the Using Hadoop with OneFS - Isilon Info Hub
This article describes the core tasks required to get a basic operational Hadoop cluster running with Isilon. Because this procedure is long, the process is described in two parts:
The article is based on the following:
- Isilon OneFS: 126.96.36.199
- CDH 5 parcel: 5.7.1-1.cdh5.7.1.p0.11
Before installing any Hadoop cluster, consult the OneFS supportability matrix
This article assumes that the following Isilon Hadoop environment is configured and operational:
- Isilon is licensed for HDFS
- A dedicated Isilon Access Zone is in use (not the system zone).
- Isilon HDFS root directory in the Access Zone exists
- The Isilon SmartConnect Zone configuration is implemented per best practice for Isilon HDFS access.
- The Isilon HDFS configuration is correctly configured.
- A simple access model exists between Hadoop and Isilon; user UID & GID and parity exists. The best approach to achieving parity is described in another article.
This article does not address the setup, configuration and deployment of the Linux hosts used to deploy Hadoop services. Consult the Cloudera documentation to set up and prepare the hosts correctly: Overview of Cloudera and the Cloudera Documentation Set
. This article also does not address advanced Cloudera installs. This article focuses on the Isilon integration into the installer and how to complete the install.
For an overview of the procedure being presented here, see Installation Path A - Automated Installation by Cloudera Manager (Non-Production Mode)
Assuming that the Isilon is setup and configured for integration with Cloudera, you can begin the deployment of the Cloudera Manager. The first task is to download the bits and install the CM.
# wget https://archive.cloudera.com/cm5/installer/latest/cloudera-manager-installer.bin
# chmod u+x cloudera-manager-installer.bin
Next, run the installer. You see the following:
Yes, Accept the Cloudera License:
Next, let Cloudera Manager install the JDK:
Yes, accept the Oracle license:
Click OK. Note the URL and the user/pass for the Cloudera Manager WebUI:
Validate that the Cloudera Manager Service is running. Iif you see problems, tail the cloudera-scm-server.log as you start the service.
# service cloudera-scm-server status
# cloudera-scm-server (pid 10487) is running...
# tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
Log in to the Cloudera Manager WebUI using user:admin, password:admin
Select the Yes check box to accept the EULA and Continue:
Select the version you wish to deploy:
Specify hosts. This example deploys to a single Linux host. The process is the same when multiple hosts are used in the Hadoop cluster.
Add the FQDN of the Linux hosts to be deployed. Use Search:
On completion of the search, select the host(s) to deploy to:
Select to use Parcels.
Select the CDH Stack you wish to deploy.
Select the Additional Parcels and Agent configuration as needed and what is supported by Isilon:
Select install the JDK.
Select install the JUSEP files if you intend to secure this cluster later.
Click Continue twice:
For this example, do not deploy in Single User Mode. Continue:
Provide the SSH credentials, either root password or SSH keys, depending on how you set up your Linux hosts and how you wish to manage them. Continue.
The installation begins.
Installation completes and the installer continues.
Parcels are downloaded.
Parcels are distributed.
Parcels are unpacked and activated.
The host inspector validates hosts, versions and additional software installed.
The installer checks and validates the hosts. If it finds any deviations, recommendations appear to optimize the hosts. If the validation checker fails, we suggest that you follow the recommendations and then re-try the validation.
Common errors are seen with:
Make the recommended changes to the hosts and run again:
This completes part 1 of the install. To continue deploying Hadoop services with Cloudera Manager, see Cloudera and Isilon Implementation Part 2 ---->