Better HDP upgrades with OneFS

Better HDP upgrades with OneFS

Note: This topic is part of the Using Hadoop with OneFS - Isilon Info Hub.

This article shows how to upgrade Hortonworks Data Platform (HDP) deployment using Ambari 2.2 when Isilon OneFS is used as the data storage for HDFS.

Choosing an upgrade guide

This guide applies to OneFS or higher, OneFS or higher, and later releases of OneFS.

There are two previous articles on this topic. One describes how to upgrade HDP, which required manual changes to work. Use that guide if you are using OneFS or lower. OneFS was subsequently updated, making the process much easier. Use the later guide if you are not yet on OneFS or, or if you are not ready to upgrade to Ambari 2.2.

The following table shows all the upgrade choices.
OneFS Ambari Guide
  • 7.2.1 family, equal or greater than
  • 8.0.0 family, equal or greater than
  • Any later family
2.2 or higher Current page
7.2.1 family equal or greater than 2.1 family. But, not 2.1.2 because of JIRA 13414 January 2016 guide (with patch) or 2.1 family November 2015 guide or 2.1.0 No upgrade possible to HDP 2.3

Ambari 2.2 mainstreamed the previously experimental skip of service checks that we used in the January 2016 guide. Also Ambari 2.2 introduced a new workflow, Express Upgrade, which speeds up the process when there is no problem for your data scientists if you disrupt jobs. This guide describes the improved Rolling Upgrade experience, and what to expect from Express Upgrade.

The Upgrade

Getting started

OneFS must report to the Ambari Server that it is at the target version. This is the 4-number release version and 4 digit build number of HDP. An example version is Determine version from the release notes of the HDP build that you are upgrading to, or by running the command 'hadoop version' on an existing deployment.

Use the following command to make the change on OneFS:

isi hdfs settings modify --odp-version=[version] --zone=[zone]

Rolling Upgrade

Begin Ambari's upgrade steps and complete installation of packages. Click the Perform Upgrade button. (If the button changed to a link named Upgrade: In Process, click on that.)

The Upgrade Options box opens:

Click the blue link to see failed pre-upgrade checks. There are two Requirements that fail because of the OneFS host configuration. Ambari expects NameNode High Availability enabled, and expects no Secondary NameNode. Those are okay because OneFS provides continuous availability internally. Hadoop's NNHA and Secondary NameNode are not truly applicable. You can ignore these errors.

If the odp-version setting in OneFS was not previously set, the following warning may appear. You can ignore it. Alternatively, correct the condition by setting odp-version on OneFS to the initial HDP version and then restarting the HDFS service in Ambari.

If any other required checks or warning messages appear, resolve those issues before continuing.

Returning to the Upgrade Options, select the "Skip all Service Check failures" checkbox, then choose Rolling Upgrade, and then click Proceed.

There may be other actions necessary to complete HDP installation, but the OneFS and HDFS aspects are complete.

Express Upgrade

No special instructions are needed for HDP upgrade using Express Upgrade with OneFS in the cluster. You may encounter the warning about the HDFS components not having version stamps, but you can ignore that warning.

Proceed through the HDP upgrade wizard until the "Upgrade finished" dialog appears.

Article ID: SLN319415

Last Date Modified: 02/28/2020 03:07 PM

Rate this article

Easy to understand
Was this article helpful?
0/3000 characters
Please provide ratings (1-5 stars).
Please provide ratings (1-5 stars).
Please provide ratings (1-5 stars).
Please select whether the article was helpful or not.
Comments cannot contain these special characters: <>()\
characters left.