There are three current OneFS versions affected: 126.96.36.199, 188.8.131.52, and 184.108.40.206. The upcoming release 220.127.116.11 will be similarly impacted. 18.104.22.168 and 22.214.171.124 would only allow upgrade from one release of HDP 2.2 to another maintenance release, eg 2.2.6 to 2.2.8. OneFS 126.96.36.199 introduced compatibility with Ambari 2.1.0 and HDP 2.3, which is the target of this post.
These steps walk through manual upgrade, based on the HDP upgrade guide. HDP 2.3.0 and 2.3.2 are both possibilities, but because of compatibility requirements of OneFS, Ambari 2.1 must be used. Also the steps can be applied to upgrades to maintenance releases such as 2.2.6 to 2.2.8, but they are specifically written for an upgrade from 2.2 to 2.3 and align with those HDP upgrade steps. The examples assume that postgresql is used for the Ambari database, and the steps were confirmed on Ambari Server installed on CentOS 6.6The procedures in this article use the following variables:
Because OneFS is not sending expected information to Ambari Server, in some cases no entry will be created during installation for it in the database table that tracks the state of HDP versions per host. In this sequence you will check if this is true and correct it if so.
In the results, there should be a line with the OneFS host and the state set to CURRENT. This is what it would look like:
insert into host_version (id, repo_version_id, host_id, state) values ('$NEWID', (select repo_version_id from repo_version where version = '$HDPVERSION-PACKAGE'), (select host_id from hosts where host_name = '$ONEFSHOST'), 'CURRENT');
In this section, you will have Ambari install packages on all of the HDP hosts, and then edit the database to reflect the state as if OneFS had provided installation results to Ambari Server.
Begin the HDP manual upgrade steps from the Hortonworks upgrade guide.Before the final step of "Installing a New Version" (2.2 step 6), the UI will be hung with the status "Installing" and the progress bar will be full but blue. Clicking on "Installing" will show that all hosts are complete and green. Ambari Server is waiting to hear version information about successful installation on the OneFS host. It will look like this:
update host_version set state = 'INSTALLED' where state in ('INSTALLING', 'INSTALL_FAILED'); update cluster_version set state = 'INSTALLED' where state in ('INSTALLING', 'INSTALL_FAILED');
Installation in the Ambari Server version page should complete at this point, transitioning to a "Perform Upgrade" button. Do not press it.
In this section you will follow the "Preparing to Upgrade" section of the HDP upgrade guide, skipping a number of irrelevant steps for HDFS. The Ambari database will need to be updated as if OneFS has completed upgrade. The normal step to complete upgrade will not work, so that is replaced with further database edits and an ambari-server command line action.
Begin 2.3, "Preparing to Upgrade"
Complete through 2.3.5 step 1
Skip 2.3.5 steps 2 to 8.
Resume from 2.3.5 step 9.
Do not perform 2.4.1 on OneFS, but do it on other HDP hosts.
At 2.4.4 step 1, move OneFS components to the new version. First stop Ambari Server from the command line then enter the database as described above. Then enter the following queries:
update hostcomponentstate set upgrade_state = 'COMPLETE' where host_id in (select host_id FROM hosts where host_name = '$ONEFSHOST') ;
update hostcomponentstate set version = '$HDPVERSION-PACKAGE' where component_name in ('SECONDARY_NAMENODE', 'NAMENODE', 'DATANODE');
Exit psql and start Ambari Server again.
In 2.4.4, skip steps 1 to 7.
At 8(b), perform Start instead of Restart All.
8(c) is irrelevant. Skip steps 9 to 13.
Resume and complete service upgrade tasks.
After 2.4.21 step 1, set all components to the current stack, move OneFS host and cluster state to UPGRADED.
update hostcomponentstate set current_stack_id = subquery.stack_id from (select stack_id from stack where stack_version = '$STACK') as subquery;
update hostcomponentstate set upgrade_state = 'NONE';
update host_version set state = 'UPGRADED' where state IN ('UPGRADING', 'INSTALLED', 'INSTALLING');
update cluster_version set state = 'UPGRADED' where state IN ('UPGRADING', 'INSTALLED', 'INSTALLING');
ambari-server set-current --cluster-name=$CLUSTER --version-display-name=$VERSIONDISPLAYNAME
Your upgrade is now complete.
Article ID: SLN319419Last Date Modified: 02/25/2020 03:58 PM