Configuring Ambari Hive View with OneFS


Configuring Ambari Hive View with OneFS




Note: This topic is part of the Using Hadoop with OneFS - PowerScale Info Hub.

This article describes how to configure Ambari Hive View to work with OneFS.


Web-based Hive with OneFS

Ambari Hive View is a lightweight tool for interacting with Hortonworks or Open Data Platform cluster's Hive database. With just a few configuration steps, you can configure it to work with PowerScale's OneFS storage.


Ambari User

When you open Hive View and execute queries, the current Ambari Server user is accessing Hive data on OneFS. If you accepted the defaults during Ambari setup, you are currently signed into Ambari Server as the user admin.

In the default OneFS configuration, admin is a System zone user with limited file system action privileges. Predictably, trying to use Hive View as admin in the System zone is not going to work. Here are a few errors you see while attempting to use Hive View as admin user in OneFS's System zone.

In Hive View:

  • Permission denied: user=admin, access=READ, path="/user/admin/.AUTO_HIVE_INSTANCE.defaultSettings"
  • E090 HDFS020 Could not write file /user/admin/hive/jobs/hive-job-15-2016-08-17_12-00/query.hql [HdfsApiException]

In OneFS hdfs.log (example is from 8.0 or higher):

  • twilight-sparkle-18: 2016-08-17T12:00:51-07:00 <30.6> twilight-sparkle-18 hdfs[5695]: [hdfs] RPC V9 getFileInfo user: admin exception: org.apache.hadoop.security.AccessControlException cause: Permission denied: user=admin, access=READ, path="/user/admin/hive/jobs/hive-job-1-2017-08-17_12-00" <FS Exception at CreateHandleFromPath():397 status: STATUS_ACCESS_DENIED = 0xC0000022 Path: user/admin/hive/jobs/hive-job-1-2016-08-17_12-00>

We recommend that you create a new user in Ambari; for example, ambari-view-user. The following procedure assumes that you are authenticating with local users.

1. Enter the Manage Ambari page using the drop down on the top right of Ambari's UI. Create the user in the Manage Users page.


Image

2. On the OneFS cluster, create admin-view-user as well. You need to create a user directory and determine an appropriate group. This example assumes the group is Users.

isi auth users create --name ambari-view-user --primary-group Users --home-directory /ifs/%zone%/%hdfs_root%/user/ambari-view-user --enabled=yes --zone=%zone%

3. Give group ownership for the new user's HDFS user directory to hadoop.

chgrp hadoop /ifs/%zone%/%hdfs_root%/user/ambari-view-user


Kerberos and Proxy users

For now, OneFS does not support Hive View in a Kerberos environment. If you correctly configure proxy users for the Hive View, you still encounter an error stating that GetDelegationToken is not supported.

The Ambari documentation for Hive View has two preparation steps regardless of where you store HDFS data: create HDFS proxy users and user directories. The user directory creation is described in the previous procedure. You now need to create the proxy user.

After OneFS webhdfs implements GetDelegationToken, the proxy user configuration is necessary in both Ambari and on OneFS. HiveServer2 checks the proxy configuration for its local HDFS client before attempting to access HDFS. OneFS needs a user account for the Ambari Server principal, and a proxy user setting that allows that account to act as ambari-view-user (or admin). The Hive proxy user needs to have ambari-view-user (or admin) added as a member. Follow the instructions in Ambari for configuring Views for Kerberos, and also for Hive View. Without the proxyusers settings added in Ambari, Hive View fails to start even if the proxy users are correctly configured on OneFS.

HDFS and WebHDFS protocol access


For Ambari 2.2.x and earlier

In Ambari 2.1 and Ambari 2.2, the Hive View works correctly when you use the default view Cluster Configuration. However, if you need to use custom Hive View Cluster Configuration, be sure to use hdfs:// protocol and port 8020 for the WebHDFS FileSystem URI. This setting is on the Views page, accessible from the Manage Ambari page.

Image


Ambari 2.4.0

Ambari 2.4.0 has a new Hive View 1.5.0. Both the old and new Hive View have changed the default HDFS protocol behavior. It now always uses WebHDFS if it is enabled, whereas views previously matched the HDFS Site's fs.defaultFS setting.

The OneFS WebHDFS is not currently compatible with Hive View. To work around this problem, use custom Cluster Configuration in the Hive View settings with hdfs:// as indicated in the previous image.

See Hortonworks' Ambari documentation for details on the required settings in Hive View Cluster Configuration.



Article ID: SLN319161

Last Date Modified: 07/08/2020 06:58 PM

Rate this article

Accurate
Useful
Easy to understand
Was this article helpful?
0/3000 characters
Please provide ratings (1-5 stars).
Please provide ratings (1-5 stars).
Please provide ratings (1-5 stars).
Please select whether the article was helpful or not.
Comments cannot contain these special characters: <>()\
characters left.