Apache Ranger deny policies with OneFS

Apache Ranger deny policies with OneFS

Note: This topic is part of the OneFS 8.0.1 Documentation - PowerScale Info Hub.

This article describes how to use the Ranger service with Hadoop and OneFS. It includes how to deploy Ranger, configuration steps, and using the deny policy features.

HDFS Access Control with Ranger

In OneFS and later, HDFS authorization policies in Ranger are supported.

The Basics

Ranger is an add-on service in the Hortonworks Hadoop distribution. In an Apache Hadoop cluster, it can be used as a substitute for file system permissions. With OneFS integration, PowerScale uses Ranger as a layer above the filesystem. This means that Ranger can be more strict on denying file and directory access, but Ranger cannot allow a user or group access that they are not already granted on the file system.

To achieve this, OneFS uses a new deny policy in Ranger. It was added in Ranger 0.6.0, which ships as part of HDP 2.5, which in turn requires Ambari 2.4. Those are the minimum versions that you need to leverage this functionality.

Deploying Ranger

Ranger must be added to an existing cluster. If you are upgrading an HDP cluster, refer to HDP 2.5 blog post to see if you need to add users or change configurations. For example, odp-version might need changes.

Regardless, Ranger requires a service account user on OneFS. This user is included in the latest configuration script, and may already be configured in your zone. Add it if it is not there. An example command is:

isi auth users create ranger --primary-group=hadoop

After following the OneFS and Hortonworks integrated installation document, begin with the steps outlined in the Hortonworks Security Guide. The following steps use steps from the HDP 2.5 version of that guide.

In 1.3.1 "Start the Installation", in Step 5, be sure that the Ranger components are not assigned to OneFS. On the next page, also be sure that TagSync and Client checkboxes are empty next to OneFS.

Continue working through section 1.3 to install Ranger.


Enabling Deny Policies

Next, complete section 1.4.1. Also, complete section 1.5.1 if you have a Kerberized cluster. Begin section 2.

If you are using Ambari Server or higher, go to Ranger settings in Ambari and add a new setting in ranger-admin-site named ranger.servicedef.enableDenyAndExceptionsInPolicies set to true. Now skip down to Allowing Users Access via Ranger.

The following steps only apply if your Ambari Server is or lower.

The UI to create a deny policy is not enabled by default in Ranger 0.6.0. The documentation for enabling and using deny policies can be found on the Apache wiki. Here is a clearer explanation of the actual mechanics:

  1. First, download the default service definition for HDFS. You'll need to connect to a machine that has HTTP access to the host with Ranger Admin installed. Run the following command:

    curl -u admin:admin -X GET -H "Accept: application/json" -H "Content-Type: application/json" http://ranger.admin.host:6080/service/public/v2/api/servicedef/name/hdfs > ~/downloaded_definition.json
  2. Next, modify the options to enable deny.

    1. Open ~/downloaded_definition.json
    2. Search for options{}
    3. Within the curly braces add the following text, including quotation marks: "enableDenyAndExceptionsInPolicies": "true"
    4. Save and close
  3. Finally, submit the file back to Ranger Admin:

    curl -u admin:admin -X PUT -v -H "Content-Type: application/json" http://ranger.admin.host:6080/service/public/v2/api/servicedef/name/hdfs -d @downloaded_definition.json

Now, any HDFS service instance in Ranger has the deny policy options included below the allow policy options for each authorization policy.

This UI, which you will work with later, appears at the bottom of the page:


Allowing users access via Ranger

Section 2.3 includes a note describing that service instances are automatically created during Ambari Ranger installation when the service is enabled. This is not true with OneFS. You need to create a new service instance. This is described in section 2.3.2Y

Step 1 of 2.3.2 shows a screenshot of the Create Service page in Ranger. Note the following:

  1. The first line, Service Name, also must be configured in OneFS. Remember the value for later use.
  2. The Username and Password are specific to the service instance. The "Test Connection" will fail until you save and reopen the service instance.
  3. Fill in the Namenode URL with this after replacing your real smart connect FQDN: hdfs://onefs.smartconnect.name:8020

The service instance is created with a default policy "all - path", granting access to all files to the user that you included in the Service Details page. You need to add all of your OneFS zone groups (or individual users) to this policy in order to provide access. If they are not included, they will implicitly be denied all access.

By default, the user and group fields include the internal Ranger accounts as well as those in the Ranger Admin host's file system. If you create local users on OneFS, or use AD, you need to change the UserSync settings in Ambari or add the users in Ranger.

Remember, OneFS file system permissions are still honored even if this policy indicates that the user or group can access everything.

Configuring OneFS

OneFS needs to know where to download the service instance policies from, what the name of the service instance is, and Ranger needs to be enabled on OneFS.

In the WebUI, these are present in the Protocols > HDFS page in a new Ranger Plugin Settings tab.


In the CLI, the following commands configure Ranger on OneFS. Note that Ranger's "Service Name" was called "Repository" prior to 0.6.0, and still is labeled as such in the Ranger REST API. The OneFS setting for Service Name is called Repository Name.

isi hdfs ranger-plugin settings modify --policy-manager-url=http://ranger.admin.host:6080

isi hdfs ranger-plugin settings modify --repository-name=STEP-2.3.2-SERVICE-NAME

isi hdfs ranger-plugin settings modify --enabled=true

Add the OneFS root user to the group for your hadoop services accounts. It is usually hadoop. For example:

isi auth groups modify hadoop --add-user=root

Creating a deny policy in Ranger

Section 2.4.3 details the steps for creating a policy. Because deny policies are enabled, your policy edit page includes "Deny Conditions" below the "Allow Conditions". Include the group or user that should have limited access to the Resource Path, and then indicate which Permissions will be denied to that path.


After you save the policy, OneFS enforces the policy at the next download. If a user attempts to take action on a path that is denied by Ranger policy, this is reported in the OneFS HDFS log at /var/log/hdfs.log like this:

gold-squadron-3: 2016-10-11T17:55:41-07:00 <30.6> gold-squadron-3 hdfs[4842]: [hdfs] RPC V9 getFileInfo user: pops exception: org.apache.ranger.authorization.hadoop.exceptions.RangerAccessControlException cause: Permission denied: user=pops, access=EXECUTE, path="/tmp"

Also, if a user who is not in a group on the "all - path" policy attempts to access the system, OneFS assumes that the user should be denied access. The denial is reported in the OneFS HDFS log like this:

gold-squadron-3: 2016-10-11T15:31:09-07:00 <30.6> gold-squadron-3 hdfs[4842]: [hdfs] WebHDFS GETFILESTATUS user: dutch exception: org.apache.ranger.authorization.hadoop.exceptions.RangerAccessControlException cause: Permission denied due to undetermined access: user=dutch, access=EXECUTE, path=/

Article ID: SLN319164

Last Date Modified: 07/09/2020 08:59 AM

Rate this article

Easy to understand
Was this article helpful?
0/3000 characters
Please provide ratings (1-5 stars).
Please provide ratings (1-5 stars).
Please provide ratings (1-5 stars).
Please select whether the article was helpful or not.
Comments cannot contain these special characters: <>()\
characters left.