PowerScale OneFS HDFS Configuration Guide

HDFS Transparent Data Encryption implements transparent end-to-end encryption. With TDE, the data can only be encrypted and decrypted by the client, and HDFS never stores or has access to unencrypted data or unencrypted data encryption keys.

Transparent data encryption satisfies two typical security, regulatory, and compliance requirements for encryption:

• Data encryption on persistent media, such as on-disk data
• In-transit encryption, for example when data is traveling over the network

How transparent data encryption works

Once configured, data read from and written to configured HDFS directories, called encryption zones, is transparently encrypted and decrypted without requiring changes to user application code. The contents of encryption zones are transparently encrypted upon write and transparently decrypted upon read.

Each encryption zone is associated with a single encryption zone key, which is specified when the zone is created. Each file within an encryption zone has its own unique data encryption key (DEK). Data encryption keys are never handled directly by HDFS. Instead, HDFS only handles an encrypted data encryption key (EDEK). HDFS clients decrypt the EDEK, and then use the subsequent DEK to read and write data. During this transfer, HDFS data nodes simply see a stream of encrypted bytes.

Limitations

• HDFS TDE for OneFS 8.2.0 is limited to HDFS client access only. Note that HDFS TDE is not a multi-protocol feature. Once the data is ingested, it can only be decrypted by the HDFS client.
• Data written to and read from HDFS encryption zones should only be accessed over the HDFS protocol.
• The OneFS web administration interface configuration is not available for Key Management Server (KMS) configuration or for encryption zone management. Encryption zones can only be created using the OneFS command-line interface (CLI). Hadoop command-line tools to create encryption zones are not currently supported in OneFS 8.2.0.
• SyncIQ will not move encryption zones for OneFS 8.2.0.

You can configure TDE using the following command-line interface (CLI) commands:

• isi hdfs crypto encryption-zones create
• isi hdfs crypto settings modify
• isi hdfs crypto settings view
• isi hdfs crypto encryption-zones list

The CLI syntax is described in the HDFS Commands section of this guide.