Data Domain: Retention Lock Frequently Asked Questions

What is retention lock?

What are the different versions of retention lock?

• Governance: The less strict of the two RL functions. Locks against files can be reverted if necessary.
• Compliance: The stricter of the two functions, introduced in DDOS 5.3. It adheres to several common regulatory standards; that is, locks against files cannot be reverted. The DD must be configured with a "security officer" user to authenticate certain commands. There are various restrictions on other functionality to prevent locked data from being removed or locks being reverted early.

• Each flavor of RL requires a separate license key.
• RL functionality is enabled on a per-mtree basis.
• A single system can use both governance and compliance mode against separate mtrees. However, it must have separate governance and compliance licenses installed.
• Do not enable any kind of RL on mtrees used to store Avamar data unless called for in the Avamar documentation as part of getting this feature running on Avamar. Enabling DD RL on any such mtree without following the right Avamar process can render the mtree unusable for backups and incur the need for lengthy recovery. This is specially true if enabling Automatic Retention Lock (ARL) for an Avamar mtree.
• Retention lock functionality may not be supported against mtrees that are used to store data using older versions of Avamar or Protection Software data on an Integrated Data Protection Appliance or PowerProtect Data Protection Series appliance. This can prevent Avamar or Protection Software inside Integrated Data Protection Appliance from functioning as expected and put them into READONLY state.

Which data access protocols are supported with retention lock?

• The NFS, CIFS, and DD Boost protocols are fully supported against mtrees using retention lock governance or compliance mode.
• The Virtual Tape Library (VTL) protocol is only supported against mtrees using retention lock governance mode. Automatic retention lock is not supported on Data Domain VTL. See the Data Domain Administration Guide to determine how to unlock retention-locked tapes so that they can be written to.

What regulatory standards does retention lock compliance mode meet?

How is retention lock governance enabled?

1. Add a retention lock governance (RLG) license to the DD.
2. Enable RLG against any required mtree:

   # mtree retention-lock enable mode governance mtree <mtree>
   

How is retention lock compliance enabled?

1. There are specific instructions for some newer Data Domain models with iDRAC.
2. Add a retention lock compliance (RLC) license to the DD.
3. Create a user with the role of "security" (assuming such a user does not exist):

   (ADMIN USER) # user add <username> role security

4. Log in to the DD with the "security" user and enable security user authorization:

   (SECURITY USER) # authorization policy set security-officer enabled

5. Configure the system for retention lock compliance mode. Once the following command runs to completion, the system reboots automatically:

   (ADMIN USER) # system retention-lock compliance configure

6. Once the system has rebooted, enable retention lock compliance mode on the system:

   (ADMIN USER) # system retention-lock compliance enable

   Note: For newer DDOS releases, this task must be performed with the Data Domain UI under Administration > Compliance
7. Enable retention lock compliance mode against any required mtrees:

   (ADMIN USER) # mtree retention-lock enable mode compliance mtree <mtree>

How is it possible to determine which mtrees have retention lock enabled?

# mtree list
Name                                Pre-Comp (GiB)   Status    Tenant-Unit
---------------------------------   --------------   -------   -----------
...
/data/col1/rich-retention-lock                 0.0   RW/RLGE   -
/data/col1/rl_test                             0.0   RW/RLGD   -
/data/col1/rl_test_comp                        0.0   RW/RLCE   -
/data/col1/test                                3.1   RW/RLGE   -
...
---------------------------------   --------------   -------   -----------
...
 RLGE : Retention-Lock Governance Enabled
 RLGD : Retention-Lock Governance Disabled
 RLCE : Retention-Lock Compliance Enabled

Can an mtree with Retention Lock Governance enabled be converted to Retention Lock Compliance?

Can an mtree with Retention Lock Compliance enabled be converted to Retention Lock Governance?

How are file retention or lock periods set?

# mtree retention-lock set min-retention-period <period> mtree <mtree>
# mtree retention-lock set max-retention-period <period> mtree <mtree>

How can existing retention periods be displayed?

# mtree retention-lock show min-retention-period mtree <mtree>
# mtree retention-lock show max-retention-period mtree <mtree>

# mtree retention-lock show min-retention-period mtree /data/col1/rl_test
Retention-lock min-retention-period of mtree /data/col1/rl_test is: 720 minutes.
# mtree retention-lock show max-retention-period mtree /data/col1/rl_test
Retention-lock max-retention-period of mtree /data/col1/rl_test is: 30 days.

How are files within an mtree with retention lock enabled locked?

• When retention lock is enabled against an mtree, existing files within the mtree are not automatically locked. That is, all preexisting files remain read or write.
• When a new file is written to an mtree with retention lock enabled, the file is not automatically retention locked. That is, the new file remains as read or write.
• Starting with DDOS 6.2.0.20, there is a feature in DDOS called "automatic retention lock," which can set a lock on all written files automatically. See the "Automatic Retention Lock" section further down in this article to learn more.
• To retention lock a specific file, the atime of the file must be modified to match the date and time the file should be retention locked. That is, the date and time until which the file should remain read-only. Until the time is modified in this way, the file cannot be retention locked (and can be modified or removed).
  • A file's atime can be changed from an NFS or CIFS client using the 'touch' command:

    # touch -a -t <expiry time> <file to be locked>

  • For example, to set atime of /data/col1/rl_test/testfileto 07:05 on 8 June:

    # touch -a -t 06080705 /data/col1/rl_test/testfile

  • The period from current time to future atime must be within the minimum and maximum retention periods for the mtree. Otherwise, an error is generated when modifying the file's atime:

    # touch -a -t 08080705 /data/col1/rl_test/testfile
    touch: setting times of `/data/col1/rl_test/testfile': Permission denied

  • A corresponding message is also shown in the Data Domain File System (DDFS) log files:

    06/07 13:44:57.197 (tid 0x2b28ee5258c0): Attempt to set atime of /data/col1/rl_test/testfile to larger than maximum retention period of mtree.

  • CIFS (Windows) clients do not, by default, include a touch command or utility. However, several such utilities are freely available for download from various third-party websites.

Which backup applications support automatically retention locking files after writing them to a DD?

What is Automatic Retention Lock?

# mtree retention-lock set
{min-retention-period | max-retention-period |
automatic-retention-period | automatic-lock-delay} <period>
mtree <mtree-path>

What can or cannot be done against a locked file?

• Retention-locked files are read-only and cannot be modified or deleted.
• Once a file's retention period expires, it is "unlocked". When in an unlocked state the file still cannot be modified, but it can be deleted. The DD does not automatically delete a file when its retention period expires - subsequent deletion must be performed from a client system or backup application.
• Once set, the retention period for a specific file cannot be reduced (that is, the file's atime cannot be brought forward).
• Retention periods can, however, be increased (atime can be increased up to the maximum retention period for the mtree).
• Ownership and permission settings for a file can continue to be modified while the file is locked.
• It is only possible to rename or delete a directory in a retention lock mtree (RLGE, RLGD, or RLCE) if that directory contains no files. If the directory contains files (even if these files are not retention locked), the rename or delete of the directory fails
• Even if it does not change the file contents itself, renaming locked files is not allowed. Renaming is also not allowed once the file's lock has expired. For files that are no longer locked, the only permitted operation is a delete. This is changed in DDOS 7.7.4 - files that are no longer locked can be renamed.

Is it possible to completely remove the retention lock against a file or set of files?

# mtree retention-lock revert <path>

# mtree retention-lock revert /data/col1/rl_test_comp/testfile
This operation is not allowed. Mtree is in retention-lock compliance mode.

What happens if a retention-locked file is attempted to be modified or removed?

# echo " test2" >> /data/col1/rl_test/testfile
bash: testfile: Permission denied
# rm testfile
rm: remove write-protected regular file `testfile'? y
rm: cannot remove `testfile': Permission denied

06/07 07:06:59.756 (tid 0x2b5a77605d50): Atime of retention-lock file /data/col1/rl_test/testfile is not expired.
06/07 07:07:42.504 (tid 0x2b5a79111390): Atime of retention-lock file /data/col1/rl_test/testfile is not expired.

Is it possible to list all files which are retention locked?

mtree retention-lock report generate retention-details
mtrees {<mtree-list> | all}
[format {text | tsv | csv}]
[output-file <file-name>]
               Report detailed information of
               retention-lock files.

# mtree retention-lock report generate retention-details mtrees /data/col1/jftest
Report generated on: Fri Jul  1 14:19:31 2016

Report for mtree: /data/col1/jftest
File Path        Mode        Size(Bytes)        Expiration Date
/data/col1/jftest/file1 governance 10521456 Sat Jul  2 22:35:48 2016
/data/col1/jftest/testdir/file2 governance 10521680 Sat Jul  2 22:35:42 2016
/data/col1/jftest/file3 governance 10521820 Sun Jul 10 22:36:09 2016
Total files: 3

Is it possible to completely disable retention lock for an mtree after it is enabled?

# mtree retention-lock disable mtree /data/col1/rl_test
Retention-lock feature is disabled (previously enabled) for mtree /data/col1/rl_test.

# mtree list
Name                                Pre-Comp (GiB)   Status    Tenant-Unit
---------------------------------   --------------   -------   -----------
...
/data/col1/rl_test                             0.0   RW/RLGD   -
...

• New files written to the mtree cannot be retention locked.
• Files that are already locked stay locked for their previously defined retention period. That is, all locks are not automatically reverted when retention lock is disabled against an mtree.
• Existing locks against files in an mtree where retention lock is disabled cannot be reverted. All necessary reversions must be done before the retention lock being disabled:

  # mtree retention-lock revert /data/col1/rl_test/testfile
  **** Retention-lock feature is disabled (previously enabled) for mtree which contains the path /data/col1/rl_test/testfile.

• Retention lock cannot be disabled for an mtree using compliance mode:

  # mtree retention-lock disable mtree /data/col1/rl_test_comp
  **** Operation is not allowed because the system is a retention-lock compliance system

• The file system may report "not responding" while the revert command runs in the background:

  # mtree retention-lock revert /data/col1/myTree
  The 'mtree retention-lock revert' command removes retention-lock on this path thereby making it unprotected.
  Are you sure:  (yes | no)  [no]:  yes
  Ok, proceeding
  Please enter sysadmin password to confirm 'mtree retention-lock revert':
  **** The filesystem is not responding.

Can replication still be used against mtrees which have retention lock enabled?

• Directory replication
  • Only supported for files using governance mode
  • Does not replicate minimum and maximum retention periods to the destination system
• MTree replication
  • Can be used for either governance or compliance mode data
  • Does replicate minimum and maximum retention periods to the destination system
• Collection replication
  • Can be used for either governance or compliance mode data
  • Does replicate minimum and maximum retention periods to the destination system

• Both the source and destination systems must have corresponding retention lock licenses installed.
• If replicating an RLC-enabled mtree, then both source and destination DDs must have RLC configured or an error occurs:
  "A compliance retention-locked mtree cannot be replicated to a destination that is not enabled for compliance retention-lock."
• MTree replication contexts must have 'replication propagate-retention-lock' set to "Enabled".
• For files replicated by directory replication, the corresponding mtrees' minimum and maximum retention periods must be manually set on the destination system.

Are there any other restrictions when replicating retention locked files?

• If mtree replication is used and the destination mtree contains retention-locked files that do not exist on the source, a resync fails.
• If directory replication is used and the destination has a retention lock that is enabled but the source does not, a resync fails.

• The source mtree does not have retention lock enabled, but the destination does.
• The destination mtree does not have retention lock enabled, but the source does.

• The mtree replication context should be broken on both source and destination systems:

  # replication break mtree://<destination system>/data/col1/<mtree>

• Retention lock compliance mode should be enabled on the source system:

  # mtree retention-lock enable mode compliance mtree <mtree>

• A new mtree replication context should be created on both source and destination systems:

  # replication add source mtree://<source system>/data/col1/<mtree> destination mtree://<destination system/data/col1/<mtree>

• The newly created replication context should be resynced on the source system:

  # replication resync mtree://<destination system>/data/col1/<mtree>

Can retention-locked files be fast copied?

Are there any other restrictions in system functionality when using retention lock?

# authorization policy set security-officer {enabled | disabled}
# filesys destroy
# user reset

# mtree delete <mtree>
# mtree retention-lock reset <min-retention-period period | max-retention-period period> mtree <mtree>
# mtree retention-lock disable mtree <mtree>
# mtree retention-lock revert

• The DD is running DDOS 7.3 or later.
• The RLCE mtree to be deleted is empty (has zero files and directories).
• An administrator successfully passes security-officer authentication.

# cloud unit del <cloud unit name>

Is the system clock important on retention lock enabled systems?

• Log in to the DD.
• Check that the system clock is set correctly.
• Enable the file system:

  # filesys enable

• When prompted, enter details of a user with the role of security to allow the security clock to be reset.

Can the system clock be synchronized with Active Directory on retention lock compliance-enabled systems?

What steps can be taken if a DD using retention-locked files fills to capacity?

• There are any files that are not retention locked which can be removed.
• There are any files that are locked using governance mode which can have their locks reverted and be removed.

• The retention period is reached for some or all the files. After this point, they can be deleted and clean run as described above.
• The system is reinstalled from a USB drive, which causes loss of all data on the DD.
• More physical storage is added to the system as described above.