VNXe: During replication quotas were exceed causing issues with replcation between VNXe and Celerra..(User Correctable)

>> Savvol on Celerra exceeds exceeds 20% (default)

The properties of both the file-systems are same but still we are getting the error.
The state failed with error: 13421840574: Disk space : quota exceeded.


Checked in the replication log and found the below instances:

cemtracer_Replication.log shows as below:

23 Aug 2016 23:39:17  - [ReplicationServices] DEBUG - {0:43864:199966906}[22730|16934|0x8694e58][convertCCMDIdToCEMCode @ /storage01/svn_wc/NEOMAIN-CP/components/providers/osls/ReplicationServices/src/ReplicationServicesCommonUtils.cpp:225]  : got a ccmdMsg briefDescription: Operation task id=29734 on NADC1NS960_CS0 at first internal checkpoint create state failed with error: 13421840574: Disk space : quota exceeded.

23 Aug 2016 23:39:17  - [ReplicationServices] DEBUG - {0:43864:200134416}[22730|16934|0x8694e58][convertCCMDIdToCEMCode @ /storage01/svn_wc/NEOMAIN-CP/components/providers/osls/ReplicationServices/src/ReplicationServicesCommonUtils.cpp:230]  : got a ccmdMsg fullDescription: The specified operation failed in the current state. If there is a failure during rollback, an event is posted to /nas/log/sys_log, and a UI alert is also generated on the specified Celerra Control Station. Check this log for more details.

23 Aug 2016 23:39:17  - [ReplicationServices] ERROR - {0:43864:200594488}[22730|16934|0x8694e58][handleNasReplicationSession @ /storage01/svn_wc/NEOMAIN-CP/components/providers/osls/ReplicationServices/src/CEMCelerraReplicationManager_ReplicationSession.cpp:2190] Creation of replication session failed.

23 Aug 2016 23:39:17  - [ReplicationServices] ERROR - {0:43864:205782296}[22730|16934|0x8694e58][CreateReplicationSession @ /storage01/svn_wc/NEOMAIN-CP/components/providers/osls/ReplicationServices/src/C4CB_ReplicationService.cpp:739] Failed to create the session




The error states that the internal process is unable to create replication session due to the quota exceed error on Celerra as we are unable to create the first internal root replication checkpoint on Celerra.


Also, found the similar instances on the Celerra end:

[nasadmin@NADC1NS960_CS0 log]$ nas_logviewer /nas/log/sys_log | grep -i exceed
Aug 22 12:57:12 2016:CS_CORE:SVFS:INFO:4:::::The checkpoint volume allocation quota has been exceeded. Current total_ckpt_usage is: 23%.
Aug 22 12:57:13 2016:CS_PLATFORM:SVFS:ERROR:21:::::Slot3:1471884997: /nas/sbin/rootnas_fs -x root_rep_ckpt_3855_891858_2 QOSsize=20000M Error 2238: Disk space : quota exceeded.
Aug 22 23:57:55 2016:CS_CORE:SVFS:INFO:4:::::The checkpoint volume allocation quota has been exceeded. Current total_ckpt_usage is: 23%.
Aug 23 18:40:00 2016:CS_CORE:SVFS:INFO:4:::::The checkpoint volume allocation quota has been exceeded. Current total_ckpt_usage is: 23%.
Aug 23 19:12:17 2016:CS_CORE:SVFS:INFO:4:::::The checkpoint volume allocation quota has been exceeded. Current total_ckpt_usage is: 23%.
Aug 23 19:39:09 2016:CS_CORE:SVFS:INFO:4:::::The checkpoint volume allocation quota has been exceeded. Current total_ckpt_usage is: 23%.

•  In celerra, by default the savvol space is used as 20% of the total space available on the system. This calculation includes SavVols used by SnapSure and other applications, such as the Celerra Replicator. Before using SnapSure, ensure that your system has enough space for the SavVols for each PFS of which you plan to create checkpoints.
• Once the savvol exceeds exceeds 20% (default), it doesn't allow the VNXe to create the first replication checkpoint on the destination Celerra and hence the creation of replication session fails.
• If the 20% value is reached, the following message appears in the command output:

Error 2238: Disk space: quota exceeded

>> To resolve the issue, first needs to check what is consuming so much space in the savvol.
- You can delete the old checkpoints with customer permission.
- You can remove some of the stale checkpoints which might be consuming the savvol space.
- If customer do not want to delete any of the old checkpoints then we would require to increase the default savvol param on the Celerra which was set to 20 and change it to 30 (Values can be 20 through 99).

Note: This should be the last resort. You need to consult with the VNX coach/L2 to see if this is an option here.

Once verified, need to follow the below steps:

- Open the /nas/site/nas_param file

- Find the line starting with ckpt in the below format:

 ckpt:10:100:20:

The last parameter, 20, is the default value for the savvol to be increased.
By increasing the value to 30, we are increasing the savvol % of the total size.

- Save and close the file.


- Cat the file.
The output should be something like below:

[nasadmin@cel1 nasadmin]$ cat /nas/site/nas_param
primary_if:el30:xxx.xxx.x.0:255.255.255.0:xxx.xxx.x.xxx:el30:
backup_if:el31:xxx.xxx.x.0:255.255.255.0:xxx.xxx.x.xxx:el31:
ckpt:10:100:30:


>> Once done, try creating the replication session from VNXe.
It should now work.