Metro Node: No space on the logging volume
Summary: This article talks to an issue when trying to create a distributed device, or attaching a remote mirror to a local device or trying to expand virtual volume errors are noted due to low or no space on the logging volume while creating distributed devices. This article provides a workaround to resolve this issue. ...
This article applies to
This article does not apply to
This article is not tied to any specific product.
Not all product versions are identified in this article.
Symptoms
- Attempting to create distributed devices or attaching a remote mirror to a local device failed with the following message:
One or more clusters has no room to allocate bitmap log settings for the raid-1 distributed-device. cause: Unable to attach '<device_name>' to device '<device_name>'. cause: Unable to attach mirror '<device_name>' to distributed Device '<device_name>'. cause: Firmware command error. cause: One or more clusters has no room to allocate bitmap log settings for the raid-1 distributed-device.
- Attempting to expand a virtual volume failed with the following message:
“Failed to expand virtual-volume ‘VPLEX_SLR_MFS05_F’ Unable to find a logging volume at cluster ‘cluster-1’ with enough free space to expand distributed device ‘dd_VPLEX_SLR-MFS05_F’.”
Cause
There is not enough space on the logging volumes.
Resolution
User must create a new logging volume with higher capacity and move all distributed devices from the existing logging volume to the new logging volume.
WORKAROUND STEPS:
This procedure has been broken down into three sections as follows:
Section-A: Create logging volume with higher capacity.
Section-B: Move a logging volume.
Section-C: Destroy the old logging volume.
Section-A: Create logging volume with higher capacity
1. Login into the VPLEX management server using the service account credentials and then access the VPlexcli.
2. From the VPlexcli main prompt check for unclaimed storage-volumes using the following command (approximately 10 GB of logging volume space for every 160 TB of distributed devices):
In the below example output look in the column "Use" for "unclaimed" for both clusters as follows:
NOTE: Repeat steps 2 and 3 for the other desired storage volumes user want to claim to be used for the new logging volume.
4. For those storage volumes you claimed create an extent for each using the following the command:
5. Note down the name of the extents created on top of the claimed storage-volume by using below command:
6. Next create the new logging volume on top of the extents created in step 4, using the following command:
Example output:
NOTE: Repeat all steps in Section A for cluster-2.
Section-B: Move a logging volume
The following procedure moves the bit map of distributed-devices from an existing logging volume ( c1_old_vol ) to new logging-volume ( c1_New_log_vol ).
1. Follow step #7 in Section-A to get the logging volume details:
3. Verify that the old logging-volume C1_Logging_vol, is being used by distributed-devices:
Example output:
5.Check that the new logging-volume is allocated for the distributed-devices:
Example Output:
6. Check that the original/old logging-volume is not being used by any distributed-devices:
NOTE: Repeat all steps in Section B for cluster-2.
Section-C: Destroy the old logging volume
1. To see the details of the new logging volume run the following command from the VPlexcli:
show-use-hierarchy clusters/cluster-1/system-volumes/<new_logging_volume_name>
2. Destroy the old logging-volume using the following command:
3. Destroy the extents behind the logging-volume using the following command:
NOTE: Repeat all steps in Section C for cluster-2.
WORKAROUND STEPS:
This procedure has been broken down into three sections as follows:
Section-A: Create logging volume with higher capacity.
Section-B: Move a logging volume.
Section-C: Destroy the old logging volume.
Section-A: Create logging volume with higher capacity
1. Login into the VPLEX management server using the service account credentials and then access the VPlexcli.
2. From the VPlexcli main prompt check for unclaimed storage-volumes using the following command (approximately 10 GB of logging volume space for every 160 TB of distributed devices):
ll /**/system-volumes/
In the below example output look in the column "Use" for "unclaimed" for both clusters as follows:
VPlexcli:/> ll /**/storage-volumes /clusters/cluster-1/storage-elements/storage-volumes: Name VPD83 ID Capacity Use Vendor IO Type Thin Provision Thin ---------------------------------------- ---------------------------------------- -------- --------- ------ Status ----------- Rebuild Type Capable ---------------------------------------- ---------------------------------------- -------- --------- ------ ------ ----------- ------- --------- ------- C1_Log VPD83T3:6006016013xxxxxxxxb4ae275e86e611 10G used DGC alive normal true legacy false C1_VNX1539_DR1_a8f VPD83T3:60060160c9c0xxxxxxxx9be88ac4e711 10G used DGC alive normal true legacy true C1_VNX1539_DR1_aaf VPD83T3:60060160c9c0xxxxxxxxxbe88ac4e711 10G used DGC alive normal true legacy true C1_thin_rebuild_svol VPD83T3:600601601330xxxxxxxxxx0e6986e611 10G used DGC alive normal true legacy false . . VPD83T3:60060160c9c02cxxxxxc47ef8ac4e711 VPD83T3:60060160c9c0xxxxxx0c47ef8ac4e711 10G unclaimed DGC alive traditional false legacy true VPD83T3:60060160c9c02cxxxxxaba254b99e711 VPD83T3:60060160c9c0xxxxxxxaba254b99e711 80G claimed DGC alive normal false legacy false VPD83T3:60060160c9c02cxxxxxxxbe88ac4e711 VPD83T3:60060160c9c0xxxxxxxx9be88ac4e711 10G unclaimed DGC alive traditional false legacy true VPD83T3:60060160c9c02cxxxxxxxbe88ac4e711 VPD83T3:60060160c9c0xxxxxxxxxbe88ac4e711 10G unclaimed DGC alive traditional false legacy true VPD83T3:60060160c9c02cxxxxxxx55a4a99e711 VPD83T3:60060160c9cxxxxxxxxxx55a4a99e711 80G meta-data DGC alive traditional false legacy false3. From each cluster where user want to claim storage-volumes, change directory (cd) to the storage-volume context level and claim the desired storage-volumes (approximately 10 GB of logging volume space for every 160 TB of distributed devices), using the following command:
storage-volume claim --name <storage volume name> -d <storage-volume VPD ID>Example Output:
VPlexcli:/clusters/cluster-1/storage-elements/storage-volumes> storage-volume claim --name Test_LogA -d VPD83T3:600601601330270xxxxa4e5be611 Claimed 1 of 1 storage-volumes.
NOTE: Repeat steps 2 and 3 for the other desired storage volumes user want to claim to be used for the new logging volume.
4. For those storage volumes you claimed create an extent for each using the following the command:
extent create --storage-volumes <name of storage volume>Example Output:
VPlexcli:/clusters/cluster-1/storage-elements/storage-volumes> extent create --storage-volumes Test_LogANOTE: Repeat step 4 for the remaining claimed storage volumes.
5. Note down the name of the extents created on top of the claimed storage-volume by using below command:
VPlexcli:/clusters/cluster-1/storage-elements/extents> llExample Output:
VPlexcli:/clusters/cluster-1/storage-elements/extents> ll Name StorageVolume Capacity Use ------------------------------------------------- ---------------------------------------- -------- ------- extent_Test_LogA Test_LogA 20G used extent_Test_LogB Test_LogB 20G used . . . </truncated>
6. Next create the new logging volume on top of the extents created in step 4, using the following command:
logging-volume create -n <name of logging volume> -e <extent1, extent2>
Example output:
VPlexcli:/clusters/cluster-1/storage-elements/storage-volumes> logging-volume create -n c1_New_log -e Test_LogA, Test_LogB7. List the newly created logging volume using the following command:
ll /clusters/cluster-1/system-volumes/Example Output:
VPlexcli:/> ll /clusters/cluster-1/system-volumes/ /clusters/cluster-1/system-volumes: Name Volume Type Operational Health Active Ready Geometry Component Block Block Capacity Slots ------------------------------- -------------- Status State ------ ----- -------- Count Count Size -------- ----- ------------------------------- -------------- ----------- ------ ------ ----- -------- --------- -------- ----- -------- ----- C1__Logging_vol logging-volume ok ok - - raid-1 1 2621440 4K 10G - C1_New_Log_vol logging-volume ok ok - - raid-1 1 2621440 4K 10G - C1_Meta meta-volume ok ok true true raid-1 2 20971264 4K 80G 64000 C1_Meta_backup_2018Apr10_120032 meta-volume ok ok false true raid-1 1 20971264 4K 80G 64000 C1_Meta_backup_2018Apr11_120018 meta-volume ok ok false true raid-1 1 20971264 4K 80G 64000
NOTE: Repeat all steps in Section A for cluster-2.
Section-B: Move a logging volume
The following procedure moves the bit map of distributed-devices from an existing logging volume ( c1_old_vol ) to new logging-volume ( c1_New_log_vol ).
1. Follow step #7 in Section-A to get the logging volume details:
VPlexcli:/clusters/cluster-1/system-volumes> ll2. Verify that the new logging-volume c1_New_Log_vol, is not being used by distributed-devices by looking to ensure under the column 'Use' it reads "free':
VPlexcli:/clusters/cluster-1/system-volumes/c1_New_log_vol/segments> ll Name Starting Block Block Count Use ------ -------------- ----------- ---- free-0 0 2623200 free
3. Verify that the old logging-volume C1_Logging_vol, is being used by distributed-devices:
VPlexcli:/clusters/cluster-1/system-volumes/C1_Logging_vol/segments> llExample output:
VPlexcli:/clusters/cluster-1/system-volumes/C1_Logging_vol/segments> ll /clusters/cluster-1/system-volumes/C1_Logging_vol/segments: Name Starting Block Use ----------------------------------------------------- Block Count ------------------------------------------- ----------------------------------------------------- -------- ------- ------------------------------------------- allocated-device_C1_VNXXX12_DR1_XXf_1 0 80 allocated for device_C1_VNXXX12_DR1_XXf_1 allocated-device_C2_VNXXX12_DR1_X2X_12017Nov20_165905 0 80 allocated for device_C2_VNXXX12_DR1_X2X_12017Nov20_165905 allocated-device_local_source_12018Mar05_063808 0 80 allocated for device_local_source_12018Mar05_063808 allocated-device_remote_target_1 0 80 allocated for device_remote_target_1 free-3204. Choose the new logging volume for all the distributed-devices in the system using the following command:
ds dd set-log --logging-volumes <name of new logging volume> --distributed-devices *
Example output:
VPlexcli:/distributed-storage/distributed-devices> ds dd set-log --logging-volumes c1_New_Log_vol --distributed-devices *
5.Check that the new logging-volume is allocated for the distributed-devices:
VPlexcli:/clusters/cluster-1/system-volumes/c1_New_Log_vol/segments> ll
Example Output:
VPlexcli:/clusters/cluster-1/system-volumes/c1_New_Log_vol/segments> ll
/clusters/cluster-1/system-volumes/C1_Logging_vol/segments:
Name Starting Block Use
----------------------------------------------------- Block Count -------------------------------------------
----------------------------------------------------- -------- ------- -------------------------------------------
allocated-device_C1_VNXXX12_DR1_XXf_1 0 80 allocated for device_C1_VNXXX12_DR1_XXf_1
allocated-device_C2_VNXXX12_DR1_X2X_12017Nov20_165905 0 80 allocated for device_C2_VNXXX12_DR1_X2X_12017Nov20_165905
allocated-device_local_source_12018Mar05_063808 0 80 allocated for device_local_source_12018Mar05_063808
allocated-device_remote_target_1 0 80 allocated for device_remote_target_1
free-320
6. Check that the original/old logging-volume is not being used by any distributed-devices:
VPlexcli:/clusters/cluster-1/system-volumes/C1_Logging_vol/segments> ll
Name Starting Block Block Count Use
------ -------------- ----------- ----
free-0 0 2623200 free
NOTE: Repeat all steps in Section B for cluster-2.
Section-C: Destroy the old logging volume
1. To see the details of the new logging volume run the following command from the VPlexcli:
show-use-hierarchy /clusters/cluster-1/storage-elements/extents/<name of extent of logging volume>OR
show-use-hierarchy clusters/cluster-1/system-volumes/<new_logging_volume_name>
show-use-hierarchy clusters/cluster-1/system-volumes/<new_logging_volume_name>Example output using the extent name:
VPlexcli:/> show-use-hierarchy /clusters/cluster-1/storage-elements/extents/extent_C1_Log_1
logging-volume: c1_New_Log_vol (20G, raid-0, cluster-1)
extent: C1_New_Log_1 (10G)
storage-volume: C1_New_Log(10G)
logical-unit: VPD83T3:600009700002987014xxxx3030354144
storage-array: EMC-SYMMETRIX-123456789
extent: C1_New_Log_2(10G)
storage-volume: New_Log2(10G)
logical-unit: VPD83T3:600009700002987xxxx3533030354145
storage-array: EMC-SYMMETRIX-987654321
2. Destroy the old logging-volume using the following command:
logging-volume destroy --logging-volume <name of old logging volume>Example output:
VPlexcli:/clusters/cluster-1/system-volumes/C1_Log_vol> logging-volume destroy --logging-volume C1_Log_vol Context '/clusters/cluster-1/system-volumes/C1_Log_vol' has been removed.
3. Destroy the extents behind the logging-volume using the following command:
extent destroy --extents extent <name of extent>Example output:
VPlexcli:/clusters/cluster-1/storage-elements/extents/C1_Log_1> extent destroy --extents extent_C1_Log_1 WARNING: The following items will be destroyed: Context ----------------------------------------------------------------------------------- /clusters/cluster-1/storage-elements/extents/extent_C1_Log_1 Do you wish to proceed? (Yes/No) Yes Extent extent_C1_Log_1' is claimed and may be in use, do you wish to destroy it anyway? (Yes/No) Yes Context '/clusters/cluster-1/storage-elements/extents/extent_C1_Log_1' has been removed. Destroyed 1 out of 1 targeted extents.
NOTE: Repeat all steps in Section C for cluster-2.
Affected Products
metro nodeProducts
metro node mn-114, metro node mn-215Article Properties
Article Number: 000224388
Article Type: Solution
Last Modified: 29 Apr 2024
Version: 1
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.