unusable disk space

Question

we have around 4TB free space on disks from disk group. But, device (17gb raid 5 3+1) creation failed with error
"The requested device(s) cannot be created because the mirrors cannot be distributed according to protection requirements¿

I would like to know that anyway the disk space can be utilized ? thanks

MarcT2 · Accepted Answer

What you're seeing there is a single large gap at the end of the disk (ie freespace).

For comparison, check out this swiss cheese layout from one of our lab boxen:

# symdisk -sid xxx show 1a:c0 -gaps
 
Symmetrix ID                 : 000
  Director                   : DF-1A
  Interface                  : C
  Target ID                  : 0
  Disk Group Number          : 0
 
  Vendor ID                  : SEAGATE
  Product ID                 : SX3300007FC
  Product Revision           : C300LPF
  Serial ID                  : 3KR27VGR
 
  Disk Blocks                : 585937499
  Block Size                 : 512
  Actual Disk Blocks         : 585937499
  Total Disk Capacity (MB)   : 286102
  Free Disk Capacity (MB)    : 231429
  Actual Disk Capacity (MB)  : 286102
  Hot Spare                  : False
 
  Failed Disk                : False
 
 
  Hypers (20):
  {
  #   Vol  Emulation        Dev  Type          Mirror Status         Cap(MB)
  --- ---- ---------------- ---- ------------- ------ -------------- --------
    1    1 VAULT_DEVICE     0000 Data            1    N/A                5120
    2    2 FBA              0300 RAID-5          1    Ready              2878
    -    - N/A                 - GAP             -    N/A                   3
    3    3 FBA              0317 RAID-5          1    Ready              2878
    -    - N/A                 - GAP             -    N/A                   3
    4    4 FBA              0362 Data            2    Ready              8702
    -    - N/A                 - GAP             -    N/A                   6
    5    5 FBA              02F0 Data            1    Ready              8634
    -    - N/A                 - GAP             -    N/A                   6
    6    6 FBA              0338 Data            1    Ready              8634
    -    - N/A                 - GAP             -    N/A                   6
    7    7 CELERRA_FBA      00C3 RAID-5          1    Ready                95
    -    - N/A                 - GAP             -    N/A                   0
    8    8 FBA              010B RAID-5          2    Ready              1233
    -    - N/A                 - GAP             -    N/A                   1
    9    9 FBA              0116 RAID-5          2    Ready              1233
    -    - N/A                 - GAP             -    N/A                   1
   10   10 FBA              0121 RAID-5          2    Ready              1233
    -    - N/A                 - GAP             -    N/A                   1
   11   11 FBA              012C RAID-5          2    Ready              1233
    -    - N/A                 - GAP             -    N/A                   1
   12   12 FBA              0137 RAID-5          2    Ready              1233
    -    - N/A                 - GAP             -    N/A                   1
   13   13 FBA              0142 RAID-5          2    Ready              1233
    -    - N/A                 - GAP             -    N/A                   1
   14   14 FBA              0173 RAID-5          1    Ready              2878
    -    - N/A                 - GAP             -    N/A                3297
   15   15 FBA              0184 RAID-5          2    Ready              1233
    -    - N/A                 - GAP             -    N/A                   1
   16   16 FBA              018F RAID-5          2    Ready              1233
    -    - N/A                 - GAP             -    N/A                   1
   17   17 FBA              019A RAID-5          2    Ready              1233
    -    - N/A                 - GAP             -    N/A                   1
   18   18 FBA              01A5 RAID-5          2    Ready              1233
    -    - N/A                 - GAP             -    N/A                   1
   19   19 FBA              01B0 RAID-5          2    Ready              1233
    -    - N/A                 - GAP             -    N/A                   1
   20   20 FBA              01BB RAID-5          2    Ready              1233
    -    - N/A                 - GAP             -    N/A              228086

You can use "symdisk list -gaps" rather than "symdisk show " to see the whole box.

To review the number of different gaps you have, try the following (Solaris):

symdisk list -gaps  -v -sid xxx | grep "Gaps" | sort -u

On a box where all the volumes are contiguous, you'll see:

Gaps   (1):

on our horific lab box, we have:

  Gaps   (17):
  Gaps   (18):
  Gaps   (19):
  Gaps   (20):
  Gaps   (21):
  Gaps   (22):
  Gaps   (23):
  Gaps   (49):
  Gaps   (9):

M

Message was edited by:
MarcT - fix spelling

bodnarg · Answer

You need to figure-out how much space you really have on each disk. 4TB sounds like a lot of space, but if the space is really unbalanced or you just have a lot of devices with very small gaps of space might be hard to use.

The only way to really determine what you can do is to take a look at the space on drives. You can use symdisk -sid xxx list which shows each drive with a free space column. Then take a look at the individual drives to see how that "free space" sits on the drive:

symdisk -sid xxx show 16B:d11 -gaps

The answer may be you have lots of gaps in your data leaving small amount of space or perhaps your 4TB number is skewed by having a few large drives that are not used which don't provide enough redundancy for a Raid-set.

Without more data hard to give you an exact answer.

wishtobeMrEMC · Answer

bodnarg, as per the following output, the gap is 23265 MB. it means the accumlated gap between hypers OR just one single large gap?

./symdisk -sid xxx show -v 1A:C2 -gaps

Symmetrix ID : xxx
Director : DF-1A
Interface : C
Target ID : 2
Disk Group Number : 1

Vendor ID : SEAGATE
Product ID : SX3146854FC
Product Revision : C146X15
Serial ID : 3KN21S7R

Disk Blocks : 286749475
Block Size : 512
Actual Disk Blocks : 286749475
Total Disk Capacity (MB) : 140014
Free Disk Capacity (MB) : 23265
Actual Disk Capacity (MB) : 140014
Hot Spare : False

Failed Disk : False

Hypers (34):
{
# Vol Emulation Dev Type Mirror Status Cap(MB)
--- ---- ---------------- ---- ------------- ------ -------------- --------
1 38 VAULT_DEVICE 0002 Data 1 N/A 5120
2 39 FBA 00A7 RAID-5 1 Ready 5761
...
...
...
...
34 71 FBA 0F0E RAID-5 1 Ready 1441
- - N/A - GAP - N/A 23265

bodnarg · Answer

That is showing that you have a single gap of 23265 GB at the end of your drive. If you had multiple gaps they would each show independently.

The question now becomes how many of these do you have and are they spread across enough disks/DAs to allow for the creation of Raid-5 groups. Also are they on the same kinds of drives (FC, etc.)?

I have never seen an explicit list of rules that symconfigure enforces for creating Raid-5 devices but I know the main ones (besides obviously having enough space to begin with) is ensuring that each of the partitions that make-up the Raid-5 volume are spread-out redundantly to avoid a single drive or loop failure from putting you at risk of the dreaded double-drive failure.

wishtobeMrEMC · Answer

thanks bodnarg .. let me figure out that whether the space is spread across reduntant disks for raid 5 creation .. for 17gb raid 5 3+1 device, it is required the free space of (17gb/4) + 33% on each of 4 disks [reduntant disks], correct? roughly around 6gb free space on each disk, right?

bodnarg · Answer

Yes. I'd suggest you round-up to 34% or even 35% as it does not end-up being a bit above 33% with (I think) some overhead.

wishtobeMrEMC · Answer

As per below output, almost there is atleast a gap either at the end of the disk or between hypers. But, as I have checked thoroughly, majority of gaps are large enough to accommadate 17GB Raid5 3+1 hyper. only few are lesser than required space. I had tried to create a single sym device (17gb raid5 3+1) but it fails though there is a lot of large gaps even on reduntant disks. Is there anyway i can let enginuity to choose the disks that safisfies the criteria to create sym dev.

I really do not want to waste 3.7TB disk space on DMX3. bcoz i always argue with DBAs to justify the storage space requirements and end up allocating only 50% of what they have requested.

bodnarg · Answer

I wonder if there is a different problem other than having good candidate drives for the allocation.

What level of microcode are you running on your DMX and are you currently using Raid on the frame? There is a limitation on older code levels (can't remember at what version this changes) where you could only run either Raid 7+1 or Raid 3+1 devices. Also at older code levels you had to have a setting enabled to even allow you to create Raid-5 devices.

Other than an oddball case like that I unfortunately can't give you the exact rules for how Raid-5 devices are selected - I've asked trust me

. I am also not aware of a method to select the exact device. Other than selecting the disk_group I don't think you can target specific drives. You would need to consult with your local CE to get them to take a peek at the configuration and figure-out why you can not use that space.

MarcT2 · Answer

Can you post the symconfigure script you're using to create the RAID5 devices?

wishtobeMrEMC · Answer

bodnarg, info as given below
Product Model : DMX3-24
Symmetrix ID : xxx

Microcode Version (Number) : 5771 (168B0000)
Microcode Date : 10.26.2007
Microcode Patch Date : 10.26.2007
Microcode Patch Level : 104
Raid-5 Configuration : RAID-5 (3+1)

MarcT, i used SMC to create a device, not symcli.

bodnarg · Answer

Well that answers that - you appear to have Raid-5 3+1 as your current setting, but you did not answer if you have any other Raid-5 devices created in this environment?

I did not think this was the case at 71 code, but there used to be something that the CE had to enable on the Service Processor to allow Raid devices to be created which you could not see. This is probably a real long shot and if you have other Raid-5 devices created not the problem.

Are you selecting anything odd in the SMC creation window (are you selecting a disk group maybe without the right storage combination, or flipping away from FBA for the device creation?)

You may want to talk to your CE and get him/her to help dig into this one unless someone else has some other suggestions.

MarcT2 · Answer

SMC actually generates a symconfigure script when you create devices. You should be able to see this in the Solutions Enabler log files, which are located in:

/var/symapi/log - for Unix
Program Files\Emc\SYMAPI\log - for Windows

I doubt SMC would get it wrong, but it would be worth reviewing the script in case any incorrect options were selected (ie the wrong disk group). You also get better error messages in the logfile.

Cut & paste the relevant section here if you wish.

xe2sdc · Answer

RAID1 and RAID5 have different requirements. Maybe your drives are "not good" to create RAID5 volumes while you may be able to create RAID1 (mirror) volumes.
However RAID1 volumes require more space on each drive and "wastes" a lot more space in a different way (you use only 50% of allocated space, obviously).

If you need big volumes and don't have big gaps you can also considere using metavolumes.. Go create small mirrored volumes and form them in bigger metas.

wishtobeMrEMC · Answer

Stefano, i can create smaller devices and form meta to make use of gaps.
not really very sure what is the optimal size of symdev. in my prev. comp, we used 5gb raid1 and 10gb and 30gb raid 5 3+1 on dmx3. but in present env, we use 16gb as the EMC suggested optimal size on DMX3. i think the disk type is same.

wishtobeMrEMC · Answer

Marc, below listed is the symapi log that was generated for the raid 5 3+1 sym dev creation failure

create dev count=3, size=18414 cyl, emulation=FBA, config=RAID-5, mvs_ssid=0, disk_group=1;
Submitting configuration changes..........................Failed.
EMC:SMC cfgLogSymmwinError cfgCongtrolSubmit call to silsw_DoSymmWinService for SWS_CONFIG_SUBMIT_CHANGE failed with code: SWRS_INVALID_SERVICE_DATA
cfgControlSubmit Local config server msg:
failed to distribute volume 0691:
looking for space in an existing physical raid group: none of the 20 disks with raid volumes have the whole physical raid group available
Closing configuration change request......................Closed.
Terminating session with configuration server.............Done.

Symmetrix

unusable disk space

Was this post helpful?