Symdev ID range allocation?

Hello roysubs,

Thanks for contacing EMC Community,

That is a great question, First of all I would recommend reading EMC KB article #56586 https://support.emc.com/kb/56586

It will answer most of your questions. For the other questions you have:

1- yes, 65535 is the max device count in SYMMETRIX Arrays. This number includes meta members, valut devices, SFS and all other device types like VDEV, BCVs, GKs, TDATs etc...

2- There is no way to push the SYMM dev into a specifc Dev ID when you create it. SYMMWIN will determine that (It is mentioned in the KB article) but i can tell you if you deleted a device with symdev ID 00A7 and you created a device with exactly same configuration, It will be created with the same Dev ID but again SYMMWIN will take care of this.

3- technically speaking, We have never reached this number before with any of our customers. There are too many factors that should be taken into consideration before reaching this number.

Hope that helps

Mohammed Salem

Mohammed, that's excellent KB, thanks !

roysubs

I suspect in your case you had a gap in your volume ID numbering starting at 057D and when symconfigure create dev was run, the number of devices created was smaller than the size of the gap. The gap most probably was creating by deleting devices.

You can have some control of your volume IDs if you find these gaps in numbering (as the KB explains) and fill them up with temporary devices, for example thin gatekeepers ( symconfigure -cmd "create gatekeeper count=32 , type=thin, emulation=fba ; "  )

In my environment use two sizes, 50G and 15G, and all 50G devices start from  1000, and all 15G start from 2000 (all in hex)

When the VMAX was installed, I had the VAULT, SFS, VCB and the TDATs taking up until about D00.  I created (0x1000 - 0x0D00 = 0x0300 ) or 768 thin gatekeepers. Then I created 4096 (0x1000) x  50G TDEVs and my 15G TDEVs started from 2000.

If we need to add more drives in the system, I will calculate how many datadevs I will need, and delete that many of the thin gatekeepers from the 0x0D00 - 0x0FFF range

This greatly simplified my provisioning scripts.

Many will argue that this waste of device IDs for gatekeepers and TDEVs that I may never use is wasting VMAX cache, but with system with half TB cache I can live in few megabytes wasted.

Thanks @Mohammed / @bhalilov

The EMC article you gave me was very useful (EMC KB article #56586 https://support.emc.com/kb/56586 )

ok, got it, 0000:FFFF (65535) is the hard maximum symdev positions for meta-heads, meta members, vault devices, SFS, VDEV, BCVs, GKs, TDATs etc. Thanks.

> There is no way to push the SYMM dev into a specifc Dev ID when you create it.

ok, perfect, and as you say SYMMWIN determines this (it's quite simple as I've noticed, if there is a gap and the create is going to create a set of devices the same size or smaller, then it will use that gap etc).

> technically speaking, We have never reached [65535] before with any of our customers. There are too many factors that should be taken into consideration before reaching this number.

ok, I can see how that would be highly unlikely

Thanks for this. It's cleared up a lot about how Symmetrix allocates for me, I see that I was getting some points wrong originally. The entire map of one of our arrays is below using the -all switch on a symdev list

   symdev -sid 123 list -all

First come the VAULT devices  (160x 9216 = 1440 GB)

Then are the SFS (FS) devices (  4x 8192 =   32 GB)

Then the single ACLX device   (  1x    3 =    3 GB)

Then some GKs                 ( 64x    3 =  182 GB)

Then the datadev (DT) devices of 4 fixed sizes, some RAID-5, some RAID-6

Then 1937x TDEV devices of various sizes

        Device Name           Directors                  Device            

--------------------------- ------------- -------------------------------------

                                                                           Cap

Sym  Physical               SA  : P DA :IT  Config        Attribute    Sts   (MB)

--------------------------- ------------- -------------------------------------

0000:009F (160x  VAULT)     ???:? 05A:C0  VAULT         N/A          N/A   9216

00A0:00A3 (   4x (FS) )     ???:? 07A:C18 2-Way Mir     N/A     (FS) RW    8192

00A4      (   1x ACLX )     10H:0 09A:C18 2-Way Mir     N/Grp'd ACLX WD       3

00A5:00E4 (  64x GKs  )     ***:* 10D:C18 2-Way Mir     N/Grp'd      RW       3

00E5:0224 ( 320x (DT) )     ???:? 11A:C0  RAID-5        N/A     (DT) RW   99607

0225:02A4 ( 128x (DT) )     ???:? 05A:C6  RAID-5        N/A     (DT) RW   70438

02A5:057C ( 738x (DT) )     ???:? 05A:C8  RAID-5        N/A     (DT) RW  103061

057D:0584 (   8x TDEV )     ***:*  NA:NA  TDEV          N/Grp'd  (M) RW  204803

^^^^ The datadev gap was created here.

0585:07F0 ( 620x (DT) )     ???:? 07A:C1A RAID-6        N/A     (DT) RW  245001

07F1:0F87 (1927x TDEV )     ***:*  NA:NA  TDEV          N/Grp'd      RW   various sizes

@bhalilov, yep, you were on the right track. In fact, 8 datadevs had been removed, which turned out to be from 057D upwards. These were being removed to be used for testing thick, but before they were rebuilt as thick, a meta was created which took those symdev ID's, so now, when I run a normal symdev list (which excludes VALUT / (FS) / (DT) devices),  I see this TDEV 8-way meta at 057D:0584 so I was confused as to why that had happened.

   symdev -sid 123 list 

I have a few other follow up questions:

- Are there always 160x VAULT devices, or does this number vary from system to system, and is this a static set always using 0000:009F on every system?

- Are there always 4x (FS) (these are SFS right?) devices, or do these vary, and if not, are they always located at 00A0:00A3 ?

- Is the ACLX always at 00A4 on every VMAX ?

- What does symgate -sid 123 define dev 00AD actually do in practice ? I understand that it marks the device with the "GK" Attribute, but does this have any significance in real-world terms (by which I mean, a device can be used as a gatekeeper even if it is not defined with the "GK" Attribute, so does this have an actual function, or is it just a flag with no meaningful function) ?

Above the VAULT, SFS, ACLX, GK, it's clear from what you've said that symdev id's are used on a first come first served basis for all TDATs, TDEVs (single devices, meta-heads, meta members), VDEV, BCVs, GKs, etc.

Sorry, I forgot to answer the symgate question.

Symgate should be rarely used actually, I will use it in situations where only i want to force define a device as a GK if my system cannot detect the device as GK (In VMAX this is rare to see, this command was primarily used in DMX and old releases of SE v5.x). To check the GK presented first run the below commands from your host:

syminq

sympd -sid xxx

stordaemon action storapid –cmd show –gk_pdevs

If all of the above fails to detect the device as GK then you will need to define it using symgate command and hence symgate list will return all defined devices.

SE will automatically detect the devices as GKs if their size is less than 10 MB. in VMAX running 5875 or later a new command was introduced to create gatekeepers automatically without defineing space requirements. in 5876 you have the option to create GK as TDEVs and use them without being bound to a thin pool.

Hope that helps

Mohammed Salem

You should have only one ACLX device and 4 SFS devices per SYMM VMAX array. ACLX and SFS should be configured as RAID-1, The should reside on any disk type but usually we create them on FC disks. Migrating them later to other disks is very easy but should be done by EMC personnel to go onsite and configure the migration. symmigrate will not work with these type of devices.

VAULT devices are different they are the first devices created on the SYMM box. They need one vault hyper per disk. The vault disks are the first five slots in each loop. Like you said the number will vary from an array to another depends on the modela nd number of disks and loops

For the dev numbering, VAULT drives should start with Dev ID 0000 while SFS and ACLX is not necessarily have same numbers in each box especially that some customers may migrate them later as I stated earlier.

Hope that helps

Mohammed Salem

Hi Mohammed, thanks, just realised that I know that we can have GKs as TDEVs in 5876, but:"in VMAX running 5875 or later a new command was introduced to create gatekeepers automatically without defineing space requirements". What is this new command ?

For  what it's worth, We talked here about symgate command, in the latest release notes for SE v7.6.2 page 5 and 6. It was mentioned that SE v7.6.2 will be the LAST version to support it along with other commands. Please take  a moment to review these features as it will be NO longer supported in future releases.

Hope that helps

Mohammed Salem

Mohammed,

Just to clarify, Solutions Enabler 7.6.x will be the last SE family to support those features and commands. So future 7.6.x versions will support those features but 8.x will not.

That's correct AranH. I am not sure if another version will be released for 7.6.The last roadmap i had was not including this.  EMC Engineering may release one last version but most probably this will be after SE v8.x official GA.

Thanks for following up

Hope that helps

Mohammed Salem

Excellent, thanks very much, useful information, I will avoid using symgate in any way then if it is deprecated.