emcmagic
3 Argentum

To show the layout of what engines and dir where each different type of disks is located?

Jump to solution

I have VMX-20 and with 4 engines. Further, it has EFD, FC-15K, FC-10K, and SATA drives, with FAST VP configuration. Can you please share with me the rule/best practice of how these drives should be spread, and to what engines/directors, and what commands to show me the layout on what drives are located in what engines / dirs.

Thanks!

Labels (1)
0 Kudos
1 Solution

Accepted Solutions
rohang1
3 Silver

Re: To show the layout of what engines and dir where each different type of disks is located?

Jump to solution

Hello emcmagic.

Ok. let me do my bit to try and get this across to you.

If you look at the #symdisk list output you'd see directors labelled 5 to 12 .. which is a total of 8 back-end directors spread across 4 engines, for your 4 engine VMAX.

Drives connecting to directors 5 and 6 are located on an engine,

Drives connecting to directors 7 and 8 are located on an engine,

Drives connecting to directors 9 and 10 are located on an engine,

Drives connecting to directors 11 and 12 are located on an engine.

A small correction to what I wrote on the previous reply :

For example - 5A is a (back-end) director .. 5A:C is cabled to 1 DAE (the same DAE is connected to director 6A:C (using LCC B -- remember that the drives are dual ported for backend redundancy) -- directors 5 and 6 are located within the same engine) and 5A:C6 is your EFD within the DAE. [Note: If it were a DMX, the DAE would have cables coming in from director ports 5A:C and 12A:C]

If you look at the figure below ..

Each engine with it's 2 back-end director ports connect to 8 unique DAEs (which is why the name "octant") .. for example - back-end directors 5 and 6 on Engine 3 will connect to 8 DAEs on storage bay 1B (Octant 3 in Orange color) using direct connections. The 8 connections to the 8 unique DAEs (from LCC A) will be 5A:C, 5A:D, 5B:C, 5B:D, 5C:C, 5C:D, 5D:C, 5D:D. The same 8 DAEs will have a connection from the redundant backend directors ports of director 6 from LCC B. (6A:C, 6A:D, 6B:C, 6B:D, 6C:C, 6C:D, 6D:C, 6D:D)

Do take a look at the complete output from a #symdisk list (not just disk group 3 which is your EFDs) ..

I'll paste a DMX back-end architecture to help you visualize this better ..

So if you look at the back-end [directors labelled DF01 and DF16], a DAE is cabled to dir port 1B:C and to a redundant back-end port 16B:C.

So these 2 director pairs can connect to 8 unique DAE's. (1 and 16 work in pairs, 2 and 15 work in pairs and so on .. for a DMX)

Similarly for a VMAX, you have directors within an engine working in pairs, connecting to 8 unique DAEs using 8 director ports. (1 and 2 work in pairs.. 3 and 4 work in pairs .. so on).

Hope this helps.

thanks.

View solution in original post

0 Kudos
11 Replies
seancummins
3 Argentum

Re: To show the layout of what engines and dir where each different type of disks is located?

Jump to solution

Ideally the each disk type should be evenly distributed across all directors/engines. You can see the disks behind each director with the "symdisk list" command.

Sent from my iPhone

0 Kudos
Quincy561
4 Beryllium

Re: To show the layout of what engines and dir where each different type of disks is located?

Jump to solution

The ideal configuration will have equal numbers of each drive type on each DA director.

The most important drive type to have balanced are the EFDs, since they do the most IOPs per drive, and therefore the most IOPs on each DA CPU.

0 Kudos
rohang1
3 Silver

Re: To show the layout of what engines and dir where each different type of disks is located?

Jump to solution

Hello emcmagic,

As sean said, balance all drives across all available DA's where possible. An ideal configuration would be multiple of 8 drives per engine-of the same type/speed (not including spares). This is important especially for EFDs, as Quincy mentioned.

The recommendation is to implement 8 EFDs per VMAX engine, or multiples of 8, across all VMAX engines (this does not includes spare EFDs). The use of the same sized EFDs within a single pool is critical to ensure even loading of EFDs and DA's. A balanced configuration of FC and SATA disks is recommended but is less critical, as each FC and SATA can only place a relatively small load on a DA, whereas a single EFD can place a significant load on the DA.

Say you have a 4 engine VMAX, which has 32 DA's but only 8 EFDs, with the 8 EFDs spread across just 1 engine as an example.

FAST VP will promote the most active extents from all of the FC and SATA disks to the 8 EFDs.The 8 EFDs in the 4 engine VMAX, which might represent less than 1% of the total useable capacity, will have a very good chance of being saturated with I/O and FAST VP will be busy promoting and demoting data. The DA's to which the 8 EFDs are configured will have a high utilization, while the remaining 24 DA's will have a low utilization, meaning that the response times will be impacted for the 8 EFDs and the FC and SATA disks attached to the same DA's as the EFDs.

Another point to note is that 2 x 200 GB EFDs are better than 1 x 400 GB EFD. The former will provide double the potential performance capability of one 400 GB EFD because both EFDs are able to support the same IOPS, so the 200 GB EFD is able to support double the I/O density, where I/O density is measured in IOPS/GB.

hope this helps.

thanks ..

emcmagic
3 Argentum

Re: To show the layout of what engines and dir where each different type of disks is located?

Jump to solution

Thank everyone here for your messages. They help a lot.

The following is excerpted from " symdisk list -by_diskgroup -sid 123", I now concentrate on EFD only. If I understand VMAX architecture correctly. The following A,B,C,D are representing directors in in each one of 4 engines, and 05-12 under SYMB are representing CPU's, and C and D under TID are representing ports, respectively. Then from this layout, it is in line with what you all just said that 32 EFD drives are evenly located under 4 engines ad 8 drives each. However, my questions are following:

1. why under director A, drives are spread under different CPU, whereas in the other cases, drives are under same CPU but different ports?

2. are there any reasons, drives are not put under some CPUs, for instance, 6B is missing?

3, why there are 9 drives under director B? I assume drive under 10A is spare drive, since Hypr is 0. If it is not, then why Hypr here is 0?

Thanks again!

Disk Group                   : 3

Disk Group Name              : DISK_GROUP_003

Technology                   : EFD

Speed (RPM)                  : 0

Form Factor                  : N/A

                                                         Capacity(MB)

Ident  Symb Int TID Vendor     Type       Hypr   Total       Free      Actual

------ ---- --- --- ---------- ---------- ---- ---------- ---------- ----------

DF-5A  05A    C   6 HITACHI    HRP0100       8      95396        173      95396

DF-6A  06A    D   6 HITACHI    HRP0100       8      95396        173      95396

DF-7A  07A    C   6 HITACHI    HRP0100       8      95396        173      95396

DF-8A  08A    D   6 HITACHI    HRP0100       8      95396        173      95396

DF-9A  09A    C   6 HITACHI    HRP0100       8      95396        173      95396

DF-10A 10A    D   6 HITACHI    HRP0100       0      95396          0      95396

DF-11A 11A    C   6 HITACHI    HRP0100       8      95396        173      95396

DF-12A 12A    D   6 HITACHI    HRP0100       8      95396        173      95396

DF-5B  05B    C   5 HITACHI    HRP0100       8      95396        173      95396

DF-5B  05B    D   6 HITACHI    HRP0100       8      95396        173      95396

DF-7B  07B    C   5 HITACHI    HRP0100       8      95396        173      95396

DF-7B  07B    D   6 HITACHI    HRP0100       8      95396        173      95396

DF-8B  08B    C   E HITACHI    HRP0100       8      95396        173      95396

DF-9B  09B    C   5 HITACHI    HRP0100       8      95396        173      95396

DF-9B  09B    D   6 HITACHI    HRP0100       8      95396        173      95396

DF-11B 11B    C   5 HITACHI    HRP0100       8      95396        173      95396

DF-11B 11B    D   6 HITACHI    HRP0100       8      95396        173      95396

DF-6C  06C    C   5 HITACHI    HRP0100       8      95396        173      95396

DF-6C  06C    D   6 HITACHI    HRP0100       8      95396        173      95396

DF-8C  08C    C   5 HITACHI    HRP0100       8      95396        173      95396

DF-8C  08C    D   6 HITACHI    HRP0100       8      95396        173      95396

DF-10C 10C    C   5 HITACHI    HRP0100       8      95396        173      95396

DF-10C 10C    D   6 HITACHI    HRP0100       8      95396        173      95396

DF-12C 12C    C   5 HITACHI    HRP0100       8      95396        173      95396

DF-12C 12C    D   6 HITACHI    HRP0100       8      95396        173      95396

DF-6D  06D    C   6 HITACHI    HRP0100       8      95396        173      95396

DF-6D  06D    D   5 HITACHI    HRP0100       8      95396        173      95396

DF-8D  08D    C   6 HITACHI    HRP0100       8      95396        173      95396

DF-8D  08D    D   5 HITACHI    HRP0100       8      95396        173      95396

DF-10D 10D    C   6 HITACHI    HRP0100       8      95396        173      95396

DF-10D 10D    D   5 HITACHI    HRP0100       8      95396        173      95396

DF-12D 12D    C   6 HITACHI    HRP0100       8      95396        173      95396

DF-12D 12D    D   5 HITACHI    HRP0100       8      95396        173      95396

                                               ---------- ---------- ----------

Total                                             3148077       5546    3148077

0 Kudos
rohang1
3 Silver

Re: To show the layout of what engines and dir where each different type of disks is located?

Jump to solution

The following A,B,C,D are representing directors in in each one of 4 engines, and 05-12 under SYMB are representing CPU's, and C and D under TID are representing ports, respectively. Then from this layout, it is in line with what you all just said that 32 EFD drives are evenly located under 4 engines and 8 drives each. However, my questions are following:

I'd like to correct you here ..

A,B,C,D are not directors. They are processors, and C and D are the ports on these processors that connect to your DAE's.

For example - 5A is a (back-end) director .. 5A:C is cabled to 1 DAE (the same DAE is connected to director 12A:C for backend redundancy) and 5A:C6 is your EFD within the DAE. [if you do a #symdisk list -da 5a -int c, you should be able to see all your drives on this DAE/loop, including the EFD at location 5A:C6]

1. why under director A, drives are spread under different CPU, whereas in the other cases, drives are under same CPU but different ports?

You should call it director 5A, or director 12D .. not director A.

From what I see, the drives are pretty much evenly distributed under the C and D ports for every back-end director.

Remember : just like you have ports 0 and 1 for your front-end director (5e:0, 5e:1, 5f:0, 5f:1, 5g..), you'd have ports C and D for your backend director.(5A:C, 5A:D, 5B:C, 5B:D, 5C: .. )

[port 0 and 1 on a slice (5e for example) share a CPU, similarly ports C and D on a slice(5A for example) share a CPU]

2. are there any reasons, drives are not put under some CPUs, for instance, 6B is missing?

your question should be : are there any reasons, drives (EFD's) are not put under some directors, for instance, 6B is missing?

3, why there are 9 drives under director B? I assume drive under 10A is spare drive, since Hypr is 0. If it is not, then why Hypr here is 0?

I see 9 drives under directors 7 and 8 (which will be contained in an engine). not sure on this.

Again, 10A is not a drive. 10A is a director and yes, 10A:D6 is a spare drive configured on the backend loop.

thanks.

emcmagic
3 Argentum

Re: To show the layout of what engines and dir where each different type of disks is located?

Jump to solution

Hi rohang,

I guess I still not fully understand the relationship of the engine and directory.

We have 32 EFD, and as you and others said, they should be evenly spread to 4 engines, so, each engine should have 8 EFD drives. Based on the output, is following correct?

DRIVES connecting on 05A-12A is located on engine,

DRIVES connecting 05B-11B on 2nd.

DRIVES connecting 06C-12C on 3rd,

DRIVES connecting 6D-12D on 4th

I know it is incorrect, because 05A-12A are directors, they should not be on the same engine. One engine only has 2 directors. I am confused.

Thank you for your patience!

0 Kudos
rohang1
3 Silver

Re: To show the layout of what engines and dir where each different type of disks is located?

Jump to solution

Hello emcmagic.

Ok. let me do my bit to try and get this across to you.

If you look at the #symdisk list output you'd see directors labelled 5 to 12 .. which is a total of 8 back-end directors spread across 4 engines, for your 4 engine VMAX.

Drives connecting to directors 5 and 6 are located on an engine,

Drives connecting to directors 7 and 8 are located on an engine,

Drives connecting to directors 9 and 10 are located on an engine,

Drives connecting to directors 11 and 12 are located on an engine.

A small correction to what I wrote on the previous reply :

For example - 5A is a (back-end) director .. 5A:C is cabled to 1 DAE (the same DAE is connected to director 6A:C (using LCC B -- remember that the drives are dual ported for backend redundancy) -- directors 5 and 6 are located within the same engine) and 5A:C6 is your EFD within the DAE. [Note: If it were a DMX, the DAE would have cables coming in from director ports 5A:C and 12A:C]

If you look at the figure below ..

Each engine with it's 2 back-end director ports connect to 8 unique DAEs (which is why the name "octant") .. for example - back-end directors 5 and 6 on Engine 3 will connect to 8 DAEs on storage bay 1B (Octant 3 in Orange color) using direct connections. The 8 connections to the 8 unique DAEs (from LCC A) will be 5A:C, 5A:D, 5B:C, 5B:D, 5C:C, 5C:D, 5D:C, 5D:D. The same 8 DAEs will have a connection from the redundant backend directors ports of director 6 from LCC B. (6A:C, 6A:D, 6B:C, 6B:D, 6C:C, 6C:D, 6D:C, 6D:D)

Do take a look at the complete output from a #symdisk list (not just disk group 3 which is your EFDs) ..

I'll paste a DMX back-end architecture to help you visualize this better ..

So if you look at the back-end [directors labelled DF01 and DF16], a DAE is cabled to dir port 1B:C and to a redundant back-end port 16B:C.

So these 2 director pairs can connect to 8 unique DAE's. (1 and 16 work in pairs, 2 and 15 work in pairs and so on .. for a DMX)

Similarly for a VMAX, you have directors within an engine working in pairs, connecting to 8 unique DAEs using 8 director ports. (1 and 2 work in pairs.. 3 and 4 work in pairs .. so on).

Hope this helps.

thanks.

View solution in original post

0 Kudos
emcmagic
3 Argentum

Re: To show the layout of what engines and dir where each different type of disks is located?

Jump to solution

Hi rohang,

Your diagrams and descriptions look great, they cleared my mind. Your help is greatly appreciated here!

I would like to take this opportunity to extend my original question, if you don't mind.  it is about FA directors.

The front ports will be located on the same directors on each engine as DA directors. They are located on CPU E,F,G, and H. Each CPU has 2 FA ports, and therefore on VMAX-20, we have 2(directors) x 4 (CPUs) x 2(ports) x 4(engines) = 64 FA ports.

Is my statement correct?

0 Kudos
Quincy561
4 Beryllium

Re: To show the layout of what engines and dir where each different type of disks is located?

Jump to solution

emcmagic, you are correct.  Each director board has 4 FA CPUs (1 per 2 ports).

0 Kudos