This post is more than 5 years old

1 Rookie

 • 

33 Posts

1753

June 25th, 2019 08:00

PV MD 3200i - VD not on preferred path

Hi,

I have read almost all posts related with this issue, and I have redistributed (all?) vdisks. Now, I have two VD (100G and 250G), the latter is storing some critcal VMs. and the first one some minor db app. First question:

It's safe to redistribute the vdisk while I/O is running on the block device? I have read on another post and one Dell tech reply that "yes". But I'm concerned, I'm running critical infrastructure here.

Firmware: 07.84.47.60 - Dell md3200i (no external enclosures)

Host os with initiators (open-iscsi) and multipathd(8): Linux. (ubuntu 18.04 AMD64).

I have paths that are OK, but one of theme looks like this: (output from multipath -l from linux):

size=100G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 rdac' wp=rw
|-+- policy='round-robin 0' prio=0 status=enabled
| |- 12:0:0:1 sdaw 67:0 failed undef unknown
| |- 15:0:0:1 sdau 66:224 failed undef unknown
| |- 16:0:0:1 sdav 66:240 failed undef unknown
| `- 18:0:0:1 sdax 67:16 failed undef unknown
`-+- policy='round-robin 0' prio=0 status=active
|- 10:0:0:1 sdaj 66:48 active undef unknown
|- 11:0:0:1 sdal 66:80 active undef unknown
|- 17:0:0:1 sdai 66:32 active undef unknown
`- 19:0:0:1 sdak 66:64 active undef unknown

 

As I understand this, it has failed over the non-preferred path, which causes the array to complain (the amber light is up, and the message "needs attention" shows up on MDSM).

 

Question #2.

If all other paths on  other vdisks from the same host (4 more) show signs of no problem then what's going on. However. I believe the issue it's not deterministic, as I have seen (before the redistribution) other vdiss were mentioned in the "Needs Attention" window, I then redistributed those (in the maintenane window, all VMs down, etc.) and haven't saw them again as reported on the non-preferred path. But I have this two now.

Yesterday I did a maintenance window and when I redistributed all vidsk directly by going to the Hardware tab, then selecting the controller (this is needed?) then from Storage/Virtual Disk/Advanced -> Redistribute Virtual Disks.

 

Everything was ok, multipathing was working, etc, then like half an hour later 1 path started failing, then another, the two vdisks I  mentioned. As far as I know this "redistribution" is a quick fix (is it?) because yesterday I did just that (but not from right  clicking on the vdisk on MDSM but directly from the Storage menu that I just put in my previus paragraph.) and then the problem appeared again.

 

What I can deduce is that my host conectivity (4 interfaces 1Gbit/s) is OK, I can ping all 8 ip addresses on the SAN (two for each interface, we use 4 subnets to reach controller 0 and controller 1, which are connected to a dell switch), the host is connected to the same switch. Also another host I have have zero problems with iSCSI, using the same switch and the same SAN, same subnets, same number of interfaces. And, since all other paths are showing as correct and are being used (round-robin) I suppose that the problem it's not on my host. MTU is jumbo as it's correctly setup both from the Linux interface and from the Switch in the middle (dell powerconnect 7024) and from the SAN. 

Can it be that I need to recreate those vdisks on the SAN side? 

Also, when doing a wireshark session I can se this message: "LUN Busy" returned from the SAN, I could see this from all interfaces, don't happen all the time but it show up eventually on all of them, is this normal?

 

Thanks I appreciate any insights into this.

Moderator

 • 

7.7K Posts

June 26th, 2019 08:00

Hello lukes357,
The first thing that I would do is to upgrade your firmware on your MD3200i, as the version that you are running was released in July 2013. The latest version of firmware is 08.20.24.60 for your MD3200i. Once you have upgraded the firmware on your Raid controllers and HDD’s then I would monitor the system to see if it happens again. If it does happen after the firmware upgrade, then we will need to review a support bundle from your MD3200i so that we can see what is going on.
Here are the steps for doing a firmware upgrade just in case you need them.

Controller Firmware
https://www.dell.com/support/home/us/en/04/drivers/driversdetails?driverid=4cp5x&oscode=w12r2&productcode=powervault-md3200i
Resource dvd
https://www.dell.com/support/home/us/en/04/drivers/driversdetails?driverid=r9g1x&oscode=w12r2&productcode=powervault-md3200i
HDD/ SDD
https://www.dell.com/support/home/us/en/04/drivers/driversdetails?driverid=x50j8&oscode=w12r2&productcode=powervault-md3200i

Firmware Upgrade steps:
1. Gather support bundle in MDSM
2. WARNING: If you have a single controller PowerVault MD3200/MD3600 series storage array you must stop all I/O operations before starting the firmware upgrade.
3. Extract the firmware to folders and remember location.
4. Burn or mount the ISO for the resource cd.
5. Uninstall the MDSM from hosts (reboot required)
6. Install MDSM from resource DVD (reboot required)
7. Clear the Major Event Log.
8. Update to 07.75.28.60 if not already there, then update to latest firmware.
o If you receive an error while checking the SPM database, ensure that you have an out of band management connection to both RAID controllers.
9. Verify connection and data.
10. Update the hard drive firmware.
11. Reboot stack and verify all is optimal.
A. Power down the server(s)
B. Power down the MD32x0(i)
C. Power down any attached storage (MD12xx)
D. Leave the power off for 2-3 minutes

Please allow for at least 60-85 minutes for the updates. Time may vary depending on how long each reboot takes.
Please let us know if you have any other questions.

1 Rookie

 • 

33 Posts

June 26th, 2019 13:00

Sam,

Concise answer thanks. The upgrade have to wait for approval from my Manager, he is now on vacation.

What I did today is create a new vdisk and moved data from the old to the new, so far that vdisk's paths are not failling. I will post results tomorrow and also when I upgrade the firmware in the future (if approved), thanks. 

1 Rookie

 • 

33 Posts

July 10th, 2019 06:00

Hi Sam,

I have approval to update the firmware. Just to be on the safe side because I don't know the version conventions used by DELL (I assume higher numbers of the fw are newer versions). With that said my fw version is 07.84.47.60 so (as per your previous answer and the Readme file) I don't need the bridge version (07.75.28.60).

That is correct?

Also, I don't want to upgrade the disk firmware (me being really careful), it's mandatory for using the latest fw?

 Thanks.

Moderator

 • 

7.7K Posts

July 10th, 2019 10:00

Hello lukes357,

That is correct, as your current version of firmware that you are using, the bridge firmware has already been applied.  You are good with going directly to 08.20.24.60 firmware.

Please let us know if you have any other questions.

1 Rookie

 • 

33 Posts

July 10th, 2019 11:00

Hi Sam,

I'm inclined to not upgrade the disk firmware (me being really careful), it's mandatory for using the latest fw?

Thanks.

Moderator

 • 

7.7K Posts

July 11th, 2019 08:00

Hello lukes357,

It is not mandatory to upgrade the firmware on your HDD’s.  However, I can say that if you don’t upgrade the HDD’s firmware you will most likely get some HDD alerts & or errors on your drives at some point that is resolved by the newer drive firmware.   

Please let us know if you have any other questions.

1 Rookie

 • 

33 Posts

July 26th, 2019 07:00

Hello Sam,

I will be updating the firmware soon. Question, I see on the PV Owner's manual Page 190 they mention the NVSRAM, which one goes first (install order) the firmware for the static ram or the firmware for the array since I have two files, one for each, thanks.

 

 

1 Rookie

 • 

33 Posts

July 29th, 2019 05:00

Sam,

Upgrade performed two days ago and so far no problems, thanks for the help, I appreciate it, have a great day.

 

 

Top