Highlighted
brad12341
2 Iron

Shutdown and move Cisco 9506 switches

Jump to solution
Hello,

In our Texas data center we have two identical 9506 switches (mds1 and mds2). Each host and Storage array is connected to both switches. We need to move both switches to a new location. We will pull multiple new FC cables for each host and array to the new location.

We will then power down mds1 and move it to the new location. Once powered back up we will plug in the newly run cable into mds1. We will then go to each host and remove one old cable (each host has 2 HBAs) and plug in the newly run cable that we just plugged inot mds1. We will do a similar thing for each array.

As long as we plug the right cables into the right ports we should be good with no zonning changes required. Once mds1 comes up in the new location we should be good. During the move of mds1 all hosts and storage will keep running through the redundant ports on mds2.

At some point later we would shutdown mds2 and move it while everything runs on mds1.

We've never done anything like this and the mds switches are fairly new to us. Of course we would back up the switches before shutdown. Looking at the MDS9000 Quick Reference I can see that their is cli command shutdown. It says that it gracefully shuts down the interface and administratively disables traffic flow.

I'm guessing that when we power up the switch at the new location it will just come up with all zonning infor in place.

Are there any issues that we should be aware of before attempting this move?
Labels (1)
0 Kudos
1 Solution

Accepted Solutions
healyj
2 Iron

Re: Shutdown and move Cisco 9506 switches

Jump to solution

Hi there,

In our effort s to clean up the forum, we came across your question / statement.

If the question / statement is still valid, not expired and you need an update please reach out again and we try to get it answered.

As for now we set it to “answered.”

Regards,

Jim

0 Kudos
29 Replies
AranH1
4 Germanium

Re: Shutdown and move Cisco 9506 switches

Jump to solution
Brad,
I have done this a few times unfortunately with MDS switches. I say unfortunately because it is never fun to relocate a data center. Last year we moved two production local data centers into one large production data center. The switches were the easiest part of the move

Make sure you backup each switch:

switch# copy run start
switch# copy start ftp:switch1_config.txt

We moved our main production data center in one weekend (300 servers, 60 SAN-attached servers, two MDS9509, four CLARiiONs, one Centera) so were not moving one switch at a time. Using that process we shut down all hosts first, then all arrays, and after copying off the startup config for the switches we powered them down. So there was no host i/o to deal with when shutting down the switches. Our secondary production data center was much smaller, two MDS9216s, one CLARiiON and three hosts, so it was very easy.

Since you are going to be shutting down a switch that is in use you will want to be more careful. The shutdown command is for ports, not the entire switch. You could use the reload command without any arguments which would restart the switch. You could then power off the switch manually as soon as it shuts down and before it boots up. If you are doing this during a low i/o period (I hope) then you could just power off the switch after copying off the config. These are cisco switches and can be powered off manually without any other command run first.

If you are using pWWN zoning and not port zoning you don't have to worry about getting the cables exactly in the same spot. I even used the move to organize the cabling better and changed a lot of the cabling locations. Purchasing new cables for all hosts and storage and running it all before hand will save you tons of time. This made things much easier for us as you can imagine.

Hope this helps you. Did I answer all your questions?

Aran
brad12341
2 Iron

Re: Shutdown and move Cisco 9506 switches

Jump to solution
AranH, Thanks for the good info.

We use wwn zoning so it sounds like things will work even if we do not get plugged back into the same ports.

Several of the apps are 24/7 so there will always be data moving through both switches. It sounds like we may be ok in just shutting off the two black rocker switches at the top of mds1 before moving it?

We've never powered these switches off, so hopefully they will come up at the new location without any problems.

Thanks - Brad
0 Kudos
AranH1
4 Germanium

Re: Shutdown and move Cisco 9506 switches

Jump to solution
Brad,
Yes, you can just toggle the black switches when you are ready to powerdown. Do you have the luxury of performing a test of the process? You could schedule a time to just shutdown one of the switches. See how all your hosts deal with the outage (find out if there is any multipathing that is not setup right :D ) and then power on the switch again.

The MDS switches are real solid in my experience and I have had to move 9509s a few times and have yet to have any issues.

Aran
0 Kudos
RRR
5 Tungsten

Re: Shutdown and move Cisco 9506 switches

Jump to solution
If you are using pWWN zoning and not port zoning you don't have to worry about getting the cables exactly in the same spot.


What about port security ? Although not commonly used, it is possible to only allow 1 wwpn to connect to a specific port. I would label the cables anyway, just to make sure.
0 Kudos
brad12341
2 Iron

Re: Shutdown and move Cisco 9506 switches

Jump to solution
I am more familar with AIX than Windows although we run both. When we move our MDS switches I will need to know which HBA goes to which switch. This becomes more difficult because I have never visited this remote data center in Texas. For the AIX host below I know it uses HBAs fcs3 and fcs6 to connect to the DMX. Below I have run the lsslot command and can see fcs3 is in slot P1-I5 and fcs6 is in P2-I5. When the time comes I can instruct the onsite operator to unplug the old cable from the correct slot and plug in the new cable that runs to the new location of the mds1 switch.

root@wfscr05:/#lsslot -c slot
# Slot Description Device(s)
U1.9-P1-I1 Logical I/O Slot pci7 fcs0
U1.9-P1-I2 Logical I/O Slot pci8 ent0
U1.9-P1-I3 Logical I/O Slot pci9 fcs1
U1.9-P1-I5 Logical I/O Slot pci16 fcs3
U1.9-P1-I10 Logical I/O Slot pci11 sisscsia0
U1.9-P1/Z1 Logical I/O Slot pci10 scsi0
U1.9-P1/Z2 Logical I/O Slot pci22 scsi4
U1.9-P2-I1 Logical I/O Slot pci12 fcs2
U1.9-P2-I2 Logical I/O Slot pci13 ent1
U1.9-P2-I5 Logical I/O Slot pci25 fcs6
U1.9-P2-I8 Logical I/O Slot pci15 ent2
U1.9-P2/Z1 Logical I/O Slot pci14 scsi1
U1.9-P2/Z2 Logical I/O Slot pci29 scsi5
root@wfscr05:/#

Windows is more difficult for me. There are 2 HBAs in each server. At the switches each cable is labeled hba1 or hba2. Is there a simple way in windows to find out which slot hba1 is in? I have not been able to find anything like the AIX lsslot command.
0 Kudos
AranH1
4 Germanium

Re: Shutdown and move Cisco 9506 switches

Jump to solution
I usually just open up the HBA manager software on the host, list the pwwn of the hbas and compare that to the zones for the host on each switch. That will tell you which HBA is plugged into which switch, and then you can flash the LEDs on the HBA to determine which physical slot is associated with the HBA.

And the port security (hard zoning) in my opinion is not worth the effort. If your data center is unsecured to the point that anyone can plug into a switch you have bigger problems :D
0 Kudos
RRR
5 Tungsten

Re: Shutdown and move Cisco 9506 switches

Jump to solution
port security is NOT hard (port) zoning !

Port zoning means you put 2 ports in a single zone and let the wwpn communicate with each other, no matter what you plug in there.
Port security means that the wwn of a device can be allowed as the only wwn able to use that port. After having set the security, you can still use wwn zoning !!

check out this document on the Cisco website: http://www.cisco.com/en/US/docs/storage/san_switches/mds9000/sw/rel_1_x/1_3/fm/configuration/guide/S...
0 Kudos
AranH1
4 Germanium

Re: Shutdown and move Cisco 9506 switches

Jump to solution
Sorry Rob :D I have heard people use the term port security intermingled with hard zoning before and didn't realize you were actually referring to Cisco port security.

But I would not implement port security and restrict a pwwn to a single port, but rather a range of ports. That way you can move the cable if needed (or by mistake) without the administrative headache of reconfiguring your port security and the fabric is secured to trusted pwwn's.
0 Kudos
brad12341
2 Iron

Re: Shutdown and move Cisco 9506 switches

Jump to solution
One more question. Currently mds1 has vsan100 and mds2 has vsan200. All hosts and storage arrays are redundant on both switches. For example a host will have 2 hbas with one going to mds1 and may zone to cx380_spa1. The other hba may go to mds2 and zone to cx380_spb1. Both switches uses two ports to form an ISL which is vsan300.

We will move mds1 one night and then move mds2 a different night. The night that we move mds1 we will remove all old cables including the two ISL ports. Mds2 will continue to run and all hosts and storage will remain up on mds2 while we power off and move mds1.

We have never removed thse ISL cables before. I am not sure what will happen on mds2 when we remove the ISL cables on mds1. It may take a while to get mds1 racked up in the new location. Once it is back up we will immediately plug in two new cables for the ISL. Should we anticipate errors/problems with mds2 when we remove the ISL cables?
0 Kudos