I just had Dell in to add an additional PS6110 enclosure into an existing array group of 6 enclosures. As part of the install they needed to update the firmware on all enclosures. As they all have multiple controllers, in my naivety I was expecting this to be able to be done without powering down all connected servers and effectively taking the whole array down. However each enclosure had to be rebooted which took all the volumes offline as they were "vertically striped" across the members.
Isn't there a method to perform these updates without so much disruption?
Also as the f/w was uploaded to the members and the restart option was taken from the GUI, the enclosures did not appear to restart and come back online. After approx. 15 mins I had to connect a serial cable and restart from the CLI? ??
A restart is always required for firmware upgrades. Same as for a Windows Server when the kernel or other system files are updated, they can't be done live.
The process is to restart the secondary CM first, then failover to it, then restart the old primary. This cuts the outage time to as short as possible.
Would need diags to see exactly what happened, but a couple of things to consider.
1.) On the support website, along with the Firmware download, is a PDF with instructions on setting the Disk Timeout Value for several OS's. In a virtual environment, both the server and VMs need this set.
2.) Do the restart in a lower IO window, if you can't actually do it in a maintenance window
3.) My personal preference, is to do upgrade via a serial terminal server. That way I can monitor the entire process. I watch both primary and secondary at the same time. It's not required, but I find it helpful.
4.) Most importantly, this allows me to stage the restarts. Which sounds like wasn't done in this case. By doing one restart at a time, it could have prevented the outage you experienced. Again without complete diags, I can't say for certain. After the restart, I wait for a console message to say the member has returned to the group, then wait about a minute or so before doing the next restart. This allows the members to settle in and sync up with each other. Since typically, a volume is only spread across 3 members by default, this also prevents a large scale outage when done one at a time.
Please open a support case if you would like a more defintive answer.
Social Media and Community Professional
Get Support on Twitter - @dellcarespro
Sorry you had trouble. I've got a PS6100XS and have applied at least 5 firmware updates to it during production over the last couple of years and never had any trouble.
As Don indicated, the firmware update requires a restart, however a restart in “EqualLogic speak” is in actuality only a control module failover and not a power cycle of the array.
So, provided that ALL your hosts iSCSI initiators (physical and/or Virtual) are configured properly to “ride out” the failover from one controller to the other you should not see any outage. But, please note that some DB applications behave strange and might need to be shut down (so doing a firmware update during off hours is needed in these cases), but this is the exception rather than the norm.
Testing the controller failover and your host connectivity is important in the initial setup of your group, requiring proper wiring, inter-switch links, etc. All this is outlined in the configuration guide:
To ensure all your physical hosts, and VM’s have the proper iSCSI disk timeout’s setup properly (so they can ride out the restart/failover), you need to ensure that the setting are correct, please refer to the “iSCSI Initiator and Operating System Considerations “ located on the firmware download page (this document can change between major version of the FW, so always refer to it prior to updating firmware).
Lastly, we do have a solution for multi-member groups to do a “rolling update”. This method is designed to prevent any outage, and handles mission critical application (that can’t be taken off line, even for a few minutes). This type of update typically takes several days to complete the FW update to all the members. Again on the support site you can find the steps to do this (login to the support.equallogic.com, search the KB for “Updating a multi-member group without down-time”)
And as Don indicted as a good practice…. “My personal preference is to do upgrade via a serial terminal server. That way I can monitor the entire process. I watch both primary and secondary at the same time. It's not required, but I find it helpful”
Social Media and Community Professional
Get Support on Twitter - @dellcarespro
Follow me on Twitter: @joesatdell
I'm actually trying to update mine from 7.1.12 (Backplane 62532) but apparently I can't download firmware updates without a support contract and my one piece of equipment is not enough to warrant one. I can't seem to find these updates anywhere else and Dell doesn't seem to offer anything for this either.
You are correct. You can only download firmware and other bundled software if you have an array with an active support contract. That contract also maintains your license to use the firmware and applications. There is no other way to get that firmware.
I guess that leaves me out of luck because they don't sell support for the PS4000 anymore either and even if it was an imaginable financial option, which it's not, I can't see going out and buying an entirely new array that I can get a support contract on just to get a download for an older end of life one.
The PS Series SAN is no longer sold. So you couldn't get a new one. You can get hardware support up to 12 years from ship date but that doesn't include firmware.
I do understand this now, though admittingly I find it all a bit absurd and ridiculous.
1. It's not a "support agreement" issue because I can't even get one. It's not something even sold so it's not a monetary issue at all.
2. Exception: If it is Dell's desire that by denying firmware updates the system which otherwise functions perfectly and could still get serviced if it broke, would become more likely to be replaced with a new system because of this denied access, then I would classify such a motivation no differently than the immoral planned obsolescence Apple and Samsung were found guilty of and regardless of its legality would ethically be reason enough in itself to motivate me to no longer do business with Dell.
3. This system can still be serviced if it broke, but that's not the problem, I simply need to update the firmware.
4. The system was under support when the firmware was released. It simply at the time was a risk in updating that was appropriate to delay until more necessary (such as now).
5. Had I known, I would have simply made sure to download each of the firmware updates as it was released and simply delayed installing them.
Regardless, at least I am now aware of the stance and how it is treated, it is my fault for not knowing to ask such things before becoming invested. I now at least know better what questions to ask and ensure I compensate for when I shop for a company to buy my next array from. Lesson learned, life moves on.
Thank you for the answers.
I am sorry that you were not aware of how the array is licensed.and sorry to lose you as a Dell customer.
Equallogic founders sought to do something very different with storage. At the time the industry tended to be a license each feature and related software individually. Then add support contracts for both the hardware and software features you licensed. Making the management of those contracts complicated and time-consuming.
If you ever had to install VMS on a Digital VAX you know what I mean. Right down to serial and network ports. Each had a long license to fill in to activate them.
The PS broke that mold. You bought the HW, got a one year warranty. You got both software and hardware support. Any new firmware or bundled software was included during that first year. Keep a support contract keep getting support and firmware. Simple, one contract for both. Many arrays were sold with four or five year contracts just to make it easier.
The "firmware" is much more than simple 'code'. It's not Linux leveraging open source software in a nice package. NetBSD boots the controller HW, and loads the EQL OS, other than telnet, ftp, ssh, and http all the code for iSCSI, networking, RAID, replication, snapshots, etc were written by EQL engineers. The bundled software too is written by EQL. Which is expensive and time consuming to write, debug, QA release and support. So giving all the away forever isn't a practical business option.
Since the HW cost was not extreme the support contract becomes the license to keep it up to date. Especially since you got all the existing features and any new features that came along. That cost has to be covered somehow. Either up front or over the lifetime of the product by contract.
Microsoft did something similar. You could get a contract for a period of time and whatever OS's were released you got a now additional cost.
Sorry to see you go, I sincerely wish you well.