Alec,

This is from a CX500 running flare 24 (running release 24 patch .014 currently) - I have merged the SPA and SPB log files together to make it a bit easier to see - the 2580 event does have the word "Testing" in each line going back to the begining of the log (June 2007). This array is in a lab and not very busy.

It's possible that the line that is missing the extra data is a problem in flare 24 -  what patch level ae you at on this array?

Did this just start happening? Does it occur on all the arrays - those running release 19 and those running release 24? Or is it confined to a subset of all the arrays - same patch level?

glen

A       01/03/10 00:50:21 Bus0 Enc0 SpsB      698 Battery Testing In Progress [Testing]    0    0        80
B       01/03/10 00:50:48 Bus0 Enc0 SpsB      62a Periodic SPS test time [OK]              0    0        2
B       01/03/10 00:50:51 Bus0 Enc0 SpsB      698 Battery Testing In Progress [Testing]    0    0        80
A       01/03/10 00:51:00                    2580 Storage Array Faulted Bus 0 Enclosure 0 SPS B : Testing Bus 0 Enclosure 0 Power B : Faulted
B       01/03/10 00:51:10                    2580 Storage Array Faulted Bus 0 Enclosure 0 SPS B : Testing Bus 0 Enclosure 0 Power B : Faulted
A       01/03/10 00:51:49 Bus0 Enc0 SpsB      637 SPS Recharging                           0    0        0
A       01/03/10 00:52:05                    2500 Storage Array is not Faulted
B       01/03/10 00:52:20 Bus0 Enc0 SpsB      637 SPS Recharging                           0    0        0
B       01/03/10 00:53:20                    2500 Storage Array is not Faulted
A       01/03/10 01:03:48 Bus0 Enc0 SpsB      638 SPS Enabled [OK]                         0    0        2
A       01/03/10 01:03:48 Bus0 Enc0 SpsA      698 Battery Testing In Progress [Testing]    0    0        80
A       01/03/10 01:04:04                    2580 Storage Array Faulted Bus 0 Enclosure 0 SPS A : Testing Bus 0 Enclosure 0 Power A : Faulted
B       01/03/10 01:04:18 Bus0 Enc0 SpsB      638 SPS Enabled                              0    41433130 2bff1842
B       01/03/10 01:04:18 Bus0 Enc0 SpsA      698 Battery Testing In Progress [Testing]    0    0        80
B       01/03/10 01:05:19                    2580 Storage Array Faulted Bus 0 Enclosure 0 SPS A : Testing Bus 0 Enclosure 0 Power A : Faulted
A       01/03/10 01:05:21 Bus0 Enc0 SpsA      637 SPS Recharging                           0    0        0
B       01/03/10 01:05:51 Bus0 Enc0 SpsA      637 SPS Recharging                           0    0        0
A       01/03/10 01:06:14                    2500 Storage Array is not Faulted
B       01/03/10 01:06:24                    2500 Storage Array is not Faulted
A       01/03/10 01:17:13 Bus0 Enc0 SpsA      638 SPS Enabled                              0    41433130 2bff1822
B       01/03/10 01:17:43 Bus0 Enc0 SpsA      638 SPS Enabled [OK]                         0    0        2

Hi, Glen

It'a all started once we've added thee second SPS to our CX300 systems. It seems that all of it running patch FLARE 24 with 0.16 (i have to verify it). All systems with FLARE19 doesn't have this problem. I've been trying to simulate this problem all day today with patch 0.15 and it worked fine, so i'll try to upgrade it to 0.16, to see if the problem is there.

Thanks

Alec

Thanks Alex, I'll see if I can upgrade our LAB array to 24 patch 16 tonight

glen

Hi, Glen

I've managed to simulated it. After upgrading the system to .016 and running few SPS tests in the row - on the 8th or 9th attempt.

The only difference is that at our "life" systems the test begins always at SPS A (and at my system it begun at SPS B) and SP B reports about the Fault (and at my system its SP A).

Alec

Alec,

I just finished upgrading to patch 16 and am currently running the SPS test every hour - hopefully it will shed some light on this issue.

glen

Please see KnowledgeBase article emc172277 - this was an event added in patch 11 that generated the error - engineering recommends that you disable the 2580 Event ID in the monitor templete - if there is a real issue the array will generate a different error code that will be sent.

http://csgateway.emc.com/primus.asp?id=emc172277

============

The event number 2580 should be unchecked in the monitor template. According to EMC Engineering, event 2580 event is always called during an SPS test even if the SPS has no problems. That is why this event is not monitored. If there is a problem with the SPS, a subsequent event will identify the problem and dial home.

Note: Navisphere may need to be in engineering mode in order to access the template properties.

============

glen

Actually there is other KnowledgeBase article emc204601 that a bit closer to what we have, but still not exactly the same.

Anyway, it seems like there is some kind of miss-interaction between HW status polling and SPS testing condition. Maybe a poll requests the status of Power supply in the exact same moment when SPS momentarily cuts the power supply feed to Power Supply, so Storage Processors assumes that there is a problem with Power Supply

Alec

It;'s not really explained in the engineering notes, but I belive that you are correct - just when a test takes place if the times on the two SP's are a bit different, it probably is not aware of the test running.

You can see the times on the SP's by using:

navicli -h xxx.xxx.xxx.xxx getsptime

glen

Thanks

Alec