PowerEdge 2950 stuck in speedstep/demand-based power management mode

Hello Softrac

Could you provide the model numbers of the CPU's you are using. Based on the information you have provided I cannot find any issues. It sounds like it is configured correctly for what you are trying to do.

Thanks

In the server that has the biggest issue they are X5450's.  One of the others has the X5260, but at least that throttles up when needed (even though it should still be at 100%).

Since Demand-Based Power Management is disabled in the BIOS there are no options to adjust processor state.  

When I did a fresh install I did enable that and made sure they were both 100%, but same issue.

Main and backup servers are Gen 3.

Here is something I found interesting on the main machine - notice the core clock and throttle are both the same.

On our main backup the core clock and throttle are correct, but still should be running at 100% all the time.  At least this one will throttle up, unlike the other.

Server 3 is a Gen 2, but has the same issue as the one above - it throttles up correctly, but should still be 100% all the time.

I'll check with our escalation groups to see if they have any ideas on this. While I'm doing that disable C states and Ce1 in the BIOS. Let me know if that has any effect.

Check the power options within control panel. Make sure the maximum processor state is set to 100%.

If there are no issues with that then check to see what revision the servers are. There will either be no numerals on the front of the chassis, indicating first generation, or II/III.

We can backflash the BIOS, but the microcode changes will not be undone. This could cause compatibility between the microcode on the CPU and the BIOS. I would only recommend this as a last resort, and I would perform it on the dual core system.

What firmware version is the BMC at? The latest is 2.37 from back in 2008.

BMC is 2.37

If nothing else I may try swapping the CPU's to see if the quad cores will throttle up on the backup machine.  I had always wondered why a much better spec'd machine was 2x slower than the backup, but just had figured it was from it being a production machine and not that it was throttled down all the time.

Have you heard of this kind of thing happening with these before?  One I could understand, but three different machines spanning two generations seems odd.

No options for those that I have seen - I know they are in some of the newer servers.  Only options are:

Virtualization Technology(VT)

Demand Based Switching(DBS)

Execute Disable(XD)

When looking back at the release notes for 2.7 i noticed:

* Updated Quad-Core Intel(R) Xeon(R) Processor 5400 series C0 Stepping Microcode (Patch ID=60F)

* Updated Quad-Core Intel(R) Xeon(R) Processor 5400 series E0 Stepping Microcode (Patch ID=A0B)

Not sure if those would even relate to this, but is it possible to download any of the older firmwares such as the previous 2.6.1 to try?

Enable DBPM and windows will not boot - crashes right away and won't even generate an error or log file on both gen3 machines.  

When I did the fresh install I had DBPM enabled and that is when I was able to change the policy for the CPU and tried high performance and made sure min and max were at 100% and the same thing happened.  When I tried the Windows 7 install I also tried with DBPM enabled - same reaction.

Finally found the other post that was similar to my issue:

http://en.community.dell.com/support-forums/servers/f/956/p/19367720/19833856.aspx

Yes, it is very odd. I have discussed this issue with two hardware escalations techs and one software escalation tech. None of us have heard of this issue. We have no internal documentation of this issue being reported either.

I would like to know what happens when DBPM is enabled and the OS is set to high performance. Could you enable it on one of the systems and let me know what happens?

There has to be some kind of odd configuration issue. I am running 2008 R2 on one of my 2950's at home without any issues, so the OS works with 2950's. Could you list your configuration and firmware levels of the devices.

What I am specifically looking for is:

Power supply wattage
Any attached USB devices
Any PCI/e/x devices
How many HDDs
How much and what type of memory

I'm guessing that there is something about your configuration that is common among these systems. From what you have described I am assuming that this problem has been going on since you have owned the servers. It sounds like you have started troubleshooting poor performance and noticed this problem, and then started updating the BIOS and performing these other steps. If that is not the case and the issue started recently then let me know.

Thanks

The issue has been going on for some time, it was just assumed to be related to something else like SQL performance - it wasn't until cleaning up the database and other tests that I noticed the vast performance different between the much faster main machine and the slower backup.

Bios and firmware have all been updated as they were released so nothing new has been done aside from reflashing.

Have you happened to run cpu-z with high performance and checked if it is indeed always running at 100%?

Dual 750W power supplies

Only usb is a KVM

Ethernet card in PCI slot

3 73GB SAS and 3 146GB SAS

16 GB memory - 8 x 2gb

Bios - 2.7

BMC - 2.37

RAC - 1.60

When the fresh test was done there were no cards installed

Have you seen this paper on processor power management, interesting paper, one of the things you might try is turning off the cstates temporarily,... odd problem indeed..

download.microsoft.com/.../ProcPowerMgmt.docx

Have you tried the machine without the KVM and the Ethernet card?

What is the memory speed? Almost sounds like the cpu is not recognised as a speedstepping cpu. Any chance of switching out ram.

Have you run the Dell diags and Intel CPU test, (even though Linux is OK)?

You could call Intel tech support, there are some phenomal techs there, they can be most helpful. Recently had a programmer had an issue with underutilized CPU core use, Intel spent an easy 8 hours on the issue.

I would not worry about the throttle delay on the workable servers, it is miniscule, and once it does throttle up it should maintain until the load truely drops.

Main server might just have a borked motherboard - swapped out the cpu and drives to the backup machine and at least they throttle up now - still not ideal since I'd prefer 100% power 100% of the time.

Tried without KVM and network card when doing the clean install test.  Swapped out the memory and even just ran a single stick.  Memory is PC2-5300.

I'll be sure to check the article.

I just wish I had found all this info two months ago before the warranty went out, but isn't that how it always works.