PowerEdge Hardware General

[Marc-CED]

T320 with Xeon E5-1410 will throttle and get stuck at low speed.  Cold rebooting server will fix.  Random issue occurred on 2 servers now.   One was set in the bios to DAPC.  The other, Windows controlled (yes Windows power settings were set to Performance).  One had the latest bios (Server 2012 R2) and the other had an older bios (Server 2008 R2).   I stress tested the cpu on both and observed no overheating.

This is a known bug with Dell PowerEdge T320 Servers with a Intel Xeon E5-1410 v2.  On the Server 2012 R2 server, it was happening randomly only once or twice a month.  On the Server 2008 R2 server it only happened a couple of times at random.  The CPU would go down to below 0.20 Ghz and the 2012 server would crawl and service availability was seriously degraded.   This also happened to another client T320 Server with Server 2008 R2 with an Intel Xeon E5-1410.  Multiplier would not increase from 12 and the server was running at ~1.2ghz.

If your CPU has been tested and is not overheating, this is the workaround:

Resolution:  In the PowerEdge Bios, change the System Profile to Performance Optimized.  This will disable C-States and C1E.   Instructions:  www.ponjavic.com/.../disable-dell-poweredge-m620-cpu-throttling

This has to be a bug in the bios or the microcode of the CPU.  The above resolution works.
My clients don't care about the minimal loss in power savings.  But all T320s ship with the default set to Performance Per Watt (DAPC).  This is a serious issue.

DELL:  This problem is hard to duplicate, but needs to be fixed. I'm surprised it hasn't been resolved already.

[DELL-Josh Cr]

Hi,

We have escalated the issue and will provide more information when it becomes available.

[Marc-CED]

Thanks.  I have been contacted by a Dell Enterprise Engineer.

[Dave F.S.]

Thank you Marc for posting this!

We have 20 plus T320's out in the field and have been plagued by what appears to be this very same issue. Our clients randomly experience this CPU slowness approximately once a month which is crippling to their business. Low and behold, your cold boot fixed the issue and the CPU returned to normal but before we start proactively changing the BIOS on all our client's Servers, I would appreciate a call from one of the Dell Engineers to discuss this issue and the fix a little further.

[Marc-CED]

Dell said they are trying to resolve the issue.  But it's hard for them to reproduce and fix.  They offered a beta bios, but I was not interested.   I made a change to my bios settings on 2 servers and the problem has not reoccurred.  I've described the exact changes in my post above.   The potential side effects in my expert opinion:  Slightly higher energy consumption (I don't care); Slightly Higher CPU temperature (still well within Intel spec); Slightly shorter CPU life (not an issue as the CPU life far exceeds it's useful life).

[42tech]

I experienced this exact issue this morning.  New Dell PowerEdge T320 with Xeon E5-2430 v2 (2.50GHz).  Deployed about a month ago.  Client called complaining of very slow access to network resources and server being very sluggish.  On site, i saw that the CPU in task manager wasn't showing much utilization, but the CPU was reporting a speed of 0.15Ghz 

I rebooted and the problem was still there.  I shut down, unplugged, powered up and checked the bios settings.  BIOS version is 2.3.3   and I changed a setting that said "Allow OS to place cpu in idle state" to disabled.  The power profile in use was the default Performance per Watt .

upon boot, server was again working at full speed, with lowest reported speed at 1.18GHz

[Marc-CED]

I don't recall a bios setting with that name. But that may not fix it.   When you cold boot the server (not a warm reboot), the problem goes away.  For a time.  Then comes back.  Your setting may work.  I can't say.  I do know that my suggested setting in System Profile to "Performance Optimized " does in fact work.

[Marc-CED · Accepted solution]

A BIOS update is available. The release notes state:  "Updated Intel Management Engine (ME) binary to Rev 2.1.5. 0x8B, to address issues that resulted in unexpected temporary or permanent Processor frequency degradation." 

I tested the bios and it fixed the issue described in this thread. 

=============================================

Dell Server BIOS PowerEdge T320 Version 2.4.2       
T320_BIOS_8H9VP_WN64_2.4.2.EXE | Update Package for Microsoft® Windows® 64-Bit. (22 MB)
BIOS
Release date 07 Apr 2015
Last Updated 07 Apr 2015  
Recommended
 Version 2.4.2,2.4.2 

Dell Server BIOS PowerEdge T320 Version 2.4.2

PowerEdge T320 BIOS Version 2.4.2

Fixes & Enhancements

Enhancements
- Updated Intel Xeon processor E5-2400 V2 product family microcode to 428.
- Updated Intel chipset configuration compliant with latest BIOS specification updates Rev (2.0.8).
- Updated Intel Management Engine (ME) binary to Rev 2.1.5. 0x8B, to address issues that resulted in unexpected temporary or permanent Processor frequency degradation.
- Updated Intel SINIT Authenticated Code Module to SINIT ACM 2.4.
- Made improvements to the System Setup user interface when Console Redirection is enabled.
- Updated iDRAC HII configurator UEFI driver to v1.65.65.04.
- Updated BIOS Copyright as 2015 Dell Inc.

Fixes
- Corrected an issue where VMware ESXi 5.1.x, QLE24xx cards stop responding during disk I/O when Intel Xeon E5-2400 V2 family of processors is used.
- Corrected a system crash issue while performing the change security key operation under 'Integrated RAID Controller PERC 710' Configuration Utility by using mouse.

Version

Version 2.4.2, 2.4.2

Category

BIOS

Release date

07 Apr 2015

Last Updated

07 Apr 2015

Importance

Recommended

I have not tested this bios to see if it fixed the throttling.

[Radiman]

Chiming in on this issue just for a refresher. I had the exact same problem and this thread saved me. Our 320 server was a DC and this has happened 3 times now. I had to shut down, unplug the power and restart to get the server to behave. My BIOS version was 2.3. I updated to the 2.4.2 BIOS and the problem is fixed.

[catogtp-gmail.com]

Having the same issue as well. Reboot will not solve this. I hope the bios change does.

Thanks for posting this, it's still helping years later.