PowerEdge R530 ram memory mix 16/32 2CPU

Hello @FelixTheCat_,

I've been studying your memory configuration for dual CPU E5-2699v4 with the following memory DIMMs:

2x 32GB DDR4 ECC 2Rx4 2666
6x 16GB DDR4 ECC 2Rx4 2133

And first thing I can point is that memory bus operating frequency can be 2400 MT/s, 2133 MT/s, 1866 MT/s, 1600 MT/s, or 1333 MT/s. This being said 2666 MT/s is not supported.

I would like to check the Memory module compatibilty after going any further. Do you have the brand and model? Maybe the DIMM P/N? I need to check DIMM Type, rank, speed and voltage.

This being said, if modules are compatible. I would suggest you first trying a 96GB design with the 6 16GB DIMMs. With the following population order: A1, A2, A3, B1, B2, B3. if this dual config works you can consider adding the 32GBs modules in the A1, B1 and move those two 16GB to A4, B4.

Regards.

Hi Felix,

The owner's manual should be of value to you. Page 68 is where the section on memory begins. That should help clear a few things up.

https://dell.to/37kESNj

Hello @Dell-DylanJ,

Yes, exactly on this manual I was looking on and I was assuming based on this document that configuration of 4x 32GB should work in dual processor configuration. 2R x4  

Because I'm running BIOS version 2.11 (which is newer than BIOS 2.8 where support for 2666 MT/s has been added) should work also with memory modules at 2666 MT/s. BIOS release version 2.11 is newer than latest owners manual which is from 02.2018. In page 3, from: https://dl.dell.com/FOLDER05781350M/1/R430R530T430_BIOS_2.10.5_RN.pdf

Thank you!

Reading your post, it looks like you have 4 DIMMs in the A bank, with none in the B bank. WIth a dual processor configuration, you need to distribute the DIMMs in both A and B following the guidelines in the manual.

If you don't have all your DIMMs in the A bank, please share what slots the DIMMs are in. 

Hello @Dell-DylanJ ,

The purpose of previous post was to show memory type that I have and that is seen by BIOS (being at 2666 Mhz is not a problem). Indeed, is on bank A, because is running only with one CPU. 

If I try to run into Dual Processor configuration, iDRAC/LifeCycle will not show the new memory configuration (A1,A2, B1, B2) which I already tested, because: 

• BIOS is in freeze state (or loop detection routine of CPU/Memory) most probably. Is not showing anything on VGA output.
• Being in a freeze state, information of new memory mapping is not update -> it shows only last known system configuration in iDRAC.

Thank you!

Hello @Dell-DylanJ ,

"EDIT: I missed part of the post." -> No problem. If you need some logs, I can post them.

Basically, single clue is this: 

(...)
2020-06-11 20:14:39	SYS1003	System CPU Resetting.
2020-06-11 20:14:38	CPU0000	Internal error has occurred check for additional logs.
2020-06-11 20:14:37	RAC0703	Requested system hardreset.
2020-06-11 20:14:37	SYS1003	System CPU Resetting.
2020-06-11 20:14:36	CPU0000	Internal error has occurred check for additional logs.
2020-06-11 20:14:34	RAC0703	Requested system hardreset.
(...)

 and if I change memory modules in slots (from Dual CPU configuration to a single CPU), the updated new hardware profile (from BIOS perspective) - logs shows in this way: 

2020-06-11 20:31:07	SYS1000	System is turning on.
2020-06-11 20:28:39	SEC0034	The chassis is closed while the power is off.
2020-06-11 20:27:39	PR36	Version change detected for Internal Dual SD Module firmware. Previous version:1.11, Current version:N/A
2020-06-11 20:27:29	HWC2004	The system board Intrusion cable or interconnect is connected.
2020-06-11 20:27:09	SYS1003	System CPU Resetting.
2020-06-11 20:27:09	SYS1001	System is turning off.
2020-06-11 20:26:39	IPA0100	The iDRAC IP Address changed from 0.0.0.0 to 172.17.20.120.
2020-06-11 20:26:33	HWC2005	The system board Intrusion cable or interconnect is not connected, or is improperly connected.
2020-06-11 20:25:15	SEC0031	The chassis is open while the power is on.
2020-06-11 20:24:16	DIS002	Auto Discovery feature disabled.
2020-06-11 20:24:15	RAC0182	The iDRAC firmware was rebooted with the following reason: ac.
2020-06-11 20:24:07	PR1	A replacement part was detected for device: DDR4 DIMM(Socket A8)
2020-06-11 20:24:07	PR1	A replacement part was detected for device: DDR4 DIMM(Socket A7)
2020-06-11 20:24:07	PR1	A replacement part was detected for device: DDR4 DIMM(Socket A6)
2020-06-11 20:24:07	PR1	A replacement part was detected for device: DDR4 DIMM(Socket A5)
2020-06-11 20:24:06	PR1	A replacement part was detected for device: DDR4 DIMM(Socket A4)
2020-06-11 20:24:06	PR1	A replacement part was detected for device: DDR4 DIMM(Socket A3)
2020-06-11 20:22:38	PSU0800	Power Supply 1: Status = 0x00, IOUT = 0x0, VOUT= 0x0, TEMP= 0x0, FAN = 0x0, INPUT= 0x0.
2020-06-11 20:14:51	CPU0000	Internal error has occurred check for additional logs.
2020-06-11 20:14:49	RAC0703	Requested system hardreset.
2020-06-11 20:14:49	SYS1003	System CPU Resetting.
2020-06-11 20:14:49	SYS1001	System is turning off.
2020-06-11 20:14:49	SYS1003	System CPU Resetting.

In first example is shown the server's logs while is not booting at all, under Dual CPU configuration and second example is a good state, when BIOS discovers a memory change (added or removed memory modules).

As I stated before, I'm stating again that both CPUs (any of them in CPU1 socket) and RAM memory are working correctly.

Thank you!

EDIT: I missed part of the post.

EDIT: "As I stated before, I'm stating again that both CPUs (any of them in CPU1 socket) and RAM memory are working correctly." in single CPU configuration.

Hello @Dell-DylanJ  and @DiegoLopez 

Generally I'm double checking in advance, especially on technical side, but this time I've been caught in offside. I suspect the issue is the BIOS itself (version 2.11), that in latest version available (public) today (June 12 2020) is not able to initialise Dual Processor configuration on Intel Xeon E5-2699v4.

I had two options: buy a new server or protect the previous investment and upgrade it as much as possible. A quick search engine: "comparison R540 and R530" , i've found: https://i.dell.com/sites/csdocuments/Product_Docs/en/server-generation-comparison-matrix-r540.pdf 

Document quite new, dated 2019-12, revision A00, so I was assuming that is quite accurate (Processor Intel  Xeon E5-2699v4 is dated around Q1'16). Took the decision to upgrade the PowerEdge R530 -  I was quite confident (standard quote: "what can go wrong". I inserted the random function, but with small entropy.)  

===

Dell EMC PowerEdge R540 and R530 server comparison

The PowerEdge R540 system has the following features compared with the R530 system.

===

The fix would be from my point of view (a little bit educated) to release a new BIOS let's say: 2.12 with support for CPU with 22 cores and 20 cores in Dual Processor configuration (not 100% sure about this claim, haven't verified myself, it could work with 20 cores). I know the priority is R540, I understand that, but business climate of today is not the happiest.  

@DiegoLopez  and @Dell-DylanJ  -> please confirm / infirm that is seen somewhere Dual Processor configuration with 22 cores and 20 cores on Intel E5 v4. 

Thank you!

Hello,

@Dell-DylanJ  @DiegoLopez 

Can you help here? I've done so far also a Clear CMOS (NVRAM_CLR) - usually this helps - in this case didn't make a difference. Removed any power consumption devices and left only 4x32 GB Memory / 2x CPU. 

Is a issue with CPUs VRM on the Mainboard? It doesn't have enough power for two CPUs so the BIOS doesn't allow to go further? Is there any BIOS version for R530 that will permit to boot up with 2 CPUs that are in high end range?

Thank you!

Hey @FelixTheCat_,

As per server specifications it should support one or two Intel Xeon processors E5-2600 v4 product family. This, of course, includes the E5-2699v4. For the memory. It supports 1333 MT/s, 1600 MT/s, 1866 MT/s, 2133 MT/s or 2400 MT/s DDR4 registered -> RDIMMs. Specifically 4 GB single rank and 8 GB, 16 GB, or 32 GB dual rank.

Regarding BIOS firmware version. You are perfect with the last one. The 2.11.0 was launched to address some vulnerabilities. This BIOS version is not intended to restrict the use of any processor models or dual configurations.

You know also the memory population rules so you can have up to 384 GB with a dual processor. And it seems you are having applied this rules properly.

So the only thing you said we have in logs is a 2nd CPU Internal Error. This can mean: error on CPU (you can discard it by swapping the processors). Fail on CPU socket (difficult to test, but theoretically you can swap CPUs and test). Fail on Memory DIMMs (same procedure: swapping) Fail on Memory Slot (easily you can check this if you get an error everytime you install a DIMM on this slot). So as far as I understand you were never able to boot on dual config right? not with a single DIMM each CPU?

Let's rewind a little to the basics just to make sure everything is ok. Please, attach some pictures of the DIMMs: both sides, I want to see model and part number.

Thanks in advanced.

Regards.

Hello @DiegoLopez 

Thank you for answering to my message. 

I confirm that the RAM memory will run at most 2400 MT/s (we already now that). Tested with 4x 32 GB in single CPU mode, banks A1, A2, A3, A4, memory modules that supports speed of 2666 MT/s. Tested just in case (it took to me around 10 minutes, including connection to iDRAC so why not). 

Ok, thank you for confirming that there is no restriction in BIOS. If is possible, can you get an confirmation that exists in the wild or in the lab an actual working R530 with this CPU configuration? Theoretically should work, in practice we have the picture bellow. 

2nd CPU Internal Error - this appeared when I installed second CPU before upgrading BIOS. With latest firmware (2.11) this error disappeared from local diagnostics LCD (and logs) under same conditions (memory arrangement, cpu..) . I've swapped the CPUs between them, just to be sure that I have 2 working CPUs. Server is running now with swapped CPU.  

(If you see some something that makes you believe that there is a serious problem with cable management or lack of it - is a possibility to be some random errors in photo ) 

In the picture is shown CPU error message, before BIOS upgrade. 

Just to confirm, I never saw running on this R530 server two CPUs. When I received the pair of CPUs, I removed the protective cap from CPU2 socket and just installed. Never worked, even in single DIMM (A1/B1) memory placement.  

Tried various memory setups that where more likely to work, including your suggestions. 

	CPU1: A1, A2 - 32GB DDR4 2666  2R x4 RDIMM  CPU2: B1, B2 - 32 GB DDR4 2666 2R x4 RDIMM Red Light on the right: SD Card module In front, memory modules of 16 GB DDR4 at 2133 MT/s 16 DDR4 RDIMM Samsung PN

 This post is not complete. I'll add more information on it later. 

This is a continuation of the previous post. 

To see all the pictures, it might be necessary to scroll left or right. 

Roate / Crop HiRes - R530 memory

I cropped and rotate the original photo. In previous picture you could not see the actual DDR RAM manufacturer and model/specs. 

Thank you!

Hello @FelixTheCat_,

Sadly, I cannot test on the lab because with the COVID-19 situation the installations are closed. Today I was reading some documentation that was worryng me about if the configuration is supported in terms of the amount of cores. Please, check this: 

"2x Intel® Xeon® E5–2600 v3 processors (14 cores maximum each CPU; 28 total maximum cores) or E5–2600 v4 processors (18 cores maximum each CPU; 36 total maximum cores)" (screenshot attached)

As you can see here, it says that for an E5–2600 v4 processors family only 18 x 2 cores are supported in dual processor configuration (total 36). As far as I know the E5-2699v4 is 22 cores. So that would make a total of 44 cores.

Can you check in BIOS (System Setup) (F2 on boot), and then System Setup Main Menu / System BIOS / Processor Settings and check the option "Number of Cores per Processor". This option controls the number of enabled cores in each processor. By default, the number of Cores per Processor option is set to All.  We are looking here for a specific amount for this configuration. For example, in my demo enviorement for a R730, the maximun is 12. Can you check and let me know if you hace a 18 cores maximun?

Regards.