Help Me Choose: Memory

 

The new DDR3 memory technology offers significant advantages over the previous DDR2 FBD technologies:
  • Each CPU has three separate memory controller hubs (MCHs) within the CPU package; memory transactions no longer need to transfer between the CPU and another device to reach RAM.
  • DDR3 memory offers higher bandwidth and lower power consumption compared to the previous DDR2 FBD technologies.
Intel® Xeon® 5500 and 5600 Series processors support two types of DDR3 memory:
  • UDIMMs: Unbuffered DIMMs
  • RDIMMs: Registered DIMMs
The register allows RDIMMs to run at higher frequencies and support more DIMMs within a memory channel.

Memory Comparison

 UDIMM UDIMM
Register/Buffer?  No Yes
 Frequencies 800, 1600, 1333MHz 800, 1600, 1333MHz
 Ranks Supported 1 or 2 1, 2 or 4
 Capacity per DIMM 1 or 2GB 1, 2, 4, 8 or 16GB
 Max # DIMMs per Channel 2  3
 DRAM technology x8 x4 or x8
 Temperature Sensor Yes Yes
 ECC Yes Yes
 SDDC Yes Yes
 Address Parity NoYes 

RDIMMs are a good choice for customers who need:
  • Large memory capacity (up to 16GB DIMMs)
  • A broader future memory-expansion road map due to the ability to achieve three DIMMs per channel
  • The latest RAS features (address parity)
UDIMMs are a good choice for customers who need:
  • A limited amount of memory
  • Power and cost savings
Power Considerations
RDIMMs use about 1W of power more per DIMM than compared to UDIMMs due to the register feature. However, the register will allow for performance improvements when the memory is being highly utilized and more than one DIMM per channel is populated. Both UDIMMs and RDIMMs use less power than DDR2 FBD DIMMs.

Speed and Population Dependency
Due to technology limitations, frequency support has some dependency on the DIMMS per channel and the ranks used in a DIMM. For example, one population restriction is that any quad-rank DIMM must be the first DIMM installed in a channel; other than that, the ranks may be mixed.

DIMM Type DIMM 0DIMM 1DIMM 2 # of DIMMs 80010661333
 SR  Yes
Yes Yes
 DR  YesYesYes
UDIMMSR SR  Yes
Yes
No
 SR SR  Yes
Yes
No
 DRDR  Yes
Yes
No
        
DIMM TypeDIMM 0DIMM 1 DIMM 2 # of DIMMs 8001066 1333
 SR  1Yes
Yes
Yes
 DR  1YesYesYes
 QRSR 2YesYesNo
 SRSR 2Yes
Yes
Yes
 SRDR 2Yes
Yes
Yes
 DRDR  2Yes
Yes
Yes
RDIMMQRSR 2Yes
No
No
 QRDR 2Yes
No
No
 QRQR 2Yes
No
No
 SRSRSR3Yes
No
No
 SRSRDR3Yes
No
No
 SRDRDR3Yes
No
No
 DRDR DR 3Yes
No
No

 

Each CPU has three integrated MCHs that have their own memory channel. Memory accesses can be interleaved across the CPU nodes or the system can be configured in nonuniform memory architecture (NUMA). This is the default BIOS configuration setting from the factory.

Memory Optimized Mode

In this mode, the MCHs run independently of each other; for example, one can be idle, one can be performing a write operation and the other can be preparing for a read operation. Memory may be installed in one, two or three channels. To fully realize the performance benefit of the memory optimized mode, all three channels per CPU should be populated. To maximize performance, it is better to add DIMMs across channels before adding more DIMMs to a channel that is already populated.

Max Bandwidth Performance
First bank fully populated with DDR3 1333MHz per MCH
  • Max capacity 48GB (96GB capacity can be reached with 16GB DIMMs, memory speed will be 1066MHz)
  • Must have Intel® Xeon® 5550 Series processor and up 
  • Two DIMMs per channel will be supported with BIOS version 1.1.4

Balanced Performance
First and second banks fully populated with DDR3 1066MHz per MCH

  • Max capacity 96GB (192GB capacity can be reached with 16GB DIMMs, memory speed will be 800MHz)
  • Must have Intel® Xeon® 5520 Series processor and up
  • Support for 1333MHz for first and second banks coming soon, requires Intel Xeon 5550 Series processor and up (Coming Soon)

Maximum Capacity
First, second and third banks fully populated with DDR3 800Mhz per MCH

  • Max capacity 144GB
  • Must have Intel Xeon 5500/5600 Series processor and up

Advanced ECC Mode
This RAS-memory mode uses two MCHs and "binds" them together to emulate a 128-bit data bus DIMM. This is primarily used to achieve a single device data correction (SDDC) for DIMMs based on x8 DRAM technology. SDDC is supported with x4-based DIMMs in every memory mode. One MCH is completely unutilized and any memory installed in this channel will generate a warning message during POST.

Mirror Mode
This RAS-memory mode uses two of the three channels. Identical data writes are performed on each channel and reads alternate between the two channels. If excessive memory errors are detected on a channel, that channel is disabled, and the system uses the other channel for future reads and writes. This is essentially a data backup and allows the system to continue running even during a catastrophic DIMM failure on a channel. Since the memory is copied between DIMMs, the operating system will only see and report half of the installed memory.

Max Bandwidth Performance RAS
First bank fully populated with DDR3 1333MHz per MCH (Only 2 MCHs can be utilized)

  • Max capacity 32GB (64GB capacity can be reached with 16GB DIMMs, memory speed will be 1066MHz, must have Intel Xeon 5520 Series processor and up)
  • Mirror Mode 16GB (32GB capacity can be reached with 16GB DIMMs, memory speed will be 1066MHz, must have Intel Xeon 5520 Series processor and up)
  • Must have Intel Xeon 5550 Series processor and up
  • Two DIMMs per channel will be supported with BIOS version 1.1.4

Balanced Performance RAS
First and second banks fully populated with DDR3 1066MHz per MCH (Only 2 MCHs can be utilized)

  • Max capacity 64GB (128GB capacity can be reached with 16GB DIMMs, memory speed will be 800MHz, must have Intel Xeon 5500 Series processor and up)
  • Mirror Mode 32GB (64GB capacity can be reached with 16GB DIMMs, memory speed will be 1066MHz, must have Intel Xeon 5500 Series processor and up)
  • Must have Intel Xeon 5550 Series processor and up
  • Support for 1333MHz for first and second banks, requires Intel Xeon 5550 Series processor and up (Coming Soon)

Maximum Capacity RAS
First, second and third banks fully populated with DDR3 800MHz per MCH (Only 2 MCHs can be utilized)

  • Max capacity 96GB
  • Mirror Mode 48GB
  • Must have Intel Xeon 5500/5600 Series Processor and up

 

 

Selection Criteria
When considering memory options, the five key considerations are price, performance, power, RAS features and scalability.
  • Performance = RDIMM
  • Power = UDIMM
  • RAS features = RDIMM
  • Scalability = RDIMM
  • Price = UDIMM

Memory Performance
Memory performance is affected by CPU speed, memory ranks and DIMMs per channel.
  • DDR3 1333MHz is only supported in one DIMM per channel per MCH; two DIMMs per channel will be supported with BIOS version 1.1.4
  • DDR3 1066MHz is only supported in two DIMMs per channel per MCH, once third DIMM is populated, memory speed will clock down to 800MHz
  • Max capacity memory speed is 800MHz

For maximum memory performance:

  • Use DDR3 1333MHz memory across all three channels per CPU
  • This speed can only support one DIMM per channel and thus a maximum of three DIMMs per CPU. Two DIMMs per channel will be supported with BIOS version 1.1.4
  • No quad-rank DIMMs may be used

For balanced memory performance:
  • Use DDR3 1066MHz memory across all three channels per CPU
  • This speed can only support two DIMMs per channel and thus a maximum of six DIMMs per CPU

For maximum memory capacity:
  • Use DDR3 800MHz memory across all three channels per CPU
  • This speed will support 3 DIMMs per channel and thus a maximum of nine DIMMs per CPU
Address Parity
The ability to detect an error on a single address line. This is not correctable.

Advanced ECC Mode
This mode uses two MCHs and "ties" them together to emulate a 128-bit data bus DIMM. This is primarily used to achieve a single device data correction (SDDC) for DIMMs based on x8 DRAM technology. SDDC is supported with x4-based DIMMs in every memory mode. One MCH is completely unutilized and any memory installed in this channel will generate a warning message during POST.

DDR3
Double Data Rate 3, memory technology.

DIMM
Dual Inline Memory Module. This is the memory "stick" that is installed in each memory slot. It is comprised of multiple memory chips and, in some cases, registers, buffers and/or temperature sensors.

ECC
This mode uses two MCHs and "ties" them together to emulate a 128-bit data bus DIMM. This is primarily used to achieve a SDDC for DIMMs based on x8 DRAM technology. SDDC is supported with x4-based DIMMs in every memory mode. One MCH is completely unutilized and any memory installed in this channel will generate a warning message during POST.

MCH
Memory Controller Hub

Memory Optimized Mode
In this mode, the MCHs run independently of each other; for example, one can be idle, one can be performing a write operation and the other can be preparing for a read operation. Memory may be installed in one, two or three channels. To fully realize the performance benefit of the memory optimized mode, all three channels per CPU should be populated. This implies that some "atypical" memory configurations, such as 3GB, 6GB or 12GB, will yield the best performance. This is the recommended mode unless specific RAS features are needed.

Mirror Mode
One of the RAS-memory modes is mirror mode. This mode uses two of the three channels. Identical data writes are performed on each channel and reads alternate between the two channels. If excessive memory errors are detected on a channel, then that channel is disabled, and the system uses the other channel for future reads and writes. This is essentially a data backup and allows the system to continue running even during a catastrophic DIMM failure on a channel. Since the memory is copied between DIMMs, the operating system will only see and report half of the installed memory.

Mirroring
The memory controller is configured to allow the same data to be written to two banks of DIMMs. Each bank's data are identical to the other; thus, if one bank fails or has multiple bit errors, there is a backup data bank. The operating system will report half of your installed memory.

Rank
Number of 64-bit wide data areas.

RAS
Reliability, Availability and Serviceability.

SDDC 
Memory systems that utilize SDCC can detect and correct multibit errors that come from a single memory chip on the DIMM.

Temperature Sensors
All DIMMs have integrated temperature sensors. This allows the server to monitor for over temperature conditions with the resolution of individual DIMMs. This can provide more efficient thermal and power management.