Unsolved
This post is more than 5 years old
5 Posts
0
37497
DELL PowerEdge servers with 4GB RAM, but OS can't use all of this memory.
Description of problem: We have few DELL PowerEdge servers with 4GB RAM, but OS can't use all of this memory. PowerEdge 6650 -------------------- 6650_1:~> free total Mem: 3375272 6650_1:~> lspci 00:00.0 Host bridge: Broadcom CMIC-HE (rev 22) 00:00.1 Host bridge: Broadcom CMIC-HE 00:00.2 Host bridge: Broadcom CMIC-HE 00:00.3 Host bridge: Broadcom CMIC-HE 00:04.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27) 00:05.0 Class ff00: Dell Remote Access Card III 00:05.1 Class ff00: Dell Remote Access Card III 00:05.2 Class ff00: Dell Remote Access Card III: BMC/SMIC device not present 00:0f.0 Host bridge: Broadcom CSB5 South Bridge (rev 93) 00:0f.1 IDE interface: Broadcom CSB5 IDE Controller (rev 93) 00:0f.2 USB Controller: Broadcom OSB4/CSB5 OHCI USB Controller (rev 05) 00:0f.3 ISA bridge: Broadcom CSB5 LPC bridge 00:10.0 Host bridge: Broadcom CIOB30 (rev 03) 00:10.2 Host bridge: Broadcom CIOB30 (rev 03) 00:11.0 Host bridge: Broadcom CIOB30 (rev 03) 00:11.2 Host bridge: Broadcom CIOB30 (rev 03) 00:12.0 Host bridge: Broadcom CIOB30 (rev 03) 00:12.2 Host bridge: Broadcom CIOB30 (rev 03) 03:01.0 PCI bridge: Intel Corporation 21154 PCI-to-PCI Bridge 04:00.0 PCI bridge: Intel Corporation 21154 PCI-to-PCI Bridge 04:01.0 SCSI storage controller: QLogic Corp. ISP12160 Dual Channel Ultra3 SCSI Processor (rev 06) 05:00.0 RAID bus controller: American Megatrends Inc. MegaRAID (rev 20) 0a:01.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5700 Gigabit Ethernet (rev 14) 0a:02.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5700 Gigabit Ethernet (rev 14) 15:01.0 PCI bridge: Intel Corporation 21154 PCI-to-PCI Bridge 16:00.0 PCI bridge: Intel Corporation 21154 PCI-to-PCI Bridge 16:01.0 SCSI storage controller: QLogic Corp. ISP12160 Dual Channel Ultra3 SCSI Processor (rev 06) 17:00.0 RAID bus controller: American Megatrends Inc. MegaRAID (rev 20) -------------------- 6650_2 :~> free total Mem: 3635360 6650_2:~> lspci 00:00.0 Host bridge: Broadcom CMIC-HE (rev 22) 00:00.1 Host bridge: Broadcom CMIC-HE 00:00.2 Host bridge: Broadcom CMIC-HE 00:00.3 Host bridge: Broadcom CMIC-HE 00:04.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27) 00:05.0 Class ff00: Dell Remote Access Card III 00:05.1 Class ff00: Dell Remote Access Card III 00:05.2 Class ff00: Dell Remote Access Card III: BMC/SMIC device not present 00:0f.0 Host bridge: Broadcom CSB5 South Bridge (rev 93) 00:0f.1 IDE interface: Broadcom CSB5 IDE Controller (rev 93) 00:0f.3 ISA bridge: Broadcom CSB5 LPC bridge 00:10.0 Host bridge: Broadcom CIOB30 (rev 03) 00:10.2 Host bridge: Broadcom CIOB30 (rev 03) 00:11.0 Host bridge: Broadcom CIOB30 (rev 03) 00:11.2 Host bridge: Broadcom CIOB30 (rev 03) 00:12.0 Host bridge: Broadcom CIOB30 (rev 03) 00:12.2 Host bridge: Broadcom CIOB30 (rev 03) 03:01.0 PCI bridge: Intel Corporation 21154 PCI-to-PCI Bridge 04:00.0 PCI bridge: Intel Corporation 21154 PCI-to-PCI Bridge 04:01.0 SCSI storage controller: QLogic Corp. ISP12160 Dual Channel Ultra3 SCSI Processor (rev 06) 05:00.0 RAID bus controller: American Megatrends Inc. MegaRAID (rev 20) 0a:01.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5700 Gigabit Ethernet (rev 14) 0a:02.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5700 Gigabit Ethernet (rev 14) PowerEdge 2560 -------------------- :~> free total Mem: 3895444 :~> lspci 00:00.0 Host bridge: Broadcom CMIC-WS Host Bridge (GC-LE chipset) (rev 13) 00:00.1 Host bridge: Broadcom CMIC-WS Host Bridge (GC-LE chipset) 00:00.2 Host bridge: Broadcom CMIC-LE 00:04.0 Class ff00: Dell Embedded Remote Access or ERA/O 00:04.1 Class ff00: Dell Remote Access Card III 00:04.2 Class ff00: Dell Embedded Remote Access: BMC/SMIC device 00:0e.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27) 00:0f.0 Host bridge: Broadcom CSB5 South Bridge (rev 93) 00:0f.1 IDE interface: Broadcom CSB5 IDE Controller (rev 93) 00:0f.2 USB Controller: Broadcom OSB4/CSB5 OHCI USB Controller (rev 05) 00:0f.3 ISA bridge: Broadcom CSB5 LPC bridge 00:10.0 Host bridge: Broadcom CIOB-X2 PCI-X I/O Bridge (rev 03) 00:10.2 Host bridge: Broadcom CIOB-X2 PCI-X I/O Bridge (rev 03) 00:11.0 Host bridge: Broadcom CIOB-X2 PCI-X I/O Bridge (rev 03) 00:11.2 Host bridge: Broadcom CIOB-X2 PCI-X I/O Bridge (rev 03) 03:06.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5701 Gigabit Ethernet (rev 15) 03:08.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5701 Gigabit Ethernet (rev 15) 04:08.0 PCI bridge: Intel Corporation 80303 I/O Processor PCI-to-PCI Bridge (rev 01) 04:08.1 RAID bus controller: Dell PowerEdge Expandable RAID Controller 3/Di (rev 01) Version-Release number of selected component: all servers listed above uses RH4U4 with 2.6.9-42.ELsmp #1 SMP Sat Aug 12 09:39:11 CDT 2006 i686 i686 i386 GNU/Linux but we've checked amount of memory using memtest86 and shows the same amount of memory as stated above Actual results: non all of memory can be used by system Expected results: all memory should be accessible by system Additional info: When i removed some unused PCIX cards I freed 256MB of memory (see difference between 6650_1 and 6650_2) I've found some hardware related info: Server Products Not All Memory is Available after Installing 4GB or More of System Memory (http://support.intel.com/support/motherboards/server/sb/CS-010458.htm) Not All Physical Memory May Be Reported By The Operating System On Certain HP ProLiant Servers http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?objectID=PSD_EL041214_CW01 but only info at DELL sites that i've found suggests to use /3GB and PAE switch with Windows.... and we are using RHEL 4U4, Centos4 and other linux like :) btw. DELL PE1850 doesn't have this problem 1850:~> free total Mem: 4147184
TJaszo
5 Posts
0
July 12th, 2007 12:00
Description of problem:
We have few DELL PowerEdge servers with 4GB RAM, but OS can't use all of this
memory.
PowerEdge 6650
--------------------
6650_1:~> free
total
Mem: 3375272
6650_1:~> lspci
00:00.0 Host bridge: Broadcom CMIC-HE (rev 22)
00:00.1 Host bridge: Broadcom CMIC-HE
00:00.2 Host bridge: Broadcom CMIC-HE
00:00.3 Host bridge: Broadcom CMIC-HE
00:04.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27)
00:05.0 Class ff00: Dell Remote Access Card III
00:05.1 Class ff00: Dell Remote Access Card III
00:05.2 Class ff00: Dell Remote Access Card III: BMC/SMIC device not present
00:0f.0 Host bridge: Broadcom CSB5 South Bridge (rev 93)
00:0f.1 IDE interface: Broadcom CSB5 IDE Controller (rev 93)
00:0f.2 USB Controller: Broadcom OSB4/CSB5 OHCI USB Controller (rev 05)
00:0f.3 ISA bridge: Broadcom CSB5 LPC bridge
00:10.0 Host bridge: Broadcom CIOB30 (rev 03)
00:10.2 Host bridge: Broadcom CIOB30 (rev 03)
00:11.0 Host bridge: Broadcom CIOB30 (rev 03)
00:11.2 Host bridge: Broadcom CIOB30 (rev 03)
00:12.0 Host bridge: Broadcom CIOB30 (rev 03)
00:12.2 Host bridge: Broadcom CIOB30 (rev 03)
03:01.0 PCI bridge: Intel Corporation 21154 PCI-to-PCI Bridge
04:00.0 PCI bridge: Intel Corporation 21154 PCI-to-PCI Bridge
04:01.0 SCSI storage controller: QLogic Corp. ISP12160 Dual Channel Ultra3 SCSI
Processor (rev 06)
05:00.0 RAID bus controller: American Megatrends Inc. MegaRAID (rev 20)
0a:01.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5700 Gigabit
Ethernet (rev 14)
0a:02.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5700 Gigabit
Ethernet (rev 14)
15:01.0 PCI bridge: Intel Corporation 21154 PCI-to-PCI Bridge
16:00.0 PCI bridge: Intel Corporation 21154 PCI-to-PCI Bridge
16:01.0 SCSI storage controller: QLogic Corp. ISP12160 Dual Channel Ultra3 SCSI
Processor (rev 06)
17:00.0 RAID bus controller: American Megatrends Inc. MegaRAID (rev 20)
--------------------
6650_2 :~> free
total
Mem: 3635360
6650_2:~> lspci
00:00.0 Host bridge: Broadcom CMIC-HE (rev 22)
00:00.1 Host bridge: Broadcom CMIC-HE
00:00.2 Host bridge: Broadcom CMIC-HE
00:00.3 Host bridge: Broadcom CMIC-HE
00:04.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27)
00:05.0 Class ff00: Dell Remote Access Card III
00:05.1 Class ff00: Dell Remote Access Card III
00:05.2 Class ff00: Dell Remote Access Card III: BMC/SMIC device not present
00:0f.0 Host bridge: Broadcom CSB5 South Bridge (rev 93)
00:0f.1 IDE interface: Broadcom CSB5 IDE Controller (rev 93)
00:0f.3 ISA bridge: Broadcom CSB5 LPC bridge
00:10.0 Host bridge: Broadcom CIOB30 (rev 03)
00:10.2 Host bridge: Broadcom CIOB30 (rev 03)
00:11.0 Host bridge: Broadcom CIOB30 (rev 03)
00:11.2 Host bridge: Broadcom CIOB30 (rev 03)
00:12.0 Host bridge: Broadcom CIOB30 (rev 03)
00:12.2 Host bridge: Broadcom CIOB30 (rev 03)
03:01.0 PCI bridge: Intel Corporation 21154 PCI-to-PCI Bridge
04:00.0 PCI bridge: Intel Corporation 21154 PCI-to-PCI Bridge
04:01.0 SCSI storage controller: QLogic Corp. ISP12160 Dual Channel Ultra3 SCSI
Processor (rev 06)
05:00.0 RAID bus controller: American Megatrends Inc. MegaRAID (rev 20)
0a:01.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5700 Gigabit
Ethernet (rev 14)
0a:02.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5700 Gigabit
Ethernet (rev 14)
PowerEdge 2560
--------------------
:~> free
total
Mem: 3895444
:~> lspci
00:00.0 Host bridge: Broadcom CMIC-WS Host Bridge (GC-LE chipset) (rev 13)
00:00.1 Host bridge: Broadcom CMIC-WS Host Bridge (GC-LE chipset)
00:00.2 Host bridge: Broadcom CMIC-LE
00:04.0 Class ff00: Dell Embedded Remote Access or ERA/O
00:04.1 Class ff00: Dell Remote Access Card III
00:04.2 Class ff00: Dell Embedded Remote Access: BMC/SMIC device
00:0e.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27)
00:0f.0 Host bridge: Broadcom CSB5 South Bridge (rev 93)
00:0f.1 IDE interface: Broadcom CSB5 IDE Controller (rev 93)
00:0f.2 USB Controller: Broadcom OSB4/CSB5 OHCI USB Controller (rev 05)
00:0f.3 ISA bridge: Broadcom CSB5 LPC bridge
00:10.0 Host bridge: Broadcom CIOB-X2 PCI-X I/O Bridge (rev 03)
00:10.2 Host bridge: Broadcom CIOB-X2 PCI-X I/O Bridge (rev 03)
00:11.0 Host bridge: Broadcom CIOB-X2 PCI-X I/O Bridge (rev 03)
00:11.2 Host bridge: Broadcom CIOB-X2 PCI-X I/O Bridge (rev 03)
03:06.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5701 Gigabit
Ethernet (rev 15)
03:08.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5701 Gigabit
Ethernet (rev 15)
04:08.0 PCI bridge: Intel Corporation 80303 I/O Processor PCI-to-PCI Bridge (rev 01)
04:08.1 RAID bus controller: Dell PowerEdge Expandable RAID Controller 3/Di (rev 01)
Version-Release number of selected component:
all servers listed above uses RH4U4 with
2.6.9-42.ELsmp #1 SMP Sat Aug 12 09:39:11 CDT 2006 i686 i686 i386 GNU/Linux
but we've checked amount of memory using memtest86 and shows the same amount of memory as stated above
Actual results:
non all of memory can be used by system
Expected results:
all memory should be accessible by system
Additional info:
When i removed some unused PCIX cards I freed 256MB of memory (see
difference between 6650_1 and 6650_2)
I've found some hardware related info:
Server Products
Not All Memory is Available after Installing 4GB or More of System Memory
(http://support.intel.com/support/motherboards/server/sb/CS-010458.htm)
Not All Physical Memory May Be Reported By The Operating System On Certain HP
ProLiant Servers
http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?objectID=PSD_EL041214_CW01
but only info at DELL sites that i've found suggests to use /3GB and PAE switch
with Windows.... and we are using RHEL 4U4, Centos4 and other linux like :)
btw. DELL PE1850 doesn't have this problem
1850:~> free
total
Mem: 4147184
mmainprize
260 Posts
0
July 18th, 2007 04:00
TJaszo
5 Posts
0
July 18th, 2007 05:00
"
Version-Release number of selected component:
all servers listed above uses RH4U4 with
2.6.9-42.ELsmp #1 SMP Sat Aug 12 09:39:11 CDT 2006 i686 i686 i386 GNU/Linux
"
And with exactly the same version of linux installed on PowerEdge 1850 shows all 4GB of RAM... so it's rather not OS issue.
"
DELL PE1850 doesn't have this problem
1850:~> free
total
Mem: 4147184
"
mmainprize
260 Posts
0
July 19th, 2007 04:00
Memory
Architecture
72-bit ECC PC-1600 compliant, DDR SDRAM registered DIMMs, with 4-way interleaving
Memory module sockets
sixteen 72-bit wide, 168-pin DIMM sockets on two memory riser cards
Memory module capacities
128-, 256-, 512-MB, or 1-GB registered SDRAM DIMMs
Minimum RAM
512 MB
Maximum RAM
16 GB
Message Edited by mmainprize on 07-19-2007 12:35 AM
Message Edited by mmainprize on 07-19-2007 12:36 AM
kghammond
20 Posts
0
July 27th, 2007 05:00
TJaszo
5 Posts
0
July 27th, 2007 11:00
sorry, I know that all of You want to help, but please read question carefully...
I ask again:
does any one of You know how to handle memory problem in PE2650 and PE6650 in configuration stated above under linux system (RHEL4U4)
I know that i can buy Windows ... and use some magic tricks, but I just can't... We are using RHEL in very stable version with other software and we just added 2GB of RAM to improve performance... and it appeared that instead of 4GB we can use less...
we tried some software tricks, but i didn't helped. I guess that maybe there is some hardware (bios?) tricks that we can try... So thats why I'm asking about it on _this_ forum...
thanks
--
Tomasz
I wish all of You happy SysAdmin Day ;)
TJaszo
5 Posts
0
July 27th, 2007 12:00
Intel and HP in similar case advices to upgrade bios - but in Dell BIOS change log I do not see any update regarding memory remapping...
so maybe there is some switch in BIOS config that we are missing... :/
kghammond
20 Posts
0
July 27th, 2007 12:00