iDRAC problem on Dell R610

Same issue here too. R610, arrived yesterday, iDRAC6 Express. I don't remember firmware version, but BIOS version was 3.0.0 (which released few days after latest iDRAC firmware 1.70, so I suppose I've latest firmware version). The setup is same as described by CLXADMIN.

Additionally I've tried to manually add ARP records for iDRAC MAC address and tried to access it from next server connected to R610 with 2 NIC ports (eth0 using hosting's switch connected to internet and eth1 using our local unmanaged switch connected only to our servers) - no luck. iDRAC never replied even on ping.

I've also tried to temporary set iDRAC's IP address as eth0:0 alias in OS booted from LiveCD - I was able to connect to OS using that IP, so this isn't routing/network configuration issue.

Here is my version info:

BIOS Revision: 3.0.0

Broadcom…: v6.0.11

MPT2BIOS-7.11.01.00

iDRAC6 Configuration Utility: 1.59

IDRAC6 Firmware Revision: 1.70.21

Lifecycle Controller version: 1.5.0.671

Primary Backplane Firmware Revision: 1.07

I've just reset iDRAC6 to factory defaults, then change: IPv4's IP+gateway+DNS(8.8.8.8), domain and host names (to existing name pointing to iDRAC's IP), and LAN user's password - nothing else was changed. Reboot several times using KVM's Ctrl-Alt-Del - iDRAC doesn't reply to ping.

The idrac express is an extention of the original BMC standard, it does not contain a web interface etc. To use it you need ipmi shell <ADMIN NOTE: Broken link has been removed from this post by Dell> you can find this on the web or the dell cd's.

I don't have neither Windows nor Dell CD. Even if it doesn't contain web interface, isn't it must reply to ping anyway?

Ok, I've tried ipmitool (IPMI over Lan is enabled in DRAC contifiguration, password is also set) on next server in same rack:

# ipmitool -v -v -I lan -H _m_._y_._i_._p_ -U root -P _my_pass_ power reset
ipmi_lan_send_cmd:opened=[0], open=[269004796]
IPMI LAN host 78.129.210.127 port 623
Sending IPMI/RMCP presence ping packet
ipmi_lan_send_cmd:opened=[1], open=[269004796]
No response from remote controller
Get Auth Capabilities command failed
ipmi_lan_send_cmd:opened=[1], open=[269004796]
No response from remote controller
Get Auth Capabilities command failed
Error: Unable to establish LAN session
Failed to open LAN interface
Unable to set Chassis Power Control to Reset
#

As for me, looks like either hardware or firmware issue.

I wasn't trying to access the web interface, the IPMI commands will do but something to do with networking isn't working on this iDRAC. FWIW, when I used ipmitool on the host with the problematic iDRAC I'm able to query it fine. It's just when I try and do it over the network that it doesn't work, because the networking doesn't work on the iDRAC for some reason.

When it's set to DHCP it doesn't send out any DHCP requests, I've had it connected directly to another server with tcpdump running - nothing shows up. And also when it's configured with a static IP address it doesn't respond to ARP requests, this has nothing to do with it having a web interface or not.

So could this be a hardware or software issue? How do you go about getting this type of issue resolved?

Actually, the iDRAC 6 Express does have a web interface.  Go into the Ctrl+E setup utility during POST, reset it to factory defaults, then give it a unique static IP address that is in your normal subnet.  Try it with IE7 or 8, and see if it responds.  If not, then shut down the server, unplug all external cables, then drain flea power by holding in the power button for 10 seconds.  Plug everything back in and crank it up, then check the settings in Ctrl+E and redo that step if necessary, then see if it works with a browser. 

JeffLM, I've already tried as you suggested but just to be sure I thought I'd give it a go again. Same result as last time - it doesn't work. I configured it with an IP address that is in the same subnet as the other hosts that are on the same LAN segment.

Ping doesn't work nor does trying to establish a TCP connection to port 80/443. From a tcpdump I can quite clearly see that the problem is down to the iDRAC not responding to ARP requests. Without this a connection will never be established as the host initiating it doesn't know the MAC address of the host to send the packet to.

It is NIC1/eth0 that the iDRAC Express piggy backs isn't it?

Device ID                 : 32
Device Revision           : 0
Firmware Revision         : 1.70
IPMI Version              : 2.0
Manufacturer ID           : 674
Manufacturer Name         : DELL Inc
Product ID                : 256 (0x0100)
Product Name              : Unknown (0x100)
Device Available          : yes
Provides Device SDRs      : yes

Any other ideas of how I can get the iDRAC working? Surely it's got to be a lot simpler than this. I had nowhere near as many problems with a load of T300's. This has got to indicate a hardware/firmware issue??

Well, you could try reseating the card, but if that doesn't do it I reckon it might be broken.  Call Tech Support.

I'm now downloading RHEL to compare driver sources with my kernel.

Can you please confirm your system uses mpt2sas driver, and provide kernel boot output/dmesg related to this driver?

Thanks for the update powerman-asdf, will try and get a Dell Engineer out to replace the motherboard in our server.

We are using Linux with a H200 controller but we're running Red Hat 5.6 on, we've also had Red Hat 6.0 on and didn't have any noticeable problems. It's currently running kernel 2.6.33.9-rt31.64.el5rt, the non RealTime kernel was also fine. The install was done via PXE/KS/NFS.

I think the problem mentioned in the bug report relates to an Ubuntu/kernel problem and not the hardware.

Below is the dmesg output, we're only using the one SAS disk.

SCSI subsystem initialized
mpt2sas version 03.100.03.00 loaded
scsi0 : Fusion MPT SAS Host
mpt2sas 0000:03:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
mpt2sas 0000:03:00.0: setting latency timer to 64
mpt2sas0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (16454708 kB)
mpt2sas 0000:03:00.0: irq 61 for MSI/MSI-X
mpt2sas0: PCI-MSI-X enabled: IRQ 61
mpt2sas0: iomem(0xd92b0000), mapped(0xffffc90012c60000), size(65536)
mpt2sas0: ioport(0xfc00), size(256)
mpt2sas0: sending diag reset !!
mpt2sas0: diag reset: SUCCESS
mpt2sas0: Allocated physical memory: size(1154 kB)
mpt2sas0: Current Controller Queue Depth(499), Max Controller Queue Depth(2607)
mpt2sas0: Scatter Gather Elements per IO(128)
mpt2sas0: LSISAS2008: FWVersion(07.15.04.00), ChipRevision(0x03), BiosVersion(07.11.01.00)
mpt2sas0: Dell PERC H200 Integrated: Vendor(0x1000), Device(0x0072), SSVID(0x1028), SSDID(0x1F1E)
mpt2sas0: Protocol=(Initiator,Target), Capabilities=(Raid,TLR,EEDP,Snapshot Buffer,Diag Trace Buffer,Task Set Full,NCQ)
mpt2sas0: sending port enable !!
mpt2sas0: host_add: handle(0x0001), sas_addr(0x5782bcb0162adb00), phys(8)
mpt2sas0: failure at drivers/scsi/mpt2sas/mpt2sas_scsih.c:3911/_scsih_add_device()!
mpt2sas0: port enable: SUCCESS
scsi 0:0:0:0: Direct-Access     FUJITSU  MBD2147RC        D809 PQ: 0 ANSI: 5
scsi 0:0:0:0: SSP: handle(0x000a), sas_addr(0x500000e11716a7d2), device_name(0xe1000050d1a71617)
scsi 0:0:0:0: SSP: enclosure_logical_id(0x5782bcb0162adb00), slot(0)
scsi 0:0:0:0: qdepth(254), tagged(1), simple(1), ordered(0), scsi_level(6), cmd_que(1)
sd 0:0:0:0: [sda] 286749480 512-byte logical blocks: (146 GB/136 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: c7 00 00 08
sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
 sda: sda1 sda2 sda3
sd 0:0:0:0: [sda] Attached SCSI disk

[root@server ~]# modinfo mpt2sas
filename:       /lib/modules/2.6.33.9-rt31.64.el5rt/kernel/drivers/scsi/mpt2sas/mpt2sas.ko
version:        03.100.03.00
license:        GPL
description:    LSI MPT Fusion SAS 2.0 Device Driver
author:         LSI Corporation

I finally got around to getting a Dell Engineer out to replace the motherboard, and would you believe, problem sorted.

I got a follow up email from Dell and they said that they've seen this issue with a number of customers and that the commonality between them is that they were all running iDRAC firmware 1.70. They also noticed that it took a long time to downgrade the firmware and had to be done offline. It's probably best to avoid that firmware.