Start a Conversation

Solved!

Go to Solution

3719

October 17th, 2021 22:00

S3048-ON died after a shutdown and reboot

I have 2 S3048-ON in stack, and after 1 year without reboot, we turn off and turn on the switches, and one switch dont back more. On startup switch show errors and reboot again and again...

I try to uninstall ONIE and install again, but I cant install because receive errors about cannot read /dev/i2c-2, unable to found serial number  and not found base mac address

Ideas about this problem?

 

errors on boot:

 

Dell EMC Networking OS Release 9.14(2.4)
NetBSD 7.1.2 (S3000) #0: Fri Nov 8 09:11:45 PST 2019
NetBSD 7.1.2 (S3000) Notice: this software is protected by copyright
Detecting hardware...: EMC230x RPM based fan controller 0
netbsd_bde_create() failed
done.
WARNING: 5 errors while detecting hardware; check system log.
ifconfig: exec_matches: Device not configured
route: writing to routing socket: File exists

 

 

When I try to install FTOS (pendrive mounted):

ONIE:/ # onie-nos-install /mnt/pendrive/FTOS-SG-9.14.1.10.bin
discover: installer mode detected.
Stopping: discover... done.
Error: Could not open file `/dev/i2c-2': No such device
Error: Could not open file `/dev/i2c-2': No such device
Notice: Invalid TLV header found. Using default contents.
Notice: Invalid TLV checksum found. Using default contents.
ONIE: Unable to find 'Base MAC Address' TLV in EEPROM data.
Error: Could not open file `/dev/i2c-2': No such device
Error: Could not open file `/dev/i2c-2': No such device
Notice: Invalid TLV header found. Using default contents.
Notice: Invalid TLV checksum found. Using default contents.
ONIE: Unable to find 'Serial Number' TLV in EEPROM data.
ONIE: Executing installer: /mnt/pendrive/FTOS-SG-9.14.1.10.bin
/var/tmp/installer: line 1: FORCE10: not found
/var/tmp/installer: line 2: ▒x▒▒10▒▒91: not found
/var/tmp/installer: line 5: syntax error: unexpected word (expecting ")")
Failure: Unable to install image: /mnt/pendrive/FTOS-SG-9.14.1.10.bin

 

On run DIAG:


Dell Networking OS Release 2.0(0.3)
NetBSD 5.1_STABLE (S3000) #0: Tue Mar 14 02:23:07 PDT 2017
multiboot: Information structure flags: 0x00001a67
multiboot: Boot loader: GRUB 1.99~rc1
multiboot: Command line: console=com console_addr=0x2f8 console_speed=115200 -v onieftosdiag=yes ▒▒
multiboot: 619 KB lower memory, 2073132 KB upper memory
multiboot: Symbol table at 0xc3370214, length 837056 bytes
multiboot: String table at 0xc343c7d4, length 1355464 bytes
total memory = 2029 MB
avail memory = 1941 MB
Dell EMC S3000 (3.24.0.0-9)
mainbus0 (root)
FADT (revision 5) is longer than ACPI 2.0 version, truncating length 0x10C to 0xF4cpu0 at mainbus0 apid 0: Intel 686-class, 1750MHz, id 0x406d8
cpu1 at mainbus0 apid 2: Intel 686-class, 1750MHz, id 0x406d8
ioapic0 at mainbus0 apid 2
acpi0 at mainbus0: Intel ACPICA 20080321
APIC (PNP0003) at acpi0 not configured
hpet0 at acpi0 (HPET, PNP0103-0)hpet0: ACPI: unable to get _CRS resources: AE_NOT_FOUND
attimer0 at acpi0 (TIMR, PNP0100): io 0x40-0x43,0x50-0x53 irq 0
IUR3 (PNP0501) at acpi0 not configured
IUR4 (PNP0501) at acpi0 not configured
pci0 at mainbus0 bus 0: configuration mode 1
pchb0 at pci0
pchb0: vendor 0x8086 product 0x1f0f (rev. 0x02)
ppb0 at pci0: vendor 0x8086 product 0x1f10 (rev. 0x02)
ppb0: unsupported PCI Express version
pci1 at ppb0 bus 1
ppb1 at pci0: vendor 0x8086 product 0x1f11 (rev. 0x02)
ppb1: unsupported PCI Express version
pci2 at ppb1 bus 2
ppb2 at pci0: vendor 0x8086 product 0x1f12 (rev. 0x02)
ppb2: unsupported PCI Express version
pci3 at ppb2 bus 3
ppb3 at pci0: vendor 0x8086 product 0x1f13 (rev. 0x02)
ppb3: unsupported PCI Express version
pci4 at ppb3 bus 4
pchb1 at pci0
pchb1: vendor 0x8086 product 0x1f14 (rev. 0x02)
ismt0 at pci0 vendor 0x8086 product 0x1f15 (miscellaneous system, revision 0x02)
ismt0: Mapped SMBAR 0xdffcb004 size 0x400
ismt0: io_rng_dma = 0x53ed000
wm0 at pci0../../../../dev/pci/if_wm.c wm_attach:1301 Not found macaddr from bios!
: I354 Gigabit SGMII, rev. 3
wm0: interrupting at ioapic0 pin 21
Unable to get the mac-addr property!
GbE version identified as 0xffff
wm0: Ethernet address 64:00:6a:ce:d6:a0
wm1 at pci0../../../../dev/pci/if_wm.c wm_attach:1301 Not found macaddr from bios!
: I354 Gigabit SGMII, rev. 3
wm1: interrupting at ioapic0 pin 21
Unable to get the mac-addr property!
wm1: Ethernet address 00:e0:ec:25:b8:54
brgphy0 at wm1 phy 1: BCM54616S 1000BASE-T media interface, rev. 2
brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
OUI 0x001be9 model 0x0011 rev 2 at wm1 phy 2 not configured
OUI 0x001be9 model 0x0011 rev 2 at wm1 phy 3 not configured
OUI 0x001be9 model 0x0011 rev 2 at wm1 phy 4 not configured
OUI 0x001be9 model 0x0011 rev 2 at wm1 phy 5 not configured
OUI 0x001be9 model 0x0011 rev 2 at wm1 phy 6 not configured
OUI 0x001be9 model 0x0011 rev 2 at wm1 phy 7 not configured
ehci0 at pci0: vendor 0x8086 product 0x1f2c (rev. 0x02)
ehci0: interrupting at ioapic0 pin 23
usb0 at ehci0: USB revision 2.0
ahcisata0 at pci0: vendor 0x8086 product 0x1f22
ahcisata0: interrupting at ioapic0 pin 19
ahcisata0: AHCI revision 0x10300, 4 ports, 32 command slots, features 0xc3208000
atabus0 at ahcisata0 channel 0
atabus1 at ahcisata0 channel 1
atabus2 at ahcisata0 channel 2
atabus3 at ahcisata0 channel 3
ahcisata1 at pci0: vendor 0x8086 product 0x1f32
ahcisata1: interrupting at ioapic0 pin 19
ahcisata1: AHCI revision 0x10300, 2 ports, 32 command slots, features 0xc3308000
atabus4 at ahcisata1 channel 0
atabus5 at ahcisata1 channel 1
ichlpcib0 at pci0
ichlpcib0: vendor 0x8086 product 0x1f38 (rev. 0x02)
ichlpcib0: 24-bit timer
ichlpcib0: TCO timer reboot disabled by hardware; hope SMBIOS properly handles it.
gpio0 at ichlpcib0: 64 pins
ichsmb0 at pci0: vendor 0x8086 product 0x1f3c (rev. 0x02)
ichsmb0: interrupting at ioapic0 pin 18
iic0 at ichsmb0: I2C bus
i2cctl0 at iic0
iic1 at ismt0: I2C bus
i2cctl1 at iic1
iicmux0 at iic1 addr 0x73 : PCA9548
iicmuxchan0 at iicmux0 channel 0
iic2 at iicmuxchan0: I2C bus
i2cctl2 at iic2
s2eeprom0 at iic2 addr 0x50 : AT24Cxx compatible EEPROM
iicmuxchan1 at iicmux0 channel 1
iic3 at iicmuxchan1: I2C bus
i2cctl3 at iic3
max6699x0 at iic3 addr 0x1a max6699x0: : flags:0x10
iicmuxchan2 at iicmux0 channel 2
iic4 at iicmuxchan2: I2C bus
i2cctl4 at iic4
iicmuxchan3 at iicmux0 channel 3
iic5 at iicmuxchan3: I2C bus
i2cctl5 at iic5
iicmuxchan4 at iicmux0 channel 4
iic6 at iicmuxchan4: I2C bus
i2cctl6 at iic6
iicmuxchan5 at iicmux0 channel 5
iic7 at iicmuxchan5: I2C bus
i2cctl7 at iic7
iicmuxchan6 at iicmux0 channel 6
iic8 at iicmuxchan6: I2C bus
i2cctl8 at iic8
iicmux1 at iic8 addr 0x71 : PCA9548
iicmuxchan8 at iicmux1 channel 0
iic10 at iicmuxchan8: I2C bus
i2cctl10 at iic10
s2eeprom1 at iic10 addr 0x53 : AT24Cxx compatible EEPROM
iicmuxchan9 at iicmux1 channel 1
iic11 at iicmuxchan9: I2C bus
i2cctl11 at iic11
max6699x1 at iic11 addr 0x1a max6699x1: : flags:0x10
iicmuxchan10 at iicmux1 channel 2
iic12 at iicmuxchan10: I2C bus
i2cctl12 at iic12
s2eeprom2 at iic12 addr 0x52 : AT24Cxx compatible EEPROM
psufanctlx0 at iic12 addr 0x5a - PSU FAN controller (MB) 0
iicmuxchan11 at iicmux1 channel 3
iic13 at iicmuxchan11: I2C bus
i2cctl13 at iic13
s2eeprom3 at iic13 addr 0x53 : AT24Cxx compatible EEPROM
psufanctlx1 at iic13 addr 0x5b - PSU FAN controller (MB) 1
iicmuxchan12 at iicmux1 channel 4
iic14 at iicmuxchan12: I2C bus
i2cctl14 at iic14
sfpx0 at iic14 addr 0x50 : SFP eeprom at addr 0x50
sfpx4 at iic14 addr 0x51 : SFP eeprom at addr 0x51
iicmuxchan13 at iicmux1 channel 5
iic15 at iicmuxchan13: I2C bus
i2cctl15 at iic15
sfpx1 at iic15 addr 0x50 : SFP eeprom at addr 0x50
sfpx5 at iic15 addr 0x51 : SFP eeprom at addr 0x51
iicmuxchan14 at iicmux1 channel 6
iic16 at iicmuxchan14: I2C bus
i2cctl16 at iic16
sfpx2 at iic16 addr 0x50 : SFP eeprom at addr 0x50
sfpx6 at iic16 addr 0x51 : SFP eeprom at addr 0x51
iicmuxchan15 at iicmux1 channel 7
iic17 at iicmuxchan15: I2C bus
i2cctl17 at iic17
sfpx3 at iic17 addr 0x50 : SFP eeprom at addr 0x50
sfpx7 at iic17 addr 0x51 : SFP eeprom at addr 0x51
iicmux2 at iic8 addr 0x72 : PCA9548
iicmuxchan16 at iicmux2 channel 0
iic18 at iicmuxchan16: I2C bus
i2cctl18 at iic18
iicmuxchan17 at iicmux2 channel 1
iic19 at iicmuxchan17: I2C bus
i2cctl19 at iic19
iicmuxchan18 at iicmux2 channel 2
iic20 at iicmuxchan18: I2C bus
i2cctl20 at iic20
s2eeprom4 at iic20 addr 0x54 : AT24Cxx compatible EEPROM
iicmuxchan19 at iicmux2 channel 3
iic21 at iicmuxchan19: I2C bus
i2cctl21 at iic21
s2eeprom5 at iic21 addr 0x54 : AT24Cxx compatible EEPROM
iicmuxchan20 at iicmux2 channel 4
iic22 at iicmuxchan20: I2C bus
i2cctl22 at iic22
s2eeprom6 at iic22 addr 0x54 : AT24Cxx compatible EEPROM
iicmuxchan21 at iicmux2 channel 5
iic23 at iicmuxchan21: I2C bus
i2cctl23 at iic23
emc230x0 at iic23 addr 0x4d : EMC230x RPM based fan controller 0
iicmuxchan22 at iicmux2 channel 6
iic24 at iicmuxchan22: I2C bus
i2cctl24 at iic24
iicmuxchan23 at iicmux2 channel 7
iic25 at iicmuxchan23: I2C bus
i2cctl25 at iic25
iicmuxchan7 at iicmux0 channel 7
iic9 at iicmuxchan7: I2C bus
i2cctl9 at iic9
isa at ichlpcib0 not configured
isa0 at mainbus0
com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
com1 at isa0 port 0x2f8-0x2ff irq 3: ns16550a, working fifo
com1: console
npx0 at isa0 port 0xf0-0xff
mmcx0 at isa0 port 0x100-0x1fe iomem irq drq
smcx0 at isa0 port 0x200-0x2fe iomem irq drq
uhub0 at usb0: vendor 0x8086 EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
wd0 at atabus0 drive 0:
wd0: 7641 MB, 15525 cyl, 16 head, 63 sec, 512 bytes/sect x 15649200 sectors
uhub1 at uhub0 port 1: vendor 0x8087 product 0x07db, class 9/0, rev 2.00/0.02, addr 2
uhub1: single transaction translator
Too many symbols for tree, skipping 19533 symbols
bshell pseudo-device
diag as pseudo-device getting attached
nbsd_ubde pseudo-device
netbsd_bde_create() failed
bcnp pseudo-device
bcmw pseudo-device
bcmwstk pseudo-device
bcmws pseudo-device
f10logger as pseudo-device getting attached
bfd pseudo-device
F10 Generic RTC pseudo-device
platform pseudo-device
crshntfy pseudo-device
f10mc pseudo-device
boot device: wd0
root on md0a dumps on wd0l
WARNING: clock gained 1678 days
WARNING: CHECK AND RESET THE DATE!
ifconfig: exec_matches: Device not configured
route: writing to routing socket: File exists
ERROR BSD Parition using default offset 62073
Partitioning 7641mb flash ....Pass!
mount: cannot open `/dev/wd0b': Device not configured
Mounting /boot
Creating boot file
Creating env file
route: writing to routing socket: File exists
add host 127.10.10.13: gateway 127.10.10.13: File exists


RELEASE IMAGE HEADER DATA :
--------------------------
Release Image SW Image Count : 3
Release Image Major-Version : 2
Release Image Minor Version : 0
Release Image Maint Version : 0
Release Image Patch-Version : 3
Release Image Header Size : 140
Release Image Data Size : 20747932
Release Image Platform Type : 24
Release Image Product Name : FTOSDIAG-SG
Release Image Checksum Value : 0xda67465d
Release Image Header Checksum: 0x9db7c9d5
Release Image Create Year : 2017
Release Image Create Month : 3
Release Image Create Date : 14
Release Image Create Hour : 2
Release Image Create Minute : 25
Release Image Create Second : 9

SOFTWARE IMAGE HEADER DATA :
----------------------------
Software Image[1] Major Version : 2
Software Image[1] Minor Version : 0
Software Image[1] Maint Version : 0
Software Image[1] Patch Version : 3
Software Image[1] Image type : 2
Software Image[1] Header Size : 100
Software Image[1] Img Data Size : 11087280
Software Image[1] Img Orig Size : 11087280
Software Image[1] Family Code : cp
Software Image[1] Img file Name : CPRPLP-RPM-AP-2.0.0.3.bin
Software Image[1] Hdr Checksum : 0x80263a67
Software Image[1] Data Checksum : 0x32e144ef

SOFTWARE IMAGE HEADER DATA :
----------------------------
Software Image[2] Major Version : 2
Software Image[2] Minor Version : 0
Software Image[2] Maint Version : 0
Software Image[2] Patch Version : 3
Software Image[2] Image type : 2
Software Image[2] Header Size : 100
Software Image[2] Img Data Size : 1561684
Software Image[2] Img Orig Size : 4229136
Software Image[2] Family Code : nbsdcprplp
Software Image[2] Img file Name : NBSDCPRPLP-RPM-DIAG-2.0.0.3.bin
Software Image[2] Hdr Checksum : 0x94742c3b
Software Image[2] Data Checksum : 0x119e4757

SOFTWARE IMAGE HEADER DATA :
----------------------------
Software Image[3] Major Version : 2
Software Image[3] Minor Version : 0
Software Image[3] Maint Version : 0
Software Image[3] Patch Version : 3
Software Image[3] Image type : 2
Software Image[3] Header Size : 100
Software Image[3] Img Data Size : 8098668
Software Image[3] Img Orig Size : 23193616
Software Image[3] Family Code : nbsdlib
Software Image[3] Img file Name : NBSDLIB-RPM-AP-2.0.0.3.bin
Software Image[3] Hdr Checksum : 0x7b45348d
Software Image[3] Data Checksum : 0xdd707039

CPUINFO =
********************************************************
*** DEVELOPMENT: Enabling WIDE-OPEN telnet access ***
*** from 10.11 subnets see hosts.allow ***
********************************************************
********************************************************
*** DEVELOPMENT: inetd TELNET at port 370 ***
*** Dell Networking OS TELNET at port 23 ***
********************************************************

ERROR:i2c_write failed, bus=3, cs2eeprom1: s2eeprom_debug_read: address write failed at 0x0
hannel=5, i2c addr:0x4d offset:0s2eeprom6: s2eeprom_debug_read: address write failed at 0x0
x20 reglen:0x1 buflen:0x1, flagss2eeprom5: s2eeprom_debug_read: address write failed at 0x0
: 0, rv:-1
ERROR:Writing reg:0xs2eeprom4: s2eeprom_debug_read: address write failed at 0x0
20 of fan cntrl0 failed
ERROR:dev:0, initialization failed.
Helix4 initialization starting.....
ERROR:sfpPlus Module:1 is not present.
ERROR:sfpPlus Module:2 is not present.
ERROR:sfpPlus Module:3 is not present.
ERROR:sfpPlus Module:4 is not present.
Unit 0 is not attached
Unit 0 is not attached
Unit 0 is not attached
Unit 0 is not attached
Unit 0 is not attached
Unit 0 is not attached
Unit 0 is not attached

16 Posts

October 21st, 2021 11:00

Good news!!!!

Today, in a call with Christian Dinarte, a Dell Network specialist from Brazil, we do  update Bios of Switch, and after this update we reinstalled ONIE and FTOS and switch appers to be live again!

Thanks all for help!!

16 Posts

October 18th, 2021 09:00

S3048-ON died after a shutdown and reboot
 

I have 2 S3048-ON in stack, and after 1 year without reboot, we turn off and turn on the switches, and one switch dont back more. On startup switch show errors and reboot again and again...

I try to uninstall ONIE and install again, but I cant install because receive errors about cannot read /dev/i2c-2, unable to found serial number  and not found base mac address

Ideas about this problem?

logs of boot, DIG and when I try to install FTOS: pastebin.com/3h9GWWH8

( I try to put logs here but my post is marked with spam )

Moderator

 • 

8.5K Posts

October 18th, 2021 13:00

LeoALage,

 

 

Give me some time to research and also do some testing for you regarding the issue with the S2048-ONs. Please be patient and I will be back with you as soon as I can.

 

 

16 Posts

October 18th, 2021 19:00

Hi Chris thanks, this problem here is about one switch Dell s3048-on
I had diferent problem in different switches on same day 

 

thanks

Moderator

 • 

3.7K Posts

October 19th, 2021 00:00

Hi, LeoALage
I understand you left two separate posts for two different servers.
Could you please direct message/private message (NOT as a reply here on this thread) Chris with your service tag as below so he can see if there's anything else to help?
1. S3048-ON died after a shutdown and reboot : service tag xxxxxx
2. S4048-ON stuck on boot loop and cant console in: service tag xxxxxx

Moderator

 • 

8.5K Posts

October 19th, 2021 08:00

LeoALage,

 

Sorry for the delay. 


What I would suggest is that you try to break the stack by disconnecting the cables between them and see if one of them will boot, while not stacked.

You may also want to run diags as seen on page 10-11 here.  Though based on the errors i wouldn't be surprised if one of the switches had a hardware failure in the storage, and everything was fine while it was running in RAM.

Let me know what you see.

 

 

16 Posts

October 19th, 2021 13:00

Thanks again @DELL-Chris H 

Follow the results:

DIAG: pastebin.com/72yxz5id

testall: pastebin.com/aSD2QuBE

 

I already disconnect switch from stack, and uninstall ONIE.

Thanks

Leo

Moderator

 • 

8.5K Posts

October 19th, 2021 15:00

Thanks Leo,

Do both of the switches in the stack have the same errors? The one you posted looks like a storage failure. Is it under warranty?

16 Posts

October 19th, 2021 20:00

Hi @DELL-Josh Cr , only one switch had problem. We removed them from stack.

The warranty expired in October 2020, I'm searching for anything to try to fix this.. do you sugest to me swap the mdsata card, for example?

Moderator

 • 

8.5K Posts

October 20th, 2021 10:00

Upon speaking with Josh, would you confirm with a picture, the card you are wanting to replace?

 

Thanks. 

 

 

Moderator

 • 

8.5K Posts

October 20th, 2021 11:00

In the logs you posted there was this

ERROR:i2c_write failed, bus=3, cs2eeprom1: s2eeprom_debug_read: address write failed at 0x0

hannel=5, i2c addr:0x4d offset:0s2eeprom6: s2eeprom_debug_read: address write failed at 0x0

x20 reglen:0x1 buflen:0x1, flagss2eeprom5: s2eeprom_debug_read: address write failed at 0x0

: 0, rv:-1

and

 

 

PPId : ERROR:Error in reading eeprom tlv header

 

PPId Revision : ERROR:Error in reading eeprom tlv header

 

Board Service Tag : ERROR:Error in reading eeprom tlv header

 

MMC Rev : 0x0

SMC Rev : 0xf

Image Build Version : 2.0(0.3)

 

 

Available free muvm_fault(0xd8fa1354, 0, 1) -> 0xe

fatal page fault in supervisor mode

 

I don’t think it is recoverable.

16 Posts

October 20th, 2021 11:00

I took another msata of another switch and try to put in this switch, and problem is the same, msata isn't a problem. Do you already saw this epron errors? Its possible recover this switch?

 

Thanks for help

No Events found!

Top