Start a Conversation

Unsolved

B

18 Posts

2710

May 1st, 2020 10:00

RAID storage initilizing

My array is stuck with initilizing, I have tried new controllers but still stuck.

I ran 'su raidtool" and got this reply;

Technical Support.
Driver Status: *Admin Intervention Requested*
RAID LUN 0 Ok.
raid status unrecoverable.
15 Drives (0,2,4,6,8,10,12,1,3,5,7,9,11,13,14)
RAID 6 (64KB sectPerSU)
Capacity 9,986,026,831,872 bytes
Available Drives List: 15

Is my RAID gone?

Any way to recover?

3 Apprentice

 • 

1.5K Posts

May 1st, 2020 11:00

Hello, 

 I just sent a reply asking for this output 

You're RAIDset is not gone. As the LUN itself is "OK" 

 Do you have a good backup? 

 You have a couple of options. If you are in a region where Dell supports a Time and Material case you can open a case and they can properly triage it. 

 If you either have a good backup or are willing to risk data loss.  You can try to clear this LUN condition and the array could finish booting and everything will come back online.  If that happens you will then need to check the filesystem/data integrity of all your volumes.  I.e. using fsck, or chkdsik/scandisk for Windows. 

 The command would at the CLI would be  CLI>clearlostdata 

  Regards, 

Don 

 

18 Posts

May 1st, 2020 12:00

Ok, on the reset. I just type in reset in the CLI and it will return to default right?

 

18 Posts

May 1st, 2020 12:00

On the chassis there are a few lights as well.  

The chassis lights are address to left, power top right then a check mark and another symbel.  

Both controller lights are all green or yellow.

18 Posts

May 1st, 2020 12:00

I did and appoligize for now having two posts for same overall issue (I'll stick to this post moving forward).  I think I answered you other question above.

18 Posts

May 1st, 2020 12:00

I have not replaced any drives but the place I bought the array from installed SSD drives that have been running fine for over a year.

3 Apprentice

 • 

1.5K Posts

May 1st, 2020 12:00

Hello, 

The yellow light on the controller simply means it's the passive CM.  Green/Green is active. And RED means the controller has a fault. 

 Your issue is not with the controllers. 

Don 

 

18 Posts

May 1st, 2020 12:00

after clearlostdata it gets stuck in this loop:

####################################################
# #
# Dell (tm), Inc. Storage Array #
# Copyright 2001-2009 #
# #
# Controller Information: #
# Part=70-0300 Rev=A06 SN=RMS0943675G1CF4 ECO=C00 #
# #
#########################################################
Bootloader Version 2.3.5 (SWINT Rev:1)
Compiled on Mon Apr 30 17:08:16 EDT 2012
(type h for help)
Enter Ctrl-P for boot prompt

Executing bootcmd0 [dload sd primary/eqlstor.gz]
0X80231000/11912 0X85f08100/31424212 entrypt 80231000

Executing bootcmd1 [run]
cpu_online_map=ffff, userapp_cpu_map ffff
psb_os_active_mask=0, psb_os_mask=0
boot1_info: userapp_cpu_map=ffff, psb_os_cpu_map=0
cpu_online_map = 0xffff
Jumping to the application... 0x80231000
------------------------------------------------------------
Preparing ffff bitmask of cpus to run
No network device to cleanup
count = 16, total = 16
All slave cpus (16) ack'ed userapp init
count = 4, total = 4
All slave cpus (4) ack'ed message ring init

Dell Inc. Storage Array

Copyright 2001-2016 Dell Inc.
SP:13.81:mips_pss_init.c:363:INFO:28.2.107:Control module in slot 1 with serial number RMS0943675G1CF4 is designated as active.
SP:1588371414.28:ppool_nvram.c:396:ERROR:15.4.1:NVRAM contains valid data. This is a PANIC RECOVERY due to a panic on a NetBSD processor.
SP:1588371414.28:ppool_nvram.c:188:ERROR:15.4.5:Saved CPU registers, CPU 0
at 0000000000000000 v0 ffffffff80b73d00 v1 0000000000000000
a0 ffffffff80b8d3e0 a1 ffffffff80b81200 a2 ffffffff80b74380 a3 ffffffff80b73d00
t0 0000000000000840 t1 ffffffff815c5770 t2 0000000000000001 t3 ffffffff80ba0000
t4 000000004000c001 t5 0000000000000007 t6 0000000000000007 t7 0000000000000000
s0 ffffffff80b8d3e0 s1 ffffffff8efd3c00 s2 0000000000000000 s3 0000000000000001
s4 0000000000000000 s5 0000000051eb851f s6 0000000000000064 s7 ffffffff8e175c80
t8 ffffffff8344c00c t9 ffffffff80414538 k0 0000000000384bdf k1 0000000000000001
gp ffffffff8110a6b0 sp 00000000c87c7820 s8 00000000000000a2 ra ffffffff809ef8f0
SP:1588371414.28:ppool_nvram.c:206:ERROR:15.4.6:Saved CP0 registers, CPU 0
sr 4000ff01 badva c0332e98 epc 8092eed0 errorepc bfc00000
cause 00000008 errctl 00000000 cacheeri 00000000 cacheerd 00000000
buserr 0000000000000000 cacheerrdpa 0000000000000000
SP:1588371414.28:ppool_nvram.c:213:ERROR:15.4.7:Saved function call stack, CPU 0
80586ab4 809ef8f0 80904abc 8091614c 80916b7c 80930d5c 809359b4 80935e78
808a22d4 8088f660 8088f9cc 8088fd00 8099b458 8099b770 80592434 00000000
SP:1588371414.28:eqllog_mbuf_Q.c:1100:ERROR:2.4.0:Panic recovery from CPU0 with reason 'kernel fatalassertion "!IOM_IS_INVALID_METADATA_VERSION(version)" failed: file "../../../NetBSD/src/sys/EQL/ioman/lv2/lv2_lv_log_read.cc", line 2112'.
MFS set up
Building databases...
SP:1588371421.98:emm.c:1333:INFO:28.2.6:Enclosure serial number: SHU0937898Y9856.
[ENTRY][BIST][EQL][TWDOG][WFP][210][CM1D][MESR][SCAN][BOOT2][PROC][BOARD][SMP][IOBUS][FLASH][HT][ETH0][ETH1][ETH2]

 

I tried to call Dell and since I bought the array from a third party and am not the current owner in their records they will not help me.  And the third party does not have a record from over a year ago of who the original owner was.  GRRRRRRRRR

3 Apprentice

 • 

1.5K Posts

May 1st, 2020 12:00

Hello, 

 Sorry but this is about as far I can take this here.  You can try to open up a T&M case or reset it and restore the data. 

 Regards, 

Don 

 

18 Posts

May 1st, 2020 12:00

I pulled the drives halfway out and restarted the chassis and ran clearlostdata:

CLI> clearlostdata
% Error - could not retrieve email notification configuration

WARNING: You are executing on the Passive Control Module.
This is normally not the right thing to do, because most of
the data this command gathers will not be available here.

Do you want to proceed anyway (y/n) [n]: n
Aborted.
Invalid request, driver is not in admin state, no lost data to clear.
CLI>

PS Series Storage Arrays
Unauthorized Access Prohibited

login: grpadmin
Password:


Welcome to Group Manager

Copyright 2001-2016 Dell Inc.

 

The storage array is still initializing. Limited commands will be available unt
il the initialization is complete. Please try again later.

CLI> 3256:1:netmgtd: 1-May-2020 18:38:24.540001:rcc_util.c:1024:INFO::25.2.9:CLI: Login to account grpadmin succeeded, using local authentication. User privilege is group-admin.
3464:3335:SP: 1-May-2020 18:38:36.493335:emm.c:2806:INFO::28.2.35:Link to the enclosure management processor has been re-established.
4167:4033:SP: 1-May-2020 18:39:17.754033:emm.c:2363:ERROR::28.4.47:Critical health conditions exist.
Correct immediately before they affect array operation.
Link to the enclosure management processor failed.
There are 1 outstanding health conditions. Correct these conditions before they affect array operation.
5401:5259:SP: 1-May-2020 18:40:29.851163:emm.c:2806:INFO::28.2.35:Link to the enclosure management processor has been re-established.
6103:5957:SP: 1-May-2020 18:41:11.101861:emm.c:2363:ERROR::28.4.47:Critical health conditions exist.
Correct immediately before they affect array operation.
Link to the enclosure management processor failed.
There are 1 outstanding health conditions. Correct these conditions before they affect array operation.
7338:7183:SP: 1-May-2020 18:42:23.213087:emm.c:2806:INFO::28.2.35:Link to the enclosure management processor has been re-established.

CLI> 8058:7898:SP: 1-May-2020 18:43:04.463802:emm.c:2363:ERROR::28.4.47:Critical health conditions exist.
Correct immediately before they affect array operation.
Link to the enclosure management processor failed.
There are 1 outstanding health conditions. Correct these conditions before they affect array operation.

 

3 Apprentice

 • 

1.5K Posts

May 1st, 2020 12:00

Hello, 

 Thank you.   Did you see my reply to your other post?

Don

 

18 Posts

May 1st, 2020 12:00

Switched to main controller:

CLI> show
The storage array is still initializing. Limited commands will be available unt
il the initialization is complete. Please try again later.

CLI> clearlostdata
% Error - could not retrieve email notification configuration

The clearlostdata command will gather information about the
state of this array for support and troubleshooting purposes.
No user information will be included in this data.

E-mail notification is not available, so you must retrieve the results
by using the "text capture" feature of your terminal emulator
or Telnet program.

You will be given information to help you do this at the end of this procedure.

Finally, please remember to include your Dell Technical Support case or incident number
in the subject line of any e-mail that you send to Dell Support. This will help
ensure that the message is routed correctly.

Do you wish to proceed with data collection? (y/n) [y]: y

Starting data collection on Fri May 1 18:46:08 EDT 2020.

Section 1 of 1: ..11231:11048:SP: 1-May-2020 18:46:09.902856:emm.c:2806:INFO::28.2.35:Link to the enclosure management processor has been re-established.

Finished in 2 seconds


You also have the option to capture the output by using the "text capture"
feature of your Telnet or terminal emulator program.

Do you wish to do this (y/n) [n]: n
Invalid request, driver is not in admin state, no lost data to clear.
netmgtd:1588373175.55:main.c:1272:AUDIT:25.7.8:User action: clearlostdata
CLI>

18 Posts

May 1st, 2020 12:00

Where can I find a LED reference for the enclosure?  I have a few lights that are flashing yellow but not sure what they mean.

18 Posts

May 1st, 2020 13:00

Any resource for the chassis lights (not controllers)?

3 Apprentice

 • 

1.5K Posts

May 1st, 2020 13:00

Hello, 

 Yes. It will force you to type "DeleteAllMyDataNow" to confirm it.  All the data and configuration info will be reset. 

 Regards, 

Don 

3 Apprentice

 • 

1.5K Posts

May 1st, 2020 16:00

Hello, 

I'll check.  Are you talking on the front panel or rear? 

 That chassis is a standard chassis from another vendor. 

  Once booted I'm not sure there's must useful information from them. 

  I'll see if there's anything about them in the HW reference guide. 

 Regards, 

Don 

 

No Events found!

Top