Unsolved
This post is more than 5 years old
12 Posts
0
1743
Celerra NS20 can't access some cifs folders
Some friend called me to help with a celerra NS20. About 1TB data is not accesible and is very worried.
This morning 1 drive faulted in group raid with two other disks already faulted, so now, there are 3 disk faulted of 5. But, i'm not sure that this is related to the problem, because i understand that with two disk faulted this raid group was innacessible since 2014 when other disk of the group become faulted.
Over this raid there are two LUNs and about 1TB data space. There is only one storage group forme by all LUNs of the system.
I connected to navisphere and in alert panel i read the log for the disk faulted but no other events shown. Since 2013 there is no events related to luns become offline.
Some folder are accesible and some others not. But i dont know how can i see the mapping between luns and data or how to try to mount the folder. Also, dont know how can i access data from navicli or control station.
I really appreciatte any help.
Thanks.
dynamox
2 Intern
2 Intern
•
20.4K Posts
0
December 22nd, 2015 19:00
if file system is corrupted, i am amazed datamover has not paniced. Have you connected to the control station and start investigating while file systems are there, which are still mounted.
borjacc
12 Posts
0
December 22nd, 2015 23:00
I connected to control station but can't found where filesystems are mounted.
dynamox
2 Intern
2 Intern
•
20.4K Posts
0
December 23rd, 2015 06:00
what commands did you use to check ?
nas_fs -l
server_df server_2
server_mount server_2
borjacc
12 Posts
0
December 28th, 2015 04:00
The first command shows me a list of directories, i only recognized two of them, i think they are the two main storage filesystems. One can be accesed, the other not.
Second command gives me an error 4000.
Is there a way of extract raw data of LUNs online? There are some programs that analyze data trying to find files like excel or doc file types.
Thank you.
borjacc
12 Posts
0
January 12th, 2016 01:00
I managed to run the commands correctly.
With nas_fs -l i can see the two main filesystems DATOS and ARCHIVO and both shows that are in use.
With server_df server_2 i can see that only ARCHIVO filesystem is mounted with root_fs _common and root_fs_2.
server_mount server_2 tries to mount the 4 filesystems, but in DATOS can't do it:
DATOS on /DATOS uxfs,perm,rw, .
I also tried to mount this filesystem with: server_mount server_2 DATOS /DATOS and get: error 4020 failed to complete the command.
Any ideas?
Thnaks
dynamox
2 Intern
2 Intern
•
20.4K Posts
0
January 12th, 2016 11:00
please post output from the commands.
Thanks
borjacc
12 Posts
0
January 12th, 2016 12:00
Sure,
[root@NASxxxxx ~]# /nas/bin/nas_fs -l
id inuse type acl volume name server
1 n 1 0 10 root_fs_1
2 y 1 0 12 root_fs_2 1
3 n 1 0 14 root_fs_3
4 n 1 0 16 root_fs_4
5 n 1 0 18 root_fs_5
6 n 1 0 20 root_fs_6
7 n 1 0 22 root_fs_7
8 n 1 0 24 root_fs_8
9 n 1 0 26 root_fs_9
10 n 1 0 28 root_fs_10
11 n 1 0 30 root_fs_11
12 n 1 0 32 root_fs_12
13 n 1 0 34 root_fs_13
14 n 1 0 36 root_fs_14
15 n 1 0 38 root_fs_15
16 y 1 0 40 root_fs_common 1
17 n 5 0 73 root_fs_ufslog
19 n 5 0 77 root_fs_d3
20 n 5 0 78 root_fs_d4
21 n 5 0 79 root_fs_d5
22 n 5 0 80 root_fs_d6
27 y 1 0 97 DATOS 1
44 y 1 0 348 ARCHIVO 1
46 n 5 0 366 root_panic_reserve
---------------------------------------------------------------------------------------------------------------
]# /nas/bin/server_df server_2
server_2 :
Filesystem kbytes used avail capacity Mounted on
ARCHIVO 3788103392 3677132224 110971168 97% /ARCHIVO
root_fs_common 13624 5280 8344 39% /.etc_common
root_fs_2 231944 87832 144112 38% /
---------------------------------------------------------------------------------------------------------------------
# /nas/bin/nas_fs -info DATOS
id = 27
name = DATOS
acl = 0
in_use = True
type = uxfs
worm = off
volume = v97
pool = xxxxxxx
member_of = root_avm_fs_group_27
rw_servers= server_2
ro_servers=
rw_vdms =
ro_vdms =
auto_ext = hwm=90%,max_size=5400000M (reached),virtual_provision=yes
deduplication = unavailable
stor_devs = CK200074400335-0011,CK200074400335-0012,CK200074400335-0013,CK200074400335-0010,CK200074400335-0015,CK200074400335-0016,CK200074400335-0014,CK200074400335-0017,CK200074400335-0019,CK200074400335-0018
disks = d8,d9,d10,d7,d11,d12,d13,d14,d15,d16
disk=d8 stor_dev=CK200074400335-0011 addr=c16t1l1 server=server_2
disk=d8 stor_dev=CK200074400335-0011 addr=c0t1l1 server=server_2
disk=d9 stor_dev=CK200074400335-0012 addr=c0t1l2 server=server_2
disk=d9 stor_dev=CK200074400335-0012 addr=c16t1l2 server=server_2
disk=d10 stor_dev=CK200074400335-0013 addr=c16t1l3 server=server_2
disk=d10 stor_dev=CK200074400335-0013 addr=c0t1l3 server=server_2
disk=d7 stor_dev=CK200074400335-0010 addr=c0t1l0 server=server_2
disk=d7 stor_dev=CK200074400335-0010 addr=c16t1l0 server=server_2
disk=d11 stor_dev=CK200074400335-0015 addr=c0t1l5 server=server_2
disk=d11 stor_dev=CK200074400335-0015 addr=c16t1l5 server=server_2
disk=d12 stor_dev=CK200074400335-0016 addr=c0t1l6 server=server_2
disk=d12 stor_dev=CK200074400335-0016 addr=c16t1l6 server=server_2
disk=d13 stor_dev=CK200074400335-0014 addr=c16t1l4 server=server_2
disk=d13 stor_dev=CK200074400335-0014 addr=c0t1l4 server=server_2
disk=d14 stor_dev=CK200074400335-0017 addr=c16t1l7 server=server_2
disk=d14 stor_dev=CK200074400335-0017 addr=c0t1l7 server=server_2
disk=d15 stor_dev=CK200074400335-0019 addr=c0t1l9 server=server_2
disk=d15 stor_dev=CK200074400335-0019 addr=c16t1l9 server=server_2
disk=d16 stor_dev=CK200074400335-0018 addr=c16t1l8 server=server_2
disk=d16 stor_dev=CK200074400335-0018 addr=c0t1l8 server=server_2
------------------------------------------------------------------------------------------------------------------------------------------------
Where disk d12 and d14 are de LUNs affected by the faulted RG.
# /nas/bin/server_mount server_2 DATOS /DATOS
server_2 :
Error 4020: server_2 : failed to complete command
------------------------------------------------------------------------------------------------------------------------------------------------
# /nas/bin/nas_fsck -start DATOS -monitor
Error 5: Input/output error
Thank you!
kelleg
4.5K Posts
0
January 12th, 2016 14:00
I think the main issue is to try to repair the faulted disks. If the raid group was a Raid 6 which can withstand a two disk failure and still function and a third disk fails, then the raid group will be faulted and access disabled to the LUNs from the raid group.
The way to fix this is to replace the last disk that failed (the 3rd one) first and let the raid group rebuild, then replace the second disk that failed, let raid group rebuild, the replace the first disk that failed. This is only safe way to fix this issue, removing or moving around any of the faulted disks could result in the correct disk getting into the wrong slot. It's very important that if you do remove a disk, always return the disk back to the same slot before during any other work.
The raid group should be OK and the LUNs should come back on-line once you replace the disks.
glen