Unsolved
This post is more than 5 years old
19 Posts
0
1540
RHEL VMs lose connection to Isilon NFS mounts randomly
Has anyone experienced hosts randomly losing connectivity to the isilon? here is a scrubbed output from the server that lost it's mount. you will see that a df-hT shows it lost connectivity. I verified the settings are correct in /etc/fstab. I unmounted and mounted and it came back lickity-split. I have around 200 connected clients and it randomly occurs. Any assistance would be appreciated. I've also opened a case, so if know one knows I'll post the resolution once found. thanks in advance
[user@rhel-host ~]$ df -hT
Filesystem Type Size Used Avail Use% Mounted on
df: `/apps/rhelFS/nfs': Input/output error
[user@rhel-host ~]$ more /etc/fstab
isilon-nfs01:/ifs/data/nfs/customer/rhelFS-nfs01 /apps/rhelFS/nfs nfs rw,vers=3,hard,bg,nolock,dirsync,acdirmax=1,acdirmin=1,lookupcache=none 0 0
[user@rhel-host ~]$ sudo su
[root@rhel-host user1]# umount /apps/rhelFS/nfs
[root@rhel-host user1]# mount -a
[root@rhel-host user1]# df -hT
Filesystem Type Size Used Avail Use% Mounted on
isilon-nfs01:/ifs/data/nfs/customer/rhelFS-nfs01
nfs 95T 9.8T 83T 11% /apps/rhelFS/nfs
Anonymous User
170 Posts
0
April 7th, 2015 10:00
Yes, we a LOT of issues when the RHEL guests were hosted on KVM on Isilon. When we migrated those same guests to VMware on NetApp filers, the problems all went away.
We saw lots of timeouts in dmesg and /var/log/messages, similar to this:
kernel: BUG: soft lockup - CPU#0 stuck for 69s!
I did have an open case with EMC about this and I repeatedly heard that the next release of OneFS was going to make it better, but the problems didn't go away until we migrated off of Isilon completely for our virtualized workload. We're still a significant Isilon customer for our NFS workloads. I love Isilon but believe in using the best tool for the job and Isilon + KVM just wasn't a good combination for us.
Peter_Sero
1.2K Posts
0
April 7th, 2015 18:00
If you changed both, vm environment and filer, you can't tell much about the individual aspects...
In contrast (and not actually comparable to the original question), using VMware Workstation on RHEL 5, we run virtual Isilon clusters with all the OneFS drives (vmdks) placed on a single NFS mount from a physical Isilon (X200).
Never worked with 6.5.x, works great with 7.0.2.9 and 7.1.1.2 (on the physical cluster).
To the original poster: Which vm environment? How is your network set up? Log messages from Isilon, vm environment and guest RHEL?
Cheers
-- Peter