dynamox
6 Thallium

Re: Ask the Expert: SMB Protocol on an Isilon Cluster

Any way to improve OSX users experience when using Isilon via CIFS. Browsing shares is much slower than on Windows. I did read this paper "docu45329_Using-Mac-OS-X-Clients-with-Isilon-OneFS-6.5" but we don't have SSD nodes and changing view in Finder did not do anything.

Thanks

0 Kudos
pabromitis
1 Nickel

Re: Ask the Expert: SMB Protocol on an Isilon Cluster

Mark, you could also do the same with wireshark.  It can roll as many logs as you require and you can roll them on size or time.

0 Kudos
pabromitis
1 Nickel

Re: Ask the Expert: SMB Protocol on an Isilon Cluster

Rocketman,

Connections are idle or active.

An active connection means that at the time you checked the stat, the client had sent a SMB request that the node had not responded to yet.

Idle connections mean that the TCP session is active but there was no SMB request sent during the time the stat was collected.

If you see "stale" connections, it really means they are idle and there is an active tcp session associated with it that is being kept alive or has not timed out yet due to inactivity.  As soon as TCP times out, the associated smb session will be cleaned up.

You could compare your smb sessions to the raw netstat output (also you have to remember that these counters are going to be a per node basis):

For example when I look node 1 of my cluster, I see that I have two smb sessions that have been idle for a long time:

isi-ess-east-1# isi smb session list 

Session [1]

        Computer                :192.168.1.10

        Username                :ISI-ESS-EAST\sli

        Client type             Smiley Very HappyOS LM 2.0

        Number of opens         :5

        Active time             :54932

        Idle time               : 0

        Guest login             :no

        Encryption              :no

Session [2]

        Computer                :192.168.1.11

        Username                :ISI-ESS-EAST\pete

        Client type             Smiley Very HappyOS LM 2.0

        Number of opens         : 0

        Active time             :1096

        Idle time               : 0

        Guest login             :no

        Encryption              :no

When I run netstat -na:

isi-ess-east-1# netstat -na |grep .445

tcp4       0      0 10.111.176.100.445     192.168.1.11.64493     ESTABLISHED

tcp4       0      0 10.111.176.100.445     192.168.1.10.10875   ESTABLISHED

tcp4       0      0 *.445                  *.*                        LISTEN

tcp46      0      0 *.445                  *.*                        LISTEN

Both of those TCP connections are in an established state meaning that the clients are keeping the tcp connection alive.

When I run netstat on my client it too shows the connection established:

C:\Users\pete>netstat -na

Active Connections

  Proto  Local Address          Foreign Address        State

TCP192.168.1.11:64493 10.111.176.100:445 ESTABLISHED

In the end though, if you would like to terminate just those sessions, you can do so via:

For 6.5:

isi smb session delete --computer-name=<ip>

For 7.X

isi smb sessions delete --computer-name=<ip>

0 Kudos
BernieC2
2 Iron

Re: Ask the Expert: SMB Protocol on an Isilon Cluster

dynamox, how much slower are you seeing browsing for Macs, and are you dealing with large and wide directory structure?  Typically that's not an ideal workflow for SMB.  NFS handles it better due to the readdirplus calls it can make, but NFS comes with its own set of challenges on the Mac (like, AppleDouble's creation of dotbar files).

While I have heard of some folks modifying /etc/nsmb.conf on Macs to disable change notification and alternate data streams to try improve performance, past experience hasn't shown much (if any) improvement gleaned by making those changes.  And, making those changes to the client require that all clients get the change.

As the document describes, retrieving metadata faster from the Isilon is the best way to get the Finder to display objects more quickly.  Reducing network latency may not be possible, which is why the document does call out the 5-7x improvement in speed when using SSDs.

At this point, it's far to early for me to say if something like the SMB2 support in OS X 10.9 is going to make much of a difference, although it is something I'm starting to test with for an update to the Mac guide.

0 Kudos
dynamox
6 Thallium

Re: Ask the Expert: SMB Protocol on an Isilon Cluster

Bernie,

Directory structure is very shallow, maybe 10-15 directories. Any experience with disabling change notification on Isilon side (per share) ? What does it do and how will it impact windows and Mac users ?

Thanks

0 Kudos
BernieC2
2 Iron

Re: Ask the Expert: SMB Protocol on an Isilon Cluster

I'll let Pete speak to the Windows experience of changing the change notification settings (I vaguely seem to recall that SMB2 requires change notification).  My experience with the Mac has shown that changing change notification from All to norecurse solves a strange behavior in the Finder where it'll occasionally flip from one subdirectory all the way back to the root of the share (without user intervention).  We have this documented in KB 89045.  But, again, I haven't seen much performance difference with it set to all or norecurse.

Disabling change notification may be a problem in some workflows where users rely on quick updates to Finder views.  Because there's no F5 in the Finder to refresh a Finder window, change notification sends an OS X fsevent up to the Finder to refresh its view.  That can be useful in cases where two graphics editors are watching the same directory, and one needs to know when the other has updated a file, or added a color label.

0 Kudos
pabromitis
1 Nickel

Re: Ask the Expert: SMB Protocol on an Isilon Cluster

Yeah, you should no longer disable change notify as it breaks Windows Vista clients and beyond.  Per KB:

https://support.emc.com/kb/91441

When the Change Notify setting is set to None, the feature is disabled for the share. Clients will not be notified of any changes that are made in the current directory or sub-directories. If this is the case, Windows clients might experience the following:

  • Windows 7 clients cannot manually refresh the directory. This is a limitation of how directories are cached locally, and of not performing a findfirst of the directory during the refresh. However, when a Windows 7 client makes a change in a directory, that change and all other changes made by other clients will be detected without the need to manually refresh.
  • Windows XP clients can manually refresh the directory by pressing F5 from within Windows Explorer, or right-clicking and selecting Refresh. When a Windows XP client makes a change in a directory, the client detects only the changes that it makes. Changes made by other clients are not detected until a manual refresh is done.
0 Kudos
dynamox
6 Thallium

Re: Ask the Expert: SMB Protocol on an Isilon Cluster

Bernie,

from your experience quota size has no impact on how fast Finder displays directory content, it's all about how many objects (directories/files) are there ?

0 Kudos
BernieC2
2 Iron

Re: Ask the Expert: SMB Protocol on an Isilon Cluster

Quota size, no.  In my experience, several factors come into play for directory listings on a Mac:

  • Number of objects in a directory (If you're using AppleDouble, that may double the number of objects in the directory; the Finder hides the dotbar files from view, but it still has to enumerate them, and read them to apply resource fork data.)
  • The view being used in the Finder (Column view can be initially as fast any other view, but if Finder needs to start refreshing each column, it'll hold up displaying lower levels in the tree until the upper levels have finished their refresh.).
  • Network latency (round-trip time)
  • The time it takes the Isilon cluster to pull metadata from disk.

The worst possible situation for SMB on the Mac is in WAN environments.  I recall seeing an environment where the Mac was in Los Angeles and the Isilon cluster was in London.  The client, even on the customer's private network, was getting 70ms round trip response times, which is pretty good RTT for such a long link.  But, with the hundreds or thousands of SMB calls it had to send over that link, the Finder just ended up spinning for minutes.  Even NFS can only help so much here.  The Finder's quite picky about its metadata (resource forks), compared to Windows.  Those resource forks are essentially additional file data that the Mac has to go enumerate and read before displaying it to the user.

In my experience with the Mac, my goal is to get metadata retrieval latencies down to minimums.  WIth the inability of SMB1 to batch together those metadata calls, SSDs make the biggest difference, as you cut out large amounts of seek latency.  But, you're still at the mercy of network latency, as I mentioned above.

0 Kudos
mcbris
2 Iron

Re: Ask the Expert: SMB Protocol on an Isilon Cluster

Thank you everyone for participating in this Ask the Expert event. Many thanks to Pete for hosting the discussion! He has posted a summary of some of the key topics here: https://community.emc.com/docs/DOC-26337.

Please feel free to continue the conversation (note that it will not be formally moderated by Pete after today) or start a new thread in the Isilon Support Forum to dicuss other topics.

Best,

Stephanie