Unsolved
This post is more than 5 years old
83 Posts
3
10208
Ask The Expert, OneFS and the Mac: Command+K to Awesome!
Welcome to this EMC Support Community Ask the Expert conversation. This is an opportunity to learn about and discuss the way Macs and OneFS interoperate.
Topics of discussion we could have in this thread may include:
- SMB2 in OS X Mavericks
- NFS vs SMB, which is better for my Mac users?
- User identity with OneFS and Mac users
- Multi-protocol and multi-platform data access with OneFS
- Maintaining high metadata performance with OS X and OneFS
This discussion begins on 11/11/2013 and concludes on 11/22/2013 Get ready by bookmarking this page or signing up to receive email notifications.
Your host is Bernie Case!
Name: | Bernie Case |
Occupation: | Tech Support Engineer IV |
Biography: | My original career path was taking me toward audio production and engineering, but I got mixed up in a "bad" crowd of IT folks during the dot-com boom of the late 1990s and that's where I've focused my talents ever since. I use my knowledge and experience with networking protocols, media and entertainment workflows, Mac OS X, *nix, Windows, and system administration to help solve interesting, complex, and sometimes urgent problems. |
URL: | http://www.emc.com |
Expertise: | Multiple flavors of Mac OS, including pre-OS X through OS X 10.9, networking protocols (OSI layer 2 and up), system administration. |
Company: | EMC Isilon |
City: | Seattle |
State: | WA |
Country: | USA |
Display Name: | Bernie Case |
Web Site: | http://www.emc.com |
MRWA
83 Posts
0
November 11th, 2013 08:00
This Ask the Expert thread is now open!
arichard1
21 Posts
0
November 11th, 2013 10:00
Hello a litte bit of topic but
Is there a plan to push Isilon as a premary VMware storage in the next onefs release? We are currently use onefs 7.1.on S200, any improuvemt soon?
last question, should we put SSD in our S200 of VMware only storage (A lot of VM conected but not a lot of files)?
BernieC
76 Posts
0
November 11th, 2013 15:00
Hi Alain!
Thanks for asking your questions. I'd recommend, in the future, to start a new thread so that we can properly answer your questions there.
I'm not completely aware of any new plans to reposition OneFS for VMware environments, although there were a number of performance improvements implemented in 7.1 that give us higher IOPs which may help. It's possible someone else can give more details.
Regarding SSDs on your S200s. If your entire workflow is nothing but VM, then there likely would not be a large benefit from SSD. But, if you stored home directories or other highly latency-sensitive metadata on your nodes, SSDs would help quite a lot.
Thanks for the questions,
Bernie
BernieC
76 Posts
1
November 11th, 2013 16:00
So let's start with something that I'm quite excited about: SMB2 support in OS X.
Apple, in OS X Lion, moved away from Samba, instead implementing its own SMB client and server called smbx. SMB2 support was something that had been rumored to be coming in Lion (10.7), but the SMB client in Lion and Mountain Lion (10.8) was sadly absent. Funny enough, however, the SMB2 server is present, and can be confirmed by running /usr/sbin/smbd --help, which explains that that SMB2 support is experimental in those releases, and even Mavericks (10.9) where it is enabled by default.
Thankfully though, the SMB2 client in Mavericks is not experimental, and has been enabled by default in Mavericks. What this means for the Mac users is that they can get much higher throughput now when connecting to OneFS 7.x clusters due to the SMB 2.1 support included in those releases. We expect that performance for Macs with OneFS 6.5.x will be largely similar to SMB1 because that version of OneFS supports a smaller (64KB) maximum read and write size. OneFS 7.x supports up to 1MB read and write sizes, which greatly boosts throughput on 10Gbit networks. For more information, see this previous post on ECN.
Apple indicated in its Mavericks Core Technologies Overview that SMB2 is automatically used to share files between two Macs running OS X Mavericks. That has not been my experience locally; AFP appears to still be used between my Macs. But, SMB2 will be used automatically against Isilon clusters that support it (6.5.x and later).
While the compatibility guides do not reflect support for OS X Mavericks, it's something I'm working with our documentation group to update. We do support OS X Mavericks in OneFS 6.5 and later, but we prefer customers use OneFS 7.0.x and later for maximum performance.
There is an issue I have seen with the SMB2 support in Mavericks. Getting a list of SMB shares may fail when the Isilon cluster is running OneFS 6.5.x, particularly in environments with a large number of shares. Connecting directly to the share, by its sharename (i.e. smb://server/share), will still work. If you are having this issue in your environment please open up a support case and ask that your issue be escalated to Isilon engineering.
I look forward to future questions and comments. Thanks!
Anonymous
5 Practitioner
5 Practitioner
•
274.2K Posts
0
November 12th, 2013 08:00
Hi Bernie!
Thanks for taking the time out for answering our questions on all things OSX!
I know you did some work with OSX and NFS 4 with kerberos/AD. Can you share with us some of your findings and oberservations.
Hints,Tips, pitfalls etc.
Look forward to the thread!
BernieC
76 Posts
1
November 13th, 2013 15:00
Hi broreg,
Thanks for the note and thanks for reading! OS X, NFSv4, and Kerberos are all very interesting topics. I've been working on an update to the Mac Best Practices Guide (see the OneFS 6.5 version here) which takes into account all of these topics. The updated guide is being reviewed by our Info Development team and should be published this quarter. But, getting back to your question, here's what I've learned...
Because of these problems with the NFSv4 idmapper in OS X, I'm recommending customers shy away from NFSv4 and OS X for the time being. As soon as these problems are taken care of, or if new information comes out about how to work around them, I'll share details in the best practices guide.
Kerberized NFSv3 against Active Directory is a little tricky, however, because the Mac can only use DES encryption against 2003 and 2008 domains (AD doesn't use des3-cbc-sha1, which is the Mac's preferred encryption type). That requires some changes to those domains, and to user accounts, to enable that encryption type, which some may be hesitant to do, due to DES's known shortcomings. We have been investigating the feasibility of adding AES support to our NFS server in the future, however, so there's hope that we can avoid DES entirely. Stay tuned for more info soon, and watch support.emc.com for the KB!
As it is currently, if customers would like to use NFS in an AD authentication environment while, at the same time, maintaining the ability to authenticate and authorize users by their AD credentials, they can do one of two things:
NFS user squash is obviously the easier option, but does not allow for a high level of permissions granularity. It may, however, work well in environments that only have a few Mac users in an otherwise larger AD domain.
Services for Unix implementation can be an involved process, particularly if no UIDs and GIDs have been added to accounts in AD. Isilon professional services can assist customers with getting UIDs and GIDs for AD users and groups from the Mac and importing them into AD (details on the Mac side of this process can be found in this KB). The Mac, by default, automatically generates UIDs and GIDs for AD accounts based on the GUID in an AD account (meaning, the UIDs and GIDs match for the same AD account, no matter which Mac you've logged into). This enables us to get a full list of UIDs and GIDs from the Mac, import them into AD accounts, and then configure OneFS to get UIDs and GIDs directly from AD. This allows OneFS to map UIDs and GIDs it retrieves from AD directly to Windows SIDs, which makes multi-protocol (NFS for Macs, SMB for Windows) environments possible, and manageable.
Again, thanks for the question, and please let me know if you have any others!
Thanks,
Bernie
BernieC
76 Posts
1
November 14th, 2013 11:00
NFS vs SMB, which is better for my Mac users?
Like a lot of things with NAS, the answer is often, "It depends..." I find that, often, the answer comes when you answer a few other questions:
The Mac Best Practices guide goes into these topics in the Workflow Consideration section, but I want to share some anecdotes that precipitated the creation of those questions.
Media & Entertainment environments typically like to see higher performance for their users, particularly in interactive or near-interactive video editing or video post-production workflows. NFSv3 has been the better choice here, given that it provides high throughput on higher throughput networks like 10GbE. SMB2 may, however, become more feasible in these environments when used with OneFS 7.x, given the performance improvements in that protocol. I will still recommend NFSv3 in environments that make heavy use of wide directories (e.g. environments that process Cineon frames) because NFSv3 can retrieve metadata from Isilon much faster due to enhancements in its protocol.
SMB can be beneficial in media & entertainment workflows where Windows clients are also accessing the cluster. Using SMB allows Mac-specific resource fork data to get stored within an Alternate Data Stream, which the Windows clients will not see. That's better for some environments that don't want to have to deal with all of the dotbar (._) files that the Mac will create on an NFS export. But, the Mac retrieves file and resource fork metadata via SMB much more slowly than it does via NFS, which can be frustrating for users in environments where thousands of items are stored within just one directory. It often helps, with SMB, to architect directory structure that limits the total number of objects stored in one directory. And, for both protocols, limit the use of the Finder's column view option! (The Finder deserves its own post, so I'll save more details for then!)
SMB is a good choice for environments that require AD authentication. Given that it will natively attempt Kerberos authentication on AD domain-joined Macs, it can be easier to get it authenticated to the cluster. But, the Mac's SMB client will not show file system ACLs when connected to SMB shares (a note on this: OS X 10.5 does show POSIX-style ACLs, but 10.6 and later do not).
NFS authentication and authorization can be a little more challenging depending on the environment, as I mentioned earlier in this thread. Care must be taken with NFS to make sure that the POSIX style permissions that are seen on the Mac are going to give the logged-in user access to an object. Programs like Final Cut Pro rely on these permissions to determine access (a forthcoming KB will share more detail about this). Finder, on the other hand, will often try the operation on the object even if the permissions do not seem to give it much access.
I think this is all I have for now. Cheers!
BernieC
76 Posts
0
November 15th, 2013 16:00
So let's talk about this guy...
The Finder, for those not too familiar with the Mac, is the graphical file manager that has been with the Mac since the beginning (24 January 1984, and that date has special significance even now as I'll explain below). The Finder has some interesting behaviors when it's used against network storage, such as:
We have a number of other KB articles about the Finder available here. Thanks for reading, and have a great weekend everybody!
BernieC
76 Posts
0
November 18th, 2013 16:00
There's one other big thing about the Finder I didn't mention, and I think it's fairly important: try to avoid the use of Column View. For those not familiar with column view, here's what it looks like in 10.9:
What this view shows you is a directory structure that's a few levels deep. My example here is very, very simple, and I have used TinkerTool to enable the display of the path up in the title bar of the window (and a Finder option to show it at the bottom, as well). Each column denotes a different level of the overall directory tree. No doubt, column view is a popular view, and shows a lot of useful information. That information comes at a price, however.
Remember in my previous post where I said that Finder won't display items to a user until it's retrieved the list of items and resource fork data for the folder? Imagine how this plays out with Column view, where not one, but multiple levels of a directory tree are on display for the user. Finder has to keep multiple directories refreshed and updated constantly, which takes more network calls between the Mac and the server. I have seen some cases, with column view, where someone was trying to load just a small directory containing just a few items, but had to wait minutes for that directory to load. When I was able to see what was happening on the network, I found that the Finder was doing directory listings and resource fork data retrieval for directories that were several levels higher in the tree. Until Finder had completed its work there was it able to display the directory. Not a great experience for the user, unfortunately.
I typically recommend list view, but I do so with a couple of extra recommendations:
There are, of course, some Finder replacements out there which greatly alleviate shortcomings in the Finder's retrieval of file and directory data. Pathfinder is one such option, and may work better for you. But, for those stuck to using Finder, like me, these simple tips should hopefully improve your Finder experience against Isilon, or any NAS for that matter.
Thanks,
Bernie
Peter_Sero
1.2K Posts
0
November 20th, 2013 02:00
Hello Bernie,
thanks for the great series of write ups in this thread!
Two more things one might be curious about (I am):
Modern OSX applications have replaced the "File->Save" function by an implicit version history per document, but only if the file system supports it. What, if at all possible, does it take to have versioned documents on an Isilon share (SMB or NFS)?
Another thing is Spotlight indexing. While it's probably a good thing to have indexing of network volumes DISabled in general, one might want to activate it in certain scenarios. I have seen OS X clients indexing NFS home folders in previous versions---and where the NFS server was an also running OS X. Is there any magic known about how the indexing of mounted network volumes could be controlled for OneFS 7.x? (The Isilon & Mac paper only talks about OneFS 6.5 and suggests using iSCSI...)
Cheers
-- Peter
BernieC
76 Posts
0
November 21st, 2013 14:00
Hi Peter,
Thanks for the comments. Let me respond to them below.
Document versioning in OS X uses a special directory at the root of a volume in addition to a SQLite database in order to track revisions. It also takes a developer who implements a specific framework in order to support this within the application. (An overview is available in the excellent Ars Technica review of 10.7 available here.) What it also appears to need is a filesystem that supports document versioning. It looks like OS X just added an API, but didn't make any actual modifications to the underlying filesystem in order to support this, from what I can tell. But, what it would need is some way to fool OS X into believing that a NAS device was eligible for document versions, and right now applications like TextEdit complain about the filesystem not supporting versioning when you make modifications to a document, save it to NAS, and then close the window.
I imagine any changes made to support document versioning may need to be made both in the framework in OS X and on the NAS device in order to understand how to handle specific calls to put data into that special directory. Interesting request, and it might be something Apple would consider in the future. By the way, I also tested this via AFP between two Macs... it's not even supported in that sort of environment. It would seem it's only designed to work with direct-attached storage.
The reason iSCSI was mentioned in the best practices guide was somewhat similar to why document versioning only works with direct-attached storage - it boils down to where the Spotlight indices are built (.Spotlight-V100 directory on the root of a direct-attached volume). But, it also boils down to the protocol being used between client and server. AFP supports the notion of network Spotlight searches, which uses a special AFP call (FPSpotlightRPC, which just happens to be undocumented) in order to search through the server's Spotlight Index over the network. Only the AFP protocol supports this call, even though Apple's own documentation for file serving in the 10.6 timeframe says that network spotlight is supported over SMB shares. (I've tested network spotlight between a Mac SMB client and Mac SMB server; it didn't work. It did work with AFP, however). In the end, network Spotlight is only supported with AFP, and iSCSI was mentioned as a way to reshare a portion of Isilon filesystem using iSCSI and AFP in order to provide network Spotlight for clients. Due to the layers of protocols and devices involved (AFP > Mac server > iSCSI > NAS), high performance would not be a guarantee. And, you'd be limited to 16TB HFS+ volumes on iSCSI LUNs. That defeats many of the benefits of scale out NAS.
I can certainly understand the interest in this topic! Running a command like mdls (from a Mac) against a file gives a wealth of Spotlight information that the Mac keeps for it. But, that same information is not created or Indexed on Isilon, and there's no way with NFS or SMB to share those Spotlight details over those protocols (the protocols just don't support the special Spotlight RPCs). One thing that could help here is if we somehow created a metadata index for the filesystem structure to speed up name-based searches, but generating that index would be a very time consuming process without something like the FSEvents system constantly keeping tabs on directories. That would be a significant undertaking. Instead, one thing we recommend are digital asset management (DAM) solutions that create their own indices based on Isilon storage and then share those out to clients directly. These sorts of solutions are very useful for media & entertainment workflows, in particular, as they offer a layer of abstraction between the client and the underlying storage, and also allow for workflow processes to be enforced.
Thanks for the questions and comments!
--Bernie
Peter_Sero
1.2K Posts
0
November 22nd, 2013 06:00
Thanks Bernie for sharing these insights!
It's kind of amazing what Apple can achieve with their pre-UNIX filesystem...
MRWA
83 Posts
0
November 22nd, 2013 08:00
This Ask the Expert event has now concluded, but the conversations continue here in the Isilon Support Forum. Thanks to Bernie Case for all his advice and insight!
Thank you to all those who participated in this event!
BernieC
76 Posts
0
November 22nd, 2013 08:00
Agreed! One such big additions to their file system was added with 10.5, when Time Machine was introduced. Using directory hard links in the file system is what makes Time Machine possible and very efficient.
And believe me, I have spent quite a lot of time trying to get Time Machine working via NFS or SMB. 10.5 and 10.6 could be tricked into doing backups via those protocols, but later versions really only support "unsupported" servers so long as they're running AFP :-(
Well, thanks for the ask the expert thread. It was fun!
ICPSR
10 Posts
0
March 13th, 2014 11:00
A couple notes...
1) We ran into problems with the finder having huge stalls when we mounted a directory with about 3,000 files. I do not remember if it was NFS or SMB, I kind of think I tried both. sorting things into smaller directories, under 1,000 each, made display about 100x faster.
2) Be very wary of running time machine against a non-apple-supported AFP server. netatalk has nominal support, but I and others I've talked to have had reliability problems with time machine and netatalk.