NTFS file attribute support - sparse

I'm pretty sure that OneFS ignores the FILE_ATTRIBUTE_SPARSE_FILE bit (as well as a few other bits). So your sparse file copy is likely just resulting in a file full of zeros on OneFS.

I'm not 100% sure on this, so your best course of action is to connect with Isilon Support so one of our Windows/SMB engineers can comment with more specifics...

--kip

Hello,

FILE_ATTRIBUTE_SPARSE_FILE

The FSCTL_SET_SPARSE IOCTL (see [MS-FSCC]) is used to mark a given file as sparse or not sparse. When a file system that supports sparse files, i.e. NTFS, receives an FSCTL_SET_SPARSE IOCTL request on a file, it should set or unset the FILE_ATTRIBUTE_SPARSE_FILE attribute of the specified file.

If you attempt to clear the FILE_ATTRIBUTE_SPARSE_FILE attribute, the operation is valid only if there is no sparse region in the file. The FSCTL_QUERY_ALLOCATED_RANGES (see [MS-FSCC]) control code is used to determine whether there are sparse regions in a file.

In Windows NTFS file systems, files are not made sparse by default. The application or user needs to explicitly mark the file sparse via the FSCTL_SET_SPARSE control code.  Unlike Unix or Linux file systems, when a file is created on Windows and some data is written at an offset, let assume at 1 MB offset, NTFS will allocate clusters and fill in all space from 0 to 1 MB.  This means, the file has to be explicitly marked sparse before it becomes sparse.

Marking a file as sparse

This can be done by calling DeviceIoControl() with the FSCTL_SET_SPARSE file system control code.

For example:

A file can be associated with more than one application at a time, such as Microsoft® Word and Microsoft® WordPad. For instance, a file structure like the following illustrates file association, but not multiple files:

program:source_file

:doc_file

:object_file

:executable_file

Reason could be because NTFS is not supported on floppy disks, when you copy an NTFS file to a floppy disk, data streams and other attributes not supported by FAT are lost without warning.

I have  pasted link below which will help you answer your query.

https://technet.microsoft.com/en-us/library/cc938926.aspx

http://blogs.msdn.com/b/openspecification/archive/2010/11/05/notes-on-sparse-files-and-file-sharing.aspx

http://www.flexhex.com/docs/articles/sparse-files.phtml

Thanks

Hemant

Hemant,

Thank you for the information.  My question may not have been as clear as I thought.  I am looking for sparse attribute support on OneFS in the SMB world.  I can create a sparse file in a share on a Windows server.  I have been able to copy that file with a data migration tool to other NAS devices and the sparse attribute is maintained.  Using a share on an Isilon cluster as the destination the sparse attribute is not maintained.

I have opened a SR with EMC and research is being done.  So far OneFS 7.0.x does not support the sparse file attribute for SMB.  Still waiting to find out about OneFS 7.1.x.

Thanks kipcranford!

I checked out our latest "head" code, and FILE_ATTRIBUTE_SPARSE_FILE is in a data structure that implies it's being ignored.  So now I'm 99.9% sure we won't set that bit (and this applies to all OneFS releases)...