Highlighted
MRWA
2 Iron

Ask the Expert: OneFS 7.1 Waikiki Preview.

Welcome to this EMC Support Community Ask the Expert conversation.

YOU MAY ALSO BE INTERESTED ON THESE ATE EVENTS...

Ask the Expert: Isilon Performance Analysis

Ask the Expert: AIMA: Everything you wanted to know but were afraid to ask.

Ask The Expert – Isilon’s New Releases: IsilonSD Edge, OneFS.NEXT and CloudPools

This is an opportunity to learn about and discuss the newest upcoming OneFS release code name WAIKIKI!

November brings the release of OneFS 7.1 for Isilon products, which includes a number of improvements and new features, including:

  • Job Engine version 2: In OneFS 7.1, the job engine can run multiple maintenance jobs simultaneously, while dynamically modifying those jobs to ensure that impact to production workflows is minimized.
  • Deduplication: OneFS 7.1 supports sharing blocks amongst multiple files via shadow stores, reducing overall space usage.
  • SMB Auditing with Varonis DatAdvantage: OneFS 7.1 supports auditing of all SMB traffic to the cluster via the Common Event Enabler (CEE) framework to Varonis DatAdvantage.
  • Data at Rest Encryption: OneFS 7.1 introduces support for nodes with Self-Encrypting Drives, or SEDs, allowing data to be protected against physical hard drive theft.

… along with a number of smaller features and fixes. This is your opportunity to get the skinny on the next release of OneFS!


This discussion begins on 10/7/2013 and concludes on 10/18/2013 Get ready by bookmarking this page or signing up to receive email notifications.

Your host: Jim Gossett

profile-image-display.jspa?imageID=8195&size=350

Work

Occupation:Technical Support Engineer V
Biography:I have a Bachelor's degree in Computer Science from the University of Washington (go Huskies!)

I've worked in development, QA, IT, and support/escalations at various companies large and tiny for the past 20 years.

I started at Isilon Systems as an escalations engineer in 2007, and came over to EMC when Isilon was acquired.
Expertise:Currently, I'm the "Future Products Subject Matter Expert", so my general area of expertise is "what is coming next for Isilon products, and how to we train and document it?"
Company:EMC
City:Seattle
State:WA
Country:USA
48 Replies
MRWA
2 Iron

Re: Ask the Expert: OneFS 7.1 Waikiki Preview.

This Ask the Expert thread is now open!!

0 Kudos
Peter_Sero
3 Zinc

Re: Ask the Expert: OneFS 7.1 Waikiki Preview.

Where are the demo Virtual Nodes?

For the actual discussion, I'd mostly be interested in performance:

- NFS performance in general?

- changes to layout/handling of (lots of) small files?

- NDMP/SyncIQ scans (could benefit from snaphot-like change tracking)?

- smarter restripe operations?

Thanks

-- Peter

jasdav02
1 Copper

Re: Ask the Expert: OneFS 7.1 Waikiki Preview.

I am fairly excited about the new job engine, is there any quick stats about how much this improves job speed on a typical cluster? One of our biggest issues is having cluster load interrupt running jobs.


I know that it wasn't mentioned but has there been any work on getting SmartConnect to behave with NFSv4? I realize that with v4 we move to a more stateful connection which like SMB doesn't really play all that nicely with transparently failing connections over node-to-node.


On that end, any updates on pNFS support?

0 Kudos
skoburn
1 Copper

Re: Ask the Expert: OneFS 7.1 Waikiki Preview.

Hi Jim,

Will deduplication require a license?

Thanks!

0 Kudos
jimgossett
1 Nickel

Re: Ask the Expert: OneFS 7.1 Waikiki Preview.

Hi Peter!

>Where are the demo Virtual Nodes?

They will be made available the same time as the code is -- at the very tail end of October or early November.

Regarding your individual points:

- NFS performance in general?

There are no changes specifically to NFS in OneFS 7.1; however, to your next point...

- changes to layout/handling of (lots of) small files?

There were some fairly invisible changes made to the way that we balance flushing journal writes to disk, and subsequent refactoring of delay inside the I/O scheduler, that should lead to IOPS improvements in lots-of-small-file workflows. Layout of small files did not change, but workflows like user home directories should see some improvement.

- NDMP/SyncIQ scans (could benefit from snaphot-like change tracking)?

SyncIQ is already using Snapshots for change tracking in any Sync job after the first; OneFS 7.1 includes "continuous" sync mode, wherein instead of running a Sync policy once per hour or day, the cluster is continuously checking for changes to a given directory (using snapshot change tracking) and syncing data as soon as possible when changes are detected. This isn't ideal for high change datasets, but it can be very helpful for low change datasets where it is necessary to propagate changes as quickly as possible.

NDMP will be using snapshot-based change tracking in OneFS 7.1 to support faster incremental backups. 

- smarter restripe operations?

The new OneFS job engine will help with restripe operations by allowing multiple jobs to run simultaneously, and by scaling impact to the cluster according to available resources (within pre-set limits). Can you provide some more detail as to what you are looking for in "smarter" restripe operations?

Jim

jimgossett
1 Nickel

Re: Ask the Expert: OneFS 7.1 Waikiki Preview.

Hi jasdav02!

>I am fairly excited about the new job engine, is there any quick stats about how much this improves job speed on a typical cluster? >One of our biggest issues is having cluster load interrupt running jobs.


Individual jobs should run at approximately the same speed they normally do in the Job Engine v2 world (unless the cluster is not seeing significant load, in which case the job engine will start to use additional resources up to a pre-defined ceiling.)


The big gain here is that in OneFS 7.1, for instance, if a drive fails and a Smartfail job is started, this will not stop regularly scheduled SnapshotDelete jobs from running alongside it; neither job will need to be paused or pre-empted, but they will run in tandem. Some classes of jobs are still not allowed to run at the same time (jobs that do similar types of work); I'll detail those a bit further in a post soon.


>I know that it wasn't mentioned but has there been any work on getting SmartConnect to behave with NFSv4?


Can you comment further on issues you saw with SmartConnect failover and NFSv4? No work was specifically done in OneFS 7.1 around this area, but there has been a lot of lock-oriented work going on in the latest 7.0.1 builds to address problems customers were seeing that may be related to your specific concerns.


> On that end, any updates on pNFS support?


It's still on our internal roadmaps, but no updates I can share at this time. Sorry!


Jim

jimgossett
1 Nickel

Re: Ask the Expert: OneFS 7.1 Waikiki Preview.

Hi skoburn!

Deduplication (or "DedupeIQ") will require a license to use in OneFS 7.1.

Jim

0 Kudos
Peter_Sero
3 Zinc

Re: Ask the Expert: OneFS 7.1 Waikiki Preview.

Hello Jim,

thanks a lot for your promising answers!

Restripes could have some headroom for improvements in my view:

MultiScan after a disk replacement shuffles around

lots of blocks in the whole pool rather than picking some

blocks from N-1 disks to be migrated to the new disk, N.

Maybe this is  a very naive view, but it might illustrate that

improvements in this direction could be possible.

Restripes also can have a strange effect (according to my

observations) on the distribution of inodes over the drives.

A pool with an even distribution of ca 2 Mio inodes/drive

(replicated; as reported by isi statist drice --long) turned into a

distribution of  25 x 2.4 Mio inodes/drv plus 8 x 800K inodes/drv.

While this is still being investigated (OneFS 6.5.5), I wonder wether

such unbalance is avoided with current/future releases of

the restriper.

-- Peter

0 Kudos
Peter_Sero
3 Zinc

Re: Ask the Expert: OneFS 7.1 Waikiki Preview.

> Deduplication (or "DedupeIQ") will require a license to use in OneFS 7.1.

Will the "dry run" for testing the space saving need a license, too?

What is the overhead cost on disk and in memory?

-- Peter

0 Kudos