CloudPools recall via file pool policy

Question

[Previously posted on the TUG, but no responses there, so...]

Hi all,

We've just started using CloudPools to archive old data from our Isilon cluster to the cloud, to free up some space on the cluster.

However, what I've been unable to work out is if it's possible to use a file pool policy to recall files from the cloud.

So, if you have a file pool policy which says "Files in directory x/y/z with modification date older than 6 months ago, archive to CloudPool A", is it possible/necessary to have another policy which says "Files in directory x/y/z with modification date newer than 6 months ago, recall to Isilon"?

All I can find in the documentation is a statement that files can be fully recalled from the cloud with "isi cloud recall" (isi cloud archive recall with newer OneFS).

So, if you do want to bring back files in a policy-based manner, can this be done with SmartPools, or do we need to put in place cron jobs on the cluster which run "isi cloud recall"?

Cheers,

John

crklosterman · Answer

So serious response here, not trying to be facetious at all. But, why would you want to? (I mean as a job, why would you want to on a regular / routine basis that could be scheduled via cron.) The egress costs to read them back can be quite egregious [pun intended], which likely cancels out any savings from archiving them off in the first place.

Correct me if I'm wrong here, but my understanding is that:

If the file is read-back by a user and the user changes that file, then the new version is written back to the cluster's local disk. If it then ages out, the new version would be archived again using cloudpools. The VNX/CTA had a similar concept.

~Chris

JohnB999 · Answer

Thanks for your reply, this following paragraph isn't what I grasp from the documentation (OneFS CloudPools Administration Guide, 8.1.0) though:

Correct me if I'm wrong here, but my understanding is that:

If the file is read-back by a user and the user changes that file, then the new version is written back to the cluster's local disk. If it then ages out, the new version would be archived again using cloudpools. The VNX/CTA had a similar concept.

The guide reads:

When a user accesses a cluster and views the OneFS file system through a supported protocol (SMB, NFS, Swift, or HDFS), SmartLink files appear to be the original files. When the user opens a SmartLink file, OneFS automatically retrieves and caches as much data as needed from the cloud. This operation is called inline access. Any modifications the user makes to a file during inline access are updated in the file data stored in the cloud.

In addition to inline access, CloudPools also provides a CLI command to enable full recall of files from the cloud, in which case the SmartLink files are replaced by the actual files.

So I think my original question remains...if a user modifies a file, should it not be possible to modify that file so that it no longer stubbed, and brought back from the cloud, as it could be considered "fresh"?

Later in the guide there is more detail:

Inline access

CloudPools enables users connecting to a cluster through supported protocols to access cloud data by opening associated SmartLink files. This process is referred to as inline access. To the user connecting to OneFS through a supported protocol, a SmartLink file appears to be the original file. When the user opens a SmartLink file, CloudPools retrieves and caches cloud data locally. The user can view and edit the file as usual. CloudPools automatically retrieves and sends any updated file data back to the cloud so that the cloud contains the latest version.

Note

CloudPools offers inline access as a user convenience. However, CloudPools is designed mainly as an archival solution, and is not intended for storing data that is frequently updated. Such data should be left on the local cluster until it stabilizes and is ready for archival.

Isilon

CloudPools recall via file pool policy

Was this post helpful?