JohnB999

11 Posts

2827

November 2nd, 2018 05:00

CloudPools recall via file pool policy

[Previously posted on the TUG, but no responses there, so...]

Hi all,

We've just started using CloudPools to archive old data from our Isilon cluster to the cloud, to free up some space on the cluster.

However, what I've been unable to work out is if it's possible to use a file pool policy to recall files from the cloud.

So, if you have a file pool policy which says "Files in directory x/y/z with modification date older than 6 months ago, archive to CloudPool A", is it possible/necessary to have another policy which says "Files in directory x/y/z with modification date newer than 6 months ago, recall to Isilon"?

All I can find in the documentation is a statement that files can be fully recalled from the cloud with "isi cloud recall" (isi cloud archive recall with newer OneFS).

So, if you do want to bring back files in a policy-based manner, can this be done with SmartPools, or do we need to put in place cron jobs on the cluster which run "isi cloud recall"?

Cheers,

John

Responses(3)

C

crklosterman

450 Posts

0

November 5th, 2018 08:00

So serious response here, not trying to be facetious at all. But, why would you want to? (I mean as a job, why would you want to on a regular / routine basis that could be scheduled via cron.) The egress costs to read them back can be quite egregious [pun intended], which likely cancels out any savings from archiving them off in the first place.

Correct me if I'm wrong here, but my understanding is that:

If the file is read-back by a user and the user changes that file, then the new version is written back to the cluster's local disk. If it then ages out, the new version would be archived again using cloudpools. The VNX/CTA had a similar concept.

~Chris

J

JohnB999

11 Posts

0

November 6th, 2018 15:00

Thanks for your reply, this following paragraph isn't what I grasp from the documentation (OneFS CloudPools Administration Guide, 8.1.0) though:

Correct me if I'm wrong here, but my understanding is that:

If the file is read-back by a user and the user changes that file, then the new version is written back to the cluster's local disk. If it then ages out, the new version would be archived again using cloudpools. The VNX/CTA had a similar concept.

The guide reads:

When a user accesses a cluster and views the OneFS file system through a supported protocol (SMB, NFS, Swift, or HDFS), SmartLink files appear to be the original files. When the user opens a SmartLink file, OneFS automatically retrieves and caches as much data as needed from the cloud. This operation is called inline access. Any modifications the user makes to a file during inline access are updated in the file data stored in the cloud.

In addition to inline access, CloudPools also provides a CLI command to enable full recall of files from the cloud, in which case the SmartLink files are replaced by the actual files.

So I think my original question remains...if a user modifies a file, should it not be possible to modify that file so that it no longer stubbed, and brought back from the cloud, as it could be considered "fresh"?

Later in the guide there is more detail:

Inline access

CloudPools enables users connecting to a cluster through supported protocols to access cloud data by opening associated SmartLink files. This process is referred to as inline access. To the user connecting to OneFS through a supported protocol, a SmartLink file appears to be the original file. When the user opens a SmartLink file, CloudPools retrieves and caches cloud data locally. The user can view and edit the file as usual. CloudPools automatically retrieves and sends any updated file data back to the cloud so that the cloud contains the latest version.

Note

CloudPools offers inline access as a user convenience. However, CloudPools is designed mainly as an archival solution, and is not intended for storing data that is frequently updated. Such data should be left on the local cluster until it stabilizes and is ready for archival.

A

Anonymous

5 Practitioner

•

274.2K Posts

0

November 13th, 2018 05:00

To directly address the initial question:

No, files can not be recalled from the cloud via FilePool policy. Files are also not recalled when they no longer match a FilePool policy. When a FilePool policy finds a file matching a policy which archives to the cloud, that file is then replaced with a SmartLink file. Any modifications to FilePool policies after that point will apply to the SmartLink file (for example, moving that SmartLink file to another pool), but will not recall data from the cloud.

isi cloud recall will call the Isilon Job Engine to do a tree walk (Via a CloudPoolsTreeWalk hidden job), which does use the job engine to mark files for recall, and initiate a related cloud recall job in the cloud job engine.

I can understand a possible use for a method to have a file recalled and no longer be a SmartLink when modified by a user, but that option does not currently exist within OneFS CloudPools.

View All

No Events found!