Hi Rob,

OneFS 7.2.0.3

InsightIQ 3.2.2

We don't have established reporting on the file data, so we don't keep just some of the numbers and delete the rest. Having the reports allows us to look back and see how and where data has changed easily (the compare function is great).

For the cost of saving the reports (few TB for a PB cluster) is an easy decision for us.

Hi Adam,

That is a very interesting retention requirement.  Could you share with us why you need to keep that many FSA result set?  Maybe there's a better way to get what you need.

Also what are your version numbers for...

• OneFS
• InsightIQ

Thanks!

Adam,

how do you retain FSA reports in InsightIQ ?  I would like to keep a year worth of reports where today it only has last 7 or so.

Thank you

Hi dynamox,

In IIQ,

1. go to Settings tab

2. click "configure" for the cluster

3. click "FSA Configuration" tab

4. Result Set Retention

Result Set Retention

Thank you Rob. Where are the FSA reports stored, on the cluster itself or on the IIQ appliance (which in my case uses an NFS export from Isilon for its datastore)

dynamox

They are stored on the cluster, in the "/ifs/.ifsvar/modules/fsa/pub/" directory.

as rob said, increase the "Maximum result set count". Also if you want to keep specific reports (stop them from being deleted automatically),

1. go to Settings tab

2. click "configure" for the cluster

3. click "Manage FSA Result Sets" tab

4. find the report you wish to keep

5. check the pinned box

nice, i have set to 500 as well. My cluster seems to complete fsanalyze job every 2 weeks so this should give me reports to go way back.

Hi dynamox,

keep in mind, I have issues with the cluster keeping lots of reports. perhaps by the time you get that many the issues will be resolved (assuming your cluster/s get kept up to date)

Hi Adam,

For your original question -- I'll let others on the forum to chime in on how many FSA results they keep.  I suspect not many customers retain as many FSA result sets as you do.  Part of the reason is that on some of the larger clusters FSA jobs can take days to weeks to complete.  The upcoming OneFS 8.0 release has something pretty awesome to address that.  Stay tuned.

I've also filed 2 bugs against Engineering -- 1 for FSA job unable to start due to unable to allocate memory, and 1 for FSA/IIQ not expiring FSA results as specified in the "Maximum result set count".

Thanks for your input.

Hi dynamox,

Adam beat me to it.

The FSA reports that IIQ relies on always reside on-cluster, and is never part of IIQ's datastore.  I don't know the reason behind the design, but I suspect security have something to do with it.  InsightIQ's own datastore stores only performance-related metrics.

The FSA result set are located at -- /ifs/.ifsvar/modules/fsa/pub

The files are sqlite3 database files, one could open them up with a DB browser that reads sqlite3 files.

Example below, where the job numbers in the directory name reflects the job engine job numbers in OneFS:

[root@hop-isi-g-1 /ifs/.ifsvar/modules/fsa/pub]# ls -l

total 548

drwxr-xr-x 2 root  wheel  3004 Dec 14 22:01 job.287

drwxr-xr-x 2 root  wheel  3004 Dec 15 22:01 job.293

drwxr-xr-x 2 root  wheel  3004 Dec 16 22:01 job.303

drwxr-xr-x 2 root  wheel  3004 Dec 17 22:01 job.309

drwxr-xr-x 2 root  wheel  3004 Dec 18 22:01 job.315

drwxr-xr-x 2 root  wheel  3004 Dec 19 22:01 job.321

drwxr-xr-x 2 root  wheel  3004 Dec 20 22:01 job.328

drwxr-xr-x 2 root  wheel  3004 Dec 21 22:01 job.334

lrwxr-xr-x 1 root  wheel     7 Dec 21 22:01 latest -> job.334

[root@hop-isi-g-1 /ifs/.ifsvar/modules/fsa/pub]# ls -lSh job.334

total 23174

-rw-r--r-- 1 root  wheel   768K Dec 21 22:01 results.db

-rw-r--r-- 1 root  wheel   256K Dec 21 22:01 list_top_n_files_by_atime.db

-rw-r--r-- 1 root  wheel   256K Dec 21 22:01 list_top_n_files_by_btime.db

-rw-r--r-- 1 root  wheel   256K Dec 21 22:01 list_top_n_files_by_ctime.db

-rw-r--r-- 1 root  wheel   256K Dec 21 22:01 list_top_n_files_by_log_size.db

-rw-r--r-- 1 root  wheel   256K Dec 21 22:01 list_top_n_files_by_phys_size.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 disk_usage_2_4299031803.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 disk_usage_4299031803_4299097997.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 disk_usage_4299097997_4299428955.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 disk_usage_4299428955_4299556229.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 disk_usage_4299556229_4299622631.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 disk_usage_4299622631_4299752760.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 disk_usage_4299752760_4299818634.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 disk_usage_4299818634_4300933193.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 disk_usage_4300933193_4301193818.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 disk_usage_4301193818_72057594037927936.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 list_top_n_dirs_by_atime.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 list_top_n_dirs_by_btime.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 list_top_n_dirs_by_ctime.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_all.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_disk_pool_atime_attribute.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_disk_pool_atime_log_size.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_disk_pool_atime_path_ext.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_disk_pool_atime_phys_size.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_disk_pool_attribute_ctime.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_disk_pool_attribute_log_size.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_disk_pool_attribute_path_ext.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_disk_pool_attribute_phys_size.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_disk_pool_ctime_log_size.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_disk_pool_ctime_path_ext.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_disk_pool_ctime_phys_size.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_disk_pool_log_size_path_ext.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_disk_pool_path_ext_phys_size.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_path_squash_atime_attribute.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_path_squash_atime_log_size.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_path_squash_atime_path_ext.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_path_squash_atime_phys_size.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_path_squash_attribute_ctime.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_path_squash_attribute_log_size.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_path_squash_attribute_path_ext.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_path_squash_attribute_phys_size.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_path_squash_ctime_log_size.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_path_squash_ctime_path_ext.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_path_squash_ctime_phys_size.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_path_squash_log_size_path_ext.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 stats_path_squash_path_ext_phys_size.db

-rw-r--r-- 1 root  wheel   128K Dec 21 22:01 system_squash_map.db

-rw-r--r-- 1 root  wheel    33K Dec 21 22:01 disk_usage_2_4299031803.db-journal

-rw-r--r-- 1 root  wheel    33K Dec 21 22:01 disk_usage_4299031803_4299097997.db-journal

-rw-r--r-- 1 root  wheel    33K Dec 21 22:01 disk_usage_4299097997_4299428955.db-journal

-rw-r--r-- 1 root  wheel    33K Dec 21 22:01 disk_usage_4299428955_4299556229.db-journal

-rw-r--r-- 1 root  wheel    33K Dec 21 22:01 disk_usage_4299556229_4299622631.db-journal

-rw-r--r-- 1 root  wheel    33K Dec 21 22:01 disk_usage_4299622631_4299752760.db-journal

-rw-r--r-- 1 root  wheel    33K Dec 21 22:01 disk_usage_4299752760_4299818634.db-journal

-rw-r--r-- 1 root  wheel    33K Dec 21 22:01 disk_usage_4299818634_4300933193.db-journal

-rw-r--r-- 1 root  wheel    33K Dec 21 22:01 disk_usage_4300933193_4301193818.db-journal

-rw-r--r-- 1 root  wheel    25K Dec 21 22:01 disk_usage_4301193818_72057594037927936.db-journal