CloudPools CLI Commands only provide a maximum of 100,000 Elements

Some of the CLI commands for CloudPools have the potential to return many entries. For example:

isi cloud jobs files list <jobid>

The output of the above command may have several times more than 100,000 elements, depending on the number of files impacted by the job. Without the fix described, only the first 100,000 entries are listed, unless a Platform API (PAPI) call is made directly.

The reason for this is that the PAPI handler was modified to handle batch and page parameters, and can list entries after 100000. However, the parameters to do so were not implemented in the CLI. The fix adds batch and page parameters to the CLI.

This limitation is fixed in OneFS v8.0.0.7 and beyond. 

With the fix, the batch, limit, and page parameters must be specified together in order for the CLI command to output the intended results.

For Example:

# isi cloud jobs files list --job-id=49 --verbose --batch=1 --limit=100000 --page=1

# isi cloud jobs files list --job-id=49 --verbose --batch=1 --limit=100000 --page=2

# isi cloud jobs files list --job-id=49 --verbose --batch=1 --limit=100000 --page=3

...

As a workaround, prior to the fix being implemented, PAPI calls can be made directly using cURL. Example syntax would be:

# curl -u root:a -H "Content-type: application/json" 'https://100.77.1.231:8080/platform/3/cloud/jobs-files/4?batch=1&limit=100000&page=1' -k > /ifs/data/Isilon_Support/job49_filelist_page1

# curl -u root:a -H "Content-type: application/json" 'https://100.77.1.231:8080/platform/3/cloud/jobs-files/4?batch=1&limit=100000&page=2' -k > /ifs/data/Isilon_Support/job49_filelist_page2

# curl -u root:a -H "Content-type: application/json" 'https://100.77.1.231:8080/platform/3/cloud/jobs-files/4?batch=1&limit=100000&page=3' -k > /ifs/data/Isilon_Support/job49_filelist_page3