Unsolved
This post is more than 5 years old
21 Posts
0
801
May 16th, 2017 23:00
Get Specific Path Directory Size - REST API Namespace (Recursion?)
Hello Everyone,
So I have a sort of weird issue I was hoping to pick your brains on- I have customers that would like to get directory specific size stats, however the directory isn't always the same one (ex. different shares, exports, etc) and the nesting isn't always the same, some of these shares are given to end uses and they build their own file structure with serious nesting.
So a basic example, "rusty" is the export (arbitrary btw)
Isilon1:/ifs/prod/nfs/rusty
However, a user could have folders like so
Isilon1:/ifs/prod/nfs/rusty/opt/appdata/temp/file_1/object_Store/anotherfolder/evenmore/etc
And there are likely files, not just directories at each layer.
I'm familiar with InsightIQ and the size calculation SmartQuotas does, but I'd like to setup a basic approach to this with a somwhat STaaS mentality where an app/server admin can go to this application/script give me the path they want and receive statistics in return.
So I observed that the REST endpoint of "?detail=default" gives some values on "Type" and "Size", when "Type" = "ojbect" it is a file and the size is relevant/correct however when "Type" = "container" the size is generic. My idea is to hit the RAN, and have a recursive function loop through each set of objects adding size to a grand total variable until there are no more "containers" or directories. But I read in the API guide that deep recursion is discouraged, is it a performance consideration or something else?
Maybe there is an easier way just reaching out...
My messy [pseudo] code so far:
$isilon_path = //Maybe data from a POST from a form or dropdown for the user- web front end?
function call_isilon(){
$http_header = array('Content-Type: application/json','Authorization: Basic HahaYeahRightNoCredsHere);
$ch = curl_init("https://".$isilon_cluster."/namespace/".$isilon_path."?limit=10000&detail=default");
curl_setopt($ch, CURLOPT_HTTPHEADER, $http_header);
curl_setopt($ch, CURLOPT_TIMEOUT, 60);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
$curl_response = curl_exec($ch);
if ($curl_response === false) {
$info = curl_getinfo($ch);
$error = curl_error($ch);
curl_close($ch);
echo "
The error:
";
print $error;
echo "
More details
";
die('error occured during curl exec. Additional info: ' . var_export($info));
}
curl_close($ch);
$decoded_json = json_decode($curl_response, true);
$all_objects[] = $decoded_json['children'];
foreach ($all_objects as $directories){
foreach($directories as $dir){
$dir_name = $dir['name'];
$dir_size = 0;
$dir_size + $dir[size];
$dir_type = $dir[type];
$dir_mark = "/";
if ($dir_type == "container"){
$isilon_path = $isilon_path.$dir_mark.$dir_name;
call_isilon($isilon_path);
}
}
}
echo $dir_size ;
}
call_isilon($isilon_path);



Peter_Sero
4 Operator
•
1.2K Posts
0
May 17th, 2017 07:00
You may find this approach to work well in a test setup, but things
can break when it comes to scale... it might be simply too slow to be useful,
hard to make it robust wrt to connection timeouts etc,
running out of memory, ...
But why not give it a shot, and stress test the tool along
the above lines, before making it nice and user-friendly.
Best of luck
-- Peter