This post is more than 5 years old
5 Practitioner
•
274.2K Posts
0
1057
Sizing: How much SSD do I need to add to a cluster for metadata acceleration?
I have been trying to track down a way to determine how much SSD needs to be added to an existing cluster for metadata acceleration based on the amount of data that exists on the cluster already. I am aware of the recommendations in regard to percentages, but determining how much space on the SSD will be utilized is what I'm looking for. Any thoughts?
curtharpold
11 Posts
0
July 2nd, 2013 11:00
While you might be able to calulate the actual amount of metadata required, you can't turn on GNA with raw SSD capacity less than 1.5% of raw HDD. For the foreseeable future, it is safest to follow the guideline of 2% capacity (though you could go as low as 1.5% in current OneFS versions), and 20% of nodes.
Anonymous
5 Practitioner
5 Practitioner
•
274.2K Posts
0
June 30th, 2013 20:00
I recently had a co-worker say that he believed you could roughly determine the amount of metadata that will reside on SSD by taking a count of all of the files, and directories, on the cluster and multiplying by 512 Bytes, since this is the amount of data needed for the metadata for each file. Any comments on this figure and whether it is accurate, or relatively accurate?
Peter_Sero
1.2K Posts
0
August 20th, 2013 00:00
> count of all of the files, and directories, on the cluster and multiplying by 512 Bytes, since this is the amount of data needed for the metadata for each file. Any comments on this figure and whether it is accurate, or relatively accurate?
I dont' think so. That would mean 50 GB of SSD could hold
the metadata for 100 Mio files+dirs,
a mark we missed by at least a factor of 10 when we tried it ;-(
(No, the try wasn't based on that rule; but we
still had underestimated the SSD usage;
read on for a better estimation.)
With 6.5.5 we have finally observed a few KB of SSD per dir
(with 30 files per dir on average).
For files, the SSD metadata ranges from about 2KB/file for "small" (250KB)
files to several ten KB/file for larger files (e.g 100 KB SSD for a 100 MB file).
While this looks much worse than the 512Byte-per-file formula,
it still means that the SSD:HDD ratio can drop to the 0.1%(!) region
for large files (on 6.5.5 at least).
This explains why "exceptions" from the 2% rule can make sense,
and in fact can be implemented! It seems one is actually
encouraged to check it out, see:
GNA - Do not be afraid to ask for an Exception
(by William Griffith, Corporate Systems Engineer, EMC)
-- Peter