CloudBoost will never create additional buckets, except as I mentioned already for Atmos or OpenStack Swift.
Also the largest CloudBoost object is less than a megabyte because incoming data is deduplicated, and compressed.
I guess that on Atmos or Swift if the incoming save set was very large CloudBoost would be forced to create a new directory or container to accommodate every new batch of 100,000 or million segments respectively.
For object stores that have per bucket performance limitations CloudBoost does have the ability to do sharding across multiple buckets but there is no way to manually set the number of buckets used for this purpose.
It is not something that you need to know about as an administrator. It is just one of the many things the engineers need to deal with when using different object stores. CloudBoost will create however many are needed to hold the data on the object storage being used. Unless you are using an object storage provider that has per bucket limits CloudBoost will only create the two buckets, one for the deduped chunks, and one for the automatic backups it does of itself for DR recoveries.
For example Atmos has a limit of 100,000 blobs per directory (similar to a bucket) and OpenStack Swift starts slowing down at about 1 million blobs per container. The public object stores on the other hand do this load-balancing at a lower layer than the bucket level so CloudBoost does not have to do it itself.
Thanks a lot for your reply. One more question, will CloudBoost create multiple buckets when storing a big object or storing multiple objects simultaneously?
CloudBoost automatically creates whatever buckets it needs. Although I should add none of the supported object stores that use an S3 API would require it.
geordiekorper
22 Posts
5939
0
Posted November 4th, 2016 06:00
CloudBoost will never create additional buckets, except as I mentioned already for Atmos or OpenStack Swift.
Also the largest CloudBoost object is less than a megabyte because incoming data is deduplicated, and compressed.
I guess that on Atmos or Swift if the incoming save set was very large CloudBoost would be forced to create a new directory or container to accommodate every new batch of 100,000 or million segments respectively.