This post is more than 5 years old
52 Posts
0
6838
How many buckets does CloudBoost create ?
Hi, everyone. I know CloudBoost create two buckets by defualt. But can I create more bucket for CloudBoost to use?
This post is more than 5 years old
52 Posts
0
6838
Hi, everyone. I know CloudBoost create two buckets by defualt. But can I create more bucket for CloudBoost to use?
Top
geordiekorper
22 Posts
0
November 4th, 2016 06:00
CloudBoost will never create additional buckets, except as I mentioned already for Atmos or OpenStack Swift.
Also the largest CloudBoost object is less than a megabyte because incoming data is deduplicated, and compressed.
I guess that on Atmos or Swift if the incoming save set was very large CloudBoost would be forced to create a new directory or container to accommodate every new batch of 100,000 or million segments respectively.
geordiekorper
22 Posts
1
November 2nd, 2016 07:00
For object stores that have per bucket performance limitations CloudBoost does have the ability to do sharding across multiple buckets but there is no way to manually set the number of buckets used for this purpose.
HePeng1
52 Posts
0
November 2nd, 2016 18:00
Do you mean CloudBoost will create multiple buckets? If so, how many buckets CloudBoost could create?
geordiekorper
22 Posts
1
November 3rd, 2016 07:00
It is not something that you need to know about as an administrator. It is just one of the many things the engineers need to deal with when using different object stores. CloudBoost will create however many are needed to hold the data on the object storage being used. Unless you are using an object storage provider that has per bucket limits CloudBoost will only create the two buckets, one for the deduped chunks, and one for the automatic backups it does of itself for DR recoveries.
For example Atmos has a limit of 100,000 blobs per directory (similar to a bucket) and OpenStack Swift starts slowing down at about 1 million blobs per container. The public object stores on the other hand do this load-balancing at a lower layer than the bucket level so CloudBoost does not have to do it itself.
HePeng1
52 Posts
0
November 4th, 2016 01:00
Thanks a lot for your reply. One more question, will CloudBoost create multiple buckets when storing a big object or storing multiple objects simultaneously?
HePeng1
52 Posts
0
February 7th, 2017 18:00
If the object storage provider with S3 API has per bucket limit, CloudBoost will create the third bucket or not?
geordiekorper
22 Posts
0
February 8th, 2017 06:00
CloudBoost automatically creates whatever buckets it needs. Although I should add none of the supported object stores that use an S3 API would require it.
HePeng1
52 Posts
0
February 8th, 2017 23:00
So do you mean that CB would create another bucket if the bucket has reached the limit of objects when the object storage is using S3 API?
geordiekorper
22 Posts
0
February 9th, 2017 06:00
No I do not mean that. There are no object stores that use the S3 API that have a limit.
HePeng1
52 Posts
1
February 9th, 2017 17:00
If the cloud storage which is using S3 API has a limit of object per bucket and objects has reached the limit, how does CB deal with the situation?