NetWorker: CloudBoost Performance Troubleshooting

Increasing the performance of CloudBoost is a frequently asked question. This article covers several conditions that enable you to improve the performance of CloudBoost. 

Hardware Requirements

CloudBoost has defined hardware requirements to provide the best performance. This is often the root cause to performance issues. 

Recommended minimum requirements: 

• CPU: Eight cores
• RAM: 32 GB
• Hard Drive: SSD with at least 1:2000 + 30% of the planned raw storage* 

SSDs are the most frequently missed item in the recommended requirements. We request SSDs because the CloudBoost metadata operations have frequent read/writes leading to a large IOPS requirement. 

Configuration Settings

The configuration of CloudBoost can lead to immediate ingress limitations during the initial upload of data and throughout the life of the product. 

Site-cache Enabled
Enabling site-cache is only recommended if you have low bandwidth (<100 Mbps) or high latency (>300 millisecond). Enabling site-cache stages data to the CloudBoost appliance where it can be queued up, pending upload to your specified cloud store. Site-cache ingress is limited to 25 MB/s. This can be slower than WAN capabilities within your network and is counterproductive for high-performance requirements, while frequent read operations may be improved.

NetWorker Client Devices
CloudBoost supports multiple client device inputs. In a high-performance environment, it is suggested to setup multiple CB-type devices within NetWorker. Configure each device with at least four PSS (parallel save streams).

You can have multiple NetWorker clients using the client-direct model to communicate directly with the cloud store. This helps offload a significant amount of load on the CloudBoost appliance, instead placing it on the client host. Ensure that any network performance validation that is completed on the CloudBoost device is also performed on each client.

Performance Adjustments

CloudBoost can make several adjustments to extend the performance. 


Client Parallel Requests
There is a client that is built into CloudBoost and NetWorker storage nodes with the ability to adjust parallel requests that it can make to the cloud store. Use the Blobstore Bench (often seen as BSB) to identify the correct parameter.**  
See KB article 9645: CloudBoost: Adjust NetWorker Storage Node for CloudBoost Performance for instructions on making this adjustment. 

CloudBoost Average Chunk Size
The average chunk size of CloudBoost can be a determining factor on performance. The general rule is that the higher the chunk, the faster the network performance. The opposite is correct when it comes to your deduplication ratio, the lower the chunk size, the better the deduplicate ratio. You must find a balance that you are comfortable with. This change could increase the total chunks that are sent to your cloud store, potentially leading to an increased cost. 
See KB article 3763: Adjusting Average Chunk Size for instructions on making this adjustment. 

Tools

Within the CloudBoost CLI, there are available and hidden tools. We go over blobstore-cli and MTR. You can use these for validating your cloud store credentials and ensure CloudBoost functionality, but you can also use these for performance validation. 

Blobstore-cli

Blobstore Validation (BSV)

1. Sign into the CloudBoost CLI as the user: admin
2. Run the command:  diagnostics blobstore-cli "--provider <ENTER PROVIDER> --identity <ENTER ACCESS KEY> --credential <ENTER SECRET KEY>" validate

See KB article 13717: CloudBoost: Validate Cloud Store Credentials (BSV) for instructions on your specific cloud store.

Blobstore Benchmark (BSB)

1. Sign into the CloudBoost CLI as the user: admin
2. Run the command:  

diagnostics blobstore-cli "--provider <PROVIDER> --endpoint <URL_IF_REQUIRED>  --identity <ID> --credential <SECRET_KEY> benchmark --num-parallel-requests 16 --blob-size 262144 --max-runtime 60 --container <CONTAINER_NAME> WRITE"

The WRITE operation is used for testing backup performance, while the READ operation is used for testing the restore performance. 

See KB article 10167: CloudBoost: Benchmark Cloud Store (BSB) for instructions and parameters to be used with BSB. 

It is suggested that you try several different size options with parallel requests first, such as 16, 64, and 128. Choose the parallel requests with the highest bandwidth and lowest acceptable latency and try with different chunk sizes, such as 64 KB, 256 KB, and 1 MB.  

Other Tools

Using the diagnostics command provides additional useful tools for troubleshooting network functionality. MTR is a frequent tool for performance tests. 

MTR
MTR combines the functionality of traceroute and ping into useful output. 

1. Sign into the CloudBoost CLI as the user: admin
2. Run the command: mtr <DOMAIN/FQDN>:<PORT>

Note: Do not include http/https within the domain. 

* 1:2000 is a rough estimate and enough for deployment. There are many variables that could cause this requirement to be increased, such as average chunk size, deduplicate ratio, and average change rate. 

** Contact Dell Customer Support for help with client performance tests using BSB. This test requires a separate binary.