In the Avamar Server software, the index is used to keep track of data chunks on the system. Think of the index like a card catalog in a library - it lets us look up where in the system a particular chunk of data is stored.
At the beginning of the backup window, we load as many index entries as we can into memory to improve backup performance. We call this the "index cache" (or "index caches"). For each data chunk that is not found in the client-side caches, the client will send a request to the server to determine if the server already has the chunk (this might happen if the same chunk was backed up by another client). This is called an "is present" lookup. These caches stay loaded in memory until the end of the blackout window because Garbage Collection performs many lookups as well. The index caches are unloaded each day when the maintenance window starts, primarily because we need to free the memory so hfscheck can use it.
The index entries are not stored contiguously on the disk so if we had to go to the disk every time we wanted look up a hash, we would have to peform a seek and this would incur a performance penalty. Each individual client communicating with the Avamar server may send thousands of lookup requests to the server and we would pay this performance penalty for each lookup. Loading the indexes into memory means we can respond to each lookup request much more quickly (ns instead of ms) and without having to seek the disk. Naturally, the process of loading these index caches into memory (or unloading them) causes a heavy I/O load on the system. The system should resume responding normally once the loading (or unloading) of the indexes completes. While the system may respond more slowly than normal, there should be no operational impact caused by the delay.
This is not related to the MCS or EMS flushes (though these flushes are backups and do take advantage of the index caches if they are loaded).
Some one else might be able to address the relation to the MC_BACKUP and EM_BACKUP, but I can describe the intended workings of the indexcacheloader.
The "indecies" stored in the index cache is the master hash index tables in the Avamar GSAN. Durring the normal backup window the hash indicies are stored in memory, this alows the Avamar server to respond very quicly to clients that are querying the Avamar server to see if a given hash entry is already stored on the Avamar server. This is called an "Is Present" call.
At the end of the backup window in the "Black out/Maintenance" window, the Index cache is removed from memory and written back to disk by the indexcacheloader. This is to free up memory for the performance of Maintnance Activities. When the master hash index tables are no longer in memory, "Is Present" calls can take longer because they require a disk seek on the avamar server. After the maintnance activities have completed, the indexcacheloader places the indecies back into memory.
ionthegeek
2 Intern
•
2K Posts
0
February 29th, 2012 12:00
In the Avamar Server software, the index is used to keep track of data chunks on the system. Think of the index like a card catalog in a library - it lets us look up where in the system a particular chunk of data is stored.
At the beginning of the backup window, we load as many index entries as we can into memory to improve backup performance. We call this the "index cache" (or "index caches"). For each data chunk that is not found in the client-side caches, the client will send a request to the server to determine if the server already has the chunk (this might happen if the same chunk was backed up by another client). This is called an "is present" lookup. These caches stay loaded in memory until the end of the blackout window because Garbage Collection performs many lookups as well. The index caches are unloaded each day when the maintenance window starts, primarily because we need to free the memory so hfscheck can use it.
The index entries are not stored contiguously on the disk so if we had to go to the disk every time we wanted look up a hash, we would have to peform a seek and this would incur a performance penalty. Each individual client communicating with the Avamar server may send thousands of lookup requests to the server and we would pay this performance penalty for each lookup. Loading the indexes into memory means we can respond to each lookup request much more quickly (ns instead of ms) and without having to seek the disk. Naturally, the process of loading these index caches into memory (or unloading them) causes a heavy I/O load on the system. The system should resume responding normally once the loading (or unloading) of the indexes completes. While the system may respond more slowly than normal, there should be no operational impact caused by the delay.
This is not related to the MCS or EMS flushes (though these flushes are backups and do take advantage of the index caches if they are loaded).
Jeff_Cameron
133 Posts
0
February 29th, 2012 12:00
Some one else might be able to address the relation to the MC_BACKUP and EM_BACKUP, but I can describe the intended workings of the indexcacheloader.
The "indecies" stored in the index cache is the master hash index tables in the Avamar GSAN. Durring the normal backup window the hash indicies are stored in memory, this alows the Avamar server to respond very quicly to clients that are querying the Avamar server to see if a given hash entry is already stored on the Avamar server. This is called an "Is Present" call.
At the end of the backup window in the "Black out/Maintenance" window, the Index cache is removed from memory and written back to disk by the indexcacheloader. This is to free up memory for the performance of Maintnance Activities. When the master hash index tables are no longer in memory, "Is Present" calls can take longer because they require a disk seek on the avamar server. After the maintnance activities have completed, the indexcacheloader places the indecies back into memory.