Avamar: Space reclamation process Part 3:  The Remove Checkpoint Process (RMCP)

This is the third and last in a series of knowledge base articles which discuss Avamar space reclamation processes.

The article focuses on the Remove Checkpoint (rmcp) process. This is an activity which runs as part of the maintenance cycle and frees up disk space by deleting old checkpoints.  

The first parts of the space reclamation process are garbage collection and crunching which have been covered in the other articles:

• Avamar: Space reclamation process Part 1: Garbage Collection
• Avamar: Space reclamation process Part 2: Crunching

What is rmcp and why is it needed?

The Avamar server rmcp process is a task which runs on data nodes to clear up stale checkpoints.

This ensures that hard drive space in the Avamar data partitions (and any integrated Data Domain storage), can be freed up and reused.

If rmcp does not run, what problems might occur?

Problems with the rmcp process are relatively rare. Where they do occur, high operating system capacity may be seen due to too many checkpoints, or failing maintenance activities due to contention with a long-running rmcp process.

When does rmcp run?

The rmcp process runs as part of the daily maintenance process and occurs after a checkpoint has been created:

dumpmaintlogs --types=cp --days=1

cp logs start for last 1 day(s) of cp activity (note: all times UTC)
2016/04/03-08:02:02.68874 {0.0} <4300> starting scheduled checkpoint maintenance
2016/04/03-08:02:02.68975 {0.0} <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
2016/04/03-08:02:02.68976 {0.0} <cpflags delete="true" keepmin="false" waittime="1800"/>
2016/04/03-08:03:01.63624 {0.0} <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
2016/04/03-08:03:01.63625 {0.0} <checkpoint tag="cp.20160403080204" isvalid="true" refcount="1" cpctime="1459670524" cpseqno="499" nodestotal="1" stripestotal="2595" hfsctime="1437733723" dirstotal="3" deletable="false" ddr="true" locked="false" sufficient="true" complete="true" checkable="true" nodecount="1">
2016/04/03-08:03:01.63626 {0.0}   <hfscheck validcheck="false" type="full" checks="unknown" starttime="0" nodestarttime="0" nodefinishedtime="0" errors="0">
2016/04/03-08:03:01.63627 {0.0}     <hfscheckerrors/>
2016/04/03-08:03:01.63628 {0.0}   </hfscheck>
2016/04/03-08:03:01.63629 {0.0}   <nodeidlist count="1">
2016/04/03-08:03:01.63630 {0.0}     <nodeidrange dcno="0" lseqno="0" useqno="0"/>
2016/04/03-08:03:01.63631 {0.0}   </nodeidlist>
2016/04/03-08:03:01.63632 {0.0}   <ddr isvalid="true" validcheck="false"/>
2016/04/03-08:03:01.63633 {0.0} </checkpoint>
2016/04/03-08:03:01.69849 {0.0} deleting extra checkpoints
2016/04/03-08:03:03.02651 {0.0} <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
2016/04/03-08:03:03.02652 {0.0} <checkpointrmlist has-approved-checkpoint="true">
2016/04/03-08:03:03.02653 {0.0}   <checkpoint tag="cp.20160402080224" deleted="false" ddr-deleted="false"/>
2016/04/03-08:03:03.02654 {0.0}   <checkpoint tag="cp.20160402081456" deleted="false" ddr-deleted="false"/>
2016/04/03-08:03:03.02655 {0.0}   <checkpoint tag="cp.20160403080204" deleted="false" ddr-deleted="false"/>
2016/04/03-08:03:03.02656 {0.0} </checkpointrmlist>
2016/04/03-08:03:03.02812 {0.0} <4301> completed checkpoint maintenance