Avamar: How to identify large files or directories consuming disk space on an Avamar node

1. Log in to the node as admin.

2. Elevate to root privilege.

3. Review the current capacity of partitions. (This example focuses on the root (/) partition, however these steps can be applied to any full partition.)

df -h

Filesystem            Size  Used Avail Use% Mounted on
/dev/sda5             7.9G  7.9G  0.0G 100% /
udev                  5.8G  252K  5.8G   1% /dev
tmpfs                 5.8G     0  5.8G   0% /dev/shm
/dev/sda1             114M   54M   55M  50% /boot
/dev/sda3             256G  131G  125G  52% /data01
/dev/sda7             1.5G  209M  1.2G  15% /var 

4. Confirm that the du command also reports that the partition (in this case /) is full:

du -xh / --max-depth=1

8.0K    /var
8.0K    /initrd
6.3M    /bin
0       /proc
0       /selinux
4.0K    /data01
2.0K    /boot
20M     /sbin
8.0K    /bootalt
4.0K    /space
8.0K    /varalt
16K     /lost+found
767M    /opt
96K     /tmp
105M    /root
3.6G    /usr
25M     /etc
16K     /mnt
300K    /.avamardata
7.9G    /

If du does not report that root (/) is full, then deleted files may still be held by running processes.

5. Identify the folders which are consuming the most space on the disk. 

This command shows the top 25 folders by size on the root partition:
(The command can be altered to check /data01, /var, or any other folder. It can also be altered to check more or less top folders)

du -xm / --max-depth=10 | sort -nr | head -25 

Sample output:

6299    /
5106    /usr
3210    /usr/local
2274    /usr/local/avamar
1478    /usr/local/avamar/bin
876     /usr/local/avamar-tomcat-7.0.59
864     /usr/local/avamar-tomcat-7.0.59/webapps
772     /usr/local/avamar/lib
509     /opt
436     /usr/share
393     /usr/lib
333     /usr/lib64
312     /usr/java
253     /root
232     /opt/EMC/TransportSystemService
232     /opt/EMC
228     /usr/bin
205     /opt/emc-third-party/platform/suse-11-x64
205     /opt/emc-third-party/platform
205     /opt/emc-third-party
196     /usr/sbin
184     /root/.avamardata/var/mc/cli_log
184     /root/.avamardata/var/mc
184     /root/.avamardata/var
184     /root/.avamardata 

An alternative approach is to look for all very large files.  

The following command will search for all files over 1GB in size on the current partition only:

find / -mount -size +1000000k -exec du -h {} \;

Sample output:

6.8G    /proc/kcore
find: /proc/19798/task/19798/fd/4: No such file or directory
find: /proc/19798/fd/4: No such file or directory
1.2G    /data01/avamar/src/downloads/VMWARE_PROXY/AvamarCombinedProxy-linux-sles11_64-6.1.102-47.ova
1.1G    /data01/avamar/src/downloads/ARCHIVE/AvamarCombinedProxy-linux-sles11_64-7.0.100-427.ova
3.5G    /data01/avamar/repo/packages/UpgradeClientDownloads-7.2.0-401.avp
1.9G    /data01/avamar/var/rabbitmq/log/rabbit@localhost.log.1
2.1G    /data01/avamar/var/rabbitmq/log/rabbit@localhost-sasl.log.1
1.1G    /data01/avamar/var/mc/server_data/postgres/data/base/16389/18598.5
1.1G    /data01/avamar/var/mc/server_data/postgres/data/base/16389/18390.3
1.1G    /data01/avamar/var/mc/server_data/postgres/data/base/16389/18598.4
1.1G    /data01/avamar/var/mc/server_data/postgres/data/base/16389/18598.1
1.1G    /data01/avamar/var/mc/server_data/postgres/data/base/16389/18598
19G     /data01/avamar/var/mc/server_data/mcs_data_dump.sql.restore
19G     /data01/avamar/var/mc/server_data/mcs_data_dump.sql
2.5G    /data01/home/admin/logs.160628.074208.tar
1.5G    /data01/home/admin/ays/usr/local/avamar/var/mc/server_data/mcs_data_dump.sql
2.5G    /data01/home/admin/logs.160719.120257.tar
3.5G    /data01/home/admin/nric/UpgradeClientDownloads-7.2.0-401.avp

6. Review the following article for a list of known scenarios or issues that can cause a disk partition to become full: Avamar: Disk partition full but not due to 'cur' or checkpoint overhead (RESOLUTION PATH)