Hi Petter,

Please check the following Powerlink article about that warning message:

https://solutions.emc.com/emcsolutionview.asp?id=esg113499

Hope this helps you.

Thanks,

Ahmed Bahaa

Hi again,

That dosn't seem to help, from what I can find 'morning_cron_run' is located in /usr/local/avamar/bin/ and only on the utility node;

root@avamar:~/#: find /usr -name morning_cron_run -print

/usr/local/avamar/bin/morning_cron_run

And the gccount=64 flag is not set in that file or in the hfscheck_cron script... did I miss anything?

#!/bin/bash

echo 'morning_cron_run starting: '

date

ssh-add ~dpn/.ssh/dpnid

cd ~dpn

mccli mcs waitforflushcomplete

sleep 300

gc_cron

sleep 3600

cp_cron

sleep 300

hfscheck_cron --throttlelevel=1 --useschedule

sleep 300

cp_cron

health_check.pl > /usr/local/avamar/var/cron/health_check.log

echo 'morning_cron_run ending: '

date

The article referenced only applies to Avamar 4.x. The crons are no longer used in Avamar 5.x or higher.

The message means that some data is being swapped out of memory and onto disk by the Linux kernel running on the storage nodes. In Avamar 5.x and 6.x, a certain amount of swapping is normal and nothing to worry about.

If one particular node is reporting a lot of swapping, there may be a DIMM failure on that node. The output of the following command (run as the admin user with the SSH keys loaded) will tell you how much memory is available to the OS on each node. Take a look at the 'total' column.

mapall --noerror 'free'

Here is some example output from a Gen2 2TB node system:

admin@avamar1:~/>: mapall --noerror 'free'

Using /usr/local/avamar/var/probe.xml

(0.0) ssh  -x  -o GSSAPIAuthentication=no admin@10.5.161.109 'free'

             total       used       free     shared    buffers     cached

Mem:      16406984   16294104     112880          0     192984   15247028

-/+ buffers/cache:     854092   15552892

Swap:      4192924        224    4192700

(0.1) ssh  -x  -o GSSAPIAuthentication=no admin@10.5.161.110 'free'

             total       used       free     shared    buffers     cached

Mem:      16406984   16300976     106008          0     191108   15234216

-/+ buffers/cache:     875652   15531332

Swap:      4192924        224    4192700

(0.2) ssh  -x  -o GSSAPIAuthentication=no admin@10.5.161.111 'free'

             total       used       free     shared    buffers     cached

Mem:      16406984   16293400     113584          0     189612   15276172

-/+ buffers/cache:     827616   15579368

Swap:      4192924        224    4192700

(0.3) ssh  -x  -o GSSAPIAuthentication=no admin@10.5.161.112 'free'

             total       used       free     shared    buffers     cached

Mem:      16406984   16290824     116160          0     296492   15085516

-/+ buffers/cache:     908816   15498168

Swap:      4192924        224    4192700

This particular system has 16GB of memory on each node.

If one node is reporting less memory than the others, please contact support for assistance in resolving that issue.

My pleasure!

Ok, thanks! :-)

Thanks, all nodes looks ok;

(0.0) ssh  -x  admin@192.168.255.2 'free'

             total       used       free     shared    buffers     cached

Mem:      37007652   36824864     182788          0       1260   29688656

-/+ buffers/cache:    7134948   29872704

Swap:      4184892    3095424    1089468

(0.1) ssh  -x  admin@192.168.255.3 'free'

             total       used       free     shared    buffers     cached

Mem:      37007652   36822308     185344          0       1240   30089124

-/+ buffers/cache:    6731944   30275708

Swap:      4184892    2707176    1477716

(0.2) ssh  -x  admin@192.168.255.4 'free'

             total       used       free     shared    buffers     cached

Mem:      37007652   36820064     187588          0       1232   29887680

-/+ buffers/cache:    6931152   30076500

Swap:      4184892    2830192    1354700

Is there a way to disable the check for "server node %s is swapping"? Or isn't that recomended?

Regards,

Petter

I would not recommend disabling this message -- if one node reports the error over and over, it could be a symptom of a bigger problem such as a memory failure.

A change that should reduce the number of swapping messages reported by the nodes will be included in Avamar 6.0.2 and Avamar 6.1.