Start a Conversation

Unsolved

This post is more than 5 years old

12084

March 5th, 2010 14:00

Celerra CAVA errors

Hi,

We got high water mark errors recently on the Celerra NS80. I checked the server logs and I found the below logs. Anybody know what this error means. Please let me know and that helps me a lot.

2010-02-05 04:43:55: VC: 3: 30: Error on CAVA server xxxx: OFFLINE, ntStatus: SUCCESS.

2010-02-05 04:44:05: VC: 3: 30: Error on CAVA server xxxxx OFFLINE, ntStatus: SUCCESS.

2010-02-05 04:44:55: VC: 3: 30: Error on CAVA serverxxxxxx: OFFLINE, ntStatus: SUCCESS.

2010-02-05 04:44:55: VC: 5: 29: Server xxxxx is online.

2010-02-05 04:44:57: VC: 5: 29: Server xxxxxx is online.

2010-02-05 05:00:22: VC: 3: 2: high water mark reached

2010-02-05 05:03:08: VC: 4: 3: low water mark reached.

Thanks for your help on advance.

Bannu.

March 5th, 2010 14:00

The first error messages make me think there's a problem with the anti-virus product on each server.  The CAVA client on each server seems to be able to communicate with the the CAVA service on the NAS - hence the ntStatus: SUCCESS.  However, the AV product seems to be unavailble to the CAVA client, hence the OFFLINE message.  At the same time, the CAVA service suddenly starts responding, hence the Server is ONLINE message.  The Low/High watermark messages were caused by an outsanding number of scan requests that came while no CAVA servers were available.

Without more info, I'd guess that the AV client might have updated its virus definitions and restarted.  Or, there could have been network issues between the CAVA servers and the NAS.  Or it might be stray backgound radiation.   Are there any messages in the Event Logs on the servers?

Thanks!

Karl

March 8th, 2010 09:00

This sounds similar to past experiences.  Under McAfee, the default maximum scan time is 45 seconds.  When the scan time exceeds this limit, the current scan is aborted.  Under McAfee 8.0i, the abort is actually the exit of the service - McAfee stops and restarts the service.  This causes CAVA to mark the AV server as offline - you'll probably have scans that were sent to this AV server and sitting in the wait queue, hence, the high water mark.

We upgraded to McAfee 8.5i patch 2 and seems like the problem has gone away.  If you can, upgrade one of your CAVA servers to 8.5i and see if things improve.  You'll probably want to open a ticket with EMC if upgrading McAfee has no effect.

Be sure to comment on your progress here.

Thanks!

99 Posts

March 8th, 2010 09:00

Hi Karl,

Thanks for the prompt response!

I saw the event logs but did not find any errors related to this issue. But, I found below EMC primus solution could be the reason. Our NAS code is 5.6, CAVA version is 3.6.2 and McAfee is 8.0i. In the primus it is saying that we have to at 8.5i patch7. What are you comments?

Thanks,

Bannu.

EMC Knowledgebase
spacer
spacer"CAVA has detected termination of the resident Network Associates."
spacer
spacerspacerspacer
ID:emc206027
Usage:7
Date Created:01/27/2009
Last Modified:03/16/2009
STATUS:Approved
Audience:Customer
Knowledgebase Solution

Environment:Product: Celerra
Environment:Feature: Celerra AntiVirus Agent (CAVA) 3.6.2
Environment:SW EMC: NAS Code 5.5
Environment:EMC Software: NAS Code 5.6
Environment:

NAI McAfee VirusScan: 8.5i Patch 2 and and later

Problem:VC: 4:16:  Virus Checking STARTED
VC: 3: 8:  Server 161.228.78.5: AV_NOT_FOUND, RPC program version 3, CAVA release: 3.6.2, AV Engine: Unknown
VC: 3: 5:  No Virus Checker Server available, STOP VIRUS CHECKING
Problem:CAVA Event Viewer Application Log shows the following event multiple times:

CAVA has detected the termination of the resident Network Associates.  CAVA will no longer process Celerra AV requests until the resident antiviurs software is restarted.

Problem:McAfee service pauses and restarts for up to 45 seconds at a time, sometimes more.  These restarts can be seen in the CAVA Event Logs.
Root Cause:The timeout between when Network Associates goes down and starts up again sometimes takes too long and CAVA therefore stops.
Fix:In this case, the McAfee software was upgraded to McAfee 8.5i Patch 7 and the problem was resolved.

spacer

99 Posts

March 11th, 2010 08:00

Hi Karl,

What are the McAfee upgrade steps:

Can I stop the viruschecker service on all DMs

Upgrade the McAfee to 8.5i patch7 and start the VC service? Will it updates the Virusdefinition file. Means will it scans all the files again.

Could you please let me know.

Thanks,

Bannu.

March 11th, 2010 11:00

Hi Bannu -

There's no need to stop the viruschecker service.  You can upgrade McAfee on each CAVA server, one at a time, until the upgrade is completed.  While McAfee is being reloaded, the CAVA agent on the server won't be able to find a valid virus scanning engine.As you reboot each CAVA server after installing McAfee 8.5i, the CAVA service will contact the DM to identify itself.  If you use the command 'server_viruschk server_2', you should see a list of all the configured CAVA servers and the version of McAfee on them.  Keep upgrading your CAVA servers and make sure that they restart CAVA correctly and are ready for scanning.  Just be sure that each CAVA server comes back up and ready to scan before shutting down the next one.  If you're too fast, you might have all your CAVA servers down at once, causing the CIFS service to stop.  Keep us posted with your progress!

Thanks!

Karl

99 Posts

March 16th, 2010 11:00

Hi Karl,

We are in the process of upgrading. I am seeing the below entries in the VC log all the day(couple of times).

  2010-03-16 00:23:47: VC: 5: 18: The VC server xxx.x.xxxx.x updated the reference time, reference=Tue Mar 16 04:23:46 2010 (GMT-00:00).
2010-03-16 00:31:45: VC: 5: 18: The VC server xxx.x.xxxx.x updated the reference time, reference=Tue Mar 16 04:31:25 2010 (GMT-00:00).
2010-03-16 01:20:45: VC: 5: 18: The VC server xxx.x.xxxx.xupdated the reference time, reference=Tue Mar 16 05:19:51 2010 (GMT-00:00).
2010-03-16 01:26:46: VC: 5: 18: The VC server xxx.x.xxxx.x updated the reference time, reference=Tue Mar 16 05:26:46 2010 (GMT-00:00).

I checked the CAVA server time and DM time which are almost same(may be 3secs diffrence). Is something that goes away after McAfee upgrade?

Any comments on the above entries?

Thanks,

Bannu.

March 16th, 2010 13:00

Hi Bannu -


The reference time updates indicate that the McAfee process updated its definition files.  This happens whenever the AutoUpdate or Startup Update task runs and updates the VirusScan definitions.  These messages are usually harmless, if they're coming from different CAVA servers at different times.  However, if all your servers are set to update their virus definitions at the same time, this can cause an outage, as all AV servers could be down at the same time.  You should consider setting the AutoUpdate task to run at a different time on each server (possibly several hours apart), so that no two servers should be updating at the same time.

Thanks!

Karl

99 Posts

March 16th, 2010 13:00

Thanks Karl for your quick response!

We have 4 AV servers having the same Autoupdate time 12AM daily. So, can I keep them 12AM, 1AM, 2AM, 3AM for AV servers. Is this a good idea?

Thanks again.

Bannu.

March 16th, 2010 14:00

Hi Bannu - it sounds like setting them to update at 12AM through 3AM would be great. Be sure to look at the settings for Randomization in the Scheduler:

Capture.PNG

Make sure that you reduce the randomization time to less than one hour.  Otherwise, there's a 'random' chance two servers might update at the same time.

Thanks!

Karl

99 Posts

March 17th, 2010 13:00

Thankyou very much Karl for your quick response.

Where can I find  E-lab for CAVA on powerlink.

Thanks,

Bannu.

March 17th, 2010 13:00

Sure thing -

Start from the Powerlink home page > Support > Interoptability and Product Lifecycle Information > E-Lab Interoperability Manager

Select Software:Antivirus Software and check 'McAfee' under NAI; Add to cart

Select Server: Operating System and check 'DART' under EMC; Add to cart; Get Results

Under 'NAS', take a look at the resulting document 'Celerra Virus Scanning Solution'

Thanks!

Karl

99 Posts

March 17th, 2010 13:00

Karl,

We have 8.5i patch8 with our admins. But according to emc primus206027 we need patch7. Is it OK to apply patch8.

Thanks,

Bannu.

99 Posts

March 17th, 2010 13:00

Hey Karl,

Thanks for your help.

I came to know this 4 CAVA AV servers are VMs. Still the operation and eveything should be same as physical machine right?

Bannu.

March 17th, 2010 13:00

Yes, Bannu - that's correct.  The operation of the VM CAVA servers is identical to the physical CAVA servers.  All of the servers have the same configuration - same amounts of RAM and processor, and behave basically the same.

Thanks!

Karl

March 17th, 2010 13:00

In my environment, we are running McAfee 8.7i patch 1.  We were running McAfee 8.5i patch 3 when we upgraded.  According to the E-Lab Interoperability Navigator, it says "8.5.i patch 2 and higher is supported".  If you're not 100% sure, install it on just one CAVA servers and see how it behaves.

Thanks!

Karl

No Events found!

Top