Isilon and ICAP integration - Policy scans do not work

Question

Hi all, I have some trouble with an Symantec ICAP integration to the Isilon. At OneFS the ICAP server is configured. The system does recognize the ICAP server, the AV service at the cluster is up and running and there are no error messages at the Isilon log files. When I do a manual single file scan through the CLI the eicar test virus is identified and quarantined. But doing a policy scan of that folder does not work. Neither the virus is identified nor any files are scanned. Looking at the av scan details of each file there is always the same output:    Last scan:   never    Scan result: never scanned    Last ISTag:     Scan status: not current    Quarantined: false For the Antivirus scan settings there are no filename restrictions configured and the file size restriction is set as recommended to a maximum file size of 2 GB. On Access Scans are disabled. My policy only contains the directory and no more other specific settings. I would expect after starting a policy that each file is sent to the ICAP server and is checked there for possible viruses. After the scan, the result of the scan is sent back to Isilon and can be displayed there via CLI for each file. But at the moment it seems that no file is scanned. I know that there is a scan result caching. The NAS antivirus service caches scanning results for each clean file. The cached information includes the date and revision number of the virus definitions that were used to perform the scan. So, if a second user requests access to a file that has already been scanned and if the virus definitions have not changed, a redundant scan is avoided. But for that caching each file must be scanned at least once, what does not happen currently... What's going wrong here?

crklosterman · Answer

Can all nodes of your cluster talk to all of the ICAP servers configured? I do mean all nodes. ICAP work is sent to every single node whether it has a network connection or not, and regardless of what subnet it's interfaces are on. That's the most common source of problems that I see, people trying to use ICAP with NANON/NENON (not all nodes on network/ not every node on network).

try this:

isi_for_array -s "ping -c 1 "

and make sure it works on every single node.

~Chris

philippspohr · Answer

No, that is not the reason. All Nodes are connected to the external network and each Node reaches the ICAP server.

scott_owens · Answer

Which Symantec product are you using?

Go.Y · Answer

Phil As far as I understand, for Policy Scan, you must wait until AVscan operation finished to get which file was affected. Is the operations already finished?

philippspohr · Answer

Hi go.y,

each policy scan only needs less than a second until it finished although there are some files and of course the test virus files at the scanned folders. Here is one report output:

Report ID: R:5732dd69:17e8

Policy ID:    5732dba617e84

Status:       Finish

Start time:   05-11-2016 09:21:13

End time:     05-11-2016 09:21:13

Duration:     -

Files:        0

Size:         0

Sent Bytes:   0

Threats:      0

Band   -

I would expect that all files of that folder and of all subfolders are sent to the ICAP server for the scan. The file count and the "Sent Bytes" should not be zero.

philippspohr · Answer

Hi scott,

we are using Symantec(TM) Protection Engine for Network Attached Storage (NAS) version 7.5.

The isi config output is

ICAP server 1:

URL:          icap://xx.xx.xx.xx (enabled)

Description: Symantec Protection Engine

Status:       alive, virus defs 20160511.055

Glob filters: disabled, include patterns

Limit real-time scans to prefixes:

/ifs/data/xxx/Test

Remediations: repair, quarantine

Max scan size: 2147483647

Scan on open: disabled

fail open:    enabled

Scan on close: disabled

Report expiry: 604800

Go.Y · Answer

Phil, Could you also provide following information if possible? # isi avscan policy # sqlite3 /ifs/.ifsvar/modules/avscan/isi_avscan.db .dump # isi verison

philippspohr · Answer

Hi go.y, of course.

The policies are

Policy id:        572e2ea317e82

Status:         enabled

Name:           Test

Paths:

/ifs/data/xxx/Test

Recursion depth: unlimited

Force:          disabled

Last run:       05-11-2016 09:07

Policy id:        5732dba617e84

Status:         enabled

Name:           Test2

Paths:

/ifs/data/xxx/Test

Recursion depth: unlimited

Force:          enabled

Last run:       05-11-2016 09:21

Here is the db dump:

PRAGMA foreign_keys=OFF;

BEGIN TRANSACTION;

CREATE TABLE Scans (ReportID text UNIQUE ON CONFLICT IGNORE, PolicyID text, Start blob, End blob, Status text, NumFiles blob, Size blob, BytesSent blob, NumInfections blob, Duration blob, RestriperID integer);

INSERT INTO "Scans" VALUES('RO572d3000','SCAN_ON_OPEN',1462636608,0,'Started',1,68,336,1,0,-1);

INSERT INTO "Scans" VALUES('RC572d3000','SCAN_ON_CLOSE',1462640831,0,'Started',0,0,0,0,0,-1);

INSERT INTO "Scans" VALUES('R:572e326d:b63c','MANUAL',1462645357,1462645358,'Succeeded',1,68,337,1,0,-1);

INSERT INTO "Scans" VALUES('R:572e34bd:baa9','572e2ea317e82',1462645949,1462645949,'Finish',0,0,0,0,0,840);

INSERT INTO "Scans" VALUES('R:572e34fc:17e8','572e2ea317e82',1462646012,1462646013,'Finish',0,0,0,0,0,841);

INSERT INTO "Scans" VALUES('R:572e353b:17e8','572e2ea317e82',1462646075,1462646075,'Finish',0,0,0,0,0,842);

INSERT INTO "Scans" VALUES('R:572e3557:bcae','MANUAL',1462646103,1462646103,'Succeeded',1,68,337,1,0,-1);

INSERT INTO "Scans" VALUES('R:572e37d2:17e8','572e2ea317e82',1462646738,1462646739,'Finish',0,0,0,0,0,843);

INSERT INTO "Scans" VALUES('R:572e3976:17e8','572e2ea317e82',1462647159,1462647159,'Finish',0,0,0,0,0,844);

INSERT INTO "Scans" VALUES('R:5732d526:17e8','572e2ea317e82',1462949158,1462949159,'Finish',0,0,0,0,0,855);

INSERT INTO "Scans" VALUES('R:5732d68d:17e8','572e2ea317e82',1462949517,1462949517,'Finish',0,0,0,0,0,856);

INSERT INTO "Scans" VALUES('R:5732d8da:1837','MANUAL',1462950106,1462950107,'Succeeded',1,68,337,1,0,-1);

INSERT INTO "Scans" VALUES('R:5732d936:17e8','5732d92e17e83',1462950198,1462950198,'Finish',0,0,0,0,0,857);

INSERT INTO "Scans" VALUES('R:5732da19:17e8','572e2ea317e82',1462950425,1462950425,'Finish',0,0,0,0,0,858);

INSERT INTO "Scans" VALUES('R:5732dce1:17e8','5732dba617e84',1462951137,1462951138,'Finish',0,0,0,0,0,859);

INSERT INTO "Scans" VALUES('R:5732dd69:17e8','5732dba617e84',1462951273,1462951273,'Finish',0,0,0,0,0,860);

CREATE TABLE Infections (ReportID text, Filename text, Time blob, Result text, FileInfected text, VirusName text);

INSERT INTO "Infections" VALUES('R:572e326d:b63c','/ifs/data/xxx/Test/eicar.com',1462645358,'Quarantined','eicar.com','EICAR Test String');

INSERT INTO "Infections" VALUES('R:572e3557:bcae','/ifs/data/xxx/Test/eicar.com',1462646103,'Quarantined','eicar.com','EICAR Test String');

INSERT INTO "Infections" VALUES('RO572d3000','/ifs/data/xxx/eicar-2.com',1462646959,'Quarantined','eicar-2.com','EICAR Test String');

INSERT INTO "Infections" VALUES('R:5732d8da:1837','/ifs/data/xxx/Test/eicar.com',1462950107,'Quarantined','eicar.com','EICAR Test String');

CREATE INDEX ReportIDIdx ON Infections (ReportID);

CREATE INDEX TimeIdx ON Infections (Time);

CREATE INDEX StatusIdx ON Scans (Status);

CREATE INDEX StartIdx ON Scans (Start);

CREATE INDEX EndIdx ON Scans (End);

CREATE INDEX PolicyIdIdx ON Scans (PolicyID);

COMMIT;

As we can see in the dump, the manual singe file scan and the scan on access seems to be working, but not the policy scans.

And here is the OneFS version:

Isilon OneFS v7.2.0.5 B_7_2_0_212(RELEASE): 0x7020050005000D4:Fri Dec 18 00:19:16 GMT 2015

root@sea-build7-03:/b/mnt/obj/b/mnt/src/sys/IQ.amd64.release clang version 3.3 (tags/RELEASE_33/final)

Go.Y · Answer

Phil,

It looks like, policy Test and Test2 seems to scan same path.

If yes, delete "Test" policy, and add a copy of a infected eicar file as "eicar3" under the path.

Then run the Test2 policy again.

If it doesn't work try following.

1. access to CLI as root user.

2. stop isi_avscan_d service

# isi services -a isi_avscan_d disable

3. rename the isi_avscan.db .dump

# cd /ifs/.ifsvar/modules/avscan/

# mv isi_avscan.db isi_avscan.db.bk

4. restart isi_avscan_d

# isi services -a isi_avscan_d enable

5. Start Test2 policy

philippspohr · Answer

Hi go,y,thanks a lot. I think stopping and restarting the service daemon fixed the issue.I have deleted all policies, disabled the service, enabled the service again, created a new policy and surprise: Now the policy is working with the same settings.

Go.Y · Answer

Phil,I'm glad to hear that.Next time same thing happen, I think you should open a SR and find the root cause with EMC.

tim.koopman · Answer

Go. Y,

I had similar issue after upgrading from 8.0.0.4 to 8.1.2.0 OneFS. My scheduled Avscan jobs would start, run, in one minute and complete successful. The issue is the job was not scanning files. I stopped the service, moved the database, and restarted the service and now the one avscan job that I have ran is working. Thank you for your earlier post.

Isilon

Was this post helpful?