AV-Comparatives November/2010 Proactive/Retrospective Tests Results

Depressing indeed.

It's also "feels" rather "quirky", in that Avast5 (which I use) nonetheless received an "advanced" certification, despite its detection rate being NEXT TO LAST !!!

Of course, as noted in the article:  "Some products maybe had the ability to detect some samples e.g. on execution or by other monitoring tools, like behaviour blocker, reputation/cloud heuristics, etc.... [which] are outside the scope of this retrospective test".   As a specific example, avast includes a "Code emulator:  When a suspicious executable is encountered (during both on-demand and on-access scanning), avast! is able to emulate the program’s code in an isolated environment. The code emulator is used for two purposes. First, it is used for generic unpacking. Secondly, it is used in the heuristics engine. Technically, this is done using dynamic translation, a method much faster than traditional emulation techniques" which would NOT have been taken into consideration in this report.

Depressing indeed. It's also "feels" rather "quirky", in that Avast5 (which I use) nonetheless received an "advanced" certification, despite its detection rate being NEXT TO LAST !!!

As AV-C explained in its methodology, any program that detected 25-50 % of malware, and had "None to few" false positives, qualified for Advanced certification. Those are pretty liberal criteria, as NO program tested rated lower than Advanced.

But your points are well-taken. These are not real-life tests of any total layered security protection strategy. Nor are they new revelations- we have been seeing low proactive detection rates for years from AV-C.

Yet my PCs, whether using Avira, avast!, NOD32, or MSE have not been compromised for many years. I suspect you could claim the same. Which tends to confirm my bias that the choice of AV is not nearly so important as other factors in preventing infection. How otherwise would we have avoided the ~40-50% of malware our AVs are not detecting proactively?

Nonetheless, these objective tests do represent a narrow slice of comparative testing, and perhaps have more relevance to those that rely on AVs only, all other things being equal.

I agree RD.

The Whole Product Dynamic test of 2009 showed much better blocking of a limited test sample by most products. In that test, avast and MSE were neck and neck in blocking.

AV-C will be releasing the 2010 Whole Product Dynamic test results shortly, using a much larger test sample conducted over several months. The major problem is that only commercial security suites seem to be tested, and I'm not a fan of these suites in general, with the possible exception of MSE+Windows native firewall. Sadly, the latter is not included in these tests. (Nor is AVG or McAfee- possibly because they fared not so well by comparison in the 2009 test?).

I am particularly disappointed that McAfee is not being tested this year, as it seems to be the predominant pre-installed Security Suite that Dell offers (at least on the systems I have been looking at recently), and might be of more interest to readers of this board.

"Some products maybe had the ability to detect some samples e.g. on execution or by other monitoring tools, like behaviour blocker, reputation/cloud heuristics, etc.... [which] are outside the scope of this retrospective test".

An important point there. In the real world, with an active Internet connection, HIPS and the technologies mentioned above enabled (if available), I have no doubt each product would perform better at detecting zero-day threats.

Never been a big fan of this particular test from AV-C, and can understand why some vendors abstained. Much prefer the whole product dynamic tests myself.

Please don't get me started on McAfee! While their software served me well in the past, I'm having trouble bringing myself to defend them anymore.

Last year, there was a major problem with their SystemGaurds (light HIPS) component that went on for several months. In the end, instead of fixing the problem, they decided it would be easier to drop SystemGaurds altogether from future home product releases.

And then in their 2010 software, some "genius" decided to introduce a feature called Smart Timer. The function of this being that if the PC is in use, updates are held off until it is idle. Do they not realize that most home users turn off their systems when not in use. I swear... I once went nearly 3 days without an update from McAfee!

I could go on but I think I've made my point :emotion-2: .