Success and Failure in Antivirus Lab Test

Malware writers constantly come up with new tricks to fight or evade antivirus protection, and antivirus writers just as constantly work on new ways to detect and defeat malware attacks. That also means that the researchers who test and evaluate antivirus products must constantly test and retest. The product that blew the doors of a test in April might totally tank in November.

German lab AV-Test Institute publishes its mainstream test results six times per year. In the latest test, four products earned a perfect score, while three others couldn't do any better than the baseline established by Microsoft's built-in antivirus.

The Tests
AV-Test rates products on three criteria: Protection, Performance, and Usability. Products can earn up to six points in each area, for a maximum of 18 points. To pass the test, a product must manage a total of 10 or better, with no zero scores in any of the three areas.

The Protection rating is based on how well a product prevented infestation by over 20,000 prevalent malware samples and around 150 extremely new zero-day threats. Quite a few products managed 100 percent or 99.9 percent detection, earning the full six points.

If your antivirus sucks up so much CPU time that it slows your computer use, you may be tempted to turn it off. Performance is thus a security issue as well. To measure each program's performance drain, AV-Test's researchers time 13 typical user actions with and without the antivirus installed. These include copying files, downloading from the Internet, and installing programs, among other things.

Naturally the antivirus must remove bad files, but it also needs to refrain from erroneously removing valid files. The more false positive identifications, the lower the antivirus's Usability. Researchers test each antivirus by scanning almost 1.5 million legitimate files. They also visit 500 valid websites, and check whether the antivirus blocks or warns about actions taken by legitimate file installers.

For more detail about test methods, check out AV-Test's website.

The Winners
In the previous test, two months ago, only Bitdefender Antivirus Plus 2016 £19.99 at Bitdefender UK managed to reach a perfect score, 18 points. This time around, there's more room at the top. Kaspersky Anti-Virus (2016)£19.99 at Kaspersky UK also attained a perfect score. Both of these have earned our Editors' Choice honor for commercial antivirus.

Avira Antivirus 2015 also managed a perfect score, as did Symantec Norton Security Premium £39.99 at NortonLifeLock. Norton is an Editors' Choice for cross-platform multi-device security.

And The Losers
Microsoft has been doing better in tests recently, which really raises the bar for other vendors. In this test and the previous one, Microsoft scored a respectable 14 points. Many previous tests saw Microsoft failing to reach the 10 point cutoff.

Comodo Antivirus 8 See it at Comodo Antivirus also got 14 points. Like Microsoft, it only scored 3.5 of 6 possible points in the important Protection category. You might offer the excuse that Comodo is a free product, but then, so is winner's circle member Avira.

The other two 14-pointers, G Data Antivirus 2015 and K7 Antivirus Plus 14 , didn't fare as poorly in the Protection test. G Data earned 5.0 points and K7 got 4.5. A 3.0 score for performance was G Data's downfall. K7 simply lost points in each of the three areas. Comodo also scored 14 points in the previous test, while G Data had 16 points.

Keep Testing
AV-Test and the other independent labs perform an invaluable service. Without them, we'd have no idea which antivirus products work best. In addition, the results of their tests let the antivirus vendors know where they need to improve. I rely on their results as a component of my own antivirus reviews. Thanks guys; keep it up!

Image courtesy of Flickr User Chris Potter.

About Our Expert

Neil J. Rubenking

Principal Writer, Security

My Experience

When the IBM PC was new, I served as the president of the San Francisco PC User Group for three years. That’s how I met PCMag’s editorial team, who brought me on board in 1986. In the years since that fateful meeting, I’ve become PCMag’s expert on security, privacy, and identity protection, putting antivirus tools, security suites, and all kinds of security software through their paces.

Before my current security gig, I supplied PCMag readers with tips and solutions on using popular applications, operating systems, and programming languages in my "User to User" and "Ask Neil" columns, which began in 1990 and ran for almost 20 years. Along the way, I wrote more than 40 utility articles, as well as Delphi Programming for Dummies and six other books covering DOS, Windows, and programming. I also reviewed thousands of products of all kinds, ranging from early Sierra Online adventure games to AOL’s precursor Q-Link.

In the early 2000s, I turned my focus to security and the growing antivirus industry. After years of working with antivirus, I’m known throughout the security industry as an expert on evaluating antivirus tools. I serve as an advisory board member for the Anti-Malware Testing Standards Organization (AMTSO), an international nonprofit group dedicated to coordinating and improving testing of anti-malware solutions.

The Technology I Use

Much of the testing I do, particularly testing with real-world ransomware, is just plain dangerous. To perform such tests safely, I sequester them inside virtual machines managed by VMWare Workstation. For cross-platform testing, I use a MacBook Air, a Google Pixel 4, and a 6th-generation iPad.

I rely on my Delphi coding skills to create and maintain small applications. These include programs to check whether an antivirus correctly handled the malware it detected, launch dangerous URLs and record the security program’s reaction, and analyze the malware that I collect for use in testing. I also wrote a tiny browser and text editor for use in testing security apps that have predefined reactions for known products.

I do my writing and research on a Dell OptiPlex desktop, relying on Microsoft Word (my fingers know all the shortcuts). Many of my articles include charts and analysis; Excel is my go-to for those. When work hours end, though, I escape the bounds of Microsoft and Windows. There’s an iPhone in my pocket, I relax with my oversized iPad, and my Kindle Oasis is always loaded with the best science fiction and fantasy.

Read the latest from Neil J. Rubenking

Read full bio