Microsoft Raises the Bar in Antivirus Test

Every modern Windows installation comes with free antivirus protection from Microsoft built in. We've never advised users to rely on the built-in Windows Defender—the best third-party antivirus applications are significantly more effective. Even the antivirus testing labs treat Microsoft as a baseline, rather than as a competitor. Lately, though, Microsoft has been faring better and better in tests, which puts the pressure on other vendors to match or beat the baseline.

Just a few years ago, Microsoft routinely tanked third-party tests, sometimes earning a below-zero score. Microsoft's own security experts argued that their telemetry shows the product really works, and therefore they don't need independent tests to validate its efficacy. Even so, current events suggest that perhaps the Microsoft team is now working to score big both in their own telemetry and in independent tests.

Testing Methods
AV-Test Institute is one of the labs that treats Microsoft's test results as a baseline. They don't come out and say this, but if a product doesn't beat the baseline, it's not doing very well.

The test in question rates products on protection, performance, and usability, with six points possible in each area. To pass the test, a product need only attain a total score of 10 points, with a non-zero score in each of the three categories. It's been a while since any product failed to reach that minimal level of success.

In last November's report, Microsoft scored 14.0 points. Of the 20 tested products, 14 scored better than that baseline. Avira, Bitdefender £19.99 at Bitdefender UK, Kaspersky £19.99 at Kaspersky UK, and Norton managed a perfect 18 point score.

ThreatTrack Vipre and Chinese antivirus Quick Heal didn't beat the baseline, though. Comodo, G Data , and K7 scored the same as Microsoft, no better.

Raising the Bar
In the latest results report from AV-Test, things look quite different. Microsoft scored a very decent 15.5 of 18 possible points. This time only 9 of the 20 products beat the baseline. Avira, Bitdefender, and Kaspersky stayed at the top, with 18 points, but Norton dropped to 17.

Vipre and Quick Heal also stayed the same—below the baseline. They're joined by AhnLab, Comodo, ESET , and Panda £4.45 at Amazon UK. Four others merely matched the baseline, F-Secure, G Data, K7, and MicroWorld eScan.

So, antivirus vendors, you're on notice. Microsoft is raising the bar. This current success and other triumphs like a AAA rating from Dennis Labs mean that third-party antivirus products must up their game, or be left in that dismal spot below the baseline.

Image (modified) courtesy of Flickr User Alexander Mueller.

About Our Expert

Neil J. Rubenking

Principal Writer, Security

My Experience

When the IBM PC was new, I served as the president of the San Francisco PC User Group for three years. That’s how I met PCMag’s editorial team, who brought me on board in 1986. In the years since that fateful meeting, I’ve become PCMag’s expert on security, privacy, and identity protection, putting antivirus tools, security suites, and all kinds of security software through their paces.

Before my current security gig, I supplied PCMag readers with tips and solutions on using popular applications, operating systems, and programming languages in my "User to User" and "Ask Neil" columns, which began in 1990 and ran for almost 20 years. Along the way, I wrote more than 40 utility articles, as well as Delphi Programming for Dummies and six other books covering DOS, Windows, and programming. I also reviewed thousands of products of all kinds, ranging from early Sierra Online adventure games to AOL’s precursor Q-Link.

In the early 2000s, I turned my focus to security and the growing antivirus industry. After years of working with antivirus, I’m known throughout the security industry as an expert on evaluating antivirus tools. I serve as an advisory board member for the Anti-Malware Testing Standards Organization (AMTSO), an international nonprofit group dedicated to coordinating and improving testing of anti-malware solutions.

The Technology I Use

Much of the testing I do, particularly testing with real-world ransomware, is just plain dangerous. To perform such tests safely, I sequester them inside virtual machines managed by VMWare Workstation. For cross-platform testing, I use a MacBook Air, a Google Pixel 4, and a 6th-generation iPad.

I rely on my Delphi coding skills to create and maintain small applications. These include programs to check whether an antivirus correctly handled the malware it detected, launch dangerous URLs and record the security program’s reaction, and analyze the malware that I collect for use in testing. I also wrote a tiny browser and text editor for use in testing security apps that have predefined reactions for known products.

I do my writing and research on a Dell OptiPlex desktop, relying on Microsoft Word (my fingers know all the shortcuts). Many of my articles include charts and analysis; Excel is my go-to for those. When work hours end, though, I escape the bounds of Microsoft and Windows. There’s an iPhone in my pocket, I relax with my oversized iPad, and my Kindle Oasis is always loaded with the best science fiction and fantasy.

Read the latest from Neil J. Rubenking

Read full bio