Every antivirus product and security suite should prevent attack by viruses and other malware. I challenge such products by deliberately trying to infect a protected test system using known malware samples. I then calculate a malware blocking score based on how successfully the product detected and prevented these attacks. I also check the antivirus's ability to prevent infection by blocking malware-hosting URLs.

Blocking Malicious URLs
Almost all modern malware reaches your system from the Internet. Many antivirus products head off infection by blocking all access to malware-hosting URLs. Others check files during or immediately after download. Last year I introduced a test specifically aimed at measuring how well a product handles blocking malicious URLs.

I start with a feed of extremely new malicious URLs supplied by MRG-Effitas. They process many thousands of URLs every day; typically the ones I use are no more than four hours old. I filter the list to specifically capture URLs pointing to an executable file.

The testing process is quite simple. One after another, I launch the URLs in Internet Explorer, using a simple utility I coded myself. For each URL, there are three possible outcomes. The security software may block all access to the URL, it may wipe out the file during or right after download, or it may do nothing. I report the overall percentage blocked, whether at the URL level or during download.

I've ben running this test since November 2013; I don't have data for products reviewed before that date.

Deliberate Malware Attack
My malware samples change over time, but the collection will typically  include adware, spyware, viruses, worms, scareware (rogue security software), rootkits, and Trojans.

I install the product on a clean test system and manually run an update, to make sure it has the very latest virus definitions. Then I simply open a folder containing the collection of samples and note how the product reacts. In many cases, the minimal access that occurs when Windows Explorer displays the filename is sufficient to trigger real-time protection. I also single-click on each file, since real-time protection in some products doesn't kick in until a click.

Scoring
Naturally the product scores a full ten points for each threat it eliminates on sight. Continuing the test, I launch any samples that survived the initial culling and note how the product reacts. Typically I'll launch three or four of them and then run my proprietary analysis tools to determine whether the threats managed to place any files on the test system.

If the threat didn't plant any executable files and installed from zero to 20 percent of its non-executable file and Registry junk I award ten points, the same as if the antivirus wiped it out on sight. An antivirus that allowed the threat to put 20 to 80 percent of its junk on the test system still gets nine points. That sinks to eight points if 80 percent or more of the junk landed on the test system.

Once the antivirus has detected a threat attempting installation it really should prevent placement of any executable files. If an executable file gets through I offer five points, or half-credit. If, despite the antivirus's best efforts, a malware component manages to run, that goes down to three points. Naturally a total failure to detect the threat earns zero points. The overall blocking score is simply the average of all the individual scores. I also break out separate scores for blocking rootkits and scareware.

The product's final rating doesn't have a one-to-one correlation with the malware blocking and removal scores. Other factors can come into play, including the results of independent lab tests, but scoring well on my malware blocking and malicious URL blocking tests certainly helps get a good rating.