Labs

Antivirus software

Viruses are a moving target. New threats appear all the time, and old ones keep coming back in new variants - rewritten, encrypted or otherwise modified so as to be no longer recognised by detection tools. To counter this, antivirus developers are constantly updating their products' detection capabilities, and this means any single test of antivirus systems can only gauge the accuracy of their database at a given point in time. In this month's Labs, we measure exactly what proportion of a sample collection of current threats is detected by 13 antivirus packages, and rank them according to their results.

Although our tests focus on virus detection, we also consider the overall user experience. An antivirus package mustn't overlook dangerous intrusions, but it's equally unacceptable for it to draw undue attention to itself, slow down your PC or nag you to buy additional products and services. Alongside our virus-detection measurements, we also take a sober look at how graciously each package integrates with your system.

Based on our findings, we assign a star rating out of six to each package across three categories - Performance, Ease of Use and Value for Money - from which an Overall score is calculated. Unless specified, facts and figures cited refer to the XP version, but every package was tested on both Windows XP and Windows Vista (with the exception of ZoneAlarm Antivirus, which remains an XP-only title). Except where indicated, performance was functionally identical on both platforms.

Real viruses and other malware

Our test methodology this month is more rigorous and objective than any we've previously applied to antivirus software. This is thanks largely to global messaging security expert MessageLabs, which for this month's test has kindly given us access to a collection of over 200 genuine viruses recently captured "in the wild" by its own servers. We've exposed each antivirus package to these viruses and measured how many of them it was able to identify and remove, enabling us to objectively rank each one's effectiveness at neutralising genuine threats.

Although we've described these items as viruses, not all are viruses in the classical sense. Strictly speaking, a virus is a program that propagates copies of itself, and some items picked up by MessageLabs don't do that. That's not to say they're benevolent, though: they'll still hide on your computer and meddle with your system without your permission. We take the view that, while programs like this may not technically be viruses, an antivirus package still ought to intercept them, so we draw no distinction between viruses, spyware, trojans and all other types of malware. We use the term "infected" to refer either to an innocuous file with a virus attached to it or to a malware installer with no other content.

Our malware collection not only includes various types of threat, it also divides evenly into executable and non-executable files. Executables represent the most obvious threat, and all antivirus software should recognise them. Non-executables can be harder to spot: they work by exploiting vulnerabilities in existing programs or system routines, and have extensions such as ANI.

The test

Each antivirus package is installed on two systems: a freshly installed Windows XP PC and a Vista Home Basic PC, each with a 2GHz processor and 512MB of RAM. To ensure an even playing field, Windows and the antivirus packages are fully updated and patched up to a uniform cut-off date, after which no further updates are permitted. Recommended settings are used to simulate a standard installation.

Each PC is then connected to our fully isolated testing LAN and the default email client is launched. More than a hundred emails, each bearing a different infected attachment, are then received over a POP3 connection from a local mail server. For each package, on each platform, we note how many of these emails are identified as infected and what measures the software takes to protect us from them.

We then use a script to download the remainder of the malicious files via HTTP from a local web server. Some of these files have EXE extensions; others are non-executables. Again, we note how many are identified as infected and what actions are taken.

Our findings provide an overall figure for the proportion of malware that's been identified and removed by each package. You'll find this figure stamped on each review as a percentage.

Performance

The Performance score is a direct function of each package's percentage score, and indicates how effectively each package protected us from viruses. The highest score goes to the package that detected the most threats, and the lowest to the package that identified the fewest.

Ease of Use

The Ease of Use score is a subjective assessment of the user experience. Unobtrusive packages that were simple to install, had minimal impact on system resources and only intruded when necessary, scored high marks. Lower scores went to those that pestered us with requesters, confused us with too many options and slowed down our PC.

Value for Money

Most companies charge an annual fee for updates of the virus database, and the Value for Money score reflects this price in light of the degree of protection and ease of use the software delivers. For free software, the score reflects how much of a "saving" the software really represents when compared to the commercial competition.

Overall

The Overall score is an average of the other three scores, although due to rounding it may appear slightly higher or lower than expected.