Labs

Internet security suites

All of our Internet security suite tests are performed on the same base machine as a control unit: an AMD Athlon 64 3400+ desktop PC with 1GB of RAM running Windows XP Professional SP 2. Between product tests, this is returned to its initial state using the same original and 'clean' drive image. If a program passes the test, it gets one point. So if it passes five out of five, it gets five stars in the Perfomance rating at the foot of each review.

Firewall: 1 point

We test the online visibility of the test computer using the PC Flank Stealth Test (www.pcflank.com), and use HackerWhacker (www.hackerwhacker.com) for a full Nmap-driven scan of all 65,534 ports checking for open status. Lastly, we conduct our own vulnerability testing, exposing the computer to a barrage of exploits using the same 'tools' as hackers themselves. All tests have to be passed to get that point.

Anti-Virus: 1 point

We first run an EICAR test (www.eicar.org) to check the anti-virus component was alive and working, before introducing the system to 10,000 emails containing a mixture of clean, virus-infected and spam messages. Any of the known threats that went unrecognised by the anti-virus scanner result in a fail.

Anti-Spam: 1 point

We don't train any of the anti-spam applications that use Bayesian detection, instead using default out-of-the-box settings coupled to a cursory sweep of the configuration options - tweaking any that are obvious and available, such as language filtering. To pass this test, the application has to detect at least 90 per cent of the spam we threw at it, without identifying more than 100 messages (1 per cent) wrongly.

Parental control: 1 point

Having set the appropriate levels of control, we log in using a 'child' account and attempt to access 50 websites across the following categories: adult, drug use, gambling, racist and violent content. Successful connection to a single one of them resulted in failure, as did not being able to configure restrictions on an individual account basis.

Anti-spyware: 1 point

For these tests, we apply a different drive image, as used in our anti-spyware group test (see issue 133, p130), pre-infected with a selection of rogue diallers, remote access trojans, browser toolbars, commercial keyloggers and a number of adware components. Since the standard to beat has already been set by Spy Sweeper 4.5 and Spyware Doctor 3.2, the products had to hit 90 per cent for detection, 85 per cent for removal and 60 per cent for blocking those removed programs in order to pass. Missing any target is enough to fail.

Integration: 1 point

The provision of an efficient central management console falls far beyond the realms of 'ease of use', which is why we include it within the performance-testing results. Being able to configure applications with consistent look and feel, managing definition updates and getting both at-a-glance overviews and in-depth detail of system security are key to creating a successful suite.