IT Forum 2005: Microsoft challenges Linux reliability

By Alun Williams, Barcelona

Posted on 16 Nov 2005 at 13:57

In what will be a red rag to the Linux bull, Microsoft is trumpeting a survey that concludes that Windows is a more reliable platform than Linux for companies.

The study compared two teams of IT administrators as they maintained Windows Server 2000 and Novell SUSE Enterprise Linux 8, and then upgraded to Windows Server 2003 and Novell SUSE Enterprise Linux 9, respectively.

The comparison, which took place over the course of a year, was carried out by Security Innovation, described as an independent security services company (unfortunately its website was down at time of writing).

When it came to reliability, the survey reports that Linux administrators took 68 per cent longer to implement new business requirements than their Windows counterparts, and the Suse Linux system apparently suffered 14 critical breakages while the Windows Server system experienced none. Furthermore, the Linux system required 4.79 times the number of patches.

In justifying the claim that the Windows platform was more 'consistent and predictable' Microsoft states that only one of the three experienced Linux IT administrators successfully met all business requirements, while all three Windows administrators succeeded in meeting requirements.

I challenged Microsoft's Senior Director of Platform Strategy EMEA, Ashim Pal, over the survey, which he was highlighting at the IT Forum.

What of the sample size - only three administrators operating the respective systems - surely that is too small a sample size? Pal pointed out that the survey was carried out over a year, and a thousand such samples, for example, could not be feasible. All the administrators had five years experience, so they were reasonably experienced to face the tasks that were set by the test.

Was it fair, I asked, to compare the efforts of three open source-experienced administrators with the best Redmond had to offer. Why not, Pal insisted? They were not challenging the admins to assemble open source code from the internet, he said, the test involved two retail software packages. Whether other competitors, such as IBM, might also offer consulting services with their Linux packages was not an issue, he insisted, as these were both out-of-the-box systems administered by experienced personnel.

Questioning the fact that the Linux systems required 4.79 times the number of patches of the Windows equivalent, Pal assured me that a visit to Windows Update or any other automated Windows patch management service did not count as 'one update' - that the survey had been realistic in counting the individual Microsoft updates.

What of the tasks that were required - was the test skewed to feature the latest group collaboration functionality that would show SharePoint, for example, to good advantage? Not so, insisted, Pal. While not mirroring one particular vertical industry, he said the tasks would characterise a serious retail operation, involving history-based targeted marketing, some data mining and general e-commerce functionality.

'It was,' Pal insisted, 'a comparison involving an extremely transparent methodology and testing process. We want to demystify this area and encourage a fact-based rather than emotional response to this comparison.'

While conceding there were areas - such as high-end high performance computing - where open source solutions would have a 'current advantage', he insisted that Microsoft's performance could stand up to any independent inspection. He challenged third parties to repeat the test and report any different findings.