Posted on November 12th, 2008 by Darien Graham-Smith

Follow-up: Benchmarking Windows 7

Well, my last blog post certainly kicked up a storm. I’m glad so many people found it stimulating: I’m always interested to hear your responses.

But a few of you have raised good questions about the tests I used to compare performance between XP, Vista and Windows 7. So let me explain them in a bit more detail.

For the feature “Memory Laid Bare” in issue 169, I ran a series of tests, using the standard PC Pro benchmark suite, to find out how much RAM you need for peak Windows performance. One of my findings was that, on a given system, our benchmarks complete significantly more quickly under XP than Vista, regardless of how much memory is installed.

Unsurprisingly, it was GDI-heavy tasks that suffered the most. On the 2GB test system I used for that feature, our Microsoft Office test took 6:51 to complete in Windows XP; under Vista it slowed down to 8:07.

But calculation-intensive operations, such as video encoding, were also affected: our Canopus ProCoder exercise took 5:01 under XP but 5:11 under Vista. While that’s a much smaller gap, it demonstrates that the OS affects efficiency even for seemingly CPU-bound tasks.

In light of these results, I was intrigued to discover that both of these applications perform identically in Vista and Windows 7 – especially with commentators consistently reporting that the new OS feels faster. To me, there’s one obvious inference: that Microsoft has responded to criticism of Vista’s performance by speeding up the interface while leaving the back-end fundamentally unchanged.

But of course, Windows 7 is still a work in progress, and while Microsoft is starting to nail down the feature set (see Barry Collins’ reports on the new tools for IT admins, Device Stage, networking, touch controls, and user interface ), its developers still have a year in hand to tidy up the code. They could yet surprise us by streamlining the internals to bring back XP-like levels of performance.

Then again, of course, they could spend that time inventing a truck-load of glitzy new services to slow the whole caboodle down even more.

Here’s hoping they really have learnt from Vista.

Tags: benchmarks, Vista, Windows 7

Posted in: Software, Windows 7

Permalink | Trackback

Follow any responses to this entry through the RSS 2.0 feed.

8 Responses to “ Follow-up: Benchmarking Windows 7 ”

Dave Lindhout Says:
November 12th, 2008 at 2:23 pm
Where’s the charts and graphs, proving your numbers?

It’s pre-beta software, only an idiot would benchmark pre-beta software.

Windows 7 runs fine on my machine!

What are you smoking?

Oh, yeah!

Just trying to save the fanbois some time. This is the third or fourth review that basically says the same thing. Under the covers Windows 7 is very similar to Vista. Both are slower than XP. Many interface issues have be fixed to make Windows 7 feel more responsive.

The Microsoft memo of talking points must have said to use this line: “Why do you insist on trashing Windows 7 when all the other reviewers are praising how great it is”. I’m still looking. ComputerWorld, eWeek, InfoWorld, etc., and you, are all saying pretty much the same thing. Can somebody point me in the right direction?
Lise Says:
November 12th, 2008 at 4:36 pm
Charts and graphs are indeed famed for their ability to prove numbers.
J Gray Says:
November 13th, 2008 at 10:32 am
Come on Dave Lindhout, I think you are being a bit unfair (I think calling people idiots on this is a bit strong for this forum). If there were no benchmarking being done, you’d probably complain about that as well. We are all aware that this is pre-beta software and as such any benchmarks may be subject to change, but it would be wrong for PCPro to not try and give any (p)review without some figures to back up their ‘feelings’ of speed.
Also, Lise, I agree – charts and graphs demonstrate data in a different form, but do nothing to prove or improve the validity of data (sorry to be pedantic!).
Matthew Hall Says:
November 13th, 2008 at 11:48 am
What, so you won’t believe him unless he posts a screenshot? It’s a blog not an article, and the website’s free – grow up mate, he’s probably busy.

“Then again, of course, they could spend that time inventing a truck-load of glitzy new services to slow the whole caboodle down even more.

Here’s hoping they really have learnt from Vista.”

Sounds like a pretty balanced opinion to me
Dave Lindhout Says:
November 13th, 2008 at 1:19 pm
My apologies to Mr. Gray and Mr. Hall.

At the beginning of my first comments, please insert the tag, [sarcasm]. After the words, ‘Oh Yeah’, insert the tag, [/sarcasm]. If you have to explain your attempt at humor, you’re not that funny. My bad.

I agree with Darien Graham-Smith and his conclusions, as they are similar to what others have said. The viewer comments in other articles have been less than complementary, and I don’t think they are justified. And I have yet to find information that substantiates what they want to believe.
Matthew Hall Says:
November 13th, 2008 at 5:37 pm
Ah, it all becomes clear now that I read the previous blog.
Can we hug it out?
HP Charges Ahead with Windows 7 on Netbooks Says:
February 11th, 2009 at 6:28 pm
[...] underlying gears. PC Pro has reported that performance for benchmarks for typical office work is identical to Vista and slower than [...]
Windows 7 « Computer Guide Says:
December 9th, 2009 at 6:49 am
[...] Graham-Smith, Darien (January 2009). “Follow-up: Benchmarking Windows 7“. Retrieved [...]