I’ve become used to Nvidia and AMD rocking up with new graphics cards on a consistently quick schedule: since 2008 and beginning with the GeForce 9000-series, Nvidia has released six generations of GPUs, and AMD has replied with three of its own, starting with the Radeon HD 4000-series.

Both firms put the finishing touches on their current ranges in the spring, and the trail’s gone quiet since. Nvidia has confirmed that it won’t release any new cards before 2012, and AMD hasn’t mentioned any potential release dates for its new Radeon HD 7000-series — presumably taking extra time to further tweak and perfect the new chips.

I can understand why Nvidia and AMD aren’t in any hurry to unveil new silicon — after all, evidence suggests that few games, and fewer gamers, will make use of the levels of power available from new cards: 2007’s Crysis is still one of the most demanding games around and, outside of Battlefield 3, few forthcoming games look like they’ll tax even last year’s graphics cards — after all, most are now ported from consoles.

Take a look at the Steam Hardware Survey from July 2011. The most popular ten graphics cards are, with no exceptions, old: top of the pile is the Nvidia GeForce 9800 GT, and it’s followed up by venerable old warhorses like the GeForce 8800 GT, Radeon HD 4870 and even the GeForce 8600 GT. The most popular current-generation chip, the GeForce GTX 560, is 25th on the list.

To find something truly high-end, keep on travelling down the list — the GTX 570 is the 31st most popular card and is used by a whopping 0.78% of Steam’s users, and the AMD Radeon HD 6950 is favoured by just 0.68%.

That’s the biggest PC gaming platform in the world and proof that, while AMD and Nvidia have spent the fast few years trying to beat each other with benchmarks, most users haven’t taken any notice. Instead, they’re too busy playing games on cards that are older and perfectly capable.

Perhaps the big guns have realised that this pixelated arms race is virtually pointless. Almost all users either aren’t bothered or can’t afford the latest chips, and the lack of big-name PC exclusives means that even a modest card will run rings around the latest games. Enthusiasts might enjoy the yearly releases, but board partners constantly tweak, overclock and improve chips to eke more performance out of older cores — and, if they’d like to run games across several screens, there’s always CrossFireX and SLI.

It looks like we won’t see any major graphics card releases until 2012 but, after several years of frenzied, benchmark-driven battling, I’m happy to wait — and it looks like most users are, too.

The Apple MacBook Pro 13in is a glorious laptop. It’s thin and light, gorgeous both to look at and to use, and it packs no small amount of power in its tiny chassis. Yet our tests have uncovered a performance issue that will affect every user.

We ran our new Real World Benchmarks on the top-end model, with a dual-core 2.7GHz Intel Core i7-2620M processor, 4GB of DDR3 and a 500GB hard disk. It’s a very fast laptop for its size, as a final score of 0.70 shows – that’s only around 20% slower than the top-end quad-core 17in model. Yet it’s not quite as fast as it should be.

We first noticed a problem when the benchmarks finished five full runs and the results popped up on screen: the times taken to complete several of the most intensive tests were rising with each run. This would suggest an overheating problem, so we ran a temperature monitor to find out how hot this Sandy Bridge CPU was getting. Here are the readings both when idle and under full load:

We should point out that 93°C is not necessarily too high for a modern CPU, but it is the root cause of the bigger performance problem. To explain, here’s that full-load temperature again, along with Intel’s own Turbo Boost monitor:

The temperature of 93°C was reached with the processor peaking at 2.7GHz. The Core i7-2620M should be able to Turbo Boost up to a maximum of 3.4GHz, but in this laptop it doesn’t top 2.7GHz at any temperature.

For comparison, here is the same reading from the Core i7-2720QM in the top-end 17in model:

Here it’s being Turbo Boosted from its stock 2.2GHz. Notice how the blisteringly fast 17in model only peaks a few degrees hotter than the 13in, and that’s with the maximum boost. As soon as the fans kicked in that settled comfortably in the high eighties.

What does this all mean? Well, if the CPU in the MacBook Pro 13in hits 93°C at stock speeds, we can only imagine how hot it would get if Turbo Boost was allowed to kick in. So we suspect Apple has disabled it completely to prevent overheating in such a tiny chassis.

It’s an issue we haven’t seen highlighted, perhaps because to the end user it doesn’t really affect the day-to-day experience – and it shouldn’t put you off buying what is in every other way a fantastic piece of kit. But Apple is promoting this on its website as a Turbo Boost-enabled laptop. We’ve asked Apple for comment and await its response.

We also now have the £999 model with its 2.3GHz Core i5 processor in the Labs. We ran the same tests on that one and can confirm that its processor is Turbo Boosting in Boot Camp as it should. The issue is unique to the i7 model.

However, secondly, we must doff our caps to Anandtech and show you our reading from a utility they used called MSR Tools:

We stand corrected on one count: it is indeed Turbo Boosting in OS X. We ran a temperature monitor for several minutes as well, and those boosted speeds occurred with a peak temperature of 93°C, the exact same as the peak in Windows without the boost.

The Turbo Boost issue therefore looks to be one of processor cooling in Windows. That affects a far smaller group of users that an OS X flaw, but it remains a mystery: does the Core i7 model of the MacBook Pro 13in run so hot under Windows drivers that Apple has chosen to disable Turbo Boost? Our tests make that a plausible scenario.

Heat is an issue that’s difficult to ignore. Even on the Core i5 model, just unpacking a large zip file had the Turbo Boosted Core i5 quickly rising to 90°C; during a stress test that hit 99°C. And that has a knock-on effect: with an IR temperature gun we measured the aluminium underside of the laptop at a thigh-scalding 60°C!

We’re more sure than ever that Apple had a real task on its hands getting such fast processors into this chassis, and from everything we’ve seen we’re leaning away from the top-end 13in model as a purchase. If the sight of our benchmarks gradually getting slower with each consecutive run wasn’t enough to highlight the cooling problem, a heat gun pointed at the metal underside certainly was.

There is one plus to the Turbo Boost mystery, however: if you intended to buy a MacBook Pro 13in and install Windows on it, you’ll probably find the cheaper model actually runs faster than the top-end one. Save yourself £300.

Our new tests don’t rely on synthetic measures: we use real, current applications such as Microsoft Office 2010 and Photoshop CS5, as well as a completely new set of responsiveness tests, to get an all-round picture of a PC’s performance.

That means the benchmark scores you’ll see from this day on are not directly comparable with older scores, but they give the best ever insight into exactly what each system can do for you.

The new tests

The new PC Pro Real World Benchmarks are split into three sections, each reflecting a different area of usage.

Responsiveness

With so many netbooks and ultraportables using lightweight processors, it’s vital to know how responsive a system is. We measure this with two new tests. We time how long it takes to open documents and switch between a series of common desktop applications, including the latest versions of Word and Excel, Acrobat Reader 10 and Internet Explorer. We also measure Windows performance, by timing how long the system takes to open, close and move dozens of Explorer windows.

Media

Our media tests expose how capable a system is at using modern software to process music, photos and video files. We start by using iTunes 10 to encode an album into both AAC and MP3 formats. Next, we set Photoshop CS5 to work on a folder of 12-megapixel RAW-format photographs. We adjust the colours and curves, apply some artistic sharpening and blurring and save the results in JPEG format. Finally, we use Sony Vegas 10 to render a short home video shot at 1080p resolution, with a picture-in-picture effect and a crossfade transition. Again, the whole process is timed.

Multitasking

For our multitasking test, we run the iTunes and Photoshop tests simultaneously and then launch our responsiveness tests over the top, giving the CPU, hard disk and memory a real workout. Finally, we time how long it takes the multi-threaded Cinebench 11.5 renderer to produce a complex 3D scene. The multitasking score thus reflects both responsiveness and performance under heavy load – just the information you need if you’re looking for a real workhorse computer.

The Overall score

Once we have our timings, we compare them with the times achieved by our new reference platform: a 3.4GHz Core i7-2600K processor, with 4GB of DDR3 RAM and a 7,200 RPM hard disk, at a display resolution of 1,920 x 1,080 (all desktops are tested at this resolution; we test laptops at native resolution).

Each score is given as a number relative to that platform, with higher scores being faster. Thus, a score of 0.50 in a particular test indicates that a system performs those tasks half as quickly as our reference system. A score of 2.00 would indicate a super-powerful PC that was twice as quick as the reference platform.

We combine the three scores into an average for easy comparison – this is the Overall score. But at the bottom of each review you’ll find a breakdown of the scores in the three tests. This shows each system’s strengths and weaknesses at a glance, so you can easily find the system that’s perfect for you.

Examples

Current desktop PCs are powerful enough to motor through the Responsiveness and Media sections with ease, and should prove the most adept at Multitasking too:

Laptops, with their slower processors but still-plentiful RAM, should show the real variation in what is being tested. Most will remain strong in the Windows Responsiveness test, demonstrating their proficiency at everyday tasks, but will be less successful when pushed by the more intensive Media and Multitasking tests:

Finally, low-power systems such as netbooks and nettops will see that Responsiveness score falling by the wayside as well, but the real drop will be in the more intensive tests as their lack of power limits them to simple linear tasks:

New battery benchmarks

We’ve changed our battery benchmarks too, to be more consistent and to better reflect the sort of stamina you can expect from your laptop while it’s actually in use.

Light use

In the light-use test, we optimise the system settings for the greatest power efficiency. Windows’ power profile is set to Power Saver mode, and we use an X-Rite i1Display 2 colorimeter to set the screen brightness as close to 75cd/m² as possible. We then disconnect the mains and run a script browsing a selection of web pages until the system shuts down, giving you a realistic idea of how much surfing time each notebook offers.

Heavy use

For the heavy-use test, we engage Windows’ High Performance power profile, set the display brightness to maximum, and allow the taxing Cinebench 3D renderer to push the processor load to the limit. This gives a worst-case figure, revealing how long you can expect the battery to last under the most demanding conditions.

The switchover

As of today, every PC and laptop that arrives in the PC Pro Labs will have these new Real World Benchmarks run on them. You’ll still see the old benchmarks on reviews already on the website and (due to press timelines) in issue 199 of PC Pro, on sale 10 March. We’re in the process of updating the benchmarks tab on website reviews, and as of issue 200 the switchover will be complete.

Over the next few months you’re going to see some big changes on PC Pro.

2011 benchmarks

In issue 200 of the magazine (on sale mid-April), we’re going to be unveiling our 2011 benchmarks suite. Written in-house using real-world applications such as Office 2010 and Photoshop CS5, the PC Pro benchmarks will deliver the most accurate measurement of PC performance to date, testing overall system responsiveness as well as typical desktop tasks and demanding multi-core workloads.

That will be graphically illustrated in issue 200’s CPU megatest, in which we’ll be testing every mainstream desktop CPU from both Intel and AMD to see which really offers the best performance for your money. We’ll explain more about how our new benchmarks work in the coming weeks.

New-look magazine

In the next few months, you’ll also see a new-look PC Pro magazine on the newsagents’ shelves or dropping through your letterbox (find out how to subscribe here). We’re fundamentally redesigning the magazine, whilst still delivering more features, more reviews and more real-world expert advice than any other title.

We’ll share more of our redesign plans with you in the coming weeks and months, but you can look forward to:

* A revamped A List that makes it easier than ever to find our recommended products

* More detailed product photography, analysis and performance scores in reviews

* Platinum Labs – comprehensive group tests of the best kit money can buy

* Real-world advice from those looking to pursue a career in IT

These are only a small fraction of the changes and new features we’re planning for the new-look PC Pro. Read Tim Danton’s column in this month’s magazine to find out how you give your feedback on the planned changes.

A new editor

A more immediate change is that, as of today, I will be succeeding Tim Danton as editor of PC Pro.

Tim’s left me with an enviable legacy: the biggest, most-read computing monthly in the country; a website with more than a million unique users every month; and, crucially, the most talented and experienced team of staff and contributors of any IT publication in the country.

Tim’s not going far: he’s taking over as editorial director of Dennis Publishing’s tech division. You’ll still hear his voice on the PC Pro podcast and read his fine words on the PC Pro blogs.

I’m sure you’ll join me in wishing him well (a man who drinks that much black coffee each day needs all the well-wishers he can get) in his new post, and I look forward to what’s going to be a momentous few months for PC Pro.

Barry Collins – Editor

This prompted one commenter (giving his name as Wilbert3) to raise an insightful point. Hyper-Threading is great for everyday multi-tasking: for example, it lets a dual-core Core i5 CPU service four concurrent processes. But it works by presenting each core’s spare execution capacity to the OS as a virtual second core. Under heavy load, where there is no spare capacity, it would seem unable to offer any benefit. In such cases we shouldn’t expect to see a Core i5 achieve performance anywhere near what a true quad-core architecture would provide.

That analysis sounds persuasive, but is it borne out by the evidence?

To find out, I’ve done some tests on our standard Core i7-920 test rig, using our highly parallel 3D rendering benchmark and selectively disabling physical and virtual cores. Here are the results, in seconds, showing how long it took to render the same scene in different CPU configurations:

For the record, this was tested on a 32-bit Vista Home Premium installation, running at 2.66GHz with 4GB of DDR3-1066 RAM and a 1TB Seagate Barracuda 7200.12 hard disk.

Clearly, Hyper-Threading does help, cutting the dual-core render time for our test scene from 185 seconds to 165 seconds. It seems even highly parallel tasks waste some execution capacity that Hyper-Threading can exploit. But when we switched off Hyper-Threading and stepped up to four physical cores, render time was slashed to just 118 seconds — a performance boost more than three times greater than that offered by Hyper-Threading.

(Update: At the request of some readers, I subsequently repeated the four-core test with Hyper-Threading enabled. This brought render time down to 107 seconds, representing a speed improvement of 10% — almost exactly the same proportional change, to within 1%, as observed in the two-core tests.)

The conclusion is clear: Hyper-Threading can eke some extra performance out of even the most demanding tasks, and to that extent it’s certainly a plus point for Intel’s architecture. It’s also clearly worth having if you need to service many parallel lightweight demands — in a server, for example. But when it comes to proper grunt-work, it’s no substitute for real silicon.

Now Windows 7 has progressed all the way to Release Candidate status I thought it might be interesting to repeat the experiment with the almost-final code. So again I’ve been running our real-world benchmarks, this time on a Core i7-based system with 3GB of RAM, to compare performance in Vista to both clean and upgrade installations of Windows 7 RC.

This time the results surprised me even more:

As you can see, in most of our tests a clean installation of Windows 7 RC remains on a par with Vista, or at worst a few seconds behind. It’s faintly odd that, in the Photoshop and 3D tests, the upgrade installation was slower than a clean installation of either Vista or Windows 7, but the gap isn’t big enough to fret over.

But what sticks out like a sore thumb is Windows 7 RC’s dreadful performance in our Office test. This test involves extensive number-crunching and graphing in Excel, page formatting and printing in Word, database sorting in Access and slide creation in PowerPoint. Our Windows 7 alpha completed it in an identical time to Vista, but the RC took 70% longer in a clean installation. In an upgraded environment execution time was almost doubled.

(In case you’re wondering, the Multi-app test entails running the Office, audio and Photoshop benchmarks all at the same time, so 7’s relatively poor scores here are probably just another symptom of poor Office performance.)

I don’t yet know what’s causing the slowdown. It’s not unique to this particular setup: I repeated the test on an Athlon X2 system, which is architecturally pretty damn different to a Core i7, and saw a comparable slow-down on this benchmark.

But I’m continuing to investigate, and I’ll let you know what I find.

But a few of you have raised good questions about the tests I used to compare performance between XP, Vista and Windows 7. So let me explain them in a bit more detail.

For the feature “Memory Laid Bare” in issue 169, I ran a series of tests, using the standard PC Pro benchmark suite, to find out how much RAM you need for peak Windows performance. One of my findings was that, on a given system, our benchmarks complete significantly more quickly under XP than Vista, regardless of how much memory is installed.

Unsurprisingly, it was GDI-heavy tasks that suffered the most. On the 2GB test system I used for that feature, our Microsoft Office test took 6:51 to complete in Windows XP; under Vista it slowed down to 8:07.

But calculation-intensive operations, such as video encoding, were also affected: our Canopus ProCoder exercise took 5:01 under XP but 5:11 under Vista. While that’s a much smaller gap, it demonstrates that the OS affects efficiency even for seemingly CPU-bound tasks.

In light of these results, I was intrigued to discover that both of these applications perform identically in Vista and Windows 7 – especially with commentators consistently reporting that the new OS feels faster. To me, there’s one obvious inference: that Microsoft has responded to criticism of Vista’s performance by speeding up the interface while leaving the back-end fundamentally unchanged.

But of course, Windows 7 is still a work in progress, and while Microsoft is starting to nail down the feature set (see Barry Collins’ reports on the new tools for IT admins, Device Stage, networking, touch controls, and user interface), its developers still have a year in hand to tidy up the code. They could yet surprise us by streamlining the internals to bring back XP-like levels of performance.

Then again, of course, they could spend that time inventing a truck-load of glitzy new services to slow the whole caboodle down even more.

Here’s hoping they really have learnt from Vista.

And the net effect? Surprisingly little. At this stage of development, over a year from release, Windows 7 looks almost identical to Vista. There are some welcome new features, as already noted by our esteemed editor and deputy editor (see their blog posts here and here); but the high profile changes (such as the snazzy new taskbar that Barry Collins saw in California the other week) are yet to be plumbed in.

Tortoise and hare

Yet Windows 7 does already offer one compelling advantage over Vista: it’s fast. Both our senior pontificators were struck by how nimble Windows 7 feels after you’re used to its predecessor. As Tim Danton writes, “Vista was never this nippy. You press on an icon and it leaps into action. . . . I can’t remember using any new OS that was this quick.”

Now if that’s not cause for celebration, I don’t know what is. Vista has had us driving with the handbrake on for the past two years, but at long last Windows 7 is coming to set us free. Admittedly it’s not actually going to get here for another year, but at least salvation is in sight.

But it’s a funny thing, you know. Because I’ve been running a few benchmarks, just to find out exactly what sort of speed boost we’re talking about. And I can exclusively reveal that the actual performance gap between Vista and Windows 7 is… nada. Absolutely nothing. Our Office benchmarks and video encoding tests complete in precisely the same time regardless of which OS is installed.¹

Perception is reality

It’s tempting to see this as a bit of a con. They’ve sped up the front end so it feels like you’re getting more done, but in terms of real productivity it’s no better than Vista.

But personally I think it’s an inspired move. Over the past few years, Microsoft has learnt the hard way the power of perception. Once the masses got hold of the idea that Vista was a lumbering step backwards, no Mojave Experiment could rescue its reputation.

Now, to borrow a phrase from Steve Ballmer, they’ve “woken up smarter.” They’ve recognised that perceptions of speed focus almost exclusively on interactive performance. Very few people notice or care whether a big mail-merge job takes thirty seconds or forty, but they sure as hell notice when they click a button and nothing seems to happen. That’s what wrecked Vista’s reputaton, not its disappointing benchmark scores; and that’s why we’re all hankering after Windows 7 despite its identical scores.

All the small things

Of course, it’s a disappointment to realise just how similar Microsoft’s new OS is, under the bonnet, to its current one – similar enough to explain why Windows 7 actually has an internal version number of 6.1. We all had high hopes of a lightweight “MinWin”, akin to what Apple is reportedly working on for OS X 10.6 (“Snow Leopard”).

But while this faster front-end seems like a superficial change, it makes a world of difference. As I’ve said before, there are plenty of things in Vista to like, but I find it impossible to enjoy them while the whole experience is weighed down by a sluggish interface. No longer. I can only echo Tim Danton’s conclusion: “Windows 7 is exactly what Vista should have been.”

Of course, it’s shameful that it’s taken so long to get here. It’s generally suspected that Vista was a rush release, but there’s no reason the improved window manager couldn’t have been dropped in via Windows Update once it was ready. Holding it back for Windows 7 is a real two fingers to users who paid for Vista, and I’m not sure it makes sense for Microsoft. The company surely realises what Vista is doing to its reputation, yet here it is giving Apple another year, on top of the two it’s already had, to thumb its nose and woo away potential customers.

All’s well that ends well

But ultimately I see Windows 7 much as I see the latest edition of Norton Internet Security. In both cases, previous versions acquired – with some justice – a reputation for terrible performance. In both cases, that’s now been fixed. We can kvetch all we want about how and when it should have been fixed, but the fact is that the battle is over. If you feel you were ripped off in the past, you can signal your displeasure by choosing a competing product now; but arguably it sends a clearer message to invest in a fixed product than to boycott it. Plus, this way you get the thing you really wanted, albeit late.

So when Windows 7 finally comes knocking next Christmas, I won’t be turning it away. Sure, there will be admonishments about how long it took to get here. And I’m sure it will bring new faults as well as benefits.

But deep down we’ll both know how much I’ve been looking forward to its arrival. And together, at long last, we’ll once and for all close the book on Vista.

Follow-up: Benchmarking Windows 7