• 
• 
• 

 1 of 3

Gallery

Performance

Posted on 13 Aug 2008 at 12:03

Mozilla is making bold claims about the performance of its new browser, but do they really bear scrutiny? We've put Firefox 3 through a series of real-world tests to examine whether it genuinely does offer the claimed "two to three times" performance boost over Firefox 2.

Memory usage

The developers of Firefox 3 claim it makes more efficient use of memory than before. But does it really matter - how much do browsers use anyway? To find out, we set up a scripted test and assessed four major browsers: IE7, Opera 9.5, Firefox 2 and Firefox 3. The script opens ten tabs and loads ten different popular websites into each, including the likes of the BBC home page, the BBC iPlayer site, YouTube and Flickr. It also includes specific sites we know are heavy on resources, in particular the Financial Times website at www.ft.com, which is full of Flash adverts and JavaScript widgets. The test script leaves the sites open for a fixed period and then closes them down again in sequence, which in theory should release the memory they use back to the operating system.

We then used Vista's Resources Monitor tool to track the individual memory usage of each application over time, and plot the results in a single graph (below right). By default, Firefox 2 and 3 won't co-exist on the same Windows installation, but we used command-line options to load different user profiles for each app.

Opinion varies on exactly which metric represents the true memory usage of an application. There are two potential measurements that Resources Monitor provides: "Working Set (Private)" and "Private Bytes". Both represent in slightly different ways the amount of memory an application has grabbed for its own exclusive use and which is unavailable to the rest of the system. Developers of Firefox have said they prefer the Private Bytes measurements. We used it to take readings in the graph opposite, but found that while the results differ slightly between the two, it's a fairly small differential.

The results of our tests show stark differences in the total memory footprint grabbed by the various browsers. Opera 9.5, Firefox 2 and Firefox 3 are fairly close in terms of memory usage, but Firefox 3 peaks higher than its older cousin. Internet Explorer 7, by contrast, steals far more, peaking at 130MB with the exact same sites open. We were disappointed by Opera 9.5, since earlier versions have proved far leaner - Firefox can now claim to be superior when in comes to overall memory footprint.

Where Firefox 3 scores well is its ability to release memory it's been using, as shown by the final third of the graph. None of the browsers drops back down to their original 20MB or so, but Firefox 3 soon reverts to around 65MB - significantly less than the others.

Firefox evangelists have made much of the new version's improved control of memory leaks, but in these tests we didn't see a huge difference. Memory leaks refer to an application taking a chunk of system memory, then losing the reference to it and failing to give it back to the operating system's free-memory pool. Over time, all free system memory can seep into the application's own resource pool and eventually lead to a crash. To simulate the conditions under which memory leaks occur, we left our test script running for a whole day in each browser, continuously opening and closing the same set of websites. In practice, all the browsers tended to use progressively more memory, but it was a small increase and none was so poor as to threaten system stability.

Real-world performance

Aside from the improved memory management, Mozilla claims that enhancements to the browser's underlying Gecko engine have resulted in huge performance gains, "making Firefox 3 two to three times faster than Firefox 2 overall".