Browser War – Part 1: Firefox2 vs Firefox3RC1

By | May 23, 2008

As we’ve mentioned before it’s about time for another ‘clash of the titans’ in the never ending web browser wars. Raja Rao of our QA team had previously built a sweet AJAX client testing framework, so we decided to pit the major browser’s current releases and nightly builds verses one another. Who will go down in this first round?

Before you go “Wait, is this particular article just Firefox vs Firefox – aren’t newer versions expected to preform better anyways?” think of it as just a warm-up to instill confidence by beating personal records in preparation for some looming, ugly battle-royal. Plus the gym was destroyed last time we had them all in one place, and our graphs just get too cluttered, so you’ll have to come back for other matches like IE7 vs IE8b and Safari 3.1 vs Safari 3 nightly builds.

FF2 vs FF3RC1 barchart  
 
For Firefox memory bloat tests have been popular lately, but we wanted some real world JavaScript tests through a set of ZWC tasks on 5.0.6 – such as logging in, composing and viewing messages, navigating around various folders, switching between our many apps, and even changing options.

 
FF3RC1 clearly won every test, so Mozilla deserves a pat on the back for advancing their browser. While completing little web-client actions there was often barely a difference. However, where heavy rendering had to be done the improvements were significant – in some places half the rendering time or even three times as fast!

What’s the test harness?
 
We used a customized OpenQA Selenium setup to calculate time-taken rendering a page after clicking a particular button/link. Test machines were running AMD Opteron 1.8GHz Dual-cores with 2GB RAM against ZCS 5.0.6 GA RHEL4.

When we compare other browsers we will be using the same set of saved actions. Try it out yourself and discuss it in the forums – if you’d like some other testing ideas you might play with the other commonly used benchmarks like VeriTest & SunSpider.
FF2 vs FF3RC1 total completion time

 
So what has made Firefox 3 so speedy? Perhaps it’s the Profile-Guided Optimization (dual pass compiling) builds now being created that are greatly improving performance, or lots of combined JavaScript engine enhancements. Have you been comparing all the nightlies leading up to RC1 or noted the exact change that has enhanced it so much? Discuss it below and stay tuned, I hear Safari is pulling out all the stops – get your bets in now.


Think working on complex AJAX apps is cool? Head over to the developer section in the community forums and see if you got what it takes. Who knows, you may just decide to come join us.
*As always performance will vary based on system configuration, network connection, and other factors like account data and preferences.


Comments are closed.