Hey, did you hear that Google released a browser? Yeah, and it’s very cool! We might have been a bit early to call Safari the Browser war winner. Based on WebKit (KHTML), this rendering framework (that Chrome uses) has really stormed the market. If you asked us five months ago who was winning the browser war, we would easily say Firefox, with Safari as a close second. With the introduction of Chrome, a new war has started.
At the start of this century, the war was about “Open-ness” and who could be more open and win the hearts of users. Now it’s a war of speed, and who’s faster. A few blogs and articles have been written, with Mozilla and Google both claiming their JS engine is faster. So who’s right? Both are faster than IE (6, 7, and 8), but in our opinion, what matters is how responsive web applications are. So who will win Zimbra’s Speed trophy?
Zimbra has a testing harness thats in alpha which we will be making available to the public in the future, that measures performance on different actions within Zimbra. This helps us understand what the end user is seeing. People can talk V8 Benchmark, Dromaeo, SunSpider, or what ever they want. What really matters is how applications perform. Our tests are pure UI performance, ie, how fast Zimbra is to the end user.
Considering that one of Zimbra’s strengths is our AJAX web interface, we decided to put Chrome to the test, along with IE, FireFox, and Safari. The control system was: Intel Core2 duo, 2.39Ghz 1.99GB RAM Windows XP
Here’s how it did (lower is faster):
Overall Performance
All Tests
Given that Chrome is built on WebKit, this didn’t come as any particularly huge surprise. In our previous tests, Safari came out the fastest renderer of the Zimbra Web Client. In our tests, Chrome came in as a very close second, and we expect it to get faster.
We want Chrome to work as good as FireFox or Internet Explorer. So, if you find an issue, please report it in the bug report below.
Want more info on the browser war? Check out these links:
With Zimbra 5.0 we’ve introduced some newer ways to make the user experience faster with the Zimbra Web Client. We’ve talked about Jetty, YUI compression, and Lazy Loading, but now there’s just one burning question: Which browser is fastest?
There’s some amazing JavaScript handling enhancements about to be pushed into the major browsers. In-case you missed previous rounds of the browser wars we’ll tell you the answer, but you should still checkout Firefox 2 vs FF3RC1, Internet Explorer 7 vs IE8b, and Safari 3.1.1 vs SF nightlies to get more insight into how each fared on the testbed.
The gloves are off, this is a bare knuckle battle-royal fight:
The winner’s Safari!
Surprised?
So were we.
We had high hopes that FF3GA would at least match Safari 3.1.1 in order to contend with Apple’s Safari 4 just around the corner. Infact that graph is simply those tested in our browser war series; the WebKit nightlies (engine for SF4) deliver a knockout blow. And it’s not just our favorite testing software that shows this; we use OpenQA Selenium which allows us to nicely calculate time rendering a page while navigating the Zimbra AJAX web-client. Other commonly used benchmarks like SunSpider & VeriTest show very similar results and Safari 4 nightlies even fare well on the Acid3 DOM and JavaScript test, beating out Firefox 3 every time. But we’ll let the judges analyze and discuss later.
The SquirrelFish JavaScript interpreter in Safari 4 is a bytecode engine which eliminates almost all of the overhead of a tree-walking JavaScript interpreter. According to the WebKit project, the SquirrelFish engine is 1.6 times faster than the engine in Safari 3.1. SquirrelFish does its magic by turning JavaScript script into so-called bytecodes, an optimized code much more suitable for run-time execution than natural language-based commands, which are longer and more complicated to interpret – and therefore are slower. It also leaves room to experiment with things like constant folding, type inference, specialization based on expression context, peephole optimization, and escape analysis.
In addition, Safari 4 adds the ability to save webpages as standalone web applications (Similar to the Mac favorite Fluid, or Mozilla’s Prism which meshes nicely as an add-on to Firefox.), CCS enhancements to gradients, masks, and reflections, as well as some additional native font rendering and HTML5 support.
That said, there’s more to browser choices than JavaScript rendering in ZCS – it’s a great day for Firefox. The new release is packed with over 15,000 updates and new features; from the underlying Gecko and JavaScript engines to Profile-Guided Optimizations (dual pass compiling) that bring dramatic improvements to performance, memory usage and speed. And you can’t forget the best add-on network.
There was previously a clamoring in the forums about participating in Firefox 3 download day Guinness World Record attempt. Go do your part and download this awesome browser!
*Test machines were running AMD Opteron 1.8GHz Dual-cores with 2GB RAM against ZCS 5.0.6 GA RHEL4. As always performance will vary based on system configuration, network connection, and other factors like account data and preferences.
Some might be thinking it seems like a shorter gap in browser versions – why not Safari 3.0 through 3.0.4, or even 3.1 to show greater self improvement? First, we opted for current supported release vs betas. We did use Firefox 2.0.0.14 and the most up-to-date IE7 we could find for previous matches. Secondly, since we planned this series 3.1.1 became GA, so we decided to do Safari 3.1.1 build 525.17 (which was at least pushed to the masses) vs WebKit r33940 (the open-source browser engine packaged close to Safari 3.1.1 build 525.20).
But as you can see Safari clearly doesn’t need the moral boost anyways:
You know the drill: We used an OpenQA Selenium setup to calculate time rendering a page; across the same set of saved actions such as logging in, composing and viewing messages, navigating around various folders, switching between our many apps, and even changing options.
Those might not look like huge jumps, but Safari builds have been churned out so fast – and nothing on that graph is over 2 seconds. The older version only wins in a handful of cases, but that could easily be due to other test-harness factors when your dealing with 1/100th’s of a second.
Safari has clearly been working out – it’s fit, and plans to stay that way by avoiding dessert.
Mac users rejoice: Apple released Safari 3.1.1 build 525.20 (~WebKit r33940) on May 28th with Mac OS X v10.5.3.
Windows machines have Safari 3.1.1 but it’s still build 525.17, so some stuff hasn’t been incorporated into Windows releases yet; and Apple either takes their time distributing updates, or they’re waiting on a Safari 4 version before next push. Don’t fret, it’s still decently fast and you’ll have the latest JS engine soon no doubt.
There’s already a developer seed of Safari 4 released. Which includes the SquirrelFish JavaScript interpreter (renamed from GlassFish to avoid confusion with Apple’s other Java stuff). SquirrelFish is a bytecode engine which eliminates almost all of the overhead of a tree-walking JavaScript interpreter. It also leaves room to experiment with things like constant folding, type inference, specialization based on expression context, peephole optimization, and escape analysis but they haven’t implemented all that yet. Safari 4 adds the ability to save webpages as standalone web applications (much like site-specific browsers such as Fluid, Adobe AIR, Bubbles, the favorite Mozilla Prism and countless others), some CCS enhancements to gradients, masks, and reflections, as well as additional native font rendering which provides a better experience for Windows users.
You can grab the nightlies (currently WebKit r34581) here. Or grab the current Safari GA by heading over to Apple’s site.
*Test machines were running AMD Opteron 1.8GHz Dual-cores with 2GB RAM against ZCS 5.0.6 GA RHEL4. As always performance will vary based on system configuration, network connection, and other factors like account data and preferences.
Round 1 covered Firefox 2 vs 3RC1 and the results were much easier to predict and extrapolate, but it wasn’t the same for Internet Explorer 7 vs 8b. In the heavy weight division IE7 is often compared to a 500-pound gorilla, but could Microsoft convince it to go on a diet for IE8?
Once again we used a customized OpenQA Selenium setup to calculate time-taken rendering a page after clicking a particular button/link; across the same set of saved actions such as logging in, composing and viewing messages, navigating around various folders, switching between our many apps, and even changing options as done on Firefox. And IE7 connected a hard blow right off the bat.
It certainly wasn’t going to be a pushover fight. IE8b wiped the blood off it’s face from the initial loading and decisively won all the little forays. However, when throwing bigger JavaScript rendering tasks at the contestants it often looked to be anyone’s fight. The really crazy thing about it is that while there’s tons of CSS handling improvements in IE7 as opposed to IE6; IE8b touts more JavaScript enhancements than IE7 (what should matter for ZCS) but I just wasn’t blown back in awe. While IE8b’s endurance clearly won in the end, IE7’s punches are gonna leave a few bruises.
It’s good that Microsoft finally realized that people were tired of switching between Internet Explorer and a separate development environment such as IEWatch or Visual Studio and polished up their debug tools for JavaScript, CSS, and HTML. (We’re gonna cover our favorites from Firebug & Charles to Eclipse, IntelliJ, & GDB later.) They also included more ways to give feedback and track feature requests for this beta period as opposed to last time. IE8b contains sparse CSS3 support, but finally complete HTML4 adherence – though we’re rounding on HTML5, and CSS2.1 compliance isn’t new – it was even aimed for IE7 but ended up sub-par. As far as I can tell most of the JavaScript rendering speed reported by others have been because of DOM enhancements (how you store meaningful amounts of client-side data in a persistent and secure manner) rather than the stated JavaScript the engine overhauls. Either that or it just took so long to warm up for a noticeable kick in our tests.
Granted IE8 is farthest out, so hopefully that gives Microsoft time to refine their closed source browser. Don’t get me wrong IE8b is an improvement; but we’re rounding on GA release of Mozilla Firefox 3 next week, while Apple pushed Safari 3.1.1 into OS X 10.5.3 and already has a game-plan for Safari 4, so Internet Explorer better step up the training regiment.
*Test machines were running AMD Opteron 1.8GHz Dual-cores with 2GB RAM against ZCS 5.0.6 GA RHEL4. As always performance will vary based on system configuration, network connection, and other factors like account data and preferences.
As we’ve mentioned before it’s about time for another ‘clash of the titans’ in the never ending web browser wars. Raja Rao of our QA team had previously built a sweet AJAX client testing framework, so we decided to pit the major browser’s current releases and nightly builds verses one another. Who will go down in this first round?
Before you go “Wait, is this particular article just Firefox vs Firefox – aren’t newer versions expected to preform better anyways?” think of it as just a warm-up to instill confidence by beating personal records in preparation for some looming, ugly battle-royal. Plus the gym was destroyed last time we had them all in one place, and our graphs just get too cluttered, so you’ll have to come back for other matches like IE7 vs IE8b and Safari 3.1 vs Safari 3 nightly builds.
For Firefox memory bloat tests have been popular lately, but we wanted some real world JavaScript tests through a set of ZWC tasks on 5.0.6 – such as logging in, composing and viewing messages, navigating around various folders, switching between our many apps, and even changing options.
FF3RC1 clearly won every test, so Mozilla deserves a pat on the back for advancing their browser. While completing little web-client actions there was often barely a difference. However, where heavy rendering had to be done the improvements were significant – in some places half the rendering time or even three times as fast!
What’s the test harness?
We used a customized OpenQA Selenium setup to calculate time-taken rendering a page after clicking a particular button/link. Test machines were running AMD Opteron 1.8GHz Dual-cores with 2GB RAM against ZCS 5.0.6 GA RHEL4.
When we compare other browsers we will be using the same set of saved actions. Try it out yourself and discuss it in the forums – if you’d like some other testing ideas you might play with the other commonly used benchmarks like VeriTest & SunSpider.
So what has made Firefox 3 so speedy? Perhaps it’s the Profile-Guided Optimization (dual pass compiling) builds now being created that are greatly improving performance, or lots of combined JavaScript engine enhancements. Have you been comparing all the nightlies leading up to RC1 or noted the exact change that has enhanced it so much? Discuss it below and stay tuned, I hear Safari is pulling out all the stops – get your bets in now.
Posted in Open Source by Ross Dargahi on April 17th, 2006
IE 6 is an inadequate platform for developing advanced Web 2.0 applications. I suspect that a number of hard core web application developers will nod their heads in agreement with this statement. From my experience, IE 6 is certainly more challenging to work with than some of its competitors, and it exhibits some very unpleasant behaviours that make it a difficult platform with which to develop advanced Web 2.0 applications.
Before I go further, I know some folks may be asking what exactly are “advanced Web 2.0 applications?” I describe them as the class of application making extensive use of AJAX and DHTML – which I like to, for convenience, group together under the AJAX umbrella – to provide a rich and compelling end user experience. Such applications can be easily identified by their ability to provide the flexibility, power, and richness of their desktop counterparts while at the same time bringing the power and benefits of the Web to the end user. The Zimbra Collaboration Suite’s web client as an example of such an application.
Hey, did you hear that Google released a browser? Yeah, and it’s very cool! We might have been a bit early to call Safari the Browser war winner. Based on WebKit (KHTML), this rendering framework (that Chrome uses) has really stormed the market. If you asked us five months ago who was winning the browser war, we would easily say Firefox, with Safari as a close second. With the introduction of Chrome, a new war has started. 
