Hey, did you hear that Google released a browser? Yeah, and it’s very cool! We might have been a bit early to call Safari the Browser war winner. Based on WebKit (KHTML), this rendering framework (that Chrome uses) has really stormed the market. If you asked us five months ago who was winning the browser war, we would easily say Firefox, with Safari as a close second. With the introduction of Chrome, a new war has started.
At the start of this century, the war was about “Open-ness” and who could be more open and win the hearts of users. Now it’s a war of speed, and who’s faster. A few blogs and articles have been written, with Mozilla and Google both claiming their JS engine is faster. So who’s right? Both are faster than IE (6, 7, and 8), but in our opinion, what matters is how responsive web applications are. So who will win Zimbra’s Speed trophy?
Zimbra has a testing harness thats in alpha which we will be making available to the public in the future, that measures performance on different actions within Zimbra. This helps us understand what the end user is seeing. People can talk V8 Benchmark, Dromaeo, SunSpider, or what ever they want. What really matters is how applications perform. Our tests are pure UI performance, ie, how fast Zimbra is to the end user.
Considering that one of Zimbra’s strengths is our AJAX web interface, we decided to put Chrome to the test, along with IE, FireFox, and Safari. The control system was: Intel Core2 duo, 2.39Ghz 1.99GB RAM Windows XP
Here’s how it did (lower is faster):
Overall Performance
All Tests
Given that Chrome is built on WebKit, this didn’t come as any particularly huge surprise. In our previous tests, Safari came out the fastest renderer of the Zimbra Web Client. In our tests, Chrome came in as a very close second, and we expect it to get faster.
We want Chrome to work as good as FireFox or Internet Explorer. So, if you find an issue, please report it in the bug report below.
Want more info on the browser war? Check out these links:
Who won the browser war? - http://www.zimbrablog.com/blog/archives/2008/06/and-the-winner-is.html
Safari vs Safari- http://www.zimbrablog.com/blog/archives/2008/06/browser-war-part-3-safari-311-nightlies.html
IE vs IE - http://www.zimbrablog.com/blog/archives/2008/06/browser-war-part-2-ie7-vs-ie8b.html
FF vs FF - http://www.zimbrablog.com/blog/archives/2008/05/round-1-ff2-vs-ff3rc1.html
Support Opera for Zimbra Web Client - http://bugzilla.zimbra.com/show_bug.cgi?id=5932
Hack the Zimbra Web Client to Support Chrome - http://www.zimbra.com/forums/users/21903-googles-chrome-browser.html
on September 4th, 2008 at 1:57 am
So, I can’t get Zimbra to run the “advanced” AJAX version in Chrome. Zimbra just reverts to the HTML version every time I load the client. We’re running the latest release. You guys seem to be running it fine. Why can’t we?
Thanks!
on September 4th, 2008 at 2:25 am
There is an open bug.. until you get a patch, you can hack it as described here:
http://bugzilla.zimbra.com/show_bug.cgi?id=31254#c4
In the meantime, for those that want to hack the client code to get the
advanced version to come up:
- Edit /ZimbraWebClient/WebRoot/public/launchZCS.jsp
- around line 199, comment out “switchToStandardClient();”
- run “ant sync” under ZimbraWebClient
on September 4th, 2008 at 2:48 am
I’ve encountered the same issue as Noah… I haven’t been able to get the advanced version to run under Chrome or Safari. Is there some hidden workaround?
on September 4th, 2008 at 3:33 am
An observation: Test no 28 caused that narrow loss, other than that on most parts Chrome is faster.
on September 4th, 2008 at 10:03 am
With with Firefox I have the iMacros addon. These macro scripts alone make Firefox the fastest browser for me, as it saves me tons of typing and clicking in Zimbra and on other websites.
on September 4th, 2008 at 11:29 am
When you say you tested ffox 3.1b1, I assume you turned on their new javascript interpreter?
on September 4th, 2008 at 12:43 pm
So, if you exclude Test 28, Chrome is the winner. What exactly do you test on that phase?
on September 4th, 2008 at 1:31 pm
Noah, the very last link in the article points to “Hack the Zimbra Web Client to Support Chrome”. That link actually takes you to a forum thread and you have to click on another link.. You can skip that and find the instructions from John here: http://www.zimbra.com/forums/administrators/21908-google-chrome-browser.html
on September 4th, 2008 at 6:02 pm
Does “overall performance” simply sum the times of the tests? Perhaps a geometric mean would be a clearer statistic.
on September 5th, 2008 at 3:33 pm
@Louis Re: Tracemonkey in FF3.1
Yes we tested with JIT turned on in about:config
@Peter Kasting
We used a customized OpenQA Selenium JavaScript test setup to calculate time taken rendering a page after clicking a particular button/link. This runs them all through the same set of ZWC tasks - such as logging in, composing and viewing messages, navigating around various folders, switching between our many apps, and even changing options. As soon as one task completes the next begins, thus total time for the entire rendering is a good measure.
on October 3rd, 2008 at 10:07 pm
Understand a few factors in play before you declare Chrome the winner of this ‘browser war.’
First off, Chrome is, as the article said, based off Webkit. Webkit is the open-source web rendering engine originally developed by Apple, on which Safari is based. Chrome is also open-sourced, and even now Chrome’s core technologies are making their way back into the Webkit repository, meaning that everything Google has done in Chrome is available in Webkit for Apple to use in their next version of Safari.
The second thing to understand is that Google simply has no interest in there being a ‘browser war,’ much less winning one. Their purpose is improve all web browsers so that they run web applications better, because web-based apps are what Google really wants to be about. Chrome’s main purpose was as a wake-up call to other browsers to take these new technologies seriously, or get replaced.
The thing I love about Google is that their definition of winning the browser war includes Safari and Firefox winning as well, and most importantly, web users everywhere winning.
on October 3rd, 2008 at 10:19 pm
There’s a problem with benchmarking Chrome which I think that even Zimbra isn’t addressing.
Chrome’s main advantages have nothing to do with raw rendering speed.
You know how after a day of browsing with dozens and dozens of open tabs, your browser crawls to a near halt? I use Safari, and after a few hours of many-tab browsing I feel like I’m crawling the internet on a 14k modem. I have to restart Safari to get speed back. Essentially inefficiencies in memory management are rapidly compounding over time until the processor is doing 100x as much work as necessary to deal with user requests.
This is the biggest issue that Chrome was developed to address, and it’s where its biggest speed gains will be. Unless benchmarks are designed that simulate the memory issues of sustained many-tabbed browsing, they won’t come remotely near to showing the speed gains Chrome creates.
on October 14th, 2008 at 2:23 am
Could anyone explain how to do the hack in layman’s terms? It is really bugging me that I can’t use advanced, especially since it was working fine up until earlier today!
on October 16th, 2008 at 12:25 pm
Hi, Peter Kasting:
Are you using Selenium IDE to automate Browser? My experience is that Selenium IDE can’t support some right-click, doule-click very well in Firefox. Do you add some modification in Selenium IDE?
—”Zimbra has a testing harness thats in alpha which we will be making available to the public in the future, that measures performance on different actions within Zimbra. ”
Do you have any plan to open the automation way/tool for various browsers?
Another curious thing is: Chrome hasn’t open its’ interactional API, how did you achive to automate it for your benchmark?