Roland pointed me to an article about the amount of Javascript on digg.com’s home page. Seems a few Digger’s found it interesting. Well here at Zimbra we’ve got tons of Javascript and CSS in our AJAX web app. I decided to try an experiment to see if the techniques we use for Zimbra, a rather large AJAX app would help a site like digg.
First some background on Zimbra’s AJAX app and the techniques we currently use. Raw and uncompressed the Javascript alone is over 2Megs. To make our app easier to download we’ve made some optimizations.
For Javascript we combine all the files in to a single _all.js file. The order that you concat these files is important since the browser will parse the files top to bottom. The same reason why the order of the script tags in your HTML document would matter to prevent dependency problems.
We then use jsmin to remove comments, white space, and excess new lines. This gives on average 25-30% reduction in size. The final and most important step is to gzip the file. This gives on average 300-400% reduction in size. Some web applications will apply gzip compression on-the-fly, which for dynamic data is the only way. Here since the Javascript is static, which is true for most AJAX applications these days, we can pre-compress which will reduce latency and CPU overhead.
For CSS we also combine all our CSS files into a single _all.css file. This combined file is then compressed with gzip.
Now back to the experiment. I took a copy of Digg’s home page and applied the techniques above. Combined, jsminified, and gziped the Javascript, then combined and gziped the CSS. Visually and functionality nothing has changed. Under the covers however the source is quite a bit smaller. I used Web Page Analyzer to compare the results. We see a 50% reduction in the number of HTTP requests and a 65% reduction in the size of the download.
Original:
Total HTTP Requests: 26
Total Size: 199246 bytes
Zimbrafied Digg:
Total HTTP Requests: 13
Total Size: 70040 bytes
Not bad at all… Took me longer to write this up than to make the changes and I think all digg user’s would benefit from such a change. Not to mention digg itself would see a > %50 reduction in bandwidth usage which for a site like digg could be very significant.
Love working on ajax optimization problems like this? Check out our careers page and drop us an email.
on January 28th, 2006 at 3:39 pm
Wow. Very nice post :).
on January 28th, 2006 at 3:56 pm
Great article - can you explain how you pre-compress a gzipped file? I have only seen how to do it with an apache mod or through PHP, etc.
on January 28th, 2006 at 11:11 pm
That’s amazingly cool. It’s great people still do things just to be helpful, not for money and the like.
Good job.
on January 29th, 2006 at 12:20 am
For this example I used the gzip ANT task in our build system. You could also use the gzip command found on most Unix type OS’s.
on January 29th, 2006 at 12:31 am
JC: Use mod_deflate.
God Bless ya!
on January 29th, 2006 at 12:36 am
Wow… Great stuff!
on January 29th, 2006 at 1:51 am
Woah. Sweet stuff!
on January 29th, 2006 at 2:20 am
is it possible to send gzipped version only if the browser supports it? (Preferrably a prebuilt gzipped version)
on January 29th, 2006 at 2:40 am
I also thought about gzip’ing CSS and Javascript files, but how do you handle these problems:
Internet Explorer May Lose the First 2,048 Bytes of Data That Are Sent Back from a Web Server That Uses HTTP Compression
http://support.microsoft.com/default.aspx?scid=kb;en-us;Q313712
http://support.microsoft.com/default.aspx?scid=kb;en-us;Q312496
This looks like a serious road block.
Also, Mozilla and IE will only decompress files, if they have allowed compression (configurable by the user in Mozilla at least). So you cannot just unconditionally deliver gzip’ed content.
Also, I have read about older (not specified which) IE versions that will fire window.onload after script tags in have loaded their source, but before the source was decompressed and parsed.
Looks like Zimbra takes the “oh well, works for >99% of the users”, it seams. Not that this is not unreasonable.
Also, there is a note on the Zimbra login page about caching problems with IE. Does the problem occur only with compression, or also without?
on January 29th, 2006 at 7:57 am
Zimbra is a great app, we use it, Digg is a great social news site. Glad to see two of my favourite things combined into something better!
on January 29th, 2006 at 9:44 am
@zanzina,@Martin
Yes we package and ship both a gzip precompressed file and a jsminified file (non-gzip). At run time we look at the browser headers(Accept) and to determine if they support gzip. So we handle the case where a browser doesn’t support gzip or has it disabled. From our testing this seems to cover the problem certain IE(6.0 SP1) browsers have.
The IE cache problem is different. This is where the IE browser cache gets damaged/corrupted and will cause every image, script, etc to be read from the server each time. The fix here is to simply clear your cache.
on January 29th, 2006 at 11:25 am
Duh… Tree, Forest and stuff. Of cause one can generate different and tags dependent on the client… Thanks for the slap
But how about the problem described in the two knowledge base articles? It cannot be detected, so there is nothing one can do about it.What does Zimbra do about this?
I guess it depends on your assessment of the risk if you do not deliver compressed files to IE at all (or rely no user agent sniffing to deliver compressed files to IE6 SP1 and up only) or if you risk the breakage.
on January 29th, 2006 at 12:00 pm
From our testing it doesn’t always cause a problem. In fact we spent the better part of last week working with IE 6.0 SP1 around this issue and an unrelated problem with image caching. The hotfix[1] for this bug has been out since 2003, and there are so many other ways to get the fix (XP SP2, IE SP2, etc) that we still send gzip content to IE if it claims to support it. We also have a URL flag ?gzip=false where user’s can disable gzip if they are unable to apply the hotfix, or get the fix via one of the many service packs.
[1] http://www.microsoft.com/downloads/details.aspx?familyid=A4EA08F9-BCCA-4CFF-AE0B-3F0FD945F586&displaylang=en
on January 30th, 2006 at 4:52 am
Very interesting. I am trying this out myself and while I have compressed our css file using gzip, any attempt to load it does not seem to work.
Do we need to edit mime types on the server etc to enable this ?
Many Thanks
on January 30th, 2006 at 7:44 am
Yes, you’ll need to set the correct content-type headers. We use a ServletFilter but this could also be done with mime extension mapping.
on January 30th, 2006 at 9:00 am
How can you have a 300% reduction in size? If you have a 100% reduction, is the size not gone? Are we talking some kind of negative dimensional incursion where the size moves to an alternate dimension, and then increases 300%?
on January 31st, 2006 at 12:13 am
I would be very interested in seeing some sample code on how exactly you determine whether or not to send the GZIPped content, this would be very much appreciated.
on February 2nd, 2006 at 12:39 pm
@anonymous - Think about it in reverse. It’s just a way of saying 3-4 times smaller.
@bartb - This code is available in our CVS. For more code related questions please join our forums.
http://www.zimbra.com/forums
on February 9th, 2006 at 2:17 pm
Very nice article. Funny, I called with someone at Zimbra today as I’m looking at getting a new mailserver for work. This blog and the way the guy was speaking on the phone shows a very passionate company. something sadly lacking in the corporate world.
on January 30th, 2007 at 5:02 am
Hi Kevin,
Nice blog. I’ve written something similar.
http://www.peterbe.com/plog/gzip-and-slimmer-optimization
Combining the .js files into one will be my next challange.
on January 31st, 2007 at 10:32 am
hi guys,
well this is all true and we are basically doing the same thing!
im also using
http://developer.mabwebdesign.com/cssoptimizer/compress.html for css
and
http://alex.dojotoolkit.org/shrinksafe/ for js.
also gzip in web server.
now, i would like to hear what do you guys do with “heavy” ajax call, do you have a recommended method fo “shrinking” the content that is sent to server ?
on February 21st, 2007 at 3:17 pm
Yeah, anonymous is right on the reduction stuff. You can’t say something gives you a 20% reduction, meaning it’s 80% of its original size and then say something else gives it more than 100% reduction. With your 300-400% reduction logic, a reduction less than 100% would have to mean the file actually got larger.
You can say the original is 300% to 400% larger than the compressed version, but greater than 100% reduction makes no sense.
on August 18th, 2007 at 5:23 am
AJAX, by now always the best!!! thanks for post.
on November 11th, 2007 at 1:52 pm
thanks for post
very useful
on December 30th, 2007 at 11:31 pm
Thank you,
AJAX, by now always the best!
on December 30th, 2007 at 11:35 pm
Very nice article. Funny, I called with someone at Zimbra today as I’m looking at getting a new mailserver for work
on May 26th, 2008 at 10:15 pm
Good post I liked it