From Gzip to JSMin to Shrinksafe to YUI Compressor

When Zimbra started out we had only a few thousand lines of JavaScript and didn’t really optimize our JavaScript at all. As we added features to the Zimbra Web Client, the code base grew pretty fast. On the client/end user side, raw Javascript in hundreds of script tags were received. This came out to be about 2MB of raw/uncompressed JavaScript, in about 75 files.

In early 2005 (before our first public beta) we took our first step at optimization. We wanted to combine the 75 or so files into 3 or 4 larger files. This made a big difference in load time as it reduced the number of requests(and http connections) we made for the JavaScript files.

The next improvement was to add gzip compression to reduce size of code we sent to the client. We introduced this in Zimbra 3.0, and the browsers of the time weren’t particularly fast at loading complex AJAX scripts, so we tried to reduce load time by reducing file size. We managed to reduce the size on the wire by about 75%. At the time we still downloaded our entire client code base at login/startup time. Zimbra continued to grow with new features, and we increased our development team.

One of the biggest complaints we heard about Zimbra, was its loading time. We now had Large Javascript files that were gziped on the fly, but loading time still wasn’t as fast as we’d hoped and wanted.

Looking to further compress and shrink the codebase we found a tool called JSMin, or the JavaScript Minifier written by Douglas Crockford. JSMin is a really basic utility that filters javscript and removes comments and whitespaces from code. This gave us a significant savings when ran prior to gizipping.

This, however, cause a problem. Since JSMin was RegEX based, we would periodically hit issues where it would choke on a certain complex JS expressions. Worse yet, it would break the working code in ways that only appeared at run-time. This meant that it would be difficult for us to debug or reproduce issues until our users found them.

JSMin is a great tool, but the Zimbra code was becoming too complex to run through the tool. By this time we’d found ShrinkSafe by Dojo. After evaluating its features, we thought it might be a good fit. It is based on Rhino so was a true JS parser and didn’t mangle working code. [It did have one issue with certain eval() calls but it was easy to work around.] In addition to removing whitespace and comments like JSMin it would rename and reduce the length of internal variable and function names. So it really compressed the javascript code in a way that helped reduce load time. This gave us about 25% compression before gzipping.

While ShinkSafe worked really well, we were always on the lookout for something better. We read about the YUI Compressor on Ajaxian. The YUI Compressor is written in Java by Julien Lecomte, like ShrinkSafe it uses Rhino to tokenize, or parse the JavaScript. It achieves a higher leverl of compression that the Dojo compressor. One of the main reasons is that it will reuse the short tokens unlike ShrinkSafe which keeps an ever increasing counter. In our testing, it didn’t break our code and is careful around eval() so avoids a couple issues the ShrinkSafe compressor hit. Since it’s Java we can call it at run-time and compress Zimlets, for example, on-the-fly. The YUI Compressor also compresses CSS and we use that to help reduce the size of our CSS files that we deliver to the browser. In the end we still gzip both CSS and JavaScript to get the most compression we can. The use of YUI for JavaScript compression is part of Zimbra’s upcoming 5.0 release.

5.0 will also include our new ‘Lazy Loading‘ method to help further decrease startup/login tmes. Learning from Dojo’s package system, Conrad Damon and Andy Clark started a project on the client team to move to component based loading system. Basic idea is you group the code into small packages or modules, and create dependencies on each. So for startup you’d require the login package and the mail core package as an example.

With Lazy Loading, and YUI Compressor, we now have loading times 3 to 4 times faster with our AJAX client when compared to the latest ZCS 4.5.x release. Interestingly enough, Safari 3 seems to be the fastest of all of the browsers we’ve tested (well save the details of that test for a future blog post).

Written by Kevin Henrikson, published by jholder