Zimbra

HackDay just kicked off, and we’ve seen all sorts of things go from idea to prototype in just 24 hours. This year has a twist: Not only are our engineers across the globe involved (with many making it an all-nighter) but everyone is invited to participate - even you. Open Hack 2008 takes place September 12-13th where anyone with an idea is encouraged to gather a team up, then spend a day building stuff that they think is cool. HackU (the University Hackdown) is even flying in its top ranks, with a few Zimbra customers among them (including Stanford & Georgia Tech) to join us at Yahoo! HQ in Sunnyvale, CA for rounds of coding; plus camaraderie, food, demos, awards, and good music. From our team KevinH & JohnH are also giving several presentations throughout the event.

Some of the things we’re ran across have given us ideas for Zimlets - so we’re hereby seeking Zimlet beta testers. This will be an ongoing project, meaning that those who volunteer will get continuous access to the latest and greatest Zimlet ideas.

We need volunteers that:

• Are using either ZCS (with the ability to deploy Zimlets - so essentially administrators of the respective system) or Zimbra Desktop users.

• Are willing to try out different services that some of these may link to, and consciously note how they affect their daily Zimbra experience.
• Of course give us feedback about their place in productivity, effectiveness, usefulness, and anything you’d like to see added or extended.

To join in just send me a PM/Email by the end of this week. (We’ll be sure to reward you for your efforts.)

We can’t take everyone, so if you not accepted don’t feel bad - there’s still plenty of cool & useful Zimlets over in the Gallery - plus they’ll soon be making it off engineer workstations and into perforce. We’re even working on a way to make them easier to install in Zimbra Desktop, but you can find current directions here.

Those on the development side will soon see a few community members marked “Zimlet Guru” - if you’ve created a few yourself, and are into helping out others in the Zimlet section of the forums, be sure to drop me a line.

Hey, did you hear that Google released a browser? Yeah, and it’s very cool! We might have been a bit early to call Safari the Browser war winner. Based on WebKit (KHTML), this rendering framework (that Chrome uses) has really stormed the market. If you asked us five months ago who was winning the browser war, we would easily say Firefox, with Safari as a close second. With the introduction of Chrome, a new war has started.

At the start of this century, the war was about “Open-ness” and who could be more open and win the hearts of users. Now it’s a war of speed, and who’s faster. A few blogs and articles have been written, with Mozilla and Google both claiming their JS engine is faster. So who’s right? Both are faster than IE (6, 7, and 8), but in our opinion, what matters is how responsive web applications are. So who will win Zimbra’s Speed trophy?

Zimbra has a testing harness thats in alpha which we will be making available to the public in the future, that measures performance on different actions within Zimbra. This helps us understand what the end user is seeing. People can talk V8 Benchmark, Dromaeo, SunSpider, or what ever they want. What really matters is how applications perform. Our tests are pure UI performance, ie, how fast Zimbra is to the end user.

Considering that one of Zimbra’s strengths is our AJAX web interface, we decided to put Chrome to the test, along with IE, FireFox, and Safari. The control system was: Intel Core2 duo, 2.39Ghz 1.99GB RAM Windows XP

Here’s how it did (lower is faster):

Overall Performance

---

All Tests

---

Given that Chrome is built on WebKit, this didn’t come as any particularly huge surprise. In our previous tests, Safari came out the fastest renderer of the Zimbra Web Client. In our tests, Chrome came in as a very close second, and we expect it to get faster.

We want Chrome to work as good as FireFox or Internet Explorer. So, if you find an issue, please report it in the bug report below.

Want more info on the browser war? Check out these links:

Who won the browser war? - http://www.zimbrablog.com/blog/archives/2008/06/and-the-winner-is.html
Safari vs Safari- http://www.zimbrablog.com/blog/archives/2008/06/browser-war-part-3-safari-311-nightlies.html
IE vs IE - http://www.zimbrablog.com/blog/archives/2008/06/browser-war-part-2-ie7-vs-ie8b.html
FF vs FF - http://www.zimbrablog.com/blog/archives/2008/05/round-1-ff2-vs-ff3rc1.html
Support Opera for Zimbra Web Client - http://bugzilla.zimbra.com/show_bug.cgi?id=5932
Hack the Zimbra Web Client to Support Chrome - http://www.zimbra.com/forums/users/21903-googles-chrome-browser.html

Part 1 covered Network Edition backup features, today’s snips apply to all editions.

First among the lesser known additions: We recently provided the possibility for a nice performance boost to some environments by adding the ability to turn on batched indexing in ZCS 5.0.3 (you can even fine tune it at the localconfig, COS, and account level). We’re not talking about when you re-index an entire account here, this is a change to the index-as-received model; now new items can sit in a ‘queue’ (really a ‘indexing deferred’ flag on the mail_items table of the pertaining mboxgroup database in MySQL) to run all at once when it reaches the zimbraBatchedIndexingSize threshold, saving you from all the tiny disk thrashing. It might not be immediately apparent that this works better, but you can mention it in the forums and we’ll show you the evidence to the contrary - it proves expecially useful for POP heavy or ZAD archive accounts.


 New & Enhanced Admin Tools

 

Your /opt/zimbra/bin & /opt/zimbra/libexec directories hold a wealth of tools to make your job easier.

zmdumpenv has been around for a long time, but underutilized - it grabs the basics that you should probably provide with every issue to help others understand where you’re coming from.

When you need to send ad-hoc SOAP commands to the server, the powerful zmsoap takes care of authenticating, generating the envelope, sending the request, and writing the response to stdout.

If your server freezes or is busy, running zmdialog can give that ‘my server hung’ support ticket a purpose. With JDK 1.5 it won’t collect a heap dump so you might also run /opt/zimbra/java/bin/jmap -heap:format=b [/opt/zimbra/log/zmmailboxd_java.pid] however zmdiaglog collects a core dump, from which it should be theoretically possible to get a heap dump. Thread dumps when you kill -QUIT/3 [pid] are helpful too. Info on ways to take them (like /opt/zimbra/libexec/zmmailboxdmgr threaddump) plus a handy script, are here.

There’s the MySQL metadata DB, the Lucene index, and the actual blob files on disk in /opt/zimbra/store. If you can click on a folder in the web UI and get results, it means MySQL is up and happy. If you open a message and get the infamous “missing blob for id” error, it means that the message files on disk aren’t where they’re supposed to be (ie: disk crash, not mounted/permissions, or you’ve recently pointed your volumes incorrectly). There’s all sorts of things you can do in that situation - here’s a more detailed list. Even if you don’t have a good grasp of the mailbox database structure yet, you can appreciate the need to easily determine if those blobs are still available (after you’ve moved whatever you can salvage of the store from corrupted drives to a better location). Enter the zmblobchk utility which can determine what files are missing - there’s also plans for a repair mode, for when you finally realize the blobs are gone and want to get rid of those UI error messages. Of course you’ll next consult the Network Edition’s restoreToTime feature or other backup solution you may have.

When you’re absolutely out of room on the disks housing your DB (seriously put in that purchase order for more storage - it’s cheap these days) in a pinch the optimizeMboxgroups.pl script (available in the public cache and added to the upcoming 5.0.10) can help you recover wasted space in your mail_item, appointment, imap_folder, imap_message, open_conversation, pop3_message, revision, and tombstone tables on each mboxgroup. Just note that it temporarily locks each table, and could use considerable IO while they’re being rebuilt. You can certainly use it pro-actively during a maintenance window to reclaim space as well.

There’s also a new wiki page on statistic collection so you can generate nice charts that help you figure out what you might need to tweak.

We could go on and on about the utilities in those folders, so throw up a test environment and experiment sometime - it may just make life easier when you have hundreds of users breathing down your neck. And if you should ever be ’stumbling around in the dark’ you can always enable additional debug logging to shed more light on the situation.

 

---

You can find help for all the above utilities over in the Community Forums or ask us a question on them below.

It’s truly amazing how excited people get over Zimbra - so thought we’d share some of the ways people show it.

We were pleasantly surprised a few days ago when a school declared a ‘national Zimbra day’ and sent us pictures of cupcakes (that we can virtually enjoy). Why the baked goods? They recently rolled out ZCS and love it. Since we are also launching 5.0.9 today it’s fitting.

As the summer is the perfect time for schools to do system upgrades when students aren’t around, Thunder04’s organization had a conversion party while they worked on their switchover - with some delicious goodies of course.


The Menlo Park City School District (which serves Menlo Park & Atherton in California) would like to wish you all a “Happy Zimbra Conversion or Upgrade Day” as ZCS 5.0.9 has just been released! (This version includes additional beta builds for Ubuntu 8.04 LTS in both x86 & 64-bit as well.)

 

Linux users have pondered: “Wouldn’t it be nice to just grab Zimbra software via repositories?”

There was just so much positive feedback over Zimbra Desktop Beta 3 (with over 900 new members to the forums last month!) we thought that Ubuntu 32-bit users should be able to grab it easily. (64-bit support is coming soon)

This brings Zimbra Desktop’s easy setup against ZCS, Yahoo! Mail, Gmail, AOL and any other IMAP/POP accounts to the huge Ubuntu community - letting them take mail, contacts, calendars, tasks, documents, and briefcase items offline and sync whatever actions they may take when they’re reconnected.

To install:
sudo synaptic (As you’re read this it’s being added to Applications > Add/Remove.)
Enable the third party packages in settings > repositories.
Reload, search for “Zimbra”, and away you go. (same as a sudo apt-get install zdesktop)

Once downloaded, start via Applications > Internet:



Many thanks to the Canonical Team for showcasing us on their frontpage: ubuntu.com/news/zimbra-desktop

Not using Ubuntu? It’s cross-platform for other Linux variants, as well as Windows and Mac - grab it here.

You may ask: “What about the entirety of the Zimbra Collaboration Suite Server in Ubuntu or other repositories?” We can’t say one way or the other at this point - but think of this as a harmonious step.

 

---

Find help for Zimbra Desktop over in the Community Forums, ask us a question below, or fill out the Ubuntu registration/feedback on it: ubuntu.com/register/zimbra/

This may be the most important post you will ever read. If you’re a Zimbra Administrator, please read, and pass this onto your colleagues who use Zimbra. If you’re a CTO or CEO, take time to ask your Zimbra Admin about the subject of this post. This blog post is about backups.

Whether your an Open Source User, Zimbra Desktop User, or Network Edition Customer, you can do backups of your data. There is nothing worse than getting a call from a customer, or a Private Message from a Forum User that says, “I need help. My HD has crashed, and all my backups were on that drive.”

Let me Digress for a moment, and share with you my personal experience with backups. It’s sort of a Legend here at Zimbra. It all begins in 2005, Zimbra was the new kid on the block and I was an inexperienced System Linux admin for Tombstone Unified School District in Arizona. This was a small district with limited funds. When I interviewed for the post, the Superintendent handed me his card, and it had a Hotmail address on it. Right then, I knew this would be quite a difficult job.

One of the first things I did, was investigate E-mail Server platforms, and TCO (Total Cost of Ownership). A quick search on the interwebs (which is a series of tubes, not to be confused with the you tubes), yielded this new thing called a Zimbra. Much like our millions of Users, I downloaded it and tried it. It worked great! We used the Zimbra Open Source Edition (then it was in Beta).

I was so proud of my cool new setup, and our staff were so excited to finally have good e-mail and calendaring. In knew the importance of keeping backups, and archiving. All of you System Admins at Hospitals and Government institutions know what I mean. You get well acquainted with the Laws and Requirements. So, I would stay up until about midnight, and stop Zimbra, and rsync /opt/zimbra into /opt/zimbra/backups. Keep in mind that this was in the early days of Zimbra, and I wasn’t even an employee yet. As a matter of fact, myself and several others, pioneered the Open Source backup procedure.

Everything was great until one night I noticed I was running low on Disk Space on my /opt partition. So, I thought, “If I just remove all my backups, and make a fresh one, that will save me a bunch of space”. So I ran the following command as root:
rm -rf /opt/zimbra backups
Now, back then, we mounted a clamav partition ramdisk for quarantine purposes. The only indication that I had that something was wrong was when I got an error saying that it couldn’t unmount the partition because it was in use. Everything else in /opt/zimbra was gone…including my backups.

As most of you admins know, preserving data is important. We were involved in several litigation matters, and I would later be cited for obstruction of justice for destroying evidence.

When I noticed what had happened, I immediately called Zimbra and talked with MarcMac here at Zimbra. He tried to recover the inodes using Midnight Commander, but it was a total loss.

Lesson Learned. So, from one admin to another, please take time to make sure your backups are not located on the same machine that Zimbra is on. Please! We never want to hear about data loss. Whether an opensource user or network user, I hope you will take a few minutes to consider your backup strategy, and fix any single points in failure.

Learn from my experience.
-John (jholder)

This week, Zimbra held a Zimbra Administration Course at the Yahoo! headquarters in Santa Clara. For those who don’t know, the administration course covers just about everything in Zimbra from A to Z.

I had the opportunity to lead the first two days of the basic/general sessions of the training course. Attendee’s learned everything from Installation and upgrades to Java Garbage Collection and Disaster Recovery.

We like to play a game during training called “stump the chump”, where attendee’s who stump me get t-shirts. Attendees also get to keep all the training materials, and exclusive access to a special training forum called “Camp Zimbra.

The Third day is the advanced course, and it was led by Anup P. Anup is a Zimbra Service Engineer who has a ton of Large Deployment experiences under his belt. He led the Third day covering things like performance graph generation, and cluster-specific options and questions.


Congratulations to the Zimbra Admin Class of June 2008 from all of us at Zimbra!

---

If you’re interested in when a Zimbra Training will be available for your region, or when the next one will be available, check out our Zimbra Training Page.

Times have been busy for us here @ Zimbra. We have been working to get some cool stuff into 5.0.3, and some other cool stuff that we’ll blog about later. In the mean time, Zimbra’s been nominated for the a Webware top 100 award.

Who do you think should win?

If you’re a Zimbra fan, show us you love us. Click to vote.:)
< /end shameless plug>

Zimbra has a toaster for Mac, and a toaster for Windows…but didn’t have a toaster for Linux until pbruna came along with his adaptation of the checkgmail program by Owen Marshall. Linux fanboys rejoice, your toaster is here (bread not included).

(more…)

And we’ve got a surprise for blog subscribers who will be attending

(more…)