Zimbra HSM Plus – Secondary Volumes on Amazon S3

In previous blog posts, we’ve seen how to download and install Zimbra Suite Plus and a quick overview of each feature. In this blog post, I want to introduce you to this new, cool feature included in Zimbra HSM Plus: secondary volumes on Amazon S3. This includes products that use the Amazon S3 API, but officially we only support Amazon S3 at the moment. Please let us know about your integrations with Zimbra HSM Plus and your storage solutions in the comments.

How Does Zimbra HSM Plus Work?

Zimbra HSM Plus includes several new features that I will cover in future blog posts, but the basic functionality of Zimbra HSM Plus is to move old items to secondary volumes based on a policy or policies. This allows you to save space on the primary storage solution, which can be SAS15K or SSD, and move old items to NFS, SATA, NL-SAS, etc.

This is a quick overview of how Zimbra HSM Plus works with Amazon S3 secondary volumes.

Create an Amazon S3 Bucket

This blog post is not a deep look into the Amazon S3 buckets, security, etc. I want to show you how to quickly create a bucket on Amazon S3.

On the Amazon Web Services Console, click S3.

Click Create Bucket.

Select a name for the bucket and the most desirable region for the project.

Once the bucket is created, you need an Access and Secret Key with access to this bucket. Please refer to Amazon Help if you have issues with this.

Activate and Configure Zimbra HSM Plus with Amazon S3

The Zimbra HSM Plus Amazon S3 functionality needs a local cache folder inside Zimbra Collaboration. This folder can be on the primary storage, as you can limit the size on the Admin Console. If you fail to create this folder, Zimbra HSM Plus will create errors on the users when they try to access their old items. Click here for more info about it.
As root user:

mkdir /opt/zimbra/cache
chown zimbra:zimbra /opt/zimbra/cache

Once logged into the Zimbra Admin Console, navigate to Zimbra Suite Plus > HSM > and under Secondary Volumes > Add

Select the second option S3 Bucket.

Now you can select the name you want for the volume, a prefix if needed, and you will be able to select a S3 profile if you have one already, or if not, you will be able to create a new S3 profile with your credentials, etc.

If you didn’t have a S3 profile, this step allows you to create a new one. Simply introduce the bucket name created on Amazon S3 AWS, an Access and a Secret key with write-access to that bucket.

Mind the option Amazon S3 or Custom. As I said, Zimbra Suite Plus should work with other storage vendors who uses the Amazon S3 API, but they are not supported yet.

Select the Region where you’ve created the bucket in the previous steps, and click Test S3 bucket.

If you’ve introduced all the correct information, the test will show a message saying the bucket is valid.

If you return to the Amazon AWS S3 Console and navigate through the bucket, you will see the test connection files. You can safely remove these files.

The wizard will not return to the previous window, but in this case you will have already selected the S3 profile. Click Next.

Check the Set as current checkbox to start saving files once the HSM policy is configured.

The Amazon S3 profiles can be edited, removed or you can add more under the menu Home – Configure – Global Settings – S3 Buckets.

Configure the Zimbra HSM Plus Policy

To move the old items to the Amazon S3 secondary volume, you need to create an HSM Policy. In this example, I’ve selected all items, and only the ones older than 30 days will be moved to the HSM secondary volume, in this case an Amazon S3 bucket.

At the same time, I want to schedule the task to be automatically executed at 02am every night.

For example, I have a demo mailbox with items older than 30 days. Per our previous policy, all the items will be automatically moved to the Amazon S3 bucket. This operation will be invisible to the end-user: it will just take a few more seconds to load old items on the web client once he or she needs them. The items on the primary volume will be accessible faster, as usual.

Once the HSM process completes the task of moving all the old items matching the policy to the secondary volume or volumes, you can navigate through the Amazon AWS S3 Console bucket and to see the directory with the regular Zimbra folder structure with the user ID, etc.

If you navigate inside a userID folder, you will see the regular .msg files, size of the msg, etc., all inside Amazon S3.

And that’s it for this Blog post. We now have all the old items protected and secured on Amazon S3, and we have freed up the local storage which should be super fast. Now, with Zimbra HSM Plus, years and years of old items are moved from an expensive storage solution to the Amazon S3 Cloud.

Remember that you can download and try it free for 30-days: Zimbra Suite Plus here.