In previous blog posts, we’ve seen how to download and install Zimbra Suite Plus and a quick overview of each feature. In this blog post, I want to introduce you to this new, cool feature included in Zimbra HSM Plus: secondary volumes on Amazon S3. This includes products that use the Amazon S3 API, but officially we only support Amazon S3 at the moment. Please let us know about your integrations with Zimbra HSM Plus and your storage solutions in the comments.
How Does Zimbra HSM Plus Work?
Zimbra HSM Plus includes several new features that I will cover in future blog posts, but the basic functionality of Zimbra HSM Plus is to move old items to secondary volumes based on a policy or policies. This allows you to save space on the primary storage solution, which can be SAS15K or SSD, and move old items to NFS, SATA, NL-SAS, etc.
This is a quick overview of how Zimbra HSM Plus works with Amazon S3 secondary volumes.
Create an Amazon S3 Bucket
This blog post is not a deep look into the Amazon S3 buckets, security, etc. I want to show you how to quickly create a bucket on Amazon S3.
On the Amazon Web Services Console, click S3.
Click Create Bucket.
Select a name for the bucket and the most desirable region for the project.
Once the bucket is created, you need an Access and Secret Key with access to this bucket. Please refer to Amazon Help if you have issues with this.
Activate and Configure Zimbra HSM Plus with Amazon S3
The Zimbra HSM Plus Amazon S3 functionality needs a local cache folder inside Zimbra Collaboration. This folder can be on the primary storage, as you can limit the size on the Admin Console. If you fail to create this folder, Zimbra HSM Plus will create errors on the users when they try to access their old items. Click here for more info about it.
As root user:
mkdir /opt/zimbra/cache chown zimbra:zimbra /opt/zimbra/cache
Once logged into the Zimbra Admin Console, navigate to Zimbra Suite Plus > HSM > and under Secondary Volumes > Add
Select the second option S3 Bucket.
Now you can select the name you want for the volume, a prefix if needed, and you will be able to select a S3 profile if you have one already, or if not, you will be able to create a new S3 profile with your credentials, etc.
If you didn’t have a S3 profile, this step allows you to create a new one. Simply introduce the bucket name created on Amazon S3 AWS, an Access and a Secret key with write-access to that bucket.
Mind the option Amazon S3 or Custom. As I said, Zimbra Suite Plus should work with other storage vendors who uses the Amazon S3 API, but they are not supported yet.
Select the Region where you’ve created the bucket in the previous steps, and click Test S3 bucket.
If you’ve introduced all the correct information, the test will show a message saying the bucket is valid.
If you return to the Amazon AWS S3 Console and navigate through the bucket, you will see the test connection files. You can safely remove these files.
The wizard will not return to the previous window, but in this case you will have already selected the S3 profile. Click Next.
Check the Set as current checkbox to start saving files once the HSM policy is configured.
The Amazon S3 profiles can be edited, removed or you can add more under the menu Home – Configure – Global Settings – S3 Buckets.
Configure the Zimbra HSM Plus Policy
To move the old items to the Amazon S3 secondary volume, you need to create an HSM Policy. In this example, I’ve selected all items, and only the ones older than 30 days will be moved to the HSM secondary volume, in this case an Amazon S3 bucket.
At the same time, I want to schedule the task to be automatically executed at 02am every night.
For example, I have a demo mailbox with items older than 30 days. Per our previous policy, all the items will be automatically moved to the Amazon S3 bucket. This operation will be invisible to the end-user: it will just take a few more seconds to load old items on the web client once he or she needs them. The items on the primary volume will be accessible faster, as usual.
Once the HSM process completes the task of moving all the old items matching the policy to the secondary volume or volumes, you can navigate through the Amazon AWS S3 Console bucket and to see the directory with the regular Zimbra folder structure with the user ID, etc.
If you navigate inside a userID folder, you will see the regular .msg files, size of the msg, etc., all inside Amazon S3.
And that’s it for this Blog post. We now have all the old items protected and secured on Amazon S3, and we have freed up the local storage which should be super fast. Now, with Zimbra HSM Plus, years and years of old items are moved from an expensive storage solution to the Amazon S3 Cloud.
Remember that you can download and try it free for 30-days: Zimbra Suite Plus here.
What is the behavior regarding Zimbra Backups for medium/large deployments ?
I whitch way it is compatible/supported with Zimbra backup ?
The Amazon S3 feature is only for HSM Plus at the moment, we are working internally to allow Backups to Amazon S3 too, but for now, it’s not supported, I wrote more about it long time ago:
In Zimbra NE and HSM, HSM is included in backups.
As it is possible to use HSM plus with Zimbra NE, I was asking if object in S3 are included in Zimbra backups and what is the impact for large deployments (as s3 is know to be slow).
Hi Jorge, I have a little question. Is possible to deny user to delete some email from his mailboxes? Actualy I need a solution for audit mail. I have a postfix with “always_bcc = audit@”, but this email is growing a lot on my default zimbra volume, I want to store all incoming/outgoing mail from users.
We have a solution on Zimbra Network Edition, where you can save this emails for audit purposes in cheaper disks, it’s called Archiving and Discovery – https://www.zimbra.com/email-server-software/email-archiving-discovery/
I think it fit with your needs, let us know.