Update on June 2022 Zimbra Patch release

Hello Zimbra partners, customers, community and friends,

I would first like to apologize for the issues triggered by the recent patch release. I know that many people rely on the services offered by our products and when we do not execute well, the impact is felt by customers. While we take efforts to prevent problems, there are times when those efforts are not enough.

What Happened:
Upon releasing Zimbra 8.8.15 patch 32 and 9.0.0 patch 25, we recognized there was a defect which would affect a number of users. Our attempts remove the availability of the patches broke everyone’s ability to install the product or update to another patch. This happened to coincide with the release of information on a security issue covered in the previous patch release (8.8.15 patch 31 and 9.0.0 patch 24). Panic over the security issue drove a number of people to attempt to update, only to find out they could not.

Where things stand:
The patches for Zimbra 8.8.15 and 9.0.0 has been made available once again with the blocking issue resolved. The package repository issues have been corrected, fixing the issues with installations and upgrades. Everyone should now be able to install and upgrade without issue.

What we have learned:
Our quality assurance release gates failed to uncover the issues present in the patch release. While we have identified the specific gaps related to the patch release, we will be undertaking a full evaluation of our processes to ensure a better outcome in the future.
Our release process did not properly account for pulling back releases due to issues. This has been corrected and we are expanding our resources who have knowledge of the process for redundancy.

Moving forward:
I want to personally thank our partners, customers, community members and friends who worked closely with us during this incident. With your help, we were able to quickly identify the major issues and eventually resolve them. Without your support and interaction, things could have been much worse.We will be taking the lessons learned from this incident and implementing adjustments to our processes to ensure we do not repeat the mistakes made here.

We have also separated out our security and patch announcements from our email newsletter. We felt this would be a better way of highlighting security issues which could arise and bringing attention to them without the distraction of other content. We believe this will help you make better decisions around the content of patches and the availability of security releases. You can sign up for the announcements here.

Thank you.

John Eastman

9 Responses to Update on June 2022 Zimbra Patch release

  1. Jim Roland June 23, 2022 at 8:23 AM #

    John, I have to disagree with you that the broken packages have been fixed. I spent long hours last night with a collegue trying to fix mailboxd that would still not start. Rebooting the server yesterday yum on CentOS updated zimbra and after reboot, zimbra was broken. I found a link on a discussion forum about downgrading packages including zimbra-core-components, etc (4 packages in total), and only after downgrading to the previous version, did Zimbra start back up again.
    Please remove the current highest version out there. There is still a jetty/java issue.

    • Avatar photo
      Barry de Graaff June 23, 2022 at 11:45 AM #

      Thanks for your feedback, we are investigating to determine the issue.

    • Alex July 4, 2022 at 2:00 AM #

      Dear Jim,

      Please share a link on a discussion forum about downgrading packages including zimbra-core-components

      Thanks

    • Darwin July 6, 2022 at 7:45 AM #

      Hi Jim, how can I downgrade those packages?? Thanks

  2. funig June 26, 2022 at 1:34 AM #

    Please fix java jetty problem as soon as possible, which waste CPU resources high percentage.
    momentally exceeds 100% and calm down, this cycle repeats.
    Something wrong compared with Patch23.

    As the same as Jim, I downgraded 4 pakages including core/jetty/ldap/JDK) and as for me, zmmailboxd can run, but java’s PID suddenly active momentally(I confirmed ‘top’ command)

    Temporary you should Patch23’s era perfectly, please.

  3. gmelis July 7, 2022 at 12:25 PM #

    As a temporary patch that will allow zimbra to deliver messages, with the cost of “soft” restarts, you could add a line like this in crontab (mind you, do not run it more than 6-8 times per hour):

    /sbin/lsof -nPi | grep ‘:7025-.*CLOSE_WAIT’ | awk ‘{print $2, NR}’ | tail -1 | while read a b; do if [ $b -gt 18 ]; then kill -HUP $a; fi; sleep 20; done

  4. Avatar photo
    Barry de Graaff July 7, 2022 at 11:48 PM #

    Please continue the discussion on the forums: https://forums.zimbra.org/viewtopic.php?f=15&t=70865

  5. Cheryl McLain July 20, 2022 at 8:48 AM #

    How can I fix error message “OUT OF RPC CACHE” My product is 8.815_GA4173 Build 20211022125231 Thank you.

    • Avatar photo
      Barry de Graaff July 25, 2022 at 2:56 AM #

      This error is shown on Classic UI if too many or recursive requests are made. Try and disable all Zimlets and remove all shares from/to the account.

Copyright © 2022 Zimbra, Inc. All rights reserved.

All information contained in this blog is intended for informational purposes only. Synacor, Inc. is not responsible or liable in any manner for the use or misuse of any technical content provided herein. No specific or implied warranty is provided in association with the information or application of the information provided herein, including, but not limited to, use, misuse or distribution of such information by any user. The user assumes any and all risk pertaining to the use or distribution in any form of any subject matter contained in this blog.

Legal Information | Privacy Policy | Do Not Sell My Personal Information | CCPA Disclosures