12/21/2023 7:57 AM
We sincerely apologize for the inconvenience caused by the incident surrounding the deployment of the December LMS365 release.
Root Cause: During the deployment of the new LMS365 release, code error caused the system to inaccurately update certificate statuses.
Resolution: The team identified and rolled out a fix for the issue across affected regions, which have resolved inaccuracies in certificate statuses and restored notification functionality.
* Continuous monitoring to ensure system stability and accuracy of the certificate status post-resolution.
* We have reviewed deployment processes to prevent similar issues in future releases.
We appreciate your understanding and patience as we work diligently to prevent any similar errors.
We would like to inform you that the fix was rolled out for the US Government Community Cloud region as well. We can confirm that the issue is now resolved.
We apologize for any inconvenience caused by this incident. If you encounter any issues or have additional concerns, please feel free to contact our support team.
We are pleased to inform you that the fix for the issue with certificate statuses has been successfully implemented in all regions except for US Government Community Cloud. As a result, the erroneous 'Expiring' statuses have been rectified, and certificates are now correctly displaying as 'Valid'.
Additionally, the notification functionality should now be operating properly.
We are actively monitoring the results of the fix to ensure stability and accuracy.
We appreciate your patience and understanding throughout this process. Should you encounter any issues or have further concerns, please don't hesitate to reach out to our support team. Thank you for your cooperation.
As we continue to address the ongoing issue, we are currently in the process of rolling out the LMS365 December release in the Australia and Japan regions.
As previously mentioned, these respective regions have not been affected by the incident other than experiencing delays in the delivery timeline.
During the deployment of the new LMS365 release, we encountered an issue related to updating certificate statuses. They were inaccurately set to 'Expiring' despite being intended to remain 'Valid'.
At present, we have pinpointed the root cause of this issue, and our team is actively working on a solution.
While this is being addressed, the email notification functionality may experience disruptions, templates for other types of notifications may not be sent (appointment invites, for example).
Additionally, we want to notify you that the deployment of the new release in Japan and Australia regions might face potential delays. However, rest assured, we are committed to making every effort to deliver the updates on schedule.
We sincerely apologize for any inconvenience this may have caused.
For current system status information about LMS365, check out our system status page. During an incident, you can also receive status updates by subscribing to updates available on our status page. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us.