Incident Report - Intermittent Delays 04/06/2024

05/06/2024 - 5:05pm - Incident Resolved 

A final set of fixes has been patched and performance is returning to normal. 

We're very sorry for the disruption caused to affected companies and appreciate your understanding as we work to improve our systems. Final details on this incident are as follows - 

Summary: A recent release that introduced new functionality for displaying and sending overtime records to PayHero caused performance issues in our production system. Despite extensive testing over the past two months, the issue occurred under production load conditions that were not identified during our testing phase.

Next Steps: Our team will conduct a full process review to ensure that similar issues are prevented in future releases.

05/06/2024 - 09:15am - An interim patch has been applied to further alleviate some of the issues. However, slow performance is still expected. Please allow each page to fully load before taking any further actions or navigating to other pages.

For companies with teams, it's essential that you filter to one team at a time before trying to navigate around the calendar or perform any actions.

If you're having any trouble approving or exporting timesheets for payroll, please contact support@droppah.com and we can provide an export of your timesheets.

Our team are continuing to work on a full resolution. 

04/06/2024 - 10:15am - We've temporarily increased the available processing resources for Droppah to help alleviate the loading issues while a permanent fix is developed. Some slowness is still expected, but errors should be reduced.

We expect to patch the fix for the underlying issues a little later today. 

04/06/2024 - 9:40am - There is currently an unexpected incident in Droppah, causing intermittent slowdowns, delays, errors, or other loading issues for some customers, especially when logging into Droppah or viewing the Timesheets.

We're urgently investigating to get our services back up and running at their normal operating speeds as soon as possible. Updates will follow here as we learn more.

Our apologies for the inconvenience this issue is causing.