Users were unable to view their existing visualizations in Visualize with AWS (Jira)
Incident Report for Utoolity Apps
Resolved
This incident has been resolved.

*Migration status*: If you have not edited any issue visualization between Feb 3rd and today, all visualizations should appear correctly again. If you have edited a single visualization on an issue between Feb 3rd and today, the visualization has been automatically migrated to the correct storage. In case you have tried to edit multiple visualizations on a single issue between Feb 3rd and today, please reach out to our support team for assistance with the required manual migration of visualizations to the correct storage.
Posted Apr 11, 2023 - 17:40 CEST
Monitoring
The app is fully operational again after Atlassian has rolled out the upstream fix for our app's production environment, refer to https://developer.status.atlassian.com/incidents/9l163ryc2wcw for details.

*Migration status*: If you have not edited any issue visualization between Feb 3rd and today, all visualizations should appear correctly again. If you have edited a single visualization on an issue between Feb 3rd and today, the visualization has been automatically migrated to the correct storage. In case you have tried to edit multiple visualizations on a single issue between Feb 3rd and today, please reach out to our support team for assistance with the required manual migration of visualizations to the correct storage.

We are monitoring the situation.
Posted Apr 11, 2023 - 14:07 CEST
Update
We have rolled out the migration code and asked Atlassian to roll out the upstream fix to ultimately address the underlying regression. We expect this to happen after their Easter break, refer to https://developer.status.atlassian.com/incidents/9l163ryc2wcw for details.

We will provide the next update by 2023-04-11 16:00 UTC, or whenever there is new information.
Posted Apr 10, 2023 - 16:32 CEST
Update
We have implemented and verified the migration code. We will now need to coordinate the rollout of the migration code and the upstream fix with Atlassian after the Easter break, refer to https://developer.status.atlassian.com/incidents/9l163ryc2wcw for details.

We will provide the next update by 2023-04-10 16:00 UTC.
Posted Apr 07, 2023 - 20:42 CEST
Update
Atlassian has implemented the fix for new apps, refer to https://developer.status.atlassian.com/incidents/9l163ryc2wcw for details. They have also enabled it for one of our development instances so that we can test the migration code.

We continue working on the migration code and will provide updates at least every twelve hours, or whenever there is new information.
Posted Apr 07, 2023 - 13:42 CEST
Update
Atlassian has meanwhile published a public incident themselves, refer to https://developer.status.atlassian.com/incidents/9l163ryc2wcw for details. Implementing a graceful migration turns out to be more complex and requires thorough testing, which is not possible until the upstream fix is rolled out.

We continue working on the migration code and will provide updates at least every twelve hours, or whenever there is new information.
Posted Apr 06, 2023 - 23:04 CEST
Update
Atlassian has prepared a rollout plan to revert the erroneous key format that caused this regression. The fix requires migration of data that has been erroneously persisted against the erroneous key though, which requires collaboration by affected partners.

We are working on the migration code and will provide updates at least every twelve hours, or whenever there is new information
Posted Apr 06, 2023 - 10:25 CEST
Update
We have received feedback from Atlassian that the team has identified the root cause and are now working on reverting the underlying change - there is no ETA yet.

We will provide updates at least every twelve hours, or whenever there is new information.
Posted Apr 05, 2023 - 17:08 CEST
Update
We have not yet received more detailed feedback from Atlassian.

We will provide updates at least every twelve hours, or whenever there is new information.
Posted Apr 05, 2023 - 14:21 CEST
Update
We have received feedback from Atlassian that the incident will be worked on by Forge engineers during their business hours in the APAC timezone and expect a more detailed update during that time.

We will provide updates at least every twelve hours, or whenever there is new information.
Posted Apr 05, 2023 - 00:31 CEST
Update
We have not yet received any feedback from Atlassian.

We will provide updates at least every twelve hours, or whenever there is new information.
Posted Apr 04, 2023 - 12:58 CEST
Update
We have not yet received any feedback from Atlassian.

We will provide updates at least every twelve hours, or whenever there is new information.
Posted Apr 04, 2023 - 02:49 CEST
Update
The Atlassian support team has reproduced the issue and escalated it internally.

We will provide updates at least every eight hours, or whenever there is new information.
Posted Apr 03, 2023 - 21:46 CEST
Identified
We have identified an upstream regression in a Forge API response format (changed ID). We assume there will not be any data loss for existing visualizations, however, you should refrain from creating new visualizations until this regression has been addressed.

We have escalated the issue to Atlassian support and will provide updates at least every eight hours, or whenever there is new information.
Posted Apr 03, 2023 - 18:26 CEST
Investigating
Users are unable to view their existing visualizations, and users cannot create multiple distinct new visualizations.

We are currently investigating this issue and will provide updates at least every eight hours or whenever there is new information.
Posted Apr 03, 2023 - 17:45 CEST
This incident affected: Utoolity Apps (Visualize with AWS (Jira)).