Users were unable to access Visualize with AWS (Confluence)
Incident Report for Utoolity Apps
Resolved
We have identified and fixed the subsequent issue with the upgraded release not being visible for administrators. The upgrade is now visible from the 'Manage apps' interface and must be applied to address the original permission issue. New installations will use the new release automatically.

We are continuing to analyze a root cause and how to best prevent this type of incident in the future.
Posted May 25, 2021 - 21:42 CEST
Identified
We have identified the core problem and verified an interim workaround via our Forge staging environment. Once the workaround is deployed, the app must be upgraded by a site administrator. So while inconvenient, the workaround conceptually addresses the problem and has also been rolled out to the Forge production environment in turn.

Unfortunately the upgraded deployment is visible in the Forge production environment and all app installations are correctly listed as 'out-of date', but other than with the staging environment, the upgraded app does not become visible and installable for the site administrator, neither as an upgrade, nor via an explicit uninstall/reinstall cycle via the Marketplace.

We are continuing to investigate this subsequent issue and will provide updates at least every four hours or whenever there is new information.
Posted May 25, 2021 - 19:36 CEST
Update
The incident has been escalated as a critical outage due to blocking users from:

* viewing and editing existing visualizations - users are asked to 'Allow access' and can follow through, but clicking 'Accept' ultimately yields an unspecified 'Something went wrong' error
* creating new visualizations - users are presented with a read-only 'Allow access' macro view, but clicking 'Edit' ultimately yields a 'Failed to load' error that asks users grant access via the inaccessible 'Allow access' button in the macro

We are continuing to investigate this issue and will provide updates at least every four hours or whenever there is new information.
Posted May 25, 2021 - 17:10 CEST
Update
The incident has been escalated as a critical outage, we will follow up with details soon.

We are continuing to investigate this issue and will provide updates at least every four hours or whenever there is new information.
Posted May 25, 2021 - 17:09 CEST
Investigating
Users are encountering an 'Allow access' flow when trying to use the app, even though they had already granted access in the past and there has not been any app deployments since. In Jira, the subsequent permission details screen correctly shows the existing consent count, and clicking 'Accept' ultimately grants access. However, in Confluence the subsequent permission details screen incorrectly shows the existing consent count as 0, and clicking 'Accept' yields an unspecified 'Something went wrong' error.

We are currently investigating this issue and will provide updates at least every four hours or whenever there is new information.
Posted May 25, 2021 - 16:36 CEST
This incident affected: Utoolity Apps (Visualize with AWS (Confluence)).