Cluster-level certificate/credential rotation was the underlying cause here. The tasks service relies on these credentials to spin up resources in its own cluster. Various locations needed to be manually addressed/re-deployed with the updated certificate/credentials to ensure full operation of the tasks service and administration UI.
Posted Dec 29, 2021 - 09:18 MST
Resolved
New tasks and existing alike are fully operational
Posted Dec 29, 2021 - 09:16 MST
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Dec 28, 2021 - 19:24 MST
Investigating
Spinning up new tasks and task groups is currently down and we're investigating. We've determined the source of the issue, but still tracking all of the updates needed to make the service fully operational.
Posted Dec 28, 2021 - 16:46 MST
This incident affected: Platform Services (Tasks).