Back to overview
Downtime

Azure service outage

Jan 10, 2025 at 3:19am UTC
Affected services
PROD PBD API
Test PBD API

Resolved
Jan 11, 2025 at 4:34am UTC

Azure regional network problems have been resolved.

Updating test API status to resolved.

Updated
Jan 10, 2025 at 8:59pm UTC

The Azure resources serving the Test API appear to be up and down. Changing status to degraded until Azure confirms East US2 is stable once more.

Updated
Jan 10, 2025 at 8:45pm UTC

The test API and supporting services / infrastructure have been restored.

Updated
Jan 10, 2025 at 6:23pm UTC

The test API will be restored once Azure has recovered in East US2.

At that time, the production API will be re-deployed to East US2.

Current Azure status can be found here.

Updated
Jan 10, 2025 at 3:22pm UTC

Production API has resumed normal operation.

Note that performance will likely still be affected until the primary region has recovered.

Updated
Jan 10, 2025 at 1:41pm UTC

All primary API services have been re-deployed to the backup region.

Some data will be out of sync temporarily (transactions, items, receipts). PBD is working to recover these services as well.

Updated
Jan 10, 2025 at 12:59pm UTC

Some back-end services are beginning to recover.

We do not have a specific ETA available at this time. More updates to follow.

Updated
Jan 10, 2025 at 3:51am UTC

Downtime related to Azure outage in some regions.

The API and supporting services are in the the process of re-deployment to our secondary region.

More updates to follow.

Created
Jan 10, 2025 at 3:19am UTC

PROD PBD API went down.