INC177794 - Digital Banking Outage - Eastern Datacenter Database connection failure
Incident Report for Central 1
Resolved
Digital banking services have remained stable throughout the day as all traffic has been successfully managed by our Vancouver data center.

The connectivity anomaly was resolved in our Toronto data center and Central 1 will moving back to an active/active connection tonight, April 2, at 1 a.m. PT (4 a.m. ET).

We will be working with our service providers on the root cause analysis with high urgency and provide a postmortem within the next two weeks.
Posted Apr 01, 2024 - 16:49 PDT
Monitoring
Central 1 has completed a failover of our TOHC data center traffic to our Vancouver Data Center (VAHC), restoring traffic and resolving the incident.

Eastern Digital Banking Clients may have experienced an online banking access outage between 11:20 to 11:50 a.m. PT (2:20 to 2:50 p.m. ET). Users would have received an error when trying to log into online banking if they reached our Toronto Data Center as that data center's connection to the database was degraded.

Central 1 is completing an investigation today on the point of failure to restore our active/active connectivity between the data centers.

Another update will be provided by 5 p.m. PT (8 p.m. ET).
Posted Apr 01, 2024 - 12:20 PDT
Identified
All Forge/MemberDirect clients, please be advised that our Eastern Datacenter in the Azure cloud (TOHC, ZCC) has lost connection to our Digital Banking Database, which will cause login to online banking to fail for all retail and/or Small business traffic flowing through the Eastern Datacenter.

Retail and Small business traffic that is flowing through our Western (VAHC) datacenter is unaffected. All Commercial banking traffic is unaffected as it all flows through our Western datacenter.

We are working to fail over all traffic to the Western datacentre to mitigate impact while we investigate the connection outage.
Posted Apr 01, 2024 - 11:48 PDT
This incident affected: Digital Banking Services and Incident Alerting.