What happened
On Thursday, August 15, 2019, Telus customers in British Columbia and Alberta lost access to telus.net email. Telus later said the cause was its data storage cloud provider, Dell EMC, which was repairing failed equipment when a flawed repair procedure accidentally took telus.net offline. By August 19, CBC News reported customers had been without reliable email for four days.
Telus said it had restored access to most inboxes by Friday, but performance problems followed as customers returned. Customers without access could use webmail on new servers to send and receive new messages, while Telus said it would take its time recovering old messages and restoring them to mailboxes, a complex and time-consuming process. A NAIT instructor told CBC the length of the outage was unprecedented for a company of Telus's size and suggested it meant mailboxes had to be restored.
Small businesses were hit hardest. A Vancouver notary said senders received no bounce-back, so he could not tell what was arriving, and that four calls to Telus produced no rerouting option. The owner of Mix The Bakery said he lost at least $2,000 and a catering job for the University of British Columbia, and was most bothered by Telus's lack of communication.
Why it happened
A repair procedure carried out by the storage vendor, Dell EMC, was flawed and took the telus.net email service offline.
Email storage appears to have sat with a single cloud provider, so one vendor mistake knocked out the whole service, as the NAIT instructor quoted by CBC argued.
Messages sent to affected addresses did not bounce, so senders and recipients could not tell what had been lost.
Customers described poor communication: a business owner said repeated calls and tweets did not produce a workaround.
The lesson
Outsourcing storage does not outsource accountability: if one vendor holds the only copy of customer mail, one repair mistake becomes your multi-day outage.
Aftermath
On Monday, August 19, Telus chief customer officer Tony Geheran said all active customers affected would receive bill credits within 48 hours, writing "We let our customers down." Telus said it was working to ensure an outage like this never happens again. One affected notary said his business would most likely switch email providers.
FOLLOW THE EVIDENCE