Rogers Communications CEO Tony Staffieri and new CTO Ron McKenzie faced the House of Commons industry committee today, to account for how its entire network failed on July 8 and what the company is doing about it. The giant telco is fighting for its reputation as it tries to convince regulators that letting it buy Shaw Communications will be good for the Canadian telecom market.

Here’s what you need to know:

‘This was truly an unprecedented incident’: McKenzie told MPs that the upgrade that broke Rogers’s system was the sixth in a series of seven that had been executed without previous problems.

He also said there was no way to anticipate it in testing, because the bug showed up when one set of equipment began sending information to another set from a different vendor.

Overwhelmed and helpless: Rogers described the problem in more detail in a submission to its main regulator, the Canadian Radio-television and Telecommunications Commission, on Friday. A configuration change that was part of the upgrade removed an internal filter, causing an “abnormally high” amount of internal traffic. “Certain network routing equipment became flooded, exceeded their memory and processing capacity and were then unable to route and process traffic, causing the common core network to shut down.”

Why it was so bad: Rogers’s “common core network” underlies all its services. When the brain fails, everything fails.

Rogers isn’t special: “What’s so important from this is [it’s] not just Rogers,” McKenzie said. “This core design of a common IP core is the consistent architecture used by operators all around the world. We all use the same equipment.”

What Rogers will do: Scrap the common core network design, Staffieri said. “To guard against a systemwide outage, we will set a higher standard by physically separating our wireless and internet networks and create an always-on network.”

What it will cost: “To be frank, this added layer of protection will be expensive. We estimate it will cost at least a quarter of a billion dollars. But we know it is the right thing to do,” Staffieri said. So, a lot.

And yet, not. In a public note released Sunday, Staffieri wrote that over the next three years, Rogers will spend $10 billion on reliability. That includes “more oversight, more testing and greater use of artificial intelligence,” he wrote. The company did not respond by deadline to questions about the part AI will play.

Rogers already planned to spend $2.8 billion to $3 billion on capital expenses this year, according to its most recent quarterly report. Staffieri told the committee that the share of Rogers’ capital spending going to its “network” is double what it used to be and will keep increasing.

The $250 million will go to “what I would describe as urgent fixes,” he said. 

Company spokespeople did not answer The Logic’s questions about how much of the $10 billion in spending wasn’t already planned three weeks ago.