Hardware issue blamed for wireless and Internet outage in Southeast
"Wireless and wireline service has been restored for all customers in parts of the Southeast affected by a hardware-related network issue," AT&T said in a statement late Tuesday. "Our engineers completed repairs and service is running normally. We apologize for any inconvenience."
AT&T would not describe the nature of the hardware problem, and said it only could speak about service for its own customers.
However, the outage was reported on social media and other sources to have affected thousands of customers for all the major carriers, lasting from about 4 p.m. to 8 p.m. ET Tuesday. It hit customers in parts of Tennessee, Kentucky, Indiana, Alabama, Georgia and Missouri, and possibly other states.
Buddy Rogers, a public information officer for the Kentucky Division of Emergency Management, said in an interview with Computerworld on Wednesday that the cause of the outage was not officially determined.
The cause was originally reported to his office on Tuesday as a fiber-optic cable cut, which could well have been in error, Rogers said. "I don't have a clue if it was a cable cut or a router outage or what," he said. Rogers' personal AT&T cellular service went down for a while, but workers nearby on other carriers continued to have service.
Two industry officials who asked not to be named told Computerworld that the source of the outage was an unexplained router failure and not a fiber cut that affected service for all of the carriers. In this outage, the router was shared or leased by all the major carriers for transmitting voice and Internet services, the officials said.
Rogers said that even though conventional wireless and Internet services might have been disrupted, there was still emergency backup for use by first responders in the region using a system called the Defense Military Affairs Radio Communications System. (DMARCS).
Analysts said it makes sense that an AT&T router would affect other carriers because AT&T is the predominant carrier in that region and it is commonplace for all the carriers to share equipment. A redundant network is the goal of all the U.S. carriers, but there are gaps in redundancy, especially in regions outside of major metropolitan areas.
"It's a difficult situation," said one industry official. "Where we have the opportunity to have redundancy in place, we build it in every point. Redundancy is absolutely a part of our DNA. Sometimes, in places like a mountain range, redundancy isn't possible to protect against all the things than can happen."
In absence of further information from AT&T or other carrier officials, it's impossible to understand precisely what happened, analysts said.
"What's surprising is that there wasn't redundancy built in," said Roger Entner, an analyst at Recon Analytics. "The only thing you can do [to protect against outages] is build in redundancy, and that's usually done in more populated areas of the country."
He added, "Reports indicate that the breakdown happened at a bottleneck and there are fewer and fewer of these nationally. There's really not anything you can do other than redundancy to defend against a cable cut, sabotage or other causes."