When Really Big Networks Fail

Updated: August 20, 2012

Recent network crashes at RIM (Research in Motoion Ltd.) and Rackspace US Inc . have come on the heels of the undersea fiber-optic-cable disruptions that cut off 70 percent of the Internet traffic between Europe and Asia, according to The Economic Times . These network failures, which happened very close together, have caused many to wonder about the security of the global Internet connection, as well as the vulnerabilities of enterprise-based networks that give users access to that connection. Is it all as fragile as it appears?

The short answer is: It's complicated. The undersea fiber-optic-cable disruption revealed a critical bottleneck for worldwide connectivity, but not all international connections that rely on undersea cables are as vulnerable. For example, 14 undersea cables connect the U.S. with Spain, France, the U.K. and Germany.

Fail-Safe?

Theoretically, this level of redundancy would make a disruption like the 70 percent loss between Europe and Asia unlikely — if not impossible — but complicatedly, it doesn't. In December 2006, an earthquake severed seven of the eight cables that connect Taiwan to the rest of Southeast Asia, proving that even good redundancy may not be enough to protect the security of the global connection.

Furthermore, even though most American Internet traffic remained unaffected by the cable disruptions in the Mediterranean Sea and the Persian Gulf, the event did cut 50 percent of the outflow from India to the U.S., revealing a critical network security flaw in India's outsourcing industry .

All of India's important fiber lines go underwater into the Arabian Sea at Mumbai, including all the traffic from Bangalore, India's outsourcing capital. From Mumbai, data can travel along three fiber cables that go west through the Red Sea and the Suez Canal into the Mediterranean Sea, or along three cables that go east, around Sri Lanka and into the web of fiber cables that connect Southeast Asia.

All of Bangalore's fiber-optic cables come from Mumbai, and no fiber lines run east from Bangalore to the Bay of Bengal, despite the fact that Bangalore is equidistant between the Bay of Bengal and the Arabian Sea. With India sending half of all its outsourcing traffic from Mumbai through the Suez Canal bottleneck, American companies might reconsider outsourcing jobs there if the reward of cheap labor becomes less than the risk of losing fiber connection due to dragging anchors or scuba-diving jihadis.

"A ubiquitous Internet requires ubiquitous connectivity, and we are doing quite well in this arena," wrote Scott Bradner, Network World columnist and the technology security officer for Harvard University. "What we do not have in too many places is the real redundancy required for high reliability."

In short, redundancy equals reliability, and the undersea-cable crisis of late January 2008 demonstrated that the critical Suez Canal bottleneck lacked such redundancy.

What Is Being Done?

New cable construction is already in the works, and it should help to prevent such outages in the future, according to TeleGeography , a company that maintains databases and publishes reports on subjects ranging from wireless carrier competition to global Internet backbone traffic. In the next few years alone, four new cables are planned to connect Europe and Egypt, a weak point in the global fiber web . These additional lines were on the drawing board long before two cables were cut in this exact area in late January 2008, proving that experts have been aware of the lack-of-redundancy problem well before the casual observer was.

The BlackBerry outage is a different story entirely.

RIM , the maker of the BlackBerry operating system, is all too familiar with network crashes, since the early February 2008 outage was the company's fourth in less than a year, according to Sky News.

The biggest problem with the latest crash was the timing. During a network upgrade on RIM servers on Monday, Feb. 11 — on the eve of the Presidential primary elections in Virginia, Maryland and Washington, D.C. — every political operative and journalist lost BlackBerry service around 3:30 p.m. EST. How did RIM notify its customers of the outage? By email, which they presumably received at 6:30 p.m. EST, when the network went back online.

RIM's public-relations response to the outage seemed identical to the playbook it followed after its April 2007 crash, which it handled by pretending nothing happened.

"It took RIM a long time to fess up that it was poor planning on its part that caused the problem …" Bradner wrote in his Network World column on April 30, 2007 . "RIM still does not have any explanation on its Web page of what happened … If RIM had been quicker to … [provide] more detail maybe more people would believe the company."

Ten months later, when the BlackBerry network crashed, RIM again shut the press out, including the Chicago Tribune, despite "repeated requests for comment" from the newspaper.

In another replay of April 2007, no mention of the outage can be found in the press section of RIM's Web site . To look at the company's press releases, no one would know that anything important happened between Feb. 7, 2008, when RIM's co-CEO announced he would speak at an investor conference, and Feb. 13, 2008, when the "ultrasleek BlackBerry Pearl" was unveiled in Hong Kong.

This desire to deny bad news instead of working to get out in front of the story does little to inspire a baseline of confidence. Networks are only as secure as the people running them, and what is denial but a manifestation of insecurity?