Config error at Boston-area hosting company takes down Reddit, others

02.07.2015
A Boston-area hosting provider briefly knocked several large services and websites dependent on Amazon and AWS offline on Tuesday night, thanks to a configuration error.

Somerville, Mass-based Axcelx said in a tweet that the problem was caused by a route leak, which is an issue that can cause a small service provider to advertise itself as a different and much, much larger one, attempting to route huge amounts of traffic across its routers and generally causing the traffic in question to grind to a halt.

+ ALSO ON NETWORK WORLD: How Cisco brings communications to disaster relief efforts + One third of enterprise iOS devices vulnerable to app, data hijacking attacks +

Reddit was briefly taken offline by the incident, as were a number of other sites and services. However, the configuration appears to have been fixed relatively quickly, and Axcelx apologized to anyone inconvenienced by the incident. The issue, the company said in subsequent Twitter posts, had to do with a misconfiguration on the part of both Axcelx and one of its ISPs, Hibernia Networks.

Network analytics firm Thousand Eyes, in an informative blog post on the incident, noted that Netflix, Yelp and Match were all affected, as were HipChat, Jobvite and even financial firms like Experian. According to Thousand Eyes, there were indicative changes in the behavior of connections to Amazon's servers, and Axcelx and Hibernia both appeared in the BGP paths, where they shouldn't have been.

Route leaks, as the products of human error, are not uncommon but their effects can be far-reaching. A similar incident in March saw an Indian ISP briefly shut down Google, and another in October 2014 resulted in significant downtime for Cloudflare. Just last month, a route leak at Telekom Malaysia caused widespread outages for large parts of Australia and New Zealand.

(www.networkworld.com)

Jon Gold