Time Warner Blacks Out Chicago--Halo Gets Resolution

How the Internet Works

It has been an interesting morning.  Around 10:00am, our clients started calling us reporting that their websites were down.  However, a quick analysis proved that all web traffic for clients other than Time Warner customers was functioning as normal.  All websites were up and accounted for.

So, we called Time Warner, which in typical fashion was slow to acknowledge an issue. Team Halo rushed into action.  Our data center team in Chicago remotely logged in to workstations in the Louisville area and began running traceroutes.  We were able to conclusively prove that all web traffic was being incorrectly looped by Time Warner.

So, what happened? According to our data center techs:

"The issues this morning were due to a Tier 2 network incorrectly routing traffic from 10am until about 11:30am.

So, if you happened to be using an ISP (such as Insight/TWC) that used that specific Tier 2 network (RoadRunner) for handshaking/routing, you were basically blocked from all of Chicago, since the Tier 2 network wasn’t routing connections up to the Tier 1 networks properly."--see image above

Around 11:30am Time Warner officially confirmed what we already knew, that a routing loop on their end caused all traffic to be delayed.  All of our traceroutes that were dying showed it was traffic they were connecting through Road Runner.  Time Warner took this information and resolved the problem.

Thank you for reporting your issues.  We resolved them to the best of our ability regarding the national behemoths that caused the problem.  We apologize for any inconvenience this caused you.