Current Filter: Network>>>>>Feature>

   

The recent outages suffered by household name websites like eBay were somewhat unusual. It's quite rare to see internet giants falling offline. They're internet giants for a reason, and they know exactly what downtime means: damaged reputations, frustrated customers and lost revenues.

eBay actually seems to suffer more than most, this being its tenth major outage this year. But you only had to look at the remarkable headlines proclaiming that the internet was full to know that this time something was different.

So what happened? In short, a lot of routers ran out of memory when the number of routes on the internet briefly exceeded 512,000.

Routers direct internet traffic by working out where to send packets on the basis of information held in lookup tables. These tables contain lists of prefixes, which are groups of IP addresses located in the same place. The number of these prefixes has been steadily growing since TCP/IP was first deployed in the early 80s. Earlier this year Cisco issued a warning that many routers will not cope with more than 512k prefixes unless they are upgraded, and it was predicted that we would reach that limit by October.

We got a taste of the consequences of failing to upgrade those routers several months early when a large American network provider accidentally split one large group of IP addresses in to 15,000 smaller groups for a period of about 10 minutes, temporarily exceeding the limit. Those 10 minutes caused chaos.

IPv4 exhaustion is contributing to this problem because as IPv4 addresses become increasingly scarce they are handed out in smaller and smaller groups, meaning even more prefixes. The glacially slow take-up of IPv6 means we are speeding towards this limit and a repeat of this issue.

The solution is fairly simple: network providers need to upgrade their routers. If this doesn't happen then this problem will reoccur, with widespread internet blackouts as the inevitable result. And next time we exceed the limit there might not be a quick fix to bring us back below the 512k threshold, prolonging the darkness and causing serious damage to businesses.

The problem is that for some reason even simple solutions are not always implemented in time by networking companies, even when they are given plenty of notice. It will be interesting to see what happens now and this issue has already moved from urgent to critical. We have even been afforded a brief taste of the looming disaster - but will the industry break the habit of a lifetime and apply the solution in time? It might not be a cheap solution but it's surely far less costly than an indefinite version of the issues we experienced recently.

The first warnings of this problem came a few months before we saw the effects, and we now know that plenty of networks are not ready. In the case of IPv4 exhaustion we have had more than a few months' notice, we've had years - more than two decades in fact. And we’ve had a solution for nearly as long. So the networking industry is totally on top of that issue at least, right? Wrong. Less than 15% of the top 1000 websites are available over IPv6.

Companies are at the mercy of network providers, but the solution is not entirely out of their hands. You can bet that eBay is talking to its network providers right now to get reassurances that this problem will not happen again. Smaller companies without eBay's muscle should ask their network provider about their level of preparedness, and seek assurances that their routers are not going to shut down when the routing table exceeds this limit of 512,000. NC