I use a modified Linksys WRT-54G wireless router as the heart of the Reinvented network here at HQ. It’s not an industrial strength router, but we’re not sending people to the Moon here, and it’s been pumping along just fine for 7 years.

Until last night at 8:00 p.m. when, for some unknown reason, it closed its pipe down to a trickle, effectively replicating dial-up bandwidth. The effects on the infrastructure here, which expects broadband, were somewhat chaotic, and as a result the mail stopped flowing, this website went offline, and a cascade of other less significant side-effects resulted from there.

The Day The Internet Stopped

It took some time to trace the problem back to the router, and all it took after that was a quick unplug/replug and the data started to flow strong and free once again.

With the exception of 178 Nagios messages in my inbox waiting for review – one of the weak points of the operation here is that the monitoring server is monitoring itself, and can’t send email out if something like this happens – all is back to normal.

Just another Saturday morning…


Dave Cairns's picture
Dave Cairns on November 14, 2010 - 03:17 Permalink

Peter ..

You running dd-wrt? If you are you may want to check your free nvram Later version of dd-wart store monthly traffic stats in nvram and after a number of months/years it can consume all of the free space.