I use a modified Linksys WRT-54G wireless router as the heart of the Reinvented network here at HQ. It’s not an industrial strength router, but we’re not sending people to the Moon here, and it’s been pumping along just fine for 7 years.

Until last night at 8:00 p.m. when, for some unknown reason, it closed its pipe down to a trickle, effectively replicating dial-up bandwidth. The effects on the infrastructure here, which expects broadband, were somewhat chaotic, and as a result the mail stopped flowing, this website went offline, and a cascade of other less significant side-effects resulted from there.

It took some time to trace the problem back to the router, and all it took after that was a quick unplug/replug and the data started to flow strong and free once again.

With the exception of 178 Nagios messages in my inbox waiting for review – one of the weak points of the operation here is that the monitoring server is monitoring itself, and can’t send email out if something like this happens – all is back to normal.

Just another Saturday morning…