Sticking with Buttondown

Last week, with great fanfare, I announced that I’d switched from using MailChimp to using Buttondown to power the ability to subscribe to a daily digest of my blog posts by email.

A week later, Buttondown suffered a technical meltdown, something that resulted in some of my 94 subscribers receiving 8 copies of that day’s email, and some—like me—not receiving it at all.

Needless to say, I was pissed off and frustrated, all the more so when the reply I received from Buttondown support was disappointingly “we’ll get back to you about this, but don’t hold your breath”:

Thank you for raising this issue about the duplicate emails going out. I have filed your experience on the Buttondown Roadmap page to be investigated and addressed. I have also associated your message to support with the ticket so that when it is resolved we can reach out directly to update you.

I was frustrated enough that I started to look elsewhere for alternative providers, to the point where I got pretty deep into setting up Moosend, a process that involved its own frustrations and limitations.

Today, before continuing down this road, I thought to look and see if Buttondown has a status page. It does. And on that page I read a very detailed blow-by-blow about what had happened, in part:

Bad configuration on one of our self-hosted SMTP servers caused a crash that proved difficult to recover from, leaving lots of emails “stuck” in varying degrees – and their being stuck manifested in a slew of unpleasant ways. We’ve fixed the configuration, are investing (literally, right at this very moment) in better tooling and alerting, and are architecting a way to prevent this from ever happening again.

and then, later on, in more detail:

Over the course of the afternoon, approximately 13,000 subscribers across 40 authors experienced some combination of the following:

• Multiple hour delays before receiving a message
• Not receiving an email at all (though we’ve redriven these.)
• Multiple sends of the same email

And that’s the fate that befell my subscribers and me.

Having managed servers professionally for more than 30 years, I am no stranger to confounding cascading technical meltdowns: some of my most challenging and frustrating hours have been spent in the wee hours, trying to diagnose some combination of web and database servers being overwhelmed. I was religious about telling my clients, when this sort of thing happened, exactly what happened and why; I treated that responsibility as seriously as my responsibility to actually fix the problem.

As I wrote last week:

I’d been reading the blog of Buttondown’s founder for awhile, and liked the cut of his jib.

Posts like this, with a photo of his office, and his bicycle, remind me that he’s a person like me, and he’s likely had a really shitty week.

This all combines to me unusually sympathetic to the Buttondown’s plight, and so I’m going to stick with them.