Apache Hung, But Why?

Posting this here just in case others find themselves in the same situation.

Since migrating this site from my own silverorange-colocated server to Amazon AWS, I’ve been experiencing seemingly random Apache server outages. The usual suspects – mismatched Apache and MySQL settings, lack of memory, etc. – didn’t seem to apply, and the symptoms were confoundingly simple: all of Apache’s processes – up to the MaxClients settings limit – would be “hung,” and the server would be unreachable. But MySQL was fine, the system load was fine, and, as near as I could tell, everything was beautiful and sunny. Except that Apache was, effectively, hung.

After several weeks of Googling away for “Apache hung processes” and “Apache process limit” and “Apache crashed” and “mysterious and confounding Apache issue,” I launched into some more serious Apache debugging and eventually traced the issue to some variation of APC and this PHP issue. The debugging that got me there was running strace on any of the child Apache processes and seeing:

futex(0x2b19a0b01070, FUTEX_WAIT, 2, NULL)

Rather than trying to solve the APC issue itself, I solved the issue – fingers cross – by upgrading from Apache 2.2 to Apache 2.4 and from PHP 5.3.29 to PHP 5.5.20. Among other things, this let me jettison APC and use PHP OPcache instead. Which appears to have addressed the larger issue, and given me access to a nice OPcache visualization to boot:

PHP OpCache

It's like The Great Storm never happened...

It snowed. For two days. 86 cm worth of snow in Charlottetown at last report.

And then it was over. Today is a bright, sunny day. Schools are closed. Businesses are closed. There is a lot of snow. Everywhere.

However, by some miracle, the sidewalk plow passed in front of our house early this morning.

And our resourceful neighbour flagged down a snow-thrower-equipped tractor yesterday and paid him to blow out our driveway.

And the walk from home to office, although without sidewalks, is passable. And Receiver Coffee is open. And the side door of the Reinventorium was free of snow.

It’s almost as if the storm never happened.

I may be unique in this viewpoint.

Sidewalk Plow

Jetta Protests My Love

So yesterday I gave my Jetta the day off, in the humid and warm confines of the Delta Prince Edward underground parking lot.

Around supper time I retrieved the car and returning it to the frigid temperatures of the Charlottetown night. I drove home, parked for a few minutes, then drove across the Hillsborough River to Stratford for a couple of hours, then headed back home.

I noticed, while heading across the bridge the first time, an unusual high-pitched whine from the engine, and also noticed that my headlights didn’t appear as bright as they usually do. On the way back home I pulled into the Riverside Irving to clear my windows and when I tried to start the car back up it refused to start, showing classic signs of a dead battery.

I rang up brother Johnny and he heroically hopped in his car and came to my rescue; by the time he arrived I’d managed to start the Jetta back up, but it continued to act strangely.

And so I decided the best course of action was to hobble over to Dave’s Service Centre – you remember Dave’s, right? – and found, to my surprise and delight, that Dave himself was still in the shop. We had a chat and he deduced that it was likely an alternator failure at the root of the issue: the car had been running on battery alone, which was why it refused to start and why the lights were dim.

So I left the car with Dave, and am waiting to hear back this morning.

Moral of the story? Never give your car the day off.

Snow Day for the Jetta

After a slow start, the snow has come to Charlottetown in droves over the past several weeks. By some miracle, we’ve managed to keep our car shoveled out and driveable, in no small part due to the generosity of mysterious forces that have conspired to blow out the space between the road and the sidewalk that gets packed tight with snow from the plows (whoever you are: thank you).

Snow Car

Our car – now 15 years old – still starts on all but the coldest of cold mornings. But in the last storm the wind snow and rain and temperature combined to deposit solid blocks of ice in the space in front of the front doors, meaning they don’t open completely and thus require the driver to execute a complex limbo-like manoeuvre to enter the car.

To solve this issue, at least for now, I’ve given the car a day off: after running Oliver to school and Catherine to an appointment, I drove down to the Delta Prince Edward and left the car in the underground heated garage for the day. The Jetta breathed a satisfying sigh of relief as soon as it was hit by the warm underground air: I’m looking forward to returning in 8 hours time to pick up a refreshed car with openable doors.

Invisibilia and the Podcasting Renaissance

Podcasting, it seems, is undergoing a renaissance. And my favourite of the new crop is Invisibilia from NPR, a podcast (and, apparently, also a radio program) that “explores the intangible forces that shape human behavior.”

They are four episodes in now, and I’ve been fascinated by something – and usually more than one something – on every episode I’ve listened to.

Their description of the three schools of psychotherapy – who knew! – in The Secret History of Thoughts and the part of the Entanglement episode discussing how distant atoms can somehow at once be the same atom, are the highlights for me so far.

I encourage you to listen.

My podcast-listening application of choice (thank goodness “podcatcher” never took off as a noun) is Pocket Casts, which is available for both iOS and Android and here’s what you’ll find in my subscription list these days (the links are the podcast subscription links that you can add to your, er, podcatcher):

  • Planet Money - also from NPR, I enjoy their approach to money and the economy; it’s a sweet spot between lowbrow and highbrow.
  • 99% Invisible - design, architecture, curiousities.
  • Transom - seldom updated, but the back catalogue of independent documentaries is rock-solid.
  • Invisibilia - as above!
  • CANADALAND - hyperbolic, but interesting nonetheless.
  • Slate’s Working - where did it go? I hope it returns, as each episode’s focus on the quotidian aspects of a different profession was gripping.
  • Reply All - technology with a humanist bent.
  • Radiolab - arguable the grandfather of Invisibilia, and a pioneering user of what might be called “cinematic” techniques in radio.
  • Longform - a collection of interviews with writers about the craft (sort of like “Working,” but broader in scope but limited to a single profession).
  • Mark Kermode and Simon Mayo’s Film Reviews - I can’t imagine going anywhere else for film reviews; my longest-standing subscription.
  • StartUp - a podcast about starting a podcasting company; I love it.
  • TravelCommons - much less regular in recent years, but I keep listening because of the focus on the quotidian aspects of business travel.
  • This American Life - the classic.

It appears, as I write those thumbnails, that “quotidian” is my passion of choice.

Here’s an OPML file of all of the above that you can import into your podcast-listener if you want to slurp them all into your life too.