Life in the Hurricane

2:00 a.m. — Finish up work for the night, mostly making small last-minute changes to the system that will tally the votes in today’s election. When I finish, I figure, being up, I might as well go out and see what’s happening outside, so I take a brief drive around the city: not much to see other than driving rain and curious onlookers.

2:30 a.m. — Return home and turn on CBC Radio to see if there’s any news from Halifax; find out that they’ve been hit pretty hard over there — ferry terminal roof gone, bridges closed, apartments evacuated. Catherine gets up with an awful barometric headache, and decides to sleep in Oliver’s bed just in case a tree falls on him. I try and get to sleep.

3:00 a.m. — Wind really picks up. I tune into the CBC again and hear that the “eye” is due to pass over Charlottetown in about 20 minutes. Andrew Sprague is on the phone to CBC Halifax talking about the situation on the ground here; right in the middle of his piece the power goes out and he disappears for a couple of seconds until the CBC’s backup power kicks in.

3:15 a.m. — Wind whistling through the front door so loud that it’s impossible to sleep.

3:45 a.m. — I finally manage to get to sleep, and sleep pretty solidly through the night (or at least what remains of it).

8:45 a.m. — Catherine and Oliver wake me up and I’m not sure where I am and what’s happened. Look out the back window and see a large tree in Angus and Karen’s back yard is down, and then that the tree in our back corner is split in half. Leaves all over the place. We look out the front window and see that the last remaining elm on Richmond Street is looking well-pruned. Power is out.

8:49 a.m. — Small discussion with Catherine about whether it’s possible to have a shower when the power is out. Given the day to come, I decide to risk it. Shower goes well.

9:00 a.m. — Although natural inclination is to jump right into things, I decide to eat breakfast: a bowl of cereal with blueberries and a glass of juice. Catherine digs out Eunice Reid’s old black rotary dial phone — the modern cordless phones don’t, of course, work without power. Brother Steve phones on his cell while on his walk to work at the CBC in Montreal.

9:10 a.m. — Meet up with Karen Rose, our new neighbour, surveying the damage next door. Find out from her that Maritime Electric expects power to be out for the day (Karen has the inside line on this, being married to Angus Orford, Maritime Electric’s spokesman. Karen says Angus has been up all night). See Norma Palmer, from Elections PEI, walking up the street on a mission.

9:15 a.m. — Decide to walk over to Elections PEI to see if they can use any help. Run into Gary on the way, and he tells me about Connaught Square (aka Jail Square) losing most of its trees, and the marina being creamed.

9:20 a.m. — Arrive at Elections PEI and find Merrill sitting in the dark, talking on the phone. Review the situation, ask him if there’s anything I can do, as I’ve got no power and thus no relevance otherwise. Norma arrives back, soaked. It starts to rain.

9:25 a.m. — Merrill sends me to deliver a message on foot to Bobby Macmillan and Richard Brown’s headquarters because they’re not answering their phones: the Civic Centre has been declared dangerous, and three polls in District 12 have been moved to Holland College.

9:30 a.m. — Torrential rain continues. I stop by the house to get a rain coat. Walk down Richmond Street to Richard’s office and find a tree across the street and power lines down. Jonah Deacon, Richard’s campaign manager, talks me through and I give him the message.

9:35 a.m. — Walk up Hillsborough St. to Bobby Macmillan’s headquarters and deliver the message to one of his people — she already knew, of course. Trees down all over the place, and many streets blocked off (which doesn’t appear to stop people from driving down them, around the barricades).

9:45 a.m. — Back home to change my pants — soaked in the rain. Realize that I would never make it in a life at sea. Catherine and Oliver okay; Emily has arrived for a previously scheduled babysitting appointment; no stopping that woman!

9:50 a.m. — Back to Elections PEI to find only Barb in the office and a sign on the door saying they’ve moved to the Confederation Centre. Barb tells me that the Centre has emergency power, and the phones have been forwarded.

9:55 a.m. — Over to the Confederation Centre. Find it oddly pleasant to be somewhere with power. Find Merrill and George in the Lecture Theatre on the phone. Review the situation: decide there’s no point trying to get the website working, as nobody will have power anyway.

10:05 a.m. — Elections Central is starting to get calls from electors looking to find where to vote. Normally the website would make this an easy civic address lookup away, but the website is down with no power. Realize that I have the raw materials to rig something up, especially as Confederation Centre has working Internet access.

10:10 a.m. — Return home to retrieve my laptop, keyboard, and CD containing PEI civic address data.

10:25 a.m. — Set up at the Confederation Centre. Get connected to the Internet. Download and install MySQL for my laptop. Dump the civic address data into a MySQL database, and rig up a small command line application to allow a civic address to be entered and a district and poll to be returned. Start answering questions that come in on the phones.

10:30 a.m. — Talk to Kevin O’Brien on the phone and find out that ISN, which is scheduled to provide one half of the Internet feed for election night, actually has power, and that Kevin’s prepared to move mountains to get things working. Gordon Johnston, who’s handling the networking for event, arrives, and says he thinks he can patch together things to the point where we can get online, and run the back-end systems, if we can get a generator to 180 Richmond St., where the servers are housed.

10:45 a.m. — Merrill okays the generator, and George Macdonald, who has been seconded from Transportation and Public Works for the election period, makes a call to get a generator. It’s easy to forget that Government has things like generators available at times like this.

10:55 a.m. — Talk to Kevin O’Brien on the phone again. He offers to power up my own servers, here in the basement of 100 Prince St., using the generator built in to his Road Trek.

11:05 a.m. — Realize that DNS, as usual, is going to be the Achilles heel of any effort to get elections results online to the public: the primary servers are in my basement, without power, and the secondary servers are in the Government complex, without power. Start to look for DNS alternatives, and find that it’s impossible to move DNS servers without 24 to 48 hours notice. Contact my domain registrar in Edmonton to see if there’s any way around this.

11:15 a.m. — As a precautionary move, I sign up for DNS service from several third-party DNS providers.

11:25 a.m. — Call Kevin back: if he can get power to my basement, I can get my DNS servers up and running, and we should be back in business. Arrange to meet Kevin at my house.

11:27 a.m. — Hear from Gordon that the generator’s in place at 180 Richmond, and that power has been restored to the Elections PEI servers. Gordon’s also arranged to borrow power from CBC’s generator in the basement of Province House to let our dedicated fiber circuit to the Confederation Centre be lit, giving us the ability to update results from the Centre.

11:45 a.m. — Meet Kevin at my house. He goes off to gas up his van, and Catherine and I dig out the extension cords we’ve accumulated over the years (you can never have too much phone cord, to many extension cords, or too many lengths of garden hose, in my experience). Kevin returns, powers up his generator, and my basement has power. And, oddly enough, Internet service on the cable modem.

11:50 a.m. — The main Reinvented server is complaining about disk problems on reboot. Spend about 30 minutes in the basement, in the dark, with Kevin, holding down the ‘Y’ key as RedHat Linux fixes the disk problems. Reminds both Kevin and I of the times we used to spent in the basement of the Confederation Centre, back in the day, when ISN’s servers were based there.

12:10 p.m. — Disk still being fixed. Kevin takes off in my car to put out more fires.

12:45 p.m. — Merrill calls: they need my address looking up services. Leave Catherine in front of the server, and head back to Elections PEI. Start answering “where do I vote?” questions. Continue to work on the DNS problem.

1:25 p.m. — Get a call from Trinic, my domain registrar in Edmonton: they confirm that changing DNS servers requires action by CIRA in Ottawa, which can take up to 24 hours.

1:45 p.m. — The generator at 180 Richmond dies. I walk over to find Gordon looking exhausted after trying to restart it. George arrives and tries to get it started. No go. George calls for another generator.

2:00 p.m. — Gordon goes off to vote, and I meet with Merrill and Lowell and we decide to move the servers from 180 Richmond into the Confederation Centre where they can run the back-end results tabluation system; we’ll leave the Internet out of the picture until we’ve got that end of the system working.

2:05 p.m. — On a lark, I try to start the generator myself: it starts on the first pull. We have power! Take Merrill’s desk lamp and hook it up in the vault so we have enough light to navigate the servers out.

2:15 p.m. — Lowell and I wrestle the servers out of the second floor of the vault at 180 Richmond (lots of buildings in Charlottetown have vaults: how many have two floors!) and wheel them over to the Lecture Theatre.

2:35 p.m. — Servers up and running in the Lecture Theatre. Set up a small private network, and get my laptop, the IBM laptop that will be printing results, and the servers all talking to each other. Find that the laptop is missing drivers for the printer, but that Charlie Mackay, ever resourceful manager of Election Central, has brought the driver CD. Install the drivers. Print. Find that the printer (a leased HP LaserJet) will only print if you press and hold a big button on the front, which makes using it on Election Night inconvenient. Merrill volunteers to go for a walk and retrieve the CD for his LaserJet, which is in place as the backup printer.

3:00 p.m. — Gordon arrives and announces that he can get us on the Internet by patching the ISN connection at 180 Richmond St. through to the Lecture Theatre with a big piece of fiber and some borrowed generator power in Province House (where the CBC has a huge rented generator to power their televsion and radio setup there). Realize the DNS issue might make this a vain effort, but we forge on.

3:05 p.m. — New printer drivers installed on the laptop: Merrill’s printer prints like a dream.

3:10 p.m. — Food, liberated by Charlie from the CBC, arrives. Fears that the CBC pasta salad may be tainted arise. We’ve already started eating it. Sigh.

4:00 p.m. — Connection to ISN is in place. Wow! Talk to Catherine on the phone: my servers are alive and kicking. Try to hit my servers from the Confed Centre, but no deal. Walk over to 100 Prince to find myself without Internet access. Sigh again. Ride my bike back to the Confederation Centre.

4:05 p.m. — Talk to Kevin on the phone. He promises to do whatever he ca to get Eastlink to restore cable modem service. We talk about the possibility of moving my servers up to ISN, but realize that we can’t route my IP addresses, which are really Eastlink IP addresses, up to Kevin’s server room on Kent St. without help from Eastlink, which appears to be in short supply.

4:25 p.m. — People who will be answering the phones at Elections Central arrive for training, and place starts to hum. Carol Murphy and Teressa Richards, who will be running the results printing operation for the night, arrive, and we briefly run over things. Realize that one of the best decisions I ever participated in when I was working with Government was to hire Teressa and Carol.

4:35 p.m. — Run through some tests with the results tabulation system, and things look like they’re all working. The results system is a web-based application that runs on a pair of Linux servers. Results come in over the phone from Returning Officers, and are written down on sheets of paper, one sheet per poll. The sheets are then walked over to my station, where they’re entered into the web application, and stored in a MySQL database on the servers. Carol and Teressa’s workstation then uses another part of the application to print sheets, with changes highlighted since the last print, and these are then photocopied, given to the media, and finally, as a backup, entered into a Lotus spreadsheet. Leaving aside Lotus, it’s a completely open-source election!

5:15 p.m. — More testing.

6:00 p.m. — Calm before the storm: phone people have gone for dinner, along with Carol and Teressa. Grab another little bite to eat (more tainted salad). Merrill rides my bike around the room.

6:25 p.m. — Phone people have come back. Arrange to have Charles, Kevin’s technical man, set up electionspei.isn.net as a backup domain name. Gordon gets the domain name spread to the media, and talks to Nick Grant, who runs the Government webservers, about getting a link online once the Government computer room, which is still without power, comes back online.

6:45 p.m. — Gordon connects his laptop to the Confederation Centre’s Internet connection, and finds he can hit the public results page. So we’re live to the Internet, albeit in a slightly hidden out of the way corner.

7:00 p.m. — Polls close.

7:09 p.m. — First call comes in. Data entry. Everything works! First poll printed and to the media.

7:11 p.m. — Lowell volunteers to sit with me and read numbers and names to make the data entry go faster; this proves an invaluable help.

7:25 p.m. — The results are coming fast and furious now. Only one glitch encountered: somehow the names of a couple of the split polls — where one too-large poll is split alphabetically in half — got missed in the proofing process, so I manually update the database to change a couple of instances of “(A-L)” to “(A-M)”.

7:45 p.m. — In a trance-like data entry state. Numbers, numbers, numbers. Realize that if I try to interpret any of the numbers, I’ll lose focus, so even though every result is flowing through my fingers, I have no idea what’s actually happening with the election.

8:41 p.m. — A small lull. Lowell and I flip the browser over to the public page and see we’re at 323 of 336 polls, and the Liberals are leading in 5 seats, the NDP in 0, and the PCs in 22.

8:45 p.m. — Lull over.

9:05 p.m. — About 10 polls still to come in. Lowell and Merrill get on the phone to chase down the last results. Sea changes slightly: 4 Liberals, 23 PCs.

9:23 p.m. — Last poll arrives. 336 of 336 reporting.

9:25 p.m. — Take a brief break as the Lotus data entry is finished up. Take a call from the production man at The Guardian: we’re providing them with a data file of the results, and he’s anxious to know when it will hit their FTP site. Tell him it should be 20 minutes.

9:45 p.m. — Compare all the results for all polls in the web system with the numbers entered into the Lotus spreadsheets to make sure everything matches. Everything’s okay. Prepare the file for the Guardian. Realize the connection to ISN has gone down, most likely because they’ve shut off the generator at Province House. End up generating the Guardian file on one network, disconnecting, hooking up to the Confederation Centre Internet feed, and FTPing the file. Try to call the Guardian, but nobody’s home. Try another number. And another. Finally someone answers, and we confirm the upload.

10:10 p.m. — Realize we’ll need to update the www.gov.pe.ca site, which came back online sometime during the evening, with results, because the local Elections PEI servers will be offline tonight.

10:15 p.m. — Copy the relevant files to my laptop, and shut down Edward and Wallis, the Elections PEI servers. They’ve performed admirably and need a rest. Brush them down, feed them some oats.

10:20 p.m. — Realize I can’t actually update www.gov.pe.ca from the Lecture Theatre, because I’m outside the newly installed firewall there. Backup plan: go back to 180 Richmond and use the feed there. Lowell and Merrill and Gordon and I are the only ones left at Election Central. We pack up the gear and wheel it over to 180 Richmond. The guard at the back door tears a strip out of me for taking my bicycle into the Centre. I take a picture of Merrill and Lowell for posterity.

10:30 p.m. — Back at 180 Richmond, we find ourselves without a network. I hear Gordon’s car door slam, and manage to catch him before he leaves. Find out that the problem is as simple as plugging back in some things in the vault that were unplugged earlier in the day.

10:40 p.m. — Some fancy footwork in BBEdit to transform the web pages into shape for www.gov.pe.ca. Upload the files. Couple more changes. We’re done.

10:45 p.m. — Head home, laptop, rain coat, keyboard, and bicycle in hand. Catherine’s in one piece — still has a headache, alas — and Oliver is fast asleep. Find out the Brackley Drive-in was destroyed in the hurricane, along with the roof of the Charlottetown Driving Park, and most of the Charlottetown marina. No Internet at home still, so no excuse not to go to sleep.

10:55 p.m. — Getting ready for bed, I realize that I’ve missed most of the media coverage of both the hurricane and the election.

11:15 p.m. — Sleep, wonderful sleep.

Comments

Chris Corrigan's picture
Chris Corrigan on October 1, 2003 - 18:09

Peter:

Hell of a job. It still boggles my mind that the USA, the bastion of democracy cannot run a fair election, while a bunch of savvy islanders with some spare garden hose and fiber optic cable can run one, tabulate the results and have it all said and done by midnight ON THE DAY OF A HURRICAINE.

Fantastsic stuff.

Rob Paterson's picture
Rob Paterson on October 3, 2003 - 19:52

Way to go Peter!

Add new comment