I first heard people talking about Z39.50 back in 1994 when I went to my first conference of librarians, Access ‘94 in St. John’s, Newfoundland.

At that conference, at at other Access conferences I attended in later years, there always seemed to be at least a few sessions about Z39.50.

Like “Z39.50 for finding it all” at Access 2002 in Windsor.

To me, a non-librarian, Z39.50 seemed like the darkest of dark library arts: a secret language that librarians whispered to their systems, out of earshot of we civilians. That it had such an arcane name only reinforced this.

Simply by osmosis over those years, I’ve managed to deduce that Z39.50 is a protocol for speaking to library catalogues: in the same way that our web browsers speak to our web servers using a protocol called HTTP, library clients can talk to library catalogue servers using Z39.50.

Z39.50 may be a dark art, but it’s an open dark art, and most automated library systems will respond to you if you talk to them using the protocol.

To speak Z39.50 you need a client – a program that will communicate via Z39.50 with a catalogue – along with the IP address and port number of the server.

The open source YAZ toolkit is a solid client, and it comes in a PHP flavour called PHPYAZ that’s a good place to start experimenting. If you have a Mac and Homebrew you can install it with:

brew install --universal yaz --without-icu4c

This will, among other things, leave you with a command line client named yaz-client.

Here in Prince Edward Island our Provincial Library System’s catalogue listens for Z39.50 on port 2200 on host address 24.224.240.217, so to start speaking to it with YAZ it’s:

yaz-client 24.224.240.217:2200

To which the catalogue responds:

Connecting...OK.
Sent initrequest.
Connection accepted by v3 target.
ID     : Unicorn 3.4.1 Bath
Name   : SIRSI Corporation
Version: 3.0
Options: search present delSet scan sort namedResultSets
Elapsed: 0.349298
Z> 

And then it sits there, waiting for a command.

So let’s go.

First, we’ll search the catalogue for records containing “Wade MacLauchlan” using the find command:

Z> find "Wade MacLauchlan"

The response is:

Sent searchRequest.
Received SearchResponse.
Search was a success.
Number of hits: 2, setno 1
records returned: 0
Elapsed: 0.062955

So there are two records in the catalogue that match; we can show them with the show command. We’ll ask for the second one:

Z> show 2
Sent presentRequest (2+1).
Records: 1
[DEFAULT]Record type: USmarc
(Length implementation at offset 22 should hold a number. Assuming 0)
02314 am a22005298i 4500
001 000042512248
003 CaOOAMICUS
005 20140328122422.0
008 140328s2014    pica     b    001 0beng  
016    $a 20149019998
020    $a 9780920434406 (bound)
040    $a CaOONL $b eng $e rda $c CaOONL
043    $a n-cn-pi
055  0 $a FC2625.1 C34 $b M33 2014
082 0  $a 971.7/04092 $2 23
090    $a PEI 921 CAM 2014
100 1  $a MacLauchlan, Wade, $e author.
245 10 $a Alex B. Campbell : $b the Prince Edward Island premier who rocked the cradle / $c H. Wade MacLauchlan.
264  1 $a Charlottetown, Prince Edward Island : $b Prince Edward Island Museum and Heritage Foundation, $c 2014.
300    $a 410 pages : $b illustrations ; $c 24 cm.
336    $a text $b txt $2 rdacontent
337    $a unmediated $b n $2 rdamedia
338    $a volume $b nc $2 rdacarrier
504    $a Includes bibliographical references and index.
600 10 $a Campbell, Alexander B., $d 1933-
651  5 $a Prince Edward Island $x Politics and government $y 1959-1979.
650  0 $a Premiers (Canada) $z Prince Edward Island $v Biography.
650  0 $a Politicians $z Prince Edward Island $v Biography.
710 2  $a Prince Edward Island Museum and Heritage Foundation, $e issuing body.
596    $a 1 3 4 5 6 7 9 10 16 17 19 22 24 25 26
926    $a ALB $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a BOR $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a BRE $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a CHA $b ANF $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a CHA $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 2
926    $a CHA $b PEICOLL $c PEI 921 CAM 2014 $d PEINCIRC $f 1
926    $a CHA $b CHECKEDOUT $c PEI 921 CAM 2014 $d PEIBOOK $e 5/7/2016 $f 2
926    $a COR $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a GEO $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a GSL $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a MON $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a MOR $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a MYH $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a SOU $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 2
926    $a STR $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a SUM $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a TIG $b CHECKEDOUT $c PEI 921 CAM 2014 $d PEIBOOK $e 21/6/2016 $f 1

This is a MARC record — another one of librarianship’s dark arts — that describes the book Alex B. Campbell : the Prince Edward Island premier who rocked the cradle. Each line of the record tells us something different about the book and copies of it. Lines starting 090, for example, are for Local Call Numbers and so the line:

090    $a PEI 921 CAM 2014

tells us that the call number for this book – what you’ll use to find it on the shelf — is PEI 921 CAM. And, sure enough, that’s what the online public catalogue tells us too.

The lines starting with 926 are about the holdings information for this book — what library branches it can be found in. So, for example, the line:

926    $a CHA $b CHECKEDOUT $c PEI 921 CAM 2014 $d PEIBOOK $e 5/7/2016 $f 2

tells us that there’s a copy that was checkout out (CHECKEDOUT) from the Confederation Centre Public Library in Charlottetown (CHA) that’s due back on July 5, 2016 (5/7/2016).

Dark arts though Z39.50 and and the MARC format be, they are well-documented, structured data, and that’s gold for someone who’s interested in automated interactions with a library catalogue.

Imagine, for example, that I’m visiting a friend and I see a copy of The Girl Who Saved the King of Sweden on their bookshelf. I take out my phone, running a mobile app that I might create for this purpose, and point it at the bar code on the back of the book:

The bar code scanner built into the app reads the bar code as as 9781443431590:

My app then speaks Z39.50 to the Provincial Library Catalogue, searching for the ISBN:

Z> find "9781443431590"
Sent searchRequest.
Received SearchResponse.
Search was a success.
Number of hits: 1, setno 1
records returned: 0
Elapsed: 0.090130

One record was found, so the app looks up the details:

Z> show 1
Sent presentRequest (1+1).
Records: 1
[DEFAULT]Record type: USmarc
(Length implementation at offset 22 should hold a number. Assuming 0)
02597 am a2200457Ii 4500
001 859586132
003 OCoLC
005 20140429084539.0
008 131007t20142013nyu           000 f eng d
020    $a 9780062329127
020    $a 006232912X
020    $a 9780007557905 (tr. pbk.)
020    $a 9781443431590 (tr. pbk.)
040    $a BTCTA $b eng $c BTCTA $d BDX $d WIM $d OCLCQ $d YDXCP $d CXP
041 1  $a eng $h swe
043    $a e-sw---
049    $a SPPA
090    $a F JON
100 1  $a Jonasson, Jonas, $d 1961-
245 14 $a The girl who saved the King of Sweden : $b a novel / $c Jonas Jonasson ; translated by Rachel Willson-Broyles.
246 1  $i Pre-publication title was given as: $a Illiterate Who Could Count.
250    $a First Edition.
260    $a New York : $b Ecco Press, $c c2014.
300    $a 387 pages ; $c 24 cm.
336    $a text $b txt $2 rdacontent.
337    $a unmediated $b n $2 rdamedia.
338    $a volume $b nc $2 rdacarrier.
520    $a "In a tiny shack in the largest township in South Africa, Nombeko Mayeki is born. Put to work at five years old and orphaned at ten, she quickly learns that the world expects nothing more from her than to die young, be it from drugs, alcohol, or just plain despair. But Nombeko has grander plans. She learns to read and write, and at just fifteen, using her cunning and fearlessness, she makes it out of Soweto with millions of smuggled diamonds in her possession. Then things take a turn for the worse... Nombeko ends up the prisoner of an incompetent engineer in a research facility working on South Africa's secret nuclear arsenal. Yet the unstoppable Nombeko pulls off a daring escape to Sweden, where she meets twins named Holger One and Holger Two, who are carrying out a mission to bring down the Swedish monarchy...by any means necessary. Nombeko's life ends up hopelessly intertwined with the lives of the twins, and when the twins arrange to kidnap the Swedish king and prime minister, it is up to our unlikely heroine to save the day--and possibly the world"--from publisher's web site.
650  0 $a South Africans $z Sweden $v Fiction.
650  0 $a Nuclear weapons $v Fiction.
650  0 $a Twins $v Fiction.
651  0 $a Sweden $x Kings and rulers $v Fiction.
700 1  $a Willson-Broyles, Rachel.
596    $a 1 6 7 16 24 25
926    $a CHA $b AF $c F JON $d BOOK $f 4
926    $a CHA $b CHECKEDOUT $c F JON $d BOOK $e 9/7/2016 $f 5
926    $a CHA $b AF $c F JON $d BOOK $f 6
926    $a COR $b AF $c F JON $d BOOK $f 1
926    $a MON $b CHECKEDOUT $c F JON $d BOOK $e 2/7/2016 $f 1
926    $a STR $b AF $c F JON $d BOOK $f 1
926    $a SUM $b AF $c F JON $d BOOK $f 1

nextResultSetPosition = 2
Elapsed: 0.401591

The app decodes the MARC record and deduces that there are two copies available at my local branch in Charlottetown:

926    $a CHA $b AF $c F JON $d BOOK $f 4
926    $a CHA $b AF $c F JON $d BOOK $f 6

At this point my app could open up a web browser to:

http://catalogue.library.pe.ca/uhtbin/cgisirsi.exe/x/0/0/5?searchdata1=9781443431590

which is a deep link to this book in the library catalogue. From this point I could place a hold on the book and then go to the library to pick up a copy:

Z39.50 is open data in its finest form, and our Provincial Library System’s catalogue has been speaking Z39.50 for many years. We owe a debt of gratitude to librarians past and present for breaking the trail.

After a delicious meal at the Red Door in Stanley Bridge we drove home through PEI National Park and stopped at the shore to watch the sun go down.

Here’s Catherine simultaneously talking on the phone to her nieces, receiving a birthday birdhouse, and blowing out the candle on her Chinese Birthday Bun.

I recorded this with my phone on approach to Charlottetown Airport last Saturday morning. I was flying in an Air Canada Beechcraft 1900D, a tiny 18 seat plane that’s used on the Halifax-Charlottetown runs. The sun was positioned perfectly for the plane to be able to chase its shadow home. The scene opens as we fly over the Atlantic Superstore and the University of Prince Edward Island.

If Ethan ever records an album, this will be the cover image.

Without any fanfare, the Province of Prince Edward Island launched a new website today, PrinceEdwardIsland.ca, and the most exciting aspect of the new site is that it telegraphs the Province’s plans for open data.

Under Online Services > Open Data, you’ll find a list of data sets.

Like this one, with data about the popularity of the various conservation license plates in recent years. There’s a link to a CSV file at the bottom of that page; open it in a spreadsheet and it looks like this:

It’s nice, clean, CSV, not just “File > Save As > Excel” data, and that makes it easy to quickly create charts like this, graphically illustrating the popularity of the plates for last year:

(I thought I was in the elite squad of Red Fox plate-owners; turns out I was just following the herd).

You won’t find all the data the province holds online – this is the first toe in the water – but it’s certainly a very good start: there’s everything from data about doctors to data about oysters. And the province’s license, principles and standards, modeled after the Government of Canada’s, appear sound. It’s hard not to love an effort that has non-discrimination as one of its core principles:

Non-discriminatory access to data should enable any person to access the data at any time without having to identify themself or provide any justification for doing so.

Wearing my Hacker in Residence hat, I spent much of last year talking to public servants about open data; although greater forces than I have conjured up the political will to make this happen, I’m happy to have a played a small nudging role in this effort. I’m excited to see what comes next, not only in terms of data sets, but in the gradual migration to an attitude in the public service where institutional street cred is derived from being more open, more often.

Back in the early 1990s, shortly after moving to Prince Edward Island, Catherine and I had our first occasion to take the Borden ferry to the mainland, as part of a trip to Halifax to visit friends.

Following the crowd from the vehicle decks up to the cafeteria, we spotted Roger Younker, then the host of CBC Compass, the supper-hour television news cast. We were (or perhaps, more accurately, I was) star struck: a God walked among us. And he was enjoying two eggs over easy.

As we became acclimated to Island life in subsequent years I came to realize that spotting Roger Younker on the ferry wasn’t as much of a deal as it initially seemed, but that impression of the cafeteria as the agora in which Islanders of all walks of life could break bread together before heading out into the cruel indignities of the mainland remained with me.

And when the ferry was replaced by the Confederation Bridge it was that absence I missed the most.

Last week I had occasion to chat with a colleague at The Old Farmer’s Almanac about his family’s vacation to the Maritimes last summer; he pulled out his mobile phone and showed me some of the photos he’d taken at Hopewell Rocks in New Brunswick, site of the highest tides in the world. I realized that, in the 23 years we’ve lived on the Island, we’ve never been across to see the Bay of Fundy, and upon returning home on Saturday this fact loomed large; I could have suggested a family trip later in the season, but I feared that, like the last 23 years, it would be pushed to the bottom of the pile and remain a dream unrealized.

And so after breakfast yesterday I suggested we take an impromptu trip to Hopewell Rocks. Right away.

Catherine and Oliver are nothing if not game for the impromptu trip, and so 15 minutes later we were on the road.

I suspect that I’m not unique in feeling, when driving toward the Bridge, that as soon as you reach the greater Crapaud-Tryon area you have escaped something of the gravitational pull of Charlottetown and attained escape velocity; a new sense of calm comes over the car at this point, as a collective sigh of relief is breathed.

Historically it was inside this breath – sometimes punctuated by an extended wait in the ferry line – that one would have breakfast with Roger Younker; on Sunday we replaced this with a visit to Scapes, the improbably-located restaurant on the fringe of Gateway Village, the simulated community at the lip of the bridge.

While the balance of Gateway Village is a simulacrum, Scapes is a very much real, delightful place to grab a bite to eat, owned and operated by Sarah Bennetto O’Brien, a passionate chef who cares deeply about local, healthy food.

We enjoyed a lunch of scratch-made fish cakes (for Catherine and me) and a BLT (for Oliver), served by friendly staff in a tiny pleasant room.

Sitting on a stool at the front of Scapes waiting for my fish cakes, I realized that it’s the new way to prepare for embarkation: the new culinary precipice from which we can gather ourselves to confront life onshore.

If Sarah could arrange for Roger Younker to drop in occasionaly, the scene would be perfect.

The more passionate among fans of The Old Farmer’s Almanac can sometimes be seen posing in front of the Almanac.com Webcam to create a souvenir of their drive through the Almanac’s home here in Dublin, New Hampshire. This morning I happened to be the first person in at the office – a victim of my body clock still being on Atlantic Daylight Time – and so I had my pick of parking spots and was able to park right in the middle of the camera’s field of view.

And so, here’s me, with my rental Toyota Yaris, at 6:59 a.m. this morning:

More than 12 hours later, after supper with colleagues, I popped back into the office to attend to some fiddly bits of advertising code, and so the camera captured me, again in an emptied parking lot, just pulling in:

I had the lucky chance to be able to share coffee with an insightful group of people last week, most of them dietitians, to talk about school food.

At one point in our conversation I admitted that I didn’t know what protein is.

Other than having a vague sense that I need protein to survive, and that I can get it by eating things like meat – he takes a moment to Google “sources of protein” – eggs, tofu, nuts, and beans, I had no idea why I might need protein to survive. Nor what eating more, or less, or better, or worse sources of protein would do to me.

Fortunately my dietitian friends greeted my admission of ignorance kindly; indeed one of them suggested that perhaps knowing about the ins and outs of protein might be a lot less important than simply understanding what eating “whole foods” is about.

But, regardless of their kindnesses, I decided I needed to learn about protein.

Fortunately I had, on my recent trip to London, come across the book Gut: The Inside Story of Our Body’s Most Underrated Organ in the London Review Book Shop and purchased a copy. While the book isn’t about protein specifically – it has far more to say about the mechanics of poop – it does helpfully provide a well-worded short course on the fundamentals of food. And it has this to say about protein:

Having examined carbohydrates and fats, there is just one more nutritional building block to consider. It is probably the least familiar: amino acids. It seems strange to imagine, but both tofu, with its neutral-to-nutty taste, and salty, savory meat are made up of lots of tiny acids. As with carbohydrates, these tiny building blocks are linked in chains. This is what gives them their different taste and a different name: proteins.

Digestive enzymes break down these chains in the small intestine and then the gut wall nabs the valuable components. There are twenty of these amino acids and an infinite variety of ways they can be linked to form proteins. We humans use them to build many substances, not the least of which is our DNA, the genetic material contained in every new cell we produce every day. The same is true of other living things, both plants and animals. That explains why everything that nature produces that we can eat contains proteins.

That’s exactly the concise explanation of what protein is that I needed: it allowed me to take the disconnected bits of protein trivia I’ve been floating through all my life and to start to bind them together.

It also makes eating protein sound pretty important to success in life.

This is only one of the wonderful things I learned from the book Gut. Beyond the aforementioned poop mechanics, I learned about fats and carbohydrates, about the immune system, about saliva, about what the appendix does. And I’m only halfway through the book. It’s entertainingly written, well illustrated, and, as with the treatise on protein, provides just the right amount of information to allow one to start making sense of how food and body work together to support life.

As the story of protein continues in the book, it gets even more interesting:

Plants construct different proteins than animals, and they often use so little of a given amino acid that the proteins they produce as known as incomplete. When our body tries to use these to make the amino acids it needs, it can continue to build the chain only until one of the amino acids runs out. Half-finished proteins are then simply broken down again, and we excrete the tiny acids in our urine or recycle them in our body.

After reading that paragraph I went from simple connecting-the-dots to a sudden and profound shift in my world view:

When we eat plants and animals we are simply taking the stuff they used to make themselves, and using it to make ourselves.

That may be self-evident to you, but this revelation has just completely rocked my world.

I’d simply never thought of plants and animals this way, and never connected the notions of eating, plants & animals, and protein all together in a systematic way.

Amino acid swapping. That’s what it’s all about. That is amazing.

I’m so glad I admitted I didn’t know what protein was, and that I set out to correct that.