Whispering Sweetly to the Provincial Library Catalogue with Z39.50

I first heard people talking about Z39.50 back in 1994 when I went to my first conference of librarians, Access ‘94 in St. John’s, Newfoundland.

At that conference, at at other Access conferences I attended in later years, there always seemed to be at least a few sessions about Z39.50.

Like “Z39.50 for finding it all” at Access 2002 in Windsor.

To me, a non-librarian, Z39.50 seemed like the darkest of dark library arts: a secret language that librarians whispered to their systems, out of earshot of we civilians. That it had such an arcane name only reinforced this.

Simply by osmosis over those years, I’ve managed to deduce that Z39.50 is a protocol for speaking to library catalogues: in the same way that our web browsers speak to our web servers using a protocol called HTTP, library clients can talk to library catalogue servers using Z39.50.

Z39.50 may be a dark art, but it’s an open dark art, and most automated library systems will respond to you if you talk to them using the protocol.

To speak Z39.50 you need a client – a program that will communicate via Z39.50 with a catalogue – along with the IP address and port number of the server.

The open source YAZ toolkit is a solid client, and it comes in a PHP flavour called PHPYAZ that’s a good place to start experimenting. If you have a Mac and Homebrew you can install it with:

brew install --universal yaz --without-icu4c

This will, among other things, leave you with a command line client named yaz-client.

Here in Prince Edward Island our Provincial Library System’s catalogue listens for Z39.50 on port 2200 on host address 24.224.240.217, so to start speaking to it with YAZ it’s:

yaz-client 24.224.240.217:2200

To which the catalogue responds:

Connecting...OK.
Sent initrequest.
Connection accepted by v3 target.
ID     : Unicorn 3.4.1 Bath
Name   : SIRSI Corporation
Version: 3.0
Options: search present delSet scan sort namedResultSets
Elapsed: 0.349298
Z> 

And then it sits there, waiting for a command.

So let’s go.

First, we’ll search the catalogue for records containing “Wade MacLauchlan” using the find command:

Z> find "Wade MacLauchlan"

The response is:

Sent searchRequest.
Received SearchResponse.
Search was a success.
Number of hits: 2, setno 1
records returned: 0
Elapsed: 0.062955

So there are two records in the catalogue that match; we can show them with the show command. We’ll ask for the second one:

Z> show 2
Sent presentRequest (2+1).
Records: 1
[DEFAULT]Record type: USmarc
(Length implementation at offset 22 should hold a number. Assuming 0)
02314 am a22005298i 4500
001 000042512248
003 CaOOAMICUS
005 20140328122422.0
008 140328s2014    pica     b    001 0beng  
016    $a 20149019998
020    $a 9780920434406 (bound)
040    $a CaOONL $b eng $e rda $c CaOONL
043    $a n-cn-pi
055  0 $a FC2625.1 C34 $b M33 2014
082 0  $a 971.7/04092 $2 23
090    $a PEI 921 CAM 2014
100 1  $a MacLauchlan, Wade, $e author.
245 10 $a Alex B. Campbell : $b the Prince Edward Island premier who rocked the cradle / $c H. Wade MacLauchlan.
264  1 $a Charlottetown, Prince Edward Island : $b Prince Edward Island Museum and Heritage Foundation, $c 2014.
300    $a 410 pages : $b illustrations ; $c 24 cm.
336    $a text $b txt $2 rdacontent
337    $a unmediated $b n $2 rdamedia
338    $a volume $b nc $2 rdacarrier
504    $a Includes bibliographical references and index.
600 10 $a Campbell, Alexander B., $d 1933-
651  5 $a Prince Edward Island $x Politics and government $y 1959-1979.
650  0 $a Premiers (Canada) $z Prince Edward Island $v Biography.
650  0 $a Politicians $z Prince Edward Island $v Biography.
710 2  $a Prince Edward Island Museum and Heritage Foundation, $e issuing body.
596    $a 1 3 4 5 6 7 9 10 16 17 19 22 24 25 26
926    $a ALB $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a BOR $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a BRE $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a CHA $b ANF $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a CHA $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 2
926    $a CHA $b PEICOLL $c PEI 921 CAM 2014 $d PEINCIRC $f 1
926    $a CHA $b CHECKEDOUT $c PEI 921 CAM 2014 $d PEIBOOK $e 5/7/2016 $f 2
926    $a COR $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a GEO $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a GSL $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a MON $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a MOR $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a MYH $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a SOU $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 2
926    $a STR $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a SUM $b PEICOLL $c PEI 921 CAM 2014 $d PEIBOOK $f 1
926    $a TIG $b CHECKEDOUT $c PEI 921 CAM 2014 $d PEIBOOK $e 21/6/2016 $f 1

This is a MARC record — another one of librarianship’s dark arts — that describes the book Alex B. Campbell : the Prince Edward Island premier who rocked the cradle. Each line of the record tells us something different about the book and copies of it. Lines starting 090, for example, are for Local Call Numbers and so the line:

090    $a PEI 921 CAM 2014

tells us that the call number for this book – what you’ll use to find it on the shelf — is PEI 921 CAM. And, sure enough, that’s what the online public catalogue tells us too.

The lines starting with 926 are about the holdings information for this book — what library branches it can be found in. So, for example, the line:

926    $a CHA $b CHECKEDOUT $c PEI 921 CAM 2014 $d PEIBOOK $e 5/7/2016 $f 2

tells us that there’s a copy that was checkout out (CHECKEDOUT) from the Confederation Centre Public Library in Charlottetown (CHA) that’s due back on July 5, 2016 (5/7/2016).

Dark arts though Z39.50 and and the MARC format be, they are well-documented, structured data, and that’s gold for someone who’s interested in automated interactions with a library catalogue.

Imagine, for example, that I’m visiting a friend and I see a copy of The Girl Who Saved the King of Sweden on their bookshelf. I take out my phone, running a mobile app that I might create for this purpose, and point it at the bar code on the back of the book:

The bar code scanner built into the app reads the bar code as as 9781443431590:

My app then speaks Z39.50 to the Provincial Library Catalogue, searching for the ISBN:

Z> find "9781443431590"
Sent searchRequest.
Received SearchResponse.
Search was a success.
Number of hits: 1, setno 1
records returned: 0
Elapsed: 0.090130

One record was found, so the app looks up the details:

Z> show 1
Sent presentRequest (1+1).
Records: 1
[DEFAULT]Record type: USmarc
(Length implementation at offset 22 should hold a number. Assuming 0)
02597 am a2200457Ii 4500
001 859586132
003 OCoLC
005 20140429084539.0
008 131007t20142013nyu           000 f eng d
020    $a 9780062329127
020    $a 006232912X
020    $a 9780007557905 (tr. pbk.)
020    $a 9781443431590 (tr. pbk.)
040    $a BTCTA $b eng $c BTCTA $d BDX $d WIM $d OCLCQ $d YDXCP $d CXP
041 1  $a eng $h swe
043    $a e-sw---
049    $a SPPA
090    $a F JON
100 1  $a Jonasson, Jonas, $d 1961-
245 14 $a The girl who saved the King of Sweden : $b a novel / $c Jonas Jonasson ; translated by Rachel Willson-Broyles.
246 1  $i Pre-publication title was given as: $a Illiterate Who Could Count.
250    $a First Edition.
260    $a New York : $b Ecco Press, $c c2014.
300    $a 387 pages ; $c 24 cm.
336    $a text $b txt $2 rdacontent.
337    $a unmediated $b n $2 rdamedia.
338    $a volume $b nc $2 rdacarrier.
520    $a "In a tiny shack in the largest township in South Africa, Nombeko Mayeki is born. Put to work at five years old and orphaned at ten, she quickly learns that the world expects nothing more from her than to die young, be it from drugs, alcohol, or just plain despair. But Nombeko has grander plans. She learns to read and write, and at just fifteen, using her cunning and fearlessness, she makes it out of Soweto with millions of smuggled diamonds in her possession. Then things take a turn for the worse... Nombeko ends up the prisoner of an incompetent engineer in a research facility working on South Africa's secret nuclear arsenal. Yet the unstoppable Nombeko pulls off a daring escape to Sweden, where she meets twins named Holger One and Holger Two, who are carrying out a mission to bring down the Swedish monarchy...by any means necessary. Nombeko's life ends up hopelessly intertwined with the lives of the twins, and when the twins arrange to kidnap the Swedish king and prime minister, it is up to our unlikely heroine to save the day--and possibly the world"--from publisher's web site.
650  0 $a South Africans $z Sweden $v Fiction.
650  0 $a Nuclear weapons $v Fiction.
650  0 $a Twins $v Fiction.
651  0 $a Sweden $x Kings and rulers $v Fiction.
700 1  $a Willson-Broyles, Rachel.
596    $a 1 6 7 16 24 25
926    $a CHA $b AF $c F JON $d BOOK $f 4
926    $a CHA $b CHECKEDOUT $c F JON $d BOOK $e 9/7/2016 $f 5
926    $a CHA $b AF $c F JON $d BOOK $f 6
926    $a COR $b AF $c F JON $d BOOK $f 1
926    $a MON $b CHECKEDOUT $c F JON $d BOOK $e 2/7/2016 $f 1
926    $a STR $b AF $c F JON $d BOOK $f 1
926    $a SUM $b AF $c F JON $d BOOK $f 1

nextResultSetPosition = 2
Elapsed: 0.401591

The app decodes the MARC record and deduces that there are two copies available at my local branch in Charlottetown:

926    $a CHA $b AF $c F JON $d BOOK $f 4
926    $a CHA $b AF $c F JON $d BOOK $f 6

At this point my app could open up a web browser to:

http://catalogue.library.pe.ca/uhtbin/cgisirsi.exe/x/0/0/5?searchdata1=9781443431590

which is a deep link to this book in the library catalogue. From this point I could place a hold on the book and then go to the library to pick up a copy:

Z39.50 is open data in its finest form, and our Provincial Library System’s catalogue has been speaking Z39.50 for many years. We owe a debt of gratitude to librarians past and present for breaking the trail.