Making a café react to its customers…

Those of you who have been reading since the Working for Free days will recall that presentation was anchored in this project for Casa Mia Café, an altruistically selfish attempt to ensure the success of the place that fed me coffee every morning.

Well, two years on and Casa Mia is still my coffee supplier, and I continue to use it as a playground for digital experiments.

My latest experiment started out with a strong aversion to Jesse Cook, an artist whose music, it seemed for awhile, was the only music Casa Mia ever played. I’m as much a lover of rhapsodic Canadian guitar as the next guy, but after 1,600 plays (there’s now evidence of this) I was ready to scream.

Which got me thinking: how could we make the café react automatically to the customers inside it at any point in time, and adjust the music playlist accordingly.

This week I’ve been working on an answer to this question. Here’s the working theory:

  1. Customer enters the café with a Bluetooth device in their pocket.
  2. The café’s music server, which is running a Bluetooth scanner, detects the presence of the device.
  3. If it’s a new device, never seen in the café before, a message is sent to it (at least for devices that have message-receiving capabilities) inviting the customer to register and associate their Last.fm account with their device’s address.
  4. If it’s a previously-seen device of someone who has registered, then the server finds their Last.fm address, and adds their musical tastes to the café’s playlist until they leave.

There are some technical bits that make this challenging.

The Bluetooth Bits

There’s not really such a thing as a “Bluetooth scanner,” at least not a friendly off-the-shelf application that you just install on a Mac. To the rescue comes LightBlue, a Python Bluetooth toolkit that lets me do neat stuff like:

# /usr/bin/python -Wignore
Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import lightblue
>>> lightblue.finddevices()
[('00:1F:5D:39:75:16', u'ruk', 5898764)]

Which tells me “somewhere near this computer is a Bluetooth device with a device address of 00:1F:5D:39:75:16 that calls itself ruk. This happens to be my Nokia N95 mobile phone, which is sitting beside me as I type.

It’s not hard to imagine, with LightBlue in hand, how the “detects the presence of the device” part of this system works, then: just set up a Python script to run in the background and to report new Bluetooth devices as they appear.

The other Bluetooth challenge is in the sending of a message of invitation to newly-detected devices. There are lots of devices that support this, but also lots that don’t. For example, there’s no way, as near as I can tell, to push a Bluetooth message to an Apple iPhone or iPod Touch. Of course for any device there’s the issue of Bluetooth being set off, or set to non-discoverable; can’t work around that, other than by letting customers who want to play know what they need to do.

The Music Bits

The music part of the system is a little more challenging: Last.fm isn’t really built for this sort of thing. Indeed once you start scratching into the Last.fm API, you realize that at every corner there’s another roadblock, part of Last.fm’s “we can’t actually let people listen to the exact track they want to listen to” licensing model.

For example, you might think a simple call to user.GetTopTracks for someone who’s just walked into the door would be a good place to start: just grab their favourite songs and add them to the café’s playlist using playlist.addTrack and then remove the tracks once they leave the café. Except that, alas, there’s no playlist.removeTrack method to allow the removal to happen. And the radio.tune method doesn’t actually support streaming playlists at all, only artists, users, and tags.

So, what to do? Well, my working model at this point is to variously dip into each present-user’s music by simply directly the Last.fm scrobbler to play Last.fm-style URLs like lastfm://user/reinvented/library (which, if you’re a Last.fm subscriber and have the scrobbler installed, should play my library for you if you click on it).

So, from a PHP script, I fade down the music server’s volume (it happens to be a Mac Mini, so I can do this using AppleScript):

function volumeFadeDown() {
      for ($volume = 7 ; $volume >= 0 ; $volume = $volume - 0.5) {
              system("osascript -e 'set volume $volume'");
       }
}

Then I “tune in” a given customer’s playlist:

system("osascript -e \"open location 
     \\\"lastfm://user/$username/library/\\\"\"");

Now that music is streaming, I wait some amount of time, likely sensitive to the number of playlists I’m juggling, and then segue to the next playlist.

Because it’s a stream, not tracks that I’m playing, the downside of this approach is that I’ll cut track off in mid-stream; I might be able to mitigate this somewhat by making calls to track.getInfo to get duration information for each currently-playing track, and then do the volume fade only once the end of the track was nearing.

None of this is particularly elegant, and it would be nice to have a more flexible track streaming service available, at cost, to allow to to either stream specific tracks or at least to add to and subtract from aggregate playlists.

I welcome any guidance on other approaches to this with Last.fm (or with alternatives to Last.fm).

Where I’m At

I’ve made good progress over the last few days with putting the pieces of this system together.

Casa Mia now has a Last.fm account, back-filled with 30,000 tracks worth of play data from iTunes. And because the Windows machine that’s playing music in the café is scrobbling everything it plays, we can now publish this information using the Last.fm API. So if you visit casamiacafe.ca, you’ll see the playlist in the right sidebar, and if you visit music.casamiacafe.ca, you’ll get a mobile-optimized version of the same.

There’s a headless Mac Mini sitting in Casa Mia running a modified version of haraldscan (a Pyhton script that’s built on LightBlue); it’s logging every Bluetooth device it sees, and we’ll let it run for a few days to get a sense of what sort of devices are walking into the café and what their capabilities are.

On the music side, I’ve cobbled together the PHP script outlined above, and have been running it here in the office with some dummy presence data fed into it to rotate through various playlists. The only dealbreaker I’ve encountered so far is the tendency of the Last.fm Mac scrobbler to stop working from time to time, reporting in a dialog box that it can’t stream; that’s difficult to deal with on a headless Mac where the dialog can’t be seen. So I’m looking at alternatives like lastfm-cli, a command-line client for Last.fm, that might allow me to stop using the GUI client altogether.

More than music…

There’s more to this than music, of course. I’ve already got the Bluetooth scanner set up to automatically check me in to Foursquare every time I stop by the café; there’s no reason why I couldn’t extend this to anyone who wanted to supply their Foursquare credentials.

Further on down the line, there’s all sorts of “automatically bring me a macchiato whenever I walk in the door” magic brewing around my head too.

As always, thoughts welcome.

Comments

Stephen pate's picture
Stephen pate on July 12, 2010 - 10:09 Permalink

Now if you could just program some customer service values into Casa Mia. I was there two years ago and couldn’t get passed the arrogance.

Peter Rukavina's picture
Peter Rukavina on July 12, 2010 - 13:29 Permalink

If you want to join in the testing, email me the Bluetooth Device Address of your mobile device (how you find this will vary from device to device; if you hang out near the back cash in the café for a few minutes the scanner will grab your address and I&#160can help connect you to it if all else fails).

Ryan Filsinger's picture
Ryan Filsinger on July 12, 2010 - 20:05 Permalink

This is a sweet idea. I usually don’t stick around casa mia for very long as I’m in to grab my coffee and then off to work. I do love the auto foursquare check in feature, now you just need to convince casa mia to give $0.25/off their coffee or something if you’re the mayor. :)

Let me know how to get that running with my nexus one. I’d love to be a guinea pig if you need some help.

Ton Zijlstra's picture
Ton Zijlstra on July 13, 2010 - 08:26 Permalink

Hi Peter,
You may also want to look into RoomWare (http://roomwareproject.org/), which is basically an open source server environment to connect sensors to (like BT) and generates XML streams of info for apps and actuators. It’s been used for Last.fm profile comparisons at parties etc.
In general it is meant as an easy way to help create apps in spaces, making rooms smarter. If you can’t use the entire thing, there may be routines in there that you can repurpose.

Peter Rukavina's picture
Peter Rukavina on July 13, 2010 - 14:19 Permalink

One issue that might be a deal-breaker, at least partially, is that it seems that “modern” devices like the Nexus One and iPhone do not support setting themselves as “discoverable” for Bluetooth except when you’re actually on the settings screen (in fact I haven’t been able to get the Mac Mini to detect an iPhone even when this is the case).


This is one example where old-school Nokia devices (and Blackberrys, for that matter) are superior.

WIlliam Schaub's picture
WIlliam Schaub on October 3, 2010 - 01:48 Permalink

You might get round the discoverable problem by using the l2ping command in Linux to ping a list of known device addresses. most devices respond to L2CAP ping requests without pairing and don’t need to be discoverable to respond to pings.

I’m working on some off the wall uses of Bluetooth myself and this is one method I’m currently looking into. I’m not sure yet how fast this method will be when searching though a long list of possible addresses though.

In the case of smart phones you should be able to create an app that can be started that will discover and talk to your server but depending on the size of the room not everyone may be within the 2 meter range of a class 2 bluetooth device

You could have more than one bluetooth device(attached to a computer) that is discoverable and advertising your service via SDP in strategic parts of the business that then reports back to your music server though.

William Schaub's picture
William Schaub on October 3, 2010 - 01:49 Permalink

Sorry that’s 10 meter range not 2