Exporting Short URLs from Bitly to Drupal

I’ve been using Bitly.com as a URL-shortening service for many years, mostly via IFTTT in a recipe that automatically tweets my blog posts here and post links to them on Facebook.

I took advantage of Bitly’s ability to configure a custom URL (they call it a “branded short domain”) to have my short links rendered under the l.ruk.ca subdomain rather than under bit.ly, something I did to allow me to repatriate the links under a system of my own control if something ever went wrong on Bitly’s end.

Recently I started to notice that my Bitly-hosted short links were showing up in a Google Search for site:ruk.ca and that made me uncomfortable, as, even though I’d been linking to whatever the destination of the link was at some time, I didn’t want the world, via Google, to think that the destinations were sites under my control.

I might post a link somewhere to the Republican Party, for example, and Bitly might give me a short URL like http://l.ruk.ca/1Y5vFet and then Google might index that and display it in search results. 

That felt creepy.

And so it was time to repatriate.

I found, to my surprise, that Bitly makes no provision, via its website, for exporting all the links I’ve shortened with it – I expected, but, alas, did not find a “export all my data” feature on bitly.com.

Fortunately Bitly does have an API that allowed me to do that export myself with a little coding.

I tracked down a handy Python script, py-bitly-exporter, to do the heavy lifting. I had to tweak the code a little to slow the export down, as I kept bumping into Bitly’s rate limit. To do this I inserted this in the head of the script:

from time import sleep

and this in the body of the script, inside the “while offset <= result_count:” loop:

sleep(0.3)

That slowed things down enough that I could export the 17,000+ links without hitting the limit.

The result was a file called links.csv that contained all my Bitly links, in comma-delimited form, like this:

http://l.ruk.ca/1OmlkZ7,"",http://ruk.ca/content/terms-union
http://l.ruk.ca/1QQHiWz,"",http://ruk.ca/content/five-islands
http://l.ruk.ca/1MQgW5h,"",http://ruk.ca/content/reinvented-vinyl

Once I had the links exported, I created a new Drupal site, installed the ShURLy module, and imported my exported data into the Drupal table shurly, putting in dummy values for fields like created and last_used, ending up with a table like this:

With that done and everything working as it should, I simply changed the DNS for the domain l.ruk.ca to point to my new Drupal site and, when I’m logged in, I can create new short URLs via a bookmarklet in my browser toolbar:

I’m missing all the Bitly razzmatazz with the migration — all the graphs and charts and other reports — but I never used the information anyway. The only thing I miss now is that Bitly support is burned into products like IFTTT and TweetDeck and these services don’t support roll-your-own link shorteners; so I’ll need to leave them to their own devices to shorten URLs as they like. At least the shortened URLs will no longer live under my domain.

To complete the process I added all manner of “no Google, not index this site nor the links on it” verbiage to the repatriated l.ruk.ca site so that, presumably, over time the links in Google’s search results will wither away and die.