Headline Testing: Behind the Music

Yesterday I wrote about how the CBC is doing A/B testing on its news headlines.

Today I dug in a little deeper, and created this little hack, which scrapes the data from the CBC’s Chartbeat endpoint, and exposes the currently-running headline tests.

Here’s what it looks like at this hour, with “More French students than ever on P.E.I.” soundly beating “Enrolment at French schools in P.E.I. is at an all-time high”:

Screen shot of my CBC Headline Testing Experiment

There’s no magic to how I did this: it’s 26 lines of PHP and HTML:

<html>
<head>
<title>CBC Prince Edward Island Headline Testing</title>
</head>
<body>
<h1>CBC Prince Edward Island Headline Testing</h1>
<h4>This is an experiment, not affiliated with the CBC. <a href="https://ruk.ca/content/cbc-ab-testing-news">Read more here</a>.</h4>
<?php
$json = file_get_contents("http://mab.chartbeat.com/mab_strategy/headline_testing/get_strategy/?host=cbc.ca&domain=cbc.ca&path=/news/canada/prince-edward-island");
$headlines = json_decode($json);
if (property_exists($headlines, 'data') and property_exists($headlines->data, 'experiments')) {
  foreach ($headlines->data->experiments as $headline) {
    $location = $headline->location;
    preg_match('#\bhttps?://[^,\s()<>]+(?:\([\w\d]+\)|([^,[:punct:]\s]|/))#', $location, $match);
    $url = $match[0];
    print "<h2><a href=\"$url\">$url</a></h2>\n";
    foreach ($headline->variants as $variant) {
      print "<h3>" . $variant->content . "</h3>\n";
      $width = intval($variant->probability * 100) * 3;
      $percentage = intval($variant->probability * 100) . "%";
      print "<p>$percentage <img src=\"1by1green.png\" width=\"$width\" height=\"25\" align=\"absmiddle\"></p>\n";
    }
  }
}
?>
</body>

The headline testing JSON object for the CBC Prince Edward Island front page is always at the same URL; my script simply grabs that in real time, turns it into a PHP object, and then iterates through each variant for each test and displays the headlines and their current “probability” measure, both as a number and as a bar graph (drawn with a 1x1 pixel PNG, 1990s-style).

Feel free to use this code as the basis for exploring your local CBC’s headline testing operation.