How to use ONS geography map outlines

This is a brief summary on creating interactive maps based on the Office for National Statistics’ (ONS) UK map outlines.


The ONS geography team helpfully provide open data for the UK administrative boundaries. I’d previously used the district outlines as an overlay for Google maps on a regional geography prototype, and I wanted to extract the outlines to create interactive charts using the data from various Statistical Bulletins.

The first stage was to find the outline data from 2011 on the Geoportal .(There may be a more straight-forward way to get this, but I couldn’t find it.) I searched for “LA district boundary” on the home page, and got 100 plus records, including the Local Authority district (GB) 2011 boundaries. There were two promising versions: Full, Extent and Generalised, Clipped.

Clicking the item opens the section, to display more options. It is possible to ‘Add To Map’, which displays the outlines in a separate window. This takes a little time to download so you need to be patient as the shape file is loaded into a new window (and there is no indication that anything is happening…) The Full, Extent version seems to contain boundaries that extend into the sea, eg the Bristol Channel and the Wash, whereas the Clipped version is neater.

Having found the correct version (Local authority district (GB) 2011 Boundaries (Generalised, Clipped)), there was the small matter of downloading the shape file. The Details section includes background information on the dataset, and includes a URL for the Distribution zip file:
[Note: revisiting the Open Geography portal, there is a ‘Download Products’ link at the top of the page. Under the ‘Boundaries’ section are a range of data files including Local_authority_district_(GB)_2011_Boundaries_(Generalised_Clipped).zip]

The zip contains a number of files, including the shape file and associated data files. The next step is to visit and process the shape file, in order to convert the area data into geoJSON. A trick here (thanks to @fryford), is to drag across ALL the data files so that the geoJSON that is generated contains the reference names and ONS area codes. This is key in order to match the areas to the data. Again, from experience, the ‘prevent shape removal’ checkbox should be ticked otherwise some area can be lost. Depending on the use, it is also reducing the coordination precision to reduce the file size. Once you have loaded the shapes files, simplifying the area polygons as needed and repair any intersections. Finally down load the geoJSON.

I used the Highcharts mapping library to display the outlines and for this, the two main steps were to convert the XLS spreadsheet data into JSON and then map the JSON data value with the ONS id in the geoJSON. The Statistical Bulletin used can be found here. After downloading the spreadsheet I extracted the data and created a JSON file in following format:

data = [
{ 'code':'E06000001', 'value':1.0},
{ 'code':'E06000002', 'value':-7.8},
{ 'code':'E06000003', 'value':-1.6}

The only other part was to create the Highchart series and map the data ‘code’
to the corresponding area in the geoJSON (LAD11CD):

series : [{
data : data,
mapData: geojson,
joinBy: ['LAD11CD', 'code'],

The final page showing the Internal Migration map is on Github, including the full code.


Must have Chrome Plugins for Developers

Prompted by Kenneth Auchenberg’s article about web development workflow, I’ve switched back to Chrome to try out the development tools.

First thing on the list is to install a short list of plugins (yes, I know that Chrome splits  them into extensions/apps/themes).

Purely for future reference, in no particular order:

I’ve started using Sublime Text 2 for a work project, so the next thing is to learn to use it more efficiently.

Still a lot to do, and to quote Paul Irish:


combine and minify css and js; inline @imports; inline small images as data-uri; remove console.log ;remove unused CSS; cache manifest creation; asset revving; dead code elimination; image compression; video/audio transcoding

Not to mention the tools, boilerplate, abstractions, frameworks, workflow, tuning, building and deployment!

Football Visualised

Following on with my previous work on a example visualisation of player data, I carried out some more research into the best colour schemes for charts. My initial design was heavily influenced on the current trend for Business Intelligence dashboards to have a dark background. Based on the work of Maureen Stone and Stephen Few, I inverted the colour scheme of the design to provide a neutral consistent canvas for the data.

Here’s the version with a black background and the same one with a light background.

croppedDesign croppedInverse

With any project, there is always a feeling that the design can be improved, but given the time constraints, I left the design work with the intention on revisiting it in the future (!).

The next step was to mark up the basic html page with a simple gridded design and then use Bootstrap css to speed up the process of styling the page. To start with I used actual content, with a plan to replace these metrics and values with placeholders. The final aim was to make the page completely dynamic so that for example, I could swap the summary data for a player based on broad roles, eg goal keeper, defender, midfield or forward.

Once the content had been defined, I modified the css to mimic the original design. I kept the css within the html page initially, purely due to the convenience of editing it. This is the html5 version. Note that the charts in the page are still image placeholders. Again the plan is use d3.js to load in the match data and process it to provide dynamic charts for each player.

Another aspects of the design work was to rough out some ideas of the structure / navigation pages for getting to the player page (with pencil and paper). At the moment the work is purely to play around with the technology and try some experiments in visualisation, but it also balanced with the eventual need to refine the data into a understandable/presentable form. (All stuff for the parts bin)

That said, the next part is to spend some more time playing around with d3 and the data. The plan is to build on the heatmap, and track a player’s position through a game and be able to see how they move around the pitch during a game.

Still no word on the #MCFCAnalytics, though I assume the January transfer window is the one of the busiest time of the year for football analysts…


Football Analytics

I stumbled on the MCFC Analytics project over the holidays, while looking for some sample data. I’ve signed up for a copy of the full data (and also the advanced XML – does that mean there is more data than the full set?) I’m waiting to hear back from them.

I did a quick scan of other data visualisations using the sample data. The Guardian Data Blog has a couple of interesting chalkboards and there appear to be a few blogs out that that I need to look into.

I also found the Opta developer project and signed up for that – apparently they are planning to make a different set of data available to developers this month, which sounds promising.

In the meantime, I pulled together some basic positioning data and had a play around with miso and Mike Bostock’s Data Driven Documents – d3.js as a way of learning how to use the technology. Here’s a first stab at a heatmap.

One of the areas that the MCFC Analytics project is interested in, is position specific metrics. With that in mind, I took a step back from the development work and put together a sample infographic design to use for player measurement from each game. It still very much work in progress at this point, and I imagine it will change significantly when I get my hands on the data to see what I can extract. One thing that springs to mind immediately is tracking players movements off the ball. This would be essential in calculating the distance a player travels during a match and also their speed.


Click the cropped image to see the full graphic.

One obvious extension piece would be to map out set pieces to show player’s positions at each event and provide an animated step through control to see how each event plays out.

Another idea is to build up the player profile through the season and track the player’s performance at home and away games.

Update your SDK

Note to self: when returning to a project (JQuery, Flash, Flex, Android, Wowza, Java, Blackberry or Haxe) it is worth spending a few minutes updating any existing SDKs rather than chasing down an obscure bug, only to find that the issue is due to a previous version of the SDK.

Also remember to keep the existing copy of the SDK so that you can go back and run any existing projects with their corresponding SDK versions.

FlashDevelop Haxe Class Not Found

I’m playing around with Haxe as part of my on-going search for a good cross platform development language. I recently got a “Class Not Found” error with FlashDevelop when I tried to add the Actuate library to my project.

I’d already installed the library via nme (haxelib install actuate) and I was able to import the code in FlashDevelop, but couldn’t build the swf. After much searching, I found some info from Joshua Granick

First include the library in the NMML project file:

<haxelib name="actuate" />

Then in FlashDevelop, go to Project > Properties, pick the Compiler Options tab and expand Libraries. Add the new library (“actuate”) on its own line. Adding it to your project file is all you need to compile properly, but FlashDevelop still needs you to add it in your project settings in order to get code completion.

Manually adding .apk package to Android

Once you have created the .apk package, use the Android Debug Bridge (adb) to transfer the package to your Android device for testing. The adb was previously found in the SDK in the tools directory. The adb is now located in platform-tools.

The command to transfer the .apk is:
adb install pathTo/yourPackageName.apk
After the .apk is installed for the first time you may get an error message when you try to install it again: Failure [INSTALL_FAILED_ALREADY_EXISTS].

The adb requires a flag to re-install the app:
adb install -r pathTo/yourPackageName.apk
Note that this keeps any existing app data.

To uninstall the app:
adb unistall yourPackageName.apk
More information can be found on the developer site ( and also via the command line adb help

Error: Resource entry main is already defined

I got this strange error after editing my main.xml file: “Error: Resource entry main is already defined”, with main.out.xml being created in the same folder.  A quick search then revealed that this is Eclipse, helpfully trying to transform the XML that I had edited. This is the default behaviour whenever the editor has a XML file open and in focus.

I tried to modify the Eclipse launch Configuration as follows:

Open Window>Preferences>Run/Debug>Launching>Launch Configuration.

Check the “Filter checked Launch configuration types:”

Then scroll through the list to “XSL” and check this box too, but the issue remained whenever the XML file had the focus.

The solution I used was to change the launch configuration by opening Window>Preferences>Run/Debug>Launching, and then checking the “Always launch the previously launched application” box.

Installing Eclipse for Android Development

I’ve set up Eclipse using the  Java Enterprise Edition(Java EE) with Java Development Kit 1.6 (JDK6 – I’m glad it is not just me losing track of what version is what) in order to play around with Live Wallpaper and native Android applications. The installation process was a breeze compared with my previous experience with the IDE.

Configuring it to use the Android Developer Tools (ADT) plug-in was also very quick and easy. The developer guide is really clear and easy to follow – nice work!

I’ve run through the samples and got some Paint on the Canvas, but I miss Flash’s smooth vector shapes 🙁 Next stop vector graphics…

But before that, I’ve installed the color theme, picked a nice dark theme (“inkpot”) and turned off the bold text.