Off to Scotland we go

It will come as no surprise to readers of this blog to hear that the Office for National Statistics is heavily involved with working in and around large amounts of data, which is why we are proud to once again sponsor Open Data Camp.

As we get ready for the trip to Aberdeen, ONS Chief Publishing Officer Andrew Dudfield, explains how ONS Digital are working to make more data more open.

The organisation is, in part, formed around a large data production process. We initially collect information. This has been traditionally collected via surveys filled in by individuals and business and is over time being augmented via a wider range of other data sources that help us better understand the world we are measuring.

Once we have the data, we need to store and process it. Again, this is an area of immense change. Not so long back, this would be in processing paper forms so that we could apply statistical methods to the results. We are now utilising a wide range of large scale data storage solutions to ensure the widest range of well organised, well labelled, consistent data is available to our teams of experts to work with and offering them a wider range of technical tools than ever before to unlock the insight within this. That insight is, in part, what is then published as statistics.

The publishing of these statistics is also transforming. We are trying to move from a world of inconsistent excel documents published on the pages of our website, to forming an openly available layer of data that anyone can easily access and use to answer a wide range of questions.

Within that, the word ‘open’ is important. We want to publish data that is open for as many people as possible to use. The use cases for this are near endless, but could include automated fact checking of statements, using the data to augment the responses given via voice controlled interfaces, producing automatically updating charts containing the latest data we publish and so on and so on.

We have started this journey by producing some Beta functionality on our website and via a new API. These are the thin edge of a much wider project and one we want to evolve in line with the needs of the users of this information. This, in part, is why I and a handful of my colleagues are heading to Aberdeen in November to attend the 6th Open Data Camp. This unconference has been touring the UK for the last few years, drawing together a group of likeminded folks who are interested in the importance and utility of the kind of open information the ONS is producing. We have sponsored the conference on several occasions and are pleased to do so again as we believe those attending and the communities of practice they represent are a vital audience in helping us deliver the best digital and data services possible.

Over the conference weekend we will be looking to learn from and with a range of professionals from a wide variety of backgrounds and will be happy to discuss what we are working on right now and share some thoughts on where we may go in the future. As we continue the transformation I described above, we will be continuing to hire digital and data staff across our three office locations (London, Newport and Titchfield) and I would be very happy to chat to people about some of the amazing roles we are hiring for as well

Looking forward to it.