Monthly Archives: May 2016

What’s changing in the open data ecosystem?

Last weekend was the 3rd Open Data Camp, in the great venue of the Bristol Watershed. Across the many sessions and discussions over the 2 days, there were some clear stories of what’s changing in the open data ecosystem, and some clear frustrations about what’s still needed.

Open Data Camp by Drawnalism
Open Data Camp by Drawnalism

The open data centre of gravity in government appears to be shifting towards Defra, at least to us observers outside government. A combination of top-level support from ministers and senior leadership is helping drive a big ramp-up in activity and data publication. At Open Data Camp there was a big turn-out from Defra and Environment Agency (although it was a bit of a home game for the Environment Agency with their Bristol HQ), and lots of discussion around data such as Lidar. With many of the current good examples of data use coming out of Defra, Environment Agency et al, next month’s Defra Open Data Market event will be a good event to take stock of how far we’ve come in opening up useful data.

There’s still a massive need for improvement in the “find, understand, use” part of the open data ecosystem. Data.gov.uk and other local open data systems are still essentially simple catalogues with only basic search tools – and have not really evolved in user-terms since open data catalogues such as our own Data4nr.net appeared in 2005. There’s little linkage between these data catalogues and “how the data has been used”, and little-to-no linkage with help on “how do I use this?”. There are some bright spots out there: Data USA  and the recently launched Data campfire are based around telling data stories, Nomis’s help forums are a truly useful source of expert help, and the Stack Exchange Open Data forum is interesting but needs more support and momentum (and perhaps a UK-specific version). I understand GDS are reviewing data.gov.uk, and it would also be good to see ONS impact in this area – the National Statistician role includes data dissemination across government, not just ONS data. If we’re serious about continuing to help users use data to improve services and businesses, it’s time we got serious about improving this part of the open data ecosystem.

It’s time to move on from asking “is open data valuable”? There are 100s of examples of open data proving its worth – from Census data (“2011 census benefits were £490 million each year”, Ian Cope ONS) to the Index of Multiple Deprivation being used to target upwards of £1billion resources per year to open transport APIs powering consumer travel apps to recent Lidar use (more on that below). Open data demonstrably provides value. Of course that doesn’t mean that every open data set is valuable – you can look at the usage statistics for data.gov.uk to see some of the less useful candidates (the CSV download at https://data.gov.uk/data/site-usage/dataset shows all datasets, and there’s a very long tail) – but can we please stop asking the “is open data valuable?” question.

Data use gets creative. For me the highlight session at Open Data Camp was John Murray’s step-by-step run through from raw Lidar height data to filtered building outlines.  The task that the Environment Agency set our Data Advisory Group in the first meeting was to prioritise which of their datasets they should release first. Lidar was absolute top of our list, and in meetings with the Lidar data team we listed roughly 50 uses for the dataset that helped make a bullet-proof case for publishing as open data – many of which we’re already seeing (although we missed the Roman roads … ). There’s a lesson here about the value of open data – although the Environment Agency EA no longer receives licensing fees from the (now) open data Lidar dataset, the return-on-investment to the Agency’s task and public realm is far more significant.

Open Data Camp was a great community-building event, very much down to the organisers for their hard work in putting it together and bringing in so many of the people doing great stuff in this field. I’m looking forward to the next.

Tom Smith is Chief Executive of OCSI and chair of the Environment Agency Data Advisory Group. @_datasmith and tom.smith@ocsi.co.uk.

People who have pitched, organising session times and locations
Horse trading after the pitching sessions
Photo of Owen Boswarva pitching a session
Owen Boswarva is real. And pitching a session.
Unleash your inner data hero - logo for Local Insight
Unleash your inner data hero
Turning LiDAR into actionable insight - first slide from John Murray's session
Turning LiDAR into actionable insight

The nuts and bolts of open data camp 3: a group review

New Doc 15_1

Feedback from day two: was mostly procedural (or what one delegate called ‘hygiene’ issues):

The goldfish bowl approach to debate?

Pro votes: 2. Anti votes: 1 Pro points: you know where the debate is coming from. Anti points: It’s against the spirit of an unconference; makes it hard to contribute; in principle you can signal that you want to take part or step out, but can you, actually?

Continue reading The nuts and bolts of open data camp 3: a group review

Open Data: the policy problem

Open Data: the policy problem

Owen Boswarva

There used to be a strategy board and an open data user group, and many other groups steering open data at the policy level. But most of these have now gone away. The one that seems to have survived in the Data Steering Group – but that has a wide range of interests – and we don’t know how interested they are in open data. Other groups seem to have evaporated. None of them have met since 2013/14.

Some sector boards still seem to be in effect. Should these surviving groups be steered from inside or outside government? There are some clearly missing. There’s a good pool of practitioners – but how do people outside the community find out about open data now? And how do we push for more release?

Continue reading Open Data: the policy problem

Open Data for Newbies

Open Data for n00bs

This session set out to create an open document (of course) full of resources and tools about open data. Leader Simon Redding said he wanted to find out where the gaps are; so they can be filled over time.

Other participants said such a resource would be useful for guiding communities towards data sources hey could use. This might address some of the issues surfaced during the expert Q&A session, which discussed concerns about the direction of the open data movement.

Continue reading Open Data for Newbies

Data standards: sampling chickens in an open data way

Fowl open data - food hygiene standards for chickens

Here’s an example of why open data standards are important: Campylobacter, which the biggest cause of food poisoning ilness amongst humans. It’s commonly found in chickens, and the Food Standards Agency is actively monitoring for it. So, how to create a useful set of data standards for it?

The standards we’re discussing are the generic ones that apply to lots of sampling. What could people suggest.

Cost? How is it determined? Specified for each dataset. For the chicken dataset, it’s the cost per entire chicken. Is it a sensible thing to have in?

Continue reading Data standards: sampling chickens in an open data way

Expert panel/Q&A

Building-the-perfect-FOI-request-DrawnalismNew-Doc-13_1.jpg

An expert panel session at Open Data Camp 3 turned out to be less of a Q&A on the minutiae of data sets and their use than a passionate debate about the direction that the open data movement is taking.

There was concern that after much excitement – even hype – about the impact that simply releasing data sets could have, disillusion was setting in as decision making processes remained unchanged and communities remained unaware of the information sources available to them – and the impact this could have. The debate concluded with some passionate calls to make it easier to uncover data and to make it less ‘scary’, so that more people could use it.

Continue reading Expert panel/Q&A

Hacking the hack: routes to community engagement with open data

Open data engagement beyond the hack day

Bristol City Council has an open data platform, and a team to work on release and engagement. They also have a mobility API for transport data. How, though, to get more people to use this lovely open data?

The already run hacks – they have one coming up on Saturday 21st May 2016. They also run round tables with community groups. They’re running a session with the University of West of England journalism course pairing journalists with coders to see what they can do. But they need more than hacks – they’re limited to people with a particular technical skill set.

Continue reading Hacking the hack: routes to community engagement with open data

How can we use better Lidar data analysis?

Analysing Lidar data for fun and business benefit

WARNING – liveblogging. Prone to error, inaccuracy and howling affronts to grammar and syntax. Posts will be improved over the next 48 hours

Google doc of session notes

Lidar is a technology that produces incredibly useful data that can be used to measure the environment. It’s a key part of self-driving cars – and the applications go much further than that.

Serge Wich is in the Amazon using Lidar on drones to aid orang-utan conservation – the software can actually tell orang-utans alongside apart from other species. But where Lidar has the most promise is in the built environment. 3D modelling with Lidar solves so many problems organisation have with building metrics.

Continue reading How can we use better Lidar data analysis?

Open data platform requirements

Open data platform requirements, Bristol, capture

What should an open data platform look like? This was the question addressed by a session triggered by Martin Howitt of ODI Devon in relation to Bristol’s plans to develop its platform.

“[There is] an open data platform that was brought in by the council [to release] data sets, but it wants to move on to next level: e.g. citizen sensing projects, citizen data sets, information from third sector and even private sector partners,” he said.

Continue reading Open data platform requirements