Skip to main content

How would you identify ‘core’ departmental data?

Posted by: , Posted on: - Categories: Government

This has been a busy couple of weeks for the Transparency Team in the Cabinet Office. On Tuesday we had the announcement of an international open data charter which saw members of the G8 sign up to orient themselves towards open data by default. Prior to that on 14th June there was the launch of the Information Economy Industrial Strategy and the Government Response to the Shakespeare Review of Public Sector Information (PSI) where the Government made a series of commitments to further opening up data. Amongst other things these included consulting on releasing part of the VAT Register as open data. The Royal Mail also announced plans to improve access to the Postcode Address File and the Charity Commission said it would make the public register of charities freely available.


A National Information Infrastructure


In the Government Response to Stephan Shakespeare’s review, we set out how we’re going to embark on the next phase of releasing further data sets as open data. Central to this will be the identification of a National Information Infrastructure, where departments and users work together to identify the important data held by government. To support this process, the Transparency Team will be working with departments to identify all data held by each department through producing inventories (potentially like the one published by HMRC), scoring these to understand their importance and including relevant datasets for release in the final 2013 Open Government Partnership UK National Action Plan in October (a draft of this will be published shortly).


This is an ambitious timetable for a first draft of inventories and the National Information Infrastructure (NII) but it will build on work already undertaken through the catalogue of data on as well as published by government on sites such as and the UK National Statistics Publication Hub. However, this process will also highlight data which is held by Government but not yet published. As suggested in the Government Response to the Shakespeare Review this will need to be undertaken with others across Whitehall including those working on publishing data under Infrastructure for Spatial Information in the European Community (INSPIRE), with the Government Office for Science and Office for National Statistics.


Key to this as well will be the contribution of users external to Government in particular in terms of identifying those datasets which contribute to economic growth and social growth. We will be looking to work with stakeholders including the Open Data User Group,  Open Data Institute, the Open Knowledge Foundation and others on this as we want to get as wide a view as possible on this approach and the ways in which we are identifying key data sets. We will also look to develop further functionality on to support the development of the National Information Infrastructure.


Commenting on criteria


In the Government Response we set out some criteria we will be asking departments to use during this process and also committed to publish these criteria for comment. The criteria we set out were as follows:


Economic Growth

  • If open, could it stimulate growth in the UK economy?
  • Is it being requested by business?
  • Would it enable more efficient functioning of markets and reduce the cost of living for citizens?

Social growth

  • Is it requested by campaigning groups?
  • If open, would it help stimulate volunteering and self-help?
  • Could it aid in promoting social development and change?

Effective public services

  • Which data are fundamental to the operation of each Department?
  • Could it be used to hold government to account?
  • If open, could it aid the efficiency of public services and the running of government?
  • Could it aid the public in making choices about which public services to use?
  • Is the government the sole owner of this information, or is uniquely well placed to provide the data?

Connective reference data

  • If open would it aid in connecting and unlocking the potential of other data sets?

Other key data

  • Is it considered to have broad importance outside the above criteria?


So if you have comments on the criteria we’re suggesting departments use to score the data they hold, please leave your feedback in the comments section below. We would be keen to get your thoughts on how we should define these criteria in more detail as well as ways in which we can combine them into an overall score.


We also committed to blogging more regularly on the progress of the domestic transparency agenda and will keep you up to date on the work on the National Information Infrastructure through these.

Sharing and comments

Share this page


  1. Comment by william perrin posted on

    Hi Ed


    can i refer you to my blog post that started to set out an 'accountability stack' of core data for civil society


    It's a short piece but gives a rudimentary framework for considering these issues.

    In the UK some of the biggest and most embarrassing gaps are around open justice.  It's an aphorism that 'justice is seen to be done' but UK courts are almost invisible online.  as much as 95% of justice is dispensed in magistrates courts but it is impossible to get hold of basic datasets that contain magistrate court listings and results without being a paid up journalist.  and then journalists are often required to have highly stringent handling conditions of what is a long way from open data.  the same information is often pinned to a board in the court house itself in a C19th system that Dickens would recognise.

    strange pockets of practice exist elsewhere - coroners courts can be opaque.

    the Supreme Court and the high Courts garner a lot of attention but it is in the mags that justice is done but not seen

    All the above data is absolutely fundamental to civic discourse and the UK must be destined to score very low in any OGP exercise.

    BTW on this blog - why does it need a 'medium password' it's only a blog.  And the cookie thank you thing is gratuitous.  and then, depsite all that faffing you ask me to do a captcha.  FHS.

    • Replies to william perrin>

      Comment by edparkes posted on

      Hi Will

      These ideas sound like something we should consider raising with MoJ. Will have a look at the blog as well. Do you think that the criteria for NII as set out would mean you could make a case for these datasets?

      As for the DGU password issue I believe this level of protection was introduced because of concerns about spam postings.

      • Replies to edparkes>

        Comment by rahmad posted on

        William, thanks for your feedback on password strength - as another forum particpant has commented on we have put this in place as part of a collection of measures to reduce/remove SPAM postings to  This includes captacha, logging in to make comments and certain requirements for password strength.

        Whilst we will always review these approaches - its essential to have these things in place to make sure te expereince of visitors wanting to take part in commenting is not ruined by irrelavant content - we will of course re-review these processes from time to time - to make sure they are still effective and usable.

  2. Comment by psychemedia posted on

    To what extent might an analysis of FAQ requests reveal information about data sets requested from particular departments? For example, a search for FOI requests made via WhatDoTheyKnow that return data files provides a snapshot over FOIable released data that might contrast with data typically published via opendata routes? 

    • Replies to psychemedia>

      Comment by edparkes posted on

      Thanks for this. We'll take a look at the website as you suggest and consider idea.