The Home Office is consulting about proposals for elected Police and Crime Commissioners to hold UK police forces to account. The consultation paper includes commitments about local crime and policing data and includes questions about what data should be published, how and by which bodies. The Local Public Data Panel is an independent panel that exists to promote and facilitate the release of public data. The Panel has played a key role in providing advice on the publication of fine grained and timely local authority spend data. This paper sets out the Panel’s views in response to the Home Office consultation. It reflects the draft principles published by the Transparency Board and seeks to apply them in the context of police and crime data. At this stage, we are not seeking to set out in detail every piece of data that should be published under the new police accountability arrangements. However, we think it would be useful to set out some of the overarching principles that should apply.
Our key points are:
- Data and information about policing, public safety and criminal justice are essential in local and strategic oversight and accountability of police forces and the wider criminal justice system.
- We propose that a whole systems approach should be taken to the publication of data about crime and criminal justice.
- The presumption should be that all data will be published unless it is private or genuinely confidential. It will be important to ensure that the data covers the range of information that is necessary for effective transparency and accountability at both strategic and local level. We set out six categories of police data that should be published. In all of these categories, data should be published at the most granular level of detail possible.
- The question of how data should be released is as important as questions about what data should be included. If data is not released in a standardised, re-usable, machine-readable form without restrictions on re-use or re-purposing, then it cannot be used to its full potential and cannot be described as genuinely open data.
- The approach that should be taken to releasing data should reflect the progression from taking the first step of publishing data in whatever form to the goal of providing linked data. The best must not become the enemy of the good – progress should be made to the next stage towards linked data at the earliest opportunity. As a first stage in this process, the data to be published by January 2011 should be published up to at least the two-star standard outlined here.
- We recognise that there are risks involved. These must be managed in relation to specific categories of information, and the fact that there are risks involved should not generally act as a brake on progress across the board. The presumption should be that as much granular data as possible will be published as close to real time as possible, with exemptions being the exception rather than the rule.
- We will be pleased to work with early adopters to support them in becoming examplars and to support others to learn from and build on their experience.
Why police, crime, public safety and criminal justice data and information are important
Data and information about policing, public safety and criminal justice are essential in local and strategic oversight and accountability of police forces and the wider criminal justice system.
There are several categories of people and organisations that are already users of crime data, and others who are likely to become users in the future. The types and format of data to be published must meet the needs of all these different groups.
- Members of the public seeking to understand and potentially act locally to tackle crime in their area.
- Developers and local community activists who will turn the published data into information that is useful for the public and other users.
- Crime and disorder reduction partnerships and bodies
- Elected members of the police and crime panels charged with holding elected police and crime commissioners to account, and other local councillors and Members of Parliament.
- Businesses and third sector organisations.
One of the features of public data is that the full extent of its likely use and value can only be discovered once publication takes place at a granular level in a standardised, re-usable and machine-readable format and without licensing restrictions preventing re-use and re-purposing. The uses of crime and policing data would be likely to include the following.
- Local, granular data will be essential in informing the proposed neighbourhood and beat meetings where local people hold their local police teams to account and put forward their views on priorities in the area. In order for this to be meaningful, comprehensive and granular data must be clearly related to strategic force-wide data in order to provide a context for discussing local priorities. The principle that should apply is that data should be published at the level of granularity at which it is held, unless there are strong reasons for doing otherwise.
- Data will also be crucial in informing the work of the proposed police and crime panels which will be responsible for holding elected Police and Crime Commissioners to account on behalf of the electorate. For this purpose, it will be important for data to be published according to consistent, standardised categories so that meaningful comparisons can be made between forces. Police and Crime Commissioners should have within their remit the ability to require police forces to publish data.
- As public services continue to move towards partnership and area-based working, crime, public safety and criminal justice data will inform agreements on local priorities and strategies for addressing the public’s concerns about crime and safety.
- If people in the ‘big society’ are to be expected to get involved in preventing and tackling crime in their local areas, they need to know what crime is taking place and where.
- Crime and related data, if published in sufficient detail, can also inform other policies and decisions within the private and public sectors, such as social services, town planning, public health, and education and skills training provision.
There are also many potential benefits for those working within police forces and the wider criminal justice system seeking to improve their services, and there are potential savings and economies that should come from the local publication of standardised data.
A presumption that all data will be released
Police forces and other public bodies hold huge amounts of detailed data about what crime takes place and what is reported, including details of the crimes, demographic information about the perpetrators and victims as well as what charges are brought, what convictions are made, what sentences are passed and to what extent the sentences are completed. The key issue is the extent to which this data can and should be published and to what degree of granular detail.
Police and crime data
The Prime Minister’s letter to government departments on opening up data, on 31 May 2010, included a commitment for ‘Crime data to be published at a level that allows the public to see what is happening on their streets from January 2011’.5 Work is underway to define what this will involve and how it will be done. It will be important to ensure that the data covers the range of information that is necessary for effective transparency and accountability at both strategic and local level. This broadly falls into six categories:
- financial data, including budgets and expenditure, which should so far as possible be broken down according to the approach taken with local authority spending data (although see our comments on risk, below – one example of risks associated with financial data would be spending relating to covert operations and informants; there will undoubtedly be others that will need to be considered and managed).
- performance data - whilst we recognise the Government’s desire to move away from targets and de-facto targets, this does not negate the need for data about performance on key issues of public concern and importance such as cleanup / sanction detection rates in relation to different categories of crime.
- human resources data, including senior officers and beat officers, together with identifiers such as badge numbers, and disciplinary data.
- crime data, including data about the location, time / date and type of crime, and demographic information about perpetrators and victims of crime.
- democratic data (eg about the police force itself and the police and crime commissioner, including geographical remit, and about decisions taken, meetings and minutes)
- communications data and information (eg community bulletins)
In all of these categories, data should be published at the most granular level of detail possible. For example, in order for the proposed local ‘beat meetings’ to work effectively, crime data would need to be published at specific location level rather than local authority, ward or sub-ward level. The presumption should be that whatever data is collected should be published unless there is a genuine reason not to do so. A good example of this granular level of data about reported crimes can be found in the USA, where forces publish location-specific data about reported crimes.
We recognise that some data held by police forces cannot or should not be published, particularly in real time, because it may compromise operations or reveal the identity of victims or suspected perpetrators of particular crimes where this would be detrimental to the victims or might compromise operations. However, in our view this should apply in limited circumstances: the presumption should be that as much data as possible will be published as close to real time as possible, with exemptions being the exception rather than the rule.
Data relating to crime held by other services and bodies
People’s experience of crime, safety and criminal justice does not adhere to bureaucratic boundaries between different agencies. The data held by police forces is one very important piece of the jigsaw, but equally important are the data held by emergency services on crimes that are not reported to the police, and data held by criminal justice, probation and other agencies about issues such as sentencing, rehabilitation of offenders and re-offending rates.
People will want to use police and crime data to understand trends and patterns in crime. This requires data held by agencies other than police forces. For example, we know that an estimated 60 per cent of crime is not reported to the police, some of which can be revealed by data held by other bodies such as ambulance services. Ambulance services hold data about call-outs relating to assaults that can inform a wider and more detailed understanding of the incidence and nature of violent crime than would be afforded by examining police figures on reported crime in isolation.
Criminal justice data – a whole systems approach
The consultation paper refers mainly to reform of police oversight and accountability arrangements, but the direction of travel is towards a more rounded and comprehensive view of policing, crime and criminal justice, for example through crime and disorder reduction partnerships that seek to tackle the causes, incidence and consequences of crime and disorder in their local areas. The Panel believes that the proposed reforms, and the Government’s commitment to open and linked data as a building block of transparency and accountability, will present a valuable opportunity to review the accountability and transparency of the wider criminal justice system.
As we move towards greater transparency and accountability of public services, there will be a transition away from the bureaucratic tradition of elected politicians and official auditors seeking to hold individual services to account against targets. There is a growing and increasingly diverse network of scrutineers, both elected and unelected, seeking to investigate problems and issues from the point of view of people and contribute to the solutions, both locally and strategically. In this context, members of the public, the media, academics, policy-makers and politicians will expect to be able to look not only at the performance of police forces, but also at the performance of the criminal justice system as a whole, from prevention to rehabilitation of offenders and everything in between.
It is currently either impossible or at least very difficult to access meaningful data that shows the full journey from a crime being committed to justice being served. The crime number only applies within the police force – once the crime enters the criminal justice system, different unique identifiers apply. This means that individual crimes cannot easily be tracked through the system using the unique identifiers.
We propose that a whole systems approach should be taken to the publication of data about crime and criminal justice. This would aid public understanding of patterns and trends in crime and could inform the public and democratic oversight and accountability of police and other relevant services.
For example, magistrates’ courts should be required to publish data online and in a standardised format, showing data about the cases and outcomes. At present this information is often available only to journalists and not democratically elected or lay scrutineers.
This may require changes to way in which different bodies identify and track individual crimes and cases, so that either the same unique identifiers apply throughout the life cycle of a case, or the unique identifiers used by different bodies are linked.
How data should be released
The question of how data should be released is as important as questions about what data should be included. If data is not released in a standardised, re-usable, machine-readable form without restrictions on re-use or re-purposing, then it cannot used to its full potential and cannot be described as genuinely open data.
We recognise that there is already a good deal of data and information published by a range of bodies in relation to police and crime. However, much of this is published in PDF or word processed documents and it is often published with a significant time lag, at ward or local authority level, within excessively broad categories and in inconsistent formats. The Home Office, working with ACPO and others, should promote, support and facilitate a move towards the consistent and comprehensive publication of standardised, re-usable, machine-readable data which can be freely re-used for any lawful purpose.
The approach that should be taken to releasing data should reflect the progression from taking the first step of publishing data in whatever form to the goal of providing linked data:
- publish the available data on the web in whatever format
- make it available as structured data, for example in a spreadsheet rather than a pdf document
- publish it in a non-proprietary format such as comma separated values (csv)
- user URLs to identify items so that people can ‘point’ to them
- link the data to other data to provide context.
The best must not become the enemy of the good – progress should be made to the next stage towards linked data at the earliest opportunity, as each step in this progression represents an improvement on the previous level. Stage ii should be a reasonable starting position, reflecting the principle that data should be published in machine-readable, re-usable format. However, any difficulty in achieving stage ii is not a reason to delay or not to do stage i.
The Government has clearly indicated that its priority is to promote the early release of data even if it is not perfect. The move is from a presumption of not publishing data in case risks might materialise to a presumption of publishing it whilst managing the risks and trying to minimise imperfections. This should be reflected in the approach taken to releasing data and the assessment and management of risks.
Clearly there are risks and potential consequences involved in releasing police and crime data, and these must be carefully managed. For example, the implications for individuals of publishing criminal justice data on the web will need to be fully thought through, given the presumption that nothing is ever fully deleted from the web. However, none of these risks is insurmountable, and most risks will apply only to specific categories or types of data rather than to police and crime data in general. Risks must therefore be managed in relation to specific categories of information, and the fact that there are risks involved should not generally act as a brake on progress across the board.
Data should be published locally by police forces and accessible through links on data.gov.uk. The case for retaining a central hub for collecting data before publication needs to be reviewed, as a central data collection hub runs the risk of creating bottlenecks in the release of data. The aim should be to minimise the time taken between data being collected within each body and it being published. Sharing hubs that do continue should add significant value to the data and should not simply be a clearing house for the data.
There must be a standard copyright, building on the Open Government License that has been developed for local authority spend data and general public sector re-use. This would enable people to re-use the data without unnecessary and restrictive copyright restrictions.
Getting there
Achieving the ambitions set out in this paper will require leadership at the highest political level, and significant work to support and promote those who want to act as exemplars.
Much of the progress that has been possible in the release of local spending data has been the result of innovative and forward-thinking work by early adopters. This should equally apply in the case of police and crime data. We have been made aware of some police authorities that have already taken steps to open up their data. We will be pleased to work with early adopters to support them in becoming examplars and to support others to learn from and build on their experience.
We recognise that there will be technical and bureaucratic barriers to the release of some types of data, as well as the need to consider and manage risks associated with certain categories of data. These risks and obstacles should be identified and discussed publicly so that the issues can be aired and solutions generated in conversation with data users. This will help to generate and sustain momentum and progress, and ultimately ensure that the data that is published is relevant and useful.
1 comment
Comment by Andy Mabbett posted on
I hope this will be of some use: http://pigsonthewing.org.uk/more-police-web-pages-about-wildlife-crime-officers-needed/