When it comes to user research on a topic with lots of unknowns, it can be a challenge to figure out how to begin.
For this discovery project, my purpose was to understand how we might improve data science literacy in the civil service to support the development of policy and delivery. This is a huge subject with the potential to go in any number of directions.
Discussing the application of terms like ‘data’ and ‘science’ at work with policy and delivery colleagues can seem daunting at the best of times, but when they join to become ‘data science’ it can appear like we’re speaking a different language.
What is data science and how can it create new insights into the way we work?
My colleague, Ed Parkes, wrote a blog post in April to frame the opportunities of a civil service that makes the most of data science. This blog post is intended to follow on from that - laying out the methodology of how we conducted our research.
Ed began by meeting with experts inside and outside government to scope the viability, desirability and feasibility of a data science literacy project in the civil service. Once he had used this to get a broad idea of next steps, he brought me in to focus on user research.
The interviews
We knew from the start that his project would cover an enormous space, and we were starting from a position where we didn’t know much about the status quo. So we would need as much data as possible from a broad selection of people: from experts to people who had little to no knowledge of the area.
In-depth interviews were therefore seen as the best way forward. Whilst they are time consuming and take a long time to write up and assess, they do offer richer, more descriptive and qualitative information over other data collection methods like surveys. We could ask questions that would yield more descriptive answers and we could really drill down to find the kind of information we needed to start our project.
We designed an interview guide specifically laid out for simplicity, depth and easy comparison with the following structure:
- Role and background
- Current use of data
- Desired use of data
- Barriers
- Learning and innovative practice
All those interviewed were asked to review and sign consent forms that allowed us to record interviews, use notes and recordings for internal research purposes, and which would protect user confidentiality.
A round of 15 interviews followed with key contacts and potential users from mid-April to late-May. This was succeeded by a workshop with GDS colleagues in June to communicate early insights and receive feedback about user research so far.
For the second round of interviews, we undertook a recruitment process primarily using GDS colleague networks. The intention was to interview non-data science experts and we focused on finding “mainstream” policy and delivery civil servants.
This phase took place from mid-June to mid-July 2016. Given time and resources we set a target of 20 interviews, but we were able to surpass that and reach 25 interviews across 12 departmental agencies. Of that cohort of interviewees, 19 were based in London and 6 were based in other regions across England and Scotland.
In total, we conducted 40 interviews before analysing the data by comparing responses to each interview theme to spot trends and variations as well as develop personas to highlight key character types. We then took our findings to two workshops with GDS and data- related colleagues from several departments to create propositions (i.e. possible solutions to user needs). Those propositions were then tested with many of the interviewees and other civil servants in focus groups to help us understand strengths, weaknesses and preferences. As a result, we now have greater clarity about the challenges, opportunities and context in which data science might play a crucial role in the development of policy and delivery in the civil service.
Ed Parkes wrote a blog post that set out some of the options for services we considered on the basis of the research. Take a look to see where we hope to go next.
Recent Comments