Transforming data sharing across government

Challenge 

Data sharing in government could release untold value, simplifying services, saving users time and generating new policy insight. But in practice it is difficult, creating data sharing arrangements can take up to 18 months due to inconsistencies in how departments create, store, promote and govern the use of their data. 

In 2022 the government published its roadmap for digital and data and with it an ambition to ensure better data to power decision making.

The vision for the Data Marketplace is that it will become a place to find and understand how to find and access data across government in a legal, ethical and effective way. 

We partnered with the Central Digital and Data Office (CDD) to put in place a full multidisciplinary team to design and build a minimum viable product (MVP) as a private beta service on Cabinet Office’s Azure platform to enable cross department data sharing.

Approach 

This was a complex project requiring a multidisciplinary team of research, design and technology expertise, from user researchers to front end and back end developers and data architects. We collaborated with 156 users across 35 departments and arms-length bodies in government. Our role was to first deliver an Alpha and successfully complete an alpha assessment. While the assessment of this service was complex due to its position at the heart of government, the depth of our understanding of user needs eventually persuaded the assessors that change was needed here and that an MVP could help to establish user needs in a working context, rather than in theory.

Following that success, we began to stand up a beta minimum viable product (MVP) that would provide a place to search for data, provide a standardised approach to sharing data and provide guidance documentation whilst being secure, measurable and interoperable.

User research

Existing research had insight gaps in how civil servants worked with data.  We did semi-structured interviews across government departments to classify who was using data and what further data and information they needed to obtain and why. Nearly every example was unique, but common features were: finding data, usefulness of data to solve a specific problem, technical requirements and the legal, cultural and policy constraints. We designed behavioural personas focusing on acquirers and suppliers of data.  Then we ran workshops and interviews to establish the challenges, blockers and pain points people found with data. We conducted comprehensive usability testing and attended established working groups to ensure robust insights.

What we learnt

Departments had real difficulty in identifying data that was available to them. Departments faced challenges in arranging data sharing because of the varying processes and governance in each department and those processes meant longer durations to get access to the right data.  There was a lack of understanding of who needed to be involved and at what stage of the process, on the whole, a really complex arrangement for data sharing.

Design

We created as-is and to-be journey maps, detailing existing and future states of the data marketplace so users could see how pain points and blockers would be removed and to give senior leadership confidence.  We helped build enthusiasm for why a design approach focused on user needs would enable the right technology build. We created a comprehensive landscape map detailing interdependencies to help define the data marketplace enterprise architecture and we worked with multiple departments across government to define and build confidence that the data marketplace would derisk their data activities.

Tech solution

Our initial goal was to ensure we were solving the right problems so we could complete an alpha phase and gain an understanding of what the tech solution needed to be then led to an extended alpha. Finally, we delivered a private beta product that began to deliver value as a central place to search for data, as well as a standardised approach to understanding what data was available and comprehensive guidance about how to share data. Ultimately what was needed was a way to get data suppliers and people who wanted data talking about how real data could be shared. We delivered an overarching user interface for the catalogue including a search and share function to get data suppliers and data acquirers talking, the “design a data share” service and login capability all using the gov.uk design system so that only authorised users could access the service.

Impact 

Throughout this project, we brought together the right expertise from different disciplines to build a multidisciplinary team at pace to tackle a complex and far reaching problem across government. Coming together with CDDO as ‘one team’ to deliver a high cadence, using open, transparent ways of working, meant we could ship code regularly and quickly  and make sure that design thinking was present at every stage, resulting in a private beta which is now being trialled with government departments

CDDO now has a working beta that provides a central place to search for data, provides a consistent and standardised approach for sharing data across government and importantly is scalable across all government departments.