Role and Responsibilities: Senior UX Researcher
The digital transformation of government has led to masses of unstructured data that cost millions to store. It is impossible to determine whether it is of value, can be used to inform future policy decisions, or is eligible to be sent to the National Archive.
Working with the Cabinet Office, we are exploring how machine learning and natural language models can be applied to help understand the contents of “digital heaps” of documents so that policymakers can learn from previous work and KIMs (knowledge and information managers) can meet their compliance obligations.
Working with the Cabinet Office and the team at Atchai, we wanted to explore how machine learning could be used to understand the contents of a single “digital heap” consisting of 1 petabyte of unstructured data. This heap included an array of file formats and content types. The aim was to create a series of prototypes that would provide ways of exploring the data and evaluating whether it is information that would enable the government to retain knowledge.
Using a one-week sprint model, I designed, developed, and tested a series of prototypes created using the GOV.UK design kit. 6 individuals from the policymaking and KIM community across the government were invited to each round to participate in contextual interviews and user testing.
The prototypes used JSON data files taken from the machine learning models the data science and engineering teams developed to give us real examples of what could be possible with this technology.
Participants used the prototypes to look for information about a significant historical event (The Anderson inquiry) to see what data existed and whether they could understand how the events unfolded.
There was overwhelmingly positive sentiment from the participants that tools like this would dramatically change their capabilities. Policymakers stated that the greatest waste in government is its inability to retain knowledge due to the loss of information when people leave. This kind of tool would reduce that risk dramatically and give policy leaders a greater ability to learn whether something had been previously explored or abandoned and, if so, why it didn’t work so that they do not repeat failed projects.
Luke Sands (Head of Digital at the Cabinet Office) described our collective work on this project as the best application of AI he’s seen within government and received positive attention from The Times newspaper.
David Canning, the Head of Digital Knowledge and Information Management in the Cabinet Office, published ‘Conquering the Digital Heap’, a post detailing the department’s journey to address the organisation of this complex data estate.