UK's grand plan to fuel AI with public data faces uphill battle

Promise meets prototype

It has been reported that the government pledged a National Data Library (NDL) as part of a 2024 Autumn Budget promise — a centrepiece meant to feed public-sector data into cutting‑edge AI and boost growth. The Department for Science, Innovation and Technology says it has mapped priorities and completed a discovery phase. But a new Open Data Institute (ODI) prototype, dubbed "NDL‑Lite", has found the data landscape less like a well‑oiled reservoir and more like a dusty filing cabinet: 38 GB of material and over 100,000 files brought together, yet many datasets are poorly labelled, out of date or missing critical metadata.

Why AI will look elsewhere

The ODI discovered that messy metadata and inconsistent standards make even broad topics — take "crime" — hard to analyse at scale. One Home Office crime dataset hasn't been updated since 2018, and the updated version is apparently not accessible via the ONS API. Professor Elena Simperl of the ODI warned that AI agents, faced with inaccessible official sources, will simply go elsewhere — news, social media, commercial feeds — which might be easier to scrape but not necessarily more reliable. The emotional sting here: a public resource built to underpin trusted analysis could be bypassed, leaving noise to fill the vacuum.

Fixes, fanfare and a race against time

A government spokesperson said efforts are underway to "maximise the benefits of public sector data" and that digital infrastructure is being overhauled under a Roadmap for Modern Digital Government. It has been reported that the ODI’s work shows the NDL can be assembled at relatively low cost — if the raw materials are fit for purpose. So the question now is simple: will ministers and data owners clean up labels, standards and APIs quickly enough, or will AI developers keep drifting toward easier, messier sources? The answer will decide whether the NDL is a foundation for trustworthy AI — or just another missed opportunity.

Sources: The Register