Should government own its mind?

Written by Jonny Williams, Chief Digital Adviser UK Public Sector, Red Hat

Imagine your housing benefit is reviewed. An automated system flags a discrepancy in your earnings, and, pending verification, your payments stop.

The civil servant handling your case may not be able to independently verify why this has happened. Not because they’re being evasive, but because the system that produced the recommendation is operated by a commercial vendor, running on infrastructure the department doesn’t control, using a model that may have changed since it was last audited.

 

How we got here

This is the logical consequence of how government has been buying artificial intelligence. To understand how we got here, you have to go back to 2013.

In the early 2010s, the government made what seemed like a sensible decision. Stop buying expensive, monolithic IT systems from a handful of enormous suppliers. Instead, government embraced commoditisation. Buying computing power like electricity – from whoever is cheapest, switching when you want, owning nothing you don’t need to own.

This was about freedom from dependency. Until it wasn’t.

 

The cloud wasn’t the problem. The stack was.

The problem was not the cloud as a concept. It was what came with it. Renting compute gradually became an interwoven stack of services, including managed databases, proprietary data pipelines, vendor-native security tooling, platform-specific development frameworks.

Consuming a stack that was tightly coupled to the underlying compute meant that government workloads did not merely run on hyperscaler infrastructure – they were woven into it. The compute was commodity, but everything built on top of it was not.

By the time anyone looked hard at the numbers, following the adoption of this stack, three companies owned 95% of UK government cloud spending. One firm alone had received nearly a billion pounds without a truly competitive tender. A policy designed to end oligopoly had arguably created one.

Now, rather than unwinding that dependency, departments are layering artificial intelligence on top of it, and pursuing sovereign AI rhetoric in public while procurement incentives deepen the same structural relationships with the same incumbent suppliers.

 

Someone else’s mind

We may have tolerated the expense and dependency of those decisions for compute. But artificial intelligence is not just someone else’s computer, it’s someone else’s mind.

What government is now outsourcing is not storage or processing power. It is judgement itself.

When a department uses a managed AI service to help determine whether you receive a benefit, how your immigration case is prioritised, or whether a fraud flag is placed on your account, a question arises that policy has not yet answered honestly. Who is actually making that decision?

If a model belongs to a vendor, the version running this morning may not be the version running tomorrow, and the department may not be notified. The training data, the internal logic, the decision-making systems all sit behind proprietary APIs, accessible to government only as outputs. The model could even disappear tomorrow, leaving government scrambling.

The department owns the consequences, while the vendor owns the stack.

 

Accountability you can’t actually demonstrate

Since March 2024, central government has been legally required to document and be accountable for automated decision-making tools that have a significant influence on decisions with public effect.

For AI running on open, inspectable infrastructure, that standard can be met with evidence. For a managed API, you fill in the compliance form using whatever the vendor told you. Critical parts of the audit chain remain vendor-controlled. The department cannot independently verify which version processed a given case, or reproduce the logic behind a specific outcome.

One of these approaches enables departmental accountability, the other mimics it.

 

The counterarguments

Some will point to the ability to import custom model weights into these managed platforms as evidence of agency, choice, and control. It is not. Owning the weights while the vendor controls the inference pipeline, the orchestration layer, and the surrounding compute is roughly analogous to owning the recipe while the restaurant controls the kitchen, the ingredients, and the bill. The department may determine what the model is, but the vendor determines how, where, and at what cost it runs. The dependency has simply moved one level down.

Those who defend the status quo tend to invoke UKCloud – a domestic provider that collapsed in 2022, costing the Cabinet Office £17 million and stranding government workloads across the country. This failure is frequently presented as proof that concentration around the large hyperscalers is an unfortunate market reality rather than a policy choice. But that interpretation arguably gets the lesson backwards. UKCloud failed because its clients were locked into its proprietary platform. Any platform lock-in is fragile, therefore the cure is portability and interoperability, not resignation.

An open stack for AI, running on commodity compute, can be picked up and moved. Change the provider, keep the capability. The infrastructure becomes interchangeable, which is, in many ways, what Cloud First was originally intended for.

 

A contradiction at the heart of policy

Which makes the government’s current position particularly difficult to defend. With one hand, it is making smart investments in sovereign AI, with supercomputers in Bristol, GPU clusters designated as national infrastructure, and a stated ambition to make Britain the global home of open source AI talent. With the other, it is signing procurement agreements that build new proprietary dependencies on top of old ones.

The government has committed over £2 billion to sovereign compute infrastructure on the premise that Britain would have genuine AI capability of its own. But if departmental AI workloads are locked into managed AI services then they cannot move. Our sovereign compute investment will return nothing if the AI workloads running across government are structurally incapable of migrating to it.

You cannot have a sovereign AI strategy and a managed-API procurement default for government. These two positions are oil and water.

 

The fix

The fix is not complicated. Departments are required to demonstrate a viable exit strategy before technology spend is approved (albeit this is frequently overlooked). That requirement should be extended to AI, with one clarification that exit means being able to move the entire working system to different infrastructure without rebuilding it. Not owning the model weights or switching between models on the same vendor’s platform. Actually being able to pick it up and run it somewhere else. That is a testable, demonstrable standard – and it is not currently being met. For high-stakes decisions, including benefits, immigration, fraud, that standard should be non-negotiable.

None of this requires leaving the cloud. Renting someone else’s tin is fine. But letting someone else own your systems of judgement for the sake of efficiency is a category error we will spend the next decade correcting, and regretting.

This is not a question of whether we should use powerful infrastructure. That should be a given. But, should using someone else’s compute leave us nationally exposed to someone else’s strategy, someone else’s commercial decisions, and, when a citizen needs to understand why an algorithm changed their life, someone else’s silence.

For a government accountable to its citizens, the answer has to be no.

It was no in 2013, when the policy so many individuals are quick to invoke was created, and it should be no now.

Except now the stakes are much, much higher.


Read More Regulation

Comments are closed.