The UK Government’s AI Obsession is a Big Risk w/ Will Dunn

Episode Summary

Executive Summary: The episode examines how the UK government has rapidly adopted generative AI, driven by hopes of efficiency, fiscal savings, and national competitiveness. Will Dunn argues this rush creates major risks: vendor lock-in, weakened democratic control, biased or politically shaped outputs, and erosion of state capacity and sovereignty as governments increasingly rely on persuasive, foreign-built systems to write documents, analyze budgets, and even draft laws.

Main Topics: UK’s AI-first state strategy (Priority: 5/5): The conversation explains why the UK wants to appear early and enthusiastic in adopting AI, including a desire to emulate past waves of industrial modernization and to signal openness to innovation and investment. Political and ideological advocates (Priority: 5/5): Paris and Will discuss the network of influencers pushing AI adoption in government, from Silicon Valley lobbyists to figures like Tony Blair and Dominic Cummings, who see AI as a way to make government faster and more decisive. AI in government operations (Priority: 5/5): The transcript details practical deployments such as document summarization, sorting citizen letters, spending-review analysis, parliamentary speech drafting, and even AI-written legal text, showing the technology is already embedded in state functions. Sovereignty, power, and vendor lock-in (Priority: 5/5): A major concern is that outsourcing core governmental judgment to US-based AI platforms shifts control away from domestic institutions and locks the state into expensive, hard-to-replace systems. Risks of persuasion and bias (Priority: 4/5): Will argues generative AI is not just a tool but a persuasive system that can shape ministerial judgment, embed political bias, and influence who gets heard in government decision-making. Economic uncertainty and overstated savings (Priority: 4/5): The episode questions claims that AI will produce major productivity gains or fiscal savings, emphasizing that the technology may instead impose new costs, energy demands, and political risks. Public and industry pushback (Priority: 3/5): The discussion notes emerging resistance from copyright holders, creatives, and local communities opposing data centers, though there is not yet a broad anti-AI mass movement in the UK.

Key Arguments: The UK is trying to become “customer number one” for AI because it lacks domestic foundational-model giants like the US and China. Labour and Conservative governments both embraced AI, but Labour inherited severe fiscal pressure that made AI look like a shortcut to savings. Dominic Cummings-style reformers see AI not just as efficiency software but as a way to bypass the civil service and make ministers more powerful. Using foreign-built AI in core state functions raises sovereignty concerns because the software embodies external norms, incentives, and potential biases. AI is already being used for real government tasks, including analyzing spending bids and drafting legal text, showing this is not speculative. The biggest danger is not only automation of work, but delegation of judgment and persuasion to systems optimized to influence human decision-makers. Vendor lock-in can begin with cheap or even nominal contracts, but later makes government dependent on expensive proprietary systems. Claims that AI will fix public finance and productivity problems are viewed as overconfident; technological gains have historically been localized rather than economy-wide. The UK’s creative sector is mounting resistance to AI training on copyrighted work, which may become a major public flashpoint. If AI adoption continues, questions about who truly holds power in government and how laws are made will become more politically salient.

Data Points: Time since ChatGPT launch before UK Labour took office: Just under 2 years - Will Dunn notes that by Labour’s 2024 arrival, AI had already become a major political topic after ChatGPT’s release. Estimated AI savings cited by advocates: £45 billion per year - Promoters told the UK government AI could save this amount annually, shaping enthusiasm for adoption. OpenAI losses last year: $20 billion - Paris notes the company is still burning massive amounts of cash, raising questions about future pricing and dependence. Former UK prime ministers from Oxford: 31 - Will uses this statistic to illustrate how easy it is to influence future UK leaders via a small number of elite institutions. UK debt level: Approaching GDP size - Will describes the UK’s fiscal strain as part of the pressure to search for a technological “silver bullet.” Public sector letters handled by AI: Thousands per year - A government tool was described as triaging large volumes of citizen letters that otherwise would take too long for human review. AI-written legal text in UK constitution: At least one instance - Will says civil servants joked the UK had already used an LLM to compose constitutional text.

Pivotal Quotes: "If you bring an incredibly persuasive machine into every level of your power system... you're taking a risk that I think we have yet to calculate with who holds power." — Paris Marks: Opening framing of the episode’s central concern: AI can alter power structures, not just automate tasks. "So if you bring an incredibly persuasive machine into every level of your power system... you're taking a risk that I think we have yet to calculate with who holds power." — Paris Marks: Paris returns to this idea later in the interview, linking AI adoption to democratic control and institutional power. "We don't want Brussels to write the laws, we want OpenAI to do it. Let Sam Altman do it." — Will Dunn: A sharp critique of the irony that the UK rejected external lawmaking through Brexit but is now outsourcing legal drafting to AI.

Implications: The episode warns that AI adoption in government can quietly shift power to vendors and systems outside democratic oversight. Listeners should watch for lock-in, hidden bias, and state dependence disguised as modernization.