PD Newsletter #98: Test, learn and grow

👋🏽👋🏻👋🏾 Hello, welcome.

We’ve just published our new book on test and learn, an approach to delivery that we’ve been promoting for as long as PD has existed. Email us at communications@public.digital to request a free copy.

And this month, we announced our partnership with the Government Outcomes Lab at the Blavatnik School of Government and New Local helping to deliver the UK Government’s Test, Learn & Grow programme.

Millie and Rosemary

🌟 View this newsletter on LinkedIn
🚀 Forwarded this email? Sign-up

✨ Introducing our guest editor

Millie Devereux is a Principal Consultant at Public Digital who guides organisations through the behavioural and systemic change needed to deliver services that work for everyone. She specialises in combining user-centred design with behavioural science to transform public services, working closely with senior leaders to ground strategic decisions in real user experience.

Before joining PD, Millie established and led the user-centred design practice at Social Finance, shaped national policy on data sharing for child safeguarding at the Government Digital Service, and delivered behavioural design across six countries at the Behavioural Insights Team.

Read Millie's piece, featured in our new book, on learnings from behavioural science in adopting test and learn.

🔄 Test and learn

A test and learn approach is the foundation of a modern organisation. It’s a way of working which is part of our DNA at Public Digital, but which, as Lara Sampson writes in the foreword to our new book, is unique in that it doesn’t belong to any single profession.

“Test and learn is a way of thinking that can be widely understood, is relevant across whole organisations and even wider systems, and is inherently inclusive - if we take care to keep it so.”

Lara Sampson

First pioneered in government with projects like the 2013 Universal Credit reset, test and learn puts the needs of people and communities at the heart of how we design and deliver services. It describes an approach which:
➡️ Is oriented around outcomes, and closes the gap between policy/strategy and delivery
➡️ Is evidence-led, focused on learning and improving through doing
➡️ Prioritises understanding user needs, building something quickly, testing with people, and improving based on what is learnt
➡️ Grows an evidence base of what works for others to build on.

To learn more about test and learn and how it can help your organisation, get in touch at workwithus@public.digital.

🔄 Test, Learn & Grow

The UK Government’s Test, Learn & Grow (TLG) programme aims to embed test and learn in the delivery of public services, tackling the biggest local challenges by working directly with communities, local government and citizens to test what works, learn from the results, deliver incrementally, and grow a public sector reform movement.

Earlier this month the TLG partnership hosted over 100 members of the test and learn community - including Parliamentary Secretary Georgia Gould and DSIT minister Feryal Clark - at the Blavatnik School of Government. Watch the video below to find out more.

🚀 Latest from Public Digital

We’ve been shortlisted in two categories of the Lloyds British Business Excellence Awards. The winners will be announced on 11 November in London.

Our Data Bites events series is taking a break over the summer, but will be back on 3 September 2025 at Broadway House, London or via YouTube livestream.
In the meantime, watch the best bits from Data Bites #57 recorded on 2 July.

Ways of working

👐 While a test and learn approach relies on working in the open, sharing learnings and failures can feel uncomfortable - if not slightly terrifying. Giles Turnbull tells us how to be open while you test and learn, starting with embedding support for openness at the highest organisational levels. For examples of how organisations work in the open, Giles has put together this resource.

🧩 Our new test and learn book looks at how to be a service leader, but design leader Ayesha Moarif fills an important gap with her post on how to be service-led when you’re forced to work in a way designed for programme delivery. She suggests treating governance as a design constraint, as well as re-framing the work: “Create a parallel story of value that’s grounded in the service outcome, not just spend and features.”

🪐 A philosophical read from Harry Metcalfe on the friction between the conceptual and the experiential in the governance of large organisations - and what a resolution might look like. Among his many insightful takes: “Good governance doesn't subsist in a collection of policies and processes. It's a habit of attention… whatever clarity we find is never absolute.”

🧭 Great stuff in this interview from Manish Jethwa, the CTO of Ordnance Survey, on his work helping to transform the 234-year-old organisation. His concept of a ‘value chain’ from sourcing data to delivering value for customers is particularly useful: “Customers don’t want lightbulbs, they want light. At the OS, customers don’t want data, they want insights.”

🙌 Finally, Canadian tech educator Xe laso has developed Anubis, an open-source software available on GitHub, designed to protect websites from AI bot-scrapers by verifying that visitors to the site are human.

Technology in focus

🚀 We’re pleased to see institutions across Latin America have co-developed their own open-source LLM, designed to better meet the linguistic and cultural needs of Latin American users compared to US models. On a smaller scale, Steve Messer’s work playing around with his own model to automate note-taking in meetings is another impressive example of AI that is ‘local and open’.

🤢 Big tech presents AI products as feeding high consumer demand, but a team of French design experts have done excellent research on how big tech force-feeds us AI, in everything from the design of platform interfaces to how AI is represented in symbols and colours. Plus, it’s not much of a surprise that the Google AI salesman regurgitates and makes up sales pitches under the guise of helpful, objective content.

⚠️ Important reading in this technical paper from members of the AI Governance Initiative on why chain-of-thought (CoT) in LLMs is not explainability: verbalised chains “can appear coherent and convincing while failing to faithfully reflect the model’s true decision process”. The implications are stark for high-stakes LLM use cases in which CoT has been thought of as a method for verification.

👔 A journalist at the Verge tried out OpenAI’s ChatGPT Agent and - predictably - discovered some issues, not least the fact that its security measures essentially limit it to research tasks. Anthropic’s recent project which involved tasking an AI to run a vending machine produced some very strange and entertaining results, including the AI “claiming it would deliver products ‘in person’ to customers while wearing a blue blazer and a red tie.” At least it didn’t defy instructions and delete an entire database, as this AI vibe coding tool did.

🌈 Our last newsletter explored DEI and the importance of psychological safety in creating inclusive workplaces. The UK Government’s comprehensive report on diversity in the tech sector makes some of the same recommendations in its examination of the diversity, skills and inclusion gaps in UK tech organisations.

Digital government

🌍 The excellent DPI map launched in June, offering the first global investigation of digital identity, payment and data exchange systems. Whether it’s the Canadian Digital Service committing to developing a federal DPI strategy, or Indonesia integrating DPI in its social protection system, the momentum for this infrastructure - and the digital autonomy it affords - is growing. At PD, we’ve been developing our thinking on digital sovereignty, often understood as a goal for technological independence, but defined by Mike Bracken as “the agency and capacity of any organisation to make intelligent, informed choices to shape its digital future by design".

💾 It’s been a year since the Japanese government ended the use of floppy disks, having scrapped as many as 1,034 regulations governing their use. As Chris Govias points out, it’s an example of the hard, unglamorous work that makes or breaks good government services: “You can't just point AI at a problem like floppy disks and write a clever prompt to make it go away.”

🔥 James Plunkett writes shrewdly about how to save bureaucracy from itself. The solution for the UK Government is not, as he points out, banning civil servants from speaking in public: an approach which risks not only damaging policy and decision-making, but undermines the principle of openness on which effective government thrives. On the subject, this piece on a rejected FOI request to the gov.uk design team looks at what happens when an organisation that is synonymous with working in the open chooses not to.

🤔 The UK Government has also attracted criticism for its recently announced partnership with OpenAI. While perhaps not on the scale of President Trump’s so-called ‘big beautiful bill’ and his AI Action Plan in their endorsement of Silicon Valley, the UK government’s partnership might - as Imogen Parker of the Ada Lovelace Institute suggests - cause the public to “reasonably wonder whether, rather than taking actions to address the power of big tech, the government is helping entrench it.”

🔄 How can you take a test and learn approach in the public sector when contracting out to external providers? In partnership with the Government Outcomes Lab, we’ve been working with UK public sector commissioners and service providers to explore how a more relational approach to public sector procurement can better support iterative service delivery. Read about our findings in PD’s test and learn book.

Something fun

🏄‍♀️ Essential advice on recapturing the joy of the early internet in how to surf the web in 2025, and why you should. TL;DR: “If you end up on social media, you’re no longer surfing.”