Ways of working for test and learn

Public Digital’s definition of digital hinges on organisations becoming adaptive enough to respond to people’s raised expectations, not least in response to AI. Like any technology, AI is constantly evolving, as are people’s assumptions about what it can do for them.

To avoid being left behind, organisations must become adaptive, and that means embracing test and learn feedback loops at pace and at scale – for both AI and other new ways of working.

Public Digital’s consultants come from a broad range of backgrounds in digital delivery. From our collective experience, and Public Digital’s work in public and private organisations, we’ve seen just how challenging it can be to create the conditions for test and learn to happen.

Here’s our guide to creating the conditions you need to succeed:

1. Start by defining the outcome you want to achieve

Make your outcome clear, so everyone can relate to it.
Make your outcome bold, so itdrives change.
Make your outcome crunchy, so it fosters accountability.
Empower teams to work out how to reach this outcome (they may need a few sub-outcomes to help them steer a path).

2. Design for user needs, not organisational convenience

Use insights from ongoing, embedded user research as your guide; make user research a team sport.
Trust data over intuition (but trust insights from user research over both).
Develop awareness about how emerging technology may alter or create behaviours by observing early adopters.

3. Test your riskiest assumptions with actual users.

Identify and test assumptions about the operating model and the technology, as well as the business model or the service design.
The best way to get value from new technologies like AI is to change your organisation’s norms such that it can test and learn from incorporating, say, LLMs into real services with real users; do so at pace, at scale and with awareness of possible consequences.
At the start, one of your riskiest assumptions is often whether your team can work well together at all. Make this explicit; another assumption to be tested.
Create a culture where teams feel they can talk about assumptions and risks, not hide them away; false certainty is enticing, but dangerous.

4. The unit of delivery is the empowered, multidisciplinary team

Hire specialists for attitude as well as aptitude. No rockstars.
Bring in people from operations from the start and make subject matter experts like policy, legal and security part of the team.
Build a small core team who are 100% dedicated, and use the Team Onion as a model of broader engagement with collaborators and supporters.
The team should own their process; don’t let any process own them.
Scale by adding teams, not by making a bigger team. Do this organically and incrementally, not by guessing upfront.
Teams won’t always map onto existing organisational structure and that’s OK, in fact it’s probably a good thing.
Design for, practice and learn from failure.
Ensure final say on prioritisation is the clear responsibility of a single product owner within each team.
Give teams the tools and physical environment they need to do their job.

5. Test and learn. Then test and learn again. And again. And again.

The test and learn loop needs to apply to policies and practices as much as it does to software and technologies..
Feedback loops are best informed by observing real users.
Don’t fixate on one potential solution to a problem – explore the possibility space through research and prototyping.
Sometimes you should test and learn to improve your existing service; sometimes you should reinvent the service completely. Pick one.
Changes need to be live in hours not months, so make changing software easy and safe. Continuous delivery of code is a non-negotiable essential. Continuous deployment is even better.
If code doesn’t have automated tests, it doesn’t work.

6. Transformation happens at the speed of trust.

Leaders must trust their teams to work out how to meet the outcomes you’ve set. This is hard, scary, and totally worth it.
Foster the psychological safety essential for people in your teams to trust each other, to trust other teams, and to trust their leaders. This will usually require investment of significant time and emotional energy.
Do the hard work to build trust across your organisational boundaries. The problems you’re trying to solve won’t respect them, so do all you can to lower them.

7. Do the hard work to make things simple

Teams should obsess about making their services simple to use, simple to understand and simple to change.
Sometimes that means starting again and redesigning everything from your desired outcome backwards.
For existing services, go back to first principles and challenge every rule, custom and practice that creates complexity or confusion for users.
Be suspicious of the latest technology “innovation”. Fix all the broken things, first. Measure, then reduce failure demand.

8. Staying secure means building for resilience

You can no longer stay secure by standing still. Agility and threat awareness are your friends.
Bake security awareness into the mindset of your teams, and the design of your service, operations and technology.
Understand potential threats, notably how data might be misused, and then make life as hard as possible for those who seek to do damage to you or your users.
Your obligations don’t end at the edge of your network. Understand, assess and manage the risks in your software and data supply chain, not just the infrastructure you control.

9. Recognise the duty of care you have to users, and to the data you hold about them

Minimise risk by minimising the data you hold about users. Practice privacy by design.
Explain to users what data you hold about them, how it is being used and who is responsible.
Put users in control of the data you hold about them, this helps to respect their rights against misuse, and protect them from fraud.
Design for minimum viable data access, not bulk data sharing.
Give users easy ways to get their problems fixed.

10. Make things open; it makes things better

Show the thing. Practice agile communication.
Early on, use prototypes to tell a story, set a vision and build a shared understanding. Then throw the prototypes away.
Use the tools of the open internet, and be open to open source.
Adopting open standards helps avoid getting locked-in by proprietary software vendors.
Value shipping products to real users over documentation, ‘target operating models’ or RAG statuses.
Use openness, regular show-and-tells and assessment via peer-review to demonstrate progress, and to know how to change the direction of a product or shut it down all together.
Effective governance means going and seeing the thing for yourself, regularly.

11. Fund product teams, not projects

Projects and programmes are often the engine of legacy creation. So provide ongoing funding to product teams instead.
Beware your business case, finance processes or contracts dictating how, or even what you deliver. A “lean/agile friendly” business case process is vital in the medium-term. Credit to the UK Government for trialing some new ideas.
Budget for continuous improvement. The mix of skills you need and the size of the team will change over time. Help your finance team adapt to this.
Don’t be afraid to stop funding something if it isn’t working (agile business cases should make it easier to call a halt earlier).

12. Redesign your governance to be an enabler

Redesign your governance processes around these 6 enduring principles.
Your governance should support teams to deliver outcomes with assurance and pace. You’re not marking their homework; you’re staying very close to their work, unblocking their progress, and applying small course corrections.
This will often require radical change from your normal governance processes. That’s OK. In fact, it’s probably essential.

You’ll know your governance is working for you when:

You are all focusing on outcomes
You have the necessary skills and capabilities to deliver the work
Leaders and teams have confidence in the outcomes and the pace that they are trying to deliver
Teams are supported to deliver (leaders know how to help)
You understand and are comfortable with the risks you are taking
You have clarity of the impact
You are working across the system and breaking down silos

Finally: break any of these rules sooner than do anything barbaric. It’s better to be pragmatic and make some progress rather than wait for perfect circumstances that will never come, and not make any progress at all.

This list isn’t set in stone. It gives you plenty of starting points, but don’t be tempted to try implementing all of them, everywhere, all at once. As always, start small, pick your first project carefully, and focus on the stuff that feels most relevant to your organisation. Iterate the bits that help your teams deliver, in your organisation.

Allow time and space to learn your own lessons, so you can make your own additions to the list in future.