Adopting the Practice of Test and Learn
This book authored by Public Digital colleagues, unpacks what we mean by test and learn and how it can be applied to reforming services.
Read more
Using test and learn to improve the resident experience at Southern Housing
The client
Southern Housing is one of the largest housing associations in the UK, providing affordable homes and services to over 167,000 people across London, the South East, the Isle of Wight and the Midlands.
The challenge
Southern Housing had recently undergone a successful merger with another large housing association. With the merger complete, they wanted to build the skills, capabilities and momentum for more innovative work, and learn how to operate as a single organisation with a new governance framework.
At the same time, they wanted to adopt more agile, user-centred ways of working, while ensuring they could continue to comply with legal and regulatory change within the housing sector.
Underlying these goals was Southern Housing’s ambition for 2030 to improve their resident experience and improve their consumer standards grade as awarded by the Regulator of Social Housing.
What we did
- 1
We helped Southern Housing design and implement a new governance framework that set it up to operate as a single organisation
- 2
Within the client’s highly regulated environment, we ran a ‘test and learn’ beacon experiment which resulted in an improved repair experience for residents and created momentum for this new way of working.
- 4
Alongside creating a roadmap towards a service-led organisation, we helped to build a collective vision for transformation, including at a leadership level, embedding culture change and equipping them with the tools to maintain momentum.
- 3
We set the organisation up to adopt a test and learn approach as a sustained way of working, enabling them to extend this approach to their adoption of AI.
We helped Southern Housing build and implement a new governance framework.
We co-developed a transformation governance framework with the client’s transformation team and colleagues from its technology and data teams through running a series of workshops on:
- Articulating outcomes and measuring success
- Making decisions and keeping everyone accountable
- Prioritisation and roadmapping
- Sharing progress and managing risks
The framework we developed enables transformation teams to adopt a test and learn approach, continuously challenging and iterating how they work to achieve the best possible outcomes for users. The framework is structured around a set of core principles, supported by high-level guidance, so that teams can apply it consistently across projects.
We ran test and learn beacon experiments which resulted in an improved repair experience and more meaningful engagement with residents.
We built on the discovery work Southern Housing had already done to map the service for residents and staff. Paired with data analysis, this helped us identify where residents were experiencing failures and gaps in the service.
We helped the client to form a cross-functional, multidisciplinary team and, based on our findings, we ran several beacon experiments aligned to the client’s 2030 ambition around improving the repair experience. These focused on situations where repairs are not resolved first-time and there is a lack of clarity about next steps for both staff and residents.
One experiment, which involved introducing more resident-focused, text-based communication around when a repair was going to happen, led to a drop in both ‘no access’ cases - where residents aren't at home to allow repair staff entry to the property - and ‘chaser calls’ residents made to the call centre. Of 28 repairs in the beacon experiment, 20 were first-time fixes.
By introducing short feedback loops and a focus on users, a test and learn approach also empowered the organisation to engage meaningfully with its residents, moving beyond typical resident engagement activities towards bold input and user research. For example, we sought feedback from members of the resident engagement board on both their experience of the service and on our developing prototypes, enabling a human-centred approach that was positively received by residents.
We helped to build a collective vision and momentum for transformation
Being able to demonstrate the impact of testing and learning something quickly - particularly in the context of heavy regulation - created momentum and buy-in for new ways of working at Southern Housing.
At the same time, we worked directly with the leadership team to help them articulate and create a collective vision for transformation.
Facilitating a workshop which convened multiple senior leaders, we helped to democratise the organisation’s transformation and equip the leadership team with the tools and motivation to make change sustainable.
We set the client up to adopt and sustain a test and learn approach, enabling them to extend it into AI and colleague-led development
We provided the client with a test and learn playbook offering practical, accessible guidance to help teams adopt a test and learn approach.
We also set up the foundations for a test and learn Community of Practice, composed of specialists from across the organisation, designed to provide a dedicated forum for sustaining the practices, approaches and governance necessary for this way of working.
Southern Housing has since applied the same principles to the adoption of artificial intelligence. Rather than pursuing large-scale technology initiatives, the organisation has encouraged colleagues to identify real business challenges and explore AI-enabled solutions in a safe, governed environment.
Through a colleague-led development approach, teams from across the organisation have been supported to test ideas, build prototypes and share learning with others. This has enabled innovation to emerge from frontline teams and business areas that understand resident and colleague needs best, while ensuring appropriate oversight through Southern Housing's AI governance framework.
The approach has helped create a growing community of colleagues who are exploring how AI can improve services, reduce administrative effort and enhance decision-making. By encouraging trials, sharing successes and learning from failures, Southern Housing is building both AI capability and confidence across the organisation. This has created momentum for wider adoption while ensuring AI is introduced in a practical, responsible and resident-focused way.
