The public sector is a beast of inertia. Doing things that were done before is the point of an institution – to sustain and replicate certain ways of doing things.
This obviously has many advantages. No one wants the underpinnings of the state to be massively changeable on a day to day basis. We all need and want some consistency in how things work.
But there are also disadvantages. One of those is that the inertia can mean things continue on when they may no longer be fit-for-purpose, such as an assumption about how the world works governing processes and practices despite it no longer being a valid understanding.
The challenge for public servants is that these defaults – whether assumptions, business rules, or processes – are mostly not explicit. By virtue of being part of our established practices, they blend into the background and become just part of how things are done. And so there can be times when the world around us changes, but the implications of that change may go unnoticed. This can get us into trouble.
It’s likely one of these times is now.
GenAI challenges our defaults
A short time ago, you could assume that in every single meaningful interaction you had, there was a real person on the other end, or that at least it was obvious if that was not the case. You could also make assumptions about them by what they wrote or said or how they interacted. That’s not to say you’d always have been right of course, but you could still rely on a mental model of someone on the other side.
But with the arrival of accessible and ubiquitous generative AI, that core default assumption of having another person at the other side of an interaction – one that has been a fairly reliable one throughout the history of humanity – is no longer as much use. GenAI tools can make it easy for authoritative sounding content to be generated quickly and at scale, in any number of permutations. GenAI tools can make it easy for someone’s voice to be replicated and to say whatever an AI agent makes it say. GenAI tools are increasingly capable of easily creating video content that is hard to distinguish from reality.
Yet many of our default processes are built upon that previous assumption. It’s been such an obvious truth that it has permeated much of how we structure and do things in the public sector.
Which defaults will matter and which won’t?
Of course, just because a specific default or assumption is no longer as true as it once was, it doesn’t necessarily mean it is significant. Lots of our assumptions probably don’t matter in the grand scheme of things. Many defaults may cater to new assumptions as well as old ones.
The trouble with defaults is that they are privileged over doing something differently. So even though our circumstances might have changed, unless they are actively challenged or questioned, they are likely to survive until such a time as they are revealed to be unsatisfactory. And in the public sector context, unsatisfactory can have some serious consequences, whether in terms of risks, costs, or messing with people’s lives and livelihoods. Waiting for things to be revealed as unsatisfactory can be costly and poor stewardship.
“But we have risk assessment processes that deal with all of this.”
Risk processes can help deal with some of these issues, but not all. The transformative change posed by a general purpose technology means that many of our risk processes are not yet calibrated to the uncertainty and the new risks that GenAI brings.
For instance, in events I’ve run, I’ve tested whether participants have used GenAI tools in applying for a job – and the vast majority have. Yet I’m reasonably confident in saying that the majority of recruitment processes have not run a risk assessment looking at what that means for their process, because recruitment is still regarded as a pretty stable thing. Everyone ‘knows’ what the recruitment process involves, so why would anyone bother to question this default? It’s easy to think it either doesn’t matter, that the people who are running the process will be able to manage it if is an issue, or if it really was an issue, then someone else would have said something.
In short – there are new risks emerging, but because they are risks to deeply established defaults, and the assumptions underlying them, they are unlikely to come to the fore until they are realised.
So the questions that arise are:
- How serious is the challenge to our assumptions and defaults from generative AI?
- If it is serious, how then might we proactively identify which of our existing assumptions might most need to be revisited?
Examples of defaults being challenged by GenAI
So how exactly is GenAI challenging our defaults, and the assumptions underlying them?
I’ve collected a few below that I think are useful in illustrating the issue.
| Example | Existing assumption | Challenge posed by GenAI | Signal / evidence point |
| Job applications | Job applications provide enough insight and intelligence about an applicant to make a meaningful call as to their relevance of their skills and capabilities | GenAI tools make it easier for people without the necessary relevant expertise or experience to provide a (more) convincing case of why they might suitable for the role | ‘Nearly 73 per cent of Australians are using AI tools in their current job search – well ahead of the 58 per cent global average – and 90 per cent of those admit using it to “exaggerate or lie about their skills,” marketplace vendor Capterra’s survey of 3,000 job seekers has found.’ (ACS) |
| Job interviews | Job interviews provide a good opportunity to test and verify the suitability of an applicant for a position | GenAI tools make it easier for people in non in-person interviews to provide the impression that they meet the relevant criteria for the job | At a minimum, we can expect many applicants to be using GenAI to help them prepare for interviews. In other cases, care may need to be taken to mitigate against the possibility that GenAI is being used more intensively: ‘27 per cent of Australian job seekers admitted using AI to complete a test assignment or skills assessment given them as part of the interview process – with 22 per cent using AI to generate answers for interview questions.’ (ACS) |
| Qualitative research | Recruiting qualitative research participants through crowdsourcing platforms is a good way to test relevant views and experiences when designing or validating service design | GenAI makes it easier for bad faith actors to enrol in such platforms and provide input that might seem valid but is fake | ‘Artificial intelligence (AI)-based large language models (LLMs) have made it easy for bad actors to automatically fill in online forms, including generating meaningful text for open-ended tasks. These technological advances threaten the data quality for studies that use online questionnaires.’ (Frontiers in Robotics and AI) |
| Voice authentication | Individual voice patterns are too hard to replicate and therefor voice authentication is a useful filter for people accessing services and providing sensitive data | GenAI tools now have the capability to replicate a voice pattern with minimal input of the original voice | OpenAI has specifically encouraged “Phasing out voice based authentication as a security measure for accessing bank accounts and other sensitive information” |
| Public submissions | Invitations by the public sector for people to submit information to an inquiry or consultation process will provide valuable insight, even though interest groups might encourage particular types of submissions or provide a self-interested viewpoint | GenAI tools now make it easy for any individual to create text in the style of expert or professional language (i.e. lawyers, accountants, scientists, etc.,) or to put forward a seemingly well thought out argument making the case for any point of view | ‘More than 100 people have used an artificial intelligence (AI) assistant set up by a campaign against the federal government’s proposed misinformation law to write submission letters opposing the bill, in what appears to be a first for the technology.’ (Crikey) |
| Procurement | Procurement processes provide a means for government agencies to rigorously test their criteria and give them a detailed understanding of what they are procuring | GenAI tools have experienced continuing updates, sometimes resulting in significant changes to their capabilities and functionality, outside of traditional product cycles | A look at the OpenAI changelog reveals how many changes have occurred for their product. As these tools are integrated into the enterprise context, traditional procurement processes might not be calibrated for ongoing dynamic updating of functionality. |
Each of these examples helps to illustrate the ways in which generative AI might disrupt some of our deeply held assumptions. Some of these disruptions will likely turn out to be relatively minor, and will self-resolve with minimal alterations to existing processes needed. However, as the transformative effects of GenAI continue to play out, more and more of these disruptions are likely to accumulate, and at least some of them are likely to be significant and force a major shift in how things are done.
A role for an assumptions audit?
One of the things some of us have been playing within the Australian context is the idea of an ‘assumptions audit’ – a systemic process for identifying the relevant assumptions and thinking about how or whether generative AI might meaningfully disrupt them. How might we audit our assumptions to ensure that we can be sensitive to when they may no longer hold true or be as useful as they were?
This document has some pointers and methods for how we can do so. I’m sure there are many others or other things that could be included, and I’d love to hear them.
This assumptions audit has no formal status, but rather is just a mental aid for trying to get to grips with the transformative impact of GenAI so that we can (hopefully) get out in front of any big issues that might arise in public administration. As public servants we need to be thinking about how people using AI might affect our processes, our practices, and our defaults, as well as the important work of thinking how our use of these tools can be responsible and ensure better outcomes for the community.
How do you proactively identify which assumptions are most vulnerable and most need to be revisited?
Many public sector agencies around the world are already starting to grapple with some of these questions, but I’m curious however to see if others have played with something similar to an assumptions audit. The transformative effects of generative AI are likely to reveal themselves for some time yet, as the technology continues to mature and become more capable. How are others challenging their core assumptions on an ongoing basis?