Identifying innovation success – knowing what failure looks like?

[Originally published on Australian Government DesignGov under a Creative Commons 3.0 BY AU licence]

How do we know if an innovation has been successful? In this post I’d like to test with you a possible approach to judging success and see whether it might offer assistance in applying innovation within the public sector.

At a recent workshop I attended there was discussion about an experimental project we were collectively involved in and someone raised the question ‘how will we know if it was successful?’

This is a common question, with suggested responses including visualizing success, thinking about how the system would be different if the initiative was successful, what would have changed and how. This is in addition to standard measures common to project management including Key Performance Indicators, project evaluations, and gauging stakeholder views.

This of course is very reasonable. After all, why would you attempt something if you don’t know what it is you’re trying to achieve?

But as I have had more to do with innovation in the public sector over the years, I have been wondering if another view might also offer value. While still very much preliminary thinking, that view revolves around answering the question ‘what will failure look like?’

I put this forward as a possible approach (and please let me know if you think I have gotten this wrong) for a number of reasons.

First, a lot of the innovation process is problem-driven. It is about making a problem go away (usually phrased as ‘solving’ a problem). But there are many possible solutions to any problem, some of which are better than others, some worse. There is very rarely just one answer. Success can take many forms.

Failure on the other hand is usually more specific – we know what the problem is now, and we will usually be able to tell whether an innovation has made the problem less bad.

Second, in the public sector, there is usually much stronger agreement around there being a ‘problem’, than there is around what success will look like. Different groups and stakeholders may agree there is a problem, but have very different ideas as to what they think will be a satisfactory result.

A focus on agreeing what failure will look like might leave open the possibility to try something and see, rather than ruling it out beforehand because it does not fit with one group’s preconception of what success will/should look like.

Third, the larger or more disruptive the scale of an innovation, the more likely it will be materially affecting a dynamic system. If you introduce a major innovation, other things will change and respond in turn (as opposed to a static system where you could make your change and everything else would stay as it was before). The bigger the change, the more likely that it will be difficult to predict what success will look like, because everything else will start to change as well. Usually though, the notion of failure will be more stable.

Fourth, a focus on a vision of success may inhibit any ability to take advantage of serendipity or advantageous unexpected discoveries. If you know that you will be measured on a set of performance indicators about what you should have achieved, you are less likely to spare any effort for additional things that may be valuable, but could jeopardise what you are ‘meant’ to be doing, even if it would offer a better outcome overall.

Fifth, the public sector can attune public servants more to avoiding failure than to achieving success. Contrary to some elements of public opinion, failure in the public sector usually has direct and immediate consequence (whether it be reputational or something more significant) whereas success is more diffuse. Success requires many people to work together and contribute – it is usually more easily to attribute responsibility for failure than it is for success.

Sixth, innovation and design are increasingly about iteration, rapid prototyping and fast failure. It is important to quickly see whether something works or not. But the definition of working is again more about what seems promising than it is about perfecting the solution then and there. It is easier to tell whether a prototype has failed than it is to know whether it will be successful.

Now this might seem a pessimistic take on innovation/the public sector. But I actually think it is liberating. It allows us to concentrate on what we are trying to fix and find our way, rather than pre-defining what it is we are trying to achieve. As design teaches us, the process of making the problem go away will be an iterative and exploratory one, with many unexpected twists and turns and insights that offer us new possibilities and learnings that we could not conceive of previously, and that can be applied in new settings and with new problems. A focus on success might actually limit the chance of achieving success.

What do you think? Is a discussion around ‘how will we know if it has failed’ a useful alternative to articulating ‘how will we know we have succeeded’?

(Of course there is a lot more nuance and exceptions to some of these statements than can be explored in a blog post. I am hoping though that the general approach may offer value when the standard approach of identifying success may be problematic.)