[Originally published on the Australian Government Public Sector Innovation Network under a Creative Commons 3.0 BY AU licence]
The other day I was lucky enough to attend a talk at the Canberra Evaluation Forum given by Dr Mark Matthews, Executive Director of the HC Coombs Policy Forum based at ANU.
His talk was titled ‘Evaluations and Innovation: Why improvements to dealing with uncertainty and risk in evaluation are a key enabler of public innovation’. The main message of his talk was that ‘if we are serious about building a more innovative public sector we need to get much better at building uncertainty and risk into our assessment and evaluation frameworks’. 1
In his talk, Dr Matthews reflected on the current use by government of the output-outcome financial framework, a system he thinks is heavily based on quantifying known performance parameters (input and outputs), which results in either ignoring or making subjective assumptions on those parameters where quantification is not easily achieved.
However, innovation by definition involves coping with and addressing uncertainties. Dr Matthews argues that when we seek to innovate we aim to translate uncertainty and ignorance into risk, and then attempt to transform risk into a manageable risk profile.
How then can we reconcile these two seemingly incongruent systems?
Dr Matthews believes that this is inherently a ‘wicked problem’ and that there are no easy answers. That being said, he suggests one of the ways we can address this issue is by using more sophisticated risk-aware approaches (as opposed to risk-averse approaches) such as:
-
- the robust use of stage-gating approaches to developing, evaluating and selecting innovative policy solutions;
-
- adopting methods that explicitly assess and measure relative investment risk in the public sector innovation process (as part of a more formal approach to managing investments in public sector innovation), and;
-
- adoption of ‘rapid spiral development’ methods derived from the software, space and military innovation environments – methods via which the public sector innovation process can become more effective by integrating far more frequent improvement cycles with incremental programme roll-out (rather than relying on more infrequent programme reviews/evaluation stages several years after programme roll-out).
Dr Matthews has kindly provided his presentation along with a paper he prepared for the Federation of Australian Scientific and Technological Societies ‘Giving Preparedness a Central Role in Science and Innovation Policy’.2
What do people think? Do you agree with Dr Matthews’ conclusions about the current reporting framework and his proposed solutions? Has anybody used some of the risk-aware approaches suggested by Dr Matthews and what were the results?
A big thank you to Dr Mark Matthews for his permission to use his presentation and paper on the blog.
- Presentation given to the Canberra Evaluation Forum on 19 May 2011; by Dr Mark Matthews – HC Coombs Policy Forum – Australian National University – Canberra.
- The presentation is not covered by the Creative Commons licence or Commonwealth Copyright. Dr Mark Matthews has requested that the use of any part of his presentation or papers be cited properly.