Crawford School of Public Policy

A ‘post-truth’ world offers fertile ground for big claims, made boldly. Real impact has different measures, Helen Sullivan writes.

How do we judge the value of universities? Nobel prizes won? Numbers of world leaders and captains of industry educated? All of these can illustrate the kinds of impact a university education can have.

In a time of constrained public finances, politicians must make difficult decisions about allocating resources. Focusing on impact appears to offer a way of doing that and as a consequence universities all over the world are now scrambling to try and demonstrate and quantify the difference they make to the economy, society, and culture. Public policy schools – which are by nature designed to have an impact – have always experienced this pressure to account for their activities. However, these schools frequently find themselves criticised for not meeting the needs of policymakers, failing to bridge the gap between policy and academia, and for publishing in academic journals that no-one reads on issues that don’t address Australia’s concerns.

Putting in place a system of impact assessment that requires universities (and public policy schools) to account for the difference their research has made to the world is an entirely rational response.

It is also entirely the wrong response.

There is no doubt that the current system of academic promotion and reward distorts research decisions by privileging publication in highly specialised research journals. But establishing a parallel system for impact will not only be costly, but also self-defeating.

What such a system will do, and indeed already is doing, is reward those scholars whose work is amenable to measurable impact, and those who are adept at self-promotion through blogging, tweeting, or public engagement. I have nothing against those things. Social media is an excellent way of communicating with busy policymakers and the public. But in our time-poor, media-hungry environment, there is an increasing risk that what gets heard is what comes in the most appealing package, not what is necessarily the best-researched idea. The era of the soundbite is not sympathetic to the cautious assessment of much social science. A ‘post-truth’ world offers fertile ground for big claims, made boldly.

What is needed is another course guided by principles of humility, listening, and evidence.

We all know the stories of scholars who single-handedly transformed policy. But these stories don’t reflect the bulk of how public policy is made nor how academics contribute. There are very few ‘eureka’ moments in the social sciences. Rather, influence and impact comes from what Carol Weiss termed ‘enlightenment’ – gradual change as the consequence of many scholarly and other inputs.

This kind of interaction – painstaking brushstrokes over time rather than bold sweeps – runs counter to the current narrative. Yet given the limits to our knowledge exposed by events such as the global financial crisis or the rise of Trump, it is more appropriate. As social scientists, it perhaps behoves us all to be rather more humble in our claims and assertions. This is not the same thing as a lack of ambition. We remain committed to addressing the major policy problems of our time. But we do so without hubris.

The good news is this sort of deep engagement is happening all the time in public policy schools. It is what we do. These interactions with policymakers are necessarily relational, developed over time, and based on shared trust and commitment to public policy. They require academics to really listen to the needs and fears of policymakers. Developing listening skills is essential if we are to improve our impact.

The world’s many policy challenges demand that we who are privileged to work in universities bring the best evidence to bear on the problems facing society. This evidence will rarely provide the ‘silver bullet’ that many politicians want. But it will be defensible and it will help build towards the solutions we need.

How do you construct an impact framework that captures this? Not easily. But maybe a place to start is to look at how public policy schools do what they do rather than focusing solely on the outcome. Supporting practices that facilitate engaged scholarship that is marked by humility, listening and evidence would be a good way to value universities.

This piece was originally published in The Australian’s Higher Education section: https://t.co/3pSAK11KEk. It is also published by Policy Forum.net, the website of the Asia and the Pacific Policy Society and Crawford School. https://www.policyforum.net/how-to-measure-impact/