In The Impact Agenda, Katherine E. Smith, Justyna Bandola-Gill, Nasar Meer, Ellen Stewart and Richard Watermeyer bring together research about the impact agenda and its policies into one critical discussion to highlight why it creates the controversies, consequences and challenges of the book’s subtitle. Calling on the UK academic community to seize the opportunity to reshape the impact agenda in more positive and sustainable ways, the book’s valuable synthesis and analysis is worthwhile reading for anyone interested in this topic, recommends Jane Tinkler.
The Impact Agenda: Controversies, Consequences and Challenges. Katherine E. Smith, Justyna Bandola-Gill, Nasar Meer, Ellen Stewart and Richard Watermeyer. Policy Press. 2020.
The news that the 2021 Research Excellence Framework (REF) is being delayed due to the COVID-19 pandemic has caused relief and frustration in equal measure. This is perhaps appropriate, given the extent that the ‘impact agenda’ provokes such a variety of reactions.
For a relatively change-averse sector, the effects of the impact agenda (defined in this book as both the REF with its impact element and the Pathways to Impact statements required by research councils) have been considerable and wide-ranging. The Impact Agenda – by authors Katherine E. Smith, Justyna Bandola-Gill, Nasar Meer, Ellen Stewart and Richard Watermeyer – seeks to bring together research about these policies into one critical discussion so as to highlight quite why it creates the controversies, consequences and challenges of the book’s subtitle. In doing so, the authors have undertaken three useful tasks that make reading it worth the time of anyone interested in this subject.
Firstly, they have synthesised the critiques and areas of concern about the impact agenda. The book looks at the rise of the impact agenda, through the policies that informed the first REF in 2014 along with the evaluations that have taken place since, as well as the Stern Review and plans for the 2021 exercise. The discussion is necessarily UK-focused, but the authors rightly argue that there are a number of other countries watching the UK’s impact experiment very closely. They hope by drawing together concerns, as well as any lessons more recent research has unearthed, the book’s findings will feed into wider discussion about impact in the UK and also in the various countries where these forms of assessment are being considered.
The authors’ second task has been to undertake a synthesis of literature from across disciplines (although predominantly social science as that is where much of the literature on policy engagement has focused). The book feels to have been a long time in the making. The rich qualitative data used throughout has been collected as part of various projects over more than fifteen years. This over-time synthesis is useful as there is no single discipline of ‘impact studies’, so the impact literature is drawn from different disciplinary or theoretical approaches and some focuses on the experience of individual academics. It can be difficult to look across this in-depth but small scale and piecemeal evidence and take a wider view. Laudably, this is exactly what this book does.
As an example, one chapter looks at how the impact agenda is seen by different disciplines. As might be expected, STEM (science, technology, engineering and mathematics) literature sees the target audiences for research use, what REF panels most value and how impact is evidenced differently to SHAPE subjects (social science/arts and humanities). However, the book also identifies four themes where there is consensus in discussions: that there is a difference between ‘genuine’ impacts and those put forward for the REF; that the impact agenda is part of increased monitoring and metricisation of academic life; that impact is more complex and collective than can be shown or valued in the REF; and that there are important differences between conceptual and instrumental impact.
All disciplines seem to agree that the impact agenda had led to changes in academic practice. One change that might be seen as positive is that collaborative or co-produced research has increased, as we assume was hoped by the architects of the REF: ‘Collaborative research is one of [the] fastest growing sources of research income in UK universities’ (HEFCE 2016). The academics involved in impact projects that the authors talked to felt their research benefitted from a wider range of partners being involved. However, critics argue this has threatened academic autonomy and critical scholarship. Conversely though, research that seeks to engage with the public as a partner can be seen as particularly problematic. Although research by the National Co-ordinating Centre for Public Engagement (2017) found that REF2014 impact cases that involved public engagement did not score lower than other cases, there is still a concern that it is seen as a ‘risky’ choice for case studies and therefore support for this type of research is more difficult to obtain.
Finally, the third task the authors have undertaken is to highlight the views on the impact agenda from those working alongside, or outside, academia. One chapter outlines evidence from those working in knowledge exchange (KE) organisations. This evidence falls into two broad categories: how the impact agenda is an enabler for KE organisations, and the ways in which it is a challenge. As might be expected, these voices gave the strongest support for the impact agenda as engagement is a core part of their work. But this group also discussed the contradictory incentives they saw within the impact agenda and an evidence base that they saw as calling into question some of the assumptions behind impact policies. Another chapter draws on interviews with the non-academic user assessors on the REF panels. The authors argue that the REF contributed to the view of academics as ‘experts’ and communities as purely beneficiaries of research. As the REF encouraged academics to claim the benefit from any co-produced work, it led to the promotion of academic effort at the expense of the work of partners.
Since the final book draft was completed, two major reports have been published on the research environment in UK Higher Education. These reports highlighted that bullying and harassment, overwork and undue pressures are sadly part of academic life for many. A Wellcome Trust report from January 2020 found the REF was felt to have a negative impact on creativity, increasing pressure to respond to metrics rather than research excellence, as well as damaging community and collective work. This was followed by a UKRI report on research integrity from June 2020 that found that 32 per cent of survey respondents thought that the REF potentially had a negative impact on research integrity, 27 per cent a positive effect, with 17 per cent viewing it as both positive and negative. Participants seemed to feel these negative effects came as much from how institutions responded to the REF as from the REF itself. This aspect – how the impact agenda has been used and abused by universities – is one the book does not cover.
Improving the wider research environment is the focus of the book’s conclusion, which outlines six alternative approaches that could be used to develop, restrict or replace the impact agenda. The most persuasive of these – rewarding impactful environments (rather than individual achievements) and assessing open and engaged universities (that are recognised as ‘anchor’ institutions in their local area) on a wider range of activities – makes even more sense during COVID-19 when HE bodies are having to urgently show their value to UK society and the economy in a bid to secure financial support. The book ends by calling on the UK’s academic community to seize this window of opportunity to reshape the impact agenda in more positive and sustainable ways.
Note: This review gives the views of the author, and not the position of the LSE Review of Books blog, or of the London School of Economics.
Image Credit: .
1 Comments