Anyone interested in the future of higher education would do well to read the NCCPE’s recent report on the social value of universities. Paul Manners, the NCCPE’s Director, sets the context for the report when he quotes research for Universities UK in 2010 which suggested that less than one in five people in the UK recognise the wider impacts that universities have on society. This alarming statistic goes some way to explaining the relative ease with which the government has introduced its controversial reforms. Where the public has shown concern it has been focused almost exclusively on the level of fees rather than on the implications for the sector and its contribution to the life and health of the nation.
The contrast with the NHS or the BBC could not be greater. These two institutions are hardly without their critics but, such is the public’s basic regard for them, politicians tread carefully when proposing their reform. Against this backdrop, there is an urgent need for universities both collectively and individually to find a way of talking about their value that engages effectively with the public and thence politicians.
The report is valuable not least for its clear and balanced assessment of the different approaches to assessing and measuring social value. There is an extensive literature here and the authors provide a useful guide to anyone wishing to understand this concept better. The main point of their report is to set out the case for their chosen method of assessing and reporting social value, the “SMEV” or “socially modified economic valuation” which aims to put a hard, numeric value on the contribution that universities make to society. We endorse this attempt at finding a rigorous and quantitative approach to assessing social value and find the authors’ arguments for doing so compelling. However, our experience at the BBC, where we developed and applied the Public Value Test, suggests that caution is in order and that the SMEV, as described, may not be as workable as the authors believe.
We also endorse their view that valuable though the SMEV approach is it needs to be located within a broader debate about how the sector creates social value. From this a framework can be created which sets out the sector’s purposes and what can be expected of it.
The SMEV model involves the following steps:
- Identifying all university outputs
- Quantifying the outputs
- Economic pricing of the outputs
- Finally, adding “social weights” to the results of the economic valuation to produce the SMEV
So, for instance, take the hypothetical example of an open university lecture which was free to those attending. The authors estimate, through a technique known as “shadow pricing”, that the economic value of the lecture would be some £357. To assess the social value of the lecture the authors turn to HM Treasury’s Green Book for guidance. The Green Book provides guideline “weights” for economic analysis which reflect what the government deems to be socially useful. In this hypothetical example the lecture succeeded in attracting a disproportionately high number of students from less advantaged backgrounds which, in so far as it is in line with government policy, produces an upweighting of the value of the lecture to between £405 and £442.
While we think the SMEV is an interesting and potentially worthwhile approach we question whether it is as workable as described in the report. The authors point to a pilot study at Strathclyde University which succeeded in identifying 220 separate outputs. However, this marks only the beginning of what is potentially a highly complex process, not least because of the difficulty of identifying the intentions behind each output. For instance, in the hypothetical example of the open lecture, it is quite conceivable such a lecture would be designed and delivered with more than one socially desirable output in mind.
We encountered precisely this problem at the BBC when trying to assess the impact of the BBC’s programming and concluded that, much as we would have liked to demonstrate its delivery of socially valuable goals, the sheer complexity of identifying the range of intentions at the point of commissioning and then assessing their impact at the point of delivery was just too great. While it might be possible to manage the complexity of identifying outputs at the level of individual institutions it would be a challenge of a different order at the sectoral level.
These reservations aside, we think the SMEV could in principle help support better informed decision-making by university administrators and civil servants. As the report’s authors say, though, the sector needs rather more than this. They refer to the need for a “narrative of value” as well as a hard, quantitative valuation precisely because numbers alone cannot “‘catch the imagination’ or give insight into the richness and diversity of higher education.” We strongly agree with this and their contention that the SMEV would need to work within a wider context that would see agreement as to what value society expects the sector to create.
To do this would mean engaging with the question of outcomes and it is here that the sector has an opportunity to seize the initiative and begin to set the terms on which its effectiveness is judged.
The report’s authors rightly point out that outcomes – e.g. a healthier, better educated society – are the products of variables that are outwith the control of any one institution or grouping of them. For this reason they limit the SMEV to a consideration of outputs. While we can understand this from a methodological point of view, we nonetheless believe they lose sight of outcomes rather too easily. The SMEV might reasonably look for evidence of how outputs are supportive of outcomes, albeit not at the expense of the hardness of the data provided by outputs.
An understanding of desired outcomes is critical to securing legitimacy in the eyes of the wider public and those with the power to grant or withhold funds. Very few will be interested in the number of research papers or open lectures a university produces – however “weighted”. But a discussion about the socially beneficial outcomes that an institution or sector is trying to promote through its various activities has the potential to engage and to do so in effective ways.
This was our experience at the BBC. In developing the Public Value Test and the attendant Public Value performance framework, we deliberately eschewed a focus on any single metric as indicative of success or otherwise and instead emphasised the need for judgment, albeit based on careful and rigorously researched evidence.
The focus of our work was on the BBC’s six public purposes and an engagement with the public about how far the BBC was meeting them in its existing or proposed services. What struck us forcibly was the ease with which the public engaged with the idea of there being products and services which had high social value and were therefore worthy of public funding but in which most individuals had little personal interest. One of the best examples of this is current affairs programming which, relatively speaking, attracts few viewers and listeners but which rates very highly as something which the BBC should continue to provide in the interests of a well-functioning democracy.
There is no doubt that the BBC has benefited greatly from adopting a public value framework. It was developed on the basis of both audience research and staff engagement and government was closely consulted in the process. The result is a framework that forms an agreed basis for engagement with government and stakeholders. It is one that is owned and understood by staff and provides the means for a continuing dialogue with the public as to how well the Corporation is meeting its purposes and where it needs to improve. The Public Value Test has also done much to address what were acute commercial concerns about the BBC’s development of new services, particularly in nascent markets.
Unlike the BBC, our universities are in the unenviable position of creating value both economic and social of which the public seems heedless. There is no simple solution to this problem, no magic bullet that can illustrate their value. But the sector as a whole and the institutions within it can change this situation if they begin to engage seriously with the question of fundamental purposes and use this as the basis for a dialogue with the public. There is much to be said for a rigorous, evidence-based approach such as that proposed in the NCCPE paper but only as part of a deeper, more far-reaching discussion about what our universities think they are for.
Image courtesy Loughborough University Library.
