In her excoriating response to the audit culture of the contemporary university, Marina Warner turns to Dante’s Purgatory and his vision of Earthly Paradise. Here, the pilgrim finds the river Eünoè, the “good mind”. Warner asks whether the real value of education can ever be measured. For her and many others, Purgatory is the audit culture of performance indicators, the superimposition of “the imagery of the market on the idea of a university – through ‘targets’, ‘benchmarks’, time-charts, league tables, ‘vision statements’, ‘content providers’”. So is there any hope for a reasonable consensus about learning analytics, for a way forward that respects the individuality and breadth of the individual’s needs and gains and that also aggregates such information to make an informed and persuasive case for a progressive future for higher education?

A friend reminded me of Warner’s essay, first published in March, while I was reading about the apparent collapse of the OECD’s Ahelo project, long in the making, which has set out to produce a universal measure of the value that students get from universities. Warner was writing before Britain’s May election, when the Teaching Excellence Framework was a throwaway line in the Conservative Party’s manifesto. The TEF has since become a reality and, presumably, the elixir of the River Eünoè still further away. Is it inevitable that common ground is impossible, that we cannot find a way of measuring the core value of education without siding with the sinners and wandering for ever in Purgatory?

Warner writes that “something has gone wrong with the way the universities are being run. Above all, I have learned that not everything that is valuable can be measured”. In contrast, the Assessment of Higher Education Learning Outcomes (Ahelo) project set out to measure of what students across the world “know and can do” at graduation. The ultimate goal of Ahelo has much in common with Warner’s commitment to widening fair access to education as a public good. Ahelo’s objective is to “transform the established hierarchy in world higher education” through a set of measurements of learning gain that cancel out the distorting effects of previous advantage. The idea is that every prospective student should know the real value that a university can offer them before making a choice and committing considerable resources to the educational journey ahead.

But despite earlier enthusiasm, significant interest groups seem to be backing off. Despite earlier endorsement by the UK’s minister responsible for universities, Britain has declined to be part of Ahelo and the American Council on Education and Universities Canada have both expressed “grave reservations”. Ahelo’s advocates have decried this opposition as that of an “established oligopoly”, “ legacy providers, who are very dominant in the research space, trying to prevent new information about education coming to light.” But is this really a conspiracy of the privileged to defend their territory, or are there flaws in the Ahelo approach?

In these kinds of debates, two separate issues tend to get conflated.

One is the question of fairness. In an unequal world, that is becoming more differentiated by wealth and access to prior education opportunities, higher ranked universities enrol a disproportionate number of students from better-off backgrounds who as a result have improved opportunities in life, perpetuating and accelerating the cycle of economic differentiation. Looked at from this angle, Ahelo is the Robin Hood of Higher Education, standing against the elite and their interests.

The second issue is that of measurement. Ahelo is sometimes presented as if it has set out to measure all aspects of university life. But its feasibility study – and its case for future commitment – is based on two disciplines, Engineering and Economics. This, coupled with the well-known difficulties in defining and measuring “generic skills”, make exercises like Ahelo seem superficial and reductionist. Marina Warner’s point is that the real value of education must be much more than this. Seen from this angle, she is the champion of truly transformative education.

Would Marina Warner sit down with the OECD to design a revised approach to measuring the real value of the university experience? This seems improbable, although they have more in common than either may realize. Here is Warner’s clear and unequivocal vision for a future world: “I believe education at every age and level is an unqualified good, unassailably beneficial to the individual and to society and the world. I believe it is as important an indicator of a society’s state of health as nutrition and housing”. Ahelo’s advocates must surely agree.

So where do the differences start? For Warner, and fellow critics of the audit culture of contemporary university, the true gains from a university education are in what could be called the interiority of the student – in their perception and understanding of the world, in the sense and application of their own abilities. The multitude of such individual gains comprises the aggregate social good. But for the OECD and the auditors, the social and economic benefits are only evident at the aggregate level, once the “noise” of class, nationality, culture and personality has been discarded. Individuality is stripped away and the student is reduced to a data point and a string of associated parameters.

Does this matter? Are these not two ends of the same telescope? Yes, it does. The problem for Ahelo is not a conspiracy by the oligarchy, although there are certainly vested interests that would not want to see the consequences of universal fairness in access to university. The deeper problem is in the assumption of comparability.

Engineering and Economics are just two of many disciplines. Their curricula are constantly changing, as they should do. Trying to find a common denominator across nations, languages, political preferences and university systems is to risk a race to the bottom and to superficial rendering of the richness of any educational experience. If our learning analytics of the future are like this they will contribute little, if anything, to claiming back higher education as a progressive public good.

Here is Warner on the experience of learning:

You can feel the whirr and hum of thought: it feels woven of reciprocity, willing, ambition, the impulse to translate fugitive thoughts into communication with others. The same can happen with an audience at a concert, with readers in a library, or with visitors looking at pictures in a gallery.

This is for Creative Writing. But it could equally be for the excitement of a Physics laboratory, or for teaching Nursing, or History or – indeed – for inducing a passion for Accounting.

Building an understanding of the value of education from the individual outwards does not, though, cancel out the obligation for transparency. Students and their families are making a huge investment in going to a university today. Transparency is required to get beyond the sham of elitism – the claim that a university must be good because it claims to be. And here, the Ahelo pilot project has a lot to offer. For example, its “Value Added Analysis”, calibrates students’ results by taking into account both differences in the contexts in which universities operate and differences in their students’ prior academic achievements. These adjusted results, appropriately aggregated, are a much better measure of what a particular universities gives than the conventional grades that are fed into newspaper league tables.

What’s the way forward for Ahelo and for similar projects concerned with learning analytics. Firstly, abandon conspiracy theories. Universities at the top of the league tables are too busy competing with one another to organize an oligarchy. Secondly, don’t start by trying to change the world; with attempts to define universal systems of measurement that have a negligible chance of achieving consensus. Rather start with the rich experience of small cohorts of students, building these up into well-informed profiles of natural groups within and across clusters of institutions.

Approached in this way, learning analytics can both deepen our understanding of the value that education brings and retain the poetics of knowledge, the sort of inspiration that Marina Warner finds in Dante:

I choose to understand this elixir, Eünoè, as good, active knowledge, not only retrospective memory; the kind of knowledge that education passes on and cultivates, that grounds further learning, develops the ability to do something well, to think clearly and freely. Doing something well requires attentiveness – the attention of others who are helping you, teaching you – from doing up your shoelaces when you first go to school to identifying and fostering your aptitudes later. Good knowledge requires inquiry on your part, your absorbed attention – and to be attentive to the point of self-forgetfulness also lightens discontents. So curiosity must be met by responsiveness – by listening, not silence. Good knowledge is also good enough knowledge: the endless insistence on excellence and competitiveness drives individual disappointment. Above all, Eünoè – good knowledge – flows from the heart of education, and will add to the sum of pleasure and mutual illumination in the world.

**

AHELO “Feasibility Study. Volume 3. Further Insights”. 2013. http://www.oecd.org/edu/skills-beyond-school/AHELOFSReportVolume3.pdf

John Morgan “World’s university ‘oligopoly’ accused of blocking OECD bid to judge learning quality”. Times Higher Education 6 August 2015. https://www.timeshighereducation.co.uk/news/world%E2%80%99s-university-%E2%80%98oligopoly%E2%80%99-accused-blocking-oecd-bid-judge-learning-quality

OECD “Testing student and university performance globally: OECD’s AHELO”. http://www.oecd.org/edu/skills-beyond-school/testingstudentanduniversityperformancegloballyoecdsahelo.htm

Marina Warner   “Learning my lesson”. London Review of Books 19 March 2015 http://www.lrb.co.uk/v37/n06/marina-warner/learning-my-lesson