There’s a lot of interest at the moment in the efficacy of funding for “non-traditional” students at universities; first-in-family to get in, students from low income households, under-represented minorities, vulnerable groups. While everyone supports “social mobility”, there’s a wide range of understanding of what this means. And just off-stage, and watching intently, are the Treasury’s accountants, asking patiently how the return on investment for special funding can be measured.

A recent study for the Institute of Fiscal Studies offers an illuminating perspective on this issue, as well as an intriguing counterintuitive; one of the surprises that big data analysis can bring.

In this report Claire Crawford, who is at the University of Warwick, looked at all students with home addresses in England who began at any UK university at the ages of 18 or 19 between 2004 and 2009, with each of these six cohorts comprising between 180 000 and 235 000 students. This is about 80% of all students attending British universities. She tracked their individual attainment from their test results for English, Mathematics and Science from the age of 11, through their GCSE results at age 16, their A-level or equivalent results at age 18, and on to the quality of their university degree. She also took on board a rich mix of socioeconomic indicators (proxies for household income, neighbourhood indicators of relative deprivation and property values, and the educational levels and occupations of people living nearby) and the characteristics of the secondary school that each student had attended: state or private; selectivity; the availability of a Sixth Form College; and school performance based on the government benchmark of the percentage of students that achieve at least 5 GCSEs at grades A*-C at age 16. Data sets for a national-level study of education do not get much bigger than this.

There were three major outcomes from this research, each of which has significant implications.

First, Crawford’s work extends what we already know about the effects of a person’s prior and present life circumstances on their prospects of getting into university and getting a well-paid graduate job. Study after study has shown that where you were born and live, combined with your family’s circumstances, will determine the probability of your getting into university as well as the kind of university that will admit you. This new analysis follows students past admission to first year undergraduate study and through to degree attainment, plotting the likelihood of getting a “good degree” (a first or an upper second). It shows that individuals from different socioeconomic backgrounds face significant differences in their prospects of making it through to the final year of study and attaining the class of degree that they will need for a well-paid graduate job.

For state schools in England, less than 10% from the most advantaged backgrounds leave university within two years without completing their qualifications, 80% complete their degree within five years and nearly 70% graduate with a first or an upper second class degree. In contrast, more than 20% of those from the most deprived backgrounds will drop out, less than 60% will complete within five years and fewer than 40% will attain the coveted “good degree”.

Second, up to 80% of these differences in university-level attainment can be explained by what Crawford calls “human capital”. This is a little confusing, because a good deal of attention has already been given to “social capital”, broadly understood in this context as the attributes of a graduate household, where material and experiential support is more readily available to students. Crawford’s definition of “human capital” is far narrower, and primarily comprises a person’s prior test and examination results; national English and Mathematics testing at Key Stage 2 (approximately age 11, and including some additional testing in Science subjects), Key Stage 4 (GCSEs at age 16) and Key Stage 5 (A-levels or vocational equivalent). Because the attainment of students at state primary and secondary schools is powerfully shaped by the socioeconomic circumstances of families in their catchment areas, this finding again extends what we already know; the school that a person went to prior to admission to university will have an enduring influence on prospects for both degree completion, for the class of degree attained and for subsequent employment opportunities.

Third – and this is the counterintuitive finding from this massive data set – when all else is held constant, students from poor performing schools do better than those from high performing schools: “when comparing pupils from similar backgrounds with the same prior attainment, those from the best-performing state schools are now 2 percentage points more likely to drop-out, 2 percentage points less likely to complete their degree and 5.2 percentage points less likely to graduate with a first or 2:1”.

Why is this? Here, we are at the limits of what the statistics are able to tell us, however large the overall data set. But reasonable inferences can be made. Remember that this category of higher-achieving students has already overcome significant barriers to get into university in the first place. Given the wide differences in access to high-ranking secondary schools and to all universities by different socioeconomic groups, this will be a relatively small sub-group of the study as a whole. They are also likely to be special; people who will, for example, have been consistently ranked at the top of their class in school. It is certainly worth finding out more about them; their stories will provide valuable lessons about what universities can do to close the attainment gap that is so evident in terms of overall socio-economic background.

Claire Crawford is primarily interested in her counter-intuitive finding as a way for universities to refine their existing admission policies. This, though, is to assume that universities should seek to narrow the attainment gap between different socioeconomic groups by taking fewer students from backgrounds for which the statistical data predict higher drop out rates and lower degree outcomes.

It’s useful to test this reasoning with a simple thought experiment. Say that all universities were able to use this sort of massive, longitudinal data to select, at admission, students who would all complete their degrees, and all achieve firsts or upper seconds. Say also that all state secondary schools providing universities with their students accomplished a similar feat, as did all primary schools from which these secondary schools recruit. The overall outcome would be a clear and exclusive pathway into university from Key Stage 2 at the age of 11, and a tight and almost impenetrable relationship between the circumstances of the household in which a person is born, their subsequent education, and their prospects in life.

That, though, is not what we profess to want from universities. Every major political party now embraces social mobility as a good thing and social mobility is, by definition, the inter-generational transformation in prospects that would be impossible if this thought experiment were to become a reality. Given this, there is a more important implication from this piece of work. Here’s how Crawford summarizes this:

“The relatively crude measures of university experience at our disposal do not help to reduce the remaining difference in degree outcomes by socio-economic background very much further. Even the addition of course fixed effects – effectively comparing students on the same courses – plus an indicator for whether the student lived and studied in the same region (and, in the case of drop-out, indicators for type of qualification and mode of study), does not reduce the difference in degree outcomes between young people from different socio-economic backgrounds by very much”.

Specifically, when the attainment of students after admission to university is controlled in terms of the academic programme of study, those with similar Key Stage 5 results but from the least privileged backgrounds are still 3.4% more likely to drop out, 5.3% less likely to graduate and 3.7% less likely to attain a “good degree”.

Given that this mirrors the disparities that characterize educational attainment throughout a person’s life, it raises an obvious question: does a university education do anything to counter prior disadvantage? Does attending a university add value?

We all, of course, claim that it does, and the cause of Widening Participation (WP) is a shibboleth for the sector. The problem up to now has been that we cannot measure in any systematic way the benefits of WP; a matter of some interest to the number-crunchers in the Treasury. And the problem now is that new research such as this suggests that, seen in terms of degree attainment, there is little apparent return on the investment in WP funding.

And this is where Claire Crawford’s methodology could have wide application. In comparing groups of students within the same cohort of study but from different backgrounds, every university has a way of measuring the value that it adds, and the sector as a whole a way of demonstrating empirically the returns for public investment in Widening Participation. This would be far more revealing than the inanities of the existing league tables, or the superficialities of the National Student Survey. Given the cuts that are rumored for this year’s Comprehensive Spending Review, this could be a timely priority to adopt.

Source

Claire Crawford, “Socio-economic differences in university outcomes in the UK: drop-out, degree completion and degree class”. IFS Working Paper W14/31. http://www.ifs.org.uk/publications/7420