Immutable: Why exams are here to stay.

Why do you come here, and why do you stick around?

Morrissey, Suedehead

Why do ideas stick around? For the past couple of years or so, I’ve spent a great deal of time asking the opposite, preoccupied with the faddishness of educational change. In our book on the subject, we chose to portray this through a cover illustration which shows a scrapyard of cars, each representing a ‘good idea’ that has been discarded onto the junk pile of Next Big Things.

But, whilst cars, like educational fads, have a relatively short shelf-life, the existence of cars as the dominant form of modern transportation persists. Why does this form of transportation grow ever-more popular? What differentiates between the ‘good ideas’ (and the innovation they spawn) that last and those that are fleeting?

In education, we may find some equivalence of the motor car in the examination system. Forever the focus of reform and the subject of attack, examinations have nonetheless stuck around for decades – indeed prospered – as the full-stop at the end of compulsory education.

The essayist Nassim Nicolas Taleb argues that the best ideas are those that last; that longevity is the ultimate indicator of quality. The best ideas, he suggests, are those that are ‘anti-fragile’, by which he means that they grow through being placed under continued stress, rather than break through repeated exposure. There may be some truth in Taleb’s claim but I find it hard to reconcile with my chosen examples. Cars (made possible by the invention of the combustion engine) meet the test of longevity and are anti-fragile in that they have proliferated despite various attempts to limit their use: by taxation, restrictions on road building, and incentives to use alternative forms of transportation. But can we claim that a technology which is contributing so significantly to resource depletion and environmental destruction is a ‘good idea’? Critics of the examination system may similarly point to harmful unintended consequences of an innovation that satisfies many needs, but at a great cost. The passing of time may be an indicator of anti-fragility, but this is not a reliable proxy for goodness or worthiness. My instinct is that ideas stick around for reasons other than their intrinsic quality.

What I find frustrating regarding the debate about the examinations system, particularly the contributions made by those who would have it dismantled or substantially reformed, is that there is little consideration given as to why it is so resistant to such attempts. It must be persistent for a reason. I am sure that these critics would agree that the systems’ longevity is not necessarily an indicator of its goodness, but without an understanding of the reasons for its continued existence their attempts at reform will be futile. Those who champion examinations make a similar mistake in arguing that the system is the best imperfect solution we have to a range of problems, without realising that the argument won’t be won by appealing to conceptions of goodness. If they want examinations to remain largely as they are, their time would better be spent understanding, and then investing in, the mechanisms that propagate this innovation.

In an attempt to inform this debate, I will consider two alternative theories for why some ideas stick around before illustrating these with reference to the examination system.

Innovations that become systemically fundamental

Have you seen the internet meme of the tree that has grown around the bicycle? Presumably, the bike was once rested against a sapling but, as the tree has grown, has been lifted and partially swallowed until it has become a metallic branch – indistinguishable but inseparable from its organic host.

Complex systems behave organically in that they evolve around alien bodies. If an innovation sticks around for long enough, we can expect the wider system to adapt to incorporate it, with the effect of propagating its continued existence. Eventually, the innovation becomes irreversible as its removal comes at too great a cost to the system. In some respects, the system becomes dependent on the existence of the innovation; perhaps it is even fundamental to its effective operation.

We can observe this effect in relation to our car example. Most obviously, the world’s infrastructure has been fundamentally shaped by the combustion engine. The road network and fuel pipelines which are required for the value of this form of transport to be realised have shaped our landscape and economy. So too have institutions evolved such that many derive their power and legitimacy from the continued existence of the motor car. We can imagine a ‘better’ world in which other forms of transport are dominant perhaps, but our only viable option is to innovate in line with the momentum of systemic change. Our best bet is to develop better cars, not to abandon them.

The examination system has become systemically fundamental both in terms of infrastructure (the way the system has configured around examinations) and the interest groups which gain their power and legitimacy from it.

Our educational system, and indeed the wider economy, requires that education perform a sorting function. We may not feel comfortable with this, but it is nonetheless the case. Before the introduction of universal schooling and, subsequently, universal examinations at age 16, we found other ways of doing this. Contemporary sorting mechanisms have evolved to incorporate examination results. For example, we require that students achieve certain grades at GCSE to be able to access A Level and other level 3 qualifications post-16. A sorting mechanism at age 16 will be necessary for as long as there are alternative pathways post-16. There are, of course, other ways of allocating students to programmes of study. Rather than universal qualifications at 16, we could introduce entry tests for particular pathways. However, these will still result in those that ‘pass’ and those that ‘fail’. The only alternative to a sorting mechanism at age 16 is that we provide the same education for all students up until the age of 18. The question therefore is not whether we sort but when and how.

Educational idealists (such as Michael Rosen) are quick to criticise the current system which, he alleges, brand a fixed proportion as ‘failures’, but fail to offer a viable alternative sorting mechanism – or to argue for the one-size-fits-all education post-16 which would also inevitably be disadvantageous for certain groups of students. It is simpler to ignore the systemic realities and make emotive appeals. Other more informed and realistic commentators (see here) observe the harmful effects on some students of the current system and call for evolution, not revolution (such as this critique of the effect of meritocratic ideals on students with special educational needs). The former long for a better world, the latter realise that we should realistically work towards a better car.

The power and legitimacy of various institutions, groups and individuals has also come to depend on the examinations system. The ‘standards agenda’ has been enabled by universal examinations and has become central to policy-making. Examinations are the third leg of the standards stool, alongside Ofsted and academisation. It is hard to imagine politicians giving up the ability to infer the quality of individual schools through simple metrics, and to rank them accordingly, no matter how misleading this may be. Examinations have even become the mechanism by which standards are improved, the assertion being that ‘tougher’ exams incentivise ambitious curricular, higher teacher expectations and more learned students. There is evidence to suggest that this mechanism can be effective, given the right conditions, therefore policy makers will not give it up easily.

But it is not only those outside of the system that derive their power and legitimacy from the examinations system. Exams give educational managers greater control over their organisations and we have seen this transfer of power over a number of decades. The performance of a teacher’s students in public exams is a factor in whether they are hired or retained, the degree of scrutiny their teaching practice receives, the professional development opportunities they can access, and the way they are perceived, and therefore supervised in their work. Examinations also create curricular alignment as they signal what should be taught and at what depth. There are significant organisational benefits to this alignment including resource efficiencies and professional development economies. (Such is the desire to achieve curricular alignment, where examinations do not exist at key stage 3, the system has substituted a whole other mechanism in the form of Ofsted to further this goal.) Finally, common measures of student achievement enable pseudo-scientific performance management processes which support a ‘tough on failure’ definition of accountability.

The transfer of power from teachers to managers that examinations has facilitated has brought many benefits. An education system paid for by the public should not allow teachers complete autonomy over what children are taught, how they are taught, or to what standard. The harmful effects of this transfer of power are also well documented. My purpose is not to come down on one side or other of this argument, but to point out that the education system has evolved to this point and we can only hope to influence the future, not to change the past. Without examinations, how will school leaders do what is asked of them? I don’t think they can. Examinations have therefore become systemically fundamental to the role and remit of thousands of school leaders.

If we get rid of the car we are also faced with dismantling the infrastructure that supports it. We must change our whole way of life. Only a crazy radical would propose this, and yet there are similar calls to ‘scrap exams’ by those who fail to comprehend how systemically fundamental they are to the education system’s operation.

Ideological alignment

Ideas are also more likely to stick around if they are aligned with ideologies that define the age – zeitgeist friendly.

When I was in the US recently with my family, I was able to take advantage of the ‘car pool’ lanes on the Californian freeways. Car pool lanes are reserved for cars with ‘two or more people’ (i.e. more than one person!) in them. They are an incentive to lift-share and reward behaviours that reduce congestion and environmental impact. What was striking was that relatively few cars were in these lanes as the vast majority had only one person in them. Most American cars are big – able to seat 5, if not 7 or 8, people. But this capacity is vastly under-used. Why?

There is an immutable ideal – personal mobility – that society has embraced in its love/hate affair with the car. Personal mobility speaks of freedom and choice. It rests on the individualistic drift of Western cultures. It is made possible by prosperity and technology, which in turn promise us that our lives will continually get better. When we call to curtail road transport, we are asking for the ideal of personal mobility to be abandoned. Public transport offers environmental salvation but at the cost of travelling like a communist!

What is the immutable ideal that perpetuates examinations? I suggest three.

First, it is our desire to measure and control our world. Science promises to reveal what was previously unseen and misunderstood. We look outwards to the very edges of our known universe, and inwards to the hidden architecture of our minds. Cognitive measurement is the best technology we have to estimate our development. Scientific management has permeated every industry, supporting vast gains in productivity and the creation of wealth. So why not apply these same tools to revolutionise education? Calls to reverse ‘progress’ in measuring educational outcomes are heard as an admission of defeat – the retreat of science and knowledge.

Second, examination results speak to our meritocratic ideals – the belief that if you work hard, good things will come to you. Meritocracy appeals most to those who have attained ‘success’ (often in the form of wealth, power, or notoriety) as they may attribute this to their ability and determination, rather than luck or the perpetuation of inequality. The examination system rides on a meritocratic wave and will not fall until this wave crashes.

Third, examinations promise greater equality. This belief builds on the meritocratic myth by asserting that education is the route out of poverty. Examinations (as the measure of education) are the means by which children may step up the social ladder, therefore schools are an instrument of social engineering. This belief is evident in educational policy which calls for schools to close attainment gaps, although the past twenty years indicates that it is not within the power of schools to do this (see here). Curiously, this idea has traction with both left and right wing politicians. Educationalists have also embraced it and ‘closing gaps’ has become a dominant theme in the school improvement narrative. Even those who have abandoned the meritocratic myth argue that exams at least make inequality visible in a way that more subjective forms of assessment will not, even if it is not within the gift of schools to solve the problem.

Equality, meritocracy, and scientific progress are the immutable ideals that help to perpetuate the examination system. What is the scope for reform when examinations are systemically fundamental and aligned with such powerful ingrained belief systems?

Better cars

For the reasons above, I believe that examinations will be with us for a long time to come. If we accept that we have limited influence over the factors that fuel their longevity we can expend our energies on realistic reforms – evolution, not revolution. Electric cars will replace those powered by fossil fuels, but the car will remain the dominant form of transport. So too should we be working towards better exams, not their elimination.

By way of conclusion to this post, I shall offer a few suggestions as to where we might focus our efforts:

1. Incentivising the right things. Examinations incentivise behaviour – for students, teachers, managers and policy makers. As far as possible, we should try to incentivise the right things. For example, what we choose to put in the test and the form the test takes both signal what learning is valued. Reformers must think carefully about what incentives they are creating, but also recognise that changes bring unintended consequences.
2. Mitigating negative effects. There are undoubtedly harmful effects of the examination system, not least the managerialism and high stakes accountability system that has proliferated, or the casualties of the sorting mechanism that we choose to employ. We can do more to mitigate these effects.
3. Don’t use exams as a lever for wider change. Examinations play a particular role (or roles) within the system. Reformers should focus on ensuring exams perform these roles as well as possible. If you want systemic change, then work to change the system. Examination reform should not be used as a lever for systemic change as it is puny in the face of systemic structures and widely held societal beliefs.
4. Recognise the limits of exams (and education) as an instrument for social change. Our belief in the power of the education system to bring about equality is misguided. That is not to say that education is not important or that we cannot make a difference to those from disadvantaged backgrounds, but societal change is well beyond the scope of schools. Pretending otherwise lets those that can do something about inequality off the hook.
5. Develop a more sophisticated and informed conception of school improvement. Our school improvement technology is poor and will remain so whilst we rely on simplistic proxies for school quality. This distracts from more important purposes that should be served.

Examinations are – whether we like it or not – part of the educational ecosystem. Their existence is immutable, but not their form. A better examination system is possible if we recognise that there are limits to what can be changed. In my view, calls for radical reform are misguided. We can work towards a better system, but should not fool ourselves that we can design and build something better from scratch.