For all the associated benefits of waking up slightly later and being able to sn
eak off to Daunt Books at the end of the day, spending Monday in a conference centre getting feedback from our exam board would not be my first choice for how to start a week. However, it turned out to be useful in ways beyond the obvious, because the whole process got me thinking about evidence again. This was important because, under the stress of teaching the new A Level and GCSE courses for the first time, I had played it pretty safe with evidence at key stages 4 and 5, sticking closely to the requirements of the examination, despite frequently wanting to roll my eyes at them. I had read some excellent blogs by Michael Fordham that covered the big picture of how the use of evidence could be overhauled in the History curriculum, but it wasn’t until last Monday that I really did some critical thinking about the way that evidence was used at A level.
In part this happened because of my reluctance to get on with my marking. It just so happened that, along with the essays that I fully intended to review during the train journey, (yeah right), I had also packed a copy of the 2015 Oxford History Aptitude Test. I had planned to read it through in the hopes of cannibalising it for an activity aimed at some students in my reading group who had expressed interest in applying next year. So it was that on my way home, having abandoned my marking beyond all hope but still feeling a little guilty about diving straight into my bookshop haul, I decided to compromise by looking at the HAT instead. In doing so I was immediately struck by some of the differences in how the HAT handled sources as compared to the A Level exam papers that I had spent the day examining.
The first element of interest was the assumptions contained within the question. On the sample A level paper we had been looking at, the question ran thus:
‘With reference to these three sources and your understanding of the historical context, assess the value of these three sources to a historian studying the Kronstadt Rising of 1921’
But on the HAT paper, this is how the question was set out:
‘What can this source tell us about Ratramnus’s understanding of the world, and the means by which he acquired it?’
There are some really important differences in emphasis here, (beyond the fact that the second question is rather more elegant than the first). The first area of difference is that the question in the HAT paper assumes that the source can and will tell us something about the area under investigation; it is inherently useful. Contrast this with the use of the phrase ‘assess the value’ in the A Level question, which opens up the possibility that one or all of the sources may have little or no value at all. Now this may seem like a nice distinction, and it must be acknowledged that at the examiner meeting we were urged to push our students to meaningfully engage with the possibilities presented by the sources, and to avoid simplistic ‘utility’ and ‘reliability’ assessments, however the shape the question takes will necessarily shape the direction of thought. It would be very hard not to find anything of value in Ratramnus’s account after being steered towards engaging seriously with the source by the question, you could not guarantee the same kind of response from the A Level question.
One of the things that makes the source questions more effective on the A level paper rather than vague ‘how useful’ or ‘how reliable’ ones is the insistence in the question that the students interrogate the value of the source for a specific purpose. This is an important step forward, but I would argue that this purpose could still be more carefully defined. For instance, though we were told that students should not ‘check off’ the contents of the source against everything they knew about the event or issue that it was linked to, the question could be interpreted as steering them in that direction. By focusing on the value of the sources for telling us about a particular event, in this case the Kronstadt Rising, the question naturally focuses on the ‘accuracy’ of the source’s account. The HAT question, conversely, avoids that ‘checking off’ process by asking what the source can tell us about Ratrmnus’s understanding of the world, not the ‘realities’ of medieval geography. By focusing on his understanding of the world it becomes meaningless to assess how accurate his account is because that is clearly not the area of concern indicated by the question.
The final, and in my view, most important, element that I drew from the question was the contrast between the location of the ‘historian’. In the A Level question, they are an abstract figure, held at second-hand, whose priorities must be considered but not personally felt when assessing the value of the sources. The HAT paper on the other hand, assumes that the historian is the reader, their focus is the one indicated in the question, and as such they will examine the source on their own behalf. This is not to argue that the students reading both questions are not novices in the discipline, but I would also make the case that, by treating them as taking a historian’s perspective in their own right, the HAT question is both more direct and invokes more seriousness of purpose.
If we look at the ways in which the answers are to be assessed, further differences can be identified. The guidelines for examiners set out in the HAT paper are clear that, rather than forming part of simplistic assessment of reliability, the ‘values that mediated his [Ratramnus’s] analysis’ must be taken seriously. The mark-scheme states that top candidates should demonstrate understanding that Ratramnus was a ‘rational, knowledgable and religious person’ who had ‘consistent organising principles’ that structured his view of the world. To do well in answering this question, it is made clear that those writing must seek evidence of Ratramnus’s preconceptions and values as much as they should determine the specific claims he makes about reality, the two being seen as interdependent. Allowances are however, made for some attempts to weld Ratramnus’s perspective onto a sense of ‘what things were really like’. When it comes to the deployment of evidence, the mark scheme specifies that ‘markers should give some latitude for off-beat or (more probably) prosaic attempts to establish the concrete basis of this discourse’. This allowance makes it even more clear that the aim of the exercise is not to ‘check off’ the claims made by the source against the student’s knowledge of the period, but to see the claims of Ratramnus as equally important historical evidence.
By contrast, when we look at the mark-scheme issued by the exam board to guide examiners in assessing student responses to the question on the Kronstadt revolt, it is made clear that the students are expected to engage in testing the claims of the source against ‘the facts’. This can be seen in the commentary on a source written by Trotsky, which while it is recommended that students engage with the ideas that might have shaped his interpretation, reference is still made to the fact that it is difficult to ‘substantiate’ a claim made by the source, and ‘the reality’ of the situation is mentioned. Likewise, in the advice we were given by the exam board, we were told to encourage students to use phrases like ‘adds’ or ‘detracts from value’ when talking about provenance in particular, the implication being that it was possible to rank the value of the sources, arguably letting questions of ‘utility’ and ‘reliability’ in through the back door.
Why is this important?
Now all this might be seen as me engaging in a bit of nit-picking, but I do think that the difference between the approaches of the HAT paper and the A Level question demonstrate that work still needs to be done on setting rigorous evidential questions. While I don’t think that A-Level students will behave in exactly the same ways that fully trained historians engaging in research will do, it makes little sense to me to teach them an approach to evidence that actively militates against many of the approaches used by historical researchers. Here’s where I fess up to the ‘provenance’ of my attitude in this matter, as someone who once did some research in the highly fashionable sub-field of intellectual history represented by the study of absolutism in New France (I can hear you yawning at the back), the approach in the HAT paper aligns much more closely with my experiences of using evidence. Of the two elements that I have isolated here – the relationship with historical ‘reality’ and the questions of the ‘reliability’ of provenance, the HAT question tacks much more closely towards the kind of questions I would have had in mind when approaching evidence. If, for example, I had encountered a piece of evidence that contradicted the accounts of events contained in the historical literature and in other pieces of evidence, this would not have been a reason to discount it as less valuable, in fact I would have paid it more attention in an effort to find out what the roots of the contradiction were. Likewise, when I was doing my MPhil research, the competing perspectives of Count Frontenac and the Intendant of New France, influenced by their competing political and philosophical perspectives, were crucial to my study of their dispute. Evidence somehow cleansed of these ‘biases’ or discounted on that basis would have provided thin gruel indeed for someone researching intellectual history, and to be honest, almost anything else.
That’s fair enough, but what bearing does it have on my A Level class? My observation would be that the HAT paper demonstrates that it is possible to set rigorous evidential questions that avoid some of the pitfalls that afflict the questions they are set in the exams. It seems a shame that some of the good evidential work that is being done at KS3 is interrupted once students hit key stages 4 and 5. Though we’re no longer in the frustrating position of having to coach students through assessing how ‘reliable’ a David Low cartoon is, there is still work to be done.
In the meantime, I’ll be preferring HAT papers along with the KitKats at the next meeting of my reading group.
Fresh Alarums 