Comparing students to give current, new specification, GCSE grades (without grade boundaries)

In the absence of grade boundaries, yet still being under pressure to complete data sheets about students, there are several courses of action that teachers have taken:

1. Make up grade boundaries (using some sort of logic)
2. Create grade descriptors for 1-9 using some of the information we have been given
3. Just make it all up

Although I have pondered grade boundaries, as I don’t use marks with my GCSE students there is no need for me to come up with % or raw mark grade boundaries. I have also avoided number 2 as it can be complex and rely on complex and often ambiguous language. In an attempt to avoid number 3, I have trialled using student comparison to help with our data entry.

I’m currently trialling ‘No more marking’ so the idea of comparison was in my head when we did this yesterday.

We took a list of our students. Here is a made up class to help me explain:

Throughout this process, I had in my mind two sets of information about grades and levels of knowledge & skills; Ofqual’s Religious Studies GCSE grade descriptors (2/5/8) and our exam board mark scheme (in our case AQA GCSE)

We initially identified the student/s that we believe could get full marks and have assumed that full marks will be a grade 9. We then identified the student that has shown the least aptitude with the course content and skills and allocated them with a 3. We believed that he could do slightly more than the grade 2 descriptor. We then identified a student that we thought would be a grade 5 and 7 using as far as possible, logical increments of knowledge and skills.

I then posed a set of questions to my colleague about the remaining students. I also had student data in front of me so I knew roughly where to start the questioning. Here is an example of the type of conversation we had:

Me: Billy’s data suggests he should be 4-6. Is he ‘stronger’ or ‘weaker’ than Sally?

Colleague: He’s really good with responses in class but in his written work has lacked detail that Sally has, so weaker.

Me: Is he stronger than Brian?

Colleague: Definitely. He can give examples in his work that Brian can’t.

The conversations were actually more detailed than this as we could both discuss specific knowledge and skills that we know they have to have. Whilst, I know this class to some extent, the discussion with my colleague meant that we could consider carefully what they can/can’t do. I can’t vouch for my colleague, but I know if someone did this with me it would certainly help to clarify what it is that each student needs to do to progress further as we discussed in terms of knowledge and skills, not in terms of how many more marks they need to achieve.

In all cases we have taken a holistic view of the student; classroom responses, accuracy of answers, test capability etc. It is not just from one test with marks and grade boundaries.

So we finally ended up with a set of class data, that was essentially a ‘rank’ of the class, using grade descriptors/mark schemes. I know that the term ‘ranking’ of a class can be controversial but this is in no way is shared with students or used with them. It had the outcome of generating a grade that we needed to enter in our data system but hopefully more usefully for my colleague we had a good discussion about the individual students and their strengths and areas that we can help them on in the coming months.

I’m not claiming that this is any better than the suggestions at the start of the blog but it certainly is another way, instead of using randomly made up grade boundaries. It also encourages a teacher has to know their class well (not just what marks they get) and helps to diagnose potential future support for a student.

The marathon of assessment

This post is inspired by Daisy Christodoulou’s book ‘Making good progress’ which I highly recommend if you’re interested in assessment in any way.

In the book she uses the metaphor of a training for a marathon when talking about assessment and I really like how this can explain what I do with my students. I thought it might be useful to share my thoughts on how I do this. I am using the final marathon race in my context as the GCSE exam in year 11.

Marathon runners rarely run whole marathons in training

This really interested me as this is exactly how I’ve been sorting my assessment for past few years. Students learn early in the GCSE what an exam paper requires them to do. We practise exam questions from the first half term. Not all of them but the simple ones to get them confident; these are ‘small’ assessments. I scatter them throughout the term. I rarely tell them they will be doing an ‘exam question’ because I don’t want to put undue emphasis on them completing them. For some odd reason it works. I could put an exam question in front of them in period 1 on Monday and they don’t even respond. I say ‘stop talking, write’ and they all do! I never have ‘is this a test?’ Or ‘you didn’t tell us?’ Or ‘I didn’t revise for this’. I just put them in front of them and I’m so confident that they can do them, it seems to rub off on them.

However, I will tell them about the longer tests but I don’t expect them to ‘revise’, I’ve been doing that for them already through their homework. Longer tests will interleave all previous topics, not just the one they’ve done. This produces the ‘medium’ sized assessment.

Not everything you do to prepare for a marathon involves running

It is important not to think that everything that is preparation is doing exams or even exam questions. A marathon runner may buy a new pair of trainers, they may choose their diet carefully and they must also rest. None of these involve running. There are so many things that I do to prepare students for an exam that aren’t an assessment. Unmeasurable stuff like class relationships, praise and confidence building are essential for students to feel they can succeed. Their homeworks are all either writing multiple choice questions, doing the multiple choice quiz or learning keywords. Whilst they won’t be questioned directly on a keyword. They have to know what they mean to access the questions. If they’re asked to evaluate  “Sanctity of life is more important than quality of life”, they have to know what these terms mean to have any hope of answering the question.

You cannot start training a few weeks before the marathon

Most of us are rubbish at planning ahead of time. We will leave things as long as possible. Most children will not consciously prepare for their exams until a couple of months or some cases weeks before their exam. I strongly believe that teachers have a duty to put this structure in for students so they are practising from day 1; we shouldn’t just leave it to chance. I start ‘revision’ with my classes from day 1 of the GCSE. This includes all their homeworks. Our teaching from day 1 should focus on the final outcome, the marathon/exam. Some teachers don’t like this idea as they think it is about teaching to the test. I think it is and it isn’t. Also, we can assume that most people that train for a marathon  do it out of choice. Whilst student may have opted for a subject, for core subjects this isn’t true. This is even more reason to put the structure in for them from the start.

Practising the whole marathon is about performance analysis 

I only give two full ‘mock’ experiences; December and March of year 11. It’s the only experience of the ‘large’ assessment they will get, with all the content and all the skills needed, as the real exam. My emphasis on these mocks is different to some others. I don’t care what they get in terms of marks or grade. It is much more useful as a diagnostic tool. It is exactly what I need to help them unpick what they need to work on in the last months; how to focus their final revision. We spend time on these papers after they’ve been marked. They improve aspects individually and I pull together common errors and plan around them.  A mock exam shouldn’t be seen as anything other than a snapshot of where they are and where they could get to. It’s a really important indication to me about my teaching (this is why I find it odd that people outsource marking) and where they’ve ‘got it’ or I need to recap.

It would be easy to continue with the metaphor in many ways but for me the key aspect that reflects what I do is the increasing levels of practice of different aspects (the ‘small’ and ‘medium’ assessments) that are key aspects of working towards the ‘large’ assessment.

Why marking Pupil Premium books first is misguided

I’ve blogged before on how I dislike people talking about groups of children in sweeping generalisations. Pupil premium is a special case as it attracts money. Money makes people do odd things. One thing that seems to have been suggested by someone and then spread throughout schools is marking pupil premium books first.

I personally find the premise behind this ‘strategy’ at best misguided and illogical and at worst discriminatory and unprofessional.

The premise/s they have used to support it are:

Mark PP student books first because you will:

• Be more awake
• The lesson will be more fresh in your mind
• Focus more on them than non-PP
• Spend more time on them
• Give more detailed feedback
• It will be your ‘best’ marking
• You will have a better attention span
• It keeps vulnerable students in your mind

 

The DfE states that:

PPG provides funding for two policies:

• raising the attainment of disadvantaged pupils of all abilities to reach their potential
• supporting children and young people with parents in the regular armed forces

However there are so many issues surrounding the strategy of marking PP (especially FSM students) books first:

Issue 1- Conflation with academic ability 

In many cases PP is conflated with ‘low ability’ or not ‘making progress’. It’s used as a piece of data that indicates academic skills, like you might use a SAT score. This is a huge misuse of data. In no way does PP indicate a child’s academic ability. There will be PP children that have low levels literacy or an SEN need or high absence. Equally there will be PP children that have beyond age reading ability, are classed as ‘above average’ ability and who attend school 100%. You cannot make a sweeping generalisation. If the former is true, marking their books first WILL NOT make any difference.

Issue 2- What about non-PP?

If you are willing to say you mark PP books first so they get better treatment you are essentially saying that non-PP are not worth better treatment. Could you really tell a non-PP child’s parent at parents evening ‘I mark your child’s book last because you’ve never claimed for free school meals’? The value judgment you are making on non-PP is never what the government intended with this funding.

Issue 3 – Who says that the first books marked are the ‘best’? Evidence?

I’ve heard people claim they’re less tired, they are more focussed and they have more time for the first books. This is a very concerning indictment of that teacher/schools marking policy or systems if a teacher publicly says they don’t mark in a similar standard throughout.

In an unscientific poll on twitter about which books do people mark first, many claim that the first books aren’t always the ‘easiest’ to mark or may not have the ‘best marking’. Teachers need to read a few books to gauge where the class has struggled etc  either way, there’s something wrong with choosing PP books to be at the ‘optimum’ stages of marking about other students.

This clearly shows that marking & feedback in schools needs a serious overhaul or complete rethink.

Issue 4 – The child whose parent/s haven’t claimed FSM but could

If you claim that PP children need more attention as they are disadvantaged and need more support you are on really dodgy ground if you have children in your school who fit the criteria but for some reason their parent hasn’t claimed FSM. You are essentially saying, because your parent hasn’t completed a form, I’m not going to give you the attention you may need. That’s immoral and unjust. Judging a student by PP and not by need in my eyes is negligent and unprofessional.

Dr Rebecca Allen, Director, Education Data Lab
points out:

“…it is important to note that many non-FSM pupils come from lower income households than FSM pupils. (Hobbs and Vignoles12 estimate that only around one-quarter to one-half of FSM pupils are in the lowest income households in 2004/5.) This is principally because the very act of receiving means-tested benefits and tax credits pushes children eligible for FSM up the household income distribution. It is the diverse nature of the non-FSM pupils across England that means that is more difficult than we might think to compare pupil premium gaps across schools. A school may substantially narrow the gap by working hard to improve the attainment of their most deprived children, or through the accident of the characteristics of their ineligible children.”

Issue 5 – Evidence it ‘makes a difference’

If any teacher claims that marking PP books first has made a significant difference to a student’s progress I would argue this is the case because:

• It may have ‘closed the gap’ because you’ve ignored non-PP.
• You’ve used effective marking strategies in those books but not in the others, being PP is irrelevant
• You’ve consciously ‘not bothered’ with detailed feedback for non-PP

None of these are admirable.

In a publication of ‘good practice’ from the DfE (thank goodness) none of the methods for minimising barriers include marking PP books first..

The reality of there is very little evidence on what makes effective marking and what improves progress. I’m doubtful that these teachers have found an effective solution that the EEF didn’t see.

Issue 6 – Focussing on the ‘gap’ instead of attainment

When asked about marking PP books first, some teachers have replied with an excellently ‘on message’ response about national statistics showing that we need to ‘close the gap’. All very admirable and I’m sure would be perfect for an interview response. However the reality is that if you focus on the gap the whole time you’re missing the point.

Dr Allen says:

“Concentrate on better results for pupil premium children, rather than narrowing the gap
. Free school meals children are clearly different from one another, but they vary far less than the group who are not eligible for free school meals, since this group includes both those with bankers and cleaners as parents.  Many schools have always had pupil premium gaps close to zero because their non-claiming pupils are no different in their social or educational background to their pupil premium children. So, although it is gaps in achievement that contribute to social class inequalities and should be the national benchmark to assessing policy success, it is better for schools to concentrate their focus on the attainment of their FSM pupils rather than the size of their own pupil premium gap.”

Issue 6 – The assumption of PP minority groups 

In some schools/classes, non-PP is the minority. Marking PP ‘first’ becomes a challenge as if you believe your marking is ‘best’ with the first few books, which PP books will you put at the top? Another sub group? This truly illustrates how marking PP books first is nonsense.

Issue 7 – Misapplication of ‘what works’

Some teachers will argue that they’ve used the EEF toolkit as evidence of ‘what works’ for their PP students. I agree that great feedback and response is essential for learning. The issue is that it’s being justified for use just for PP students. What works for PP students works for everyone. Good teaching is good teaching for all. All students deserve quality feedback. If a teacher cannot equally spread that amongst their students the teacher and/or school need to rethink what they’re policy or system demands.

Dr Allen confirms:

“As it turns out, great schools tend to be great schools for all children in the school – the statistical correlation between who does well for FSM children and who does well for non- FSM children is very high. Moreover, schools can make a difference to the life chances of FSM children – there are huge differences in attainment for these children across schools, far larger than there are for children from wealthy backgrounds who do pretty well in all schools.”

The nFER report on disadvantage makes it very clear is that what works is for all students:

Issue 8- Assuming PP means ‘disadvantaged’

One of the biggest things that annoys me about some teachers attitude to PP students is the assumption that they are disadvantaged. I can’t help feeling that this stereotypical response is what causes the ‘gap’ in the first place. Whether they are disadvantaged or not it is highly unlikely that marking their book first will deal with any disadvantage they do have.

Issue 9 – Lack of focus on the individual child

Finally, the clumping of PP students has almost become a differentiation strategy in itself. It is absolutely not. Within that group of students there are individuals. Each with strengths and needs. Marking their books first seems to be a proxy for meeting their needs. I know the teachers/schools that mark their books first do many other things but I would argue we should mark all books equally and spend that ‘extra’ time with those that are struggling or not making progress, PP or non-PP.

 

Some teachers and schools have somehow got caught up in a rhetoric surrounding PP students. I teach so that ALL my students can make great progress whilst looking at each student individually for how I can support them. A policy of marking their books first, in my opinion, is a policy made to ‘tick’ a pupil premium action box. How many of these teachers/schools will ditch this method if Ofsted stopped focussing on pp students?

Instead we should be focussing on great teaching (and feedback) for all students.

 

 

References & related documents 

Click to access Funding-for-disadvantaged-pupils.pdf

Click to access Pupil-Premium-Summit-Report-FINAL-EDIT.pdf

https://www.gov.uk/government/publications/the-pupil-premium-how-schools-are-spending-the-funding-successfully

Click to access DFE-RR411_Supporting_the_attainment_of_disadvantaged_pupils.pdf

Click to access EEF_Marking_Review_April_2016.pdf

Those of you that read/mark student work….

Whose work do you read/mark first?

Those that…

— Miss (@missdcox) April 9, 2017

Research in education is great…until you start to try and use it.

One reason that I’ve been interested in education research in the past couple of years is because, by luck/coincidence it mainly supports what I already do. I’ve been setting quick 1-5 recap quizzes for years, well before I read cognitive science research on the positive impact of retrieval practice. I’ve not really done much to change how I teach yet it ‘works’ with the research.

However, it’s not always as simple as confirming what you already do. Research in education is a minefield. Might it be better to leave well alone?

Here is a (not comprehensive) summary of why using research is problematical:

1.Not enough specific research has been done for context specific conclusions

In the EEF ‘A marked improvement’ publication on marking, the authors make the following comment, very early on:

‘The quality of existing evidence focused specifically on written marking is low. This is surprising and concerning bearing in mind the importance of feedback to pupils’ progress and the time in a teacher’s day taken up by marking. Few large-scale, robust studies, such as randomised controlled trials, have looked at marking. Most studies that have been conducted are small in scale and/or based in the fields of higher education or English as a foreign language (EFL), meaning that it is often challenging to translate findings into a primary or secondary school context or to other subjects. Most studies consider impact over a short period, with very few identifying evidence on long-term outcomes.’  p5

This is true for many areas of teaching yet teachers and schools are being encouraged to use research more.

If I work in a coastal school, with majority white, FSM boys, will any research be directly applicable to my context?

Is it just a waste of time with the current lack of useful and applicable research?

2. It can go against our instincts
   

In ‘Make it stick’, The Science of Successful Learning by Peter C. Brown, Henry L. Roediger III and Mark A. McDaniel, they make a point about interleaved practice and teacher discomfort:

Teachers prefer for themselves and their students to feel comfortable as they learn. Clearly some research informed practice won’t always ‘feel’ comfortable.

3.Ofsted may not agree

Ofsted have previously commented on what they see and think ‘works’, not always what research may suggest is the case. There are examples in Ofsted reports that make a statement on something that a school is doing that may not correlate with research. In recent years, as research has become more prominent, comments in some cases have declined especially in cases where practice has been ‘debunked’ and Ofsted have attempted to make it what they are aren’t looking for when inspecting. Should inspectors continue to make  judgements from their collective or personal view or would it mean that reports would be impossible to write?

Ofsted report

And perhaps more controversially, should an Ofsted team comment on a school that is actively going against what research points to as being effective?

Is the issue here consistency or that research may not support it?

4.It’s not always convenient for leaders

One reason why so many schools have been reluctant to ditch grading lesson observations (even though evidence shows they’re unreliable to consistently make valid judgements) is not because they think grading is a valuable tool for teachers but a perfect system for a spreadsheet.

5.It doesn’t support a school’s priorities

Schools that want to prioritise English and Maths in the curriculum will not be interested in research that suggests that the Arts or Philosophy might help learning, even when there might be evidence that they can benefit English & Maths. The political gaming of league tables will always mean that leaders want what seems to be more direct action such as giving more curriculum time to core subjects.

6. It’s not the way we’ve always done it
   

Teachers can be habitual. We like to teach in a way that we know, even if it isn’t hugely successful; we are reluctant to change. It’s comfortable to stick with the way we’ve always done things.

Who cares if research has a better way?

7.We can cotton pick what we want…

The EEF toolkit rates ‘peer tutoring’ as having a positive possible effect. I could see this and tell my staff ‘I want to see ‘peer tutoring’ in all your classes because that will enhance learning by ‘+5’ months.

However, the evidence behind this summary wouldn’t support this action. It specifies that the tutoring is most effective with cross-age tutoring, with two years between the students. That wouldn’t be the case in one class in the UK.

And crucially it also states:

‘Peer tutoring appears to be less effective when the approach replaces normal teaching, rather than supplementing or enhancing it, suggesting that peer tutoring is most effectively used to consolidate learning, rather than to introduce new material.’

Research in the wrong hands and with superficial or no in-depth analysis can be dangerous….

8…And ignore what we don’t like

I don’t like teaching using group work. I’ll ignore any research about its possible learning benefits.

9.Research suggests…..

Even in the EEF toolkit each intervention has a list of caveats. Nothing is simple. Looking at the effect sizes is one way but even that is full of complications.  All we can ever say is that ‘Research suggests….’ and always present it with the scepticism that it deserves.

Should we be directing teachers how to teach using research that can, at best, make suggestions of what might work?

10. Sources contradict each other

Read any literature review and you will almost certainly find a variety of previous papers that have different and often contradictory findings. Interpretation bias can also make a difference. You can pretty much use research to prove or disprove your point if you want to.

As a leader, would you reduce class sizes or not?

EEF Toolkit & Hattie’s effect sizes (0.4 is seen as the point where something ‘makes a difference’)

11.It’s not what we want it to be

Research on classroom displays suggests that, especially for younger children, too much stimulation can hinder learning yet in primary schools across the country teachers spend hours on colourful bordettes, elaborate scenes and ‘welcoming’ boards.

Whilst there is research that some targeted displays can actually accelerate progress they may not be the wonderfully creative and attention grabbing scenes that some teachers love to construct.

Sources & references

• http://visible-learning.org/nvd3/visualize/hattie-ranking-interactive-2009-2011-2015.html
• https://educationendowmentfoundation.org.uk/resources/teaching-learning-toolkit
• http://www.evidencebasedteaching.org.au/hattie-effect-size-2016-update/
• http://askforevidence.org/articles/neuromyths-and-why-they-persist-in-the-classroom