Assessing without levels – A case of the emperor’s new clothes?

Standard

Inspired by discussions about levels and @AlBrine reminding me of this document from the Dfe on assessing without levels, I’ve been thinking about some of the new models I’ve seen for assessment.

My understanding of why we’re getting rid of levels is:

  • They label children – “I’m a 4a”
  • They became over complex – They were only ever supposed to be whole numbers not broken down into a/b/c
  • As a result in some cases they became laborious to use
  • Also as a result the language used in them became -un-user friendly – teachers spent hours putting them into ‘pupil speak’
  • Parent’s didn’t understand them
  • They became a measuring tool for leaders to measure progress rather than focus on the learning each individual does

I’ve seen the following models:

  • Using GCSE (A*-G) grades from year 7
  • Going from two strands of assessment to five strands using 1-9
  • Using stages of learning i.e from novice to expert
  • Systems that use -1,0,1,2 to measure student achievement
  • Using the same descriptors i.e Blooms, but without the levels
  • Every assessed topic has it’s own set of criteria to show advancement of knowledge & skills

Whilst I have been selective in these models, all I can see the same thing happening again.

Whether a child thinks they are ‘4a’ or ‘F’ or an ‘expert’ or a ‘-1’ , it is surely still going to end in labelling? For example, “I’m ‘developing’  in Biology”. I know I’d be fed up being called a ‘novice’ or an ‘I4’.

Aren’t these just as complex as levels? Subjects will still need to use strands and each strand each new grade/number.

Do we really think that parents will understand them more? Especially as there is a high chance that the systems used between primary and secondary will be different and if they move schools almost certainly different.

Has any school announced with their new systems that they will record progress in a manner that doesn’t fit with a spreadsheet? I doubt it. So regardless, teachers will be under pressure to use these in the same way that levels have been to ensure your class is making progress.

If teachers are making up new criteria for every topic isn’t this going to take hours? God forbid someone wants to change which part of the curriculum you want to directly assess. What if one school says an ‘expert’ is one thing and another school calls that a ‘novice’?

I have also read some of the case studies and watched the clips from schools that have shared their systems via the DfE and TES. In the majority of these (perhaps with the exception of SOLO taxonomy) they seem to be a very similar system but with new labels and ‘made by the school’. In some cases they’re are similarly or more complex than the original levels. In the reports the authors have made some sweeping statements about levels and how their new systems are better but they’re not tallying up. Stating ‘levels put a ceiling on achievement’ isn’t true and to then share a system where there is a ‘top’ level which assumably is also a ceiling seems bizarre. Have people got carried away with their own rhetoric?

I’m not seeing anything that hasn’t really been done before; skills passports with ‘I can’ statements, using overriding statements to describe a student’s knowledge & understanding. None of this is new.

Some of the comments I’ve heard and seen about new systems really worry me. Comments such as ‘I can now regularly focus on progress’ , didn’t you do this before?! Nothing I’m hearing is new if you were assessing effectively anyway.

Have we got overexcited about being given the freedom to do what we want without being dictated to and ended up with exactly the same type of systems ?

A shift in mindset?

“The distinction between assessment of learning and assessment for learning is basically about the intention behind the assessment. So, if you’re assessing in order to help you teach better, that’s assessment for learning, and if you’re assessing in order to grade students, to rank them or to give them a score on a test, then that’s assessment of learning.”
Dylan Wiliam, ‘Assessment for Learning: why, what and how

(found in Alex Quigley’s blog on levelling)

All I’m seeing in some of these new models is assessment of learning.

So is the difference between the levels system and any new system just in our mindset? or more the mindset of the leaders in our schools?

If we have leaders who are still interested in children making exceptional progress then surely any method we introduce will go the same way as levels?

Any system that truly has assessment for learning at heart will probably not fit into a spreadsheet or SIMS. It may not be neat and our spreadsheets might not gleam with green boxes.

How many leaders will stop teachers having to enter data into a system at least once a half term? Or will it still be a case of the tail wagging the dog?

So should we be spending a lot of time and effort rewriting things that we think will focus on deeper learning and creating new data systems or are they just a case of the emperor’s new clothes?

Advertisements

8 thoughts on “Assessing without levels – A case of the emperor’s new clothes?

  1. Great blog. When levels were first introduced it seemed so much fairer to move to a form of criterion referencing and assess generic ‘progress’ rather than using judgements that just compare quality across a cohort. The problem is criterion referencing such as levels doesn’t work because it is actually really difficult to create meaningful generic criteria.
    The obsession with something called ‘progress’ just has to be dropped. In many subjects (such as history) it just can’t be meaningfully measured.
    It is such a basic point. YOU CAN’T EFFECTIVELY MEASURE ‘PROGRESS’ unless in your subject there is a meaningful generic and fixed hierarchy of skills. It just about works in maths and in primary with literacy skills and reading but not much anywhere else.

  2. Great piece. First article (of many) I have read about this in which there has been reference to spreadsheets and schools’ information systems. Sad, but true, that this will have to be considered if/when new models are adopted.

  3. Miss D. I think you have hit the crux in this thoughtful piece with the Wiliam’s quote. We are being asked to do two things with assessment and they are to some degree in opposition. Ideally assessment should be not only for learning (AfL) but as learning AaL). The assessment should be a fluid part of the learning, remembering, reinforcing and developing of the knowledge and skills of the subject in hand and should primarily be about our own individual development. Ideally this should be continual and have no ceiling apart from the one that we set and that we are happy with. To give an analogy I could be happy thrashing my guitar in the bedroom at the weekend or I could only be happy when I playing the O2 to thousands of screaming fans my achievement is driven by my own ambition (and grit and determination – and probably by a limit on my skills but lets not get obsessed by genetic pre-determinism here!)

    However from the early days of assessment it has also had to serve another master that of the external verifier and the comparator. So it is not enough for me to know:

    – what can I do?
    – how do I know I can do this?
    – how do I improve?

    which we might call the key questions for an AfL / AaL process but also am I doing as well as other people and how can I prove this to an external verifier – in crude terms, though this question is still at the heart of the OfSTED process – am I giving value for money. This is not entirely unreasonable we are spenders not creators of the public wealth (though of course we are central to the creation of future public wealth) but if the dominance is the latter then we will need systems which can be pseudo-quantified in order to show the kinds of process which, as you say, can be charted and graphed.

    My ultimate cynicism is that this is part of a process to bring back the ultimate in the Assessment of Learning the end of year examination. We have already seen both GCSE and A levels moving back towards this kind of uber-summative assessment and I hope that we never have the lists of students results on the board and the “place in class” but again I see some signs of such things coming along.

    So, can we take some charge of the process. I think we can by not rushing to invent “new levels” but being happy with the ones we have which were originally designed as development ladders not measures (like you I despair of the use of sub or even decimalised levels – I have seen schools where they have an average of level 5.32 in Y7 etc… the ultimate in pseudo-quantitisation) and really focus on helping children to think about, and be involved in, the three questions above and use technology to help them:

    – assess their own learning
    – provide evidence about this
    – set targets for the next stage of learning

    This self-assessment (with scaffolding and support from their teachers) can be very motivating (Dweck and Duckworthy’s work on this is very useful) and becomes about individual progress. We will, of course, still have to worry about preparation for the GCSE and A level examinations but these can be built in my the teacher scaffolding of targets. I do hope that this is an opportunity to take back some of the control.

    As for the proving that the whole school is doing OK well maybe that should be the job of the senior teachers and they could leave the learning to take place in the classroom! – a different kind of revolution!

  4. I think there are systems out there that move away from assessment for the purposes of grading, but only in primary as far as I’ve seen. I think the probably with secondary schools is the desire to have a common system that can feed into spreadsheets, but which must then be designed by separate departments.
    The Key Objectives model I propose for primary (and the similar Key Performance Indicators model from the NAHT) take assessment back to its real purpose – judging whether or not a child knows/can do a specific thing. Of course, from that you can compile spreadsheets and charts if you wish, but it should all come back to the common list of objectives, about which we can say to a child; these are the things you are secure in; these are the things you need to perfect. Likewise, that same advice can be given to parents.
    I agree, that as soon as you reduce reporting to ‘E’, or ‘Novice’ or any similar banding, then you lose that quality information.

  5. Excellent blog that sums up what’s going wrong. It’s currently making me quite despondent, as, driven by this ridiculous injunction to always be ‘showing progress’ we’re now (in my primary school) literally quantifying the statements, working out the % ‘meeting’ for each pupil, each half term, and recording that as a number or a ‘grade’ e.g. 5M (meeting at least 80% of the statements for y5). Once those are written on our tracking system, the senior leaders actually believe it and use that data to discuss year groups, school progress, specific groups, etc. There are so many things wrong with that model I don’t really know where to start, but some of the issues are:

    The statements taken from the New NC are not assessment criteria, they are what they wish us to be teaching.
    The statements themselves can be ludicrous or ridiculously vague, e.g. ‘continuing to read and discuss an increasingly wide range…’
    The statements are sometimes meaningless in terms of real learning, for example, from the year 5 writing, ‘to choose a suitable writing implement’. Really?
    There are too many to adequately and reliably assess, so we obtain a sub-set, thus introducing a new arbitrary variable.
    We have not actually assessed all of the hundreds of statements at every junction where we are to give evidence, because there is not time, so we use ‘rule of thumb’. We all know how reliable that is!
    We are still focussed on endless measuring, when we should be focussed on teaching and on the actual content of what we’re teaching. Is what we’re assessing even worthwhile?
    The statements are not a continuum. If we’re really going to be serious about showing a pupil where they are going, we need them to have access to what the different stages on the path look like.
    We’re back to cohort-based assessment, regardless of age, background, language, location, life history. How is that defensible? Imagine taking a course where the groups were based on your year of birth, not your experience in that subject.

    There are so many exciting developments in the pipeline that could enhance teaching and learning but the new curriculum and the farce of assessment without levels, as it is being (mis)conceived in England, are really getting in the way.

  6. Pingback: Separating formative & summative assessment – Avoiding the pitfalls of levels | missdcoxblog

  7. Pingback: Five checks to see if you’ve replaced levels with levels | missdcoxblog

  8. Pingback: Are you brave enough to ditch data? The battle of Data, Ofsted and Research  | missdcoxblog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s