Tag Archive | assessment

Don’t you just hate “Target Grades”

Well perhaps not “hate” but they do concern me for a number of reasons.

It won’t be long before the UK starts school again but I would bet even now there will be computers crunching Key Stage or end of year test results running them through algorithms to predict future performance in order to set target grades.

This worries me, does it you?

I am no expert in statistics but I assume that with enough data and analysis you could begin to predict what could happen in the future based on what has gone before. But – this is only a prediction, a percentage chance that something could or could not happen. Life or car insurance must be much the same with certain categories resulting in much higher premiums as they are more likely to have an accident than others. The difference being as, far as insurance goes, if the model is right the company makes a profit and if not a loss. With target grades the model outcome is quite different, you are setting a challenge, an expectation and the conditions for failure or success. This is not a profit or loss situation, it is life chances.

What I am interested in as a teacher is the psychological impact of target grades on the students as well as how it changes the teaching and learning landscape.

Here are my questions so far.

  • Having a target for anything suggests accountability. “Why did you not reach the target?” is accusatory and you are on the defensive straight away.  Is this good for the teaching and learning environment and relationship?
  • Who decides, and how, on a suitable target?
  • Is prior performance a good predictor of future performance, can there be a linear gradient link between the two?
  • Do we understand and acknowledge the psychological effects of target setting and work to mitigate any negative aspects?
  • Are we clear and consistent in our use of the term “target grade”. Are there sub categories for example “aspirational target grade” or “minimum target grade” that confuse?
  • What happens if you exceed a target grade before the allotted time period or assessment point?
  • How do we account for and what are the implications for exceeding or not reaching a target grade?
  • If we accept that progress is not linear when do we begin to concern ourselves about reaching a target grade?
  • Who’s fault is it (or who gets blamed?) if a target grade is not achieved, is it the student, the teacher or whoever arrived at the target grade, or is it all three?
  • Boys and girls appear to react differently to target grades*. If this is the case why?
  • Do we ever ignore potential because of a set lower target grade?
  • Does setting targets based on a statistical model actually raise individual achievement?
  • Is everything needed to meet a target within the control of the teacher or the learner?
  • Is the time and effort spent assessing, recording and monitoring targets getting in the way of teaching? In short is it worth it?

It also occurs to me that those setting the target have a responsibility in terms of understanding the impact such an action has. I recognise that when students achieve or exceed a target we celebrate this and there are recognised systems in place to do so but where I am not so confident is in seeing similar systems for those who fail to reach a target grade.

If you feel like commenting on either the original interest topic or the questions please do so I would be glad to hear of your perspective, experience or view.

*See the article “Why many boys only do just enough

 

Advertisements

Part 4: The one and only learning theory that counts is …

Part 4.  The impact of the no one learning environment cont.

blame

A blame culture, the ultimate outcome of the “one way”.

Earlier I explored the impact of the one way not working. I described how in my experience it leads to the tightening of monitoring and checking systems,  inflexible frameworks and the limiting of creativity (or in some cases finding “creative” ways around inflexibility).  Now we turn to whose fault is it the one way is not working.

If the one way to learn, the prescribed approach, is not working then it is the fault of someone. Who is that “someone”? At the start there are always a lot of things to point the finger at, after time though the number dwindles. That someone was the Local Education Authority, trendy (lazy) teachers, progressive teaching methods, low aspirations, parents, disruptive students etc. Now it is either the leadership of the school or the teacher or a lack of effort on the part of the learner (also the fault of the teacher). In such cases it is easy to get into a cycle of finger pointing or a blame culture.

We in the UK are definitely into a blame culture and as we move further and further into it the language used by government gives this away. We hear things like “we are introducing a new check”, “pupils at risk of falling behind” , “target those areas” and “children aren’t being given a fair shot to succeed”. More the language of war you would think (the outcome of desperation?) than education perhaps.  Then there is the “takeover” manoeuvre (there is that war analogy again!), the one where those who were “in charge” or responsible are no longer trusted and a new regime is installed. In the UK it is academy trusts who take over “failing schools” but these are also failing (as we would expect if the one way does not work!). It’s certainly a dilemma for any government that persists on the one way path. I suppose with so much invested in the one way, both personally, as well as politically, it is hard if not impossible to even consider another way let alone more than one way.

What we do know is the learning environment created by the pursuit at all costs of the one way is very toxic for those involved in leadership, teaching, and learning.  Finding a way to deal with this environment is the key to improving teaching and learning. We know that through regulation and inspection leadership and teachers have their hands tied so this leaves the learner.  A simple analogy that describes how we may proceed in dealing with a toxic environment that is not going to change is living somewhere really cold and wanting to be warm. You can ask for sunnier days, less snow and ice each year or longer summers and shorter winters until you are blue in the face (ignoring climate change). You are asking for the unlikely if not impossible. The more successful way is to acclimatise yourself to the environment and seek ways of managing it in order to get what you want – to be warm. So you learn what clothes to wear and how to wear them, you practice ways of getting and keeping warm and after a while you are warm, despite the environment.

If we take the same approach in teaching and learning then it’s not about changing the learning environment to meet the needs of the learner it’s about equipping the learner to manage the learning environment to meet their learning needs.  This is important not only because of the one way  problem but because we do not learn just in schools or managed environments. We have the opportunity to learn in a number of different environments. For example at home, in work, during leisure and in a social setting are all potential learning environments.  My experience is that some learners do not do well in one school environment but thrive in another, some do not do well in any formal education environment but thrive when on work placements, and some excel in leisure pursuits but do less well in school. They are the same person but achieve differently in different environments. If we wanted evidence that we need to equip learners with the skills, attitudes, attributes, and behaviours (SAAB) to manage their learning environment then we need look no further than these examples. Where their SAAB matches the environment they flourish, where it does not they struggle.

My claim is that in these situations the learner possesses the appropriate SAAB profile for the environment in which they thrive but not the profile for those where they struggle. It occurs to me that we need to broaden or develop the SAAB profile of the learner such that they can thrive in any learning environment. We need to work with the learner to explore their learning needs and how this impacts on their learning beliefs.  To build in the learner the ability to see a difficulty to learn not as a personal weakness but as a result of the environment they are in and not having the SAAB to mange it effectively.

Links to earlier parts are:

Part 1

Part 2

Part 3

diagram of LQ and SAAB

Assessment Without Levels. Is it Possible?

banned sign

The flux in education in the UK at the moment is all about levels; the lack of levels, the change of levels and the validity of levels.  This is a great time to take a totally different approach to preparing students for examinations, one that focuses on learning and improves performance.

Section 7 of the NCSL publication “Beyond Levels” concluded that:

Pupil involvement in the learning process: the importance of placing pupils at the centre of the assessment process; and involving their active participation and views was a recurring outcome. Enabling young people to have a clear understanding of what they were learning and needed to learn next, was recognised as important.”

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/349266/beyond-levels-alternative-assessment-approaches-developed-by-teaching-schools.pdf

This article is an outline of the principles a professional development course for schools and available from Advocating Creativity that satisfies that need.

Although the course, “Assessment Without Levels” by Advocating Creativity Ltd,  was developed before the popularity of theories and research by people such as Dweck (Growth mindset), Langer (Mindful learning) and Hattie (Table of effects) it brings all three together in an effective approach. The outcome of the course has been shown to include improved:

  • learning,
  • student performance and
  • teacher planning.

Assessment for Learning

My experience shows that when we focus on establishing what a student knows or understands at any given point  then we can better plan future learning.  This is because we have a clearer understanding of what has been achieved and understood so far. The term “Assessment for Learning” (AfL) encompasses this approach. The difficulty has always been getting the students to see assessment this way , partially because they have been taught to focus on levels, be they targets or achievement  grades.

Setting tests

As a teacher you know the way it works. You set a test or revision paper and the buzz in the class at feedback time is about what mark or grade was achieved. Students are quick to spot any discrepancies in marking and insist on the right mark being awarded in line with another student in the class. Getting students to review papers in light of what they are familiar with but need to review and what they do not understand or know is at times difficult.  A great deal of valuable feedback can be lost if we do not find a way of using the performance in a formative manner. This can be a significant challenge for the teacher as few students want to revisit a test they did poorly, or even well, in.

The problem of targets

What if the setting of target grades is a way of limiting performance not enhancing it? In my experience few students see a target grade as anything more than a line to cross. Still fewer still see it as a line to be surpassed. What is more, the mechanism of setting target grades to raise attainment can be a limiting factor. This is in part  due to  the use of algorithms that are used to calculate and then predict future performance based on past performance.

There is, I believe, a particular issue with boys in respect of setting target grades.  Experience suggests that by nature many boys who are not fully engaged in an activity nearly always do just enough and little more. They see little, if any, benefit in doing more than just what is required of them. See this article if you wish to explore this idea further. “Why many boys only do just enough” http://wp.me/p2LphS-2J

Life after levels and grades

Moving away from providing feedback or targets via grades or levels may be scary but it is a course of action that if grasped with both hands will put the focus back where it needs to be, on learning.

Implementing the approach of the course, Assessment Without Levels, results in a different emphasis when setting and “marking” assessments. The traditional focus on what mark or grade to award is eliminated and replaced instead by a traffic light system indicating a level of learning. The only way for a student to assess how well they have done is by reviewing their responses[i].

better way to mark capture

Setting the test requires a slightly different approach in the way it is planned and structured. Each question or task requires identifying with a particular set of learning points. The author of the test must start with defining the learning points and consider how the learner will demonstrate understanding. They can then go on to design the question or task in a way that will allow the learner to demonstrate understanding (not just recall).

Differentiation of response is converted into a mark scheme by the teacher but this remains unseen throughout by the learner.  A question is awarded a series of marks according to the learning criteria demonstrated. Whilst there are marks involved at this stage of the process they are merely to determine the thresholds of the three possible outcomes per topic or question. The threshold marks are required by the conditional formatting process employed at the spread sheet stage.

When marking the allowed outcomes are as follows:

1) the student  has demonstrated a sound grasp of the concepts/topic and could transfer this knowledge and understanding to similar situations without difficulty[ii]This is a green traffic light.

2) the student  may understand the topic but there are some areas that need revision in order to gain mastery and be able to transfer knowledge and understanding to similar situations. This is an amber traffic light

3) the student has not shown sufficient understanding of a large number of aspects about the topic and requires a review rather than a revision of material along with possible coaching or a different approach or example used at the teaching stage. This is a red traffic light.

The learner perspective

On reviewing a test, and faced with the traffic lights per question or topic, the student can immediately identify where they need to put their efforts in order to improve. Red takes priority, followed by amber. Green responses fall into the occasional revision category. This gives a focus to further work and an ability to apply future efforts in a more effective and efficient manner.

The teacher perspective

A further analysis, this time from the teacher perspective, is also very informative.  Preparing a matrix of questions and student responses the teacher is immediately able to see if there are any common trends. Red or amber areas of concern across a number of students may suggest a topic that has not been well understood by the class. Red traffic lights indicate priorities for review, amber traffic lights revision. As a result the teacher is then better informed as how to proceed in planning revision or review after the test.  This has the benefits of making teaching more effective and resource allocation more efficient leading to the economic advantages of saved time.

How to carry out the analysis

Luckily for us Excel is able to do the analysis and presentation for us once we set the thresholds of performance. Here is an example of a typical class test set out in this way.

TRAFFIC LIGHT MARKING

 

In the example conditional formatting has been used to set the traffic lights according to performance thresholds set by the teacher. For example Q1, based on the analysis of information, is scored out of 10. The teacher has determined that a score of 8 or more is acceptable in demonstrating a sound understanding. A score of 6 or 7 suggests the second category, that of requiring revision. Anything less than 6 is regarded as a red traffic light and requires a whole scale review. What is more the teacher can model scenarios by adjusting the thresholds seeing who or what topic moves into what zone. This exercise can yield a great deal of information and inform future planning as well as helping to target resources more efficiently and effectively.

Reading the data

There is a great deal that can be gained from this type of analysis of performance. Below are a few examples of what you may find.

A brief look at the chart tells us the following

  • David needs to focus on “writing a response”.
  • Mark has need of further coaching and support in all areas.
  • Lucy needs to focus on two areas, those of: “preparing an argument” and “writing a response”

You can see how powerful this is in analysing performance of individual students.  What would you say about Angel’s performance?

For Angel It would be easy to say everything needs attention but we can see “Preparing and argument” is priority. Perhaps they have not had enough practice or the lesson planning and resources need review.

Now let’s look once again at the class performance from the teacher perspective.

We can see that:

  • all the class need to look again at “writing a response”. Perhaps they have not had enough practice or the lesson planning and resources need review.
  • possible strategies for whole class activity may include peer review and we know which students to pair up.

Returning the test

Here is an example of how a student may receive their results in the form of the front page rubric[iii]. Circling the appropriate smiley face gives direct feedback on where their efforts are needed or where they have demonstrated understanding and should celebrate their achievements.

[i] Front page of test from TuitionGuru Coaching

rubric

A word of warning. Unless you have prepared your students for this type of response to their test or assignments you will be asked “What grade did I get then?” Once you have got over this hurdle I have found that the question becomes “How can I change from an amber to a green?” When this happens you know you have AfL. I also find it useful for students to look at green or amber answers as part of self review. In this way they get to see what they are aiming for in terms of an answer and where their understanding is lacking.

Returning to the original question

Assessment without Levels. Is it possible? Not only is possible to assess without giving learners grades or levels it benefits learning to do so. Benefits can be seen in teaching and learning and in performance. Schools can be more effective and efficient in the way in which they deploy resources. Finally there are economic benefits to be gained from better use of resources.

Want to adopt or explore this approach?

If you would like your staff and learners to benefit from the approach outlined here then contact

.

*My thanks to Paul Whitehead for his feedback and comments on this article. His questions and observations have led to a review that I hope makes the process and advantages clearer.

[i] From the Advocating Creativity in Education “Teaching Ideas” series. You can download a copy  from the website  along with other titles in the Teaching Ideas series.

[ii] This is determined  as a result of the way in which the question or task is designed

[iii] Front page of test from TuitionGuru Coaching http://www.coachingmathsandenglish.co.uk

Learning Intelligence (LQ) and Lesson Planning

The start of a new term or semester often means the start of a new module, new project, or chapter in learning for the student. It has also meant a lot of lesson planning for the teacher has already taken place and it is time to test out the material. There is a lot riding on how well this has been done, the resources collected together and how it will be introduced.  Get it right and you have engaged, interested, and enthusiastic learners. Get it wrong and the consequences range from disinterest to conflict and behaviour issues.

How can LQ play a part in lesson planning?

LQ round

 This question came about because of my current research and thinking for the LQ book I am presently working on.  Although I said I would not be posting anything new on LQ I wanted to “air” my ideas on this particular aspect of Teaching and Learning and see if there was any “feedforward.

We know that the successful teacher models learning behaviours. They often have a “project” in which they are involved, they are engaged in learning and remember what it feels like to learn something for the first time. These feelings often find their way into the planning cycle because the teacher will reflect on the experiences that will be faced by the students.

The teacher/learner is not merely presenting stuff to learn they understand they must guide the student through the learning experience too and their planning will reflect this. If you have read the earlier articles on LQ you will understand why I believe LQ thinking to be important when lesson planning.

Here is an LQ take on the lesson planning process.

(Heading in blue suggest LQ and those in red traditional planning considerations)

What do I need to teach is often the starting point.

What is the unit about, what will it cover and what do I want the students to learn? We can see aims and objectives being written in response to this question. No departure from normal lesson planning.

Where are my students?

What do they know and what “anchors” can I use to help “fix” the new learning? In other words prior learning, what do they know and how do I know what they know? A teacher should always start at this point, however, some assume rather than find out and this can mean bored learners or learners who are unable to access the learning. We are planning on poor foundations. No departure from good practice so far.

How do my students feel about what they have learnt already?

How confident are they in taking on a new challenge or applying what they know already? Will they be able to find the courage to try, to face possible struggles and in some cases failure at the first, second or even third attempt? Here we are beginning to open the LQ box of questions. To include this aspect in lesson planning is not too difficult and there are strategies that can be employed to help learners overcome confidence issues, to become learning heroes and understand the challenges faced in the quest to conquer the unknown or new.

How do I begin by sharing the learning challenges ahead?

In planning terms we may refer to this as the “Introduction” but only if we focus on the content and not the process. Sharing the challenges and involving the learner in planning to meet them is part of the LQ approach in planning and it is sometimes referred to as learner centred teaching.  New topics can be approached in a number of ways and asking the learners to identify the most appropriate (even if this involves an element of guiding) helps share the ownership and responsibility for learning.  It also develops LQ since lessons can be learnt from the how of learning as well as the knowledge or understanding itself. Sharing this aspect of planning is a little like offering a choice at meal time, it is difficult to push the plate away and say, “I don’t like this” when you have chosen it!

Here are some more LQ planning questions and requirements for you to consider:

  • How do I share my enthusiasm for this topic?
  • How do I elicit and include the ideas of the learners in my planning, preparation and resourcing?
  • How do I describe achievement and how will the students recognise it?
  • How do we work together to achieve and in doing so share the challenges?
  • What will my role be in the learning process be and how do I signal this to the students?
  • How will we celebrate achievement together and as individuals?
  • How does the student go about reviewing their achievement against their learning map (what they believe they can and cannot learn) in order to re draw it to include new information about themselves (LQ)?
  • What resources will be required to support them emotionally through the learning challenges?

LQ involves considering emotions and feeling about learning and coming to terms with them as a natural part of the learning environment.

One emotion that features a great deal at the start of something new is fear. Fear is often associated with rejection, of no longer being part of a group with which we want to be identified. If you have ever experienced rejection you will see why failure is so feared.

Having a sense of belonging* is one of our four basic needs as learners without it we find learning much harder. We need to recognise that this emotional state is often the starting point for many learners when faced with a new challenge. If we fail to consider it in our planning then we are being rather cruel and possibly limiting the success of learners.

I firmly believe LQ is an antidote to the fear of failure and leads to the sense of inclusion that builds belonging and leads to successful learning experiences.

If you want to find out more about LQ then follow this blog and Tweets from @4c3d. Please also remember if you would like to provide a workshop or organise a talk about LQ then your organisation can contact me by e-mail to make the necessary arrangements.

*Belonging is part of the “Please Be Child Friendly” approach developed by ace-d and stands for the 4 learning needs:

Power – Belonging – Choice and Fun.

LQ and PBCF

 

 

Learning Intelligence (LQ) and the link to Assessment

assessment

What does an assessment mean to you?

a)      Revision

b)      A time of dread

c)       A signpost and recognition of achievements

d)      A mark or grade and little more

e)      A way of helping you to choose where to focus future efforts

f)       A way of setting future targets or goals

Before suggesting or considering an answer we need to reflect on John Hattie’s table of effect sizes which claims to answer the question “what has the greatest influence on student learning?”[i]  Here are the top 11. An effect size of 0.5 is the equivalent to a one grade increase at GCSE.

Influence Effect Size Source of Influence
Feedback 1.13 Teacher
Student’s prior cognitive ability 1.04 Student
Instructional quality 1.00 Teacher
Direct instruction .82 Teacher
Acceleration .72 Student
Remediation/feedback .65 Teacher
Student’s disposition to learn .61 Student
Class environment .56 Teacher
Challenge of Goals .52 Teacher
Peer tutoring .50 Teacher
Mastery learning .50 Teacher

The term feedback is a broad one and includes providing commentary on what they (the learner) have done well and where they need to improve. Importantly, and this is where the link to LQ comes in, Hattie believes students can receive feedback on the process or strategy they have used to complete the task and regulate (the LQ term for which is manage) their own learning.

With this in mind what did you suggest an assessment meant to you? You will recognise, no doubt, the elements of an assessment which are of greatest value link directly to Hattie’s effect table and especially that of the top effect “Feedback”. You will also note that the source of influence for feedback is associated with the teacher. So how do you as a teacher give feedback and how do you get learners to see the importance of the different elements?

Hands up if, when you give back the marked test papers, your students do the following:

1)      Look at the overall mark or grade first followed by a question about what is acceptable/a pass/a fail etc.

2)      Check your arithmetic in the hope of challenging you on the total or grade given.

3)      Check with their friends to see you have given the same mark for a similar answer, just in case they can score a few more points.

4)      Check to see what others got to establish either ranking or comfort themselves.

You would be no different to many other teachers if this is how your class of learners respond. How many of these fall into Hattie’s description of feedback? None perhaps! If you are a learner then once again you are probably no different to many other learners who respond in an identical way when they get their papers back.

The challenge is to change the way we respond to assessments, especially the rich source of feedback that is so often overlooked or missed. This is even more of a concern when you think it could make the difference of two grades at GCSE. We are failing to make the most of a valuable resource but why? In part it has something to do with targets and the focus on achieving them. If we know we should get a “B” for example, and we score a B then we are reluctant to dig any deeper. If we get an A then we rejoice and investigate no further. If we get a C or less perhaps we lick our wounds and look to what or who we can blame for the failure. This last course of action is where the most damage is done and a worthy read is Dweck’s work on “attribution theory.” A learner may attribute their success or lack of it on external factors such as luck or decide it was an exceptionally hard test. This type of response does not help in future growth or learning as it suggests the outcome had nothing to do with their efforts.

LQ is a valuable asset in managing the process of feedback for both teachers and learners but both must work co-operatively to achieve the greatest benefit. The simplest and most effective strategy for the teacher is to mark and comment on an assessment but not total the overall score or give a grade. Try it and see where you get for you have just prevented the most common reactions detailed above. So how will the learner respond, well much depends on how much work you have done regarding LQ and in discussing managing the learning environment. If you have done little or nothing then uproar and rebellion may result. There may be a demand to give them the standard results they expected, a grade, or total score, but you must resist. Instead you can now get them to focus on what they did well in, what they need to improve and set their own personal targets for future learning. This is a relatively easy task but requires some preparation and a little bit of colouring in! Here is how you do it.

Decide for each question or assessment task what, in terms of response or score, defines the following:

a)      a sound answer demonstrating a good grasp of what is required. The sort of response that suggests mastery or understanding and an ability to apply what has been learnt in an unfamiliar situation. You would consider it unnecessary for the student to spend more time on this topic or material.

b)      an answer which shows a developing understanding of what is required. Typically the key concepts or ideas are applied but there may be mistakes or errors that indicate a degree of uncertainty and a need for further practice or revision.

c)       An answer which shows a lack of understanding of what is required.  This may be characterised by large unanswered sections, the wrong facts or approach used. This would indicate a need for more than just revision but instead re visiting the topic and building understanding from a known point.

In essence this is all you and the student should be interested in. Your next task is to decide what these answers look like in terms of marks or grades if that is how they would normally be marked. For example, out of 10 each category may look like this:

a)      8+ marks,

b)      between 5 and 7 marks and

c)       anything less than 5 marks.

To help visualise what this means in terms of LQ and quality feedback we can colour code each category of answer or provide smiley faces to ring or delete at the front of the assessment is an ideal way to do this.[2]

TuitionGuru assessmentIf as a teacher we consider the class response in terms of outcomes to the assessment and, if this represents multiple questions, we get a clear direction of where we need to focus and, when considering LQ, how we need to help learners meet the challenges of the learning environment. For example if in the above example of a final assessment all students received a red category (the lowest) in Q6 but the rest were orange and green then we would know we need to address a specific topic or area. It may not be the topic that limited the learning but the resources, teaching style or learning environment in which it is taught. Thinking about these areas will help the teacher provide quality feedback and help in deciding if it would be appropriate to approach a class or a topic in a different way.

What this means for the Teacher

A key principle is when designing assessments is to start with the understanding, knowledge, or skill you wish to assess and work backwards to the method, question or strategy you will use to allow the student to demonstrate what they have learnt and for you to gain your assessment of them.

When administrating assessments take time to explain the purpose is to establish what has been learnt and not to gain a particular mark or grade.

What this means for the learner

If you are presented with a traditional assessment mark or grade try to unpick from your answers what you are a) competent in, b) need to revise and c) need to revisit to gain a better understanding.

Think about your preparation and look for alternative approaches to those things you need to revise or revisit. This is LQ in action, learning to manage your learning environment to meet your needs.

Finally

You may also be interested in the “Teaching Ideas” series to be found on my website. The link is at the top of the blog.

[i] Professor John Hattie’s Table of Effect Sizes can be found at: http://www.teacherstoolbox.co.uk/T_effect_sizes.html

[2] TuitionGuru www.coachingmathsandenglish.co.uk

%d bloggers like this: