Wednesday, October 16, 2013

The insidious Dunning-Kruger effect, or why our skills don't look like skills, even to ourselves

I'm an English professor, right? And it frustrates me to no end that very few people seem to see the value of an English degree. We've all heard it from our relatives and colleagues in the real world: "So, are you going to be a teacher?" Or, "I'd better watch what I say around the grammar police!" Or, "Do you want fries with that?" Or, "How are you ever going to get a real job with no training?" Etc.

What's even worse, for me, is hearing people who've completed English degrees (I'm looking at you, Margaret Wente, and also, distressingly, a lot of people I myself went to school with) bemoaning the "fact" that they have no discernible skills to show for it.

That hurts. I can maybe wrap my head around why people who have no experience of university-level training in English to maybe wonder how many poetry repair shoppes its graduates can reasonable expect to sustain. But our own grads? Failing to see what excellent writers and thinkers they are?

People, this week I learned about the Dunning-Kruger effect, which is basically that most people are really really bad at assessing their own level of competence in a given field of endeavour both in absolute terms, and relative to others. Go read the link. Go. It's mindblowing, yet somehow obvious, work. This line of thinking has been crucial to totally reframing my understanding of why people in general and even people who should know better, are so eager to bash the humanities.

In brief, students who really don't get the material don't get it at such a level that they don't even know how little they get. Got it? So, a student in the bottom quartile of a class might tell a researcher with perfect guilelessness that she's a B- student. The idea is that there are some degrees of unknowing that are so deep that the unknower can't even see the difference between their own (bottom quartile) efforts and an example paper from the top-achieving student. The research suggests that showing these student an exemplar paper does not help them learn: they think that's what they already did.

To me, this explains how some quite skilled and smart people who don't have English degrees somehow cannot see the differences between what they write, and what skilled and trained writers write. They literally cannot see the difference, and so to them, their skills are equal to the trained writers. So it's not nastiness, it's a metacognitive thing.

More amazing to me still was the corollary finding that top performers also consistently misrate ... their relative level of skill. So an 85 student might say she thinks she has a roughly 85-level understanding, but where she makes her error is in thinking that most other students are probably getting 83 or 84. When in fact, they are getting 70. So the skilled student also falls into the error not of misrating her own knowledge, but of failing to recognize this skilled performace as skilled. So she is likely to downplay her talents as a baseline level of accomplishment that most people have. When they don't.

So where this led me was to at least beginning to think I understand why the world at large, and even, heart-breakingly, our own graduates, undervalue skills in writing, and critical thinking, and analysis that we work so hard to foster. They can't see the difference.

Now. How to fix?

2 comments:

  1. Well, I guess an extra layer of explanation you need in your story is why this happens in humanities but not in, e.g., math. In more formalized classes, it seems to me, one still sees the people at the top thinking "well, this is easy ... so easy everyone must be doing as well as me." What's different is that for those who are not getting it, the signals that they are not getting it are, shall we say, harder to disguise. This is related to why the marks tend to be bi-modal in more technical subjects. So part of the story, too, is the difference between a big red X at the bottom of every math assignment from grade 3 on and a little blue "C" at the end of an essay ... one's a much clearer way of saying "Oh dear, this is not good at all."

    Of course, even if that's right, there's no useful answer to your "what is to be done" question in this. "Be harsh in feedback on poor work" is hardly a strategy that deserves to be recommended. But on the basis of about 35 seconds of thought, it does seem to me like part of the story.

    ReplyDelete
  2. Today I had a student complain to me that the assignment guidelines made him do poorly. He blamed me for his grade, because he thinks that he is very good at the subject. I see this reasoning all the time. Good students, on the other hand, are very often harder on themselves than I would ever be. How to solve it? I honestly don't know. Our educational system is structured to allow both types of behaviour. I think that perhaps we should rethink what a learning objective is, even at the post-seondary level.

    ReplyDelete

Drop us a line! We're angling for vigorous commentary, but we will cut loose any vitriol dragged up from the depths.