Archive for July 31st, 2012

A recent article at Slate by Karla Starr has me considering the effects of order on my grading practices. The subject of the article is the question of whether Olympic gymnasts benefit from performing later in the program and the answer appears to be, “yes”. Starr writes:

Step-by-step evaluations by highly trained judges also suffer from a myriad of biases. In Olympic gymnastics, evaluators are given direct instructions by the International Gymnastics Federation to base the participants’ scores on an ideal version of a performance with the same elements. The first performances are thus typically judged against a mythic, Platonic idea of perfection. Early in competitions, judges also tend to dole out moderate scores in the event that later routines will be even more deserving of high marks.

Later performances are scored according to the judges’ revised standard of performance: namely, that established by the first performances. In addition to inadvertently lowering their standards, judges tend to focus on the unique, positive traits of the later performances—something that’s impossible for them to do for the first performers. One of Bruine de Bruin’s studies, which analyzed figure-skating results from 1994 to 2004, found that the last to perform had a 14 percent chance of winning, compared to a mere 3 percent for the first participants.

Performers also suffer or benefit from social comparison, an effect that’s been verified by researchers Lysann Damisch and Thomas Mussweiler from the University of Cologne. Their analysis of the 2004 Olympic gymnastics competition concluded that a major influence on a gymnast’s score was the performance of the previous gymnast, which can alter a score by up to two-tenths of a point. As long as your performance isn’t marred by errors, going after a skilled gymnast actually increases your score.

The paragraphs above could easily be applied to the process of grading. I wonder about the extent to which most instructors take the time to counteract these effects and whether there is a more elegant (and time-effective) solution than grading papers and exams multiple times in different orders. If not, maybe we can make the process more fun by wearing different hats on each run through and pretending that we’re graders from different nations. Then we can leave bizarre comments on student work. For example: “The French judge may have liked your paper, but the Russian judge says it stinks!”

Read Full Post »