Finding the good in bad reviews

Following up on my post about reviewing bad papers that Jeremy gave the Scatterplot bump, I completed my review while being as gentle as I could.  Although I did not search for the paper’s title before completing my review, I did so afterward and came up with nothing, supporting my belief that this work had not been previously presented in any form and leading me to suspect that it was written by a graduate student.  While most students in my own graduate program would not dream of submitting a paper for review that has not been read by numerous graduate students and professors, it is becoming increasingly clear that not all graduate student experiences are the same.

At any rate, shortly after completing the review I received a message from the journal’s editor containing the decision (a much-needed rejection) and the text of all of the reviews.  I thought that this was excellent, as it gave me an opportunity to see how others had framed their attempts to let the author down gently and also to compare the points that were raised.  It actually took me a moment to identify which of the reviews was my own because a number of them began by pointing out how interesting the idea was and how promising the research could be if done in a completely different way.

Incidentally, a friend of mine reviewed a paper for the same journal around the same time and was horrified that she received the same type of message because she worried that others might negatively judge her review.  Of course, the reviews were blind, so I had a hard time seeing how a chance to compare her own review with those of others could have possibly reflected negatively on her.  I tried pointing out how excited I had been at the opportunity to see how others approached the review but this failed to convince her that the practice was a good one.  (Maybe she has her own pseudonymous blog with a bizarro-world version of this post.)  Given my relative lack of reviewing experience, I’m not sure how common this practice is but my sense is that the answer is “not very.”  While opinions about the practice are split based on my sample size of two, as a young scholar I greatly appreciated the chance to gain a bit more insight into the reviewing process.