As if academic false consciousness wasn’t bad enough at our institutions and among graduate students, it appears that it is also present at academic journals. Last week, Olderwoman at Scatterplot posted about receiving a packet of five reviews of the same article, stating:
Although five was over the top and freaked me out, it has become pretty common now for me as a reviewer to get a packet with four reviews. No wonder we regular reviewers are feeling under the gun. The old calculation of two or even three reviews per article has gone by the wayside. The pressure for fast turnaround and the high turn-down or non-response rate among potential reviewers has led editors to send out articles to extra reviewers in the hopes of ending up with at least the minimum two or three.
But this is a death spiral. As a frequently-sought reviewer I get at least four requests a month, sometimes as many as eight, and I used to get more before I got so crabby. When I was young and eager, I was reviewing an article a week [and thus, by the way, having a huge influence on my specialty area], and I know some people who are keeping that pace. But at some point you burn out and say “no more.” I, like all other frequently-sought reviewers I know, turn down outright the requests from journals I don’t know for articles that sound boring, and then save up the other requests and once a month pick which articles I want to review. So the interesting-sounding articles from good journals get too many reviewers, while the boring-sounding articles from no-name journals get none. If journal editors respond to the non-response by reviewers to boring-sounding articles by sending out even more reviewer requests per article, our mailboxes will be flooded even more and the non-response rate and delayed-response rate by reviewers will go up even more. Senior scholars are asked to review six to eight (or more?) articles per month. You have to say no to most of the requests.
And then we have the totally out of hand R&R problem. I think it is completely immoral to send an R&R to ANY new reviewers. I know a young scholar with a perfectly good paper who is now on the 4th (!!!!) iteration of an R&R from ASR. Not because she has not satisfied the original reviewers, but because the editors keep sending each revision to a new set of reviewers in addition to the original reviewers and, of course, the new reviewers have a different perspective and a new set of suggestions for the paper, some of which cover ground that was gone over in one or more of the previous revisions. Not to mention the problem that R&R memos are now longer than the original articles!! We are no longer a discipline of article publishing, we are turning into a discipline of R&R memo-writing.
She proposes several ground rules that she thinks would help the problem and that reminded me a bit of Gary Fine’s discussion of similar problems as editor of Social Psychology Quarterly.
Fabio followed her post with one of his own, talking specifically about ASR and the number of R&Rs that are given:
This issue has arisen with respect to the American Sociological Review, the flagship journal of the American Sociological Association. The ASR has been giving R&R’s to many submitted articles, much more than average, and they are soliciting many reviews per article. It has also been sending articles through multiple rounds of revisions, leading to articles being held at the journal for years. Since they seem to accept to same number of articles per year (about 40), that implies that the multiple rounds of revision do not lead to publication for many authors. Here is my response to that post:
I am asking the American Sociological Review to curtail this practice. In writing this, I have no personal stake in this matter. I do not have any papers under review, nor has the ASR accepted my previous submissions. I only write as a member of the profession, senior faculty at a top 20 program, a former managing editor of an ASA journal (Sociological Methodology), former associate editor of the American Journal of Sociology, occasional board member for various journals, author, and reviewer.
The inflated R&R policy is damaging sociology in a few ways. First, by continually R&R’ing papers that have little chance of publication, the ASR is “trapping” papers that may be perfectly suitable for specialty journals or other outlets. Thus, inflated R&Rs keep good research out of the public eye for years. You are suppressing science.
Second, inflated R&Rs damage the reputation of the ASR itself. The goal of a flagship journal is to be very picky. When people hear that a paper has been invited for revision, they believe that the editors think that the paper is of great merit and wide relevance. Inflated R&Rs undermine that perception.
Third, you are damaging people’s careers. By trapping papers, you preventing papers from being resubmitted to other journals that can help their careers. Also, R&R invitations are often seen as signs of intellectual progress, especially for doctoral students and junior faculty. By lumping together strong and weak papers, you are debasing the “currency” of the R&R. When people see “R&R at American Sociological Review,” they no longer know what to think and that pollutes the junior level job market.
Fourth, you are wasting precious time. Reviewers are usually full time faculty who teach, mentor graduate and undergraduate students, do administrative work, conduct research, and have full family lives. Thus, when you ask for a fourth reviewer, or a invite a paper for a third round of R&R, you are taking up many, many scarce resources.
Olderwoman, Fine, and Fabio all make valid points that need to be addressed by editors as well as their reviewers. More than any of the other instances of academic false consciousness, this seems like something that can be addressed quickly and relatively easily. Let’s do it.
Read Full Post »
Memoirs of a SLACer 