The broken review system

A while back, Philip Cohen posted on some of the problems with the peer review system in sociology, sharing the story of a paper that underwent 13 peer reviews over several years in the publication gauntlet. Although the paper’s findings were essentially unchanged by this process, each reviewer apparently thought that the paper could be framed in a different, and better, way. As Cohen says:

Most (or all) of the reviewers were sociologists, and most of what they suggested, complained about, or objected was about the way the paper was “framed,” that is, how we establish the importance of the question and interpret the results. Of course framing is important – it’s why you’re asking your question, and why readers should care (see Mark Granovetter’s note on the rejected version of “the Strength of Weak Ties”). But it takes on elevated importance when we’re scrapping over limited slots in academic journals, so that to get published you have to successfully “frame” your paper as more important than some other poor slob’s.

Cohen also cites problems with the journal system and its speed and arbitrary nature, but I think that the issue of framing is particularly important because journal editors appear to be letting reviewers asking a version of the “why didn’t you write the paper I would have written?” conference question arbitrarily prevent the publication of otherwise-worthy papers. This is particularly problematic for graduate students and those of us who work at teaching-oriented institutions and don’t typically have numerous papers under review at once.

Cohen proposes an alternate peer review system, but barring major changes in the system, I think that editors can take immediate steps to address this issue. When somebody asks the “why didn’t you write the paper I would have written?” question at a conference, everybody else in the audience rolls their eyes and recognizes the problem. Journal editors need to be more forceful about recognizing these problems themselves, providing clearer review guidelines and ensuring that framing is not the single most important factor in their decisions.

Via: Scatterplot

—

“Like” Memoirs of a SLACer on Facebook to receive updates and links via your news feed and use the comments to complain about their framing.

Read Full Post »

The reality of writing at teaching-oriented institutions

The other day Tenured Radical had a great post about the reality of writing in academia in general, but especially at teaching-oriented institutions:

The truth is that the vast majority of academic jobs, and some of the jobs that people want most because they conform to our romance of what higher education ought to be, are the least likely to forward one’s life as a writer and a scholar. Do you believe in faculty governance? OK, then, slice about six to eight hours out of your week for it, unless you are in the faculty senate or on some other major committee, and then take out another five hours. Are you a dedicated teacher? Six to seven hours a week, per class, until you start to enter Grading Hell about the middle of February, and then you can double that commitment. Do you like students? Well, then they will love you! Reserve another four to six hours a week for scheduled and unscheduled office hours, Mr. Chips, and this doesn’t even begin to count the hours you will spend advising and writing letters of recommendation.

I became increasingly aware over the last 25 years that peers who did not work at teaching intensive colleges had a great deal more time to spend on their writing. Yet strangely people act as if all full-time academic jobs are more or less the same, and that we all are similarly accomplished. We act as though there are not more than a very few people who work under the conditions that allow them to write more. In fact, I would argue that there is a kind of accelerator effect in academia, in which people who have access to the best fellowships and best jobs coming out of graduate school will, increasingly have access to more time to write than other people. It is these people who set the standard for excellence that, in the end, the vast majority of academics are expected by their institutions, and expect themselves, to meet.

This is a problem that I’ve discussed before in terms of academic false consciousness and the publication gauntlet. I am also hurrying to get some writing done before the crush of grading makes it practically impossible. While I think her identification of the problem is right on, most of her proposed solutions are aimed at helping faculty write more rather than addressing the fact that expectations no longer match a practical definition of reality for most academics.

False consciousness strikes again!

Read Full Post »

Academic phase transition in the liberal arts

Fabio’s post at Orgtheory today about academic phase transition, in which academics go from being in low demand to being in high demand very quickly, made me think about the experiences of one of my acquaintances from grad school in the publication gauntlet and, to a lesser extent, my own recent experiences.

Quite a few years ago at ASA I was talking to an acquaintance who had graduated and started working at a liberal arts school where he was about to go up for tenure. He was somewhat concerned because the school did not clearly define what the publication expectation was for junior faculty. At the time, he had published one peer reviewed article since starting his job and the official word of the administration was that junior faculty did not need at least two publications but that they did need more than one.

A few years later I was wondering if he had been able to get the necessary (but not required) second publication and checked his profile on the school’s web page. He had published two papers in the year he went up for tenure and two more in the year after receiving tenure. Checking his profile today he has published at least one paper in every year since.

As much as we like to think that we come out of graduate school as fully-formed academics, I suspect that for most people this process is not complete when they receive their Ph.D.s. Personally, it took years before I was able to settle into my current position where I am able to balance teaching and service with getting a bit of research done.  Although my publication productivity has been relatively low up to this point, I hope that I am on the cusp of an academic phase transition of my own.

Read Full Post »

Running the publication gauntlet

I’ve talked about aspects of the publication gauntlet before, but this account of Rick Trebino’s quest to publish a response to, and correction of, an incorrect publication makes you wonder if anybody would go to these lengths if the paper in question did not falsely invalidate their previous research.  For example:

Ouch.  Maybe I was being foolish when I assumed that my consideration of jobs pumping septic tanks would correspond with the end of graduate school.

Via: Wicked Anomie

Read Full Post »