Service to the discipline is a crucial part of our professional lives. The Blog of the APA encourages members to propose or contribute posts about any aspect of service activities, including, but not limited to: journal refereeing and editing, conference organizing, maintaining scholarly resources, administrative roles in scholarly societies, advising, and mentoring. If you would like to propose or submit a contribution, we’d love to hear from you. Please contact us via the submission form here.
*
by Kathryn J. Norlock
Look at all those philosophical journals, gosh. There are hundreds.With so many journals producing thousands of articles each year, surely there is room enough for the publications of philosophers, but you wouldn’t know it from our high rejection rates (Philosophy averages a 92% rejection rate compared to psychology’s 78%, according to Lee and Schunn 2010). Anglophone philosophy journals remain so dedicated to peer review and super-selectivity that many philosophers do a great deal of refereeing with an eye to justification for rejection of a submission.
As more than one online commenter has noted, the result is the occasional peer-review that can be summarized as follows: “I can imagine objections to points or arguments in this essay. Therefore, I recommend rejection.” Thankfully, not all referees do this—if they did, almost nothing would be published in the field. Philosophers have good imaginations, and we can object to all but the most spare of logical proofs. Not all objections are grounds for rejection.
In a better world, no peer review would take the above form. I look in the mirror, and think that referee reports like these may be the faults of the journal editors who could do more to prevent them. Most of us in journal editing are not overwhelmed with eager referees who would love to do more service. Sometimes we take what we can get after five or fifteen or twenty-five requests to referee are declined. But we should more often state, in instructions to referees, “Please remember that not all objections are grounds for rejection.” Not every referee needs the reminder, but referees can include the inexperienced, the uncertain, and even the wicked.
Well, I doubt a reminder can stop the wicked. But for those merely new to refereeing or uncertain in their intuitions, here are some questions to help sort your objections:
- Is the author correct, on balance, about the general state of the literature with respect to their chosen topic? Perhaps they didn’t cite you and/or your favorite author, your advisor, or your loved one. Irks, right? I’ve gotten papers in my specialty that don’t cite me, and here’s my first reaction every time: HMPH! This is an understandable response. However, it is an indicator, and only an indicator, that the author did not make exactly the contribution to the subfield that they think they did. A paper that is correct, on balance, about their description of the landscape from a distance might be improved by citing you or any neglected others. It doesn’t need to be rejected.
- Assuming the author does (1) all right, then does the submission serve up a clear contribution that matters to future readers in the area of inquiry, and preferably on a silver platter, with multiple courses and hopefully a dessert? If the essay makes a clear contribution, you should be able to restate it easily in your response to the editor. If you cannot, if the author said there would be a platter and never brought it out, or kept tossing morsels at you but never identified the meal they were supposed to amount to, this is a fair reason to reject a paper. However, most papers do not exhaust all possibilities, so “I want more” is not an objection that always entails a rejection. Ask yourself if you want more because the author didn’t establish the author’s own claims and the relationship of their chosen supports to existing literature—reason for rejection—or if it’s really that you’d like to write a response to this paper when it comes out. If the latter, then you probably disagree with the content rather than find the development to be insufficient.
- Serious question: Do you find yourself writing objections to adverbs? An author that lays on the adverbs can make me cross. I get completely suspicious when authors totally overstate their cases that they are the only authors who notice a past publication is absolutely wrong. You Are My Nemesis, I find myself thinking. And then I must remind myself that over a decade ago, Dale Jacquette pointed out that “an author is not a referee’s adversary, even when the two sharply disagree philosophically.” So I take a deep breath, and do the following: I ask myself if the paper would be publishable were the adverbs removed. If so, then again, the objection does not justify rejection. The author may not even realize how overused were the adverbs. Advise revising.
- Is your objection to a submission more attentive to a problem in its structure, that is, the inferences, logic, or organization of ideas, or is it more focused on the content of an idea? Sadly, most of the rejections I have recommended were based on the former, and even more sadly for me, most of the rejections that I’ve received to my own work were based on the former. The former is a good reason to reject even a promising paper which bodes well for a future contribution. A publishable paper is not just intriguing in its ideas; it should be compelling in its procedure and persuasively arranged. But the objections that revolve more around the content of an idea come much closer to being reasons for supporting revision or publication, so that you can argue against the ideas in a fair and public way.
I’m in good company with others who have written on refereeing, and I haven’t tried to summarize all the excellent observations that already exist. Do add your own thoughts about considerations that help others sort objections from rejections.
Kathryn J. Norlock holds the Kenneth Mark Drain Endowed Chair in Ethics in the Philosophy Department at Trent University and is co-founder and co-editor of Feminist Philosophy Quarterly.
*
If you would like to submit a contribution, we’d love to hear from you. Please contact us via the submission form here.
Junior scholars are contacting me off-site to say they would have benefited from reading this when refereeing in graduate school and in their first year after the Ph.D. I feel a rant coming on for a future post, but will limit myself to this comment for now: Editors, limit the extent to which you’re asking graduate students and newly minted PhDs for refereeing service! And senior colleagues, when you’re asked to suggest other possible referees, your graduate students should not be your go-to. Some groundbreakers are the exception, I realize, but most of the time, come on, senior scholars can and should referee. /endrant
I feel much happier now I write shorter referee reports with max 3-4 things that I think are important for the paper, and about 1/2 page, at most 3/4 page of text. It takes significantly less time to complete this, and also, being on the receiving end, I feel that many comments is just micromanaging and doesn’t always benefit the paper.
I completely agree with Norlock’s remarks. The mere fact that one can think of an objection is not by itself ground to recommend rejection. I would go further: it is very often not even grounds to recommend revision. A good philosopher writing a paper may very readily be aware of thirty objections to what they say but, simply to keep the paper to manageable length (and perhaps to prevent it becoming unreadably tedious), pick out for discussion only the three or four they judge most important and interesting. The objection you thought of is very likely one of the other twenty-six. So I think it always important when refereeing to ask oneself, Have I just come up with an critical thought about this paper that might perhaps make an interesting contribution to a discussion of it at a seminar; or have I identified a serious hole in the argument of which the author seems clearly unaware? When I referee nowadays I quite often find myself saying things to the effect, Please pass on the appended comments to the author in case he/she may find them helpful or interesting but I don’t recommend addressing them should be made a condition of acceptance.
Thanks for your comments, Helen De Cruz and Jimmy Lenman. Both offerings here are markedly different from the emails I’m getting from newer/junior scholars regarding the massive work some of them do for referee reports. I actually feel more worried for new referees now than I did when I wrote the blog post! Allow me to offer here a few concrete suggestions that I’ve emailed the overburdened, which not everyone will agree with, but which may benefit someone:
1. Remind yourself that your first job is to help the editor make a decision. Helping the author is important but secondary. Think about the reason that you were sent this submission, by an editor, and what function the editor expects you to fulfill. They need a decision and *some* rationale. Ideally, we editors need an unambiguous call, and if recommending revisions, just two or three pages from referees. More is not necessarily better.
2. Relatedly, the primary (though not sole) responsibility for getting the author’s work to publishable quality rests with the author, not the referee or the editor. Your report should offer some transparency of reasons to the author but does not have to show the author exactly how to revise the paper. Reasons WHY are not the same as HOW. Your task is primarily to provide the WHY.
3. I got doubly worried at the correspondent who reported the time it takes to download and read the articles in the submission’s Works Cited, so I’m going to overstate the following to make my point: Don’t read all the works referred, because you don’t have to arbitrate whether or not the submission-author interpreted all the literature correctly. That’s an area of dispute over the content, better resolved in public, in publications, by other specialists. What matters more for publication-ability is whether or not the author makes a well-supported, self-contained, articulated case for their assertion that they offer a clear contribution to, and excellently appreciation of, the ongoing conversations on the topic, excellently written and organized, in a way that others can recognize as engaged with the literature. If you’ve got expertise in the area, you can usually go with what you know — but see (4).
4. Referees doing (3) because of unfamiliarity with most of the main supports for the paper are so beautifully well-intentioned, but you can just back out of refereeing after all, instead. I am always grateful when a colleague agrees to referee something for FPQ, such a relief! But I’d be accepting and understanding if I got back a message that reports, “Upon getting the entire work, I find it is not nearly as close to my areas of expertise as I had expected. I must retroactively decline to referee.” Okay, I wouldn’t be thrilled. But I’d be accepting and understanding anyway. (Also, your more senior colleagues say no to me all the time. They save themselves the trouble of finding out that something’s not in their area of expertise. You didn’t spare yourself the same trouble. Editors appreciate that, but we really do want you at your best. If you feel that you can’t be, just tell us you take it back. Be like the No-Sayers.)
In response to the question in the post title: if there is a particular objection that is likely to occur to many readers and the author shows no awareness of that objection, I think that is a good reason why the paper should not be published in its present form. However, if this is the most serious problem with the paper, an outright rejection is probably not warranted: give an R&R and see what the author comes up with in response!
Although I’m at a pretty early career stage, I’ve both given and received quite a few referee reports. (I probably should turn down more requests than I do.) Here are two other thoughts:
(1) It would be a good thing – a really good thing – if more journals would give specific instructions to referees and make these instructions available to authors. Most referee requests say little more than “tell us whether we should publish this article.” But many journals have only very vague statements of scope and aims on their web-sites; what really are the criteria supposed to be? In most cases, I have my own impression of what kind of journal this is and what kinds of things they publish, and my own opinion about what kinds of things a journal with their stated aims should publish, but am I really thinking about this in the same way the author and the editors are? I don’t know.
(2) In agreement with point 1 of Kate Norlock’s most recent comment, the more referee reports I receive the more I’m persuaded that the referee’s number one duty is to make his/her recommendation intelligible. It’s nice if you can actually persuade the editor and/or the author that your recommendation is correct, and it’s also nice if you can help the author improve the paper, but job number one is making sure that the reader of your report understands why you recommended what you did. The most irritating reports are the ones where it’s hard to figure out why the referee recommended rejection (and of course the editors usually go along with the referee even in cases like this).