Hostname: page-component-6bf8c574d5-w79xw Total loading time: 0 Render date: 2025-02-21T06:13:15.542Z Has data issue: false hasContentIssue false

Seeing Your Name in Print: Unpacking the Mysteries of the Review Process at Political Science Scholarly Journals

Published online by Cambridge University Press:  10 July 2007

Andrew J. Polsky
Affiliation:
Hunter College and the Graduate Center, CUNY
Rights & Permissions [Opens in a new window]

Extract

“Publish or perish.” Every graduate student has heard the phrase. Many junior scholars understand that it reflects the cold reality of professional survival in any political science department that bases tenure and promotion decisions, in whole or in part, on a record of demonstrated scholarly achievement. Despite occasional pronouncements by college presidents or faculty committees that teaching should be given greater weight in personnel decisions, the pressure to publish will not soon subside. If anything, it may be becoming more acute, as graduate students entering the job market struggle to publish some portion of their dissertation in a respected scholarly journal so as to distinguish themselves from the pack of applicants presenting otherwise similar credentials.I wish to thank Jack Jacobs, Lenny Markovitz, Michael Hiscox, and the graduate students at the CUNY Graduate Center and Harvard University for many useful suggestions and comments on the presentations that served as the basis for this article. I have incorporated ideas offered by Jim Jackson, Marianne Stewart, and John Geer at an editors' roundtable in which we participated at the 2006 Midwest Political Science Association meeting. Bob Lineberry and two anonymous referees helped me to clarify certain points in the manuscript. I am responsible for any errors that remain.

Type
THE PROFESSION
Copyright
© 2007 The American Political Science Association

“Publish or perish.” Every graduate student has heard the phrase. Many junior scholars understand that it reflects the cold reality of professional survival in any political science department that bases tenure and promotion decisions, in whole or in part, on a record of demonstrated scholarly achievement. Despite occasional pronouncements by college presidents or faculty committees that teaching should be given greater weight in personnel decisions, the pressure to publish will not soon subside. If anything, it may be becoming more acute, as graduate students entering the job market struggle to publish some portion of their dissertation in a respected scholarly journal so as to distinguish themselves from the pack of applicants presenting otherwise similar credentials.

For all the anxiety that surrounds publication, the process by which journals review manuscripts and the basis upon which they reach decisions remains obscure. I propose here to peel back the lid. I do so with a constructive purpose—to offer some guidance to prospective authors, especially graduate students who hope to see their byline in print by the time they seek an academic appointment. Many manuscripts fail because authors commit serious but avoidable (indeed, sometimes obvious) errors. Although the standard review process of “double-blind” refereeing contains unpredictable elements, authors can take steps to improve their prospects. I also believe an understanding of the process can help ease the minds of those scholars who fear submitting their work to anonymous scrutiny, even as the tenure clock ticks away.

My comments reflect my experience as author, editorial board member, and editor. As with most of my peer editors, I was an author of journal articles before I assumed my present position in July 2005 as editor of Polity. My experience as author spanned the full range of decision possibilities: rejection at one journal, acceptance at another; revise-and-resubmit (“R&R”) resulting in both acceptance and rejection; R&R multiple times at the same journal; and acceptance pending minor revisions. I have heard of the mythic academic superstar who has never received anything other than a clear accept decision for an initial manuscript submission. Alas, most of us mere mortals have suffered our share of rebuffs. Editors such as myself, then, understand what it means to be on the receiving end of bad news. Since becoming editor, moreover, I have gained a better appreciation of what referees expect of manuscripts and learned to identify pitfalls that can doom potentially interesting and significant submissions.

When Is a Piece Ready to Submit?

A manuscript should be vetted by other scholars working in the same field before you submit it to a journal. Beware the enthusiastic professor in a graduate course who pronounces your seminar paper ready for publication. It isn't. Typically, a piece will be presented first at one or more scholarly conferences before submission. This gives you an opportunity to receive critical feedback that may identify obvious flaws and/or alert you to important literature you have overlooked. Your argument will be stronger for it. Note, however, that the process of presenting and revising is not the equivalent of receiving anonymous reviews: often graduate school peers, faculty colleagues, and discussants on panels will soften their criticisms so as not to wound or offend.

At the opposite extreme from those who rush prematurely to submit their papers for publication are the junior scholars who delay, delay, delay. They fine tune and polish in the elusive quest for perfection. Even as a tenure decision looms, they hesitate, intending just one more revision. If this describes you, bear one thing in mind: perfectionists don't publish.

My advice to the perfectionist is straightforward. Do not hold yourself to a higher standard than you expect of others. We all read work in graduate school that is flawed, yet still makes a significant, even seminal contribution to the discipline. Similarly, when we teach we include on course syllabi scholarship that is open to criticism—indeed, that is one means by which we teach our students to think critically. And in our own research, we cite literature that we have found useful even when we take issue with it. If imperfect scholarship is good enough for you to use routinely in your teaching and your research, your own imperfect work is good enough to submit for publication.

Where Should You Submit?

With a multitude of scholarly journals publishing work by political scientists, you will likely face a number of possibilities as you consider where to submit a manuscript. You must make a choice. Scholarly journals insist upon exclusive consideration, so you may submit your piece to only one journal at a time. If a target journal is not familiar to you or is not one of the general political science journals accepting material across the discipline, do your homework to confirm that it publishes scholarship similar to yours. Most journals have an online home page linking you to the table of contents of recent issues; a journal may make some articles available free for a limited period to boost interest. (The home page is also the place to find submission guidelines and policies, such as page limits.) University libraries have paper copies and may also subscribe to services such as JSTOR, which give their patrons electronic access to journal articles a few years after publication.

Opinions vary about the importance of a journal's reputational status for aspiring scholars and junior faculty. My own experience suggests that search committees are more interested in the fact that a job candidate has published than in the prestige of the journal. But the standards are sure to vary across institutions. Much the same applies to junior faculty facing periodic reviews and tenure decisions. In that case, at least, you should be able to learn from senior colleagues the relative value of publishing in particular journals. As a general rule of thumb, publishing in the most highly rated scholarly journals will do more for your career, but their acceptance rates will be quite low. Some top journals in the discipline accept fewer than 5% of submissions.

Of equal concern to job candidates and junior scholars is the review turnaround time. When journals relied on paper submissions and mailed hard copies to referees (a system common until the last few years), it often took five or six months to complete the review process. If a manuscript was rejected at the end of that period, the author had to begin again, faced with the same extended wait. New electronic review procedures at many journals have shortened the review cycle dramatically. Manuscripts can be sent to referees as email attachments; if a referee declines to review a piece, it can be sent to someone else a few days later. Reviews come back just as quickly. (On occasion we have received referee reports within 24 hours of the initial request.) At Polity, we guarantee authors a decision within two months of submission, and it rarely takes that long. Some journals that still rely on paper submissions have also found ways to shorten the review cycle, such as by adding a third referee at the outset. The American Political Science Review, which continues to require hard copy submissions, completes reviews within two months. Where time matters, then, you should certainly look first to journals that commit to prompt turnaround.

The Manuscript Review Process

When you submit your manuscript, you should receive an acknowledgment from the journal. It may contain a tracking number, an explanation of the review procedure, and a timetable for the review process. If you do not get confirmation within two or three weeks that your submission has been received, follow up to make certain that it has not gone astray. Never assume that silence means everything is going as it should. Editors differ over whether you should bother with a cover letter explaining why the piece is important or appropriate for the journal. In many cases the editor will never see the note (graduate students on the editorial staff may log new entries into the system or the process may be fully automated). I disregard a cover note because I believe the article either stands or falls on its own merits. Some editors, by contrast, find a note useful to help them situate a manuscript on a subject with which they are not familiar.

Journal editors undertake an initial assessment of manuscript submissions, a practice that varies widely across journals and by editors. This step is used as a filter to eliminate manuscripts that editors deem unsuitable for full review. A manuscript may be rejected up front for several reasons: it exceeds the journal's page maximum; it is essentially political and/or ideological rather than scholarly; the journal has a backlog of accepted manuscripts on the same subject or in the same field (an unpredictable factor that means looking at recent issues may actually mislead a prospective author); the piece is poorly written; or the editor believes the piece is deficient in some obvious way that would lead referees to reject it (see the discussion below of common manuscript flaws). Some editors send out most manuscripts for full review while others exercise a significant check during the initial evaluation. The process may also change when a new editor takes over at a journal—I exercise significantly greater discretion in screening out manuscripts upon arrival than did my predecessor.

Rejection prior to external review is one way in which editors function as gatekeepers for professional scholarship, and it raises a question about the openness of the review process. I justify the practice on several grounds. First and most important, no journal exercises a monopoly over publication, even within a subfield. This matters because, however much experience we may have, our judgment is fallible; we may err and miss the substantial merits of a piece where some obvious flaw has caught our attention. Were there no alternative, it would be hard to defend the screening practice. But authors always have other journals to which they can submit a piece when one editor has declined to consider it. Second, we need to be mindful of the burden we place on referees. They volunteer their time out of a sense of professional commitment. To send them a manuscript of poor quality or, depending on the journal's objectives, manifestly unsuitable content would represent an abuse of their dedication. Moreover, although the pool of potential reviewers may be sizable, it is still finite. We risk depleting the available supply if we draw upon it recklessly.

Manuscripts that pass the initial screening will typically be subjected to “double-blind” peer review—that is, at least in theory, neither the referee nor the author knows the identity of the other. I say “in theory” because in the present web-connected world it has become very difficult to conceal the identity of authors. If you have presented a paper at a conference and then used the title (or a close cousin) for your manuscript, a reviewer can do a simple Internet search to find out who you are. Similarly, working papers and graduate student colloquia schedules are often posted online. I believe, then, that we are fast approaching a situation in which journals can hope at best to achieve single-blind reviewing in which the referee's identity is confidential.

External referees will typically be chosen from two pools of scholars. Most journals have an editorial board whose members are expected to participate in the review process in some capacity. Sometimes manuscripts will be read exclusively by the editorial board, although that places a very heavy burden upon its members. At Polity, most manuscripts will be read by one editorial board member; because of the general scope of the journal, I ask the board referee to consider especially the appeal of a manuscript for a wider political science audience.

The second pool consists of scholars with expertise in the subject addressed by the manuscript. Journals have several methods for finding such specialists. Some journals cultivate a stable of proven referees, scholars who are reliable and conscientious, and maintain a data bank organized by subject competence. A manuscript itself may be a useful guide—the citations point to potential referees. Online search engines such as Google Scholar help editors identify academics who have recently published or presented work on related subjects. Lastly, authors are welcome to suggest potential reviewers (at Polity we usually invite them to do so), recognizing, of course, that the journal is not obligated to use those names. It does no harm to identify potential reviewers with your initial submission.

When potential referees are asked to review a manuscript, they are under no obligation to agree. They receive no compensation and often no public recognition for their services. Nevertheless, many agree to take on the task. Indeed, some do so without ever informing the journal, so the first indication that they have agreed comes when they return the review several weeks after they received the manuscript. (This is why at Polity we sometimes have received as many as five referee reports for a single manuscript.) Referee reports are often long and detailed, especially when critical of a manuscript. Having read hundreds of reports over the past two years, I have been impressed with the constructive thrust of the vast majority of reviews: most scholars seek to improve the work they evaluate. Referees are also conscientious about identifying ethical issues, such as when they believe they know who the author of a piece is or when they have previously reviewed the same manuscript for another journal.

Journals prefer two or more reviews for each manuscript placed under full review. Two reviews suffice when they are in clear agreement, but often referees differ. At Polity we always seek commitments from three referees (including the editorial board member) to evaluate a manuscript. Three referee reports offer a better chance for a majority recommendation. Authors benefit in another way, too: a third report may provide valuable additional feedback on the manuscript. Standard practice within political science (though, interestingly, not in some of the humanities) calls for journals to share referee reports with authors.

What Reviewers Want and Common Manuscript Weaknesses

Certain attributes make it much more likely that a manuscript will be accepted. First, though the point may be obvious, the piece should have something to say. Presentation also counts. Referees are partial to a manuscript that is “reader friendly.” They like a clear structure: the author presents the central claims and establishes their potential significance in the opening section, identifies where the argument fits in the current scholarly conversation or debate, explains and justifies the methodology and choice of evidence, presents the evidence in a logical manner, and concludes by connecting the pieces of the argument and restating the significance of the findings. I do not mean to suggest that only manuscripts following this format will be accepted. However, if you deviate from convention, you should do so for good reason and make that reason explicit.

When I undertake the initial assessment of a manuscript submitted to Polity, I first look for a clear statement of the “value added” of the piece. That is, I want to know what the manuscript will contribute of significance to our understanding of political phenomena and to scholarly discourse within a field or across the discipline. Assuming the author can demonstrate through the use of appropriate evidence his/her central claims, how does the result advance some ongoing scholarly conversation? This is, of course, a basic question we pose about all scholarship—the “so what?” challenge. Graduate students are told they will face this question at any job interview. It should not surprise you, then, to learn that journal editors and referees ask the same question. If on first reading I cannot find some clear indication of the value-added of a manuscript, I usually decline to send it out for full review.

Establishing the significance of the research involves in part a judicious review of the relevant and recent literature. Scholarly conversations may be advanced through a variety of contributions. You may intervene in a discourse to clarify a theoretical point that is underdeveloped, to resolve a conflict or tension between two competing theoretical positions by bringing to bear compelling new evidence, to suggest that productive insights may be derived from combining two approaches that heretofore have not been made to speak to each other, or for some other purpose. It is essential that your point of entry into scholarly debate be made clear. Your discussion of the literature also needs to be current. Referees typically do not condemn a manuscript that is missing some recent literature, but they have little patience when the discussion of scholarship is generally outdated.

Referees also appreciate a manuscript that is well written. This point bears some emphasis because as a discipline political science is not known for lucid, graceful prose. Yet many referees comment on the quality of writing. They prefer clear, succinct expression (as do editors struggling with page limits) and dislike repetitive phrasing and excessive use of jargon.

Much as certain qualities increase the likelihood of acceptance, common weaknesses doom many submissions, even though they may have some merit. There is no excuse for ignoring a journal's stated rules about maximum length or formatting conventions. (Editors may not care whether a submission follows the journal's citation style, so long as authors realize that their manuscript will have to conform should it be accepted. If your manuscript does not use the journal's style, check before you submit to determine whether your piece will be considered.) Similarly, a manuscript stands little chance of acceptance if it has been poorly copy-edited or it suffers from many careless writing mistakes. Proofread before submission. Better yet, get a second set of eyes to do so, too.

Many manuscripts fail the value-added or significance test. Sometimes the failure occurs because the project has not been effectively situated. An author may “talk around” a scholarly conversation without making plain the manuscript's unique contribution to that discourse. Vaguely associating your work with some body of literature does not suffice. And be explicit about how you want your work to be perceived: do not expect referees to make a connection for you. Some manuscripts appear to engage a scholarly debate but on closer inspection prove to have waged a heroic battle against a straw-person caricature.

Authors also err in assuming that an analysis of an unstudied or understudied phenomenon is inherently important. At a recent panel of journal editors, John Geer, editor of the Journal of Politics, noted that when he comes across the word “gap,” it sets off alarm bells. To fill a gap does nothing by itself to establish the importance of what is being studied. This applies to work in political theory as much as to empirical research—a manuscript on an obscure political thinker is not important merely because no one has paid attention to his/her writings before.

Problems of case selection and case-theory relations also undermine many manuscripts. Some research projects are “case-driven,” that is, motivated by an interest in a particular empirical phenomenon or text, rather than by a theoretical question. One result is the “case-heavy” manuscript, rich with description and anecdotal explanations but devoid of theoretical context. It is highly unlikely that such work will be accepted by any major general political science or highly-regarded specialized journal. Prospects are not much better for “theory as an afterthought,” case-driven manuscripts. Here the author recognizes that the project needs to be joined to some theoretical conversation. Unfortunately, however, because the case has not been selected to test some theoretical claim, the empirical evidence and the theoretical context are poorly integrated. The author often fails to justify the case selection, to explain how it is appropriate to adjudicate between contending theoretical perspectives or how it lets us eliminate competing explanations. In the case of political theory manuscripts, the author offers a new reading of familiar texts without explaining why we should prefer it to other interpretations.

A different kind of problem may arise with manuscripts that rest entirely upon a review of existing literature rather than upon original research. I believe such pieces may have value in suggesting new ways to view familiar phenomena, drawing upon published work in a way that generates fresh insights and that is likely to be productive in sparking future research. (Review essays fall in the same category.) In a discipline given over to highly specialized scholarship, we need work that can knit together the many scattered insights that emerge from our particular inquiries. That said, however, there is no place in major journals for the literature review that merely summarizes and restates what has been published elsewhere.

Editorial Decisions

Referees return their reports to the editor, typically accompanied by a recommendation to accept, reject, or revise-and-resubmit. (Delays may result if a referee fails to return a review by the prescribed deadline and the journal is forced to seek another evaluation.) Journal decision rules vary, from a unanimity requirement for a piece to be accepted to the more common majority principle. But the application of a decision rule may call for significant editorial discretion and represents a key point at which an editor may exercise decisive judgment. Many scholars are loathe to reject a piece outright, preferring instead to call for a drastic recasting of a weak or seriously flawed manuscript. I am guided more by the substance of the referee report. If the report effectively says the author might have the kernel of an interesting idea but essentially needs to start over from scratch, I interpret that as a rejection. Moreover, three-way split decisions are not uncommon: one reviewer recommends acceptance, one calls for rejection, and the third wants to invite resubmission after revision. Here the editor has to decide whether the positives outweigh the negatives sufficiently to justify revise-and-resubmit. Finally, not all referees are created equal. Journals may attach added weight to a report from a member of the editorial board (Polity's practice) or from a reviewer with a proven track record.

As the recommendation hierarchy implies, three types of decisions are possible. First, a manuscript may be accepted or accepted pending minor revisions. Where minor revisions are needed, the piece will not be sent back to the referees for their approval. The editor decides whether the changes are satisfactory. In all but a tiny fraction of cases, a manuscript accepted pending minor revisions will be published.

Second, a manuscript may be rejected. For most journals, the editor's decision is final—there is no group to whom you can appeal a rejection. Editorial boards do not function to oversee editors; nor do the governing associations of regional journals such as Polity operate as a kind of appellate court for individual manuscripts. It is unlikely that you will be able to persuade an editor to reverse a rejection decision, though you might attempt to do so if you believe a referee report to be unsound or grossly unfair and you can give compelling reasons why the editor should discard it and seek another review. On the rare occasion when I made the case to an editor that a referee misconstrued something I had written or seemed to be pursuing his/her own agenda, I found the editor to be open-minded. That said, asking an editor to reconsider should be done only under exceptional circumstances. Unless the other referee reports are significantly more positive, it is a waste of time. Pursue some other publication option instead.

Third, and most problematic, a manuscript may be given a revise-and-resubmit verdict. R&R can be troubling for editors and authors alike because it sends ambiguous signals. For some journals, straightforward acceptance upon initial submission is rare and R&R signifies a positive response and a high probability of acceptance. An editor would be reluctant to invite resubmission, then, for anything other than strong manuscripts. At other journals, the invitation to submit a new version of a piece connotes nothing more than a willingness to send the revision through the full review process again. Authors may be puzzled about whether they are being encouraged to try again and what the likelihood of success is. To add to the confusion, referees may call for quite different and possibly incompatible changes.

I try to bring as much clarity to the R& R decision as possible. First, I suggest the most important criticisms to address and to identify common critical themes running through two or more referee reports. In my experience as an author, I have found this type of editorial guidance to be common, and very helpful. Second, I state clearly the review process I intend to use for a resubmission. Most journals send revised manuscripts back to the original referees. I will indicate to an author the referees (identified only by letters) to whom I plan to send the revision and how many must give an affirmative recommendation to publish in order for me to accept the piece. In the case of a “low” R&R, the manuscript may go back to all of the original referees. Some journal editors routinely explain how they will treat a resubmission, but the practice is not universal. It cannot hurt to ask how a revised manuscript would be reviewed as that may help you decide whether to pursue publication elsewhere.

A revise-and-resubmit decision places the author under no obligation. You may take your manuscript to another journal or, if major changes are required or your research focus shifts, choose to abandon it. Journals may set a deadline after which they will not consider a revision or simply treat it as a new submission. Few journals will follow up to inquire about your intention to resubmit.

Dealing with Rejection

Sooner or later a manuscript you submit will be rejected. Indeed, it may well be the first piece you submit precisely because you lack experience and are thus prone to the mistakes beginners make. And since you also may not have the self-confidence that prior success helps engender, rejection may hit you very hard. Draw comfort from the fact that we have all gone through it. You do need to develop a thick skin because many manuscripts have to be submitted to more than one journal before being accepted.

If you treat rejection as a means by which to secure useful critical feedback, moreover, it can have a constructive value. The day you receive the bad news and read the referee reports for the first time, you may feel a certain defensiveness. I speak from experience here. I have read anonymous readers' reports that (I was certain!) misconstrued or misunderstood my arguments, chastised me for not having read something I should have read (sometimes written, I churlishly suspected, by none other than the referee), or suggested I revise my submission to produce the entirely different article the referee thought I should have been writing. But when you look again at the reports after a few days, you may well have a different reaction. You will see that several scholars have taken a good deal of time to offer detailed and thoughtful comments that point out inconsistencies in your analysis, note that your evidence falls well short of confirming your initial claims, suggest different ways of looking at your subject that may yield fruitful insights, call your attention to vital scholarship you overlooked or to work in related subfields that may shed new light on the case(s) you have examined, and encourage you to be bolder (or perhaps more responsible) in your conclusions. In my two years as editor of Polity, I have been impressed with the consistent professionalism of the scholars who offer their time to help improve manuscripts even when recommending rejection.

Recognizing that the referee reports you received represent the opinions only of those scholars, you may want to send the piece out immediately to another journal. I advise you to take some time first to incorporate some of the suggestions and address some of the criticisms the referees have raised. There is one very practical reason for doing so. The manuscript may end up in the hands of the same referee(s), solicited now by Journal Two. Although editors may prefer to have a manuscript seen by fresh eyes, we also like to know whether an author responds to feedback constructively. Few referees will be positively inclined toward a piece the second time if the author has simply ignored their suggestions from the first review. Further, almost any piece will be improved through a revision based on thoughtful critical comments. Why send a flawed manuscript to another journal when the opportunity to improve it has been handed to you?

Conclusion

The reviewing process at scholarly journals is designed to put manuscripts through a rigorous process of peer evaluation. Although it is less than perfect, it yields scholarship of high quality and helps improve that scholarship through the review process itself. It is not hard to see where error can occur. Human judgment plays an important part in the evaluation of a piece from the initial submission through the blind review by outside evaluators to the final decision by the editor. An editor may be too quick to dismiss a manuscript up front; referees may harbor an animus to a certain type of scholarship that they choose not to disclose to an editor; or an editor may give undue weight to one negative report that is detailed to the point of nitpicking at the expense of briefer but more positive assessments. For all that, most scholars want to encourage good work, even where they disagree with an author's claims. I have read favorable reports by scholars whose own views were under attack in a manuscript. On the other side, I can count the number of reviews that struck a demeaning and unprofessional tone on the fingers of one hand.

As with many human activities, one can only learn so much about getting published from reading about it or listening to those who have done it. You need to try it yourself—to put your scholarship out there to be judged by others and steel yourself for the early lumps you may take along the way. Expect some frustration. But understand, too, that when your manuscript is accepted, you will have accomplished something quite remarkable—you will have made an original contribution to what the community of scholars knows about politics.

Author Bio

Andrew J. Polsky, professor of political science at Hunter College and the Graduate Center, CUNY, serves as the editor of Polity. He is the author of The Rise of the Therapeutic State (1991). His articles have appeared in Studies in American Political Development, Journal of Theoretical Politics, American Politics Research, and other journals.