Contact

Teaching‎ > ‎

Tips on Writing Reviews of Academic Manuscripts

This short tip-sheet summarizes some helpful things to keep in mind when reviewing manuscripts for academic journals.  Reviewing is an important part of the discipline of being an academic. Helpful, critical reviews go a long way in making a paper better and improving the quality of scholarship in the journal you are reviewing for. Harsh, uncharitable reviews (I've written a few!) might make the reviewer feel better, but are not helpful for the paper's author. A reviewer's goal is both to protect the integrity of the discipline and the journal, but also to help colleagues improve upon their work.

How Long? How Long!?
Editors often have a hard time getting reviewers. The review of a paper takes a long time, often 5-7 months for the first review to come back. Reviews should be about 1-2 single space pages. Many people write 1/2 page reviews - these may be appropriate if the paper has major flaws or is a mismatch for the journal in question. In this case, you are writing more a summary judgment than detailed comments. Many journals send everything out for review, so often you will receive manuscripts to review that clearly don't fit in the discipline or the journal. Don't waste your time writing a detailed review. In these situations, it is also appropriate to write a private note the editor indicating that you feel this article has major issues.

But, if the paper is matched to the journal, and the author has spent time and attention on the subject, then you should write a full review. Don't write a half-page review for a paper that is appropriate for the journal and reasonable well-researched. This sends the wrong message to the author - that your time is so valuable that you could only write a few general remarks - again, part of the discipline of academia is the exchange of ideas between smart people.  That said, don't micro-manage their paper. It is, after all, not your paper, and you need to give authors the freedom to do things their own way. Do your job, be rigorous, but don't kill yourself.

Antagonizing Editors...
Don't say whether or not you want to reject/accept the paper in the comments to the editor. This annoys editors. Don't annoy editors. Your job is to review the paper, not accept or reject it. You will privately indicate to the editor if you think the paper should be accepted, RnR'd or rejected. Your review is supposed to justify your apparent decision. Present your case as to why this paper is or is not up to the standards of this journal. Be critical. But be nice. Explain how flaws are or are not fatal. But, if there are fatal flaws, it's OK to point them out. Some limitations might sink the paper. Try and think of suggestions to salvage the paper.

Your guiding thought throughout the review should be: How do I help this paper to get in? Think of ways to help the paper get better.  That said, some papers just need a stake driven through their hearts. But try to be nice while you pound it in!

Start Nice
Lots of people summarize the paper in the first few sentences of the review. If you want to summarize the paper, you can do that quickly and concisely. I don't generally do this. We all know what the paper is about. I just read it, and the author has spent a lot of time antagonizing over it. The editor has read the paper, too, and is a smart person. I try and start the review by writing something nice. If you really want the paper to get in even though it has big problems, go a little overboard on the niceties. 

Sociologists are notoriously bad writers. Not as bad as economists, mind you, but few sociologists are out their winning prizes for literature. If you come across a well-written paper, say so! Tell them how much you appreciate the care and attention they paid to the writing. 

Critique
Choose a handful of things that are wrong with the paper that can be fixed. Don't critique things that you know can't be fixed. If  the only way the author could fix your problem is by resurrecting the respondents from the dead and re-interviewing them using a 7 point scale, rather than a yes/no question about their attitudes about hospice care, then don't say it. Sometimes these issues come up. You have to decide if this constitutes a fatal flaw, or if, in spite of this limitation, the paper still makes a valuable contribution.

As you write the review, follow the progression of the paper. I usually structure my critiques this way: General complaints; inaccurate take on literature of the past; methodological issues (poor measurement of DV, poor controls, problems with representativeness, etc); problems translating the results into substantive results; wrong conclusions reached. I always think about whether the author has considered the historical and/or national context of their study and whether other contexts might produce different results. Sociologists like to generalize and can forget the importance of context. I also recommend places to shorten the paper. I look at the figures and tables and suggest ways to make them clearer and more readable.

I generally end with a set of bullet points that are detailed. These are things like spelling mistakes, grammatical errors, errors in tables, paragraphs to reorder, etc. Just don't do too much of this.

Finally, when you get a paper that takes an approach you don't agree with, you have a choice: either don't review it, or try and review it on its own merit. Scholarship is a democratic process. We need ideas we don't agree with to sharpen our own. If you are very strident on one side of an issue then you really have to think about whether you can review the paper onjectively.

Do's and Don't's
  • Don't write a line by line review. It's not feasible to micro-manage a paper. Make general points about whether the writing needs work. Also comment on length.
  • Don't make vague complaints about "a big literature" that is neglected. Add a citation or two.
  • Don't yell at the author and don't be mean. The temptation is there because it is anonymous.
  • Use clear, dispassionate language.
  • Don't impugn an author's character.
Responding to reviews.
The vast majority of papers are rejected at competitive journals. The game has gotten tougher at the top. It is normal to get cruel, negative reviews. Often important details are missed. To change this situation, the onus is on you to write better reviews!

Once you get your comments back, then write a cover letter back to the editor and tell them what you want to do with the paper. Concede on almost ALL minor issues, and don't pick fights on little things. Improve the paper to make it stronger and worthy of acceptance.

Also, write a blinded letter that responds to the comments of the reviewer. Itemize the reviewer concerns and then list how you responded to each of the reviewer's concerns. It's advisable to make the memo to the point and precise. Keep things concise and point to places in the text where things were fixed. Number or bullet ALL the concerns raised. Below each bullet explain how you dealt with the issue in question. You don't have to change the paper in the way suggested, but you have to justify why you didn't change it. Be certain you are right when you don't want to do what they ask.

You can disagree on a few issues, pick your battles VERY, VERY carefully. You want to resolve the argument and close it down. Say "I tried your method and it didn't work very well." 

Be respectful and courteous. You don't know who your reviewers are, how influential they are, and how many times this person will encounter your work. With the internet, it is hard to hide your identity from the reviewers (unless you make no reference to this paper on your CV, blog, or in conference proceedings). Sure, reviewers aren't supposed to try and find out who the author is, but assume it happens. The editor does know who you are. If you handle negative reviews professionally, this leaves a good impression. Editors are usually important people in the discipline. The review process is a professional conversation between you, the editor and the reviewers.