Peer Review

Peer review is meant to assess the quality of a paper. Peer-reviewed articles are supposed to be a trusted form of scientific communication. It is important that peer-reviewed work isn't necessarily correct or conclusive since scientific knowledge is cumulative, we run into troubles if we look at work in isolation. The peers (reviewers) are supposed to specialize in the same area as the authors and be impartial while reviewing (journals help anonymize authorship/reviewers through double and single blind review.

Process

1. Scientists research a study and write up an article. That article is submitted to a journal.
2. Depending on the journal, the article then enters the review process. At high impact journals, action editors decide whether or not the article deserves to be sent out to reviewers. At other journals, the editor contacts experts in the subfield of the article topic and asks for them to be reviewers.
3. Those reviewers will then provide feedback on the article and provide the editor with assessment whether the work and the article should be accepted for publication. Reviewers evaluate the article for its writing, the soundness of the methods, the appropriateness of the approach taken and the place the paper fits in the literature and the interpretation of the results.
4. The editor will then take the feedback from the reviewers and decide on how to move forward with the paper, usually choosing to either reject the paper or ask for the authors to respond to the feedback from the reviewers.
5. Articles that eventually meet some minimum of scientific standards (e.g., rigorous, well-designed studies, appropriately conducted and analyzed and thoughtfully placed in the literature) are accepted for publication.
6. You can find more in-depth information on the process from the Wiley publishers

Potential Issues with peer review

• Former BMJ editor Dr. Smith provides an overview of some of the issues with peer review. Below are a few critiques.
• Peer reviewers just aren't that good. 607 peer reviewers were randomized into two intervention groups receiving training on how to review and neither group performed well catching major errors in the papers assigned to them.
• Bias. One of the largest perceptions of issues with peer-review is bias.
• Objectivity. Scientists are humans, complete with human biases and errors. If a scientist views themselves as a "gatekeeper" of science, what drives their decision process?
• No reward for reviewers. Peer review is an essential, but time consuming venture.

How to Peer Review

Below are a number of guides to help you through the review process.

How to become good at peer review: A guide for young scientists

A guide from Jennifer Raff

10 tips for reviewing scientific manuscripts – and 5 red flags

A guide from the publisher Elsevier written by Joseph Alpert.

Top tips for peer reviewers

A guide from the publisher Wiley. Check out some of the links to other documentation about reviewing.

How to Review a Paper

A guide from Benos, Kirk and Hall published as a personal view in the journal Advanced Physiology Education

How to Review a Paper

A collection of advice from multiple scientists at different stages of the review process.

The Review

The review consists of two documents: one recommendation letter to the editor and one report to the study authors.

The letter is only viewed by the editor and used to make the editorial decision. It contains:

• Your name
• Submission identifying information: usually these are handled through the on-line submission repositories, but its nice to have the paper's title and journal ID code
• Your recommendation and justification

Your report contains the feedback for the authors. It contains:

• Summary - summarize the paper so that you show the authors and editor that you read and understood the paper.
• General Comments - Give big picture ideas. Remember the journal that you are reviewing for. Explore the role of the paper in the literature with the scope of the journal in mind. Comment on its place and contribution to the literature
• Constructive criticism - Go through section by section, ask question/clarification, offer ways to improve science, methodology, and organization/clarity. The point of this section is not to gatekeep, but to learn from the authors and help improve the contribution of the work to the field. Again remember the context of both the paper and the journal. Are you asking for something that is another entire study?
• Nitpicks - be brief here, but point out any glaring errors or omissions

References

Hojat, M., Gonnella, J.S., Caelleigh, A.S. (2003). Impartial judgment by the "gatekeepers" of science: fallibility and accountability in the peer review process. Adv Health Sci Educ Theory Pract, 8(1), 75-96.

Resnik, D.B., Guiterrez-Ford, C. & Peddada, S. (2008). Perceptions of ethical problems with scientific journal peer review: An exploratory study. Sci Eng Ethics, 14, 305-310.

Schroter, S., Black, N., Evans, S., Godlee, F., Osorio, L., & Smith, R. (2008). What errors do peer reviewers detect, and does training improve their ability to detect them? Journal of the Royal Society of Medicine, 101(10), 507-514.

Smith, R. (2006). Peer review: a flawed process at the heart of science and journals. Journal of the Royal Society of Medicine, 99(4), 178-182.

Bionaz, M. (2013). What scientific journals can do to improve the peer review process: Reward the reviewer! J Nutr Food Sci, 3, (120), doi: 10.4172/2155-9600.1000e120.

Wiley Author Services. The peer review process. Retrieved from https://authorservices.wiley.com/Reviewers/journal-reviewers/what-is-peer-review/the-peer-review-process.html