A beginner ’ s guide to reviewing manuscripts in ecology and conservation

Peer review is a hallmark of scientific publishing, yet finding peers to conduct reviews is increasingly challenging. Some attribute this challenge to the “tragedy of the referee commons,” wherein selfish behaviors related to the publication of scientific manuscripts lead to unsustainable use of potential referees and a reduction in the quality of scientific publications. To address the tragedy and maintain the quality of scientific publiccations, we could increase the degree to which junior referees are involved in the peer review process. Because these potential referees often have limited relevant experience and limited access to useful resources, we have developed a guide to reviewing manuscripts submitted for publication in journals that focus on conservation, ecology, or evolution. Herein, we provide an overview of the peer review process, discuss the role of junior reviewers in this process, and discuss how reviews should be conducted. This guide can serve to train both new and junior referees as well as to provide a refresher for senior referees.

We are suffering from the "tragedy of the referee commons" (Hochberg et al. 2009, Hochberg 2010).The selfish behaviors of authors, journals, and review bodies have increased the rate of manuscript submission, promoted unsustainable use of potential referees, and reduced the quality of scientific publications (Hochberg et al. 2009, Aarssen and Lortie 2009, Hochberg 2010)a collective disaster befitting Hardin's (1968) terminology.What can we do?We could increase the size of the referee pool by involving junior referees such as senior graduate students and postdoctoral scientists, increase the size of the referee pool by implementing a system of rewards and punishments related to review service, send fewer manuscripts out for review, reduce manuscript submission rate with high submission fees, or require submitting authors to provide peer reviews and vouch for their quality (Aarssen and Lortie 2010, Hochberg et al. 2009, Donaldson et al. 2010, Hochberg 2010).We believe that the first suggestion is the most pragmatic and least objectionable on philosophical, logistical, and ethical grounds.Indeed, some editors already solicit reviews from junior referees on a regular basis (Donaldson et al. 2010).Still, some are concerned that the reviews from junior referees are inferior in quality because these referees have limited experience with scientific research, mentors do not provide sufficient training in manuscript review, books on the review process are inaccessible and intimidating in scale (e.g., Hames 2007), and most journals provide limited guidelines for reviews.We address these concerns-and hopefully the tragedy-with the following guide to reviewing manuscripts submitted for publication in journals that focus on conservation, ecology, or evolution.While we focus on these three types of journals, the guide is applicable to other scientific fields and disciplines as well.

Understanding the peer review process
The rate at which science advances is positively correlated with the quality of published research manuscripts.To maintain the quality of these manuscripts via selective publication, the vast majority of journals have employed peer review for at least 150 years (DeVries et al. 2009, Despeaux 2011, Jackson et al. 2011).The basic process and value of peer review have received the attention of many able authors (e.g., Kliewer et al. 2004) and are probably understood by most junior referees, so we will restrict our comments to the mechanics of the review process relevant to most conservation, ecology, and evolution journals.Likewise, there are several worthwhile commentaries on how to conduct peer review that may be useful to readers (e.g., Hoppin 2002, Benos et al. 2003, Roberts et al. 2004), but these have focused on other disciplines or have not sought to introduce junior referees to the review process.
The peer review process begins when an Editor-in-Chief receives a manuscript to consider for publication in his/her journal (Figure 1).If the Editor-in-Chief feels the manuscript is appropriate for publication in the journal, he/she forwards the manuscript to an editor.The editor requests reviews from referees (perhaps including you), collects reviews, and makes publication recommendations to the Editor-in-Chief.If the decision involves (1) acceptance pending revision or (2) a request for substantial revision and resubmission, the editor or the editor and the referees may see the manuscript a second or third time.In short, the referees are the workhorses of the peer review process; without them, the system will fail.

Accepting the review
If you have received a request for a review, you should consider your expertise, availability, and bias before accepting or declining the offer.In other words, do you have the appropriate topical and/or statistical training or experience, enough time to complete a thorough review before the given deadline, and sufficient objectivity regarding the topic and the authors?A single negative answer should elicit a prompt and polite decline.However, a few comments on availability and bias are warranted.As a general rule, a review of an unpublished manuscript will require three to six times longer (excluding breaks between bouts of activity) than a read of a more polished, published manuscript.Carefully and candidly assess the amount of time you require to read, comment on, and submit your review.Then, decide if you have time to complete the review by the allotted deadline and communicate that decision to the journal as soon as possible; delayed responses by potential reviewers can substantially lengthen the review period and time to publication.Lastly, prevent bias both for and against the manuscript.If your opinion of the topic, knowledge of the authors or their institutions, or affiliation with authors or their institutions could color your decision on the manuscript's scientific merit or relevance to the journal or its audience, decline the review.Some journals (e.g., Landscape and Urban Planning and Society and Natural Resources) substantially reduce the potential for bias by employing a "double blind" system in which you and the authors are prevented from seeing (i.e.identifying) each other in correspondence.However, few journals in ecology and conservation are double blind (Budden et al. 2008).As a result, whether intended or not, bias has been a problem in the review process (Grod et al. 2008) and should be kept to a minimum.Finally, it is important to keep in mind that by accepting the review, you have promised to submit a review that is thorough, thoughtful, and on time.Your inability to fulfill this promise might allow the publication of low quality science or impede the publication of high quality science.

Reading the manuscript
Once you have accepted a review, you must access and read the manuscript.Until roughly eight years ago, you would have received a paper copy of the manuscript in the mail.Since that time, it has become more common to receive the manuscript as an email attachment or via access to a secure website.Regardless of your specific methods, we suggest reading the manuscript twice with one to seven days between reads.The short break encourages reflection and a deeper, less reactive review.Some seasoned referees prefer to perform their initial read without recording any comments, while others prefer to record comments during both reads.You can record your comments on the hardcopy, in the digital copy, or in a separate document.Whatever you prefer, relate each comment to a specific location in the manuscript (e.g., with line number; page, paragraph, and sentence; or digital comment) and realize that you will probably be required to submit a digital copy of your review.

Assessing the manuscript's format, content, and writing
Once you have accepted a review and read the manuscript, you are ready to provide what the Editor-in-Chief needs-an assessment of the manuscript's quality.You must assess quality with respect to format, content, and writing.Format is the easiest to assess, as most manuscripts must have seven sections in the following sequence: abstract, introduction, methods, results, discussion, literature cited, and tables and figures.If all sections are present, we recommend that novice reviewers begin their read with the introduction and proceed through the discussion, referencing literature cited, tables, and figures sections as necessary.The abstract should be read last for reasons that we discuss below.
The introduction must provide the framework for and the goal of the study.The framework must be established with a combination of logical argument and supporting literature.The latter should not be excessive, as most journals ask for an introduction of less than six paragraphs and the authors can and should cover more literature in the discussion.In fact, the entire manuscript must be concise, as all journals attempt to reduce manuscript length as much as possible so they can publish more studies per year.The goal of the study could be to describe a particular phenomenon or to test a hypothesis.Either way, the goal must be clearly stated (Please see Box 1 for a quick guide to content assessment).If a manuscript does not contain an obvious goal, then it is important to ask whether or not the study is actually engaged in scientific research or is simply constructing a report.While beyond the scope of these guidelines, it is essential that reviewers have an understanding of the philosophy of science and what are appropriate means of asking questions in science.
The methods must communicate how the study was conducted so that others could replicate the study.Typical topics (and subheadings) include study design, study sites, sampling units (e.g., individuals, communities, or landscapes) and replication, data collection and manipulation, and statistical analyses.While reading through this section, the reviewer should consider whether the approaches taken are adequately described and appropriate given the study's goal."Appropriate" need not mean "best," because studies with sub-optimal designs and datasets can still make meaningful contributions to science, particularly in developing research areas.Similarly, some flaws in methods can be corrected with revision.Note that the subsections on experimental design and statistical analysis may not be completely interpretable to some novice reviewers.If such a lack of interpretation occurs, there are several strategies to deal with this problem.First, one can attempt to learn about the described methods in standard bio-statistical texts (e.g., Gotelli and Ellison 2004, Zar 2009, Sokal and Rohlf 2011).If this is not possible or you do not understand the material, then a second option is to consider talking with a statistician, colleague, mentor, or supervisor to see if they are knowledgeable about the methods being used.Asking for assistance at this point can be valuable both in terms of learning the methods yourself and by improving your ability to review future manuscripts.Finally, if neither of these options is viable, then we suggest contacting the journal editor to ask if they would prefer you to review the manuscript without consideration of these elements or if you should simply stop the review.
The results section that follows must convey the patterns in the data that address the study's goal.In most cases, the patterns are best described with some combination of non-redundant text, tables, and figures and should be accompanied by statistical output.The mode by which a pattern is described should be dictated by both economy of space and clarity of presentation.There may be instances where a figure uses more space than text, but shows the pattern much more clearly (see Tufte 2001 for a thorough discussion on figure design).This section must not include interpretation or discussion of patterns.
At a minimum, the discussion section must interpret the results in light of the study's goal and provide additional context with citations to more, relevant literature.Because interpretation can be subjective, the bulk of a reviewer's comments may pertain to this section.These comments can and should focus on the appropriate interpretation of statistics, the strength with which a hypothesis was tested (sensu Platt 1964), and the scope and portrayal of the literature review.For some journals, this section may end with a sub-section on "management implications" or "future research directions." The last section with text, the abstract, must communicate the basic information in each of the aforementioned sections in a couple hundred words so that the casual reader can quickly determine whether they want to read the paper or add it to their database.For most journals, literature citations and statistical results are not reported here.Thus, the abstract should succinctly convey a bit of background, the main goal(s), questions, hypothesis, methodology, main results, and a sentence or two of interpretation or discussion.
The issues of content that have just been covered are more important than writing quality, because the latter can be improved more easily with revision.However, writing quality still needs to be reasonable in order to communicate said content.At a minimum, a reviewer should evaluate the manuscript's grammar (see Goldwasser 1998 for notable grammatical points to consider in ecological writing), logic, flow, and tone.The last element is important enough to deserve its own heading.

Establishing a respectful tone
One way to encourage a respectful tone in the manuscript is to establish a respectful tone in your review.You can achieve this goal by following the International Journal of Wildlife Law and Policy's "Golden Rule of reviewing," for which many journal editors advocate (McPeek et al. 2009).This rule requires that you "review the work of others as you would have them review your own work, i.e. carefully and with respect."Box 1. Quick reference guide for reviewing a manuscript.Answers to these questions should not all receive equal weight, but several "no's" should correspond to a more critical review.If "no's" primarily result from concerns over formatting or writing, try not to let these shortcomings color your assessment of content; all but the most severe problems with formatting and writing can be fixed with editing.The same is not generally true of problems with content. Does the writing follow all applicable rules of grammar? Are the concepts connected in a logical sequence? Is the tone respectful to those who have written on this topic or who will find the conclusions relevant?Overall Evaluation  Will the manuscript be interesting for a large proportion of the journal's audience? Will the manuscript substantially advance the relevant field of study by conducting a strong test and, perhaps, reducing the set of viable hypotheses?

Required content by section of the manuscript
Although this practice may seem self-evident, some referees struggle with it because they equate "review" with "critical" and assign a negative connotation to the latter.This problem is exacerbated by inclusion of both negative definitions (i.e.fault finding) and positive definitions (i.e.exercising careful judgment) of "critical" in the same dictionary (Oxford English Dictionary).This latter definition is far closer to how peer review should be approached, as it is constructive rather than destructive; it encourages authors to revise and improve their manuscript and engenders a positive attitude toward the journal and publishing in general.
Negative criticism is destructive because it can lead to a confrontationally toned revision and response to referees, a retreat into depression, anger, and-perhaps the worst-the decision to submit to a different journal without revision.The last effect only adds to the "tragedy of the referee commons."To illustrate the difference between positively-and negatively-toned comments, we have included several examples of constructive and destructive comments from reviews of our manuscripts, reviews we have received as editors, and from our colleagues (Table 1).
We could also encourage constructive criticism by changing how we critique manuscripts in coursework.For instance, many classes in graduate school have regular discussions of papers in which the students and professor completely pull apart a paper and may nitpick every item in a methodology.While it is instructive to learn to deconstruct papers and discuss flaws, it is also important to realize that this type of exercise is not the same as reviewing a manuscript for publication.Without this distinction, it is easy to train students to become 'pit bull' reviewers (Walbot 2009).In fact, a better mechanism is to distinguish the process of reading or criticizing published work from that of peer review through the use of peer review exercises in the classroom (e.g., Liu et al 2002).

Completing the review
As you approach completion of your review, you will need to format your review, make a specific recommendation regarding the manuscript, and choose whether or not to sign the review.The most common way to structure the review is to divide it into comments Table 1.Examples of positive (constructive) and negative (destructive) comments to authors Positive Negative "…the authors could remove most, if not all, of the core information without significant loss to the [manuscript].They can simply emphasize important points when necessary.""The Introduction would benefit from a more specific statement (or series of statements) regarding the overall objectives.""The accuracy assessment was done through validation where you reserved a portion of the data.This is a definite strength and should be noted as such.Also, how does the 83% pixel accuracy compare with other studies of similar nature?""By the end of Page 8, I was pretty lost in terms of lists and datasets…A table with the following information would help tremendously: dataset name; abbreviation; # of years the dataset covers; type of data collected…" "The rationale for the natural history analyses was not sufficiently developed in the manuscript (nor even mentioned in the introduction), making it seem more like a fishing expedition for patterns.A better approach would be to clearly define a priori predictions that are tightly linked to the 3 hypotheses in the introduction and then test these predictions.""I wouldn't call this 'progress' in the field of oceanography."For the journal, Progress in Oceanography.
"I really think you should reconsider what you are measuring, and what theory is appropriate.Therefore I will not evaluate the discussion.""A particular irritation (a point that instantly rouses my suspicions about the authors' understanding of the statistics used) is the misuse of the term "multivariate"." The second paragraph begins "In recent years the use and importance of houses in ecology, conservation, and management, has gained widespread appeal."This implies that managers stick houses out there as a management tool.My undergraduates can write more clearly than that, and they aren't very good writers.
I am not impressed with this chapter… It is over-written, badly structured, insufficiently questioning and inconclusive.Its chief intellectual fault is a consistently unskeptical acceptance of numerous untested assumptions about amphibian biology which, collectively, render moot any novel conclusions that might otherwise have been reached.It is thus uncritical and overtly strives to justify methods already in use rather than test their efficacy.This boosterism and lack of critical inquiry about assumptions and methods not only makes for a bad chapter but is potentially damaging.
to the editor and comments to the author (this is often required by the journal).These two sets of comments often overlap a great deal, but allow more candid comments to the editor.The comments to the author are often divided into general comments, such as those on major strengths and weaknesses, and specific comments, such as those on text flow and grammar.Organizing your review in this way, as well as providing it in digital format, will save the editor time and speed up the review process.Again, relate each specific comment to the relevant text's location.In the comments to the editor, you should also include your recommendation regarding the journal's treatment of the manuscript.In most cases, this recommendation will be to reject, reject with request to revise and resubmit, to accept with major revision, or to accept with minor revision.
Your final task will often be to decide whether to sign the comments to the authors.The idealist in us would like to tell you to sign your review because such attribution encourages the referees to be honest and supportive.However, the realist in us knows this is naïve; authors can harbor grudges that can affect your career in unfore-seen ways.This risk of signing far outweighs its benefit unless you are an established scientist who does not need to read these guidelines.
After you submit your review to the journal, you will likely receive a short note indicating that it has been received.Depending upon your review, the reviews of other referees, and the decisions of the editor and the Editor-in-Chief, you may or may not be asked to review this manuscript again in the future.Regardless of whether or not you are asked to review the manuscript again, it is good practice to save your reviews for future reference.

Final Thoughts
We wrote this guide to train junior referees and, thereby, avoid the tragedy of the reviewer commons.However, during the editing process, we realized that the guide might also have two other uses.First, if it encourages junior scientists to participate in the review process and improves the quality of those early experiences, this guide could increase the likelihood that these reviewers will, in turn, mentor junior referees.Second, this guide may serve as a model for classroom exercises on the peer review process.Such a model is useful, as "it is rare to find a scientist whose formal training has incorporated instruction in the art of reviewing" (Benos et al. 2003).If this guide is used in even one of the three ways just described, it will help to maintain the quality of scientific publication and the rate of scientific progress.

Response to Referee
As Donaldson (2011) elegantly notes, the process of learning how to peer review presents a dilemma.Without opportunities to review manuscripts, it is difficult to learn the review process, and if opportunities are given with no training, then it is difficult to produce a review of high quality.In our cases, we were both fortunate to have graduate advisors that not only gave us manuscripts to review, but taught us the process.However, based upon our professional experience, this is not the norm; many faculty members do not provide opportunities to review or a process to learn how.Thus, as Donaldson notes, the key to success is to find opportunities for review experience, real or simulated.
We concur with Donaldson on the numerous routes one can take to become involved in the peer review process.For instance, once a junior scientist has published a paper in a peer reviewed journal, he/she will likely be contacted in the future to perform a review.However, he/she need not wait to be called upon by a journal, but can also seek out opportunities.First, they can discuss reviewing manuscripts with their advisors or other graduate committee members and express their interest in learning how to conduct peer review.Second, they can directly contact journals or editors (i.e.unsolicited) and indicate their interest in reviewing for the journals.The editors may ask for statements of interest or CVs in order to assess areas of expertise.Finally, junior scientists can offer to review manuscripts written by colleagues or friends before they submit them to journals.
One aspect that Donaldson mentions and that bears further mention is how we can weave peer review into formal coursework.As he notes, several methods exist.Both of us have had success using some combination of the following in undergraduate, graduate, science, and general education courses: lectures on peer review process, presentation of examples of peer reviews, and supervised review of work by fellow students.
Our editorial experience has impressed us with the need to address the tragedy of the reviewer commons.Our teaching experience and Donaldson's ideas suggest several ways to prepare for that first, real peer review.Now, our job is to implement these ideas at more institutions.Please join us.

Figure 1 .
Figure 1.Directional flow of information between elements of the peer review process.Information may flow through a given arrow more than once.The arrow to the right of the Editor-in-Chief represents acceptance for publication and may or may not be achieved by any given manuscript written by an author or authors.
Donaldson, M.R. 2011.The junior referee conundrum: how do you gain experience with the peer review process?Ideas in Ecology and Evolution 4: 32-33.CrossRef