- A review should always
be polite, respectful to the author(s), and helpful for improving the paper,
regardless of whether you recommend acceptance or rejection.
- Start the review with
one or two sentences summarizing the paper.
- Communicate clearly
the strengths and weaknesses of the paper.
- Make sure that your
comments give the reasons for your recommendation.
- Be especially clear
in justifying a recommendation of rejection and provide feedback the author(s)
can use to improve the paper/work.
- If you feel the author(s)
should be aware of related work, try to provide specific references.
- Try to give, in terms
of quality, the kind of review you would like to receive for your own work.
- Keep the original paper
you reviewed for a few months (3 in case of conferences, up to 6 in case of
journal papers) so that you can discuss your review in case any controversy
arises.
- Do not distribute submitted
papers, they are meant to be confidential.
- Here are some (transcribed)
tips for paper reviewing from Allan Newell:
When we get a paper
to review, at the beginning we should always have as the default that we
accept the paper. While reading the paper, we may start raising specific
objections along the issues in the review form, namely the goals are not
stated, the system is not well described, the approach is not novel or not
validated, etc. etc.
Each objection weighs
a little against our initial default acceptance. Rejecting a paper is to
see if these objections weigh more than our threshold, based on our experience
with other other conferences, papers, and advice from the specific conference.
(Of course, the review can also raise the initial default acceptance, and
then it's even a clearer accept.)
One word of care that
I recall: It may happen that we raise either unjustified objections or support
to a paper. In particular in AI, it may be rather common that we completely
"disagree" or "agree" with the paper's approach/results. We have to be very
careful, as much as possible, not to include objections or support that
corresponds to subjective, or sometimes dogmatic, opinions about the work.
|
