Hints for referees

D. Knuth

In a relatively new field such as computing there is bound to be a lot of trash published since there are too few people available to recognize the poor quality of much of the material. But this discourages people in the computing profession from reading the literature and causes a poor image for the profession in the eyes of others.

The only way to prevent this is to have a strong refereeing system. Although the job of refereeing is not simple, it is an important responsibility, nearly the most important thing anyone could be doing for the field of computer science. Papers generally will fall into the following categories:

(1) Publish essentially as is; the only changes necessary are very simple typographical matters which can be changed by the editor.
(2) Publish after author's minor revisions. The referee suggests points which must be changed before the paper meets the standards for publication.
(3) Publish only if the author makes major revisions. (Perhaps the paper is much too long or is badly written. The revised paper will be refereed again.)
(4) Reject. (There is nothing salvageable.)

The goals of a referee are to keep the quality of publication as high as possible and also to help the author to produce better papers in the future. Your referee's report should be designed to give the author the maximum benefit, yet not compromise on quality. Try to get every author to put out the best paper he is capable of writing; a paper rarely falls in category 1 above.

Never put a paper in category 1, if you feel the author can do better, even if the paper as it stands is reasonably good! A paper should only be put into category 3, if the substance of the paper is considered significant enough to warrant the additional amount of labor to rewrite and reconsider the paper. To judge the publishability of the paper you certainly know what is good and what is bad but the following brief list is included here anyway.

The paper should contribute to the state of the art and/or should be a good expository paper. If it is purely expository it should be clearly designated as such.
All technical material must be accurate (e.g. no incorrect equations, etc.). A referee should check this carefully.
The article must be understandable, readable, and written in good English style.
The bibliography should be adequate.

It is tempting to postpone refereeing tasks by putting the paper aside for a few days. But it takes no longer to do it today than it will in a week's time. If you feel that you are for some reason unable to referee the paper please return it immediately. Otherwise, the referee's report is expected in no more than four weeks. Remember that the refereeing cycle is "critical path time" in the publication process.

Return the manuscript to the editor; please don't mark it up. You should submit the report in duplicate. Remember that one copy will be sent directly to the author; it is up to you whether you want to mention your name on it or not. If you desire, you may write an accompanying letter to the Editor which of course will not be passed on to the author. This letter, however, must not constitute the referee's report.