Dept. of Computer Science and Institute for Advanced Computer Studies
Univ. of Maryland, College Park
This article stems from discussions among the program committee for SIGPLAN'91 PLDI. The program committee thought it might be useful to put together some advice for authors. To give some context to these suggestions, I've also provided a brief description of the process by which the conference papers were reviewed, partially from my perspective. This process is similar to the way most SIGPLAN conferences are run, although the details differ for each conference.
How the Papers Were Reviewed:
There were 169 extended abstracts submitted to the SIGPLAN '91 PLDI conference. At the request of the program committee chair, program committee members (and their graduate students) refrained from submitting any abstracts to the conference. This allowed us to avoid having to deal with direct conflict of interests.
Each program committee member was assigned 60 abstracts, based on his or her areas of expertise. Since all abstracts were sent to all committee members, members could review any abstracts they wished, so long as they reviewed at least the abstracts assigned to them. Program committee members could review the abstracts themselves or have others review them, although in most cases the program committee members at least briefly reviewed all the abstracts they were assigned, even if they had colleagues review some of them in detail for them.
I wasn't able to read any abstracts until after the semester ended in mid-December, and I allowed myself a week off from reading abstracts for Christmas. Thus, I had about four weeks to read the abstracts, and I couldn't spend much more than 20 hours a week reading them (due to limitations both on available time and the amount of reviewing I could do in a day before I suffered burnout). Since I read more abstracts than I was assigned, this came down to an average of one hour per abstract.
In reading an abstract, I had to try to understand the work presented, the significance of it, and possible problems with it. I spent at least 30-40 minutes on almost every abstract, sometimes coming back to an abstract several times. I spent over four hours each on several abstracts. In one case this was because the abstract looked interesting but was badly written; in another case, because the abstract dealt with a dense subject. In several cases, I spent several hours on a paper simply because I had expertise or interest in the topic described by the paper.
The program committee met for two days to discuss the submitted abstracts and choose those to be accepted. A preliminary numerical ranking provided by the reviews received in advance of the meeting helped structure our discussions. On each of several passes through all the submissions, some papers were eliminated from consideration, others were retained for further discussion and some were accepted. Finally, we had a total of 28 accepted papers.
What is an Extended Abstract?
An extended abstract is not simply a long abstract. An extended abstract should contain references, comparisons to related work, proofs of key theorems and other details expected in a research paper but not in an abstract.
An extended abstract is a research paper whose ideas and significance can be understood in less than an hour. Writing an extended abstract can be more demanding than writing a research paper.
Some things that can be omitted from an extended abstract: future work, details of proofs or implementation that should seem plausible to reviewers, ramifications not relevant to the key ideas of the abstract.
Some Issues Considered by the Committee:
A related problem is not citing relevant work in the area. Don't rely on the program committee realizing that X's work in this area doesn't apply because you are considering a slightly different problem that renders X's methods unusable.
If you have additional current papers on topics related to your submission (accepted by or submitted to other conferences or journals), be sure to discuss the contribution of your submission over that of your other papers.
If you submit an extended abstract involving a specialized application, be sure the significant contributions of your work don't get lost in the details of your application.
Several program committee members stated that after reading 10 pages worth of material, they felt free to stop reading at any point if they were not truly excited by the paper.
Don't let the page count limit prevent you from providing figures or examples that make the paper easier to understand. The page count limit should be considered an upper bound on the number of full pages of text, exclusive of figures and examples. One program committee member disagreed and thought that the page limit should be strictly adhered to, noting that if a picture is worth 1000 words, a picture is worth more than the 200 words it displaces.
In exceptional cases, it may be appropriate to put additional material in an appendix that extends past the length limit. This is acceptable only if the extended abstract itself stands on its own without the additional material. Given their time limitations, most reviewers probably will ignore the appendix. Acceptable material for an appendix could include background material for committee members not familiar with the details of the research area and details of proofs and implementations omitted from the body of the abstract.
The program committee was sympathetic about not expecting data that ought to have been very difficult to collect. However, the committee was disappointed in several instances by abstracts that failed to report data that ought to have been easy to collect and would have answered obvious questions about the work.
Final Comments for Authors:
Before you submit an abstract, give it to a programming languages colleague who is not familiar with the details of your research or your research area and ask him or her how much they can get out of it in less than an hour.