Some Advice for Program Committee Chairs Based on my ISCA 2005 Experience

Mark D. Hill

Computer Sciences Department
University of Wisconsin-Madison

Filed at

4 April 2005, revised 20 May 2005

Many computer science researchers accept the honor and burden of being program committee (PC) chair of a major conference only once. Thus, many PC chairs are performing the task for the first time. While the task is informed by being a PC member, its duties and responsibilities are considerably different. Once PC chairs have gained their experience, it is often lost.

To pass on some of what I learned serving as PC chair of the 2005 International Symposium on Computer Architecture (ISCA), I have written two short documents:

Some Things That Worked (Reasonably Well)

Here on some things that worked well, in order of importance.

Use established software package and assign a web master. I used Grunwald's Conference Review Package (CRP) . It served well. I had one of my Ph.D. students deal with the software, including implementing minor changes. Since I knew this was a large task, I gave him credit as Web Master, an official ISCA 2005 officer. In the end, he reports that he worked 120 hours on both CRP and the conference web page.

Seek reasons to accept papers. At the PC meeting, I had the PC member who gave the highest recommendation begin the discussion by summarizing the paper and then critiquing it. Next, other PC members who read the paper spoke. Finally, we had a discussion. This set a positive tone, and, in the long run, helped focus PC members on looking to what is good (since some PC members summarized many more papers than others). I also kept mentioning 42 papers as a target, reminding members to look for papers to accept. In the end, we took 45 papers.

Keep PC meeting discussion focused on making decisions. I was ruthless about focusing on decisions. I cut people off when they rambled. I cut off discussion when it appeared consensus was reached. See "egg timer" below. We covered the top 50 papers explicitly and then moved to a model where PC members could "save" papers (to cover 30 more).

Following SIGARCH guidelines. I strictly followed Guidelines for SIGARCH Sponsored Conferences. I found it helpful to build on the wisdom of others. In particular, I fould it valuable to follow the guideline on restricting each PC member to at most two submissions. This reduced the number of PC papers that we would likely accept (reducing appearance and reality of inbreeding) and saved time at the PC meeting (since PC paper discussions take longer than others).

Being very clear on conflict of interest rules. Doing so makes the process more fair and, equally important, appear more fair. In the PC meeting, for example, I had two other researchers run the discussion for papers for which I had a conflict of interest. This was critical, since on the rest of the papers, I would often cut off discussion. If I did that for a conflicting paper, it might look like I was influencing the decision (because I would be). I found that, with reminders, authors were good at listing conflicts, but PC members were not.

Limited internet access. I had only two Internet connections. We used these to print a few key related papers. Keeping PC members from their email was critical for keeping their attention. We brought two laptops and one printer.

Getting all external reviews (2 per paper). I personally solicted non-PC reviews for most papers and, for papers where I had a conflict of interest, designated two PC members to act as PC chairs to solict reviews (indirectly, with me sending the actual email). With some nagging I got all external reviews. The reviews were less correlated with PC reviews (but were correlated with me). I believe they carried more weight at the PC meeting, since they were mostly from known people rather than junior students. This process cost me two-person weeks (but I am good at nagging).

Rebuttals. I fould rebuttals helpful. They serve as a hard pre-PC-meeting deadline for both external reviewers and PC members. They make authors feel better. They ocasionally make a difference in the PC's decision. I recommend keeping them short (e.g., 200 words) to reduce both author and PC burden.

Other recommendations.

Things That Did NOT Work (Well)

I am biased, but, in my view, there were only minor things I would change if I could start over.


Thanks for feedback from David Albonesi, Remzi Arpaci-Dusseau, Margaret Martonosi, Kathryn McKinley and Caitlin Scopel.


Mark D. Hill, Program Chair's Message for 2005 International Symposium on Computer Architecture (ISCA),

Mark D. Hill, Charge to the Referees of the 2005 International Symposium on Computer Architecture (ISCA),