The goal of the technical program committee is the selection of papers based on their technical merit and suitability for the conference.
The TPC enacts goals and criteria of the conference.
The TPC guarantees the technical correctness of the results published at the conference.
- This is a service to the technical community and society.
- The experts of the TPC check, so readers and users don’t have to.
The TPC takes decisions jointly as a body.
- Every TPC member’s voice counts equally, independent of area, seniority etc.
- The PC can, and in some cases should, take decisions differently from the reviewers’ opinions.
- Discussions outside the entire TPC, e.g., before meeting (“online discussion”), risk (de)emphasizing opinions based on outspokenness, seniority, etc.
- “the one quiet person with the divergent opinion could be the correct one”
Define and communicate goals and criteria clearly to authors and PC members, reviewers, e.g., theory vs practice, enforce during reviewing and discussions.
- The Call for Papers should be sole basis for evaluation.
- Then, authors and TPC members have same understanding.
- Those not “in the game” are not disadvantaged by lack of inside knowledge.
- Changes of criteria during the evaluation process, which benefit those “inside” can be identified and stopped.
Decisions have to be based on technical correctness and novelty, and possibly categories defined for the profile of the conference.
- Careful with criteria and opinions including “don’t see that catching on”, “does not excite me” etc., we just don’t know at this point, only time will tell
- Looking in retrospect which papers had impact can be enlightening, also comparing with Best Papers, i.e., what we thought would have impact and then what really had
Confidentiality regarding all aspects of paper selection, as well respecting conflicts-of-interest is paramount.
Minor practical points:
- Making reviews available to other reviewers before the deadline creates temptations to not to do independent reviewing, but base it on the other existing ones, biasing the reviews, e.g., by not doing a thorough job, or preemptively change review tones to fit in – see also comments above.
- Similar arguments for making reviewer names available before the TPC meeting.
- Individual reviews and their scores represent individual reviewers’ opinions and are only input to the TPC discussions, in fact can be overturned, they do not necessarily reflect the final result. I suggest removing them from the information given to authors.
- Summaries of the TPC discussions leading to the decision, e.g., called meta reviews, can provide valuable information to authors.