I do appreciate the intention of rebuttals, improving the evaluation process, but am not convinced about the benefits vs. the efforts involved.
Let me contrast two publications forms. The comments are specific to computer science, other areas, e.g., physics see the relations quite differently.
- Conference –
- intent: bring quality results to the public quickly
- short turn around time from submission to publication
- quality control
- due to time limits error prone
- if not good enough reject – submit new version to other conference
- only very limited modifications possible after evaluation
- only one submission round
- Journal – intent: publish established quality results
- long turn around time possible
- quality control
- advanced possible, sufficient time to include additional reviewers, expertise, etc
- if not good enough several reviewing rounds possible
- major modifications possible and possibly required
In my view, rebuttals, also shepherding, etc. can dilute the specific purpose and profile of conference and create high efforts for the involved. In a way, they try to cram the quality procedures of journals, eg, multiple rounds, into the (short) time line of a conference. Questions and answers to rebuttals require significant effort of authors and evaluators in a very short time interval, increasing the pressure, which affects disproportionately junior researchers and TPC members.
I believe rebuttals can work for very specific clarifications. They require very disciplined evaluators, and monitoring by the TPC chair, to be efficient and fair. As example: I have seen rebuttal questions ala “please provide more detailed comparison to other work and description of the proof” – in very few, e.g., 500 words. This is simply not possible, stresses authors, and will frustrate evaluators.
RTNS24 has a multi-submission model which provides for major revision and resubmission, ala major revision in journals, but at a much extended time scale, which I think is an interesting idea.