TPC Chair: Preparation is everything, monitor processes, stay on top of things

The evaluation process of conferences can be tricky and complex, quickly overwhelming if things go wrong (and some will). My advice to TPC chairs is to prepare as much as possible in all steps, including

• Communicate guidelines, evaluation criteria, procedures etc clearly to the TPC. Many TPC members are overloaded, have done it many times, might not pay attention, so will do “how they always do it”. I found most TPC to be members constructive, but forgetful. See also this post on TPC – chair communication.
• Monitor reviews as they come in, so you are aware of situations of papers, and can guide further reviewing, possibly asking for additional opinions.
• once reviewing is complete, before TPC meeting:
  • go through the papers and reviews (yes that is a lot, but see it as investment for the TPC discussions), ensure that they are done according to guidelines and evaluation criteria.
  • weed out the reviews: check for issues such as reviewer identifying wording, inappropriate language, reviews not based on facts, arrogant opinions etc
  • check if there is material, e.g., inaccessible papers, artifacts missing for the reviews, provide anonymized versions for TPC
  • once you know the reviews, you can have your own opinions (hard to lead a discussion when you have to rely only on others’ opinions) and anticipate special situations. Which paper has very diverging opinions, where is a single review skewing the evaluation etc, so you can prepare yourself and possibly add extra help.
  • make a prediction for yourself on which papers will go in, which you think would be good for the conference, etc.. This helps you
    • counter TPC discussions first going extra strict, then not having enough papers, then trying to compensate on the spot
    • monitor progress and possible counter steer too restrictive trends
  • when setting deadlines, consider community friendly timing
• prepare for the TPC meeting
  • do a practice run with some non TPC volunteers of the TPC meeting, ensure you are on top of the tools (evaluation software, spreadsheets, etc), try out handovers between hosts, CoI chairs, etc – the meeting requires your full attention, you cannot do it well if the tools take time, get help for the logistics. Be aware of default settings of software, e.g., closed captions, which may be active next time, even if you turned them off in the trial.
  • check that your setup keeps your requirements, e.g., confidentiality – zoom e.g., does transcripts for all to read, “AI companion” turn it off before
  • provide break rooms for those with CoIs in particular for online meetings, provides at least some networking opportunities, makes the meeting more bearable
  • plan for breaks, these meetings can tense, also for you to change or check out things offline
  • avoid the phasing beginning strict – then panik not enough papers – then more lenient than beginning; evaluation criteria should depend on when the discussion happens
  • outsource as much of the logistics, e.g., moving TPC members between zoom rooms, to others, so you can focus on the paper selection, or at least breadth while someone else does the logistics
  • include your TPC co-chairs and others leading discussions in the trail as well
  • if you will use others to chair, substituting you when you have CoIs, ensure they properly understand the procedures, just as good as you do. I recommend selecting those who already have been TPC chair at the conference, that means that had community approval before, have experience, and understand the importance of accepting papers.
• at the TPC meeting
  • communicate and repeat, get the TPC supportive on your goals and guideline
    • this can be tricky, sometimes TPC members don’t read your information or even don’t pay attention when you tell them at the TPC meeting intro
    • you can try a trick to check attention inspired by Van Halen’s rider exluding brown M&Ms, e.g. naming conventions at the online meeting
    • repeat and apply during the meeting
    • keep the entire TPC involved, in particular as the meeting goes on
  • use a structure for presentation of reviews, similar to the reviewing form – first very brief summary, positives, negatives, etc – to avoid biasing and freeform discussions
  • imbue in TPC members a sense that being a good TPC member is not about rejecting papers to show high standards, but making the effort to find reasons to accept
  • do not go overboard accommodating TPC members special request, e.g, w.r.t timing of discussions; it is quite tricky to schedule without giving away information, TPC members have committed to the meeting, your constraints as TPC chair are more important than those of TPC members trying to get out of things
  • ensure conflicts-of-interests (COIs) are enforced at all times, including excluding COI members from discussions, references to other papers in the discussion etc.
  • follow proper voting procedures: as the final result has to be “accept” or “reject”, a simple majority vote between e.g., “accept” – “shepherd” – “reject” can have a wrong result, e.g., “accept”:4, “shepherd”:3, “reject”:5 does not mean reject, although it has the highest numbers of votes: 7 votes for paper in > 5 votes for out. One way to fix this is to first vote between “accept” or “reject”. If that results in “accept”, a vote between “accept” or “shepherding” can be taken. Show or report the resulting numbers to the TPC for transparency. Record the numbers outside the software for documentation.
  • guide the discussion, ensure contributions lead to a goal, i.e., decision – without it, TPC members will be happy to talk as long and as on many points as they can (hei! it was a lot of work reviewing, happy to talk about it), not only will that prolong the meeting, chances are you will loose other TPC members’ attention. It is about getting a decision, not about everyone making full statements.
  • desire to speak seems to be stronger at the beginning of the meeting, best put papers with need for discussion first
  • in the beginning, you are setting the tone and style of the meeting, perhaps choose a paper to serve your ideas, with constructive and efficient TPC members leading the discussion. Make sure the “brief summaries” to start discussions are really brief.
  • don’t forget breaks, there is only so long attention can be held in such intense settings, quality of discussions and decisions will be affected over time
• after the TPC meeting
  • ensure constructive and polite feedback is provided to the authors
  • informative meta-reviews can be very helpful to authors for future versions of submissions
  • I am not sure about the value of sending out reviewer scores to authors, they are just input to the discussions and decisions in the TPC
  • when asking TPC members to provide information to authors, e.g., in the meta-review, providing a template can be helpful to ensure quality standards and set a proper tone (as proposed by Hyoseung Kim at RTSS25) and save efforts for TPC members

A TPC, without a force applied by the TPC chair, will reject any paper.
Task of the TPC chair is to get papers accepted against the trend of the TPC.
(source unknown)