TPC Chair: Preparation is everything, monitor processes, stay on top of things

The evaluation process of conferences can be tricky and complex, quickly overwhelming if things go wrong (and some will). My advice to TPC chairs is to prepare as much as possible in all steps, including

• Communicate guidelines, evaluation criteria, procedures etc clearly to the TPC. Many TPC members are overloaded, have done it many times, might not pay attention, so will do “how they always do it”. I found most TPC to be members constructive, but forgetful. See also this post on TPC – chair communication.
• Monitor reviews as they come in, so you are aware of situations of papers, and can guide further reviewing, possibly asking for additional opinions.
• once reviewing is complete, before TPC meeting:
  • go through the papers and reviews (yes that is a lot, but see it as investment for the TPC discussions), ensure that they are done according to guidelines and evaluation criteria.
  • weed out the reviews: check for issues such as reviewer identifying wording, inappropriate language, reviews not based on facts, arrogant opinions etc
  • check if there is material, e.g., inaccessible papers, artifacts missing for the reviews, provide anonymized versions for TPC
  • once you know the reviews, you can have your own opinions (hard to lead a discussion when you have to rely only on others’ opinions) and anticipate special situations. Which paper has very diverging opinions, where is a single review skewing the evaluation etc, so you can prepare yourself and possibly add extra help.
  • make a prediction for yourself on which papers will go in, which you think would be good for the conference, etc.. This helps you
    • counter TPC discussions first going extra strict, then not having enough papers, then trying to compensate on the spot
    • monitor progress and possible counter steer too restrictive trends
• prepare for the TPC meeting
  • do a practice run with some non TPC volunteers of the TPC meeting, ensure you are on top of the tools (evaluation software, spreadsheets, etc), try out handovers between hosts, CoI chairs, etc – the meeting requires your full attention, you cannot do it well if the tools take time, get help for the logistics
  • check that your setup keeps your requirements, e.g., confidentiality – zoom e.g., does transcripts for all to read, “AI companion” turn it off before
  • provide break rooms for those with CoIs in particular for online meetings, provides at least some networking opportunities, makes the meeting more bearable
  • plan for breaks, these meetings can tense, also for you to change or check out things offline
  • avoid the phasing beginning strict – then panik not enough papers – then more lenient than beginning; evaluation criteria should depend on when the discussion happens
  • outsource as much of the logistics, e.g., moving TPC members between zoom rooms, to others, so you can focus on the paper selection, or at least breadth while someone else does the logistics
• at the TPC meeting
  • communicate and repeat, get the TPC supporitve on your goals and guideline
    • this can be tricky, sometimes TPC members don’t read your information or even don’t pay attention when you tell them at the TPC meeting intro
    • you can try a trick to check attention inspired by Van Halen’s rider exluding brown M&Ms, e.g. naming conventions at the online meeting
    • repeat and apply during the meeting
    • keep the entire TPC involved, in particular as the meeting goes on
  • use a structure for presentation of reviews, similar to the reviewing form – first very brief summary, positives, negatives, etc – to avoid biasing and freeform discussions
  • imbue in TPC members a sense that being a good TPC member is not about rejecting papers to show high standards, but making the effort to find reasons to accept
  • do not go overboard accommodating TPC members special request, e.g, w.r.t timing of discussions; it is quite tricky to schedule without giving away information, TPC members have committed to the meeting, your constraints as TPC chair are more important than those of TPC members trying to get out of things
  • ensure conflicts-of-interests (COIs) are enforced at all times, including excluding COI members from discussions, references to other papers in the discussion etc.
  • Follow proper voting procedures: as the final result has to be “accept” or “reject”, voting should go first between “accept” or “reject”. If that results in “accept”, a vote between “accept” or “shepherding” can be taken. Having the vote in the beginning between all three unfairly tilts the result to “reject”.

A TPC, without a force applied by the TPC chair, will reject any paper.
Task of the TPC chair is to get papers accepted against the trend of the TPC.
(sources unknown)