Close

Quantitative Solutions don’t solve Qualitative Problems

A reflex to problems often seem to be “do more of x”.

Let’s look at an example: “Not happy with the quality of the review process? Do more reviews per paper.” Lack of quality in paper selection procedures can often be attributed to bad reviews by uncommitted reviewers or TPC members. Such reviewers should not serve on TPCs, a quality problem of TPC composition. Such reviews should not be left unchecked, also a quality assurance problem, e.g., by the TPC chair, who maybe should have been instructed better. All about quality.
Trying a quantitative solution, i.e., larger number of reviews per paper, is unlikely to fix this with such persons and procedures in place. In fact, it might even decrease quality, as work load on reviewers increases, and sense of personal responsibility, “so many other reviews, mine won’t matter” decreases.

A side effect can be quantitative race: if having n of X in conference A, having n+1 of X makes conference B appear better, etc.

Rather, revise quality assurance procedures in selection and guidance of reviewers and reviews.

Leave a Reply

© 2025 1 is the Only Acceptable Acceptance Ratio | WordPress Theme: Annina Free by CrestaProject.