Using item response theory to rate (not only) programmers

From IOI Wiki
Jump to: navigation, search
Pub article.png
  • Article: Using item response theory to rate (not only) programmers
  • Author(s): Michal Forišek
  • Journal: Olympiads in Informatics 3 (2009), 3-16

Abstract: We show how Item Response Theory (IRT) can be used to define a new type of rating system, one that is especially suitable for programming competitions (and other types of competitions where difficulty of competitions varies between rounds). We show some useful theoretical properties of this rating system, including the ability to argue about hardness of past competition tasks, and about the precision of contestants' skill estimates. We also define an objective method of comparing different rating systems. In the final section of the paper we apply our methods on real competition data.

Keywords: item response theory, ranking, rating, programming competitions

Download: (free)