Supporting Finding and Re-Finding - People | MIT CSAIL

Although numbers can’t be directly compared, due to variation in the number of possible ratings and the size of the result set evaluated, inter-rater agreement appears to be substantially higher for TREC (e.g., greater than 94% (Koenmann & Belkin, 1996)) and previous studies of the Web (e.g., 85% (Eastman & Jansen, 2003)). ................
................