Tag: Benchmarking
-
When the Ruler Moves With the System
A changing score does not necessarily mean the system changed. The evaluator may have changed too. Datasets evolve, parsers are corrected, thresholds move, admissibility rules shift, and reporting conventions are revised. Once both the artifact and the evaluator are moving, a numerical difference can be completely real and still fail to tell us what actually…