Sensitivity of the International Skating Union’s Mathematical Criteria to Flag Potential Scoring Anomalies

Abstract

This article describes the “mathematical criteria” employed by the International Skating Union (ISU) to identify potential judging anomalies within competitive figure skating. The mathematical criteria have greater sensitivity to identify scoring anomalies for technical element scores than for the program component scores. This article provides illustrations of how hypothetical subtle judging biases could not be flagged for two female skaters at the 2014 Winter Olympics. As a result, we offer four suggestions to improve the sensitivity of the ISU’s mathematical criteria: (1) alter the way the Total Deviation Points are computed for the program component scores (i.e., use “absolute” error); (2) lessen the amount of error allowed on average for each program component; (3) reconsider the cut-off value for Total Deviation points for the technical elements; and (4) reduce the cut-off value for the range of program component scores. Moreover, we suggest that “absolute” error is a more appropriate descriptor for which both technical element and program component scores can be flagged for potential judging anomalies.

Keywords:

• judging anomalies
• absolute error
• judge bias
• constant error

Notes

¹ The description of how the Average GOE is computed is not stately clearly in ISU Communication No. 1631, p. 5. All of the judges’ scores and the Referee’s score are used to get a weighted average where the intent is to heavily weight the referee’s score by giving it a factor of 2. In Equation 1, the referee’s score contributes 67% to the Average GOE. An International Skating Referee agrees with our interpretation of the ISU’s description of the how the Average GOE is computed. An alternative equation would not heavily weight the referee’s score as intended:

The referee’s score would only contribute 18% to the Average GOE which serves as the standard by which potential judging errors are flagged. Equation 1 is the most reasonable choice for determining the “fair/true score” for a skater in an event.

² The description of how Average Program Component is determined is not stated clearly on p. 6 of ISU Communication No. 1631. Our interpretation as illustrated in Equation 2 is consistent with feedback we received from one ISU referee. This interpretation allows the referee’s score to contribute 44.4% and the OAC member’s average to contribute 33.3% to the Average Program Component. An alternative equation would not heavily weight the referee’s and OAC members’ scores as intended:

Their scores would only contribute 16% and 12 % respectively to the Average Program Component, which serves as the standard by which potential judging errors are flagged. Equation 2 is the most reasonable choice for determining the “fair/true score” for a skater in an event.