Leaderboard formula

Submissions will be evaluated according to classification accuracy in each of the 3 typing conditions. Classification accuracy (ACC1) is the proportion of correctly labeled samples. For each submission, the following formula is used:

  1. For each participant, the highest ACC1 in each condition (both, left, right) is taken over all submissions.
  2. A rank is determined for each condition. Each participant receives a rank for each condition.
  3. The leaderboard is determined by the sum of ranks. Thus, the best score is (1 + 1 + 1) = 3 for having the highest ACC1 in each condition. This would achieve first position in the leaderboard. Ties are broken by the median ACC1

Final Leaderboard

This leaderboard is calculated using 100% of the test data:

Rank Name Both Left Right
1 Patrick Bours 0.8276 0.3053 0.4015
2 Sudalai Rajkumar S 0.8276 0.2748 0.3212
3 ATVS Universidad Autónoma de Madrid 0.6946 0.1679 0.2044
4 Andrea Parker 0.0542 0.0305 0.0292
5 Benchmark Score 0.0246 0.0458 0.0292