Rating › Prediction accuracy
How accurate are the rating predictions?
This checks how well the win probabilities derived from this site's ratings match actual results, using past games.
Each prediction is calculated only from results before that game; it is not fitted after the fact. Only games between professional players are counted (games involving amateurs or Shoreikai members are excluded).
Accuracy by period
| Period | Games | Higher-rated player won | Average prediction (higher side) | Prediction error |
|---|---|---|---|---|
| FY1964–2006 | 78,973 | 64.4% | 64.8% | 0.6277 |
| FY2007–2026 | 49,049 | 65.5% | 65.5% | 0.6195 |
| Last 3 fiscal years (FY2024–2026) | 6,946 | 65.4% | 65.7% | 0.6194 |
- "Higher-rated player won" is how often the player with the higher pre-game rating actually won. At 50%, the rating would be of no use at all.
- "Average prediction" is the mean predicted win probability of the higher side. The closer it is to the actual win rate, the more honest the probabilities.
- "Prediction error" measures how far off the predictions are (log loss); smaller is better. Predicting every game as 50-50 gives 0.6931.
Actual win rate by predicted probability
For games predicted at "70%", how often does the result really land near 70%? Shown by probability band (all games from FY1964 through FY2026). The blue bar is the actual win rate and the red line is the average prediction; the closer the red line is to the end of the bar, the better the prediction.
| Prediction (higher side's win probability) | Games | Average prediction | Actual win rate | |
|---|---|---|---|---|
| 50〜55% | 24,955 | 52.5% | 52.8% | |
| 55〜60% | 23,613 | 57.5% | 57.4% | |
| 60〜65% | 21,306 | 62.4% | 62.3% | |
| 65〜70% | 18,696 | 67.4% | 66.3% | |
| 70〜75% | 15,144 | 72.4% | 72.0% | |
| 75〜80% | 11,278 | 77.4% | 76.8% | |
| 80〜85% | 7,582 | 82.3% | 81.4% | |
| 85〜90% | 4,050 | 87.2% | 87.8% | |
| 90〜95% | 1,277 | 91.8% | 92.7% | |
| 95〜100% | 121 | 95.9% | 98.3% |
How to read it, and limits
- Games between evenly matched players (predicted 50–55%) are close to a coin flip. Even strong players lose, so there is a limit to how well ratings can call a result.
- First/second move, recent form, matchups, time controls and the importance of the game are not included.
- The rating design (the K factor and so on) was chosen by looking at how well predictions did from FY2007 onward. The figures from FY2007 on may therefore look slightly better than FY1964–2006, which was not used for the design.
- FY1963 and earlier are not included here, because only Junisen games exist for that period and their dates are estimated.
Method ／ Upcoming games (with win probability) ／ Tournament simulation