Thursday, 21 February 2008

My rating is inaccurate!

Oooh, ratings fight! A post at the Closet Grandmasters blog has provoked the usual back and forth about the accuracy/problems with the rating system used in Australia (Glicko2). While I'm firmly on the side of statistical accuracy (as opposed to the "this looks wrong but I can't tell you why" side of the argument) a number of years ago I did come up with an alternative system which should make everyone happy.

New Ratings System Proposal

Premise: A rating system should be easy to understand, transparent, and reflect the perceived strength of the participants.

Rule 1: When I beat someone rated above me my rating should go up
Rule 2: When I beat someone rated way above me my rating should go up heaps
Rule 3: When I lose to someone rated below me my rating should only drop a little as they were probably lucky or I was off form, and anyway they shouldn't be beating me, otherwise they would be rated above me.
Rule 4: When I lose to someone rated way below me they are clearly an underrated player and the system is at fault so I shouldn't lose any ratings points, and indeed should get a couple of extra for the embarrassment and inconvenience.
Rule 5:If I have a good tournament where I perform above my rating, then this tournament reflects my true strength and my rating should be adjusted accordingly.
Rule 6:If I have a couple of bad tournaments these results should be discounted as they don't reflect my true strength (see rule 5) and to include them would make a mockery of the ratings system.
Rule 7: It is an historical fact that players in my state of "insert name here" are underrated compared to all the other states and an adjustment factor is required for each list

Some proposed values
Rule 1: +10 points
Rule 2: +50 points
Rule 3: -1 point
Rule 4: +5 points
Rule 5: Players performance rating
Rule 6: 0 points
Rule 7:+100 points

A simple system, easy to understand.
Comments invited.


DeNovoMeme said...

I like it. All you have to do is correct for inflation.
By-law to your rules.
1. The population mean rating change (upwards!) is calculated.
2. Players who did not play in that period are allocated that figure to their rating.
3. The entire rating list is then transformed by deflating it by the rate of inflation.

1. Everyone is happy about the points gained or lost to their betters/inferiors.
2. Relativity is maintained.

Southern Chess Player said...

It seems that the main problem with the Gliko rating system is that average Joe chess player doesn't understand it.

Basically the debate on Closet Grandmasters blog is over the rating of a returning player not being very accurate after one bad result and hence she is under-rated. The people involved seemed to have something personal against the rating system but their main complaint was implying that opponents who subsequently played against her would suffer the consequences of her under-rating causing them to lose out on the rating points scale.

Anyone who knows how the Gliko system works will realise that this is not the case. Her rating is currently very unstable because of the limited number of rated games. What this means is that in games between her and other players the opponents rating will be insignificantly changed by the result of their games, at least compared to a game against a player who has a rating based on a much larger number of games.

Also note that her rating will fluctuate quite a lot as a result of future events and if what they are saying is correct (she is under-rated and had a poor tournament) then this will be in a positive direction.

Maybe the difficulty of understanding the Gliko system is it's main weakness, but it has a very solid statistical basis so you can hardly criticise it on mathematical grounds without challenging that. None of the negative (of the Gliko system) commentary on Closet Grandmaster has any foundation.

Anonymous said...

vhxbwgjzmbvwepnygwjt, yahoo Scanner, VrnJTOo.