Sunday, 5 August 2007

Quick Ratings Revisited

Earlier this year I made a post about the Quick and Dirty Ratings System, and promised to report back on any findings.
Well the system I described has been in operation for last two terms at my local junior chess club, which is probably sufficient time to make some sort of judgement.

The first observation is that it is popular with the kids. No surprises there, as chess wouldn't be Chess without a ranking system. Of course it is more popular with the kids at the top of the list, although I try and mitigate the effects of overt boasting by making the list 'semi-private' ie you can check your own rating on request.
Secondly, the existent of ratings does have an effect on who you play. Last term I grouped players into round-robins based on rating but found that that wasn't the best way to do things. (eg the bottom player in any group often scored 0/5 and went home feeling bad). So this term I allowed players to choose their opponents. What has happened is that players at the top will usually play each other anyway, as they have worked out that playing lower rated opponents doesn't help their own ratings (even if they win). I'm not convinced this is a good thing (especially for the lower rated players) but it has had the effect of players being able to change their potential field, especially if they have got of to a bad (or exceptional) start.

In terms of numbers there are some other observations. As every new player starts with a rating of 500, and the ratings calculation are zero sum, then the mean of the pool is always 500 (no deflation!). Over the last 20 weeks 37 players have played at the club. Although the mean is 500, the median rating (ie the rating of the player in the middle) is 473. In terms of distribution there are 15 players rated above 500, with 22 players rated below. In fact 15 players are rated between 400 and 500.
The majority of these players were "short termers" ie players who came for a couple of weeks before dropping out. My belief is that like most activities, interest is driven by enjoyment/success and they discovered that losing wasn't that much fun. If they had stuck at it maybe they would have turned that around, but we are talking about primary school kids, not noted for their long term planning.
Overall the system has worked well, with my initial observations about the volatility of the system now being tempered by the use of a different tournament structure. It certainly has ranked the players at the club fairly well in terms of results/abilities and has done a good job of rewarding the players for actually playing. (eg kids who have won a few games are keen to play some more, to "get more rating points")
So after 20 weeks of use, I would happily recommend it to anyone looking for a simple internal club rating system, whether it is for juniors or adults.

1 comment:

Anonymous said...

What are your thoughts of having a policy where a weaker player can challenge a stronger player, but not vice versa?