Wednesday, August 3, 2011

Rating Inflation?

WARNING: Headache-inducing Math involved

For those of you who are ignoring the warning, read here.

Basically, it is a paper where two guys analyzed a large database of chess games move by move to determine chess players' "intrinsic rating" and see if the "strength of their moves" have improved compared to rating levels in the past. In simple terms, they are testing to see if a 2700 player today is playing the same quality moves as a 2700 player 20 years ago. If the move qualities are the same, it shows that there is no rating inflation.

So what did they find?
A smooth correspondence is shown between statistical results and the century points on the Elo scale, and ratings are shown to have stayed quite constant over time. That is, there has been little or no ‘rating inflation’.

........

In the 1970’s there were only two players with ratings over 2700, namely Bobby Fischer and Anatoly Karpov, and there were years as late as 1981 when no one had a rating over 2700. In the past decade there have usually been thirty or more players with such ratings. Thus lack of inflation implies that those players are better than all but Fischer and Karpov were.
Note that they are using the strength of the moves of the chess players and not just results, which is probably a much more accurate measure of playing strength. I sincerely believe the study can be extended to using rating to predict playing strength. But as in all statistical studies, higher ratings will only increase the chances of winning. Playing strength is not absolute. As they say, "you win some, you lose some".

6 comments:

  1. A Newton in the 21st century may be equal to 2 Newtons of the past. The difference is the old Newton had to do pioneering work (so to speak, where no man had been before) whereas a Newton today can rely on many others' work as well as super computers. This is inflation as basically there was only one man, Newton.

    ReplyDelete
  2. Just for argument's sake, you are assuming that building on Newton's work is easier than Newton's initial work. We have no reason to believe such is true. Of course, the Newton today does not have to "rediscover the wheel", but I think that although the initial work has open up a lot of possibilities, work has not necessarily become easier. Assume that Newton "discovered" the early forms of calculus. Now, someone comes along and uses calculus to discover something else. Does that mean that someone is "not as smart" as Newton?

    Translating this to chess, just because someone has discovered a particular variation does not necessarily make that person stronger than another chess player who uses it in the future. It is not directly clear that there is inflation.

    With regards to the supercomputer, more novelties may have been discovered, but playing good moves on the board has not become easier. Maybe computers have improved the opening stage of many players, but over the course of the game, there are still a lot of decisions to be made without the assistance of a computer. Though, I think it may skew the data a little bit. It was unclear from which move did they start their evaluation.

    Another concern is that, what constitutes a "strong" move? Even computers evaluate positions differently. Fundamentally, it is just algorithms input by programmers to assign a score to some qualitative features (assuming no material difference). We don't even know how to assess the score difference between positions rated +0.50 vs +0.40 and the difference between positions rated +0.30 vs +0.20. Although their differences are both 0.10, we don't even know if the scale is linear.

    Technicalities aside, I have always liked Newton's quote:

    "If I have been able to see further than others, it is because I have stood on the shoulders of giants."

    :)

    ReplyDelete
  3. Chess is a finite game. We can calculate the whole possibilities of the game, no matter how complex. BUT, there is also the element of time. You are given a fixed time limit to complete a game. If Newton was a chess player back then, he would have achieved a certain playing strength. Now, bring him to the present (if that were possible), feed him with all the accumulated chess theory and give him a chess engine, he would easily beat the Newton of centuries past in a finite time game ... but he is beating himself. Many players nowadays cannot do anything without his Chessbase and pc. Take those away and you could effectively reduce his playing strength by say 100 points, if he could still play. The point is can a modern player do without "chess support"? I am sure no top player can do that. Therefore, any claim to being stronger than a past player is partially due to available resources, and I think that contribute to "inflation" of chess rating.

    ReplyDelete
  4. Granted, it is finite. But presently, no computer is able to solve that yet. It is estimated to have about 10^16 possible combinations.

    Well, it is a hypothesis that cannot be proven can it? At least not yet.

    ReplyDelete
  5. Going off tangent, chess can never be solved on earth; at least, it's solution won't be storable.

    5-man Nalimov tablebases take up some 7GB. 6-man takes up over 1TB. Assuming exponential growth(but there could very well be factorial growth) in tablebase size, a 32-man tablebase which solves chess would be something like 140^26=6.3*10^55TB.

    To put that in perspective, all the storage space in the world amounts to less than 1ZB, or 1 billion TB. So we would need 6.3*10^34 times the available hard disk space that the whole world has today to store the 32-man tablebases alone (actually we also need to consider the space occupied by the 31,30...3-man tablebases but let's not bother). In words, 63 million billion billion billion times the available hard disk storage space throughout the world today.

    It's been proven that the maximum number of possible positions is less than 10^47. There are 10^50 atoms on earth. So if it takes more than 1000 atoms to store a single position, the solution won't even fit on our planet even if we turned it into a giant hard disk.

    There are many things mankind has managed to do that were thought to be impossible. But seriously, chess is not one of them. Not until we manage to turn planets into hard drives.

    ReplyDelete
  6. Why would anyone want to store all the possible chess positions? You don't need to memorise all historical facts to be a historian, nor all mathematical data to be an excellent mathematician. If you need to, how can S. Hawkings solve all the problems in his head? Similarly, Anand surely doesn't need to know all possible continuations to be world champion.

    The point is plenty of today's young GM's are "slaves" to Chessbase. That is what makes them so good because of the "stored" experience which saves them years of learning and hard work.

    ReplyDelete