{ "fetched_at": "2026-10-19T14:34:16Z", "source_url": "https://www.kaggle.com/competitions/ChessRatings2/data", "pages": [ { "name": "Background", "content": "The Elo rating system was developed by the Hungarian physicist Arpad Elo in the 1950's and adopted by the world chess federation (FIDE) in 1970.  For more than four decades the FIDE Elo system has served as the primary yardstick in the world for measuring the strength of chess players.  FIDE ratings are used for determining invitations to chess tournaments including the world championship cycle, calculating specific pairings in most chess tournaments, and granting titles such as International Master (IM) or Grandmaster (GM).  In fact the Elo system is so popular that it has been adapted to many other applications beyond chess, including team sports rankings, other board games, and online video game systems.

However, despite the popularity of the Elo system, it has never really been demonstrated that the Elo approach is technically superior to other approaches.  Much of the appeal of the Elo system comes from its simplicity and familiarity, and it was ideally suited to a time when the computation of ratings was a significant practical challenge even for an annual list of a few hundred players.  Elo's formula was derived theoretically, in an era without large amounts of historical data or anything approaching today's computing power.  With the benefit of powerful computers and large game databases, we can easily investigate approaches that might do better than Elo at predicting chess results.  Such an investigation could have major implications on the theory and practice of ratings methodology, both for chess and also for the world beyond chess.


As an initial step in this process, Kaggle held an \"Elo versus the Rest of the World\" contest in the fall of 2010, requiring participants to develop predictive models that could forecast the results of chess games with great accuracy.  The contest was immensely popular among both chess enthusiasts and data scientists, drawing more than 3,500 submissions from 258 participating teams across 41 countries.  Although the prize fund was minimal, there was tremendous competitive drive spurring many participants to sustained effort.  For instance, despite a restriction keeping teams from making more than two submissions per day, there were twenty teams that made more than 50 submissions across the entire duration of the contest.  The underlying data is so simple and straightforward that it is very easy to begin playing with prediction models, but the overall problem of maximizing accuracy is so challenging that even the massive efforts of the first contest were not sufficient to identify a clearly superior approach.  There was a wide variety in the methodologies of just the top ten prizewinners, all of whom documented their approaches in significant detail after the completion of the contest.  The benchmark submission of the Elo formula finished far, far, behind, in 141st place out of 258, and there were 39 teams whose predictions were at least 5% more accurate than the Elo system.  It is clear that the Elo system is not the most accurate one, but it remains unclear which system is superior.

It is no longer a question of \"Elo versus the Rest of the World\"; we must now hold a second contest and focus on finding a suitable replacement for Elo.  Possibly the best approach will be a modification of the Elo approach, or perhaps it will be something completely novel.  This second contest will have several significant improvements relative to the first contest.  It will have a more robust scoring function, one selected after extensive analysis of the results from the first contest as well as consultation with worldwide leaders in chess rating theory and categorical data analysis.  Further, FIDE has provided a complete dataset of multiple years of game-by-game results that were used for calculating the official FIDE ratings.  Until now it has never been possible to perform this level of analysis, because the dataset had not been assembled, and this contest website is the only place in the world where the data is available  The second contest provides more than 30 times as many games as the first contest did, and a much larger population of chess players as well, reflecting the whole distribution of player strength rather than just the fraction of top players covered by the first contest.
", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Custom", "content": "All three training datasets (the primary training dataset, the secondary training dataset, and the tertiary training dataset) contain individual game outcomes that can help you train your prediction model and estimate the playing strength of all players.  However, only the primary training dataset tells you who had the white pieces and who had the black pieces in each game.  In the other datasets, the players are simply identified as \"Player1\" and \"Player2\" since it is not known who had the white pieces or the black pieces.  The secondary training dataset does contain known game results (only the piece color is unknown), whereas the games provided in the tertiary training dataset are not actual game outcomes.  Instead, they are estimated matchups and outcomes that satisfy known summary statistics for each player in a tournament.  You will have to decide whether the secondary and tertiary training datasets are useful to you.

In order to understand these different training sets, it is important to first understand a bit of the history behind the collection of chess game results (from tournament organizers) by FIDE.  For many years, the only information submitted to FIDE for each tournament was summary data: for each player in the tournament, you would know their pre-tournament rating, how many games they played against rated opponents, what their total score was in those games, and the average rating of those opponents.  This was sufficient to calculate ratings according to the rules of the FIDE Elo system back then, but it did not make it possible to identify the outcomes of individual games.

Then for a couple of years, tournament submission to FIDE did require an opponent-by-opponent breakdown of results, but it did not require identifying who had White and Black in each game.  Finally, a few years ago, the requirement for complete game-by-game specification (including piece color) was implemented.  Therefore we have three descending levels of certainty about different sets of games in the FIDE data:

(1) Complete knowledge of the game outcome, including who had White/Black (months 102-135)

(2) Knowledge of the game outcome but not knowing who had White/Black (months 85-107)

(3) Summary knowledge of tournament totals but not knowing individual game results (months 1-95)

If we had just used the game-by-game data (with color) submitted to FIDE in the past few years, the training dataset would be less than three years long.  Rating systems need time to settle down, and also to differentiate themselves from each other during validation and scoring.  In order to support a long training period, the contest organizer, Jeff Sonas, made a concerted effort to improve the level of certainty for many of the tournaments from levels (2) and (3).  There are external sources of game-by-game results, most prominently the game databases published by Chessbase.  Jeff cross-referenced players and events between the FIDE summary results, and the Chessbase databases, in order to \"promote\" the summary results (level 3) or unknown-color results (level 2) into complete game records (level 1).

After this work of cross-referencing, all of the games with level 1 certainty were assembled into the primary training dataset, constituting 1,840,124 games spanning months 1-132.  The games from the test dataset (months 133-135) also are all level 1 certainty.  From examining the primary training dataset, you will see that there are 54,332 games in the first year, and the yearly count increases gradually up to 95,566 games in the eighth year.  All of those games in the first eight years (months 1-96) of the primary training dataset, and some of the games in the ninth year (months 97-108), came from the cross-referencing process.  The primary training data from the tenth and eleventh years (months 109-132) were directly collected by FIDE and did not require any enhancements.

However, this does leave us with a lot of useful data still remaining.  There were an additional 312,511 \"level 2\" games available from months 85-107, spanning the period where individual game outcomes were submitted but the color was unknown.  These games have been assembled into the secondary training dataset.  In these games, it is not known who had the white pieces and who had the black pieces.  Therefore, the players in the secondary training dataset are simply identified as \"Player 1\" and \"Player 2\", where the ID# of Player1 is always less than the ID# of Player2.  Otherwise the data in the secondary training dataset is analogous to that in the primary training dataset.  Please also remember that the data for each game in the secondary training datast also indicates values of Player1Prev and Player2Prev, representing how many games the two players had played in the primary training dataset during the previous 24 months.

Finally, there were still a lot of remaining tournaments where we don't know game-by-game results, but we do know a lot of \"level 3\" summary details (each player's rating, how many games each player had against rated opponents, and their total score and average opponent rating in those games).  Ideally we could work backward from those summary results into a set of games that satisfy the summary statistics, and these games would be helpful in stabilizing the rating pool for any rating system.  Of course there is a multitude of possible solutions for each tournament, or even no exact solutions due to errors in the reported numbers, but Jeff nevertheless tried to create an optimal set of games that were assembled into the tertiary training dataset.  Therefore these are \"imputed\" games; the games do not necessarily reflect the actual matchups or the actual results, but they do satisfy the known summary statistics for each player in the tournament.  This dataset, spanning 265,577 games from months 1-95, may or may not be useful to contest participants.

Just as with the secondary training dataset, it is not known who had the white pieces or the black pieces, and so the players in the tertiary training dataset are simply identified as \"Player 1\" and \"Player 2\", where the ID# of Player1 is always less than the ID# of Player2.  And the data for each game in the tertiary training datast also indicates values of Player1Prev and Player2Prev, representing how many games the two players had played in the primary training dataset during the previous 24 months.
", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Custom", "content": "In preparation for the contest, thirteen different \"benchmark\" entries were built and submitted.  They are summarized below.  These represent the implementation of rating calculation algorithms that are already well-known and well-documented.  Calling attention to these approaches can give contest participants a head-start in their research into optimal approaches, and the presence of these entries on the leaderboard will help you to gauge your own progress.  In addition, we will be asking the top ten finishers from the previous contest to execute their algorithms on the newer dataset, in order to prepare benchmark entries corresponding to their prizewinning approaches.  Those benchmarks will be posted to the leaderboard during the first month of the contest.
 
Simplistic Benchmarks

(1) All Draws Benchmark - This is the simplest possible entry, using an expected white score of 50% for every game in the test set.  As you would expect, it is the worst-performing of any of the benchmarks.  However, you can use an approach like this to verify proper submission format and to investigate basic behavior of the evaluation metric.

(2) White Advantage Benchmark - It is always a slight advantage to have the white pieces in a game of chess, because White always gets the first move.  The advantage of that extra move is highest among closely-matched top players, and less advantageous among weaker or mismatched players.  It is worth roughly 25-40 rating points on average, giving White an expected score of about 54% in general.  The White Advantage Benchmark is another extremely simplistic entry, using an expected white score of 53.875% for all games.  This is slightly more accurate than the All Draws Benchmark, and so it performs better, but only marginally so.

(3) Initial FIDE Ratings Benchmark - The initial ratings list provided as part of the contest gives ratings for more than 14,000 of the players in the contest, using the actual FIDE rating list published just prior to month 1 of the training set.  You might think, why not just use those ratings to make predictions for the test set?  Well, there are several problems with this.  For one thing, the initial rating list is 11+ years prior to the test period and is perhaps not that useful in making predictions for the games in the test set.  Further, almost 75% of players in the contest datasets did not have an initial rating, and so more than half of the test games are played between players who both lacked an initial rating.  Thus this list is of minimal direct use in making predictions.  Nevertheless it is possible to make predictions on the test set using the initial ratings when available, or a constant rating otherwise.  This approach certainly does perform a little better than the All Draws Benchmark or the White Advantage Benchmark.
 
Elo-related Benchmarks

(4) FIDE Benchmark (Primary training only) - The current FIDE regulations for calculating ratings were implemented exactly, including the details of how new ratings are obtained (although see parameter (n), below). See the FIDE Ratings Handbook here for more details about this (note that for purposes of rating calculations, all games for a player in a particular month are treated as being played in a single Swiss-style tournament).  Also see here for discussion from the previous contest about benchmarking Elo.  Ratings were calculating across the 11-year training period, starting from the initial ratings list (provided as part of the contest) and using only the primary training dataset.  These 15 system parameter values were used:

  (a) LowKFactor = 10 (this is the scaling factor used for updating the ratings of players in the \"Low K Factor\" group)

  (b) MediumKFactor = 15 (this is the scaling factor used for updating the ratings of players in the \"Medium K Factor\" group)

  (c) HighKFactor = 25 (this is the scaling factor used for updating the ratings of players in the \"High K Factor\" group)

  (d) MinGamesForRating = 9 (this is the minimum total games you need to play in order to receive a rating)

  (e) MinMonthGamesForUnrated = 3 (months only count toward an initial rating if the player plays at least this many rated games during the month; otherwise their games in the month are ignored)

  (f) MinMonthScoreForUnrated = 0.5 (months only count toward an initial rating if the player scores at least this many points in rated games during the month; otherwise their games in the month are ignored)

  (g) MinMonthScoreFirstUnrated = 1.0 (months only count toward an initial rating if the player scores at least this many points in rated games during their first month of play; otherwise their games in the month are ignored)

  (h) MaxEffectiveRatingDifference = 400 (rating differences larger than this between players in a single game are treated as this instead, for purposes of updating ratings)

  (i) InterpolateElo = 0 (if this is zero, use the exact lookup Elo table from the FIDE handbook; if this is one, interpolate linearly in order to calculate expected score or performance rating to additional precision)

  (j) RatingCutoffForLowK = 2400 (once a player has a rating at this or above, they will always stay in the Low K Factor group)

  (k) CareerGamesCutoffForHighK = 30 (once a player has played at least this many games in their career, they will always stay in the Low K Factor or Medium K Factor group, rather than the High K Factor group)

  (l) RatingDigitsAfterDecimal = 0 (this is the precision of the ratings that are tracked - zero means integer precision, one means ratings like 2345.6 are possible, two means ratings like 2345.67 are possible, etc.)

  (m) MinimumRating = 1200 (ratings dropping below this value result in players losing their rating and having to start over again, and also when players are unrated and trying to reach sufficient career rated games so they can receive an initial rating, their aggregate rating must never fall below this value cumulatively or the latest month results do not count)

  (n) MaxMonthSpanOfNewRating = 24 (when players are unrated and trying to reach sufficient career rated games so they can receive an initial rating, the various results can only span this many months; once their earliest results are older than this many months, the older results are discarded.  Note that this functionality was not implemented for the FIDE/Elo benchmarks but is a part of the FIDE rating regulations so it is listed here)

  (o) IncrementalNewRatingPointsPerPlus = 12.5 (when players are unrated and trying to reach a partial or initial rating, their rating performance when winning more games than they lose is the average rating of their opponents, plus this value times the difference between # wins and # losses)

The FIDE ratings handbook, linked to above, provides more details about the implementation of all these parameters.  When making predictions, all rating differences were compressed to 93% of their original value, then white was given 30 rating points, then the Elo formula (interpolated) was used to calculate expected rating difference.  Note that if a prediction was required for an unrated player, their rating was assumed to be 1909 if they had no rated games yet, or 1948 + (11*NumGames) if they had some rated games already (but fewer than 9).

(5) FIDE Benchmark (Primary + secondary) - Everything was done identically to the FIDE Benchmark (Primary training only), except that the primary and secondary training datasets were used (rather than just the primary training dataset).  Therefore you can see that the secondary training dataset did provide significant additional useful information.  All of the same system parameters were used.  When making predictions, all rating differences were compressed to 93% of their original value, then white was given 30 rating points, then the Elo formula (interpolated) was used to calculate expected rating difference.  Note that if a prediction was required for an unrated player, their rating was assumed to be 1853 if they had no rated games yet, or 1900 + (10*NumGames) if they had some rated games already (but fewer than 9).

(6) FIDE Benchmark - Everything was done identically to the FIDE Benchmark (Primary training only) and FIDE Benchmark (Primary + secondary), except that the primary and secondary and tertiary training datasets were used.  Therefore you can see that the tertiary training dataset did provide significant additional useful information.  All of the same system parameters were used.  When making predictions, all rating differences were compressed to 94% of their original value, then white was given 30 rating points, then the Elo formula (interpolated) was used to calculate expected rating difference.  Note that if a prediction was required for an unrated player, their rating was assumed to be 1821 if they had no rated games yet, or 1844 + (14*NumGames) if they had some rated games already (but fewer than 9).

(7) Actual FIDE Ratings Benchmark - This is the only benchmark where additional information, above and beyond what was provided for the contest, was used.  For this benchmark, the actual FIDE ratings of the players, at the time of the test games, were used for predicting.  When making predictions, all rating differences were compressed to 88.8% of their original value, then white was given 30 rating points, then the Elo formula (interpolated) was used to calculate expected rating difference.  Such an entry would be clearly disqualified for violating the rules, but it does provide a useful gauge for how well the various entries perform when compared to the accuracy of the actual existing FIDE rating system.  A large number of the games used during the 11-year training period for calculating FIDE ratings are not available in this contest, and so it makes sense that a benchmark using the actual FIDE ratings might be more accurate than a benchmark that faithfully implements the FIDE regulations but only uses the available data in the contest.  And so we see that the FIDE Benchmark does not do as well as the Actual FIDE Ratings Benchmark.  In fact the Actual FIDE Ratings Benchmark was not surpassed by anyone in the first contest (which had far less data provided), but it will clearly be surpassed in this contest, considering that both the Glicko Benchmark and Chessmetrics Benchmark have already surpassed it significantly, and the relatively simplistic Optimized Elo Benchmark also did slightly better.

(8) Optimized Elo Benchmark - Everything was done identically to the FIDE Benchmark, except that some system parameters were changed:

  (a) LowKFactor = 14 (all K-factors were increased by 40%)

  (b) MediumKFactor = 21 (all K-factors were increased by 40%)

  (c) HighKFactor = 35 (all K-factors were increased by 40%)

  (d) MinGamesForRating = 6 (making it easier to get a rating, so that predictions are not so influenced by treatment of unrated players)

  (e) MinMonthGamesForUnrated = 2 (making it easier to get a rating, so that predictions are not so influenced by treatment of unrated players)

  (g) MinMonthScoreFirstUnrated = 0.5 (making it easier to get a rating, so that predictions are not so influenced by treatment of unrated players)

When making predictions, all rating differences were compressed to 91% of their original value, then white was given 30 rating points, then the Elo formula (interpolated) was used to calculate expected rating difference.  Note that if a prediction was required for an unrated player, their rating was assumed to be 1694 if they had no rated games yet, or 1745 + (10*NumGames) if they had some rated games already (but fewer than 6).

Benchmarks of Other Approaches

(9) PCA Benchmark (Primary training only) - This approach was previously documented in the first contest and was implemented again on the new contest.  It suffers from the fact that new ratings are quite slow to accumulate under this system, and so a large portion of games in the test set involved predicting results for unrated players.  This is particularly problematic in this contest because the quantity of games and new players increases so rapidly in the final three years, and systems that are slower to accumulate new ratings will be harmed most strongly by this.  When making predictions, all rating differences were compressed to 75% of their original value, then white was given 30 rating points, then the Elo formula (interpolated) was used to calculate expected rating difference.  Also see here for discussion from the previous contest about benchmarking PCA.
(10) Glicko Benchmark (Primary training only) - This approach was previously documented in the first contest and was implemented again on the new contest.  Also see here for discussion from the previous contest about benchmarking Glicko.  Ratings were calculated across the 11-year training period, starting from the initial ratings list (provided as part of the contest) and using only the primary training dataset.  Players on that initial rating list were given an initial RD of 100 and a rating equal to their FIDE rating - 600, and everyone else was given an initial RD of 350 and a rating of 1500.  The Glicko constant system parameter known as c was 63.2.  The white advantage was 25 rating points, and the formula for predicting expected value, given the ratings r1 (for White) and r2  (for Black), and the rating deviations RD1 and RD2 for the two players, was  1/(1 + 10^{-g(sqrt(RD1^2 + RD2^2))(25.00 + r1 - r2)/400}), where g(x) = 1/sqrt(1+3q^2x^2/pi^2) and q=(ln 10)/400 and pi=3.14159265...  Unlike the previously-mentioned approaches, it was not necessary to compress the rating difference before making predictions, which apparently is a characteristic of a more accurate rating system.  Also note that Glicko-2 was not benchmarked for this contest because of a floating point error that can happen on rare occasions during the iterative optimization of a parameter, and this problem prevented complete implementation of the Glicko-2 approach.  In preliminary tests, however, Glicko-2 (with a Tau value of 0.5) appeared to perform very slightly better than Glicko, just as in the first contest.

(11) Glicko Benchmark - Everything was done identically to the Glicko Benchmark (Primary training only), except that it used the primary, secondary and tertiary datasets together as the training dataset.  As expected, this improved the accuracy of the predictions, although not as dramatically as for the FIDE benchmarks.  Note that the Glicko system is the best-performing benchmarked approach that meets the restrictive conditions for a \"practical chess rating system\".  The Glicko-2 system would fail to qualify because of its need to iteratively determine a volatility-related constant for each player, and the Chessmetrics system would fail to qualify because of its need to retain a longer game history for each player, rather than just maintaining a few parameters as is done in Elo or Glicko.

(12) Chessmetrics Benchmark (Primary training only) - This approach was previously documented in the first contest and was implemented again on the new contest, with just the primary training dataset.  A few important parameters had to be modified in order to work well with entire pool of players.  Previously it seemed that a linear relationship between rating difference and expected score was most appropriate, but this appears to only be true when you restrict your pool of players to the strongest group (as was done in the first contest).  When applied to the entire pool of players, as in this contest, it is clear that the logistic expectancy relationship is better (actually the interpolated Elo table from the FIDE rating handbook was used) and so that was used (rather than a linear relationship) for the Chessmetrics system.  In addition, rather than several multiple \"fake draws\", the best performance was obtained when using one fake draw against your average opponent rating, and one fake draw against a hypothetical opponent rated 2000.  When making predictions, the advantage for the white pieces was 28 rating points, and it was not necessary to assume a rating for any unrated/provisional players, because every player in the test set already had a regular Chessmetrics rating.  Also see here for discussion from the previous contest about benchmarking Chessmetrics.

(13) Chessmetrics Benchmark - Everything was done identically to the Chessmetrics Benchmark (Primary training only), except that it used the primary, secondary and tertiary datasets together as the training dataset.  In fact this did not improve the accuracy of the predictions.  The extra game data is extremely helpful for most rating systems (such as Glicko, FIDE/Elo, or PCA) where the only way you can get a new rating is by playing other players who are already rated.  Adding more game data, especially early in the training period, gets people their ratings sooner and so other people can get ratings too.  That ultimately means fewer unrated/provisional players that you have to make uncertain predictions about, and also gives everyone's rating more time in which to reach an equilibrium value.  But for Chessmetrics, which doesn't care whether you were rated at the time of the game and just uses everyone's games from the past four years, having extra games early on (prior to month 85) doesn't matter at all.  And apparently the quality of the primary training games was already sufficient, so that adding the extra secondary/tertiary games from the final four years of the training period didn't help predictions for Chessmetrics.

", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "data-description", "content": "The data files cover a recent 135-month period of professional chess game outcomes, extracted from the database of the world chess federation (FIDE).  The training period spans 132 months, followed immediately by a three-month test period (months 133-135).  The specific dates are not provided; all games simply have an integer Month ID# ranging from 1 to 135.  There are 54,205 different chess players included in the data; those players are uniquely identified by an ID ranging from 1 to 54,205.

There are eight different files provided as part of the contest:

(1-3) The primary training dataset (including 1,840,124 games) can be used to train your prediction system.  It is split up into three separate files: part 1 (months 1-96), part 2 (months 97-118), and part 3 (months 119-132).

(4-5) A secondary training dataset (including 312,511 games) and tertiary training dataset (including 265,577 games) provide optional additional training data that may be useful in validating your system or predicting game results.  The contest does not require you to use these additional training datasets, but most systems will benefit significantly from using the secondary and tertiary training datasets. More details can be found on the Additional Training Datasets page.

(6) A test dataset (including 100,000 games) identifies the chess games during the test period that must be predicted.  Also note that an example submission file is provided on the Submission Instructions page.

(7) An initial rating list (including 14,118 players) provides an initial list of players' FIDE ratings going into the start of the training period.  This represents approximately 25% of the pool of players; initial ratings for the remaining 75% of players must be calculated based on their initial games during the training period.  Remember that anyone wishing to qualify for the FIDE prize category must use this list to determine the initial ratings of these 14,118 players.

(8) An example submission file (including 100,001 rows) is formatted appropriately for submission to the contest, containing a prediction of an expected score of 50% for all 100,000 games in the test set.  This is identical to the All Draws Benchmark.  It includes a header row followed by predictions for test games #1, #2, #3, ..., #99,998, #99,999, #100,000, for a total of 100,001 rows.  Your actual submissions should be exactly like this file except that it will contain your actual predictions for every game, instead of a universal 50%.

Primary Training Dataset

There are 1,840,124 games in the primary training set, spanning a recent eleven-year period (known as Month #1 through Month #132).  They represent 35-40% of the almost five million game results that were used by FIDE (the world chess federation) to calculate player ratings during that time.  Many of those five million game results are not individually available in computer files, and so they had to be partially reconstructed from other sources for this contest; this is why there are \"only\" 1.84 million games in the primary training set.  For much of the training period, the primary training set contains about 25% of the full set of FIDE games used for official rating calculations.  The percentage is much higher in the final two or three years of the training period, thanks to recent changes in FIDE's regulations for how tournament organizers must submit their tournament's results to FIDE.  Recent data is much more readily available.

There are three possible outcomes to a completed chess game - either White wins, or the game is drawn, or Black wins.  These correspond to a \"score\" for White of 1.0, 0.5, or 0.0 points, and conversely a score for Black of 0.0, 0.5, or 1.0 points.  Tournament scoring typically adds up a player's total score, so someone who wins three games, draws five games, and loses one game would have a total score of 5.5 out of 9 games, and (despite having fewer wins) would finish ahead of someone else who wins five games, draws zero games and loses four games (total score of 5.0 out of 9).  Thus it is important to consider not just how many games a player manages to win, but also how many losses they manage to avoid.  White always gets to move first and so there is a slight advantage to having the white pieces (more pronounced among the strongest players).  For instance, in the primary training set, there are 125,000 games where the absolute difference in FIDE ratings between the players was fewer than 20 points, and the average score for White in those games was 53.7%.

The primary training dataset includes 1,840,124 total games, each of which is assigned a unique Primary Training Game ID# (called PTID) between 1 and 1,840,124.  For each game, the PTID, and the Month ID (between 1 and 132), and the player ID #'s for the White and Black players, are provided, as well as the outcome of the game from White's perspective (a score of either 1.0, 0.5, or 0.0).  The FIDE ratings of the players are not provided in the primary training dataset; part of the challenge is for you to make your own assessments of the players' changing strengths over time.

Finally, the primary training dataset provides two redundant values (named WhitePlayerPrev and BlackPlayerPrev) for each game result, indicating the quantity of games (in the primary training set) played by each of the two players within the previous 24 months.  These values could actually be calculated by reviewing the data from earlier months in the primary training dataset, rather than being explicitly provided, but for your convenience they have been explicitly provided for each training game.

Why are WhitePlayerPrev and BlackPlayerPrev relevant?  Well, the pool of active players is always increasing in size.  Thus in any given month, many of the games that are played involve players who have previously played a very small number of games (or none at all).  It seemed unfair to include many of those games in the test set, since the predictions would necessarily be quite uncertain.  So in the creation of the test set for months 133-135, there was a filter applied.  Instead of including all games played during months 133-135, the test dataset only includes games in which both players had at least 12 games played during the final 24 months of the primary training dataset (i.e. months 109-132).  Participants may wish to perform cross-validation, drawing samples from the primary training set that would have similar characteristics to the test dataset, in order to develop a model or optimize parameters in the model.  In order to create your own validation sets from the training data, you would presumably want to apply the same sampling filter that was used to create the test set (requiring WhitePlayerPrev > 12 and BlackPlayerPrev > 12).  Including the values of WhitePlayerPrev and BlackPlayerPrev in the training games allows you to do this.

For instance, if your validation test set was drawn from the games of month 130 only, and you only selected games from that month having WhitePlayerPrev > 12 and BlackPlayerPrev > 12, then you would only be getting games in which both players had at least 12 games played in the previous 24 months of the primary training set (i.e. months 106-129).  And this filter is analogous to what was done to filter the games for the test set.

Test Dataset

The test dataset (covering 100,000 games between months 133-135) is very similar to the primary training dataset, although it intentionally omits the WhiteScore column.  Of course the point of the contest is to have participants accurately predict the WhiteScore for all games in the test dataset, so the test dataset only indicates the identities of the players, and the month in which the game was played, and does not tell you the game outcomes.  The test dataset includes all known games played in months 133-135 where both players had played 12 or more games in the final 24 months of the primary training dataset.

The test dataset includes 100,000 total games, each of which is assigned a unique Test Game ID# (called TEID) between 1 and 100,000.  For each game, the TEID, and the Month ID (between 133 and 135), and the player ID's for the White and Black players, are provided.  The TEID is particularly important because you will use it to uniquely identify each test game when submitting predictions, and to sort the rows in your submission file.

Please note that you should NOT use the test dataset as an additional source of clues about a player's strength.  The predictions for months 133-135 should be based upon the players' estimated playing abilities at the end of month 132, and these predictions must be completely prospective, as though you made the predictions right at the end of month 132.  We could have asked for a complete cross product (54,205 x 54,205 x 3) of predictions of all possible games, instead of just the 100,000 games from the test set, but this would have been too large a file to submit.  Instead, in order to discourage participants from \"mining\" the test dataset for additional information, the test dataset of 100,000 games includes many thousands of spurious matchups.  The predictions for those particular games will be discarded and not used when scoring submissions.  The method for generating the spurious matchups is not random, and is not being revealed.  The spurious games are intentionally being included, in order to encourage people to only predict prospectively.

Pool of Players

The contest datasets do not include games for all players in the FIDE rating pool.  There are many isolated pockets of chess players around the world, playing only within their geographical region or age group, who are not well-connected with the rest of the pool of players.  We wanted to focus on a large pool of reasonably well-connected, reasonably active players in this contest, spanning the whole range of playing strength from weakest to strongest.  Therefore an iterative selection process was performed, starting with the four most prominent world champions (Viswanathan Anand, Garry Kasparov, Vladimir Kramnik, and Veselin Topalov) during that span.  They formed the initial pool of four players, and then additional players were iteratively added to the existing pool if they were well-connected to it.  The final pool represents the complete population of players whose games are included in the contest.  For these players, all known games during the appropriate time periods are included; no data was omitted intentionally for them.

So first there were four players in the pool, and then we looked to see how many other players were \"well-connected\" to those four initial players.  There are 19 players who played against all four of those opponents, having at least 15 games total against those four opponents as a group, during the eleven-year period.  Those 19 players, who can be thought of as \"one degree removed from a world champion\", were added to the pool, bringing it up to 23 total players.  Then, another iteration was performed against the pool of 23 players, identifying an additional 141 players who faced at least four different opponents in the pool of 23 players, and also played at least 15 games total against that pool, during the eleven-year period.  Those 141 players, \"two degrees removed from a world champion\", were added to the pool, bringing it up to a total of 164 players.  This process was performed eight additional times, always pulling in additional players who had played at least four different opponents in the existing pool, and at least 15 games total against existing pool members.  The final iteration brought in 2,007 new players (all of whom can be thought of as being \"ten degrees separated from a world champion\") resulting in a final pool of 54,205 players: exactly the players who are included in the contest datasets.  Each player was randomly assigned an ID# from 1 to 54,205, and players in the contest are always referenced by this ID# rather than by name.

Please note that the source games included in the secondary training dataset and tertiary training dataset were not used during this iterative process of forming the pool of chess players.  The secondary and tertiary training datasets are optional and therefore it was desirable to make sure the pool of players was well-connected within the primary training dataset only.  Analysis of the primary training dataset will confirm that all players in the contest have at least 15 games in the primary training dataset, including at least 4 different opponents.

Initial Rating List

Most rating systems require an initial rating list in order to start the system.  In these systems, players can only receive new ratings through playing games against opponents who already have ratings themselves, and therefore the system cannot start from nothing, as nobody would ever get the first rating.  A notable exception is the Chessmetrics rating algorithm, which does not use prior rating lists and always generates a rating list using only game results.

In the previous contest we did not provide an initial rating list; participants were required to generate their own ratings using their own approach.  This essentially created a \"blended\" contest, where you had to develop a good algorithm to generate initial ratings and also a separate algorithm to update existing ratings.  Some people even implemented the Chessmetrics algorithm just to create their initial ratings.  In order to avoid the \"blended\" competition, and to focus mainly on optimizing the process for updating players' existing ratings, we decided to include initial ratings for everyone who actually had a FIDE rating going into month #1.  Almost 75% of the 54,205 players in the contest did not yet have a rating at that point, so the initial rating list only contains 14,118 players, but it still should suffice to seed the rating pool sufficiently so that it can increase to include all (or almost all) of the test players by the end of the training period.  In addition, the initial ratings have been randomly adjusted up or down by several rating points in order to slightly anonymize the list and discourage contest participants from decoding the list and determining the identities of each player.  Note that this behavior is explicitly forbidden by the rules of the contest.
\r\n

\r\nUPDATE: Follow-up Data (files starting with \"followup_\")\r\n

\r\n

\r\nThese files represent a slightly larger dataset than is the case for the contest that was just completed. There were 54,205 players in the actual contest, with a training period spanning 132 months. The pool of players in the actual contest was formed by starting with four core world champions (Kasparov, Anand, Kramnik, Topalov) and then iteratively augmented by adding all players who had a total of 10+ games played (and at least 4 distinct opponents) against existing pool members across the first nine years of the training period, and only ten iterations of this were executed, leading to the 54,205 players. Instead, for this dataset, the restrictions were eased, as it allowed only 2+ distinct opponents (rather than 4+) and also the first eleven years of the training period were considered as a source of games (rather than just the first nine years). Four players who had no games after month #1 were removed, leaving exactly 88,000 players, exactly 19,000 of whom had initial ratings. And so with more players, we have more games.

\r\n

\r\nOther than having more players and more games, the data in these files is pretty much analogous to that of the actual contest, with a few minor exceptions:\r\n(1) The training period spans months 1-135 rather than 1-132\r\n(2) The test period spans months 136-138 rather than 133-135\r\n(3) In the previous training dataset, the PTID of the primary training games went from 1 to 1,840,124, and the STID of the secondary training games went from 2,000,001 to 2,312,511, and the TTID of the tertiary training games went from 3,000,001 to 3,265,577. This would allow you to uniquely identify each game by this ID even if you combined them into a single training dataset. However, in the larger dataset, there are more than 2 million primary training games, and so the same ID would have been shared by the end of the primary training games and the start of the secondary training games. Therefore the starting ID# for the secondary training set was changed to start at 2,500,001 instead. So now, the PTID of the primary training games goes from 1 to 2,327,683, and the STID of the secondary training games goes from 2,500,001 to 2,918,349, and the TTID of the tertiary training games goes from 3,000,001 to 3,394,324.\r\n(4) Player ID#'s are all exactly five digits, ranging from 10,001 to 98,000, rather than starting at 1 (as they did in the actual contest, where ID's ranged from 1 to 54,205)\r\n(5) Also note that the player ID#'s have been randomized again; there is no relationship between players' ID#'s in the actual contest and in these files\r\n(6) Once again there are many spurious games in the test set, and this time a much higher percentage of test set games are spurious.\r\n

", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Description", "content": "The datasets for this contest include real historical data provided by the world chess federation (FIDE).  Contest participants will train their rating systems using a training dataset of over 1.84 million game results for more than 54,000 chess players across a recent eleven year period.  Participants then use their method to predict the outcome of a further 100,000 games played among those same players during the following three months.  Contest entries will be scored automatically by the website, based on the accuracy of their predictions.  Two entries per day can be submitted by each team, and prizes will be determined according to each team's best-scoring single submission.

The contest's sponsor, Deloitte Australia, has provided the $10,000 prize to be awarded to the team that submits the most accurate predictions.  Deloitte is a preeminent provider of analytics globally and helps companies capture, manage and analyze their data as part of their overall business strategy.

Chessbase has donated chess software with signatures by famous players for the three teams finishing in 2nd/3rd/4th place.

In addition, FIDE representatives will award a special \"FIDE prize\" to what they consider to be the most promising approach, out of the ten most accurate entries that meet a restrictive definition of a \"practical chess rating system\".  This restrictive definition is specified within the rules of the contest.  For the winner, FIDE will provide air fare for a round trip flight to Athens, Greece, and full board for three nights in Athens, and payment toward other expenses, for one person to present and discuss their system during a special FIDE meeting of chess rating experts in Athens.
", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Evaluation", "content": "Contest submissions will contain the predicted White score in each of the 100,000 games in the test set.  The predictions for individual games will be scored separately and the aggregate \"deviance\" calculated as the average deviance across all games.  Also note that several thousand of the games are spurious (fake) matchups, included in order to discourage participants from \"mining\" the test set for additional information about each player's strength.  These games are ignored by the evaluation function.

The evaluation function used for scoring submissions will be the \"Binomial Deviance\" (or \"Log Likelihood\") statistic suggested by Mark Glickman, namely the mean of:

-[Y*LOG10(E) + (1-Y)*LOG10(1-E)]

per game, where Y is the game outcome (0.0 for a black win, 0.5 for a draw, or 1.0 for a white win) and E is the expected/predicted score for White, and LOG10() is the base-10 logarithm function.

The winning single submission will be the one having the minimum Binomial Deviance.  It is easy to see that predicting an expected score of 0% or 100% has an undefined Binomial Deviance, since LOG10(0) is undefined.  Further, even if White wins, the marginal benefit for a prediction of slightly above 99% is minimal compared to a 99% prediction.  And similarly, even if Black wins, the marginal benefit for a prediction of slightly below 1% is minimal compared to a 1% prediction.  Therefore there is really no good reason to predict an expected score above 99% or below 1%.  So for purposes of scoring, all predictions of a White score above 99% will be treated as 99%, and all predictions of a White score below 1% will be treated as 1%.  You can \"cap\" your predictions at 1% and 99% yourself, or you can let the scoring function do it for you; either way your score will be the same.

Here is an example of the calculation of the evaluation function for five sample games in a test set, where the participant was asked to predict the outcome for five games, one of which was actually a spurious (fake) matchup:

Game #1 (predicted White score = 0.650, actual White score = 0.50)
Game #2 (predicted White score = 0.220, actual White score = 0.00)
Game #3 (predicted White score = 0.000, actual White score = 0.50)
Game #4 (predicted White score = 0.330, game is spurious and therefore not scored)
Game #5 (predicted White score = 0.999, actual White score = 1.00)

The individual binomial deviances for each game would be:

Deviance for #1 = -[0.50*LOG10(0.65) + (1-0.50)*LOG10(1-0.65)] = 0.321509
Deviance for #2 = -[0.00*LOG10(0.22) + (1-0.00)*LOG10(1-0.22)] = 0.107905
Deviance for #3 = -[0.50*LOG10(0.01) + (1-0.50)*LOG10(1-0.01)] = 1.002182
Deviance for #4 = (ignored)
Deviance for #5 = -[1.00*LOG10(0.99) + (1-1.00)*LOG10(1-0.99)] = 0.004365

And the overall binomial deviance score for that submission would be the average of 0.321509, 0.107905, 1.002182, and 0.004365, or 0.358990.  Therefore the overall score of this submission (across these five rows from the test set) would be 0.358990.  Please notice how the predictions in games #3 and #5 were \"capped\" at 0.01 and 0.99 for purposes of the binomial deviance calculation, and how game #4 was not scored because it was spurious.

This graph provides a visual indication of the relative deviance contributions for each combination of actual outcomes and predicted scores:



\"Deviance

\r\n\r\n

\r\n

", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Prizes", "content": "Main Prizes

The main prizes are for the four teams whose single best submissions have the best score (i.e. the lowest Binomial Deviance score).  The contest's sponsor, Deloitte Australia, has provided the $10,000 prize to be awarded to the first place winner.  Deloitte is a preeminent provider of analytics globally and helps companies capture, manage and analyze their data as part of their overall business strategy.

   1st place: $10,000 prize provided by Deloitte Australia

In addition, Chessbase has donated three additional main prizes (prizewinners can select from among several available options):

   2nd place: Chess software with signature(s) by famous player(s)

   3rd place: Chess software with signature(s) by famous player(s)

   4th place: Chess software with signature(s) by famous player(s)
There are further details about eligibility and conditions for receiving prizes, on the Rules page.  Remember that in order to be eligible for the main prizes, participants must publicly reveal all details of their methodology, and must submit one additional set of predictions via email after the contest is complete.

In addition, the world chess federation (FIDE) has defined an additional prize category and a special prize for the contest:

FIDE Prize Category

There are many algorithms based on probability models that use the full history of all known games.  Although these approaches are quite accurate, they are not useful in the practical setting of a world chess federation's rating system.  Players want to be able to track and calculate their own rating and world ranking, and a popular chess rating system must therefore be reasonably simple and not require the full set of games from all players in order to calculate an updated rating for one player.  A practical chess rating system should only track a few parameters for each player, and should only require the game outcomes and the parameters of the player and their opponents (in the current rating period) in order to completely update the player's rating (and associated parameters).  If FIDE is to consider modifying its existing rating system in order to improve accuracy, the system must remain transparent and understandable.  Therefore we are defining what is meant by a \"practical chess rating system\" and providing a special prize, sponsored by FIDE, for the most promising submission from a practical chess rating system. 

On a regular basis, a small group of chess rating experts, from around the world, meets for a two-day \"experts meeting\" funded and organized by FIDE near their offices in Athens, Greece.  The meeting, located across the street from the Temple of Zeus and a short walk from the Parthenon/Acropolis, is used for discussion of current key issues pertaining to the FIDE chess rating system.  The organizer of this contest, Jeff Sonas, attended the experts meetings in June 2009 and June 2010.

The ten participants who submit the most accurate entries that meet the definition of a \"practical chess rating system\" will be eligible for the FIDE prize.  After reviewing the submissions (including description of methodology and accuracy of predictions), FIDE representatives will select one of the ten finalists to be the recipient of the FIDE prize.  For the prizewinner, FIDE will provide air fare for a round trip to Athens, Greece, and full board for three nights in Athens, Greece, and payment toward other expenses, for one person to attend the next experts meeting (tentatively scheduled for June 2011), and present/explain their prizewinning approach.

The details about what constitutes a \"practical chess rating system\" can be found on the Rules page.  Remember that in order to be eligible for the FIDE prize, participants must publicly reveal all details of their methodology, and must submit a full log of all players' rating parameters across the entire duration of the training period, as well as one additional set of predictions, via email after the contest is complete.
", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Rules", "content": "\r\n\t\r\n

One account per participant

\r\n

You cannot sign up to Kaggle from multiple accounts and therefore you cannot submit from multiple accounts.

\r\n

No private sharing outside teams

\r\n

\r\n Privately sharing code or data outside of teams is not permitted.\r\n It's okay to share code if made available to all participants on the forums.\r\n

\r\n

Team Mergers

\r\n

\r\n Team mergers are allowed and can be performed by the team leader. In order to merge, the combined team must have a total submission count less than or equal to the maximum allowed as of the merge date. The maximum allowed is the number of submissions per day multiplied by the number of days the competition has been running.
\r\n

\r\n\r\n

Team Limits

\r\n

There is no maximum team size.

\r\n

Submission Limits

\r\n

You may submit a maximum of 2 entries per day.

\r\n

You may select up to 5 final submissions for judging.

\r\n

Competition Timeline

\r\n

Start Date: 2/7/2011 12:00 AM UTC

\r\n

Merger Deadline: None

\r\n

Entry Deadline: None

\r\n

End Date: 5/4/2011 3:00 PM UTC

\r\n\t\r\nThis competition requires you to choose up to five entries that count towards the final result. To choose your five entries visit your submissions page and click the star next to each relevant entry to select it. You can do this at any time before the contest ends. If you do not choose any entries, your last five entries will be chosen by default.

There are two categories of prizes for this contest.  The qualification rules are different for the two categories, although the same team can win one prize in each category.  Please remember that each contest participant can only participate as a member of one team, and all team members must be registered on Kaggle for their contribution to be recognized.  Each team can make two submissions during each 24-hour period, throughout the duration of the contest.  Please also note that employees of Deloitte, FIDE, or Kaggle are welcome to compete but are not eligible to win prizes.

Main Prizes

The main prizes (including the $10,000 first prize provided by Deloitte Australia) will be awarded to the teams having the most accurate single submissions.  It doesn't matter whether a team submits other entries that are less accurate; only the most accurate entry submitted by each team will count toward the final standings.  The $10,000 first prize, provided by Deloitte Australia, will go to the team having the submission with the lowest Binomial Deviance score out of any submission in the contest.  2nd, 3rd, and 4th prizes, donated by Chessbase, will go to the teams that submitted the entries with the next-lowest Binomial Deviance scores.  These prizes will be chess software with signatures by famous players.

There are no restrictions on the methodology used to create accurate predictions and claim the main prizes, with one exception: any submissions will be disqualified if they include determining the actual identity of the chess players and using their publicly-known results during the test period, or their publicly-known FIDE ratings at any time after the start of the training period, in order to inform their predictions.

In order to receive a main prize, the top finishers must publicly reveal all details of their methodology within seven days after the completion of the contest, and must submit an additional set of predictions via email within seven days after the completion of the contest.  In order to make these additional predictions, prizewinners will be provided with additional training data and a new test dataset, within 24 hours of the completion of the contest, and must train their system and make their predictions according to the same methodology and system parameters that were used for their prizewinning submission.
 
FIDE Prize Category

The ten participants who submit the most accurate entries that meet the definition of a \"practical chess rating system\" will be eligible for the FIDE prize.  The contest organizer will work with top-performing participants during the final weeks of the contest in order to determine which participants appear to meet this definition.

After reviewing summaries of the finalists' systems (including description of methodology and accuracy of predictions), FIDE representatives will select one of the ten finalists to be the winner.  For the winner, FIDE will provide air fare for a round trip flight to Athens, Greece, and full board for three nights in Athens, and payment toward other expenses, for one person to present and discuss their system during a special FIDE meeting of chess rating experts in Athens.

For the purposes of this contest, a \"practical chess rating system\" is a prediction system for which all of the following are true:

(1) A rating vector V(X,M) of one or more numbers (maximum of ten) is maintained and updated on a monthly basis for each player X, representing the components of player X's rating at the start of month M.

(2) The initial rating vectors V(X,1), representing the components of player X's rating at the start of month 1, can only be a function of one or more of the following:
 (a) Player X's rating on the initial FIDE rating list provided as part of the contest
 (b) Player X's K-factor on the initial FIDE rating list provided as part of the contest
 (c) Player X's career # of games on the initial FIDE rating list provided as part of the contest
 (d) System constant parameters

(3) The predicted score E(X,Y,M) for player X in a single game against player Y during a particular month M in the test period (months 133-135), can only be a function of one or more of the following:
 (a) The rating vector V(X,133) for player X, representing the components of player X's rating at the start of month 133
 (b) The rating vector V(Y,133) for player Y, representing the components of player Y's rating at the start of month 133
 (c) System constant parameters
 (d) The details of whether player X has the white pieces, or player Y has the white pieces, in the game
 (e) The value of M (either 133, 134, or 135)

(4) When updating the rating vector for player X, from V(X,M) to the new values V(X,M+1), based on games G1, G2, ..., GN played by player X against opponents Y1, Y2, ..., YN during a particular month M, where N>0, those updates can only be a function of one or more of the following:
 (a) The rating vector V(X,M) representing the components of player X's rating at the start of month M
 (b) System constant parameters
 (c) For each game Gi and opponent Yi out of games G1, G2, ..., GN played by player X against opponents Y1, Y2, ..., YN during month M:
   (i) The rating vector V(Yi,M) representing the components of player Yi's rating at the start of month M
   (ii) The game outcome (win/draw/loss) for player X in game Gi
   (iii) The details of whether player X had the white pieces, or player Yi had the white pieces, or the color of pieces was unknown.
   (iv) The details of whether game Gi came from the primary training dataset, secondary training dataset, or tertiary training dataset
 
(5) Rating vector updates cannot involve any iterative computation that is carried out to convergence in order to solve an optimality criterion (it can require, at most, two iterations of such an algorithm).

(6) Any player X who is not included in the initial FIDE rating list provided as part of the contest, and therefore \"enters\" the system due to their first month M where they played any games, can either receive an initial rating vector V(X,1) that is maintained unchanged until month M+1, or they can receive their first rating vector V(X,M+1) as part of the rating updating algorithm described above in (4); either approach is acceptable.

(7) System constant parameters may be optimized by the contestant in any fashion they desire, but such values are to remain unchanged and independent of month M when applying the rating updating algorithm described above.

The ten finalists for the FIDE prize must demonstrate their qualification as a \"practical chess rating system\", and their eligibility for the FIDE prize, by providing a data log to the contest organizer (via email within seven days of the completion of the contest), including the entire set of rating vectors for all players across all months, as well as publicly revealing all details of their methodology.  Any submissions will be disqualified if they include determining the actual identity of the chess players and using their publicly-known results during the test period, or their published FIDE ratings at any time after the start of the training period, in order to inform their predictions.

Finalists for the FIDE prize must publicly reveal all details of their methodology within seven days after the completion of the contest, and must submit an additional set of predictions via email within seven days after the completion of the contest.  In order to make these additional predictions, finalists will be provided with additional training data and a new test dataset, within 24 hours of the completion of the contest, and must train their system and make their predictions according to the same methodology and system parameters that were used for their winning submission.

Disclaimers

Every effort has been made to be thorough in the definition of the contest rules prior to the beginning of the contest.  However, it is possible there are errors or omissions.  The organizers of the contest reserve the right to make changes to the rules, or the data files, after the contest begins, if sufficient need arises.

Please note that the contest organizers and sponsors also reserve the right to use the submitted data for the purpose of comparative analysis.  Prior to the start of the contest, 25 different chess tournaments were identified from within the test dataset, that are considered to be of particular analytical interest.  These tournament games make up approximately 15% of the entire test dataset, including 3,700 games from eight elite tournaments (all having average FIDE rating of participants well above 2400), 1,300 games from two large women's tournaments, 1,100 games from two large seniors' tournaments, and 350 games from three junior national championships, as well as 7,300 games from ten different open tournaments of varying strength, size, and geographical location.  The contest organizer intends to assess the relative performance of the contest predictions for those tournaments, compared to the predictions that are possible using the players' actual FIDE ratings at the time of the tournaments, and to see whether there are particular types of player (weak, strong, young, etc.) where the winning methodology provides the largest increase in predictive accuracy, compared to the FIDE approach.

", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "SubmissionInstructions", "content": "The test dataset lists 100,000 games played during the test period (months 133-135).  Each of these games is uniquely identified by a TEID (Test Game ID) ranging from 1 to 100000.  The test dataset provides sufficient information about each of these games, including the month it was played in and the ID numbers for White and Black, for you to make predictions of the outcome of each game.

When submitting predictions, you must include a file with all 100,000 games listed, along with a header row.  For each game, you will provide the TEID and the expected score for White (a decimal value between 0 and 1).  Please note that the scoring function will treat all expected scores below 0.01 as 0.01, and will treat all expected scores above 0.99 as 0.99.

You must make sure your games are listed in ascending numerical order, according to the TEID.  The first row is your header row, the second row is your prediction for TEID#1, the third row is your prediction for TEID#2, and so on, ending with the final row having your prediction for TEID#100000.  Therefore all files should have exactly 100,001 rows.

Files should be formatted in Comma-Separated-Value (CSV) format, having a \".CSV\" filename extension and separating data values on the same row using a comma.

As an example, if you submitted a file containing predictions of 0.500 for White's expected score in the first three games, the first four rows would look like this:

TEID,WhiteExpect
1,0.500000
2,0.500000
3,0.500000
", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false } ] }