In preparation for the contest, thirteen different "benchmark" entries were built and submitted.  They are summarized below.  These represent the implementation of rating calculation algorithms that are already well-known and well-documented.  Calling attention to these approaches can give contest participants a head-start in their research into optimal approaches, and the presence of these entries on the leaderboard will help you to gauge your own progress.  In addition, we will be asking the top ten finishers from the previous contest to execute their algorithms on the newer dataset, in order to prepare benchmark entries corresponding to their prizewinning approaches.  Those benchmarks will be posted to the leaderboard during the first month of the contest.
Simplistic Benchmarks
(1) All Draws Benchmark - This is the simplest possible entry, using an expected white score of 50% for every game in the test set.  As you would expect, it is the worst-performing of any of the benchmarks.  However, you can use an approach like this to verify proper submission format and to investigate basic behavior of the evaluation metric.
(2) White Advantage Benchmark - It is always a slight advantage to have the white pieces in a game of chess, because White always gets the first move.  The advantage of that extra move is highest among closely-matched top players, and less advantageous among weaker or mismatched players.  It is worth roughly 25-40 rating points on average, giving White an expected score of about 54% in general.  The White Advantage Benchmark is another extremely simplistic entry, using an expected white score of 53.875% for all games.  This is slightly more accurate than the All Draws Benchmark, and so it performs better, but only marginally so.
(3) Initial FIDE Ratings Benchmark - The initial ratings list provided as part of the contest gives ratings for more than 14,000 of the players in the contest, using the actual FIDE rating list published just prior to month 1 of the training set.  You might think, why not just use those ratings to make predictions for the test set?  Well, there are several problems with this.  For one thing, the initial rating list is 11+ years prior to the test period and is perhaps not that useful in making predictions for the games in the test set.  Further, almost 75% of players in the contest datasets did not have an initial rating, and so more than half of the test games are played between players who both lacked an initial rating.  Thus this list is of minimal direct use in making predictions.  Nevertheless it is possible to make predictions on the test set using the initial ratings when available, or a constant rating otherwise.  This approach certainly does perform a little better than the All Draws Benchmark or the White Advantage Benchmark.
Elo-related Benchmarks
(4) FIDE Benchmark (Primary training only) - The current FIDE regulations for calculating ratings were implemented exactly, including the details of how new ratings are obtained (although see parameter (n), below). See the FIDE Ratings Handbook here for more details about this (note that for purposes of rating calculations, all games for a player in a particular month are treated as being played in a single Swiss-style tournament).  Also see here for discussion from the previous contest about benchmarking Elo.  Ratings were calculating across the 11-year training period, starting from the initial ratings list (provided as part of the contest) and using only the primary training dataset.  These 15 system parameter values were used:
(a) LowKFactor = 10 (this is the scaling factor used for updating the ratings of players in the "Low K Factor" group)
(b) MediumKFactor = 15 (this is the scaling factor used for updating the ratings of players in the "Medium K Factor" group)
(c) HighKFactor = 25 (this is the scaling factor used for updating the ratings of players in the "High K Factor" group)
(d) MinGamesForRating = 9 (this is the minimum total games you need to play in order to receive a rating)
(e) MinMonthGamesForUnrated = 3 (months only count toward an initial rating if the player plays at least this many rated games during the month; otherwise their games in the month are ignored)
(f) MinMonthScoreForUnrated = 0.5 (months only count toward an initial rating if the player scores at least this many points in rated games during the month; otherwise their games in the month are ignored)
(g) MinMonthScoreFirstUnrated = 1.0 (months only count toward an initial rating if the player scores at least this many points in rated games during their first month of play; otherwise their games in the month are ignored)
(h) MaxEffectiveRatingDifference = 400 (rating differences larger than this between players in a single game are treated as this instead, for purposes of updating ratings)
(i) InterpolateElo = 0 (if this is zero, use the exact lookup Elo table from the FIDE handbook; if this is one, interpolate linearly in order to calculate expected score or performance rating to additional precision)
(j) RatingCutoffForLowK = 2400 (once a player has a rating at this or above, they will always stay in the Low K Factor group)
(k) CareerGamesCutoffForHighK = 30 (once a player has played at least this many games in their career, they will always stay in the Low K Factor or Medium K Factor group, rather than the High K Factor group)
(l) RatingDigitsAfterDecimal = 0 (this is the precision of the ratings that are tracked - zero means integer precision, one means ratings like 2345.6 are possible, two means ratings like 2345.67 are possible, etc.)
(m) MinimumRating = 1200 (ratings dropping below this value result in players losing their rating and having to start over again, and also when players are unrated and trying to reach sufficient career rated games so they can receive an initial rating, their aggregate rating must never fall below this value cumulatively or the latest month results do not count)
(n) MaxMonthSpanOfNewRating = 24 (when players are unrated and trying to reach sufficient career rated games so they can receive an initial rating, the various results can only span this many months; once their earliest results are older than this many months, the older results are discarded.  Note that this functionality was not implemented for the FIDE/Elo benchmarks but is a part of the FIDE rating regulations so it is listed here)
(o) IncrementalNewRatingPointsPerPlus = 12.5 (when players are unrated and trying to reach a partial or initial rating, their rating performance when winning more games than they lose is the average rating of their opponents, plus this value times the difference between # wins and # losses)
The FIDE ratings handbook, linked to above, provides more details about the implementation of all these parameters.  When making predictions, all rating differences were compressed to 93% of their original value, then white was given 30 rating points, then the Elo formula (interpolated) was used to calculate expected rating difference.  Note that if a prediction was required for an unrated player, their rating was assumed to be 1909 if they had no rated games yet, or 1948 + (11*NumGames) if they had some rated games already (but fewer than 9).
(5) FIDE Benchmark (Primary + secondary) - Everything was done identically to the FIDE Benchmark (Primary training only), except that the primary and secondary training datasets were used (rather than just the primary training dataset).  Therefore you can see that the secondary training dataset did provide significant additional useful information.  All of the same system parameters were used.  When making predictions, all rating differences were compressed to 93% of their original value, then white was given 30 rating points, then the Elo formula (interpolated) was used to calculate expected rating difference.  Note that if a prediction was required for an unrated player, their rating was assumed to be 1853 if they had no rated games yet, or 1900 + (10*NumGames) if they had some rated games already (but fewer than 9).
(6) FIDE Benchmark - Everything was done identically to the FIDE Benchmark (Primary training only) and FIDE Benchmark (Primary + secondary), except that the primary and secondary and tertiary training datasets were used.  Therefore you can see that the tertiary training dataset did provide significant additional useful information.  All of the same system parameters were used.  When making predictions, all rating differences were compressed to 94% of their original value, then white was given 30 rating points, then the Elo formula (interpolated) was used to calculate expected rating difference.  Note that if a prediction was required for an unrated player, their rating was assumed to be 1821 if they had no rated games yet, or 1844 + (14*NumGames) if they had some rated games already (but fewer than 9).
(7) Actual FIDE Ratings Benchmark - This is the only benchmark where additional information, above and beyond what was provided for the contest, was used.  For this benchmark, the actual FIDE ratings of the players, at the time of the test games, were used for predicting.  When making predictions, all rating differences were compressed to 88.8% of their original value, then white was given 30 rating points, then the Elo formula (interpolated) was used to calculate expected rating difference.  Such an entry would be clearly disqualified for violating the rules, but it does provide a useful gauge for how well the various entries perform when compared to the accuracy of the actual existing FIDE rating system.  A large number of the games used during the 11-year training period for calculating FIDE ratings are not available in this contest, and so it makes sense that a benchmark using the actual FIDE ratings might be more accurate than a benchmark that faithfully implements the FIDE regulations but only uses the available data in the contest.  And so we see that the FIDE Benchmark does not do as well as the Actual FIDE Ratings Benchmark.  In fact the Actual FIDE Ratings Benchmark was not surpassed by anyone in the first contest (which had far less data provided), but it will clearly be surpassed in this contest, considering that both the Glicko Benchmark and Chessmetrics Benchmark have already surpassed it significantly, and the relatively simplistic Optimized Elo Benchmark also did slightly better.
(8) Optimized Elo Benchmark - Everything was done identically to the FIDE Benchmark, except that some system parameters were changed:
(a) LowKFactor = 14 (all K-factors were increased by 40%)
(b) MediumKFactor = 21 (all K-factors were increased by 40%)
(c) HighKFactor = 35 (all K-factors were increased by 40%)
(d) MinGamesForRating = 6 (making it easier to get a rating, so that predictions are not so influenced by treatment of unrated players)
(e) MinMonthGamesForUnrated = 2 (making it easier to get a rating, so that predictions are not so influenced by treatment of unrated players)
(g) MinMonthScoreFirstUnrated = 0.5 (making it easier to get a rating, so that predictions are not so influenced by treatment of unrated players)
When making predictions, all rating differences were compressed to 91% of their original value, then white was given 30 rating points, then the Elo formula (interpolated) was used to calculate expected rating difference.  Note that if a prediction was required for an unrated player, their rating was assumed to be 1694 if they had no rated games yet, or 1745 + (10*NumGames) if they had some rated games already (but fewer than 6).
Benchmarks of Other Approaches
(9) PCA Benchmark (Primary training only) - This approach was previously documented in the first contest and was implemented again on the new contest.  It suffers from the fact that new ratings are quite slow to accumulate under this system, and so a large portion of games in the test set involved predicting results for unrated players.  This is particularly problematic in this contest because the quantity of games and new players increases so rapidly in the final three years, and systems that are slower to accumulate new ratings will be harmed most strongly by this.  When making predictions, all rating differences were compressed to 75% of their original value, then white was given 30 rating points, then the Elo formula (interpolated) was used to calculate expected rating difference.  Also see here for discussion from the previous contest about benchmarking PCA.
(10) Glicko Benchmark (Primary training only) - This approach was previously documented in the first contest and was implemented again on the new contest.  Also see here for discussion from the previous contest about benchmarking Glicko.  Ratings were calculated across the 11-year training period, starting from the initial ratings list (provided as part of the contest) and using only the primary training dataset.  Players on that initial rating list were given an initial RD of 100 and a rating equal to their FIDE rating - 600, and everyone else was given an initial RD of 350 and a rating of 1500.  The Glicko constant system parameter known as c was 63.2.  The white advantage was 25 rating points, and the formula for predicting expected value, given the ratings r1 (for White) and r2  (for Black), and the rating deviations RD1 and RD2 for the two players, was  1/(1 + 10^{-g(sqrt(RD1^2 + RD2^2))(25.00 + r1 - r2)/400}), where g(x) = 1/sqrt(1+3q^2x^2/pi^2) and q=(ln 10)/400 and pi=3.14159265...  Unlike the previously-mentioned approaches, it was not necessary to compress the rating difference before making predictions, which apparently is a characteristic of a more accurate rating system.  Also note that Glicko-2 was not benchmarked for this contest because of a floating point error that can happen on rare occasions during the iterative optimization of a parameter, and this problem prevented complete implementation of the Glicko-2 approach.  In preliminary tests, however, Glicko-2 (with a Tau value of 0.5) appeared to perform very slightly better than Glicko, just as in the first contest.
(11) Glicko Benchmark - Everything was done identically to the Glicko Benchmark (Primary training only), except that it used the primary, secondary and tertiary datasets together as the training dataset.  As expected, this improved the accuracy of the predictions, although not as dramatically as for the FIDE benchmarks.  Note that the Glicko system is the best-performing benchmarked approach that meets the restrictive conditions for a "practical chess rating system".  The Glicko-2 system would fail to qualify because of its need to iteratively determine a volatility-related constant for each player, and the Chessmetrics system would fail to qualify because of its need to retain a longer game history for each player, rather than just maintaining a few parameters as is done in Elo or Glicko.
(12) Chessmetrics Benchmark (Primary training only) - This approach was previously documented in the first contest and was implemented again on the new contest, with just the primary training dataset.  A few important parameters had to be modified in order to work well with entire pool of players.  Previously it seemed that a linear relationship between rating difference and expected score was most appropriate, but this appears to only be true when you restrict your pool of players to the strongest group (as was done in the first contest).  When applied to the entire pool of players, as in this contest, it is clear that the logistic expectancy relationship is better (actually the interpolated Elo table from the FIDE rating handbook was used) and so that was used (rather than a linear relationship) for the Chessmetrics system.  In addition, rather than several multiple "fake draws", the best performance was obtained when using one fake draw against your average opponent rating, and one fake draw against a hypothetical opponent rated 2000.  When making predictions, the advantage for the white pieces was 28 rating points, and it was not necessary to assume a rating for any unrated/provisional players, because every player in the test set already had a regular Chessmetrics rating.  Also see here for discussion from the previous contest about benchmarking Chessmetrics.
(13) Chessmetrics Benchmark - Everything was done identically to the Chessmetrics Benchmark (Primary training only), except that it used the primary, secondary and tertiary datasets together as the training dataset.  In fact this did not improve the accuracy of the predictions.  The extra game data is extremely helpful for most rating systems (such as Glicko, FIDE/Elo, or PCA) where the only way you can get a new rating is by playing other players who are already rated.  Adding more game data, especially early in the training period, gets people their ratings sooner and so other people can get ratings too.  That ultimately means fewer unrated/provisional players that you have to make uncertain predictions about, and also gives everyone's rating more time in which to reach an equilibrium value.  But for Chessmetrics, which doesn't care whether you were rated at the time of the game and just uses everyone's games from the past four years, having extra games early on (prior to month 85) doesn't matter at all.  And apparently the quality of the primary training games was already sufficient, so that adding the extra secondary/tertiary games from the final four years of the training period didn't help predictions for Chessmetrics.