Each day your team is able to submit up to 5 agents (bots) to the competition. Each submission will play episodes (games) against other bots on the leaderboard that have a similar skill rating. Over time, skill ratings will go up with wins, down with losses, or evened out with ties.

Every bot submitted will continue to play episodes until the end of the competition, with newer bots selected to play more frequently. Once you have reached two active submissions, older ones will be deactivated. On the leaderboard, only your best scoring bot will be shown, but you can track the progress of all of your submissions on your Submissions page.

Each submission has an estimated skill rating which is modeled by a Gaussian N(μ,σ2) where μ is the estimated skill and σ represents the uncertainty of that estimate which will decrease over time.

When you upload a submission, we first play a validation episode where that submission plays against copies of itself to make sure it works properly. If the episode fails, the submission is marked as error and you can download the agent logs to help figure out why. Otherwise, we initialize the submission with μ0=600 and it joins the pool of for ongoing evaluation. At this time we also deactivate older agents if the total number of active agents is greater than three.

###Ranking System
After an episode finishes, we'll update the rating estimate for all bots in the episode. If one bot pair won, we'll increase their μ and decrease the opponent's μ -- if the result was a tie, then we'll move the μ values closer towards their mean. The updates will have magnitude relative to the deviation from the expected result based on the previous μ values, and also relative to each bot’s uncertainty σ. We also reduce the σ terms relative to the amount of information gained by the result. The score by which your bot wins or loses an episode does not affect the skill rating updates.

###Final Evaluation
At the submission deadline on February 11, 2025, submissions will be locked and the leaderboard will be reset. From February 11, 2025 to (approximately) February 25th, 2025 (update: March 6th, 2025) we will continue to run episodes. During this period only your active submissions will be eligible for the leaderboard. At the conclusion of this period, the leaderboard is final.