{
  "id": 573496,
  "title": "So… We cooked a model 🤷",
  "content": "<p>Congratulations to the winners and huge thanks to the AIMO team for hosting such an intense competition that pushed the frontier of open-source reasoning models. I'd like to express my deep gratitude to the best teammates I could have ever asked for, especially <a href=\"https://www.kaggle.com/yeoyunsianggeremie\" target=\"_blank\">@yeoyunsianggeremie</a>, <a href=\"https://www.kaggle.com/xyzdivergence\" target=\"_blank\">@xyzdivergence</a>, and <a href=\"https://www.kaggle.com/conjuring92\" target=\"_blank\">@conjuring92</a>. They did most of the hard heavy lifting work.</p>\n<p>The results were released on 4/9, and we got rank 49 — what a coincidence 😭. Although we didn't achieve the leaderboard results we hoped for, it has been incredibly fun working on this competition, so we'd like to share what we learned and enjoyed tinkering with.<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2270821%2F6b78a883ef96aefae690d3bee57d454a%2Fmodel_family_compare_8to3_dpi150.png?generation=1745039112225316&amp;alt=media\" alt=\"Majority voting accuracy (Maj@32) on the uncontaminated AIME 2025 math olympiad of our models compared to the base DeepSeek-R1-Distill models. Token budget for all models is set to 16384 (CoTs are cut at this length).\"></p>\n<h1>Evaluation</h1>\n<p>Let's start with evaluation results first, since most people are too lazy to scroll down 😅. Like many teams, our local CV consists of 40 problems: <a href=\"https://huggingface.co/datasets/opencompass/AIME2025\" target=\"_blank\">30 problems from AIME 2025</a> and <a href=\"https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-2/data?select=reference.csv\" target=\"_blank\">10 problems from AIMO2 reference</a>. For each model, we collect 64 reasoning traces per question for evaluation.</p>\n<h2>14B models</h2>\n<ul>\n<li><strong>Merged-14B:</strong> the model we submitted to the Private LB for both of our final submissions. It's a merge of several SFT trained at 16K context and GRPO on top of SFT with 6K context checkpoints. We uploaded the AWQ on Huggingface: <a href=\"https://huggingface.co/bogoconic1/aimo2-final-merged-model-14b\" target=\"_blank\">bogoconic1/aimo2-final-merged-model-14b</a></li>\n<li><strong>Last-GRPO-14B:</strong> trained in the final days of the competition, 16K context. It showed slightly worse results on our local End-to-End whole pipeline validation, so we never submitted it to LB.</li>\n</ul>\n<table>\n<thead>\n<tr>\n<th>Token budget</th>\n<th>Model Name</th>\n<th>CV Pass@1</th>\n<th>CV Maj@8</th>\n<th>CV Maj@32</th>\n<th>AIME'25 Pass@1</th>\n<th>AIME'25 Maj@8</th>\n<th>AIME'25 Maj@32</th>\n<th>Avg. length</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>12800</td>\n<td>DeepSeek-R1-Distill-14B</td>\n<td>0.412</td>\n<td>0.575</td>\n<td>0.613</td>\n<td>0.41</td>\n<td>0.596</td>\n<td>0.648</td>\n<td>9473.66</td>\n</tr>\n<tr>\n<td></td>\n<td>Light-R1-14B-DS</td>\n<td>0.442</td>\n<td>0.613</td>\n<td>0.664</td>\n<td>0.45</td>\n<td>0.625</td>\n<td>0.671</td>\n<td>9787.01</td>\n</tr>\n<tr>\n<td></td>\n<td><strong>Our Merged 14B</strong></td>\n<td>0.477</td>\n<td><strong>0.686</strong></td>\n<td><strong>0.747</strong></td>\n<td>0.455</td>\n<td>0.673</td>\n<td><strong>0.731</strong></td>\n<td><strong>9251.56</strong></td>\n</tr>\n<tr>\n<td></td>\n<td><strong>Our Last GRPO 14B</strong></td>\n<td><strong>0.482</strong></td>\n<td><strong>0.684</strong></td>\n<td>0.736</td>\n<td><strong>0.468</strong></td>\n<td><strong>0.679</strong></td>\n<td><strong>0.738</strong></td>\n<td>9312.03</td>\n</tr>\n<tr>\n<td>16384</td>\n<td>DeepSeek-R1-Distill-14B</td>\n<td>0.449</td>\n<td>0.614</td>\n<td>0.664</td>\n<td>0.447</td>\n<td>0.625</td>\n<td>0.671</td>\n<td>10910.2</td>\n</tr>\n<tr>\n<td></td>\n<td>Light-R1-14B-DS</td>\n<td>0.498</td>\n<td>0.664</td>\n<td>0.713</td>\n<td>0.502</td>\n<td>0.679</td>\n<td>0.731</td>\n<td>11432.1</td>\n</tr>\n<tr>\n<td></td>\n<td><strong>Our Merged 14B</strong></td>\n<td>0.525</td>\n<td>0.706</td>\n<td>0.759</td>\n<td>0.504</td>\n<td>0.698</td>\n<td>0.746</td>\n<td><strong>10552.1</strong></td>\n</tr>\n<tr>\n<td></td>\n<td><strong>Our Last GRPO 14B</strong></td>\n<td><strong>0.541</strong></td>\n<td><strong>0.717</strong></td>\n<td><strong>0.762</strong></td>\n<td><strong>0.521</strong></td>\n<td><strong>0.704</strong></td>\n<td><strong>0.758</strong></td>\n<td><strong>10511.6</strong></td>\n</tr>\n</tbody>\n</table>\n<p>We built on top of <a href=\"https://huggingface.co/datasets/qihoo360/Light-R1-SFTData\" target=\"_blank\">Light-R1</a> by using a subset of their stage 2 data. We are incredibly grateful to the Light-R1 team for their open-source contributions. This comparison is done only to show how much we were able to improve upon our baselines.</p>\n<h2>7B models</h2>\n<p>We trained our 7B GRPO on top of SFT in 2 stages: first with 8K context, then with 16K context. Our best 7B model is a merge of the SFT and GRPO models, but we never submitted it to the LB as we moved on to 14B models.</p>\n<table>\n<thead>\n<tr>\n<th>Token budget</th>\n<th>Model name</th>\n<th>CV Pass@1</th>\n<th>CV Maj@8</th>\n<th>CV Maj@32</th>\n<th>AIME'25 Pass@1</th>\n<th>AIME'25 Maj@8</th>\n<th>AIME'25 Maj@32</th>\n<th>Avg. length</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>12800</td>\n<td>DeepSeek-R1-Distill-7B</td>\n<td>0.345</td>\n<td>0.502</td>\n<td>0.55</td>\n<td>0.353</td>\n<td>0.512</td>\n<td>0.562</td>\n<td><strong>9553.74</strong></td>\n</tr>\n<tr>\n<td></td>\n<td>Light-R1-7B-DS</td>\n<td>0.36</td>\n<td>0.539</td>\n<td>0.606</td>\n<td>0.371</td>\n<td>0.531</td>\n<td>0.606</td>\n<td>9831.07</td>\n</tr>\n<tr>\n<td></td>\n<td><strong>Our Final SFT 7B</strong></td>\n<td>0.379</td>\n<td>0.569</td>\n<td>0.627</td>\n<td>0.374</td>\n<td>0.523</td>\n<td>0.604</td>\n<td>9751</td>\n</tr>\n<tr>\n<td></td>\n<td><strong>Our Merged 7B</strong></td>\n<td><strong>0.383</strong></td>\n<td><strong>0.575</strong></td>\n<td><strong>0.658</strong></td>\n<td><strong>0.38</strong></td>\n<td><strong>0.558</strong></td>\n<td><strong>0.637</strong></td>\n<td><strong>9583.42</strong></td>\n</tr>\n<tr>\n<td>16384</td>\n<td>DeepSeek-R1-Distill-7B</td>\n<td>0.368</td>\n<td>0.519</td>\n<td>0.58</td>\n<td>0.377</td>\n<td>0.537</td>\n<td>0.583</td>\n<td><strong>11104.1</strong></td>\n</tr>\n<tr>\n<td></td>\n<td>Light-R1-7B-DS</td>\n<td>0.391</td>\n<td>0.564</td>\n<td>0.611</td>\n<td>0.401</td>\n<td>0.565</td>\n<td>0.631</td>\n<td>11511.7</td>\n</tr>\n<tr>\n<td></td>\n<td><strong>Our Final SFT 7B</strong></td>\n<td>0.412</td>\n<td>0.592</td>\n<td>0.678</td>\n<td>0.398</td>\n<td>0.567</td>\n<td><strong>0.658</strong></td>\n<td>11449.6</td>\n</tr>\n<tr>\n<td></td>\n<td><strong>Our Merged 7B</strong></td>\n<td><strong>0.422</strong></td>\n<td><strong>0.619</strong></td>\n<td><strong>0.686</strong></td>\n<td><strong>0.409</strong></td>\n<td><strong>0.575</strong></td>\n<td><strong>0.654</strong></td>\n<td><strong>11145.9</strong></td>\n</tr>\n</tbody>\n</table>\n<h2>Evaluation setting</h2>\n<p>Solutions are generated in <code>BFloat16</code>. Length and <code>Pass@1</code> metrics were averaged across 64 solutions per question. For <code>Maj@K</code> (majority voting), we sampled <code>K</code> traces per problem from our generated solutions. We repeat this process 16 times and reported the average to reduce noise. You can reproduce our evals scorecard using our <a href=\"https://www.kaggle.com/code/chankhavu/aime25-aimo2-evals\" target=\"_blank\">evaluation code</a> and our <a href=\"https://www.kaggle.com/datasets/chankhavu/reasoning-traces-aime25-aimo25\" target=\"_blank\">dataset of collected reasoning traces</a>.</p>\n<h1>Data</h1>\n<ul>\n<li>First, we filtered math word problems from <a href=\"https://huggingface.co/datasets/AI-MO/NuminaMath-1.5\" target=\"_blank\">NuminaMath-1.5</a>, keeping only integer-answer problems from olympiads, AoPS forums, AMC, AIME of previous years, olympiad references, and number theory sources. Our goal was to get harder problems in topics that DeepSeek models would likely struggle with. We then joined with correct R1 reasoning traces from <a href=\"https://huggingface.co/datasets/open-r1/OpenR1-Math-220k\" target=\"_blank\">OpenR1-Math-220k</a>. This filtered 800K problems down to 27K.</li>\n<li>To filter further, we sampled 8 solutions per problem with <code>max_len</code> of 8K tokens using <a href=\"https://huggingface.co/casperhansen/deepseek-r1-distill-qwen-7b-awq\" target=\"_blank\">deepseek-r1-distill-qwen-7b-awq</a> and removed easy problems. We kept only problems with 7 or fewer correct solutions, leaving 8K problems. To get harder problems for GRPO, we used the 14B AWQ model for similar filtering.</li>\n<li>Later, we added a subset of <a href=\"https://huggingface.co/datasets/qihoo360/Light-R1-SFTData\" target=\"_blank\">Light-R1</a> stage 2 data (after filtering out non-integer answers and removing CoT with more than 16K tokens) and removed duplicates with our dataset.</li>\n</ul>\n<h1>Modeling</h1>\n<h2>SFT</h2>\n<p>We used <a href=\"https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B\" target=\"_blank\">DeepSeek-R1-Distill-Qwen-14B</a> and <a href=\"https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B\" target=\"_blank\">DeepSeek-R1-Distill-Qwen-7B</a> as our base models and fine-tuned them for 6 epochs at 16K context length.</p>\n<h2>GRPO</h2>\n<p>Like other teams, we identified generation length as the main problem with SFT models. During the Reinforcement Learning process, we tried to steer our model's behavior toward shorter reasoning CoTs. We applied these improvements to GRPO:</p>\n<ul>\n<li>Removed rewards scaling, following the <a href=\"https://arxiv.org/abs/2503.20783\" target=\"_blank\">Dr. GRPO paper</a>, to eliminate length bias.</li>\n<li>Actor rollouts is the most painful part of GRPO. For quicker convergence, we re-used each batch twice during training.</li>\n</ul>\n<p>We spent considerable time on 7B models, hoping our findings would translate to 14B, only to relearn the bitter lesson repeatedly — good tricks at smaller scales don't always work at larger ones.</p>\n<ul>\n<li>For the 7B model, we performed GRPO in two stages: first on 8K context, then on 16K context. We found LoRA converged faster than FFT while being more VRAM-efficient. We used DAPO's clipping and online sample filtering, and length penalty worked well for 7B.</li>\n<li>For 14B, length penalty severely hurts accuracy, so we removed it. Also, training on much shorter contexts significantly reduced accuracy at intended inference lengths. Model merging helped regain some accuracy, so our final submission is a merged SFT and GRPO on 6K context. As mentioned, we had another 14B GRPO on 16K context trained on the last day.</li>\n</ul>\n<h2>Merging</h2>\n<p>For 7B models, merging improved overall performance: the merged model outperformed both SFT and all GRPO checkpoints in accuracy and token economy. With 14B, merging became more of a trade-off — we gained back some accuracy compared to GRPO but lost some token efficiency.</p>\n<h1>Inference post-mortem</h1>\n<p>We used SGLang since it proved faster than vLLM in our tests. Quantization is a major issue we faced — we lose a lot of our gains during training after quantization. We did not use early stopping of reasoning rollouts in our sub.</p>\n<hr>\n<p>If you've read the whole post, thanks a ton! Anyways, see you all again in AIMO3 this Autumn 👋</p>",
  "comments": [
    {
      "id": 3180995,
      "author_name": "Geremie Yeo",
      "author_url": "",
      "post_date": "2025-04-17T10:57:51.787000",
      "content": "<p>In addition, the team has also open-sourced a tool to perform <strong>batch inference</strong> on math problems (it's in the very early phase of development though!) which integrates certain parts of our inference logic.</p>\n<p>I will continue to enhance this tool in the months leading up to AIMO3! Feel free to give it a try and provide your feedback! The setup steps are on the README itself. You will need to run it on an instance with GPUs, it is really expensive to host it for public access 🤣</p>\n<p><a href=\"https://github.com/bogoconic1/aimo-inference-service\" target=\"_blank\">https://github.com/bogoconic1/aimo-inference-service</a></p>",
      "votes": 4,
      "replies": []
    },
    {
      "id": 3181174,
      "author_name": "Chew Kok Wah",
      "author_url": "",
      "post_date": "2025-04-17T14:35:45.297000",
      "content": "<p>Thanks for sharing. <br>\nIs your final model GPRO also trained with LORA? If no, how big is the different in term of accuracy between your LORA test vs FFT ?</p>\n<p>Btw your evaluation setting is one of the most comprehensive one that was shared so far, did you use different temperature and seed for different run or use the standard recommended  temperature?</p>",
      "votes": 1,
      "replies": [
        {
          "id": 3181206,
          "author_name": "Chan Kha Vu",
          "author_url": "",
          "post_date": "2025-04-17T15:18:55.747000",
          "content": "<blockquote>\n  <p>Is your final model GPRO also trained with LORA? If no, how big is the different in term of accuracy</p>\n</blockquote>\n<p>No, it was trained with FFT. We didn't have a chance to perform proper full ablation, so it's hard to say what makes a training run better and by how much. I think 9th place team mentioned that LoRA worked much worse for them.</p>\n<p>I personally view LoRA more as a regularization form to confine changes to a sub-space, which can be beneficial if all we want to do is just to steer the model's behavior.</p>\n<blockquote>\n  <p>your evaluation setting is one of the most comprehensive one that was shared so far, did you use different temperature and seed for different run or use the standard recommended temperature?</p>\n</blockquote>\n<p>Thanks for the kind words! We use <code>temperature=0.75</code> and <code>top_p=0.95</code> for all models. We don't fix any random seeds. You can actually find our sampling parameters and prompt examples in <code>config.yaml</code> files of each model in our <a href=\"https://www.kaggle.com/datasets/chankhavu/reasoning-traces-aime25-aimo25\" target=\"_blank\">dataset of collected reasoning traces</a>. Will add the inference script to that dataset later for documentation.</p>",
          "votes": 2,
          "replies": []
        }
      ]
    }
  ],
  "comment_count": 3,
  "messages": [
    {
      "id": 3179954,
      "postDate": "2025-04-16T00:51:33.253Z",
      "content": "<p>Congratulations to the winners and huge thanks to the AIMO team for hosting such an intense competition that pushed the frontier of open-source reasoning models. I'd like to express my deep gratitude to the best teammates I could have ever asked for, especially <a href=\"https://www.kaggle.com/yeoyunsianggeremie\" target=\"_blank\">@yeoyunsianggeremie</a>, <a href=\"https://www.kaggle.com/xyzdivergence\" target=\"_blank\">@xyzdivergence</a>, and <a href=\"https://www.kaggle.com/conjuring92\" target=\"_blank\">@conjuring92</a>. They did most of the hard heavy lifting work.</p>\n<p>The results were released on 4/9, and we got rank 49 — what a coincidence 😭. Although we didn't achieve the leaderboard results we hoped for, it has been incredibly fun working on this competition, so we'd like to share what we learned and enjoyed tinkering with.<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2270821%2F6b78a883ef96aefae690d3bee57d454a%2Fmodel_family_compare_8to3_dpi150.png?generation=1745039112225316&amp;alt=media\" alt=\"Majority voting accuracy (Maj@32) on the uncontaminated AIME 2025 math olympiad of our models compared to the base DeepSeek-R1-Distill models. Token budget for all models is set to 16384 (CoTs are cut at this length).\"></p>\n<h1>Evaluation</h1>\n<p>Let's start with evaluation results first, since most people are too lazy to scroll down 😅. Like many teams, our local CV consists of 40 problems: <a href=\"https://huggingface.co/datasets/opencompass/AIME2025\" target=\"_blank\">30 problems from AIME 2025</a> and <a href=\"https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-2/data?select=reference.csv\" target=\"_blank\">10 problems from AIMO2 reference</a>. For each model, we collect 64 reasoning traces per question for evaluation.</p>\n<h2>14B models</h2>\n<ul>\n<li><strong>Merged-14B:</strong> the model we submitted to the Private LB for both of our final submissions. It's a merge of several SFT trained at 16K context and GRPO on top of SFT with 6K context checkpoints. We uploaded the AWQ on Huggingface: <a href=\"https://huggingface.co/bogoconic1/aimo2-final-merged-model-14b\" target=\"_blank\">bogoconic1/aimo2-final-merged-model-14b</a></li>\n<li><strong>Last-GRPO-14B:</strong> trained in the final days of the competition, 16K context. It showed slightly worse results on our local End-to-End whole pipeline validation, so we never submitted it to LB.</li>\n</ul>\n<table>\n<thead>\n<tr>\n<th>Token budget</th>\n<th>Model Name</th>\n<th>CV Pass@1</th>\n<th>CV Maj@8</th>\n<th>CV Maj@32</th>\n<th>AIME'25 Pass@1</th>\n<th>AIME'25 Maj@8</th>\n<th>AIME'25 Maj@32</th>\n<th>Avg. length</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>12800</td>\n<td>DeepSeek-R1-Distill-14B</td>\n<td>0.412</td>\n<td>0.575</td>\n<td>0.613</td>\n<td>0.41</td>\n<td>0.596</td>\n<td>0.648</td>\n<td>9473.66</td>\n</tr>\n<tr>\n<td></td>\n<td>Light-R1-14B-DS</td>\n<td>0.442</td>\n<td>0.613</td>\n<td>0.664</td>\n<td>0.45</td>\n<td>0.625</td>\n<td>0.671</td>\n<td>9787.01</td>\n</tr>\n<tr>\n<td></td>\n<td><strong>Our Merged 14B</strong></td>\n<td>0.477</td>\n<td><strong>0.686</strong></td>\n<td><strong>0.747</strong></td>\n<td>0.455</td>\n<td>0.673</td>\n<td><strong>0.731</strong></td>\n<td><strong>9251.56</strong></td>\n</tr>\n<tr>\n<td></td>\n<td><strong>Our Last GRPO 14B</strong></td>\n<td><strong>0.482</strong></td>\n<td><strong>0.684</strong></td>\n<td>0.736</td>\n<td><strong>0.468</strong></td>\n<td><strong>0.679</strong></td>\n<td><strong>0.738</strong></td>\n<td>9312.03</td>\n</tr>\n<tr>\n<td>16384</td>\n<td>DeepSeek-R1-Distill-14B</td>\n<td>0.449</td>\n<td>0.614</td>\n<td>0.664</td>\n<td>0.447</td>\n<td>0.625</td>\n<td>0.671</td>\n<td>10910.2</td>\n</tr>\n<tr>\n<td></td>\n<td>Light-R1-14B-DS</td>\n<td>0.498</td>\n<td>0.664</td>\n<td>0.713</td>\n<td>0.502</td>\n<td>0.679</td>\n<td>0.731</td>\n<td>11432.1</td>\n</tr>\n<tr>\n<td></td>\n<td><strong>Our Merged 14B</strong></td>\n<td>0.525</td>\n<td>0.706</td>\n<td>0.759</td>\n<td>0.504</td>\n<td>0.698</td>\n<td>0.746</td>\n<td><strong>10552.1</strong></td>\n</tr>\n<tr>\n<td></td>\n<td><strong>Our Last GRPO 14B</strong></td>\n<td><strong>0.541</strong></td>\n<td><strong>0.717</strong></td>\n<td><strong>0.762</strong></td>\n<td><strong>0.521</strong></td>\n<td><strong>0.704</strong></td>\n<td><strong>0.758</strong></td>\n<td><strong>10511.6</strong></td>\n</tr>\n</tbody>\n</table>\n<p>We built on top of <a href=\"https://huggingface.co/datasets/qihoo360/Light-R1-SFTData\" target=\"_blank\">Light-R1</a> by using a subset of their stage 2 data. We are incredibly grateful to the Light-R1 team for their open-source contributions. This comparison is done only to show how much we were able to improve upon our baselines.</p>\n<h2>7B models</h2>\n<p>We trained our 7B GRPO on top of SFT in 2 stages: first with 8K context, then with 16K context. Our best 7B model is a merge of the SFT and GRPO models, but we never submitted it to the LB as we moved on to 14B models.</p>\n<table>\n<thead>\n<tr>\n<th>Token budget</th>\n<th>Model name</th>\n<th>CV Pass@1</th>\n<th>CV Maj@8</th>\n<th>CV Maj@32</th>\n<th>AIME'25 Pass@1</th>\n<th>AIME'25 Maj@8</th>\n<th>AIME'25 Maj@32</th>\n<th>Avg. length</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>12800</td>\n<td>DeepSeek-R1-Distill-7B</td>\n<td>0.345</td>\n<td>0.502</td>\n<td>0.55</td>\n<td>0.353</td>\n<td>0.512</td>\n<td>0.562</td>\n<td><strong>9553.74</strong></td>\n</tr>\n<tr>\n<td></td>\n<td>Light-R1-7B-DS</td>\n<td>0.36</td>\n<td>0.539</td>\n<td>0.606</td>\n<td>0.371</td>\n<td>0.531</td>\n<td>0.606</td>\n<td>9831.07</td>\n</tr>\n<tr>\n<td></td>\n<td><strong>Our Final SFT 7B</strong></td>\n<td>0.379</td>\n<td>0.569</td>\n<td>0.627</td>\n<td>0.374</td>\n<td>0.523</td>\n<td>0.604</td>\n<td>9751</td>\n</tr>\n<tr>\n<td></td>\n<td><strong>Our Merged 7B</strong></td>\n<td><strong>0.383</strong></td>\n<td><strong>0.575</strong></td>\n<td><strong>0.658</strong></td>\n<td><strong>0.38</strong></td>\n<td><strong>0.558</strong></td>\n<td><strong>0.637</strong></td>\n<td><strong>9583.42</strong></td>\n</tr>\n<tr>\n<td>16384</td>\n<td>DeepSeek-R1-Distill-7B</td>\n<td>0.368</td>\n<td>0.519</td>\n<td>0.58</td>\n<td>0.377</td>\n<td>0.537</td>\n<td>0.583</td>\n<td><strong>11104.1</strong></td>\n</tr>\n<tr>\n<td></td>\n<td>Light-R1-7B-DS</td>\n<td>0.391</td>\n<td>0.564</td>\n<td>0.611</td>\n<td>0.401</td>\n<td>0.565</td>\n<td>0.631</td>\n<td>11511.7</td>\n</tr>\n<tr>\n<td></td>\n<td><strong>Our Final SFT 7B</strong></td>\n<td>0.412</td>\n<td>0.592</td>\n<td>0.678</td>\n<td>0.398</td>\n<td>0.567</td>\n<td><strong>0.658</strong></td>\n<td>11449.6</td>\n</tr>\n<tr>\n<td></td>\n<td><strong>Our Merged 7B</strong></td>\n<td><strong>0.422</strong></td>\n<td><strong>0.619</strong></td>\n<td><strong>0.686</strong></td>\n<td><strong>0.409</strong></td>\n<td><strong>0.575</strong></td>\n<td><strong>0.654</strong></td>\n<td><strong>11145.9</strong></td>\n</tr>\n</tbody>\n</table>\n<h2>Evaluation setting</h2>\n<p>Solutions are generated in <code>BFloat16</code>. Length and <code>Pass@1</code> metrics were averaged across 64 solutions per question. For <code>Maj@K</code> (majority voting), we sampled <code>K</code> traces per problem from our generated solutions. We repeat this process 16 times and reported the average to reduce noise. You can reproduce our evals scorecard using our <a href=\"https://www.kaggle.com/code/chankhavu/aime25-aimo2-evals\" target=\"_blank\">evaluation code</a> and our <a href=\"https://www.kaggle.com/datasets/chankhavu/reasoning-traces-aime25-aimo25\" target=\"_blank\">dataset of collected reasoning traces</a>.</p>\n<h1>Data</h1>\n<ul>\n<li>First, we filtered math word problems from <a href=\"https://huggingface.co/datasets/AI-MO/NuminaMath-1.5\" target=\"_blank\">NuminaMath-1.5</a>, keeping only integer-answer problems from olympiads, AoPS forums, AMC, AIME of previous years, olympiad references, and number theory sources. Our goal was to get harder problems in topics that DeepSeek models would likely struggle with. We then joined with correct R1 reasoning traces from <a href=\"https://huggingface.co/datasets/open-r1/OpenR1-Math-220k\" target=\"_blank\">OpenR1-Math-220k</a>. This filtered 800K problems down to 27K.</li>\n<li>To filter further, we sampled 8 solutions per problem with <code>max_len</code> of 8K tokens using <a href=\"https://huggingface.co/casperhansen/deepseek-r1-distill-qwen-7b-awq\" target=\"_blank\">deepseek-r1-distill-qwen-7b-awq</a> and removed easy problems. We kept only problems with 7 or fewer correct solutions, leaving 8K problems. To get harder problems for GRPO, we used the 14B AWQ model for similar filtering.</li>\n<li>Later, we added a subset of <a href=\"https://huggingface.co/datasets/qihoo360/Light-R1-SFTData\" target=\"_blank\">Light-R1</a> stage 2 data (after filtering out non-integer answers and removing CoT with more than 16K tokens) and removed duplicates with our dataset.</li>\n</ul>\n<h1>Modeling</h1>\n<h2>SFT</h2>\n<p>We used <a href=\"https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B\" target=\"_blank\">DeepSeek-R1-Distill-Qwen-14B</a> and <a href=\"https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B\" target=\"_blank\">DeepSeek-R1-Distill-Qwen-7B</a> as our base models and fine-tuned them for 6 epochs at 16K context length.</p>\n<h2>GRPO</h2>\n<p>Like other teams, we identified generation length as the main problem with SFT models. During the Reinforcement Learning process, we tried to steer our model's behavior toward shorter reasoning CoTs. We applied these improvements to GRPO:</p>\n<ul>\n<li>Removed rewards scaling, following the <a href=\"https://arxiv.org/abs/2503.20783\" target=\"_blank\">Dr. GRPO paper</a>, to eliminate length bias.</li>\n<li>Actor rollouts is the most painful part of GRPO. For quicker convergence, we re-used each batch twice during training.</li>\n</ul>\n<p>We spent considerable time on 7B models, hoping our findings would translate to 14B, only to relearn the bitter lesson repeatedly — good tricks at smaller scales don't always work at larger ones.</p>\n<ul>\n<li>For the 7B model, we performed GRPO in two stages: first on 8K context, then on 16K context. We found LoRA converged faster than FFT while being more VRAM-efficient. We used DAPO's clipping and online sample filtering, and length penalty worked well for 7B.</li>\n<li>For 14B, length penalty severely hurts accuracy, so we removed it. Also, training on much shorter contexts significantly reduced accuracy at intended inference lengths. Model merging helped regain some accuracy, so our final submission is a merged SFT and GRPO on 6K context. As mentioned, we had another 14B GRPO on 16K context trained on the last day.</li>\n</ul>\n<h2>Merging</h2>\n<p>For 7B models, merging improved overall performance: the merged model outperformed both SFT and all GRPO checkpoints in accuracy and token economy. With 14B, merging became more of a trade-off — we gained back some accuracy compared to GRPO but lost some token efficiency.</p>\n<h1>Inference post-mortem</h1>\n<p>We used SGLang since it proved faster than vLLM in our tests. Quantization is a major issue we faced — we lose a lot of our gains during training after quantization. We did not use early stopping of reasoning rollouts in our sub.</p>\n<hr>\n<p>If you've read the whole post, thanks a ton! Anyways, see you all again in AIMO3 this Autumn 👋</p>",
      "rawMarkdown": "Congratulations to the winners and huge thanks to the AIMO team for hosting such an intense competition that pushed the frontier of open-source reasoning models. I'd like to express my deep gratitude to the best teammates I could have ever asked for, especially @yeoyunsianggeremie, @xyzdivergence, and @conjuring92. They did most of the hard heavy lifting work.\n\nThe results were released on 4/9, and we got rank 49 — what a coincidence 😭. Although we didn't achieve the leaderboard results we hoped for, it has been incredibly fun working on this competition, so we'd like to share what we learned and enjoyed tinkering with.\n![Majority voting accuracy (Maj@32) on the uncontaminated AIME 2025 math olympiad of our models compared to the base DeepSeek-R1-Distill models. Token budget for all models is set to 16384 (CoTs are cut at this length).](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2270821%2F6b78a883ef96aefae690d3bee57d454a%2Fmodel_family_compare_8to3_dpi150.png?generation=1745039112225316&alt=media)\n\n# Evaluation\n\nLet's start with evaluation results first, since most people are too lazy to scroll down 😅. Like many teams, our local CV consists of 40 problems: [30 problems from AIME 2025](https://huggingface.co/datasets/opencompass/AIME2025) and [10 problems from AIMO2 reference](https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-2/data?select=reference.csv). For each model, we collect 64 reasoning traces per question for evaluation.\n\n## 14B models\n\n- **Merged-14B:** the model we submitted to the Private LB for both of our final submissions. It's a merge of several SFT trained at 16K context and GRPO on top of SFT with 6K context checkpoints. We uploaded the AWQ on Huggingface: [bogoconic1/aimo2-final-merged-model-14b](https://huggingface.co/bogoconic1/aimo2-final-merged-model-14b)\n- **Last-GRPO-14B:** trained in the final days of the competition, 16K context. It showed slightly worse results on our local End-to-End whole pipeline validation, so we never submitted it to LB.\n\n| Token budget | Model Name | CV Pass@1 | CV Maj@8 | CV Maj@32 | AIME'25 Pass@1 | AIME'25 Maj@8 | AIME'25 Maj@32 | Avg. length |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| 12800 | DeepSeek-R1-Distill-14B | 0.412 | 0.575 | 0.613 | 0.41 | 0.596 | 0.648 | 9473.66 |\n|  | Light-R1-14B-DS | 0.442 | 0.613 | 0.664 | 0.45 | 0.625 | 0.671 | 9787.01 |\n|  | **Our Merged 14B** | 0.477 | **0.686** | **0.747** | 0.455 | 0.673 | **0.731** | **9251.56** |\n|  | **Our Last GRPO 14B** | **0.482** | **0.684** | 0.736 | **0.468** | **0.679** | **0.738** | 9312.03 |\n| 16384 | DeepSeek-R1-Distill-14B | 0.449 | 0.614 | 0.664 | 0.447 | 0.625 | 0.671 | 10910.2 |\n|  | Light-R1-14B-DS | 0.498 | 0.664 | 0.713 | 0.502 | 0.679 | 0.731 | 11432.1 |\n|  | **Our Merged 14B** | 0.525 | 0.706 | 0.759 | 0.504 | 0.698 | 0.746 | **10552.1** |\n|  | **Our Last GRPO 14B** | **0.541** | **0.717** | **0.762** | **0.521** | **0.704** | **0.758** | **10511.6** |\n\n\nWe built on top of [Light-R1](https://huggingface.co/datasets/qihoo360/Light-R1-SFTData) by using a subset of their stage 2 data. We are incredibly grateful to the Light-R1 team for their open-source contributions. This comparison is done only to show how much we were able to improve upon our baselines.\n\n## 7B models\n\nWe trained our 7B GRPO on top of SFT in 2 stages: first with 8K context, then with 16K context. Our best 7B model is a merge of the SFT and GRPO models, but we never submitted it to the LB as we moved on to 14B models.\n\n| Token budget | Model name | CV Pass@1 | CV Maj@8 | CV Maj@32 | AIME'25 Pass@1 | AIME'25 Maj@8 | AIME'25 Maj@32 | Avg. length |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| 12800 | DeepSeek-R1-Distill-7B | 0.345 | 0.502 | 0.55 | 0.353 | 0.512 | 0.562 | **9553.74** |\n|  | Light-R1-7B-DS | 0.36 | 0.539 | 0.606 | 0.371 | 0.531 | 0.606 | 9831.07 |\n|  | **Our Final SFT 7B** | 0.379 | 0.569 | 0.627 | 0.374 | 0.523 | 0.604 | 9751 |\n|  | **Our Merged 7B** | **0.383** | **0.575** | **0.658** | **0.38** | **0.558** | **0.637** | **9583.42** |\n| 16384 | DeepSeek-R1-Distill-7B | 0.368 | 0.519 | 0.58 | 0.377 | 0.537 | 0.583 | **11104.1** |\n|  | Light-R1-7B-DS | 0.391 | 0.564 | 0.611 | 0.401 | 0.565 | 0.631 | 11511.7 |\n|  | **Our Final SFT 7B** | 0.412 | 0.592 | 0.678 | 0.398 | 0.567 | **0.658** | 11449.6 |\n|  | **Our Merged 7B** | **0.422** | **0.619** | **0.686** | **0.409** | **0.575** | **0.654** | **11145.9** |\n\n## Evaluation setting\n\nSolutions are generated in `BFloat16`. Length and `Pass@1` metrics were averaged across 64 solutions per question. For `Maj@K` (majority voting), we sampled `K` traces per problem from our generated solutions. We repeat this process 16 times and reported the average to reduce noise. You can reproduce our evals scorecard using our [evaluation code](https://www.kaggle.com/code/chankhavu/aime25-aimo2-evals) and our [dataset of collected reasoning traces](https://www.kaggle.com/datasets/chankhavu/reasoning-traces-aime25-aimo25).\n\n# Data\n\n- First, we filtered math word problems from [NuminaMath-1.5](https://huggingface.co/datasets/AI-MO/NuminaMath-1.5), keeping only integer-answer problems from olympiads, AoPS forums, AMC, AIME of previous years, olympiad references, and number theory sources. Our goal was to get harder problems in topics that DeepSeek models would likely struggle with. We then joined with correct R1 reasoning traces from [OpenR1-Math-220k](https://huggingface.co/datasets/open-r1/OpenR1-Math-220k). This filtered 800K problems down to 27K.\n- To filter further, we sampled 8 solutions per problem with `max_len` of 8K tokens using [deepseek-r1-distill-qwen-7b-awq](https://huggingface.co/casperhansen/deepseek-r1-distill-qwen-7b-awq) and removed easy problems. We kept only problems with 7 or fewer correct solutions, leaving 8K problems. To get harder problems for GRPO, we used the 14B AWQ model for similar filtering.\n- Later, we added a subset of [Light-R1](https://huggingface.co/datasets/qihoo360/Light-R1-SFTData) stage 2 data (after filtering out non-integer answers and removing CoT with more than 16K tokens) and removed duplicates with our dataset.\n\n# Modeling\n\n## SFT\n\nWe used [DeepSeek-R1-Distill-Qwen-14B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B) and [DeepSeek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B) as our base models and fine-tuned them for 6 epochs at 16K context length.\n\n## GRPO\n\nLike other teams, we identified generation length as the main problem with SFT models. During the Reinforcement Learning process, we tried to steer our model's behavior toward shorter reasoning CoTs. We applied these improvements to GRPO:\n\n- Removed rewards scaling, following the [Dr. GRPO paper](https://arxiv.org/abs/2503.20783), to eliminate length bias.\n- Actor rollouts is the most painful part of GRPO. For quicker convergence, we re-used each batch twice during training.\n\nWe spent considerable time on 7B models, hoping our findings would translate to 14B, only to relearn the bitter lesson repeatedly — good tricks at smaller scales don't always work at larger ones.\n\n- For the 7B model, we performed GRPO in two stages: first on 8K context, then on 16K context. We found LoRA converged faster than FFT while being more VRAM-efficient. We used DAPO's clipping and online sample filtering, and length penalty worked well for 7B.\n- For 14B, length penalty severely hurts accuracy, so we removed it. Also, training on much shorter contexts significantly reduced accuracy at intended inference lengths. Model merging helped regain some accuracy, so our final submission is a merged SFT and GRPO on 6K context. As mentioned, we had another 14B GRPO on 16K context trained on the last day.\n\n## Merging\n\nFor 7B models, merging improved overall performance: the merged model outperformed both SFT and all GRPO checkpoints in accuracy and token economy. With 14B, merging became more of a trade-off — we gained back some accuracy compared to GRPO but lost some token efficiency.\n\n# Inference post-mortem\n\nWe used SGLang since it proved faster than vLLM in our tests. Quantization is a major issue we faced — we lose a lot of our gains during training after quantization. We did not use early stopping of reasoning rollouts in our sub.\n\n----------------------------------------------------\n\nIf you've read the whole post, thanks a ton! Anyways, see you all again in AIMO3 this Autumn 👋",
      "votes": 18
    },
    {
      "id": 3180995,
      "postDate": "2025-04-17T10:57:51.787Z",
      "content": "<p>In addition, the team has also open-sourced a tool to perform <strong>batch inference</strong> on math problems (it's in the very early phase of development though!) which integrates certain parts of our inference logic.</p>\n<p>I will continue to enhance this tool in the months leading up to AIMO3! Feel free to give it a try and provide your feedback! The setup steps are on the README itself. You will need to run it on an instance with GPUs, it is really expensive to host it for public access 🤣</p>\n<p><a href=\"https://github.com/bogoconic1/aimo-inference-service\" target=\"_blank\">https://github.com/bogoconic1/aimo-inference-service</a></p>",
      "rawMarkdown": "In addition, the team has also open-sourced a tool to perform **batch inference** on math problems (it's in the very early phase of development though!) which integrates certain parts of our inference logic.\n\nI will continue to enhance this tool in the months leading up to AIMO3! Feel free to give it a try and provide your feedback! The setup steps are on the README itself. You will need to run it on an instance with GPUs, it is really expensive to host it for public access 🤣\n\nhttps://github.com/bogoconic1/aimo-inference-service",
      "votes": 4
    },
    {
      "id": 3181174,
      "postDate": "2025-04-17T14:35:45.297Z",
      "content": "<p>Thanks for sharing. <br>\nIs your final model GPRO also trained with LORA? If no, how big is the different in term of accuracy between your LORA test vs FFT ?</p>\n<p>Btw your evaluation setting is one of the most comprehensive one that was shared so far, did you use different temperature and seed for different run or use the standard recommended  temperature?</p>",
      "rawMarkdown": "Thanks for sharing. \nIs your final model GPRO also trained with LORA? If no, how big is the different in term of accuracy between your LORA test vs FFT ?\n\nBtw your evaluation setting is one of the most comprehensive one that was shared so far, did you use different temperature and seed for different run or use the standard recommended  temperature?",
      "votes": 1,
      "replies": [
        {
          "id": 3181206,
          "postDate": "2025-04-17T15:18:55.747Z",
          "content": "<blockquote>\n  <p>Is your final model GPRO also trained with LORA? If no, how big is the different in term of accuracy</p>\n</blockquote>\n<p>No, it was trained with FFT. We didn't have a chance to perform proper full ablation, so it's hard to say what makes a training run better and by how much. I think 9th place team mentioned that LoRA worked much worse for them.</p>\n<p>I personally view LoRA more as a regularization form to confine changes to a sub-space, which can be beneficial if all we want to do is just to steer the model's behavior.</p>\n<blockquote>\n  <p>your evaluation setting is one of the most comprehensive one that was shared so far, did you use different temperature and seed for different run or use the standard recommended temperature?</p>\n</blockquote>\n<p>Thanks for the kind words! We use <code>temperature=0.75</code> and <code>top_p=0.95</code> for all models. We don't fix any random seeds. You can actually find our sampling parameters and prompt examples in <code>config.yaml</code> files of each model in our <a href=\"https://www.kaggle.com/datasets/chankhavu/reasoning-traces-aime25-aimo25\" target=\"_blank\">dataset of collected reasoning traces</a>. Will add the inference script to that dataset later for documentation.</p>",
          "rawMarkdown": "> Is your final model GPRO also trained with LORA? If no, how big is the different in term of accuracy\n\nNo, it was trained with FFT. We didn't have a chance to perform proper full ablation, so it's hard to say what makes a training run better and by how much. I think 9th place team mentioned that LoRA worked much worse for them.\n\nI personally view LoRA more as a regularization form to confine changes to a sub-space, which can be beneficial if all we want to do is just to steer the model's behavior.\n\n> your evaluation setting is one of the most comprehensive one that was shared so far, did you use different temperature and seed for different run or use the standard recommended temperature?\n\nThanks for the kind words! We use `temperature=0.75` and `top_p=0.95` for all models. We don't fix any random seeds. You can actually find our sampling parameters and prompt examples in `config.yaml` files of each model in our [dataset of collected reasoning traces](https://www.kaggle.com/datasets/chankhavu/reasoning-traces-aime25-aimo25). Will add the inference script to that dataset later for documentation.",
          "votes": 2
        }
      ]
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-20T06:41:16Z",
  "competition_id": 86023,
  "competition_slug": "ai-mathematical-olympiad-progress-prize-2",
  "team_id": 12785107,
  "solution_url": "/competitions/ai-mathematical-olympiad-progress-prize-2/writeups/so-we-cooked-a-model"
}