{ "id": 573086, "title": "11th place solution", "content": "

I’m grateful to Kaggle and the competition organizers for hosting this challenge. I’d also like to extend my thanks to the participants who shared insightful discussions and valuable datasets.

\n

Summary

\n

Using deepseek-r1-distill-qwen-14B-awq-casperhansen with SC-TIR.
\nBy modifying vllm, the inference can end early when Self-Consistency (SC) converges or diverges.
\nEvaluation is done using the most recent 100 AIME problems. No machine resources were available to do GRPO.

\n

Strategy

\n

1. Model

\n\n

2. Self-Consistency

\n\n

3. Custom vllm

\n\n

\"\"

\n\n

4. Validation dataset

\n\n

5. one-shot

\n\n
You are a Python code assistant. \n\nYou will be  a mathematical problem  has  solutions.\nYour task   convert this complex math problem  Python code.\nLet's have Python do  tedious calculations  us!\n\nThere are multiple ways  solve this problem, so find  most efficient one.\n\n- The final answer should be an .\n- The final answer should be modulo \n- Please  Python code only following  format .\n\n\\`\\`\\`python\nimport math \n\n...\n\n...\n\nprint(<answer> % )\n\\`\\`\\`\n\nHere  an example.\n\nUser:\nFind    ordered pairs $(m, n)$ such  $m$  $n$ are positive integers    $\\{, , ..., \\}$   greatest common divisor  $^m + $  $^n - $   $$ .\n\nAssistant:\n\\`\\`\\`python\nimport math\n\nanswer = \n\n\n m  range(, ):\n     n  range(, ):\n        \n         math.gcd(**m + , **n - ) != :\n            answer += \n\n\nprint(answer % )\n\\`\\`\\`\n
\n

Public LB: 29
\nPrivate LB: 28
\nInference Notebook: https://www.kaggle.com/code/farsail/aimo2-inference
\nCustom vllm: https://www.kaggle.com/code/farsail/pip-install-aimo2-custom-vllm

", "comments": [ { "id": 3178028, "author_name": "xz", "author_url": "", "post_date": "2025-04-13T16:49:04.263000", "content": "

I am not seeing python code extraction and execution from the inference notebook. Am I missing something?

", "votes": 1, "replies": [ { "id": 3178305, "author_name": "wang", "author_url": "", "post_date": "2025-04-14T02:52:41.223000", "content": "

Thank you for your question. I forgot to include the custom vllm. here it is.

", "votes": 2, "replies": [ { "id": 3178309, "author_name": "", "author_url": "", "post_date": "2025-04-14T02:57:57.767000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 3178310, "author_name": "wang", "author_url": "", "post_date": "2025-04-14T02:58:22.910000", "content": "

I mainly made improvements to the def _run_engine function.

", "votes": 1, "replies": [] } ] }, { "id": 3178153, "author_name": "", "author_url": "", "post_date": "2025-04-13T19:33:50.780000", "content": "", "votes": 0, "replies": [] } ], "comment_count": 5, "messages": [ { "id": 3177894, "postDate": "2025-04-13T12:17:55.397Z", "content": "

I’m grateful to Kaggle and the competition organizers for hosting this challenge. I’d also like to extend my thanks to the participants who shared insightful discussions and valuable datasets.

\n

Summary

\n

Using deepseek-r1-distill-qwen-14B-awq-casperhansen with SC-TIR.
\nBy modifying vllm, the inference can end early when Self-Consistency (SC) converges or diverges.
\nEvaluation is done using the most recent 100 AIME problems. No machine resources were available to do GRPO.

\n

Strategy

\n

1. Model

\n\n

2. Self-Consistency

\n\n

3. Custom vllm

\n\n

\"\"

\n\n

4. Validation dataset

\n\n

5. one-shot

\n\n
You are a Python code assistant. \n\nYou will be  a mathematical problem  has  solutions.\nYour task   convert this complex math problem  Python code.\nLet's have Python do  tedious calculations  us!\n\nThere are multiple ways  solve this problem, so find  most efficient one.\n\n- The final answer should be an .\n- The final answer should be modulo \n- Please  Python code only following  format .\n\n\\`\\`\\`python\nimport math \n\n...\n\n...\n\nprint(<answer> % )\n\\`\\`\\`\n\nHere  an example.\n\nUser:\nFind    ordered pairs $(m, n)$ such  $m$  $n$ are positive integers    $\\{, , ..., \\}$   greatest common divisor  $^m + $  $^n - $   $$ .\n\nAssistant:\n\\`\\`\\`python\nimport math\n\nanswer = \n\n\n m  range(, ):\n     n  range(, ):\n        \n         math.gcd(**m + , **n - ) != :\n            answer += \n\n\nprint(answer % )\n\\`\\`\\`\n
\n

Public LB: 29
\nPrivate LB: 28
\nInference Notebook: https://www.kaggle.com/code/farsail/aimo2-inference
\nCustom vllm: https://www.kaggle.com/code/farsail/pip-install-aimo2-custom-vllm

", "rawMarkdown": "I’m grateful to Kaggle and the competition organizers for hosting this challenge. I’d also like to extend my thanks to the participants who shared insightful discussions and valuable datasets.\n\n\n# Summary\n\nUsing [**deepseek-r1-distill-qwen-14B-awq-casperhansen**](https://www.kaggle.com/models/huikang/deepseek-r1/Transformers/deepseek-r1-distill-qwen-14b-awq-casperhansen/1) with **SC-TIR**.\nBy **modifying vllm**, the inference can end early when Self-Consistency (SC) converges or diverges. \nEvaluation is done using the **most recent 100 AIME problems**. No machine resources were available to do GRPO. \n\n\n\n# Strategy\n\n### 1. Model\n\n- Adopted **deepseek-r1-distill-qwen-14B-awq-casperhansen** \n- Compared 7B, 14B, and 32B:\n - 32B is too slow and not perfect.\n - 14B > 7B (in terms of accuracy), 14B < 7B (in terms of speed) \n - Increasing the number of generations for 14B made it more competitive\n\n### 2. Self-Consistency\n\n- Its effectiveness was already demonstrated in AIMO1. \n- It was also a standard approach in AIMO2.\n\n### 3. Custom vllm\n\n- Employed 14B and modified **vllm** to allow as many generates as possible.\n- For easy problems → outputs don’t vary much, so there’s no need to generate up to `max_num_seqs`. \n- For overly difficult problems → the output often ends up being 0 and -1 or diverging (no unique solution). Even generating the full `max_num_seqs` won't lead to a correct answer in many such cases. \n- As a result, the number of problems that actually require the full `max_num_seqs` length to determine the outcome is surprisingly small.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16562148%2F2e93dcdb0b0c106ea78145d8054ee7e2%2F2025-04-13%2020.31.32.png?generation=1744546636576459&alt=media)\n\n- **There are too many -1 and 0** \n - If there are a certain number of -1 or 0 (invalid responses), it ends inference early. \n - 5 ~ 7 responses → terminate if 5 or more are -1/0 \n - 8 ~ 11 responses → terminate if 4 or more are -1/0 \n - 12 ~ 16 responses → terminate if 6 or more are -1/0\n\n- **There responses have diverged** \n - Once there are at least 8 responses, if there are 6 or more different valid responses, I consider it too scattered and terminate.\n\n- **The winner has been decided** \n - If the difference in counts between the most frequent answer and the second-most frequent answer exceeds a certain threshold, the winner is considered determined, and inference stops. \n - 4 ~ 7 responses → difference ≥ 3 \n - 8 ~ 11 responses → difference ≥ 2 \n - 12 ~ 16 responses → difference ≥ 1 \n\n### 4. Validation dataset\n\n- Used the 100 most recent AIME problems.\n- Because the score varies, I tested several times and took the average for pipeline evaluation.\n- The [public notebook](https://www.kaggle.com/code/huikang/thought-engineering-r1-distill-qwen-7b-awq) score was about 63–68, while my pipeline scored around 75–79.\n\n### 5. one-shot\n\n- Even problems that are mathematically difficult to solve might still be solved by a **TIR** exhaustive search.\n- I included the **exhaustive search approach** in the few-shot prompt.\n- Prompts such as `Let's have Python do the tedious calculations for us!` and \n `There are multiple ways to solve this problem, so find the most efficient one.` \n contributed slightly to improved scores.\n\n```\nYou are a Python code assistant. \n\nYou will be given a mathematical problem that has integer solutions.\nYour task is to convert this complex math problem into Python code.\nLet's have Python do the tedious calculations for us!\n\nThere are multiple ways to solve this problem, so find the most efficient one.\n\n- The final answer should be an integer.\n- The final answer should be modulo 1000.\n- Please return Python code only following the format below.\n\n\\`\\`\\`python\nimport math \n\n...\n# Intermediate calculations\n...\n\nprint( % 1000)\n\\`\\`\\`\n\nHere is an example.\n\nUser:\nFind the number of ordered pairs $(m, n)$ such that $m$ and $n$ are positive integers in the set $\\{1, 2, ..., 30\\}$ and the greatest common divisor of $2^m + 1$ and $2^n - 1$ is not $1$ .\n\nAssistant:\n\\`\\`\\`python\nimport math\n\nanswer = 0\n\n# Check all m, n in [1, 30]\nfor m in range(1, 31):\n for n in range(1, 31):\n # Compute GCD of (2^m + 1) and (2^n - 1)\n if math.gcd(2**m + 1, 2**n - 1) != 1:\n answer += 1\n\n# Print the result modulo 1000\nprint(answer % 1000)\n\\`\\`\\`\n```\n\n\n\n\nPublic LB: 29\nPrivate LB: 28\nInference Notebook: https://www.kaggle.com/code/farsail/aimo2-inference\nCustom vllm: https://www.kaggle.com/code/farsail/pip-install-aimo2-custom-vllm", "votes": 19 }, { "id": 3178028, "postDate": "2025-04-13T16:49:04.263Z", "content": "

I am not seeing python code extraction and execution from the inference notebook. Am I missing something?

", "rawMarkdown": "I am not seeing python code extraction and execution from the inference notebook. Am I missing something?", "votes": 1, "replies": [ { "id": 3178305, "postDate": "2025-04-14T02:52:41.223Z", "content": "

Thank you for your question. I forgot to include the custom vllm. here it is.

", "rawMarkdown": "Thank you for your question. I forgot to include the custom vllm. [here](https://www.kaggle.com/code/farsail/pip-install-aimo2-custom-vllm) it is.", "votes": 2, "replies": [ { "id": 3178309, "postDate": "2025-04-14T02:57:57.767Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 3178310, "postDate": "2025-04-14T02:58:22.910Z", "content": "

I mainly made improvements to the def _run_engine function.

", "rawMarkdown": "I mainly made improvements to the `def _run_engine` function.", "votes": 1 } ] }, { "id": 3178153, "postDate": "2025-04-13T19:33:50.780Z", "rawMarkdown": "", "isDeleted": true } ], "complete": true, "source": "live", "fetched_at": "2026-10-20T06:41:52Z", "competition_id": 86023, "competition_slug": "ai-mathematical-olympiad-progress-prize-2", "team_id": 12867124, "solution_url": "/competitions/ai-mathematical-olympiad-progress-prize-2/writeups/farsail-11th-place-solution" }