{ "id": 575172, "title": "20th place solution - generate lots of code with a 32b model", "content": "

I see that all 10 published so far higher ranked solutions are using 14b models, and (maybe except for the top solution) they seem to be focused on generating reasoning paths long enough so that they will end with generating a text (boxed) answer.

\n

My solution is a little different. I was using a 32b model and I gave up on trying to generate sufficient amount of tokens to obtain a boxed answer.
\nInstead, at 5k-6k tokens I force the model to generate code multiple times. In total, up to 108 programs in Python per task are generated.
\nThe final answer is chosen not by majority voting, but by a weighting algorithm with carefully chosen weights.

\n

First solution (version 118 of the notebook)
\nprivate score 27, public score 26

\n

Second solution (version 137 of the notebook)
\nprivate score 25, public score 25

\n

How to run it locally:

\n
    \n
  1. Copy the set of questions from the cell 2, without the first line, to the appropriate file (dq_data5e.py or dq_data4h.py)
    2. \n
  2. Copy the code from the cell 3 (begins with T = True) to the file solve.py
    3. \n
  3. Change to:
    \nsubmit = F
    \nconf_pc = T
    4. \n
  4. Change llm_path to the location of the model
    5. \n
  5. Either set prior_time_per_task to the average time per task from previous runs, or set use_control = F
    6. \n
  6. Run with the command:
    \npython3 solve.py save_dir=1 save_prefix=A runs=1 gpus=0,1
    \nChange save_prefix for every run.
    7. \n
\n

How it works:

\n

Quick summary:

\n\n

Additionally:

\n\n

In parallel to generating the code I continue generating the reasoning path that sometimes will give a good boxed answer.
\nI structure launching prompts in a way that all unfinished reasoning paths and the last wave of unfinished code chunks end at the same token number, so that the tree of generated tokens is cut at a fixed depth.
\nAll that is fast, because vLLM can reuse KV cache entries for prompts with the same prefix.

\n

The program automatically uses less time for easier tasks, because if the LLM returns a boxed answer early, then the whole subtree will be generated sooner, and that will speed up generating the remaining subtrees.

\n

I used a model from a source popular in this contest, without fine-tuning:
\nhttps://huggingface.co/casperhansen/deepseek-r1-distill-qwen-32b-awq
\nThis is Deepseek R1 distilled to the Qwen 2.5 architecture with 32 billion weights, with 4-bit AWQ.

\n

Obvious possible improvements:

\n", "comments": [], "comment_count": 0, "messages": [ { "id": 3187811, "postDate": "2025-04-26T15:08:43.387Z", "content": "

I see that all 10 published so far higher ranked solutions are using 14b models, and (maybe except for the top solution) they seem to be focused on generating reasoning paths long enough so that they will end with generating a text (boxed) answer.

\n

My solution is a little different. I was using a 32b model and I gave up on trying to generate sufficient amount of tokens to obtain a boxed answer.
\nInstead, at 5k-6k tokens I force the model to generate code multiple times. In total, up to 108 programs in Python per task are generated.
\nThe final answer is chosen not by majority voting, but by a weighting algorithm with carefully chosen weights.

\n

First solution (version 118 of the notebook)
\nprivate score 27, public score 26

\n

Second solution (version 137 of the notebook)
\nprivate score 25, public score 25

\n

How to run it locally:

\n
    \n
  1. Copy the set of questions from the cell 2, without the first line, to the appropriate file (dq_data5e.py or dq_data4h.py)
    2. \n
  2. Copy the code from the cell 3 (begins with T = True) to the file solve.py
    3. \n
  3. Change to:
    \nsubmit = F
    \nconf_pc = T
    4. \n
  4. Change llm_path to the location of the model
    5. \n
  5. Either set prior_time_per_task to the average time per task from previous runs, or set use_control = F
    6. \n
  6. Run with the command:
    \npython3 solve.py save_dir=1 save_prefix=A runs=1 gpus=0,1
    \nChange save_prefix for every run.
    7. \n
\n

How it works:

\n

Quick summary:

\n\n

Additionally:

\n\n

In parallel to generating the code I continue generating the reasoning path that sometimes will give a good boxed answer.
\nI structure launching prompts in a way that all unfinished reasoning paths and the last wave of unfinished code chunks end at the same token number, so that the tree of generated tokens is cut at a fixed depth.
\nAll that is fast, because vLLM can reuse KV cache entries for prompts with the same prefix.

\n

The program automatically uses less time for easier tasks, because if the LLM returns a boxed answer early, then the whole subtree will be generated sooner, and that will speed up generating the remaining subtrees.

\n

I used a model from a source popular in this contest, without fine-tuning:
\nhttps://huggingface.co/casperhansen/deepseek-r1-distill-qwen-32b-awq
\nThis is Deepseek R1 distilled to the Qwen 2.5 architecture with 32 billion weights, with 4-bit AWQ.

\n

Obvious possible improvements:

\n", "rawMarkdown": "I see that all 10 published so far higher ranked solutions are using 14b models, and (maybe except for the top solution) they seem to be focused on generating reasoning paths long enough so that they will end with generating a text (boxed) answer.\n\nMy solution is a little different. I was using a 32b model and I gave up on trying to generate sufficient amount of tokens to obtain a boxed answer.\nInstead, at 5k-6k tokens I force the model to generate code multiple times. In total, up to 108 programs in Python per task are generated.\nThe final answer is chosen not by majority voting, but by a weighting algorithm with carefully chosen weights.\n\n[First solution](https://www.kaggle.com/code/arkadiuszpaterek/aimo-2-new-solution-10?scriptVersionId=230095668) (version 118 of the notebook)\nprivate score 27, public score 26\n\n[Second solution](https://www.kaggle.com/code/arkadiuszpaterek/aimo-2-new-solution-10?scriptVersionId=230875528) (version 137 of the notebook)\nprivate score 25, public score 25\n\nHow to run it locally:\n1. Copy the set of questions from the cell 2, without the first line, to the appropriate file (dq_data5e.py or dq_data4h.py)\n2. Copy the code from the cell 3 (begins with T = True) to the file solve.py\n3. Change to:\nsubmit = F\nconf_pc = T\n4. Change llm_path to the location of the model\n5. Either set prior_time_per_task to the average time per task from previous runs, or set use_control = F\n6. Run with the command:\npython3 solve.py save_dir=1 save_prefix=A runs=1 gpus=0,1\nChange save_prefix for every run.\n\nHow it works:\n\nQuick summary:\n* 32b model served with vLLM\n* begin with initial 7 prompts\n* generate up to 108 programs per task, and up to 9 boxed answers\n* end generating after around 7k tokens\n* balance the importance of boxed and code answers\n - shrink the weights (decrease importance) of groups of code answers that have long common prefix\n - large bonus for identical boxed and code answer \n - penalty for short code\n - penalty for small value of the answer\n\nAdditionally:\n* temperature decreases from 0.7 to 0.2, and code is generated with temperature 0.15\n* some prompts rewrite the question before forcing generating the code\n* some initial prompts ask the LLM to first rewrite the question in mathematical notation\n* async-await is utilized to run code with 12s timeout, simultaneously with generating text by the LLM\n* adaptive control chooses the max number of tokens to generate, depending on time remaining and on a regularized estimate of time used per task\n\nIn parallel to generating the code I continue generating the reasoning path that sometimes will give a good boxed answer.\nI structure launching prompts in a way that all unfinished reasoning paths and the last wave of unfinished code chunks end at the same token number, so that the tree of generated tokens is cut at a fixed depth.\nAll that is fast, because vLLM can reuse KV cache entries for prompts with the same prefix.\n\nThe program automatically uses less time for easier tasks, because if the LLM returns a boxed answer early, then the whole subtree will be generated sooner, and that will speed up generating the remaining subtrees.\n\nI used a model from a source popular in this contest, without fine-tuning:\nhttps://huggingface.co/casperhansen/deepseek-r1-distill-qwen-32b-awq\nThis is Deepseek R1 distilled to the Qwen 2.5 architecture with 32 billion weights, with 4-bit AWQ.\n\nObvious possible improvements:\n* Use a better model. The code works with models of any size, just the constant prior_time_per_task needs to be adjusted.\n* As of today, LLM inference engines don't fully utilize multiple GPUs, and particuarly when using speculative decoding with a smaller draft model, both models quantized to 4-bit. More than 2x faster token generation should be possible here.", "votes": 9 } ], "complete": true, "source": "live", "fetched_at": "2026-10-20T06:41:02Z", "competition_id": 86023, "competition_slug": "ai-mathematical-olympiad-progress-prize-2", "team_id": 12783352, "solution_url": "/competitions/ai-mathematical-olympiad-progress-prize-2/writeups/arek-paterek-20th-place-solution-generate-lots-of-" }