The AI Mathematical Olympiad - Progress Prize 3 is a challenging competition that encourages participants to develop open-source AI systems capable of solving high-level mathematical reasoning problems. For me, this competition was not only a leaderboard challenge, but also a valuable opportunity to study how modern large language models reason, fail, recover, and improve under different inference strategies.
\nMy solution is built upon a strong public baseline notebook, and my main goal was to test and reflect on related approaches, conduct several experimental attempts, and gradually identify the strategy that I considered the most suitable. In particular, I focused on prompt design, response analysis, answer extraction, and inference-time aggregation, hoping to better understand how small changes in reasoning guidance and sampling behavior can influence the final performance. I hope that sharing my observations and research can provide some value to the community.
\nMy solution is based on the strong public baseline notebook shared by Andreas Bisiadis and Parthenos.
\nThis solution is mainly an inference-time optimization attempt. No additional training or fine-tuning is used. The goal is to make a large open-weight language model solve AIMO3 mathematical problems as reliably as possible under the fixed competition time budget, with final answers in the range [0, 99999].\nThe pipeline at a high level:
\nThe final version favors simplicity and stability. I tested more complex prompt and entropy variants, but the submitted version kept the components that were easier to interpret and more stable in my submissions.
\nMy submission mainly explores two practical questions:
\nOne practical question I explored was how prompt design can help the model avoid common mistakes in mathematical reasoning. In olympiad-style problems, a wrong answer is often not caused by a complete lack of relevant knowledge, but by smaller failures during the reasoning process: choosing a solution path too early, making arithmetic or algebraic mistakes, overlooking constraints, ignoring boundary cases, or failing to verify the final result.
\nInspired by Parthenos's five-step protocol — UNDERSTAND, EXPLORE, PLAN, EXECUTE, and VERIFY — I tested a more complex prompting strategy in some experimental versions. These prompts encouraged the model to follow a more deliberate problem-solving process: first understand the problem, then explore possible strategies, make a plan, execute the solution, and finally verify the result. I also added reminders to check arithmetic, test special cases, consider constraints, and use Python for calculations or verification when appropriate. The goal was not to teach the model new mathematics, but to make its reasoning process less careless and more self-checking.
\nIn practice, longer prompts introduced extra instructions and did not always give a stable advantage. They also consumed more context and sometimes made the reasoning process less direct. Therefore, in the final version, I chose a simpler prompt that focused on the most important constraints: the model should act as a strong olympiad-level solver, the answer must be a non-negative integer in the required range, and the final answer must be placed inside \\boxed{}.
\nsystem_prompt = (\n 'You are a world-class International Mathematical Olympiad (IMO) competitor. '\n 'The final answer must be a non-negative integer between 0 and 99999. '\n 'You must place the final integer answer inside \\\\boxed{}. '\n)\n\ntool_prompt = (\n 'Use this tool to execute Python code. '\n 'The environment is a stateful Jupyter notebook. '\n 'You must use print() to output results.'\n)\n\npreference_prompt = (\n 'You have access to math, numpy and sympy to solve the problem.'\n)\nThis choice reflects a balance between guidance and simplicity. The prompt provides the essential task framing and output constraints, while Python tool usage, multiple independent attempts, answer extraction, and entropy-weighted aggregation handle much of the practical reliability. In this way, prompt design helped reduce common errors mainly by clarifying the task, enforcing the answer format, and encouraging a careful mathematical-solving mindset, without making the final system overly complicated.
\nI should note that this was not a rigorous ablation study of prompt engineering. Rather, it was a practical exploration during the competition, and the final prompt was selected because it was simple, stable, and consistent with the rest of the inference pipeline.
\n| Prompt Strategy | \nMain Design | \nEntropy Strategy Setting | \nNumber of Submissions | \nPublic LB Score Range | \nObservation | \n
|---|---|---|---|---|---|
| Final submitted prompt | \nShort and direct prompt focusing on IMO-level role, valid answer range, and boxed-answer format | \nSame entropy strategy: simple inverse-entropy weighting | \n6 | \n36–40 (Scored 40 twice) | \nMore stable in practice and better aligned with the rest of the inference pipeline | \n
| More structured error-aware prompt | \nExplicitly guides the model through understanding, exploration, planning, execution, and verification | \nSame entropy strategy: simple inverse-entropy weighting | \n6 | \n35–39 | \nHelpful in principle, but did not show a stable advantage in my submissions | \n
Based on these submission results, I finally chose the shorter prompt for the final submission. Although the more structured prompt was designed to reduce common reasoning errors, such as premature conclusions, missed constraints, and insufficient verification, it did not provide a consistently better leaderboard result in my tests. One possible reason is that a longer prompt introduces more instructions for the model to follow, which may consume context, make the reasoning process less direct, and increase sensitivity to sampling randomness. In contrast, the shorter prompt kept only the most important constraints and worked more smoothly with the rest of the system, including Python tool use, multiple independent attempts, answer extraction, and entropy-weighted aggregation.
\nIn addition to the simple inverse-entropy weighting strategy, I also tested a more complex entropy metric inspired by Parthenos. Instead of using only the mean entropy of generated tokens, this variant tried to model several aspects of reasoning confidence: the average uncertainty level, the confidence of later tokens closer to the final answer, the variance of entropy across the reasoning process, the proportion of sustained high-entropy tokens, and the existence of long low-entropy streaks.
\nAlthough the complex entropy metric was more expressive, I finally selected the simpler inverse-entropy weighting strategy for the final submission. The main reason was robustness. The complex metric introduced several hand-designed components and hyperparameters, such as the position decay factor, high-entropy threshold, low-entropy threshold, and the weights of each component. With only a limited number of leaderboard submissions and a small number of attempts per problem, these extra choices were difficult to tune reliably.
\n| Aggregation Strategy | \nSame Prompt Setting | \nMain Idea | \nNumber of Submissions | \nPublic LB Score Range | \nObservation | \n
|---|---|---|---|---|---|
| Simple inverse-entropy weighting | \nYes | \nUse mean token entropy as an uncertainty signal, then weight each valid answer by 1 / entropy | \n6 | \n36–40 (Scored 40 twice) | \nSimple, stable, and easy to interpret | \n
| More complex entropy metric | \nYes | \nCombine mean entropy, position-weighted entropy, entropy variance, high-entropy penalty, and low-entropy streak reward | \n3 | \n36–39 | \nMore expressive, but did not show a stable advantage | \n
In addition, token-level entropy is only an approximate confidence signal. A more complicated transformation of this signal may amplify noise rather than provide better calibration. In my tests, the complex strategy achieved scores in the range of 36–39, while the simpler strategy achieved 36–40. Therefore, I did not observe a sufficiently stable advantage from the complex variant. I chose the simple inverse-entropy method because it was easier to interpret, less dependent on tuning, and more consistent with the practical constraints of the competition.
\nDuring development, I also explored several practical variants that were not selected for the final submission:
\nBecause of the limited number of submissions and other practical constraints, my prompt exploration was not exhaustive. I still believe that better prompt designs could potentially bring meaningful improvements. However, among the variants I actually tested, the alternative prompts did not show a clear performance gain. Therefore, my final choice was based on empirical testing and the actual scores observed in practice.
\n| Item | \nDetail | \n
|---|---|
| GPU | \n1× H100 80GB | \n
| Model | \nGPT-OSS 120B, FP8 KV-cache, up to 81,920 context tokens | \n
| Attempts per problem | \n8 parallel attempts | \n
| Jupyter kernels | \n16 persistent Jupyter kernels | \n
| Avg time per problem | \n~300–400 seconds | \n
| Total runtime (50 problems) | \n~4–5 hours | \n
| Training | \nNone | \n
I would like to sincerely thank Kaggle and the host team for organizing this challenging and inspiring competition. I am also deeply grateful to the outstanding contributors in the community, especially Andreas Bisiadis, parthenos, and many others who shared strong notebooks, thoughtful discussions, and detailed write-ups. Their work provided valuable references for my own testing, reflection, and improvement. I have deep respect for their contributions, and I hope that sharing my observations and research can provide some value to the community.
\nThanks to Parthenos and Andreas Bisiadis for openly sharing the base notebook, to the AIMO competition organizers for hosting the competition, and to the Kaggle community for the valuable discussions and contributions.
", "comments": [], "comment_count": 0, "messages": [ { "id": 3466678, "postDate": "2026-06-04T13:37:54.543Z", "content": "The AI Mathematical Olympiad - Progress Prize 3 is a challenging competition that encourages participants to develop open-source AI systems capable of solving high-level mathematical reasoning problems. For me, this competition was not only a leaderboard challenge, but also a valuable opportunity to study how modern large language models reason, fail, recover, and improve under different inference strategies.
\nMy solution is built upon a strong public baseline notebook, and my main goal was to test and reflect on related approaches, conduct several experimental attempts, and gradually identify the strategy that I considered the most suitable. In particular, I focused on prompt design, response analysis, answer extraction, and inference-time aggregation, hoping to better understand how small changes in reasoning guidance and sampling behavior can influence the final performance. I hope that sharing my observations and research can provide some value to the community.
\nMy solution is based on the strong public baseline notebook shared by Andreas Bisiadis and Parthenos.
\nThis solution is mainly an inference-time optimization attempt. No additional training or fine-tuning is used. The goal is to make a large open-weight language model solve AIMO3 mathematical problems as reliably as possible under the fixed competition time budget, with final answers in the range [0, 99999].\nThe pipeline at a high level:
\nThe final version favors simplicity and stability. I tested more complex prompt and entropy variants, but the submitted version kept the components that were easier to interpret and more stable in my submissions.
\nMy submission mainly explores two practical questions:
\nOne practical question I explored was how prompt design can help the model avoid common mistakes in mathematical reasoning. In olympiad-style problems, a wrong answer is often not caused by a complete lack of relevant knowledge, but by smaller failures during the reasoning process: choosing a solution path too early, making arithmetic or algebraic mistakes, overlooking constraints, ignoring boundary cases, or failing to verify the final result.
\nInspired by Parthenos's five-step protocol — UNDERSTAND, EXPLORE, PLAN, EXECUTE, and VERIFY — I tested a more complex prompting strategy in some experimental versions. These prompts encouraged the model to follow a more deliberate problem-solving process: first understand the problem, then explore possible strategies, make a plan, execute the solution, and finally verify the result. I also added reminders to check arithmetic, test special cases, consider constraints, and use Python for calculations or verification when appropriate. The goal was not to teach the model new mathematics, but to make its reasoning process less careless and more self-checking.
\nIn practice, longer prompts introduced extra instructions and did not always give a stable advantage. They also consumed more context and sometimes made the reasoning process less direct. Therefore, in the final version, I chose a simpler prompt that focused on the most important constraints: the model should act as a strong olympiad-level solver, the answer must be a non-negative integer in the required range, and the final answer must be placed inside \\boxed{}.
\nsystem_prompt = (\n 'You are a world-class International Mathematical Olympiad (IMO) competitor. '\n 'The final answer must be a non-negative integer between 0 and 99999. '\n 'You must place the final integer answer inside \\\\boxed{}. '\n)\n\ntool_prompt = (\n 'Use this tool to execute Python code. '\n 'The environment is a stateful Jupyter notebook. '\n 'You must use print() to output results.'\n)\n\npreference_prompt = (\n 'You have access to math, numpy and sympy to solve the problem.'\n)\nThis choice reflects a balance between guidance and simplicity. The prompt provides the essential task framing and output constraints, while Python tool usage, multiple independent attempts, answer extraction, and entropy-weighted aggregation handle much of the practical reliability. In this way, prompt design helped reduce common errors mainly by clarifying the task, enforcing the answer format, and encouraging a careful mathematical-solving mindset, without making the final system overly complicated.
\nI should note that this was not a rigorous ablation study of prompt engineering. Rather, it was a practical exploration during the competition, and the final prompt was selected because it was simple, stable, and consistent with the rest of the inference pipeline.
\n| Prompt Strategy | \nMain Design | \nEntropy Strategy Setting | \nNumber of Submissions | \nPublic LB Score Range | \nObservation | \n
|---|---|---|---|---|---|
| Final submitted prompt | \nShort and direct prompt focusing on IMO-level role, valid answer range, and boxed-answer format | \nSame entropy strategy: simple inverse-entropy weighting | \n6 | \n36–40 (Scored 40 twice) | \nMore stable in practice and better aligned with the rest of the inference pipeline | \n
| More structured error-aware prompt | \nExplicitly guides the model through understanding, exploration, planning, execution, and verification | \nSame entropy strategy: simple inverse-entropy weighting | \n6 | \n35–39 | \nHelpful in principle, but did not show a stable advantage in my submissions | \n
Based on these submission results, I finally chose the shorter prompt for the final submission. Although the more structured prompt was designed to reduce common reasoning errors, such as premature conclusions, missed constraints, and insufficient verification, it did not provide a consistently better leaderboard result in my tests. One possible reason is that a longer prompt introduces more instructions for the model to follow, which may consume context, make the reasoning process less direct, and increase sensitivity to sampling randomness. In contrast, the shorter prompt kept only the most important constraints and worked more smoothly with the rest of the system, including Python tool use, multiple independent attempts, answer extraction, and entropy-weighted aggregation.
\nIn addition to the simple inverse-entropy weighting strategy, I also tested a more complex entropy metric inspired by Parthenos. Instead of using only the mean entropy of generated tokens, this variant tried to model several aspects of reasoning confidence: the average uncertainty level, the confidence of later tokens closer to the final answer, the variance of entropy across the reasoning process, the proportion of sustained high-entropy tokens, and the existence of long low-entropy streaks.
\nAlthough the complex entropy metric was more expressive, I finally selected the simpler inverse-entropy weighting strategy for the final submission. The main reason was robustness. The complex metric introduced several hand-designed components and hyperparameters, such as the position decay factor, high-entropy threshold, low-entropy threshold, and the weights of each component. With only a limited number of leaderboard submissions and a small number of attempts per problem, these extra choices were difficult to tune reliably.
\n| Aggregation Strategy | \nSame Prompt Setting | \nMain Idea | \nNumber of Submissions | \nPublic LB Score Range | \nObservation | \n
|---|---|---|---|---|---|
| Simple inverse-entropy weighting | \nYes | \nUse mean token entropy as an uncertainty signal, then weight each valid answer by 1 / entropy | \n6 | \n36–40 (Scored 40 twice) | \nSimple, stable, and easy to interpret | \n
| More complex entropy metric | \nYes | \nCombine mean entropy, position-weighted entropy, entropy variance, high-entropy penalty, and low-entropy streak reward | \n3 | \n36–39 | \nMore expressive, but did not show a stable advantage | \n
In addition, token-level entropy is only an approximate confidence signal. A more complicated transformation of this signal may amplify noise rather than provide better calibration. In my tests, the complex strategy achieved scores in the range of 36–39, while the simpler strategy achieved 36–40. Therefore, I did not observe a sufficiently stable advantage from the complex variant. I chose the simple inverse-entropy method because it was easier to interpret, less dependent on tuning, and more consistent with the practical constraints of the competition.
\nDuring development, I also explored several practical variants that were not selected for the final submission:
\nBecause of the limited number of submissions and other practical constraints, my prompt exploration was not exhaustive. I still believe that better prompt designs could potentially bring meaningful improvements. However, among the variants I actually tested, the alternative prompts did not show a clear performance gain. Therefore, my final choice was based on empirical testing and the actual scores observed in practice.
\n| Item | \nDetail | \n
|---|---|
| GPU | \n1× H100 80GB | \n
| Model | \nGPT-OSS 120B, FP8 KV-cache, up to 81,920 context tokens | \n
| Attempts per problem | \n8 parallel attempts | \n
| Jupyter kernels | \n16 persistent Jupyter kernels | \n
| Avg time per problem | \n~300–400 seconds | \n
| Total runtime (50 problems) | \n~4–5 hours | \n
| Training | \nNone | \n
I would like to sincerely thank Kaggle and the host team for organizing this challenging and inspiring competition. I am also deeply grateful to the outstanding contributors in the community, especially Andreas Bisiadis, parthenos, and many others who shared strong notebooks, thoughtful discussions, and detailed write-ups. Their work provided valuable references for my own testing, reflection, and improvement. I have deep respect for their contributions, and I hope that sharing my observations and research can provide some value to the community.
\nThanks to Parthenos and Andreas Bisiadis for openly sharing the base notebook, to the AIMO competition organizers for hosting the competition, and to the Kaggle community for the valuable discussions and contributions.
", "rawMarkdown": "The AI Mathematical Olympiad - Progress Prize 3 is a challenging competition that encourages participants to develop open-source AI systems capable of solving high-level mathematical reasoning problems. For me, this competition was not only a leaderboard challenge, but also a valuable opportunity to study how modern large language models reason, fail, recover, and improve under different inference strategies.\n\nMy solution is built upon a strong public baseline notebook, and my main goal was to test and reflect on related approaches, conduct several experimental attempts, and gradually identify the strategy that I considered the most suitable. In particular, I focused on prompt design, response analysis, answer extraction, and inference-time aggregation, hoping to better understand how small changes in reasoning guidance and sampling behavior can influence the final performance. I hope that sharing my observations and research can provide some value to the community.\n\n# 1.Overview\nMy solution is based on the strong public baseline notebook shared by Andreas Bisiadis and Parthenos.\n\nThis solution is mainly an inference-time optimization attempt. No additional training or fine-tuning is used. The goal is to make a large open-weight language model solve AIMO3 mathematical problems as reliably as possible under the fixed competition time budget, with final answers in the range [0, 99999].\nThe pipeline at a high level:\n- Use GPT-OSS 120B as the main reasoning model\n- Use a concise IMO-style prompt with strict boxed-answer format\n- For each problem, run up to 8 independent reasoning attempts\n- Allow the model to use Python tools for calculation and verification\n- Extract valid integer answers from \\boxed{}\n- Aggregate answers with simple inverse-entropy weighting\n- Use frequency-based early stopping to save inference time\n- Control the per-problem budget under the global notebook time limit\n\nThe final version favors simplicity and stability. I tested more complex prompt and entropy variants, but the submitted version kept the components that were easier to interpret and more stable in my submissions.\n\n# 2. Details of the Submission\n## 2.1 Main Focus\nMy submission mainly explores two practical questions:\n- How does prompt complexity affect mathematical reasoning performance, and is a highly structured prompt necessarily better than a concise task-focused prompt?\n- How should multiple generated answers be aggregated under the competition time limit, especially when comparing simple entropy-weighted aggregation with more complex entropy-based strategies?\n\n### 2.1.1 Prompt Complexity vs. Stability\nOne practical question I explored was how prompt design can help the model avoid common mistakes in mathematical reasoning. In olympiad-style problems, a wrong answer is often not caused by a complete lack of relevant knowledge, but by smaller failures during the reasoning process: choosing a solution path too early, making arithmetic or algebraic mistakes, overlooking constraints, ignoring boundary cases, or failing to verify the final result.\n\nInspired by Parthenos's five-step protocol — UNDERSTAND, EXPLORE, PLAN, EXECUTE, and VERIFY — I tested a more complex prompting strategy in some experimental versions. These prompts encouraged the model to follow a more deliberate problem-solving process: first understand the problem, then explore possible strategies, make a plan, execute the solution, and finally verify the result. I also added reminders to check arithmetic, test special cases, consider constraints, and use Python for calculations or verification when appropriate. The goal was not to teach the model new mathematics, but to make its reasoning process less careless and more self-checking.\n\nIn practice, longer prompts introduced extra instructions and did not always give a stable advantage. They also consumed more context and sometimes made the reasoning process less direct. Therefore, in the final version, I chose a simpler prompt that focused on the most important constraints: the model should act as a strong olympiad-level solver, the answer must be a non-negative integer in the required range, and the final answer must be placed inside \\boxed{}.\n\n```python\nsystem_prompt = (\n 'You are a world-class International Mathematical Olympiad (IMO) competitor. '\n 'The final answer must be a non-negative integer between 0 and 99999. '\n 'You must place the final integer answer inside \\\\boxed{}. '\n)\n\ntool_prompt = (\n 'Use this tool to execute Python code. '\n 'The environment is a stateful Jupyter notebook. '\n 'You must use print() to output results.'\n)\n\npreference_prompt = (\n 'You have access to math, numpy and sympy to solve the problem.'\n)\n```\n\nThis choice reflects a balance between guidance and simplicity. The prompt provides the essential task framing and output constraints, while Python tool usage, multiple independent attempts, answer extraction, and entropy-weighted aggregation handle much of the practical reliability. In this way, prompt design helped reduce common errors mainly by clarifying the task, enforcing the answer format, and encouraging a careful mathematical-solving mindset, without making the final system overly complicated.\n\nI should note that this was not a rigorous ablation study of prompt engineering. Rather, it was a practical exploration during the competition, and the final prompt was selected because it was simple, stable, and consistent with the rest of the inference pipeline.\n\n| Prompt Strategy | Main Design | Entropy Strategy Setting | Number of Submissions | Public LB Score Range | Observation |\n|---|---|---|---:|---|---|\n| Final submitted prompt | Short and direct prompt focusing on IMO-level role, valid answer range, and boxed-answer format | Same entropy strategy: simple inverse-entropy weighting | 6 | 36–40