I participated in this competition primarily for my own learning and study. My submission is built upon a well-known public baseline notebook, and my primary goal was to improve the prompt design using an original, structured approach.
\nCompared to other outstanding notebooks and comprehensive WriteUPs, I truly realize how much I still have to learn. However, I worked hard to analyze the competition's dynamics, design my own experimental framework, and iterate on improvements, so I would love to share my findings with the community.
\nI plan to add some models and documentation in the coming days.\nAs I believe the presentation slides are much easier to read, I would highly appreciate it if you could read the attached presentation file.
\nBelow is a summary of the key points.
\nMy solution is based on the strong public baseline notebook shared by \"ZaynYu\".
\ngpt-oss-120b as-is without any additional fine-tuning or parameter training. The model is empowered with a Python code execution tool.To improve the prompt without overfitting to the Public Leaderboard (LB) or the reference.csv set, I established an offline validation process:
\nllm_history.csv). I then leveraged more capable LLMs to cross-check and categorize why gpt-oss-120b failed on incorrect instances.The main improvement was in the prompt design. Specifically:
\nI used an LLM that was more capable than gpt-oss-120b, because I expected it to analyze the responses \nmore accurately.However, a stronger LLM can also suggest solution methods that are too advanced for gpt-oss-120b to \nfollow.To avoid this, I also provided examples of correct responses generated by gpt-oss-120b.This helped me search for \neffective solution patterns that were still feasible for gpt-oss-120b to execute.
\nTo reduce the risk of overfitting, I used a separate set of 50 problems outside the official competition data while improving the solution.\nThe goal was to avoid relying too heavily on the public leaderboard or the reference problems.
\nWhen I started this experimental workflow, the public notebooks had already reached a very high score around 40/50. While it was impossible to know the exact individual accuracy for each of these 50 problems on the leaderboard, I envisioned the situation as follows:\n[Hypothesis]
\nUnder these circumstances, if I relied solely on the public leaderboard score to evaluate whether my model was improving, I felt I would end up overfitting to just those 10 unstable problems. Given that the score calculations are heavily influenced by the inherent randomness of LLM text generation and problem ordering, I wanted to avoid placing too much emphasis on this metric.
\nSince the final private evaluation would evaluate the models on completely unseen problems, my goal was to implement enhancements that targeted all branches of mathematics and various dimensions of difficulty. Using only 10 reference problems felt far too restrictive.
\nFurthermore, I reasoned that analyzing the full-text content of the responses would allow me to discover ways to make the outputs even more stable for problems the model was already answering correctly. I believed this qualitative evaluation would alleviate the risk of overfitting.
\n(I would like to express my utmost respect to the competitors who pushed the scores to such an incredibly high level around 40/50. Needless to say, the leaderboard score was a critically important metric for guiding the community's progress up to that point.)
\nDuring the development cycle, I explored several ideas that ultimately did not yield improvements and were discarded:
\n system_prompt = (\n 'You are a world-class mathematical problem solver.'\n 'The final answer is a non-negative integer in [0, 99999].'\n 'Return only the verified final answer in \\boxed{}.'\n 'Before solving, you must:'\n '1) extract all explicit constraints from the problem,'\n '2) identify the mathematical structure (algebra / number theory / combinatorics / geometry / functional equation),'\n '3) choose a solution plan,'\n '4) use Python for verification when symbolic reasoning is insufficient.'\n 'You must never guess.'\n 'If a candidate answer is found, verify it against all constraints before finalizing.'\n 'Prefer concise reasoning and avoid long unnecessary explanations.'\n )\n\n tool_prompt = (\n 'Use Python as a verification and search tool.'\n 'The environment is a stateful Jupyter notebook. '\n 'You must use print() to output results.'\n 'Always:'\n '- print intermediate key values needed for verification,'\n '- validate candidate answers against all stated constraints,'\n '- avoid brute force unless the search space is explicitly bounded or pruned,'\n '- estimate complexity before large loops,'\n '- keep code minimal and robust.'\n )\n\n preference_prompt = ( \n 'Solve the problem with the following protocol:'\n 'Step 1: Extract constraints and target form.'\n 'Step 2: Derive reductions before coding (parity, modular constraints, bounds, symmetry, invariants, monotonicity, geometric relations, etc.).'\n 'Step 3: Use Python only for:'\n '- checking candidate formulas,'\n '- bounded search after pruning,'\n '- validating edge cases,'\n '- confirming the final answer.'\n 'Step 4: If using search, explain the pruning rule first.'\n 'Step 5: Before final output, verify:'\n '- integer,'\n '- in [0, 99999],'\n '- all constraints satisfied.'\n 'Step 6: Return only the final verified answer in \\boxed{}.'\n 'Keep reasoning compact. Never guess. If verification fails, discard the candidate and continue.'\n )\nNice write-up. The structured error analysis and constraint-focused prompt design were particularly interesting. Clear and practical approach.
", "votes": -1, "replies": [] }, { "id": 3463420, "author_name": "Wasiq Ali", "author_url": "", "post_date": "2026-05-26T21:04:06.793000", "content": "The structure of this writeup is a step_by_step procedure which is best for learners
", "votes": -1, "replies": [] }, { "id": 3463276, "author_name": "Fff343", "author_url": "", "post_date": "2026-05-26T13:42:32.053000", "content": "Thanks for sharing. I didn't understand what u changes other than prompt
", "votes": 0, "replies": [ { "id": 3463613, "author_name": "SKobayak", "author_url": "", "post_date": "2026-05-27T12:47:13.723000", "content": "In the final submission notebook, I only changed the prompt. There are no other changes.
\nBesides the prompt, I also experimented with parallel computation methods and voting systems, but I did not include them in the submission because the score appeared to decrease.
", "votes": 1, "replies": [ { "id": 3463616, "author_name": "Mohammad Shadab Alam", "author_url": "", "post_date": "2026-05-27T13:04:17.280000", "content": "Hello @skobayak did you manage to observe the variance in the winning submission on the public LB ?
", "votes": 0, "replies": [] }, { "id": 3464485, "author_name": "SKobayak", "author_url": "", "post_date": "2026-05-30T00:41:55.650000", "content": "I submitted my final submission notebook three times, and the public LB scores were 36, 37, and 39.
\n(If I am not mistaken, as the submission page display is a bit confusing…)
", "votes": 0, "replies": [] } ] } ] }, { "id": 3462748, "author_name": "@kaggleqrdl", "author_url": "", "post_date": "2026-05-24T19:00:33.443000", "content": "Well done, thanks for the writeup. Quite possible the prompt played a big factor in the win. Will be interesting to see what the other projects did
", "votes": 0, "replies": [] } ], "comment_count": 7, "messages": [ { "id": 3462604, "postDate": "2026-05-24T05:35:22.770Z", "content": "I participated in this competition primarily for my own learning and study. My submission is built upon a well-known public baseline notebook, and my primary goal was to improve the prompt design using an original, structured approach.
\nCompared to other outstanding notebooks and comprehensive WriteUPs, I truly realize how much I still have to learn. However, I worked hard to analyze the competition's dynamics, design my own experimental framework, and iterate on improvements, so I would love to share my findings with the community.
\nI plan to add some models and documentation in the coming days.\nAs I believe the presentation slides are much easier to read, I would highly appreciate it if you could read the attached presentation file.
\nBelow is a summary of the key points.
\nMy solution is based on the strong public baseline notebook shared by \"ZaynYu\".
\ngpt-oss-120b as-is without any additional fine-tuning or parameter training. The model is empowered with a Python code execution tool.To improve the prompt without overfitting to the Public Leaderboard (LB) or the reference.csv set, I established an offline validation process:
\nllm_history.csv). I then leveraged more capable LLMs to cross-check and categorize why gpt-oss-120b failed on incorrect instances.The main improvement was in the prompt design. Specifically:
\nI used an LLM that was more capable than gpt-oss-120b, because I expected it to analyze the responses \nmore accurately.However, a stronger LLM can also suggest solution methods that are too advanced for gpt-oss-120b to \nfollow.To avoid this, I also provided examples of correct responses generated by gpt-oss-120b.This helped me search for \neffective solution patterns that were still feasible for gpt-oss-120b to execute.
\nTo reduce the risk of overfitting, I used a separate set of 50 problems outside the official competition data while improving the solution.\nThe goal was to avoid relying too heavily on the public leaderboard or the reference problems.
\nWhen I started this experimental workflow, the public notebooks had already reached a very high score around 40/50. While it was impossible to know the exact individual accuracy for each of these 50 problems on the leaderboard, I envisioned the situation as follows:\n[Hypothesis]
\nUnder these circumstances, if I relied solely on the public leaderboard score to evaluate whether my model was improving, I felt I would end up overfitting to just those 10 unstable problems. Given that the score calculations are heavily influenced by the inherent randomness of LLM text generation and problem ordering, I wanted to avoid placing too much emphasis on this metric.
\nSince the final private evaluation would evaluate the models on completely unseen problems, my goal was to implement enhancements that targeted all branches of mathematics and various dimensions of difficulty. Using only 10 reference problems felt far too restrictive.
\nFurthermore, I reasoned that analyzing the full-text content of the responses would allow me to discover ways to make the outputs even more stable for problems the model was already answering correctly. I believed this qualitative evaluation would alleviate the risk of overfitting.
\n(I would like to express my utmost respect to the competitors who pushed the scores to such an incredibly high level around 40/50. Needless to say, the leaderboard score was a critically important metric for guiding the community's progress up to that point.)
\nDuring the development cycle, I explored several ideas that ultimately did not yield improvements and were discarded:
\n system_prompt = (\n 'You are a world-class mathematical problem solver.'\n 'The final answer is a non-negative integer in [0, 99999].'\n 'Return only the verified final answer in \\boxed{}.'\n 'Before solving, you must:'\n '1) extract all explicit constraints from the problem,'\n '2) identify the mathematical structure (algebra / number theory / combinatorics / geometry / functional equation),'\n '3) choose a solution plan,'\n '4) use Python for verification when symbolic reasoning is insufficient.'\n 'You must never guess.'\n 'If a candidate answer is found, verify it against all constraints before finalizing.'\n 'Prefer concise reasoning and avoid long unnecessary explanations.'\n )\n\n tool_prompt = (\n 'Use Python as a verification and search tool.'\n 'The environment is a stateful Jupyter notebook. '\n 'You must use print() to output results.'\n 'Always:'\n '- print intermediate key values needed for verification,'\n '- validate candidate answers against all stated constraints,'\n '- avoid brute force unless the search space is explicitly bounded or pruned,'\n '- estimate complexity before large loops,'\n '- keep code minimal and robust.'\n )\n\n preference_prompt = ( \n 'Solve the problem with the following protocol:'\n 'Step 1: Extract constraints and target form.'\n 'Step 2: Derive reductions before coding (parity, modular constraints, bounds, symmetry, invariants, monotonicity, geometric relations, etc.).'\n 'Step 3: Use Python only for:'\n '- checking candidate formulas,'\n '- bounded search after pruning,'\n '- validating edge cases,'\n '- confirming the final answer.'\n 'Step 4: If using search, explain the pruning rule first.'\n 'Step 5: Before final output, verify:'\n '- integer,'\n '- in [0, 99999],'\n '- all constraints satisfied.'\n 'Step 6: Return only the final verified answer in \\boxed{}.'\n 'Keep reasoning compact. Never guess. If verification fails, discard the candidate and continue.'\n )\nNice write-up. The structured error analysis and constraint-focused prompt design were particularly interesting. Clear and practical approach.
", "rawMarkdown": "Nice write-up. The structured error analysis and constraint-focused prompt design were particularly interesting. Clear and practical approach.", "votes": -1 }, { "id": 3463420, "postDate": "2026-05-26T21:04:06.793Z", "content": "The structure of this writeup is a step_by_step procedure which is best for learners
", "rawMarkdown": "The structure of this writeup is a step_by_step procedure which is best for learners ", "votes": -1 }, { "id": 3463276, "postDate": "2026-05-26T13:42:32.053Z", "content": "Thanks for sharing. I didn't understand what u changes other than prompt
", "rawMarkdown": "Thanks for sharing. I didn't understand what u changes other than prompt", "replies": [ { "id": 3463613, "postDate": "2026-05-27T12:47:13.723Z", "content": "In the final submission notebook, I only changed the prompt. There are no other changes.
\nBesides the prompt, I also experimented with parallel computation methods and voting systems, but I did not include them in the submission because the score appeared to decrease.
", "rawMarkdown": "In the final submission notebook, I only changed the prompt. There are no other changes.\n\nBesides the prompt, I also experimented with parallel computation methods and voting systems, but I did not include them in the submission because the score appeared to decrease.", "votes": 1, "replies": [ { "id": 3463616, "postDate": "2026-05-27T13:04:17.280Z", "content": "Hello @skobayak did you manage to observe the variance in the winning submission on the public LB ?
", "rawMarkdown": "Hello @skobayak did you manage to observe the variance in the winning submission on the public LB ?" }, { "id": 3464485, "postDate": "2026-05-30T00:41:55.650Z", "content": "I submitted my final submission notebook three times, and the public LB scores were 36, 37, and 39.
\n(If I am not mistaken, as the submission page display is a bit confusing…)
", "rawMarkdown": "I submitted my final submission notebook three times, and the public LB scores were 36, 37, and 39. \n\n(If I am not mistaken, as the submission page display is a bit confusing...)" } ] } ] }, { "id": 3462748, "postDate": "2026-05-24T19:00:33.443Z", "content": "Well done, thanks for the writeup. Quite possible the prompt played a big factor in the win. Will be interesting to see what the other projects did
", "rawMarkdown": "Well done, thanks for the writeup. Quite possible the prompt played a big factor in the win. Will be interesting to see what the other projects did" } ], "complete": true, "source": "live", "fetched_at": "2026-10-20T04:26:54Z", "competition_id": 118448, "competition_slug": "ai-mathematical-olympiad-progress-prize-3", "team_id": 15030302, "solution_url": "/competitions/ai-mathematical-olympiad-progress-prize-3/writeups/3rd-place-solution-for-the-aimo3-competition" }