This solution was developed for the AI Mathematical Olympiad - Progress Prize 3 competition. My selected notebook, AIMO3-NeverGiveUp, achieved a private score of 42 using gpt-oss-120b without fine-tuning.
The notebook is an original, independent implementation. It draws high-level inspiration from publicly shared AIMO3 community approaches that explored the combination of vLLM serving, Harmony tool-calling, Python sandbox execution, and multi-attempt reasoning. One reference that helped me understand this general pattern was:
\n[44/50] Skills Optional, Luck NecessaryAll code structure, class design, naming, prompts, deterministic seeding, aggregation logic, timeout handling, and implementation details in my notebook were written independently and materially differ from the referenced work.
\nAlso, my thanks to the AIMO organizers, Kaggle, XTX Markets, and the AIMO3 community for making this competition possible and for encouraging detailed technical sharing.
\nMy solution focuses on inference-time engineering rather than model fine-tuning. The solver runs gpt-oss-120b on a single Kaggle H100 through vLLM, using fp8 KV cache, prefix caching, Harmony encoding, and a stateful Python/Jupyter tool layer.
For each problem, the system launches 8 independent reasoning attempts. Each attempt receives the same problem, but uses deterministic per-problem seeding to encourage controlled diversity across attempts. The model can call a Python sandbox during generation to perform computation, verification, symbolic manipulation, or brute-force checks.
\nOnce attempts produce candidate answers, the solver applies a majority-first aggregation strategy. If one answer receives the highest agreement, it is selected. If multiple answers tie, the solver uses the lowest mean token entropy as the tie-breaker.
\nThe main idea is simple: a strong mathematical reasoning model still needs a reliable execution system around it. In this competition, performance depended not only on the model, but also on how inference was orchestrated, how tools were exposed, how time was managed, and how candidate answers were selected.
\nThe solution has four main components:
\ngpt-oss-120bThe model is served through a local OpenAI-compatible vLLM endpoint. The solver then communicates with the server through the OpenAI client API.
\nBefore starting inference, the notebook prewarms model files by reading the model checkpoint files into the OS page cache. This reduces server startup friction. The notebook then initializes 16 Python kernels, each preloaded with common mathematical libraries.
\n| Parameter | \nValue | \nNotes | \n
|---|---|---|
| Model | \ngpt-oss-120b | \nKaggle-hosted model | \n
| Inference engine | \nvLLM | \nOpenAI-compatible local server | \n
| Hardware | \n1× Kaggle H100 | \nCompetition environment | \n
| KV cache | \nfp8_e4m3 | \nMemory-efficient serving | \n
| Context length | \n65,536 tokens | \nmax_model_len | \n
| Reasoning effort | \nHIGH | \nHarmony system content | \n
| Temperature | \n0.9 | \nUsed in selected notebook | \n
min_p | \n0.02 | \nSampling filter | \n
| Attempts per problem | \n8 | \nParallel independent attempts | \n
| Worker kernels | \n16 | \nPython sandbox pool | \n
| Turns per attempt | \n96 | \nChosen to limit drift and cost | \n
| Early-stop threshold | \n4 agreements | \nStop when enough attempts converge | \n
| Base timeout | \n300 s | \nReserved per problem | \n
| Max timeout | \n900 s | \nIf enough global time remains | \n
| Notebook runtime limit | \n17,400 s | \nConservative limit below full wall clock | \n
| Aggregation | \nMajority-first + entropy tie-break | \nAgreement first, confidence second | \n
| Seed | \n42 | \nBase seed | \n
| Answer range | \n[0, 99999] | \nCompetition format | \n
The prompts were intentionally compact, and the system was intended to define the task, answer format, and tool availability without forcing the model into a rigid reasoning template.
\nYou are an elite math problem solver.\nSolve the problem carefully and verify your result.\nIf computation helps, call the python tool.\n\n### Rules:\n- Final answer must be an integer in [0, 99999]\n- Put final answer in \\boxed{...}\nTool prompt
\nPython tool: use it for computation, brute force, algebra (sympy), or checks.\nExplain briefly what we're doing, then run code. Use print() for outputs.\nPreference prompt
\nUse math/numpy/sympy/mpmath. Prefer exact reasoning, then verify.\nThe design principle was to give the model enough structure to behave reliably, but not so much structure that the prompt starts controlling the reasoning path too aggressively. In my experiments, compact prompting was easier to manage under strict runtime limits.
\nEach reasoning attempt has access to a stateful Python sandbox through Harmony tool messages. Every sandbox preloads:
\nThis tool layer is important because many AIMO-style problems require exact arithmetic, symbolic simplification, modular checks, combinatorial enumeration, or high-precision numerical validation.
\nAlso, added a helper that wraps the final expression in print() when appropriate. This avoids silent Python cells and makes outputs visible to the model.
\nFor each problem, the solver launches 8 independent attempts. Each attempt has the same model, prompt, and tool access, but receives a deterministic seed based on the problem text and attempt index.
\nThe seed is computed as:
\nh = hashlib.md5(problem_text.encode(\"utf-8\")).hexdigest()\nbase = int(h[:8], 16)\nseed = (base + 9973 * attempt_idx) % (2**31 - 1)\nUsed this design because two things were intended at the same time:
\nThis was especially useful because stochastic inference can produce noticeable leaderboard variance.
\nThe model output is streamed token by token. During streaming, the solver scans recent text whenever a closing brace appears. This allows an attempt to terminate as soon as a valid answer is found instead of waiting for the full message to finish.
\nThe primary answer pattern is:
\n\\boxed{...}\nThe regex used is:
\nr\"\\\\boxed\\s*\\{\\s*([0-9,]+)\\s*\\}\"\nIf no boxed answer is found, the fallback pattern checks for:
\nfinal answer is <integer>\nOnly integers in [0, 99999] are accepted. Attempts that do not produce a valid integer answer do not contribute to the final vote.
\nThe final submission uses a majority-first aggregation rule.
\nThe process is:
\nThe relevant idea is:
\ncounts = Counter(answers)\ntop_cnt = counts.most_common(1)[0][1]\ntop_answers = [a for a, c in counts.items() if c == top_cnt]\nIf there is only one top answer, it is selected directly. If there is a tie, entropy is used as a confidence signal.
\nThis is chosen because agreement between independent attempts is an interpretable reliability signal. Entropy is useful, but in this system, it's used as a tie-breaker rather than allowing one low-entropy answer to override a stronger majority.
\nThe solver stops early once 4 valid attempts agree on the same answer.
\nif Counter(valid).most_common(1)[0][1] >= self.cfg.early_stop:\n stop_evt.set()\nThis helped save time on problems where the answer stabilized quickly. It also helped preserve runtime for harder problems later in the evaluation.
\nThe notebook uses a global runtime limit of 17,400 seconds. For each problem, it computes the remaining time and reserves a base budget for the remaining problems.
\nThe budget is capped between:
\nThis prevents one difficult problem from consuming too much of the global budget.
\nThe budgeting logic is:
\nelapsed = time.time() - self.start_time\nleft = self.cfg.notebook_limit - elapsed\nothers = max(0, self.problems_remaining - 1)\nreserve = others * self.cfg.base_problem_timeout\n\nbudget = left - reserve\nbudget = min(budget, self.cfg.high_problem_timeout)\nbudget = max(budget, self.cfg.base_problem_timeout)\nThe strongest parts of the system were:
\nThe model performed best when it used Python to verify computations instead of relying only on text reasoning.
\nSeveral directions are identified for further exploration in future iterations:
\nThe current system starts with the same number of attempts for every problem. A stronger version could estimate problem difficulty from early attempts and allocate extra compute only when needed.
\nThe current verifier is implicit: agreement and entropy are used as signals. A future version could verify consistency between the final answer, Python outputs, and the reasoning trace.
\nThe current notebook focuses on producing valid submissions. A research version should log attempt-level traces, Python call counts, entropy, time per attempt, and answer agreement patterns.
\nIf early attempts strongly disagree, a second wave could be launched selectively. I did not include this in the selected notebook because the additional runtime risk was not clearly worth it.
\nThe selected submission was run in the Kaggle competition environment.
\n| Split | \nScore | \n
|---|---|
| Private score | \n42 / 50 | \n
| Public score | \n42 / 50 | \n
This solution shows that a competitive mathematical reasoning system can be built without fine-tuning the base model. The main contribution is the inference harness around the model: vLLM serving, Harmony tool use, Python sandboxing, deterministic attempt diversity, early stopping, time budgeting, and majority-first aggregation.
\nMy main takeaway from AIMO3 is that strong LLM reasoning is not only a modeling problem. It is also a systems problem. The model must be paired with the right execution environment, tool interface, answer extraction logic, and aggregation strategy to perform reliably under competition constraints.
", "comments": [], "comment_count": 0, "messages": [ { "id": 3447849, "postDate": "2026-04-24T04:53:15.140Z", "content": "This solution was developed for the AI Mathematical Olympiad - Progress Prize 3 competition. My selected notebook, AIMO3-NeverGiveUp, achieved a private score of 42 using gpt-oss-120b without fine-tuning.
The notebook is an original, independent implementation. It draws high-level inspiration from publicly shared AIMO3 community approaches that explored the combination of vLLM serving, Harmony tool-calling, Python sandbox execution, and multi-attempt reasoning. One reference that helped me understand this general pattern was:
\n[44/50] Skills Optional, Luck NecessaryAll code structure, class design, naming, prompts, deterministic seeding, aggregation logic, timeout handling, and implementation details in my notebook were written independently and materially differ from the referenced work.
\nAlso, my thanks to the AIMO organizers, Kaggle, XTX Markets, and the AIMO3 community for making this competition possible and for encouraging detailed technical sharing.
\nMy solution focuses on inference-time engineering rather than model fine-tuning. The solver runs gpt-oss-120b on a single Kaggle H100 through vLLM, using fp8 KV cache, prefix caching, Harmony encoding, and a stateful Python/Jupyter tool layer.
For each problem, the system launches 8 independent reasoning attempts. Each attempt receives the same problem, but uses deterministic per-problem seeding to encourage controlled diversity across attempts. The model can call a Python sandbox during generation to perform computation, verification, symbolic manipulation, or brute-force checks.
\nOnce attempts produce candidate answers, the solver applies a majority-first aggregation strategy. If one answer receives the highest agreement, it is selected. If multiple answers tie, the solver uses the lowest mean token entropy as the tie-breaker.
\nThe main idea is simple: a strong mathematical reasoning model still needs a reliable execution system around it. In this competition, performance depended not only on the model, but also on how inference was orchestrated, how tools were exposed, how time was managed, and how candidate answers were selected.
\nThe solution has four main components:
\ngpt-oss-120bThe model is served through a local OpenAI-compatible vLLM endpoint. The solver then communicates with the server through the OpenAI client API.
\nBefore starting inference, the notebook prewarms model files by reading the model checkpoint files into the OS page cache. This reduces server startup friction. The notebook then initializes 16 Python kernels, each preloaded with common mathematical libraries.
\n| Parameter | \nValue | \nNotes | \n
|---|---|---|
| Model | \ngpt-oss-120b | \nKaggle-hosted model | \n
| Inference engine | \nvLLM | \nOpenAI-compatible local server | \n
| Hardware | \n1× Kaggle H100 | \nCompetition environment | \n
| KV cache | \nfp8_e4m3 | \nMemory-efficient serving | \n
| Context length | \n65,536 tokens | \nmax_model_len | \n
| Reasoning effort | \nHIGH | \nHarmony system content | \n
| Temperature | \n0.9 | \nUsed in selected notebook | \n
min_p | \n0.02 | \nSampling filter | \n
| Attempts per problem | \n8 | \nParallel independent attempts | \n
| Worker kernels | \n16 | \nPython sandbox pool | \n
| Turns per attempt | \n96 | \nChosen to limit drift and cost | \n
| Early-stop threshold | \n4 agreements | \nStop when enough attempts converge | \n
| Base timeout | \n300 s | \nReserved per problem | \n
| Max timeout | \n900 s | \nIf enough global time remains | \n
| Notebook runtime limit | \n17,400 s | \nConservative limit below full wall clock | \n
| Aggregation | \nMajority-first + entropy tie-break | \nAgreement first, confidence second | \n
| Seed | \n42 | \nBase seed | \n
| Answer range | \n[0, 99999] | \nCompetition format | \n
The prompts were intentionally compact, and the system was intended to define the task, answer format, and tool availability without forcing the model into a rigid reasoning template.
\nYou are an elite math problem solver.\nSolve the problem carefully and verify your result.\nIf computation helps, call the python tool.\n\n### Rules:\n- Final answer must be an integer in [0, 99999]\n- Put final answer in \\boxed{...}\nTool prompt
\nPython tool: use it for computation, brute force, algebra (sympy), or checks.\nExplain briefly what we're doing, then run code. Use print() for outputs.\nPreference prompt
\nUse math/numpy/sympy/mpmath. Prefer exact reasoning, then verify.\nThe design principle was to give the model enough structure to behave reliably, but not so much structure that the prompt starts controlling the reasoning path too aggressively. In my experiments, compact prompting was easier to manage under strict runtime limits.
\nEach reasoning attempt has access to a stateful Python sandbox through Harmony tool messages. Every sandbox preloads:
\nThis tool layer is important because many AIMO-style problems require exact arithmetic, symbolic simplification, modular checks, combinatorial enumeration, or high-precision numerical validation.
\nAlso, added a helper that wraps the final expression in print() when appropriate. This avoids silent Python cells and makes outputs visible to the model.
\nFor each problem, the solver launches 8 independent attempts. Each attempt has the same model, prompt, and tool access, but receives a deterministic seed based on the problem text and attempt index.
\nThe seed is computed as:
\nh = hashlib.md5(problem_text.encode(\"utf-8\")).hexdigest()\nbase = int(h[:8], 16)\nseed = (base + 9973 * attempt_idx) % (2**31 - 1)\nUsed this design because two things were intended at the same time:
\nThis was especially useful because stochastic inference can produce noticeable leaderboard variance.
\nThe model output is streamed token by token. During streaming, the solver scans recent text whenever a closing brace appears. This allows an attempt to terminate as soon as a valid answer is found instead of waiting for the full message to finish.
\nThe primary answer pattern is:
\n\\boxed{...}\nThe regex used is:
\nr\"\\\\boxed\\s*\\{\\s*([0-9,]+)\\s*\\}\"\nIf no boxed answer is found, the fallback pattern checks for:
\nfinal answer is <integer>\nOnly integers in [0, 99999] are accepted. Attempts that do not produce a valid integer answer do not contribute to the final vote.
\nThe final submission uses a majority-first aggregation rule.
\nThe process is:
\nThe relevant idea is:
\ncounts = Counter(answers)\ntop_cnt = counts.most_common(1)[0][1]\ntop_answers = [a for a, c in counts.items() if c == top_cnt]\nIf there is only one top answer, it is selected directly. If there is a tie, entropy is used as a confidence signal.
\nThis is chosen because agreement between independent attempts is an interpretable reliability signal. Entropy is useful, but in this system, it's used as a tie-breaker rather than allowing one low-entropy answer to override a stronger majority.
\nThe solver stops early once 4 valid attempts agree on the same answer.
\nif Counter(valid).most_common(1)[0][1] >= self.cfg.early_stop:\n stop_evt.set()\nThis helped save time on problems where the answer stabilized quickly. It also helped preserve runtime for harder problems later in the evaluation.
\nThe notebook uses a global runtime limit of 17,400 seconds. For each problem, it computes the remaining time and reserves a base budget for the remaining problems.
\nThe budget is capped between:
\nThis prevents one difficult problem from consuming too much of the global budget.
\nThe budgeting logic is:
\nelapsed = time.time() - self.start_time\nleft = self.cfg.notebook_limit - elapsed\nothers = max(0, self.problems_remaining - 1)\nreserve = others * self.cfg.base_problem_timeout\n\nbudget = left - reserve\nbudget = min(budget, self.cfg.high_problem_timeout)\nbudget = max(budget, self.cfg.base_problem_timeout)\nThe strongest parts of the system were:
\nThe model performed best when it used Python to verify computations instead of relying only on text reasoning.
\nSeveral directions are identified for further exploration in future iterations:
\nThe current system starts with the same number of attempts for every problem. A stronger version could estimate problem difficulty from early attempts and allocate extra compute only when needed.
\nThe current verifier is implicit: agreement and entropy are used as signals. A future version could verify consistency between the final answer, Python outputs, and the reasoning trace.
\nThe current notebook focuses on producing valid submissions. A research version should log attempt-level traces, Python call counts, entropy, time per attempt, and answer agreement patterns.
\nIf early attempts strongly disagree, a second wave could be launched selectively. I did not include this in the selected notebook because the additional runtime risk was not clearly worth it.
\nThe selected submission was run in the Kaggle competition environment.
\n| Split | \nScore | \n
|---|---|
| Private score | \n42 / 50 | \n
| Public score | \n42 / 50 | \n
This solution shows that a competitive mathematical reasoning system can be built without fine-tuning the base model. The main contribution is the inference harness around the model: vLLM serving, Harmony tool use, Python sandboxing, deterministic attempt diversity, early stopping, time budgeting, and majority-first aggregation.
\nMy main takeaway from AIMO3 is that strong LLM reasoning is not only a modeling problem. It is also a systems problem. The model must be paired with the right execution environment, tool interface, answer extraction logic, and aggregation strategy to perform reliably under competition constraints.
", "rawMarkdown": "# NeverGiveUp: Majority-First Tool-Augmented Reasoning for AIMO3\n\n## Acknowledgments\n\nThis solution was developed for the AI Mathematical Olympiad - Progress Prize 3 competition. My selected notebook, **AIMO3-NeverGiveUp**, achieved a private score of **42** using `gpt-oss-120b` without fine-tuning.\n\nThe notebook is an original, independent implementation. It draws high-level inspiration from publicly shared AIMO3 community approaches that explored the combination of vLLM serving, Harmony tool-calling, Python sandbox execution, and multi-attempt reasoning. One reference that helped me understand this general pattern was:\n\n- **Inspired by:** `[44/50] Skills Optional, Luck Necessary`\n- **Author:** Andreas Bisiadis\n- **Link:** https://www.kaggle.com/code/nihilisticneuralnet/44-50-skills-optional-luck-necessary\n\nAll code structure, class design, naming, prompts, deterministic seeding, aggregation logic, timeout handling, and implementation details in my notebook were written independently and materially differ from the referenced work.\n\nAlso, my thanks to the AIMO organizers, Kaggle, XTX Markets, and the AIMO3 community for making this competition possible and for encouraging detailed technical sharing.\n\n---\n\n## Overview\n\nMy solution focuses on inference-time engineering rather than model fine-tuning. The solver runs `gpt-oss-120b` on a single Kaggle H100 through vLLM, using fp8 KV cache, prefix caching, Harmony encoding, and a stateful Python/Jupyter tool layer.\n\nFor each problem, the system launches 8 independent reasoning attempts. Each attempt receives the same problem, but uses deterministic per-problem seeding to encourage controlled diversity across attempts. The model can call a Python sandbox during generation to perform computation, verification, symbolic manipulation, or brute-force checks.\n\nOnce attempts produce candidate answers, the solver applies a majority-first aggregation strategy. If one answer receives the highest agreement, it is selected. If multiple answers tie, the solver uses the lowest mean token entropy as the tie-breaker.\n\nThe main idea is simple: a strong mathematical reasoning model still needs a reliable execution system around it. In this competition, performance depended not only on the model, but also on how inference was orchestrated, how tools were exposed, how time was managed, and how candidate answers were selected.\n\n---\n\n## System Architecture\n\nThe solution has four main components:\n\n1. Offline dependency setup using the AIMO3 utility wheels\n2. Local vLLM server for `gpt-oss-120b`\n3. Harmony prompt wrapper and Python tool interface\n4. Multi-attempt reasoning engine with majority-first aggregation\n\nThe model is served through a local OpenAI-compatible vLLM endpoint. The solver then communicates with the server through the OpenAI client API.\n\nBefore starting inference, the notebook prewarms model files by reading the model checkpoint files into the OS page cache. This reduces server startup friction. The notebook then initializes 16 Python kernels, each preloaded with common mathematical libraries.\n\n---\n\n## Configuration\n\n| Parameter | Value | Notes |\n|---|---:|---|\n| Model | `gpt-oss-120b` | Kaggle-hosted model |\n| Inference engine | vLLM | OpenAI-compatible local server |\n| Hardware | 1× Kaggle H100 | Competition environment |\n| KV cache | `fp8_e4m3` | Memory-efficient serving |\n| Context length | 65,536 tokens | `max_model_len` |\n| Reasoning effort | HIGH | Harmony system content |\n| Temperature | 0.9 | Used in selected notebook |\n| `min_p` | 0.02 | Sampling filter |\n| Attempts per problem | 8 | Parallel independent attempts |\n| Worker kernels | 16 | Python sandbox pool |\n| Turns per attempt | 96 | Chosen to limit drift and cost |\n| Early-stop threshold | 4 agreements | Stop when enough attempts converge |\n| Base timeout | 300 s | Reserved per problem |\n| Max timeout | 900 s | If enough global time remains |\n| Notebook runtime limit | 17,400 s | Conservative limit below full wall clock |\n| Aggregation | Majority-first + entropy tie-break | Agreement first, confidence second |\n| Seed | 42 | Base seed |\n| Answer range | `[0, 99999]` | Competition format |\n\n---\n\n## Prompts\n\nThe prompts were intentionally compact, and the system was intended to define the task, answer format, and tool availability without forcing the model into a rigid reasoning template.\n\n### System prompt\n\n```text\nYou are an elite math problem solver.\nSolve the problem carefully and verify your result.\nIf computation helps, call the python tool.\n\n### Rules:\n- Final answer must be an integer in [0, 99999]\n- Put final answer in \\boxed{...}\n```\n\n**Tool prompt**\n```\nPython tool: use it for computation, brute force, algebra (sympy), or checks.\nExplain briefly what we're doing, then run code. Use print() for outputs.\n```\n\n**Preference prompt**\n```\nUse math/numpy/sympy/mpmath. Prefer exact reasoning, then verify.\n```\n\nThe design principle was to give the model enough structure to behave reliably, but not so much structure that the prompt starts controlling the reasoning path too aggressively. In my experiments, compact prompting was easier to manage under strict runtime limits.\n\n---\n\n## Python Tool Layer\n\nEach reasoning attempt has access to a stateful Python sandbox through Harmony tool messages. Every sandbox preloads:\n\n- math\n- numpy\n- sympy\n- itertools\n- collections\n- mpmath with 64-digit precision\n\nThis tool layer is important because many AIMO-style problems require exact arithmetic, symbolic simplification, modular checks, combinatorial enumeration, or high-precision numerical validation.\n\nAlso, added a helper that wraps the final expression in print() when appropriate. This avoids silent Python cells and makes outputs visible to the model.\n\n---\n\n## Parallel Attempts\n\nFor each problem, the solver launches 8 independent attempts. Each attempt has the same model, prompt, and tool access, but receives a deterministic seed based on the problem text and attempt index.\n\nThe seed is computed as:\n```\nh = hashlib.md5(problem_text.encode(\"utf-8\")).hexdigest()\nbase = int(h[:8], 16)\nseed = (base + 9973 * attempt_idx) % (2**31 - 1)\n```\n\nUsed this design because two things were intended at the same time:\n\n- Diversity between attempts\n- More stable behavior across reruns\n\nThis was especially useful because stochastic inference can produce noticeable leaderboard variance.\n\n---\n\n## Streaming Answer Extraction\n\nThe model output is streamed token by token. During streaming, the solver scans recent text whenever a closing brace appears. This allows an attempt to terminate as soon as a valid answer is found instead of waiting for the full message to finish.\n\nThe primary answer pattern is:\n```\n\\boxed{...}\n```\n\nThe regex used is:\n```\nr\"\\\\boxed\\s*\\{\\s*([0-9,]+)\\s*\\}\"\n```\n\nIf no boxed answer is found, the fallback pattern checks for:\n```\nfinal answer is