# NeverGiveUp: Majority-First Tool-Augmented Reasoning for AIMO3

NeverGiveUp: Majority-First Tool-Augmented Reasoning for AIMO3
Acknowledgments
This solution was developed for the AI Mathematical Olympiad - Progress Prize 3 competition. My selected notebook,
AIMO3-NeverGiveUp
, achieved a private score of
42
using
gpt-oss-120b
without fine-tuning.
The notebook is an original, independent implementation. It draws high-level inspiration from publicly shared AIMO3 community approaches that explored the combination of vLLM serving, Harmony tool-calling, Python sandbox execution, and multi-attempt reasoning. One reference that helped me understand this general pattern was:
Inspired by:
[44/50] Skills Optional, Luck Necessary
Author:
Andreas Bisiadis
Link:
[https://www.kaggle.com/code/nihilisticneuralnet/44-50-skills-optional-luck-necessary](https://www.kaggle.com/code/nihilisticneuralnet/44-50-skills-optional-luck-necessary)
All code structure, class design, naming, prompts, deterministic seeding, aggregation logic, timeout handling, and implementation details in my notebook were written independently and materially differ from the referenced work.
Also, my thanks to the AIMO organizers, Kaggle, XTX Markets, and the AIMO3 community for making this competition possible and for encouraging detailed technical sharing.
Overview
My solution focuses on inference-time engineering rather than model fine-tuning. The solver runs
gpt-oss-120b
on a single Kaggle H100 through vLLM, using fp8 KV cache, prefix caching, Harmony encoding, and a stateful Python/Jupyter tool layer.
For each problem, the system launches 8 independent reasoning attempts. Each attempt receives the same problem, but uses deterministic per-problem seeding to encourage controlled diversity across attempts. The model can call a Python sandbox during generation to perform computation, verification, symbolic manipulation, or brute-force checks.
Once attempts produce candidate answers, the solver applies a majority-first aggregation strategy. If one answer receives the highest agreement, it is selected. If multiple answers tie, the solver uses the lowest mean token entropy as the tie-breaker.
The main idea is simple: a strong mathematical reasoning model still needs a reliable execution system around it. In this competition, performance depended not only on the model, but also on how inference was orchestrated, how tools were exposed, how time was managed, and how candidate answers were selected.
System Architecture
The solution has four main components:
Offline dependency setup using the AIMO3 utility wheels
Local vLLM server for
gpt-oss-120b
Harmony prompt wrapper and Python tool interface
Multi-attempt reasoning engine with majority-first aggregation
The model is served through a local OpenAI-compatible vLLM endpoint. The solver then communicates with the server through the OpenAI client API.
Before starting inference, the notebook prewarms model files by reading the model checkpoint files into the OS page cache. This reduces server startup friction. The notebook then initializes 16 Python kernels, each preloaded with common mathematical libraries.
Configuration
Parameter
Value
Notes
Model
gpt-oss-120b
Kaggle-hosted model
Inference engine
vLLM
OpenAI-compatible local server
Hardware
1× Kaggle H100
Competition environment
KV cache
fp8_e4m3
Memory-efficient serving
Context length
65,536 tokens
max_model_len
Reasoning effort
HIGH
Harmony system content
Temperature
0.9
Used in selected notebook
min_p
0.02
Sampling filter
Attempts per problem
8
Parallel independent attempts
Worker kernels
16
Python sandbox pool
Turns per attempt
96
Chosen to limit drift and cost
Early-stop threshold
4 agreements
Stop when enough attempts converge
Base timeout
300 s
Reserved per problem
Max timeout
900 s
If enough global time remains
Notebook runtime limit
17,400 s
Conservative limit below full wall clock
Aggregation
Majority-first + entropy tie-break
Agreement first, confidence second
Seed
42
Base seed
Answer range
[0, 99999]
Competition format
Prompts
The prompts were intentionally compact, and the system was intended to define the task, answer format, and tool availability without forcing the model into a rigid reasoning template.
System prompt
You are an elite math problem solver.
Solve the problem carefully and verify your result.
If computation helps, call the python tool.

### Rules:
- Final answer must be an integer in [0, 99999]
- Put final answer in \boxed{...}
Tool prompt
Python tool: use it for computation, brute force, algebra (sympy), or checks.
Explain briefly what we're doing, then run code. Use print() for outputs.
Preference prompt
Use math/numpy/sympy/mpmath. Prefer exact reasoning, then verify.
The design principle was to give the model enough structure to behave reliably, but not so much structure that the prompt starts controlling the reasoning path too aggressively. In my experiments, compact prompting was easier to manage under strict runtime limits.
Python Tool Layer
Each reasoning attempt has access to a stateful Python sandbox through Harmony tool messages. Every sandbox preloads:
math
numpy
sympy
itertools
collections
mpmath with 64-digit precision
This tool layer is important because many AIMO-style problems require exact arithmetic, symbolic simplification, modular checks, combinatorial enumeration, or high-precision numerical validation.
Also, added a helper that wraps the final expression in print() when appropriate. This avoids silent Python cells and makes outputs visible to the model.
Parallel Attempts
For each problem, the solver launches 8 independent attempts. Each attempt has the same model, prompt, and tool access, but receives a deterministic seed based on the problem text and attempt index.
The seed is computed as:
h = hashlib.md5(problem_text.encode("utf-8")).hexdigest()
base = int(h[:8], 16)
seed = (base + 9973 * attempt_idx) % (2**31 - 1)
Used this design because two things were intended at the same time:
Diversity between attempts
More stable behavior across reruns
This was especially useful because stochastic inference can produce noticeable leaderboard variance.
Streaming Answer Extraction
The model output is streamed token by token. During streaming, the solver scans recent text whenever a closing brace appears. This allows an attempt to terminate as soon as a valid answer is found instead of waiting for the full message to finish.
The primary answer pattern is:
\boxed{...}
The regex used is:
r"\\boxed\s*\{\s*([0-9,]+)\s*\}"
If no boxed answer is found, the fallback pattern checks for:
final answer is <integer>
Only integers in [0, 99999] are accepted. Attempts that do not produce a valid integer answer do not contribute to the final vote.
Aggregation
The final submission uses a majority-first aggregation rule.
The process is:
Collect valid answers from all completed attempts.
Count how many times each answer appears.
Select the answer with the highest count.
If multiple answers tie, select the tied answer with the lowest mean token entropy.
The relevant idea is:
counts = Counter(answers)
top_cnt = counts.most_common(1)[0][1]
top_answers = [a for a, c in counts.items() if c == top_cnt]
If there is only one top answer, it is selected directly. If there is a tie, entropy is used as a confidence signal.
This is chosen because agreement between independent attempts is an interpretable reliability signal. Entropy is useful, but in this system, it's used as a tie-breaker rather than allowing one low-entropy answer to override a stronger majority.
Early Stopping
The solver stops early once 4 valid attempts agree on the same answer.
if Counter(valid).most_common(1)[0][1] >= self.cfg.early_stop:
    stop_evt.set()
This helped save time on problems where the answer stabilized quickly. It also helped preserve runtime for harder problems later in the evaluation.
Runtime Budgeting
The notebook uses a global runtime limit of 17,400 seconds. For each problem, it computes the remaining time and reserves a base budget for the remaining problems.
The budget is capped between:
300 seconds minimum
900 seconds maximum
This prevents one difficult problem from consuming too much of the global budget.
The budgeting logic is:
elapsed = time.time() - self.start_time
left = self.cfg.notebook_limit - elapsed
others = max(0, self.problems_remaining - 1)
reserve = others * self.cfg.base_problem_timeout

budget = left - reserve
budget = min(budget, self.cfg.high_problem_timeout)
budget = max(budget, self.cfg.base_problem_timeout)
What Worked
The strongest parts of the system were:
Compact prompts
Tool-augmented reasoning
8 parallel attempts
Deterministic per-problem seeds
Majority-first aggregation
Early stopping at 4 agreements
Runtime budgeting across the evaluation
The model performed best when it used Python to verify computations instead of relying only on text reasoning.
Future Improvements
Several directions are identified for further exploration in future iterations:
Difficulty-aware scheduling
The current system starts with the same number of attempts for every problem. A stronger version could estimate problem difficulty from early attempts and allocate extra compute only when needed.
Better verification
The current verifier is implicit: agreement and entropy are used as signals. A future version could verify consistency between the final answer, Python outputs, and the reasoning trace.
More detailed logging
The current notebook focuses on producing valid submissions. A research version should log attempt-level traces, Python call counts, entropy, time per attempt, and answer agreement patterns.
Adaptive second wave
If early attempts strongly disagree, a second wave could be launched selectively. I did not include this in the selected notebook because the additional runtime risk was not clearly worth it.
Reproducibility
The selected submission was run in the Kaggle competition environment.
Hardware: 1× NVIDIA H100
Model: gpt-oss-120b
Model source: Kaggle model input
Utility wheels: AIMO 3 Utils
Inference engine: vLLM
Tool interface: Harmony + Jupyter kernel
Fine-tuning: None
Results
Split
Score
Private score
42 / 50
Public score
42 / 50
Conclusion
This solution shows that a competitive mathematical reasoning system can be built without fine-tuning the base model. The main contribution is the inference harness around the model: vLLM serving, Harmony tool use, Python sandboxing, deterministic attempt diversity, early stopping, time budgeting, and majority-first aggregation.
My main takeaway from AIMO3 is that strong LLM reasoning is not only a modeling problem. It is also a systems problem. The model must be paired with the right execution environment, tool interface, answer extraction logic, and aggregation strategy to perform reliably under competition constraints.