# Scaling Inference-Time Compute for Mathematical Reasoning

Scaling Inference-Time Compute for Mathematical Reasoning: A Systematic Study of 60+ Experiments on IMO-Level Problems
Pawan Mali
AIMO Progress Prize 3 Competition Writeup - Updated May 2026
Abstract
This writeup documents three months of inference-time scaling work on the AIMO Progress Prize 3 competition, where the task is to solve 50 hidden IMO-level mathematical problems on a single NVIDIA H100 within a 4.8-hour budget. Across 60+ submitted versions we converged on a configuration we call V143: GPT-OSS-120B with
temp=1.0
, an 8-attempt parallel-sampling pipeline, early-stopping on 4-vote consensus, a 5-component weighted entropy score for confidence ranking, and a 65K context window with FP8 KV cache. V40 (an earlier iteration of the same config) scored 42/50, and V143 reproduced 41/50 on the hidden test set. After the competition closed we rebuilt the pipeline outside Kaggle with per-attempt instrumentation and rented an H100 80GB to validate against the public 10-problem reference set; that run scored
10/10 in 31 minutes
, total cost $5.50, with a full per-attempt CSV preserved. The most surprising finding from the instrumented run: on one reference problem all 8 parallel attempts produced 8 different answers, and the entropy ranking still picked the correct one based on confidence alone. The rest of this writeup catalogues both what worked and the considerably longer list of things that didn't verbose prompts (-9), Bayesian voting (-11), self-refinement (-4), fine-tuning without matching the inference pipeline (-6), bundled "improvements" (-6), and a context-length finding that turned out to be confounded with throughput.
1. Introduction
AIMO Progress Prize 3 sits at the intersection of two hard problems: mathematical reasoning at IMO level, and inference-time scaling under realistic compute constraints. Fifty hidden problems, answers must be non-negative integers between 0 and 99,999, evaluated on a single H100 with roughly 4.8 hours of wall-clock budget. A $2.2M prize was on offer for 47/50; no team reached it. The winner (ippeiogawa) achieved 46/50; a cluster of teams sat at 44–45.
This writeup documents 60+ submitted versions between January and April 2026. The peak was 42/50 (V40), and we settled on a stable 41/50 baseline (V125, V132, V133, V143). The numerical headline is not the most useful thing here it's that across the same number of versions trying to improve on that baseline, roughly the same number of changes regressed it. Most regressions were not subtle: -6 to -11 points on a single configuration change.
The contributions of this work are three:
A reproducible 10/10 reference-set result for the V143 configuration, with byte-exact seeding and a per-attempt CSV (16 columns) available alongside this writeup.
A characterisation of what does and doesn't work for inference-time scaling on this kind of task, with deltas calibrated against an observed ±3.5-point single-run variance band so readers can see which findings are signal and which are noise.
An empirical case study of a confidence-only voting decision: a problem where 8 attempts produced 8 different answers, and a 5-component entropy ranking nevertheless picked the correct one.
![](assets/249ab982925e78e10e32149e.png)
Figure 1: Our submissions in the context of the final AIMO3 leaderboard. The top of the leaderboard (gray) clusters between 44 and 46 with the winner ippeiogawa at 46/50. Our peak V40 (42/50, dark green) and stable V143 baseline (41/50, mid green) sit just below the leader cluster, with V148 (40/50) slightly behind. Our two largest regressions V135 (131K context, 35/50) and V142 (Bayesian + VOI voting, 30/50) are highlighted in purple. The shaded band shows the ±3.5-point variance observed across re-runs of V143; the red dashed line marks the 47/50 prize threshold that no team reached.
2. Related Work
Several lines of work inform the design used here.
Chain-of-thought prompting
(Wei et al., 2022) demonstrated that step-by-step reasoning markedly improves mathematical performance for large models.
Self-consistency
(Wang et al., 2022) extended that by sampling multiple chains and selecting the majority answer, which is the conceptual foundation of our 8-attempt voting pipeline. More recent test-time-compute work (Snell et al., 2024; Brown et al., 2024) systematically explores how inference budget translates into accuracy gains, and finds that sample-and-vote scales favourably until a diminishing-returns regime. Our experiments are consistent with that, but with the additional finding that the
quality
of the aggregation function (here, 5-component weighted entropy rather than simple majority) matters at this size.
Tool-integrated reasoning
letting the model emit Python that gets executed in a sandbox has proven particularly effective on mathematical tasks. Our pipeline runs each attempt with access to an isolated Jupyter kernel; the model generates code, the kernel returns results, the model continues. This combines pattern recognition with arithmetic precision.
Self-refinement
approaches have shown mixed results. Huang et al. (2023) found that LLMs cannot reliably self-correct reasoning errors without external feedback, and that self-refinement often introduces errors into previously-correct answers. Our V134 experiment is consistent with this: adding a self-refinement step regressed our score by 4 points (within the variance band, but reliably negative across multiple submissions).
In the AIMO competition series specifically, the first progress prize topped out around 29/50, the second edition at roughly 35/50 (with the winning solution introducing a GenSelect-style LLM-as-judge over candidates), and AIMO-3 leaderboard winners reached 46/50. Public AIMO-3 notebooks  particularly nihilisticneuralnet (44/50) and huikang established baselines that influenced our search space.
3. Methodology
3.1 Model Selection
We tested several base models before committing to GPT-OSS-120B. DeepSeek-R1-32B reasoned well but its capacity capped scores around 28–32. Qwen2.5-Math-72B was strong on math but its 32K context limited the length of tool-using reasoning chains, putting it in the 30–35 range. GPT-OSS-120B at 120B parameters and 131K native context consistently scored 38–42 across our pipeline iterations, and was our choice for everything subsequent.
The model is MXFP4-quantized in its public release (
openai/gpt-oss-120b
), which requires a GPU with compute capability ≥ 8.0 (Ampere or newer). On the H100 the entire 60 GB of weights fits comfortably with a 6 GB FP8 KV cache and an 8 GB safety buffer in 80 GB.
3.2 System Architecture
The pipeline has three stages per problem:
Parallel generation.
Eight attempts are launched concurrently against a vLLM server hosting GPT-OSS-120B. Each attempt has its own Jupyter sandbox; the model is free to emit Python and consume the tool result before continuing. Generation proceeds until a
\boxed{}
answer is produced or the deadline is hit.
Per-attempt scoring.
As tokens stream back we accumulate per-token entropy from the top-k logprobs (
k=5
). At the end of each attempt we summarise the entropy sequence with a 5-component score (§3.3) and store it.
Weighted voting.
Each attempt contributes
1 / max(E, ε)
to its extracted answer; the answer with the highest summed weight is selected. Early-stopping triggers when any answer reaches 4 votes, cancelling remaining attempts and freeing the budget for the next problem.
3.3 Five-Component Entropy Voting
For each token
t
with top-k logprobs we compute Shannon entropy
H_t = −Σᵢ pᵢ · log₂(pᵢ)
. The per-attempt entropy sequence
{H₁ … Hₙ}
is summarised by a five-component score:
E = 0.3 · mean(H)
  + 0.4 · position_weighted_mean(H, decay=0.995)
  + 0.2 · std(H)
  + 0.9 · |{t : H_t > 2.0}| / n
  − 0.1 · longest_run(H_t < 0.5) / n
The components emphasise different aspects of the confidence signal. Position weighting (40% of the score, exponential decay 0.995 favouring recent tokens) reflects an empirical observation: mathematical solutions exhibit exploratory uncertainty early but should converge to confident conclusions, and the final-answer tokens carry most of the signal. The high-entropy-fraction term (
r_high
, threshold 2.0 bits) penalises attempts with extended uncertain stretches even when the overall mean is moderate. The low-entropy-streak bonus (
s_low
, threshold 0.5 bits) is a small reward for sustained-confidence regions.
The vote weight for each attempt is
1 / max(E, ε)
. Lower E means more confident, contributes more weight.
Ablation against a simple-mean-entropy baseline (V136 vs V125) showed a 6-point regression outside the noise band, the largest defensible single-axis result we have.
![](assets/b425a70b6724148ec1408fd0.png)
Figure 2: The 5-component entropy mechanism in action on real V143 attempt logs. Left panel: all 8 V143 attempts on problem 86e8e5 from the reference reproduction, showing the per-attempt extracted answer and the 5-component entropy score E. Every attempt produced a different answer (50505, 54680, 74763, 8687, 51379, 41754, 93764, 13657), so there was no consensus to vote on; attempt 3 (green) was selected because its E of 0.500 was the lowest, and its answer 8687 was the reference value. Right panel: the per-component decomposition of the winning attempt versus the runner-up (attempt 5, ans=41754, E=0.514). The winning margin of 0.014 is driven by the position-weighted term (40% of the score, favouring confident final-answer tokens) and the standard-deviation term (20%, penalising inconsistent confidence).
3.4 Prompting Strategy
Our best-performing configuration (V40, 42/50) uses a three-sentence system prompt: identify the model as an IMO competitor, specify the answer range, require the boxed format. Adding structured problem-solving frameworks, verification checklists, or detailed mathematical guidance consistently regressed the score by 4–9 points. We hypothesise three contributing factors: (a) GPT-OSS-120B was trained on extensive mathematical content and already possesses effective problem-solving strategies that elaborate prompts may override; (b) prompt tokens displace solution tokens within a fixed context window; (c) detailed instructions can introduce conflicting guidance.
4. Experimental Results
4.1 Overall Performance
Table 1 summarises our key submissions on the
hidden 50-problem competition test set
. V40 was our best at 42/50; V125, V132, V133, V143 established a reproducible 41/50 baseline.
Post-competition reference reproduction.
To address the reproducibility question and to provide per-problem evidence for the analyses in §4.2 and §4.4, we re-ran the V143 configuration on the public 10-problem AIMO3 reference set after the competition closed. The run used a single NVIDIA H100 80GB SXM rented from RunPod (AP-IN-1 data centre, $2.99/hr) with the standalone runner published in our
runpod/
directory. Result:
10/10 correct in 1869 seconds (31 minutes)
, mean wall time 187 s per problem, total spend $5.50. The pipeline generated 1.2 million response tokens, made 1056 Python tool invocations, recorded 202 Python errors (which the voting absorbed without issue), and stored the five entropy components, completion status, and timing for each of 80 attempts.
This is the first end-to-end reproducible evidence we can publish; the hidden 50-problem test set is closed post-competition and cannot be re-scored externally.
![](assets/56e7a40d8887c5a4fa839928.png)
Figure 3: Score on the hidden 50-problem test set for every submission from V8 to V158 (January to April 2026). The shaded band is the ±3.5-point variance observed across re-runs of the V143 configuration; the dashed line marks the V143 baseline (41/50) and the dotted line marks our V40 ceiling (42/50). Points are coloured by outcome class: V40 (dark green) is our only run above the baseline band; V125 / V132 / V133 / V143 (mid green) sit on the baseline; light green points are above the baseline but inside the band; gray points are within the noise band; light purple are mild regressions (V126 / V127 / V128 / V131 / V134 / V135 / V139 / V141 / V144); dark purple are severe regressions (V124, V142). The orange X markers along the bottom are OOM or crashed runs (14 of them). Only V142 (-11) and V127 (-9) clearly escape the band downward, and no run cleanly clears V40 upward.
Table 1: Key Versions on the Hidden Test Set
Version
Date
Score
Configuration
V40
Feb 6
42/50
Simple prompts, temp=1.0, 5-comp entropy (best ever)
V111
Mar 2
40/50
5-comp entropy isolated
V125
Mar 19
41/50
temp=1.0 + top_p=0.8
V132
Mar 26
41/50
No top_p, 5-comp entropy
V143
Apr 6
41/50
Stable baseline restored
V127
Mar 22
33/50
Verbose prompts (-9)
V128
Mar 24
35/50
Fine-tuned model (-6)
V131
Mar 25
35/50
6 techniques bundled (-6)
V135
Mar 29
35/50
131K context (-6, confounded see §4.2)
V142
Apr 5
30/50
Bayesian voting + VOI (-11)
4.2 Ablation Studies (Single-Run Snapshots)
These comparisons are single-run snapshots drawn from production iteration over three months. Identical configurations exhibited ±3-4 point variance across re-runs (see §6), so deltas smaller than roughly 4 points should be treated as suggestive rather than significant. Larger deltas (8+) are confidently outside noise. We did not have time during the competition to run controlled multi-seed ablations; we acknowledge this as a limitation and have since added the per-attempt instrumentation that would enable it.
![](assets/6bf25e3ba3de38ac0a9667f3.png)
Figure 4: Four-panel ablation summary, with every data point labelled by the V-number that produced it. (a) Temperature: V41/V139 at 0.5 scored 34/50, V112 at 0.7 scored 38, V146/V147 at 0.8 scored 40, V125/V143 at 1.0 scored 41. Only temp=0.5 (-7) clears the variance band. (b) Attempt count: V40/V125/V143 at 8 attempts scored 41; V144 at 12 attempts regressed to 35. (c) Voting strategy: V125/V143 (5-component inverse entropy) at 41/50; V136 (simple-mean entropy) at 35; V100/V148 (GenSelect) at 40; V142 (Bayesian + VOI) at 30 the only delta unambiguously outside the variance band. (d) Context length: V125 at 65K scored 41, V135 at 131K scored 35, with the twin axis showing vLLM max concurrency falling from 7.11x to 3.35x at the same
max_num_seqs
. The 96K intermediate (white circle) was not tested; it is the natural follow-up to disentangle "capacity hurt" from "throughput hurt" in (d).
Temperature.
Higher temperature produced better aggregate scores.
temp=1.0
(V40, V125) gave 41–42;
temp=0.5
(V41, V139) gave 34. An 8-point gap, comfortably outside the noise band. The mechanism is straightforward: voting benefits from diverse candidate answers, and low temperature causes the 8 attempts to converge on the same answer (sometimes wrong), leaving no correct alternative for the vote to find.
Context length confounded.
Doubling the context window from 65K (V125) to 131K (V135) dropped the aggregate score from 41/50 to 35/50. Earlier versions of this writeup framed this as "context hurts"; that framing was overreach. The vLLM server logs show that at fixed
max_num_seqs=256
, max concurrency fell from 7.11x at 65K to 3.35x at 131K effectively halving the parallel attempt pool. For a pipeline whose accuracy comes out of an 8-way vote, halving the pool can on its own cost 4–6 points before any individual problem becomes harder or easier. We cannot from a single aggregate score per submission disentangle "capacity loss" from "throughput loss." The honest conclusion: with batch-matched effective parallelism not held constant, we are not measuring context-window effects alone. A controlled comparison (65K @ batch=256 vs 131K @ batch=128, plus an intermediate 96K @ batch=192, with multiple seeds per cell) would resolve the question. We did not run this within the competition window.
Entropy weighting.
Replacing the 5-component weighted entropy with simple mean entropy (V136) gave 35/50 against 41/50 baseline (V125), a 6-point regression. The position-weighted term carries most of this: mean entropy treats exploratory uncertainty and answer uncertainty symmetrically, diluting the signal that matters.
Prompt complexity.
Verbose multi-paragraph prompts with explicit problem-solving frameworks (V127, 33/50) scored 9 points below the same configuration with a three-line system prompt (V40, 42/50). This is one of the larger isolated effects we measured. Distinguishing the three mechanisms hypothesised in §3.4 (override of learned strategies, token displacement, conflicting guidance) would require controlled prompt-length and prompt-content ablations we did not complete.
4.3 Failed Experiments
Several intuitively appealing approaches produced negative results.
Self-refinement.
V134 added a "review your answer" step after each attempt. -4 points. The model corrected correct answers into wrong ones with alarming frequency, consistent with Huang et al. 2023.
Answer verification.
V133 asked the model to verify each candidate answer with a CORRECT/WRONG decision. No improvement (matched V132 at 41/50). The model was equally willing to verify wrong answers as correct ones.
Bayesian voting + VOI.
V142 replaced the inverse-entropy aggregation with a Bayesian posterior plus Value-of-Information early-stopping. -11 points, the worst regression we observed. The complex aggregation over-penalised uncertainty in ways uncorrelated with actual correctness.
Fine-tuned model swap.
V128 used huikang/gpt-oss-120b-aimo3 without its matching inference pipeline. -6 points. Fine-tuned weights are optimised for a specific deployment context; swapping in the weights without the rest of the pipeline breaks the optimisation.
Bundled changes.
V131 combined six promising techniques simultaneously. -6 points. We learned to test one variable at a time after this.
4.4 Per-Attempt Instrumentation: Real Voting Behaviour
The single-aggregate-score nature of competition submissions made many ablations difficult to interpret. After the competition we added per-attempt CSV logging that captures, for every attempt of every problem: response token count, completion status, timeout flag, the five entropy components individually, Python tool calls and errors, elapsed wall time, and the final voting weight. The instrumented V143 run on the 10-problem reference set produced 80 attempt rows; the patterns visible in this data are:
Consensus regimes.
Eight of ten reference problems reached complete consensus among the attempts that completed (all completed attempts returned the same answer; early-stop typically triggered after 4 of 8). Two problems (P3, P10) had one outlier answer among the completed attempts; the entropy-weighted majority still selected correctly. One problem
86e8e5
, our hardest produced eight different answers from eight attempts, none agreeing. The 5-component entropy correctly identified the single attempt whose final-answer tokens were most confident; that attempt's answer (8687) was the reference value. The margin over the runner-up was 0.014 entropy units.
![](assets/abfad91401f658b034b86b7d.png)
Figure 5: Vote-weight distribution per problem from the V143 reference run (10/10 correct, 31 minutes on an H100 80GB SXM). Each panel is one of the ten reference problems with the reference answer in the title; each bar is a distinct answer produced by some attempt, height is the summed inverse-entropy vote weight, count is the number of attempts that produced that answer. Green bars are the selected (correct) answers; purple bars are wrong answers. Eight problems (P1, P2, P4, P5, P7, P8, P9) show a single green bar and unanimous consensus among completed attempts; two problems (P3, P10) have a single purple outlier next to the dominant green; only P6 (86e8e5, highlighted in purple in its title) has eight distinct purple-and-one-green bars, meaning the winning margin came from confidence alone rather than agreement.
Throughput band.
Figure 6 plots attempt elapsed time against tokens generated. The points cluster on a near-linear band of approximately 100 tokens per second, consistent with vLLM's throughput on a single H100 at this batch size and context length. Hard problems (
86e8e5
,
641659
) generate 3–5× more tokens per attempt and consume the full deadline budget; easy problems finish well under a minute.
![](assets/8ae418e1a37e55213abb9aea.png)
Figure 6: V143 per-attempt throughput on the H100 80GB SXM every one of the 80 attempts from the reference reproduction plotted as elapsed wall time vs response tokens generated, coloured by problem. Circles are attempts that produced a
\boxed{}
answer; X markers are attempts cancelled by early-stop or terminated at the deadline. The dashed gray line is the fitted throughput band at ~120 tokens/second, consistent with vLLM's reported max concurrency for our 65K-context / fp8_e4m3-KV-cache setup. P6 (brown, top right) is the clear outlier: 8 attempts in the 33-59K token range, all of which ran to the 900-second deadline the same problem that produced the 8 unique answers in Figures 2 and 5.
Limitations of this reproduction.
It is a single run, so it does not give us a variance estimate on V143's reference-set score (though the unanimous-vote pattern on 8/10 problems suggests it would be stable). A controlled 65K-vs-131K comparison under matched effective parallelism which would directly answer the question from §4.2 remains future work. The reference set is 10 problems; the hidden test set is 50, and that score cannot be recomputed.
5. Analysis and Discussion
5.1 Why Simple Prompts Win
Three contributing factors are consistent with our data. First, GPT-OSS-120B was trained on extensive mathematical content and already possesses effective problem-solving strategies; elaborate prompts can override these. Second, verbose prompts displace solution tokens within a fixed context. Third, detailed instructions can introduce conflicting guidance that fights the model's existing dispositions.
5.2 Diversity Drives Voting
The temperature-1.0 win over 0.5 reflects a general principle for ensemble systems: diversity among candidates matters more than the accuracy of any individual candidate, up to a point. Low temperature causes premature convergence; the 8 attempts make the same mistake together. High temperature without a confidence filter would produce noise. The combination temperature 1.0 with min_p=0.02 and inverse-entropy voting gets us both diversity and selection.
5.3 Position-Weighted Confidence
The 5-component score substantially outperforms simple averaging because mathematical reasoning has heterogeneous confidence patterns across a response. Early steps are exploratory and naturally uncertain; the final answer should be confident when correct. The position-weighted term captures this; simple averaging dilutes it.
5.4 The Gap to the Top of the Leaderboard
The winner (ippeiogawa) achieved 46/50, four points above us. A cluster of teams sat at 44–45. Public 44/50 notebooks suggest those top configurations use temperature 0.5 combined with simpler entropy calculations the opposite of what works for us. We can't resolve this fully without more controlled experiments, but the most likely explanation is that the optimal temperature depends on the other pipeline components, particularly the entropy weighting scheme. With a simpler aggregation, you may need lower temperature to keep the candidates close to correct, whereas a 5-component score can tolerate higher diversity.
6. Lessons Learned
A few practical guidelines from this work:
Test one variable at a time.
V131 bundled six "improvements" and regressed -6. The interactions between changes are unpredictable enough that any compound delta is unreadable.
Respect the variance band.
The same configuration produced scores varying by 3–4 points across re-runs on the hidden test set. Reading any single submission as evidence in either direction requires that the delta exceed this band. Most published "improvements" in our space don't.
Capable models don't need verbose prompts.
Our biggest measurable wins were from removing things, not adding them.
Match the inference pipeline to the model.
Fine-tuned weights from someone else's pipeline broke our score. The same surely applies in reverse.
Be skeptical of clever ideas.
Bayesian voting, GenSelect, CISC, VOI, self-refinement, answer verification every one of these regressed or didn't help. Simple inverse-entropy weighted voting beat them all.
Instrument early.
The instrumentation we added after the competition would have answered most of the questions our ablation tables couldn't.
7. Reproducibility
The V143 configuration is the canonical baseline for everything reported here. To reproduce the post-competition 10/10 reference-set result on independent hardware:
Clone the repository and
cd runpod/
.
Provision an NVIDIA H100 80GB or A100 80GB instance with 100 GB of attached storage.
Run
bash setup.sh
(~35 min): installs vLLM 0.16.0, PyTorch 2.9.1+cu128, openai-harmony, jupyter_client, plus downloads
openai/gpt-oss-120b
with
ignore_patterns=['metal/*','original/*']
to skip 135 GB of unneeded variants.
Run
bash start_server.sh
in one terminal and wait for
Application startup complete
.
Run
bash run.sh
in another terminal. Full 10-problem run completes in ~31 min on an H100 80GB; expect ~50 min on an A100 80GB.
Outputs land in
/workspace/results/
:
submission.csv
- id, answer per problem
scored.csv
- adds the reference answer and a correctness flag
per_attempt_log.csv
- 80 rows, 16 columns, schema defined in §4.4
run_summary.txt
- score, wall time, mean time per problem
All sampling is seeded (CFG seed = 42; per-attempt seeds are
(42 + i)²
). On the same vLLM version and hardware the results are byte-exact reproducible; we verified this by replaying the first five problems in a smoke run before the full reproduction (all five matched bit-for-bit). The V143 configuration as deployed on Kaggle scored 41/50 on the hidden 50-problem test set during the competition; that score cannot be recomputed externally.
8. Conclusion
Three months and 60+ versions produced one durable insight that I didn't expect going in: simple beats sophisticated, almost everywhere we measured. Three-line prompts beat verbose ones by 9 points. Inverse-entropy voting beat Bayesian aggregation by 11. Standard sampling beat low-temperature settings by 4–8. Adding verification, refinement, or longer context regressed the score. The model already knows how to do mathematics; our job was to sample effectively from it and trust the confidence signal in its own probabilities.
The four-point gap to the top of the leaderboard remains open. Public top solutions appear to use a different temperature/voting pair from ours, suggesting that the optimum is path-dependent on the rest of the pipeline. Closing that gap would likely require either MCTS-style search over solution steps, problem-classification routing to different configurations, or an honest ensemble across multiple model architectures.
For the broader community of practitioners working on inference-time scaling, the catalog of failures here is probably more valuable than the 42/50 headline. We've documented every clever idea we tried that didn't work, with deltas calibrated against the noise band, so the next team can skip what we already tested.
Acknowledgments
Thanks to OpenAI and Daniel Hanchen for the GPT-OSS-120B model; to the AIMO Prize Foundation for organising the competition; to the Kaggle community particularly nihilisticneuralnet and huikang whose public notebooks shaped our search space; and to Sam Bealing for the careful review and the questions that pushed me to instrument what I should have instrumented months ago.
References
AIMO Prize Foundation. AI Mathematical Olympiad Progress Prize 3. Kaggle Competition, 2026.
[https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3](https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3)
Wei, J., Wang, X., Schuurmans, D., et al. Chain-of-thought prompting elicits reasoning in large language models.
NeurIPS
35, 2022.
Wang, X., Wei, J., Schuurmans, D., et al. Self-consistency improves chain of thought reasoning in language models.
ICLR
, 2023.
Snell, C., Lee, J., Xu, K., Kumar, A. Scaling LLM test-time compute optimally can be more effective than scaling model parameters. arXiv:2408.03314, 2024.
Brown, B., Juravsky, J., Ehrlich, R., et al. Large language monkeys: Scaling inference compute with repeated sampling. arXiv:2407.21787, 2024.
Huang, J., Chen, X., Mishra, S., et al. Large language models cannot self-correct reasoning yet.
ICLR
, 2024.
Kwon, W., Li, Z., Zhuang, S., et al. Efficient memory management for large language model serving with PagedAttention.
SOSP
, 2023.
Madaan, A., Tandon, N., Gupta, P., et al. Self-Refine: Iterative refinement with self-feedback.
NeurIPS
36, 2023.
Cobbe, K., Kosaraju, V., Bavarian, M., et al. Training verifiers to solve math word problems. arXiv:2110.14168, 2021.
Hendrycks, D., Burns, C., Kadavath, S., et al. Measuring mathematical problem solving with the MATH dataset.
NeurIPS Datasets and Benchmarks
, 2021.
OpenAI. GPT-OSS-120B.
[https://huggingface.co/openai/gpt-oss-120b](https://huggingface.co/openai/gpt-oss-120b)
Toshniwal, S., Du, W., Moshkov, I., et al. OpenMathInstruct-2. arXiv:2410.01560, 2024.
AIMO Prize Foundation. AIMO Progress Prize 2 winning solutions. Numina, 2024.
nihilisticneuralnet. AIMO3 Solution Notebook (44/50). Kaggle, 2026.
huikang. Streaming inference with tool calling for mathematical reasoning. Kaggle, 2026.
datasciencegrad. Stable 42/50 AIMO3 solution. Kaggle, 2026.
Supplementary Materials
Per-attempt CSV from the H100 reference reproduction
: see
runpod_results/results/per_attempt_log.csv
in the linked repository
Standalone runner (Kaggle-free)
:
runpod/run_v143.py
Companion notebook with full version history
:
[Chasing 47/50: AIMO3 Journey of 60+ Experiments](https://www.kaggle.com/code/pawanmali/chasing-47-50-aimo3-journey-of-50-experiments)
Original implementation code
:
[AIMO3 GPT-OSS-120B](https://www.kaggle.com/code/pawanmali/aimo3-gpt-oss-120b)
Updated post-rebuttal, May 2026.