Team: techtech69 | Public Leaderboard Score: 43/50
\nWe describe our submission to the AI Mathematical Olympiad Progress Prize 3 (AIMO3), a competition requiring automated solutions to 50 olympiad-style integer-answer mathematics problems within a five-hour compute budget on a single NVIDIA H100 GPU. Our approach uses the gpt-oss-120b model served via vLLM, with 8 parallel sampling attempts per problem, entropy-weighted majority voting across attempts, and a tool-augmented reasoning loop that allows the model to execute Python code for symbolic computation and numerical verification. Our system scores 43/50 on the public test set, with a mean of 39.1 across nine submissions and a peak of 43. This writeup documents the full system design, all ablation experiments, the root causes of each failure mode we encountered, and the precise relationship between our observed scores and the theoretical pass@k framework published by the competition hosts.
\nThe AIMO3 competition challenges participants to build systems that solve International Mathematical Olympiad-style problems. Each problem has an integer answer in the range [0, 99999]. Submissions are run twice on a private test set; a problem scores 1.0 if both runs agree and are correct, 0.5 if exactly one run is correct, and 0.0 if neither is correct. This two-run scoring scheme creates an important distinction between maximising single-run accuracy and maximising the probability that both runs agree on the correct answer.
\nThe competition allows any open-weight model released before March 15, 2026, and provides a single NVIDIA H100 85GB GPU with a five-hour wall-clock limit. Participants may use any model that fits within these constraints.
\nOur system is built around three design choices that emerged from careful analysis of the problem structure:
\nModel selection. We use gpt-oss-120b, a 120-billion-parameter dense model with a 65,536-token context window, which the competition hosts identify as \"Model B\" in their published pass@n benchmarks. The hosts report that this model achieves pass@8 approximately equal to 43 and pass@20 approximately equal to 47 on the public test set.
Parallel sampling with majority voting. Rather than running a single inference pass, we generate 8 independent answers per problem and select the answer with the highest entropy-weighted vote count. This converts a per-attempt accuracy of roughly 85% on easy problems and 5% on hard problems into a reliable consensus.
Tool-augmented reasoning. The model has access to a persistent Python execution environment via the openai_harmony tool-calling framework. This allows it to use sympy, mpmath, numpy, and Python's standard library for symbolic algebra, high-precision arithmetic, and combinatorial enumeration.
The central challenge we faced throughout the competition was identifying a way to increase the effective number of attempts per problem from 8 toward 20, which the host data indicates would raise the expected score from 43 to approximately 47. Every attempt to increase attempt count failed due to hardware constraints: the H100 has approximately 15-16 GB of KV cache headroom after loading the 65 GB model weights, limiting safe concurrent request count to approximately 8. Sequential attempts suffer from budget collapse when hard problems consume their full 900-second allocation early in the problem sequence.
\nWe serve gpt-oss-120b using vLLM 0.11.2 with the following configuration:
\nfp8_e4m3), weights loaded in native dtype (auto-detected as bfloat16)--no-enable-prefix-caching) to avoid a known gibberish-output bug on Blackwell (B200) GPUs with tensor parallelism greater than 1, as documented by the competition hostsTORCHAO_DISABLE_TRITON=1 is set to prevent a CUDA initialisation crash in the torchao library at vLLM import timeModel weights are preloaded into the Linux page cache before vLLM starts by reading every weight file in 1 GB chunks using a 16-thread executor. This reduces vLLM startup time from approximately 400 seconds to approximately 130 seconds, a saving of roughly 270 seconds that is available for problem-solving.
\nFor each problem, the system executes the following loop:
\nbudget = max(base_problem_timeout,\n min(high_problem_timeout, time_left - reserved))\nwhere time_left = notebook_limit - elapsed, reserved = (problems_remaining - 1) * base_problem_timeout, base_problem_timeout = 300, high_problem_timeout = 900, and notebook_limit = 17400.
Launch 8 parallel attempts using a ThreadPoolExecutor with max_workers=16. Each attempt runs independently with a different seed derived from seed^2 = (42 + attempt_index)^2.
Each attempt follows a multi-turn reasoning loop (up to 128 turns):
\n\\boxed{...} patterns and record the first valid integer in [0, 99999]After 5 of 8 attempts agree on the same answer (early_stop=5), set a stop event to cancel remaining futures and proceed to voting.
Select the final answer using entropy-weighted majority voting: compute the mean token entropy of each attempt's generation, weight each vote by 1 / max(entropy, 1e-9), and select the answer with the highest total weight.
Each of the 16 workers has an associated persistent Jupyter kernel. Kernels are initialised in parallel at startup and reused across problems. After each attempt completes, the kernel state is reset using %reset -f followed by re-importing math, numpy, sympy, itertools, collections, and mpmath with 64 decimal places of precision.
The persistent kernel pool eliminates import overhead for every tool call and preserves numpy and sympy state within a single attempt's multi-turn conversation.
\nThe _scan_for_answer function uses three sequential regex patterns:
\\boxed{[0-9,]+}: primary extraction, handles comma-separated numbers\\boxed{\\d+}: fallback for plain integersfinal answer is \\d+: last resort for answers stated in plain textA fallback scan over the last 1,500 characters of the full response is performed if no answer was found during streaming. This catches cases where the model states its answer after a final reasoning step that does not trigger the streaming scan.
\nWe conducted nine submission-level ablation experiments. Each ablation changed exactly one or two parameters from the baseline. Table 1 summarises all results.
\n| Configuration | \nScore | \nKey Change | \nRoot Cause of Failure | \n
|---|---|---|---|
| Baseline (temp=1.0, att=8, w=16, bpt=300) | \n43 peak, 39.1 avg | \nReference | \nn/a | \n
| temperature=0.6 | \n35 | \nTemp change | \nModel trained at temp=1.0; lower temp causes repetitive output and fewer solution paths | \n
| attempts=16, workers=16 | \n34 | \n2x concurrent | \n16 concurrent KV requests saturate the 15 GB KV budget; requests preempted mid-generation | \n
| early_stop=8 | \n37 | \nStop threshold | \nDoes not address root cause; variance comes from GPU non-determinism, not threshold | \n
| Prompt engineering | \n38 | \nSystem prompt | \nopenai_harmony ReasoningEffort.HIGH already injects chain-of-thought; redundant prompts conflict | \n
| Iterative rounds (new executor per round) | \n28 | \nRound structure | \nNew ThreadPoolExecutor per round; shutdown(wait=True) blocked 350s per problem on running threads | \n
| attempts=24, workers=24 | \n21 | \n3x concurrent | \n24 concurrent requests = 3x KV capacity; model preempted almost every request | \n
| attempts=24, workers=8, bpt=100 | \n22 | \nLow budget floor | \n100s budget generates only ~10,000 tokens; AIME problems require ~20,000 tokens to reach boxed answer | \n
| attempts=24, workers=8, bpt=200 | \n28 | \nMedium budget floor | \nBudget collapses when hard problems cluster early; reserved = 44 x 200 = 8,800s exceeds time_left | \n
| attempts=24, workers=8, bpt=300 | \n30 | \nHigh budget floor | \nSandbox timeout: after early_stop, newly released thread slots trigger sandbox_timeout=3s failure | \n
The most consequential hardware constraint is the available KV cache memory. gpt-oss-120b loads approximately 65 GB of weight data in FP8 format, leaving approximately 15-16 GB for KV cache. At context length 65,536, each concurrent request requires approximately 2 GB of KV cache storage. This limits safe concurrency to approximately 7-8 requests.
\nThe original baseline uses workers=16 with attempts=8, which results in exactly 8 concurrent vLLM requests (since 8 tasks fill all available thread slots before any complete). This is precisely at the safe boundary.
Our attempt with workers=24, attempts=24 placed 24 concurrent requests on a server that could accommodate 8, causing vLLM to preempt the majority of requests. This explains the catastrophic drop from 43 to 21.
The time budget formula creates a dynamic where hard problems (consuming their full 900-second allocation) progressively reduce the budget available to subsequent problems. With base_problem_timeout=300, the reserved time for 44 remaining problems is 13,200 seconds. After 6 hard problems have consumed 5,400 seconds of compute plus 700 seconds of setup, time_left = 11,300s and budget = max(300, min(900, 11300-13200)) = 300s.
With a 300-second budget, any second round of 8 parallel attempts (which would start at approximately T = 350 seconds into the problem) exceeds the deadline before producing any answers. This is why attempts=24 with the original bpt=300 still only achieves pass@8 in practice.
\nReducing base_problem_timeout to 200 or 100 alleviates the budget formula collapse but introduces a new failure mode: the minimum budget floor becomes too short for the model to generate a complete answer on medium-difficulty problems. At 100 seconds, the model generates approximately 10,000 tokens, which is insufficient for most AIME-level problems that require 15,000-30,000 tokens to reach a boxed answer. At 200 seconds, budget collapse still occurs when hard problems cluster at the beginning of the problem sequence in the random competition ordering.
Our attempt to run sequential rounds of 8 attempts within a problem's time budget scored 28 due to a Python concurrency bug. The implementation created a new ThreadPoolExecutor inside a while loop. After early_stop triggered on an easy problem, the code set a stop event and called executor.shutdown(wait=True). However, wait=True blocks until all currently running threads complete their work, including ongoing streaming requests to vLLM. For easy problems with early_stop=5, approximately 3 threads were still streaming responses of 200-400 seconds duration. Multiplied across 50 problems, this added up to thousands of seconds of wasted blocking time, causing the notebook to time out before solving the majority of problems.
The correct fix for sequential rounds is to use a single ThreadPoolExecutor for the entire problem, with the queue naturally managing which attempts run. However, as shown in Table 1, this still fails due to budget collapse unless the per-attempt time is substantially reduced, which requires a faster model.
The competition evaluates each submission by running the notebook twice and applying the following per-problem scoring:
\nThe expected score for a problem with per-run accuracy p is:
\nE[score] = p^2 * 1.0 + 2*p*(1-p) * 0.5 + (1-p)^2 * 0.0 = p\nThis means the expected total score equals the sum of per-run per-problem accuracies, which in turn equals the expected single-run score. The two-run mechanism does not change the expected value of the score; it only increases variance relative to a single-run evaluation.
\nThe observed variance in our scores (36 to 43 across 9 submissions, standard deviation 2.13) arises from two sources:
\nGPU floating-point non-determinism. When 8 requests are batched together in vLLM, the exact completion order depends on GPU scheduling. Different completion orders produce slightly different softmax outputs due to floating-point associativity, leading to different sampled tokens even with the same seed.
Early stopping stochasticity. When early_stop triggers after 5 of 8 attempts agree, the 3 remaining attempts (which might have produced different answers) are cancelled. The specific 5 that finish first varies between runs due to GPU scheduling.
The competition hosts published pass@n data for Model B (equivalent to gpt-oss-120b class) on the public test set. Table 2 reproduces this data alongside our theoretical predictions.
\n| n | \nHost-reported score | \nTheoretical (our model) | \n
|---|---|---|
| 1 | \n27 | \n27.5 | \n
| 3 | \n37 | \n38.1 | \n
| 5 | \n41 | \n41.3 | \n
| 8 | \n43 | \n43.2 | \n
| 20 | \n47 | \n46.4 | \n
| 100 | \n49.5 | \n49.9 | \n
Our theoretical model uses three difficulty tiers: easy (q=0.90, n=20 problems), medium (q=0.45, n=20 problems), and hard (q=0.05, n=10 problems), where q is the per-attempt probability of a correct answer. These parameters fit the host data well across the full range of n.
\nThe host data shows that pass@8 achieves approximately 43 and pass@20 achieves approximately 47. The gap of 4 points arises almost entirely from hard problems: at q=0.05, pass@8 = 0.337 while pass@20 = 0.642, an improvement of 0.305 per hard problem times 10 hard problems equals approximately 3 additional correct answers.
\nThis analysis makes clear that the path from 43 to 47 requires roughly 20 effective attempts on hard problems. On 1xH100 with gpt-oss-120b, the KV cache constraint limits us to 8 safe concurrent requests, and the budget formula prevents reliable sequential rounds. The 4-point gap is therefore a hardware constraint given this specific model and GPU, not a software issue.
\n| Model | \nProblems Solved (of 10) | \nNotes | \n
|---|---|---|
| GPT-5 Pro | \n10/10 | \nCommercial, not eligible | \n
| GPT-5.1 (high) | \n10/10 | \nCommercial, not eligible | \n
| DeepSeek-v3.1-terminus | \n9/10 | \n671B parameters, does not fit on 1x H100 | \n
| grok-4 | \n9/10 | \nCommercial, not eligible | \n
| Gemini-2.5-Pro | \n8/10 | \nCommercial, not eligible | \n
| Qwen3-Next-80B (262k context) | \n8/10 | \nOpen-weight, 3.9B active params | \n
| GPT-5 (high) | \n8/10 | \nCommercial | \n
| gpt-oss-120b (ours) | \n4/10 | \nOpen-weight, eligible, used in our submission | \n
| QWO-32B | \n3/10 | \nOpen-weight | \n
| R1-Distill-Llama-70B | \n3/10 | \nOpen-weight | \n
| Qwen3-Next-80B (30k context) | \n4/10 | \nSame model, reduced context | \n
| Claude-Sonnet-4.5 (32k) | \n4/10 | \nCommercial | \n
The reference chart reveals that gpt-oss-120b is significantly behind the frontier on hard olympiad problems. The most capable open-weight model for this task is Qwen3-Next-80B-A3B-Thinking, which achieves 8/10 on the hard reference problems at pass@3. This model has 80B total parameters but only 3.9B active parameters per token due to its sparse Mixture-of-Experts architecture, making it approximately 5x faster than gpt-oss-120b on an H100, while fitting within the 85GB VRAM budget.
\nThe difference between the 262k and 30k context versions of Qwen3-Next-80B is striking: the model with 30k context solves only 4/10 reference problems (identical to gpt-oss-120b), while the same model with 262k context solves 8/10. This demonstrates that extended reasoning context, not just model size, is critical for hard olympiad problems.
\nGiven the KV cache constraint (8 safe concurrent requests) and the budget formula (prevents sequential rounds after hard problems), the effective pass@k achievable with gpt-oss-120b on 1x H100 is approximately pass@8 on the first 6-8 problems and pass@5-8 on remaining problems, for a theoretical ceiling of approximately 43-44. Our peak score of 43 confirms this analysis.
\nThe baseline system is well-designed for the specific constraints of the competition. The entropy-weighted majority voting is more robust than simple plurality voting because it down-weights answers produced by high-entropy (uncertain) generations. The openai_harmony tool-calling framework integrates cleanly with the vLLM serving infrastructure. The persistent Jupyter kernel pool eliminates per-call import overhead. The budget formula correctly handles the tradeoff between giving hard problems adequate time and ensuring all problems receive at least a minimum allocation.
\nThe fundamental limitation is that gpt-oss-120b solves only 4 of the 10 hard AIMO3 reference problems even with unlimited attempts. The remaining 6 hard problems are simply outside the model's capability distribution at any reasonable pass@k. Increasing attempts from 8 to 20 would improve expected score from 43 to 47, but every approach to increasing attempts on this hardware either saturates the KV cache or causes budget collapse.
\nBased on the reference chart and the competition host's published pass@n data, the reliable path to the 47/50 threshold required for the Overall Progress Prize is one of:
\nUse Qwen3-Next-80B-A3B-Thinking with its native 262k context window. This model is eligible (released before March 15, 2026), fits on a single H100 (approximately 40 GB in FP8 plus small KV cache due to 3.9B active parameters), and solves 8/10 hard reference problems at pass@3 compared to gpt-oss-120b's 4/10.
Use multiple H100 GPUs with tensor parallelism, allowing safe concurrent requests to scale proportionally. 4x H100 with tensor_parallel_size=4 would enable 32 concurrent requests and pass@32 per problem within the same time budget.
Fine-tune gpt-oss-120b on high-quality olympiad solutions to improve its per-attempt accuracy on hard problems from q=0.05 to q=0.15 or higher. The competition provides access to 128 H100 GPUs for fine-tuning as part of its ecosystem offering.
Our submission achieves 43/50 on the AIMO3 public test set using gpt-oss-120b with parallel sampling and entropy-weighted majority voting. The system is clean, reproducible, and operates reliably within the hardware constraints. Our extensive ablation study identifies the KV cache limitation (maximum 8 safe concurrent requests on 1x H100 with gpt-oss-120b) as the binding constraint preventing reliable pass@20 performance.
\nThe competition host's published pass@n data confirms that the gap between our 43 and the 47 threshold is achievable with the same model given more compute, or with Qwen3-Next-80B given the same compute. We document all failure modes in detail so that future participants can avoid the same pitfalls.
\nThe complete submission code is reproduced below. The system requires the following Kaggle datasets:
\n/kaggle/input/gpt-oss-120b/transformers/default/1: model weights/kaggle/input/aimo-3-utils/wheels.tar.gz: vLLM and openai_harmony wheels/kaggle/input/ai-mathematical-olympiad-progress-prize-3/test.csv: competition test set%pip uninstall --yes 'keras' 'matplotlib' 'scikit-learn' 'tensorflow'\nimport warnings\nwarnings.simplefilter('ignore')\nimport os\nimport sys\nimport subprocess\n\ndef set_env(input_archive, temp_dir):\n if not os.path.exists(temp_dir):\n os.makedirs(temp_dir, exist_ok=True)\n subprocess.run(['tar', '-xzf', input_archive, '-C', temp_dir], check=True)\n subprocess.run([\n sys.executable, '-m', 'pip', 'install', '--no-index',\n '--find-links', f'{temp_dir}/wheels',\n 'unsloth', 'trl', 'vllm', 'openai_harmony'\n ], check=True)\n\nset_env(\n input_archive='/kaggle/input/aimo-3-utils/wheels.tar.gz',\n temp_dir='/kaggle/tmp/setup'\n)\nsubprocess.run(['ls', '/kaggle/tmp/setup/tiktoken_encodings'])\n\nos.environ['TRANSFORMERS_NO_TF'] = '1'\nos.environ['TRANSFORMERS_NO_FLAX'] = '1'\nos.environ['CUDA_VISIBLE_DEVICES'] = '0'\nos.environ['TOKENIZERS_PARALLELISM'] = 'false'\nos.environ['TRITON_PTXAS_PATH'] = '/usr/local/cuda/bin/ptxas'\nos.environ['TIKTOKEN_ENCODINGS_BASE'] = '/kaggle/tmp/setup/tiktoken_encodings'\n\n# CFG: all key parameters documented inline\nclass CFG:\n system_prompt = (\n 'You are a world-class International Mathematical Olympiad (IMO) competitor. '\n 'The final answer must be a non-negative integer between 0 and 99999. '\n 'You must place the final integer answer inside \\\\boxed{}.'\n )\n tool_prompt = (\n 'Use this tool to execute Python code. '\n 'The environment is a stateful Jupyter notebook. '\n 'You must use print() to output results.'\n )\n preference_prompt = (\n 'You have access to math, numpy and sympy in Python to solve the problem, '\n 'but check their version first before starting to use them.'\n )\n served_model_name = 'gpt-oss'\n model_path = '/kaggle/input/gpt-oss-120b/transformers/default/1'\n kv_cache_dtype = 'fp8_e4m3'\n dtype = 'auto'\n high_problem_timeout = 900 # maximum seconds per problem\n base_problem_timeout = 300 # minimum seconds reserved per remaining problem\n notebook_limit = 17400 # total seconds available (5 hours minus buffer)\n server_timeout = 180 # seconds to wait for vLLM startup\n session_timeout = 960 # OpenAI client timeout\n jupyter_timeout = 10 # seconds per code execution\n sandbox_timeout = 3 # seconds to wait for sandbox from pool\n stream_interval = 200 # vLLM streaming chunk interval\n context_tokens = 65536 # maximum context length\n buffer_tokens = 512 # minimum tokens before refusing new turn\n search_tokens = 32 # chunks to scan back for boxed answer\n top_logprobs = 5 # logprobs per token for entropy computation\n batch_size = 256 # vLLM max-num-seqs\n temperature = 1.0 # sampling temperature (model tuned at 1.0)\n min_p = 0.02 # minimum probability filter\n early_stop = 5 # votes needed to stop early\n attempts = 8 # parallel attempts per problem\n workers = 16 # thread pool workers\n turns = 128 # max tool-call turns per attempt\n seed = 42 # base random seed\n gpu_memory_utilization = 0.96 # fraction of GPU memory for vLLM\nThe notebook startup sequence consumes approximately 700 seconds:
\n| Phase | \nDuration | \nCumulative | \n
|---|---|---|
| pip install from local wheels | \n90s | \n90s | \n
| Model weight preload (16 threads, 65 GB) | \n80-100s | \n190s | \n
| vLLM server startup | \n120-140s | \n330s | \n
| Jupyter kernel pool init (16 kernels) | \n3s | \n333s | \n
| Buffer/overhead | \n~370s | \n700s | \n
After startup, 16,700 seconds remain for 50 problems, giving an average of 334 seconds per problem.
\nThe budget formula ensures that all problems receive at least base_problem_timeout seconds, while the total wall-clock time stays within notebook_limit. The formula behaves as follows under different workload distributions:
All easy problems (each takes 20s, early_stop fires quickly):\nEvery problem receives budget=900s but uses only 20s. Total time: 700s setup + 50 x 20s = 1,700s. Well within the limit.
\nAll hard problems (each uses full 900s):\nProblems 1-45 receive budget=900s. Problems 46-50 receive budget=max(300, min(900, time_left - reserved)). After 45 hard problems, elapsed = 700 + 45 x 900 = 41,200s, which exceeds the 17,400s limit. In practice, vLLM will enforce the deadline by cutting off streaming, so each problem actually uses at most its allocated budget. The formula prevents later problems from being starved.
\nMixed workload (realistic competition scenario):\nWith approximately 6 very hard, 14 hard, 20 medium, and 10 easy problems in random order, the total compute time is approximately 6 x 900 + 14 x 350 + 20 x 100 + 10 x 20 = 5,400 + 4,900 + 2,000 + 200 = 12,500s plus 700s setup = 13,200s. This fits within the 17,400s limit.
\ngpt-oss-120b architecture estimate:
\nThis is clearly much larger than the available memory, which means vLLM allocates a fixed pool and limits concurrent requests accordingly. With 0.96 GPU memory utilization and 65 GB weights, approximately 16 GB is available for KV cache. At 2.5 MB per token, vLLM can support approximately 6,400 concurrent tokens, which corresponds to approximately 7-8 requests at typical generation lengths. This explains the observed limit of 8 safe concurrent requests.
\nChange: temperature = 1.0 -> 0.6
Expected outcome: More consistent answers across runs, reduced variance.
\nActual outcome: Score dropped from 43 to 35, losing 8 problems compared to the baseline.
\nRoot cause: gpt-oss-120b was trained and fine-tuned with temperature=1.0 for mathematical reasoning tasks. At temperature=0.6, the model's sampling distribution is sharpened, causing:
\nThe lesson is that temperature is not a free parameter for aligned models; it must match the distribution used during RLHF fine-tuning.
\nChange: attempts = 8 -> 16, workers = 16
Expected outcome: 16 parallel attempts, wider solution space coverage.
\nActual outcome: Score dropped to 34.
\nRoot cause: 16 concurrent requests x approximately 2 GB KV cache each = 32 GB, which exceeds the available 15-16 GB. vLLM's preemption policy aborts generation for lower-priority requests when KV memory is exhausted. Most of the 16 requests were preempted before generating a boxed answer, reducing the effective pass@k to approximately pass@3 or pass@4.
\nChange: early_stop = 5 -> 8
Expected outcome: More stable voting, reduced run-to-run variance.
\nActual outcome: Score dropped to 37.
\nRoot cause: The actual source of variance is not the early_stop threshold but GPU floating-point non-determinism in batched execution. Setting early_stop=8 (equal to attempts=8) means early stopping never triggers, so all 8 attempts always complete. This slightly reduces performance on easy problems (where all 8 were already agreeing before the 5th completed) and has no effect on hard problems. The slight score reduction suggests that waiting for all 8 attempts on easy problems occasionally leads to a wrong late answer being included in the vote with high weight.
\nChange: Sequential rounds of 8 attempts using a while loop with new ThreadPoolExecutor per round.
Expected outcome: Multiple rounds of 8 attempts within the 900s budget, achieving approximately 20 effective attempts on hard problems.
\nActual outcome: Score dropped to 28.
\nRoot cause: The implementation created a new ThreadPoolExecutor(max_workers=16) inside the while loop. When early_stop=5 triggered for an easy problem at approximately T=20 seconds, the code called stop_event.set() and then executor.shutdown(wait=True). The wait=True argument blocks until all currently running threads complete their work. For easy problems, 3 threads were still streaming responses, which take approximately 200-400 seconds to complete even after the stop event is checked. This 350-second blocking overhead per easy problem, multiplied across 50 problems, caused the notebook to run out of time after solving only approximately 20-25 problems.
The correct fix requires using a single long-lived executor for the entire problem duration and relying on f.cancel() plus the stop event check inside the streaming loop to abort running attempts quickly.
Change: attempts = 24, workers = 24
Expected outcome: 24 parallel attempts, achieving approximately pass@20 through wider coverage.
\nActual outcome: Score dropped to 21, the lowest score in our experiment series.
\nRoot cause: 24 concurrent KV cache allocations = approximately 48 GB, which is 3x the available 15-16 GB. vLLM preempted the vast majority of requests, with only approximately 3-4 completing per problem. This is worse than the original 8 attempts because the throughput per request is reduced due to memory pressure, and the voting pool is very small with high variance.
\nChange: base_problem_timeout = 100
Expected outcome: More budget available for earlier problems, enabling second rounds of 8 attempts.
\nActual outcome: Score dropped to 22.
\nRoot cause: A 100-second budget generates approximately 100s x 100 tok/s = 10,000 tokens. Analysis of the AIME-level problems in our reference set shows that the model requires approximately 15,000-30,000 tokens to reach a boxed answer. Problems receiving a 100-second budget return None from the answer extractor and default to 0, which is almost always wrong. The number of problems affected depends on problem ordering; with hard problems early, approximately 15-20 later problems received 100-second budgets.
Change: base_problem_timeout = 200, attempts = 24, workers = 8
Expected outcome: Budget floor high enough to generate complete answers, while the queueing mechanism provides extra attempts on hard problems.
\nActual outcome: Score dropped to 28.
\nRoot cause: A detailed simulation shows that budget collapse still occurs when hard problems (taking 900s each) come early in the random problem ordering. After 6 hard problems:
\nAfter 9 hard problems:
\nWith 400s budget and workers=8, the first round of 8 attempts completes at T=350s, leaving 50s for the second round. 50s is insufficient for the second batch of 8 attempts to generate answers on hard problems. Problems receiving 200-400s budgets with this configuration generate partial responses that occasionally produce answers but with high miss rates.
\n| Variable | \nValue | \nPurpose | \n
|---|---|---|
| TRANSFORMERS_NO_TF | \n1 | \nSuppress TensorFlow import | \n
| TRANSFORMERS_NO_FLAX | \n1 | \nSuppress Flax import | \n
| CUDA_VISIBLE_DEVICES | \n0 | \nSingle GPU operation | \n
| TOKENIZERS_PARALLELISM | \nfalse | \nPrevent tokenizer fork warnings | \n
| TRITON_PTXAS_PATH | \n/usr/local/cuda/bin/ptxas | \nTriton compilation target | \n
| TIKTOKEN_ENCODINGS_BASE | \n/kaggle/tmp/setup/tiktoken_encodings | \nLocal encoding cache | \n
| Argument | \nValue | \nRationale | \n
|---|---|---|
| --seed | \n42 | \nReproducible base randomness | \n
| --tensor-parallel-size | \n1 | \nSingle GPU | \n
| --max-num-seqs | \n256 | \nMaximum request queue size | \n
| --gpu-memory-utilization | \n0.96 | \nLeave 4% for other processes | \n
| --kv-cache-dtype | \nfp8_e4m3 | \nHalves KV memory relative to fp16 | \n
| --max-model-len | \n65536 | \nContext window | \n
| --stream-interval | \n200 | \nToken streaming granularity | \n
| --async-scheduling | \n(flag) | \nEnables asynchronous request scheduling | \n
| --no-enable-prefix-caching | \n(flag) | \nAvoids B200 gibberish bug | \n
| Parameter | \nValue | \nRationale | \n
|---|---|---|
| temperature | \n1.0 | \nMatches model's fine-tuning distribution | \n
| min_p | \n0.02 | \nFilters very low probability tokens | \n
| top_logprobs | \n5 | \nUsed for entropy computation | \n
| Per-attempt seed | \n(42 + i)^2 | \nDiverse seeds while reproducible | \n
Based on analysis of the 50 public test problems and comparison with the AIMO3 reference problem solutions:
\nVery hard (6 problems, q approximately 0.05): These problems require advanced combinatorial identities, Hermite's identity, Legendre's formula, spiral similarity, or deep number theory. gpt-oss-120b solves these approximately 5% of the time per attempt. Examples from the reference set include problems involving tournaments with 2^20 players, functional equations on integers with mod constraints, and problems requiring the Lifting the Exponent lemma.
\nHard (8-10 problems, q approximately 0.10-0.15): Problems requiring extended computation, case analysis, or non-obvious transformations. The model often identifies the correct approach but makes arithmetic errors or misses edge cases.
\nMedium (14-16 problems, q approximately 0.40-0.55): AIME-level problems where the model reliably identifies the solution approach and usually executes it correctly. Python code assistance is particularly valuable here for modular arithmetic and combinatorial counting.
\nEasy (18-22 problems, q approximately 0.85-0.95): Problems where the model identifies and solves the problem on the first attempt with high probability. Early stopping typically triggers within the first 5 attempts.
\nThe Python execution tool plays an essential role for three problem types:
\nExact large integer arithmetic. Python integers have arbitrary precision, making them ideal for problems involving factorials, binomial coefficients, and modular exponentiation at scales where floating-point precision is insufficient. The mpmath library with 64 decimal digits of precision handles high-precision floating-point needs.
Combinatorial verification. Problems involving counts of valid configurations are often solvable by direct enumeration for small cases, allowing the model to verify patterns before generalising. The model uses sympy for symbolic manipulation and itertools for combinatorial generation.
Polynomial and number-theoretic computation. Problems involving cyclotomic polynomials, Fibonacci sequences, digit sum functions, and similar structures benefit from symbolic computation in sympy, which can handle exact rational arithmetic and polynomial factorisation.
To reproduce our submission exactly:
\nCreate a Kaggle notebook with the following datasets attached:
\ngpt-oss-120b (model weights)aimo-3-utils (wheel packages)Set accelerator to GPU T4 x2 or P100 (for testing); the competition environment uses H100.
The notebook requires no external internet access; all packages are installed from the local wheel archive.
The complete set of parameters that define our submission:
\ntemperature = 1.0\nmin_p = 0.02\nattempts = 8\nworkers = 16\nearly_stop = 5\nturns = 128\nseed = 42\ngpu_memory_utilization = 0.96\nkv_cache_dtype = 'fp8_e4m3'\ncontext_tokens = 65536\nhigh_problem_timeout = 900\nbase_problem_timeout = 300\nnotebook_limit = 17400\nWhen run on the 3-problem calibration set (0x10, 1-1, 4+x=4):
\nWhen run on the full competition set, expect:
\nTotal runtime should be approximately 12,000-16,000 seconds depending on problem ordering.
\nDue to GPU floating-point non-determinism in batched execution, exact score reproduction is not guaranteed. The 36-43 range observed across nine submissions represents the natural variance of the system. A score in the range 36-43 should be considered a successful reproduction.
\n[1] Competition hosts. \"AIMO3 Public vs Private Problem Set Difficulty Comparison.\" Kaggle Discussion, 2025.
\n[2] Competition hosts. \"AIMO3 B200 and Prefix Caching Bug Fix Notice.\" Kaggle Discussion, 2025.
\n[3] Competition hosts. \"AIMO3 March 15 Cutoff Rule FAQ.\" Kaggle Discussion, 2025.
\n[4] Competition hosts. \"Welcome to AIMO3: IMO-level problems, H100s, ecosystem, extra prizes.\" Kaggle Discussion, 2025.
\n[5] AIMO Progress Prize 2 Reference Problems and Solutions. October 2024.
\n[6] NemoSkills team. \"NemoSkills: AIMO2 Winning Solution.\" arXiv:2504.16891, 2025.
\n[7] vLLM Project. \"vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention.\" SOSP, 2023.
\n[8] Qwen Team. \"Qwen3 Technical Report.\" arXiv:2505.09388, 2025.
\n[9] Qwen Team. \"Qwen3-Next Blog Post.\" Alibaba Cloud, September 2025.
\n[10] Chen, M. et al. \"Evaluating Large Language Models Trained on Code.\" arXiv:2107.03374, 2021. (pass@k framework)
\n[11] Lightman, H. et al. \"Let's Verify Step by Step.\" arXiv:2305.20050, 2023.
\n[12] openai_harmony documentation. Internal competition API reference, AIMO3.
\nThe following table serves as a data table for a pass@k performance curve. The x-axis is n (number of attempts), the y-axis is expected score out of 50.
\n| n (attempts) | \nExpected Score | \nSource | \n
|---|---|---|
| 1 | \n27.0 | \nHost-published | \n
| 3 | \n37.0 | \nHost-published | \n
| 5 | \n41.0 | \nHost-published | \n
| 8 | \n43.0 | \nHost-published / Our peak score | \n
| 16 | \n45.6 | \nTheoretical (q: easy=0.90, med=0.45, hard=0.05) | \n
| 20 | \n47.0 | \nHost-published | \n
| 24 | \n47.1 | \nTheoretical | \n
| 100 | \n49.5 | \nHost-published | \n
The curve demonstrates three distinct regimes: rapid improvement from 1 to 8 attempts (easy and medium problems saturate quickly), slower improvement from 8 to 20 attempts (hard problems still benefit), and near-plateau above 20 attempts (only very hard problems remain unsolved).
\nThe following table describes a waterfall chart showing score changes from each ablation:
\n| Ablation | \nScore | \nDelta from Baseline | \n
|---|---|---|
| Baseline (original 43-scorer) | \n43 (peak) / 39.1 (avg) | \n0 | \n
| temperature = 0.6 | \n35 | \n-8 | \n
| attempts=16, workers=16 | \n34 | \n-9 | \n
| early_stop=8 | \n37 | \n-6 | \n
| Prompt engineering | \n38 | \n-5 | \n
| Iterative rounds (executor bug) | \n28 | \n-15 | \n
| attempts=24, workers=24 (KV OOM) | \n21 | \n-22 | \n
| attempts=24, workers=8, bpt=100 (token timeout) | \n22 | \n-21 | \n
| attempts=24, workers=8, bpt=200 (budget collapse) | \n28 | \n-15 | \n
| attempts=24, workers=8, bpt=300 (sandbox timeout) | \n30 | \n-13 | \n
Every modification to the baseline decreased the score. The three most damaging changes (KV OOM at -22, token timeout at -21, iterative rounds executor bug at -15) all involved attempts to increase the number of attempts per problem beyond what the hardware safely supports.
\nThe following table illustrates the budget formula output under a worst-case scenario (all hard problems appearing first) with base_problem_timeout=300:
\n| Problem | \nElapsed After | \ntime_left | \nreserved | \nbudget | \n
|---|---|---|---|---|
| 1 (hard, 900s) | \n1600s | \n15800s | \n14700s | \n900s | \n
| 2 (hard, 900s) | \n2500s | \n14900s | \n14400s | \n500s | \n
| 3 (hard, 900s) | \n3400s | \n14000s | \n14100s | \n300s (floor) | \n
| 4 (hard, 900s) | \n4300s | \n13100s | \n13800s | \n300s (floor) | \n
| 5 (hard, 900s) | \n5200s | \n12200s | \n13500s | \n300s (floor) | \n
| 6 (medium, 300s) | \n5500s | \n11900s | \n13200s | \n300s (floor) | \n
| 7 (medium, 300s) | \n5800s | \n11600s | \n12900s | \n300s (floor) | \n
Once the reserved buffer exceeds the remaining time, the budget is clamped to the minimum floor for all subsequent problems, regardless of their difficulty. With base_problem_timeout=200, the collapse is slightly delayed but the same phenomenon occurs when hard problems are numerous early in the sequence.
\n| Component | \nMemory | \nNotes | \n
|---|---|---|
| gpt-oss-120b weights (FP8) | \n~65 GB | \nRead from model path | \n
| CUDA runtime and drivers | \n~2 GB | \nFixed overhead | \n
| PyTorch and vLLM overhead | \n~2 GB | \nActivation memory | \n
| Available for KV cache | \n~16 GB | \nAt 0.96 GPU utilization | \n
| KV per request (65k context, FP8) | \n~2 GB | \nPer-request allocation | \n
| Safe concurrent requests | \n~8 | \n16 GB / 2 GB | \n
This breakdown explains why the original attempts=8 configuration works reliably while attempts=16 and attempts=24 fail catastrophically.
\nOur submission uses only gpt-oss-120b, a model that was available and eligible well before the March 15, 2026 AMLT cutoff. No models released after March 15 are used at inference time. The openai_harmony library is a competition-provided tool. All Python packages are installed from the competition-provided wheel archive with no external downloads. The submission notebook contains no hard-coded answers or problem-specific logic; all answers are generated by the model at runtime.
\nTeam: techtech69 | Public Leaderboard Score: 43/50
\nWe describe our submission to the AI Mathematical Olympiad Progress Prize 3 (AIMO3), a competition requiring automated solutions to 50 olympiad-style integer-answer mathematics problems within a five-hour compute budget on a single NVIDIA H100 GPU. Our approach uses the gpt-oss-120b model served via vLLM, with 8 parallel sampling attempts per problem, entropy-weighted majority voting across attempts, and a tool-augmented reasoning loop that allows the model to execute Python code for symbolic computation and numerical verification. Our system scores 43/50 on the public test set, with a mean of 39.1 across nine submissions and a peak of 43. This writeup documents the full system design, all ablation experiments, the root causes of each failure mode we encountered, and the precise relationship between our observed scores and the theoretical pass@k framework published by the competition hosts.
\nThe AIMO3 competition challenges participants to build systems that solve International Mathematical Olympiad-style problems. Each problem has an integer answer in the range [0, 99999]. Submissions are run twice on a private test set; a problem scores 1.0 if both runs agree and are correct, 0.5 if exactly one run is correct, and 0.0 if neither is correct. This two-run scoring scheme creates an important distinction between maximising single-run accuracy and maximising the probability that both runs agree on the correct answer.
\nThe competition allows any open-weight model released before March 15, 2026, and provides a single NVIDIA H100 85GB GPU with a five-hour wall-clock limit. Participants may use any model that fits within these constraints.
\nOur system is built around three design choices that emerged from careful analysis of the problem structure:
\nModel selection. We use gpt-oss-120b, a 120-billion-parameter dense model with a 65,536-token context window, which the competition hosts identify as \"Model B\" in their published pass@n benchmarks. The hosts report that this model achieves pass@8 approximately equal to 43 and pass@20 approximately equal to 47 on the public test set.
Parallel sampling with majority voting. Rather than running a single inference pass, we generate 8 independent answers per problem and select the answer with the highest entropy-weighted vote count. This converts a per-attempt accuracy of roughly 85% on easy problems and 5% on hard problems into a reliable consensus.
Tool-augmented reasoning. The model has access to a persistent Python execution environment via the openai_harmony tool-calling framework. This allows it to use sympy, mpmath, numpy, and Python's standard library for symbolic algebra, high-precision arithmetic, and combinatorial enumeration.
The central challenge we faced throughout the competition was identifying a way to increase the effective number of attempts per problem from 8 toward 20, which the host data indicates would raise the expected score from 43 to approximately 47. Every attempt to increase attempt count failed due to hardware constraints: the H100 has approximately 15-16 GB of KV cache headroom after loading the 65 GB model weights, limiting safe concurrent request count to approximately 8. Sequential attempts suffer from budget collapse when hard problems consume their full 900-second allocation early in the problem sequence.
\nWe serve gpt-oss-120b using vLLM 0.11.2 with the following configuration:
\nfp8_e4m3), weights loaded in native dtype (auto-detected as bfloat16)--no-enable-prefix-caching) to avoid a known gibberish-output bug on Blackwell (B200) GPUs with tensor parallelism greater than 1, as documented by the competition hostsTORCHAO_DISABLE_TRITON=1 is set to prevent a CUDA initialisation crash in the torchao library at vLLM import timeModel weights are preloaded into the Linux page cache before vLLM starts by reading every weight file in 1 GB chunks using a 16-thread executor. This reduces vLLM startup time from approximately 400 seconds to approximately 130 seconds, a saving of roughly 270 seconds that is available for problem-solving.
\nFor each problem, the system executes the following loop:
\nbudget = max(base_problem_timeout,\n min(high_problem_timeout, time_left - reserved))\nwhere time_left = notebook_limit - elapsed, reserved = (problems_remaining - 1) * base_problem_timeout, base_problem_timeout = 300, high_problem_timeout = 900, and notebook_limit = 17400.
Launch 8 parallel attempts using a ThreadPoolExecutor with max_workers=16. Each attempt runs independently with a different seed derived from seed^2 = (42 + attempt_index)^2.
Each attempt follows a multi-turn reasoning loop (up to 128 turns):
\n\\boxed{...} patterns and record the first valid integer in [0, 99999]After 5 of 8 attempts agree on the same answer (early_stop=5), set a stop event to cancel remaining futures and proceed to voting.
Select the final answer using entropy-weighted majority voting: compute the mean token entropy of each attempt's generation, weight each vote by 1 / max(entropy, 1e-9), and select the answer with the highest total weight.
Each of the 16 workers has an associated persistent Jupyter kernel. Kernels are initialised in parallel at startup and reused across problems. After each attempt completes, the kernel state is reset using %reset -f followed by re-importing math, numpy, sympy, itertools, collections, and mpmath with 64 decimal places of precision.
The persistent kernel pool eliminates import overhead for every tool call and preserves numpy and sympy state within a single attempt's multi-turn conversation.
\nThe _scan_for_answer function uses three sequential regex patterns:
\\boxed{[0-9,]+}: primary extraction, handles comma-separated numbers\\boxed{\\d+}: fallback for plain integersfinal answer is \\d+: last resort for answers stated in plain textA fallback scan over the last 1,500 characters of the full response is performed if no answer was found during streaming. This catches cases where the model states its answer after a final reasoning step that does not trigger the streaming scan.
\nWe conducted nine submission-level ablation experiments. Each ablation changed exactly one or two parameters from the baseline. Table 1 summarises all results.
\n| Configuration | \nScore | \nKey Change | \nRoot Cause of Failure | \n
|---|---|---|---|
| Baseline (temp=1.0, att=8, w=16, bpt=300) | \n43 peak, 39.1 avg | \nReference | \nn/a | \n
| temperature=0.6 | \n35 | \nTemp change | \nModel trained at temp=1.0; lower temp causes repetitive output and fewer solution paths | \n
| attempts=16, workers=16 | \n34 | \n2x concurrent | \n16 concurrent KV requests saturate the 15 GB KV budget; requests preempted mid-generation | \n
| early_stop=8 | \n37 | \nStop threshold | \nDoes not address root cause; variance comes from GPU non-determinism, not threshold | \n
| Prompt engineering | \n38 | \nSystem prompt | \nopenai_harmony ReasoningEffort.HIGH already injects chain-of-thought; redundant prompts conflict | \n
| Iterative rounds (new executor per round) | \n28 | \nRound structure | \nNew ThreadPoolExecutor per round; shutdown(wait=True) blocked 350s per problem on running threads | \n
| attempts=24, workers=24 | \n21 | \n3x concurrent | \n24 concurrent requests = 3x KV capacity; model preempted almost every request | \n
| attempts=24, workers=8, bpt=100 | \n22 | \nLow budget floor | \n100s budget generates only ~10,000 tokens; AIME problems require ~20,000 tokens to reach boxed answer | \n
| attempts=24, workers=8, bpt=200 | \n28 | \nMedium budget floor | \nBudget collapses when hard problems cluster early; reserved = 44 x 200 = 8,800s exceeds time_left | \n
| attempts=24, workers=8, bpt=300 | \n30 | \nHigh budget floor | \nSandbox timeout: after early_stop, newly released thread slots trigger sandbox_timeout=3s failure | \n
The most consequential hardware constraint is the available KV cache memory. gpt-oss-120b loads approximately 65 GB of weight data in FP8 format, leaving approximately 15-16 GB for KV cache. At context length 65,536, each concurrent request requires approximately 2 GB of KV cache storage. This limits safe concurrency to approximately 7-8 requests.
\nThe original baseline uses workers=16 with attempts=8, which results in exactly 8 concurrent vLLM requests (since 8 tasks fill all available thread slots before any complete). This is precisely at the safe boundary.
Our attempt with workers=24, attempts=24 placed 24 concurrent requests on a server that could accommodate 8, causing vLLM to preempt the majority of requests. This explains the catastrophic drop from 43 to 21.
The time budget formula creates a dynamic where hard problems (consuming their full 900-second allocation) progressively reduce the budget available to subsequent problems. With base_problem_timeout=300, the reserved time for 44 remaining problems is 13,200 seconds. After 6 hard problems have consumed 5,400 seconds of compute plus 700 seconds of setup, time_left = 11,300s and budget = max(300, min(900, 11300-13200)) = 300s.
With a 300-second budget, any second round of 8 parallel attempts (which would start at approximately T = 350 seconds into the problem) exceeds the deadline before producing any answers. This is why attempts=24 with the original bpt=300 still only achieves pass@8 in practice.
\nReducing base_problem_timeout to 200 or 100 alleviates the budget formula collapse but introduces a new failure mode: the minimum budget floor becomes too short for the model to generate a complete answer on medium-difficulty problems. At 100 seconds, the model generates approximately 10,000 tokens, which is insufficient for most AIME-level problems that require 15,000-30,000 tokens to reach a boxed answer. At 200 seconds, budget collapse still occurs when hard problems cluster at the beginning of the problem sequence in the random competition ordering.
Our attempt to run sequential rounds of 8 attempts within a problem's time budget scored 28 due to a Python concurrency bug. The implementation created a new ThreadPoolExecutor inside a while loop. After early_stop triggered on an easy problem, the code set a stop event and called executor.shutdown(wait=True). However, wait=True blocks until all currently running threads complete their work, including ongoing streaming requests to vLLM. For easy problems with early_stop=5, approximately 3 threads were still streaming responses of 200-400 seconds duration. Multiplied across 50 problems, this added up to thousands of seconds of wasted blocking time, causing the notebook to time out before solving the majority of problems.
The correct fix for sequential rounds is to use a single ThreadPoolExecutor for the entire problem, with the queue naturally managing which attempts run. However, as shown in Table 1, this still fails due to budget collapse unless the per-attempt time is substantially reduced, which requires a faster model.
The competition evaluates each submission by running the notebook twice and applying the following per-problem scoring:
\nThe expected score for a problem with per-run accuracy p is:
\nE[score] = p^2 * 1.0 + 2*p*(1-p) * 0.5 + (1-p)^2 * 0.0 = p\nThis means the expected total score equals the sum of per-run per-problem accuracies, which in turn equals the expected single-run score. The two-run mechanism does not change the expected value of the score; it only increases variance relative to a single-run evaluation.
\nThe observed variance in our scores (36 to 43 across 9 submissions, standard deviation 2.13) arises from two sources:
\nGPU floating-point non-determinism. When 8 requests are batched together in vLLM, the exact completion order depends on GPU scheduling. Different completion orders produce slightly different softmax outputs due to floating-point associativity, leading to different sampled tokens even with the same seed.
Early stopping stochasticity. When early_stop triggers after 5 of 8 attempts agree, the 3 remaining attempts (which might have produced different answers) are cancelled. The specific 5 that finish first varies between runs due to GPU scheduling.
The competition hosts published pass@n data for Model B (equivalent to gpt-oss-120b class) on the public test set. Table 2 reproduces this data alongside our theoretical predictions.
\n| n | \nHost-reported score | \nTheoretical (our model) | \n
|---|---|---|
| 1 | \n27 | \n27.5 | \n
| 3 | \n37 | \n38.1 | \n
| 5 | \n41 | \n41.3 | \n
| 8 | \n43 | \n43.2 | \n
| 20 | \n47 | \n46.4 | \n
| 100 | \n49.5 | \n49.9 | \n
Our theoretical model uses three difficulty tiers: easy (q=0.90, n=20 problems), medium (q=0.45, n=20 problems), and hard (q=0.05, n=10 problems), where q is the per-attempt probability of a correct answer. These parameters fit the host data well across the full range of n.
\nThe host data shows that pass@8 achieves approximately 43 and pass@20 achieves approximately 47. The gap of 4 points arises almost entirely from hard problems: at q=0.05, pass@8 = 0.337 while pass@20 = 0.642, an improvement of 0.305 per hard problem times 10 hard problems equals approximately 3 additional correct answers.
\nThis analysis makes clear that the path from 43 to 47 requires roughly 20 effective attempts on hard problems. On 1xH100 with gpt-oss-120b, the KV cache constraint limits us to 8 safe concurrent requests, and the budget formula prevents reliable sequential rounds. The 4-point gap is therefore a hardware constraint given this specific model and GPU, not a software issue.
\n| Model | \nProblems Solved (of 10) | \nNotes | \n
|---|---|---|
| GPT-5 Pro | \n10/10 | \nCommercial, not eligible | \n
| GPT-5.1 (high) | \n10/10 | \nCommercial, not eligible | \n
| DeepSeek-v3.1-terminus | \n9/10 | \n671B parameters, does not fit on 1x H100 | \n
| grok-4 | \n9/10 | \nCommercial, not eligible | \n
| Gemini-2.5-Pro | \n8/10 | \nCommercial, not eligible | \n
| Qwen3-Next-80B (262k context) | \n8/10 | \nOpen-weight, 3.9B active params | \n
| GPT-5 (high) | \n8/10 | \nCommercial | \n
| gpt-oss-120b (ours) | \n4/10 | \nOpen-weight, eligible, used in our submission | \n
| QWO-32B | \n3/10 | \nOpen-weight | \n
| R1-Distill-Llama-70B | \n3/10 | \nOpen-weight | \n
| Qwen3-Next-80B (30k context) | \n4/10 | \nSame model, reduced context | \n
| Claude-Sonnet-4.5 (32k) | \n4/10 | \nCommercial | \n
The reference chart reveals that gpt-oss-120b is significantly behind the frontier on hard olympiad problems. The most capable open-weight model for this task is Qwen3-Next-80B-A3B-Thinking, which achieves 8/10 on the hard reference problems at pass@3. This model has 80B total parameters but only 3.9B active parameters per token due to its sparse Mixture-of-Experts architecture, making it approximately 5x faster than gpt-oss-120b on an H100, while fitting within the 85GB VRAM budget.
\nThe difference between the 262k and 30k context versions of Qwen3-Next-80B is striking: the model with 30k context solves only 4/10 reference problems (identical to gpt-oss-120b), while the same model with 262k context solves 8/10. This demonstrates that extended reasoning context, not just model size, is critical for hard olympiad problems.
\nGiven the KV cache constraint (8 safe concurrent requests) and the budget formula (prevents sequential rounds after hard problems), the effective pass@k achievable with gpt-oss-120b on 1x H100 is approximately pass@8 on the first 6-8 problems and pass@5-8 on remaining problems, for a theoretical ceiling of approximately 43-44. Our peak score of 43 confirms this analysis.
\nThe baseline system is well-designed for the specific constraints of the competition. The entropy-weighted majority voting is more robust than simple plurality voting because it down-weights answers produced by high-entropy (uncertain) generations. The openai_harmony tool-calling framework integrates cleanly with the vLLM serving infrastructure. The persistent Jupyter kernel pool eliminates per-call import overhead. The budget formula correctly handles the tradeoff between giving hard problems adequate time and ensuring all problems receive at least a minimum allocation.
\nThe fundamental limitation is that gpt-oss-120b solves only 4 of the 10 hard AIMO3 reference problems even with unlimited attempts. The remaining 6 hard problems are simply outside the model's capability distribution at any reasonable pass@k. Increasing attempts from 8 to 20 would improve expected score from 43 to 47, but every approach to increasing attempts on this hardware either saturates the KV cache or causes budget collapse.
\nBased on the reference chart and the competition host's published pass@n data, the reliable path to the 47/50 threshold required for the Overall Progress Prize is one of:
\nUse Qwen3-Next-80B-A3B-Thinking with its native 262k context window. This model is eligible (released before March 15, 2026), fits on a single H100 (approximately 40 GB in FP8 plus small KV cache due to 3.9B active parameters), and solves 8/10 hard reference problems at pass@3 compared to gpt-oss-120b's 4/10.
Use multiple H100 GPUs with tensor parallelism, allowing safe concurrent requests to scale proportionally. 4x H100 with tensor_parallel_size=4 would enable 32 concurrent requests and pass@32 per problem within the same time budget.
Fine-tune gpt-oss-120b on high-quality olympiad solutions to improve its per-attempt accuracy on hard problems from q=0.05 to q=0.15 or higher. The competition provides access to 128 H100 GPUs for fine-tuning as part of its ecosystem offering.
Our submission achieves 43/50 on the AIMO3 public test set using gpt-oss-120b with parallel sampling and entropy-weighted majority voting. The system is clean, reproducible, and operates reliably within the hardware constraints. Our extensive ablation study identifies the KV cache limitation (maximum 8 safe concurrent requests on 1x H100 with gpt-oss-120b) as the binding constraint preventing reliable pass@20 performance.
\nThe competition host's published pass@n data confirms that the gap between our 43 and the 47 threshold is achievable with the same model given more compute, or with Qwen3-Next-80B given the same compute. We document all failure modes in detail so that future participants can avoid the same pitfalls.
\nThe complete submission code is reproduced below. The system requires the following Kaggle datasets:
\n/kaggle/input/gpt-oss-120b/transformers/default/1: model weights/kaggle/input/aimo-3-utils/wheels.tar.gz: vLLM and openai_harmony wheels/kaggle/input/ai-mathematical-olympiad-progress-prize-3/test.csv: competition test set%pip uninstall --yes 'keras' 'matplotlib' 'scikit-learn' 'tensorflow'\nimport warnings\nwarnings.simplefilter('ignore')\nimport os\nimport sys\nimport subprocess\n\ndef set_env(input_archive, temp_dir):\n if not os.path.exists(temp_dir):\n os.makedirs(temp_dir, exist_ok=True)\n subprocess.run(['tar', '-xzf', input_archive, '-C', temp_dir], check=True)\n subprocess.run([\n sys.executable, '-m', 'pip', 'install', '--no-index',\n '--find-links', f'{temp_dir}/wheels',\n 'unsloth', 'trl', 'vllm', 'openai_harmony'\n ], check=True)\n\nset_env(\n input_archive='/kaggle/input/aimo-3-utils/wheels.tar.gz',\n temp_dir='/kaggle/tmp/setup'\n)\nsubprocess.run(['ls', '/kaggle/tmp/setup/tiktoken_encodings'])\n\nos.environ['TRANSFORMERS_NO_TF'] = '1'\nos.environ['TRANSFORMERS_NO_FLAX'] = '1'\nos.environ['CUDA_VISIBLE_DEVICES'] = '0'\nos.environ['TOKENIZERS_PARALLELISM'] = 'false'\nos.environ['TRITON_PTXAS_PATH'] = '/usr/local/cuda/bin/ptxas'\nos.environ['TIKTOKEN_ENCODINGS_BASE'] = '/kaggle/tmp/setup/tiktoken_encodings'\n\n# CFG: all key parameters documented inline\nclass CFG:\n system_prompt = (\n 'You are a world-class International Mathematical Olympiad (IMO) competitor. '\n 'The final answer must be a non-negative integer between 0 and 99999. '\n 'You must place the final integer answer inside \\\\boxed{}.'\n )\n tool_prompt = (\n 'Use this tool to execute Python code. '\n 'The environment is a stateful Jupyter notebook. '\n 'You must use print() to output results.'\n )\n preference_prompt = (\n 'You have access to math, numpy and sympy in Python to solve the problem, '\n 'but check their version first before starting to use them.'\n )\n served_model_name = 'gpt-oss'\n model_path = '/kaggle/input/gpt-oss-120b/transformers/default/1'\n kv_cache_dtype = 'fp8_e4m3'\n dtype = 'auto'\n high_problem_timeout = 900 # maximum seconds per problem\n base_problem_timeout = 300 # minimum seconds reserved per remaining problem\n notebook_limit = 17400 # total seconds available (5 hours minus buffer)\n server_timeout = 180 # seconds to wait for vLLM startup\n session_timeout = 960 # OpenAI client timeout\n jupyter_timeout = 10 # seconds per code execution\n sandbox_timeout = 3 # seconds to wait for sandbox from pool\n stream_interval = 200 # vLLM streaming chunk interval\n context_tokens = 65536 # maximum context length\n buffer_tokens = 512 # minimum tokens before refusing new turn\n search_tokens = 32 # chunks to scan back for boxed answer\n top_logprobs = 5 # logprobs per token for entropy computation\n batch_size = 256 # vLLM max-num-seqs\n temperature = 1.0 # sampling temperature (model tuned at 1.0)\n min_p = 0.02 # minimum probability filter\n early_stop = 5 # votes needed to stop early\n attempts = 8 # parallel attempts per problem\n workers = 16 # thread pool workers\n turns = 128 # max tool-call turns per attempt\n seed = 42 # base random seed\n gpu_memory_utilization = 0.96 # fraction of GPU memory for vLLM\nThe notebook startup sequence consumes approximately 700 seconds:
\n| Phase | \nDuration | \nCumulative | \n
|---|---|---|
| pip install from local wheels | \n90s | \n90s | \n
| Model weight preload (16 threads, 65 GB) | \n80-100s | \n190s | \n
| vLLM server startup | \n120-140s | \n330s | \n
| Jupyter kernel pool init (16 kernels) | \n3s | \n333s | \n
| Buffer/overhead | \n~370s | \n700s | \n
After startup, 16,700 seconds remain for 50 problems, giving an average of 334 seconds per problem.
\nThe budget formula ensures that all problems receive at least base_problem_timeout seconds, while the total wall-clock time stays within notebook_limit. The formula behaves as follows under different workload distributions:
All easy problems (each takes 20s, early_stop fires quickly):\nEvery problem receives budget=900s but uses only 20s. Total time: 700s setup + 50 x 20s = 1,700s. Well within the limit.
\nAll hard problems (each uses full 900s):\nProblems 1-45 receive budget=900s. Problems 46-50 receive budget=max(300, min(900, time_left - reserved)). After 45 hard problems, elapsed = 700 + 45 x 900 = 41,200s, which exceeds the 17,400s limit. In practice, vLLM will enforce the deadline by cutting off streaming, so each problem actually uses at most its allocated budget. The formula prevents later problems from being starved.
\nMixed workload (realistic competition scenario):\nWith approximately 6 very hard, 14 hard, 20 medium, and 10 easy problems in random order, the total compute time is approximately 6 x 900 + 14 x 350 + 20 x 100 + 10 x 20 = 5,400 + 4,900 + 2,000 + 200 = 12,500s plus 700s setup = 13,200s. This fits within the 17,400s limit.
\ngpt-oss-120b architecture estimate:
\nThis is clearly much larger than the available memory, which means vLLM allocates a fixed pool and limits concurrent requests accordingly. With 0.96 GPU memory utilization and 65 GB weights, approximately 16 GB is available for KV cache. At 2.5 MB per token, vLLM can support approximately 6,400 concurrent tokens, which corresponds to approximately 7-8 requests at typical generation lengths. This explains the observed limit of 8 safe concurrent requests.
\nChange: temperature = 1.0 -> 0.6
Expected outcome: More consistent answers across runs, reduced variance.
\nActual outcome: Score dropped from 43 to 35, losing 8 problems compared to the baseline.
\nRoot cause: gpt-oss-120b was trained and fine-tuned with temperature=1.0 for mathematical reasoning tasks. At temperature=0.6, the model's sampling distribution is sharpened, causing:
\nThe lesson is that temperature is not a free parameter for aligned models; it must match the distribution used during RLHF fine-tuning.
\nChange: attempts = 8 -> 16, workers = 16
Expected outcome: 16 parallel attempts, wider solution space coverage.
\nActual outcome: Score dropped to 34.
\nRoot cause: 16 concurrent requests x approximately 2 GB KV cache each = 32 GB, which exceeds the available 15-16 GB. vLLM's preemption policy aborts generation for lower-priority requests when KV memory is exhausted. Most of the 16 requests were preempted before generating a boxed answer, reducing the effective pass@k to approximately pass@3 or pass@4.
\nChange: early_stop = 5 -> 8
Expected outcome: More stable voting, reduced run-to-run variance.
\nActual outcome: Score dropped to 37.
\nRoot cause: The actual source of variance is not the early_stop threshold but GPU floating-point non-determinism in batched execution. Setting early_stop=8 (equal to attempts=8) means early stopping never triggers, so all 8 attempts always complete. This slightly reduces performance on easy problems (where all 8 were already agreeing before the 5th completed) and has no effect on hard problems. The slight score reduction suggests that waiting for all 8 attempts on easy problems occasionally leads to a wrong late answer being included in the vote with high weight.
\nChange: Sequential rounds of 8 attempts using a while loop with new ThreadPoolExecutor per round.
Expected outcome: Multiple rounds of 8 attempts within the 900s budget, achieving approximately 20 effective attempts on hard problems.
\nActual outcome: Score dropped to 28.
\nRoot cause: The implementation created a new ThreadPoolExecutor(max_workers=16) inside the while loop. When early_stop=5 triggered for an easy problem at approximately T=20 seconds, the code called stop_event.set() and then executor.shutdown(wait=True). The wait=True argument blocks until all currently running threads complete their work. For easy problems, 3 threads were still streaming responses, which take approximately 200-400 seconds to complete even after the stop event is checked. This 350-second blocking overhead per easy problem, multiplied across 50 problems, caused the notebook to run out of time after solving only approximately 20-25 problems.
The correct fix requires using a single long-lived executor for the entire problem duration and relying on f.cancel() plus the stop event check inside the streaming loop to abort running attempts quickly.
Change: attempts = 24, workers = 24
Expected outcome: 24 parallel attempts, achieving approximately pass@20 through wider coverage.
\nActual outcome: Score dropped to 21, the lowest score in our experiment series.
\nRoot cause: 24 concurrent KV cache allocations = approximately 48 GB, which is 3x the available 15-16 GB. vLLM preempted the vast majority of requests, with only approximately 3-4 completing per problem. This is worse than the original 8 attempts because the throughput per request is reduced due to memory pressure, and the voting pool is very small with high variance.
\nChange: base_problem_timeout = 100
Expected outcome: More budget available for earlier problems, enabling second rounds of 8 attempts.
\nActual outcome: Score dropped to 22.
\nRoot cause: A 100-second budget generates approximately 100s x 100 tok/s = 10,000 tokens. Analysis of the AIME-level problems in our reference set shows that the model requires approximately 15,000-30,000 tokens to reach a boxed answer. Problems receiving a 100-second budget return None from the answer extractor and default to 0, which is almost always wrong. The number of problems affected depends on problem ordering; with hard problems early, approximately 15-20 later problems received 100-second budgets.
Change: base_problem_timeout = 200, attempts = 24, workers = 8
Expected outcome: Budget floor high enough to generate complete answers, while the queueing mechanism provides extra attempts on hard problems.
\nActual outcome: Score dropped to 28.
\nRoot cause: A detailed simulation shows that budget collapse still occurs when hard problems (taking 900s each) come early in the random problem ordering. After 6 hard problems:
\nAfter 9 hard problems:
\nWith 400s budget and workers=8, the first round of 8 attempts completes at T=350s, leaving 50s for the second round. 50s is insufficient for the second batch of 8 attempts to generate answers on hard problems. Problems receiving 200-400s budgets with this configuration generate partial responses that occasionally produce answers but with high miss rates.
\n| Variable | \nValue | \nPurpose | \n
|---|---|---|
| TRANSFORMERS_NO_TF | \n1 | \nSuppress TensorFlow import | \n
| TRANSFORMERS_NO_FLAX | \n1 | \nSuppress Flax import | \n
| CUDA_VISIBLE_DEVICES | \n0 | \nSingle GPU operation | \n
| TOKENIZERS_PARALLELISM | \nfalse | \nPrevent tokenizer fork warnings | \n
| TRITON_PTXAS_PATH | \n/usr/local/cuda/bin/ptxas | \nTriton compilation target | \n
| TIKTOKEN_ENCODINGS_BASE | \n/kaggle/tmp/setup/tiktoken_encodings | \nLocal encoding cache | \n
| Argument | \nValue | \nRationale | \n
|---|---|---|
| --seed | \n42 | \nReproducible base randomness | \n
| --tensor-parallel-size | \n1 | \nSingle GPU | \n
| --max-num-seqs | \n256 | \nMaximum request queue size | \n
| --gpu-memory-utilization | \n0.96 | \nLeave 4% for other processes | \n
| --kv-cache-dtype | \nfp8_e4m3 | \nHalves KV memory relative to fp16 | \n
| --max-model-len | \n65536 | \nContext window | \n
| --stream-interval | \n200 | \nToken streaming granularity | \n
| --async-scheduling | \n(flag) | \nEnables asynchronous request scheduling | \n
| --no-enable-prefix-caching | \n(flag) | \nAvoids B200 gibberish bug | \n
| Parameter | \nValue | \nRationale | \n
|---|---|---|
| temperature | \n1.0 | \nMatches model's fine-tuning distribution | \n
| min_p | \n0.02 | \nFilters very low probability tokens | \n
| top_logprobs | \n5 | \nUsed for entropy computation | \n
| Per-attempt seed | \n(42 + i)^2 | \nDiverse seeds while reproducible | \n
Based on analysis of the 50 public test problems and comparison with the AIMO3 reference problem solutions:
\nVery hard (6 problems, q approximately 0.05): These problems require advanced combinatorial identities, Hermite's identity, Legendre's formula, spiral similarity, or deep number theory. gpt-oss-120b solves these approximately 5% of the time per attempt. Examples from the reference set include problems involving tournaments with 2^20 players, functional equations on integers with mod constraints, and problems requiring the Lifting the Exponent lemma.
\nHard (8-10 problems, q approximately 0.10-0.15): Problems requiring extended computation, case analysis, or non-obvious transformations. The model often identifies the correct approach but makes arithmetic errors or misses edge cases.
\nMedium (14-16 problems, q approximately 0.40-0.55): AIME-level problems where the model reliably identifies the solution approach and usually executes it correctly. Python code assistance is particularly valuable here for modular arithmetic and combinatorial counting.
\nEasy (18-22 problems, q approximately 0.85-0.95): Problems where the model identifies and solves the problem on the first attempt with high probability. Early stopping typically triggers within the first 5 attempts.
\nThe Python execution tool plays an essential role for three problem types:
\nExact large integer arithmetic. Python integers have arbitrary precision, making them ideal for problems involving factorials, binomial coefficients, and modular exponentiation at scales where floating-point precision is insufficient. The mpmath library with 64 decimal digits of precision handles high-precision floating-point needs.
Combinatorial verification. Problems involving counts of valid configurations are often solvable by direct enumeration for small cases, allowing the model to verify patterns before generalising. The model uses sympy for symbolic manipulation and itertools for combinatorial generation.
Polynomial and number-theoretic computation. Problems involving cyclotomic polynomials, Fibonacci sequences, digit sum functions, and similar structures benefit from symbolic computation in sympy, which can handle exact rational arithmetic and polynomial factorisation.
To reproduce our submission exactly:
\nCreate a Kaggle notebook with the following datasets attached:
\ngpt-oss-120b (model weights)aimo-3-utils (wheel packages)Set accelerator to GPU T4 x2 or P100 (for testing); the competition environment uses H100.
The notebook requires no external internet access; all packages are installed from the local wheel archive.
The complete set of parameters that define our submission:
\ntemperature = 1.0\nmin_p = 0.02\nattempts = 8\nworkers = 16\nearly_stop = 5\nturns = 128\nseed = 42\ngpu_memory_utilization = 0.96\nkv_cache_dtype = 'fp8_e4m3'\ncontext_tokens = 65536\nhigh_problem_timeout = 900\nbase_problem_timeout = 300\nnotebook_limit = 17400\nWhen run on the 3-problem calibration set (0x10, 1-1, 4+x=4):
\nWhen run on the full competition set, expect:
\nTotal runtime should be approximately 12,000-16,000 seconds depending on problem ordering.
\nDue to GPU floating-point non-determinism in batched execution, exact score reproduction is not guaranteed. The 36-43 range observed across nine submissions represents the natural variance of the system. A score in the range 36-43 should be considered a successful reproduction.
\n[1] Competition hosts. \"AIMO3 Public vs Private Problem Set Difficulty Comparison.\" Kaggle Discussion, 2025.
\n[2] Competition hosts. \"AIMO3 B200 and Prefix Caching Bug Fix Notice.\" Kaggle Discussion, 2025.
\n[3] Competition hosts. \"AIMO3 March 15 Cutoff Rule FAQ.\" Kaggle Discussion, 2025.
\n[4] Competition hosts. \"Welcome to AIMO3: IMO-level problems, H100s, ecosystem, extra prizes.\" Kaggle Discussion, 2025.
\n[5] AIMO Progress Prize 2 Reference Problems and Solutions. October 2024.
\n[6] NemoSkills team. \"NemoSkills: AIMO2 Winning Solution.\" arXiv:2504.16891, 2025.
\n[7] vLLM Project. \"vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention.\" SOSP, 2023.
\n[8] Qwen Team. \"Qwen3 Technical Report.\" arXiv:2505.09388, 2025.
\n[9] Qwen Team. \"Qwen3-Next Blog Post.\" Alibaba Cloud, September 2025.
\n[10] Chen, M. et al. \"Evaluating Large Language Models Trained on Code.\" arXiv:2107.03374, 2021. (pass@k framework)
\n[11] Lightman, H. et al. \"Let's Verify Step by Step.\" arXiv:2305.20050, 2023.
\n[12] openai_harmony documentation. Internal competition API reference, AIMO3.
\nThe following table serves as a data table for a pass@k performance curve. The x-axis is n (number of attempts), the y-axis is expected score out of 50.
\n| n (attempts) | \nExpected Score | \nSource | \n
|---|---|---|
| 1 | \n27.0 | \nHost-published | \n
| 3 | \n37.0 | \nHost-published | \n
| 5 | \n41.0 | \nHost-published | \n
| 8 | \n43.0 | \nHost-published / Our peak score | \n
| 16 | \n45.6 | \nTheoretical (q: easy=0.90, med=0.45, hard=0.05) | \n
| 20 | \n47.0 | \nHost-published | \n
| 24 | \n47.1 | \nTheoretical | \n
| 100 | \n49.5 | \nHost-published | \n
The curve demonstrates three distinct regimes: rapid improvement from 1 to 8 attempts (easy and medium problems saturate quickly), slower improvement from 8 to 20 attempts (hard problems still benefit), and near-plateau above 20 attempts (only very hard problems remain unsolved).
\nThe following table describes a waterfall chart showing score changes from each ablation:
\n| Ablation | \nScore | \nDelta from Baseline | \n
|---|---|---|
| Baseline (original 43-scorer) | \n43 (peak) / 39.1 (avg) | \n0 | \n
| temperature = 0.6 | \n35 | \n-8 | \n
| attempts=16, workers=16 | \n34 | \n-9 | \n
| early_stop=8 | \n37 | \n-6 | \n
| Prompt engineering | \n38 | \n-5 | \n
| Iterative rounds (executor bug) | \n28 | \n-15 | \n
| attempts=24, workers=24 (KV OOM) | \n21 | \n-22 | \n
| attempts=24, workers=8, bpt=100 (token timeout) | \n22 | \n-21 | \n
| attempts=24, workers=8, bpt=200 (budget collapse) | \n28 | \n-15 | \n
| attempts=24, workers=8, bpt=300 (sandbox timeout) | \n30 | \n-13 | \n
Every modification to the baseline decreased the score. The three most damaging changes (KV OOM at -22, token timeout at -21, iterative rounds executor bug at -15) all involved attempts to increase the number of attempts per problem beyond what the hardware safely supports.
\nThe following table illustrates the budget formula output under a worst-case scenario (all hard problems appearing first) with base_problem_timeout=300:
\n| Problem | \nElapsed After | \ntime_left | \nreserved | \nbudget | \n
|---|---|---|---|---|
| 1 (hard, 900s) | \n1600s | \n15800s | \n14700s | \n900s | \n
| 2 (hard, 900s) | \n2500s | \n14900s | \n14400s | \n500s | \n
| 3 (hard, 900s) | \n3400s | \n14000s | \n14100s | \n300s (floor) | \n
| 4 (hard, 900s) | \n4300s | \n13100s | \n13800s | \n300s (floor) | \n
| 5 (hard, 900s) | \n5200s | \n12200s | \n13500s | \n300s (floor) | \n
| 6 (medium, 300s) | \n5500s | \n11900s | \n13200s | \n300s (floor) | \n
| 7 (medium, 300s) | \n5800s | \n11600s | \n12900s | \n300s (floor) | \n
Once the reserved buffer exceeds the remaining time, the budget is clamped to the minimum floor for all subsequent problems, regardless of their difficulty. With base_problem_timeout=200, the collapse is slightly delayed but the same phenomenon occurs when hard problems are numerous early in the sequence.
\n| Component | \nMemory | \nNotes | \n
|---|---|---|
| gpt-oss-120b weights (FP8) | \n~65 GB | \nRead from model path | \n
| CUDA runtime and drivers | \n~2 GB | \nFixed overhead | \n
| PyTorch and vLLM overhead | \n~2 GB | \nActivation memory | \n
| Available for KV cache | \n~16 GB | \nAt 0.96 GPU utilization | \n
| KV per request (65k context, FP8) | \n~2 GB | \nPer-request allocation | \n
| Safe concurrent requests | \n~8 | \n16 GB / 2 GB | \n
This breakdown explains why the original attempts=8 configuration works reliably while attempts=16 and attempts=24 fail catastrophically.
\nOur submission uses only gpt-oss-120b, a model that was available and eligible well before the March 15, 2026 AMLT cutoff. No models released after March 15 are used at inference time. The openai_harmony library is a competition-provided tool. All Python packages are installed from the competition-provided wheel archive with no external downloads. The submission notebook contains no hard-coded answers or problem-specific logic; all answers are generated by the model at runtime.
\n