\nPessimistic Scenario (35 Points):\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3967558%2F267b2a25eed33216ccdabca963d65786%2F35.png?generation=1776903669663427&alt=media\")
Abstract
\nThe deployment of Large Language Models (LLMs) for complex mathematical reasoning is heavily constrained by available compute and inference latency. In the AI Mathematical Olympiad (AIMO) Progress Prize 3, systems are restricted to a strict 5-hour execution window on a single NVIDIA H100 (80GB VRAM) GPU. While recent research has demonstrated that scaling model size and test-time computation improves absolute accuracy on mathematical benchmarks, the degradation in reasoning efficiency—and its subsequent impact under strict compute constraints—remains underexplored. In this paper, we evaluate the reasoning efficiency of state-of-the-art mid-size models, demonstrating that without optimization, the advantages of high-accuracy models are frequently nullified by time constraints. Furthermore, we analyze severe score variance (non-determinism) observed during evaluation, identifying algorithmic time-management strategies and agentic execution faults as the primary culprits. By implementing strict time-allocation controls, robust state recovery mechanisms, and inference optimizations including Speculative Decoding and FP8 KV-cache quantization, we stabilized leaderboard variance and doubled throughput. Finally, to validate these optimizations, we introduce a novel, contamination-free evaluation methodology, ensuring rigorous benchmarking against post-training data sets.
\n1 Introduction
\nThe AI Mathematical Olympiad (AIMO) Progress Prize 3 introduces a highly constrained execution environment, requiring systems to resolve complex mathematical problems within a strict 5-hour time limit using a single NVIDIA H100 GPU.
\nPrevious research aimed at improving the performance of LLMs in mathematical reasoning has predominantly been conducted in environments with unrestricted compute. Consequently, while these methodologies improve absolute accuracy on standard mathematical benchmarks, there is limited study on the associated costs to inference efficiency. When deployed in compute-bound environments, models with higher theoretical accuracy often engage in computational \"overthinking,\" leading to timeouts that negate their capability advantages.
\nSimultaneously, LLM inference within agentic frameworks exhibits significant non-determinism. In AIMO3, the effect of this variance appeared disproportionately severe, with public leaderboard scores fluctuating by up to 10 points (a ~25% variance) across identical code submissions.
\nTo address these challenges, this paper presents a comprehensive analysis of inference efficiency and non-determinism in mathematical problem-solving agents. Our core contributions include:
\n1. Analysis and Mitigation of Non-Determinism: Identifying the root causes of execution variance and proposing algorithmic and system-level solutions to stabilize evaluation scores.
\n2. Compute-Constrained Efficiency Optimization: Evaluating model efficiency and implementing techniques to maximize throughput and pass rates within a strict hardware budget.
\n3. Contamination-Free Evaluation Framework: Proposing a robust methodology for evaluating LLMs using high-difficulty, guaranteed zero-leakage datasets.
\nIn the spirit of open-source research and reproducibility, we have released our full solution codebase, models, datasets, and reasoning traces.
\nCodes & Model:
\nDataset:
\n2. Analysis and Mitigation of LLM Non-Determinism
\n2.1 Algorithmic Time Management as a Source of Variance
\nInitial observations indicated massive score fluctuations (between 35 and 45 points) on the Public Leaderboard despite using identical deterministic code and sampling parameters. Analysis revealed that this variance was not strictly due to the inherent stochasticity of LLMs, but rather an artifact of the dynamic per-question time management algorithm utilized in the execution pipeline. The specific implementation responsible for this variance is detailed below:
high_problem_timeout = 900\n base_problem_timeout = 300\n\n elapsed_global = time.time() - self.notebook_start_time\n time_left = self.cfg.notebook_limit - elapsed_global\n problems_left_others = max(0, self.problems_remaining - 1)\n reserved_time = problems_left_others * self.cfg.base_problem_timeout\n\n budget = time_left - reserved_time\n budget = min(budget, self.cfg.high_problem_timeout)\n budget = max(budget, self.cfg.base_problem_timeout)\nThis heuristic operates on two main principles: it guarantees a minimum baseline execution time (300 seconds) for all problems, and it dynamically allocates any \"saved\" time from rapidly solved problems to the current active problem, up to a maximum threshold (900 seconds).
\nThis greedy allocation strategy creates severe sensitivity to question ordering. We can illustrate this via two extreme simulated scenarios across 50 questions that give rise to the extreme 35- and 45-point variance when the question order changes:
\nTo demonstrate this, let us define a mock evaluation set of 50 questions categorized into three difficulty tiers based on the total execution time required to generate an answer:
\n| Question Type | \nTotal Questions | \nExecution time per question | \n
|---|---|---|
| Hard | \n5 | \n> 900 seconds | \n
| Intermediate | \n10 | \n300 to 900 seconds | \n
| Easy | \n35 | \n< 300 seconds | \n
\nPessimistic Scenario (35 Points):\n
i) 5 Hard Questions are encountered in the first 5 slots, the system fully exhausts the maximum time allowance per question, accumulating 4,500 seconds of compute without yielding correct answers.
\nii) Consequently, the remaining 45 questions are bottlenecked by the minimum 300-second allocation.
\niii) The 10 Intermediate Questions failed to produce a correct answer in 300 seconds.
\niv) As a result, only 35 points have been scored.
Optimistic Scenario (45 Points):\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3967558%2Fe1812c9f0f8250438fbd643c9581042d%2F45.png?generation=1776903557734112&alt=media\")
i) Hard Questions arrive during the last 5 slot.
\nii) The rest of 45 questions arrived in the earlier slot and all have enough time to finish execution, yielding a score of 45 points.
Mitigation: By entirely deprecating the dynamic time management code and enforcing a static compute budget per question, the systemic variance was eliminated, resulting in highly stable, reproducible leaderboard scores within a narrow 1-point margin (39 to 40 points).
\n2.2.Resolving Execution Faults in Agentic Environments
\nThe Tool-Integrated Reasoning environment utilizes a stateful Python Code Interpreter (via Jupyter kernels) to execute intermediate mathematical verifications. Initial implementations of this agentic environment suffered from severe systemic instability, frequently resulting in \"Notebook Inference Server Errors\" or silent process hangs. These issues scaled exponentially under high parallelization (e.g., managing 16–20 concurrent workers).
\nThrough rigorous debugging of the agent loop and sandbox architecture, we identified several race conditions, memory leaks, and uncatchable exceptions. To eliminate these intermittent crashes and guarantee back-to-back stable leaderboard scores, we re-architected the sandbox environment with the following mitigation strategies:
\nDynamic Port Allocation to Prevent Race Conditions:
\nThe baseline architecture manually assigned communication ports (shell, iopub, stdin, hb, control) using a locked incremental counter. Under high-concurrency spawning, this frequently led to port collisions. We eliminated manual port assignment, delegating port resolution entirely to the KernelManager to dynamically bind to available OS ports. This resolved initialization bottlenecks and silent kernel startup failures.
Zombie Process Mitigation and Hard Kills:
\nStandard kernel interruptions (interrupt_kernel()) often fail when Python is executing compiled C-extensions (e.g., complex matrix operations in numpy or deep symbolic recursions in sympy). These orphaned processes would continue to consume RAM, eventually causing out-of-memory (OOM) crashes. We implemented a cascading shutdown protocol: if a kernel ignores a standard interrupt after a 1-second grace period, the environment intercepts the process ID (PID) and forces an OS-level SIGKILL. This guarantees the termination of rogue memory hogs.
Active Health Monitoring via UUID Injection:
\nBecause a kernel can silently enter a deadlocked state without terminating, relying on process status is insufficient. We introduced an active is_healthy() heartbeat protocol. Before any tool execution, the sandbox injects a dynamically generated UUID into a print() statement with a strict 2-second timeout constraint. If the exact UUID is not returned, the kernel is definitively classified as unresponsive.
Self-Healing and Graceful State Recovery:
\nTo maintain inference continuity without crashing the broader agentic loop, we implemented a self-healing mechanism. If the health check fails—or if a kernel is hard-killed—the sandbox pool automatically discards the dead instance and spawns a fresh Jupyter kernel. Crucially, to prevent the LLM from hallucinating variable availability after a restart, the system automatically injects a state-reset warning back into the LLM's context window: [SYSTEM NOTE: The kernel crashed and was restarted. Previous variables are lost.] This prompts the model to intelligently re-declare necessary variables rather than failing on NameError exceptions.
By coupling these low-level sandbox optimizations with strict health thresholds, the agentic execution environment achieved near-perfect reliability, eliminating server-side inference errors within the strict 5-hour Kaggle compute window.
\n3.Optimizing Model Efficiency
\n3.1 Model Efficiency Evaluation
\nUnder the AIMO3 constraints, end-to-end execution time—defined as the total time required for an agent to generate a reasoning trace, execute the associated code, and produce a correct integer answer—is the primary bottleneck. We shortlisted three high-capability mid-size models for evaluation: GPT-OSS 120B, NVIDIA-Nemotron-3-Super, and Qwen3.5-122B-A10B.
\nEvaluations were conducted using two recent competitive datasets (HMMT February 2026 and AIME February 2026).
HMMT 2026 February
\n| Candidate Model | \nAttempt per Question | \nTokens per Question | \nEnd-to-end Time per Question (seconds) | \n
|---|---|---|---|
| GPT OSS 120B | \n8 | \n24332 | \n383 | \n
| NVIDIA-Nemotron-3-Super 120B | \n8 | \n45880 | \n433 | \n
| Qwen3.5-122B-A10B | \n8 | \n25103 | \n411 | \n
\nAIME 2026 February
| Candidate Model | \nAttempt per Question | \nTokens per Question | \nEnd-to-end Time per Question (seconds) | \n
|---|---|---|---|
| GPT OSS 120B | \n8 | \n20119 | \n204 | \n
| NVIDIA-Nemotron-3-Super 120B | \n8 | \n25233 | \n213 | \n
| Qwen3.5-122B-A10B | \n8 | \n21225 | \n235 | \n
\n3.2 Throughput Enhancements
To maximize the number of valid reasoning paths explored per problem within the time limit, raw inference throughput must be optimized without degrading precision. We implemented Speculative Decoding to accelerate token generation and applied FP8 KV-Cache Quantization.
\nThese memory and compute optimizations yielded a 2x increase in token throughput. Crucially, this efficiency gain allowed the system to scale its sampling budget beyond the standard 8 attempts per question. Increasing the number of reasoning trajectories directly correlated with higher, more stable leaderboard performance as evidence by screenshot below:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3967558%2F06d587a1d31903743b9e18d9675caf1c%2Fimage_2026-04-23_214528479.png?generation=1776951929929550&alt=media\")
4. Contamination-Free High-Difficulty Evaluation Dataset
\nA persistent challenge in evaluating mathematical LLMs is the risk of data contamination, where benchmark questions are inadvertently included in the model's pre-training corpus. To ensure highly reliable comparisons, we constructed a guaranteed contamination-free evaluation dataset.
\nMethodology:
\nTemporal Isolation: We restricted candidate questions exclusively to those published after June 2024, which serves as the training data cutoff date for our primary model (GPT-OSS 120B).
\nFormat Alignment: Rather than discarding questions with non-integer answers, we manually reformulated them into integer-output constraints, matching the precise stylistic guidelines of AIMO problem setters. This yielded an initial pool of 513 candidate questions.
\nDifficulty Filtering: To isolate high-difficulty reasoning challenges, we benchmarked the dataset using the top-performing public models. A question was retained only if it met one of two strict criteria:
\nThe model failed at least once across 8 independent sampling attempts.
The model succeeded in all 8 attempts, but the cumulative execution time exceeded 500 seconds, indicating high computational friction.
This rigorous filtering process resulted in a final dataset of 68 high-difficulty, verifiable, and zero-contamination mathematical problems.
\nStatistic on Source of Questions
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3967558%2F118c592d9e7670bb19e831a7b913e7d7%2Fimage_2026-04-23_212123084.png?generation=1776950484782196&alt=media\")
Having conducted a more careful review of the precise timings around this submission, we have determined that the post on the discussion thread was made around one hour after the 22nd April 23:59 UTC cutoff and therefore unfortunately this submission is not eligible for the Write Up prize. We still thank you for your contribution and it is of course available for the community to take any useful pointers from.
", "votes": 0, "replies": [] }, { "id": 3459451, "author_name": "Sam Bealing", "author_url": "", "post_date": "2026-05-17T20:36:00.670000", "content": "Apologies for the delay in posting these questions. We're reviewing whether there's a reason this submission was missed from our initial pass; in any case, I've spent the evening reading through your write-up and have the questions below.
\nGiven the late notice, we can give you until Monday, 18th May, 23:59 UTC to respond as a minimum (we'll update if we decide to extend that). Rest assured we'll take the time to give the submission the attention it deserves, and note that we're yet to make a final decision on the winners so you're very much still in consideration here.
\nFor wider context on the rebuttal phase, please see the updated discussion post.
\nnvidia_lc, amazon_p, nvidia_v2, nvidia_tp) materially better or worse than wenliang?Look forward to reading your responses on these.
", "votes": 0, "replies": [] } ], "comment_count": 2, "messages": [ { "id": 3447318, "postDate": "2026-04-22T23:49:31.460Z", "content": "Abstract
\nThe deployment of Large Language Models (LLMs) for complex mathematical reasoning is heavily constrained by available compute and inference latency. In the AI Mathematical Olympiad (AIMO) Progress Prize 3, systems are restricted to a strict 5-hour execution window on a single NVIDIA H100 (80GB VRAM) GPU. While recent research has demonstrated that scaling model size and test-time computation improves absolute accuracy on mathematical benchmarks, the degradation in reasoning efficiency—and its subsequent impact under strict compute constraints—remains underexplored. In this paper, we evaluate the reasoning efficiency of state-of-the-art mid-size models, demonstrating that without optimization, the advantages of high-accuracy models are frequently nullified by time constraints. Furthermore, we analyze severe score variance (non-determinism) observed during evaluation, identifying algorithmic time-management strategies and agentic execution faults as the primary culprits. By implementing strict time-allocation controls, robust state recovery mechanisms, and inference optimizations including Speculative Decoding and FP8 KV-cache quantization, we stabilized leaderboard variance and doubled throughput. Finally, to validate these optimizations, we introduce a novel, contamination-free evaluation methodology, ensuring rigorous benchmarking against post-training data sets.
\n1 Introduction
\nThe AI Mathematical Olympiad (AIMO) Progress Prize 3 introduces a highly constrained execution environment, requiring systems to resolve complex mathematical problems within a strict 5-hour time limit using a single NVIDIA H100 GPU.
\nPrevious research aimed at improving the performance of LLMs in mathematical reasoning has predominantly been conducted in environments with unrestricted compute. Consequently, while these methodologies improve absolute accuracy on standard mathematical benchmarks, there is limited study on the associated costs to inference efficiency. When deployed in compute-bound environments, models with higher theoretical accuracy often engage in computational \"overthinking,\" leading to timeouts that negate their capability advantages.
\nSimultaneously, LLM inference within agentic frameworks exhibits significant non-determinism. In AIMO3, the effect of this variance appeared disproportionately severe, with public leaderboard scores fluctuating by up to 10 points (a ~25% variance) across identical code submissions.
\nTo address these challenges, this paper presents a comprehensive analysis of inference efficiency and non-determinism in mathematical problem-solving agents. Our core contributions include:
\n1. Analysis and Mitigation of Non-Determinism: Identifying the root causes of execution variance and proposing algorithmic and system-level solutions to stabilize evaluation scores.
\n2. Compute-Constrained Efficiency Optimization: Evaluating model efficiency and implementing techniques to maximize throughput and pass rates within a strict hardware budget.
\n3. Contamination-Free Evaluation Framework: Proposing a robust methodology for evaluating LLMs using high-difficulty, guaranteed zero-leakage datasets.
\nIn the spirit of open-source research and reproducibility, we have released our full solution codebase, models, datasets, and reasoning traces.
\nCodes & Model:
\nDataset:
\n2. Analysis and Mitigation of LLM Non-Determinism
\n2.1 Algorithmic Time Management as a Source of Variance
\nInitial observations indicated massive score fluctuations (between 35 and 45 points) on the Public Leaderboard despite using identical deterministic code and sampling parameters. Analysis revealed that this variance was not strictly due to the inherent stochasticity of LLMs, but rather an artifact of the dynamic per-question time management algorithm utilized in the execution pipeline. The specific implementation responsible for this variance is detailed below:
high_problem_timeout = 900\n base_problem_timeout = 300\n\n elapsed_global = time.time() - self.notebook_start_time\n time_left = self.cfg.notebook_limit - elapsed_global\n problems_left_others = max(0, self.problems_remaining - 1)\n reserved_time = problems_left_others * self.cfg.base_problem_timeout\n\n budget = time_left - reserved_time\n budget = min(budget, self.cfg.high_problem_timeout)\n budget = max(budget, self.cfg.base_problem_timeout)\nThis heuristic operates on two main principles: it guarantees a minimum baseline execution time (300 seconds) for all problems, and it dynamically allocates any \"saved\" time from rapidly solved problems to the current active problem, up to a maximum threshold (900 seconds).
\nThis greedy allocation strategy creates severe sensitivity to question ordering. We can illustrate this via two extreme simulated scenarios across 50 questions that give rise to the extreme 35- and 45-point variance when the question order changes:
\nTo demonstrate this, let us define a mock evaluation set of 50 questions categorized into three difficulty tiers based on the total execution time required to generate an answer:
\n| Question Type | \nTotal Questions | \nExecution time per question | \n
|---|---|---|
| Hard | \n5 | \n> 900 seconds | \n
| Intermediate | \n10 | \n300 to 900 seconds | \n
| Easy | \n35 | \n< 300 seconds | \n
\nPessimistic Scenario (35 Points):\n
i) 5 Hard Questions are encountered in the first 5 slots, the system fully exhausts the maximum time allowance per question, accumulating 4,500 seconds of compute without yielding correct answers.
\nii) Consequently, the remaining 45 questions are bottlenecked by the minimum 300-second allocation.
\niii) The 10 Intermediate Questions failed to produce a correct answer in 300 seconds.
\niv) As a result, only 35 points have been scored.
Optimistic Scenario (45 Points):\n
i) Hard Questions arrive during the last 5 slot.
\nii) The rest of 45 questions arrived in the earlier slot and all have enough time to finish execution, yielding a score of 45 points.
Mitigation: By entirely deprecating the dynamic time management code and enforcing a static compute budget per question, the systemic variance was eliminated, resulting in highly stable, reproducible leaderboard scores within a narrow 1-point margin (39 to 40 points).
\n2.2.Resolving Execution Faults in Agentic Environments
\nThe Tool-Integrated Reasoning environment utilizes a stateful Python Code Interpreter (via Jupyter kernels) to execute intermediate mathematical verifications. Initial implementations of this agentic environment suffered from severe systemic instability, frequently resulting in \"Notebook Inference Server Errors\" or silent process hangs. These issues scaled exponentially under high parallelization (e.g., managing 16–20 concurrent workers).
\nThrough rigorous debugging of the agent loop and sandbox architecture, we identified several race conditions, memory leaks, and uncatchable exceptions. To eliminate these intermittent crashes and guarantee back-to-back stable leaderboard scores, we re-architected the sandbox environment with the following mitigation strategies:
\nDynamic Port Allocation to Prevent Race Conditions:
\nThe baseline architecture manually assigned communication ports (shell, iopub, stdin, hb, control) using a locked incremental counter. Under high-concurrency spawning, this frequently led to port collisions. We eliminated manual port assignment, delegating port resolution entirely to the KernelManager to dynamically bind to available OS ports. This resolved initialization bottlenecks and silent kernel startup failures.
Zombie Process Mitigation and Hard Kills:
\nStandard kernel interruptions (interrupt_kernel()) often fail when Python is executing compiled C-extensions (e.g., complex matrix operations in numpy or deep symbolic recursions in sympy). These orphaned processes would continue to consume RAM, eventually causing out-of-memory (OOM) crashes. We implemented a cascading shutdown protocol: if a kernel ignores a standard interrupt after a 1-second grace period, the environment intercepts the process ID (PID) and forces an OS-level SIGKILL. This guarantees the termination of rogue memory hogs.
Active Health Monitoring via UUID Injection:
\nBecause a kernel can silently enter a deadlocked state without terminating, relying on process status is insufficient. We introduced an active is_healthy() heartbeat protocol. Before any tool execution, the sandbox injects a dynamically generated UUID into a print() statement with a strict 2-second timeout constraint. If the exact UUID is not returned, the kernel is definitively classified as unresponsive.
Self-Healing and Graceful State Recovery:
\nTo maintain inference continuity without crashing the broader agentic loop, we implemented a self-healing mechanism. If the health check fails—or if a kernel is hard-killed—the sandbox pool automatically discards the dead instance and spawns a fresh Jupyter kernel. Crucially, to prevent the LLM from hallucinating variable availability after a restart, the system automatically injects a state-reset warning back into the LLM's context window: [SYSTEM NOTE: The kernel crashed and was restarted. Previous variables are lost.] This prompts the model to intelligently re-declare necessary variables rather than failing on NameError exceptions.
By coupling these low-level sandbox optimizations with strict health thresholds, the agentic execution environment achieved near-perfect reliability, eliminating server-side inference errors within the strict 5-hour Kaggle compute window.
\n3.Optimizing Model Efficiency
\n3.1 Model Efficiency Evaluation
\nUnder the AIMO3 constraints, end-to-end execution time—defined as the total time required for an agent to generate a reasoning trace, execute the associated code, and produce a correct integer answer—is the primary bottleneck. We shortlisted three high-capability mid-size models for evaluation: GPT-OSS 120B, NVIDIA-Nemotron-3-Super, and Qwen3.5-122B-A10B.
\nEvaluations were conducted using two recent competitive datasets (HMMT February 2026 and AIME February 2026).
HMMT 2026 February
\n| Candidate Model | \nAttempt per Question | \nTokens per Question | \nEnd-to-end Time per Question (seconds) | \n
|---|---|---|---|
| GPT OSS 120B | \n8 | \n24332 | \n383 | \n
| NVIDIA-Nemotron-3-Super 120B | \n8 | \n45880 | \n433 | \n
| Qwen3.5-122B-A10B | \n8 | \n25103 | \n411 | \n
\nAIME 2026 February
| Candidate Model | \nAttempt per Question | \nTokens per Question | \nEnd-to-end Time per Question (seconds) | \n
|---|---|---|---|
| GPT OSS 120B | \n8 | \n20119 | \n204 | \n
| NVIDIA-Nemotron-3-Super 120B | \n8 | \n25233 | \n213 | \n
| Qwen3.5-122B-A10B | \n8 | \n21225 | \n235 | \n
\n3.2 Throughput Enhancements
To maximize the number of valid reasoning paths explored per problem within the time limit, raw inference throughput must be optimized without degrading precision. We implemented Speculative Decoding to accelerate token generation and applied FP8 KV-Cache Quantization.
\nThese memory and compute optimizations yielded a 2x increase in token throughput. Crucially, this efficiency gain allowed the system to scale its sampling budget beyond the standard 8 attempts per question. Increasing the number of reasoning trajectories directly correlated with higher, more stable leaderboard performance as evidence by screenshot below:
\n
4. Contamination-Free High-Difficulty Evaluation Dataset
\nA persistent challenge in evaluating mathematical LLMs is the risk of data contamination, where benchmark questions are inadvertently included in the model's pre-training corpus. To ensure highly reliable comparisons, we constructed a guaranteed contamination-free evaluation dataset.
\nMethodology:
\nTemporal Isolation: We restricted candidate questions exclusively to those published after June 2024, which serves as the training data cutoff date for our primary model (GPT-OSS 120B).
\nFormat Alignment: Rather than discarding questions with non-integer answers, we manually reformulated them into integer-output constraints, matching the precise stylistic guidelines of AIMO problem setters. This yielded an initial pool of 513 candidate questions.
\nDifficulty Filtering: To isolate high-difficulty reasoning challenges, we benchmarked the dataset using the top-performing public models. A question was retained only if it met one of two strict criteria:
\nThe model failed at least once across 8 independent sampling attempts.
The model succeeded in all 8 attempts, but the cumulative execution time exceeded 500 seconds, indicating high computational friction.
This rigorous filtering process resulted in a final dataset of 68 high-difficulty, verifiable, and zero-contamination mathematical problems.
\nStatistic on Source of Questions
\n
Having conducted a more careful review of the precise timings around this submission, we have determined that the post on the discussion thread was made around one hour after the 22nd April 23:59 UTC cutoff and therefore unfortunately this submission is not eligible for the Write Up prize. We still thank you for your contribution and it is of course available for the community to take any useful pointers from.
", "rawMarkdown": "Having conducted a more careful review of the precise timings around this submission, we have determined that the post on the [discussion thread](https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3/discussion/689703) was made around one hour after the 22nd April 23:59 UTC cutoff and therefore unfortunately this submission is not eligible for the Write Up prize. We still thank you for your contribution and it is of course available for the community to take any useful pointers from." }, { "id": 3459451, "postDate": "2026-05-17T20:36:00.670Z", "content": "Apologies for the delay in posting these questions. We're reviewing whether there's a reason this submission was missed from our initial pass; in any case, I've spent the evening reading through your write-up and have the questions below.
\nGiven the late notice, we can give you until Monday, 18th May, 23:59 UTC to respond as a minimum (we'll update if we decide to extend that). Rest assured we'll take the time to give the submission the attention it deserves, and note that we're yet to make a final decision on the winners so you're very much still in consideration here.
\nFor wider context on the rebuttal phase, please see the updated discussion post.
\nnvidia_lc, amazon_p, nvidia_v2, nvidia_tp) materially better or worse than wenliang?Look forward to reading your responses on these.
", "rawMarkdown": "Apologies for the delay in posting these questions. We're reviewing whether there's a reason this submission was missed from our initial pass; in any case, I've spent the evening reading through your write-up and have the questions below.\n\nGiven the late notice, we can give you until Monday, 18th May, 23:59 UTC to respond as a minimum (we'll update if we decide to extend that). Rest assured we'll take the time to give the submission the attention it deserves, and note that we're yet to make a final decision on the winners so you're very much still in consideration here.\n\nFor wider context on the rebuttal phase, please see the [updated discussion post](https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3/discussion/689703).\n\n1. **Empirical evidence for the variance fix:** The variance diagnosis in §2.1 is one of the more interesting results in the writeup. The supporting figures are analytical worst/best case derivations on a mock 50-problem distribution rather than measured LB outcomes — do you have any empirical evidence (e.g. a set of identical-code reruns, or on other benchmark datasets where you can get more detailed results) that supports the thought experiment?\n2. **Per-component breakdown of the 2× throughput claim:** §3.2 attributes a 2× throughput gain to FP8 KV-cache quantization + Speculative Decoding combined. Do you have a feel (or any data) for the split between these to optimisations. Also, are any of the other Eagle3 draft models you mention in the kernel (`nvidia_lc`, `amazon_p`, `nvidia_v2`, `nvidia_tp`) materially better or worse than `wenliang`?\n3. **Visualising the results:** Do you feel any charts would help bring the writeup to life, and if so are there any you'd want to add?\n\nLook forward to reading your responses on these.\n" } ], "complete": true, "source": "live", "fetched_at": "2026-10-20T04:24:46Z", "competition_id": 118448, "competition_slug": "ai-mathematical-olympiad-progress-prize-3", "team_id": 14801648, "solution_url": "/competitions/ai-mathematical-olympiad-progress-prize-3/writeups/road-to-deterministic-and-efficient-inference" }