![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1057275%2F4b2072e5a2071ec795ba49812ebac056%2Ffigure_5digit_v1_benchmark_composition.svg?generation=1779119240850704&alt=media\")
Recent large language models have achieved strong performance on mathematical reasoning benchmarks. For instance, gpt-oss-120b, the most utilized model in this competition, achieves an accuracy of approximately 98% on AIME'25. This makes it increasingly important to understand how already-strong models can be improved further. In this regime, post-training plays a central role. Additional adaptation after pre-training can still lead to meaningful differences in mathematical reasoning performance, especially when the goal is to push strong models beyond already high baselines.
\nAmong post-training approaches, supervised fine-tuning (SFT) and reinforcement learning (RL) are two of the most important and widely used paradigms. Both have shown promising results for mathematical reasoning, yet their roles in the high-accuracy regime remain insufficiently understood. When base models are already highly capable, it is often unclear how much improvement each method can provide, what kinds of behaviors each method induces, and how their benefits differ in practice. These questions are further complicated by the fact that commonly used benchmarks are becoming increasingly saturated, making it difficult to cleanly distinguish methods or to analyze the exact source of improvements (Balunović et al., 2025; An et al., 2025).
\nAt the same time, gains from post-training cannot be fully understood from final accuracy alone. In many settings, inference must be carried out under explicit constraints on latency, computation, or cost, and mathematical reasoning performance can be highly sensitive to inference-time configurations. Decoding hyperparameters, speculative decoding, and output selection or post-processing can significantly affect both solution quality and computational efficiency (Leviathan et al., 2023). As a result, the practical value of a post-trained model depends not only on whether it achieves higher accuracy but also on how effectively that accuracy can be realized under limited inference budgets. This motivates studying post-training and inference-time design jointly, rather than as isolated components.
\nIn this work, we investigate how to further improve already-strong mathematical reasoning models through post-training under realistic inference constraints. We strengthen evaluation beyond standard benchmarks with an expanded set of challenging mathematical problems that better discriminate performance in the high-accuracy regime. Building upon this evaluation framework, we compare two major post-training strategies, supervised fine-tuning and reinforcement learning, and analyze how they differ in the quality gains they provide, the behaviors they induce, and their interactions with inference-time choices. We further examine practical inference-time mechanisms, including decoding temperature, speculative decoding, and post-processing or selection, to study how these factors jointly affect the attainable quality-resource frontier. Through this perspective, we aim to elucidate not only how to enhance strong models but also how to maximize the utility of those improvements under explicit inference constraints.
\nOur study yields several key findings:\nFirst, even in the already-strong regime, additional post-training can produce meaningful gains, and the magnitude and characteristics of those gains differ significantly between supervised fine-tuning and reinforcement learning. \nSecond, the value of post-training depends intrinsically on the inference procedure utilized at test time. Decoding and selection choices can substantially shift the effective operating point of a model, amplifying or diminishing the gains obtained from training. \nThird, comparisons based solely on unconstrained final accuracy can obscure critical differences between methods. When evaluated under explicit resource constraints, approaches with equivalent accuracy may exhibit substantially disparate efficiency profiles. \nTaken together, these results suggest that improving mathematical reasoning in the high-performance regime necessitates a coupled approach of effective post-training and careful inference design.
\nOur core contributions are as follows:
\nFocus on Integer Problems and Limitations of Existing Benchmarks\nTo tightly control the evaluation setting and isolate specific mathematical reasoning capabilities, we restrict our focus exclusively to integer-answer problems. Traditionally, benchmarks such as AIME, HMMT, and reference problems from AIMO 1-3 have served as the standard for this difficulty tier. However, recent state-of-the-art models have achieved near complete coverage on these legacy datasets. This saturation strongly suggests potential data contamination within pre-training corpora, rendering these existing benchmarks insufficient for accurately assessing true zero-shot reasoning capabilities (Balunović et al., 2025; An et al., 2025).
\nTo address this limitation and construct a robust, uncontaminated evaluation suite, we explicitly incorporated the recent 2025 editions of AIME and HMMT, alongside integer problems systematically extracted from the MathArena benchmark (Balunović et al., 2025). Furthermore, to diversify the problem distribution and guarantee extreme difficulty without contamination, we aggregated and rigorously filtered problem sets from four additional distinct sources.
\n1. Japanese Mathematical Olympiad (JMO)\nWe incorporated integer problems from the Japanese Mathematical Olympiad (JMO) to introduce novel, high-quality challenges that possess a significantly lower probability of inclusion in English-dominant training datasets.
\n2. Filtering from Nemotron-Math-v2\nWe curated a challenging subset from the Nemotron-Math-v2 dataset based on the empirical performance of the gpt-oss-120b model under varying reasoning effort configurations (low, medium, and high) (NVIDIA, 2025; OpenAI, 2025). To isolate high-complexity problems, we aggregated all instances where the model failed to answer correctly under either the low or medium effort configurations (i.e., Accuracy (low) = 0 \\lor Accuracy (medium) = 0). For these selected instances, we retained the problem text, the expected ground-truth answer, and the unique identifier. This filtering effectively eliminates trivial tasks, yielding a refined testbed for advanced reasoning assessment.
3. Curating the AoPS 2025 Contest Collection\nWe leveraged the 2025 subset of the NuminaMath dataset, sourced from the Art of Problem Solving (AoPS) Contest Collection (AI-MO, 2024). To prevent overlap with existing benchmarks, problems from widely recognized competitions (e.g., AIME) were explicitly excluded. An LLM-in-the-loop pipeline was employed to rewrite problem statements, automatically verify validity, and extract ground-truth answers from forum discussions. Subsequently, we applied a strict empirical filtering criterion using gpt-oss-120b configured with high reasoning effort and Tool-Integrated Reasoning (TIR). By sampling 8 inference trajectories per problem, we retained only instances with a success rate strictly between 0% and 100%. While the upper bound excludes trivial tasks, the lower bound (>0%) serves as a critical quality control mechanism; manual inspection confirmed that problems with a 0% success rate frequently contained flawed statements or incorrect ground-truth labels.
4. Adaptation of the IMO-Answer Bench into Value-Finding Tasks\nTo ensure an exceptionally difficult evaluation set, we utilized IMO-AnswerBench from IMO-Bench (Luong et al., 2025). We identified problems within this benchmark that could be logically restructured into definitive integer-valued tasks. By adapting these problems into an integer-answer format, we established a rigorous testbed that strictly assesses advanced reasoning while remaining highly resistant to standard pre-training memorization.
\nFinal Dataset Composition\nThe compilation of these carefully curated sources results in our final evaluation benchmark, comprising 285 high-quality integer problems. The composition is detailed in Figure 1.
\n
Figure 1: Composition of the Final Evaluation Benchmark
\nEvaluating reasoning capabilities without strict computational bounds yields an inaccurate representation of practical model utility, as scaling test-time compute naturally inflates performance. To establish a rigorous baseline, we explicitly constrain our experimental environment to a single NVIDIA H100 GPU with a strict execution time limit of 5 hours per evaluation run. This constraint models the official environment of the Kaggle AI Mathematical Olympiad Progress Prize 3, ensuring the evaluation reflects realistic, resource-bound scenarios (Kaggle, 2025).
\nTo assess joint performance and inference efficiency within this environment, we constructed an additional targeted evaluation subset consisting of 50 problems.These instances were strategically sampled from the Crystal Math Preview dataset (Chen, 2026) such that the total expected inference time closely aligns with the empirical runtime observed on the public leaderboard. This specialized subset provides a controlled testbed for evaluating the trade-off between reasoning accuracy and computational speed.
\nGiven that gpt-oss-120b already achieves high performance (OpenAI, 2025), particularly when configured with high reasoning effort and tool-integrated reasoning (TIR), the methodology for further improving such an extensively post-trained model is non-trivial. In this study, we investigate the application of supervised fine-tuning (SFT) and reinforcement learning (RL) to enhance the original gpt-oss-120b model, analyzing the distinct performance gains and behavioral shifts induced by each approach.
Our preliminary experiments on Nemotron-Math-v2 yielded two primary observations. First, training the smaller gpt-oss-20b model on trajectories generated by gpt-oss-120b improved downstream mathematical performance. Second, applying the identical self-distillation recipe to gpt-oss-120b itself produced negligible improvements. Consequently, we generated novel training trajectories using several external strong reasoning models, utilizing these traces to introduce reasoning patterns not reliably exhibited by the base model.
Motivated by the extended-SFT paradigms observed in recent literature (Yoshihara et al., 2025), we adopted a prolonged fine-tuning approach. We hypothesized that repeated supervised training on a carefully curated, high-quality mixture of mathematical traces drives sustained improvements in reasoning reliability.
\nWe constructed the SFT dataset with two primary objectives: aggressive filtering for difficulty, and trajectory generation via stronger models.
\nFor difficulty filtering, we utilized Nemotron-Math-v2 and Crystal-Math-Preview. Nemotron-Math-v2 contains approximately 347k problems with released solution traces. The availability of traces varies because the dataset was generated across multiple inference settings (with and without TIR, varying effort levels), retaining only correct trajectories. Thus, problems with fewer retained traces serve as a proxy for higher difficulty. We isolated problems possessing at most four retained trajectories and generated entirely new solution traces for them. For Crystal-Math-Preview, we utilized the verified rewritten subset. We strictly excluded any problems that overlapped with our evaluation benchmark.
\nNew trajectories were synthesized using Kimi-K2.5, DeepSeek-V3.2-Speciale, GLM-4.7, and GLM-5. Generation was conducted separately for no-TIR and TIR configurations. For each problem, we sampled four candidate solutions, retaining only those that terminated normally and yielded the correct answer. Correctness was verified via exact string matching for integer problems, and via gpt-oss-120b acting as an LLM judge for non-integer formats.
Table 1 summarizes the curated trajectories utilized for fine-tuning following preprocessing.
\nTable 1: SFT Dataset Composition and Token Distribution
\n| Source Dataset | \nInference Mode | \nGenerator Models | \nCurated Trajectories | \nTotal Tokens | \n
|---|---|---|---|---|
| Nemotron-Math-V2 | \nNo TIR | \nKimi-K2.5, DeepSeek-V3.2-Speciale | \n8,403 | \n195,721,010 | \n
| Nemotron-Math-V2 | \nTIR | \nKimi-K2.5, GLM-4.7 | \n1,296 | \n32,778,255 | \n
| Crystal-Math-Preview | \nNo TIR | \nKimi-K2.5, GLM-5 | \n8,073 | \n290,345,130 | \n
| Crystal-Math-Preview | \nTIR | \nKimi-K2.5 | \n4,515 | \n150,876,560 | \n
| Total Unique Data | \n- | \n- | \n22,287 | \n669,720,955 | \n
We fine-tuned the gpt-oss-120b base model utilizing full-parameter supervised fine-tuning. The maximum sequence length was set to 65,536 tokens, with sequence packing enabled to maximize throughput. Training was conducted for 10 epochs using a global batch size of 256. Optimization was performed using the Adam optimizer with a peak learning rate of 8e-6, a minimum learning rate of 8e-7, a cosine decay schedule, a 5% linear warmup, a weight decay of 0.1, and gradient clipping capped at 1.0.
To emphasize the acquisition of tool-use capabilities, the dataset mixture heavily upsampled TIR trajectories. During each epoch, no-TIR data were sampled once, whereas each TIR trajectory was sampled three times. Consequently, while TIR examples constituted only 5,811 of the 22,287 unique trajectories, they accounted for 551 million of the 1.037 billion effective tokens processed per epoch. Across the full 10-epoch training run, the model was exposed to approximately 10.37 billion tokens.
\nModels were evaluated on the comprehensive, high-difficulty benchmark constructed in Section 2. To robustly estimate performance, we generated 16 independent inference trajectories per problem. We report the pass@1 metric (the empirical average accuracy across the 16 samples) and the majority voting accuracy at k=8 (Maj@8), computed via 2,000 bootstrap iterations over the generated samples. Finally, we report the coverage (Pass@16), which denotes the percentage of problems solved correctly at least once among the 16 samples.
\nTable 2 compares the base model against the fine-tuned models at different stages of the training process (2, 5, and 10 epochs). All metrics are reported as absolute percentages.
\nTable 2: Performance Evolution Across SFT Epochs
\n| Inference Mode | \nTraining Epochs | \nEffective Token Exposure | \npass@1 ↑ | \nMaj@8 ↑ | \nCoverage (Pass@16) ↑ | \n
|---|---|---|---|---|---|
| TIR | \nBase | \n\n | 72.61 | \n77.86 | \n92.28 | \n
| \n | 2 | \n2.07B | \n68.60 | \n75.88 | \n87.02 | \n
| \n | 5 | \n5.19B | \n70.88 | \n76.85 | \n88.77 | \n
| \n | 10 | \n10.37B | \n73.38 | \n78.62 | \n90.88 | \n
| No TIR | \nBase | \n\n | 64.65 | \n72.04 | \n84.56 | \n
| \n | 2 | \n2.07B | \n62.04 | \n69.43 | \n83.51 | \n
| \n | 5 | \n5.19B | \n63.86 | \n71.67 | \n84.21 | \n
| \n | 10 | \n10.37B | \n65.79 | \n72.31 | \n83.51 | \n
The models fine-tuned for 2 and 5 epochs exhibit performance degradation relative to the base model on both pass@1 and Maj@8 across both inference paradigms. Conversely, fine-tuning for 10 epochs yields systematic improvements over the base model, achieving 73.38% pass@1 and 78.62% Maj@8 with tools enabled, alongside 65.79% pass@1 and 72.31% Maj@8 without tools.
\nNotably, maximum coverage (Pass@16) remains highest for the base model. This demonstrates that the primary mechanism of improvement derived from prolonged SFT is not an expansion of the model's absolute solvable problem space, but rather a significant enhancement in the average correctness and majority-vote reliability of the generated solutions. These results validate the efficacy of extended supervised fine-tuning as a mechanism for repeated exposure to a compact, verified set of difficult mathematical trajectories.
\nAs demonstrated in previous evaluations, the gpt-oss-120b model already achieves strong mathematical reasoning performance, particularly when utilizing tool-integrated reasoning (TIR) under high reasoning effort. Improving this baseline via RL requires careful data selection because training on easily solved problems wastes compute. Consequently, our RL strategy relies on strict data filtering to isolate problems at the boundary of the model's competence.
We initialized our RL training dataset using the mathematical subset of the allenai/Dolci-Think-RL-7B dataset (Allen Institute for AI, 2025), which originally comprises 60,360 problems. To construct a highly efficient learning curriculum, we applied a strict empirical filtering to eliminate both trivial tasks where the model requires no further optimization.
The filtering was conducted using the base gpt-oss-120b model configured with high reasoning effort and TIR enabled. To determine the empirical pass rate for filtering, we ran 16 independent inference trials for each of the candidate problems with a maximum generation length of 65,536 tokens.
We applied the following retention criteria:
\nThis boundary condition successfully filters out problems that are reliably solved, forcing the RL algorithm to optimize on instances where the model exhibits high variance or uncertainty. Following this procedure, the dataset was reduced to 13,358 highly discriminative problems, establishing a concentrated training set.
\nWe adopted Group Relative Policy Optimization (GRPO) (Shao et al., 2024), enhanced with Clip-Higher and Dynamic Sampling from DAPO (Yu et al., 2025), Truncated Importance Sampling (TIS) (Yao et al., 2025), and removal of the KL loss term. We maintained a substantial maximum sequence length of 65,536 tokens during rollouts. To avoid bottlenecks caused by long-tail trajectories during the rollout phase, we applied the partial rollout technique (Kimi Team, 2025). During training, we generated 16 trajectory samples per prompt to estimate the relative advantages, strictly matching our evaluation methodology. The final evaluated RL model was selected from the checkpoint at training step 70. Comprehensive details regarding the distributed training infrastructure, parallelism strategies, and RL-specific hyper-parameters are deferred to Appendix A.
\nThe RL model was evaluated on our comprehensive benchmark. Table 3 presents the absolute performance in percentage points.
\nTable 3: Performance of the RL-Tuned Model
\n| Inference Mode | \nTraining Strategy | \npass@1 ↑ | \nMaj@8 ↑ | \nCoverage (Pass@16) ↑ | \n
|---|---|---|---|---|
| TIR | \nBase | \n72.61 | \n77.86 | \n92.28 | \n
| \n | RL | \n73.61 | \n78.78 | \n90.17 | \n
| No TIR | \nBase | \n64.65 | \n72.04 | \n84.56 | \n
| \n | RL | \n65.61 | \n71.71 | \n83.85 | \n
When evaluated without tools, RL improves pass@1 but slightly degrades Maj@8 and Coverage. Under the TIR setting, the RL process slightly improves the expected pass@1 accuracy to 73.61% and Maj@8 to 78.78%. However, this comes with a marginal reduction in overall coverage (90.17%) relative to the base model. This divergence highlights a critical dynamic in the high-accuracy regime. While RL effectively shifts the probability mass toward correct answers on an individual sample basis (pass@1, Maj@8), it can also induce a narrowing of the exploration space, slightly reducing the broader trajectory diversity required for maximum coverage at high sampling budgets.
\nIn this section, we discuss the trade-off between accuracy and latency governed by temperature, particularly when utilizing speculative decoding. In the following experiments and our submission, we used Eagle3 as the draft model. For a comparison with an alternative speculative decoding method, DFlash, which we found less optimal at higher temperatures, please refer to Appendix D.
\nWe observe that increasing the decoding temperature extends inference time when utilizing speculative decoding (Leviathan et al., 2023). To quantify this effect, we evaluated our model using the Time-Constrained Evaluation Benchmark established earlier. This specialized 50-problem subset allows us to accurately approximate the strict runtime limits of the simulated submission environment. For each problem, we generated 8 concurrent candidate solutions on a single NVIDIA H100 GPU. The measured runtime metrics across different temperature settings with speculative decoding are presented in Table 4.
\nTable 4 Runtime and Speculative Decoding Efficiency by Temperature
\n| Temperature | \nRuntime (s/run) | \nTotal Wall Time (min) | \nTokens / Run | \nMean accept rate | \nMean accept length | \n
|---|---|---|---|---|---|
| 0.5 | \n247 | \n226 | \n31,500 | \n0.504 | \n3.02 | \n
| 0.6 | \n246 | \n230 | \n32,200 | \n0.492 | \n2.97 | \n
| 0.7 | \n248 | \n232 | \n32,400 | \n0.481 | \n2.93 | \n
| 0.8 | \n265 | \n242 | \n33,200 | \n0.461 | \n2.85 | \n
| 0.9 | \n273 | \n235 | \n33,500 | \n0.431 | \n2.72 | \n
| 1.0 | \n291 | \n253 | \n33,500 | \n0.370 | \n2.48 | \n
| 1.0 without speculative decoding | \n391 | \n342 | \n33,300 | \nN/A | \nN/A | \n
The table indicates that the reduction in inference speed is attributable to two primary factors. First, higher temperatures induce longer average output lengths. Second, elevated temperatures decrease the token acceptance rate of speculative decoding, thereby diminishing its acceleration benefits.
\nTo assess the corresponding impact on reasoning quality, we evaluated model performance across varying temperatures on our comprehensive benchmark. We generated 16 independent trajectories for each of the 285 problems per temperature setting. Figure 2 reports the mean accuracy, bootstrap majority-vote accuracy, coverage, and generation token length based on the proportion of correct solutions among the total generated trajectories.
\nFigure 2 Temperature sweep for the final SFT model on our evaluation set.\nWe evaluate decoding temperatures from 0.5 to 1.5 and report mean accuracy, bootstrap majority-vote accuracy, coverage (pass@16), and generation length. Bootstrap majority accuracy is computed with majority voting over 8 sampled responses. Error bars show one standard deviation across bootstrap estimates. Coverage denotes the fraction of problems solved by at least one sampled response. Token length is shown as the mean number of generated tokens, with the shaded region indicating \\pm 1 standard deviation across generations. Gray dashed lines indicate the baseline model.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1057275%2F9a6d8f1dde8d9b59fea82a27a36059d8%2Ffigure_temperature_sweep_4panel_neurips.svg?generation=1779115807151259&alt=media\")
The results demonstrate a clear trend where higher temperatures generally yield improved reasoning performance. Synthesizing these findings reveals a fundamental trade-off under constrained computing environments. Increasing the decoding temperature improves mathematical reasoning accuracy but incurs a measurable cost in inference latency, primarily due to degraded speculative decoding efficiency and increased generation lengths.
\nIn addition to tuning the decoding temperature, we investigated advanced test-time scaling methods that prompt the model to select or refine its own solutions. Specifically, we evaluated \"Self Aggregation Refinement (SAR)\" and \"GenSelect.\" While both approaches demonstrated the potential to outperform standard majority voting under certain conditions, they significantly increased pipeline complexity and proved highly sensitive to truncation caused by context length constraints. Ultimately, we concluded that simple temperature scaling offered the most robust trade-off between inference time and accuracy within the strict five-hour time limit, and we did not adopt these advanced aggregation methods in our final submission. Detailed experimental setups and results are provided in Appendix C.
\nFigure 3: Performance comparison between the RL and SFT models across varying decoding temperatures (T). The baseline model is evaluated at the default setting of T=1.0. Metrics are reported as Pass@1 / Maj@8 / Coverage (%). The highest achievable performance under the strict 5-hour runtime constraint is highlighted in bold for both models (the RL model is computationally constrained to T \\le 0.8, whereas the SFT model can operate safely at T=1.0).
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1057275%2Fd78e0d71f6094f7fa8aaf6208a2d69d4%2Ffigure7_temperature_sweep.svg?generation=1779094994681550&alt=media\")
Both SFT and RL demonstrate strong efficacy as post-training methodologies, yielding highly competitive models that effectively enhance the reasoning capabilities of the base model. In broader empirical evaluations, both approaches consistently achieved public scores exceeding 40, with the RL-tuned model attaining a peak score of 44 and the SFT model reaching a maximum of 43. Despite these strong absolute performances, we observed a minor behavioral divergence during generation. The RL process exhibited a tendency to induce slightly prolonged inference times.
\nUltimately, we conducted empirical timing evaluations within the official Kaggle submission environment. To guarantee that inference would complete reliably across both the public and private evaluation splits, we sought generation configurations that preserved a safety margin of at least 30 minutes prior to the strict runtime deadline. Our analysis revealed that to ensure the RL model finished within this acceptable window, its decoding temperature could be set no higher than 0.8. In contrast, the SFT model comfortably completed all inference tasks even when the temperature was elevated to 1.0. As demonstrated by the results in Figure 3, the SFT model operating at T=1.0 yields superior performance compared to the RL model constrained to T=0.8. Consequently, we selected the SFT model for our final submission.
\nTraining was performed with Megatron-SWIFT on 4 nodes with 8 GPUs per node. We configured the distributed setup with pipeline parallelism 4 (PP=4) and context parallelism 1 (CP=1). Specifically, to optimize the MoE architecture, we applied tensor parallelism 4 (TP=4) to the attention layers, while configuring the MoE layers with expert parallelism 8 (EP=8) and no tensor parallelism (TP=1). The model was trained in bfloat16 precision from a Megatron-Core checkpoint of gpt-oss-120b.
Table A.1: SFT Hyper-parameters
\n| Parameter | \nValue | \n
|---|---|
| Max sequence length | \n65,536 | \n
| Packing | \nEnabled | \n
| Epochs | \n10 | \n
| Global batch size | \n256 | \n
| Optimizer | \nAdam | \n
| Adam beta1 / beta2 / eps | \n0.9 / 0.95 / 1e-8 | \n
| Learning rate | \n8e-6 | \n
| Minimum learning rate | \n8e-7 | \n
| LR schedule | \nCosine decay | \n
| Warmup | \n5% of training steps | \n
| Weight decay | \n0.1 | \n
| Gradient clipping | \n1.0 | \n
| Attention softmax | \nfp32 | \n
| Attention dropout | \n0.0 | \n
Reinforcement learning was executed on an identical physical infrastructure of 4 nodes with 8 GPUs per node. For the RL workload, we maintained pipeline parallelism 4 (PP=4) and context parallelism 1 (CP=1). The parallelism strategy was decoupled across layer types: the attention layers utilized tensor parallelism 8 (TP=8), whereas the MoE layers were distributed using expert parallelism 8 (EP=8) with tensor parallelism 1 (TP=1). The model was optimized in bfloat16 precision, initialized directly from the gpt-oss-120b bfloat16 checkpoint.
Table A.2: RL (GRPO) Hyper-parameters
\n| Parameter | \nValue | \n
|---|---|
| Algorithm | \nGRPO | \n
| Max sequence length | \n65,536 | \n
| Samples per prompt | \n16 | \n
| Rollout batch size | \n64 | \n
| Global batch size | \n1024 | \n
| Selected checkpoint | \nStep 70 | \n
| Optimizer | \nAdamW | \n
| Adam beta1 / beta2 | \n0.9 / 0.98 | \n
| Learning rate | \n1e-6 | \n
| LR schedule | \nConstant | \n
| Weight decay | \n0.1 | \n
| GRPO clip range | \n[0.2, 0.28] | \n
| KL loss coefficient | \n0.0 | \n
| Entropy coefficient | \n0.0 | \n
To execute Python code in a stateful manner, we constructed a custom sandbox environment using Docker. During local training, data generation, and Kaggle notebook submissions, we utilized identical local Jupyter kernel sessions to ensure strict environmental consistency.
\nWe implemented several optimizations within this Python execution environment to maximize efficiency and stability:
\nnumpy, scipy, sympy, mpmath, networkx, numba, ortools, pulp, python-sat, z3-solver, and shapely.--color=NoColor flag. This simple adjustment drastically reduced the token length of Python error tracebacks (e.g., reducing a standard KeyError traceback from 805 tokens to 112 tokens, an 86.1% reduction).PREFIX...[truncated]...POSTFIX to allow the LLM to recover and proceed in the subsequent turn.When curating the synthetic reasoning traces for Supervised Fine-Tuning (SFT), we applied specific preprocessing steps to align the training data with our test-time inference strategy:
\n\\boxed{} Tags: In environments where latency is critical, many implementations utilize an early-stopping mechanism that halts generation and extracts the answer the moment the first \\boxed{} tag appears. However, source datasets like Nemotron-Math-v2 typically rely on the final \\boxed{} tag for evaluation. Consequently, models frequently use \\boxed{} to highlight incorrect intermediate guesses before reaching the correct final answer. To prevent early-stopping failures at inference time, we systematically sanitized the training traces by removing or masking any incorrect \\boxed{} tags appearing in intermediate reasoning steps.!pip install commands or attempted to search the web for the problem statement. Because the target Kaggle submission environment strictly prohibits internet access, models cannot execute these actions at test time. We explicitly filtered out all traces containing network requests or dynamic library installations to prevent the model from learning strategies it cannot execute.Our data generation pipeline utilized a modified prompt structure tailored to our tool-integrated reasoning environment. We specifically augmented the tool prompt to explicitly notify the model of the pre-installed libraries available in our custom environment, reducing unnecessary installation attempts.
\nUser Prompt Structure:
\n{problem_text}\n\nPlease reason step by step to solve the math problem. Finally, Return only the verified final answer in `\\boxed{}`, where the answer is an integer in [0, 99999]. Never guess.\n\nYou may use the python tool for any non-trivial computation or verification.\nTool System Prompt:
\nUse this tool to execute Python code in your chain of thought. The code will not be shown to the user. This tool should be used for internal reasoning, but not for code that is intended to be visible to the user (e.g. when creating plots, tables, or files).\n\nWhen you send a message containing Python code to python, it will be executed in a stateful Jupyter notebook environment so the past imported library and past defined variables can be accessed without re-importing or re-defining them.\n\nPython will respond with the output of the execution or time out after {timeout_sec} seconds. So you must not implement a super compute heavy logic which cannot be calculated within the timeout {timeout_sec} seconds.\n\nYou can use the built-in python libraries and pre-installed libraries below:\n`numpy`: array operations\n`sympy`: symbolic mathematics\n`mpmath`: float precision\n`networkx`: graph theory\n`numba`: JIT compilation\n`ortools`: linear programming\n`pulp`: mixed integer programming\n`python-sat`: SAT solver\n`scipy`: scientific computing\n`shapely`: geometric operations\n\nInternet access is enabled but it is only for the purpose of python library installation when really necessary. If you need other libraries than the ones listed above, you may use the code \"!pip install <library_name>\" to try installing the library. You must not search the internet to find the problem itself and its answer.\nTo maximize pass@N performance within the strict five-hour inference budget, we evaluated two advanced selection and aggregation methods.
\nWe propose Self Aggregation Refinement (SAR), an approach built upon the recently introduced Recursive Self Aggregation (RSA) framework (Siddarth Venkatraman et al., 2026). To minimize latency, SAR modifies the recursive nature of the original RSA method into a single-turn refinement process. We explicitly instructed the model to identify and verify the most mathematically sound reasoning among the existing candidates, without synthesizing a new approach from scratch unless all provided candidates were demonstrably incorrect.
\nTo evaluate SAR, we conducted 4 experimental runs on a local benchmark of 133 high-difficulty problems, generating 8 candidate traces per problem in each run (totaling 532 trials). To optimize for practical time constraints, SAR was activated only when the standard majority voting failed to reach the early-stopping threshold of 4 votes (indicating high uncertainty). When activated, we applied hyperparameters $N=6$ and $K=4$, meaning we randomly selected 4 of the 8 traces and generated 6 parallel refined solutions, ultimately using entropy voting to determine the final answer.
\nAcross the 532 trials, the activation condition was met 126 times. Table C.1 details the performance changes categorized by the margin (the difference in votes between the first and second majority answers) prior to SAR activation.
\nTable C.1: Performance changes upon SAR activation, categorized by vote margin. \"Improved\" indicates instances where SAR corrected a previously wrong answer. \"Hurt\" indicates where SAR overturned a correct answer.
\n| Margin | \nTotal Activations | \nImproved ↑ | \nHurt ↓ | \nNet Gain | \nNet Gain % | \n
|---|---|---|---|---|---|
| 0 | \n53 | \n11 | \n2 | \n+9 | \n+16.9% | \n
| 1 | \n53 | \n7 | \n5 | \n+2 | \n+3.7% | \n
| 2+ | \n20 | \n0 | \n3 | \n-3 | \n-15.0% | \n
Applying SAR to problems with a margin of 0 yielded an expected net gain of +16.9%. However, because this method incorporates multiple full reasoning traces into the prompt, the context length expands rapidly. We observed that our SFT model's performance degraded when exceeding its training context length of 65,536 tokens, necessitating aggressive truncation of the input to 60k tokens. This context limitation remained a significant practical bottleneck.
\nWe also evaluated \"GenSelect,\" an approach proposed by the winning team of AIMO 2 (Ivan Moshkov et al., 2025). This method simultaneously presents multiple candidate solutions to the language model, prompting it to directly compare them and select the most mathematically sound option. For larger candidate pools, this process is structured as a tournament.
\nFor the GenSelect evaluation, we pre-generated a pool of 32 traces (with tool-integrated reasoning enabled) for each of the 133 high-difficulty benchmark problems. From this pool, we bootstrap-sampled 8 traces, repeating this process four times per problem. To simulate submission conditions, we restricted GenSelect to activate only when the size of the simple majority among the 8 traces was 4 or fewer.
\nGiven the increased difficulty of problems in this competition, individual reasoning traces were substantially longer, meaning a direct application of the original method would easily exceed the context window. To mitigate this, we allocated a budget of 48,000 tokens exclusively for the candidate solutions within the model's 65,536-token context window. This budget was divided evenly by the \"match size\" (the number of solutions compared simultaneously), and any text exceeding the per-solution allowance was truncated from the right. For each activated sample, we repeated the GenSelect process 4 times. Table C.2 illustrates the behavior of GenSelect across match sizes of 2, 4, and 8.
\nTable C.2: GenSelect accuracy across different match sizes.
\n| Match Size | \nAccuracy (Pass) | \nAccuracy (Simple Majority) | \nAccuracy (GenSelect Majority) | \n
|---|---|---|---|
| 2 | \n0.788 | \n0.372 | \n0.379 | \n
| 4 | \n0.788 | \n0.372 | \n0.452 | \n
| 8 | \n0.788 | \n0.372 | \n0.328 | \n
When the match size was set to 2 or 4, the GenSelect majority outperformed the simple majority voting accuracy of 0.372, achieving a notable improvement to 0.452 at a match size of 4. However, increasing the match size to 8 severely constrained the token budget per trace. The resulting truncation discarded critical logical structures and mathematical steps, causing the performance to drop below even the simple majority baseline.
\nFor faster inference, we tried a speculative decoding method called DFlash (Jian Chen et al., 2026). Unlike EAGLE3, which predicts one token at a time, DFlash uses a small diffusion model with around 3–8 layers to predict roughly 4–16 tokens simultaneously, and in some cases can run faster than EAGLE3. In the environment used for our final submission, DFlash was faster than EAGLE3 when the inference temperature was lowered, for example at T=0.5; however, at the final setting we chose, T=1.0, EAGLE3 was faster.
", "comments": [ { "id": 3447559, "author_name": "ivan", "author_url": "", "post_date": "2026-04-23T10:56:59.753000", "content": "Congrats on successfully fine-tuning gpt-oss with sft and rl! But I have a question about Eagle3 model. Did you fine-tune it by yourself? How it behaves on long trajectories (after 30k tokens)? I also see you use time management as in public notebook. Do you think it's optimal? Did you experiment with different time allocation?
", "votes": 5, "replies": [ { "id": 3483541, "author_name": "Inoichan", "author_url": "", "post_date": "2026-06-29T06:44:19.980000", "content": "First of all, I am so incredibly sorry for the super late reply!
\nRegarding your questions about the Eagle3 model, we further fine-tuned wenliangtlh's draft model on our custom data, which gave us a small boost of a few percentage points compared with the original model. We observed that the acceptance rate tends to decrease as trajectories get longer, but this was partially mitigated by training with more long traces of around 65,536 tokens.
\nAs for the time allocation, we did try a few different settings, but ultimately settled on the current one. In our experiments, we mostly kept the time allocation fixed and spent more time focusing on tweaking the temperature and devising post-processing techniques. Although we didn't end up using it in our final solution, we described our post-processing approach in Appendix C. Please feel free to check it out if you are interested!
", "votes": 1, "replies": [] } ] }, { "id": 3450492, "author_name": "Yi-Chia Chen", "author_url": "", "post_date": "2026-04-29T21:43:16.743000", "content": "Great writeup! I’m glad our CrystalMath could be of some help to your work. Do you plan to release the full training code? I believe it would be very valuable for the community if available.
", "votes": 3, "replies": [ { "id": 3454910, "author_name": "corochann", "author_url": "", "post_date": "2026-05-08T07:00:38.840000", "content": "Dataset quality is really important and preparing difficult yet answerable problems are getting harder as LLM's performance improves. Thank you for the effort to create high quality dataset.
\nWe have published the SFT dataset including Crystal-Math-Preview's solution trajectory
\n\nWe found it difficult to improve gpt-oss-120b's math performance further with the existing open dataset, and I also believe this will be valuable to the future research community 🐠🐟
", "votes": 3, "replies": [] } ] }, { "id": 3459443, "author_name": "Sam Bealing", "author_url": "", "post_date": "2026-05-17T20:23:33.690000", "content": "Apologies for the delay in posting these questions. We're reviewing whether there's a reason this submission was missed from our initial pass; in any case, I've spent the evening reading through your write-up and have the questions below.
\nGiven the late notice, we can give you until Monday, 18th May, 23:59 UTC to respond as a minimum (we'll update if we decide to extend that). Rest assured we'll take the time to give the submission the attention it deserves, and note that we're yet to make a final decision on the winners so you're very much still in consideration here.
\nFor wider context on the rebuttal phase, please see the updated discussion post.
\nLook forward to reading your responses on these.
", "votes": 1, "replies": [ { "id": 3460015, "author_name": "Inoichan", "author_url": "", "post_date": "2026-05-18T14:46:25.047000", "content": "We sincerely thank @sambealing for this insightful question and the opportunity to clarify our inference strategy.\nFollowing your feedback, we extended our evaluation to include decoding temperatures up to 1.5. To ensure maximum reliability, we also increased the sampling budget from 4 to 16 trajectories per problem across all temperature settings. We have visualized these updated results in the newly added Figure 2 in our write-up (replacing the previous tabular format to better illustrate the trends).
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1057275%2F512d44671ce23c451d9d898d854edfbd%2Ffigure_temperature_sweep_4panel_neurips.svg?generation=1779115239451193&alt=media\")
As the results show, performance steadily improves from T=0.5, peaking exactly at T=1.0 for both mean accuracy and majority voting accuracy, before degrading toward T=1.5. Coverage (pass@16) peaks early at T=0.6 and remains relatively stable up to T=1.0. Concurrently, token length (and therefore inference latency) increases monotonically with temperature. Based on this extended sweep, we are confident that T=1.0 is indeed the optimal operating point, maximizing reasoning performance without incurring the severe latency costs seen at higher temperatures.
\nRegarding the apparent latency headroom (finishing in ~4.2 hours against the 5-hour cap), this reflects a deliberate risk-mitigation strategy. In our local evaluations, we observed that failing to allocate sufficient inference time to each problem causes severe score degradation. Anticipating that the Private LB would feature a significantly harder problem distribution, we prioritized strict pipeline reliability to ensure every problem received its full computational budget. While advanced post-processing methods like GenSelect or SAR (detailed in Appendix C) showed potential to boost accuracy, incorporating LLM-as-a-judge steps introduced significant variance in execution time. The severe penalty of hitting the time limit and leaving problems unattempted far outweighed the marginal accuracy gains. Therefore, we prioritized execution stability and opted for the highly predictable temperature scaling approach.
\nWe greatly appreciate you pointing out this fundamental and crucial aspect of the pipeline. Managing trajectory quality was indeed one of the areas we focused on most intensively throughout the competition.
\nFirst, we would like to detail our preprocessing pipeline for the AoPS data. We implemented an enhanced version of the Nemotron-Math-v2 data curation approach. We categorized problems by type (e.g., integer, non-integer, proof, multiple-choice, binary, image-required) and domain (Geometry, Algebra, Number Theory, Combinatorics, etc.). Crucially, forum posts often contained multiple problems, or single problems contained multiple sub-questions. We parsed these and extracted them individually to ensure each formed a strictly self-contained problem. Given the problem statement and the forum comments, we extracted the corresponding answer and subsequently ran a validation step to check if the problem statement was logically sound and contained all necessary information. To execute this complex pipeline precisely, we utilized Kimi-K2.5 in a non-reasoning configuration.
\nNext, we evaluated these curated problems using gpt-oss-120b (with TIR enabled), generating 8 traces per problem. This yielded an interesting finding: for problems where gpt-oss-120b had a 0% success rate (0/8), the ground-truth answers were mostly incorrect. A manual inspection of about 100 such problems revealed the ground truth was correct only ~30% of the time, typically due to flawed forum comments or the extraction of incomplete intermediate steps. Interestingly, in these \"0% success\" cases, if gpt-oss-120b showed a high internal consensus (agreeing with itself $\\ge 6/8$ times on a different answer), the model's majority answer was usually the truly correct one. Conversely, for problems solved exactly once (1/8), the ground truth was correct about 70% of the time.
\nUltimately, AoPS 2025 was used exclusively for our evaluation benchmark and not for SFT. Given time and compute constraints, our top priority was establishing a highly difficult, contamination-free benchmark. Because the evaluation set size was limited, we selected problems that gpt-oss-120b solved between 0 and 5 times, and manually verified their validity and correct answers.
\nAs for pre-2025 AoPS problems, we attempted to use them for training but only obtained preliminary results. We suspect this is because we were only able to extract a limited number (~2k trajectories) of genuinely high-quality problem-answer pairs, which failed to provide sufficient diversity. Within the competition deadline, we could not find a setting using this data that surpassed our final submission, though further experiments are ongoing.
\nRegarding Nemotron-Math-v2, we agree that a non-negligible fraction of broken problem statements or obviously flawed answers exists, particularly among the subset where gpt-oss-120b struggles. Specifically, when Nemotron-Math-v2 fails to extract an answer, it falls back to using the gpt-oss-120b majority answer as the ground truth. Our manual checks confirmed that this fallback subset is highly enriched for low-quality data. However, when we aggressively filtered out this subset, our post-SFT performance actually degraded slightly. Given this empirical result, our final submission model relies only on the filtering criteria described in our writeup. We hypothesize that because the models used to synthesize our SFT data (e.g., Kimi-K2.5) are more capable than gpt-oss-120b, instances where the extracted answer, the gpt-oss-120b fallback answer, and the strong LLM's generated trajectory all align still provide a robust, meaningful learning signal, essentially validating the synthetic data.
\nFor Crystal-Math-Preview, our manual spot checks revealed almost no fundamentally broken problem statements, so no additional filtering was applied.
\nIn conclusion, while we have no doubt that a small percentage of problem-answer pairs in our final SFT mix retain quality issues, we are confident that we successfully constructed a dataset of sufficient quality to meaningfully enhance the model's reasoning capabilities.
\nThank you for the constructive suggestion. We completely agree that visual aids bring the quantitative findings to life and improve overall readability. Following your advice, we have updated the writeup and replaced several tabular results with plots.
", "votes": 2, "replies": [] } ] }, { "id": 3459220, "author_name": "Inoichan", "author_url": "", "post_date": "2026-05-17T14:15:55.747000", "content": "Edited on May 17, 2026 for the rebuttal phase:
\nCongrats on successfully fine-tuning a model 2 years in a row! Knowing how hard and expensive it was with a 14B model (many thousands for experiments), I get goosebumps imagining what it would take to fine-tune a model this year lol.
", "votes": 2, "replies": [] }, { "id": 3447807, "author_name": "ayushokaay", "author_url": "", "post_date": "2026-04-24T01:34:25.237000", "content": "nice fine tuning.
", "votes": 0, "replies": [] } ], "comment_count": 9, "messages": [ { "id": 3447117, "postDate": "2026-04-22T18:13:33.410Z", "content": "Recent large language models have achieved strong performance on mathematical reasoning benchmarks. For instance, gpt-oss-120b, the most utilized model in this competition, achieves an accuracy of approximately 98% on AIME'25. This makes it increasingly important to understand how already-strong models can be improved further. In this regime, post-training plays a central role. Additional adaptation after pre-training can still lead to meaningful differences in mathematical reasoning performance, especially when the goal is to push strong models beyond already high baselines.
\nAmong post-training approaches, supervised fine-tuning (SFT) and reinforcement learning (RL) are two of the most important and widely used paradigms. Both have shown promising results for mathematical reasoning, yet their roles in the high-accuracy regime remain insufficiently understood. When base models are already highly capable, it is often unclear how much improvement each method can provide, what kinds of behaviors each method induces, and how their benefits differ in practice. These questions are further complicated by the fact that commonly used benchmarks are becoming increasingly saturated, making it difficult to cleanly distinguish methods or to analyze the exact source of improvements (Balunović et al., 2025; An et al., 2025).
\nAt the same time, gains from post-training cannot be fully understood from final accuracy alone. In many settings, inference must be carried out under explicit constraints on latency, computation, or cost, and mathematical reasoning performance can be highly sensitive to inference-time configurations. Decoding hyperparameters, speculative decoding, and output selection or post-processing can significantly affect both solution quality and computational efficiency (Leviathan et al., 2023). As a result, the practical value of a post-trained model depends not only on whether it achieves higher accuracy but also on how effectively that accuracy can be realized under limited inference budgets. This motivates studying post-training and inference-time design jointly, rather than as isolated components.
\nIn this work, we investigate how to further improve already-strong mathematical reasoning models through post-training under realistic inference constraints. We strengthen evaluation beyond standard benchmarks with an expanded set of challenging mathematical problems that better discriminate performance in the high-accuracy regime. Building upon this evaluation framework, we compare two major post-training strategies, supervised fine-tuning and reinforcement learning, and analyze how they differ in the quality gains they provide, the behaviors they induce, and their interactions with inference-time choices. We further examine practical inference-time mechanisms, including decoding temperature, speculative decoding, and post-processing or selection, to study how these factors jointly affect the attainable quality-resource frontier. Through this perspective, we aim to elucidate not only how to enhance strong models but also how to maximize the utility of those improvements under explicit inference constraints.
\nOur study yields several key findings:\nFirst, even in the already-strong regime, additional post-training can produce meaningful gains, and the magnitude and characteristics of those gains differ significantly between supervised fine-tuning and reinforcement learning. \nSecond, the value of post-training depends intrinsically on the inference procedure utilized at test time. Decoding and selection choices can substantially shift the effective operating point of a model, amplifying or diminishing the gains obtained from training. \nThird, comparisons based solely on unconstrained final accuracy can obscure critical differences between methods. When evaluated under explicit resource constraints, approaches with equivalent accuracy may exhibit substantially disparate efficiency profiles. \nTaken together, these results suggest that improving mathematical reasoning in the high-performance regime necessitates a coupled approach of effective post-training and careful inference design.
\nOur core contributions are as follows:
\nFocus on Integer Problems and Limitations of Existing Benchmarks\nTo tightly control the evaluation setting and isolate specific mathematical reasoning capabilities, we restrict our focus exclusively to integer-answer problems. Traditionally, benchmarks such as AIME, HMMT, and reference problems from AIMO 1-3 have served as the standard for this difficulty tier. However, recent state-of-the-art models have achieved near complete coverage on these legacy datasets. This saturation strongly suggests potential data contamination within pre-training corpora, rendering these existing benchmarks insufficient for accurately assessing true zero-shot reasoning capabilities (Balunović et al., 2025; An et al., 2025).
\nTo address this limitation and construct a robust, uncontaminated evaluation suite, we explicitly incorporated the recent 2025 editions of AIME and HMMT, alongside integer problems systematically extracted from the MathArena benchmark (Balunović et al., 2025). Furthermore, to diversify the problem distribution and guarantee extreme difficulty without contamination, we aggregated and rigorously filtered problem sets from four additional distinct sources.
\n1. Japanese Mathematical Olympiad (JMO)\nWe incorporated integer problems from the Japanese Mathematical Olympiad (JMO) to introduce novel, high-quality challenges that possess a significantly lower probability of inclusion in English-dominant training datasets.
\n2. Filtering from Nemotron-Math-v2\nWe curated a challenging subset from the Nemotron-Math-v2 dataset based on the empirical performance of the gpt-oss-120b model under varying reasoning effort configurations (low, medium, and high) (NVIDIA, 2025; OpenAI, 2025). To isolate high-complexity problems, we aggregated all instances where the model failed to answer correctly under either the low or medium effort configurations (i.e., Accuracy (low) = 0 \\lor Accuracy (medium) = 0). For these selected instances, we retained the problem text, the expected ground-truth answer, and the unique identifier. This filtering effectively eliminates trivial tasks, yielding a refined testbed for advanced reasoning assessment.
3. Curating the AoPS 2025 Contest Collection\nWe leveraged the 2025 subset of the NuminaMath dataset, sourced from the Art of Problem Solving (AoPS) Contest Collection (AI-MO, 2024). To prevent overlap with existing benchmarks, problems from widely recognized competitions (e.g., AIME) were explicitly excluded. An LLM-in-the-loop pipeline was employed to rewrite problem statements, automatically verify validity, and extract ground-truth answers from forum discussions. Subsequently, we applied a strict empirical filtering criterion using gpt-oss-120b configured with high reasoning effort and Tool-Integrated Reasoning (TIR). By sampling 8 inference trajectories per problem, we retained only instances with a success rate strictly between 0% and 100%. While the upper bound excludes trivial tasks, the lower bound (>0%) serves as a critical quality control mechanism; manual inspection confirmed that problems with a 0% success rate frequently contained flawed statements or incorrect ground-truth labels.
4. Adaptation of the IMO-Answer Bench into Value-Finding Tasks\nTo ensure an exceptionally difficult evaluation set, we utilized IMO-AnswerBench from IMO-Bench (Luong et al., 2025). We identified problems within this benchmark that could be logically restructured into definitive integer-valued tasks. By adapting these problems into an integer-answer format, we established a rigorous testbed that strictly assesses advanced reasoning while remaining highly resistant to standard pre-training memorization.
\nFinal Dataset Composition\nThe compilation of these carefully curated sources results in our final evaluation benchmark, comprising 285 high-quality integer problems. The composition is detailed in Figure 1.
\n
Figure 1: Composition of the Final Evaluation Benchmark
\nEvaluating reasoning capabilities without strict computational bounds yields an inaccurate representation of practical model utility, as scaling test-time compute naturally inflates performance. To establish a rigorous baseline, we explicitly constrain our experimental environment to a single NVIDIA H100 GPU with a strict execution time limit of 5 hours per evaluation run. This constraint models the official environment of the Kaggle AI Mathematical Olympiad Progress Prize 3, ensuring the evaluation reflects realistic, resource-bound scenarios (Kaggle, 2025).
\nTo assess joint performance and inference efficiency within this environment, we constructed an additional targeted evaluation subset consisting of 50 problems.These instances were strategically sampled from the Crystal Math Preview dataset (Chen, 2026) such that the total expected inference time closely aligns with the empirical runtime observed on the public leaderboard. This specialized subset provides a controlled testbed for evaluating the trade-off between reasoning accuracy and computational speed.
\nGiven that gpt-oss-120b already achieves high performance (OpenAI, 2025), particularly when configured with high reasoning effort and tool-integrated reasoning (TIR), the methodology for further improving such an extensively post-trained model is non-trivial. In this study, we investigate the application of supervised fine-tuning (SFT) and reinforcement learning (RL) to enhance the original gpt-oss-120b model, analyzing the distinct performance gains and behavioral shifts induced by each approach.
Our preliminary experiments on Nemotron-Math-v2 yielded two primary observations. First, training the smaller gpt-oss-20b model on trajectories generated by gpt-oss-120b improved downstream mathematical performance. Second, applying the identical self-distillation recipe to gpt-oss-120b itself produced negligible improvements. Consequently, we generated novel training trajectories using several external strong reasoning models, utilizing these traces to introduce reasoning patterns not reliably exhibited by the base model.
Motivated by the extended-SFT paradigms observed in recent literature (Yoshihara et al., 2025), we adopted a prolonged fine-tuning approach. We hypothesized that repeated supervised training on a carefully curated, high-quality mixture of mathematical traces drives sustained improvements in reasoning reliability.
\nWe constructed the SFT dataset with two primary objectives: aggressive filtering for difficulty, and trajectory generation via stronger models.
\nFor difficulty filtering, we utilized Nemotron-Math-v2 and Crystal-Math-Preview. Nemotron-Math-v2 contains approximately 347k problems with released solution traces. The availability of traces varies because the dataset was generated across multiple inference settings (with and without TIR, varying effort levels), retaining only correct trajectories. Thus, problems with fewer retained traces serve as a proxy for higher difficulty. We isolated problems possessing at most four retained trajectories and generated entirely new solution traces for them. For Crystal-Math-Preview, we utilized the verified rewritten subset. We strictly excluded any problems that overlapped with our evaluation benchmark.
\nNew trajectories were synthesized using Kimi-K2.5, DeepSeek-V3.2-Speciale, GLM-4.7, and GLM-5. Generation was conducted separately for no-TIR and TIR configurations. For each problem, we sampled four candidate solutions, retaining only those that terminated normally and yielded the correct answer. Correctness was verified via exact string matching for integer problems, and via gpt-oss-120b acting as an LLM judge for non-integer formats.
Table 1 summarizes the curated trajectories utilized for fine-tuning following preprocessing.
\nTable 1: SFT Dataset Composition and Token Distribution
\n| Source Dataset | \nInference Mode | \nGenerator Models | \nCurated Trajectories | \nTotal Tokens | \n
|---|---|---|---|---|
| Nemotron-Math-V2 | \nNo TIR | \nKimi-K2.5, DeepSeek-V3.2-Speciale | \n8,403 | \n195,721,010 | \n
| Nemotron-Math-V2 | \nTIR | \nKimi-K2.5, GLM-4.7 | \n1,296 | \n32,778,255 | \n
| Crystal-Math-Preview | \nNo TIR | \nKimi-K2.5, GLM-5 | \n8,073 | \n290,345,130 | \n
| Crystal-Math-Preview | \nTIR | \nKimi-K2.5 | \n4,515 | \n150,876,560 | \n
| Total Unique Data | \n- | \n- | \n22,287 | \n669,720,955 | \n
We fine-tuned the gpt-oss-120b base model utilizing full-parameter supervised fine-tuning. The maximum sequence length was set to 65,536 tokens, with sequence packing enabled to maximize throughput. Training was conducted for 10 epochs using a global batch size of 256. Optimization was performed using the Adam optimizer with a peak learning rate of 8e-6, a minimum learning rate of 8e-7, a cosine decay schedule, a 5% linear warmup, a weight decay of 0.1, and gradient clipping capped at 1.0.
To emphasize the acquisition of tool-use capabilities, the dataset mixture heavily upsampled TIR trajectories. During each epoch, no-TIR data were sampled once, whereas each TIR trajectory was sampled three times. Consequently, while TIR examples constituted only 5,811 of the 22,287 unique trajectories, they accounted for 551 million of the 1.037 billion effective tokens processed per epoch. Across the full 10-epoch training run, the model was exposed to approximately 10.37 billion tokens.
\nModels were evaluated on the comprehensive, high-difficulty benchmark constructed in Section 2. To robustly estimate performance, we generated 16 independent inference trajectories per problem. We report the pass@1 metric (the empirical average accuracy across the 16 samples) and the majority voting accuracy at k=8 (Maj@8), computed via 2,000 bootstrap iterations over the generated samples. Finally, we report the coverage (Pass@16), which denotes the percentage of problems solved correctly at least once among the 16 samples.
\nTable 2 compares the base model against the fine-tuned models at different stages of the training process (2, 5, and 10 epochs). All metrics are reported as absolute percentages.
\nTable 2: Performance Evolution Across SFT Epochs
\n| Inference Mode | \nTraining Epochs | \nEffective Token Exposure | \npass@1 ↑ | \nMaj@8 ↑ | \nCoverage (Pass@16) ↑ | \n
|---|---|---|---|---|---|
| TIR | \nBase | \n\n | 72.61 | \n77.86 | \n92.28 | \n
| \n | 2 | \n2.07B | \n68.60 | \n75.88 | \n87.02 | \n
| \n | 5 | \n5.19B | \n70.88 | \n76.85 | \n88.77 | \n
| \n | 10 | \n10.37B | \n73.38 | \n78.62 | \n90.88 | \n
| No TIR | \nBase | \n\n | 64.65 | \n72.04 | \n84.56 | \n
| \n | 2 | \n2.07B | \n62.04 | \n69.43 | \n83.51 | \n
| \n | 5 | \n5.19B | \n63.86 | \n71.67 | \n84.21 | \n
| \n | 10 | \n10.37B | \n65.79 | \n72.31 | \n83.51 | \n
The models fine-tuned for 2 and 5 epochs exhibit performance degradation relative to the base model on both pass@1 and Maj@8 across both inference paradigms. Conversely, fine-tuning for 10 epochs yields systematic improvements over the base model, achieving 73.38% pass@1 and 78.62% Maj@8 with tools enabled, alongside 65.79% pass@1 and 72.31% Maj@8 without tools.
\nNotably, maximum coverage (Pass@16) remains highest for the base model. This demonstrates that the primary mechanism of improvement derived from prolonged SFT is not an expansion of the model's absolute solvable problem space, but rather a significant enhancement in the average correctness and majority-vote reliability of the generated solutions. These results validate the efficacy of extended supervised fine-tuning as a mechanism for repeated exposure to a compact, verified set of difficult mathematical trajectories.
\nAs demonstrated in previous evaluations, the gpt-oss-120b model already achieves strong mathematical reasoning performance, particularly when utilizing tool-integrated reasoning (TIR) under high reasoning effort. Improving this baseline via RL requires careful data selection because training on easily solved problems wastes compute. Consequently, our RL strategy relies on strict data filtering to isolate problems at the boundary of the model's competence.
We initialized our RL training dataset using the mathematical subset of the allenai/Dolci-Think-RL-7B dataset (Allen Institute for AI, 2025), which originally comprises 60,360 problems. To construct a highly efficient learning curriculum, we applied a strict empirical filtering to eliminate both trivial tasks where the model requires no further optimization.
The filtering was conducted using the base gpt-oss-120b model configured with high reasoning effort and TIR enabled. To determine the empirical pass rate for filtering, we ran 16 independent inference trials for each of the candidate problems with a maximum generation length of 65,536 tokens.
We applied the following retention criteria:
\nThis boundary condition successfully filters out problems that are reliably solved, forcing the RL algorithm to optimize on instances where the model exhibits high variance or uncertainty. Following this procedure, the dataset was reduced to 13,358 highly discriminative problems, establishing a concentrated training set.
\nWe adopted Group Relative Policy Optimization (GRPO) (Shao et al., 2024), enhanced with Clip-Higher and Dynamic Sampling from DAPO (Yu et al., 2025), Truncated Importance Sampling (TIS) (Yao et al., 2025), and removal of the KL loss term. We maintained a substantial maximum sequence length of 65,536 tokens during rollouts. To avoid bottlenecks caused by long-tail trajectories during the rollout phase, we applied the partial rollout technique (Kimi Team, 2025). During training, we generated 16 trajectory samples per prompt to estimate the relative advantages, strictly matching our evaluation methodology. The final evaluated RL model was selected from the checkpoint at training step 70. Comprehensive details regarding the distributed training infrastructure, parallelism strategies, and RL-specific hyper-parameters are deferred to Appendix A.
\nThe RL model was evaluated on our comprehensive benchmark. Table 3 presents the absolute performance in percentage points.
\nTable 3: Performance of the RL-Tuned Model
\n| Inference Mode | \nTraining Strategy | \npass@1 ↑ | \nMaj@8 ↑ | \nCoverage (Pass@16) ↑ | \n
|---|---|---|---|---|
| TIR | \nBase | \n72.61 | \n77.86 | \n92.28 | \n
| \n | RL | \n73.61 | \n78.78 | \n90.17 | \n
| No TIR | \nBase | \n64.65 | \n72.04 | \n84.56 | \n
| \n | RL | \n65.61 | \n71.71 | \n83.85 | \n
When evaluated without tools, RL improves pass@1 but slightly degrades Maj@8 and Coverage. Under the TIR setting, the RL process slightly improves the expected pass@1 accuracy to 73.61% and Maj@8 to 78.78%. However, this comes with a marginal reduction in overall coverage (90.17%) relative to the base model. This divergence highlights a critical dynamic in the high-accuracy regime. While RL effectively shifts the probability mass toward correct answers on an individual sample basis (pass@1, Maj@8), it can also induce a narrowing of the exploration space, slightly reducing the broader trajectory diversity required for maximum coverage at high sampling budgets.
\nIn this section, we discuss the trade-off between accuracy and latency governed by temperature, particularly when utilizing speculative decoding. In the following experiments and our submission, we used Eagle3 as the draft model. For a comparison with an alternative speculative decoding method, DFlash, which we found less optimal at higher temperatures, please refer to Appendix D.
\nWe observe that increasing the decoding temperature extends inference time when utilizing speculative decoding (Leviathan et al., 2023). To quantify this effect, we evaluated our model using the Time-Constrained Evaluation Benchmark established earlier. This specialized 50-problem subset allows us to accurately approximate the strict runtime limits of the simulated submission environment. For each problem, we generated 8 concurrent candidate solutions on a single NVIDIA H100 GPU. The measured runtime metrics across different temperature settings with speculative decoding are presented in Table 4.
\nTable 4 Runtime and Speculative Decoding Efficiency by Temperature
\n| Temperature | \nRuntime (s/run) | \nTotal Wall Time (min) | \nTokens / Run | \nMean accept rate | \nMean accept length | \n
|---|---|---|---|---|---|
| 0.5 | \n247 | \n226 | \n31,500 | \n0.504 | \n3.02 | \n
| 0.6 | \n246 | \n230 | \n32,200 | \n0.492 | \n2.97 | \n
| 0.7 | \n248 | \n232 | \n32,400 | \n0.481 | \n2.93 | \n
| 0.8 | \n265 | \n242 | \n33,200 | \n0.461 | \n2.85 | \n
| 0.9 | \n273 | \n235 | \n33,500 | \n0.431 | \n2.72 | \n
| 1.0 | \n291 | \n253 | \n33,500 | \n0.370 | \n2.48 | \n
| 1.0 without speculative decoding | \n391 | \n342 | \n33,300 | \nN/A | \nN/A | \n
The table indicates that the reduction in inference speed is attributable to two primary factors. First, higher temperatures induce longer average output lengths. Second, elevated temperatures decrease the token acceptance rate of speculative decoding, thereby diminishing its acceleration benefits.
\nTo assess the corresponding impact on reasoning quality, we evaluated model performance across varying temperatures on our comprehensive benchmark. We generated 16 independent trajectories for each of the 285 problems per temperature setting. Figure 2 reports the mean accuracy, bootstrap majority-vote accuracy, coverage, and generation token length based on the proportion of correct solutions among the total generated trajectories.
\nFigure 2 Temperature sweep for the final SFT model on our evaluation set.\nWe evaluate decoding temperatures from 0.5 to 1.5 and report mean accuracy, bootstrap majority-vote accuracy, coverage (pass@16), and generation length. Bootstrap majority accuracy is computed with majority voting over 8 sampled responses. Error bars show one standard deviation across bootstrap estimates. Coverage denotes the fraction of problems solved by at least one sampled response. Token length is shown as the mean number of generated tokens, with the shaded region indicating \\pm 1 standard deviation across generations. Gray dashed lines indicate the baseline model.
\n
The results demonstrate a clear trend where higher temperatures generally yield improved reasoning performance. Synthesizing these findings reveals a fundamental trade-off under constrained computing environments. Increasing the decoding temperature improves mathematical reasoning accuracy but incurs a measurable cost in inference latency, primarily due to degraded speculative decoding efficiency and increased generation lengths.
\nIn addition to tuning the decoding temperature, we investigated advanced test-time scaling methods that prompt the model to select or refine its own solutions. Specifically, we evaluated \"Self Aggregation Refinement (SAR)\" and \"GenSelect.\" While both approaches demonstrated the potential to outperform standard majority voting under certain conditions, they significantly increased pipeline complexity and proved highly sensitive to truncation caused by context length constraints. Ultimately, we concluded that simple temperature scaling offered the most robust trade-off between inference time and accuracy within the strict five-hour time limit, and we did not adopt these advanced aggregation methods in our final submission. Detailed experimental setups and results are provided in Appendix C.
\nFigure 3: Performance comparison between the RL and SFT models across varying decoding temperatures (T). The baseline model is evaluated at the default setting of T=1.0. Metrics are reported as Pass@1 / Maj@8 / Coverage (%). The highest achievable performance under the strict 5-hour runtime constraint is highlighted in bold for both models (the RL model is computationally constrained to T \\le 0.8, whereas the SFT model can operate safely at T=1.0).
\n
Both SFT and RL demonstrate strong efficacy as post-training methodologies, yielding highly competitive models that effectively enhance the reasoning capabilities of the base model. In broader empirical evaluations, both approaches consistently achieved public scores exceeding 40, with the RL-tuned model attaining a peak score of 44 and the SFT model reaching a maximum of 43. Despite these strong absolute performances, we observed a minor behavioral divergence during generation. The RL process exhibited a tendency to induce slightly prolonged inference times.
\nUltimately, we conducted empirical timing evaluations within the official Kaggle submission environment. To guarantee that inference would complete reliably across both the public and private evaluation splits, we sought generation configurations that preserved a safety margin of at least 30 minutes prior to the strict runtime deadline. Our analysis revealed that to ensure the RL model finished within this acceptable window, its decoding temperature could be set no higher than 0.8. In contrast, the SFT model comfortably completed all inference tasks even when the temperature was elevated to 1.0. As demonstrated by the results in Figure 3, the SFT model operating at T=1.0 yields superior performance compared to the RL model constrained to T=0.8. Consequently, we selected the SFT model for our final submission.
\nTraining was performed with Megatron-SWIFT on 4 nodes with 8 GPUs per node. We configured the distributed setup with pipeline parallelism 4 (PP=4) and context parallelism 1 (CP=1). Specifically, to optimize the MoE architecture, we applied tensor parallelism 4 (TP=4) to the attention layers, while configuring the MoE layers with expert parallelism 8 (EP=8) and no tensor parallelism (TP=1). The model was trained in bfloat16 precision from a Megatron-Core checkpoint of gpt-oss-120b.
Table A.1: SFT Hyper-parameters
\n| Parameter | \nValue | \n
|---|---|
| Max sequence length | \n65,536 | \n
| Packing | \nEnabled | \n
| Epochs | \n10 | \n
| Global batch size | \n256 | \n
| Optimizer | \nAdam | \n
| Adam beta1 / beta2 / eps | \n0.9 / 0.95 / 1e-8 | \n
| Learning rate | \n8e-6 | \n
| Minimum learning rate | \n8e-7 | \n
| LR schedule | \nCosine decay | \n
| Warmup | \n5% of training steps | \n
| Weight decay | \n0.1 | \n
| Gradient clipping | \n1.0 | \n
| Attention softmax | \nfp32 | \n
| Attention dropout | \n0.0 | \n
Reinforcement learning was executed on an identical physical infrastructure of 4 nodes with 8 GPUs per node. For the RL workload, we maintained pipeline parallelism 4 (PP=4) and context parallelism 1 (CP=1). The parallelism strategy was decoupled across layer types: the attention layers utilized tensor parallelism 8 (TP=8), whereas the MoE layers were distributed using expert parallelism 8 (EP=8) with tensor parallelism 1 (TP=1). The model was optimized in bfloat16 precision, initialized directly from the gpt-oss-120b bfloat16 checkpoint.
Table A.2: RL (GRPO) Hyper-parameters
\n| Parameter | \nValue | \n
|---|---|
| Algorithm | \nGRPO | \n
| Max sequence length | \n65,536 | \n
| Samples per prompt | \n16 | \n
| Rollout batch size | \n64 | \n
| Global batch size | \n1024 | \n
| Selected checkpoint | \nStep 70 | \n
| Optimizer | \nAdamW | \n
| Adam beta1 / beta2 | \n0.9 / 0.98 | \n
| Learning rate | \n1e-6 | \n
| LR schedule | \nConstant | \n
| Weight decay | \n0.1 | \n
| GRPO clip range | \n[0.2, 0.28] | \n
| KL loss coefficient | \n0.0 | \n
| Entropy coefficient | \n0.0 | \n
To execute Python code in a stateful manner, we constructed a custom sandbox environment using Docker. During local training, data generation, and Kaggle notebook submissions, we utilized identical local Jupyter kernel sessions to ensure strict environmental consistency.
\nWe implemented several optimizations within this Python execution environment to maximize efficiency and stability:
\nnumpy, scipy, sympy, mpmath, networkx, numba, ortools, pulp, python-sat, z3-solver, and shapely.--color=NoColor flag. This simple adjustment drastically reduced the token length of Python error tracebacks (e.g., reducing a standard KeyError traceback from 805 tokens to 112 tokens, an 86.1% reduction).PREFIX...[truncated]...POSTFIX to allow the LLM to recover and proceed in the subsequent turn.When curating the synthetic reasoning traces for Supervised Fine-Tuning (SFT), we applied specific preprocessing steps to align the training data with our test-time inference strategy:
\n\\boxed{} Tags: In environments where latency is critical, many implementations utilize an early-stopping mechanism that halts generation and extracts the answer the moment the first \\boxed{} tag appears. However, source datasets like Nemotron-Math-v2 typically rely on the final \\boxed{} tag for evaluation. Consequently, models frequently use \\boxed{} to highlight incorrect intermediate guesses before reaching the correct final answer. To prevent early-stopping failures at inference time, we systematically sanitized the training traces by removing or masking any incorrect \\boxed{} tags appearing in intermediate reasoning steps.!pip install commands or attempted to search the web for the problem statement. Because the target Kaggle submission environment strictly prohibits internet access, models cannot execute these actions at test time. We explicitly filtered out all traces containing network requests or dynamic library installations to prevent the model from learning strategies it cannot execute.Our data generation pipeline utilized a modified prompt structure tailored to our tool-integrated reasoning environment. We specifically augmented the tool prompt to explicitly notify the model of the pre-installed libraries available in our custom environment, reducing unnecessary installation attempts.
\nUser Prompt Structure:
\n{problem_text}\n\nPlease reason step by step to solve the math problem. Finally, Return only the verified final answer in `\\boxed{}`, where the answer is an integer in [0, 99999]. Never guess.\n\nYou may use the python tool for any non-trivial computation or verification.\nTool System Prompt:
\nUse this tool to execute Python code in your chain of thought. The code will not be shown to the user. This tool should be used for internal reasoning, but not for code that is intended to be visible to the user (e.g. when creating plots, tables, or files).\n\nWhen you send a message containing Python code to python, it will be executed in a stateful Jupyter notebook environment so the past imported library and past defined variables can be accessed without re-importing or re-defining them.\n\nPython will respond with the output of the execution or time out after {timeout_sec} seconds. So you must not implement a super compute heavy logic which cannot be calculated within the timeout {timeout_sec} seconds.\n\nYou can use the built-in python libraries and pre-installed libraries below:\n`numpy`: array operations\n`sympy`: symbolic mathematics\n`mpmath`: float precision\n`networkx`: graph theory\n`numba`: JIT compilation\n`ortools`: linear programming\n`pulp`: mixed integer programming\n`python-sat`: SAT solver\n`scipy`: scientific computing\n`shapely`: geometric operations\n\nInternet access is enabled but it is only for the purpose of python library installation when really necessary. If you need other libraries than the ones listed above, you may use the code \"!pip install <library_name>\" to try installing the library. You must not search the internet to find the problem itself and its answer.\nTo maximize pass@N performance within the strict five-hour inference budget, we evaluated two advanced selection and aggregation methods.
\nWe propose Self Aggregation Refinement (SAR), an approach built upon the recently introduced Recursive Self Aggregation (RSA) framework (Siddarth Venkatraman et al., 2026). To minimize latency, SAR modifies the recursive nature of the original RSA method into a single-turn refinement process. We explicitly instructed the model to identify and verify the most mathematically sound reasoning among the existing candidates, without synthesizing a new approach from scratch unless all provided candidates were demonstrably incorrect.
\nTo evaluate SAR, we conducted 4 experimental runs on a local benchmark of 133 high-difficulty problems, generating 8 candidate traces per problem in each run (totaling 532 trials). To optimize for practical time constraints, SAR was activated only when the standard majority voting failed to reach the early-stopping threshold of 4 votes (indicating high uncertainty). When activated, we applied hyperparameters $N=6$ and $K=4$, meaning we randomly selected 4 of the 8 traces and generated 6 parallel refined solutions, ultimately using entropy voting to determine the final answer.
\nAcross the 532 trials, the activation condition was met 126 times. Table C.1 details the performance changes categorized by the margin (the difference in votes between the first and second majority answers) prior to SAR activation.
\nTable C.1: Performance changes upon SAR activation, categorized by vote margin. \"Improved\" indicates instances where SAR corrected a previously wrong answer. \"Hurt\" indicates where SAR overturned a correct answer.
\n| Margin | \nTotal Activations | \nImproved ↑ | \nHurt ↓ | \nNet Gain | \nNet Gain % | \n
|---|---|---|---|---|---|
| 0 | \n53 | \n11 | \n2 | \n+9 | \n+16.9% | \n
| 1 | \n53 | \n7 | \n5 | \n+2 | \n+3.7% | \n
| 2+ | \n20 | \n0 | \n3 | \n-3 | \n-15.0% | \n
Applying SAR to problems with a margin of 0 yielded an expected net gain of +16.9%. However, because this method incorporates multiple full reasoning traces into the prompt, the context length expands rapidly. We observed that our SFT model's performance degraded when exceeding its training context length of 65,536 tokens, necessitating aggressive truncation of the input to 60k tokens. This context limitation remained a significant practical bottleneck.
\nWe also evaluated \"GenSelect,\" an approach proposed by the winning team of AIMO 2 (Ivan Moshkov et al., 2025). This method simultaneously presents multiple candidate solutions to the language model, prompting it to directly compare them and select the most mathematically sound option. For larger candidate pools, this process is structured as a tournament.
\nFor the GenSelect evaluation, we pre-generated a pool of 32 traces (with tool-integrated reasoning enabled) for each of the 133 high-difficulty benchmark problems. From this pool, we bootstrap-sampled 8 traces, repeating this process four times per problem. To simulate submission conditions, we restricted GenSelect to activate only when the size of the simple majority among the 8 traces was 4 or fewer.
\nGiven the increased difficulty of problems in this competition, individual reasoning traces were substantially longer, meaning a direct application of the original method would easily exceed the context window. To mitigate this, we allocated a budget of 48,000 tokens exclusively for the candidate solutions within the model's 65,536-token context window. This budget was divided evenly by the \"match size\" (the number of solutions compared simultaneously), and any text exceeding the per-solution allowance was truncated from the right. For each activated sample, we repeated the GenSelect process 4 times. Table C.2 illustrates the behavior of GenSelect across match sizes of 2, 4, and 8.
\nTable C.2: GenSelect accuracy across different match sizes.
\n| Match Size | \nAccuracy (Pass) | \nAccuracy (Simple Majority) | \nAccuracy (GenSelect Majority) | \n
|---|---|---|---|
| 2 | \n0.788 | \n0.372 | \n0.379 | \n
| 4 | \n0.788 | \n0.372 | \n0.452 | \n
| 8 | \n0.788 | \n0.372 | \n0.328 | \n
When the match size was set to 2 or 4, the GenSelect majority outperformed the simple majority voting accuracy of 0.372, achieving a notable improvement to 0.452 at a match size of 4. However, increasing the match size to 8 severely constrained the token budget per trace. The resulting truncation discarded critical logical structures and mathematical steps, causing the performance to drop below even the simple majority baseline.
\nFor faster inference, we tried a speculative decoding method called DFlash (Jian Chen et al., 2026). Unlike EAGLE3, which predicts one token at a time, DFlash uses a small diffusion model with around 3–8 layers to predict roughly 4–16 tokens simultaneously, and in some cases can run faster than EAGLE3. In the environment used for our final submission, DFlash was faster than EAGLE3 when the inference temperature was lowered, for example at T=0.5; however, at the final setting we chose, T=1.0, EAGLE3 was faster.
", "rawMarkdown": "# Introduction\n\nRecent large language models have achieved strong performance on mathematical reasoning benchmarks. For instance, gpt-oss-120b, the most utilized model in this competition, achieves an accuracy of approximately 98% on AIME'25. This makes it increasingly important to understand how already-strong models can be improved further. In this regime, post-training plays a central role. Additional adaptation after pre-training can still lead to meaningful differences in mathematical reasoning performance, especially when the goal is to push strong models beyond already high baselines.\n\nAmong post-training approaches, supervised fine-tuning (SFT) and reinforcement learning (RL) are two of the most important and widely used paradigms. Both have shown promising results for mathematical reasoning, yet their roles in the high-accuracy regime remain insufficiently understood. When base models are already highly capable, it is often unclear how much improvement each method can provide, what kinds of behaviors each method induces, and how their benefits differ in practice. These questions are further complicated by the fact that commonly used benchmarks are becoming increasingly saturated, making it difficult to cleanly distinguish methods or to analyze the exact source of improvements ([Balunović et al., 2025](https://arxiv.org/abs/2505.23281); [An et al., 2025](https://arxiv.org/abs/2510.26768)).\n\nAt the same time, gains from post-training cannot be fully understood from final accuracy alone. In many settings, inference must be carried out under explicit constraints on latency, computation, or cost, and mathematical reasoning performance can be highly sensitive to inference-time configurations. Decoding hyperparameters, speculative decoding, and output selection or post-processing can significantly affect both solution quality and computational efficiency ([Leviathan et al., 2023](https://arxiv.org/abs/2211.17192)). As a result, the practical value of a post-trained model depends not only on whether it achieves higher accuracy but also on how effectively that accuracy can be realized under limited inference budgets. This motivates studying post-training and inference-time design jointly, rather than as isolated components.\n\nIn this work, we investigate how to further improve already-strong mathematical reasoning models through post-training under realistic inference constraints. We strengthen evaluation beyond standard benchmarks with an expanded set of challenging mathematical problems that better discriminate performance in the high-accuracy regime. Building upon this evaluation framework, we compare two major post-training strategies, supervised fine-tuning and reinforcement learning, and analyze how they differ in the quality gains they provide, the behaviors they induce, and their interactions with inference-time choices. We further examine practical inference-time mechanisms, including decoding temperature, speculative decoding, and post-processing or selection, to study how these factors jointly affect the attainable quality-resource frontier. Through this perspective, we aim to elucidate not only how to enhance strong models but also how to maximize the utility of those improvements under explicit inference constraints.\n\nOur study yields several key findings:\nFirst, even in the already-strong regime, additional post-training can produce meaningful gains, and the magnitude and characteristics of those gains differ significantly between supervised fine-tuning and reinforcement learning. \nSecond, the value of post-training depends intrinsically on the inference procedure utilized at test time. Decoding and selection choices can substantially shift the effective operating point of a model, amplifying or diminishing the gains obtained from training. \nThird, comparisons based solely on unconstrained final accuracy can obscure critical differences between methods. When evaluated under explicit resource constraints, approaches with equivalent accuracy may exhibit substantially disparate efficiency profiles. \nTaken together, these results suggest that improving mathematical reasoning in the high-performance regime necessitates a coupled approach of effective post-training and careful inference design.\n\nOur core contributions are as follows:\n1. We systematically investigate methods for further improving already-strong mathematical reasoning models through post-training in the high-accuracy regime.\n2. We construct an enhanced evaluation framework that better resolves performance differences among strong models, addressing the saturation of standard benchmarks.\n3. We provide a comparative analysis of supervised fine-tuning and reinforcement learning, characterizing their distinct benefits and induced behaviors.\n4. We analyze how inference-time design choices interact with post-training under explicit inference-budget constraints.\n\n\n# Evaluation Benchmark Construction\n\n## Comprehensive Performance Benchmark\n\n**Focus on Integer Problems and Limitations of Existing Benchmarks**\nTo tightly control the evaluation setting and isolate specific mathematical reasoning capabilities, we restrict our focus exclusively to integer-answer problems. Traditionally, benchmarks such as AIME, HMMT, and reference problems from AIMO 1-3 have served as the standard for this difficulty tier. However, recent state-of-the-art models have achieved near complete coverage on these legacy datasets. This saturation strongly suggests potential data contamination within pre-training corpora, rendering these existing benchmarks insufficient for accurately assessing true zero-shot reasoning capabilities ([Balunović et al., 2025](https://arxiv.org/abs/2505.23281); [An et al., 2025](https://arxiv.org/abs/2510.26768)).\n\nTo address this limitation and construct a robust, uncontaminated evaluation suite, we explicitly incorporated the recent 2025 editions of AIME and HMMT, alongside integer problems systematically extracted from the MathArena benchmark ([Balunović et al., 2025](https://arxiv.org/abs/2505.23281)). Furthermore, to diversify the problem distribution and guarantee extreme difficulty without contamination, we aggregated and rigorously filtered problem sets from four additional distinct sources.\n\n**1. Japanese Mathematical Olympiad (JMO)**\nWe incorporated integer problems from the Japanese Mathematical Olympiad (JMO) to introduce novel, high-quality challenges that possess a significantly lower probability of inclusion in English-dominant training datasets.\n\n**2. Filtering from Nemotron-Math-v2**\nWe curated a challenging subset from the Nemotron-Math-v2 dataset based on the empirical performance of the `gpt-oss-120b` model under varying reasoning effort configurations (low, medium, and high) ([NVIDIA, 2025](https://huggingface.co/datasets/nvidia/Nemotron-Math-v2); [OpenAI, 2025](https://arxiv.org/abs/2508.10925)). To isolate high-complexity problems, we aggregated all instances where the model failed to answer correctly under either the low or medium effort configurations (i.e., Accuracy (low) = 0 \\lor Accuracy (medium) = 0). For these selected instances, we retained the problem text, the expected ground-truth answer, and the unique identifier. This filtering effectively eliminates trivial tasks, yielding a refined testbed for advanced reasoning assessment.\n\n**3. Curating the AoPS 2025 Contest Collection**\nWe leveraged the 2025 subset of the NuminaMath dataset, sourced from the Art of Problem Solving (AoPS) Contest Collection ([AI-MO, 2024](https://huggingface.co/datasets/AI-MO/NuminaMath-CoT)). To prevent overlap with existing benchmarks, problems from widely recognized competitions (e.g., AIME) were explicitly excluded. An LLM-in-the-loop pipeline was employed to rewrite problem statements, automatically verify validity, and extract ground-truth answers from forum discussions. Subsequently, we applied a strict empirical filtering criterion using `gpt-oss-120b` configured with high reasoning effort and Tool-Integrated Reasoning (TIR). By sampling 8 inference trajectories per problem, we retained only instances with a success rate strictly between 0% and 100%. While the upper bound excludes trivial tasks, the lower bound (>0%) serves as a critical quality control mechanism; manual inspection confirmed that problems with a 0% success rate frequently contained flawed statements or incorrect ground-truth labels. \n\n**4. Adaptation of the IMO-Answer Bench into Value-Finding Tasks**\nTo ensure an exceptionally difficult evaluation set, we utilized IMO-AnswerBench from IMO-Bench ([Luong et al., 2025](https://arxiv.org/abs/2511.01846)). We identified problems within this benchmark that could be logically restructured into definitive integer-valued tasks. By adapting these problems into an integer-answer format, we established a rigorous testbed that strictly assesses advanced reasoning while remaining highly resistant to standard pre-training memorization.\n\n**Final Dataset Composition**\nThe compilation of these carefully curated sources results in our final evaluation benchmark, comprising 285 high-quality integer problems. The composition is detailed in Figure 1.\n\n\n\n\n**Figure 1: Composition of the Final Evaluation Benchmark**\n\n\n## Time-Constrained Evaluation Benchmark\n\n### Computational Setup and Inference Constraints\nEvaluating reasoning capabilities without strict computational bounds yields an inaccurate representation of practical model utility, as scaling test-time compute naturally inflates performance. To establish a rigorous baseline, we explicitly constrain our experimental environment to a single NVIDIA H100 GPU with a strict execution time limit of 5 hours per evaluation run. This constraint models the official environment of the Kaggle AI Mathematical Olympiad Progress Prize 3, ensuring the evaluation reflects realistic, resource-bound scenarios ([Kaggle, 2025](https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3)).\n\nTo assess joint performance and inference efficiency within this environment, we constructed an additional targeted evaluation subset consisting of 50 problems.These instances were strategically sampled from the Crystal Math Preview dataset ([Chen, 2026](https://huggingface.co/datasets/ycchen/Crystal-Math-Preview)) such that the total expected inference time closely aligns with the empirical runtime observed on the public leaderboard. This specialized subset provides a controlled testbed for evaluating the trade-off between reasoning accuracy and computational speed.\n\n\n# Training\n\nGiven that `gpt-oss-120b` already achieves high performance ([OpenAI, 2025](https://arxiv.org/abs/2508.10925)), particularly when configured with high reasoning effort and tool-integrated reasoning (TIR), the methodology for further improving such an extensively post-trained model is non-trivial. In this study, we investigate the application of supervised fine-tuning (SFT) and reinforcement learning (RL) to enhance the original `gpt-oss-120b` model, analyzing the distinct performance gains and behavioral shifts induced by each approach.\n\n## Supervised Fine-Tuning (SFT)\n\nOur preliminary experiments on Nemotron-Math-v2 yielded two primary observations. First, training the smaller `gpt-oss-20b` model on trajectories generated by `gpt-oss-120b` improved downstream mathematical performance. Second, applying the identical self-distillation recipe to `gpt-oss-120b` itself produced negligible improvements. Consequently, we generated novel training trajectories using several external strong reasoning models, utilizing these traces to introduce reasoning patterns not reliably exhibited by the base model.\n\nMotivated by the extended-SFT paradigms observed in recent literature ([Yoshihara et al., 2025](https://arxiv.org/abs/2507.08267)), we adopted a prolonged fine-tuning approach. We hypothesized that repeated supervised training on a carefully curated, high-quality mixture of mathematical traces drives sustained improvements in reasoning reliability. \n\n### SFT Dataset Construction\n\nWe constructed the SFT dataset with two primary objectives: aggressive filtering for difficulty, and trajectory generation via stronger models.\n\nFor difficulty filtering, we utilized Nemotron-Math-v2 and Crystal-Math-Preview. Nemotron-Math-v2 contains approximately 347k problems with released solution traces. The availability of traces varies because the dataset was generated across multiple inference settings (with and without TIR, varying effort levels), retaining only correct trajectories. Thus, problems with fewer retained traces serve as a proxy for higher difficulty. We isolated problems possessing at most four retained trajectories and generated entirely new solution traces for them. For Crystal-Math-Preview, we utilized the verified rewritten subset. We strictly excluded any problems that overlapped with our evaluation benchmark.\n\nNew trajectories were synthesized using Kimi-K2.5, DeepSeek-V3.2-Speciale, GLM-4.7, and GLM-5. Generation was conducted separately for no-TIR and TIR configurations. For each problem, we sampled four candidate solutions, retaining only those that terminated normally and yielded the correct answer. Correctness was verified via exact string matching for integer problems, and via `gpt-oss-120b` acting as an LLM judge for non-integer formats. \n\nTable 1 summarizes the curated trajectories utilized for fine-tuning following preprocessing.\n\n**Table 1: SFT Dataset Composition and Token Distribution**\n\n| Source Dataset | Inference Mode | Generator Models | Curated Trajectories | Total Tokens |\n| :--- | :---: | :--- | ---: | ---: |\n| Nemotron-Math-V2 | No TIR | Kimi-K2.5, DeepSeek-V3.2-Speciale | 8,403 | 195,721,010 |\n| Nemotron-Math-V2 | TIR | Kimi-K2.5, GLM-4.7 | 1,296 | 32,778,255 |\n| Crystal-Math-Preview | No TIR | Kimi-K2.5, GLM-5 | 8,073 | 290,345,130 |\n| Crystal-Math-Preview | TIR | Kimi-K2.5 | 4,515 | 150,876,560 |\n| **Total Unique Data** | - | - | **22,287** | **669,720,955** |\n\n\n### Training Configuration\n\nWe fine-tuned the `gpt-oss-120b` base model utilizing full-parameter supervised fine-tuning. The maximum sequence length was set to 65,536 tokens, with sequence packing enabled to maximize throughput. Training was conducted for 10 epochs using a global batch size of 256. Optimization was performed using the Adam optimizer with a peak learning rate of 8e-6, a minimum learning rate of 8e-7, a cosine decay schedule, a 5% linear warmup, a weight decay of 0.1, and gradient clipping capped at 1.0.\n\nTo emphasize the acquisition of tool-use capabilities, the dataset mixture heavily upsampled TIR trajectories. During each epoch, no-TIR data were sampled once, whereas each TIR trajectory was sampled three times. Consequently, while TIR examples constituted only 5,811 of the 22,287 unique trajectories, they accounted for 551 million of the 1.037 billion effective tokens processed per epoch. Across the full 10-epoch training run, the model was exposed to approximately 10.37 billion tokens.\n\n### Results\n\nModels were evaluated on the comprehensive, high-difficulty benchmark constructed in Section 2. To robustly estimate performance, we generated 16 independent inference trajectories per problem. We report the pass@1 metric (the empirical average accuracy across the 16 samples) and the majority voting accuracy at k=8 (Maj@8), computed via 2,000 bootstrap iterations over the generated samples. Finally, we report the coverage (Pass@16), which denotes the percentage of problems solved correctly at least once among the 16 samples.\n\nTable 2 compares the base model against the fine-tuned models at different stages of the training process (2, 5, and 10 epochs). All metrics are reported as absolute percentages.\n\n**Table 2: Performance Evolution Across SFT Epochs**\n\n| Inference Mode | Training Epochs | Effective Token Exposure | pass@1 ↑ | Maj@8 ↑ | Coverage (Pass@16) ↑ |\n| :--- | :---: | :--- | :--- | :--- | :--- |\n| **TIR** | Base | | 72.61 | 77.86 | **92.28** |\n| | 2 | 2.07B | 68.60 | 75.88 | 87.02 |\n| | 5 | 5.19B | 70.88 | 76.85 | 88.77 |\n| | 10 | 10.37B | **73.38** | **78.62** | 90.88 |\n| **No TIR** | Base | | 64.65 | 72.04 | **84.56** |\n| | 2 | 2.07B | 62.04 | 69.43 | 83.51 |\n| | 5 | 5.19B | 63.86 | 71.67 | 84.21 |\n| | 10 | 10.37B | **65.79** | **72.31** | 83.51 |\n\nThe models fine-tuned for 2 and 5 epochs exhibit performance degradation relative to the base model on both pass@1 and Maj@8 across both inference paradigms. Conversely, fine-tuning for 10 epochs yields systematic improvements over the base model, achieving 73.38% pass@1 and 78.62% Maj@8 with tools enabled, alongside 65.79% pass@1 and 72.31% Maj@8 without tools. \n\nNotably, maximum coverage (Pass@16) remains highest for the base model. This demonstrates that the primary mechanism of improvement derived from prolonged SFT is not an expansion of the model's absolute solvable problem space, but rather a significant enhancement in the average correctness and majority-vote reliability of the generated solutions. These results validate the efficacy of extended supervised fine-tuning as a mechanism for repeated exposure to a compact, verified set of difficult mathematical trajectories.\n\n\n\n## Reinforcement Learning (RL)\n\nAs demonstrated in previous evaluations, the `gpt-oss-120b` model already achieves strong mathematical reasoning performance, particularly when utilizing tool-integrated reasoning (TIR) under high reasoning effort. Improving this baseline via RL requires careful data selection because training on easily solved problems wastes compute. Consequently, our RL strategy relies on strict data filtering to isolate problems at the boundary of the model's competence.\n\n### RL Dataset Preparation and Empirical Filtering\n\nWe initialized our RL training dataset using the mathematical subset of the `allenai/Dolci-Think-RL-7B` dataset ([Allen Institute for AI, 2025](https://huggingface.co/datasets/allenai/Dolci-Think-RL-7B)), which originally comprises 60,360 problems. To construct a highly efficient learning curriculum, we applied a strict empirical filtering to eliminate both trivial tasks where the model requires no further optimization.\n\nThe filtering was conducted using the base `gpt-oss-120b` model configured with high reasoning effort and TIR enabled. To determine the empirical pass rate for filtering, we ran 16 independent inference trials for each of the candidate problems with a maximum generation length of 65,536 tokens. \n\nWe applied the following retention criteria:\n1. **Format compliance:** The ground-truth answer y must be an integer within the range 0 \\le y \\le 99999.\n2. **Difficulty threshold:** The model must solve the problem correctly in no more than 11 out of the 16 trials.\n\nThis boundary condition successfully filters out problems that are reliably solved, forcing the RL algorithm to optimize on instances where the model exhibits high variance or uncertainty. Following this procedure, the dataset was reduced to 13,358 highly discriminative problems, establishing a concentrated training set.\n\n### Training Configuration\nWe adopted Group Relative Policy Optimization (GRPO) ([Shao et al., 2024](https://arxiv.org/abs/2402.03300)), enhanced with Clip-Higher and Dynamic Sampling from DAPO ([Yu et al., 2025](https://arxiv.org/abs/2503.14476)), Truncated Importance Sampling (TIS) ([Yao et al., 2025](https://fengyao.notion.site/Your-Efficient-RL-Framework-Secretly-Brings-You-Off-Policy-RL-Training-237721e3f6c48094ad67dad3ac091c56)), and removal of the KL loss term. We maintained a substantial maximum sequence length of 65,536 tokens during rollouts. To avoid bottlenecks caused by long-tail trajectories during the rollout phase, we applied the partial rollout technique ([Kimi Team, 2025](https://arxiv.org/abs/2507.20534)). During training, we generated 16 trajectory samples per prompt to estimate the relative advantages, strictly matching our evaluation methodology. The final evaluated RL model was selected from the checkpoint at training step 70. Comprehensive details regarding the distributed training infrastructure, parallelism strategies, and RL-specific hyper-parameters are deferred to Appendix A.\n\n\n### Results\n\nThe RL model was evaluated on our comprehensive benchmark. Table 3 presents the absolute performance in percentage points.\n\n**Table 3: Performance of the RL-Tuned Model**\n\n| Inference Mode | Training Strategy | pass@1 ↑ | Maj@8 ↑ | Coverage (Pass@16) ↑ |\n| :--- | :--- | :--- | :--- | :--- |\n| **TIR** | Base | 72.61 | 77.86 | **92.28** |\n| | RL | **73.61** | **78.78** | 90.17 |\n| **No TIR** | Base | 64.65 | 72.04 | 84.56 |\n| | RL | **65.61** | 71.71 | 83.85 |\n\nWhen evaluated without tools, RL improves pass@1 but slightly degrades Maj@8 and Coverage. Under the TIR setting, the RL process slightly improves the expected pass@1 accuracy to 73.61% and Maj@8 to 78.78%. However, this comes with a marginal reduction in overall coverage (90.17%) relative to the base model. This divergence highlights a critical dynamic in the high-accuracy regime. While RL effectively shifts the probability mass toward correct answers on an individual sample basis (pass@1, Maj@8), it can also induce a narrowing of the exploration space, slightly reducing the broader trajectory diversity required for maximum coverage at high sampling budgets.\n\n\n# Inference\n\n## The Trade-off Between Performance and Latency Across Decoding Temperatures\n\nIn this section, we discuss the trade-off between accuracy and latency governed by temperature, particularly when utilizing speculative decoding. In the following experiments and our submission, we used Eagle3 as the draft model. For a comparison with an alternative speculative decoding method, DFlash, which we found less optimal at higher temperatures, please refer to Appendix D.\n\nWe observe that increasing the decoding temperature extends inference time when utilizing speculative decoding ([Leviathan et al., 2023](https://arxiv.org/abs/2211.17192)). To quantify this effect, we evaluated our model using the Time-Constrained Evaluation Benchmark established earlier. This specialized 50-problem subset allows us to accurately approximate the strict runtime limits of the simulated submission environment. For each problem, we generated 8 concurrent candidate solutions on a single NVIDIA H100 GPU. The measured runtime metrics across different temperature settings with speculative decoding are presented in Table 4.\n\n**Table 4 Runtime and Speculative Decoding Efficiency by Temperature**\n\n| Temperature | Runtime (s/run) | Total Wall Time (min) | Tokens / Run | Mean accept rate | Mean accept length |\n| --- | ---: | ---: | ---: | ---: | ---: |\n| 0.5 | 247 | **226** | **31,500** | **0.504** | **3.02** |\n| 0.6 | **246** | 230 | 32,200 | 0.492 | 2.97 |\n| 0.7 | 248 | 232 | 32,400 | 0.481 | 2.93 |\n| 0.8 | 265 | 242 | 33,200 | 0.461 | 2.85 |\n| 0.9 | 273 | 235 | 33,500 | 0.431 | 2.72 |\n| 1.0 | 291 | 253 | 33,500 | 0.370 | 2.48 |\n| 1.0 without speculative decoding | 391 | 342 | 33,300 | N/A | N/A |\n\nThe table indicates that the reduction in inference speed is attributable to two primary factors. First, higher temperatures induce longer average output lengths. Second, elevated temperatures decrease the token acceptance rate of speculative decoding, thereby diminishing its acceleration benefits.\n\nTo assess the corresponding impact on reasoning quality, we evaluated model performance across varying temperatures on our comprehensive benchmark. We generated 16 independent trajectories for each of the 285 problems per temperature setting. Figure 2 reports the mean accuracy, bootstrap majority-vote accuracy, coverage, and generation token length based on the proportion of correct solutions among the total generated trajectories.\n\n**Figure 2 Temperature sweep for the final SFT model on our evaluation set.**\nWe evaluate decoding temperatures from 0.5 to 1.5 and report mean accuracy, bootstrap majority-vote accuracy, coverage (pass@16), and generation length. Bootstrap majority accuracy is computed with majority voting over 8 sampled responses. Error bars show one standard deviation across bootstrap estimates. Coverage denotes the fraction of problems solved by at least one sampled response. Token length is shown as the mean number of generated tokens, with the shaded region indicating \\pm 1 standard deviation across generations. Gray dashed lines indicate the baseline model.\n\n\n\nThe results demonstrate a clear trend where higher temperatures generally yield improved reasoning performance. Synthesizing these findings reveals a fundamental trade-off under constrained computing environments. **Increasing the decoding temperature improves mathematical reasoning accuracy but incurs a measurable cost in inference latency**, primarily due to degraded speculative decoding efficiency and increased generation lengths.\n\n## Advanced Test-Time Scaling Strategies\nIn addition to tuning the decoding temperature, we investigated advanced test-time scaling methods that prompt the model to select or refine its own solutions. Specifically, we evaluated \"Self Aggregation Refinement (SAR)\" and \"GenSelect.\" While both approaches demonstrated the potential to outperform standard majority voting under certain conditions, they significantly increased pipeline complexity and proved highly sensitive to truncation caused by context length constraints. Ultimately, we concluded that simple temperature scaling offered the most robust trade-off between inference time and accuracy within the strict five-hour time limit, and we did not adopt these advanced aggregation methods in our final submission. Detailed experimental setups and results are provided in Appendix C.\n\n**Figure 3: Performance comparison between the RL and SFT models across varying decoding temperatures (T).** The baseline model is evaluated at the default setting of T=1.0. Metrics are reported as Pass@1 / Maj@8 / Coverage (%). The highest achievable performance under the strict 5-hour runtime constraint is highlighted in bold for both models (the RL model is computationally constrained to T \\le 0.8, whereas the SFT model can operate safely at T=1.0).\n\n\n\n# Final Submission\n\nBoth SFT and RL demonstrate strong efficacy as post-training methodologies, yielding highly competitive models that effectively enhance the reasoning capabilities of the base model. In broader empirical evaluations, both approaches consistently achieved public scores exceeding 40, with the RL-tuned model attaining a peak score of 44 and the SFT model reaching a maximum of 43. Despite these strong absolute performances, we observed a minor behavioral divergence during generation. The RL process exhibited a tendency to induce slightly prolonged inference times. \n\nUltimately, we conducted empirical timing evaluations within the official Kaggle submission environment. To guarantee that inference would complete reliably across both the public and private evaluation splits, we sought generation configurations that preserved a safety margin of at least 30 minutes prior to the strict runtime deadline. Our analysis revealed that to ensure the RL model finished within this acceptable window, its decoding temperature could be set no higher than 0.8. In contrast, the SFT model comfortably completed all inference tasks even when the temperature was elevated to 1.0. As demonstrated by the results in Figure 3, the SFT model operating at T=1.0 yields superior performance compared to the RL model constrained to T=0.8. Consequently, we selected the SFT model for our final submission.\n\n\n## References\n\n- OpenAI. [gpt-oss-120b & gpt-oss-20b Model Card](https://arxiv.org/abs/2508.10925). arXiv, 2025.\n- Mislav Balunović, Jasper Dekoninck, Ivo Petrov, Nikola Jovanović, and Martin Vechev. [MathArena: Evaluating LLMs on Uncontaminated Math Competitions](https://openreview.net/forum?id=y0zL9IZxZ7). NeurIPS 2025 Datasets and Benchmarks Track poster, 2025.\n- Shengnan An et al. [AMO-Bench: Large Language Models Still Struggle in High School Math Competitions](https://arxiv.org/abs/2510.26768). arXiv, 2025.\n- Kaggle. [AI Mathematical Olympiad - Progress Prize 3](https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3). 2025.\n- NVIDIA. [Nemotron-Math-v2](https://huggingface.co/datasets/nvidia/Nemotron-Math-v2). Hugging Face dataset, 2025.\n- Yi-Chia Chen. [CrystalMath: Curating High-Difficulty Contest Math with Contrast-Augmented Verification](https://huggingface.co/datasets/ycchen/Crystal-Math-Preview). Hugging Face dataset, 2026.\n- AI-MO. [NuminaMath-CoT](https://huggingface.co/datasets/AI-MO/NuminaMath-CoT). Hugging Face dataset, 2024.\n- Thang Luong et al. [Towards Robust Mathematical Reasoning](https://aclanthology.org/2025.emnlp-main.1794/). EMNLP, 2025.\n- Allen Institute for AI. [Dolci-Think-RL-7B](https://huggingface.co/datasets/allenai/Dolci-Think-RL-7B). Hugging Face dataset, 2025.\n- Hiroshi Yoshihara, Taiki Yamaguchi, and Yuichi Inoue. [A Practical Two-Stage Recipe for Mathematical LLMs: Maximizing Accuracy with SFT and Efficiency with Reinforcement Learning](https://arxiv.org/abs/2507.08267). arXiv, 2025.\n- Zhihong Shao et al. [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://arxiv.org/abs/2402.03300). arXiv, 2024.\n- Qiying Yu et al. [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://arxiv.org/abs/2503.14476). arXiv, 2025.\n- Feng Yao et al. [Your Efficient RL Framework Secretly Brings You Off-Policy RL Training](https://fengyao.notion.site/Your-Efficient-RL-Framework-Secretly-Brings-You-Off-Policy-RL-Training-237721e3f6c48094ad67dad3ac091c56). Notion article, 2025.\n- Kimi Team. [Kimi K2: Open Agentic Intelligence](https://arxiv.org/abs/2507.20534). arXiv, 2025.\n- Yaniv Leviathan, Matan Kalman, and Yossi Matias. [Fast Inference from Transformers via Speculative Decoding](https://openreview.net/forum?id=C9NEblP8vS). ICML, 2023.\n- Siddarth Venkatraman, Vineet Jain, Sarthak Mittal, Vedant Shah, Johan Obando-Ceron, Yoshua Bengio, Brian R. Bartoldson, Bhavya Kailkhura, Guillaume Lajoie, Glen Berseth, Nikolay Malkin, and Moksh Jain. [Recursive Self-Aggregation Unlocks Deep Thinking in Large Language Models](https://arxiv.org/abs/2509.26626), arXiv, 2026.\n- Ivan Moshkov, Darragh Hanley, Ivan Sorokin, Shubham Toshniwal, Christof Henkel, Benedikt Schifferer, Wei Du, Igor Gitman. [AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning dataset](https://arxiv.org/abs/2504.16891). arXiv, 2025.\n- Jian Chen, Yesheng Liang, Zhijian Liu. [DFlash: Block Diffusion for Flash Speculative Decoding](https://arxiv.org/abs/2602.06036). arXiv, 2026.\n\n---\n\n# Appendix\n\n# Resources\n\n- **Submission notebook:** [https://www.kaggle.com/code/takkyu/aimo-submission-final](https://www.kaggle.com/code/takkyu/aimo-submission-final)Congrats on successfully fine-tuning gpt-oss with sft and rl! But I have a question about Eagle3 model. Did you fine-tune it by yourself? How it behaves on long trajectories (after 30k tokens)? I also see you use time management as in public notebook. Do you think it's optimal? Did you experiment with different time allocation?
", "rawMarkdown": "Congrats on successfully fine-tuning gpt-oss with sft and rl! But I have a question about Eagle3 model. Did you fine-tune it by yourself? How it behaves on long trajectories (after 30k tokens)? I also see you use time management as in public notebook. Do you think it's optimal? Did you experiment with different time allocation?", "votes": 5, "replies": [ { "id": 3483541, "postDate": "2026-06-29T06:44:19.980Z", "content": "First of all, I am so incredibly sorry for the super late reply!
\nRegarding your questions about the Eagle3 model, we further fine-tuned wenliangtlh's draft model on our custom data, which gave us a small boost of a few percentage points compared with the original model. We observed that the acceptance rate tends to decrease as trajectories get longer, but this was partially mitigated by training with more long traces of around 65,536 tokens.
\nAs for the time allocation, we did try a few different settings, but ultimately settled on the current one. In our experiments, we mostly kept the time allocation fixed and spent more time focusing on tweaking the temperature and devising post-processing techniques. Although we didn't end up using it in our final solution, we described our post-processing approach in Appendix C. Please feel free to check it out if you are interested!
", "rawMarkdown": "First of all, I am so incredibly sorry for the super late reply!\n\nRegarding your questions about the Eagle3 model, we further fine-tuned [wenliangtlh's draft model](https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3/discussion/671839) on our custom data, which gave us a small boost of a few percentage points compared with the original model. We observed that the acceptance rate tends to decrease as trajectories get longer, but this was partially mitigated by training with more long traces of around 65,536 tokens.\n\nAs for the time allocation, we did try a few different settings, but ultimately settled on the current one. In our experiments, we mostly kept the time allocation fixed and spent more time focusing on tweaking the temperature and devising post-processing techniques. Although we didn't end up using it in our final solution, we described our post-processing approach in Appendix C. Please feel free to check it out if you are interested!", "votes": 1 } ] }, { "id": 3450492, "postDate": "2026-04-29T21:43:16.743Z", "content": "Great writeup! I’m glad our CrystalMath could be of some help to your work. Do you plan to release the full training code? I believe it would be very valuable for the community if available.
", "rawMarkdown": "Great writeup! I’m glad our CrystalMath could be of some help to your work. Do you plan to release the full training code? I believe it would be very valuable for the community if available.", "votes": 3, "replies": [ { "id": 3454910, "postDate": "2026-05-08T07:00:38.840Z", "content": "Dataset quality is really important and preparing difficult yet answerable problems are getting harder as LLM's performance improves. Thank you for the effort to create high quality dataset.
\nWe have published the SFT dataset including Crystal-Math-Preview's solution trajectory
\n\nWe found it difficult to improve gpt-oss-120b's math performance further with the existing open dataset, and I also believe this will be valuable to the future research community 🐠🐟
", "rawMarkdown": "Dataset quality is really important and preparing difficult yet answerable problems are getting harder as LLM's performance improves. Thank you for the effort to create high quality dataset.\n\nWe have published the SFT dataset including Crystal-Math-Preview's solution trajectory\n - https://huggingface.co/datasets/SakanaAI/FishMath-SFT-Data\n\nWe found it difficult to improve gpt-oss-120b's math performance further with the existing open dataset, and I also believe this will be valuable to the future research community 🐠🐟", "votes": 3 } ] }, { "id": 3459443, "postDate": "2026-05-17T20:23:33.690Z", "content": "Apologies for the delay in posting these questions. We're reviewing whether there's a reason this submission was missed from our initial pass; in any case, I've spent the evening reading through your write-up and have the questions below.
\nGiven the late notice, we can give you until Monday, 18th May, 23:59 UTC to respond as a minimum (we'll update if we decide to extend that). Rest assured we'll take the time to give the submission the attention it deserves, and note that we're yet to make a final decision on the winners so you're very much still in consideration here.
\nFor wider context on the rebuttal phase, please see the updated discussion post.
\nLook forward to reading your responses on these.
", "rawMarkdown": "Apologies for the delay in posting these questions. We're reviewing whether there's a reason this submission was missed from our initial pass; in any case, I've spent the evening reading through your write-up and have the questions below.\n\nGiven the late notice, we can give you until Monday, 18th May, 23:59 UTC to respond as a minimum (we'll update if we decide to extend that). Rest assured we'll take the time to give the submission the attention it deserves, and note that we're yet to make a final decision on the winners so you're very much still in consideration here.\n\nFor wider context on the rebuttal phase, please see the [updated discussion post](https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3/discussion/689703).\n\n1. **Temperatures above 1.0:** Your temperature sweeps in Tables 5-7 stop at T=1.0, and T=1.0 ties T=0.7 as the accuracy optimum on the 285-problem benchmark — the curve does not look saturated. Table 5 also shows the SFT model at T=1.0 finishing in 253 minutes with speculative decoding, which leaves real headroom against the 5h cap. Did you ever test T > 1.0 (e.g. 1.1, 1.2) for the SFT model, and if so what did the accuracy-vs-latency frontier look like there?\n2. **Trajectory-level answer quality control:** The SFT data ultimately blends trajectories from Nemotron-Math-v2, Crystal-Math-Preview, and AoPS 2025, each with their own ground-truth quality issues (e.g. Crystal-Math had some issues which were flagged and partially fixed [here](https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3/discussion/672592#3452172); AoPS forum answers can be inconsistent). Your filtering keeps trajectories that \"terminated normally and yielded the correct answer\". Beyond the explicit 0%-success-rate exclusion for AoPS, did you do any independent answer-key spot-checking on the ~22k retained trajectories, and do you have a feel for the residual label-error rate in the final SFT mix?\n3. **Visualising the results:** The writeup has a lot of quantitative findings, but everything is presented in tables with no graphs/charts. Do you feel any charts would help bring the writeup to life, and if so are there any you'd want to add?\n\nLook forward to reading your responses on these.", "votes": 1, "replies": [ { "id": 3460015, "postDate": "2026-05-18T14:46:25.047Z", "content": "We sincerely thank @sambealing for this insightful question and the opportunity to clarify our inference strategy.\nFollowing your feedback, we extended our evaluation to include decoding temperatures up to 1.5. To ensure maximum reliability, we also increased the sampling budget from 4 to 16 trajectories per problem across all temperature settings. We have visualized these updated results in the newly added Figure 2 in our write-up (replacing the previous tabular format to better illustrate the trends).
\n
As the results show, performance steadily improves from T=0.5, peaking exactly at T=1.0 for both mean accuracy and majority voting accuracy, before degrading toward T=1.5. Coverage (pass@16) peaks early at T=0.6 and remains relatively stable up to T=1.0. Concurrently, token length (and therefore inference latency) increases monotonically with temperature. Based on this extended sweep, we are confident that T=1.0 is indeed the optimal operating point, maximizing reasoning performance without incurring the severe latency costs seen at higher temperatures.
\nRegarding the apparent latency headroom (finishing in ~4.2 hours against the 5-hour cap), this reflects a deliberate risk-mitigation strategy. In our local evaluations, we observed that failing to allocate sufficient inference time to each problem causes severe score degradation. Anticipating that the Private LB would feature a significantly harder problem distribution, we prioritized strict pipeline reliability to ensure every problem received its full computational budget. While advanced post-processing methods like GenSelect or SAR (detailed in Appendix C) showed potential to boost accuracy, incorporating LLM-as-a-judge steps introduced significant variance in execution time. The severe penalty of hitting the time limit and leaving problems unattempted far outweighed the marginal accuracy gains. Therefore, we prioritized execution stability and opted for the highly predictable temperature scaling approach.
\nWe greatly appreciate you pointing out this fundamental and crucial aspect of the pipeline. Managing trajectory quality was indeed one of the areas we focused on most intensively throughout the competition.
\nFirst, we would like to detail our preprocessing pipeline for the AoPS data. We implemented an enhanced version of the Nemotron-Math-v2 data curation approach. We categorized problems by type (e.g., integer, non-integer, proof, multiple-choice, binary, image-required) and domain (Geometry, Algebra, Number Theory, Combinatorics, etc.). Crucially, forum posts often contained multiple problems, or single problems contained multiple sub-questions. We parsed these and extracted them individually to ensure each formed a strictly self-contained problem. Given the problem statement and the forum comments, we extracted the corresponding answer and subsequently ran a validation step to check if the problem statement was logically sound and contained all necessary information. To execute this complex pipeline precisely, we utilized Kimi-K2.5 in a non-reasoning configuration.
\nNext, we evaluated these curated problems using gpt-oss-120b (with TIR enabled), generating 8 traces per problem. This yielded an interesting finding: for problems where gpt-oss-120b had a 0% success rate (0/8), the ground-truth answers were mostly incorrect. A manual inspection of about 100 such problems revealed the ground truth was correct only ~30% of the time, typically due to flawed forum comments or the extraction of incomplete intermediate steps. Interestingly, in these \"0% success\" cases, if gpt-oss-120b showed a high internal consensus (agreeing with itself $\\ge 6/8$ times on a different answer), the model's majority answer was usually the truly correct one. Conversely, for problems solved exactly once (1/8), the ground truth was correct about 70% of the time.
\nUltimately, AoPS 2025 was used exclusively for our evaluation benchmark and not for SFT. Given time and compute constraints, our top priority was establishing a highly difficult, contamination-free benchmark. Because the evaluation set size was limited, we selected problems that gpt-oss-120b solved between 0 and 5 times, and manually verified their validity and correct answers.
\nAs for pre-2025 AoPS problems, we attempted to use them for training but only obtained preliminary results. We suspect this is because we were only able to extract a limited number (~2k trajectories) of genuinely high-quality problem-answer pairs, which failed to provide sufficient diversity. Within the competition deadline, we could not find a setting using this data that surpassed our final submission, though further experiments are ongoing.
\nRegarding Nemotron-Math-v2, we agree that a non-negligible fraction of broken problem statements or obviously flawed answers exists, particularly among the subset where gpt-oss-120b struggles. Specifically, when Nemotron-Math-v2 fails to extract an answer, it falls back to using the gpt-oss-120b majority answer as the ground truth. Our manual checks confirmed that this fallback subset is highly enriched for low-quality data. However, when we aggressively filtered out this subset, our post-SFT performance actually degraded slightly. Given this empirical result, our final submission model relies only on the filtering criteria described in our writeup. We hypothesize that because the models used to synthesize our SFT data (e.g., Kimi-K2.5) are more capable than gpt-oss-120b, instances where the extracted answer, the gpt-oss-120b fallback answer, and the strong LLM's generated trajectory all align still provide a robust, meaningful learning signal, essentially validating the synthetic data.
\nFor Crystal-Math-Preview, our manual spot checks revealed almost no fundamentally broken problem statements, so no additional filtering was applied.
\nIn conclusion, while we have no doubt that a small percentage of problem-answer pairs in our final SFT mix retain quality issues, we are confident that we successfully constructed a dataset of sufficient quality to meaningfully enhance the model's reasoning capabilities.
\nThank you for the constructive suggestion. We completely agree that visual aids bring the quantitative findings to life and improve overall readability. Following your advice, we have updated the writeup and replaced several tabular results with plots.
", "rawMarkdown": "# Response to Question 1: Temperatures above 1.0\n\nWe sincerely thank @sambealing for this insightful question and the opportunity to clarify our inference strategy.\nFollowing your feedback, we extended our evaluation to include decoding temperatures up to 1.5. To ensure maximum reliability, we also increased the sampling budget from 4 to 16 trajectories per problem across all temperature settings. We have visualized these updated results in the newly added Figure 2 in our write-up (replacing the previous tabular format to better illustrate the trends).\n\n\n\nAs the results show, performance steadily improves from T=0.5, peaking exactly at T=1.0 for both mean accuracy and majority voting accuracy, before degrading toward T=1.5. Coverage (pass@16) peaks early at T=0.6 and remains relatively stable up to T=1.0. Concurrently, token length (and therefore inference latency) increases monotonically with temperature. Based on this extended sweep, we are confident that T=1.0 is indeed the optimal operating point, maximizing reasoning performance without incurring the severe latency costs seen at higher temperatures.\n\nRegarding the apparent latency headroom (finishing in ~4.2 hours against the 5-hour cap), this reflects a deliberate risk-mitigation strategy. In our local evaluations, we observed that failing to allocate sufficient inference time to each problem causes severe score degradation. Anticipating that the Private LB would feature a significantly harder problem distribution, we prioritized strict pipeline reliability to ensure every problem received its full computational budget. While advanced post-processing methods like GenSelect or SAR (detailed in Appendix C) showed potential to boost accuracy, incorporating LLM-as-a-judge steps introduced significant variance in execution time. The severe penalty of hitting the time limit and leaving problems unattempted far outweighed the marginal accuracy gains. Therefore, we prioritized execution stability and opted for the highly predictable temperature scaling approach.\n\n# Response to Question 2: Trajectory-level answer quality control\n\nWe greatly appreciate you pointing out this fundamental and crucial aspect of the pipeline. Managing trajectory quality was indeed one of the areas we focused on most intensively throughout the competition.\n\nFirst, we would like to detail our preprocessing pipeline for the AoPS data. We implemented an enhanced version of the Nemotron-Math-v2 data curation approach. We categorized problems by type (e.g., integer, non-integer, proof, multiple-choice, binary, image-required) and domain (Geometry, Algebra, Number Theory, Combinatorics, etc.). Crucially, forum posts often contained multiple problems, or single problems contained multiple sub-questions. We parsed these and extracted them individually to ensure each formed a strictly self-contained problem. Given the problem statement and the forum comments, we extracted the corresponding answer and subsequently ran a validation step to check if the problem statement was logically sound and contained all necessary information. To execute this complex pipeline precisely, we utilized Kimi-K2.5 in a non-reasoning configuration.\n\nNext, we evaluated these curated problems using gpt-oss-120b (with TIR enabled), generating 8 traces per problem. This yielded an interesting finding: for problems where gpt-oss-120b had a 0% success rate (0/8), the ground-truth answers were mostly incorrect. A manual inspection of about 100 such problems revealed the ground truth was correct only ~30% of the time, typically due to flawed forum comments or the extraction of incomplete intermediate steps. Interestingly, in these \"0% success\" cases, if gpt-oss-120b showed a high internal consensus (agreeing with itself $\\ge 6/8$ times on a different answer), the model's majority answer was usually the truly correct one. Conversely, for problems solved exactly once (1/8), the ground truth was correct about 70% of the time.\n\nUltimately, AoPS 2025 was used exclusively for our evaluation benchmark and not for SFT. Given time and compute constraints, our top priority was establishing a highly difficult, contamination-free benchmark. Because the evaluation set size was limited, we selected problems that gpt-oss-120b solved between 0 and 5 times, and manually verified their validity and correct answers.\n\nAs for pre-2025 AoPS problems, we attempted to use them for training but only obtained preliminary results. We suspect this is because we were only able to extract a limited number (~2k trajectories) of genuinely high-quality problem-answer pairs, which failed to provide sufficient diversity. Within the competition deadline, we could not find a setting using this data that surpassed our final submission, though further experiments are ongoing.\n\nRegarding Nemotron-Math-v2, we agree that a non-negligible fraction of broken problem statements or obviously flawed answers exists, particularly among the subset where gpt-oss-120b struggles. Specifically, when Nemotron-Math-v2 fails to extract an answer, it falls back to using the gpt-oss-120b majority answer as the ground truth. Our manual checks confirmed that this fallback subset is highly enriched for low-quality data. However, when we aggressively filtered out this subset, our post-SFT performance actually degraded slightly. Given this empirical result, our final submission model relies only on the filtering criteria described in our writeup. We hypothesize that because the models used to synthesize our SFT data (e.g., Kimi-K2.5) are more capable than gpt-oss-120b, instances where the extracted answer, the gpt-oss-120b fallback answer, and the strong LLM's generated trajectory all align still provide a robust, meaningful learning signal, essentially validating the synthetic data.\n\nFor Crystal-Math-Preview, our manual spot checks revealed almost no fundamentally broken problem statements, so no additional filtering was applied.\n\nIn conclusion, while we have no doubt that a small percentage of problem-answer pairs in our final SFT mix retain quality issues, we are confident that we successfully constructed a dataset of sufficient quality to meaningfully enhance the model's reasoning capabilities.\n\n# Response to Question 3: Visualising the results\n\nThank you for the constructive suggestion. We completely agree that visual aids bring the quantitative findings to life and improve overall readability. Following your advice, we have updated the writeup and replaced several tabular results with plots.", "votes": 2 } ] }, { "id": 3459220, "postDate": "2026-05-17T14:15:55.747Z", "content": "Edited on May 17, 2026 for the rebuttal phase:
\nCongrats on successfully fine-tuning a model 2 years in a row! Knowing how hard and expensive it was with a 14B model (many thousands for experiments), I get goosebumps imagining what it would take to fine-tune a model this year lol.
", "rawMarkdown": "Congrats on successfully fine-tuning a model 2 years in a row! Knowing how hard and expensive it was with a 14B model (many thousands for experiments), I get goosebumps imagining what it would take to fine-tune a model this year lol.", "votes": 2 }, { "id": 3447807, "postDate": "2026-04-24T01:34:25.237Z", "content": "nice fine tuning.
", "rawMarkdown": "nice fine tuning. " } ], "complete": true, "source": "live", "fetched_at": "2026-10-20T04:24:13Z", "competition_id": 118448, "competition_slug": "ai-mathematical-olympiad-progress-prize-3", "team_id": 14835489, "solution_url": "/competitions/ai-mathematical-olympiad-progress-prize-3/writeups/pushing-the-limits-post-training-high-capability" }