# AIMO3: Inference-Time Reasoning, Optional RAG, and Dynamic Deadline Allocation

AIMO3: Inference-Time Reasoning, Optional Hybrid RAG, and Dynamic Deadline Allocation
Abstract.
We build on
openai/gpt-oss-120b
in its
base
(released) form because the community has already shown it is strong on this task: the public baseline notebook
[44-50 Let Me Over Cook](https://www.kaggle.com/code/nihilisticneuralnet/44-50-let-me-over-cook)
—using the same family of model and tooling—achieved
44/50
on the public leaderboard, i.e. most problems are within reach of the
unfine-tuned
checkpoint with a good harness. In our own model comparisons,
Qwen3
and
Qwen3.5
did not match that bar and were discarded; we also set aside
NVIDIA Nemotron
(released later) after it looked weaker in our internal checks.
Tinker SFT
on
gpt-oss-120b
with a
merged
adapter, by contrast,
reduced
our public scores (
37
,
26
vs
38
,
40
for the same submission family; see
§7.3
), so our final submission keeps the
base
weights. This submission solves AIMO-style problems with that fixed backbone served via vLLM, the same
system prompt
as
[44-50 Let Me Over Cook](https://www.kaggle.com/code/nihilisticneuralnet/44-50-let-me-over-cook)
, a short
preference prompt
that pushes step-by-step reasoning and heavy use of a
stateful Python (Jupyter) tool
, and
parallel stochastic attempts
with entropy-aware aggregation. The main methodological contribution is
dynamic deadline allocation
: a three-stage time budget that extends only when agreement among completed attempts is insufficient at stage boundaries—reducing wasted wall-clock on problems that already stabilize early. We also implemented a
hybrid dense + BM25 retriever
over a compact math corpus (techniques, example problems, and hints); it is
disabled in the final submission
when empirical checks showed net harm. We summarize
other approaches we tried
(continuations, layered developer prompts, global time-banking, and related
SFT
experiments) that
did not
match the stability of the final pipeline (
[finalnotebook](https://www.kaggle.com/code/jeannkouagou/finalnotebook)
). The
shipped
notebook is
[finalnotebook](https://www.kaggle.com/code/jeannkouagou/finalnotebook)
with base
gpt-oss-120b
; details on merged-SFT public scores are in
§7
. This writeup is
methodology-first
; we do not rely on extra plots or exhaustive numeric tables.
Reproducibility statement.
All implementation details below are grounded in this submission’s Kaggle notebook
[finalnotebook](https://www.kaggle.com/code/jeannkouagou/finalnotebook)
. Tinker training runs are summarized in §7 (configs and metrics were saved with each run).
EDIT on May 13th, 2026
. After the release of the private leaderboard, our final submission scored
41.0/50
, which aligns with our arguments in this writeup about stability since we scored
40/50
on the public leaderboard. Also added a link to the
[code on GitHub](https://github.com/Jean-KOUAGOU/Tinker-SFT-AIMO3)
.
1. Introduction and problem setting
AIMO3 requires producing a
single integer answer
in ({0,\ldots,99999}) under a
global notebook time limit
and per-problem compute constraints.
Backbone and fine-tuning.
The competition implementation standardizes on
openai/gpt-oss-120b
: a public AIMO3 notebook (
[44-50 Let Me Over Cook](https://www.kaggle.com/code/nihilisticneuralnet/44-50-let-me-over-cook)
) already demonstrates
44/50
on the public leaderboard with this model, which motivates investing in
inference-time
design rather than replacing the base weights. We evaluated
Qwen3
/
Qwen3.5
and
NVIDIA Nemotron
as alternatives; they did not justify switching away from
gpt-oss-120b
under our resource and tooling constraints. We also
fine-tuned
gpt-oss-120b
(merged SFT adapter) and found
worse
public results than the base model with the same harness (
§7.3
), so the
primary submission
uses the
original
checkpoint.
The approach treats each problem as a
multi-attempt inference
problem: several independent runs explore the space of reasoning trajectories; we then
stop early
when a strong consensus emerges, and otherwise
allocate more time
only when the evidence suggests the problem is still “open.”
Key Contributions (as implemented in the submission).
Prompting
identical to
[nihilisticneuralnet/44-50-let-me-over-cook](https://www.kaggle.com/code/nihilisticneuralnet/44-50-let-me-over-cook)
for the system message; an additional
preference prompt
biases the model toward tool-assisted verification.
Optional hybrid RAG
(BM25 + dense embeddings) with structured corpus entries;
turned off
for the shipped configuration when it hurt end-to-end performance.
Dynamic deadline allocation
across three stages, driven by
vote counts
at stage boundaries rather than by a fixed per-problem cap only.
Tool output renderer (truncation):
raw Python
stdout
can be extremely long—
on the order of tens of thousands of model-side tokens
in severe cases if passed through verbatim.
Without
truncation, tool returns alone can
consume the context / generation budget
, and otherwise valuable attempts may
stop early
(e.g., context exhaustion) before a final answer is produced.
The
competition submission
is
[finalnotebook](https://www.kaggle.com/code/jeannkouagou/finalnotebook)
(
38
/
40
) (§7.3).
2. Model and serving stack
Backbone:
gpt-oss-120b
(base checkpoint) from the Kaggle model path in the notebook (
CFG.model_path
)—the same line already shown to reach most problems in public (e.g.
44/50
in
[44-50 Let Me Over Cook](https://www.kaggle.com/code/nihilisticneuralnet/44-50-let-me-over-cook)
); we avoid merged SFT here because it
hurt
our public scores relative to this base (§1, §7.3).
Server:
vLLM
OpenAI-compatible API (
python -m vllm.entrypoints.openai.api_server
), with Harmony encoding for GPT-OSS (
HarmonyEncodingName.HARMONY_GPT_OSS
).
Tooling:
A
stateful Jupyter kernel
per worker; the
tool prompt
(§3.2) is written for
efficient computation
—explicit
print()
-based output, SymPy/NumPy guidance, concise dumps, and timeout-aware splitting—plus a
length cap
on tool returns (§5.5).
The end-to-end path is
OpenAI-compatible client → Harmony template (GPT-OSS) → streaming completion with tool calls → Jupyter execution
. We omit standalone architecture figures; the flow is summarized in Appendix E.
3. Prompting
3.1 System prompt (same as public notebook)
The
system_prompt
in
CFG
is
unchanged
from the public AIMO3 starter notebook
[44-50 Let Me Over Cook](https://www.kaggle.com/code/nihilisticneuralnet/44-50-let-me-over-cook)
, by nihilisticneuralnet: it defines an IMO-level solver persona, a structured problem-solving loop (understand → explore → plan → execute → verify), verification requirements, and the required output format (
\boxed{}
with an integer in ([0,99999])).
3.2 Tool prompt
The
tool_prompt
is tuned for
efficient computation
in a
stateful
Python (Jupyter) notebook: it lists available libraries (
math
,
numpy
,
sympy
, etc.) and instructs the model to
print()
results
(last expression is not auto-displayed), to use
SymPy
for exact symbolic work and
NumPy
for fast numeric work, to
estimate complexity
before large searches (prefer better-than-naive algorithms), to keep
printed output concise
(summarize large dumps), and to
split or simplify
work when a cell hits the per-call time limit. This steers tool use toward fewer wasteful megabyte-scale printouts and more targeted code.
3.3 Preference prompt (short, task-specific)
A compact
preference_prompt
is appended to the user side of the conversation:
Solve step-by-step. Use the python tool heavily: compute, don't guess. Verify your answer with a separate computation before writing
\boxed{}
.
This is intentionally short: it
does not replace
the system prompt; it
steers
sampling toward explicit computation and independent verification—aligned with competition scoring (correctness over eloquence).
4. Retrieval-augmented generation (hybrid BM25 + dense)
4.1 Corpus and retriever design
The notebook defines
CorpusRetriever
, a
hybrid
retriever:
BM25:
Stdlib-only indexing with
field-weighted
token counts (e.g., higher weight on
topics
,
techniques
,
recognition_signals
).
Dense:
Qwen3-Embedding
via
sentence-transformers
when the embedding model path exists; cosine similarity
min-max normalized
and fused with BM25:
[
\text{score} = 0.5 \cdot \text{bm25}
{\text{norm}} + 0.5 \cdot \text{dense}
{\text{norm}}
]
(falls back to BM25-only if dense is unavailable).
Retrieved items are formatted as short “strategy blocks” including technique/meta/similar-problem text, topics, insights, optional solution sketch, and
computational hints
.
Corpus size.
The submission narrative uses a
compact corpus
on the order of
~200 entries
(techniques, worked patterns, hints). The notebook logs
len(self.retriever.entries)
when a corpus file is loaded.
4.2 Example log when RAG is enabled
When retrieval is turned on, the top strategies are serialized into a
[REFERENCE STRATEGIES]
block (with closing
[END REFERENCE]
) and prepended to the user turn so the model sees structured hints, topics, techniques, and
per-strategy relevance
(BM25, dense, and fused
combined
scores). Below is a
representative
excerpt from logs (two retrieved entries; line breaks as in stdout):
[REFERENCE STRATEGIES]

--- Strategy 1 [AIME-style] ---
Similar Problem: Triangle ABC inscribed in circle omega. Tangent to omega at B and C intersect at D. AD meets omega again at P. If AB=5, BC=9, AC=10, find AP (as m/n, answer m+n).
Topics: geometry, circles
Techniques: power_of_a_point, similar_triangles
Recognition Signals:
  - tangent to circle
  - inscribed triangle
  - tangent intersection
Key Insights:
  - D is the intersection of tangents at B,C, so DB=DC (tangent lengths)
  - Power of point D: DB^2 = DA * DP
  - D lies on the perpendicular bisector of BC
Strategy: Place D as pole of BC w.r.t. omega. DB^2=DC^2=power of D. By power of a point: DA*DP = DB^2. Compute: use coordinates with BC on x-axis, or use the formula DB^2 = (circumradius)^2 * ... Alternatively: by the tangent-secant relation and similar triangles DAB ~ DPA (since angle DBA = inscribed angle = angle DPA). Then DA/DB = DB/DP gives DP = DB^2/DA. AP = DA - DP or DA + DP depending on configuration.
Code Hint: Use circumradius R = abc/(4*Area). Area by Heron's formula. Coordinate geometry for exact computation.
Relevance: combined=1.000, bm25=1.000, dense=1.000

--- Strategy 2 [Competition] ---
Similar Problem: Point P lies on the circumcircle of square ABCD such that PA*PC = 56 and PB*PD = 90. Find the area of ABCD.
Topics: geometry, circles
Techniques: radical_axis, power_of_a_point
Recognition Signals:
  - circumcircle of square
  - PA * PC
  - PB * PD
Key Insights:
  - For a point on the circumcircle of a square with side s: use Ptolemy or power of a point
  - By Ptolemy on cyclic quadrilateral PABC: PA*BC + PC*AB = PB*AC
  - The square's circumradius is s*sqrt(2)/2
Strategy: Let side = s. Circumradius R = s/sqrt(2). Place square with center at origin. For P on circumcircle: PA*PC and PB*PD relate to the position of P. Using complex numbers with vertices at R*e^{i*k*pi/2}: compute products. By Ptolemy on ABPC: PA*s + PC*s = PB*s*sqrt(2). Combined with PA*PC=56, PB*PD=90 and symmetry relations, solve for s^2 = area.
Code Hint: Use complex number representation: place square vertices at s/sqrt(2) * exp(i*k*pi/2 + i*theta) and parameterize P.
Relevance: combined=0.819, bm25=0.797, dense=0.841

[END REFERENCE]
4.3 Why RAG was disabled in the final submission
Empirical observation:
injecting retrieved context sometimes
reduced
reliability—e.g., misleading analogies, overcrowding the context window, or steering the model away from the cleanest path for a specific problem.
In the notebook’s
default
_load_corpus
configuration, the corpus path is set to a
non-existent placeholder
(
"NOFILE"
), so the retriever is
not
loaded and
solve_problem
uses:
user_input = f'{problem} {self.cfg.preference_prompt}'
.
To enable RAG in a fork, point
corpus_candidates
to your JSON corpus and ensure the Qwen3 embedding model path exists on Kaggle.
Stability, not just mean score.
With hybrid retrieval turned on, we observed
worse stability
of end-to-end performance: for example, the
over-engineered prompts + RAG
public-leaderboard line in
§7.3
spans
34
to
42
out of 50—a
8-point
spread across submits—whereas runs without RAG (or with retrieval off in the shipped stack) were typically
tighter
for us. We saw
similar high-variance behavior
in
other
notebooks that enabled RAG but are
not
detailed in this report. We did not run a dedicated offline
A/B
at fixed seeds; still, the combination of
unreliable peaks
and
heavy tail risk
led us to ship
without
retrieval and invest budget in
dynamic deadlines
and voting instead.
5. Core method: dynamic deadline allocation
![](assets/0a77abbe5c8b7e07228b78f9.png)
5.1 Motivation
Parallel attempts have
heterogeneous completion times
. On some problems,
several attempts finish quickly
with a stable answer (we observed this pattern anecdotally for problems whose final answers included
57447
,
160
, and
520
).
Empirically
, on one representative run the eight parallel attempts showed
Response Length
(assistant stream size, in tokens) from about
26k to 51k
and
Completion Time
from about
292s to 557s
—same harness, large spread—so some trajectories finish much earlier than others. Remaining attempts may still
consume tail latency
without changing the decision. A static “always run until the global max” policy
wastes time
that is better reallocated across the full contest slate.
Design goal:
treat “difficulty”
operationally
—not with a separate classifier—but via
whether the attempt ensemble has converged
by each stage boundary.
5.2 Stage budgets and global cap
For each problem, the solver computes a
remaining notebook budget
and derives a
max_budget
for that problem (function of time left, problems remaining, and configured caps). It then defines three checkpoints:
Stage 1 deadline:
first_deadline = start + first_budget
, with
first_budget = min(first_problem_timeout, max_budget)
.
Stage 2 deadline:
second_deadline = start + second_budget
, with
second_budget = min(second_problem_timeout, max_budget)
.
Stage 3 (final) deadline:
final_deadline = start + final_budget
where
final_budget = max_budget
.
Configured constants in
CFG
include (see notebook):
first_problem_timeout
,
second_problem_timeout
,
final_problem_timeout
,
base_problem_timeout
,
notebook_limit
, and agreement thresholds
first_agreement_count
,
second_agreement_count
.
5.3 Extension rules (agreement-gated)
A background loop polls completed attempts. When
real time
crosses the
current
stage deadline (with a small grace window in
is_past_deadline
), the policy compares the
top vote count
among valid numeric answers to thresholds:
Stage
If condition holds
Else
1
Finalize
when the leading answer has
≥
first_agreement_count
votes
and
len(valid_answers) > first_agreement_count
.
Extend
to stage 2 (or toward final if the stage-2 checkpoint is not later than the current deadline).
2
Finalize
when the leading answer has
≥
second_agreement_count
votes
.
Extend
to
stage 3
(final budget).
Intuition mapping (informal labels):
Easy:
enough agreement early ⇒
finalize at stage 1
—no extension.
Medium:
insufficient agreement after stage 1 ⇒
extend to stage 2
.
Hard:
still insufficient after stage 2 ⇒
extend to stage 3
(full remaining problem budget).
This matches the motivation:
do not
pay for long tail attempts when the vote mass has already stabilized.
5.4 Diagram: deadline policy
![Problem Retrieval Strategy](assets/b5b2d3d4eb118bfedd45c148.png)
5.5 Parallel attempts, early stop, and aggregation
Attempts:
CFG.attempts
parallel threads (with
workers
pool size).
Early global stop:
if any answer reaches
early_stop
copies, the solver sets a stop event and cancels pending futures.
Unanimous shortcut:
if there are at least four valid answers and one answer appears
≥4
times, return it immediately.
Additional boxed-answer prompt (finalization fallback):
when an attempt finishes without a parseable final integer, we issue a short follow-up prompt that requests
only
the final answer in
\boxed{}
format; this recovers trajectories that solved most of the problem but failed to emit the final box cleanly.
Tool output renderer (truncation):
raw Python
stdout
can be extremely long—
on the order of tens of thousands of model-side tokens
in severe cases if passed through verbatim.
Without
truncation, tool returns alone can
consume the context / generation budget
, and otherwise valuable attempts may
stop early
(e.g., context exhaustion) before a final answer is produced. The notebook’s
AIMO3Tool
therefore applies
_truncate_output
: if a tool return exceeds
CFG.max_tool_output_chars
(default
8192
characters), it keeps the
head
and
tail
and inserts a marker
...[python output truncated]...
so the model still sees the beginning and end of the run without flooding the context.
Otherwise:
build
candidates
with ≥2 votes; if still ambiguous,
_select_answer
ranks answers by a
sum of inverse-entropy weights
across attempts (lower entropy ⇒ higher weight), then picks the top score.
We did not produce separate
distribution plots
of time-to-consensus or stage-finalization rates; the design rationale above is the primary evidence for this component.
6. Explored variants and negative results
Beyond
[finalnotebook](https://www.kaggle.com/code/jeannkouagou/finalnotebook)
, we implemented and submitted several
alternative designs
that explored richer prompting, continuation-based search, global time-banking, and a
merged SFT
checkpoint.
Public leaderboard scores were not consistently strong
across reruns and configurations. Empirically, the main failure modes were
over-long or over-specified preference / system-adjacent text
(which diluted the stable baseline persona and steered tool use inefficiently) and
time-management schemes that were harder to tune
than the simpler
dynamic per-problem deadline
policy in the shipped solution.
Chosen solution.
We selected
[finalnotebook](https://www.kaggle.com/code/jeannkouagou/finalnotebook)
as the primary submission: it was submitted
twice
and scored
38
and
40
(50 points max), which is both
more stable
and
better
than the ranges below. More importantly, this approach holds enormous potential due to its time saving strategy.
Approach
Approx. score range (of 50)
Main idea (brief)
Two-phase continuations
33–40
Initial wave of attempts, then
forked
continuation trajectories (doubling the number of attempts 8==>16) with a dedicated continuation user message; optional
compute-heavy
system variant; long structured preference text; tooling penalties and deadline-aware forced
\boxed{}
extraction.
Over-engineered prompts + RAG
34–42
Layered
developer_prompt
, rewritten system copy, verbose preference steering;
hybrid retrieval
on (strategy blocks in the user turn).
§7.3
table.
Merged SFT model (Tinker)
26–37
SFT
on
gpt-oss-120b
, adapter
merged
into full weights;
CFG.model_path
points at the merged HF directory.
Public leaderboard:
37
,
26
(two submits);
§7.3
.
Banked time + hard reserve
30–40
Global time bank
and
hard-problem reserve:
adaptive per-problem budgets draw from a bank while protecting a reserve—more complex scheduling than fixed or stage-wise caps.
Why these did not work.
The headline pattern was
variance
, not a clear ceiling lift:
preference prompts
that were long or multi-part often
hurt
consistency (either by distracting the model from the baseline reasoning contract, or by encouraging expensive tool loops without a matching gain in agreement).
Time policies
that depended on global bank state or multi-phase continuation schedules were
sensitive to problem order and tail latency
, so they did not reliably beat the simpler
agreement-gated, stage-wise deadline extension
in
[finalnotebook](https://www.kaggle.com/code/jeannkouagou/finalnotebook)
. The shipped solution keeps the
short preference prompt
and focuses innovation on
deadline allocation
and
parallel voting
—a combination that still has
considerable headroom
(e.g., tighter thresholds, retrieval gating, or calibrated early stopping), but already
matches or exceeds
the best outcomes from the exploratory line above.
7. Training (Tinker), evaluation scope, and why the submission is base-model inference
7.1 Tinker fine-tuning experiments
We trained
openai/gpt-oss-120b
with
Tinker
using
LoRA
(rank
32
in saved run configs) and the
gpt_oss_no_sysprompt
renderer. Each run produced configs, metrics, checkpoints, and timing metadata (typical Tinker / Tinker Cookbook layout). The code is here
[Code on GitHub](https://github.com/Jean-KOUAGOU/Tinker-SFT-AIMO3)
. Representative settings:
Run (label)
Regime
Notes
SFT (on mixture of
[AIMO3 Dataset](https://www.kaggle.com/datasets/jeannkouagou/aimo3-tool-integrated-reasoning)
and
[Astral Math](https://www.kaggle.com/datasets/nguyennguyen599/astral-math-v1)
), 100k sample size
SFT
learning_rate
1e-4, linear schedule,
1
epoch, batch
8
, max length
32768
, JSONL SFT dataset,
train_on_what
: last assistant message
SFT (on 10k examples of
[AIMO3 Dataset](https://www.kaggle.com/datasets/jeannkouagou/aimo3-tool-integrated-reasoning)
)
SFT
Same LR / rank;
4000
steps; JSONL SFT dataset
SFT loss (from logs).
Our long SFT run on Tinker (1 epoch on 100k samples) shows training NLL generally trending downward (about
0.77 → 0.35
, with noisy batch-level fluctuations). A compact view from sampled steps:
![Figure 4: SFT train_mean_nll vs step (raw + moving average)](assets/4f24ad8428f58e48010a416e.png)
These runs document
what we tried
; they are
not
the primary story of the final submission.
Merged SFT model submission.
For supervised fine-tuning we
merged
the trained adapter into base
gpt-oss-120b
using the
Tinker Cookbook
path (
tinker_cookbook.weights.build_hf_model
). That yields a
different checkpoint
; serving it means
CFG.model_path
points at the
merged
directory.
§7.3
and
Figures 1–3
list
merged SFT
as its
own
row/category (orange in Figure 1).
7.2 What the public leaderboard comparison shows
§7.3
aggregates repeated
public leaderboard
submits:
base-model
stacks (continuation,
over-engineered prompts + RAG
, banked time),
merged SFT
(different weights), and our
two
[finalnotebook](https://www.kaggle.com/code/jeannkouagou/finalnotebook)
runs (
38
,
40
). Qualitatively,
continuation-enabled
can reach
40
but also
33
–
34
;
merged SFT
shows a
wide
two-run spread (
26
–
37
) and a
lower mean
than shipped;
banked-time
includes very low scores (
30
–
32
). We do not claim SOTA against all open-weight or commercial systems; the relevant comparison is
within our own runs
and the
AIMO3
leaderboard context.
7.3 Empirical results: public leaderboard scores
Table: raw
public leaderboard
scores (out of
50
).
Base-model
rows use
gpt-oss-120b
in
CFG.model_path
(not the merged SFT directory).
Merged SFT
is a
different checkpoint
(Tinker adapter merged into full weights)—listed on its
own
row. Multiple submits reflect iterative tuning and harness variance—not i.i.d. experiments—so we emphasize
ranges
,
means
, and
visuals
rather than (p)-values.
Approach
Scores (each submit)
(n)
Min
Max
Mean
Continuation-enabled
40, 37, 38, 39, 33, 34, 34
7
33
40
36.4
Over-engineered prompts + RAG
37, 35, 42, 34, 41, 38, 38, 38
8
34
42
37.9
Banked time + reserve
30, 39, 40, 35, 34, 32, 33, 40
8
30
40
35.4
Merged SFT (Tinker)
37, 26
2
26
37
31.5
Shipped:
[finalnotebook](https://www.kaggle.com/code/jeannkouagou/finalnotebook)
(base)
38, 40
2
38
40
39
Figure 1 — Distribution per approach.
Box plots (median in red, mean as blue dashed line) with jittered points for every submit.
Merged SFT (Tinker)
uses an
orange
box (different model family). The
shipped
[finalnotebook](https://www.kaggle.com/code/jeannkouagou/finalnotebook)
row is
green
; with
two
submits it is a short, high band.
![Figure 1: box plot of public leaderboard scores by approach](assets/53f091d3ef571112ac8a253f.png)
Figure 2 — Mean score with approximate 95% CI for the mean
(more submits ⇒ narrower intervals). The
shipped
mean is
39
(two submits at
38
and
40
).
![Figure 2: mean public leaderboard score by approach with error bars](assets/e7fe00c2b55c28d3235a3ea3.png)
Figure 3 — Range (min–max bar) and mean (dot).
Highlights
risk
:
banked time
spans
30
–
40
;
merged SFT
spans
26
–
37
on only
two
submits;
over-engineered + RAG
spans
34
–
42
;
continuation-enabled
spans
33
–
40
.
![Figure 3: min–max range and mean by approach](assets/f109fa02fd2655b55d632748.png)
Reading.
Among
base-model
exploratory lines,
continuation-enabled
reaches
40
on some submits but
33
–
34
on others;
over-engineered + RAG
(mean ~
37.9
) and
banked-time
show wide variance.
Merged SFT
(different weights), with only
two
public scores (
37
,
26
), mean ~
31.5
, includes the
lowest
observed point in the whole panel (
26
)—so our
SFT model submission
did
not
beat
[finalnotebook](https://www.kaggle.com/code/jeannkouagou/finalnotebook)
(
38
,
40
) under this evaluation budget. We
preferred
[finalnotebook](https://www.kaggle.com/code/jeannkouagou/finalnotebook)
as the
primary submission
for
strong scores
, a
clean inference story
(§3–6), and
avoidance
of the
worst tails
under
banked-time
(
30
–
32
),
continuation
(
33
–
34
), and
merged SFT
(
26
).
7.4 What this writeup does not include
We intentionally
do not
include exhaustive ablation tables (fixed vs dynamic deadlines, grid over
first_agreement_count
/
second_agreement_count
,
early_stop
sweeps) or
per-topic
error breakdowns: those would require a frozen offline harness and labels beyond the scope of this narrative. The
methodology
in §3–6 is the reproducible artifact readers can build on.
8. Limitations and future work
RAG quality:
curated corpora can distract; future work could use
confidence gating
. Here
BM25
and
dense
each produce a
ranked list
of corpus entries for the same query;
they disagree
when those lists
do not point to the same evidence
—e.g.
small overlap
between BM25’s top-3 and the dense embedder’s top-3, or a
high fusion score
on a document that is
not
near the top for
both
methods. This could be a practical signal that
relevance is ambiguous
, so injecting strategies may help; when both lists
largely agree
, retrieval may add little and can be
skipped or shortened
. One could also inject only when
parallel attempts disagree
on the answer (low self-consistency).
Difficulty proxy:
vote thresholds are simple; a
lightweight runtime probe
(e.g., entropy of partial trajectories) might refine stage transitions.
Tail latency:
stricter cancellation policies may interact with tool timeouts; tuning would need
measured
trade-offs on held-out problems.
9. Ethics and attribution
The
system prompt
follows the
public
AIMO3 notebook text in
[https://www.kaggle.com/code/nihilisticneuralnet/44-50-let-me-over-cook](https://www.kaggle.com/code/nihilisticneuralnet/44-50-let-me-over-cook)
. The
submission implementation
(dynamic deadlines, tooling, aggregation) is documented in
[finalnotebook](https://www.kaggle.com/code/jeannkouagou/finalnotebook)
. Tooling and inference patterns build on the competition’s reference stack (
kaggle_evaluation
, vLLM, Harmony).
Appendix (supporting material; expandable beyond main word limit)
A. Reproducibility checklist
Item
Notebook reference / value
Model path
CFG.model_path
(
gpt-oss-120b
Kaggle input path)
Served name
CFG.served_model_name
Seeds
CFG.seed
; per-attempt seed derived in
_process_attempt
Time limits
notebook_limit
,
first_problem_timeout
,
second_problem_timeout
,
final_problem_timeout
,
base_problem_timeout
Attempts / workers
attempts
,
workers
Tool limits
jupyter_timeout
,
max_tool_output_chars
, etc.
Retrieval
CorpusRetriever
+
_load_corpus
path list; default disables retriever
Stop / consensus
early_stop
, unanimous rule,
_select_answer
weighting
Primary submission
[finalnotebook](https://www.kaggle.com/code/jeannkouagou/finalnotebook)
— two
public leaderboard
submissions scored
38
and
40
(of 50)
Exploratory approaches (negative / unstable)
Continuations; layered developer prompts;
merged SFT
model submission; banked time — see §6
Tinker SFT
Run configs and metrics as in §7.1 (saved alongside each training job)
Merge SFT adapter → full HF
Tinker Cookbook
build_hf_model
; merged directory as
CFG.model_path
for
merged SFT
submissions (§7)
Exact
Kaggle notebook revision IDs
and
pip freeze
outputs may change per session; reproduce from the published
[finalnotebook](https://www.kaggle.com/code/jeannkouagou/finalnotebook)
and the competition
Docker / input
snapshot at submit time.
B. Figures (public leaderboard)
§7.3
includes
three
figures: (1) box plot + jittered submits (
merged SFT
in orange, shipped in green), (2) mean with uncertainty on the mean, (3) min–max range with mean dot.
C. Extended examples
Full tool transcripts are long and environment-specific. Readers should run
[finalnotebook](https://www.kaggle.com/code/jeannkouagou/finalnotebook)
on Kaggle and inspect stdout / displayed attempt tables for end-to-end behavior (retrieval off in the default configuration).
D. Bibliography
nihilisticneuralnet.
44-50 Let Me Over Cook
— public AIMO3 baseline notebook whose
system_prompt
we reuse unchanged:
[https://www.kaggle.com/code/nihilisticneuralnet/44-50-let-me-over-cook](https://www.kaggle.com/code/nihilisticneuralnet/44-50-let-me-over-cook)
.
jeannkouagou.
finalnotebook
— this team’s primary submission (implementation of dynamic deadlines, optional RAG, voting):
[https://www.kaggle.com/code/jeannkouagou/finalnotebook](https://www.kaggle.com/code/jeannkouagou/finalnotebook)
.
AIMO2 winning solution writeup (example of academic-style competition report):
[arXiv:2504.16891](https://arxiv.org/abs/2504.16891)
.
OpenAI.
gpt-oss
model card / documentation (base model used for inference): see Hugging Face / OpenAI model hub entry for
openai/gpt-oss-120b
as referenced in Tinker configs.
vLLM:
[https://github.com/vllm-project/vllm](https://github.com/vllm-project/vllm)
(OpenAI-compatible serving). Sentence Transformers:
[https://www.sbert.net/](https://www.sbert.net/)
(dense retrieval). SymPy:
[https://www.sympy.org/](https://www.sympy.org/)
(symbolic checks in tooling).