Two concrete, publicly verifiable contributions to the AIMO3 open-source lineage:
\nA. SC + TIR scaffold on top of parthenos's streaming baseline. I published Self-Consistency Strategy (notebook slug: self-consistency-strategy, V12, 170 upvotes). It merges parthenos's streaming-inference reflexion prompting notebook (score 27) with shelterw's Python-tool line, and adds what those notebooks lacked: (i) the openai_harmony chat template with ReasoningEffort.HIGH, (ii) a per-thread persistent Jupyter kernel (BlockingKernelClient + port-pool allocator), (iii) K=8 parallel self-consistency with in-stream \\boxed{…} scanning and a majority-threshold early-stop, (iv) a 16-thread OS-page-cache weight preloader that cuts vLLM startup from ~400 s to ~130 s. This notebook became the foundation ZaynYu forked into [40/50] GPT-OSS-120B TIR+DynamicTime+KernelPool (615 upvotes, Public LB 40).
B. Python-verification-weighted voting. In the ZaynYu → daniil1423 → my fork chain, the only non-trivial change I made to the voting logic was to weight each attempt's vote by whether it used the Python tool:
\nTie-break key: (weighted_votes, raw_votes, total_calls). This pushed my best same-code submission from the 35–40 band (ZaynYu) to 42/50 peak over n=25 V2 submissions (μ = 38.44, σ = 1.58).
Neither contribution changes the ceiling by much — the ceiling is set by gpt-oss-120b's pass@k on this problem set — but both are small, reproducible, and the scaffold in (A) is demonstrably load-bearing downstream.
AIMO3 gives 50 IMO-style problems with integer answers in [0, 99999]. Each submission is scored twice; a problem earns 1.0 if both runs agree and are correct, 0.5 if one run is correct, and 0 otherwise. Inference runs offline on a single NVIDIA H100-85GB under a 5-hour wall-clock limit. Models must have been released before 2026-03-15.
I used gpt-oss-120b (120B parameters, MXFP4 weights, ~65 GB VRAM) served via vLLM==0.11.2. The KV cache budget on 1×H100 with this model is about 15–16 GB, which constrains safe concurrent requests to roughly 8.
This is the relevant portion of the public fork tree. Arrows point from upstream to downstream; numbers in parentheses are approximate Public LB peaks reported in each notebook's title or comments.
\nKawchar Husain (upstream prompt template)\n │\n ▼\nparthenos · aimo3-streaming-inference-reflexion-prompting (V1, 27)\n │ streaming inference + reflexion prompting; no TIR; no Harmony\n │\nshelterw · way-to-30-50-gpt-oss-w-python-tool (parallel upstream: simple Python tool loop)\n │\n ▼\ncanqiang (me) · self-consistency-strategy (V12, 32) ← Contribution A\n │ + Harmony protocol (ReasoningEffort.HIGH)\n │ + per-thread persistent Jupyter kernel pool\n │ + K=8 parallel SC with in-stream \\boxed scanning + early-stop\n │ + 16-thread OS-page-cache weight preloader\n │\n ▼\nZaynYu · [40/50] GPT-OSS-120B TIR+DynamicTime+KernelPool (V26, 40)\n │ + dynamic per-problem time budget\n │ + class refactor (AIMO3Solver / AIMO3Sandbox / AIMO3Tool)\n │ + refined system prompts (\"world-class IMO competitor\", \"Never guess\")\n │\n ▼\ndaniil1423 · entropy-trajectory-scoring (40)\n │ + logprob/entropy-based trajectory scoring (I did not use this part)\n │\n ▼\ncanqiang (me) · fork of ETS (\"Weighted Voting with Python Verification\" V2, 42) ← Contribution B\n + python-call-count & error-rate weighted voting\n (kept the rest of the ETS/ZaynYu infrastructure unchanged)\nTwo things I want to be explicit about:
\ngpt-oss-120b, native MXFP4 weights, served by vllm 0.11.2.--kv-cache-dtype fp8_e4m3, --max-model-len 65536, --gpu-memory-utilization 0.96, --max-num-seqs 256, --async-scheduling.openai_harmony for the chat template. System message is constructed with SystemContent.new().with_reasoning_effort(ReasoningEffort.HIGH).with_tools(python_tool.tool_config), which is the native format gpt-oss-120b was post-trained on.temperature=1.0, top_p=1.0, min_p=0.02. Temperature is not a free knob for aligned models — techtech69's writeup (85th, public 43) documents an 8-point drop when they went to 0.6. My ablations agree.Each incoming problem is handled by K=8 worker threads sharing a ThreadPoolExecutor. Each thread:
AIMO3Sandbox (Jupyter kernel) from a pool.stream=True, return_token_ids=True, streams tokens while:stop_event and the per-problem deadline,\\boxed{…} and bailing out as soon as a valid integer in [0, 99999] is found.python tool call, the sandbox executes the code in a stateful kernel and the output is fed back as a new conversation message. python_calls and python_errors are tracked per attempt.early_stop = 5), the stop event fires and the remaining threads are cancelled.After the parallel sweep, votes are aggregated with four small rules on top of raw majority:
\n# AIMO3Solver._select_answer, used in the 42-score fork of ETS V2\nfor result in detailed_results:\n answer = result['Answer']\n if answer is None:\n continue\n weight = 1.0\n calls = result['Python Calls']\n errors = result['Python Errors']\n if calls >= 3:\n weight += 0.5 # strong Python-verified answer\n elif calls >= 1:\n weight += 0.25 # at least one Python check\n if calls > 0 and errors / calls > 0.5:\n weight -= 0.3 # high error rate → unreliable\n stats[answer]['weighted_votes'] += weight\n stats[answer]['votes'] += 1\n stats[answer]['calls'] += calls\n stats[answer]['errors'] += errors\n\n# Tie-break: weighted votes > raw votes > total Python calls\nreturn max(stats.items(),\n key=lambda x: (x[1]['weighted_votes'],\n x[1]['votes'],\n x[1]['calls']))[0]\nThe intuition is trivial: if an answer was arrived at by an attempt that ran Python code several times and those calls mostly returned without an exception, it is more likely to be correct than an answer the model guessed without verification. The coefficients (0.25 / 0.5 / 0.3 / threshold 0.5) are not tuned; they are round numbers that fell out of the first thing I tried. The contribution is the structure of the weighting, not the specific values.
\nRunning the same notebook twice on Kaggle does not give the same score. This is not a subtle effect and it is not my claim — it is reported by both techtech69 and ZaynYu in their own writeups / comments. On vLLM, batched completion order depends on GPU scheduling, floating-point associativity perturbs the softmax, and downstream token sampling diverges.
\nFrom public comments on parthenos's aimo3-streaming-inference-reflexion-prompting V1 (author score 27):
| Forker | \nScore | \n
|---|---|
| Poemcourt (copy, no changes) | \n26 | \n
| Dipankar Mitra (copy, no changes) | \n21 | \n
| CloudYU | \n22 | \n
| Octavi Grau | \n16 | \n
| Artem Goncharov | \n18 | \n
| swordsman (after increasing timeout) | \n22 | \n
Many more hit \"Notebook threw exception\". Spread is roughly 16–26 against an author-reported 27.
\nFrom the comment on my self-consistency-strategy:
\n\n\"i scored 35 with no changes…\" — cm391
\n
Author peak on that notebook was 32 on LB; the commenter got 35 with zero changes, and downstream forks that layered additional infrastructure (ZaynYu, daniil1423) reached 40+.
\nZaynYu's [40/50] GPT-OSS-120B TIR+DynamicTime+KernelPool (615 upvotes, 1,362 copies) has 57 comments and is the most-forked downstream child of my SC scaffold. A selection of the scores people reported, quoted literally from that comment thread:
| Forker | \nVariant | \nPublic LB | \n
|---|---|---|
| Tong Hui Kang | \nV1 (direct copy) | \n38 | \n
| Tong Hui Kang | \nV2 | \n35 | \n
| Tong Hui Kang | \nV10 | \n36 | \n
| Tong Hui Kang | \nV12 | \n36 | \n
| Siancy (\"Same as parthenos's, no modifications\") | \nV1 | \n41 | \n
| THUNDER THUNDER | \nV01 | \n40 | \n
| THUNDER THUNDER | \nV02 | \n36 | \n
| Abhijeet Bhargava | \nV01 / V03 / V05 | \n38 / 35 / 34 | \n
| chenjixuan666 (fork of Siancy) | \nV01 | \n38 | \n
| parthenos (on ZaynYu's V15) | \n— | \n37 | \n
| Dipankar Mitra | \nsubmitted twice | \n33 / 33 | \n
ZaynYu himself on his own notebook:
\n\n\n\"The score of the LB by running the same notebook is not guaranteed to be the same.\"
\n
So, empirically, the scaffold I contributed (A) sits downstream of parthenos's 16–26 band and supports a community reproduction band of roughly 34–41 on top of the same hardware.
\nI submitted the same V2 code 25 times to measure the noise floor:
\nA single run scoring 42 is well inside the tail; a single run scoring 35 is also inside the tail. This is why I frame the Python-weighted voting as \"pushed the ceiling up by ~2 points\", not \"solves more problems\". The voting change is not the dominant source of score; the model and the TIR loop are.
\nEverything in this table is a single-submission LB score from my own submission list. They are not independent trials — each number is a draw from a noisy distribution with σ ≈ 1–2 on top of a ~35–42 base. I report them with that caveat.
\n| Configuration | \nBest | \nMean | \nN | \nNotes | \n
|---|---|---|---|---|
| Weighted Voting w/ Python Verification (V2, contribution B) | \n42 | \n38.44 | \n25 | \nPeak and baseline for my work | \n
| ans_verifys (verifier variant) | \n41 | \n37.67 | \n6 | \nDifferent verification logic, similar spread | \n
| entropy-trajectory-scoring (daniil1423's logprob-entropy scoring) | \n39 | \n36.78 | \n9 | \nWithout my weighted voting, entropy scoring alone | \n
| Dual Entropy-Weighted Voting | \n37 | \n35.71 | \n7 | \nCombining two entropy signals | \n
| ZaynYu-style direct fork (36/40 variant) | \n34 | \n32.67 | \n3 | \nBefore my voting change | \n
| self-consistency-strategy V12 (contribution A, public) | \n32 | \n— | \n2 | \nBefore any voting change; scaffold only | \n
| parthenos direct fork (streaming + reflexion, no TIR) | \n26 | \n23 | \n2 | \nUpstream baseline | \n
| improved-tir-solution | \n25 | \n21 | \n3 | \nDifferent TIR formulation — didn't pay off | \n
The progression from parthenos → SC+TIR scaffold → ZaynYu's DynamicTime → weighted voting is visible across my own submissions, not just across the public fork tree.
\nsympy/numpy/mpmath call disappears. BlockingKernelClient + a per-thread port allocator avoids collisions that bite you as soon as you try to scale past K=2.ReasoningEffort.HIGH + min_p=0.02. The model's native training format outperforms any hand-crafted prompt I tried, including \"step by step\" and \"think carefully\" variants. This is also what techtech69 reports.vllm starts saves ~270 s of startup. Every second saved here is a second available to the hardest problems.\\boxed{…} scanning. Once a valid integer appears in the stream, close the connection. Don't wait for the model to finish.gpt-oss-120b. 16 and 24 concurrent attempts are preempted by vLLM and score catastrophically worse. techtech69 reports −22 at K=24; I observed the same pattern at smaller scale.base_problem_timeout with higher K. 100 s is not enough tokens to finish a hard problem. You lose more to truncated outputs than you gain from additional samples.+0.5 / +0.25 / −0.3 with a 0.5 error-rate threshold is just the first thing I tried. I don't know whether +0.3 / +0.15 / −0.2 would be better, worse, or identical within the noise.queue.Queue, the refined system prompts — the weighted-vote rule has nothing to rest on. The right way to read this writeup is \"I added two small bricks to a wall that other people built most of\", not \"I solved AIMO3 my way\".(problem_id, attempt_i, answer, python_calls, python_errors, token_len) tuple to a file that's in the submission output, so I can do real ablation offline.gpt-oss-120b caps out at ~43–44 on this hardware because its per-attempt accuracy on hard problems is ~5%. Both techtech69's and the host's pass@n data strongly suggest the gain from switching models is larger than the gain from any more voting tricks.Notebooks:
\ncanqiang/self-consistency-strategy (V12, 170 upvotes).daniil1423/entropy-trajectory-scoring with the _select_answer method replaced by the 20-line rule in §4.3. Currently private; I'll open-source it shortly.Key configuration for the 42-score submission:
\nclass Config:\n model_path = '/kaggle/input/gpt-oss-120b/transformers/default/1'\n served_model_name = 'gpt-oss'\n context_tokens = 65536\n kv_cache_dtype = 'fp8_e4m3'\n gpu_memory_utilization = 0.96\n batch_size = 256\n\n temperature = 1.0\n top_p = 1.0\n min_p = 0.02\n seed = 42\n attempts = 8\n workers = 16\n early_stop = 5\n turns = 128\n\n high_problem_timeout = 900\n base_problem_timeout = 300\n notebook_limit = 17400\n sandbox_timeout = 3\n jupyter_timeout = 10\nHardware: 1×H100 85GB, Kaggle competition runtime, 5h wall clock.
\nExpected variance: μ ≈ 38, σ ≈ 1.6 across identical-code submissions. A single run scoring in [35, 42] should be considered a successful reproduction.
Most of what is in this notebook is not mine. I want to name the people whose work I built on, in the order I inherited it:
\naimo3-streaming-inference-reflexion-prompting — the streaming-inference reflexion prompt scaffold that my SC+TIR notebook merges with shelterw's line.[Way to 30+/50] GPT-OSS - w/ python tool, the other upstream I merged into the SC notebook.If you are one of these people and I have miscredited any part of your work, please tell me and I will correct this writeup.
\nWriteups / notebooks directly referenced above:
\nExternal methodology references:
\nTwo concrete, publicly verifiable contributions to the AIMO3 open-source lineage:
\nA. SC + TIR scaffold on top of parthenos's streaming baseline. I published Self-Consistency Strategy (notebook slug: self-consistency-strategy, V12, 170 upvotes). It merges parthenos's streaming-inference reflexion prompting notebook (score 27) with shelterw's Python-tool line, and adds what those notebooks lacked: (i) the openai_harmony chat template with ReasoningEffort.HIGH, (ii) a per-thread persistent Jupyter kernel (BlockingKernelClient + port-pool allocator), (iii) K=8 parallel self-consistency with in-stream \\boxed{…} scanning and a majority-threshold early-stop, (iv) a 16-thread OS-page-cache weight preloader that cuts vLLM startup from ~400 s to ~130 s. This notebook became the foundation ZaynYu forked into [40/50] GPT-OSS-120B TIR+DynamicTime+KernelPool (615 upvotes, Public LB 40).
B. Python-verification-weighted voting. In the ZaynYu → daniil1423 → my fork chain, the only non-trivial change I made to the voting logic was to weight each attempt's vote by whether it used the Python tool:
\nTie-break key: (weighted_votes, raw_votes, total_calls). This pushed my best same-code submission from the 35–40 band (ZaynYu) to 42/50 peak over n=25 V2 submissions (μ = 38.44, σ = 1.58).
Neither contribution changes the ceiling by much — the ceiling is set by gpt-oss-120b's pass@k on this problem set — but both are small, reproducible, and the scaffold in (A) is demonstrably load-bearing downstream.
AIMO3 gives 50 IMO-style problems with integer answers in [0, 99999]. Each submission is scored twice; a problem earns 1.0 if both runs agree and are correct, 0.5 if one run is correct, and 0 otherwise. Inference runs offline on a single NVIDIA H100-85GB under a 5-hour wall-clock limit. Models must have been released before 2026-03-15.
I used gpt-oss-120b (120B parameters, MXFP4 weights, ~65 GB VRAM) served via vLLM==0.11.2. The KV cache budget on 1×H100 with this model is about 15–16 GB, which constrains safe concurrent requests to roughly 8.
This is the relevant portion of the public fork tree. Arrows point from upstream to downstream; numbers in parentheses are approximate Public LB peaks reported in each notebook's title or comments.
\nKawchar Husain (upstream prompt template)\n │\n ▼\nparthenos · aimo3-streaming-inference-reflexion-prompting (V1, 27)\n │ streaming inference + reflexion prompting; no TIR; no Harmony\n │\nshelterw · way-to-30-50-gpt-oss-w-python-tool (parallel upstream: simple Python tool loop)\n │\n ▼\ncanqiang (me) · self-consistency-strategy (V12, 32) ← Contribution A\n │ + Harmony protocol (ReasoningEffort.HIGH)\n │ + per-thread persistent Jupyter kernel pool\n │ + K=8 parallel SC with in-stream \\boxed scanning + early-stop\n │ + 16-thread OS-page-cache weight preloader\n │\n ▼\nZaynYu · [40/50] GPT-OSS-120B TIR+DynamicTime+KernelPool (V26, 40)\n │ + dynamic per-problem time budget\n │ + class refactor (AIMO3Solver / AIMO3Sandbox / AIMO3Tool)\n │ + refined system prompts (\"world-class IMO competitor\", \"Never guess\")\n │\n ▼\ndaniil1423 · entropy-trajectory-scoring (40)\n │ + logprob/entropy-based trajectory scoring (I did not use this part)\n │\n ▼\ncanqiang (me) · fork of ETS (\"Weighted Voting with Python Verification\" V2, 42) ← Contribution B\n + python-call-count & error-rate weighted voting\n (kept the rest of the ETS/ZaynYu infrastructure unchanged)\nTwo things I want to be explicit about:
\ngpt-oss-120b, native MXFP4 weights, served by vllm 0.11.2.--kv-cache-dtype fp8_e4m3, --max-model-len 65536, --gpu-memory-utilization 0.96, --max-num-seqs 256, --async-scheduling.openai_harmony for the chat template. System message is constructed with SystemContent.new().with_reasoning_effort(ReasoningEffort.HIGH).with_tools(python_tool.tool_config), which is the native format gpt-oss-120b was post-trained on.temperature=1.0, top_p=1.0, min_p=0.02. Temperature is not a free knob for aligned models — techtech69's writeup (85th, public 43) documents an 8-point drop when they went to 0.6. My ablations agree.Each incoming problem is handled by K=8 worker threads sharing a ThreadPoolExecutor. Each thread:
AIMO3Sandbox (Jupyter kernel) from a pool.stream=True, return_token_ids=True, streams tokens while:stop_event and the per-problem deadline,\\boxed{…} and bailing out as soon as a valid integer in [0, 99999] is found.python tool call, the sandbox executes the code in a stateful kernel and the output is fed back as a new conversation message. python_calls and python_errors are tracked per attempt.early_stop = 5), the stop event fires and the remaining threads are cancelled.After the parallel sweep, votes are aggregated with four small rules on top of raw majority:
\n# AIMO3Solver._select_answer, used in the 42-score fork of ETS V2\nfor result in detailed_results:\n answer = result['Answer']\n if answer is None:\n continue\n weight = 1.0\n calls = result['Python Calls']\n errors = result['Python Errors']\n if calls >= 3:\n weight += 0.5 # strong Python-verified answer\n elif calls >= 1:\n weight += 0.25 # at least one Python check\n if calls > 0 and errors / calls > 0.5:\n weight -= 0.3 # high error rate → unreliable\n stats[answer]['weighted_votes'] += weight\n stats[answer]['votes'] += 1\n stats[answer]['calls'] += calls\n stats[answer]['errors'] += errors\n\n# Tie-break: weighted votes > raw votes > total Python calls\nreturn max(stats.items(),\n key=lambda x: (x[1]['weighted_votes'],\n x[1]['votes'],\n x[1]['calls']))[0]\nThe intuition is trivial: if an answer was arrived at by an attempt that ran Python code several times and those calls mostly returned without an exception, it is more likely to be correct than an answer the model guessed without verification. The coefficients (0.25 / 0.5 / 0.3 / threshold 0.5) are not tuned; they are round numbers that fell out of the first thing I tried. The contribution is the structure of the weighting, not the specific values.
\nRunning the same notebook twice on Kaggle does not give the same score. This is not a subtle effect and it is not my claim — it is reported by both techtech69 and ZaynYu in their own writeups / comments. On vLLM, batched completion order depends on GPU scheduling, floating-point associativity perturbs the softmax, and downstream token sampling diverges.
\nFrom public comments on parthenos's aimo3-streaming-inference-reflexion-prompting V1 (author score 27):
| Forker | \nScore | \n
|---|---|
| Poemcourt (copy, no changes) | \n26 | \n
| Dipankar Mitra (copy, no changes) | \n21 | \n
| CloudYU | \n22 | \n
| Octavi Grau | \n16 | \n
| Artem Goncharov | \n18 | \n
| swordsman (after increasing timeout) | \n22 | \n
Many more hit \"Notebook threw exception\". Spread is roughly 16–26 against an author-reported 27.
\nFrom the comment on my self-consistency-strategy:
\n\n\"i scored 35 with no changes…\" — cm391
\n
Author peak on that notebook was 32 on LB; the commenter got 35 with zero changes, and downstream forks that layered additional infrastructure (ZaynYu, daniil1423) reached 40+.
\nZaynYu's [40/50] GPT-OSS-120B TIR+DynamicTime+KernelPool (615 upvotes, 1,362 copies) has 57 comments and is the most-forked downstream child of my SC scaffold. A selection of the scores people reported, quoted literally from that comment thread:
| Forker | \nVariant | \nPublic LB | \n
|---|---|---|
| Tong Hui Kang | \nV1 (direct copy) | \n38 | \n
| Tong Hui Kang | \nV2 | \n35 | \n
| Tong Hui Kang | \nV10 | \n36 | \n
| Tong Hui Kang | \nV12 | \n36 | \n
| Siancy (\"Same as parthenos's, no modifications\") | \nV1 | \n41 | \n
| THUNDER THUNDER | \nV01 | \n40 | \n
| THUNDER THUNDER | \nV02 | \n36 | \n
| Abhijeet Bhargava | \nV01 / V03 / V05 | \n38 / 35 / 34 | \n
| chenjixuan666 (fork of Siancy) | \nV01 | \n38 | \n
| parthenos (on ZaynYu's V15) | \n— | \n37 | \n
| Dipankar Mitra | \nsubmitted twice | \n33 / 33 | \n
ZaynYu himself on his own notebook:
\n\n\n\"The score of the LB by running the same notebook is not guaranteed to be the same.\"
\n
So, empirically, the scaffold I contributed (A) sits downstream of parthenos's 16–26 band and supports a community reproduction band of roughly 34–41 on top of the same hardware.
\nI submitted the same V2 code 25 times to measure the noise floor:
\nA single run scoring 42 is well inside the tail; a single run scoring 35 is also inside the tail. This is why I frame the Python-weighted voting as \"pushed the ceiling up by ~2 points\", not \"solves more problems\". The voting change is not the dominant source of score; the model and the TIR loop are.
\nEverything in this table is a single-submission LB score from my own submission list. They are not independent trials — each number is a draw from a noisy distribution with σ ≈ 1–2 on top of a ~35–42 base. I report them with that caveat.
\n| Configuration | \nBest | \nMean | \nN | \nNotes | \n
|---|---|---|---|---|
| Weighted Voting w/ Python Verification (V2, contribution B) | \n42 | \n38.44 | \n25 | \nPeak and baseline for my work | \n
| ans_verifys (verifier variant) | \n41 | \n37.67 | \n6 | \nDifferent verification logic, similar spread | \n
| entropy-trajectory-scoring (daniil1423's logprob-entropy scoring) | \n39 | \n36.78 | \n9 | \nWithout my weighted voting, entropy scoring alone | \n
| Dual Entropy-Weighted Voting | \n37 | \n35.71 | \n7 | \nCombining two entropy signals | \n
| ZaynYu-style direct fork (36/40 variant) | \n34 | \n32.67 | \n3 | \nBefore my voting change | \n
| self-consistency-strategy V12 (contribution A, public) | \n32 | \n— | \n2 | \nBefore any voting change; scaffold only | \n
| parthenos direct fork (streaming + reflexion, no TIR) | \n26 | \n23 | \n2 | \nUpstream baseline | \n
| improved-tir-solution | \n25 | \n21 | \n3 | \nDifferent TIR formulation — didn't pay off | \n
The progression from parthenos → SC+TIR scaffold → ZaynYu's DynamicTime → weighted voting is visible across my own submissions, not just across the public fork tree.
\nsympy/numpy/mpmath call disappears. BlockingKernelClient + a per-thread port allocator avoids collisions that bite you as soon as you try to scale past K=2.ReasoningEffort.HIGH + min_p=0.02. The model's native training format outperforms any hand-crafted prompt I tried, including \"step by step\" and \"think carefully\" variants. This is also what techtech69 reports.vllm starts saves ~270 s of startup. Every second saved here is a second available to the hardest problems.\\boxed{…} scanning. Once a valid integer appears in the stream, close the connection. Don't wait for the model to finish.gpt-oss-120b. 16 and 24 concurrent attempts are preempted by vLLM and score catastrophically worse. techtech69 reports −22 at K=24; I observed the same pattern at smaller scale.base_problem_timeout with higher K. 100 s is not enough tokens to finish a hard problem. You lose more to truncated outputs than you gain from additional samples.+0.5 / +0.25 / −0.3 with a 0.5 error-rate threshold is just the first thing I tried. I don't know whether +0.3 / +0.15 / −0.2 would be better, worse, or identical within the noise.queue.Queue, the refined system prompts — the weighted-vote rule has nothing to rest on. The right way to read this writeup is \"I added two small bricks to a wall that other people built most of\", not \"I solved AIMO3 my way\".(problem_id, attempt_i, answer, python_calls, python_errors, token_len) tuple to a file that's in the submission output, so I can do real ablation offline.gpt-oss-120b caps out at ~43–44 on this hardware because its per-attempt accuracy on hard problems is ~5%. Both techtech69's and the host's pass@n data strongly suggest the gain from switching models is larger than the gain from any more voting tricks.Notebooks:
\ncanqiang/self-consistency-strategy (V12, 170 upvotes).daniil1423/entropy-trajectory-scoring with the _select_answer method replaced by the 20-line rule in §4.3. Currently private; I'll open-source it shortly.Key configuration for the 42-score submission:
\nclass Config:\n model_path = '/kaggle/input/gpt-oss-120b/transformers/default/1'\n served_model_name = 'gpt-oss'\n context_tokens = 65536\n kv_cache_dtype = 'fp8_e4m3'\n gpu_memory_utilization = 0.96\n batch_size = 256\n\n temperature = 1.0\n top_p = 1.0\n min_p = 0.02\n seed = 42\n attempts = 8\n workers = 16\n early_stop = 5\n turns = 128\n\n high_problem_timeout = 900\n base_problem_timeout = 300\n notebook_limit = 17400\n sandbox_timeout = 3\n jupyter_timeout = 10\nHardware: 1×H100 85GB, Kaggle competition runtime, 5h wall clock.
\nExpected variance: μ ≈ 38, σ ≈ 1.6 across identical-code submissions. A single run scoring in [35, 42] should be considered a successful reproduction.
Most of what is in this notebook is not mine. I want to name the people whose work I built on, in the order I inherited it:
\naimo3-streaming-inference-reflexion-prompting — the streaming-inference reflexion prompt scaffold that my SC+TIR notebook merges with shelterw's line.[Way to 30+/50] GPT-OSS - w/ python tool, the other upstream I merged into the SC notebook.If you are one of these people and I have miscredited any part of your work, please tell me and I will correct this writeup.
\nWriteups / notebooks directly referenced above:
\nExternal methodology references:
\n