![\"Strategy](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13116038%2F06f46268a18bb97ef7be6bc5a5fc015d%2Faimo3_strategy_figure1_v3.svg?generation=1776812581458776&alt=media\")
Competition: AIMO Progress Prize 3
\nSubmission basis: Private leaderboard (private leaderboard score 41.5)
\nKaggle notebook:
\nFor final submission (verbose log output not reduced),
\n\nAdded Links:
\nGoogle Colab Notebook for final submission code
\nmodel in HuggingFace, which is MXFP4 of GPT-OSS120B
\nSource of Figure 2: Generated text
\nGoogle Colab Notebook to create Figure 3,4,5 and A1
\nKaggle Notebook to create Figure 6
\nSource of Appendix B: Generated text
\nRewritten on May 17, 2026 after rebuttal phase:
\nbb_read() (the pull system) was changed to express twin-failure-mode framing explicitly.Thanks to Simon Frieder, Ryan Holbrook, and the Kaggle staff for organising; to XTX Markets for hosting; to Fields Model Initiative and Tinker for computational resources; and to many competitors for discussions.
\nI learned a great deal from:
\nWe present an inference-time coordination system for mathematical olympiad problem-solving, built on top of OpenAI GPT-OSS 120B, served without fine-tuning. The core contribution is a shared-memory mechanism (\"Blackboard\") that propagates intermediate computational discoveries between parallel inference threads. We show empirically that what is shared matters substantially: sharing reasoning text—which often includes predicted answers—leads to blind copying of answers, whereas sharing only recent tool inputs and outputs produces better outcomes. We further introduce a per-thread forking mechanism for answer verification, in which each thread that produces a candidate answer spawns an independent copy; both branches continue reasoning to confirm or revise. Analysis of over 3,000 answer-producing thread instances across 328 problems shows that double-checking recovers incorrect answers approximately four times more often than it corrupts correct ones. The system achieved the highest public leaderboard score of 46 at its peak, with an average of approximately 40–42 and 41 for the final submission.
\nAutomated mathematical reasoning at the olympiad level remains one of the most challenging benchmarks for large language models. The AIMO Progress Prize competition series benchmarks progress on olympiad-level problems, where solutions often require chained deductions, creative problem decompositions, and exact integer answers.
\nThe central question we address is: given an open-source reasoning model capable of solving most of the problems and a fixed time budget, how should multiple parallel inference attempts coordinate to maximise the probability of producing a correct answer? We explore this through three mechanisms: (1) a shared-memory Blackboard for propagating computational discoveries between threads, (2) a per-thread forking and double-checking protocol for answer verification, and (3) adaptive resource management for sandboxed Python tool use. The combination achieves a notable improvement.
\nA secondary challenge throughout this work was evaluation under high score variance: with only 50 problems, which makes it difficult to attribute observed changes to specific interventions. We discuss this limitation explicitly and qualify all comparisons accordingly.
\nFigure 1 illustrates the full inference pipeline for a single problem.
\n
Figure 1. Inference pipeline for final submission. Threads share a Blackboard; each forks independently on first answer for double-checking. Individual and global stop conditions govern termination.
\nModel: OpenAI GPT-OSS 120B (MXFP4 quantized). Kaggle model card: danielhanchen/gpt-oss-120b. Original model on HuggingFace: openai/gpt-oss-120b.
\nServing: vLLM [version: 0.11.2], running on Kaggle's GPU environment (1× NVIDIA H100). Key vLLM parameters: kv_cache_dtype='fp8_e4m3', gpu_memory_utilization=0.96. No fine-tuning was applied.
Sampling: Temperature 1.0, min_p 0.02, with streaming.
Prompt format: The model uses the openai_harmony chat template. A system prompt instructs the model to use Python for computation; some threads are permitted extended time limits.
\nGlobal system prompt:
\n\n\n\"You are a world-class International Mathematical Olympiad (IMO) competitor. The final answer must be a non-negative integer between 0 and 99999. You must place the final integer answer inside \\boxed{}.\"
\n
Tool usage instruction:
\n\n\n\"Use this tool to execute Python code. The environment is a stateful Jupyter notebook. You must use print() to output results. Default time limit is 6s. You have access to
\nmath,numpyandsympyto solve the problem.\"
\n\n\"Use this tool to execute Python code. The environment is a stateful Jupyter notebook. You must use print() to output results. Default time limit is 6s. To change the time limit, insert the desired number after each tool call
\nanalysis to=python code, for example for 20s,analysis to=python code20, then your code. Set this in every tool call if you want, since it is automatically reset after each cell execution. Time limit should not exceed 60s. Do not set it too high because time is limited. You have access tomath,numpy,sympy,pulp,ortools,pysat,mipandz3to solve the problem. Suppress log output. When you use pulp, set msg=False. When you use cp_model.CpSolver() from ortools.sat.python, set cp_model.CpSolver().parameters.log_to_stdout = False. When you use m=mip.Model(), set m.verbose=0. Manage total run time not to exceed the tool environment time limit.\"
Half of the threads use the shorter prompt version and the other half use the longer version. Tool execution is implemented with adaptive time limit control (see notebook for details). This adaptive time limit was motivated by the observation that, for some problems, tool calls frequently time out within 6 seconds. However, always allowing longer limits results in unnecessary delays on irrelevant tool executions.
\nEach problem is allocated to 4 parallel inference threads. Threads are implemented with concurrent.futures.ThreadPoolExecutor sharing a single vLLM engine.
Stop conditions operate at two levels:
\nAt termination, the final answer is selected by majority vote across all threads' last reported answers. On a tie, the answer from the thread with lower prediction entropy (computed from the token log-probabilities of the answer-producing generation step) is preferred.
\nEach thread's Python sandbox is an isolated subprocess. The sub-sandbox (reserved for forking) runs only import statements and function definitions to avoid heavy CPU load. When a thread forks, the fork inherits the parent's variable state via globals() serialisation and takes the reserved sandbox.
Sandboxes are health-checked on every tool call: a probe is sent to the subprocess, and if no response arrives within a timeout, the sandbox is replaced from the queue. All sandboxes are destroyed and re-initialised between problems to prevent state leakage and accumulation of crashed processes.
\nPre-installed Python libraries available to the model: pulp, ortools, pysat-solver, z3-solver, plus the standard scientific Python stack (numpy, sympy, scipy). The GPT-OSS model naturally attempts to use constraint solvers on combinatorial problems; pre-installation avoids failed import attempts.
Tool execution time is adaptively capped between 6 and 60 seconds depending on an explicit time directive in the prompt (see \"Longer version\" tool instruction in Section 2.1) or extraction of time_limit=** from the code script. The lower bound prevents trivially fast tools from hogging the response loop; the upper bound prevents constraint-solver runs from consuming the entire problem time budget. When a tool call exceeds the cap, it is interrupted and a timeout message is returned to the model with intermediate outputs.
Total time per problem varies between 300 and 950 seconds, allocated dynamically based on the remaining total budget. The first 8 problems each receive at least 800 seconds to handle the possibility of difficulty front-loading in the test set. When the per-problem deadline is within approximately 60 seconds, a prompt is injected instructing the model to commit to its best current answer. Without this, since GPT-OSS is well trained to verify answers thoroughly before committing, threads tend to continue indefinitely. The system targets completion well under the 5-hour scoring limit; in practice, wall-clock times varied between approximately 4h 15m and 5h 10m across runs.
\nMotivation. When a problem is difficult, the probability that any single thread finds the correct reasoning path is low. However, if one thread discovers a useful intermediate result — a correct bound, a working code approach, a key insight — sharing it with other threads can efficiently guide those threads without requiring them to rediscover the same result independently. The Blackboard system is designed to enable this sharing.
\nImplementation. The Blackboard is a shared sqlite3 database, chosen because it is thread-safe and allows concurrent reads and writes without blocking. All threads read from and write to the same database file on disk.
Two sharing modes were explored: push (automatic injection) and pull (model-driven retrieval).
\nAfter each tool call completes, a portion of another thread's recent tool input and output — the last 500 characters in the final submission (ranging from 400–800 characters across earlier notebook versions — is automatically inserted into the current thread's context immediately following the tool response. The model sees this as a system message formatted as:
\n### Concurrent discoveries from other agents:\n\nAttempt{k} discoveries:\n{recent tool code and output from thread k}\nFigure 2 shows a successful example: one thread's z3-solver output (finding that m=243 is the threshold) is injected into a peer thread, which immediately redirects its reasoning toward targeted verification rather than independent search.
\nInjected:
\n<|start|>system\n<|message|>### Concurrent discoveries from other agents:\n\nAttempt3 discoveries:\nvars = {n: Bool(f\"x{n}\") for n in range(3, m+1)}\ns = Solver()\nfor a in range(3, m+1):\n for b in range(3, m+1):\n c = a*b\n if c > m: continue\n s.add(Not(And(vars[a], vars[b], vars[c])))\n s.add(Not(And(Not(vars[a]), Not(vars[b]), Not(vars[c]))))\nreturn s.check() == sat\n\nfor m in range(240, 247):\n print(m, is_colorable(m))\n\n240 True 241 True 242 True 243 False 244 False 245 False 246 False\n(Elapsed 1.7s / Limit 6s)\n\n<|end|>\nResponse:
\n<|start|>assistant\n<|message|>Great! The other concurrent agent found threshold around 243.\nSo minimal m where unsatisfiable appears is 243.\nLet's verify ourselves using CP-SAT for m=242 and 243.\nFigure 2. Successful push: a peer thread's z3-solver result is inserted, leading the receiving thread to redirect from independent search to verification. Correct answer: 243 (problem pacore-00181 from the CrystalMath dataset).
\nBased on manual observation, when injected information is not relevant to the current thread's reasoning path, it is typically ignored. The model occasionally shows brief distraction (\"let me note what the other agent found…\") before continuing its own approach.
\nThe sharing topology for the final submission follows a ring structure: each thread receives pushes from exactly 2 peers, offset by +1 and +2 modulo 4, with a slight difference for forked threads. Concretely:
\n| Thread | \nReceives from | \n
|---|---|
| 0 | \n1, 2 | \n
| 1 | \n2, 3 | \n
| 2 | \n3, 0 | \n
| 3 | \n0, 1 | \n
| 4 (fork from 0) | \n1, 5 | \n
| 5 (fork from 1) | \n2, 6 | \n
| 6 (fork from 2) | \n3, 7 | \n
| 7 (fork from 3) | \n0, 4 | \n
After many experiments varying which threads and how many to receive from — including random selection and receiving from only one thread — no clearly superior topology was found. Receiving from many threads offers the advantage of diverse information, but also risks interrupting a thread's own reasoning. Receiving from one or two peers appeared to perform better, though further research is needed. The ring topology ensures no thread is isolated while preventing any single thread's output from reaching all others simultaneously.
\nSharing reasoning text — which often includes predicted answers — caused threads to simply copy the first answer produced, regardless of correctness. This is further discussed in 3.1.2.1 \"Why tool I/O, not reasoning text\".
\n| Shared content | \nObserved behaviour | \n
|---|---|
| Reasoning text (often including predicted answer) | \nBlind copying of answer; errors propagate | \n
| Recent tool I/O, ring topology | \nBetter results; errors less likely to cascade | \n
Several approaches share reasoning text between threads — either raw chain-of-thought, a model-generated summary, or the conversation history preceding a branch point. We chose to share only tool I/O events. This subsection states the empirical evidence and the mechanistic reasons.
\nEmpirical evidence. In an experiment, we shared each tool I/O event together with the few hundred characters of reasoning text immediately preceding it, hypothesizing that the context would help recipients interpret the result. Public score dropped from an average around 41 (I/O-only) to 37 (I/O + preceding context) — lower than the lowest of several I/O-only trials, albeit on a single run. We take this as an indication that the text component was actively harmful rather than merely redundant.
\nMechanism 1: non-directionality of code. Shared reasoning text routinely contains assertive closers — \"so the answer is X\", \"therefore…\", \"this confirms…\" — and the model's natural continuation tendencies favour agreement with such assertions when they appear as quasi-demonstrations in context. Code carries no such closers: a snippet that returns 242 reports a value, not a verdict. In our traces, recipients of shared tool I/O reliably verify before adopting an implication of the result (see the very last line of Fig. 2, where the recipient model responds to the I/O insertion with \"Let's verify ourselves using CP-SAT for m=242 and 243\"); recipients of shared text typically continue from the text's conclusion without re-verification. This is, we believe, the reason text sharing produces a wrong-answer collapse and tool-I/O sharing does not.
Mechanism 2: information density and context economy. A short Python snippet plus its output can encode an algorithmic attempt, a verification, or a pattern probe in a few hundred characters; the natural-language reasoning needed to convey the same intent is typically several times longer. Long insertions also distract the recipient from its own reasoning thread — even if the recipient ultimately judges the insertion irrelevant, recovering its prior context is non-trivial. Short code minimises both costs, and gpt-oss-120b reads the intent of short code fluently.
Mechanism 3: built-in salience. Every tool I/O event marks a discrete reasoning step the producing thread judged worth executing — a calculation, a verification, a hypothesis test. The set of \"events worth broadcasting\" is therefore self-defining, requiring neither a learned filter nor an auxiliary summarization pass (the latter would also be expensive under our inference budget).
\nTaken together, tool I/O changes both the signal shared between threads (no embedded verdict) and the behaviour it induces in recipients (verification rather than continuation). Reasoning-text sharing modifies neither, which leads to wrong-answer collapse.
\nAn alternative mode allowed threads to actively retrieve peer discoveries on demand by calling bb_read() (\"Blackboard_read\") from within the Python sandbox:
# Injected into each sandbox's initialisation\nimport sqlite3, json\n\ndef bb_read(qid, key):\n conn = sqlite3.connect(\"aimo_blackboard.db\")\n r = conn.execute(\n \"SELECT value FROM bb WHERE qid=? AND key=?\",\n (qid, key)\n ).fetchone()\n conn.close()\n return json.loads(r[0]) if r else None\nWhen bb_read() was mentioned once in the initial system prompt, it was called very rarely — once or twice across all threads over dozens of problems. To encourage more frequent use, a call snippet was inserted alongside every push message. This produced the opposite failure: threads repeatedly called bb_read() for all peer threads in sequence, spending most of their token budget reading others' conversations rather than reasoning (Appendix B). The pull system was therefore discarded entirely.
Motivation. During early trials on a difficult reference problem (target answer: 8687), we observed that even after a thread produced an incorrect answer, prompting it with \"The answer you provided was wrong. Try again.\" sometimes led to the correct answer. From this observation, several prompt variants were tested.
\nMechanism. When a thread first produces an answer, it forks independently: the original thread and a copy both continue reasoning, each treating the candidate answer as a hypothesis to confirm or revise. Each follows its own chain of thought, runs its own tool calls, and stops under the same individual stop condition (3 consecutive identical answers). Other threads that have not yet produced an answer continue unaffected.
\nThe double-check prompt is selected from the following options:
\nMain prompts:
\nOptional sub-prompts (appended to the mild prompt):
\nEmpirical analysis. We analysed 3,497 answer-producing thread instances across 328 problems. Figure 3 shows the breakdown of outcome categories.
\n![\"Figure](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13116038%2Fc3c567b5c7683db1197d1ba2479c5fb4%2Foutputframe-2.png?generation=1776849656505736&alt=media\")
Figure 3. Distribution of outcome categories: Correct_single (no fork), Wrong_single (no fork), Correct→Correct, Wrong→Wrong, Wrong→Correct (WC), Correct→Wrong (CW), and rare multi-change sequences.
\nCorrect-single and Wrong-single instances were not double-checked due to the time limit or a global majority stop. The key finding is that wrong-to-correct (WC) transitions occur approximately 4× more often than correct-to-wrong (CW) transitions, suggesting that well-reasoned correct answers are more robust to verification pressure than poorly-reasoned wrong answers. Multi-change sequences (C→W→C or W→C→W) were rare enough to be negligible.
\nWC transitions occur in approximately 4% of threads on a moderately difficult problem set. Extrapolated to 50 test problems with up to 8 threads each (400 thread instances total), this is a small absolute count, but on hard problems — where perhaps 10 or more problems in the test set may be at the difficulty level where WC transitions occur at higher rates — one or two problems out of 50 may be rescued per run. The prediction is rough, and one or two problems is within the base score variance, but the direction is positive.
\nPrompt comparison. Figure 4 compares the two main prompts. The aggressive prompt achieves a higher WC rate (13.9% vs 4.9%) but also a higher CW rate (4.0% vs 1.8%). Figure 5 compares sub-prompt variants within the mild-prompt family; the counterexample sub-prompt improves the WC rate without a proportionate increase in the CW rate.
\n![\"Figure](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13116038%2F53cfbac13e9ea036772e6e60c12b0acd%2Fretryprompteffect-2.png?generation=1776849882320669&alt=media\")
Figure 4. Grouped bar chart comparing WC% and CW% for aggressive vs. mild prompt. N labelled per condition.
\n![\"Figure](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13116038%2F3707bf695425871b896cd6c51582d0eb%2Fretryprompteffect2-2.png?generation=1776850274438829&alt=media\")
Figure 5. Grouped bar chart comparing WC% and CW% for mild, mild+counterexample, and mild+structure-check. N labelled per condition.
\nThe prompt for each double-check call in the final submission is selected from aggressive, mild, or mild+counterexample based on a simple heuristic rule.
\nThis analysis was conducted on data from recent runs, not a designed experiment: problem distributions and prompt assignments were not balanced across conditions. A problem-weighted normalisation (each problem's CC, CW, WC and WW rates are computed per-problem and weighted by frequency) leaves the results qualitatively unchanged; see Appendix A.
\nFor combinatorial and discrete optimisation problems, GPT-OSS 120B naturally attempts to use constraint-solving libraries. Pre-installing pulp, ortools, pysat-solver, and z3-solver avoids import-time failures and latency. Threads are selectively permitted to use these libraries via their system prompts. Because these libraries can run for many seconds or cause sandbox instability, tool calls were given flexible time limits and sandbox health was monitored continuously.
\n\nNote on variance: With only 50 public test problems, score differences of 1–3 points are within noise. All comparisons below are directional, not statistically definitive.
\n
The best observed public score from this work was 46, which was the highest on the public leaderboard at the time. The final submitted notebook, which incorporated the fixes described in Section 4.3, scored 41 on the public leaderboard with an average of approximately 40–42 across repeated runs of similar strategies.
\nFigure 6 shows our public leaderboard score history across all submissions, grouped by identical notebook versions.
\n![\"Figure](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13116038%2Fc7247667ad629bfdc5b3b296272c7a7c%2Fpublicscoreprogress1-2.png?generation=1776850788432132&alt=media\")
Figure 6. Public score history. Identical versions of the same notebook are grouped with the same markers (e.g., Notebook #73 was submitted five times and received public scores [45, 46, 39, 39, 42], shown with green circles). The notable jump corresponds to the introduction of the Blackboard push system, which was preliminarily tested in Notebook #73 and fully applied in Notebook #76 (alongside the simultaneous RLM-like pull system).
\nA clear score increase was observed when the Blackboard push and pull systems were introduced simultaneously, making individual attribution difficult. Since the pull system was later discarded without a clear drop in average score, the push system is believed to be the main contributor — though this could not be confirmed statistically.
\nTwo thread-count variants were compared:
\n| Variant | \nDescription | \nAverage score | \nPeak score | \nNotebook # in Figure 6 | \n
|---|---|---|---|---|
| Final (4-thread + fork) | \n4 threads, fork on first answer | \n~40–42 | \n46 | \n#76, #93, #105, final submission, and others | \n
| Alternative (6-thread, no fork) | \n6 threads, no fork, early stop at 3 identical | \n~41–42 | \n43 | \n#78, #101, and others | \n
The alternative showed a slightly higher average score but a lower peak, and was not selected.
\nThe following issues were identified in the notebook that achieved the best public score and were partially or fully fixed before the final submission:
\nNameError failures on tool calls. Variable inheritance via globals() serialisation was added.bb_read() (the pull system) was called very rarely without explicit per-step prompting, and called excessively when such prompting was added — the model began querying siblings in place of its own reasoning. We could not find a useful middle ground, and it was discarded entirely.ortools, etc.) sometimes became unresponsive with no error returned; health-check replacement was made more aggressive.We presented an inference-time coordination system for mathematical olympiad reasoning that combines a shared Blackboard for propagating intermediate computational discoveries, a per-thread forking protocol for answer verification, and adaptive resource management for Python tool use. The most impactful contribution is believed to be the Blackboard push system — automatically inserting a portion of another thread's recent tool activity into the current thread's context in a ring topology — combined with the per-thread forking system for answer double-checking. Analysis of over 3,000 answer-change events shows that double-checking recovers incorrect answers approximately 4× more often than it corrupts correct ones, providing quantitative support for the verification approach. Adaptive sandbox management and constraint-solver support contributed to robustness.
\nTime management is a central constraint: double-checking and heavy solver libraries both consume significant time, and starting with 8 threads was not affordable. Starting with 4 threads (forking to at most 8) was the selected balance.
\nImproving thread coordination, reducing evaluation variance through larger reference problem sets, and more robust sandbox management under heavy solver load remain open directions.
\nRecursive Language Models. Zhang et al.\nhttps://arxiv.org/html/2512.24601v1
\nThe double-check prompt comparison in Section 3.2 uses unbalanced observational data (different problems appear at different rates across prompt conditions). To control for problem difficulty, per-problem WC and CW rates are computed for each problem and weighted by frequency. The weighted results are qualitatively consistent with the unweighted analysis: the rank ordering of prompts by CC (Correct→Correct), CW (Correct→Wrong), WC (Wrong→Correct), and WW (Wrong→Wrong) rate is unchanged, and approximate magnitudes are similar.
\n![\"Figure](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13116038%2F91d682d8bd0e913ff0948322fa563116%2Fmacroafveprompteffect-2.png?generation=1776850381361832&alt=media\")
Figure A1. Weighted CC%, CW%, WC%, and WW% per prompt type. Data count and problem count for each prompt are shown as n=*|**problems. Prompt sets not shown in Figure 5 due to low data count are included here.
\nAs described in Section 3.1.3, encouraging frequent use of bb_read() by inserting a call snippet alongside every push message produced an adverse effect: threads repeatedly called bb_read() for all peer threads in sequence, spending most of their token budget reading others' conversations rather than reasoning independently. An example is shown below.
When Attempt 1 (Thread 1) discoveries were shared using bb_read(\"attempt1\"), extracting the \"content\" and \"text\" fields as shown below — where the block beginning with \"from functools …\" represents real Attempt 1 tool I/O —
<|start|>system\n<|message|>###Concurrent discoveries from other attempts:\n\nAttempt1 discoveries by running\nmsgs = bb_read(\"attempt1\")\nfor m in msgs[-2:]:\n t = m.get(\"content\")\n if t:\n print(t[0].get(\"text\",\"\")[-800:])\n:\nfrom functools import lru_cache\n\n# board 5x5, index cells 0..24\nsize = 5\ncell_coords = [(i, j) for i in range(size) for j in range(size)]\n\n# compute subgrids (top-left corners)\n\n..... [Attempt1 tool input and output shared].....\n\n(Elapsed0.0s/Limit6s)\n<|end|>\nThe model then attempted to retrieve more from Attempt 1 via bb_read, first checking the message count:
<|start|>assistant\n<|channel|>analysis\n<|message|>Search more.\n<|end|>\n<|start|>assistant to=python\n<|channel|>analysis code\n<|message|>msgs = bb_read('attempt1')\nlen(msgs)\n\n<|call|>\n<|start|>python to=assistant\n<|channel|>analysis\n<|message|>5\n(Elapsed0.0s/Limit6s)\n<|end|>\nThe model then read a message and received output:
\n<|start|>assistant\n<|channel|>analysis\n<|message|>Let's view the later messages.\n\n<|end|>\n<|start|>assistant to=python\n<|channel|>analysis code 0-5 -0.5s \n<|message|>for i,m in enumerate(msgs):\n if m.get('role')!='assistant':\n continue\n print(i, m.get('content')[:500])\n\n<|call|>\n<|start|>python to=assistant\n<|channel|>analysis\n<|message|>2 [{'type': 'text', 'text': 'We have a combinatorial game: two players .........\n(Elapsed0.0s/Limit6s)\n<|end|>\nThe model continued reading further messages from Attempt 1:
\n<|start|>assistant\n<|channel|>analysis\n<|message|>Let's see further.\n\n<|end|>\n\n......continuing reading attempt1's message......\nThe model determined that Attempt 1 had not yet found a solution and proceeded to read from Attempt 2, then Attempt 3, then back to Attempt 1:
\n<|start|>assistant\n<|channel|>analysis\n<|message|>So attempt1 hasn't solved.\n\nNow attempt2, attempt3? Let's list all attempts.\n<|end|>\n<|start|>assistant to=python\n<|channel|>analysis code 0-8 -0.4s \n<|message|>msgs = bb_read('attempt2')\nlen(msgs)\n\n<|call|>\n<|start|>python to=assistant\n<|channel|>analysis\n<|message|>29\n(Elapsed0.0s/Limit6s)\n<|end|>\n<|start|>assistant\n<|channel|>analysis\n<|message|>Let's view attempt2 maybe later messages include final answer.\n\n<|end|>\n<|start|>assistant to=python\n<|channel|>analysis code 0-9 -0.2s \n<|message|>for i,m in enumerate(msgs):\n if m.get('role')=='assistant':\n print(i, m.get('content')[:200])\n\n<|call|>\n<|start|>python to=assistant\n<|channel|>analysis\n<|message|>2 [{'type': 'text', 'text': 'We have a set S size 35. Let F be a ...........\n(Elapsed0.0s/Limit6s)\n<|end|>\n<|start|>assistant\n<|channel|>analysis\n<|message|>So attempt2 not relevant.\n\nAttempt3 maybe.\n\n<|end|>\n<|start|>assistant to=python\n<|channel|>analysis code 0-10 -0.4s \n<|message|>msgs = bb_read('attempt3')\nlen(msgs)\n\n<|call|>\n\n.........continuing to get Attempt3 and then Attempt1 message again......\nThe model then attempted to read from Attempt 5. In this way, Thread 4 spent most of its token budget gathering information from other threads rather than reasoning independently. When this behaviour was observed repeatedly in the Blackboard pull system with encouraged bb_read() use, further trials with this approach were abandoned.
<|start|>assistant\n<|channel|>analysis\n<|message|>Let's see further after index 9? Already we looked.\n\nNow attempt1 has some analysis but not final answer.\n\nWe also have attempt5? Let's see.\n\n............continuing read Attempt5 message\nI tried sharing reasoning traces or keeping a \"summary bank\" with lemmas and facts. Everything failed - either too slow, or models starts collapsing onto \"loudest wrong ideas\".
\nYour code sharing idea is so neat! And it's almost for free (given how fast prefill is with Flash Attention 3), compared to more expensive prefix sharing or summarization. You solved a problem that I wasn't able to solve!
\nHave the same question as @flamingice2801 below though - ring topology may only slow down the propagation of wrong ideas, not stopping them completely. Have you investigated the diversity?
", "votes": 1, "replies": [ { "id": 3448821, "author_name": "ippeiogawa", "author_url": "", "post_date": "2026-04-26T17:16:49.273000", "content": "@chankhavu \nThank you for the comment. \nI was fortunate that code sharing was the first approach I attempted — it was\nsimple to implement, and everything I tried afterward produced worse results.
\nOn diversity: if you mean topological diversity beyond the ring — yes, but not much.\nAll-to-all sharing diminished efficiency.\nRough statistics suggested that receiving from 2–3 threads performed better, but the variance across runs made it difficult to\ndraw firm conclusions. \nRing + random and pure random both showed similar average performance\n to the plain ring.
\nAbout propagation of wrong ideas, it sometimes happens but very rare. I observed in most cases, injected content is not blindly copied but is actively verified by the receiving thread.
", "votes": 1, "replies": [] } ] }, { "id": 3445577, "author_name": "Utkarsh Sharma", "author_url": "", "post_date": "2026-04-20T05:56:47.427000", "content": "@ippeiogawa The Blackboard design - sharing tool I/O but not answers, in a unidirectional ring - is a genuinely new idea, and the ablation table showing that answer-sharing causes blind copying while bidirectional sharing causes false consensus is exactly the kind of design insight that helps the whole community.
\nThe 46 is remarkable. Two questions:
\nBlackboard failure modes on systematic errors. The ring topology prevents circular reinforcement, but a wrong intermediate computation can still propagate downstream through two hops (T1→T0 via direct push, and T1→T2→T3 via relay). On problems where the model systematically computes a wrong intermediate value (e.g., miscounts configurations, gets a wrong base case), did you observe cases where the Blackboard hurt by propagating that wrong computation to threads that might otherwise have computed it correctly independently? In other words - what's the Blackboard's error rate: how often does it flip a correct thread to wrong vs. flip a wrong thread to correct?
The forking verification vs. just running 8 independent threads. Your final system runs 4 threads that fork to 8. An alternative is 8 fully independent threads with majority vote (the standard approach). The fork gives you adversarial \nself-checking (\"the answer was wrong, try again\"), but at the cost of halving the initial diversity (4 independent starts \ninstead of 8). Did you compare 4-fork-to-8 against 8-independent on the same problem set? Your ablation mentions 6 threads-no-fork as an alternative - was 8-threads-no-fork also tested?
The constraint solver pre-installation (z3, ortools, pulp) is an addition I haven't seen in any other writeup. On which problem types did the model actually use these, and did you observe a measurable accuracy difference on those problems vs. sympy-only?
\nAlso, some of the figures are not being rendered, perhaps a bad link to the image?
", "votes": 1, "replies": [ { "id": 3445649, "author_name": "", "author_url": "", "post_date": "2026-04-20T09:19:22.733000", "content": "", "votes": 0, "replies": [] }, { "id": 3448814, "author_name": "ippeiogawa", "author_url": "", "post_date": "2026-04-26T16:43:56.173000", "content": "\nThank you for the kind words and the thoughtful questions.
\nAbout the questions,
\nI had not initially collected systematic statistics on this, since casual\nobservation suggested that error propagation was rare relative to successful\nsharing. To provide a more rigorous answer, I gathered data from recent runs\ncovering over 9,000 thread-generated samples. Note that these samples vary\nin topology (ring vs. random) and injection frequency, so the numbers below\nshould be treated as indicative rather than definitive.
\nResponse rate. Not every injected message influences the receiving thread.\nUsing the heuristic that a response contains \"Attempt\" or \"agent\" within the\nfirst 500 characters after an injection, such as \"According to discoveries from other agents, …\", approximately 25% of injections\nproduced a visible response (15,656 / 63,436). Three out of four injections\nappear to be ignored. Whether 25% is the optimal response rate is unclear — a higher rate would bring more peer information into each thread's reasoning, but at the cost of greater distraction from the thread's own chain of thought. Finding the right balance, or a mechanism that incorporates peer information without interrupting ongoing reasoning, remains an open direction.
\nPer-thread analysis. To measure whether the Blackboard helped or hurt, I\nfiltered to threads where the true answer was ≥ 101 (to avoid false positives\nfrom small numbers appearing incidentally in reasoning text), leaving\n5,165 thread samples. I then checked whether the true answer had already\nappeared in the thread's text before each injection — threads where it had\nare labelled \"already found\"; those where it had not are labelled \"not yet\nfound.\"
\nWithin these 5,165 samples:
\n| Outcome | \nCount | \nRate | \n
|---|---|---|
| Injection introduced correct answer (not previously mentioned) and correct answer was ultimately submitted | \n66 | \n1.3% | \n
| Injection contained a wrong answer which was ultimately submitted | \n116 | \n2.2% | \n
| Of the wrong-answer cases above: thread had already found the correct answer before injection | \n61 | \n1.2% | \n
The raw success rate (1.3%) and error rate (2.2%) are similar in magnitude,\nwhich might suggest the Blackboard is slightly harmful on net. However, this\nanalysis undercounts the Blackboard's contribution in at least two ways:
\nI acknowledge that the heuristics used here are imprecise, and a cleaner\nmeasurement would require controlled experiments. This remains an open\nquestion.
\nThe primary reason for preferring 4 threads with forking over 8 independent\nthreads was throughput. Measured token generation speed with 4 threads\nis approximately 1.4× higher than with 8 threads, because all threads share\na single vLLM engine and longer KV caches reduce per-step throughput\nsubstantially. \nOn hard problems where each thread can exceed 60,000 tokens, 8 fully independent threads would slow the system significantly.
\nComparing 4-thread to 8-thread runs across submissions showed almost no consistent score difference,\nsuggesting that beyond 4 threads, additional starting diversity gives diminishing returns within the 5-hour time constraint.
\nI set the practical goal of completing all 50 problems within approximately 4h 30m on the public set.\nThe 4-thread-with-fork and 6-thread-no-fork configurations both met\nthis target on most runs; the 8-thread-no-fork configuration almost never\ndid. Given that private-set problems were stated to be of similar difficulty but could prove more time-consuming in practice, I\nprioritised time reliability over initial thread diversity.
\nI did not measure accuracy difference between constraint-solver-enabled and\nsympy-only runs, so a direct comparison is not available. What I observed is\nthat GPT-OSS 120B naturally reaches for these libraries on problems that\ninvolve finding the minimum or maximum integer satisfying a combinatorial\nconstraint. Representative examples from the CrystalMat dataset:
\n\"bad link to the image?\" => Fixed!!
", "votes": 2, "replies": [ { "id": 3448907, "author_name": "Utkarsh Sharma", "author_url": "", "post_date": "2026-04-27T02:02:06.357000", "content": "Thank you for the detailed data - the 9,000+ thread analysis is exactly what I was hoping for.
\nThe numbers are interesting: 1.3% introduction of correct answers vs 2.2% propagation of wrong answers looks net-negative on paper, but the 14.8% confidence reinforcement and the uncaptured indirect guidance (sharing a useful code approach rather than an answer) are likely where the real value lives. The fact that 75% of injections are simply ignored is actually reassuring - it means the model is selective about what it incorporates, which limits damage from wrong propagations.
\nThe throughput argument for 4-fork vs 8-independent is compelling and something I hadn't considered. At 60,000+ tokens per thread on hard problems, 8 concurrent threads would indeed bottleneck the vLLM engine badly. Our pipeline uses 8 threads but with early_stop=4, so in practice many threads are cancelled before reaching full context - which may explain why we don't see the same throughput degradation.
\nThe constraint solver examples (dapo-3046, beyondaime-00070) are useful - these are exactly the combinatorial optimization problems where sympy alone would struggle. Worth trying for future competitions.
", "votes": 1, "replies": [] } ] } ] }, { "id": 3459004, "author_name": "ippeiogawa", "author_url": "", "post_date": "2026-05-17T06:55:19.227000", "content": "Edited on [May 17, 2026] for the rebuttal phase:
\nFollowing up on the questions from @philipvonderlind, here are some additional questions to help us assess your submission:
\nLook forward to reading your responses on these.
", "votes": 0, "replies": [ { "id": 3458988, "author_name": "ippeiogawa", "author_url": "", "post_date": "2026-05-17T06:08:48.930000", "content": "@sambealing Thank you for the detailed questions — we appreciate the close reading.
\nA fair caveat first: with only five runs of Notebook #76 it is hard to draw rigorous conclusions about score variance. Our intuition, however, is that the blackboard system does not meaningfully increase or decrease run-to-run variance, and that what we are seeing is the background variance characteristic of any high-scoring solution on this leaderboard under the 5-hour budget. Three observations support this:
\nPublic notebooks show comparable or greater spread. The widely used 44/50 notebook has been reported scoring as low as 34 on the same evaluation (discussion thread \"How stable is the 44/50 public notebook for you?\") — a 10-point spread, larger than ours, albeit aggregated over many more runs.
Pre-blackboard variants of our own pipeline showed similar variance. §4.1 (Score History and Strategy Comparison) documents that even simpler baselines without the blackboard exhibited multi-point spreads on small sample sizes — e.g. Notebook #62 showed a 5-point spread across just two runs.
A plausible mechanistic source of variance is time-budget pressure, not the sharing mechanism. Notebook #76 runs close to the 5-hour cap. Recorded run times for our five #76 submissions were 5h5m, 5h25m, 5h25m, 4h57m, with the last unrecorded; on average the pipeline appears to run at or beyond the 5-hour limit (per-run timing reported on AIMO3 Leaderboard History at aimo.huikang.dev/leaderboard is itself noisy under competition rules and should be treated as approximate). The time management in notebook #76 has inefficiencies (§4.3) — most notably tool execution running for ~60s even after the final answer is reached. Since the remaining overall time is allocated equally to the remaining problems, we suspect the lower-scoring runs are runs in which some final hard problems — those solvable only with substantial compute — received insufficient remaining budget, while the higher-scoring runs benefited from a more favourable problem ordering that left more budget for the hardest items. If this is right, the variance is being driven by tail-end budget allocation rather than by stochasticity in the blackboard's sharing decisions, and a similar mechanism would affect any solution running close to the time cap.
We have not isolated the blackboard's contribution to variance with a controlled ablation (e.g. running the same pipeline with PUSH disabled across many seeds). So this remains a hypothesis rather than a measurement.
\nBoth of the mechanisms you propose appear to be operating, and we believe they reinforce each other. We see two distinct effects:
\nCode lacks the persuasive surface that prose has. Shared reasoning text routinely contains assertive closers — \"so the answer is X\", \"this confirms that…\", \"therefore…\" — and the model's natural continuation patterns favour agreement with such assertions, especially when they appear in what looks like an in-context demonstration. Code does not carry these closers. A snippet that returns 242 says only that 242 was returned; it does not say 242 is correct, and it does not pattern-match to \"follow the conclusion.\"
Code is explicitly verified by the recipient. In our traces, after a code-I/O event is shared, the recipient thread reliably verifies before adopting any implication of the result. Fig. 2 gives a concrete example: after consuming a sibling's I/O, the receiving thread emits \"Let's verify ourselves using CP-SAT for m=242 and 243.\" This verification step is, in our observation, often absent when reasoning text is shared — the recipient simply continues from the text's conclusion.
Put together: code sharing changes both the signal (no embedded verdict) and the behaviour (the recipient verifies). Text sharing leaves both intact and so reproduces the original thread's conclusion downstream, correct or not. We will fold this mechanistic account into the new §3.1.2.1 in the rewrite.
", "votes": 0, "replies": [] } ] }, { "id": 3456797, "author_name": "Philip Vonderlind", "author_url": "", "post_date": "2026-05-12T15:52:27.803000", "content": "Hi, thank you for your submission to the Writeup Prize. @sambealing and I will post some questions below that help us in assessing your submission more closely, as described in the updated discussion post.
\nYour solution is interesting and seems to outperform just sharing context between different completions/agents.\nI have a few questions about directions that could have been interesting as well:
\nPULL method from the blackboard, so gpt-oss-120b would better understand how to use this mechanic to manage its own context? From what you wrote, the main failure mode seems to be the model not using it.Looking forward to your response :)
", "votes": 0, "replies": [ { "id": 3458985, "author_name": "ippeiogawa", "author_url": "", "post_date": "2026-05-17T06:01:24.137000", "content": "@philipvonderlind Thank you for the careful read of the writeup, and for separating these two threads of questioning. We'll answer each in turn.
\nNo, we did not attempt fine-tuning for PULL. Before answering the broader question, one clarification: your reading — \"the main failure mode seems to be the model not using it\" — captures one half of what we actually observed; our statement on this in the writeup was not as clear as it should have been, and we will revise it (see the change at the end of this answer).
\nWith sufficiently frequent in-context exemplars (as illustrated in Appendix B), gpt-oss-120b can be coaxed into issuing PULL calls. The problem is what happens once it does. Under aggressive encouragement, the model shifts from solving the problem to querying its siblings — repeatedly pulling partial results and substituting inspection of others' work for its own reasoning. So the failure mode is two-sided: rare and unused at low encouragement, frequent and crowding out reasoning at high encouragement. We discarded PULL not because it could not fire, but because we could not find a setting in which it fired the right amount — and we could not determine what the right amount would be.
This makes the problem one of timing and selectivity, not capability. A useful PULL would fire only when the model is stuck or has detected an inconsistency in its own trace — a meta-cognitive signal. Teaching this via vanilla SFT seems hard for two reasons (the direction is not impossible and is worth pursuing, but constructing the necessary training setup was beyond what we could attempt within the remaining competition period). First, the supervision signal (\"when should I have pulled?\") is non-trivial to construct, since it requires labelled trajectories of stuck-vs-progressing reasoning aligned with the available state of sibling threads at that moment. Second, the policy depends on information the model cannot observe at decision time — namely, whether any sibling currently holds useful information.
\nThis is also part of the motivation for emphasizing PUSH in the design. PUSH inverts the asymmetry: the producing thread, which has just generated a tool I/O event, is the one best positioned to know that something informative just happened, and when it happened (immediately, as it was produced). PUSH therefore needs no learned meta-cognitive trigger — the event itself is the trigger.
\nWe will tighten the language in §4.3 (Issues Fixed or Changed Between Best Public Score and Final Submission) so the twin-failure-mode framing is explicit.
\nWriteup change (§4.3):
\nbb_read() (the pull system) was called very rarely without explicit per-step prompting; it was discarded entirely.\"bb_read() (the pull system) was called very rarely without explicit per-step prompting, and called excessively when such prompting was added — the model began querying siblings in place of its own reasoning. We could not find a useful middle ground, and it was discarded entirely.\"Several teams converged on the intuition that completions should not be fully independent, but the what and how of sharing differs substantially. We position our approach against the one you mentioned and against the broader family of text-based sharing.
\nvs. Adaptive Tree-of-Thoughts with EAGLE3 (the writeup you linked). The ToT approach inherits Jupyter state and prior conversation from a parent thread into a child thread, sequentially expanding a search tree of reasoning trajectories. Our approach differs along two axes.
\nvs. text-based sharing in general (e.g. the summarization-based approach in \"Attempts to get gpt-oss-120b to collaborate\", or simply injecting recent context from another thread). Two contrasts.
\nA more detailed mechanistic comparison — why tool I/O specifically resists the wrong-answer-collapse mode that text sharing produces — is given in our response to Q2 from @sambealing. We are also adding a new subsection §3.1.2.1 (Why tool I/O, not reasoning text) to make this contrast explicit in the writeup itself.
", "votes": 0, "replies": [] } ] } ], "comment_count": 11, "messages": [ { "id": 3445280, "postDate": "2026-04-19T12:52:47.983Z", "content": "Competition: AIMO Progress Prize 3
\nSubmission basis: Private leaderboard (private leaderboard score 41.5)
\nKaggle notebook:
\nFor final submission (verbose log output not reduced),
\n\nAdded Links:
\nGoogle Colab Notebook for final submission code
\nmodel in HuggingFace, which is MXFP4 of GPT-OSS120B
\nSource of Figure 2: Generated text
\nGoogle Colab Notebook to create Figure 3,4,5 and A1
\nKaggle Notebook to create Figure 6
\nSource of Appendix B: Generated text
\nRewritten on May 17, 2026 after rebuttal phase:
\nbb_read() (the pull system) was changed to express twin-failure-mode framing explicitly.Thanks to Simon Frieder, Ryan Holbrook, and the Kaggle staff for organising; to XTX Markets for hosting; to Fields Model Initiative and Tinker for computational resources; and to many competitors for discussions.
\nI learned a great deal from:
\nWe present an inference-time coordination system for mathematical olympiad problem-solving, built on top of OpenAI GPT-OSS 120B, served without fine-tuning. The core contribution is a shared-memory mechanism (\"Blackboard\") that propagates intermediate computational discoveries between parallel inference threads. We show empirically that what is shared matters substantially: sharing reasoning text—which often includes predicted answers—leads to blind copying of answers, whereas sharing only recent tool inputs and outputs produces better outcomes. We further introduce a per-thread forking mechanism for answer verification, in which each thread that produces a candidate answer spawns an independent copy; both branches continue reasoning to confirm or revise. Analysis of over 3,000 answer-producing thread instances across 328 problems shows that double-checking recovers incorrect answers approximately four times more often than it corrupts correct ones. The system achieved the highest public leaderboard score of 46 at its peak, with an average of approximately 40–42 and 41 for the final submission.
\nAutomated mathematical reasoning at the olympiad level remains one of the most challenging benchmarks for large language models. The AIMO Progress Prize competition series benchmarks progress on olympiad-level problems, where solutions often require chained deductions, creative problem decompositions, and exact integer answers.
\nThe central question we address is: given an open-source reasoning model capable of solving most of the problems and a fixed time budget, how should multiple parallel inference attempts coordinate to maximise the probability of producing a correct answer? We explore this through three mechanisms: (1) a shared-memory Blackboard for propagating computational discoveries between threads, (2) a per-thread forking and double-checking protocol for answer verification, and (3) adaptive resource management for sandboxed Python tool use. The combination achieves a notable improvement.
\nA secondary challenge throughout this work was evaluation under high score variance: with only 50 problems, which makes it difficult to attribute observed changes to specific interventions. We discuss this limitation explicitly and qualify all comparisons accordingly.
\nFigure 1 illustrates the full inference pipeline for a single problem.
\n
Figure 1. Inference pipeline for final submission. Threads share a Blackboard; each forks independently on first answer for double-checking. Individual and global stop conditions govern termination.
\nModel: OpenAI GPT-OSS 120B (MXFP4 quantized). Kaggle model card: danielhanchen/gpt-oss-120b. Original model on HuggingFace: openai/gpt-oss-120b.
\nServing: vLLM [version: 0.11.2], running on Kaggle's GPU environment (1× NVIDIA H100). Key vLLM parameters: kv_cache_dtype='fp8_e4m3', gpu_memory_utilization=0.96. No fine-tuning was applied.
Sampling: Temperature 1.0, min_p 0.02, with streaming.
Prompt format: The model uses the openai_harmony chat template. A system prompt instructs the model to use Python for computation; some threads are permitted extended time limits.
\nGlobal system prompt:
\n\n\n\"You are a world-class International Mathematical Olympiad (IMO) competitor. The final answer must be a non-negative integer between 0 and 99999. You must place the final integer answer inside \\boxed{}.\"
\n
Tool usage instruction:
\n\n\n\"Use this tool to execute Python code. The environment is a stateful Jupyter notebook. You must use print() to output results. Default time limit is 6s. You have access to
\nmath,numpyandsympyto solve the problem.\"
\n\n\"Use this tool to execute Python code. The environment is a stateful Jupyter notebook. You must use print() to output results. Default time limit is 6s. To change the time limit, insert the desired number after each tool call
\nanalysis to=python code, for example for 20s,analysis to=python code20, then your code. Set this in every tool call if you want, since it is automatically reset after each cell execution. Time limit should not exceed 60s. Do not set it too high because time is limited. You have access tomath,numpy,sympy,pulp,ortools,pysat,mipandz3to solve the problem. Suppress log output. When you use pulp, set msg=False. When you use cp_model.CpSolver() from ortools.sat.python, set cp_model.CpSolver().parameters.log_to_stdout = False. When you use m=mip.Model(), set m.verbose=0. Manage total run time not to exceed the tool environment time limit.\"
Half of the threads use the shorter prompt version and the other half use the longer version. Tool execution is implemented with adaptive time limit control (see notebook for details). This adaptive time limit was motivated by the observation that, for some problems, tool calls frequently time out within 6 seconds. However, always allowing longer limits results in unnecessary delays on irrelevant tool executions.
\nEach problem is allocated to 4 parallel inference threads. Threads are implemented with concurrent.futures.ThreadPoolExecutor sharing a single vLLM engine.
Stop conditions operate at two levels:
\nAt termination, the final answer is selected by majority vote across all threads' last reported answers. On a tie, the answer from the thread with lower prediction entropy (computed from the token log-probabilities of the answer-producing generation step) is preferred.
\nEach thread's Python sandbox is an isolated subprocess. The sub-sandbox (reserved for forking) runs only import statements and function definitions to avoid heavy CPU load. When a thread forks, the fork inherits the parent's variable state via globals() serialisation and takes the reserved sandbox.
Sandboxes are health-checked on every tool call: a probe is sent to the subprocess, and if no response arrives within a timeout, the sandbox is replaced from the queue. All sandboxes are destroyed and re-initialised between problems to prevent state leakage and accumulation of crashed processes.
\nPre-installed Python libraries available to the model: pulp, ortools, pysat-solver, z3-solver, plus the standard scientific Python stack (numpy, sympy, scipy). The GPT-OSS model naturally attempts to use constraint solvers on combinatorial problems; pre-installation avoids failed import attempts.
Tool execution time is adaptively capped between 6 and 60 seconds depending on an explicit time directive in the prompt (see \"Longer version\" tool instruction in Section 2.1) or extraction of time_limit=** from the code script. The lower bound prevents trivially fast tools from hogging the response loop; the upper bound prevents constraint-solver runs from consuming the entire problem time budget. When a tool call exceeds the cap, it is interrupted and a timeout message is returned to the model with intermediate outputs.
Total time per problem varies between 300 and 950 seconds, allocated dynamically based on the remaining total budget. The first 8 problems each receive at least 800 seconds to handle the possibility of difficulty front-loading in the test set. When the per-problem deadline is within approximately 60 seconds, a prompt is injected instructing the model to commit to its best current answer. Without this, since GPT-OSS is well trained to verify answers thoroughly before committing, threads tend to continue indefinitely. The system targets completion well under the 5-hour scoring limit; in practice, wall-clock times varied between approximately 4h 15m and 5h 10m across runs.
\nMotivation. When a problem is difficult, the probability that any single thread finds the correct reasoning path is low. However, if one thread discovers a useful intermediate result — a correct bound, a working code approach, a key insight — sharing it with other threads can efficiently guide those threads without requiring them to rediscover the same result independently. The Blackboard system is designed to enable this sharing.
\nImplementation. The Blackboard is a shared sqlite3 database, chosen because it is thread-safe and allows concurrent reads and writes without blocking. All threads read from and write to the same database file on disk.
Two sharing modes were explored: push (automatic injection) and pull (model-driven retrieval).
\nAfter each tool call completes, a portion of another thread's recent tool input and output — the last 500 characters in the final submission (ranging from 400–800 characters across earlier notebook versions — is automatically inserted into the current thread's context immediately following the tool response. The model sees this as a system message formatted as:
\n### Concurrent discoveries from other agents:\n\nAttempt{k} discoveries:\n{recent tool code and output from thread k}\nFigure 2 shows a successful example: one thread's z3-solver output (finding that m=243 is the threshold) is injected into a peer thread, which immediately redirects its reasoning toward targeted verification rather than independent search.
\nInjected:
\n<|start|>system\n<|message|>### Concurrent discoveries from other agents:\n\nAttempt3 discoveries:\nvars = {n: Bool(f\"x{n}\") for n in range(3, m+1)}\ns = Solver()\nfor a in range(3, m+1):\n for b in range(3, m+1):\n c = a*b\n if c > m: continue\n s.add(Not(And(vars[a], vars[b], vars[c])))\n s.add(Not(And(Not(vars[a]), Not(vars[b]), Not(vars[c]))))\nreturn s.check() == sat\n\nfor m in range(240, 247):\n print(m, is_colorable(m))\n\n240 True 241 True 242 True 243 False 244 False 245 False 246 False\n(Elapsed 1.7s / Limit 6s)\n\n<|end|>\nResponse:
\n<|start|>assistant\n<|message|>Great! The other concurrent agent found threshold around 243.\nSo minimal m where unsatisfiable appears is 243.\nLet's verify ourselves using CP-SAT for m=242 and 243.\nFigure 2. Successful push: a peer thread's z3-solver result is inserted, leading the receiving thread to redirect from independent search to verification. Correct answer: 243 (problem pacore-00181 from the CrystalMath dataset).
\nBased on manual observation, when injected information is not relevant to the current thread's reasoning path, it is typically ignored. The model occasionally shows brief distraction (\"let me note what the other agent found…\") before continuing its own approach.
\nThe sharing topology for the final submission follows a ring structure: each thread receives pushes from exactly 2 peers, offset by +1 and +2 modulo 4, with a slight difference for forked threads. Concretely:
\n| Thread | \nReceives from | \n
|---|---|
| 0 | \n1, 2 | \n
| 1 | \n2, 3 | \n
| 2 | \n3, 0 | \n
| 3 | \n0, 1 | \n
| 4 (fork from 0) | \n1, 5 | \n
| 5 (fork from 1) | \n2, 6 | \n
| 6 (fork from 2) | \n3, 7 | \n
| 7 (fork from 3) | \n0, 4 | \n
After many experiments varying which threads and how many to receive from — including random selection and receiving from only one thread — no clearly superior topology was found. Receiving from many threads offers the advantage of diverse information, but also risks interrupting a thread's own reasoning. Receiving from one or two peers appeared to perform better, though further research is needed. The ring topology ensures no thread is isolated while preventing any single thread's output from reaching all others simultaneously.
\nSharing reasoning text — which often includes predicted answers — caused threads to simply copy the first answer produced, regardless of correctness. This is further discussed in 3.1.2.1 \"Why tool I/O, not reasoning text\".
\n| Shared content | \nObserved behaviour | \n
|---|---|
| Reasoning text (often including predicted answer) | \nBlind copying of answer; errors propagate | \n
| Recent tool I/O, ring topology | \nBetter results; errors less likely to cascade | \n
Several approaches share reasoning text between threads — either raw chain-of-thought, a model-generated summary, or the conversation history preceding a branch point. We chose to share only tool I/O events. This subsection states the empirical evidence and the mechanistic reasons.
\nEmpirical evidence. In an experiment, we shared each tool I/O event together with the few hundred characters of reasoning text immediately preceding it, hypothesizing that the context would help recipients interpret the result. Public score dropped from an average around 41 (I/O-only) to 37 (I/O + preceding context) — lower than the lowest of several I/O-only trials, albeit on a single run. We take this as an indication that the text component was actively harmful rather than merely redundant.
\nMechanism 1: non-directionality of code. Shared reasoning text routinely contains assertive closers — \"so the answer is X\", \"therefore…\", \"this confirms…\" — and the model's natural continuation tendencies favour agreement with such assertions when they appear as quasi-demonstrations in context. Code carries no such closers: a snippet that returns 242 reports a value, not a verdict. In our traces, recipients of shared tool I/O reliably verify before adopting an implication of the result (see the very last line of Fig. 2, where the recipient model responds to the I/O insertion with \"Let's verify ourselves using CP-SAT for m=242 and 243\"); recipients of shared text typically continue from the text's conclusion without re-verification. This is, we believe, the reason text sharing produces a wrong-answer collapse and tool-I/O sharing does not.
Mechanism 2: information density and context economy. A short Python snippet plus its output can encode an algorithmic attempt, a verification, or a pattern probe in a few hundred characters; the natural-language reasoning needed to convey the same intent is typically several times longer. Long insertions also distract the recipient from its own reasoning thread — even if the recipient ultimately judges the insertion irrelevant, recovering its prior context is non-trivial. Short code minimises both costs, and gpt-oss-120b reads the intent of short code fluently.
Mechanism 3: built-in salience. Every tool I/O event marks a discrete reasoning step the producing thread judged worth executing — a calculation, a verification, a hypothesis test. The set of \"events worth broadcasting\" is therefore self-defining, requiring neither a learned filter nor an auxiliary summarization pass (the latter would also be expensive under our inference budget).
\nTaken together, tool I/O changes both the signal shared between threads (no embedded verdict) and the behaviour it induces in recipients (verification rather than continuation). Reasoning-text sharing modifies neither, which leads to wrong-answer collapse.
\nAn alternative mode allowed threads to actively retrieve peer discoveries on demand by calling bb_read() (\"Blackboard_read\") from within the Python sandbox:
# Injected into each sandbox's initialisation\nimport sqlite3, json\n\ndef bb_read(qid, key):\n conn = sqlite3.connect(\"aimo_blackboard.db\")\n r = conn.execute(\n \"SELECT value FROM bb WHERE qid=? AND key=?\",\n (qid, key)\n ).fetchone()\n conn.close()\n return json.loads(r[0]) if r else None\nWhen bb_read() was mentioned once in the initial system prompt, it was called very rarely — once or twice across all threads over dozens of problems. To encourage more frequent use, a call snippet was inserted alongside every push message. This produced the opposite failure: threads repeatedly called bb_read() for all peer threads in sequence, spending most of their token budget reading others' conversations rather than reasoning (Appendix B). The pull system was therefore discarded entirely.
Motivation. During early trials on a difficult reference problem (target answer: 8687), we observed that even after a thread produced an incorrect answer, prompting it with \"The answer you provided was wrong. Try again.\" sometimes led to the correct answer. From this observation, several prompt variants were tested.
\nMechanism. When a thread first produces an answer, it forks independently: the original thread and a copy both continue reasoning, each treating the candidate answer as a hypothesis to confirm or revise. Each follows its own chain of thought, runs its own tool calls, and stops under the same individual stop condition (3 consecutive identical answers). Other threads that have not yet produced an answer continue unaffected.
\nThe double-check prompt is selected from the following options:
\nMain prompts:
\nOptional sub-prompts (appended to the mild prompt):
\nEmpirical analysis. We analysed 3,497 answer-producing thread instances across 328 problems. Figure 3 shows the breakdown of outcome categories.
\n
Figure 3. Distribution of outcome categories: Correct_single (no fork), Wrong_single (no fork), Correct→Correct, Wrong→Wrong, Wrong→Correct (WC), Correct→Wrong (CW), and rare multi-change sequences.
\nCorrect-single and Wrong-single instances were not double-checked due to the time limit or a global majority stop. The key finding is that wrong-to-correct (WC) transitions occur approximately 4× more often than correct-to-wrong (CW) transitions, suggesting that well-reasoned correct answers are more robust to verification pressure than poorly-reasoned wrong answers. Multi-change sequences (C→W→C or W→C→W) were rare enough to be negligible.
\nWC transitions occur in approximately 4% of threads on a moderately difficult problem set. Extrapolated to 50 test problems with up to 8 threads each (400 thread instances total), this is a small absolute count, but on hard problems — where perhaps 10 or more problems in the test set may be at the difficulty level where WC transitions occur at higher rates — one or two problems out of 50 may be rescued per run. The prediction is rough, and one or two problems is within the base score variance, but the direction is positive.
\nPrompt comparison. Figure 4 compares the two main prompts. The aggressive prompt achieves a higher WC rate (13.9% vs 4.9%) but also a higher CW rate (4.0% vs 1.8%). Figure 5 compares sub-prompt variants within the mild-prompt family; the counterexample sub-prompt improves the WC rate without a proportionate increase in the CW rate.
\n
Figure 4. Grouped bar chart comparing WC% and CW% for aggressive vs. mild prompt. N labelled per condition.
\n
Figure 5. Grouped bar chart comparing WC% and CW% for mild, mild+counterexample, and mild+structure-check. N labelled per condition.
\nThe prompt for each double-check call in the final submission is selected from aggressive, mild, or mild+counterexample based on a simple heuristic rule.
\nThis analysis was conducted on data from recent runs, not a designed experiment: problem distributions and prompt assignments were not balanced across conditions. A problem-weighted normalisation (each problem's CC, CW, WC and WW rates are computed per-problem and weighted by frequency) leaves the results qualitatively unchanged; see Appendix A.
\nFor combinatorial and discrete optimisation problems, GPT-OSS 120B naturally attempts to use constraint-solving libraries. Pre-installing pulp, ortools, pysat-solver, and z3-solver avoids import-time failures and latency. Threads are selectively permitted to use these libraries via their system prompts. Because these libraries can run for many seconds or cause sandbox instability, tool calls were given flexible time limits and sandbox health was monitored continuously.
\n\nNote on variance: With only 50 public test problems, score differences of 1–3 points are within noise. All comparisons below are directional, not statistically definitive.
\n
The best observed public score from this work was 46, which was the highest on the public leaderboard at the time. The final submitted notebook, which incorporated the fixes described in Section 4.3, scored 41 on the public leaderboard with an average of approximately 40–42 across repeated runs of similar strategies.
\nFigure 6 shows our public leaderboard score history across all submissions, grouped by identical notebook versions.
\n
Figure 6. Public score history. Identical versions of the same notebook are grouped with the same markers (e.g., Notebook #73 was submitted five times and received public scores [45, 46, 39, 39, 42], shown with green circles). The notable jump corresponds to the introduction of the Blackboard push system, which was preliminarily tested in Notebook #73 and fully applied in Notebook #76 (alongside the simultaneous RLM-like pull system).
\nA clear score increase was observed when the Blackboard push and pull systems were introduced simultaneously, making individual attribution difficult. Since the pull system was later discarded without a clear drop in average score, the push system is believed to be the main contributor — though this could not be confirmed statistically.
\nTwo thread-count variants were compared:
\n| Variant | \nDescription | \nAverage score | \nPeak score | \nNotebook # in Figure 6 | \n
|---|---|---|---|---|
| Final (4-thread + fork) | \n4 threads, fork on first answer | \n~40–42 | \n46 | \n#76, #93, #105, final submission, and others | \n
| Alternative (6-thread, no fork) | \n6 threads, no fork, early stop at 3 identical | \n~41–42 | \n43 | \n#78, #101, and others | \n
The alternative showed a slightly higher average score but a lower peak, and was not selected.
\nThe following issues were identified in the notebook that achieved the best public score and were partially or fully fixed before the final submission:
\nNameError failures on tool calls. Variable inheritance via globals() serialisation was added.bb_read() (the pull system) was called very rarely without explicit per-step prompting, and called excessively when such prompting was added — the model began querying siblings in place of its own reasoning. We could not find a useful middle ground, and it was discarded entirely.ortools, etc.) sometimes became unresponsive with no error returned; health-check replacement was made more aggressive.We presented an inference-time coordination system for mathematical olympiad reasoning that combines a shared Blackboard for propagating intermediate computational discoveries, a per-thread forking protocol for answer verification, and adaptive resource management for Python tool use. The most impactful contribution is believed to be the Blackboard push system — automatically inserting a portion of another thread's recent tool activity into the current thread's context in a ring topology — combined with the per-thread forking system for answer double-checking. Analysis of over 3,000 answer-change events shows that double-checking recovers incorrect answers approximately 4× more often than it corrupts correct ones, providing quantitative support for the verification approach. Adaptive sandbox management and constraint-solver support contributed to robustness.
\nTime management is a central constraint: double-checking and heavy solver libraries both consume significant time, and starting with 8 threads was not affordable. Starting with 4 threads (forking to at most 8) was the selected balance.
\nImproving thread coordination, reducing evaluation variance through larger reference problem sets, and more robust sandbox management under heavy solver load remain open directions.
\nRecursive Language Models. Zhang et al.\nhttps://arxiv.org/html/2512.24601v1
\nThe double-check prompt comparison in Section 3.2 uses unbalanced observational data (different problems appear at different rates across prompt conditions). To control for problem difficulty, per-problem WC and CW rates are computed for each problem and weighted by frequency. The weighted results are qualitatively consistent with the unweighted analysis: the rank ordering of prompts by CC (Correct→Correct), CW (Correct→Wrong), WC (Wrong→Correct), and WW (Wrong→Wrong) rate is unchanged, and approximate magnitudes are similar.
\n
Figure A1. Weighted CC%, CW%, WC%, and WW% per prompt type. Data count and problem count for each prompt are shown as n=*|**problems. Prompt sets not shown in Figure 5 due to low data count are included here.
\nAs described in Section 3.1.3, encouraging frequent use of bb_read() by inserting a call snippet alongside every push message produced an adverse effect: threads repeatedly called bb_read() for all peer threads in sequence, spending most of their token budget reading others' conversations rather than reasoning independently. An example is shown below.
When Attempt 1 (Thread 1) discoveries were shared using bb_read(\"attempt1\"), extracting the \"content\" and \"text\" fields as shown below — where the block beginning with \"from functools …\" represents real Attempt 1 tool I/O —
<|start|>system\n<|message|>###Concurrent discoveries from other attempts:\n\nAttempt1 discoveries by running\nmsgs = bb_read(\"attempt1\")\nfor m in msgs[-2:]:\n t = m.get(\"content\")\n if t:\n print(t[0].get(\"text\",\"\")[-800:])\n:\nfrom functools import lru_cache\n\n# board 5x5, index cells 0..24\nsize = 5\ncell_coords = [(i, j) for i in range(size) for j in range(size)]\n\n# compute subgrids (top-left corners)\n\n..... [Attempt1 tool input and output shared].....\n\n(Elapsed0.0s/Limit6s)\n<|end|>\nThe model then attempted to retrieve more from Attempt 1 via bb_read, first checking the message count:
<|start|>assistant\n<|channel|>analysis\n<|message|>Search more.\n<|end|>\n<|start|>assistant to=python\n<|channel|>analysis code\n<|message|>msgs = bb_read('attempt1')\nlen(msgs)\n\n<|call|>\n<|start|>python to=assistant\n<|channel|>analysis\n<|message|>5\n(Elapsed0.0s/Limit6s)\n<|end|>\nThe model then read a message and received output:
\n<|start|>assistant\n<|channel|>analysis\n<|message|>Let's view the later messages.\n\n<|end|>\n<|start|>assistant to=python\n<|channel|>analysis code 0-5 -0.5s \n<|message|>for i,m in enumerate(msgs):\n if m.get('role')!='assistant':\n continue\n print(i, m.get('content')[:500])\n\n<|call|>\n<|start|>python to=assistant\n<|channel|>analysis\n<|message|>2 [{'type': 'text', 'text': 'We have a combinatorial game: two players .........\n(Elapsed0.0s/Limit6s)\n<|end|>\nThe model continued reading further messages from Attempt 1:
\n<|start|>assistant\n<|channel|>analysis\n<|message|>Let's see further.\n\n<|end|>\n\n......continuing reading attempt1's message......\nThe model determined that Attempt 1 had not yet found a solution and proceeded to read from Attempt 2, then Attempt 3, then back to Attempt 1:
\n<|start|>assistant\n<|channel|>analysis\n<|message|>So attempt1 hasn't solved.\n\nNow attempt2, attempt3? Let's list all attempts.\n<|end|>\n<|start|>assistant to=python\n<|channel|>analysis code 0-8 -0.4s \n<|message|>msgs = bb_read('attempt2')\nlen(msgs)\n\n<|call|>\n<|start|>python to=assistant\n<|channel|>analysis\n<|message|>29\n(Elapsed0.0s/Limit6s)\n<|end|>\n<|start|>assistant\n<|channel|>analysis\n<|message|>Let's view attempt2 maybe later messages include final answer.\n\n<|end|>\n<|start|>assistant to=python\n<|channel|>analysis code 0-9 -0.2s \n<|message|>for i,m in enumerate(msgs):\n if m.get('role')=='assistant':\n print(i, m.get('content')[:200])\n\n<|call|>\n<|start|>python to=assistant\n<|channel|>analysis\n<|message|>2 [{'type': 'text', 'text': 'We have a set S size 35. Let F be a ...........\n(Elapsed0.0s/Limit6s)\n<|end|>\n<|start|>assistant\n<|channel|>analysis\n<|message|>So attempt2 not relevant.\n\nAttempt3 maybe.\n\n<|end|>\n<|start|>assistant to=python\n<|channel|>analysis code 0-10 -0.4s \n<|message|>msgs = bb_read('attempt3')\nlen(msgs)\n\n<|call|>\n\n.........continuing to get Attempt3 and then Attempt1 message again......\nThe model then attempted to read from Attempt 5. In this way, Thread 4 spent most of its token budget gathering information from other threads rather than reasoning independently. When this behaviour was observed repeatedly in the Blackboard pull system with encouraged bb_read() use, further trials with this approach were abandoned.
<|start|>assistant\n<|channel|>analysis\n<|message|>Let's see further after index 9? Already we looked.\n\nNow attempt1 has some analysis but not final answer.\n\nWe also have attempt5? Let's see.\n\n............continuing read Attempt5 message\nI tried sharing reasoning traces or keeping a \"summary bank\" with lemmas and facts. Everything failed - either too slow, or models starts collapsing onto \"loudest wrong ideas\".
\nYour code sharing idea is so neat! And it's almost for free (given how fast prefill is with Flash Attention 3), compared to more expensive prefix sharing or summarization. You solved a problem that I wasn't able to solve!
\nHave the same question as @flamingice2801 below though - ring topology may only slow down the propagation of wrong ideas, not stopping them completely. Have you investigated the diversity?
", "rawMarkdown": "I tried sharing reasoning traces or keeping a \"summary bank\" with lemmas and facts. Everything failed - either too slow, or models starts collapsing onto \"loudest wrong ideas\".\n\nYour code sharing idea is so neat! And it's almost for free (given how fast prefill is with Flash Attention 3), compared to more expensive prefix sharing or summarization. You solved a problem that I wasn't able to solve!\n\nHave the same question as @flamingice2801 below though - ring topology may only slow down the propagation of wrong ideas, not stopping them completely. Have you investigated the diversity?", "votes": 1, "replies": [ { "id": 3448821, "postDate": "2026-04-26T17:16:49.273Z", "content": "@chankhavu \nThank you for the comment. \nI was fortunate that code sharing was the first approach I attempted — it was\nsimple to implement, and everything I tried afterward produced worse results.
\nOn diversity: if you mean topological diversity beyond the ring — yes, but not much.\nAll-to-all sharing diminished efficiency.\nRough statistics suggested that receiving from 2–3 threads performed better, but the variance across runs made it difficult to\ndraw firm conclusions. \nRing + random and pure random both showed similar average performance\n to the plain ring.
\nAbout propagation of wrong ideas, it sometimes happens but very rare. I observed in most cases, injected content is not blindly copied but is actively verified by the receiving thread.
", "rawMarkdown": "@chankhavu \nThank you for the comment. \nI was fortunate that code sharing was the first approach I attempted — it was\nsimple to implement, and everything I tried afterward produced worse results.\n\nOn diversity: if you mean topological diversity beyond the ring — yes, but not much.\nAll-to-all sharing diminished efficiency.\nRough statistics suggested that receiving from 2–3 threads performed better, but the variance across runs made it difficult to\ndraw firm conclusions. \nRing + random and pure random both showed similar average performance\n to the plain ring.\n\nAbout propagation of wrong ideas, it sometimes happens but very rare. I observed in most cases, injected content is not blindly copied but is actively verified by the receiving thread.", "votes": 1 } ] }, { "id": 3445577, "postDate": "2026-04-20T05:56:47.427Z", "content": "@ippeiogawa The Blackboard design - sharing tool I/O but not answers, in a unidirectional ring - is a genuinely new idea, and the ablation table showing that answer-sharing causes blind copying while bidirectional sharing causes false consensus is exactly the kind of design insight that helps the whole community.
\nThe 46 is remarkable. Two questions:
\nBlackboard failure modes on systematic errors. The ring topology prevents circular reinforcement, but a wrong intermediate computation can still propagate downstream through two hops (T1→T0 via direct push, and T1→T2→T3 via relay). On problems where the model systematically computes a wrong intermediate value (e.g., miscounts configurations, gets a wrong base case), did you observe cases where the Blackboard hurt by propagating that wrong computation to threads that might otherwise have computed it correctly independently? In other words - what's the Blackboard's error rate: how often does it flip a correct thread to wrong vs. flip a wrong thread to correct?
The forking verification vs. just running 8 independent threads. Your final system runs 4 threads that fork to 8. An alternative is 8 fully independent threads with majority vote (the standard approach). The fork gives you adversarial \nself-checking (\"the answer was wrong, try again\"), but at the cost of halving the initial diversity (4 independent starts \ninstead of 8). Did you compare 4-fork-to-8 against 8-independent on the same problem set? Your ablation mentions 6 threads-no-fork as an alternative - was 8-threads-no-fork also tested?
The constraint solver pre-installation (z3, ortools, pulp) is an addition I haven't seen in any other writeup. On which problem types did the model actually use these, and did you observe a measurable accuracy difference on those problems vs. sympy-only?
\nAlso, some of the figures are not being rendered, perhaps a bad link to the image?
", "rawMarkdown": "@ippeiogawa The Blackboard design - sharing tool I/O but not answers, in a unidirectional ring - is a genuinely new idea, and the ablation table showing that answer-sharing causes blind copying while bidirectional sharing causes false consensus is exactly the kind of design insight that helps the whole community.\n\nThe 46 is remarkable. Two questions:\n\n1. **Blackboard failure modes on systematic errors.** The ring topology prevents circular reinforcement, but a wrong intermediate computation can still propagate downstream through two hops (T1→T0 via direct push, and T1→T2→T3 via relay). On problems where the model systematically computes a wrong intermediate value (e.g., miscounts configurations, gets a wrong base case), did you observe cases where the Blackboard *hurt* by propagating that wrong computation to threads that might otherwise have computed it correctly independently? In other words - what's the Blackboard's error rate: how often does it flip a correct thread to wrong vs. flip a wrong thread to correct?\n\n2. **The forking verification vs. just running 8 independent threads.** Your final system runs 4 threads that fork to 8. An alternative is 8 fully independent threads with majority vote (the standard approach). The fork gives you adversarial \nself-checking (\"the answer was wrong, try again\"), but at the cost of halving the initial diversity (4 independent starts \ninstead of 8). Did you compare 4-fork-to-8 against 8-independent on the same problem set? Your ablation mentions 6 threads-no-fork as an alternative - was 8-threads-no-fork also tested?\n\nThe constraint solver pre-installation (z3, ortools, pulp) is an addition I haven't seen in any other writeup. On which problem types did the model actually use these, and did you observe a measurable accuracy difference on those problems vs. sympy-only?\n\nAlso, some of the figures are not being rendered, perhaps a bad link to the image?", "votes": 1, "replies": [ { "id": 3445649, "postDate": "2026-04-20T09:19:22.733Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3448814, "postDate": "2026-04-26T16:43:56.173Z", "content": "\nThank you for the kind words and the thoughtful questions.
\nAbout the questions,
\nI had not initially collected systematic statistics on this, since casual\nobservation suggested that error propagation was rare relative to successful\nsharing. To provide a more rigorous answer, I gathered data from recent runs\ncovering over 9,000 thread-generated samples. Note that these samples vary\nin topology (ring vs. random) and injection frequency, so the numbers below\nshould be treated as indicative rather than definitive.
\nResponse rate. Not every injected message influences the receiving thread.\nUsing the heuristic that a response contains \"Attempt\" or \"agent\" within the\nfirst 500 characters after an injection, such as \"According to discoveries from other agents, …\", approximately 25% of injections\nproduced a visible response (15,656 / 63,436). Three out of four injections\nappear to be ignored. Whether 25% is the optimal response rate is unclear — a higher rate would bring more peer information into each thread's reasoning, but at the cost of greater distraction from the thread's own chain of thought. Finding the right balance, or a mechanism that incorporates peer information without interrupting ongoing reasoning, remains an open direction.
\nPer-thread analysis. To measure whether the Blackboard helped or hurt, I\nfiltered to threads where the true answer was ≥ 101 (to avoid false positives\nfrom small numbers appearing incidentally in reasoning text), leaving\n5,165 thread samples. I then checked whether the true answer had already\nappeared in the thread's text before each injection — threads where it had\nare labelled \"already found\"; those where it had not are labelled \"not yet\nfound.\"
\nWithin these 5,165 samples:
\n| Outcome | \nCount | \nRate | \n
|---|---|---|
| Injection introduced correct answer (not previously mentioned) and correct answer was ultimately submitted | \n66 | \n1.3% | \n
| Injection contained a wrong answer which was ultimately submitted | \n116 | \n2.2% | \n
| Of the wrong-answer cases above: thread had already found the correct answer before injection | \n61 | \n1.2% | \n
The raw success rate (1.3%) and error rate (2.2%) are similar in magnitude,\nwhich might suggest the Blackboard is slightly harmful on net. However, this\nanalysis undercounts the Blackboard's contribution in at least two ways:
\nI acknowledge that the heuristics used here are imprecise, and a cleaner\nmeasurement would require controlled experiments. This remains an open\nquestion.
\nThe primary reason for preferring 4 threads with forking over 8 independent\nthreads was throughput. Measured token generation speed with 4 threads\nis approximately 1.4× higher than with 8 threads, because all threads share\na single vLLM engine and longer KV caches reduce per-step throughput\nsubstantially. \nOn hard problems where each thread can exceed 60,000 tokens, 8 fully independent threads would slow the system significantly.
\nComparing 4-thread to 8-thread runs across submissions showed almost no consistent score difference,\nsuggesting that beyond 4 threads, additional starting diversity gives diminishing returns within the 5-hour time constraint.
\nI set the practical goal of completing all 50 problems within approximately 4h 30m on the public set.\nThe 4-thread-with-fork and 6-thread-no-fork configurations both met\nthis target on most runs; the 8-thread-no-fork configuration almost never\ndid. Given that private-set problems were stated to be of similar difficulty but could prove more time-consuming in practice, I\nprioritised time reliability over initial thread diversity.
\nI did not measure accuracy difference between constraint-solver-enabled and\nsympy-only runs, so a direct comparison is not available. What I observed is\nthat GPT-OSS 120B naturally reaches for these libraries on problems that\ninvolve finding the minimum or maximum integer satisfying a combinatorial\nconstraint. Representative examples from the CrystalMat dataset:
\n\"bad link to the image?\" => Fixed!!
", "rawMarkdown": "@flamingice2801\n\nThank you for the kind words and the thoughtful questions.\n\nAbout the questions, \n\n### 1. Blackboard Failure Modes: Error Rate vs. Recovery Rate\n\nI had not initially collected systematic statistics on this, since casual\nobservation suggested that error propagation was rare relative to successful\nsharing. To provide a more rigorous answer, I gathered data from recent runs\ncovering over 9,000 thread-generated samples. Note that these samples vary\nin topology (ring vs. random) and injection frequency, so the numbers below\nshould be treated as indicative rather than definitive.\n\n**Response rate.** Not every injected message influences the receiving thread.\nUsing the heuristic that a response contains \"Attempt\" or \"agent\" within the\nfirst 500 characters after an injection, such as \"According to discoveries from other agents, ...\", approximately 25% of injections\nproduced a visible response (15,656 / 63,436). Three out of four injections\nappear to be ignored. Whether 25% is the optimal response rate is unclear — a higher rate would bring more peer information into each thread's reasoning, but at the cost of greater distraction from the thread's own chain of thought. Finding the right balance, or a mechanism that incorporates peer information without interrupting ongoing reasoning, remains an open direction.\n\n**Per-thread analysis.** To measure whether the Blackboard helped or hurt, I\nfiltered to threads where the true answer was ≥ 101 (to avoid false positives\nfrom small numbers appearing incidentally in reasoning text), leaving\n5,165 thread samples. I then checked whether the true answer had already\nappeared in the thread's text before each injection — threads where it had\nare labelled \"already found\"; those where it had not are labelled \"not yet\nfound.\"\n\nWithin these 5,165 samples:\n\n| Outcome | Count | Rate |\n|---|---|---|\n| Injection introduced correct answer (not previously mentioned) and correct answer was ultimately submitted | 66 | 1.3% |\n| Injection contained a wrong answer which was ultimately submitted | 116 | 2.2% |\n| Of the wrong-answer cases above: thread had already found the correct answer before injection | 61 | 1.2% |\n\nThe raw success rate (1.3%) and error rate (2.2%) are similar in magnitude,\nwhich might suggest the Blackboard is slightly harmful on net. However, this\nanalysis undercounts the Blackboard's contribution in at least two ways:\n\n- **Confidence reinforcement.** In 763 of 5,165 cases (14.8%), the inserted\n text contained the correct answer that the thread had *already* found. This\n likely reinforces the thread's confidence in a correct answer without\n introducing new information — a benefit not captured by the \"newly\n introduced correct answer\" count above.\n- **Indirect guidance.** The Blackboard can help by sharing a useful *code\n approach or intermediate result* rather than a final answer directly. These\n cases are not captured by answer-string matching at all.\n\nI acknowledge that the heuristics used here are imprecise, and a cleaner\nmeasurement would require controlled experiments. This remains an open\nquestion.\n\n---\n\n### 2. Forking (4→8) vs. 8 Independent Threads\n\nThe primary reason for preferring 4 threads with forking over 8 independent\nthreads was **throughput**. Measured token generation speed with 4 threads\nis approximately 1.4× higher than with 8 threads, because all threads share\na single vLLM engine and longer KV caches reduce per-step throughput\nsubstantially. \nOn hard problems where each thread can exceed 60,000 tokens, 8 fully independent threads would slow the system significantly.\n\nComparing 4-thread to 8-thread runs across submissions showed almost no consistent score difference,\nsuggesting that beyond 4 threads, additional starting diversity gives diminishing returns within the 5-hour time constraint.\n\nI set the practical goal of completing all 50 problems within approximately 4h 30m on the public set.\nThe 4-thread-with-fork and 6-thread-no-fork configurations both met\nthis target on most runs; the 8-thread-no-fork configuration almost never\ndid. Given that private-set problems were stated to be of similar difficulty but could prove more time-consuming in practice, I\nprioritised time reliability over initial thread diversity.\n\n---\n\n### 3. Constraint Solver Libraries (z3, ortools, pulp)\n\nI did not measure accuracy difference between constraint-solver-enabled and\nsympy-only runs, so a direct comparison is not available. What I observed is\nthat GPT-OSS 120B naturally reaches for these libraries on problems that\ninvolve finding the minimum or maximum integer satisfying a combinatorial\nconstraint. Representative examples from the [CrystalMat dataset](https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3/discussion/672592):\n\n- **dapo-3046:** Maximum number of lattice points in a 101×101 grid with no\n 4-point isosceles trapezoid whose base is axis-parallel.\n- **dapo-3655:** Maximum red points in a critical 100×100 grid colouring\n with a specific neighbourhood constraint.\n- **beyondaime-00070:** Maximum edges in a 160-vertex simple graph satisfying\n a local adjacency condition.\n\n\n\"bad link to the image?\" => Fixed!!", "votes": 2, "replies": [ { "id": 3448907, "postDate": "2026-04-27T02:02:06.357Z", "content": "Thank you for the detailed data - the 9,000+ thread analysis is exactly what I was hoping for.
\nThe numbers are interesting: 1.3% introduction of correct answers vs 2.2% propagation of wrong answers looks net-negative on paper, but the 14.8% confidence reinforcement and the uncaptured indirect guidance (sharing a useful code approach rather than an answer) are likely where the real value lives. The fact that 75% of injections are simply ignored is actually reassuring - it means the model is selective about what it incorporates, which limits damage from wrong propagations.
\nThe throughput argument for 4-fork vs 8-independent is compelling and something I hadn't considered. At 60,000+ tokens per thread on hard problems, 8 concurrent threads would indeed bottleneck the vLLM engine badly. Our pipeline uses 8 threads but with early_stop=4, so in practice many threads are cancelled before reaching full context - which may explain why we don't see the same throughput degradation.
\nThe constraint solver examples (dapo-3046, beyondaime-00070) are useful - these are exactly the combinatorial optimization problems where sympy alone would struggle. Worth trying for future competitions.
", "rawMarkdown": "Thank you for the detailed data - the 9,000+ thread analysis is exactly what I was hoping for.\n\nThe numbers are interesting: 1.3% introduction of correct answers vs 2.2% propagation of wrong answers looks net-negative on paper, but the 14.8% confidence reinforcement and the uncaptured indirect guidance (sharing a useful code approach rather than an answer) are likely where the real value lives. The fact that 75% of injections are simply ignored is actually reassuring - it means the model is selective about what it incorporates, which limits damage from wrong propagations.\n\nThe throughput argument for 4-fork vs 8-independent is compelling and something I hadn't considered. At 60,000+ tokens per thread on hard problems, 8 concurrent threads would indeed bottleneck the vLLM engine badly. Our pipeline uses 8 threads but with early_stop=4, so in practice many threads are cancelled before reaching full context - which may explain why we don't see the same throughput degradation.\n\nThe constraint solver examples (dapo-3046, beyondaime-00070) are useful - these are exactly the combinatorial optimization problems where sympy alone would struggle. Worth trying for future competitions.", "votes": 1 } ] } ] }, { "id": 3459004, "postDate": "2026-05-17T06:55:19.227Z", "content": "Edited on [May 17, 2026] for the rebuttal phase:
\nFollowing up on the questions from @philipvonderlind, here are some additional questions to help us assess your submission:
\nLook forward to reading your responses on these.
", "rawMarkdown": "Following up on the questions from @philipvonderlind, here are some additional questions to help us assess your submission:\n\n1. **Variance vs other public notebooks:** Your 5 submissions of Notebook #73 scored 45, 46, 39, 39, 42 (around a 7-point spread on identical code), which is a lot of variance. Do you have a feel for whether the Blackboard system itself increases or decreases this run-to-run variance compared to the background variation we've seen on other high-scoring public notebooks (e.g. the 44/50 widely used notebook)?\n2. **Why code sharing avoids the wrong-answer collapse text sharing produces:** Your writeup notes that sharing reasoning text \"caused threads to simply copy the first answer produced, regardless of correctness\", but code sharing works without this failure. What's the mechanistic difference? Is it that code gets executed and so is implicitly verified, or that code carries less of the persuasive surface that makes prose claims sticky?\n\nLook forward to reading your responses on these.", "replies": [ { "id": 3458988, "postDate": "2026-05-17T06:08:48.930Z", "content": "@sambealing Thank you for the detailed questions — we appreciate the close reading.
\nA fair caveat first: with only five runs of Notebook #76 it is hard to draw rigorous conclusions about score variance. Our intuition, however, is that the blackboard system does not meaningfully increase or decrease run-to-run variance, and that what we are seeing is the background variance characteristic of any high-scoring solution on this leaderboard under the 5-hour budget. Three observations support this:
\nPublic notebooks show comparable or greater spread. The widely used 44/50 notebook has been reported scoring as low as 34 on the same evaluation (discussion thread \"How stable is the 44/50 public notebook for you?\") — a 10-point spread, larger than ours, albeit aggregated over many more runs.
Pre-blackboard variants of our own pipeline showed similar variance. §4.1 (Score History and Strategy Comparison) documents that even simpler baselines without the blackboard exhibited multi-point spreads on small sample sizes — e.g. Notebook #62 showed a 5-point spread across just two runs.
A plausible mechanistic source of variance is time-budget pressure, not the sharing mechanism. Notebook #76 runs close to the 5-hour cap. Recorded run times for our five #76 submissions were 5h5m, 5h25m, 5h25m, 4h57m, with the last unrecorded; on average the pipeline appears to run at or beyond the 5-hour limit (per-run timing reported on AIMO3 Leaderboard History at aimo.huikang.dev/leaderboard is itself noisy under competition rules and should be treated as approximate). The time management in notebook #76 has inefficiencies (§4.3) — most notably tool execution running for ~60s even after the final answer is reached. Since the remaining overall time is allocated equally to the remaining problems, we suspect the lower-scoring runs are runs in which some final hard problems — those solvable only with substantial compute — received insufficient remaining budget, while the higher-scoring runs benefited from a more favourable problem ordering that left more budget for the hardest items. If this is right, the variance is being driven by tail-end budget allocation rather than by stochasticity in the blackboard's sharing decisions, and a similar mechanism would affect any solution running close to the time cap.
We have not isolated the blackboard's contribution to variance with a controlled ablation (e.g. running the same pipeline with PUSH disabled across many seeds). So this remains a hypothesis rather than a measurement.
\nBoth of the mechanisms you propose appear to be operating, and we believe they reinforce each other. We see two distinct effects:
\nCode lacks the persuasive surface that prose has. Shared reasoning text routinely contains assertive closers — \"so the answer is X\", \"this confirms that…\", \"therefore…\" — and the model's natural continuation patterns favour agreement with such assertions, especially when they appear in what looks like an in-context demonstration. Code does not carry these closers. A snippet that returns 242 says only that 242 was returned; it does not say 242 is correct, and it does not pattern-match to \"follow the conclusion.\"
Code is explicitly verified by the recipient. In our traces, after a code-I/O event is shared, the recipient thread reliably verifies before adopting any implication of the result. Fig. 2 gives a concrete example: after consuming a sibling's I/O, the receiving thread emits \"Let's verify ourselves using CP-SAT for m=242 and 243.\" This verification step is, in our observation, often absent when reasoning text is shared — the recipient simply continues from the text's conclusion.
Put together: code sharing changes both the signal (no embedded verdict) and the behaviour (the recipient verifies). Text sharing leaves both intact and so reproduces the original thread's conclusion downstream, correct or not. We will fold this mechanistic account into the new §3.1.2.1 in the rewrite.
", "rawMarkdown": "@sambealing Thank you for the detailed questions — we appreciate the close reading.\n\n### Q1. Run-to-run variance\n\nA fair caveat first: with only five runs of Notebook #76 it is hard to draw rigorous conclusions about score variance. Our intuition, however, is that the blackboard system does not meaningfully increase or decrease run-to-run variance, and that what we are seeing is the background variance characteristic of any high-scoring solution on this leaderboard under the 5-hour budget. Three observations support this:\n\n1. **Public notebooks show comparable or greater spread.** The widely used 44/50 notebook has been reported scoring as low as 34 on the same evaluation ([discussion thread \"How stable is the 44/50 public notebook for you?\"](https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3/discussion/683612)) — a 10-point spread, larger than ours, albeit aggregated over many more runs.\n\n2. **Pre-blackboard variants of our own pipeline showed similar variance.** §4.1 (*Score History and Strategy Comparison*) documents that even simpler baselines without the blackboard exhibited multi-point spreads on small sample sizes — e.g. Notebook #62 showed a 5-point spread across just two runs.\n\n3. **A plausible mechanistic source of variance is time-budget pressure, not the sharing mechanism.** Notebook #76 runs close to the 5-hour cap. Recorded run times for our five #76 submissions were 5h5m, 5h25m, 5h25m, 4h57m, with the last unrecorded; on average the pipeline appears to run at or beyond the 5-hour limit (per-run timing reported on [AIMO3 Leaderboard History at aimo.huikang.dev/leaderboard](https://aimo.huikang.dev/leaderboard) is itself noisy under competition rules and should be treated as approximate). The time management in notebook #76 has inefficiencies (§4.3) — most notably tool execution running for ~60s even after the final answer is reached. Since the remaining overall time is allocated equally to the remaining problems, we suspect the lower-scoring runs are runs in which some final hard problems — those solvable only with substantial compute — received insufficient remaining budget, while the higher-scoring runs benefited from a more favourable problem ordering that left more budget for the hardest items. If this is right, the variance is being driven by tail-end budget allocation rather than by stochasticity in the blackboard's sharing decisions, and a similar mechanism would affect any solution running close to the time cap.\n\nWe have not isolated the blackboard's contribution to variance with a controlled ablation (e.g. running the same pipeline with PUSH disabled across many seeds). So this remains a hypothesis rather than a measurement.\n\n### Q2. Mechanism behind tool-I/O sharing's resistance to wrong-answer collapse\n\nBoth of the mechanisms you propose appear to be operating, and we believe they reinforce each other. We see two distinct effects:\n\n1. **Code lacks the persuasive surface that prose has.** Shared reasoning text routinely contains assertive closers — \"so the answer is X\", \"this confirms that...\", \"therefore...\" — and the model's natural continuation patterns favour agreement with such assertions, especially when they appear in what looks like an in-context demonstration. Code does not carry these closers. A snippet that returns `242` says only that `242` was returned; it does not say `242` is correct, and it does not pattern-match to \"follow the conclusion.\"\n\n2. **Code is explicitly verified by the recipient.** In our traces, after a code-I/O event is shared, the recipient thread reliably verifies before adopting any implication of the result. Fig. 2 gives a concrete example: after consuming a sibling's I/O, the receiving thread emits \"Let's verify ourselves using CP-SAT for m=242 and 243.\" This verification step is, in our observation, often absent when reasoning text is shared — the recipient simply continues from the text's conclusion.\n\nPut together: code sharing changes both the *signal* (no embedded verdict) and the *behaviour* (the recipient verifies). Text sharing leaves both intact and so reproduces the original thread's conclusion downstream, correct or not. We will fold this mechanistic account into the new §3.1.2.1 in the rewrite." } ] }, { "id": 3456797, "postDate": "2026-05-12T15:52:27.803Z", "content": "Hi, thank you for your submission to the Writeup Prize. @sambealing and I will post some questions below that help us in assessing your submission more closely, as described in the updated discussion post.
\nYour solution is interesting and seems to outperform just sharing context between different completions/agents.\nI have a few questions about directions that could have been interesting as well:
\nPULL method from the blackboard, so gpt-oss-120b would better understand how to use this mechanic to manage its own context? From what you wrote, the main failure mode seems to be the model not using it.Looking forward to your response :)
", "rawMarkdown": "Hi, thank you for your submission to the Writeup Prize. @sambealing and I will post some questions below that help us in assessing your submission more closely, as described in the [updated discussion post](https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3/discussion/689703).\n\nYour solution is interesting and seems to outperform just sharing context between different completions/agents.\nI have a few questions about directions that could have been interesting as well:\n\n1. **Learning the PULL mechanic**: Did you try to apply fine-tuning for the `PULL` method from the blackboard, so `gpt-oss-120b` would better understand how to use this mechanic to manage its own context? From what you wrote, the main failure mode seems to be the model not using it.\n2. **Comparison to similar approaches**: [Another contestant has taken a similar approach](https://www.kaggle.com/writeups/khoinguyennguyen/adaptive-tree-of-thoughts-with-eagle3-accelerated) to yours, essentially sharing the Jupyter state directly between completions. At a high level, how would you say your approach differs from theirs, and why is it better?\n\nLooking forward to your response :)", "replies": [ { "id": 3458985, "postDate": "2026-05-17T06:01:24.137Z", "content": "@philipvonderlind Thank you for the careful read of the writeup, and for separating these two threads of questioning. We'll answer each in turn.
\nNo, we did not attempt fine-tuning for PULL. Before answering the broader question, one clarification: your reading — \"the main failure mode seems to be the model not using it\" — captures one half of what we actually observed; our statement on this in the writeup was not as clear as it should have been, and we will revise it (see the change at the end of this answer).
\nWith sufficiently frequent in-context exemplars (as illustrated in Appendix B), gpt-oss-120b can be coaxed into issuing PULL calls. The problem is what happens once it does. Under aggressive encouragement, the model shifts from solving the problem to querying its siblings — repeatedly pulling partial results and substituting inspection of others' work for its own reasoning. So the failure mode is two-sided: rare and unused at low encouragement, frequent and crowding out reasoning at high encouragement. We discarded PULL not because it could not fire, but because we could not find a setting in which it fired the right amount — and we could not determine what the right amount would be.
This makes the problem one of timing and selectivity, not capability. A useful PULL would fire only when the model is stuck or has detected an inconsistency in its own trace — a meta-cognitive signal. Teaching this via vanilla SFT seems hard for two reasons (the direction is not impossible and is worth pursuing, but constructing the necessary training setup was beyond what we could attempt within the remaining competition period). First, the supervision signal (\"when should I have pulled?\") is non-trivial to construct, since it requires labelled trajectories of stuck-vs-progressing reasoning aligned with the available state of sibling threads at that moment. Second, the policy depends on information the model cannot observe at decision time — namely, whether any sibling currently holds useful information.
\nThis is also part of the motivation for emphasizing PUSH in the design. PUSH inverts the asymmetry: the producing thread, which has just generated a tool I/O event, is the one best positioned to know that something informative just happened, and when it happened (immediately, as it was produced). PUSH therefore needs no learned meta-cognitive trigger — the event itself is the trigger.
\nWe will tighten the language in §4.3 (Issues Fixed or Changed Between Best Public Score and Final Submission) so the twin-failure-mode framing is explicit.
\nWriteup change (§4.3):
\nbb_read() (the pull system) was called very rarely without explicit per-step prompting; it was discarded entirely.\"bb_read() (the pull system) was called very rarely without explicit per-step prompting, and called excessively when such prompting was added — the model began querying siblings in place of its own reasoning. We could not find a useful middle ground, and it was discarded entirely.\"Several teams converged on the intuition that completions should not be fully independent, but the what and how of sharing differs substantially. We position our approach against the one you mentioned and against the broader family of text-based sharing.
\nvs. Adaptive Tree-of-Thoughts with EAGLE3 (the writeup you linked). The ToT approach inherits Jupyter state and prior conversation from a parent thread into a child thread, sequentially expanding a search tree of reasoning trajectories. Our approach differs along two axes.
\nvs. text-based sharing in general (e.g. the summarization-based approach in \"Attempts to get gpt-oss-120b to collaborate\", or simply injecting recent context from another thread). Two contrasts.
\nA more detailed mechanistic comparison — why tool I/O specifically resists the wrong-answer-collapse mode that text sharing produces — is given in our response to Q2 from @sambealing. We are also adding a new subsection §3.1.2.1 (Why tool I/O, not reasoning text) to make this contrast explicit in the writeup itself.
", "rawMarkdown": "@philipvonderlind Thank you for the careful read of the writeup, and for separating these two threads of questioning. We'll answer each in turn.\n\n### Q1. On fine-tuning the PULL mechanic\n\nNo, we did not attempt fine-tuning for PULL. Before answering the broader question, one clarification: your reading — \"the main failure mode seems to be the model not using it\" — captures one half of what we actually observed; our statement on this in the writeup was not as clear as it should have been, and we will revise it (see the change at the end of this answer).\n\nWith sufficiently frequent in-context exemplars (as illustrated in Appendix B), `gpt-oss-120b` can be coaxed into issuing PULL calls. The problem is what happens once it does. Under aggressive encouragement, the model shifts from solving the problem to querying its siblings — repeatedly pulling partial results and substituting inspection of others' work for its own reasoning. So the failure mode is two-sided: rare and unused at low encouragement, frequent and crowding out reasoning at high encouragement. We discarded PULL not because it could not fire, but because we could not find a setting in which it fired the right amount — and we could not determine what the right amount would be.\n\nThis makes the problem one of *timing and selectivity*, not capability. A useful PULL would fire only when the model is stuck or has detected an inconsistency in its own trace — a meta-cognitive signal. Teaching this via vanilla SFT seems hard for two reasons (the direction is not impossible and is worth pursuing, but constructing the necessary training setup was beyond what we could attempt within the remaining competition period). First, the supervision signal (\"when should I have pulled?\") is non-trivial to construct, since it requires labelled trajectories of stuck-vs-progressing reasoning aligned with the available state of sibling threads at that moment. Second, the policy depends on information the model cannot observe at decision time — namely, whether any sibling currently holds useful information.\n\nThis is also part of the motivation for emphasizing PUSH in the design. PUSH inverts the asymmetry: the producing thread, which has just generated a tool I/O event, is the one best positioned to know that something informative just happened, and *when* it happened (immediately, as it was produced). PUSH therefore needs no learned meta-cognitive trigger — the event itself is the trigger.\n\nWe will tighten the language in §4.3 (*Issues Fixed or Changed Between Best Public Score and Final Submission*) so the twin-failure-mode framing is explicit.\n\n**Writeup change (§4.3):**\n\n- *Before:* \"`bb_read()` (the pull system) was called very rarely without explicit per-step prompting; it was discarded entirely.\"\n- *After:* \"`bb_read()` (the pull system) was called very rarely without explicit per-step prompting, and called excessively when such prompting was added — the model began querying siblings in place of its own reasoning. We could not find a useful middle ground, and it was discarded entirely.\"\n\n### Q2. Comparison to similar information-sharing approaches\n\nSeveral teams converged on the intuition that completions should not be fully independent, but the *what* and *how* of sharing differs substantially. We position our approach against the one you mentioned and against the broader family of text-based sharing.\n\n**vs. [Adaptive Tree-of-Thoughts with EAGLE3 (the writeup you linked)](https://www.kaggle.com/writeups/khoinguyennguyen/adaptive-tree-of-thoughts-with-eagle3-accelerated).** The ToT approach inherits Jupyter state and prior conversation from a *parent* thread into a *child* thread, sequentially expanding a search tree of reasoning trajectories. Our approach differs along two axes.\n\n1. *Concurrency*: in our setup, threads run in parallel and exchange information mid-flight via the blackboard; in ToT, sharing is a parent-to-child handoff at branching points.\n2. *Granularity of what flows*: ToT inherits a thread's full state — including the natural-language conversation that preceded it — to seed diverse continuations, whereas the blackboard broadcasts only newly produced tool I/O events as they are produced, without the surrounding reasoning text.\n\n**vs. text-based sharing in general** (e.g. [the summarization-based approach in \"Attempts to get gpt-oss-120b to collaborate\"](https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3/writeups/attempts-to-get-gpt-oss-120b-to-collaborate), or simply injecting recent context from another thread). Two contrasts.\n\n1. *Compute*: when sharing requires a model call to summarize, this is expensive under the AIMO3 inference budget; PUSH of raw tool I/O is essentially free.\n2. *Content*: reasoning text, even summarized, is highly *directional* — it tends to embed a conclusion (\"so the answer is X\"), and downstream threads anchor on it without independent verification. Tool I/O alone exposes *what was tried* and *what the kernel returned*, not *what to conclude*; this nudges rather than forces the recipient, and in observation the recipient typically re-derives or verifies before adopting the hint (see the very last line of Fig. 2 in our writeup, where the recipient model responds with \"Let's verify ourselves using CP-SAT for m=242 and 243\").\n\nA more detailed mechanistic comparison — why tool I/O specifically resists the wrong-answer-collapse mode that text sharing produces — is given in our response to Q2 from @sambealing. We are also adding a new subsection §3.1.2.1 (*Why tool I/O, not reasoning text*) to make this contrast explicit in the writeup itself." } ] } ], "complete": true, "source": "live", "fetched_at": "2026-10-20T04:21:15Z", "competition_id": 118448, "competition_slug": "ai-mathematical-olympiad-progress-prize-3", "team_id": 14801984, "solution_url": "/competitions/ai-mathematical-olympiad-progress-prize-3/writeups/aimo3test" }