On the AIMO-3 competition (open-source math olympiad solving under a 5-hour GPU H100 budget), I observed that counting distinct answers across concurrent attempts is a useful runtime indicator of problem difficulty. Easy problems converge on one answer within a few completions; hard problems typically fragment into 5–8 distinct integers with no clear plurality. I used this signal to route strategy: problems flagged as hard (via a max-vote-share threshold below 40%) switched from plurality voting to logprob-weighted scoring, and received category-specific mathematical guidance in the prompt, routed by a lightweight fine-tuned classifier over five problem types (algebra, geometry, number theory, combinatorics, analysis).
\nThis writeup describes the system, reports a public leaderboard score of 39/50 (up from 36 before the classifier-routed hints), and acknowledges the absence of paired ablations that would cleanly attribute gains to specific components. The headline findings: 1. Answer diversity is a surprisingly clean runtime difficulty indicator. 2. Category-specific strategy hints moved the leaderboard more than did the configuration tuning.
\nAIMO3 is the third competition in the AI Mathematical Olympiad Progress Prize series, backed by a $10M fund dedicated to advancing open-source models toward International Mathematical Olympiad (IMO)-caliber mathematical reasoning. The competition consists of 110 original problems across algebra, combinatorics, geometry, and number theory, with difficulty scaling from National Olympiad up to full IMO level on the Kaggle platform. It is framed around closing a measurable gap between open and closed systems: in 2025, frontier closed-source models reached gold-medal IMO performance and swept all 50 AIMO2 leaderboard problems, whereas the best open-source Kaggle entry on AIMO2 solved only 34/50.
\nParticipants submit notebooks on Kaggle that perform inference within fixed compute constraints of 5 hours on a GPU H100 80GB accelerator (for more details, see https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3). For final scoring, each notebook is executed twice on the private test set and graded under a penalized accuracy scheme: 1 point when both runs produce the correct answer, 0.5 when only one does, and 0 otherwise. Models are rewarded not only for getting answers correctly, but for getting the correct answer reliably.
\nRunning gpt-oss-120B on the AIMO-3 reference dataset with 8 attempts per problem, the answer distribution across attempts turned out to be a strong signal of problem difficulty. Hard problems produced 5–8 distinct integer answers with no clear plurality; easy problems converged quickly, often triggering the 3-consecutive-unanimous early-stop within the first few completions. You can detect difficulty during runtime, using only the model's own outputs, and route your strategy accordingly.
\nI define \"high diversity\" as:
\nif unique answers > 4 then\nhigh_diversity = max_vote_share < 0.40\nwhere max_vote_share = max(answer_votes.values()) / n_valid_attempts
\nIf unique answers is greater than 4 and no single answer captures at least 40% of valid attempts, the problem is flagged as hard. High answer diversity turned out to be a reasonable indicator that the model was struggling with a problem. In the 20 retained runs of the n-Norwegian problem spanning solver versions v343–v478, most showed the pattern the threshold is designed to catch: 5+ distinct answers, max vote share in the 25–38% range. A smaller number of runs had 3 attempts converge on the same answer, pushing max vote share above 40% — in which case the default path ran. On this problem, that sometimes produced the correct answer and sometimes didn't.
\nTwo things follow from the hard flag: a scoring-rule change and a prompt-strategy change.
\nFor completeness: the default (low-diversity) path uses plurality voting with inverse-entropy tiebreak. The high-diversity path switches to logprob-weighted scoring, borrowed from the ARC Prize 2024 winning solution, with a median cap that prevents a single very-confident attempt from dominating. In the 20 runs I have on the n-Norwegian problem, the logprob-weighted rule selected 8687 in 14/20. One caveat: these 20 runs are partially cherry-picked — during development I didn't consistently save outputs from runs that didn't produce 8687, so the 14/20 figure likely over-represents favorable runs, and the true selection rate on the final solver version is probably lower.\nI didn't run paired ablations isolating the scoring rule from the rest of the system, so I can't attribute a specific accuracy delta to it. The stronger contribution to improving the leaderboard score was on the prompt engineering side.
\nThe more impactful change was what the model saw in its prompt. A lightweight fine-tuned classifier (bge-base-en-v1.5, CPU-resident) categorizes the problem into one of five types: algebra, geometry, number_theory, combinatorics, analysis. When classifier confidence is ≥0.90, I inject a category-specific strategy hint into the user message.
\nThese aren't generic \"try harder\" instructions, they are concrete mathematical guidance that competition problems of that type actually reward. A sample from the number-theory hint:\nMODULAR EXPONENTIATION — Use the three-argument form pow(base, exp, mod) for all modular powers. CORRECT: pow(2, 1018, 109 + 7) WRONG: 2(1018) % (10**9 + 7) # allocates an astronomically large integer\nDIOPHANTINE EQUATIONS — Before writing any loop, reduce the equation modulo a small number (try 3, 4, and 8 in that order) to eliminate impossible residue classes and shrink the search space to a tractable size.\nGeometry gets a different hint emphasizing coordinate/complex representations and a warning to avoid sympy.geometry (which can hang). Combinatorics gets guidance on functools.lru_cache for DP and strict iteration limits. Analysis gets domain-restriction reminders.\nThe hints are specific enough to cite in code. They live in CFG.TYPE_STRATEGY_HINTS as ~200-word blocks. When classifier confidence drops below 0.90, the system falls back to the default pool — which empirically worked well on the easy problems.
\nI observed a +3 improvement on the public leaderboard after adding the classifier-routed hints — from 36 to 39. That's a single data point across two submissions, not a controlled comparison: other changes shipped in the same submission window, and individual problem-grading outcomes can shift by 1–2 on any given run. I can't confidently attribute the full +3 to strategy hints, but the direction was consistent with development-time observations: more 3-consecutive-unanimous early stops on hard problems after the hints went in, fewer context_exhausted NAs, more runs finishing inside the per-problem budget.
\nThe stronger qualitative signal was at the attempt level rather than the leaderboard level. With hints enabled, attempts that converged were more likely to converge on the same answer, reducing the vote spread that forced the high-diversity path in the first place. In effect, the dynamically generated hints narrowed the range of the model's reasoning paths — attempts landed closer to each other, which made aggregation more reliable downstream.
\nNo ablation. The leaderboard delta conflates hints with whatever else shipped in that submission. To attribute the gain cleanly I'd need runs on a fixed subset of the reference problems with and without hints, holding everything else constant.
\nThe hints could be improved. They reflect my intuitions about what each problem type rewards, based on manually inspecting attempts that failed on the reference set. Someone with deeper mathematical expertise might write better ones; a systematic way to generate or tune them is an open direction.\nThe classifier is small. It was fine-tuned on a small labeled dataset; categorization accuracy on held-out problems wasn't rigorously measured. The 0.90-confidence threshold is what I used empirically, not what a held-out validation curve would suggest.\n\"Easy vs hard\" is a blunt split. The 40% vote-share threshold is a crude binary. In principle difficulty is continuous, and a more graded routing (e.g., linearly interpolating between scoring rules) might do better.
High answer diversity turned out to be a reasonable indicator that the model was struggling with a problem. Easy problems converge on one answer within a few completions; hard problems typically fragment into 5–8 distinct integers. It's not a perfect detector — on some runs a few attempts converge early and bypass the hard-problem path even on genuinely hard problems — but the correlation was strong enough to be a useful routing input. The signal is available as soon as a few attempts complete, well before the full budget is spent, and it lets the solver route strategy without needing to know problem category, difficulty, or correctness ahead of time. The 40% vote-share threshold is a stepping stone; a more graded routing is an obvious next step.
\nThese did more work than the scoring rule on top of them. The hints aren't generic scaffolding — they're concrete mathematical guidance that rewards the specific failure modes each problem type exhibits: pow(base, exp, mod) for number theory, coordinate geometry over sympy.geometry, functools.lru_cache for combinatorial DP, domain restrictions for analysis. Manual curation was tractable for five categories but doesn't scale; a systematic way to generate or mine such hints from olympiad solution archives seems worth trying.
\nI spent meaningful time adjusting attempts per problem, worker pool size, batch size, GPU memory utilization, context window, early-stop thresholds, temperature schedules and the results of any individual change were usually hard to distinguish from run-to-run variance. Settings that appeared worse in one submission sometimes scored better in the next, and vice versa. The values in the final submission (8 attempts, 16 workers, 65K context, 0.955 GPU memory utilization) are the ones that felt stable across several consecutive runs, not ones I can claim are optimal. On a capable model with a 5-hour budget, the parameter space is large, submission noise is real, and without a rigorous evaluation loop (see Future Directions), informal tuning didn't do much to improve the leaderboard score.
\nLaunching 4 additional attempts when high diversity was detected did not score high on the public leaderboard. In theory, spending more compute on problems flagged as hard should raise accuracy. In practice, the launch condition required a larger time window that rarely materialized within the 5-hour budget; the feature fired on only a small fraction of problems, and when it did I couldn't distinguish its effect from noise.
\nFirst, a small ablation matrix isolating hints, classifier routing, and scoring rule independently. For 10 problems × 5 runs, this would resolve most of the attribution questions in this writeup. Second, stronger statistical evaluation on a per-problem basis. A single problem at n=20 runs barely distinguishes 70% success from chance (p≈0.058). Scaling to ≥30 runs per problem across all five types — roughly 6 hours of H100 time per evaluation — would produce attribution claims that hold up under peer review. Third, a more systematic way to construct strategy hints. For example, mining them from the reasoning traces of successful runs on each category and distilling the common patterns into hint blocks.
\nCompute. All experiments ran inside the Kaggle AIMO-3 competition kernel: a single NVIDIA H100 80GB GPU with a 5-hour time limit per notebook. The reasoning model (gpt-oss-120B) was served via vLLM on the GPU; the problem-type classifier (fine-tuned bge-base-en-v1.5) ran on CPU to avoid competing for GPU memory.
\nServing. gpt-oss-120B was served through vLLM's OpenAI-compatible completions endpoint with prefix caching enabled, 65,536-token context window, fp8 KV cache, and up to 256 concurrent sequences. Per-problem inference fanned out to 8 attempts running in parallel over a pool of 16 persistent Jupyter kernels (for tool-use Python execution), with per-attempt timeouts and a dynamic per-problem budget scaled to fit 50 problems within the overall 5-hour limit.
\nThe original base notebook score of 42 on the public leaderboard\nhttps://www.kaggle.com/code/andreasbis/aimo-3-gpt-oss-120b-with-tools
\nHighest score of 44 on the public leaderboard improved the original base notebook by andreasbis through prompt engineering by nihilisticneuralnet\nhttps://www.kaggle.com/code/nihilisticneuralnet/44-50-let-me-over-cook
\nThanks to the competition hosts and all the participants!
\nOverview about AIMO3\nhttps://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3/overview
n-Norwegian Problem 10 from AIMO3_Reference_Problems.pdf\nhttps://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3/data?select=AIMO3_Reference_Problems.pdf
Inference reference dataset\nhttps://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3/data?select=reference.csv
Adding to the comments by @sambealing below, I have two additional questions:
\nClassifiers: Did you experiment with different types of classifiers, such as FastText or simple Regex-based classifiers, or different injection prompts? If so, it would be valuable to share the results with the community. As Sam mentioned, even if the experiments are not fully documented, it would still be helpful to describe what you tried and what you observed.
Reproducibility: It would be great if you could share the code used to train the classifiers and construct the pipeline, so others can reproduce them from scratch or adapt them for their own systems.
Hi, thank you for your submission to the Writeup Prize. @philipvonderlind and I will post some questions below that help us in assessing your submission more closely, as described in the updated discussion post.
\nLook forward to reading your responses on these.
", "votes": 0, "replies": [] } ], "comment_count": 2, "messages": [ { "id": 3446675, "postDate": "2026-04-22T00:08:06.517Z", "content": "On the AIMO-3 competition (open-source math olympiad solving under a 5-hour GPU H100 budget), I observed that counting distinct answers across concurrent attempts is a useful runtime indicator of problem difficulty. Easy problems converge on one answer within a few completions; hard problems typically fragment into 5–8 distinct integers with no clear plurality. I used this signal to route strategy: problems flagged as hard (via a max-vote-share threshold below 40%) switched from plurality voting to logprob-weighted scoring, and received category-specific mathematical guidance in the prompt, routed by a lightweight fine-tuned classifier over five problem types (algebra, geometry, number theory, combinatorics, analysis).
\nThis writeup describes the system, reports a public leaderboard score of 39/50 (up from 36 before the classifier-routed hints), and acknowledges the absence of paired ablations that would cleanly attribute gains to specific components. The headline findings: 1. Answer diversity is a surprisingly clean runtime difficulty indicator. 2. Category-specific strategy hints moved the leaderboard more than did the configuration tuning.
\nAIMO3 is the third competition in the AI Mathematical Olympiad Progress Prize series, backed by a $10M fund dedicated to advancing open-source models toward International Mathematical Olympiad (IMO)-caliber mathematical reasoning. The competition consists of 110 original problems across algebra, combinatorics, geometry, and number theory, with difficulty scaling from National Olympiad up to full IMO level on the Kaggle platform. It is framed around closing a measurable gap between open and closed systems: in 2025, frontier closed-source models reached gold-medal IMO performance and swept all 50 AIMO2 leaderboard problems, whereas the best open-source Kaggle entry on AIMO2 solved only 34/50.
\nParticipants submit notebooks on Kaggle that perform inference within fixed compute constraints of 5 hours on a GPU H100 80GB accelerator (for more details, see https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3). For final scoring, each notebook is executed twice on the private test set and graded under a penalized accuracy scheme: 1 point when both runs produce the correct answer, 0.5 when only one does, and 0 otherwise. Models are rewarded not only for getting answers correctly, but for getting the correct answer reliably.
\nRunning gpt-oss-120B on the AIMO-3 reference dataset with 8 attempts per problem, the answer distribution across attempts turned out to be a strong signal of problem difficulty. Hard problems produced 5–8 distinct integer answers with no clear plurality; easy problems converged quickly, often triggering the 3-consecutive-unanimous early-stop within the first few completions. You can detect difficulty during runtime, using only the model's own outputs, and route your strategy accordingly.
\nI define \"high diversity\" as:
\nif unique answers > 4 then\nhigh_diversity = max_vote_share < 0.40\nwhere max_vote_share = max(answer_votes.values()) / n_valid_attempts
\nIf unique answers is greater than 4 and no single answer captures at least 40% of valid attempts, the problem is flagged as hard. High answer diversity turned out to be a reasonable indicator that the model was struggling with a problem. In the 20 retained runs of the n-Norwegian problem spanning solver versions v343–v478, most showed the pattern the threshold is designed to catch: 5+ distinct answers, max vote share in the 25–38% range. A smaller number of runs had 3 attempts converge on the same answer, pushing max vote share above 40% — in which case the default path ran. On this problem, that sometimes produced the correct answer and sometimes didn't.
\nTwo things follow from the hard flag: a scoring-rule change and a prompt-strategy change.
\nFor completeness: the default (low-diversity) path uses plurality voting with inverse-entropy tiebreak. The high-diversity path switches to logprob-weighted scoring, borrowed from the ARC Prize 2024 winning solution, with a median cap that prevents a single very-confident attempt from dominating. In the 20 runs I have on the n-Norwegian problem, the logprob-weighted rule selected 8687 in 14/20. One caveat: these 20 runs are partially cherry-picked — during development I didn't consistently save outputs from runs that didn't produce 8687, so the 14/20 figure likely over-represents favorable runs, and the true selection rate on the final solver version is probably lower.\nI didn't run paired ablations isolating the scoring rule from the rest of the system, so I can't attribute a specific accuracy delta to it. The stronger contribution to improving the leaderboard score was on the prompt engineering side.
\nThe more impactful change was what the model saw in its prompt. A lightweight fine-tuned classifier (bge-base-en-v1.5, CPU-resident) categorizes the problem into one of five types: algebra, geometry, number_theory, combinatorics, analysis. When classifier confidence is ≥0.90, I inject a category-specific strategy hint into the user message.
\nThese aren't generic \"try harder\" instructions, they are concrete mathematical guidance that competition problems of that type actually reward. A sample from the number-theory hint:\nMODULAR EXPONENTIATION — Use the three-argument form pow(base, exp, mod) for all modular powers. CORRECT: pow(2, 1018, 109 + 7) WRONG: 2(1018) % (10**9 + 7) # allocates an astronomically large integer\nDIOPHANTINE EQUATIONS — Before writing any loop, reduce the equation modulo a small number (try 3, 4, and 8 in that order) to eliminate impossible residue classes and shrink the search space to a tractable size.\nGeometry gets a different hint emphasizing coordinate/complex representations and a warning to avoid sympy.geometry (which can hang). Combinatorics gets guidance on functools.lru_cache for DP and strict iteration limits. Analysis gets domain-restriction reminders.\nThe hints are specific enough to cite in code. They live in CFG.TYPE_STRATEGY_HINTS as ~200-word blocks. When classifier confidence drops below 0.90, the system falls back to the default pool — which empirically worked well on the easy problems.
\nI observed a +3 improvement on the public leaderboard after adding the classifier-routed hints — from 36 to 39. That's a single data point across two submissions, not a controlled comparison: other changes shipped in the same submission window, and individual problem-grading outcomes can shift by 1–2 on any given run. I can't confidently attribute the full +3 to strategy hints, but the direction was consistent with development-time observations: more 3-consecutive-unanimous early stops on hard problems after the hints went in, fewer context_exhausted NAs, more runs finishing inside the per-problem budget.
\nThe stronger qualitative signal was at the attempt level rather than the leaderboard level. With hints enabled, attempts that converged were more likely to converge on the same answer, reducing the vote spread that forced the high-diversity path in the first place. In effect, the dynamically generated hints narrowed the range of the model's reasoning paths — attempts landed closer to each other, which made aggregation more reliable downstream.
\nNo ablation. The leaderboard delta conflates hints with whatever else shipped in that submission. To attribute the gain cleanly I'd need runs on a fixed subset of the reference problems with and without hints, holding everything else constant.
\nThe hints could be improved. They reflect my intuitions about what each problem type rewards, based on manually inspecting attempts that failed on the reference set. Someone with deeper mathematical expertise might write better ones; a systematic way to generate or tune them is an open direction.\nThe classifier is small. It was fine-tuned on a small labeled dataset; categorization accuracy on held-out problems wasn't rigorously measured. The 0.90-confidence threshold is what I used empirically, not what a held-out validation curve would suggest.\n\"Easy vs hard\" is a blunt split. The 40% vote-share threshold is a crude binary. In principle difficulty is continuous, and a more graded routing (e.g., linearly interpolating between scoring rules) might do better.
High answer diversity turned out to be a reasonable indicator that the model was struggling with a problem. Easy problems converge on one answer within a few completions; hard problems typically fragment into 5–8 distinct integers. It's not a perfect detector — on some runs a few attempts converge early and bypass the hard-problem path even on genuinely hard problems — but the correlation was strong enough to be a useful routing input. The signal is available as soon as a few attempts complete, well before the full budget is spent, and it lets the solver route strategy without needing to know problem category, difficulty, or correctness ahead of time. The 40% vote-share threshold is a stepping stone; a more graded routing is an obvious next step.
\nThese did more work than the scoring rule on top of them. The hints aren't generic scaffolding — they're concrete mathematical guidance that rewards the specific failure modes each problem type exhibits: pow(base, exp, mod) for number theory, coordinate geometry over sympy.geometry, functools.lru_cache for combinatorial DP, domain restrictions for analysis. Manual curation was tractable for five categories but doesn't scale; a systematic way to generate or mine such hints from olympiad solution archives seems worth trying.
\nI spent meaningful time adjusting attempts per problem, worker pool size, batch size, GPU memory utilization, context window, early-stop thresholds, temperature schedules and the results of any individual change were usually hard to distinguish from run-to-run variance. Settings that appeared worse in one submission sometimes scored better in the next, and vice versa. The values in the final submission (8 attempts, 16 workers, 65K context, 0.955 GPU memory utilization) are the ones that felt stable across several consecutive runs, not ones I can claim are optimal. On a capable model with a 5-hour budget, the parameter space is large, submission noise is real, and without a rigorous evaluation loop (see Future Directions), informal tuning didn't do much to improve the leaderboard score.
\nLaunching 4 additional attempts when high diversity was detected did not score high on the public leaderboard. In theory, spending more compute on problems flagged as hard should raise accuracy. In practice, the launch condition required a larger time window that rarely materialized within the 5-hour budget; the feature fired on only a small fraction of problems, and when it did I couldn't distinguish its effect from noise.
\nFirst, a small ablation matrix isolating hints, classifier routing, and scoring rule independently. For 10 problems × 5 runs, this would resolve most of the attribution questions in this writeup. Second, stronger statistical evaluation on a per-problem basis. A single problem at n=20 runs barely distinguishes 70% success from chance (p≈0.058). Scaling to ≥30 runs per problem across all five types — roughly 6 hours of H100 time per evaluation — would produce attribution claims that hold up under peer review. Third, a more systematic way to construct strategy hints. For example, mining them from the reasoning traces of successful runs on each category and distilling the common patterns into hint blocks.
\nCompute. All experiments ran inside the Kaggle AIMO-3 competition kernel: a single NVIDIA H100 80GB GPU with a 5-hour time limit per notebook. The reasoning model (gpt-oss-120B) was served via vLLM on the GPU; the problem-type classifier (fine-tuned bge-base-en-v1.5) ran on CPU to avoid competing for GPU memory.
\nServing. gpt-oss-120B was served through vLLM's OpenAI-compatible completions endpoint with prefix caching enabled, 65,536-token context window, fp8 KV cache, and up to 256 concurrent sequences. Per-problem inference fanned out to 8 attempts running in parallel over a pool of 16 persistent Jupyter kernels (for tool-use Python execution), with per-attempt timeouts and a dynamic per-problem budget scaled to fit 50 problems within the overall 5-hour limit.
\nThe original base notebook score of 42 on the public leaderboard\nhttps://www.kaggle.com/code/andreasbis/aimo-3-gpt-oss-120b-with-tools
\nHighest score of 44 on the public leaderboard improved the original base notebook by andreasbis through prompt engineering by nihilisticneuralnet\nhttps://www.kaggle.com/code/nihilisticneuralnet/44-50-let-me-over-cook
\nThanks to the competition hosts and all the participants!
\nOverview about AIMO3\nhttps://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3/overview
n-Norwegian Problem 10 from AIMO3_Reference_Problems.pdf\nhttps://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3/data?select=AIMO3_Reference_Problems.pdf
Inference reference dataset\nhttps://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3/data?select=reference.csv
Adding to the comments by @sambealing below, I have two additional questions:
\nClassifiers: Did you experiment with different types of classifiers, such as FastText or simple Regex-based classifiers, or different injection prompts? If so, it would be valuable to share the results with the community. As Sam mentioned, even if the experiments are not fully documented, it would still be helpful to describe what you tried and what you observed.
Reproducibility: It would be great if you could share the code used to train the classifiers and construct the pipeline, so others can reproduce them from scratch or adapt them for their own systems.
Hi, thank you for your submission to the Writeup Prize. @philipvonderlind and I will post some questions below that help us in assessing your submission more closely, as described in the updated discussion post.
\nLook forward to reading your responses on these.
", "rawMarkdown": "Hi, thank you for your submission to the Writeup Prize. @philipvonderlind and I will post some questions below that help us in assessing your submission more closely, as described in the [updated discussion post](https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-3/discussion/689703).\n\n1. **Which changes mattered most:** I appreciate you didn't do a careful step-through to isolate each change. But across the various components you added, do you have a rough feel for which of them moved the needle and which were closer to noise?\n2. **Is detecting hard problems useful:** Your diversity rule (unique answers > 4 AND max_vote_share < 0.40) flags problems where the 8 attempts have fragmented across many distinct answers. But if the distribution looks like that, it seems the model is unlikely to be close to the correct answer regardless of which scoring rule or hints you switch to. What's the evidence that the post-detection interventions actually recover correctness on those hard problems, rather than just picking a different wrong answer?\n\nLook forward to reading your responses on these.\n" } ], "complete": true, "source": "live", "fetched_at": "2026-10-20T04:23:27Z", "competition_id": 118448, "competition_slug": "ai-mathematical-olympiad-progress-prize-3", "team_id": 14812158, "solution_url": "/competitions/ai-mathematical-olympiad-progress-prize-3/writeups/detecting-hard-problems-by-answer-diversity" }