![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7385153%2F0931de09fd02a26e35246c9c2278422b%2F2025-04-14%20%2021.47.04.png?generation=1744656490623805&alt=media\")
Notebook: https://www.kaggle.com/code/mavicbf/3rd-place-solution-aliev
\nModel: https://www.kaggle.com/models/shelterw/deepseek-r1/Transformers/deepseek-r1-distill-qwen-14b-awq/1
First of all, I’d like to thank Aimo and Kaggle for organizing this competition, and all the participants who shared their insights and ideas throughout. Thanks to you, I learned a lot and had an incredible amount of fun during the entire journey.
\nAlthough my solution was relatively simple, I ended up securing 3rd place.
\nPublic set score: 25/50
\nPrivate set score: 30/50
No additional training, no DPO, GRPO, or anything like that — I just took the most popular model (possibly after the 7B version) used in open solutions — Deepseek-R1-Distill-Qwen-14B with AWQ quantization by ShelterW.
\nWhile running the model on the 10 validation problems, I was really annoyed by the fact that during the solving of simple tasks, where most solution branches usually lead to the same answer, we spend way too much time. I’d really like to dynamically control the moment when further generation becomes pointless. This is especially relevant in cases where 9 out of 10 solutions are already correct, and the last one gets stuck in a loop and keeps generating until the final token.
I also followed the following intuition - most likely, the closer we are to the beginning of the solution, the more common ground there is between all solution branches. It seems that the probability of hallucinating increases as the generation progresses. Based on my experience manually reviewing the validation problems, the first ~4096 tokens don’t differ much from each other in any fundamental way.
So let’s get to the idea – based on the motivation I implemented the following algorithm:
\n1) We start with 5 solution branches and generate the first 4096 tokens for each.
\n2) Then we duplicate each of the branches (so we’ll have 10 solutions in the next iteration) and generate another 4096 tokens.
\n3) At this point, it's likely that we already have enough solutions to make a decision, so I implemented the following rule:
\n - If after 8192 tokens we already have more than 6 completed solutions (detected using a regular expression for \\boxed{}) and one of the answers makes up more than 70% of all, we stop the generation. Indeed, the probability that the model will produce a different answer from this point on is quite low, and we can save valuable tokens (especially considering the increasing computational complexity of attention with longer context).
\n - If not, we move to step 4.
\n4) Since duplicating every solution would result in 20 branches, which is too much, I randomly select 7 unfinished branches, duplicate them - resulting in 14 branches - and generate the next 4096 tokens.
\n5) We then collect all of the solutions and apply a majority vote.
Since in the worst case we’d have to recompute the same KV cache (even twice) — it’s important to set the flag enable_prefix_caching=True in vllm.LLM.
This way, we can generate 14 solutions of up to 12k tokens + those 0-3 branches that might have reached an answer at 8k tokens (in fact, even at 4k tokens, the code often remains the same, but I’m omitting that detail due to how rarely such a case occurs and for the sake of simplicity). I didn’t collect full statistics, but based on my observations, this usually takes about 6–7 minutes. Considering that we’ll often get early answers, on average we should be within the time limits.
\nThis optimization isn’t without flaws — for example, the solutions will still be somewhat correlated due to the shared prefix. However, it seems this isn’t too critical and partially condensed by different prompts across the branches.
\nAlso, if we're running out of time (based on the cutoff_times array), we start with 6 branches, duplicate them after 4096 tokens, and stop at 8192 tokens.
All extra details (temperature, prompts and so on) you can check on the notebook.
\nMy score was partly due to luck, however, the core idea seems interesting, and I’d be happy to see it further developed.
\nI’ll be glad to answer any questions you may have, so feel free to ask!
", "comments": [ { "id": 3180054, "author_name": "", "author_url": "", "post_date": "2025-04-16T04:19:34.977000", "content": "Most chad notebook in this competition. 30 without any fine tuning and complex algorithms and stuff is really impressive.
\nRespect.
Congratulations on repeating the feat in the previous competition - getting 3rd place with without finetuning a model
\nJust curious, could you share the exact notebook that you made the final submission with - the one that displays Private Score 30?
We, as a community, need to carry this wonderful tradition of winning 3rd place without fine tuning on to AIMO3 😂
", "votes": 7, "replies": [ { "id": 3459801, "author_name": "Chan Kha Vu", "author_url": "", "post_date": "2026-05-18T10:10:41.157000", "content": "This aged well
", "votes": 0, "replies": [] } ] }, { "id": 3180144, "author_name": "Chew Kok Wah", "author_url": "", "post_date": "2025-04-16T07:52:13.173000", "content": "Winner did not open source his final winning Submission notebook as required in the Rules (he only share a self-claimed cleaned version of another notebook), and there is no response after over 32 hours to public request on open sourcing the actual winning Submission Notebook.
\nIs this following AIMO rules? Isn't it the Submission where Winner is supposed to open source refer to the ACTUAL notebook that was submitted for Competition, not another self-claimed cleaned notebook?
\nShould AIMO mandate that all Winners open source their Actual Winning final Submission notebook for transparency and avoidance of speculation?
\nEdit:
\n20 minutes after my post, this Winner had open sourced the actual Submission.
I know when the deadlines for uploading all the requirements are - I uploaded the cleaned code and text solution early to share with people, and not for you to count down the hours from the moment of publication. Let's respect someone else's time. The original code is already available, so enjoy watching it.
", "votes": 5, "replies": [] } ] }, { "id": 3180153, "author_name": "Aznaur Aliev", "author_url": "", "post_date": "2025-04-16T08:13:51.910000", "content": "Thx for congrats, nice predictions btw :)
\noriginal notebook in final submission - https://www.kaggle.com/code/mavicbf/notebook939675eb41
", "votes": 3, "replies": [] } ] }, { "id": 3196619, "author_name": "Namu Desu", "author_url": "", "post_date": "2025-05-07T08:26:36.410000", "content": "A simple solution that I can actually understand in one read, this is really a smart approach, congratulations!
", "votes": 1, "replies": [] }, { "id": 3179213, "author_name": "john", "author_url": "", "post_date": "2025-04-15T07:12:28.867000", "content": "The idea of duplication is interesting. I have a question about it.
\nFor these duplicated prompts in Step 2 and Step 3, what's the probability that they will give the same answer (no matter if correct or wrong)?
I happent to step across this. And its an amazing simple result derived (I think) of a lot of work. Thanks for sharing
", "votes": 0, "replies": [] }, { "id": 3194555, "author_name": "Magicyu93", "author_url": "", "post_date": "2025-05-06T03:43:58.230000", "content": "This is really interesting! Congratulations!
", "votes": 0, "replies": [] }, { "id": 3183807, "author_name": "JP", "author_url": "", "post_date": "2025-04-21T10:21:13.963000", "content": "Congratulations on reaching third place @mavicbf! This was an impressive result considering the fact that you used an off-the-shelf model without any further fine-tuning, and it's quite encouraging for those of us who are GPU poor 😅.
\nOne thing I was wondering is whether you checked the score variability of your approach? Was it relatively stable on the public set? All of the public notebooks (as far as I'm aware) were quite unreliable in producing a stable public LB score. What I suspected is that, even though Deepseek models tend to be a little unreliable for some problems (some times they find the correct solution and other times they don't), the main issue was that since the problems were being served in random order, if for one submission, you just got all the \"easier\" problems at the beginning, then the model would score higher in the public LB, but if you got harder problems in the beginning, then the model would run out of time towards the end, resulting in a lower score. Therefore, I think finding a reliable way to stop generation early (or even choosing when to skip a problem) was key, and I was wondering whether your approach managed to achieve this to some degree.
", "votes": 0, "replies": [] }, { "id": 3180444, "author_name": "", "author_url": "", "post_date": "2025-04-16T16:21:19.487000", "content": "Heartful Congratulations!, That's Amazing# 🙌
", "votes": 0, "replies": [] }, { "id": 3179715, "author_name": "", "author_url": "", "post_date": "2025-04-15T16:05:31.633000", "content": "Thanks for sharing It is good solution
", "votes": 1, "replies": [] } ], "comment_count": 14, "messages": [ { "id": 3178919, "postDate": "2025-04-14T19:25:38.100Z", "content": "Notebook: https://www.kaggle.com/code/mavicbf/3rd-place-solution-aliev
\nModel: https://www.kaggle.com/models/shelterw/deepseek-r1/Transformers/deepseek-r1-distill-qwen-14b-awq/1
First of all, I’d like to thank Aimo and Kaggle for organizing this competition, and all the participants who shared their insights and ideas throughout. Thanks to you, I learned a lot and had an incredible amount of fun during the entire journey.
\nAlthough my solution was relatively simple, I ended up securing 3rd place.
\nPublic set score: 25/50
\nPrivate set score: 30/50
No additional training, no DPO, GRPO, or anything like that — I just took the most popular model (possibly after the 7B version) used in open solutions — Deepseek-R1-Distill-Qwen-14B with AWQ quantization by ShelterW.
\nWhile running the model on the 10 validation problems, I was really annoyed by the fact that during the solving of simple tasks, where most solution branches usually lead to the same answer, we spend way too much time. I’d really like to dynamically control the moment when further generation becomes pointless. This is especially relevant in cases where 9 out of 10 solutions are already correct, and the last one gets stuck in a loop and keeps generating until the final token.
I also followed the following intuition - most likely, the closer we are to the beginning of the solution, the more common ground there is between all solution branches. It seems that the probability of hallucinating increases as the generation progresses. Based on my experience manually reviewing the validation problems, the first ~4096 tokens don’t differ much from each other in any fundamental way.
So let’s get to the idea – based on the motivation I implemented the following algorithm:
\n1) We start with 5 solution branches and generate the first 4096 tokens for each.
\n2) Then we duplicate each of the branches (so we’ll have 10 solutions in the next iteration) and generate another 4096 tokens.
\n3) At this point, it's likely that we already have enough solutions to make a decision, so I implemented the following rule:
\n - If after 8192 tokens we already have more than 6 completed solutions (detected using a regular expression for \\boxed{}) and one of the answers makes up more than 70% of all, we stop the generation. Indeed, the probability that the model will produce a different answer from this point on is quite low, and we can save valuable tokens (especially considering the increasing computational complexity of attention with longer context).
\n - If not, we move to step 4.
\n4) Since duplicating every solution would result in 20 branches, which is too much, I randomly select 7 unfinished branches, duplicate them - resulting in 14 branches - and generate the next 4096 tokens.
\n5) We then collect all of the solutions and apply a majority vote.
Since in the worst case we’d have to recompute the same KV cache (even twice) — it’s important to set the flag enable_prefix_caching=True in vllm.LLM.
This way, we can generate 14 solutions of up to 12k tokens + those 0-3 branches that might have reached an answer at 8k tokens (in fact, even at 4k tokens, the code often remains the same, but I’m omitting that detail due to how rarely such a case occurs and for the sake of simplicity). I didn’t collect full statistics, but based on my observations, this usually takes about 6–7 minutes. Considering that we’ll often get early answers, on average we should be within the time limits.
\nThis optimization isn’t without flaws — for example, the solutions will still be somewhat correlated due to the shared prefix. However, it seems this isn’t too critical and partially condensed by different prompts across the branches.
\nAlso, if we're running out of time (based on the cutoff_times array), we start with 6 branches, duplicate them after 4096 tokens, and stop at 8192 tokens.
All extra details (temperature, prompts and so on) you can check on the notebook.
\nMy score was partly due to luck, however, the core idea seems interesting, and I’d be happy to see it further developed.
\nI’ll be glad to answer any questions you may have, so feel free to ask!
", "rawMarkdown": "**Notebook**: https://www.kaggle.com/code/mavicbf/3rd-place-solution-aliev\n**Model**: https://www.kaggle.com/models/shelterw/deepseek-r1/Transformers/deepseek-r1-distill-qwen-14b-awq/1\n \nFirst of all, I’d like to thank Aimo and Kaggle for organizing this competition, and all the participants who shared their insights and ideas throughout. Thanks to you, I learned a lot and had an incredible amount of fun during the entire journey.\n\nAlthough my solution was relatively simple, I ended up securing **3rd** place.\n**Public set score: 25/50**\n**Private set score: 30/50**\n\n \n## 1. Model\nNo additional training, no DPO, GRPO, or anything like that — I just took the most popular model (possibly after the 7B version) used in open solutions — *Deepseek-R1-Distill-Qwen-14B* with AWQ quantization by *ShelterW*.\n \n## 2. Motivation\n- While running the model on the 10 validation problems, I was really annoyed by the fact that during the solving of simple tasks, where most solution branches usually lead to the same answer, we spend way too much time. I’d really like to dynamically control the moment when further generation becomes pointless. This is especially relevant in cases where 9 out of 10 solutions are already correct, and the last one gets stuck in a loop and keeps generating until the final token.\n \n- I also followed the following intuition - most likely, the closer we are to the beginning of the solution, the more common ground there is between all solution branches. It seems that the probability of hallucinating increases as the generation progresses. Based on my experience manually reviewing the validation problems, the first ~4096 tokens don’t differ much from each other in any fundamental way.\n \n## 3. The CORE part of the solution:\n \nSo let’s get to the idea – based on the motivation I implemented the following algorithm: \n \n \n1) We start with 5 solution branches and generate the first 4096 tokens for each. \n2) Then we duplicate each of the branches (so we’ll have 10 solutions in the next iteration) and generate another 4096 tokens. \n3) At this point, it's likely that we already have enough solutions to make a decision, so I implemented the following rule: \n - If after 8192 tokens we already have more than 6 completed solutions (detected using a regular expression for `\\boxed{}`) and one of the answers makes up more than 70% of all, we stop the generation. Indeed, the probability that the model will produce a different answer from this point on is quite low, and we can save valuable tokens (especially considering the increasing computational complexity of attention with longer context). \n - If not, we move to step 4. \n4) Since duplicating every solution would result in 20 branches, which is too much, I randomly select 7 unfinished branches, duplicate them - resulting in 14 branches - and generate the next 4096 tokens. \n5) We then collect all of the solutions and apply a majority vote.\n\n\n\n \nSince in the worst case we’d have to recompute the same KV cache (even twice) — it’s important to set the flag `enable_prefix_caching=True` in `vllm.LLM`.\n \nThis way, we can generate 14 solutions of up to 12k tokens + those 0-3 branches that might have reached an answer at 8k tokens (in fact, even at 4k tokens, the code often remains the same, but I’m omitting that detail due to how rarely such a case occurs and for the sake of simplicity). I didn’t collect full statistics, but based on my observations, this usually takes about 6–7 minutes. Considering that we’ll often get early answers, on average we should be within the time limits.\n \nThis optimization isn’t without flaws — for example, the solutions will still be somewhat correlated due to the shared prefix. However, it seems this isn’t too critical and partially condensed by different prompts across the branches.\n \nAlso, if we're running out of time (based on the `cutoff_times` array), we start with 6 branches, duplicate them after 4096 tokens, and stop at 8192 tokens.\n\nAll extra details (temperature, prompts and so on) you can check on the notebook.\n\nMy score was partly due to luck, however, the core idea seems interesting, and I’d be happy to see it further developed.\n\nI’ll be glad to answer any questions you may have, so feel free to ask!", "votes": 58 }, { "id": 3180054, "postDate": "2025-04-16T04:19:34.977Z", "content": "Most chad notebook in this competition. 30 without any fine tuning and complex algorithms and stuff is really impressive.
\nRespect.
Congratulations on repeating the feat in the previous competition - getting 3rd place with without finetuning a model
\nJust curious, could you share the exact notebook that you made the final submission with - the one that displays Private Score 30?
We, as a community, need to carry this wonderful tradition of winning 3rd place without fine tuning on to AIMO3 😂
", "rawMarkdown": "We, as a community, need to carry this wonderful tradition of winning 3rd place without fine tuning on to AIMO3 😂", "votes": 7, "replies": [ { "id": 3459801, "postDate": "2026-05-18T10:10:41.157Z", "content": "This aged well
", "rawMarkdown": "This aged well" } ] }, { "id": 3180144, "postDate": "2025-04-16T07:52:13.173Z", "content": "Winner did not open source his final winning Submission notebook as required in the Rules (he only share a self-claimed cleaned version of another notebook), and there is no response after over 32 hours to public request on open sourcing the actual winning Submission Notebook.
\nIs this following AIMO rules? Isn't it the Submission where Winner is supposed to open source refer to the ACTUAL notebook that was submitted for Competition, not another self-claimed cleaned notebook?
\nShould AIMO mandate that all Winners open source their Actual Winning final Submission notebook for transparency and avoidance of speculation?
\nEdit:
\n20 minutes after my post, this Winner had open sourced the actual Submission.
I know when the deadlines for uploading all the requirements are - I uploaded the cleaned code and text solution early to share with people, and not for you to count down the hours from the moment of publication. Let's respect someone else's time. The original code is already available, so enjoy watching it.
", "rawMarkdown": "I know when the deadlines for uploading all the requirements are - I uploaded the cleaned code and text solution early to share with people, and not for you to count down the hours from the moment of publication. Let's respect someone else's time. The original code is already available, so enjoy watching it.", "votes": 5 } ] }, { "id": 3180153, "postDate": "2025-04-16T08:13:51.910Z", "content": "Thx for congrats, nice predictions btw :)
\noriginal notebook in final submission - https://www.kaggle.com/code/mavicbf/notebook939675eb41
", "rawMarkdown": "Thx for congrats, nice predictions btw :)\n\noriginal notebook in final submission - https://www.kaggle.com/code/mavicbf/notebook939675eb41", "votes": 3 } ] }, { "id": 3196619, "postDate": "2025-05-07T08:26:36.410Z", "content": "A simple solution that I can actually understand in one read, this is really a smart approach, congratulations!
", "rawMarkdown": "A simple solution that I can actually understand in one read, this is really a smart approach, congratulations!", "votes": 1 }, { "id": 3179213, "postDate": "2025-04-15T07:12:28.867Z", "content": "The idea of duplication is interesting. I have a question about it.
\nFor these duplicated prompts in Step 2 and Step 3, what's the probability that they will give the same answer (no matter if correct or wrong)?
I happent to step across this. And its an amazing simple result derived (I think) of a lot of work. Thanks for sharing
", "rawMarkdown": "I happent to step across this. And its an amazing simple result derived (I think) of a lot of work. Thanks for sharing" }, { "id": 3194555, "postDate": "2025-05-06T03:43:58.230Z", "content": "This is really interesting! Congratulations!
", "rawMarkdown": "This is really interesting! Congratulations! " }, { "id": 3183807, "postDate": "2025-04-21T10:21:13.963Z", "content": "Congratulations on reaching third place @mavicbf! This was an impressive result considering the fact that you used an off-the-shelf model without any further fine-tuning, and it's quite encouraging for those of us who are GPU poor 😅.
\nOne thing I was wondering is whether you checked the score variability of your approach? Was it relatively stable on the public set? All of the public notebooks (as far as I'm aware) were quite unreliable in producing a stable public LB score. What I suspected is that, even though Deepseek models tend to be a little unreliable for some problems (some times they find the correct solution and other times they don't), the main issue was that since the problems were being served in random order, if for one submission, you just got all the \"easier\" problems at the beginning, then the model would score higher in the public LB, but if you got harder problems in the beginning, then the model would run out of time towards the end, resulting in a lower score. Therefore, I think finding a reliable way to stop generation early (or even choosing when to skip a problem) was key, and I was wondering whether your approach managed to achieve this to some degree.
", "rawMarkdown": "Congratulations on reaching third place @mavicbf! This was an impressive result considering the fact that you used an off-the-shelf model without any further fine-tuning, and it's quite encouraging for those of us who are GPU poor 😅.\n\nOne thing I was wondering is whether you checked the score variability of your approach? Was it relatively stable on the public set? All of the public notebooks (as far as I'm aware) were quite unreliable in producing a stable public LB score. What I suspected is that, even though Deepseek models tend to be a little unreliable for some problems (some times they find the correct solution and other times they don't), the main issue was that since the problems were being served in random order, if for one submission, you just got all the \"easier\" problems at the beginning, then the model would score higher in the public LB, but if you got harder problems in the beginning, then the model would run out of time towards the end, resulting in a lower score. Therefore, I think finding a reliable way to stop generation early (or even choosing when to skip a problem) was key, and I was wondering whether your approach managed to achieve this to some degree." }, { "id": 3180444, "postDate": "2025-04-16T16:21:19.487Z", "content": "Heartful Congratulations!, That's Amazing# 🙌
", "rawMarkdown": "Heartful Congratulations!, That's Amazing**# 🙌**", "isDeleted": true }, { "id": 3179715, "postDate": "2025-04-15T16:05:31.633Z", "content": "Thanks for sharing It is good solution
", "rawMarkdown": "Thanks for sharing It is good solution ", "votes": 1 } ], "complete": true, "source": "live", "fetched_at": "2026-10-20T06:43:11Z", "competition_id": 86023, "competition_slug": "ai-mathematical-olympiad-progress-prize-2", "team_id": 13374982, "solution_url": "/competitions/ai-mathematical-olympiad-progress-prize-2/writeups/aliev-3rd-place-solution-report" }