I was surprised that my \"just give it a try\" submission made within a week ended up in the gold zone for this competition. Though this is pure luck (public 23 -> private 29), I would like to share what I learned from this competition.
\nIn this competition, we were given L4x4 GPUs for inference. The good is we didn’t have to suffer with the ancient T4s in 2025. The bad is L4s aren't ideal for LLM decoding either—they only provide 300GB/s on paper memory bandwidth each, totaling 1200GB/s with 4 GPUs.
\nLong-context decoding (for reasoning) and batched decoding (for majority voting) were both important for this task. Under this kind of inference settings, as context length grows, KV cache can become the main bottleneck (larger than the model weights), further increasing the demand on GPU bandwidth.
\nI tested the inference speeds of LMDeploy, SGLang, and vLLM. With 4-bit quantized weights and fp16 KV cache, the speed ranking was: LMDeploy > SGLang > vLLM v1 >> vLLM.
\nLMDeploy supports int8/int4 KV cache quantization, while SGLang and vLLM offer FP8 KV cache quantization. Using KV cache quantization enables larger decoding batch size within the same time budget.
\nUnfortunately, int4 KV cache quantization caused too much accuracy loss, so I ended up choosing LMDeploy with int8 KV cache quantization for inference.
Before submission, I profiled inference time of decoding to 16k tokens (worst case scenario) under different batch sizes. During the actual submission, I tracked the actual inference time for each question, then add planned time - actual time into the time budget for the next question. Batch size for the next question was selected on that new time budget and my earlier profiling results.
This is a fairly conservative strategy, using smaller batch sizes for first received questions. And I didn’t profile for longer max decoding lengths, so there’s room for improvement.
\nThe inference notebook is public here. Simply a rushed implementation using casperhansen/deepseek-r1-distill-qwen-14b-awq and my submission strategy. I haven’t done enough testing so there might be bugs.
Gz! Your submission strategy, dynamically adjusting batch size based on real-time inference performance and remaining time budget, is quite clever and resource-conscious. It's a practical approach to maximize throughput within the competition's constraints.
", "votes": 1, "replies": [] }, { "id": 3176417, "author_name": "Tong Hui Kang", "author_url": "", "post_date": "2025-04-11T10:23:12.353000", "content": "Apparently you only had 1 public LB submission?
\nhttps://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-2/discussion/572784
", "votes": 1, "replies": [ { "id": 3176423, "author_name": "Wanderer", "author_url": "", "post_date": "2025-04-11T10:28:17.673000", "content": "This is what he does.😂 He often joins competitions near the deadline, submits only a few times, and then ranks very high on the private leaderboard.
", "votes": 3, "replies": [] }, { "id": 3176436, "author_name": "john", "author_url": "", "post_date": "2025-04-11T10:49:01.060000", "content": "Take a look at eedi 72nd place. I also joined eedi, so i remember this. Habit? 😀
\nhttps://www.kaggle.com/competitions/eedi-mining-misconceptions-in-mathematics/leaderboard
Congratulations on 7th place in this competition.
\n I also tried 14b model couple of times with vLLM, however, I got Notebook Timeout.
\nWhats your opinion for the model update from 7b to 14b.
I think validation should be done like this:
\nany, majority, and average correct.any, majority, and average correct. Repeat this process many times and check the stats.The 7B model might perform not bad on any, but the 14B model have much better majority and average. The 7B model has a higher risk of overfitting to the public leaderboard.
With 4-bit quantized weights and fp16 KV cache, the speed ranking was: LMDeploy > SGLang > vLLM v1 >> vLLM.
\nCan you show us the comparison results?
Congrats and thank you for sharing!
\nNow not only the training optimization, but the inference speed optimization becomes more important.
Congrats! I tried 14b with lmdeploy and int8 kv quant once and got 26 on the public LB. The performance degradation seem to be minimal. vLLM's fp8 kv quant on the other hand gave me gibberish (could have been a bug in the version I installed).
", "votes": 2, "replies": [] }, { "id": 3182950, "author_name": "Sarah Arshad", "author_url": "", "post_date": "2025-04-20T06:40:10.260000", "content": "Congratulations on your well-deserved success…. Thanks for inspiring us with your hard work and dedication.
", "votes": 0, "replies": [] }, { "id": 3176323, "author_name": "Optimo", "author_url": "", "post_date": "2025-04-11T08:39:08.860000", "content": "Amazingly simple! Congrats!
\nWhat was your public score ?
", "votes": 0, "replies": [ { "id": 3176387, "author_name": "tascj", "author_url": "", "post_date": "2025-04-11T09:52:39.547000", "content": "It was 23. That's why I call it pure luck.
", "votes": 1, "replies": [ { "id": 3176481, "author_name": "Optimo", "author_url": "", "post_date": "2025-04-11T11:52:17.590000", "content": "The private LB did not seem too random to me, you had a strong solution! Congrats!
", "votes": -2, "replies": [] }, { "id": 3176869, "author_name": "Man of the year", "author_url": "", "post_date": "2025-04-11T20:22:45.220000", "content": "I've seen that you usually win medals with just 1-2 submissions. Could you please share any tips about your strategy or advice? How do you do this?
\nPersonally, I cannot really call it \"luck\", when you were able to repeat this so many times.
", "votes": 2, "replies": [] }, { "id": 3178643, "author_name": "tascj", "author_url": "", "post_date": "2025-04-14T12:30:12.200000", "content": "\"Luck\" is specifically referring to this competition.
\nFor me, making fewer submissions helps me better balance work-Kaggle-life. I enjoy problem solving but don't enjoy leaderboard climbing much. I usually have a first-day plan, and even without submitting, there are many interesting things to experiment with. Regardless of the leaderboard score, 'executing the plan'' has been very rewarding for me.
\nThe strategy of 'only submitting in the final days' has led to many failures for me in past competitions. If you're concerned about final ranking, making more submissions is the better approach.
", "votes": 7, "replies": [] }, { "id": 3178917, "author_name": "Man of the year", "author_url": "", "post_date": "2025-04-14T19:22:35.803000", "content": "Thank you very much, your approach is very cool. I also wanted to ask you about your general validation strategy. As I understood, during your experiments you rely on your validation score. How do you create your validation set? Sometimes, local validation might not correlate well with the public leaderboard and some competitions have this domain shift. How do you deal with such competitions?
", "votes": 0, "replies": [] }, { "id": 3180152, "author_name": "Chan Kha Vu", "author_url": "", "post_date": "2025-04-16T08:08:35.643000", "content": "Exactly, whenever I see @tascj0 in the gold zone solo, they usually have less than 10 subs
", "votes": 0, "replies": [] }, { "id": 3181581, "author_name": "tascj", "author_url": "", "post_date": "2025-04-18T04:21:05.223000", "content": "@manwithaflower
\nI don't have any special validation strategies. I fail in trust LB competitions.
As long as local validation enough for the my planned experiments, I'm comfortable with it. What I planned to do through a competition might not even be related to LB.
", "votes": 3, "replies": [] } ] } ] }, { "id": 3176320, "author_name": "Quickpanda", "author_url": "", "post_date": "2025-04-11T08:37:07.573000", "content": "Thanks for sharing — your solution looks very clean, and the budget planning seems well thought out!
", "votes": 0, "replies": [] }, { "id": 3178146, "author_name": "", "author_url": "", "post_date": "2025-04-13T19:27:49.460000", "content": "", "votes": 0, "replies": [] }, { "id": 3176364, "author_name": "", "author_url": "", "post_date": "2025-04-11T09:32:06.237000", "content": "", "votes": 0, "replies": [] }, { "id": 3180408, "author_name": "Palvinder", "author_url": "", "post_date": "2025-04-16T15:29:02.453000", "content": "Congratulations! Thank you for sharing!
", "votes": 0, "replies": [] }, { "id": 3179711, "author_name": "", "author_url": "", "post_date": "2025-04-15T16:02:41.577000", "content": "Thanks for wonderful solutions
", "votes": 0, "replies": [] } ], "comment_count": 23, "messages": [ { "id": 3176314, "postDate": "2025-04-11T08:24:01.080Z", "content": "I was surprised that my \"just give it a try\" submission made within a week ended up in the gold zone for this competition. Though this is pure luck (public 23 -> private 29), I would like to share what I learned from this competition.
\nIn this competition, we were given L4x4 GPUs for inference. The good is we didn’t have to suffer with the ancient T4s in 2025. The bad is L4s aren't ideal for LLM decoding either—they only provide 300GB/s on paper memory bandwidth each, totaling 1200GB/s with 4 GPUs.
\nLong-context decoding (for reasoning) and batched decoding (for majority voting) were both important for this task. Under this kind of inference settings, as context length grows, KV cache can become the main bottleneck (larger than the model weights), further increasing the demand on GPU bandwidth.
\nI tested the inference speeds of LMDeploy, SGLang, and vLLM. With 4-bit quantized weights and fp16 KV cache, the speed ranking was: LMDeploy > SGLang > vLLM v1 >> vLLM.
\nLMDeploy supports int8/int4 KV cache quantization, while SGLang and vLLM offer FP8 KV cache quantization. Using KV cache quantization enables larger decoding batch size within the same time budget.
\nUnfortunately, int4 KV cache quantization caused too much accuracy loss, so I ended up choosing LMDeploy with int8 KV cache quantization for inference.
Before submission, I profiled inference time of decoding to 16k tokens (worst case scenario) under different batch sizes. During the actual submission, I tracked the actual inference time for each question, then add planned time - actual time into the time budget for the next question. Batch size for the next question was selected on that new time budget and my earlier profiling results.
This is a fairly conservative strategy, using smaller batch sizes for first received questions. And I didn’t profile for longer max decoding lengths, so there’s room for improvement.
\nThe inference notebook is public here. Simply a rushed implementation using casperhansen/deepseek-r1-distill-qwen-14b-awq and my submission strategy. I haven’t done enough testing so there might be bugs.
Gz! Your submission strategy, dynamically adjusting batch size based on real-time inference performance and remaining time budget, is quite clever and resource-conscious. It's a practical approach to maximize throughput within the competition's constraints.
", "rawMarkdown": "Gz! Your submission strategy, dynamically adjusting batch size based on real-time inference performance and remaining time budget, is quite clever and resource-conscious. It's a practical approach to maximize throughput within the competition's constraints.", "votes": 1 }, { "id": 3176417, "postDate": "2025-04-11T10:23:12.353Z", "content": "Apparently you only had 1 public LB submission?
\nhttps://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-2/discussion/572784
", "rawMarkdown": "Apparently you only had 1 public LB submission?\n\nhttps://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-2/discussion/572784", "votes": 1, "replies": [ { "id": 3176423, "postDate": "2025-04-11T10:28:17.673Z", "content": "This is what he does.😂 He often joins competitions near the deadline, submits only a few times, and then ranks very high on the private leaderboard.
", "rawMarkdown": "This is what he does.😂 He often joins competitions near the deadline, submits only a few times, and then ranks very high on the private leaderboard.", "votes": 3 }, { "id": 3176436, "postDate": "2025-04-11T10:49:01.060Z", "content": "Take a look at eedi 72nd place. I also joined eedi, so i remember this. Habit? 😀
\nhttps://www.kaggle.com/competitions/eedi-mining-misconceptions-in-mathematics/leaderboard
Congratulations on 7th place in this competition.
\n I also tried 14b model couple of times with vLLM, however, I got Notebook Timeout.
\nWhats your opinion for the model update from 7b to 14b.
I think validation should be done like this:
\nany, majority, and average correct.any, majority, and average correct. Repeat this process many times and check the stats.The 7B model might perform not bad on any, but the 14B model have much better majority and average. The 7B model has a higher risk of overfitting to the public leaderboard.
With 4-bit quantized weights and fp16 KV cache, the speed ranking was: LMDeploy > SGLang > vLLM v1 >> vLLM.
\nCan you show us the comparison results?
Congrats and thank you for sharing!
\nNow not only the training optimization, but the inference speed optimization becomes more important.
Congrats! I tried 14b with lmdeploy and int8 kv quant once and got 26 on the public LB. The performance degradation seem to be minimal. vLLM's fp8 kv quant on the other hand gave me gibberish (could have been a bug in the version I installed).
", "rawMarkdown": "Congrats! I tried 14b with lmdeploy and int8 kv quant once and got 26 on the public LB. The performance degradation seem to be minimal. vLLM's fp8 kv quant on the other hand gave me gibberish (could have been a bug in the version I installed). ", "votes": 2 }, { "id": 3182950, "postDate": "2025-04-20T06:40:10.260Z", "content": "Congratulations on your well-deserved success…. Thanks for inspiring us with your hard work and dedication.
", "rawMarkdown": "Congratulations on your well-deserved success.... Thanks for inspiring us with your hard work and dedication." }, { "id": 3176323, "postDate": "2025-04-11T08:39:08.860Z", "content": "Amazingly simple! Congrats!
\nWhat was your public score ?
", "rawMarkdown": "Amazingly simple! Congrats!\n\nWhat was your public score ?", "replies": [ { "id": 3176387, "postDate": "2025-04-11T09:52:39.547Z", "content": "It was 23. That's why I call it pure luck.
", "rawMarkdown": "It was 23. That's why I call it pure luck.", "votes": 1, "replies": [ { "id": 3176481, "postDate": "2025-04-11T11:52:17.590Z", "content": "The private LB did not seem too random to me, you had a strong solution! Congrats!
", "rawMarkdown": "The private LB did not seem too random to me, you had a strong solution! Congrats!", "votes": -2 }, { "id": 3176869, "postDate": "2025-04-11T20:22:45.220Z", "content": "I've seen that you usually win medals with just 1-2 submissions. Could you please share any tips about your strategy or advice? How do you do this?
\nPersonally, I cannot really call it \"luck\", when you were able to repeat this so many times.
", "rawMarkdown": "I've seen that you usually win medals with just 1-2 submissions. Could you please share any tips about your strategy or advice? How do you do this? \n\nPersonally, I cannot really call it \"luck\", when you were able to repeat this so many times.", "votes": 2 }, { "id": 3178643, "postDate": "2025-04-14T12:30:12.200Z", "content": "\"Luck\" is specifically referring to this competition.
\nFor me, making fewer submissions helps me better balance work-Kaggle-life. I enjoy problem solving but don't enjoy leaderboard climbing much. I usually have a first-day plan, and even without submitting, there are many interesting things to experiment with. Regardless of the leaderboard score, 'executing the plan'' has been very rewarding for me.
\nThe strategy of 'only submitting in the final days' has led to many failures for me in past competitions. If you're concerned about final ranking, making more submissions is the better approach.
", "rawMarkdown": "\"Luck\" is specifically referring to this competition.\n\nFor me, making fewer submissions helps me better balance work-Kaggle-life. I enjoy problem solving but don't enjoy leaderboard climbing much. I usually have a first-day plan, and even without submitting, there are many interesting things to experiment with. Regardless of the leaderboard score, 'executing the plan'' has been very rewarding for me. \n\nThe strategy of 'only submitting in the final days' has led to many failures for me in past competitions. If you're concerned about final ranking, making more submissions is the better approach.", "votes": 7 }, { "id": 3178917, "postDate": "2025-04-14T19:22:35.803Z", "content": "Thank you very much, your approach is very cool. I also wanted to ask you about your general validation strategy. As I understood, during your experiments you rely on your validation score. How do you create your validation set? Sometimes, local validation might not correlate well with the public leaderboard and some competitions have this domain shift. How do you deal with such competitions?
", "rawMarkdown": "Thank you very much, your approach is very cool. I also wanted to ask you about your general validation strategy. As I understood, during your experiments you rely on your validation score. How do you create your validation set? Sometimes, local validation might not correlate well with the public leaderboard and some competitions have this domain shift. How do you deal with such competitions?" }, { "id": 3180152, "postDate": "2025-04-16T08:08:35.643Z", "content": "Exactly, whenever I see @tascj0 in the gold zone solo, they usually have less than 10 subs
", "rawMarkdown": "Exactly, whenever I see @tascj0 in the gold zone solo, they usually have less than 10 subs" }, { "id": 3181581, "postDate": "2025-04-18T04:21:05.223Z", "content": "@manwithaflower
\nI don't have any special validation strategies. I fail in trust LB competitions.
As long as local validation enough for the my planned experiments, I'm comfortable with it. What I planned to do through a competition might not even be related to LB.
", "rawMarkdown": "@manwithaflower \nI don't have any special validation strategies. I fail in trust LB competitions.\n\nAs long as local validation enough for the my planned experiments, I'm comfortable with it. What I planned to do through a competition might not even be related to LB.", "votes": 3 } ] } ] }, { "id": 3176320, "postDate": "2025-04-11T08:37:07.573Z", "content": "Thanks for sharing — your solution looks very clean, and the budget planning seems well thought out!
", "rawMarkdown": "Thanks for sharing — your solution looks very clean, and the budget planning seems well thought out!" }, { "id": 3178146, "postDate": "2025-04-13T19:27:49.460Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3176364, "postDate": "2025-04-11T09:32:06.237Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3180408, "postDate": "2025-04-16T15:29:02.453Z", "content": "Congratulations! Thank you for sharing!
", "rawMarkdown": "Congratulations! Thank you for sharing!" }, { "id": 3179711, "postDate": "2025-04-15T16:02:41.577Z", "content": "Thanks for wonderful solutions
", "rawMarkdown": "Thanks for wonderful solutions" } ], "complete": true, "source": "live", "fetched_at": "2026-10-20T06:43:46Z", "competition_id": 86023, "competition_slug": "ai-mathematical-olympiad-progress-prize-2", "team_id": 13549099, "solution_url": "/competitions/ai-mathematical-olympiad-progress-prize-2/writeups/tascj-7th-place-solution-pure-luck" }