This competition required optimizing both efficiency and reasoning performance. Our final solution consists of three main parts:
\nOur training scripts are based on Light-R1 project, we are really grateful to their work.
We choose DeepSeek-R1-Distill-Qwen-14B as the base model considering its great performance in mathematics, coding and reasoning.
\nWe combine the stage2 data from Light-R1 and training data from Limo together (duplicates removed), which are both high-difficulty math problems' reasoning trajectories generated from deepseek-r1
\nWe finetune the base model for 8 epochs on single 8×A800 machine, taking 11 hours:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F23572158%2F641a0798dbca8be9adbf1b010cb49159%2F2e6707889efebafd252bd2ad0443d232.png?generation=1744460831585737&alt=media\")
\nThe accuracy improves but the output length also improves significantly.
We use DPO to reduce the output length of the model
\nWe choose the default subset of OpenR1-Math-220k to build our dataset
\nSpecifically, we try to use the following four criteria to construct DPO pairs (y_w, y_l means chosen response and rejected response):
360-LLaMA-Factory because they add sequence parallelism (SP) technology to support longer context training with limited memory.deepseek-14b-sft-dpo2 and deepseek-14b-sft-dpo4training data![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F23572158%2Fb0a29a66664d9236e04b532fbc0e03c8%2F7187e4e6d0daeae809ac6559ffa3b103.png?generation=1744460986607785&alt=media\")
We choose lmdeploy as the LLM inference framework. Compared with vllm, the lmdeploy framework with the TurboMind engine can provide higher throughput and shorter model initialization time.
\nThe lmdeploy version we use is 0.7.0.post1
\nThe first picture comes from here
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F23572158%2F5e8c198ab84475ca77c1b8b27adfcdd9%2F5f126916d91bb2519cb69dd4b3256ef3.png?generation=1744461014286477&alt=media\")
We apply 4-bit AWQ weight quantization, and 8-bit KV Cache quantization (setting the configuration main_model.inference_cfg.quant_policy to 8 to use 8-bit KV Cache quantization implemented by lmdeploy).
\nOur two models for submissions can be found here: model1, model2
\nSome efficiency results:
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F23572158%2F1f2074a4fa450da09049e99f3d97f2ab%2Fc1af1024b94a2d7511b234b44df1ab6e.png?generation=1744461043302217&alt=media\")
The inference workflow is shown in the figure below: A question is provided as the input. We first prepare two types of prompts, including the CoT prompt and the Code prompt (\"Prompt Preparation Task\"). Then, we let the LLM start batchify generation of multiple samples with lmdeploy (\"LLM Generation Task\"). In the mean time, we continuouly try extract the answer from the streaming output of each sample, aggregate the answers of multiple samples, and judge whether to early stop some generation:
stream_infer(...) call, and judge whether to early stop the generation of the corresponding sample. The Python code executor and answer extractor components are used here.![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F23572158%2Fef0d82c4b38753bb20a279e0f9c84148%2F4ff9e80d8347169f359c9f8a32a7331b.png?generation=1744461072712397&alt=media\")
Method:
\n\n \n \n\n \n \nSome experiments:
\n| Model | \nQuantization | \nTotal solving time | \nAvg outlen | \nAggregated correct questions (/30) | \nAverage correct samples (/32) | \nCode error break down (/16) | \n|
|---|---|---|---|---|---|---|---|
| dpsk-qwen-14b | \nKV16 | \n11838.22 | \n9776.94 | \n20.00 | \n14.63 | \nNo code: 1.93; Exec error: 2.97; Fail parseint: 0.13; Wrong number: 5.30 | \n|
| dpsk-qwen-14b-awq | \nAWQ4 KV8 | \n6844.75 | \n10118.54 | \n21.00 | \n14.40 | \nNo code: 3.30; Exec error: 2.53; Fail parseint: 0.33; Wrong number: 4.57 | \n|
| dpsk-qwen-14b-finetune-v1-epoch4 | \nKV16 | \n12971.18 | \n11151.10 | \n21.00 | \n18.90 | \nNo code: 11.00; Exec error: 0.83; Fail parseint: 0.03; Wrong number: 1.43 | \n|
| dpsk-qwen-14b-finetune-v1-epoch4-awq | \nAWQ4 KV8 | \n7963.94 | \n11557.06 | \n21.00 | \n16.80 | \nNo code: 11.07; Exec error: 1.30; Fail parseint: 0.03; Wrong number: 0.90 | \n
Motivation: Usually, the reasoning model will self-doubt a lot after obtaining the answer early, even if it usually gives out the same answer finally. And in most cases, after giving the answer between <think></think>, the model will rewrite the solution again (at least twice). Can we reduce the wasting of tokens?
\nMethod: Although we experimented with the active probing method from \"Fu et al., \"Efficiently Serving LLM Reasoning Programs with Certaindex, arXiv 2412\", we ultimately adopted a much simpler sample-level early stopping technique to simplify our inference workflow. Specifically, once we detect either the first successfully executable code or the first answer in \"\\boxed{…}\", we stop the generation process for that sample.
\nSome experiments: A natural question is whether this might harm the potential to revise an initially incorrect answer into a correct one later. In our local tests, we use scripts/analyze_early_stop.py to verify that such cases are relatively rare, as shown in the figure below.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F23572158%2Fe0046e83f954d42e3133ea88947b4b89%2Fdc372cc270acc1d5dc01f37c533e0bc8.png?generation=1744461139162863&alt=media\")
We use the commonly used self-consistency method for answer aggregation. We use a question-level early-stop strategy as follows.
\nMotivation: The difficulty varies across problems, so we aim to avoid spending too much time on easy ones. As shown in the figure below, the output length varies considerably across samples for a single problem. This suggests that for some problems, we may obtain several correct answers early on but still need to wait for the longest sample to finish -- resulting in significant time waste (e.g., q1, q11, q16–q20, etc.).
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F23572158%2Ff42ad387359f025fcd84406d2522fc07%2F4856497485f90b1ee37ee3d907f90419.png?generation=1744461161694426&alt=media\")
\nMethod: We can stop generation early for a question if sufficient certainty is achieved by examining the existing answers. Specifically, we terminate generation at the question level when a majority of the outputs are consistent, e.g., if 5 out of 7 answers agree. See the configurations in early_stop_strategy.consistency_rules in imagination_aimo2/local_eval_kaggle.py.
Motivation: The time to solve questions of different difficulty levels varies greatly, we design a adjust_speed module to dynamically adjust some hyperparameters.
\nMethod: As reasoning progresses, our adjust_speed module calculates the remaining time and number of remaining questions, and dynamically adjusts the model's sampling number and early stopping strategy accordingly.
\nFor example, the default speed is 3(normal), if the system detects that the average remaining time for each question is less than 5 minutes, it automatically adjust the speed to 1(fastest) .This means the number of samples is decreased to 10 and the maximum reasoning time for each question is also decreased. Please refer to our code for detailed implementation.
W_k and W_q to get more channel-balanced key for better KV cache quantization. This brings a consistent but small reasoning performance improvement in our local test, especially in the KV4 setting. However, as we don't have enough submission quota to tune other hyperparameters for KV4 setting, our final solution doesn't use KV4.Impressive work! A well-structured and innovative approach—great balance of reasoning optimization and inference efficiency. Congrats on the 2nd place win! 👏
", "votes": 3, "replies": [] }, { "id": 3177636, "author_name": "tascj", "author_url": "", "post_date": "2025-04-13T04:45:06.333000", "content": "Congratulations!
\nI have to say this is one of the best solution write-ups I've ever read on Kaggle.
", "votes": 4, "replies": [ { "id": 3177645, "author_name": "Yichen You", "author_url": "", "post_date": "2025-04-13T05:02:26.983000", "content": "Thanks a million!
", "votes": 1, "replies": [] } ] }, { "id": 3179861, "author_name": "Yuki", "author_url": "", "post_date": "2025-04-15T19:41:57.103000", "content": "Congratulations to your team and thanks for this amazing write-up! I learnt a lot from your post-decision processing. I have one question regarding your training process, the training data extract and built from Limo and openr1-math-220k is simply chain-of-thought prompt (given the math problem, respond with chain-of-thought and solution and python code is not required). In the last year, the top2 winner from aimo both applies tora corpus and tool-integrated training examples. May I ask what's your opinion and suggestion regarding tora corpus training and why this is not considered in your solution. Thanks!
", "votes": 1, "replies": [ { "id": 3183591, "author_name": "Yichen You", "author_url": "", "post_date": "2025-04-21T03:41:19.140000", "content": "Thanks! We did not find a suitable dataset (solving math problems with Python code), and when we tried to use deepseek-r1, we found it difficult to use prompts to let the model solve math problems with code (it always tends to think directly);I think if you use tool integration training, there will be further improvement in our problem-solving framework.
", "votes": 1, "replies": [] } ] }, { "id": 3177382, "author_name": "Quickpanda", "author_url": "", "post_date": "2025-04-12T16:43:20.640000", "content": "great work! Thanks for sharing! Your work make this competition more meaningful, it is the real value of this competition! Really thanks for sharing. Thanks kaggle and organizer @friederrr !
", "votes": 1, "replies": [] }, { "id": 3177301, "author_name": "Sangram Patil", "author_url": "", "post_date": "2025-04-12T14:45:37.167000", "content": "First of all, congratulations to your team — you guys have built a great pipeline!
\nCan I ask how you converted the models to triton_models format, because I didn’t see that code anywhere?
We have updated: https://github.com/imagination-research/aimo2/blob/master/scripts/run_convert.sh
", "votes": 1, "replies": [] } ] }, { "id": 3177243, "author_name": "ducnh279", "author_url": "", "post_date": "2025-04-12T13:25:44.680000", "content": "Congrats on the great pipeline! 🙌
\nI’m curious about your choice of 8 epochs for SFT. Given that Light-R1 and LIMO are high-quality datasets, I would think that more epochs might lead to additional improvements, similar to how transfer learning benefits from extended training on refined data.
\nThis reminds me of the Phi-4 technical report, where increasing epochs on synthetic data significantly boosted performance - more than just adding mixed lower-quality (web) data. Did you consider this approach, or was the optimal stopping point determined purely by validation performance?
\nAlso, using DPO to control response length is actually a wise strategy, especially with the challenges of unstable RL training for concise and effective reasoning!
\nBig congrats again!
", "votes": 1, "replies": [ { "id": 3177314, "author_name": "Yichen You", "author_url": "", "post_date": "2025-04-12T15:00:38.327000", "content": "We found that fine-tuning too many epochs will significantly increase the output length of the model. 8 epochs is enough.
", "votes": 5, "replies": [] } ] }, { "id": 3183920, "author_name": "Vinu1729", "author_url": "", "post_date": "2025-04-21T12:47:25.043000", "content": "Thanks Guys for sharing your all analysis on this competition , the thing u tried i done some of them but due to lack of knowledge i got confused and this was my 1st competition , yet i dont submitted on this competition , but i have an idea like why can we make a ai maths agent which would think like a general mathematian ( find whats given , what to find , how to find , any formulas/theorem ets)… like that , i have a idea to make a ramanujam model which can answer the best of any , just i need knowlege and a better can we make this , and i forget to say , \"Congrats Guys\" , here is my email: shindevinayak233@gmail.com reach me out there and here is linkedin :
\nhttps://www.linkedin.com/in/vinayak-shinde-1aa968223/ , Thanks for that Idea
Congratulations to your team, and thanks for the amazing write-up! One of the most thoughtful and clever solution I've ever seen on Kaggle. The attention to details and the effort your team has put into your solution is insane!
\nThe effort and attention to the inference strategies of your team is truly next level!
\nI tried to evaluate your deepseek-14b-sft-dpo4 model on AIME'25, in BF16, with 64 rollouts per prompt, max_len is 32768 tokens, temperature 0.75 and top_p 0.95. I made sure that the prompt I'm using is the same as your CoT prompt, for example:
<｜begin▁of▁sentence｜>You are a helpful math assistant. Please reason step by step to put the answer in \\boxed{}.<｜User｜>Find the sum of the roots of the equation $x^2 - 6x + 9 = 0$. \nYou excel at reasoning.\nYou must put the final answer in \\boxed{}.\nIf the final answer is greater than1000, then take the modulo of 1000.\nThink carefully and thoroughly, avoid duplication.<｜Assistant｜>\nHowever, here are my numbers (your model is last column):
\n| \n | DeepSeek-R1-Distill-14B | \nLight-R1-14B-DS | \nOne of our team's GRPO | \n2nd-Place-DPO4 | \n
|---|---|---|---|---|
| accuracy | \n0.484896 | \n0.581771 | \n0.542188 | \n0.557813 | \n
| avg_len | \n13071.5 | \n13827.4 | \n11274.5 | \n12183 | \n
| maj@5 | \n0.591667 | \n0.679167 | \n0.695833 | \n0.679167 | \n
| maj@8 | \n0.591667 | \n0.7 | \n0.7125 | \n0.679167 | \n
| maj@12 | \n0.6125 | \n0.716667 | \n0.741667 | \n0.708333 | \n
PS: the majority voting accuracy maj@k numbers are averaged over multiple \"virtual\" runs, where I sample traces from a bank of reasoning traces (64 to be exact).
My numbers for Deepseek-R1-14B and Light-R1-14B matches the reported numbers. But your model's accuracy in my tests is 0.558 (averaged over 64 rollouts per problem), but you reported 0.589. Was that number pure COT performance or it's mixed with TIR?
Thanks! This number is mixed with TIR, so pure COT may be lower, we mentioned it here.
\nBesides, we applied the sample-level early stop strategy (as we mentioned) for evaluation.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F23572158%2F691182fd63eb695112d9d70135e6a5a3%2F92241a106152d3a12616bdbe40d76631.png?generation=1744552045434973&alt=media\")
Thanks, that makes a lot more sense. Now I feel silly for even trying your model with CoT only 😅
", "votes": 1, "replies": [] } ] } ] }, { "id": 3182290, "author_name": "William Lien", "author_url": "", "post_date": "2025-04-19T04:47:08.450000", "content": "Very impressive thought & solution.
", "votes": 0, "replies": [] }, { "id": 3180988, "author_name": "john", "author_url": "", "post_date": "2025-04-17T10:52:08.413000", "content": "Congratulations. Thanks for the write-up & excellent plots!
\nI have one question after read the quant_awq.py code.
\nThe default max_calib_seq_len of autoawq is 512. What's the reason for choosing max_calib_seq_len=128 & max_calib_samples=1024?
Well done!
", "votes": 0, "replies": [] } ] }, { "id": 3180684, "author_name": "planet2020", "author_url": "", "post_date": "2025-04-17T02:12:16.843000", "content": "After spending hours reading the notebook and codes, I am still confused. Can you please share the original notebook used to submit for the private LB?
", "votes": 0, "replies": [ { "id": 3180699, "author_name": "Yichen You", "author_url": "", "post_date": "2025-04-17T02:39:40.807000", "content": "This backup notebook is exactly the same as the notebook we finally scored 31, there may be some irregularities in the writing, we still recommend you to read our cleaned version, which is more structured.
\nhttps://github.com/imagination-research/aimo2/blob/master/imagination_aimo2/backup_submission.ipynb
Thanks you.
", "votes": 0, "replies": [] }, { "id": 3181491, "author_name": "Yichen You", "author_url": "", "post_date": "2025-04-18T00:34:41.533000", "content": "version112 got 33/31 on public/private leaderboard and version94 got 34/29.
\nhttps://www.kaggle.com/code/youyc22/imagination-research-original-submission
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F23572158%2Fd22cae24037ff5ebd06f0bf30918e69c%2Fc08c0758b2569345f9507f101ca9b13a.png?generation=1744936765179862&alt=media\")
Thank you for sharing this notebook🙂
", "votes": 0, "replies": [] } ] } ] }, { "id": 3179940, "author_name": "planet2020", "author_url": "", "post_date": "2025-04-16T00:12:56.353000", "content": "Congratulations! I copied your notebook and run it in Kaggle. The program was terminated very quickly, leaving only the following reply. Is it not possible to run it directly on Kaggle?
\n2025-04-15 23:52:14,096 - lmdeploy - WARNING - supported_models.py:106 - /kaggle/input/dpsk-14b-sft-dpo-3-13-awq-tb/keras/default/1/dpsk-14b-sft-dpo4-3-13-awq-tb seems to be a turbomind workspace, which can only be ran with turbomind engine.
\n[TM][WARNING] [LlamaTritonModel] max_context_token_num is not set, default to 33320.
\n[WARNING] gemm_config.in is not found; using default GEMM algo
\n[WARNING] gemm_config.in is not found; using default GEMM algo
\n[WARNING] gemm_config.in is not found; using default GEMM algo
\n[WARNING] gemm_config.in is not found; using default GEMM algo
\n2025-04-15 23:53:08,504 - lmdeploy - WARNING - tokenizer.py:647 - The token <|endoftext|>, its length of indexes [27, 91, 8691, 723, 427, 91, 29] is over than 1. Currently, it can not be used as stop words
\nUpdated active prompts. Target: 15, Actual: 15
\nbbd91e
\n057f8a
\n192e23
\n1fce4b
\n71beb6
\n1acac0
\na1d40b
\n88c219
\n480182
\n349493
Thanks! You need to comment out the first three lines in predict function![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F23572158%2Fd4c766569358a8e6ef6caef86d98f06d%2Feb2e127fcc17081930a6e5bcbccdafd5.png?generation=1744769914560504&alt=media\")
It works. Thank you very much. The outputs are so detailed👍
", "votes": 0, "replies": [] } ] }, { "id": 3213597, "author_name": "", "author_url": "", "post_date": "2025-05-30T08:12:20.890000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 3179762, "author_name": "GURRAM GOPICHANDH", "author_url": "", "post_date": "2025-04-15T16:48:46.030000", "content": "Impressive thoughts and solution.
", "votes": 0, "replies": [] }, { "id": 3179704, "author_name": "", "author_url": "", "post_date": "2025-04-15T15:59:29.057000", "content": "Thanks for sharing your solution. It was very helpful .
", "votes": 0, "replies": [] }, { "id": 3178493, "author_name": "YANGJIAHUA", "author_url": "", "post_date": "2025-04-14T08:53:59.600000", "content": "Congratulations to your team, and thank you for the excellent paper! I’ve learned a lot from it! Would it be possible for you to share the code for the SFT stage or the DPO stage? I’d be very grateful if you could—I'd love to study and learn more about it!
", "votes": 0, "replies": [ { "id": 3178564, "author_name": "Foxfi Ning", "author_url": "", "post_date": "2025-04-14T10:31:25.723000", "content": "Thanks. We use 360-LLama-Factory's code (a git submodule in our repository) to run SFT and DPO. You can check our repository https://github.com/imagination-research/aimo2, the scripts are given at scripts/run_sft.sh and scripts/run_dpo.sh.
Congratulations!! This is really great, the way it's written is soo clean and it's spelled out well. For sure your work would be used to build and iterate in the future competitions.
\nI have a couple of questions though:
\nUpon the completion of the training, did your team think that 8 epochs is a bit too much or what is appropriately enough?
\nAnd can you share the training losses you got while fine tuning if possible :)
And I wanted to ask in general, I see that everyone uses both the system and user role in their prompts. But the model card in huggingface says this thing below. Did you experiment by putting the instruction+question in the user role and ditching the system role? I went through your repo (which is really cool btw) and I can see that you used both the roles. Any specific reason? I mean pretty much everyone did this too, I really want to know what's this about.
\nUsage Recommendations\nWe recommend adhering following configurations when utilizing DeepSeek-R1 series models, including benchmarking, achieve expected performance:\n\nAvoid adding prompt; all instructions should be contained user prompt.\nThanks!
\nAbout Q1, 8-epoch SFT is appropriately enough in our case. More epochs will lead to longer token length.
\nAbout Q3, thank you for letting us to know this note. We don't know this note, so we are just following early practice of using both the system prompt and user prompt.
\nAbout Q2, I need to ping Yichen to plot the loss curve @youyc22
", "votes": 1, "replies": [] }, { "id": 3178571, "author_name": "Yichen You", "author_url": "", "post_date": "2025-04-14T10:46:27.350000", "content": "SFT loss curve is as follows
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F23572158%2Fde64ae59cdb37dfe61fc77bef5508759%2Fd132ee85970b588f4e7070d91ea47d9b.png?generation=1744627553753428&alt=media\")
Congratulations! Great job!
\nHave you tried model merging ?
We have tried model merging, between the two checkpoints and another checkpoint of a new DPO run. We can get an FP16 model that has a slightly higher average sample accuracy and an obviously higher (+2/+3 out of 30 problems) aggregated accuracy in our local test. However, when quantized, the model is no better than the original quantized checkpoints in the local test, and as the submission quota is limited to test multiple time of that model, we didn't submit it.
", "votes": 0, "replies": [] } ] }, { "id": 3177948, "author_name": "Chew Kok Wah", "author_url": "", "post_date": "2025-04-13T14:23:05.743000", "content": "Can you share your original winning submission notebook. (Do understand the cleaned version could be similar but community might be able to learn something from the original one)
", "votes": 0, "replies": [] }, { "id": 3177891, "author_name": "Wadiel Rahman", "author_url": "", "post_date": "2025-04-13T12:14:56.563000", "content": "Fantastic work! I would love to see more of it!
", "votes": 0, "replies": [] }, { "id": 3177723, "author_name": "Jason Bourne", "author_url": "", "post_date": "2025-04-13T07:23:50.573000", "content": "最实至名归的一集。从前至后各阶段的实验安排条理清晰，模型提升的方法思路很让人受启发。期待你们下一次取得更好的成绩💪。
\n羡慕8块A800啊，穷人真是玩不起大模型😂
Thanks a lot!
\n感谢支持！
Congratulations!
\nCan you provide the data deduplication script for dataset duplicates removal?
Please share the FP16 model weight for Stage 1 SFT as well as Stage 2 DPO similar to the way AIMO1 winner did it.
\nhttps://huggingface.co/AI-MO
We have updated here:https://github.com/imagination-research/aimo2/blob/master/scripts/run_deduplicate.sh
\nStage2 DPO model can be found on huggingface:https://huggingface.co/collections/imagination-research/aimo2-67f92097e2a4e04d995f3d75, which can be found at the beginning of our write-up
\nStage1 SFT model will be uploaded sooner
", "votes": 1, "replies": [] } ] }, { "id": 3177560, "author_name": "RabotniKuma", "author_url": "", "post_date": "2025-04-13T01:09:37.850000", "content": "Congrats! Such a beautiful and well-designed solution.
\nDo you have records of how Public LB improved after each tranining step?
Thanks! We got 28 without any finetuning, scored 33 after sft (8 epochs) and dpo (2 epochs), scored 34 after dpo for 4 epochs(ranging between 32~34). We didn't submit the model that only had SFT applied.
This is an absolutely remarkable solution, and thank you more for sharing your approach to improve the quality of future contributions in the field. Looking forward to more works from you
", "votes": 0, "replies": [] }, { "id": 3177335, "author_name": "xz", "author_url": "", "post_date": "2025-04-12T15:43:50.377000", "content": "Congrats! What was the score with the stage 1 SFT model and did DPO give significant improvement over it?
", "votes": 0, "replies": [ { "id": 3177599, "author_name": "Yichen You", "author_url": "", "post_date": "2025-04-13T02:55:24.770000", "content": "Thanks! We didn't submit the stage 1 SFT model, base model got 28, dpo-2 got 33 and dpo-4 got 34, but on private leaderboard they got 31 and 29 respectively.
Amazing work, congratulations.
\nI am interested in quantization-aware training, I had quite a bit of trouble setting up QAT. Can you share how you did it?
", "votes": 0, "replies": [ { "id": 3177590, "author_name": "Foxfi Ning", "author_url": "", "post_date": "2025-04-13T02:36:16.660000", "content": "Thanks for your question. We just do a very cursory and incomplete trial, the following is just for your reference. We use the 360-LLama-Factory library, which uses autoawq to load the AWQ model. In order to tune the scales of AWQ-quantized model, we do the following ugly fixes:
(1) In autoawq's awq.modules.linear.WQLinear_GEMM.__init__: We register the scales as parameters instead of buffers, in order for let the optimizer update them:
.register_parameter(\n ,\n torch.nn.Parameter(torch.zeros(\n (in_features // .group_size, out_features),\n dtype=torch.float16,\n device=dev,\n )),\n )\n(2) In autoawq's awq.modules.linear.WQLinear_GEMM.forward: Instead of calling WQLinearMMFunction.apply, we change it to use the naive implementation of decoding+matrix-multiplication. The original implementation uses a more efficient Triton-implemented decoding+matrix-multiplication, but they don't implement gradient backward for the scale.
\n \n \n \n \n \n \n \n \n \n user_has_been_warned:\n warnings.warn( + msg)\n user_has_been_warned = \n torch.cuda.amp.autocast():\n out = dequantize_gemm(.qweight, .qzeros, .scales, .w_bit, .g\\\nroup_size)\n out = torch.matmul(x, out)\n out = out + .bias .bias out\n out = out.reshape(out_shape)\n \n (out.shape) == :\n out = out.unsqueeze()\n(3) In the 360-Llama-Factory lib, we add the following call to the llamafactory.train.dpo.workflow.run_dpo function after loading the model, to set the requires_grad of scales to be True.
\n\n\n\n\n\n\n\n\n def run_dpo(\n model_args: \"ModelArguments\",\n data_args: \"DataArguments\",\n def run_dpo(\n template = get_template_and_fix_tokenizer(tokenizer, data_args)\n dataset_module = get_dataset(template, model_args, data_args, training_args, stage=\"rm\", **tokenizer_module)\n model = load_model(tokenizer, model_args, finetuning_args, training_args.do_train, full_determinism=training_args.full_determinism)\n\nAs you can see, we do some ugly fixes instead of writing a clean code. A detail is that due to our computational capability limitation, we use of cutoff length of 16k (instead of 32k) to do the training on 4 GPUs. This small quick attempt doesn't get improvements. We think tuning the weights might still be needed, and longer context length might be needed, but we don't have the GPU & development capacity to further try that.
", "votes": 1, "replies": [ { "id": 3177792, "author_name": "Kawa", "author_url": "", "post_date": "2025-04-13T09:36:32.430000", "content": "Thanks for detail. Seems AWQ model so hard to finetune(especially reasoning model). My tests came out pretty bad, 15-18 on public lb.
", "votes": 0, "replies": [] }, { "id": 3180012, "author_name": "Chan Kha Vu", "author_url": "", "post_date": "2025-04-16T03:23:31.467000", "content": "Wow! this is actually really cool, so much care going into each detail…
", "votes": 0, "replies": [] } ] } ] }, { "id": 3177315, "author_name": "Danyal Abbas", "author_url": "", "post_date": "2025-04-12T15:01:28.947000", "content": "Very incredibly structured approach
\nCongratulations on the 2nd place win!
It's a huge work for your team, and it takes a long time to ultimately win the 2nd place, truly well-deserved. Great performance your Tsinghua and Microsoft.
\n\n\nRAG: we use this dataset as a document bank. When solving a question, we fetch 1~2 most similar questions from the bank, use them and their corresponding solutions as the few-shot demonstrations. But we did not get improvement on AIME2025 nor online submission.
\n
Once I thought RAG was a key solution for this competition, but in reality, it didn’t work when I tried it myself. I’m a bit confused, maybe the dataset is just too difficult?
", "votes": 0, "replies": [ { "id": 3177312, "author_name": "", "author_url": "", "post_date": "2025-04-12T14:58:53.647000", "content": "", "votes": 0, "replies": [] }, { "id": 3177321, "author_name": "Foxfi Ning", "author_url": "", "post_date": "2025-04-12T15:10:41.440000", "content": "Thanks!
\nAbout RAG, we try RAG in an early stage of our experiments. Back then, we qualitatively inspect some runs (no statistics), and find out that the questions that have similar embedding not necessarily have similar solving strategies (e.g., have similar words instead of reasoning structure). To further explore this, maybe we can try better embedding or retrieval strategies, e.g., (1) use a more appropriate feature as the embedding (we just use the mean of the last layer's feature across tokens), which might be more reflective of the reasoning state instead of language-related information, or (2) rely on the model to do some analyses and improve the embedding based on intermediate analyses, or (3) actively search for reference and reason in a more flexible agentic way.
\nHowever, we don’t further experiment with these possible mitigations. Therefore, we cannot say we have reached a reliable negative conclusion for this pathway. It still needs further inspection and exploration.
", "votes": 3, "replies": [] }, { "id": 3177594, "author_name": "", "author_url": "", "post_date": "2025-04-13T02:41:34.490000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 3181250, "author_name": "", "author_url": "", "post_date": "2025-04-17T16:07:19.640000", "content": "", "votes": 0, "replies": [] }, { "id": 3178152, "author_name": "", "author_url": "", "post_date": "2025-04-13T19:32:39.093000", "content": "", "votes": 0, "replies": [] }, { "id": 3178757, "author_name": "", "author_url": "", "post_date": "2025-04-14T15:11:50.600000", "content": "", "votes": 0, "replies": [] } ], "comment_count": 59, "messages": [ { "id": 3177222, "postDate": "2025-04-12T12:39:42.440Z", "content": "This competition required optimizing both efficiency and reasoning performance. Our final solution consists of three main parts:
\nOur training scripts are based on Light-R1 project, we are really grateful to their work.
We choose DeepSeek-R1-Distill-Qwen-14B as the base model considering its great performance in mathematics, coding and reasoning.
\nWe combine the stage2 data from Light-R1 and training data from Limo together (duplicates removed), which are both high-difficulty math problems' reasoning trajectories generated from deepseek-r1
\nWe finetune the base model for 8 epochs on single 8×A800 machine, taking 11 hours:
\n
\nThe accuracy improves but the output length also improves significantly.
We use DPO to reduce the output length of the model
\nWe choose the default subset of OpenR1-Math-220k to build our dataset
\nSpecifically, we try to use the following four criteria to construct DPO pairs (y_w, y_l means chosen response and rejected response):
360-LLaMA-Factory because they add sequence parallelism (SP) technology to support longer context training with limited memory.deepseek-14b-sft-dpo2 and deepseek-14b-sft-dpo4training dataWe choose lmdeploy as the LLM inference framework. Compared with vllm, the lmdeploy framework with the TurboMind engine can provide higher throughput and shorter model initialization time.
\nThe lmdeploy version we use is 0.7.0.post1
\nThe first picture comes from here
\n
We apply 4-bit AWQ weight quantization, and 8-bit KV Cache quantization (setting the configuration main_model.inference_cfg.quant_policy to 8 to use 8-bit KV Cache quantization implemented by lmdeploy).
\nOur two models for submissions can be found here: model1, model2
\nSome efficiency results:
The inference workflow is shown in the figure below: A question is provided as the input. We first prepare two types of prompts, including the CoT prompt and the Code prompt (\"Prompt Preparation Task\"). Then, we let the LLM start batchify generation of multiple samples with lmdeploy (\"LLM Generation Task\"). In the mean time, we continuouly try extract the answer from the streaming output of each sample, aggregate the answers of multiple samples, and judge whether to early stop some generation:
stream_infer(...) call, and judge whether to early stop the generation of the corresponding sample. The Python code executor and answer extractor components are used here.Method:
\n\n \n \n\n \n \nSome experiments:
\n| Model | \nQuantization | \nTotal solving time | \nAvg outlen | \nAggregated correct questions (/30) | \nAverage correct samples (/32) | \nCode error break down (/16) | \n|
|---|---|---|---|---|---|---|---|
| dpsk-qwen-14b | \nKV16 | \n11838.22 | \n9776.94 | \n20.00 | \n14.63 | \nNo code: 1.93; Exec error: 2.97; Fail parseint: 0.13; Wrong number: 5.30 | \n|
| dpsk-qwen-14b-awq | \nAWQ4 KV8 | \n6844.75 | \n10118.54 | \n21.00 | \n14.40 | \nNo code: 3.30; Exec error: 2.53; Fail parseint: 0.33; Wrong number: 4.57 | \n|
| dpsk-qwen-14b-finetune-v1-epoch4 | \nKV16 | \n12971.18 | \n11151.10 | \n21.00 | \n18.90 | \nNo code: 11.00; Exec error: 0.83; Fail parseint: 0.03; Wrong number: 1.43 | \n|
| dpsk-qwen-14b-finetune-v1-epoch4-awq | \nAWQ4 KV8 | \n7963.94 | \n11557.06 | \n21.00 | \n16.80 | \nNo code: 11.07; Exec error: 1.30; Fail parseint: 0.03; Wrong number: 0.90 | \n
Motivation: Usually, the reasoning model will self-doubt a lot after obtaining the answer early, even if it usually gives out the same answer finally. And in most cases, after giving the answer between <think></think>, the model will rewrite the solution again (at least twice). Can we reduce the wasting of tokens?
\nMethod: Although we experimented with the active probing method from \"Fu et al., \"Efficiently Serving LLM Reasoning Programs with Certaindex, arXiv 2412\", we ultimately adopted a much simpler sample-level early stopping technique to simplify our inference workflow. Specifically, once we detect either the first successfully executable code or the first answer in \"\\boxed{…}\", we stop the generation process for that sample.
\nSome experiments: A natural question is whether this might harm the potential to revise an initially incorrect answer into a correct one later. In our local tests, we use scripts/analyze_early_stop.py to verify that such cases are relatively rare, as shown in the figure below.
\n
We use the commonly used self-consistency method for answer aggregation. We use a question-level early-stop strategy as follows.
\nMotivation: The difficulty varies across problems, so we aim to avoid spending too much time on easy ones. As shown in the figure below, the output length varies considerably across samples for a single problem. This suggests that for some problems, we may obtain several correct answers early on but still need to wait for the longest sample to finish -- resulting in significant time waste (e.g., q1, q11, q16–q20, etc.).
\n
\nMethod: We can stop generation early for a question if sufficient certainty is achieved by examining the existing answers. Specifically, we terminate generation at the question level when a majority of the outputs are consistent, e.g., if 5 out of 7 answers agree. See the configurations in early_stop_strategy.consistency_rules in imagination_aimo2/local_eval_kaggle.py.
Motivation: The time to solve questions of different difficulty levels varies greatly, we design a adjust_speed module to dynamically adjust some hyperparameters.
\nMethod: As reasoning progresses, our adjust_speed module calculates the remaining time and number of remaining questions, and dynamically adjusts the model's sampling number and early stopping strategy accordingly.
\nFor example, the default speed is 3(normal), if the system detects that the average remaining time for each question is less than 5 minutes, it automatically adjust the speed to 1(fastest) .This means the number of samples is decreased to 10 and the maximum reasoning time for each question is also decreased. Please refer to our code for detailed implementation.
W_k and W_q to get more channel-balanced key for better KV cache quantization. This brings a consistent but small reasoning performance improvement in our local test, especially in the KV4 setting. However, as we don't have enough submission quota to tune other hyperparameters for KV4 setting, our final solution doesn't use KV4.Impressive work! A well-structured and innovative approach—great balance of reasoning optimization and inference efficiency. Congrats on the 2nd place win! 👏
", "rawMarkdown": "Impressive work! A well-structured and innovative approach—great balance of reasoning optimization and inference efficiency. Congrats on the 2nd place win! 👏", "votes": 3 }, { "id": 3177636, "postDate": "2025-04-13T04:45:06.333Z", "content": "Congratulations!
\nI have to say this is one of the best solution write-ups I've ever read on Kaggle.
", "rawMarkdown": "Congratulations!\n\nI have to say this is one of the best solution write-ups I've ever read on Kaggle.", "votes": 4, "replies": [ { "id": 3177645, "postDate": "2025-04-13T05:02:26.983Z", "content": "Thanks a million!
", "rawMarkdown": "Thanks a million!", "votes": 1 } ] }, { "id": 3179861, "postDate": "2025-04-15T19:41:57.103Z", "content": "Congratulations to your team and thanks for this amazing write-up! I learnt a lot from your post-decision processing. I have one question regarding your training process, the training data extract and built from Limo and openr1-math-220k is simply chain-of-thought prompt (given the math problem, respond with chain-of-thought and solution and python code is not required). In the last year, the top2 winner from aimo both applies tora corpus and tool-integrated training examples. May I ask what's your opinion and suggestion regarding tora corpus training and why this is not considered in your solution. Thanks!
", "rawMarkdown": "Congratulations to your team and thanks for this amazing write-up! I learnt a lot from your post-decision processing. I have one question regarding your training process, the training data extract and built from Limo and openr1-math-220k is simply chain-of-thought prompt (given the math problem, respond with chain-of-thought and solution and python code is not required). In the last year, the top2 winner from aimo both applies tora corpus and tool-integrated training examples. May I ask what's your opinion and suggestion regarding tora corpus training and why this is not considered in your solution. Thanks! ", "votes": 1, "replies": [ { "id": 3183591, "postDate": "2025-04-21T03:41:19.140Z", "content": "Thanks! We did not find a suitable dataset (solving math problems with Python code), and when we tried to use deepseek-r1, we found it difficult to use prompts to let the model solve math problems with code (it always tends to think directly);I think if you use tool integration training, there will be further improvement in our problem-solving framework.
", "rawMarkdown": "Thanks! We did not find a suitable dataset (solving math problems with Python code), and when we tried to use deepseek-r1, we found it difficult to use prompts to let the model solve math problems with code (it always tends to think directly);I think if you use tool integration training, there will be further improvement in our problem-solving framework.", "votes": 1 } ] }, { "id": 3177382, "postDate": "2025-04-12T16:43:20.640Z", "content": "great work! Thanks for sharing! Your work make this competition more meaningful, it is the real value of this competition! Really thanks for sharing. Thanks kaggle and organizer @friederrr !
", "rawMarkdown": "great work! Thanks for sharing! Your work make this competition more meaningful, it is the real value of this competition! Really thanks for sharing. Thanks kaggle and organizer @friederrr !", "votes": 1 }, { "id": 3177301, "postDate": "2025-04-12T14:45:37.167Z", "content": "First of all, congratulations to your team — you guys have built a great pipeline!
\nCan I ask how you converted the models to triton_models format, because I didn’t see that code anywhere?
We have updated: https://github.com/imagination-research/aimo2/blob/master/scripts/run_convert.sh
", "rawMarkdown": "We have updated: https://github.com/imagination-research/aimo2/blob/master/scripts/run_convert.sh", "votes": 1 } ] }, { "id": 3177243, "postDate": "2025-04-12T13:25:44.680Z", "content": "Congrats on the great pipeline! 🙌
\nI’m curious about your choice of 8 epochs for SFT. Given that Light-R1 and LIMO are high-quality datasets, I would think that more epochs might lead to additional improvements, similar to how transfer learning benefits from extended training on refined data.
\nThis reminds me of the Phi-4 technical report, where increasing epochs on synthetic data significantly boosted performance - more than just adding mixed lower-quality (web) data. Did you consider this approach, or was the optimal stopping point determined purely by validation performance?
\nAlso, using DPO to control response length is actually a wise strategy, especially with the challenges of unstable RL training for concise and effective reasoning!
\nBig congrats again!
", "rawMarkdown": "Congrats on the great pipeline! 🙌\n\nI’m curious about your choice of 8 epochs for SFT. Given that Light-R1 and LIMO are high-quality datasets, I would think that more epochs might lead to additional improvements, similar to how transfer learning benefits from extended training on refined data.\n\nThis reminds me of the Phi-4 technical report, where increasing epochs on synthetic data significantly boosted performance - more than just adding mixed lower-quality (web) data. Did you consider this approach, or was the optimal stopping point determined purely by validation performance?\n\nAlso, using DPO to control response length is actually a wise strategy, especially with the challenges of unstable RL training for concise and effective reasoning!\n\nBig congrats again!", "votes": 1, "replies": [ { "id": 3177314, "postDate": "2025-04-12T15:00:38.327Z", "content": "We found that fine-tuning too many epochs will significantly increase the output length of the model. 8 epochs is enough.
", "rawMarkdown": "We found that fine-tuning too many epochs will significantly increase the output length of the model. 8 epochs is enough.", "votes": 5 } ] }, { "id": 3183920, "postDate": "2025-04-21T12:47:25.043Z", "content": "Thanks Guys for sharing your all analysis on this competition , the thing u tried i done some of them but due to lack of knowledge i got confused and this was my 1st competition , yet i dont submitted on this competition , but i have an idea like why can we make a ai maths agent which would think like a general mathematian ( find whats given , what to find , how to find , any formulas/theorem ets)… like that , i have a idea to make a ramanujam model which can answer the best of any , just i need knowlege and a better can we make this , and i forget to say , \"Congrats Guys\" , here is my email: shindevinayak233@gmail.com reach me out there and here is linkedin :
\nhttps://www.linkedin.com/in/vinayak-shinde-1aa968223/ , Thanks for that Idea
Congratulations to your team, and thanks for the amazing write-up! One of the most thoughtful and clever solution I've ever seen on Kaggle. The attention to details and the effort your team has put into your solution is insane!
\nThe effort and attention to the inference strategies of your team is truly next level!
\nI tried to evaluate your deepseek-14b-sft-dpo4 model on AIME'25, in BF16, with 64 rollouts per prompt, max_len is 32768 tokens, temperature 0.75 and top_p 0.95. I made sure that the prompt I'm using is the same as your CoT prompt, for example:
<｜begin▁of▁sentence｜>You are a helpful math assistant. Please reason step by step to put the answer in \\boxed{}.<｜User｜>Find the sum of the roots of the equation $x^2 - 6x + 9 = 0$. \nYou excel at reasoning.\nYou must put the final answer in \\boxed{}.\nIf the final answer is greater than1000, then take the modulo of 1000.\nThink carefully and thoroughly, avoid duplication.<｜Assistant｜>\nHowever, here are my numbers (your model is last column):
\n| \n | DeepSeek-R1-Distill-14B | \nLight-R1-14B-DS | \nOne of our team's GRPO | \n2nd-Place-DPO4 | \n
|---|---|---|---|---|
| accuracy | \n0.484896 | \n0.581771 | \n0.542188 | \n0.557813 | \n
| avg_len | \n13071.5 | \n13827.4 | \n11274.5 | \n12183 | \n
| maj@5 | \n0.591667 | \n0.679167 | \n0.695833 | \n0.679167 | \n
| maj@8 | \n0.591667 | \n0.7 | \n0.7125 | \n0.679167 | \n
| maj@12 | \n0.6125 | \n0.716667 | \n0.741667 | \n0.708333 | \n
PS: the majority voting accuracy maj@k numbers are averaged over multiple \"virtual\" runs, where I sample traces from a bank of reasoning traces (64 to be exact).
My numbers for Deepseek-R1-14B and Light-R1-14B matches the reported numbers. But your model's accuracy in my tests is 0.558 (averaged over 64 rollouts per problem), but you reported 0.589. Was that number pure COT performance or it's mixed with TIR?
Thanks! This number is mixed with TIR, so pure COT may be lower, we mentioned it here.
\nBesides, we applied the sample-level early stop strategy (as we mentioned) for evaluation.
\n
Thanks, that makes a lot more sense. Now I feel silly for even trying your model with CoT only 😅
", "rawMarkdown": "Thanks, that makes a lot more sense. Now I feel silly for even trying your model with CoT only 😅", "votes": 1 } ] } ] }, { "id": 3182290, "postDate": "2025-04-19T04:47:08.450Z", "content": "Very impressive thought & solution.
", "rawMarkdown": "Very impressive thought & solution." }, { "id": 3180988, "postDate": "2025-04-17T10:52:08.413Z", "content": "Congratulations. Thanks for the write-up & excellent plots!
\nI have one question after read the quant_awq.py code.
\nThe default max_calib_seq_len of autoawq is 512. What's the reason for choosing max_calib_seq_len=128 & max_calib_samples=1024?
Well done!
", "rawMarkdown": "Well done!" } ] }, { "id": 3180684, "postDate": "2025-04-17T02:12:16.843Z", "content": "After spending hours reading the notebook and codes, I am still confused. Can you please share the original notebook used to submit for the private LB?
", "rawMarkdown": "After spending hours reading the notebook and codes, I am still confused. Can you please share the original notebook used to submit for the private LB?", "replies": [ { "id": 3180699, "postDate": "2025-04-17T02:39:40.807Z", "content": "This backup notebook is exactly the same as the notebook we finally scored 31, there may be some irregularities in the writing, we still recommend you to read our cleaned version, which is more structured.
\nhttps://github.com/imagination-research/aimo2/blob/master/imagination_aimo2/backup_submission.ipynb
Thanks you.
", "rawMarkdown": "Thanks you." }, { "id": 3181491, "postDate": "2025-04-18T00:34:41.533Z", "content": "version112 got 33/31 on public/private leaderboard and version94 got 34/29.
\nhttps://www.kaggle.com/code/youyc22/imagination-research-original-submission
\n
Thank you for sharing this notebook🙂
", "rawMarkdown": "Thank you for sharing this notebook🙂" } ] } ] }, { "id": 3179940, "postDate": "2025-04-16T00:12:56.353Z", "content": "Congratulations! I copied your notebook and run it in Kaggle. The program was terminated very quickly, leaving only the following reply. Is it not possible to run it directly on Kaggle?
\n2025-04-15 23:52:14,096 - lmdeploy - WARNING - supported_models.py:106 - /kaggle/input/dpsk-14b-sft-dpo-3-13-awq-tb/keras/default/1/dpsk-14b-sft-dpo4-3-13-awq-tb seems to be a turbomind workspace, which can only be ran with turbomind engine.
\n[TM][WARNING] [LlamaTritonModel] max_context_token_num is not set, default to 33320.
\n[WARNING] gemm_config.in is not found; using default GEMM algo
\n[WARNING] gemm_config.in is not found; using default GEMM algo
\n[WARNING] gemm_config.in is not found; using default GEMM algo
\n[WARNING] gemm_config.in is not found; using default GEMM algo
\n2025-04-15 23:53:08,504 - lmdeploy - WARNING - tokenizer.py:647 - The token <|endoftext|>, its length of indexes [27, 91, 8691, 723, 427, 91, 29] is over than 1. Currently, it can not be used as stop words
\nUpdated active prompts. Target: 15, Actual: 15
\nbbd91e
\n057f8a
\n192e23
\n1fce4b
\n71beb6
\n1acac0
\na1d40b
\n88c219
\n480182
\n349493
Thanks! You need to comment out the first three lines in predict function
It works. Thank you very much. The outputs are so detailed👍
", "rawMarkdown": "It works. Thank you very much. The outputs are so detailed👍" } ] }, { "id": 3213597, "postDate": "2025-05-30T08:12:20.890Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 3179762, "postDate": "2025-04-15T16:48:46.030Z", "content": "Impressive thoughts and solution.
", "rawMarkdown": "Impressive thoughts and solution." }, { "id": 3179704, "postDate": "2025-04-15T15:59:29.057Z", "content": "Thanks for sharing your solution. It was very helpful .
", "rawMarkdown": "Thanks for sharing your solution. It was very helpful ." }, { "id": 3178493, "postDate": "2025-04-14T08:53:59.600Z", "content": "Congratulations to your team, and thank you for the excellent paper! I’ve learned a lot from it! Would it be possible for you to share the code for the SFT stage or the DPO stage? I’d be very grateful if you could—I'd love to study and learn more about it!
", "rawMarkdown": "Congratulations to your team, and thank you for the excellent paper! I’ve learned a lot from it! Would it be possible for you to share the code for the SFT stage or the DPO stage? I’d be very grateful if you could—I'd love to study and learn more about it!", "replies": [ { "id": 3178564, "postDate": "2025-04-14T10:31:25.723Z", "content": "Thanks. We use 360-LLama-Factory's code (a git submodule in our repository) to run SFT and DPO. You can check our repository https://github.com/imagination-research/aimo2, the scripts are given at scripts/run_sft.sh and scripts/run_dpo.sh.
Congratulations!! This is really great, the way it's written is soo clean and it's spelled out well. For sure your work would be used to build and iterate in the future competitions.
\nI have a couple of questions though:
\nUpon the completion of the training, did your team think that 8 epochs is a bit too much or what is appropriately enough?
\nAnd can you share the training losses you got while fine tuning if possible :)
And I wanted to ask in general, I see that everyone uses both the system and user role in their prompts. But the model card in huggingface says this thing below. Did you experiment by putting the instruction+question in the user role and ditching the system role? I went through your repo (which is really cool btw) and I can see that you used both the roles. Any specific reason? I mean pretty much everyone did this too, I really want to know what's this about.
\nUsage Recommendations\nWe recommend adhering following configurations when utilizing DeepSeek-R1 series models, including benchmarking, achieve expected performance:\n\nAvoid adding prompt; all instructions should be contained user prompt.\nThanks!
\nAbout Q1, 8-epoch SFT is appropriately enough in our case. More epochs will lead to longer token length.
\nAbout Q3, thank you for letting us to know this note. We don't know this note, so we are just following early practice of using both the system prompt and user prompt.
\nAbout Q2, I need to ping Yichen to plot the loss curve @youyc22
", "rawMarkdown": "Thanks!\n\nAbout Q1, 8-epoch SFT is appropriately enough in our case. More epochs will lead to longer token length.\n\nAbout Q3, thank you for letting us to know this note. We don't know this note, so we are just following early practice of using both the system prompt and user prompt.\n\nAbout Q2, I need to ping Yichen to plot the loss curve @youyc22 ", "votes": 1 }, { "id": 3178571, "postDate": "2025-04-14T10:46:27.350Z", "content": "SFT loss curve is as follows
\n
Congratulations! Great job!
\nHave you tried model merging ?
We have tried model merging, between the two checkpoints and another checkpoint of a new DPO run. We can get an FP16 model that has a slightly higher average sample accuracy and an obviously higher (+2/+3 out of 30 problems) aggregated accuracy in our local test. However, when quantized, the model is no better than the original quantized checkpoints in the local test, and as the submission quota is limited to test multiple time of that model, we didn't submit it.
", "rawMarkdown": "We have tried model merging, between the two checkpoints and another checkpoint of a new DPO run. We can get an FP16 model that has a slightly higher average sample accuracy and an obviously higher (+2/+3 out of 30 problems) aggregated accuracy in our local test. However, when quantized, the model is no better than the original quantized checkpoints in the local test, and as the submission quota is limited to test multiple time of that model, we didn't submit it." } ] }, { "id": 3177948, "postDate": "2025-04-13T14:23:05.743Z", "content": "Can you share your original winning submission notebook. (Do understand the cleaned version could be similar but community might be able to learn something from the original one)
", "rawMarkdown": "Can you share your original winning submission notebook. (Do understand the cleaned version could be similar but community might be able to learn something from the original one)" }, { "id": 3177891, "postDate": "2025-04-13T12:14:56.563Z", "content": "Fantastic work! I would love to see more of it!
", "rawMarkdown": "Fantastic work! I would love to see more of it!\n" }, { "id": 3177723, "postDate": "2025-04-13T07:23:50.573Z", "content": "最实至名归的一集。从前至后各阶段的实验安排条理清晰，模型提升的方法思路很让人受启发。期待你们下一次取得更好的成绩💪。
\n羡慕8块A800啊，穷人真是玩不起大模型😂
Thanks a lot!
\n感谢支持！
Congratulations!
\nCan you provide the data deduplication script for dataset duplicates removal?
Please share the FP16 model weight for Stage 1 SFT as well as Stage 2 DPO similar to the way AIMO1 winner did it.
\nhttps://huggingface.co/AI-MO
We have updated here:https://github.com/imagination-research/aimo2/blob/master/scripts/run_deduplicate.sh
\nStage2 DPO model can be found on huggingface:https://huggingface.co/collections/imagination-research/aimo2-67f92097e2a4e04d995f3d75, which can be found at the beginning of our write-up
\nStage1 SFT model will be uploaded sooner
", "rawMarkdown": "We have updated here:https://github.com/imagination-research/aimo2/blob/master/scripts/run_deduplicate.sh\n\nStage2 DPO model can be found on huggingface:https://huggingface.co/collections/imagination-research/aimo2-67f92097e2a4e04d995f3d75, which can be found at the beginning of our write-up\n\nStage1 SFT model will be uploaded sooner", "votes": 1 } ] }, { "id": 3177560, "postDate": "2025-04-13T01:09:37.850Z", "content": "Congrats! Such a beautiful and well-designed solution.
\nDo you have records of how Public LB improved after each tranining step?
Thanks! We got 28 without any finetuning, scored 33 after sft (8 epochs) and dpo (2 epochs), scored 34 after dpo for 4 epochs(ranging between 32~34). We didn't submit the model that only had SFT applied.
This is an absolutely remarkable solution, and thank you more for sharing your approach to improve the quality of future contributions in the field. Looking forward to more works from you
", "rawMarkdown": "This is an absolutely remarkable solution, and thank you more for sharing your approach to improve the quality of future contributions in the field. Looking forward to more works from you" }, { "id": 3177335, "postDate": "2025-04-12T15:43:50.377Z", "content": "Congrats! What was the score with the stage 1 SFT model and did DPO give significant improvement over it?
", "rawMarkdown": "Congrats! What was the score with the stage 1 SFT model and did DPO give significant improvement over it?", "replies": [ { "id": 3177599, "postDate": "2025-04-13T02:55:24.770Z", "content": "Thanks! We didn't submit the stage 1 SFT model, base model got 28, dpo-2 got 33 and dpo-4 got 34, but on private leaderboard they got 31 and 29 respectively.
Amazing work, congratulations.
\nI am interested in quantization-aware training, I had quite a bit of trouble setting up QAT. Can you share how you did it?
", "rawMarkdown": "Amazing work, congratulations.\n\nI am interested in quantization-aware training, I had quite a bit of trouble setting up QAT. Can you share how you did it?", "replies": [ { "id": 3177590, "postDate": "2025-04-13T02:36:16.660Z", "content": "Thanks for your question. We just do a very cursory and incomplete trial, the following is just for your reference. We use the 360-LLama-Factory library, which uses autoawq to load the AWQ model. In order to tune the scales of AWQ-quantized model, we do the following ugly fixes:
(1) In autoawq's awq.modules.linear.WQLinear_GEMM.__init__: We register the scales as parameters instead of buffers, in order for let the optimizer update them:
.register_parameter(\n ,\n torch.nn.Parameter(torch.zeros(\n (in_features // .group_size, out_features),\n dtype=torch.float16,\n device=dev,\n )),\n )\n(2) In autoawq's awq.modules.linear.WQLinear_GEMM.forward: Instead of calling WQLinearMMFunction.apply, we change it to use the naive implementation of decoding+matrix-multiplication. The original implementation uses a more efficient Triton-implemented decoding+matrix-multiplication, but they don't implement gradient backward for the scale.
\n \n \n \n \n \n \n \n \n \n user_has_been_warned:\n warnings.warn( + msg)\n user_has_been_warned = \n torch.cuda.amp.autocast():\n out = dequantize_gemm(.qweight, .qzeros, .scales, .w_bit, .g\\\nroup_size)\n out = torch.matmul(x, out)\n out = out + .bias .bias out\n out = out.reshape(out_shape)\n \n (out.shape) == :\n out = out.unsqueeze()\n(3) In the 360-Llama-Factory lib, we add the following call to the llamafactory.train.dpo.workflow.run_dpo function after loading the model, to set the requires_grad of scales to be True.
\n\n\n\n\n\n\n\n\n def run_dpo(\n model_args: \"ModelArguments\",\n data_args: \"DataArguments\",\n def run_dpo(\n template = get_template_and_fix_tokenizer(tokenizer, data_args)\n dataset_module = get_dataset(template, model_args, data_args, training_args, stage=\"rm\", **tokenizer_module)\n model = load_model(tokenizer, model_args, finetuning_args, training_args.do_train, full_determinism=training_args.full_determinism)\n\nAs you can see, we do some ugly fixes instead of writing a clean code. A detail is that due to our computational capability limitation, we use of cutoff length of 16k (instead of 32k) to do the training on 4 GPUs. This small quick attempt doesn't get improvements. We think tuning the weights might still be needed, and longer context length might be needed, but we don't have the GPU & development capacity to further try that.
", "rawMarkdown": "Thanks for your question. We just do a very cursory and incomplete trial, the following is just for your reference. We use the 360-LLama-Factory library, which uses `autoawq` to load the AWQ model. In order to tune the scales of AWQ-quantized model, we do the following ugly fixes:\n\n(1) In `autoawq`'s `awq.modules.linear.WQLinear_GEMM.__init__`: We register the scales as parameters instead of buffers, in order for let the optimizer update them:\n```python\n self.register_parameter(\n \"scales\",\n torch.nn.Parameter(torch.zeros(\n \t (in_features // self.group_size, out_features),\n\t\tdtype=torch.float16,\n device=dev,\n )),\n\t)\n```\n(2) In `autoawq`'s `awq.modules.linear.WQLinear_GEMM.forward`: Instead of calling `WQLinearMMFunction.apply`, we change it to use the naive implementation of decoding+matrix-multiplication. The original implementation uses a more efficient Triton-implemented decoding+matrix-multiplication, but they don't implement gradient backward for the scale.\n```python\n # out = WQLinearMMFunction.apply(\n # x,\n # self.qweight,\n # self.qzeros,\n # self.scales,\n # self.w_bit,\n # self.group_size,\n # self.bias,\n # self.out_features,\n # )\n\tif not user_has_been_warned:\n warnings.warn(\"Using naive (slow) implementation.\" + msg)\n\t user_has_been_warned = True\n with torch.cuda.amp.autocast():\n out = dequantize_gemm(self.qweight, self.qzeros, self.scales, self.w_bit, self.g\\\nroup_size)\n out = torch.matmul(x, out)\n \t out = out + self.bias if self.bias is not None else out\n\t out = out.reshape(out_shape)\n # always want 3D tensor if tensor is 2D\n\tif len(out.shape) == 2:\n\t out = out.unsqueeze(0)\n```\n(3) In the 360-Llama-Factory lib, we add the following call to the `llamafactory.train.dpo.workflow.run_dpo` function after loading the model, to set the `requires_grad` of scales to be True.\n```\n+from awq.modules.linear.gemm import WQLinear_GEMM\n+def make_awq_scales_trainable(model):\n+ num = 0\n+ for name, module in model.named_modules():\n+ if isinstance(module, WQLinear_GEMM):\n+ module.scales.requires_grad_()\n+ num += 1\n+ print(f\"Set {num} AWQ scales to be trainable\")`\n+\n def run_dpo(\n model_args: \"ModelArguments\",\n data_args: \"DataArguments\",\n@@ -45,6 +75,8 @@ def run_dpo(\n template = get_template_and_fix_tokenizer(tokenizer, data_args)\n dataset_module = get_dataset(template, model_args, data_args, training_args, stage=\"rm\", **tokenizer_module)\n model = load_model(tokenizer, model_args, finetuning_args, training_args.do_train, full_determinism=training_args.full_determinism)\n+ make_awq_scales_trainable(model)\n```\n\nAs you can see, we do some ugly fixes instead of writing a clean code. A detail is that due to our computational capability limitation, we use of cutoff length of 16k (instead of 32k) to do the training on 4 GPUs. This small quick attempt doesn't get improvements. We think tuning the weights might still be needed, and longer context length might be needed, but we don't have the GPU & development capacity to further try that. ", "votes": 1, "replies": [ { "id": 3177792, "postDate": "2025-04-13T09:36:32.430Z", "content": "Thanks for detail. Seems AWQ model so hard to finetune(especially reasoning model). My tests came out pretty bad, 15-18 on public lb.
", "rawMarkdown": "Thanks for detail. Seems AWQ model so hard to finetune(especially reasoning model). My tests came out pretty bad, 15-18 on public lb." }, { "id": 3180012, "postDate": "2025-04-16T03:23:31.467Z", "content": "Wow! this is actually really cool, so much care going into each detail…
", "rawMarkdown": "Wow! this is actually really cool, so much care going into each detail..." } ] } ] }, { "id": 3177315, "postDate": "2025-04-12T15:01:28.947Z", "content": "Very incredibly structured approach
\nCongratulations on the 2nd place win!
It's a huge work for your team, and it takes a long time to ultimately win the 2nd place, truly well-deserved. Great performance your Tsinghua and Microsoft.
\n\n\nRAG: we use this dataset as a document bank. When solving a question, we fetch 1~2 most similar questions from the bank, use them and their corresponding solutions as the few-shot demonstrations. But we did not get improvement on AIME2025 nor online submission.
\n
Once I thought RAG was a key solution for this competition, but in reality, it didn’t work when I tried it myself. I’m a bit confused, maybe the dataset is just too difficult?
", "rawMarkdown": "It's a huge work for your team, and it takes a long time to ultimately win the 2nd place, truly well-deserved. Great performance your Tsinghua and Microsoft.\n\n>RAG: we use this dataset as a document bank. When solving a question, we fetch 1~2 most similar questions from the bank, use them and their corresponding solutions as the few-shot demonstrations. But we did not get improvement on AIME2025 nor online submission.\n\nOnce I thought RAG was a key solution for this competition, but in reality, it didn’t work when I tried it myself. I’m a bit confused, maybe the dataset is just too difficult?", "replies": [ { "id": 3177312, "postDate": "2025-04-12T14:58:53.647Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3177321, "postDate": "2025-04-12T15:10:41.440Z", "content": "Thanks!
\nAbout RAG, we try RAG in an early stage of our experiments. Back then, we qualitatively inspect some runs (no statistics), and find out that the questions that have similar embedding not necessarily have similar solving strategies (e.g., have similar words instead of reasoning structure). To further explore this, maybe we can try better embedding or retrieval strategies, e.g., (1) use a more appropriate feature as the embedding (we just use the mean of the last layer's feature across tokens), which might be more reflective of the reasoning state instead of language-related information, or (2) rely on the model to do some analyses and improve the embedding based on intermediate analyses, or (3) actively search for reference and reason in a more flexible agentic way.
\nHowever, we don’t further experiment with these possible mitigations. Therefore, we cannot say we have reached a reliable negative conclusion for this pathway. It still needs further inspection and exploration.
", "rawMarkdown": "Thanks! \n\nAbout RAG, we try RAG in an early stage of our experiments. Back then, we qualitatively inspect some runs (no statistics), and find out that the questions that have similar embedding not necessarily have similar solving strategies (e.g., have similar words instead of reasoning structure). To further explore this, maybe we can try better embedding or retrieval strategies, e.g., (1) use a more appropriate feature as the embedding (we just use the mean of the last layer's feature across tokens), which might be more reflective of the reasoning state instead of language-related information, or (2) rely on the model to do some analyses and improve the embedding based on intermediate analyses, or (3) actively search for reference and reason in a more flexible agentic way. \n\nHowever, we don’t further experiment with these possible mitigations. Therefore, we cannot say we have reached a reliable negative conclusion for this pathway. It still needs further inspection and exploration.\n", "votes": 3 }, { "id": 3177594, "postDate": "2025-04-13T02:41:34.490Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 3181250, "postDate": "2025-04-17T16:07:19.640Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3178152, "postDate": "2025-04-13T19:32:39.093Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3178757, "postDate": "2025-04-14T15:11:50.600Z", "content": "Great work!
", "rawMarkdown": "Great work!" } ], "complete": true, "source": "live", "fetched_at": "2026-10-20T06:42:18Z", "competition_id": 86023, "competition_slug": "ai-mathematical-olympiad-progress-prize-2", "team_id": 12926868, "solution_url": "/competitions/ai-mathematical-olympiad-progress-prize-2/writeups/imagination-research-2nd-place-solution-team-imagi" }